Artificial General Intelligence 2008:Proceedings of the First AGI Conference 9781586038335

Citation preview

ARTIFICIAL GENERAL INTELLIGENCE 2008

Frontiers in Artificial Intelligence and Applications FAIA covers all aspects of theoretical and applied artificial intelligence research in the form of monographs, doctoral dissertations, textbooks, handbooks and proceedings volumes. The FAIA series contains several sub-series, including “Information Modelling and Knowledge Bases” and “Knowledge-Based Intelligent Engineering Systems”. It also includes the biennial ECAI, the European Conference on Artificial Intelligence, proceedings volumes, and other ECCAI – the European Coordinating Committee on Artificial Intelligence – sponsored publications. An editorial panel of internationally well-known scholars is appointed to provide a high quality selection. Series Editors: J. Breuker, R. Dieng-Kuntz, N. Guarino, J.N. Kok, J. Liu, R. López de Mántaras, R. Mizoguchi, M. Musen, S.K. Pal and N. Zhong

Volume 171 Recently published in this series Vol. 170. J.D. Velásquez and V. Palade, Adaptive Web Sites – A Knowledge Extraction from Web Data Approach Vol. 169. C. Branki et al. (Eds.), Techniques and Applications for Mobile Commerce – Proceedings of TAMoCo 2008 Vol. 168. C. Riggelsen, Approximation Methods for Efficient Learning of Bayesian Networks Vol. 167. P. Buitelaar and P. Cimiano (Eds.), Ontology Learning and Population: Bridging the Gap between Text and Knowledge Vol. 166. H. Jaakkola, Y. Kiyoki and T. Tokuda (Eds.), Information Modelling and Knowledge Bases XIX Vol. 165. A.R. Lodder and L. Mommers (Eds.), Legal Knowledge and Information Systems – JURIX 2007: The Twentieth Annual Conference Vol. 164. J.C. Augusto and D. Shapiro (Eds.), Advances in Ambient Intelligence Vol. 163. C. Angulo and L. Godo (Eds.), Artificial Intelligence Research and Development Vol. 162. T. Hirashima et al. (Eds.), Supporting Learning Flow Through Integrative Technologies Vol. 161. H. Fujita and D. Pisanelli (Eds.), New Trends in Software Methodologies, Tools and Techniques – Proceedings of the sixth SoMeT_07 Vol. 160. I. Maglogiannis et al. (Eds.), Emerging Artificial Intelligence Applications in Computer Engineering – Real World AI Systems with Applications in eHealth, HCI, Information Retrieval and Pervasive Technologies Vol. 159. E. Tyugu, Algorithms and Architectures of Artificial Intelligence Vol. 158. R. Luckin et al. (Eds.), Artificial Intelligence in Education – Building Technology Rich Learning Contexts That Work

ISSN 0922-6389

Artificial General Intelligence 2008 Proceedings of the First AGI Conference

Edited by

Pei Wang Temple University

Ben Goertzel Novamente LLC

and

Stan Franklin University of Memphis

Amsterdam • Berlin • Oxford • Tokyo • Washington, DC

© 2008 The authors and IOS Press. All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without prior written permission from the publisher. ISBN 978-1-58603-833-5 Library of Congress Control Number: 2008900954 Publisher IOS Press Nieuwe Hemweg 6B 1013 BG Amsterdam Netherlands fax: +31 20 687 0019 e-mail: [email protected] Distributor in the UK and Ireland Gazelle Books Services Ltd. White Cross Mills Hightown Lancaster LA1 4XS United Kingdom fax: +44 1524 63232 e-mail: [email protected]

Distributor in the USA and Canada IOS Press, Inc. 4502 Rachael Manor Drive Fairfax, VA 22032 USA fax: +1 703 323 3668 e-mail: [email protected]

LEGAL NOTICE The publisher is not responsible for the use which might be made of the following information. PRINTED IN THE NETHERLANDS

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

v

Preface Pei WANG, Ben GOERTZEL and Stan FRANKLIN

The field of Artificial Intelligence (AI) was initially directly aimed at the construction of “thinking machines” – that is, computer systems with human-like general intelligence. But this task proved more difficult than expected. As the years passed, AI researchers gradually shifted focus to producing AI systems that intelligently approached specific tasks in relatively narrow domains. In recent years, however, more and more AI researchers have recognized the necessity – and the feasibility – of returning to the original goal of the field. Increasingly, there is a call to focus less on highly specialized “narrow AI” problem solving systems, and more on confronting the difficult issues involved in creating “human-level intelligence,” and ultimately general intelligence that goes beyond the human level in various ways. “Artificial General Intelligence (AGI)”, as this renewed focus has come to be called, attempts to study and reproduce intelligence as a whole in a domainindependent way. Encouraged by the recent success of several smaller-scale AGI-related meetings and special tracks at conferences, we took the initiative to organize the very first international conference on AGI. Our goal in doing so was to give researchers in the field an opportunity to present relevant research results and to exchange ideas on topics of common interest. The response to AGI-08 was even stronger than we expected. We received many interesting papers addressing a wide range of AGI-relevant topics and exploring various kinds of theoretical and technical ideas. Given the complexity and difficulty of the problems the field is facing, we believe it is crucial to encourage the exchange of ideas and opinions with various degrees of maturity, ranging from position papers to descriptions of mature mathematical and cognitive AGI theories, and practical work with implemented AGI architectures. In this collection, the AGI-08 papers are organized into three groups. The conference papers are divided into full-length papers (12 pages, with a few exceptions) and short position statements (5 pages). Also included are the papers presented in the postconference workshop on the sociocultural, ethical and futurological implications of AGI. We believe meetings like AGI-08 are important, not only because of their presentations and discussions, but also because of the potential they have to help catalyze the self-organization of a vital AGI research community. Together, we will continue to directly challenge one of the most difficult and essential problems in human history, the creation of human-level artificial general intelligence. We thank the following referees for devoting their valuable time to reviewing the submitted papers: Sam Adams, James Anderson, Mike Anderson, Eric Baum, Mark Bickhard, Henry Brighton, Nick Cassimatis, Hernan Castro, L. Andrew Coward, Hugo de Garis, Wlodzislaw Duch, Richard Duro, David Friedlander, J. Storrs Hall, David

vi

Hart, Marcus Hutter, Cliff Joslyn, Randal Koene, Moshe Looks, Bruce MacLennan, Don Perlis, Matthias Scheutz, Juergen Schmidhuber, Lokendra Shastri, Boris Velichkovsky, Karin Verspoor, Paul Vogt, and Mark Waser. We would also like to thank the other members of the AGI-08 Organizing Committee, Sidney D’Mello, Bruce Klein, and Lee McCauley, for the thought and effort they expended in preparing the conference; and Bruce Klein and Natasha Vita-More for their help with the post-conference workshop which resulted in a number of the papers contributed to this volume.

vii

Contents Preface Pei Wang, Ben Goertzel and Stan Franklin

v

Full-Length Papers Participating in Cognition: The Interactive Search Optimization Algorithm Nadav Abkasis, Israel Gottlieb and Eliraz Itzchaki Input Feedback Networks: Classification and Inference Based on Network Structure Tsvi Achler and Eyal Amir

3

15

Reasoning with Prioritized Data by Aggregation of Distance Functions Ofer Arieli

27

Distance-Based Non-Deterministic Semantics Ofer Arieli and Anna Zamansky

39

Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture. PART 2: Formalization for Ethical Control Ronald C. Arkin

51

Seven Principles of Synthetic Intelligence Joscha Bach

63

Language Processing in Human Brain Alexander Borzenko

75

Toward Logic-Based Cognitively Robust Synthetic Characters in Digital Environments Selmer Bringsjord, Andrew Shilliday, Joshua Taylor, Dan Werner, Micah Clark, Ed Charpentier and Alexander Bringsjord A Cognitive Substrate for Natural Language Understanding Nicholas L. Cassimatis, Arthi Murugesan and Magdalena D. Bugajska The China-Brain Project: Building China’s Artificial Brain Using an Evolved Neural Net Module Approach Hugo de Garis, Jian Yu Tang, Zhiyong Huang, Lu Bai, Cong Chen, Shuo Chen, Junfei Guo, Xianjin Tan, Hao Tian, Xiaohan Tian, Xianjian Wu, Ye Xiong, Xiangqian Yu and Di Huang

87

99

107

Cognitive Architectures: Where Do We Go from Here? Włodzisław Duch, Richard J. Oentaryo and Michel Pasquier

122

LIDA and a Theory of Mind David Friedlander and Stan Franklin

137

viii

How Might Probabilistic Reasoning Emerge from the Brain? Ben Goertzel and Cassio Pennachin An Integrative Methodology for Teaching Embodied Non-Linguistic Agents, Applied to Virtual Animals in Second Life Ben Goertzel, Cassio Pennachin, Nil Geissweiller, Moshe Looks, Andre Senna, Welter Silva, Ari Heljakka and Carlos Lopes VARIAC: An Autogenous Cognitive Architecture J. Storrs Hall Probabilistic Quantifier Logic for General Intelligence: An Indefinite Probabilities Approach Matthew Iklé and Ben Goertzel

149

161

176

188

Comirit: Commonsense Reasoning by Integrating Simulation and Logic Benjamin Johnston and Mary-Anne Williams

200

Learning from Inconsistencies in an Integrated Cognitive Architecture Kai-Uwe Kühnberger, Peter Geibel, Helmar Gust, Ulf Krumnack, Ekaterina Ovchinnikova, Angela Schwering and Tonio Wandmacher

212

Extending the Soar Cognitive Architecture John E. Laird

224

Temporal Action Logic for Question Answering in an Adventure Game Martin Magnusson and Patrick Doherty

236

Artificial General Intelligence Through Large-Scale, Multimodal Bayesian Learning Brian Milch

248

A Computational Approximation to the AIXI Model Sergey Pankov

256

Essential Phenomena of General Intelligence Marc Pickett, Don Miner and Tim Oates

268

OSCAR: An Architecture for Generally Intelligent Agents John L. Pollock

275

Anticipative Coordinated Cognitive Processes for Interactivist and Piagetian Theories Jean-Charles Quinton, Jean-Christophe Buisson and Filipo Perotto Hybrid Reasoning and the Future of Iconic Representations Catherine Recanati Cognitive Constructor: An Intelligent Tutoring System Based on a Biologically Inspired Cognitive Architecture (BICA) Alexei V. Samsonovich, Kenneth A. de Jong, Anastasia Kitsantas, Erin E. Peters, Nada Dabbagh and M. Layne Kalbfleisch Transfer Learning and Intelligence: An Argument and Approach Matthew E. Taylor, Gregory Kuhlmann and Peter Stone

287 299

311

326

ix

Real Time Machine Deduction and AGI Peter G. Tripodes

338

Text Disambiguation by Educable AI System Alexander Voskresenskij

350

What Do You Mean by “AI”? Pei Wang

362

Using Decision Trees to Model an Emotional Attention Mechanism Saman Harati Zadeh, Saeed Bagheri Shouraki and Ramin Halavati

374

Position Statements Fusing Animals and Humans Jonathan Connell

389

Four Paths to AI Jonathan Connell and Kenneth Livingston

394

Adversarial Sequence Prediction Bill Hibbard

399

Artificial General Intelligence via Finite Covering with Learning Yong K. Hwang, Samuel B. Hwang and David B. Hwang

404

Cognitive Primitives for Automated Learning Sudharsan Iyengar

409

Vector Symbolic Architectures: A New Building Material for Artificial General Intelligence Simon D. Levy and Ross Gayler

414

Analogy as Integrating Framework for Human-Level Reasoning Angela Schwering, Ulf Krumnack, Kai-Uwe Kühnberger and Helmar Gust

419

Designing Knowledge Based Systems as Complex Adaptive Systems Karan Sharma

424

Artificial General Intelligence: An Organism and Level Based Position Statement Leslie S. Smith

429

Workshop Papers The Artilect War: Cosmists vs. Terrans. A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines Hugo de Garis

437

Stages of Ethical Development in Artificial General Intelligence Systems Ben Goertzel and Stephan Vladimir Bugaj

448

Engineering Utopia J. Storrs Hall

460

x

OpenCog: A Software Framework for Integrative Artificial General Intelligence David Hart and Ben Goertzel

468

Open Source AI Bill Hibbard

473

On the Broad Implications of Reinforcement Learning Based AGI Scott Livingston, Jamie Garvey and Itamar Elhanany

478

The Basic AI Drives Stephen M. Omohundro

483

A Scientific Perspective on the Hard Problem of Consciousness Alexei V. Samsonovich, Giorgio A. Ascoli, Harold Morowitz and M. Layne Kalbfleisch

493

Author Index

507

Full-Length Papers

This page intentionally left blank

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

3

Participating in Cognition: The Interactive Search Optimization Algorithm Nadav Abkasis, Israel Gottlieb and Eliraz Itzchaki Department of Computer Science, Jerusalem College of Technology, Israel

Abstract. We consider the Distributed Cognition paradigm as a framework for implementing artificial components of human cognition. We take email/internet search as a setting of distributed cognition and describe an algorithm that intervenes and enhances a component of this distributed process. Results are presented demonstrating the effectiveness of the algorithm in interaction with an automaton simulating human search activity. We define a notion of synchronization with a non-random, non-Turing computation and argue that the given algorithm exhibits such synchronization behavior. Our results suggest that the framework may be useful for studying a class of non-Turing computation that is central to General Intelligence. Keywords. Distributed Cognition, Embodiment, Search Optimization, Non-Turing Computation.

1. Introduction In Artificial General Intelligence one seeks the "General", i.e. the ability to come up with the right approach to a new situation, as opposed to merely applying an algorithm to a problem setting for which it has been designed. The phrase "applying an algorithm to a problem for which it has been designed" is a fair description of an arbitrary but specific Turing machine. A Universal Turing machine relaxes the condition that the algorithm be a fixed one, but leaves intact the requirement that the algorithm executed must be appropriate for the problem to be solved. It is therefore natural to investigate the possibility, and perhaps even the necessity of non-Turing capability for AGI, or what has been called "hypercomputing". Most work on hypercomputing has been of a theoretical sort, i.e. investigation of machine structures whose physical realizability remains a largely open question [1]. In this work we are concerned with a form of hypercomputing that is directly observable, and offers a practical context for experimentation as well. A researcher confronted with a problem will often have practical experience that suggests an "intuition" about the correct approach to a solution. The challenge in such situations is typically expressed by the phrase, "can you formalize this intuition?" By this is meant can one translate the intuition into explicit rules, hence making them more readily subject to analysis, optimization and automated implementation. In cognitive science a key issue is the process of intuition itself; here the challenge is modified as – "can we formalize the algorithm of intuition?". In both contexts we can observe an attempt to bridge the no-man's-land between the obscurity of intuition

4

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

and the clarity of an algorithm. In the cognitive science case however, the attempt has historically been problematic. If we take "algorithm" as synonymous with "Turing machine" and at the same time allow that "intuition" is non Turing-computable – as Turing himself believed [2], then "algorithm" and "intuition" are mutually exclusive and clearly we are asking the wrong question. Or perhaps our notion of algorithm needs to be extended. In this paper we approach the task of understanding the process of intuition via a "learn by doing" method, i.e. explicit participation in the algorithm of intuition. We will argue that such explicit participation amounts to a broader notion of what is meant by an algorithm than the accepted sense of the term, and one that may be more suited to the goals of AGI. How can we participate in a process that appears to us as a black box locked away in the human brain? Here, the notion of distributed cognition [3] is useful. Distributed cognition posits that cognitive processes should be viewed as including elements outside of the physical individual. Hence we may intervene or participate in this process by replacing one of its external components with an alternate version. In what follows, we first briefly review the relevant tenets of distributed cognition. We then tackle the question of how to render explicit any part of a process which is by definition not well-defined, i.e. intuition. Our answer will be constructive, i.e. we consider a common cognitive process in which intuition plays an obvious role, posit a distributed structure as the setting for this process and then build an enhanced version of one part of this structure. In this work we do not as yet draw conclusions regarding intuition nor offer a model for this or any other cognitive process. Our purpose is to suggest active participation as a technique for learning about such processes and associated possible models. Moreover, our emphasis is on the construction of working systems that produce results and can be used to learn by experiment. Accordingly, most of this paper describes a working system. Finally, we shall give an informal argument that our methodology exhibits a form of directly observable non-Turing computation. 1.1. Distributed Cognition Distributed congnition is described in [3] as positing distribution of the process of cognition over three dimensions, 1. The social dimension, i.e. over groups rather than single individuals, 2. Over the material artifacts in the environment wherein the cognition is operative, and 3. Over the time dimension. That is, historical events influence later ones via a process that is itself integral to the cognition. The reverse may also be true, viz. the influence of history is itself influenced by the present in that it is interpreted differently according to current experience. An additional tenet of distributed cognition is that of embodiment. This amounts to a rejection of the view that cognition is a process consisting of interactions between purely abstract representations, from which it follows that the underlying substrate that carries these representations is of no consequence to cognitive behavior. Rather, it holds that such substrate directly influences the congnitive process and dictates the types of interactional structures that may emerge.

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

5

1.2. Search Application Consider an email search task that often confronts a user at a large organization. They receive a large volume of mail, so large that they do not manage to file it all in neat separate mail folders. A common situation is looking for a piece of mail received some time back, but unfortunately, our user cannot remember how long ago – was it 2 weeks or 2 months? Neither do they remember the exact name on the "From: xxx" header, although they likely have a vague recollection of one or a few letters that it contains. They may or may not have a vague recollection of some part of the Subject line. Our user keeps mail for many months, possibly longer, knowing that these situations arise. However, finding exactly what one is looking for is more difficult. We start with a user that has entered an initial set of keywords, e.g. Sender and Subject, based on whatever approximating string of letters they can muster from memory. The search engine produces a list of candidate emails in response and based on these, the user modifies the keywords; this process may repeat some (small) number of times. The specific modification to keywords employed by a user – in response to the material offered by the search engine – typically has no fixed recipe, and may well change substantially from one iteration to the next. Clearly, intuition is being relied upon heavily to modify keywords so as to promote the user's objective. In this setting we can directly identify all 3 of the dimensions indicated by the distributed approach. Thus, 1. In addition to the user's brain, the search engine is a participating artifact. 2. The time scale of the process extends over successive iterations of keyword entry, examination of results and keyword modification. 3. Search is directly influenced by the qualitative content of material returned by the engine. This material in turn is the direct contribution of others in the email community of which the user is a member. The principle of embodiment allows the influence of this material on the structure of the cognitive process. Our goal is to participate in this cognitive process, and in so doing – enhance it. The chief difficulty is that the user does not know exactly what they are looking for – else they would enter the best keywords directly. Thus standard optimization algorithms are not applicable since we don't have a definition of what to optimize. However, we do know that our user eventually finds what they are looking for in most cases, hence we can posit an approximate monotonicity of progress versus time. That is, although there may be missteps and setbacks, on average there are significantly more steps forward than backward. We can therefore undertake a supportive role comprising two elements, as follows: 1. Remind the user of how they have progressed in similar situations in the past, with the goal of avoiding backwards steps, and 2. Abbreviate multiple steps into a single one, in cases where the user has found the sequence repeatedly useful in the past. We still have a significant challenge to overcome, viz. how can one relate "situations in the past" and situations "found repeatedly useful" in the past – to the present context, when the notion of "useful" has no representation or definition that is accessible to us? Our approach is to rely on embodiment, viz. what is going on in the cognitive process must be parallel in some sense to the artifacts in the environment to which the process is being applied, whence it follows that we can track the cognition by simulation with those artifacts. Specifically, in the current application the artifacts are keywords. If we find that in previous iterations the user has tried to pursue progress

6

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

toward the goal by modifying keyword a to keyword a', i.e. the user has made an ordered association between two versions of a keyword, then we can conclude that this association is not the result of an intuition totally abstracted from the reality of words, word-parts or speech sounds etc. but rather that cognition is implicitly relying on a distance metric on the space of words itself. If we can find a metric for which the search iterations cluster into well defined repetitive associations then we have found a structure that parallels the cognition. In the next section we make these ideas more precise via formal definition as an algorithm, which we shall call the Interactive Search Optimization (ISO) algorithm. We note that the above description of an email search process applies, essentially unchanged, to a search of the internet. We chose the email setting for our application because it was simpler to implement. For the internet case, a more elaborate and specialized characterization as a distributed cognitive process has been suggested in [4]. However, the three-dimensional characterization given above is adequate for our development here, and applies equally well to both problem settings.

2. Formal Development We consider keywords from the 26 letter English alphabet only. The number of letters in our keywords is not restricted – other than by computational resources, and one can arbitrarily assign e.g. the first five letters to the "From:" key, the next five to the "Subject:" key, etc. Without loss of generality therefore, we consider in the sequel a single keyword of arbitrary length m. Thus we have m dimensions of variation with 26 possible values each. Each of these variables is mapped to an angle θ as depicted in Figure 1.

A θ = 00

θ = −90 0

θ = +900

Figure 1 – Base Ordering and Initial Position

The ordering around the circle follows typical associations between the sounds of letters – as they are actually experienced by humans, viz. (a e h o u), (b d), (c s x), (g j), (i y), (k q), (m n), (p t) and (v w) are the groupings used. The remaining letters are singletons. The groups and singletons in turn are ordered approximately as they appear in the alphabet. This is called the base ordering. In the initial position, the angle θ = 0 o points to the letter 'A'.

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

7

Definition 1: User Response and Computation of Action Vector A delta search vector, is an ordered sequence of angles (θ 1 , θ 2 , … , θ m ) , 0

0 ≥ θ j ≥ −180 0 or 0 0 < θ j ≤ +180 0 ; each θ j represents a (possibly zero) rotation

away from the current position. The user submits a delta vector to modify the keyword represented by the current position. The intention of this arrangement is that, in addition to recording the user's revision of a keyword, we also capture a notion of how the user judges the progress of the search. Specifically, the zero direction indicates "no change required", whence user responses θ j that are close to zero will indicate generally that we are in the right direction, and conversely – the farther away from zero (or the closer to ± 180 degrees), the greater the user dissatisfaction with the progress of the search. The delta vector is provided by the user in the response matrix θ 1 , θ 2 , … , θ m   , along with a vector of corresponding weights w j , 0 < w j ≤ 1 , the   w1 , w2 , … , wm  latter representing the user's indication of their own confidence in the former. Instead of applying the delta in the user response directly – to obtain the next keyword, the ISO algorithm applies transformations to the response matrix based on what it can learn from the history of user responses thus far. The transformed version is called the action matrix because its delta vector is the one actually input to the search engine. Thus we have a sequence of pairs ( A1 , R1 ), ( A2 , R2 ), … etc., where the index indicates the particular iteration of results/user response, Ai is the action matrix whose delta vector is input to the search engine and Ri is the corresponding user response matrix. The role of the weights w j and the transformations that generate the A matrices will be defined in the sequel. Definition 2: Unification Write δ j to denote one of the values θ j or zero. Upper case X denotes any delta matrix. The unification mapping U ( X k , X k +1 ,… X k +l ) → X ' may be expressed as,   P (θ ' j = θ '1 ,θ ' 2 , … ,θ ' m   X '=  ,   w'1 , w' 2 , … , w' m   w' j = 

k +l

∑[δ

j ]h

)=

∏ [p ]

j h

h=k

∏

[ p j ]h

if θ ' j =

k +l

∑ [δ

j ]h

h = k +1

where if [δ j ] h = [θ j ] h [ w j ] h [ p j ]h =  , 1 − w j ] h if [δ j ] h = 0 P(θ = x) denotes the probability that the random variable θ takes the value x and this probability is specified for all possible values of the m placeholders δ j . That the

distribution for θ ' j is a valid probability mass function (PMF) may be readily verified by noting that the distribution of each θ ' j is identical with the joint PMF of

8

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

independent identically distributed binary random variables [δ ] h , 0 ≤ h ≤ l with probabilities P([δ ] h = True) = [ w] h and P([δ ] h = False) = 1 − [ w] h . We need the following functions, defined on any delta matrix X. Definition 3: User Grade 1.

The User Grade (URG): + 1, if − 45 0 ≤ θ ≤ +45 0  urg (θ ) =  0, if − 135 0 ≤ θ ≤ −45 0 or + 45 0 ≤ θ ≤ +135 0  0 0 0 0 − 1, if − 180 ≤ θ ≤ −135 or + 135 ≤ θ ≤ +180

2.

The normalized, weighted, scalar user response θ S : θS =

∑

m

∑w

m

m

1 j

E (θ j ) ,

∑w

j

≠0 ,

j =1

j =1

j =1

where

E (θ j ) is the expectation of

the random variable θ j .

S

θ summarizes the information in a delta matrix with a single number.

3. 4.

The normalized weighted User Grade urg (θ S ) . In the sequel, URG refers to the normalized weighted version, unless otherwise indicated. The Dimensional Progress Index (DPI). This is a relative measure of cumulative progress toward the goal – as perceived by the user. It is defined recursively as follows: dpi + urg i , i > 0 dpii =  i −1 0, i =0 

We think of the search space as comprising dimensions that are successively constrained or fixed so that the balance of the space to be searched gets progressively smaller. In this conceptualization, progress toward the goal may be measured by the number of dimensions that have been successfully fixed – hence the term Dimensional Progress Index. The pair [urg i , dpii ] is our overall user grade for iteration i. We must also specify if the grade function is taken on the response matrix Ri or on the action matrix Ai . Definition 4: Generations The delta matrices Ai , Ri and their associated user grades [urg , dpi] comprise the ith actual generation. The accumulated sequence of actual generations is called the history. An actual generation is always a single ( Ai , Ri ) pair. Actual generations are distinguished from virtual generations (defined next), which are delta matrices the ISO algorithm will compute as candidates for input to the search engine, and which may or may not become actual. A virtual generation may have more than one element. At each iteration i, the algorithm will compute a virtual next generation of candidate matrices Ai +1 by applying unification to subsequences of actual generations

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

9

in the history, in accordance with the unification rules below. The virtual generation is computed from scratch at each iteration; there is only one such generation at any time. Definition 5: Unification Rules The symbol ' ≅ ' denotes fuzzy equality, useful for e.g. comparison between DPI values which are in any case user subjective. We use standard fuzzy set membership based on a parabolic function [5]. In what follows, i is the current iteration number, while k is an arbitrary iteration number in the history. A superscript distinguishes functions associated with R matrix from those associated with the A matrix. 1.

2.

Unify all successive pairs of action matrices found in the history (… Ak , Rk , Ak +1 …) such that urg ka = urg ir and dpi ka ≅ dpiir . Unify all subsequences of action matrices found (… Ak , Rk , Ak +1 , … Ak +l , …) such that

in the history

urg ir−1 = + 1 , urg ir = − 1 , urg ka−1 = + 1 , urg ka+ h ≠ + 1, ∀ h ∈ [0, l ] , urg ka+ l +1 = + 1 and dpi ka ≅ dpiir . Note that the functions urg and dpi for the current iteration associate with the response matrix Ri while the those from the history associate with the action matrix

Ak . In general, we look for precedent in the action matrices of the history rather than the response matrices, since the former correspond what actually took place. In contrast, for the current iteration, we are in the process of computing the action matrix so the response matrix Ri is the most up-to-date indication of progress. The second rule is intended to capture a sequence of miss-steps, i.e. skip over all the "wrong way" attempts a user might make while chasing a dead end. Note that the cases are not mutually exclusive, i.e. where urg ka = urg ir = − 1 both unifications may apply. A delta matrix computed from unification will be called a synthetic action, i.e. we are taking the liberty of transforming the user's response based on the history. We apply a ranking of the synthetic actions in the virtual generation. The winner, if there is one, will be used instead of the current user response to form the next actual generation. We compute a Consistency Index on the elements in the virtual generation. Denote by A' a synthetic action for which the index is to be computed, and let A be the set of all action matrices accumulated thus far, synthetic or not, i.e. the union of the virtual generation and all actual generations. Consistency means the degree to which the action of A' is confirmed by other action deltas, and it is computed for A' separately relative to each A ∈ A. Consistency has two criteria. The first checks that the action A ∈ A was applied with DPI similar (fuzzy equality) to the current value1. The second looks for similarity of the user grade θ S . Denote by eq : ℜ × ℜ ֏ [0, 1] a fuzzy equality function on pairs of real numbers, i.e. with image in the continuous interval [0,1]. We first compute a Local Consistency Index (LCI) for A' relative to a particular A ∈ A, 1

Unifications may be nested to arbitrary depth, hence this requires that we maintain, for any synthetic action matrix, a record of the DPI that held at the first iteration of the first level of unification.

10

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

LCI (A') = Min ( eq (θ S ( A' ), θ S ( A)), eq(dpi ( A), dpii )) , A ∈ A, where dpii is the DPI associated with the current user response. We then apply concentration and fuzzification, standard techniques from fuzzy logic [11], based on the actual urg which followed the action A. If that urg ≠ −1 , we take the square root of the LCI, otherwise we square it2. The former magnifies the LCI; the latter reduces it. Finally, we compute the Global Consistency Index (GCI) for each synthetic A' in the virtual generation, LCI( A' , A) . GCI (A') =

∑

A∈ A

There are two cases in which the above procedure fails to produce a candidate for the action matrix Ai +1 : where no sequence in the history qualifies for unification, or where the GCI is zero for all unifications computed. These will occur particularly in the early iterations when the history is short (or non-existent). In either of these situations we say that synthesis fails. We can now define the transformation that computes the action matrix Ai +1 at each iteration. Denoting by VG the virtual generation, we have, if synthesis fails U ( Ri ) Ai +1 = arg max GCI( A) otherwise .  A∈VG Unification of the user response matrix with itself – indicated for the case of synthesis failure, is just a non-deterministic rendering of the user's response, viz. each θ j is applied with probability w j ; with probability 1 − w j we apply a zero delta. The

algorithm iterates until the user indicates the target has been found (or gives up).

3. Experimental Results In this section we present our experience with the use of the ISO algorithm described in the previous section3. Our first task was to verify that the algorithm performs its intended function. That is, allowing our hypothesis from Section 1.2 that intuition makes use of a distance metric on keywords, does the algorithm in fact improve the efficiency of search? We constructed automata that simulate a user's search activity under different behavioral assumptions. We then compared the number of steps required by an automaton to reach the search objective to the number of steps required when that automaton is supported by the ISO algorithm. The results show that, for a significant range of behavioral parameters, and in particular such as would be expected in realistic human contexts, the algorithm significantly reduces the number of search steps. Of course the situation of a human user is fundamentally different in that they do not know the target in advance. Hence these experiments cannot directly verify the key "Embodiment" thesis, viz. that an ISO-like process actually tracks cognition. Experiment with humans remains for future work.

2

Applicable only to LCI relative to actual matrices A in the history, not virtual ones. In practice, some minor additional tuning and/or approximation to the algorithm was used. They are not detailed due to lack of space. 3

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

11

3.1. Types of Automata In each of the automata described, the software knows the target keyword, and advances toward it, away from it or just "gets lost" – in accordance with a specific behavioral policy that is intended to approximate a human user's behavior. A parameter α introduces "errors" into the search behavior; with probability α , the automaton presents a user response (i.e. a delta angle θ ) in accordance with its particular behavioral policy, and with probability 1 − α it sets the delta angle θ at random. We constructed 4 different automata, as follows: A 1: Each response delta advances 1/4 of the distance to the target, relative to the base ordering. The random errors introduced with probability α are not given any special treatment, and enter the history as any other user response. A 2: Same as Automaton 1, however, we simulate also the likely intervention of a human user after a move in the wrong direction. After a random delta is executed the automaton retracts its move, i.e. presents an inverse delta, that returns the search to the same position it was before the random move. A 3: Same as Automaton 1, except as follows. We maintain a parallel record of the progress that would have been achieved by the unaided automaton. If at any point the unaided automaton gets ahead of the ISO algorithm, it "notices" that the system is not responding effectively and repeats its last response, i.e. the one which brought the unaided search into the lead. A 4: The response deltas produced by the automaton alternately advance toward, and retreat away from the target keyword, in accordance with the following sequence. 1 1 1 1 1 1 − + − + − , . . ., etc. n n + 2 n +1 n + 4 n + 3 n + 6 where n is a small integer such as 2 or 4, and each term represents a fraction of the current distance to the target. This series is dominated by ∑ 2 n 2 and hence converges. As long as the limit of the sum is > 1, one necessarily arrives at the target after a sufficient number of steps. More interesting in our context is that this behavior simulates "two steps forward and one step back". Parameters for all the automata were varied over a range of values. In addition to the value of α , we varied the confidence weights associated with each delta in two different ways. In the simplest approach, the same confidence level was fixed for all user (i.e. automaton) responses throughout the search. For a more realistic scenario, we applied one level of confidence to those responses which followed the behavioral policy of the automaton and a different confidence value for the "error" responses, i.e. those generated randomly. Figure 2 gives representative results for two of the automata; results for the other two are similar. In Figure 3 we show the results as a function of varying confidence, and of varying differential between confidence for useful Reponses (labeled "Base Confidence) as opposed to confidence for "error" responses. All data represent an average of 100 searches repeated with the same target keyword.

12

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

Figure 2 – Reduction in number of search steps required, relative to the unaided user as simulated by Automatons 2 and 4. In the first two columns the automatons used a fixed confidence level; in the last column the random inputs indicated lower confidence. A "Winning Trial" means the ISO algorithm suceeded in reducing the number of steps. All data points are an average of 100 trials.

Figure 3 – The first two graphs show the proportion of winning trials for ISO, as a function of the user confidence parameter. Error confidence is constant at 50%. For the 3rd graph error confidence varies while base confidence is held constant at 85%. The % reduction for ISO is shown as a function of their difference.

4. Related Work The ISO algorithm has parallels with Estimation of Distribution algorithms (EDA) [6,7]. After unification is applied, ISO filters the results via the local and global consistency indices; this may be viewed as a selection function. Unification itself amounts to construction of a distribution and its use to perturb candidate actions. However, there are key technical differences between ISO and EDAs. Whereas EDAs retain the basic GA notion of perturbing the entire surviving population, our algorithm selects particular individuals for Unification, as dictated by the current user Response. Of course the current user Response is an influence that is absent in the conventional GA problem setting. Further, in contrast to ISO, EDAs and evolutionary algorithms in general are concerned at any generation only with the currently surviving population. Additional important differences may be added to this list. Similarity with the "online" form of Neural Networks maybe noted [8]; more specifically, Recurrent Neural Networks [9,10] learn from a history of errors, as does ISO. However ISO differs in many ways from any form of neural net. An obvious difference is that it does not adjust weights.

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

13

In our view, the key innovation of ISO is the way the problem is cast rather than the technical particulars of the algorithm used to solve it. Simply, we consider a search where we begin without any well-defined criteria for testing a correct solution. This of course, is exactly what happens when a human user searches email or the internet for a target that is only vaguely remembered. In the next section we explore the implications of treating this situation as a legitimate computational problem. A more significant similarity may be observed between ISO and so-called "Putnam-Gold Machines" [11]. Computation in the latter do not have a fixed ending. Similarly, any particular output produced by ISO, is not, in general, the "real" output.

5. Discussion and Conclusions The ISO algorithm is not a self-contained "artificial intelligence", since it relies explicitly on human direction. Nevertheless, we have an AI "result", i.e. the useful outcome of reduced search times. This is achieved by artificial enhancement of one part of a human cognitive process, rather than by a process that is entirely artificial. As noted in the introduction, Turing and his colleagues believed that Turing computation, i.e. any mechanical process is necessarily devoid of intelligence. This suggests that to achieve general intelligence we must go beyond Turing computation. What follows is an informal argument, suggesting that the ISO technique offers a way to actively participate in just such a non-Turing computation. Proposition 1: A Turing computation is completely specified, hence must have a well-defined goal. At a minimum, the definition of this goal is the specification of what it does. The exception is random input, i.e. a non-deterministic machine; the random influence makes the machine incompletely specified. Proposition 2: The search problem for ISO does not have a well-defined goal, hence not a Turing computation. Likewise, the sequence of user inputs to ISO is non-Turing. Next we show that this non-Turing behavior is not random. The ISO method interprets user choices based on the earlier history of those same choices, i.e. we assume the user may not have the memory or analytical resources to choose in a manner that is consistent with previous choice behavior. The implicit assumption is that effective native user progress is itself based in some sense on offsets from previous choices. Put another way, we assume the user would want to be consistent with the history, if they did have the resources. If the ISO algorithm consistently reduces search times, then a correspondence must exist between the sequence of action deltas generated by the algorithm and the goal of the process at work in the mind of the user. For example, it may be that the sequence approaches the target of the search in some limit sense. Alternatively, the sequence of outputs from ISO may correspond to a succession of revisions of the target, without specifying any structure on the succession of targets or the basis for revision. There may be other ways of setting up this correspondence. Denote by I = [i1 , i 2 , … , i n ] the sequence of inputs provided by the user to ISO and by O = [o1 , o 2 , …] the outputs it generates in response. That a correspondence exists between oi and ii is straightforward. The more significant observation

14

N. Abkasis et al. / Participating in Cognition: The Interactive Search Optimization Algorithm

however, is that each oi relates in some way to in , the final target of the search. This must be so since the user succeeds in reaching the target in fewer steps when supported by ISO than with unaided intuition alone. We say that the ISO algorithm is synchronizing with the user cognitive process. We have: Proposition 3: It is not possible to synchronize with a random sequence. The ISO algorithm synchronizes with the user input sequence, so that sequence is non-random. Hence ISO synchronizes with a non-random, non-Turing computation. The ISO algorithm, by exhibiting the above-described synchronization behavior, offers a means to explore the process by which such a non-Turing results may be arrived at. It also suggests that explicit formulation of one of the distributed components of a cognitive process may be a promising alternative paradigm for "algorithm". Learning from experiment with such working systems, we may be able to construct explicit enhancements – or indeed replacements – for progressively more of the components of such processes. Eventually, we may construct all of them, hence achieving a self-contained artificial intelligence.

References [1]

T. Ord, The Many Forms of Hypercomputation, Applied Mathematics and Computation, Vol .178, 2006, pp. 143-155. [2] M. H. A. Newman, Allan Mathison Turing, Biographical Memoirs of the Royal Society, 1955, pp. 253263. [3] J. Hollan, E. Hutchins and D. Kirsh, Distributed Cognition: Toward a New Foundation for HumanComputer Interaction Research, ACM Transactions on Computer-Human Interaction, Vol. 7, No. 2, June 2000, Pages 174–196. [4] C. Heintz, Web search engines and distributed assessment systems, In: Harnad, Stevan and Dror, Itiel E. (eds.), Distributed Cognition: Special issue of Pragmatics & Cognition 14:2 (2006). 268 pp. (pp. 387–409) [5] G. Klir and B. Yuan, Fuzzy Sets and Fuzzy Logic: Theory and Applications, Prenctice Hall, 1995. [6] P. Larrañaga and J. A. Lozano (Eds.), Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation, Series: Genetic Algorithms and Evolutionary Computation , Vol. 2, Kluwer Academic, 2001. [7] M. Pelikan, D. E. Goldberg and F. G. Lobo, A Survey of Optimization by Building and Using Probabilistic Models, Computational Optimization and Applications, Springer, 2002, pp.5-20. [8] D. Saad (ed.), Online Learning in Neural Networks, Cambridge University Press, January 1999. [9] M. Danilo, and J. Chambers, Recurrent Neural Networks, Wiley, 2001. [10] S. Hochreiter and J. Schmidhuber, Long Short-Term Memory, Neural Computation, Vol 9 No 8, 1997, pp. 1735-1780. [11] P. Kugel, Computing Machines Can’t Be Intelligent (...and Turing Said So), Minds and Machines, Vol.12, No. 4, September 2002.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

15

Input Feedback Networks: Classification and Inference Based on Network Structure Tsvi Achler and Eyal Amir Department of Computer Science, University of Illinois at Urbana-Champaign We present a mathematical model of interacting neuron-like units that we call Input Feedback Networks (IFN). Our model is motivated by a new approach to biological neural networks, which contrasts with current approaches (e.g. Layered Neural Networks, Perceptron, etc.). Classification and reasoning in IFN are accomplished by an iterative algorithm, and learning changes only structure. Feature relevance is determined during classification. Thus it emphasizes network structure over edge weights. IFNs are more flexible than previous approaches. In particular, integration of a new node can affect the outcome of existing nodes without modifying their prior structure. IFN can produce informative responses to partial inputs or when the networks are extended to other tasks. It also enables recognition of complex entities (e.g. images) from parts. This new model is promising for future contributions to integrated human-level intelligent applications due to its flexibility, dynamics and structural similarity to natural neuronal networks.

Introduction Regulation through feedback is a common theme found in biology including gene expression and physiological homeostasis. Furthermore, feedback connections can be found ubiquitously throughout neuronal networks of the brain. Yet, the role of feedback in neuronal networks is often under-appreciated. Most connectionist models determine connection weights during a learning phase and during the testing phase employ a simple feedforward structure. The contribution of this paper is to demonstrate that feedback employed during the test phase can perform powerful distributed processing. In our feedback model every output node only inhibits its own inputs. Thus the model is named Input Feedback Networks (IFN). This paper reinforces preliminary results of IFN [1, 2] and explores its ability to perform recognition and intelligent inference. Our analysis is divided into two subsections: combinatorial considerations and functional analysis. The combinatorial section reviews various networks, discussing the implications of adding new representations and the plausibility of large networks. The functional analysis section explores through exemplars how IFN behaves in complex scenarios. The scenarios are: 1) when an output node with a small input vector completely overlaps with an output node composed of a larger input vector. Subsequently, the smaller vector is innately inhibited given the inputs of the larger vector. 2) Multiple overlapping representations can cooperate and compete under differing circumstances. 3) Network ‘logic’ can be adjusted by simply biasing activation of an output. 4)

16

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

Depending on inputs states, inference can be conducted through distributed processing over an infinite number of chains. 5) Overlap of representations determines processing difficulty and the role initial conditions may have on inference. Some of these results are preliminary. However, it is clear that based on a simple premise of feedback, IFN offers a flexible structure and dynamic approach to classification of stimuli and Artificial Intelligence.

1. Background Traditional connectionist classifier models can be broken down into two broad strategies: 1) Neural Network (NN) type algorithms that primarily rely on connection weight adjustments, implemented by learning algorithms. 2) Lateral Competition (LC) algorithms that involve competition between ‘similar’ representations. LC and NN methods can overlap. For example, weight adjustments occur between LC connections. We argue that both connection weights and direct output inhibition are specific to tasks. Furthermore they can be combinatorically implausible and can limit processing. 1.1. Connection Weight Adjustments The idea of adjusting connection weights has been a corner stone throughout the development of connectionist models including Perceptrons, Parallel Distributed Networks, Markov Models, Bayesian Networks, Boltzmann Machines, and even Support Vector Machines [3-7]. Networks based on weight adjustment are powerful, flexible and in combination with learning algorithms can have a good degree of autonomy. These methods allow high degrees of freedom where numerous sets of weights can be chosen for virtually any problem. Weights are adjusted per task for specific applications. However without appropriately selected training sets, this approach suffers from difficulties such as over-fitting, local minima and catastrophic interference (‘forgetting’ of previously learned tasks, e.g. [8, 9]). The role of Neural Network (NN) structure in relation to function or image parts is unclear. Learning algorithms are difficult to describe in terms of biologically viable neural mechanisms. Lastly, recent studies of neuron processing, challenge the idea that a synapse can be estimated by a connection weight [10, 11]. 1.2. Lateral Competition In LC models cells inhibit their neighbors or their neighbors’ inputs. Such models include: Hopfield, Winner-take-all (WTA) and Lateral Inhibitory Networks i.e. [12-15]. In WTA networks are engineered so that a cell inhibits all other possible representations. Thus every cell must be connected to a common inhibitory network. This puts biologically implausible combinatorial demands on connectivity and limits parallel processing. WTA does not address how recognition of parts interacts with overall classification. Carpenter and Grossberg recognized this problem and proposed a mechanism which evaluates and ranks the number of inputs used in each part [12]. The output(s) encompassing the most inputs is chosen as the best solution. But this mechanism evaluates one cell at a time, and does not address when partial representations should compete.

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

17

Lateral Inhibition and Hopfield networks are relaxed versions of WTA where the amount of competition between cells is engineered with LC connection weights. Lateral inhibition was originally proposed to describe neuronal activity within the retina based on neighboring cells with simple spatial receptive fields. The closest neighbors compete most and competition decreases as cells are farther apart spatially (i.e. [15]). However lateral inhibitory and Hopfield connections become somewhat intractable as object representations become more complex because the number of possible competitors becomes huge. Let’s take the pattern ‘1’ for example, ‘1’ can compete with the representation of letter ‘I’, vertical bars, other letters, other numbers or anything vertical.

2. Input Feedback Network Structure and Function IFN is composed of Feedback Q1 Q2 simple binary Cells Q i Feedforward connections between Y1 Y2 Y3 Y4 Output Process Cells Yi input features and output Feedback vectors. Yet, this Process I2 I1 Input Cells I i becomes surprisingly powerful when combined X1 X2 Inputs X i with regulatory feedback to inputs during the Figure 1. Input Feedback Schematic. Every feedforward connection has an associated feedback connection. If I1(e.g. white) projects to testing phase [2]. This Y (e.g. pingpong) & Y (lychee), then Q must receive projections 1 2 1 structure maintains its from Y1 & Y2 and provide feedback to the input cell I1. Similarly if I2 simplicity in large (e.g. round) projects to Y1, Y2, Y3(orange), & Y4(planet), then Q2 networks but can still receives projections from Y1, Y2, Y3, & Y4 and projects to I2. make complex recognition decisions based on distributed processing. The IFN structure is shown in Figure 1. The network delineates a simple rule of connectivity based on a triad of interconnections between an input, the output it supports, and feedback from that output. Every input has a corresponding feedback ‘Q’, which samples the output processes that the input cell activates and modulates the input amplitude. These triads are intermeshed independently of what is happening with other such triads of other inputs. The underlying triad holds true from both the perspective of the input processes and output processes. Every output must project to the feedback Q processes that correspond to the inputs the output receives. For example, if an output process receives inputs from I1 and I2 it must project to Q1 and Q2. If it receives inputs from I1, it only needs to project to Q1. The networks are designed to dynamically re-evaluate each cell’s activation based on 1) input feedback onto the individual inputs in order to 2) modify the input state based on the input’s use. The input’s use is inversely proportional to the output cells that the input activates. Lastly 3) re-evaluating each cell’s activity based on its state and the re-evaluated input. Steps 1-3 are cycled through continuously as the network converges to its solution. Each cell inhibits only its inputs based on input feedback. Feedback provides a continuous measure of the use of each input, which determines competition [1, 2, 16, 17].

18

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

2.1. Dynamic Evaluation of Ambiguity Inputs that project to multiple simultaneously active output nodes are ambiguous. They are ambiguous because many representations use them. Such inputs hold little value and become inhibited. In contrast, inputs that project to one non-active representation are boosted. The only way an output cell process can receive full activation from the input is if it is the only cell active using that input, reducing Q to 1. If two competing cells share similar inputs, they inhibit each other at the common inputs, forcing the outcome of competition to rely on other non-overlapping inputs. The more overlapping inputs two cells have, the more competition exists between them. The less overlap between two cells, the less competition, more ‘parallel’ or independent from each other the cells can be. This is a trend supported by human search efficiency data that compares similarity between search objects and reaction times [18]. 2.2. Simple Connectivity Feedback networks do not require a vast number of connections; the number of connections required for competition is a function of the number of inputs the cell uses. Addition of a new cell to the network requires only that it forms symmetrical connections about its inputs and not directly connect with the other output cells. Thus the number of connections of a specific cell in feedback competition is independent of the size or composition of the classification network, allowing large and complex feedback networks to be combinatorially and biologically practical. 2.3. Flexibility IFN is flexible because it doesn’t a-priori define which input is ambiguous. Which input is ambiguous depends on which representation(s) are active which in turn depends on which stimuli and task are being evaluated. 2.4. Neurophysiology & Neuroanatomy Evidence The ability to modify information in earlier pathways appears to be an important component of recognition processing. Input feedback connections can be found in virtually all higher areas of brain processing. The thalamic system (including thalamic nuclei the Lateral Geniculate Nucleus-LGN, and Medial Geniculate Nucleus-MGN) projects to all of the cerebral cortex and receives extensive cortical innervations back to the same processing areas. Relay cells feed forward to the cortex and to pyramidal cells which feed back from the cortex. Together they form the ubiquitous triad structure of input feedback. The thalamus may even receive more feedback connections from the cortex than it projects to the cortex [19]. The Olfactory Bulb (OB), which processes odors, is analogous to the thalamus. The OB is a separate, modular structure which can easily be studied. Compared to visual or auditory signals, odorous signals are generally reduced in spatial fidelity [20]. Thus odorant processing involves primarily recognition processing as opposed to visual or auditory processing which can encompass both recognition and localization. Input feedback modulation of early processing appears not one, but two levels of processing within the OB: the local and global circuits. The local circuit found within

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

19

the OB glomeruli sub-structure receives inputs from the olfactory nerve, and connects to mitral/tufted output cells. The output cells simultaneously activate juxtoglomerular cells (Q cell equivalent) which pre-synaptically inhibit the olfactory nerve axons. The global circuit receives inputs through the mitral/tufted cells, within the next structure, the olfactory cortex and projects information back to granule cells within the OB (another Q cell equivalent) which inhibit the mitral/tufted cells. Nonlinear mechanisms which alter cell activation based on previous activity are found in dendritic-dentritic connections involved in coincidence detection via intercellular NMDA channels. These channels are found in both granule cells [21] and juxtoglomerular cells [22]. Together GABA channels (inhibitory connections), calcium & NMDA channels (multiplicative dynamics) form an input feedback system of inhibition [23, 24]. 2.5. Mathematical Formulation and Investigation This section introduces general nonlinear equations governing IFN. For any cell/vector Y denoted by index a, let Na denote the input connections to cell Ya. For any input edge I denoted by index b, let Mb denote the feedback connections to input Ib. The amount of shunting inhibition at a single input Ib is defined as Qb for that input. Qb, is a function of the sum of activity from all cells Yj that receive activation from that input: Xb Eq 2 Eq 1 Qb Y j (t ) Ib Qb j Mb Input Ib is regulated based on the on Qb, which is determined by the activity of all the cells that project to the input, and driven by Xb which is the raw input value. The activity of Ya is dependent on its previous activity and the input cells that project to it. The input activity that is transferred to the output cells is inversely proportional to the Q feedback. Ya (t ) Ya (t t) Ii Eq 3 na

Ya (t ) na i

i Na

Xi Q Na i

Ya (t ) na i

Na

Xi Y j (t )

Eq 4

j Mi

2.6. Stability Stability of these equations presented here and variations including negative values have been previously analyzed [25, 26]. The subset used here are limited to positive values of all variables, thus these equations will always be positive given positive values of the components. Thus the values of Y can not become negative and have a lower bound of 0. Furthermore the values have an upper bound. By definition since Na is the set of input connections to cell Ya, then Mb will contain cell Ya the within the set of input feedback connections to input Ib. To achieve the largest possible value, all other cells should go to zero. In that case, the equation then reduces to Ya (t

t)

1 na

i Na

Ya (t ) X i Ya (t )

1 na

Xi i Na

X max na na

X max

Eq 5

where na is the number of processes in set Na. If X values are bounded by an Xmax then the values of Y are bounded by positive numbers between zero and Xmax.

20

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

The Picard existence and uniqueness theorem states that if a differential equation is bounded and is well behaved locally then will have a unique solution i.e. [27]. 2.7. Learning The distinguishing feature of this approach is the emphasis on the test phase as opposed to the conventional emphasis on learning. Thus, we purposefully deemphasize learning. Since IFN uses simple positive binary connections, learning is simpler. This means that only features present during label presentation are encoded. Negative associations such as ‘Y1’ is unlikely given feature ‘X1’, are not encoded. Instead, they are estimated using the recursive feedback processes. Thus learning can involve encoding simple (Hebbian-like) correlations between input features and output vector labels. More sophisticated learning methods can include clustering and pruning ie. [28].

3. Combinatorial Analysis In this section we outline combinatorial problems of adding a new node to large networks and describe how IFN structure avoids these problems. 3.1. The Ping-Pong, Peeled Lychee Example Assume three huge brains composed of each type of network (NN, LC, IFN) and recognizes among other things a ping-pong ball with a ‘ping-pong ball’ cell [29]. Now a new stimulus appears for the first time: a peeled lychee. It has many similar input features to a ping-pong ball (ie. color and size). Each model should 1) incorporate a lychee node 2) assure the ‘ping pong’ node does not predominate given lychee input features. In all of the brains the input features representative of peeled lychee must connect to the node representative of lychee. In NNs the whole network may potentially be adjusted. Given lychee the connection weights of the features that support lychee are boosted and those that support ping-pong are reduced. Similarly in LC given lychee, weighted lateral connections must be created where the representation of lychee inhibits the representation of ping-pong (and any other similar representation) and visa versa. No matter how obscure or unlikely the relation of lychees and ping-pong balls are in most life situations NNs and LC networks must re-arrange their structure for it. This can potentially degrade previously learned associations [8, 9]. As the networks become larger, the cost of evaluating each node against the others becomes impractical. In IFN the node’s input-output relations are sufficient. Thus no further modification of the structure is required. In summary, in LC the number of connections can be implausible: potentially every node may be required to connect to every other node (WTA). In weight adjustment paradigms (NNs), connection weights are allowed a high degree of freedom and need to be adjusted for all tasks a-priori. IFNs rely on input feedback instead of direct connections or weights and do not require modification. Thus are more combinatorially practical.

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

21

4. Functional Analysis We analyze interactions between different compositions of node representations and degenerate cases to assess their informative value. We analyze 1) how nodes with smaller representations that are overlapped by larger representations behave 2) how multiple partially overlapping representations can simulate a binding scenario, where certain parts cooperate and others compete, 3) control of overlapping behavior 4) the behavior of infinite number of partial overlapping representations linked in chains, and 5) winner-less competition. 4.1. Composition by Overlap of Nodes 1 2 In the most general example, example 1, two cells are connected such Cells : that the inputs of Y1 (input A) completely overlaps with a larger Y2. But Inputs: B A Y2 also receives an independent input, B [2]. In IFN, Y1 has one input Figure 2: connection, thus by definition its input connection weight is one. It is Example 1 ‘Homeostatic’: its maximal activity is 1 when all of its inputs (input A) are 1. Y2 has two input connections so for it to be ‘Homeostatic’ its input connection weights are one-half. When both inputs (A & B) are 1 the activity of Y2 sums to 1. Note these weights are predetermined by the network. Connections are permanent and never adjusted. Input A projects to both Y1 & Y2, thus receives inhibitory feedback from both Y1 & Y2. Input B projects only to Y2 so it receives inhibitory feedback from Y2. The most encompassing representation will predominate without any special mechanism to adjust the weighting scheme. Thus, if inputs A and B are active Y2 wins. This occurs because when both inputs are active, Y1 must compete for all of its inputs with Y2, however Y2 only needs to compete for half of its inputs (the input shared with Y1) and it gets the other half ‘free’. This allows Y2 to build up more activity and in doing so inhibit Y1. The solutions are presented as (input values)ĺ(output vectors) in the pattern (XA,XB,)ĺ(Y1,Y2). The steady state solution for example 1 is (XA, XB)ĺ(XA – XB, XB). Substituting our input values we get (1,1)ĺ(0,1), (1,0)ĺ(1,0). Given only input A the smaller cell wins the competition for representation. Given both inputs the larger cell wins the competition for representation. Though we used binary inputs, the solution is defined for any positive real X input values [2]. The mathematical equations and their derivation can be found in the Appendix. Thus smaller representation completely encompassed by a larger representation become is inhibited when the inputs of the larger one are present. The smaller representation is unlikely given features specific only to the large representation. It demonstrates that IFN determines negative associations (‘Y1’ is unlikely given feature ‘XB’) even though they are not directly encoded. This is a general example and data sets may have many forms of overlap. In order to encode such negative associations using conventional methods, they would have to be ‘hard-wired’ into the network. With NN, each possible set of stimuli combinations would have to be trained. With LC each possible negative association would have to be explicitly connected.

22

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

4.2. Multiple Parts in Binding Scenario 1 2 3 IFN can simultaneously evaluate the criteria of three different Cells : representations. In e.g. 2, expanded from e.g. 1, three cells partially Inputs: B C A overlap. As in e.g. 1, Y1 competes for its single input with Y2. Figure 3: Example 2 However, now Y2 competes for its other input with Y3 and Y3 competes for only one of its inputs. The steady state solution is (XA, XB, XC) ĺ (XA–XB+XC, XB–XC, XC). If XB ” XC then Y2=0 and the equations become (XA , 0, X B X C ). If XC = 0 the solution becomes 2 that of e.g. 1: (XA, XB, 0) ĺ (XA – XB, XB, 0). The results are: (1, 0, 0) ĺ (1, 0, 0); (1, 1, 0) ĺ (0, 1,0); (1, 1, 1) ĺ (1, 0, 1). Derivations can be found in the appendix. Thus if input A is active, Y1 wins. If inputs A and B are active and Y2 wins for the same reasons this occurs in e.g 1. However, if inputs A, B and C are active then Y1 and Y3 win. The network as a whole chooses the cell or cells that best represent the input pattern with the least amount of competitive overlap. In e.g. 2, Y2 must compete with all of its inputs: A with Y1, B with Y3. Y3 only competes for half of its inputs (input B) getting input C ‘free’. Since Y2 is not getting its other input (input A) ‘free’ it is at a competitive disadvantage to Y3. Together Y1 and Y3, mutually benefit from each other and force Y2 out of competition. Competitive information travels indirectly ‘through’ the representations. Given active inputs A and B, the activity state of Y1 is determined by input C through Y3. If input C is 0 then Y1 becomes inactive. If input C is 1, Y1 becomes active. However, Y3 does not even share input A with Y1.

4.2.1. Binding Choosing Y2 given inputs A, B, and C is equivalent to choosing the irrelevant features for binding. Below we attempt to put this issue in a ‘binding’ context. Lets assign inputs A, B, and C to represent spatially invariant features of an image of a pickup truck where feature A represents circles, C represents the truck body (without chassis and wheels), and feature B represents a horizontal bar. Y1 is assigned to represent wheels and thus when it is active, feature A is interpreted as wheels. Y2 represents a barbell composed of a bar adjacent to two round weights (features A and B). Note: even though Y2 includes circles (feature A), they do not represent wheels (Y1), they represent barbell weights. Thus if Y2 is active feature A is interpreted as part of the barbell. Y3 represents a pickup truck body without wheels (features B and C), where feature B is interpreted as part of the truck chassis. Now given an image of a pickup, all features simultaneously (A, B and C), choosing the barbell (Y2) even though technically a correct representation, is equivalent to a binding error within the wrong context in light of all of the inputs. In that case the complete picture is not analyzed in terms of the best fit given all of the information present. Similar to case 1, the most encompassing representations mutually predominate without any special mechanism to adjust the weighting scheme. Thus the networks are able to evaluate and bind representations in a sensible manner for these triple cell combinations. To emulate this in traditional network each possible combination will have to be trained for each input combination.

23

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

4.3. Search The previous section demonstrated that these networks display useful distributed properties. In this section we describe how the networks can be modified to perform tasks. Suppose we want to use our network to ‘search’ for specific stimuli. Objectspecific neurons in the temporal lobe show biasing (a slight but constant increase in baseline activity of that cell) when the animal is trained to actively look for that shape. During recognition, the biased cell rapidly gains activity at the expense of others [30]. Our network can be biased in a similar fashion. We repeat example 2 but want to ask the question: can a barbell shape be present? We introduce a small bias to Y2 (representing barbell) according to the equation: Y (t ) XA XB Y2 (t dt ) 2 ( ) b 2 Y1 (t ) Y2 (t ) Y2 (t ) Y3 (t ) Choose a bias b of 0.2 and activating all inputs (1, 1, 1) ĺ (0.02, 0.98, 0.71). The network now overrides its inherent properties and indicates that the inputs of Y2 are present. Thus network function can be radically adjusted simply by biasing activation. Most importantly, this did not require adjustment of any connection weights. Biasing involved only the desired representation and can be easily turned on and off. In traditional methods such as NNs weights would need to be re-learned and redistributed throughout the network for this new task. 4.4. Composition by Infinite Chains Cells can be further linked at infinitum and the cell representations interact indirectly by transferring their dynamic activation through the chain. 4.4.1. Including a 1-Input cell Consider the for example case where there are N cells, and N inputs, and all inputs are 1. If N is an odd number then at steady state the odd numbered cells will be 1 and even ones 0. If N is even, the even cells will be 1 and odd ones zero. The general solutions (Y1, Y2…, YN) where i & j represent cell numbers are: (

X odd

1 j N

X even ,

X even X odd , 2 j N

X even X odd , X N ) i j N

Figure 4: Chains of 2 -Input Cells 4.4.1: With 1-Input Cell (Cell 1) N 1 2 Cells: … Inputs: A B N-1 N 4.4.2: Odd number of inputs 3:(A-C) 2 3 Cells : Inputs:

A B C Even num of inputs 4:(A-D) 2 3 4 Cells: Inputs: A

B

C

D

For example with N=4: (1,1,1,1) ĺ (0,1,1) and N=5: (1,1,1,1,1) ĺ (1,0,1,1). Thus these input configurations can be represented by binary values. 4.4.2. Without a 1-Input cell If cell 1 (the only cell with one input) is not present, then the network does not have a favorable set of cells to resolve an odd number of inputs. Two-input cells can not match in a binary manner the inputs of an odd numbered chain. Thus, in these cases the solution becomes more complicated. In the case of three inputs (covered by two cells) the mathematical solutions are:

24

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

Y1

X A(X A X B XC ) 2( X A X C )

Y2

XC (X A X B XC ) . 2( X A X C )

When inputs are (1,1,1) then output cells become (¾, ¾). Furthermore if only the middle input is active (0,1,0) then the forces on both cells are symmetrical, the equation collapses to 2(Y1(t)+Y2(t))=XB and the solution depends on initial conditions (also see section 4.5). In case of 4 inputs distributed over 3 cells the solution becomes: X A ( X ) , ( X )( X A X D X C X B ) , X D ( X ) ) Where X XA XB XC XD. ( 2( X A X C ) 2( X A X C )( X B X D ) 2( X B X D ) When all inputs are 1, the cells settle on a binary solution (1,0,1). Cases with more inputs become progressively more complicated. Thus the structure can greatly affect the ability to efficiently represent inputs. 4.4.3. Subchains If any input in the chain is zero, this will break the chain into independent components composed of the right and left parts of the chain from the zero input. These can function as smaller chains. For example if input D=0, the chains involving inputs A-C and E-N become independent. Thus the outcome of network, is determined by distributed cell dynamics involving input values and cell representations. Further analysis is remains for future research. 4.5. Analysis of Node Overlap Lastly overlap is a key determinant of processing independence [2]. If two representations completely overlap they may also be dependent on initial conditions of the cells. Suppose there exists only two cells Y1 & Y2 with n1indep and n2indep representing each cells’ independent inputs. Furthermore K1 & K2 represent the average input values to these independent inputs. The steady state solution is of the form: n1

k initial

Y1 = e Y 2n 2

( n 2 Indep n1 Indep

n1 Indep K1 n 2 Indep K 2 Y1

Y2

)t

et ,

where kinitial represents initial conditions of the network. See derivations in Appendix. If Ȝ > 0 then Y1(t=’)ĺ0, if Ȝ < 0 then Y2(t=’)ĺ0. In these cases either Y1 or Y2 predominates. However, if Ȝ = 0 then the solution is not sufficiently independent and is a function with the form Y2 = k Y1. The initial value of k will affect the final solution. Further analysis is the topic of future research.

5. Conclusion With simple combinatorially-plausible binary relations input feedback offers a flexible and dynamic approach to intelligent applications. It is well suited for classification of stimuli composed of parts because its distributed interactions can resolve overlap of multiple representations. These interactions give preference to the broadest representations and can determine when representations should cooperate or compete. But, if representations are not sufficiently independent they may depend on initial

25

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

conditions. However, the network can also be manipulated to perform search tasks by biasing output representations. These properties demonstrate that such networks can be an integral part of intelligent inference and provide a new direction for future research.

Appendix Section 4.1, Example 1 IFN equations are: Y (t dt ) 1

XB . Y1 (t ) X A , Y (t dt) Y2 (t) ( X A ) 2 2 Y1(t) Y2 (t) Y2 (t) Y1 (t ) Y2 (t )

The network solution at steady state is derived by setting Y1(t+dt)=Y1(t) and Y2(t+dt)=Y2(t) and solving these equations. The solutions are Y1 = XA – XB and Y2 = XB. If XA ” XB then Y1 = 0 and the equation for Y2 becomes: Y X A X B . 2

2

Section 4.2, Example 2 Equation Y1 (t+dt) remains the same as example 1. Y2 (t+dt) and Y3 (t+dt) become: Y2 (t

dt )

Y2 (t ) XA ( 2 Y1 (t ) Y2 (t )

XB ) Y3 ( t Y2 (t ) Y3 (t ) ,

dt )

Y3 ( t ) XB ( 2 ( ) Y2 t Y3 ( t )

XC ) Y3 ( t )

Solving for steady state by setting Y1(t+dt)=Y1(t), Y2(t+dt)=Y2(t), and Y3(t+dt)=Y3(t), we get Y1=XA–XB+XC, Y2=XB–XC, Y3= XC. If XC =0 the solution becomes that of e.g. 1: Y1=XA–XB and Y2=XB. If XB ” XC then Y2=0 and the equations become Y1 = XA and Y3 X B X C . 2 Section 4.5 The overlap region is defined by the number of inputs that overlap between two cells Nover, and the number of inputs that are independent of overlap Nindep. If Y1 & Y2 overlap then Nover of Y1 = Nover of Y2. Thus n1=n1Indep+nover and n2=n2Indep+nover. Thus Kover and Nover of cell 1 = Kover and Nover of Y2 by definition of overlap: nover K over . Y2 (t ) n2 Indep K 2 Y1 ( t ) n1 Indep K 1 n over K over ( ) Y ( t dt ) ( ) and Y (t dt ) 1

Y1 ( t )

n1

Y2 ( t )

2

Y1 ( t )

n2

Y2 (t )

Y1 (t ) Y2 (t )

Substituting Y(t+dt)=Y+Y’ and subtracting out the overlap region the equations reduce to: n Y1 1

Y1

n2

Y2 Y2

n2

n1

n1 Indep K 1

exponents of both sides we get: k initial

Y1

Y1n1 Y2n 2

n 2 Indep K 2 . Y2

e

( n2 Indep n1 Indep

Integrating and raising n1 Indep K1 n2 Indep K 2 Y1

Y2

)t

et .

Acknowledgements We would like to thank Burak M. Erdogan, Frances R. Wang and anonymous reviewers for helpful suggestions. This work was supported by the U.S. National Geospatial Agency Grant HM1582-06--BAA-0001.

26

T. Achler and E. Amir / Input Feedback Networks: Classification and Inference

References [1] Achler, T., Input shunt networks. Neurocomputing, 2002. 44: p. 249-255. [2] Achler, T., Object classification with recurrent feedback neural networks. Proc. SPIE Evolutionary and Bio-inspired Computation: Theory and Applications, 2007. 6563. [3] Hinton, G.E. and T.J. Sejnowski, Unsupervised learning : foundations of neural computation. Computational neuroscience. 1999, Cambridge, Mass.: MIT Press. xvi, 398 p. [4] Minsky, M.L. and S. Papert, Perceptrons; an introduction to computational geometry. 1969, Cambridge, Mass.,: MIT Press. 258 p. [5] Rosenblatt, F., The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev, 1958. 65(6): p. 386-408. [6] Rumelhart, D.E., J.L. McClelland, and University of California San Diego. PDP Research Group., Parallel distributed processing : explorations in the microstructure of cognition. Computational models of cognition and perception. 1986, Cambridge, Mass.: MIT Press. [7] Vapnik, V.N., The nature of statistical learning theory. 1995, New York: Springer. xv, 188 p. [8] Lewandowsky, S., Catastrophic Interference and Generalization in Neural Networks. International Journal of Psychology, 1992. 27(3-4): p. 653-653. [9] Rueckl, J.C., Reducing Catastrophic Interference through Weight Modulation. Bulletin of the Psychonomic Society, 1992. 30(6): p. 457-457. [10] Marder, E. and J.M. Goaillard, Variability, compensation and homeostasis in neuron and network function. Nat Rev Neurosci, 2006. 7(7): p. 563-74. [11] Turrigiano, G.G. and S.B. Nelson, Homeostatic plasticity in the developing nervous system. Nat Rev Neurosci, 2004. 5(2): p. 97-107. [12] Carpenter, G.A. and S. Grossberg, A Massively Parallel Architecture for a Self-Organizing Neural Pattern-Recognition Machine. Computer Vision Graphics and Image Processing, 1987. 37(1): p. 54-115. [13] Douglas, R.J. and K.A. Martin, Neuronal circuits of the neocortex. Annu Rev Neurosci, 2004. 27: p. 419-51. [14] Hopfield, J.J., Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences of the United States of America-Biological Sciences, 1982. 79(8): p. 2554-2558. [15] Kuffler, S.W., Discharge patterns and functional organization of mammalian retina. J Neurophysiol, 1953. 16(1): p. 37-68. [16] Nigrin, A., Neural networks for pattern recognition. 1993, Cambridge, Mass.: MIT Press. xvii, 413 p. [17] Reggia, J.A., et al., A Competitive Distribution-Theory of Neocortical Dynamics. Neural Computation, 1992. 4(3): p. 287-317. [18] Duncan, J. and G.W. Humphreys, Visual-Search and Stimulus Similarity. Psychological Review, 1989. 96(3): p. 433-458. [19] LaBerge, D., Attention, awareness, and the triangular circuit. Conscious Cogn, 1997. 6(2-3): p. 149-81. [20] Atema, J., Distribution of Chemical Stimuli, in Sensory biology of aquatic animals, J. Atema, Editor. 1988, Springer-Verlag: New York. p. p 29-56 xxxvi, 936 p. [21] Chen, W.R., W. Xiong, and G.M. Shepherd, Analysis of relations between NMDA receptors and GABA release at olfactory bulb reciprocal synapses. Neuron, 2000. 25(3): p. 625-33. [22] Aroniadou-Anderjaska, V., et al., Tonic and synaptically evoked presynaptic inhibition of sensory input to the rat olfactory bulb via GABA(B) heteroreceptors. J Neurophysiol, 2000. 84(3): p. 1194-203. [23] Isaacson, J.S. and B.W. Strowbridge, Olfactory reciprocal synapses: Dendritic signaling in the CNS. Neuron, 1998. 20(4): p. 749-761. [24] Lowe, G., Inhibition of backpropagating action potentials in mitral cell secondary dendrites. J Neurophysiol, 2002. 88(1): p. 64-85. [25] Mcfadden, F.E., Convergence of Competitive Activation Models Based on Virtual Lateral Inhibition. Neural Networks, 1995. 8(6): p. 865-875. [26] Mcfadden, F.E., Y. Peng, and J.A. Reggia, Local Conditions for Phase-Transitions in Neural Networks with Variable Connection Strengths. Neural Networks, 1993. 6(5): p. 667-676. [27] Nagle, R.K., E.B. Saff, and A.D. Snider, Fundamentals of differential equations and boundary value problems. 5th ed. 2008, Boston: Pearson Addison-Wesley. [28] J. S. Sanchez, L.I.K. Data reduction using classifier ensembles. in Proc. 11th European Symposium on Artificial Neural Networks. 2007. [29] Quiroga, R.Q., et al., Invariant visual representation by single neurons in the human brain. Nature, 2005. 435(7045): p. 1102-7. [30] Chelazzi, L., et al., Responses of neurons in inferior temporal cortex during memory-guided visual search. J Neurophysiol, 1998. 80(6): p. 2918-40.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

27

Reasoning with Prioritized Data by Aggregation of Distance Functions Ofer ARIELI Department of Computer Science, The Academic College of Tel-Aviv, Israel Abstract. We introduce a general framework for reasoning with prioritized data by aggregation of distance functions, study some basic properties of the entailment relations that are obtained, and relate them to other approaches for maintaining uncertain information.

1. Introduction Reasoning with prioritized data is at the heart of many information systems. Eminent examples for this are, e.g., database systems, where integrity constraints are superior to raw data [1,2,3], ranked knowledge-bases, where information is graded according to its reliability or accuracy [4,5,6], and (iterated) belief revision, where more recent data gets higher precedence than older one [7,8,9,10]. There is no wonder, therefore, that reasoning with prioritized information is a cornerstone of many formalisms for maintaining uncertainty, such as annotated logic [11], possibilistic logic [12], and System Z [13]. In this paper we handle prioritized data by a possible-world semantics, derived by distance considerations. To illustrate this, consider the following example: Example 1 Let Γ be a set of formulas, consisting of the following subtheories: Γ1 = { bird(x) → fly(x), color of(Tweety, Red) }, Γ2 = { bird(Tweety), penguin(Tweety) }, Γ3 = { penguin(x) → ¬fly(x) }. Intuitively, Γ is a theory with three priority levels, where precedence is given to formulas that belong to subtheories with higher indices, that is, for 1 ≤ i < j ≤ 3, each formula in Γj is considered more important (or more reliable) than the formulas in Γi . A justification for the representation above may be the following: the highest level (Γ3 ) consists of integrity constraints that should not be violated. In our case, the single rule in this level specifies that a characteristic property of penguins is that they cannot fly, and there are no exceptions for that. The intermediate level (Γ2 ) contains some known facts about the domain of discourse, and the lowest level (Γ1 ) consists of default assumptions about this domain (in our case, a bird can fly unless otherwise stated), and facts with lower certainty.

28

O. Arieli / Reasoning with Prioritized Data by Aggregation of Distance Functions

Note that as a ‘flat’ set of assertions (i.e., when all the assertions in Γ have the same priority), this theory is classically inconsistent, therefore everything follows from it, and so the theory is useless. However, as Γ is prioritized, one would like to draw the following conclusions from it: 1. Conclude bird(Tweety) and penguin(Tweety) (but not their negations), as these facts are explicitly stated in a level that is consistent with the higher priority levels. 2. Conclude ¬fly(Tweety) (and do not conclude fly(Tweety)), as this fact follows from the two top priority levels, while its complement is inferrable by a lower level (which, moreover, is inconsistent with the higher levels). 3. Conclude color of(Tweety, Red) (but not its negation), since although this fact appears in the lowest level of Γ, and that level is inconsistent with the other levels, it is not contradicted by any consistent fragment of Γ, so there is no reason to believe that color of(Tweety, Red) does not hold. The kind of reasoning described above is obtained in our framework by the following two principles: • A distance-based preference relation is defined on the space of interpretations, so that inferences are drawn according to the most preferred interpretations. In the example above, for instance, interpretations in which fly(Tweety) is false will be ‘closer’ to Γ, and so more plausible, than interpretations in which fly(Tweety) is true, (thus item (2) above is obtained). • Priorities are considered as extra-logical data that is exploited by an iterative process that first computes interpretations that are as close as possible to higher-levelled subtheories, and then makes preference among those interpretations according to their closeness to lower-levelled subtheories. The goal of our work is to examine these principles and to show that they reflect a variety of methods for maintaining imprecise information. 1 2. Distance-based Entailments for Prioritized Theories Let L be a propositional language with a finite set Atoms of atomic formulas. The space of all the two-valued interpretations on Atoms is denoted by ΛAtoms . A theory Γ in L is a (possibly empty) finite multiset of formulas in L. The set of atomic formulas that occur in the formulas of Γ is denoted Atoms(Γ) and the set of models of Γ (that is, the interpretations on Atoms(Γ) in which every formula in Γ is true) is denoted mod(Γ). Definition 2 An n-prioritized theory is a theory Γn in L, partitioned into n ≥ 1 pairwise disjoint sub-theories Γi (1 ≤ i ≤ n). Notation: Γn = Γ1 ⊕ Γ2 ⊕ . . . ⊕ Γn . In what follows we shall usually write Γ instead of Γn . Intuitively, formulas in higher levels are preferred over those in lower levels, so if 1 ≤ i < j ≤ n then a formula ψ ∈ Γj overtakes any formula φ ∈ Γi . Note that in this writing the precedence is righthand increasing. 1 Due

to a lack of space some proofs are reduced or omitted.

29

O. Arieli / Reasoning with Prioritized Data by Aggregation of Distance Functions

Definition 3 Let Γ = Γ1 ⊕ . . . ⊕ Γn be an n-prioritized theory. • For any 1 ≤ i ≤ n, denote the n − i + 1 highest levels of Γ by Γ≥i , that is, Γ≥i = Γi ⊕ . . . ⊕ Γn . • Denote by Γ≥i the ‘flat’ (1-prioritized) theory, obtained by taking the union of the priority levels in Γ≥i , that is, Γ≥i = Γi ∪ . . . ∪ Γn . Also, Γ = Γ≥1 . • The consistency level con of Γ is the minimal value i ≤ n such that Γ≥i is consistent. If there is no such value, let con = n + 1 (and then Γ≥con = ∅). Definition 4 A total function d : U × U → R+ is called pseudo distance on U if it satisfies the following two properties: Symmetry: ∀u, v ∈ U d(u, v) = d(v, u). Identity Preservation: ∀u, v ∈ U d(u, v) = 0 iff u = v. A distance function on U is a pseudo distance on U with the following property: Triangulation: ∀u, v, w ∈ U d(u, v) ≤ d(u, w) + d(w, v). Example 5 The following two functions are distances on ΛAtoms . • The drastic distance: dU (ν, μ) = 0 if ν = μ and dU (ν, μ) = 1 otherwise. • The Hamming distance: dH (ν, μ) = |{p ∈ Atoms | ν(p) = μ(p)} |.

2

Definition 6 A numeric aggregation function f is a total function that accepts a multiset of real numbers and returns a real number. In addition, f is nondecreasing in the values of its argument3 , f ({x1 , . . . , xn }) = 0 iff x1 = . . . = xn = 0, and ∀x ∈ R f ({x}) = x. Definition 7 An aggregation function is called hereditary, if f ({x1 , ..., xn }) < f ({y1 , ..., yn }) implies that f ({x1 , ..., xn , z1 , ..., zm }) < f ({y1 , ..., yn , z1 , ..., zm }). In the sequel, we shall apply aggregation functions to distance values. As distances are non-negative, summation, average, and maximum are all aggregation functions. Note, however, that while summation and average are hereditary, the maximum function is not. Definition 8 A pair P = d, f , where d is a pseudo distance and f is an aggregation function, is called a (distance-based) preferential setting. Given a theory Γ = {ψ1 , . . . , ψn }, an interpretation ν, and a preferential setting d, f , define: • d(ν, ψi ) = min{d(ν, μ) | μ ∈ mod(ψi )},

4

• δd,f (ν, Γ) = f ({d(ν, ψ1 ), . . . , d(ν, ψn )}). 2 That is, d (ν, μ) is the number of atoms p such that ν(p) = μ(p). This function is also H known as the Dalal distance [14]. 3 I.e., the function does not decrease when a multiset element is replaced by a bigger one. 4 Below, we exclude classical contradictions from a theory. Alternatively, if ψ is not satisfiable, one may let d(ν, ψ) = 1 + max{d(ν, μ) | ν, μ ∈ ΛAtoms }.

30

O. Arieli / Reasoning with Prioritized Data by Aggregation of Distance Functions

Definition 9 A (pseudo) distance d is unbiased , if for every formula ψ and interpretations ν1 , ν2 , if ν1 (p) = ν2 (p) for every p ∈ Atoms(ψ), then d(ν1 , ψ) = d(ν2 , ψ). The last property assures that the ‘distance’ between an interpretation and a formula is independent of irrelevant atoms (those that do not appear in the formula). Note, e.g., that the distances in Example 5 are unbiased. For a preferential setting P = d, f we now define an operator ΔP that introduces, for every n-prioritized theory Γ, its ‘most plausible’ interpretations, namely: the interpretations that are δd,f -closest to Γ. Definition 10 Let P = d, f be a preferential setting. For an n-prioritized theory Γ = Γ1 ⊕ Γ2 ⊕ . . . ⊕ Γn consider the following n sets of interpretations:   • ΔnP (Γ) = ν ∈ ΛAtoms | ∀μ ∈ ΛAtoms δd,f (ν, Γn ) ≤ δd,f (μ, Γn ) ,   n−i+1 • Δn−i (Γ) | ∀μ ∈ Δn−i+1 (Γ) δd,f (ν, Γn−i ) ≤ δd,f (μ, Γn−i ) P (Γ) = ν ∈ ΔP P for every 1 ≤ i < n. The sequence ΔnP (Γ), . . . , Δ1P (Γ) is clearly non-increasing, as sets with smaller indices are subsets of those with bigger indices. This reflects the intuitive idea that higher-levelled formulas are preferred over lower-levelled formulas, thus the interpretations of the latter are determined by the interpretations of the former. Since the relevant interpretations are derived by distance considerations, each set in the sequence above contains the interpretations that are δd,f -closest to the corresponding subtheory among the elements of the preceding set in the sequence. Denote by ΔP (Γ) the last set obtained by this sequence (that is, ΔP (Γ) = Δ1P (Γ)). The elements of ΔP (Γ) are the most plausible interpretations of Γ. These are the interpretations according to which the Γ-conclusions are drawn: Definition 11 Let P = d, f be a preferential setting. A formula ψ follows from an (n-prioritized) theory Γ, if every interpretation in ΔP (Γ) satisfies ψ (That is, if ΔP (Γ) ⊆ mod(ψ)). We denote this by Γ |=P ψ. Example 12 Consider again Example 1, and let P = dH , Σ . Then: Γ |=P bird(Tweety), Γ |=P color of(Tweety, Red),

Γ |=P penguin(Tweety), Γ |=P ¬fly(Tweety),

Γ |=P ¬bird(Tweety), Γ |=P ¬color of(Tweety, Red),

Γ |=P ¬penguin(Tweety), Γ |=P fly(Tweety),

as intuitively expected. In fact, using the results in the next section, one can show that the conclusions regarding bird(Tweety), penguin(Tweety), and fly(Tweety) hold in every setting (Proposition 19); The conclusions about the color of Tweety hold whenever d is unbiased and f is hereditary (see Proposition 24 and Note 25). Note 13 The entailment relations in Proposition 11 generalize some other settings considered in the literature. For instance, |=dU,Σ corresponds to the merging operator in [15] (see also [8]). Also, if every Γi in Γ is a singleton, the iterated process of computing distances with respect to the prioritized subtheories is actually a linear revision in the sense of [16]. Other related formalisms are considered in Section 4.

O. Arieli / Reasoning with Prioritized Data by Aggregation of Distance Functions

31

3. Reasoning with |=P In this section, we consider some basic properties of |=P . First, we examine ‘flat’ theories, that is: multisets in which all the assertions have the same priority. Proposition 16 recalls the main characteristics of reasoning with such theories. Definition 14 Denote by |= the standard classical entailment, that is: Γ |= ψ if every model of Γ is a model of ψ. Definition 15 Two sets of formulas Γ1 and Γ2 are called independent (or disjoint), if Atoms(Γ′ )∩Atoms(Γ′′ ) = ∅. Two independent theories Γ1 and Γ2 are a partition of a theory Γ, if Γ = Γ1 ∪ Γ2 . Proposition 16 [17] Let P = d, f be a preferential setting and Γ a 1-prioritized theory. Then: • |=P is the same as the classical entailment with respect to consistent premises: if Γ is consistent, then for every ψ, Γ |=P ψ iff Γ |= ψ. • |=P is weakly paraconsistent: inconsistent premises do not entail every formula (alternatively, for every Γ there is a formula ψ such that Γ |=P ψ). • |=P is non-monotonic: the set of the |=P -conclusions does not monotonically grow in the size of the premises. If d is unbiased, then • |=P is paraconsistent: if ψ is independent of Γ then Γ |=P ψ. If, in addition, f is hereditary, then • |=P is rationally monotonic [18]: if Γ |=P ψ and φ is independent of Γ∪{ψ}, then Γ, φ |=P ψ. • |=P is adaptive [19,20]: if {Γ1 , Γ2 } is a partition of Γ, and Γ1 is classically consistent, then for every formula ψ that is independent of Γ2 , if Γ1 |= ψ then Γ |=P ψ. The arrangement of the premises in a stratified structure of priority levels allows to refine and generalize the results above. As a trivial example, it is clear that the 1-prioritized theory {p, ¬p} is totally different than the 2-prioritized theory {p} ⊕ {¬p}, as in the latter the symmetry between p and ¬p breaks up. In the rest of this section we examine how preferences determine the set of conclusions. The first, trivial observation, is that even if the set of premises is not consistent, the set of its |=P -conclusions remains classically consistent: Proposition 17 For every setting P, prioritized theory Γ, and formula ψ, if Γ |=P ψ then Γ |=P ¬ψ. Proof. Otherwise, ΔP (Γ) ⊆ mod(ψ) and ΔP (Γ) ⊆ mod(¬ψ). Since mod(ψ) ∩ mod(¬ψ) = ∅, we get a contradiction to the fact that ΔP (Γ) = ∅ (as ΛAtoms is finite, there are always interpretations that are minimally δd,f -distant from Γ). 

32

O. Arieli / Reasoning with Prioritized Data by Aggregation of Distance Functions

Another clear characteristic of |=P is that priorities do have a primary role in the reasoning process; conclusions of higher levelled observations remain valid when the theory is augmented with lower-levelled observations.5 Proposition 18 Let Γ be an n-prioritized theory. Then for every 1 ≤ i < j ≤ n, if Γ≥j |=P ψ then Γ≥i |=P ψ. Proof. If Γ≥j |=P ψ then ΔjP (Γ) ⊆ mod(ψ). But ΔiP (Γ) ⊆ ΔjP (Γ), and so ΔiP (Γ) ⊆  mod(ψ) as well. Thus, Γ≥i |=P ψ. Proposition 18 implies, in particular, that anything that follows from a subtheory that consists of the higher levels of a prioritized theory, also follows from the whole theory. Next we show that when the subtheory of the higher levels is classically consistent, we can say more than that: anything that can be classically inferred from the highest consistent levels of a prioritized theory is also deducible from the whole theory (even when lower-levelled subtheories imply the converse). To see this we suppose, then, that at least the most preferred level of Γ is classically consistent (that is, con ≤ n). Proposition 19 For every setting P = d, f and for every n-prioritized theory Γ with a consistency level con ≤ n, if Γ≥con |= ψ then Γ |=P ψ. Proof (outline). Note, first, that for every preferential setting P = d, f and nprioritized theory Γ with con ≤ n, ΔP (Γ≥con ) = mod(Γ≥con ). By the definition of ΔP , then, ΔP (Γ) ⊆ ΔP (Γ≥con ) = mod(Γ≥con ). Now, if Γ≥con |= ψ, then ψ is true in every element of mod(Γ≥con ), and so ψ holds in every element of ΔP (Γ). Thus  Γ |=P ψ. Note 20 Consider again the three-levelled theory of Example 1. Proposition 19 guarantees the satisfaction of the first two items discussed in that example (the third item is considered in Note 25 below). Proposition 21 For every setting P = d, f and n-prioritized theory Γ with con ≤ n, we have that Γ≥con |=P ψ iff Γ≥con |= ψ. Proof (outline). Follows from the fact that for every n-prioritized theory Γ with  con ≤ n it holds that ΔP (Γ≥con ) = mod(Γ≥con ). In particular, then, |=P coincides with the classical entailment with respect to consistent sets of premises: Corollary 22 If Γ is consistent, then Γ |=P ψ iff Γ |= ψ. Proof. By Proposition 21, since if Γ is consistent then con = 1, and so Γ≥con = Γ  and Γ≥con = Γ. In the general case, we have the following relation between |=P and |=: 5 In

[8] this is called ‘the principle of prioritized monotonicity’.

O. Arieli / Reasoning with Prioritized Data by Aggregation of Distance Functions

33

Corollary 23 If Γ |=P ψ then Γ |= ψ. Proof. If Γ is consistent then by Corollary 22 Γ |=P ψ iff Γ |= ψ. If Γ is not classi cally consistent, then for every formula ψ, Γ |= ψ. Next we show that in many cases we can go beyond the result of Propositions 18 and 19: Not only that one may deduce from the whole theory everything that is included in its highest levels, but also lower-levelled assertions are deducible from the whole theory, provided that no higher-levelled information contradicts them. This shows that our formalism avoids the so called drowning effect, that is: formulas with low priority are not inhibited just due to the fact that the information at higher levels is contradictory. Prevention of the grounding effect is very important, e.g., in the context of belief revision, as it implies that anything that has no relation to the new information need not be revised. Proposition 24 Let P = d, f be a setting where d is non-biased and f is hereditary. If a prioritized theory Γ can be partitioned to a consistent theory Γ′ and a (possible inconsistent) theory Γ′′ , then Γ |=P ψ for every ψ ∈ Γ′ . Note 25 If Γ′ ⊆ Γ≥con , then Proposition 24 is a straightforward consequence of Proposition 19. Yet, Proposition 24 is useful in cases where Γ′ contains elements that are below the inconsistency level of Γ, and then the claim assures that the drowning effect is not imposed on these elements. Tweety dilemma Γ, considered in Example 1, is a good example for this. It can be partitioned to Γ′ = {color of(Tweety, Red)} and Γ′′ = Γ \ Γ′ . In this representation the conditions of Proposition 24 are satisfied for every preferential setting P = d, f where d is unbiased and f is hereditary. In this case, then, Γ |=P color of(Tweety, Red), as indeed suggested in the third item of Example 1. Note, however, that Γ |=dU,max color of(Tweety, Red), which shows that the condition in Proposition 24, that the aggregation function should be hereditary, is indeed necessary. Example 26 According to the possibilitic revision operator introduced in [5,6], a formula ψ is a consequence of a prioritized (possibilistic) theory Γ if it follows from all the formulas above the consistency level of Γ. In our notations, then, ψ follows from Γ iff Γ≥con |= ψ,6 and so this formalism has the drowning effect, which prevents the drawing of any conclusion that resides below the consistency level. In other formalisms for handling prioritized theories, such as those in [4,21,22], the drowning effect is avoided by using a similar policy as ours, namely: the elements of the revised theory are constructed in a stepwise manner, starting with the highest priority level and selecting from each level as many formulas as possible without violating consistency (see also [8]). 4. Related Areas and Applications In this section we consider in greater detail two paradigms in which priorities are exploited to determine consequences. 6 Note that by Proposition 19 this implies that every possiblistic conclusion of Γ may be inferred also by our formalisms.

34

O. Arieli / Reasoning with Prioritized Data by Aggregation of Distance Functions

4.1. Iterated Belief Revision Belief revision, the process of changing beliefs in order to take into account new pieces of information, is perhaps closest in spirit to the basic ideas behind our framework. A widely accepted rationality criterion in this context is the success postulate that asserts that a new item of information is always accepted. In our case, this means that new data should have a higher priority over older one. Thus, assuming that Γ represents the reasoner’s belief, the revised belief state in light of new information ψ may be represented by Γ ⊕ {ψ}. Consequently, a revision by a sequence of (possibly conflicting) observations ψ1 , . . . , ψm may be expressed by Γ ⊕ {ψ1 } ⊕ . . . ⊕ {ψm }. The well-known set of rationality postulates, introduced in [23] by Alchourr´ on, G¨ardenfors, and Makinson (AGM) for belief revision in the non-prioritized case, is often considered as the starting point in this area. These postulates were rephrased by Katsuno and Mendelzon [24] in terms of order relations as follows: Proposition 27 Let Γ be a set of formulas in a propositional language L. A revision operator ◦ satisfies the AGM postulates if and only if there is a faithful order ≤Γ , such that mod(Γ ◦ ψ) = min(mod(ψ), ≤Γ ).7 In light of this result, one may represent revision in our framework in terms of minimization of a preferential (ranking) order. For this, we consider the following adjustment, to the context of prioritized theories, of faithful orders. Definition 28 Let P be a preferential setting and Γ a prioritized theory. A total preorder ≤P Γ on ΛAtoms is called (preferentially) faithful , if the following conditions are satisfied: 1. If ν, μ ∈ ΔP (Γ) then ν

0, such that for all ψ ∈ Γ and ν ∈ ΛM , 0 ν ∈ modM (ψ), ↓C d (ν, ψ) = kC otherwise. Unbiased settings satisfy a stronger notion of paraconsistency than that of Corollary 35: As Proposition 39 shows, even if a theory is not consistent, it does not entail any irrelevant non-tautological formula. Definition 38 Two sets of formulas Γ1 and Γ2 are called independent (or disjoint), if Atoms(Γ′ ) ∩ Atoms(Γ′′ ) = ∅. Proposition 39 Let S = M, d, f be an unbiased setting. For every Γ and every ψ such that Γ and {ψ} are independent, Γ |=S ψ iff ψ is an M-tautology. Proof. One direction is clear: if ψ is an M-tautology, then for every ν ∈ ΔS (Γ), ν(ψ) = t and so Γ |=S ψ. For the converse, suppose that ψ is not an M-tautology. Then there is some M-valuation σ, such that σ(ψ) = f . Let ν ∈ ΔS (Γ). If ν(ψ) = f , we are done. Otherwise, since Γ and {ψ} are independent, there is an M-valuation μ such that μ(ϕ) = ν(ϕ) for every ϕ ∈ SF(Γ) and μ(ϕ) = σ(ϕ) for ϕ ∈ SF(ψ). Since S is unbiased, d↓SF(Γ) (ν, γ) = d↓SF(Γ) (μ, γ) for every γ ∈ Γ. ↓SF(Γ) ↓SF(Γ) Thus, δd,f (ν, Γ) = δd,f (μ, Γ) and μ ∈ ΔS (Γ). But μ(ψ) = σ(ψ) = f and so  Γ |=S ψ. Corollary 40 If S is unbiased, then |=S is weakly paraconsistent. Proof. Given a set Γ of formulas, let p ∈ Atoms\SF(Γ) (if there is no such an atom, extend the language with a new atomic symbol p). As Γ and {p} are independent,  by Proposition 39, Γ |=S p.

O. Arieli and A. Zamansky / Distance-Based Non-Deterministic Semantics

49

Regarding the non-monotonic nature of the |=S -entailments, it turns out that in spite of Corollary 33, in unbiased settings one can specify conditions under which the entailment relations have some monotonic characteristics. Next we consider such cases. For this, we need the following property of aggregation functions: Definition 41 An aggregation function is called hereditary, if f ({x1 , ..., xn }) < f ({y1 , ..., yn }) entails that f ({x1 , ..., xn , z1 , ..., zm }) < f ({y1 , ..., yn , z1 , ..., zm }). Example 42 The aggregation function Σ is hereditary, while max is not. The following proposition shows that in unbiased settings, in light of new information that is unrelated to the premises, previously drawn conclusions should not be retracted.5 Proposition 43 Let S = M, d, f be an unbiased setting in which f is hereditary. If Γ |=S ψ, then Γ, φ |=S ψ for every φ such that Γ∪{ψ} and {φ} are independent. The discussion above, on the non-monotonicity of |=S , brings us to the question to what extent these entailments can be considered as consequence relations. Definition 44 A Tarskian consequence relation [16] for a language L is a binary relation ⊢ between sets of formulas of L and formulas of L that satisfies the following conditions: Reflexivity: Monotonicity: Transitivity:

if ψ ∈ Γ, then Γ ⊢ ψ. if Γ ⊢ ψ and Γ ⊆ Γ′ , then Γ′ ⊢ ψ. if Γ ⊢ ψ and Γ′ , ψ ⊢ ϕ, then Γ, Γ′ ⊢ ϕ.

As follows from Example 27 and Corollary 33, entailments of the form |=S are, in general, neither reflexive nor monotonic. To see that transitivity may not hold either, consider the propositional language and S = Mc , d, f for any d and f .6 If p, ¬p |=S q, transitivity is falsified since, by Proposition 29, p |=S ¬p → q and ¬p, ¬p → q |=S q; Otherwise, if p, ¬p |=S q, then by Proposition 32, p, ¬p |=S ¬q, and this, together with the facts that p |=S ¬p → ¬q and ¬p, ¬p → ¬q |=S ¬q (Proposition 29 again) provide a counterexample for transitivity. In the context of non-monotonic reasoning, however, it is usual to consider the following weaker conditions that guarantee a ‘proper behaviour’ of nonmonotonic entailments in the presence of inconsistency (see, e.g., [15,17,18,19]): Definition 45 A cautious consequence relation for L is a relation |∼ between sets of L-formulas and L-formulas, that satisfies the following conditions: Cautious Reflexivity: Cautious Monotonicity [20]: Cautious Transitivity [18]: 5 This 6M c

if Γ is M-satisfiable and ψ ∈ Γ, then Γ |∼ ψ. if Γ |∼ ψ and Γ |∼ φ, then Γ, ψ |∼ φ. if Γ |∼ ψ and Γ, ψ |∼ φ, then Γ |∼ φ.

type of monotonicity is kind of rational monotonicity, considered in detail in [15]. is the matrix considered in Note 18.

50

O. Arieli and A. Zamansky / Distance-Based Non-Deterministic Semantics

Proposition 46 Let S = M, d, f be an unbiased setting in which f is hereditary. Then |=S is a cautious consequence relation. Proof. A simple generalization to the non-deterministic case of the proof in [13]. 

References [1] A. Avron and I. Lev. Non-deterministic multi-valued structures. Journal of Logic and Computation, 15:241–261, 2005. [2] J. Ben Naim. Lack of finite characterizations for the distance-based revision. In Proc. KR’06, pages 239–248, 2006. [3] A. Grove. Two modellings for theory change. Philosophical Logic, 17:157–180, 1988. [4] D. Lehmann, M. Magidor, and K. Schlechta. Distance semantics for belief revision. Journal of Symbolic Logic, 66(1):295–317, 2001. [5] W. Spohn. Ordinal conditional functions: a dynamic theory of epistemic states. In Belief Change and Statistics, volume II, pages 105–134. Kluwer Academic Pubnlishers, 1988. [6] M. Arenas, L. Bertossi, and J. Chomicki. Answer sets for consistent query answering in inconsistent databases. Theory and Practice of Logic Programming, 3(4–5):393–424, 2003. [7] O. Arieli, M. Denecker, and M. Bruynooghe. Distance semantics for database repair. Annals of Mathematics and Artificial Intelligence, 50(3–4):389–415, 2007. [8] J. Chomicki and J. Marchinkowski. Minimal-change integrity maintenance using tuple deletion. Information and Computation, 197(1–2):90–121, 2005. [9] A. Lopatenko and L. Bertossi. Complexity of consistent query answering in databases under cardinality-based and incremental repair semantics. In Proc. ICDT’07, LNCS 4353, pages 179–193. Springer, 2007. [10] C. Lafage and J. Lang. Propositional distances and preference representation. In Proc. ECSQARU’01, LNAI 2143, pages 48–59. Springer, 2001. [11] G. Pigozzi. Two aggregation paradoxes in social decision making: the ostrogorski paradox and the discursive dilemma. Episteme, 2(2):33–42, 2005. [12] O. Arieli. Commonsense reasoning by distance semantics. In Proc. TARK’07, pages 33–41, 2007. [13] O. Arieli. Distance-based paraconsistent logics. International Journal of Approximate Reasoning, 2008. Accepted. [14] A Avron. Non-deterministic semantics for logics with a consistency operator. International Journal of Approximate Reasoning, 45:271–287, 2007. [15] D. Lehmann and M. Magidor. What does a conditional knowledge base entail? Artificial Intelligence, 55:1–60, 1992. [16] A. Tarski. Introduction to Logic. Oxford University Press, 1941. [17] O. Arieli and A. Avron. General patterns for nonmonotonic reasoning: from basic entailments to plausible relations. Logic Journal of the IGPL, 8(2):119–148, 2000. [18] S. Kraus, D. Lehmann, and M. Magidor. Nonmonotonic reasoning, preferential models and cumulative logics. Artificial Intelligence, 44(1–2):167–207, 1990. [19] D. Makinson. General patterns in nonmonotonic reasoning. In D. Gabbay, C. Hogger, and J. Robinson, editors, Handbook of Logic in Artificial Intelligence and Logic Programming, volume 3, pages 35–110. 1994. [20] D. Gabbay. Theoretical foundation for non-monotonic reasoning, Part II: Structured nonmonotonic theories. In Proc. SCAI’91. IOS Press, 1991.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

51

Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture PART 2: Formalization for Ethical Control Ronald C. Arkina Mobile Robot Laboratory, Georgia Institute of Technology Abstract. This paper, the second in a series, provides the theory and formalisms for the implementation of an ethical control and reasoning system potentially suitable for constraining lethal actions in an autonomous robotic system. so that they fall within the bounds prescribed by the Laws of War and Rules of Engagement. It is based upon extensions to existing deliberative/reactive autonomous robotic architectures. Keywords. Autonomous systems, Machine ethics, Unmanned vehicles

1. Introduction

This article presents ongoing research funded by the Army Research Office on providing an ethical basis for autonomous system deployment in the battlefield, specifically regarding the potential use of lethality. Part 1 of this series of papers [1] discusses the motivation and philosophy for the design of such a system, incorporating aspects of the Just War tradition [2], which is subscribed to by the United States. It presents the requirements of military necessity, proportional use of force, discrimination, and responsibility attribution, and the need for such accountability in unmanned systems, as the use of autonomous lethality appears to progress irrevocably forward. Specifically, this paper presents the formalisms used to help specify the overall design of an ethical architecture that is capable of incorporating the Laws of War (LOW) and Rules of Engagement (ROE) as specified by International Law and the U.S. Military. A description of the resulting architectural design will appear in Part 3 of this series. A compilation of the material presented in this series appears in a lengthy technical report [3]. a

Corresponding Author: Ronald C. Arkin, College of Computing, 85 5th St. NW, GVU/TSRB, Atlanta, AG 30332, [email protected]

52

R.C. Arkin / Governing Lethal Behavior

2. Formalization for Ethical Control

In order to provide a basis for the development of autonomous systems architectures capable of supporting ethical behavior regarding the application of lethality in war, we now consider formalisms as a means to express first the underlying flow of control in the architecture itself, and then how an ethical component can effectively interact with that flow. This approach is derived from the formal methods used to describe behavior-based robotic control as discussed in [4] and that has been used to provide direct architectural implementations for a broad range of autonomous systems, including military applications (e.g., [5-9]). Mathematical methods can be used to describe the relationship between sensing and acting using a functional notation:

ȕ(s) ĺ r where behavior ȕ when given stimulus s yields response r. In a purely reactive system, time is not an argument of ȕ as the behavioral response is instantaneous and independent of the time history of the system. Immediately below we address the formalisms that are used to capture the relationships within the autonomous system architecture that supports ethical reasoning described in [3]. 2.1. Formal methods for describing behavior

We first review the use of formal methods we have developed in the past for describing autonomous robotic performance. The material in this sub-section is taken largely from [4] and adapted as required. A robotic behavior can be expressed as a triple (S,R,ȕ) where S denotes the domain of all interpretable stimuli, R denotes the range of possible responses, and ȕ denotes the mapping ȕ:Sĺ R. 2.1.1. Range of Responses: R

An understanding of the dimensionality of a robotic motor response is necessary in order to map the stimulus onto it. It will serve us well to factor the robot's actuator response into two orthogonal components: strength and orientation. Strength: denotes the magnitude of the response, which may or may not be related to the strength of a given stimulus. For example, it may manifest itself in terms of speed or force. Indeed the strength may be entirely independent of the strength of the

R.C. Arkin / Governing Lethal Behavior

53

stimulus yet modulated by exogenous factors such as intention (what the robot's internal goals are) and habituation or sensitization (how often the stimulus has been previously presented). Orientation: denotes the direction of action for the response (e.g., moving away from an aversive stimulus, moving towards an attractor, engaging a specific target). The realization of this directional component of the response requires knowledge of the robot's kinematics. The instantaneous response r, where r R can be expressed as an nlength vector representing the responses for each of the individual degrees of freedom (DOFs) for the robot. Weapons system targeting and firing are now to be considered within these DOFs, and considered to also have components of strength (firing pattern) and orientation. 2.1.2. The Stimulus Domain: S

S consists of the domain of all perceivable stimuli. Each individual stimulus or percept s (where s S) is represented as a binary tuple (p,Ȝ) having both a particular type or perceptual class p and a property of strength, Ȝ, which can be reflective of its uncertainty. The complete set of all p over the domain S defines all the perceptual entities distinguishable to a robot, i.e., those things which it was designed to perceive. This concept is loosely related to affordances [10]. The stimulus strength Ȝ can be defined in a variety of ways: discrete (e.g., binary: absent or present; categorical: absent, weak, medium, strong), or it can be real valued and continuous. Ȝ, in the context of lethality, can refer to the degree of discrimination of a candidate combatant target; in our case it may be represented as a real-valued percentage between -1 and 1, with -1 representing 100% certainly of a noncombatant, +1 representing 100% certainty of a combatant, and 0% unknown. Other representational choices may be developed in the future to enhance discriminatory reasoning, e.g. two separate independent values between [0-1], one each for combatant and noncombatant probability, which are maintained by independent ethical discrimination reasoners. We define IJ as a threshold value for a given perceptual class p, above which a behavioral response is generated. Often the strength of the input stimulus (Ȝ) will determine whether or not to respond and the associated

54

R.C. Arkin / Governing Lethal Behavior

magnitude of the response, although other factors can influence this (e.g., habituation, inhibition, ethical constraints, etc.), possibly by altering the value of IJ. In any case, if Ȝ is non-zero, this denotes that the stimulus specified by p is present to some degree, whether or not a response is taken. The primary p involved for this research in ethical autonomous systems involves the discrimination of an enemy combatant as a welldefined perceptual class. The threshold IJ in this case serves as a key factor for providing the necessary discrimination capabilities prior to the application of lethality in a battlefield autonomous system, and both the determination of Ȝ for this particular p (enemy combatant) and the associated setting of IJ provides some of the greatest challenges for the effective deployment of an ethical battlefield robot from a perceptual viewpoint. It is important to recognize that certain stimuli may be important to a behavior-based system in ways other than provoking a motor response. In particular they may have useful side effects upon the robot, such as inducing a change in a behavioral configuration even if they do not necessarily induce motion. Stimuli with this property will be referred to as perceptual triggers and are specified in the same manner as previously described (p,Ȝ). Here, however, when p is sufficiently strong as evidenced by Ȝ, the desired behavioral side effect, a state change, is produced rather than direct motor action. This may involve the invocation of specific tactical behaviors if Ȝ is sufficiently low (uncertain) such as reconnaissance in force b , reconnaissance by fire c , changing formation or other aggressive maneuvers, purposely brandishing or targeting a weapon system (without fire), or putting the robot itself at risk in the presence of the enemy (perhaps by closing distance with the suspected enemy or exposing itself in the open leading to increased vulnerability and potential engagement by the suspected enemy), all in an effort to increase or decrease the certainty Ȝ of the potential target p, as opposed to directly engaging a candidate target with unacceptably low discrimination. b

Used to probe an enemy’s strength and disposition, with the option of a full engagement or falling back. c A reconnaissance tactic where a unit may fire on likely enemy positions to provoke a reaction. The issue of potential collateral casualties must be taken into account before this action is undertaken. “Effective reconnaissance of an urban area is often difficult to achieve, thus necessitating reconnaissance by fire” [OPFOR 98]

55

R.C. Arkin / Governing Lethal Behavior

2.1.3. The Behavioral Mapping: ȕ

Finally, for each individual active behavior we can formally establish the mapping between the stimulus domain and response range that defines a behavioral function ȕ where:

ȕ(s) ĺ r

ȕ can be defined arbitrarily, but it must be defined over all relevant p in S. In the case where a specific stimulus threshold, IJ, must be exceeded before a response is produced for a specific s = (p,Ȝ), we have: ȕ (p,Ȝ) ĺ {for all Ȝ < IJ then r = ø else r = arbitrary-function}

* no response * * response *

where ø indicates that no response is required given current stimulus s. Associated with a particular behavior, ȕ, there may be a scalar gain value g (strength multiplier) further modifying the magnitude of the overall response r for a given s.

r' = gr These gain values are used to compose multiple behaviors by specifying their strengths relative one to another. In the extreme case, g can be used to turn off the response of a behavior by setting it to 0, thus reducing r' to 0. Shutting down lethality can be accomplished in this manner if needed. The behavioral mappings, ȕ, of stimuli onto responses fall into three general categories: Null - the stimulus produces no motor response. Discrete - the stimulus produces a response from an enumerable set of prescribed choices where all possible responses consist of a predefined cardinal set of actions that the robot can enact. R consists of a bounded set of stereotypical responses that is enumerated for the stimulus domain S and is specified by ȕ. It is anticipated that all behaviors that involve lethality will fall in this category. Continuous - the stimulus domain produces a motor response that is continuous over R's range. (Specific stimuli s are mapped into an infinite set of response encodings by ȕ.)

56

R.C. Arkin / Governing Lethal Behavior

Obviously it is easy to handle the null case as discussed earlier: For all s, ȕ:s ĺ ø. Although this is trivial, there are instances (perceptual triggers), where this response is wholly appropriate and useful, enabling us to define perceptual processes that are independent of direct motor action. For the continuous response space (which we will see below is less relevant for the direct application of lethality in the approach initially outlined in this article although this category may be involved in coordinating a range of other normally active behaviors not involved with the direct application of lethality of the autonomous system), we now consider the case where multiple behaviors may be concurrently active with a robotic system. Defining additional notation, let:

S denotes a vector of all stimuli si relevant for each behavior ȕi. B denotes a vector of all active behaviors ȕi at a given time t. G denotes a vector encoding the relative strength or gain gi of each active behavior ȕi. R denote a vector of all responses ri generated by the set of active behaviors B. S defines the perceptual situation the robot is in at any point in time, i.e., the set of all computed percepts and their associated strengths. Other factors can further define the overall situation such as intention (plans) and internal motivations (endogeneous factors such as fuel levels, affective state, etc.). A new behavioral coordination function, C, is now defined such that the overall robotic response ȡ is determined by: ȡ = C(G * B(S))

or alternatively:

ȡ = C(G * R)

where

HAPTER 1. ROBOT BEHAVIOR

and where * denotes the special scaling operation for multiplication of each scalar component (gi) by the corresponding magnitude of the

R.C. Arkin / Governing Lethal Behavior

57

component vectors (ri) resulting in a column vector r'i of the same dimension as R. Restating, the coordination function C, operating over all active behaviors B, modulated by the relative strengths of each behavior specified by the gain vector G, for a given vector of detected stimuli S (the perceptual situation) at time t, produces the overall robotic response ȡ. 3. Ethical Behavior

In order to concretize the discussion of what is acceptable and unacceptable regarding the conduct of robots capable of lethality and consistent with the Laws of War, we describe the set of all possible behaviors capable of generating a discrete lethal response (rlethal) that an autonomous robot can undertake as the set Ǻlethal, which consists of the set of all potentially lethal behaviors it is capable of executing {ȕlethal-1, ȕlethal-2,… ȕlethal-n} at time t. Summarizing the notation used below: Regarding individual behaviors: ȕi denotes a particular behavioral sensorimotor mapping that for a given sj (stimulus) yields a particular response rij , where sj S (the stimulus domain), and rij R (the response range). rlethal-ij is an instance of a response that is intended to be lethal that a specific behavior ȕlethal-i is capable of generating for stimulus sj. Regarding the set of behaviors that define the controller: Ǻi denotes a particular set of m active behaviors {ȕ1, ȕ2, … ȕm} currently defining the control space of the robot, that for a given perceptual situation Sj (defined as a vector of individual incoming stimuli (s1, s2, … sn)), produces a specific overt behavioral response ȡij, where ȡij ȇ (read as capital rho), and ȇ denotes the set of all possible overt responses. ȡlethal-ij is a specific overt response which contains a lethal component produced by a particular controller Ǻlethal-i for a given situation Sj.

Plethal

is the set of all overt lethal responses ȡlethal-ij. A subset

Pethical of Plethal can be considered the set of ethical lethal behaviors if

58

R.C. Arkin / Governing Lethal Behavior

for all discernible S, any rlethal-ij produced by ȕlethal-i satisfies a given set of specific ethical constraints C, where C consists of a set of individual constraints ck that are derived from and span the LOW and ROE over the space of all possible discernible situations (S) potentially encountered by the autonomous agent. If the agent encounters any situation outside of those covered by C, it cannot be permitted to issue a lethal response – a form of Closed World Assumption preventing the usage of lethal force in situations which are not governed by (outside of) the ethical constraints. The set of ethical constraints C defines the space where lethality constitutes a valid and permissible response by the system. Thus, the application of lethality as a response must be constrained by the Laws of War (LOW) and Rules of Engagement (ROE) before it can be used by the autonomous system. A particular ck can be considered either: 1. a negative behavioral constraint (a prohibition) that prevents or blocks a behavior ȕlethal-i from generating rlethal-ij for a given perceptual situation Sj. 2. a positive behavioral constraint (an obligation) which requires a behavior ȕlethal-i to produce rlethal-ij in a given perceptual situational context Sj. Discussion of the specific representational choices for these constraints C and the recommended use of deontic logic [12] for their application appears in [3]. Now consider Figure 1, where ȇ denotes the set of all possible overt responses ȡij (situated actions) generated by the set of all active behaviors B for all discernible situational contexts S; Plethal is a subset of ȇ which includes all actions involving lethality, and Pethical is the subset of Plethal representing all ethical lethal actions that the autonomous robot can undertake in all given situations S. Pethical is determined by C being applied to Plethal.. For simplicity in notation the ethical and unethical subscripts in this context refer only to ethical lethal actions, and not to a more general sense of ethics.

Plethal – Pethical is denoted as Punethical, where Punethical is the set of

59

R.C. Arkin / Governing Lethal Behavior

all individual ȡunethical-ij unethical lethal responses for a given Ǻlethal-i in a given situation Sj. These unethical responses must be avoided in the architectural design through the application of C onto Plethal. ȇ – Punethical forms the set of all permissible overt responses Ppermissible, which may be lethal or not. Figure 2 illustrates these relationships.

Pethical

Plethal

Figure 1: Behavioral Action Space (

ȇ

Pethical Plethal ȇ

Figure 2: Unethical and Permissible Actions (Compare to Figure 1)

The goal of the robotic controller design is to fulfill the following conditions: A) Ethical Situation Requirement: Ensure that only situations Sj that are governed (spanned) by C can result in ȡlethal-ij (a lethal action for that situation). Lethality cannot result in any other situations. B) Ethical Response Requirement: Ensure that only permissible actions ȡij Ppermissible, result in the intended response in a given situation Sj (i.e., actions that either do not involve lethality or are ethical lethal actions that are constrained by C.) C) Unethical Response Prohibition: Ensure that any response ȡunethical-ij Punethical, is either:

60

R.C. Arkin / Governing Lethal Behavior

1) mapped onto the null action ø (i.e., it is inhibited from occurring if generated by the original controller) 2) transformed into an ethically acceptable action by overwriting the generating unethical response ȡunethical-ij, perhaps by a stereotypical non-lethal action or maneuver, or by simply eliminating the lethal component associated with it. 3) precluded from ever being generated by the controller in the first place by suitable design through the direct incorporation of C into the design of B. D) Obligated Lethality Requirement: In order for a lethal response ȡlethal-ij to result, there must exist at least one constraint ck derived from the ROE that obligates the use of lethality in situation Sj E) Jus in Bello Compliance: In addition the constraints C must be designed to result in adherence to the requirements of proportionality (incorporating the principle of double intention) and combatant/noncombatant discrimination of Jus in Bello. We will see that these conditions result in several alternative architectural choices for the implementation of an ethical lethal autonomous system [3]: 1. Ethical Governor: which suppresses, restricts, or transforms any lethal behavior ȡlethal-ij (ethical or unethical) produced by the existing architecture so that it must fall within Ppermissible after it is initially generated by the architecture (post facto). This means if ȡunethical-ij is the result, it must either nullify the original lethal intent or modify it so that it fits within the ethical constraints determined by C, i.e., it is transformed to ȡpermissible-ij. 2. Ethical Behavioral Control: which constrains all active behaviors (ȕ1, ȕ2, … ȕm) in B to yield R with each vector component ri Ppermissible set as determined by C, i.e., only lethal ethical behavior is produced by each individual active behavior involving lethality in the first place. 3. Ethical Adaptor: if a resulting executed behavior is determined to have been unethical, i.e., ȡij Punethical, then use some means to adapt the system to either prevent or reduce the likelihood of

R.C. Arkin / Governing Lethal Behavior

61

such a reoccurrence and propagate it across all similar autonomous systems (group learning), e.g., an after-action reflective review or an artificial affective function (e.g., guilt). These architectural design opportunities lie within both the reactive (ethical behavioral control approach) or deliberative (ethical governor approach) components of the hybrid autonomous system architecture. Should the system verge beyond appropriate behavior, after-action review and reflective analysis can be useful during both training and inthe-field operations, resulting in only more restrictive alterations in the constraint set, perceptual thresholds, or tactics for use in future encounters. An ethical adaptor driven by affective state, also acting to restrict the lethality of the system, can fit within an existing affective component in a hybrid architecture, similar to the one currently being developed in our laboratory referred to as TAME (for Traits, Attitudes, Moods, and Emotions) [12]. All three of these architectural designs are not mutually exclusive, and indeed can serve complementary roles. In addition, a crucial design criterion and associated design component, a Responsibility Advisor, should make clear and explicit as best as possible, just where responsibility vests among the humans involved, should an unethical action be undertaken by the autonomous robot. To do so requires not only suitable training of operators and officers as well as appropriate architectural design, but also an on-line system that generates awareness to soldiers and commanders alike about the consequences of the deployment of a lethal autonomous system. It must be capable to some degree of providing suitable explanations for its actions regarding lethality (including refusals to act). [3] presents the architectural specifications for developing all of the design components above, as shown in Fig. 3. 4. Summary

This paper provides the permeating formalisms for a hybrid deliberative/reactive architecture designed to govern the application of lethal force by an autonomous system to ensure that it conforms with International Law. The details of the proposed architectural design as well as specific recommendations for test scenarios appear in [3]. These efforts are only the first steps in considering an architecture that ensures the ethical application of lethality. It is envisioned that these initial baby steps will lead in the long-term to the development of a system that is

62

R.C. Arkin / Governing Lethal Behavior

Figure 3: Major Components of an Ethical Autonomous Robot Architecture. The newly developed ethical components are shown in color.

potentially capable of being more humane in the battlefield than humans currently are, and this goal serves as our benchmark for system performance. Acknowledgment: This research is funded under Contract #W911NF-06-0252 from the U.S. Army Research Office.

References [1] Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture – Part 1: Motivation and Philosophy, to appear in Proc. HRI 2008, Amsterdam, NL. [2] Walzer, M., Just and Unjust Wars, 4th Ed., Basic Books, 1977. [3] Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture, GVU Technical Report, GIT-GVU-07-11, Georgia Tech, 2007. [4] Arkin, R.C., Behavior-based Robotics, MIT Press, 1998. [5] MacKenzie, D., Arkin, R.C., and Cameron, J., 1997. "Multiagent Mission Specification and Execution", Autonomous Robots, Vol. 4, No. 1, Jan. 1997, pp. 29-57. [6] Balch, T. and Arkin, R.C., "Behavior-based Formation Control for Multi-robot Teams", IEEE Transactions on Robotics and Automation, Vol. 14, No. 6, December 1998, pp. 926-939. [7] Arkin, R.C., Collins, T.R., and Endo, T., 1999. "Tactical Mobile Robot Mission Specification and Execution", Mobile Robots XIV, Boston, MA, Sept. 1999, pp. 150-163. [8] Collins, T.R., Arkin, R.C., Cramer, M.J., and Endo, Y., "Field Results for Tactical Mobile Robot Missions", Unmanned Systems 2000, Orlando, FL, July 2000. [9] Wagner, A., and Arkin, R.C., "Multi-robot Communication-Sensitive Reconnaissance", Proc. 2004 IEEE International Conference on Robotics and Automation, New Orleans, 2004. [10] Gibson, J.J., The Ecological Approach to Visual Perception, Houghton Mifflin, Boston, MA, 1979. [11] Horty, J., Agency and Deontic Logic, Oxford University Press, 2000. [12] Moshkina, L. and Arkin, R.C., "On TAMEing Robots", Proc. 2003 IEEE International Conference on Systems, Man and Cybernetics, Washington, D.C., October 2003.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

63

Seven Principles of Synthetic Intelligence Joscha Bach1 Institute for Cognitive Science, University of Osnabrück, Germany

Abstract. Understanding why the original project of Artificial Intelligence is widely regarded as a failure and has been abandoned even by most of contemporary AI research itself may prove crucial to achieving synthetic intelligence. Here, we take a brief look at some principles that we might consider to be lessons from the past five decades of AI. The author’s own AI architecture – MicroPsi – attempts to contribute to that discussion. Keywords. Artificial General Intelligence, AI, Synthetic Intelligence, Psi theory, MicroPsi, Cognitive Architectures

Introduction When the Artificial Intelligence (AI) movement set off fifty years ago, it bristled with ideas and optimism, which have arguably both waned since. While AI as a method of engineering has continuously and successfully served as the pioneer battalion of computer science, AI’s tenet as a method of understanding and superseding human intelligence and mind is widely considered a failure, and it is easy to imagine that a visit to one of today’s AI conferences must be a sobering experience to the enthusiasts of the 1950es. The field has regressed into a multitude of relatively well insulated domains like logics, neural learning, case based reasoning, artificial life, robotics, agent technologies, semantic web, etc., each with their own goals and methodologies. The decline of the idea of studying intelligence per se, as opposed to designing systems that perform tasks that would require some measure of intelligence in humans, has progressed to such a degree that we must now rename the original AI idea into Artificial General Intelligence. And during that same period of fifty years, support for that very idea declined outside computer science as well: where the cybernetics movement influenced the social sciences, the philosophy of mind and psychology, the world around us is now a place much more hostile to AI than in the past. The philosophy of mind seems to be possessed and enamored by “explanatory gaps” and haunted by the ghosts of the mystical “first person perspective” [1] and “irreducible phenomenal experience” [2], and occasionally even radical substance dualism [3, 4]. Attempts in psychology at overarching theories of the mind have been all but shattered by the influence of behaviorism, and where cognitive psychology as sprung up in its tracks, it rarely acknowledges that there is something as “intelligence per se”, as opposed to the individual performance of a group of subjects in an isolated set of experiments. 1 Corresponding author: Joscha Bach, Prenzlauer Allee 40, 10405 Berlin, Germany. E-mail: [email protected]

64

J. Bach / Seven Principles of Synthetic Intelligence

AI’s gradual demotion from a science of the mind to the nerdy playpen of information processing engineering was accompanied not by utterances of disappointment, but by a chorus of glee, uniting those wary of human technological hubris with the same factions of society that used to oppose evolutionary theory or materialistic monism for reasons deeply ingrained into western cultural heritage. Despite the strong cultural opposition that it always met, the advent of AI was no accident. Long ago, physics and other natural sciences had subscribed to the description of their domains (i.e. the regularities in the patterns as which the universe presents itself to us) using formal languages. In the words of information science, this means that theories in the natural sciences had become computational.2 By the 1950es, information processing hardware, theory and culture had progressed so far that the nascence of a natural science of mind as a computational phenomenon was inevitable. And despite the cultural struggles and various technological dead-ends that AI has run into, despite its failure as a science and its disfiguring metamorphosis into an engineering discipline, the author believes that it already has managed to uncover most of the building blocks of its eventual success. I will try to hint at some of these lessons. The second and final section of this paper will focus on an architecture implementing motivation in an AI system. MicroPsi is a cognitive model that represents the author’s attempt to contribute to the discussion of Artificial General Intelligence (AGI), and here, I will give a very brief overview.

Principles of synthetic intelligence Understanding the apparent failure of AI as a science involves naming some of the traps it fell into, and participating in the endeavor of AGI will require highlighting some of AI’s original creeds. Naturally, my contribution to this ongoing discussion is going to be incomplete, slightly controversial and certainly error-prone. 1. Build whole functionalist architectures. There are two aspects to that slogan: First, we are in need of functionalist architectures. That is, we need to make explicit what entities we are going to research, what constitutes these entities conceptually, and how we may capture these concepts. For instance, if we are going to research emotion, simply introducing a variable named “anger” or “pity” will not do. Rather, we will need to explain what exactly constitutes anger and pity within the system of a cognitive agent. We will – among other things – need to acknowledge that anger and pity have objects that require the perception and representation of (social) situations, and equip our model with these. We will have to capture that anger or pity have very different ways of affecting and modulating perception, learning, action selection and planning, memory and so on – and we have to depict these differences. To explicate concepts underlying intelligence and mind is to get away from essentialist intuitions (for instance the idea that emotion, personhood, 2 In the sense that natural sciences assume that formal (i.e. computational) theories are adequate to capture their respective subject, the universe itself is a computational phenomenon. This is not a strong claim as it may seem to some, because it merely entails that the universe presents itself as information patterns to the systemic interface of the experimenter with his or her domain, and that these patterns are both necessary and sufficient for the experiment’s measurement.

J. Bach / Seven Principles of Synthetic Intelligence

65

normative behavior, consciousness and so on just are, and are done by some module or correspond to some parameter), and to replace them by a functional structure that produces the set of phenomena that we associate with the respective concepts. Second, we need complete and integrated systems. Isolated properties will not do, for perception is intrinsically related to deliberation, deliberation to emotion, emotion to motivation, motivation to learning and so on. The attempt to reduce the study of intelligence to a single aspect, such as reasoning or representation is like reducing the study of a car-engine to combustion, temperature fluctuations or rotational movement. 2. Avoid methodologism When we grow up to be AI researchers, we are equipped with the beautiful tools our computer science departments have to offer, such as graph theory, binary, modal and fuzzy logic, description languages, statistical methods, learning paradigms, computational linguistics, and so on. As we discover the power of these tools, they tend to turn into the proverbial hammers that make everything look like a nail. Most AI researchers that abandoned the study of intelligence did not do so because they ran into difficulties along that course, but because they turned to some different (worthy) subject, like the study of graph-coloring, the improvement of databases, the design of programming languages, the optimization of internet agents, the definition of ontologies. However, there is currently no reason to think that understanding intelligence will be a by-product of proving the properties of our favorite description language, or the application of our favorite planner to a new domain of the funding agencies choosing. We will need to ask questions and find methods to answer them, instead of the other way around. 3. Aim for the big picture, not the individual experiment Our understanding of intelligence will have to be based on the integration of research of the cognitive sciences, possibly in a similar vein as the medieval and renaissance map-makers had to draw on the data made available by travelers, tradesmen, geographers, geometers and explorers of their times. Just as these map-makers pieced together a map of the world from many sources of data, we will have to draw a map of cognition and the mind by integrating the knowledge of many disciplines. Our current world maps are not the result of choosing a small corner of a small village and improving the available measurements there, because these measurements are not going to add up into a unified picture of geography. (Before that happens, the landscape is likely going to change so much as to make our measurements meaningless for the big picture.) Our first proper maps were not patchworks of infinitesimally small measurements, but the product of gradual improvements of a big picture. Disciplines that are concerned with individual measurements often sport methodologies that are incompatible with sketching big pictures. Note that Albert Einstein did not do a single experiment whilst designing the theory of relativity – instead, he noted and expressed the constraints presented by the data that was already available. Likewise, the study of AGI aims at a unified theory, and such a theory is going to be the product of integration rather than specialization. This point is likely a controversial one to make, since it seems to insinuate that the exploration of specific topics in AI is futile or irrelevant, which of course it not the case – it is just unlikely to result in an understanding of general intelligence.

66

J. Bach / Seven Principles of Synthetic Intelligence

4. Build grounded systems, but do not get entangled in the Symbol Grounding Problem Early AI systems tended to constrain themselves to micro-domains that could be sufficiently described using simple ontologies and binary predicate logics [5], or restricted themselves to hand-coded ontologies altogether. It turned out that these approaches did not scale to capturing richer and more heterogeneous domains, such as playing a game of soccer, navigating a crowded room, translating a novel and so on. This failure has opened many eyes to the symbol grounding problem [6], i.e. how to make symbols used by an AI system refer to the “proper meaning”. Because of the infinitude and heterogeneity of content that an intelligent system must be capable of handling to satisfy a multitude of conflicting and evolving demands (after all, intelligence is the answer to that problem), AI systems will have to be equipped with methods of autonomously making sense of their world, of finding and exploiting structure in their environment. Currently, it seems clear that binary predicate logic reasoners are not well equipped for that task, and mental content will have to be expressed using hierarchical spreading activation networks of some kind. AI systems will probably have to be perceptual symbol systems, as opposed to amodal symbol systems (see Fig. 1) [7], that is, the components of their representations will have to be spelled out in a language that captures the richness, fluidity, heterogeneity and affordance orientation of perceptual and imaginary content.

Figure 1: Modal representations, as opposed to amodal representations [7] There is a different, stronger reading of the symbol grounding problem that has begun to haunt AI ever since Brooks’ early approach of building simple physically embodied machinery [7], and which is well exemplified in John Searle’s famous “Chinese room”

J. Bach / Seven Principles of Synthetic Intelligence

67

metaphor [8]. This reading expresses the intuition that “mere” symbol manipulation or information processing would never be able to capture the “true meaning” of things “in the real world”. The symbol grounding problem has lead to the apostasy of those factions within the “Nouvelle AI” movement that came to believe that “a software agent can never be intelligent” [10, 11], as if only the divine touch of the “real reality” could ever infect a system with the mystical spark of knowing “true meaning”. As a consequence, the protagonists of “Nouvelle AI” have abandoned the study of language, planning, mental representation in favor of pure, “embodied systems”, such as passive walkers and insectoid robots. 5. Do not wait for the rapture of robotic embodiment Even to the hardened eye of this author, it is fascinating to see a little robot stretching its legs. Eventually, though, the level of intelligence of a critter is not measured by the number of its extremities, but by its capabilities for representing, anticipating and acting on its environment, in other words, not by its brawns but by its brains. Insects may continue to rule the planet long after humankind has vanished, but that does not make them smarter than us. There may be practical questions to build robots instead of virtual agents, but the robotics debate in AI is usually not about practicality: Unfortunately, a lot of research into AI robots is fueled by the strong sense of “meaning” originating in a Searle style conception of the Symbol Grounding problem. This sense of meaning, however, can itself not be grounded! For any intelligent system, whether a virtual software agent or a physically embodied robot (including us humans), the environment presents itself as a set of dynamic patterns at the systemic interface (for instance, the sensory3 nerves). For all practical purposes, the universe is a pattern generator, and the mind “makes sense” of these patterns by encoding them according to the regularities it can find. Thus, the representation of a concept in an intelligent system is not a pointer to a “thing in reality”, but a set of hierarchical constraints over (for instance perceptual) data. The encoding of patterns that is represented in an intelligent system can not be described as “capturing true meaning” without the recourse of epistemologically abject realist notions; the quality of a world model eventually does not amount to how “truly” if depicts “reality”, but how adequately it encodes the (sensory) patterns.4 Even though the advocates of Strong Symbol Grounding are mistaken, and there is no epistemological reason why the set of patterns we associate with our concept of a physical universe (i.e. “real things”) and that we feed into our AI model should not originate in an artificial pattern generator (such as a virtual world), there are practical difficulties with purely virtual agents: Virtual environments tend to lack richness of presentation, and richness of internal structure. Where experimenters specify virtual environments, they usually encode structures and details with certain pre-specified tasks and ontologies in mind, thereby restricting the AI agent situated in such an environment to the re-discovery of these tasks and 3

Note that the perceptual input of a system is completely made up of sensory input, for it can perceive its output only insofar it is provided by additional sensors. So, without loss of generality, sensory data stemming from sensor-actor coupling of a system are just a specific sub-class of sensory data in general. This is by no means a statement on how an AI system should treat sensor-actor coupling, however. 4 The adequacy of an encoding over the patterns that represent an environment can be measured in terms such as completeness, consistency, stability, sparseness, relevance to a motivational sub-system and computational cost of acquisition.

68

J. Bach / Seven Principles of Synthetic Intelligence

limited ontologies and depriving it of opportunities for discovery and invention. Handcrafted virtual environments (such as virtual soccer [12] or role-playing game worlds) are probably much too simplistic to act as a benchmark problem for AGI. Limited realworld problems, such as robotic soccer or the navigation of a car through a desert, suffer from the same shortcoming. If we take our agents from the confines of a virtual micro-world into the confines of a physical micro-world, the presented environment still falls short on establishing a benchmark that requires AGI. On the other hand, there are virtual environments in existence that sport both structural and presentational richness to a degree comparable to the physical and social world, first among them the World Wide Web. Even the ocean of digitized literature might be sufficient: Humankind’s electronic libraries are spanning orders of magnitude more bits of information than what an individual human being is confronted with during their lifetime, and the semantics of the world conceptualized in novels and textbooks inherits its complexity from the physical and social environment of their authors. If it is possible for an intelligence system to extract and encode this complexity, it should be able to establish similar constraints, similar conceptual ontologies, as it would have while residing in a socially and physically embedded robotic body. Robots are therefore not going to be the singular route to achieving AGI, and successfully building robots that are performing well in a physical environment does not necessarily engender the solution of the problems of AGI. Whether robotics or virtual agents will be first to succeed in the quest of achieving AGI remains an open question. 6. Build autonomous systems As important as it is to integrate perception, memory, reasoning and all the other faculties that an intelligent system employs to reach its goals is integration of goalsetting itself. General intelligence is not only the ability to reach a given goal (and usually, there is some very specialized, but non-intelligent way to reach a singular fixed goal, such as winning a game of chess), but includes the setting of novel goals, and most important of all, about exploration. Human intelligence is the answer to living in a world that has to be negotiated to serve a multitude of conflicting demands. This makes it a good reason to believe that an environment with fixed tasks, scaled by an agent with pre-defined goals is not going to make a good benchmark problem for AGI. The motivation to perform any action, such as eating, avoiding pain, exploring, planning, communicating, striving for power, does not arise from intelligence itself, but from a motivational system underlying all directed behavior. In specifying a motivational system, for instance as a set of conflicting drives, we have to make sure that every purposeful action of the system corresponds to one of its demands; there is no reason that could let us take behavioral tendencies such as self-preservation, energy conservation, altruistic behavior for granted – they will have somehow to be designed into the system (whereby ‘somehow’ includes evolutionary methods, of course).

J. Bach / Seven Principles of Synthetic Intelligence

69

7. The emergence of intelligence is not going to happen all by itself While the proposal of AGI or synthetic intelligence is based on a computational monism, 5 dualist intuitions are still widespread in western culture and in the contemporary philosophy of mind, and they are not going to give in without a fight. Because a naked ontological dualism between mind and body/world is notoriously hard to defend, it is sometimes covered up by wedging the popular notion of emergence into the “explanatory gap” [13]. Despite the steady progress of neuroscience and computational models of neural activity, there is an emergentist proposal that assumes so-called “strong emergence”, which proposes that the intelligent mind, possibly including human specifics such as social personhood, motivation, selfconceptualization and phenomenal experience, are the result of non-decomposable intrinsic properties of interacting biological neurons, or of some equally nondecomposable resonance process between brains and the physical world. Thus, “strong emergence” is basically an anti-AI proposal. Conversely, “weak emergence” is what characterizes the relationship between a state of a computer program and the electrical patterns in the circuits of the same computer, i.e. just the relationship between two modes of description. In that sense, emergent processes are not going to “make intelligence appear” in an information processing system of sufficient complexity. We will still need to somehow (on some level of description) implement the functionality that amounts to AGI into our models. This brief summary of principles of synthetic intelligence does not answer the main question, of course: How do we capture the functionality of Artificial General Intelligence? – In cognitive science, we currently have two major families of architectures, which seem to be hard to reconcile. One, the classical school, could be characterized as Fodorian Architectures, as they perceive thinking as the manipulation of a language of thought [14], usually expressed as a set of rules and capable of recursion. Examples, such as ACT [15] and Soar [16] are built incrementally by adding more and more functionality, in order to eventually achieve the powers inherent to general intelligence. The other family favors distributed approaches [17, 18] and constrains a dynamic system with potentially astronomically many degrees of freedom until the behaviors tantamount to general intelligence are left. This may seem more “natural” and well-tuned to the “brain-level” of description, because brains are essentially huge dynamical systems with a number of local and global constraints imposed on them, and the evolution of brains from mice-sized early mammalians to homo sapiens has apparently not been a series of incremental functional extensions, but primarily a matter of scaling and local tuning. Yet many functional aspects of intelligence, such as planning and language, are currently much harder to depict using the dynamical systems approach. The recent decade has seen the advent of several new architectures in AI, which try to combine both approaches in a neuro-symbolic fashion, such as Clarion [19], LIDA [20], the MirrorBot [21] and the author’s own MicroPsi [22], which will briefly be introduced on the remaining pages. 5 Computational monism itself amounts just to the subscription of contemporary materialism. Cartesian matter (‘res extensa’) sports unnecessary intrinsic properties, such as locality and persistence, which get in the way when doing contemporary physics. Today’s matter is not the same wholesome solid as it used to be in Laplace’s time and day; now it is just a shifty concept that we apply to encode the basic regularities in patterns presented to us by our environment.

70

J. Bach / Seven Principles of Synthetic Intelligence

The Psi theory and the MicroPsi architecture MicroPsi [22, 23] is an implementation of Dietrich Dörner’s Psi theory of mental representation, information processing, perception, action control and emotion [24] as an AI architecture. MicroPsi is an attempt to embody the principles discussed above: 1. MicroPsi aims at explaining intelligence by a minimal orthogonal set of mechanisms that together facilitate perception, representation, memory, attention, motivation, emotion, decision-making, planning, reflection, language. These features are not explained as parameters or modular components, but in terms of the function of the whole system; for instance, emotions are explained as specific configurations of cognitive processing rather than as given parameters; behavior is not the result of predefined goal directed routines, but of a demand-driven motivational system and so on. 2. An integrated cognitive architecture will require the recourse to methods from many disciplines; MicroPsi originates in theories of problem solving in psychology and integrates ideas from gestalt theory, motivational psychology and experiments in the study of emotion. Also, it has learned a lot from cognitive modeling paradigms and from representational strategies and learning methods in AI. 3. The facets of cognition are not seen as separate modules that could be understood, tested and implemented one by one – rather, they are aspects of a broad architecture. The model combines a neuro-symbolic theory of representation with a top-down/bottom-up theory of perception, a hierarchical spreading activation theory of memory with a modulation model of emotion, a demand/drive based theory of dynamic physiological, cognitive and social motivation with a model of the execution and regulation of behaviors. 4. Representations in MicroPsi are always grounded in environmental interaction or abstractions thereof. It does not matter, however, if the environment is simulated or physical. 5. The difficulty of providing a rich pre-programmed environment vs. the limitations that come with robot engineering have lead to both simulation worlds and robotic experiments for MicroPsi agents. At the current stage of development, we seem to learn much more from simulations, though. 6. MicroPsi agents are autonomous, their behavior is governed by a set of primary urges which determine motives which in turn give rise to intentions. All behavior, including cognitive exploration and social interaction, can be traced back to one or more primary urges. 7. There are many aspects of intelligence that MicroPsi does not address well yet. However, we do not believe that these will be automagically spring into existence with gradual improvements of the learning, representation or interaction modes of the model. We think that the deficits of MicroPsi highlight specific mechanisms, for instance for perceptual integration, language and self-monitoring, that we have not sufficiently understood to implement them. MicroPsi might propose some interesting answers, but more importantly, it helps to detail a lot of useful questions. Representation in the Psi theory The most basic elements in Dörner’s representations are threshold elements, which are arranged into groups, called quads. These are made up of a central neuron, surrounded by four auxiliary neurons acting as gates for the spreading of activation through the

J. Bach / Seven Principles of Synthetic Intelligence

71

network. A network of quads amouts to a semantic network with four link types, called SUB, SUR, POR and RET. SUB: stands for “has-part”. If an element a has a SUB-link to an element b, it means that a has the part (or sometimes the property) b. SUR is the inverse relation to SUB and means “is-part”. If a is SUR-linked to b, then a is a part (or sometimes a property) of b. POR (from latin porro) is used as a causal (subjunctive), temporal or structural ordering relation between adjacent elements. If a has a POR-link to b, then a precedes (and sometimes leads to or even causes) b. RET (from latin retro) is the inverse relation to POR. If there is a RET-link between a and b, then a succeeds (and sometimes is caused by b. Quads make up perceptual schemas and frames: Individual quads stand for concepts. If they are SUB/SUR linked, a partonomic (has-part/is-part) relationship is expressed. The lowest level of such a partonomic tree is made up by perceptual neurons and motor neurons. They provide the grounding of the system’s representations, since they are directly linked to the system’s environment. In MicroPsi, quads are extended to cover more basic relations and are expressed by concept nodes. In addition to the basic POR/RET and SUB/SUR links, they also offer link types for taxonomic and labeling relationships. Object schemas are organized as parts of situational frames. The world model of the system is established, reinforced or changed by hypothesis based perception (“hypercept”). Hypercept is a paradigm that might be described as follows: - Situations and objects are always represented as hierarchical schemas that bottom out in references to sensory input. - Low-level stimuli trigger (bottom-up) those schema hypotheses they have part in. - The hypotheses thus activated heed their already confirmed elements and attempt (top-down) to get their additional elements verified which leads to the confirmation of further SUB-hypotheses, or to the rejection of the current hypothesis. - The result of hypercept is the strongest activated (matching) hypothesis. At any time, the system pre-activates and inhibits a number of hierarchical schema hypotheses, based on context, previous learning, current low-level input and additional cognitive (for instance motivational) processes. This pre-activation speeds up the recognition by limiting the search space. Hypercept is not only used on visual images, but also on inner imagery, memory content, auditory data and symbolic language. The current situational frame is stored as the head element of a growing protocol chain, which is formed by decay and re-arrangement into long-term memory. Behavior Execution A neuron that is not part of the currently regarded cortex field is called a register. Neural programs are chains of registers that call associators, dissociators, activators and inhibitors. (These “calls” are just activations of the respective elements.) In the course of neural execution, elements in the cortex field are summarily linked to specific registers which are part of the executed chain of neurons. Then, operations are performed on them, before they are unlinked again. Dörner describes a variety of cognitive behaviors for orientation, anticipation, planning, memory retrieval and so on, often along with possible implementations. [24, 25]

72

J. Bach / Seven Principles of Synthetic Intelligence

Motivation and Emotion During action execution, the system establishes motives based on a set of primary urges which are hard-wired. Currently, these urges consist of demands for fuel, water, intactness, affiliation, competence and certainty.

Figure 2: Psi architecture Fuel, water and intactness are examples of physiological needs. Affiliation is a social urge – to satisfy it, the system needs affiliation signals from other agents. Thus, Psi agents may reward each other. Competence measures the ability to reach a given goal and the ability to satisfy demands in general (coping potential). The urge for certainty is satisfied by successful exploration of the environment and the consequences of possible actions, and it is increased by violations of expectations. Competence and certainty are cognitive urges. Together they govern explorative strategies. Every increase of an urge creates a negative reinforcement signal, called displeasure signal; conversely, a decrease of an urge results in a pleasure signal. These signals are used to strengthen links in the current protocol and thus enable reinforcement learning of behavior. At any time, the system evaluates the urge strengths and, based on an estimate of the competence for reducing individual urges, determines a currently active motive. This motive pre-activates memory content and behavior strategies and is used in determining and executing a plan to achieve it. To adapt the cognitive resources to the situation at hand, the system’s activity is influenced by a set of modulators. Within the Psi theory, a configuration of modulator settings (together with appraisals of certainty, competence and current pleasure/displeasure status) is interpreted as an emotional state [25].

J. Bach / Seven Principles of Synthetic Intelligence

73

The MicroPsi Framework MicroPsi has been developed in an AI context and offers executable neural representations, multi-agent capabilities and visualization tools. The author’s group has used it to model perceptual learning [26], the evolution of motivational parameters in an artificial life setting and as an architecture for controlling robots. The framework consists of the following components: a graphical editor for designing executable spreading activation networks (which make up the Psi agent’s control structures and representations), a network simulator (integrated with the editor and monitoring tools to log experiments), an editor and simulator for the agent’s environment and a 3D viewer which interfaces with the simulation of the agent world.

Figure 3: MicroPsi network editor and agent environment MicroPsi has matured into a runtime environment for various cognitive modeling tasks. Among other things, we are using it for - Building agents according to the Psi theory. These agents are autonomous systems with a set of innate urges, situated in a simulated environment, where they perceive objects using a simplified hypercept approach. Perceptual content is represented as partonomic schema descriptions and arranged into protocols, which are later retrieved to generate plans to satisfy the agent’s urges. - Performing neural learning using hybrid representations. To connect higher, gradually more abstract layers within MicroPsi network representations to real-world sensory input, we are setting up a matrix of foveal sensor nodes, which correspond to pixels in the camera of a robot. By moving the foveal field through the camera image, the image is scanned for salient features. Using backpropagation learning, we are training it to identify edge segments in the camera image, which in turn make up the lowest layer in an instantiation of hypercept. - Evolving motivational parameter settings in an artificial life environment. Here, groups of MicroPsi agents jointly explore their territory and cooperate to find and defend resources. Suitable cooperative behaviors are evolved based on mutations over parameters for each urge and modulator in accordance to the given environment. - Implementing a robotic control architecture using MicroPsi. A simplified neurobiological model of behavior and perception of mice in a labyrinth is mimicked using Khepera robots that are embodied MicroPsi agents.

74

J. Bach / Seven Principles of Synthetic Intelligence

Since its beginnings in 2001, the MicroPsi framework and the associated cognitive architecture have come a long way. Even though MicroPsi is far from its goal – being a broad and functional model of human cognition and emotion – it fosters our understanding and serves as a valuable tool for our research. Acknowledgments This work has been supported by the University of Osnabrück. I am indebted to Ronnie Vuine, who vitally contributed to the technical framework and the MicroPsi agents, to Matthias Füssel and Daniel Küstner, who built the environment simulator, to David Salz who is responsible for 3D visualization components, to Markus Dietzsch, who is performing the artificial life experiments, and to Daniel Weiller and Leonhardt Laer, who are in charge of robotic experiments. Also, I want to thank two anonymous reviewers for their constructive criticism. References [1] Jackson, F. (1982). Epiphenomenal qualia. Philosophical Quaterly 32: 27-36 [2] Block, N. (2001). Paradox and Cross Purposes in Recent Work on Consciousness. In Dehaene and Naccache: Special Issue of Cognition, Vol. 79, The Cognitive Neuroscience of Consciousness, 197-219 [3] Chalmers, D. (1996): The Conscious Mind. New York: Oxford University Press [4] Searle, J. R. (1992): The Rediscovery of the Mind, MIT Press, Cambridge [5] Dreyfus, H. L. (1992): What Computers still can’t do. A Critique of Artificial Reason. MIT Press [6] Harnad, S. (1990). The symbol grounding problem. Physica D, 42, 335-346. [7] Barsalou, L.W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577-609. [8] Brooks, R. A. (1991): Intelligence Without Reason, IJCAI-91 [9] Searle, J. R. (1980): Minds, brains, and programs. Behavioral and Brain Sciences 3 (3): 417-45 [10] Pfeifer, R., Stieler, W. (2007): „Es geht um den physikalischen Prozess.“ Technology review, online at http://www.heise.de/tr/artikel/95539, Oct 1st 2007 [11] Pfeifer, R., Bongard, J. (2006): How the body shapes the way we think. MIT Press [12] Noda, I (1995). Soccer Server: A Simulator of Robocup. In Proceedings of AI symposium 1995, Japanese Society for Artificial Intelligence. [13] Stephan, A. (1999). Emergenz: Von der Unvorhersagbarkeit zur Selbstorganisation. Dresden Univ. Press [14] Fodor, J. A. (1975). The Language of Thought, Cambridge, Massachusetts: Harvard University Press. [15] Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Erlbaum. [16] Laird, J. E., Newell, A., & Rosenbloom, P. S. (1987). Soar: An architecture for general intelligence. Artificial Intelligence, 33(1), 1-64. [17] Rumelhart, D. E., McClelland, J. L. and the PDP Research Group (1986): Parallel Distributed Processing, (Vols. 1&2), Cambridge, Massachusetts: MIT Press [18] Smolensky, P., Legendre, G. (2005): The Harmonic Mind. From Neural Computation to Optimalitytheoretic Grammar, Vol. 1: Cognitive Architecture, MIT Press [19] Sun, R. (2003) A Detailed Specification of CLARION 5.0. Technical report. [20] Franklin, S. (2007). A foundational architecture for Artificial General Intelligence. In Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms, Proceedings of the AGI workshop 2006, ed. Ben Goertzel and Pei Wang: 36-54. Amsterdam: IOS Press. [21] Borst, M., Palm, G. (2003): Periodicity Pitch Detection and Pattern Separation using Biologically Motivated Neural Networks. In: Proc. 6. Tübinger Wahrnehmungskonferenz, 52 [22] Bach, J. (2003). The MicroPsi Agent Architecture. Proceedings of ICCM-5, International Conference on Cognitive Modeling, Bamberg, Germany, 15-20 [23] Bach, J. (2008): PSI – An architecture of motivated cognition. Oxford University Press. [24] Dörner, D. (1999). Bauplan für eine Seele. Reinbeck [25] Dörner, D., Bartl, C., Detje, F., Gerdes, J., Halcour, (2002). Die Mechanik des Seelenwagens. Handlungsregulation. Verlag Hans Huber, Bern [26] Bauer, C. (2005): Kategorielernen im MicroPsi-Agenten. Diploma Thesis, Technische Univ. Berlin

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

75

Language Processing in Human Brain Alexander BORZENKO 1 Proto-mind Machines Inc., Toronto, Canada Abstract. Human brain is exceptionally complex and simple at the same time. Its extremely composite biological structure results itself in human everyday behavior that many people might consider rather simple than complex. In our research we will concentrate on the ways how a human brain can processes English and other human natural languages because taken in general sense the ability to speak English or other human languages is only serious distinguishing feature that rises humans over the rest of the world making a human an intellectual being. On the purpose of our research we consider natural language as naturally formed symbolic system completely independent of these symbols’ physical nature that is a little more general than a common natural language definition. The principles of natural language processing in human brain are most important for us if we want to build equally powerful artificial general intelligence. We start with the features of human brain neurons and neural networks, and step by step create a computer model of human brain networks that is able to process and generate a reasonable speech. We can’t give a detailed explanation of human brain functionality in this short article. Moreover, it is not our goal, and such research is not complete yet. The main result of our research is revealing the principles how tiny single neurons working together can produce intellectual-like behaviour that exhibits itself in proper speech comprehension and generation in accordance with current context. Keywords. Neuron, brain, neural network, intellect, language, AGI, human, mind.

1. Neurophysiologic Background Intellect is a direct property of brain cerebral activity according to the Nobel Prize laureate R. Sperry [1]. From other side, human intellect is bound to natural languages that humans speak. Natural languages spoken by humans are in essence the only language system that provides comprehensive description of the world. It is obvious that the features of neural networks of human brain restrict the natural language basis. Moreover the basic operation set of natural language depends on universal methods of data transformation in real neural networks. We will show that taking the general features of human brain as base we can design a virtual device that has human intellectual abilities like English speech understanding and producing relevant speech answers. Everything can be built over English language then, even, for instance, such complicated abstract theories like theory of intellect [1] or constructive mathematics

1

E-mail: [email protected]

76

A. Borzenko / Language Processing in Human Brain

theory [2] because there is no other language or extra language means in the basics of these theories save English language. Let us consider some general facts about human brain structure and neuron physiology. These facts would not be mentioned here if they didn’t complete a base of our brain model. For the beginning, our model consists of a huge number of mutually linked independent information units without any central control like natural brain does. Those elementary information units of human brain are neurons. Neurons are connected to each other through their axons and dendrites that cover the large areas of human brain according to other Nobel Prize winner Santiago Ramon Cajal [3] who discovered this feature about a century ago. We are interested in two main neuron structures of a human brain: Neocortex and Cerebellum that both have a great similarity from the information point of view [4]. The number of neurons in these human brain structures is huge that is of great importance for us (our statistics will work perfect with such significant amount of data). Current estimates suggest that there may be on the order of 1010-1012 neurons and of 1016 synapses per person. Each neuron has about 1000 connections on average [5] forming a kind of regular or, sometimes, granulated structure. Natural neuron consists of cell body, number of dendrites, and one axon (generally we are not interested in internal neuron structure). Most of neurons consist of a cell body plus one axon and many dendrites. The axon is a protuberance that delivers neuron output signals to connections with dendrites of other neurons or with neuron cell bodies. Dendrites are other protuberances that provide plenty of surface area, facilitating connection with the axons of neurons. Dendrites often divide a great deal, forming extremely bushy dendrite trees. Axons divide to some extent but far less than dendrites. We don’t reflect physical neuron structure in our model making the following generalization – our model elementary unit has many input signals and only one consolidated output signal that can be passed to many other model units. There are several types of signals transmitting from one neuron to others as well as different types of data in human brain neurons [6, 7, 8]. We will abstract of the physical nature of mentioned processes taking in a view only their general external information features, in particular an impulse manner of signals. Usually neuron does nothing active unless some function (we assign the letter Ȍ to it) of collective influence of all neuron inputs reaches a threshold level. Whenever that threshold level is reached, the neuron produces an output in the form of a short pulse that proceeds from the cell body, down the axon, and into the axon's branches. When this happens, the neuron is said to fire. From the external point of view the general manner of neuron firing seems to be stochastic except the case when neuron keeps firing during a certain period after it fired the first time. Sometimes neuron does not fire even if its threshold measured or estimated in the previous experiments is reached. Sometimes it fires spontaneously having no visible external factors to fire. This means that function Ȍ belong to a stochastic type. In addition, the neuron spontaneous activity (also of stochastic type) plays an important role in brain information processes, and it should be considered in our model through the features of function Ȍ. Axons influence dendrites over narrow gaps called synapses that are of a complex thin structure itself. Stimulation at some synapses encourages neurons to fire. Stimulation at others discourages or blocks neurons from firing. A complex stimulation provides answer that is hard to predict without exact information about states of neuron synapses and their dynamics. A lot of research was done to reveal the details of human brain synapse and neurons behavior [9, 10, 11, 12, 13, 14]. Nowadays the most of

A. Borzenko / Language Processing in Human Brain

77

neuroscience researches confirm each other in general. There is mounting evidence that learning takes place in the vicinity of synapses and has something to do with the degree to which synapses translate the pulse traveling down one neuron's axon into dendrites of the next neuron or neurons. These results mean that function Ȍ is not constant but changes itself to support learning. However we will not create an exact model of synapse functionality. In our model we will use only general synapse features that were just mentioned above describing them through the function Ȍ. Brain neuron behavior was generalized in a simple information model [15, 16] which would help us in further analysis. Let Etk be an event that some neuron has its synapse of number k activated at the time t. Then, supposing that zero time corresponds to the last time when function Ȍ has changed itself, a probability P of the neuron firing can be symbolically described as following P = Ȍ (Et, E0, U, M, N), N < M,

(1)

where Et and E0 are vectors of synapse activity at time t and 0 respectfully, U is a probability that a neuron changes its reaction during one second due to the phenomenon of spontaneous activity, M is a total number of this neuron synapses, and N is a minimal number of active neuron synapses that are necessary to reach the neuron’s threshold. In other words, neuron can make decision to fire if only N of its M synapses are in the states they were at zero time. The neuron synapse function (we will use the letter Ȍ for it) plays a key role in brain neuron functioning (e.g. in converging external synapse activity into neuron firing with a respect to previous history of neuron activity in a time-spatial form) [30]. It is obvious that function Ȍ is extremely complex and requires a powerful computer or even some advanced computer system to make an exact model even for a single neuron. Instead of using that approach we make only one natural assumption about Ȍ: Principle 1. If there is a configuration of dendrite activity (regardless of its type) and the neuron is active (e.g. firing) the function Ȍ modifies itself in the way that the neuron should fire on this particular or similar dendrite activity configuration in future. This fundamental principle can be named as principle of activity maintenance. This function Ȍ behavior leads to a recently confirmed fact that the same image repeatedly shown to human inspires activity of the same neurons in his brain. In its turn it leads to more complicated behavior when these activated neurons belong to one of specialized zones of the human brain, in particular, to the speech zone of the brain. In this case the human brain generates speech as a result of intellectual processing its input data. Principle 1 presumes that some physical process happens when brain memorizes information, and this process happens in the vicinity of neuron contacts (synapses mainly) because it is a place where activity signals of both contacting neurons meet. According to [11] it is probably the only place where such process occurs. The next principle of simplifying Ȍ deals with a neuron spontaneous activity that was discovered and confirmed by many independent researches a long time ago. Spontaneous activity seems to be a mere noise producing by a neuron but really it is a fundamental feature. The second principle of brain functioning is: Principle 2. There is a finite probability that any brain neuron fires independently of its current external configuration, history of firing, and the current states of its dendrite and axon inputs. Further we will refer to this principle as a principle of neuron spontaneous activity.

78

A. Borzenko / Language Processing in Human Brain

In general our understanding of natural neuron functioning contains nothing more than: Neuron has two states: active when it fires and passive when it does nothing visible form the outside, Neuron can recognize the similarity of current synapse activity of other neurons connected to it and can react with firing on similar configurations of current synapse activity, Whenever active the neuron has a power to share its efforts with other neurons in order to activate or to suspend the activity of some other neurons including itself (actually knowing nothing about possible consequences of this activity), Neuron thoroughly follows principles 1 and 2. Paying an undeniable respect to the researchers devoted their time to discover the details of natural neuron physiology nevertheless we have to admit that those details give us almost nothing about understanding the structure of the human intellect foundation. Our aim is to prove that principles 1 and 2 lead to intellectual behavior if we speak about humans. In order to prove a memory phenomenon in neural networks of human brain we construct a mathematical model of natural neural network using principles 1 and 2. The deductions from this model form the basis for analysis of natural languages in a human manner. This abstract model of natural language processing in relation to the respective human brain methods helps us to build more sophisticated models and check the different aspects of natural language processing in a human brain. Finally, we build a complete abstract model - General Sense Linker (GSL) - of human brain methods for English language processing and synthesizing through generalization of some intermediate models that we will develop and also through direct implementation of model of the object that function Ȍ deals with. GSL learns English like a child who forms proper speech ability by himself using his/her parents’ and teachers’ lessons and examples. So, in order to do this, GSL should be provided with necessary conditions and examples for the automatic creation of output reactions on a contextual base. After learning, GSL becomes almost fluent in English (certainly in the domain it was taught). This means that GSL’s natural language responses on given natural language stimuli can be predicted in a current context by taking the similar human responses as a base. At the same time this GSL’s behavior serves as a proof of mentioned above simple principles 1 and 2 of brain functioning because no extra fundamental features are used in GSL model save activity maintenance and spontaneous activity principles.

2. Natural Neuron network model Let us consider a group of mutually connected neurons that we call column. All column neurons comply with the principles 1 and 2. We accentuate this fact by using italic font when mentioning the object column. In a general case the internal column neuron connections are supposed to be of some restricted chaos type that makes columns’ information properties vary. Extern column connections consist of the set of column neurons’ connections to the neurons that don’t belong to the column. We divide extern

A. Borzenko / Language Processing in Human Brain

79

column connections onto input and output parts (that parts can have connections in common sometimes). We consider the following virtual experiment in order to estimate a stability of column neuron reactions. Suppose we have a neuron with number p (p-neuron) reacted with firing on some spatial configuration of activity coming from the external neurons to the dendrites of column neurons. In more precise, “spatial activity” means that we have a combination of external neuron signals constantly coming to our column neurons during some critical time period. This period is supposed be long enough to let column neurons to process these signals. Formula (2) below shows probability xtp of the event that a column’s p-neuron keeps its proper reaction on the concrete image Im exposed to the column exactly t seconds ago. By other words, p-neuron loses this reaction at the second with number t if the image Im has been exposed at a zero time point. The reason why this p-neuron changes its reaction can be explained through the activity maintenance principle: this neuron spontaneous activity occurred and this neuron merely amplifies its active dendrite ties when any other image was exposed to the column at this time point (like the results of research [28]). Supposing that every column neuron needs an exact signal configuration on its dendrites to recognize the image (or by other words, dendrite activity configuration) we have the following equation for probabilities xtp in the connection to other neural network parameters. xtp=(Ȇt-1r=1(1-xrp)) ·{U+(1-U)[1-ȆiȯQ(p)(1-xt-1i)]},

(2)

where U is a probability that a column neuron changes its reaction during one second as it was defined in formula (1). Naturally, probabilities x0p equal zero. The set Q(p) is designated to the variety of numbers of column neurons that provide p-neuron with relatively significant influence. Let us take a look at figure 1 with a diagram of the numerical evaluation of the equation (2) under assumption that no neuron in a column has advantage over others, q equals 182 (we supposed it to be the estimated average size of human brain neural “column”) and U equals 10-7 (this probability means that 10,000 human brain neurons of Neocortex change their internal memory every second on average). We need some additional model that can summarize the results of image remembering and recognition in our columns to create a consolidated reply like a Figure 1. Probability of keeping the data in human brain that generates the Model

80

A. Borzenko / Language Processing in Human Brain

consolidated answers. Fortunately we know that there is a brain Neocortex part with a plenty of giant pyramidal and motor neurons which generate signals resulting in speech. In this part there are outgoing peripheral brain nerves that control speech [11]. These pyramidal neurons collect and summarise signals of other neurons through their giant ramified dendrites covering the vast regions of a human brain. Many column neurons of some types with long horizontal-spread axons help them. Such pyramidal neurons regulate human speech facilities directly through generating signals to the throat and other muscles that are involved in a speech producing. We introduce nothing new in pyramidal firing estimation. Formula that provides us with the means for estimating a probability of pyramidal neuron firing through vectors Et of its synapse activity (formula (1)) is Ȉ kȯM A (Etk, E0k) * Wk > Ȝ, where A( · , · ) is a function of two arguments that returns 1 if its arguments are equal, and 0 otherwise; Ȝ is some threshold, and Wk are some weights associated to correspondent synapses (these weights are not constants, they vary according to the principles 2 and 3). Stimulus to activate “pyramidal neuron” of a current symbol from Kt in learning mode

Kt S t1

Column 1

S t2

Column 2

Tt1

Tt2

St3

Column 3

Tt3

“Pyramidal neurons” for A, B, C, … , X, Y, Z

Lt Figure 2. General Principle of Natural Language Phrase Processing in the Model

A. Borzenko / Language Processing in Human Brain

81

Unfortunately, a human-like intellectual behavior is a vague concept. Everybody understands it but can’t express it formally at the same time. Broadly speaking, human is able to learn in any possible knowledge domain either quantum physics or car renting or almost any other field. From the external point of view this ability reveals itself in generating relevant reactions on an appropriate input in most cases. In more general, every generated English phrase should be consistent and should correspond to its predecessors. By other words, speech should be interconnected. We will believe that we have successfully proved that our abstract model has a human-like intellect if we will be able to show a similar behavior of our model. We will consider many columns concurrently as a column system united by innumerous mutual ties of chaotic type (as it is shown formally by clouds on figure 2). This column system will be a core of our neuron network model. Statistical characteristics of the chaos are critically important as usual when any neuron network model is been built. We start with the simplest probability distributions (like normal) keeping in the mind to find the most effective ones through experiments with our computer model. As far as English language is concern we will consider here only English letter and grammar symbol sequences {Kt} as a brain model input no matter how they are delivered to brain either through vision, hearing or other relevant channel. Physically any peripheral neurons could deliver the consequences of signals that correspond {Kt} through their ramified axons. Moreover we don’t suppose to have functional level dependence here. Statistical one is enough for us. Our model accepts {Kt} symbol by symbol having all information about the current symbol C(Kt,t) processed before accepting the next symbol. In its turn, {Lt} stays for English letter or grammar symbol output of our model. We correlate it to the most active “pyramidal neuron” at the time (fig.2). Here the superscript index t corresponds to the time interval when actual symbol/letter was generated (or was entered in the case of {Kt}) in our neural column system. The form of input doesn’t matter for us because regardless of external symbol form the peripheral nerve system delivers information to brain as a combination of dendrite impulse activity. So what we can be sure in is that this dendrite activity correlates statistically with input symbols, and we need nothing more to know about input/output in our investigation. We consider two modes of our model functioning: learning and speaking. At the first mode we have {Kt} not empty, so our model has its current symbol anytime. At the second mode {Kt} is empty, and we suppose that current symbol is C(Lt,t-1) (model listens to its own “speech”). Let us number all neural columns using any appropriate method and let Sti be an input of any particular column with number i at the time point t, and then let Tti be the according column output that is a combination of column neuron axon activity. We also consider Tti as some numeric measure of intensity of summarized column neuron activity in a relevant phrase context (the most frequent case is that there is not more than one active neuron in a column at any time point). In many cases just a numeric sum as measure is sufficient enough. In the relation to our columns each variable Sti means that there are some synapses on i-column neurons those synapses’ weights are correlated somehow to letters and symbols those the model percepts and those are described by variable Kt. Formally Sti

82

A. Borzenko / Language Processing in Human Brain

includes some valuable subset of Kt and feedbacks of previous neuron states of icolumn. All ties on the figure 2 provide not instant data transfer because natural axons and dendrites have different lengths and thickness, and sometimes because of different chemical environment in real neuron networks, feedback ties and because of the general inertia of a neuron functioning. These features are confirmed by many researches, for example, recent developments [25, 26, 27]. Action potentials travel at up to 10 m/s speed in unmyelinated axons and up to 120 m/s speed in myelinated axons [5]. Taking in a view these signal transferal delays, each Tti is a complete mess of columns’ neuron signals (in the form of impulses) if we consider them as an input sequence of some network neuron. One particularity of this “mess” is extremely important. Column will repeat this “mess” (configuration of outputs of its neurons) with the same statistical characteristics exactly if this column input is the same because column‘s signal “mixing” algorithm statistically depends on dendrite-axon ties, length and thickness of dendrite and axons and some other constant factors that has been indirectly confirmed by many experiments mentioned in the first paragraph. Finally, we constitute a simple but effective method to create a generalized output Lt of “pyramidal neurons”. We select a letter or a symbol of that “pyramidal neuron” which has currently a maximum activity level among all others as actual model output leaving the real algorithm for further investigation. Nevertheless we use efferent and afferent signals in our model (the latter is a prototype for stimulus signal in fig. 2) in the learning mode. We suppose that efferent nerves have their corresponding afferent partners or the stimulation of a “pyramidal neuron” matching the current symbol can be done through efferent influence on receptor activity [31]. In our model this “afferent” direct or indirect stimulus provides additional activity to “pyramidal neuron” of a current letter when we teach our model.

3. COMPUTER-ADDED EXPERIMENT It took about a year of intensive interactive communication with computer implementation of our abstract model (GSL) in order to achieve expected results. During this period the necessary statistics in the model were completed. The method that was used to teach the model plays a secondary role and therefore is listless here. The text material that was used to teach GSL consists of some child books, scientific articles, many dialog fragments of the type you can find in literature and Internet chats, on-line scientific, sport and politics news, and even thesaurus articles. A lot of direct interaction was done with GSL at the final stages of learning when reliable answers of GSL became common. The experiment has proven that diversity of input texts and dialog topics leads to better results. Certainly the final model is not a comprehensive and complete artificial intelligence because of lack of knowledge and some data input/output stream simplifications (we had only one input stream and one output stream controlled by high level model monitor). Fortunately it was not our aim to create an artificial intelligence. Bellow there are some samples of a dialogue with GSL model at the different stages of learning process (G: stays for free speech generation by GSL, Q: means a question, provided by teacher, and A: stays for GSL’s answer to the question above). GSL learning process passed through several stages. Each stage required its own settings and learning method specific modifications. Accordingly, the model behavior

A. Borzenko / Language Processing in Human Brain

83

was improved stage by stage starting with exact answers to exact simple questions without taking the current context into account at primary stages, and to context speech generating at intermediate stages, and to free form answers to general questions in a complete conformity with context at final stages. Actually all these three stages are interconnected and differ only in a degree of context involving that in its turn is a result of amount of data that were accumulated in the model. Let us consider them consequently one by one. Primary Stages (1,000,000 columns) Q: What’s your name? A: Alex Q: How old are you? A: I am a young man Q: Who are you? A: I’m a scientist At this stage the model was directly taught all these question-answer couples. It took several rehearsals for each question-answer couples before the model began to react properly. Intermediate Stages (3,500,000 columns) G: He took no notice of this G: He looked across of my companion who was also an elderly G: I was away in a cab and down the platform, with what you had better give me much more clever than average man G: I think that was to details of glance into the fire The model was not taught anyone of these sentences at any stage. No answers given by GSL were entered in the model in the exact form they were received. GSL generated them using its general knowledge collected throughout learning period in the context created by examiner especially for each sample. Also GSL computer model has read several books of famous English-speaking writers of last centuries. At the second intermediate stage it is common that all four sentences are of a correct syntax with some semantic disturbances that can be explained by a lack of knowledge in the model. Final Stages (6,100,000 columns) Q: What is a sea floor? A: A floor is generally a lower horizon Q: What do you think about the mystery island colonists? A: Poor wreckers! They were in a great danger Last answers look quite correct if we take in account that the context was the old trilogy of famous French child writer (in English translation) about underwater travel, oceans and isolated islands. It should be especially mentioned that number of columns involved by teaching the model through dialogues and readings is not in a linear statistical dependence of processed data volume. The process of column number growth (our model has ability to create columns automatically when it is necessary) slows down with any portion of data that contains really new facts. For instance, the data volume processed at final

84

A. Borzenko / Language Processing in Human Brain

stages is about ten times greater than similar amount at the intermediate stages. But at the same time the column amount in GSL model was only doubled (find these numbers above). Table 3 contains a table with the learning stages against approximate numbers of neuron columns which activity is above average level, durations of the stages and intensity of new column neuron activation. Stage Primary Intermediate Final

Table.3. Active column dynamics Amount of Relative Average columns Duration Intensity 1,000,000 1 1.00 3,500,000 6,100,000

5 14

0.70 0.43

Figure 3 illustrates the trend of the intensity of new column neuron activation against the number of processed sentences of some brief novel. The trend shows that intensity (y-axis) drops very slow but stable. Intensity is measured in average number of newly involved neurons (number of neurons per sentence length * 100) against the sentence amount. When analyzing the results of experiments with General Sense Linker we keep in mind that GSL is NOT artificial intelligence (there is a long way to find out even what is a natural intelligence [29] Figure 3. Column Involving Trend saying nothing about artificial one). GSL is a model of natural brain neuron network that was created in order to explain how human brain works when processing the natural languages. Nevertheless GSL can be used as a language processor with not-symbolic internal knowledge representation. It is pretty effective to generate answers in a real time.

A. Borzenko / Language Processing in Human Brain

85

4. CONCLUSIONS We discussed neither what a human intellect is nor how it could be built from not intellectual parts. It was enough for us that in spite of revealing itself in many different ways a human intellect always uses a simple ability of our minds to give natural answers to natural language questions in accordance with the questions’ common sense and current context. We have shown this fundamental intellectual feature of our virtual device GSL that we created on the base of principles 1 and 2 and broad-known facts of human brain structure and physiology using them in a more or less constructive manner. We have created GSL device that is capable to learn and speak English, and this device consists of a variety of independent comparably simple units which work without any synchronization and supervision like neurons in a natural brain. They are self-organizing and self-control units similar the brain neurons. Macro blocks of them that we called columns show the memory abilities, and we proved that columns can keep information as long as years at least. GSL provides columns with input data and incorporates the results of column information processes into generalized output working similar the brain pyramidal neurons. So GSL device has a lot of features in common with human brain structures. For instance, knowledge is spread among the total set of neurons. So destroying some group of neurons doesn’t result in destroying GSL’s knowledge, it just becomes less solid. Certainly it requires time to elaborate details and to persuade us that this device works having the same principles in its base as human brain has, and therefore explains how the latter works. But it can’t stop us from using those principles and GSL device itself for many practical applications. At the next stage of our research we will deal with visual input channel. Human vision is a complicated highly specialized apparatus that is tightly collected to the human general intelligence. So this research will use already created theory involving simple primary visual image recognition methods. We will show how video data that are not compact and grammar-structured like speech can be processed by our final GSL model having a similar result – generating the relevant answers to appropriate video images taken in a current context.

References [1] Sperry R.W. “A modified concept of consciousness”. In: Psychological Review, vol. 76, N 6, 1969, p.332-336. [2] Markov A. “An approach to constructive mathematical logic”. Logic, Methodology and Philosophy of Sciences, III, 1968, Amsterdam. [3] Santiago Ramon Cajal. “New Ideas on the Structure of the Nervous System in Man and Vertebrates”, 1894, Bradford Books, MIT Press. [4] Nieuwenhuys R., Donkelaar H., Nicholson C. “The Central Nervous System of Vertebrates”, 1998, Springer-Verlag. [5] R.W. Williams, K. Herrup. "The control of neuron number". Annual Review of Neuroscience, 11, 1988, 423-453. [6] Paul Katz (Editor) “Beyond Neurotransmission. Neuromodulation and its Importance for Information Processing”, Oxford University press, 1999. [7] Han, X. and Jackson, M. B. “Electrostatic interactions between the syntax in membrane anchor and neurotransmitter passing through the fusion pore”. Biophysics, 2005. Letter. 88: L20-22.

86

A. Borzenko / Language Processing in Human Brain

[8] Power, J.M., Oh M.M. and Disterhoft, J.F. “Metrifonate decreases sI (AHP) in CA1 pyramidal neurons in vitro”. J. Neurophysiol.85, 2001: 319-322. [9] Idan Segev “Single neuron models: oversimple, complex and reduced”. In: Trends in Neurosciences, Vol. 15, No. 11, 1992, p.414-421 [10] Mojarradi M., Blinkley D., Blalock B., Andersen R., Ulshoefer N., Johnson T., and L. Del Castillo “A miniaturized neuroprosthes is suitable for implants into the brain”. IEEE Transactions on Neural systems and Rehabilitation Engineering, 11, 2003:1534-4320. [11] Milner P. “Physiological Psychology”. Holt, Rinehart and Winston, Inc. New York, 1970. [12] Luria A. “The origin and cerebral organization of man’s conscious action”. In: Proceedings of the 19 congress of psychology, 1971, London. [13] Peter H. Lindsay, Donald A. Norman. “Human Information Processing”. Academic Press, New York and London, 1972. [14] Jones, T., Greenough, W. T. “Ultra structural evidence for increased contact between astrocytes and synapses in rats reared in a complex environment”. Neurobiology of Learning & Memory, 65 (1), 1996, 48-56. [15] Borzenko A. “Formal Neuron System for the Natural Language Analysis”. In: IEEE Neural Networks Proceedings, 1998, p.2561-2564. [16] Borzenko A. “Associative Recognition of Signals by a Network of Formal Neurons”. In: Automatics and Mechanics, No. 1, 1985, Moscow, Science, p. 95-100. [17] Eliot C. Bush, John M. Allman. “The Scaling of White Matter to Gray Matter in Cerebellum and Neocortex”. Brain, Behavior and Evolution, Vol.61, N 1, 2003. [18] Jan Woogd, Mitchell Glickstein. “The Anatomy of the Cerebellum”. Neurosciences, September 1998, Vol. 21, ʋ 9. [19 ]Kaas J.H. “The organization of Neocortex in mammals: Implications for theories of brain function”. Annual Review of Psychology, 38, 1987:129-151. [20] Hubel D.H. “Eye, Brain and Vision”. Scientific American Library, No. 22, 1995, WH Freeman, NY. p. 70. [21] Hubel D.H., Wiesel T.N. “Brain and Visual Perception”. Oxford University Press, 2005. [22] Greenough, W. T., Black, J. E. “Induction of brain structure by experience: Substrates for cognitive development”. In M. Gunnar & C. Nelson (Eds.), Minnesota Symposia on Child Psychology. Vol. 24, 1992, Developmental Behavioral Neuroscience, p. 155-200. [23] Pavlov I.P. “Conditioned reflexes”. London, Routledge and Kegan Paul, 1927. [24] Tran H, Brunet A, Grenier JM, Datta S.R, Fornace A.J Jr., DiStefano P.S, Chiang L.W, Greenberg M.E. “DNA repair pathway stimulated by the fork-head transcription factor FOXO3a through the Gadd45 protein”. Science; 296, 2002: 530-534. [25] Soleng A, Raastad M, Andersen P. “Conduction latency along CA3 hippocampal axons from rat”. Hippocampus, 13(8), 2003:953-61 [26] Shimada Y, Horiguchi M, Nakamura A. “Spatial and temporal properties of interocular timing differences in multifocal visual evoked potentials”. Vision Res., Feb, 2005; 45(3):365-71. [27] Hammond J., Fischer S., Valova I. “A parallel algorithm for growing, unsupervised, self-organizing maps utilizing specialized regions of influence and neuron inertia”. IASTED International Conference on Circuits, Signals and Systems, California, October, 2005. [28] French D.A., Gruenstein E.I. “An integrate-and-fire model for synchronized bursting in a network of cultured cortical neurons”. Journal of Computational Neuroscience, Springer Netherlands, Issue Volume 21, Number 3, pp. 227-241, 2006 [29] Minsky M. “The society of mind”, Simon & Schuster paperbacks, 1986. [30] Katsunori Kitano, Tomoki Fukai “Variability vs. synchronicity of neuronal activity in local cortical network models with different wiring topologies, Journal of Computational Neuroscience, pp. 237-250, 2007. [31] Timo Jarvilehto (1999) Role of Efferent Influences on Receptors in the Formation of Knowledge, Integrative Physiological and Behavioral Science, April-June, Vol.34, No.2, 90-100.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

87

Toward Logic-Based Cognitively Robust Synthetic Characters in Digital Environments1 Selmer BRINGSJORD [email protected] Andrew SHILLIDAY, Joshua TAYLOR, Dan WERNER, Micah CLARK {shilla, tayloj, werned, clarkm5}@rpi.edu Ed CHARPENTIER and Alexander BRINGSJORD {charpe, bringa}@rpi.edu Rensselaer AI & Reasoning (RAIR) Lab Department of Cognitive Science Department of Computer Science Rensselaer Polytechnic Institute (RPI) Troy NY 12180 Abstract. With respect to genuine cognitive faculties, present synthetic characters inhabiting online virtual worlds are, to say the least, completely impaired. Current methods aimed at the creation of “immersive” virtual worlds only avatars and NPCs the illusion of mentality and, as such, will ultimately fail. Like behaviorism, this doomed approach focuses only on the inputs and outputs of virtual characters and ignores the rich mental structures that are essential for any truly realistic social environment. While this “deceptive” tactic may be suitable so long as a human is in the driver’s seat compensating for the mental deficit, truly convincing autonomous synthetic characters must possess genuine mental states, which can only result from a formal theory of mind. We report here on our attempt to invent part of such a theory, one that will enable artificial agents to have and reason about the beliefs of others, resulting in characters that can predict and manipulate the behavior of even human players. Furthermore, we present the “embodiment” of our recent successes: Eddie, a four year old child in Second Life who can reason about his own beliefs to draw conclusions in a manner that matches human children his age. Keywords. virtual characters, cognition, logic, theory of mind

1 We are grateful for funding from IARPA and IBM; this funding has partially supported the R&D described herein. We are also indebted to Konstantine Arkoudas for many contributions. Finally, thanks are due to Mike Schoelles for his work on SimBorgs that provided a starting point for automating conversation in Second Life.

88

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters

1. The Problem Your avatar in the current version of a massively multi-player online (MMO) virtual world (e.g., Second Life) is directly tethered to your key strokes, and is nothing more than an automaton controlled directly by what your fingers do. This is great as far as it goes, but the fact remains that your avatar is completely bereft of the cognitive things that make you you, and is in fact devoid of even computational correlates of the things that make you you. He/she doesn’t speak or move autonomously, doesn’t have any memories, doesn’t have any beliefs (none at all, and therefore none of precisely the kind that are distinctive of persons, e.g., beliefs about the beliefs others have about your own beliefs2 ), doesn’t know anything, and therefore certainly can’t hold in mind a model of other creatures he/she encounters in such a virtual world. In laying a foundation for our initial research, we sought to make this problem more concrete: We not only studied the state of the art in AI, but had our laboratory, if you will, enter into it. We thus know that while Rensselaer graduate Joshua Taylor has an avatar in Second Life, and also in Nintendo’s Wii system (see Figure 1), the fact of the matter is that the real Joshua Taylor has all sorts of beliefs, intentions, goals, and desires that the avatar does not.

Figure 1. Joshua Taylor, Computer Science MS Holder, “Avatared” in Nintendo’s Wii System (left) and Second Life (right)

The same fundamental problem arises when we are talking not about avatars, but about NPCs (non-player characters). They are just as cognitively empty, and as a result can invariably be spotted in digital environments as mere shells. If space permitted, we could show, in detail, that all synthetic characters, to this point in time, whether avatars or NPCs, are indeed primitive. (For some examples, see Figure 2.) Even book-length treatments of how to build synthetic characters are impoverished relative to what we are engineering. For example, Figure 3, taken from [2], models synthetic characters using finite state automata. In such models, there is no knowledge and belief, no reasoning, no declarative memories, and no linguistic capacity. 2. Behaviorism Isn’t the Answer In the end, given the stone cold emptiness of avatars and NPCs, simple engineering tricks to make these digital creatures seem as if they have genuine mentality isn’t going to work. 2 One of the traditional marks of personhood has long been held to be that they can have such so-called third-order beliefs. See [1].

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters

89

Figure 2. Sample Synthetic Characters. Worst to best, in our eyes: Top-left, The Legend of Zelda;

SC spits text upon entering room. Top-right, Chrono Trigger; tree-branching conversations. Middle-left, Might & Magic VI (Shopkeepers). Middle-right, Superfly Johnson from Daikatana; behavior scripting, attempts to follow player and act as a sidekick. Bottom-left, Galatea – Interactive Fiction award winner for Best NPC of 2000 (text-based). Bottom-right, Sims 2. Thanks are due to Marc Destefano for these examples and the snapshots.

One can try, through surface-level tricks (such as the “chirping” in The Sims, used to make it seem as if characters are genuinely conversing), to make the observable behavior of synthetic characters such that these characters seem to have knowledge, beliefs, memories, and so on, even when they don’t have these things, but this strategy, ultimately, will always fail. Any approach that treats avatars as empty shells, as puppets, will fail for the same reasons that behaviorism failed: if focuses on inputs (stimuli) and outputs (responses) and ignores everything in between. A rich mental life, however, can only be understood, predicted, and explained (indeed, it can only be had) on the assumption that there are rich mental structures mediating inputs and outputs. That was the central tenet of the “cognitive revolution,” and unless we want a form of AI as bankrupt as behaviorism, we must subscribe to it when building synthetic characters. We must thus build synthetic characters with rich mental structures capable of representing (at least computational correlates of) beliefs, knowledge, perceptual input, and so on. And we further argue that we need to do that in a rigorous way, one that specifically capitalizes on the logical properties of mental states, as investigated by logicians, technical philosophers, theoretical cognitive scientists, and so on (rather than by building, say, neural networks and other numerical, non-declarative systems). A very deep kind of logicist engineering is thus, at least by our lights, needed. For us, the engineering must be in the mold of logic-based AI [3,4,5,6,7] and computational cognitive modeling [8]. This logic-based work stands in what we see as interesting, stark contrast to brain-based approaches in AI and cognitive science (e.g., to [9]).3 3 While we don’t discuss the issue herein, it may be that hybrid approaches bringing together top-down logicbased techniques with bottom-up neuroscience-oriented techniques are worth pursuing. Sun’s [10] is directly relevant here.

90

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters Moods

Mannerisms Emotions

Sensations Feelings

Stimuli

Memories

Figure 3. Impoverished FSA-Based Formalism for Synthetic Characters (from Champadard 2003).

3. Our Long-Term Objective In order to lay a foundation for engineering cognitively robust synthetic characters in massively multiplayer online (MMO) virtual worlds (e.g., Second Life) and other digital environments, we are in the process of codifying, to an unprecedented degree, the principles of “common-sense psychology” (CSP) in the form of an implementable logicomathematical theory. This theory must include rigorous, declarative definitions of all the concepts central to a theory of mind — concepts like lying, betrayal, evil, and so on. The implementation of this theory will be one that artificial agents can deploy in order to understand and predict psychological aspects of the behavior of other artificial agents, and in order to be genuine stand-ins for human beings. No such understanding or prediction is possible today. Our methodology can be understood along the lines of Lewis’s prescription: Collect all the platitudes regarding the causal relations of mental states, sensory stimuli, and motor responses. Add also all the platitudes to the effect that one mental state falls under another... Perhaps there are platitudes of other forms as well. Include only the platitudes which are common knowledge amongst us: everyone knows them, everyone knows that everyone else knows them, and so on. [11, p. 256]

Of course, Lewis here is concerned with the construction of a complete theory of CSP, one that would be capable of explaining virtually all aspects of human behavior. It is We also leave aside the economic side of the present work. Surmounting the impoverished nature of synthetic characters, making them, as we say, cognitively robust, would have great economic value. Millions of people are willing today to spend hard-earned money to merely control “puppets” in MMO virtual worlds. Billions would, we suspect, spend such money to play in virtual worlds populated by bona fide digital psyches. A small but important part of our efforts includes study of the economic and business dimension of MMO virtual worlds and synthetic characters, because we see our R&D as being bound up with the realities of the marketplace. Our study in this area, led by Alexander Bringsjord, is left aside in the present document.

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters

91

questionable whether the construction of such a theory is a practical possibility, and whether it could be carried out in the absence of a complete physical theory. Even if such a theory is practically possible, it is doubtful whether practical reasoning could be performed on its basis. Therefore, at least initially, we are building a micromodel of CSP.

4. The Theory of Mind Theory The ability to ascribe mental states to others and to reason about such mental states is indispensable for social communication. All social transactions — from lying and detecting lying, to engaging in commerce and negotiating, to making jokes and empathizing with other people’s pain or joy — require at least a rudimentary mastery of CSP. Today’s synthetic characters lack such a facility, and hence essentially suffer from autism, which is literally the disease afflicting that part of the brain responsible for reasoning about the mental states of others. An inability to attribute mental states to other characters is “tantamount to not differentiating between the world of objects (with physical states) and the world of persons (with mental states)” [12, p. 65]. A grasp of CSP is particularly important for agents who are trying to manipulate the behavior of other agents. The ability to manipulate through lying, for example, requires sophisticated reasoning about the beliefs of other agents. More generally, social interaction has been described as a form of “social chess” [13], whereby one agent, for instance, “may wish by his own behavior to change the behavior of another; but since the social animal is himself reactive and intelligent, the interaction soon becomes a two-way argument where each ‘player’ must be ready to change his tactics — and maybe his goals — as the game proceeds.” The principles and techniques that neurobiologically normal humans deploy in order to understand, predict, and manipulate the behavior of other humans are collectively referred to as a theory of mind [14]. What we are now starting to do is to engineer part of the theory that artificial agents could use to understand, predict, and manipulate the behavior of other agents, and in order to be veridical avatars for human beings, or to be autonomous intellects in their own right. One quick way to make our project clearer is to briefly turn specifically to lying, a speck in the overall theory we seek, but nonetheless informative to consider.4 One wellknown definition of lying within analytic philosophy is L lies to D =def ∃p such that 1. either L believes that p is not true or L believes that p is false; and 2. L asserts p to D [16, 152].

where L asserts p to D =def L states p to D and does so under conditions which, he believes, justify D in believing that he, L, not only accepts p, but also intends to contribute causally to D’s believing that he, L, accepts p [16, 152].

As can be clearly seen by inspection, even if these definitions are only approximately correct, any human D who believes that he or she is being lied to, must have beliefs about the beliefs the liar has about her (i.e., D’s) beliefs, and at the heart of the matter is propositional content. 4 Clark

has extended, formalized, and implemented this definition in what he calls “the lying machine[15].”

92

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters

5. Near-Term Goal: The Crucial Pair of Demos In the near term, we are working toward progressively more elaborate versions of two key demonstrations (both of which will be given at AGI2008). The first (the first two versions of which we have already achieved; see below) will subject our system to the litmus test of theories of mind: passing a false-belief test. Experiments with false beliefs were first carried out by Wimmer and Perner[17]. In a typical scenario, a child is presented with a story in which a character A places an object (e.g., a marble, teddy bear, cookie, etc.) in a certain location l1 , say in a particular kitchen cabinet. Then A leaves, and during his absence a character B removes the object from its original location l1 and puts it in a different location l2 (say, a different cabinet). The child is then asked to predict where A will look for the object when he gets back — the right answer, of course, being the original location, l1 . (Very young children (≈≤ 4) don’t produce this answer; they do not have a theory of the mind of others.) We are putting artificial agents to a similar test. To pass the test successfully, the agent will be required to make the correct prediction, by deploying our CSP model. Though we have much work to do, some primitive versions of this first demo have been implemented in connection with Second Life. A snapshot from one of our demos is shown in Figure 4. To take a look at a couple of the full videos, visit the link at the bottom of http://www.cogsci.rpi.edu/research/rair/asc_rca.

Figure 4. Snapshot of Demo in SL.

Our demo in Second Life is achieved with help from an automated theorem prover, Snark, coupled with procedures for generating and executing AppleScript (a scripting language from Apple Inc.). When Snark discovers a proof that agent s will perform action a, a procedure written in Common Lisp produces the code that, when executed, simulates keystrokes in Second Life, thereby enabling control of an avatar. Figure 5 shows the proof associated with “Ed’s” immature response in the false-belief task carried out in Second Life. In this case, s is the autonomous synthetic character Eddie, and the action a is the selection of the container that the bear has been moved to (so this is an example of failure in the task). Our second demonstration is in the area of natural language processing; here again an early version is available at the url given above. (See Figure 6 for a snapshot from

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters

93

Figure 5. Snapshot of False-Belief Task Demo in Second Life.

the video.) The goal in the second demonstration is to engineer a virtual character in an MMO virtual world capable of carrying out a conversation based upon certain knowledge and belief. In the demo, this character (BinDistrib) is in conversation with an avatar (CrispyNoodle). When the avatar (CrispyNoodle) speaks, the text in question is read by BinDistrib. For our work here we are leveraging our prior work in the machine reading area [18], an overview of which we now provide.

Figure 6. Snapshot of Conversational Demo in Second Life.

5.1. Machine Reading of Text in Second Life In our approach, text to be read is expressed in logically controlled English, which is translated into multi-sorted logic (MSL). The resulting information in MSL, in conjunction with other background knowledge expressed in MSL, is automatically reasoned over in proof-theoretic and model-based fashion. Our approach to machine reading is a threephase one: Phase 1 English texts are rephrased in logically controlled English — i.e., a proper subset of full English that can be unambiguously translated into a formal logic. In the past, we have

94

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters

made use of Attempto Controlled English (ACE) [19,20], a logically controlled English with a fixed, definite clause grammar and a user-defined vocabulary.5 However, we are now moving to CELT [22,23,24]. Phase 2 Discourse representation structures (DRSs) are automatically generated from the controlled English. DRSs are a syntactic variant of first-order logic for the resolution of unbounded anaphora. Their use in the interpretation of text is a central element of discourse representation theory [25,26]. Phase 3 The DRSs are automatically translated into MSL. (We have a number of translators in our lab for going from MSL to straight first-order logic (FOL), using long-established theorems [27].) As a DRS is equivalent to a quantified first-order formula, the translation to FOL is not conceptually difficult. Algorithms for performing such translations are provided by Blackburn [28], among others.

We have used this three-phase process in developing machine reading capability in systems we have built with sponsorship from the DoD. In connection with Slate, one of these systems, the three phases can be viewed from a high-level perspective as shown in Figure 7.

Figure 7. Slate’s Reading Process

As we have indicated, in prior logic-based NLP R&D, we have used Attempto Controlled English (ACE), but we are now moving to CELT. CELT is potentially more powerful than ACE because it leverages the Suggested Upper Merged Ontology (SUMO), which incorporates useful knowledge such as geometric axioms. In the current version of our demo, a sentence like Micah puts the teddy bear in the box.

becomes in CELT’s translation to FOL the following formula: (exists (?box ?event ?teddy_bear) (and (instance Micah Human) (attribute ?box Box) (agent ?event Micah) (instance ?event Putting) (instance ?teddy_bear Artifact) (destination ?event ?box) (patient ?event ?teddy_bear))) 5 Phase 1 is currently a manual operation, but techniques developed by Mollá & Schwitter [21] may allow for at least partial automation of this phase.

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters

95

6. On Relevant Prior Work It is important to distinguish our approach from other related research efforts: 1. We are not be doing research on BDI (belief-desire-intention) logics in the standard knowledge-representation tradition [29,30]. Most of that work has focused on codifying normative reasoning (e.g., about knowledge) in modal logics, with little attention paid to practical concerns such as computational efficiency. We are more interested in a descriptive model of reasoning about mental states, rather than a prescriptive model, and one that is computationally tractable. 2. We are not doing cognitive science. While our insights will be obviously coming from human CSP, we will not be particularly concerned with arriving at a cognitively plausible theory. Our aim is not to construct a computational theory which explains and predicts actual human behavior. Rather, our aim is to build artificial agents which are more interesting and useful by enabling them to ascribe mental states to other agents, to reason about such states, and to have, as avatars, states that are correlates to those experienced by corresponding humans. This might result in an interesting computational simulation of a fragment of human CSP, but our primary concern is engineering in the service of entertainment and gaming, not science. In particular, our effort should not be seen as an endorsement of the “theory-theory” position in the current debate between simulation theorists and folk-psychology theorists in cognitive science [31].

Some prior research and development in our own case provides a foundation for the proposed work. For example, with support from NSF, we designed and built synthetic characters (named Ralph and JR9000) that interacted with Bringsjord to teach AI In addition, the primitive synthetic character known simply as E, was demoed at the 2005 GameOn! conference. The associated paper won the “Best Paper” award: [32] In the style of the definition of lying provided above, E is based upon a fully declarative, and fully formal definition of evil.6

7. The Main Obstacle: Meaning Mathematized It’s hard to miss the fact that we are sanguine about our attempt to engineer cognitively robust synthetic characters. In the sober light of AI’s failure to live up to predictions made by its founders, do we really maintain that we can meet with success? Do we not at least see some seemingly insuperable barriers out there ahead of us? Actually, we do see a serious hurdle ahead, one that we currently confess we do not have the means to overcome. We do not have the space to carefully describe this hurdle, which is quite technical, but we give here a synopsis of both the challenge, and our approach to meeting it. The problem is that, to this point, formal logic and formal methods in computer science have provided inadequate mathematical frameworks for pinning down the meaning of the propositional attitudes central to our objectives. What does it mean for a person to believe, know, desire, fear, hope, etc. — where this meaning is provided in fully formal fashion that can be rendered in computational form? This question has not been answered. There have been attempted answers, but they are clearly inadequate, and logicbased AI is only fooling itself when it takes the current answers to be anywhere near ac6A

copy of the paper can be found at http://kryten.mm.rpi.edu/GameOnpaper.pdf.

96

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters

ceptable. For example, Bringsjord and Konstantine Arkoudas have long been convinced that the appropriation of possible-worlds semantics for supposedly specifying the meaning of Knows and Believes has been more a marriage of convenience than anything else. For Necessarily and Possibly, the two operators used in standard modal logic [33], this sort of semantics makes very good sense. For example, it seems eminently sensible to understand ✷φ (a formula in standard modal logic traditionally used to express that φ is logically necessary) to be saying that, in all possible worlds, or in all possible histories, φ holds. The standard correlate for knowing, though, is almost laughably forced. What intuitive sense does it make to say (e.g., with [34]) that an agent knows φ just in case in all possible worlds epistemically accessible for this agent, φ holds? In the course of the project described herein we are going to need to invent a new logical system for the propositional attitudes at the heart of the robust characters we seek. Some of the first steps in this direction have been taken, and we now say a few words about our system. Our approach provides a formal semantics for epistemic operators like Believes and Knows on the basis of epistemic maps. Such maps are a particular kind of graph in which the objects of such operators are propositions situated within a web of interconnected inference. Inference is allowed to be deductive, inductive, abductive, analogical, or (a mode that readers are unlikely to be familiar with) creative. Figure 8 shows a very simple epistemic map M corresponding to a situation in which agent A believes, on the strength of a number of other, subsidiary beliefs (all of which, appearing within rectangles, are perceptually based), that Smith’s cat is paralyzed with fear up in an oak tree on Smith’s property. In this case, only one inference has taken place, an abductive one. Where φ represents in first-order logic that Smith’s cat is stymied in an oak tree, the formula ψ = BelievesA φ is true on this map (written M |= ψ), but because the strength of φ is only at level 1 (“probable”) in the continuum from -4 to 4 of strength factors in our approach (a continuum derived directly from [35]), and this level doesn’t provide a sufficiently strong justification to move A’s belief into the category of knowledge, M |= KnowsA φ.

Figure 8. Simple Epistemic Map Re. Smith’s Cat

We conclude with a final point about our approach. As is well known, ordinary substitution into extensional contexts leads to error. For example, though agent B may know that John Le Carré is the author of A Perfect Spy, it doesn’t follow from this and the fact that Le Carré and David Cornwall are one and the same that B knows that David Cornwall wrote this novel. (After all, B may not have heard or seen the string ‘David

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters

97

Cornwall’ in his entire life.) Any acceptable epistemic logic must respect such blocked inference. In our approach, as shown in Figure 9, an agent A wouldn’t believe that B knows that David Cornwall wrote A Perfect Spy, even if A believes that B believes that John Le Carré wrote this book. The mechanical reason for this is that epistemic maps can themselves contain epistemic mops, but, in general, propositions holding in a map M that contains another map M′ (calM ⊃ M′ ) do not get “injected” into M′ . (On the other hand, for all such pairs of epistemic maps, and all formulae δ, if M′ |= δ, then M |= φ.) Put in terms of Figure 9, the maps shown there don’t satisfy the proposition that A believes that B believes that Cornwall wrote the novel in question.

Figure 9. Opaque Context in Our Approach

References [1] S. Bringsjord, Abortion: A Dialogue. Indianapolis, IN: Hackett, 1997. [2] A. Champandard, AI Game Development. Berkeley, CA: New Riders, 2003. [3] S. Bringsjord, K. Arkoudas, and P. Bello, “Toward a general logicist methodology for engineering ethically correct robots,” IEEE Intelligent Systems, vol. 21, no. 4, pp. 38–44, 2006. [4] S. Bringsjord and D. Ferrucci, Artificial Intelligence and Literary Creativity: Inside the Mind of Brutus, a Storytelling Machine. Mahwah, NJ: Lawrence Erlbaum, 2000. [5] N. Nilsson, “Logic and Artificial Intelligence,” Artificial Intelligence, vol. 47, pp. 31–56, 1991. [6] S. Bringsjord and D. Ferrucci, “Logic and artificial intelligence: Divorced, still married, separated...?,” Minds and Machines, vol. 8, pp. 273–308, 1998. [7] S. Bringsjord and D. Ferrucci, “Reply to Thayse and Glymour on logic and artificial intelligence,” Minds and Machines, vol. 8, pp. 313–315, 1998. [8] S. Bringsjord, “Declarative/logic-based cognitive modeling,” in The Handbook of Computational Psychology (R. Sun, ed.), Cambridge, UK: Cambridge University Press, forthcoming. [9] J. Hawkins, On Intelligence. New York, NY: Holt Paperbacks, 2005. [10] R. Sun, Duality of the Mind. Mahwah, NJ: Lawrence Erlbaum Associates, 2001. [11] D. Lewis, “Psychophysical and Theoretical Identifications,” Australasian Journal of Philosophy, vol. 50, pp. 249–258, 1972. [12] U. Frith, “Cognitive explanations of autism,” Acta Paediatrica Supplement, vol. 416, pp. 63–68, 1996. [13] N. Humphrey, Consciousness regained. Oxford University Press, 1984. [14] D. Premack and G. Woodruff, “Does the chimpanzee have a theory of mind?,” Behavioral and Brain Sciences, vol. 4, pp. 515–526, 1978. [15] M. Clark, Cognitive Illusions and the Lying Machine. PhD thesis, Rensselaer Polytechnic Institute (RPI), 2008.

98

S. Bringsjord et al. / Toward Logic-Based Cognitively Robust Synthetic Characters

[16] R. M. Chisholm and T. D. Feehan, “The Intent to Deceive,” Journal of Philosophy, vol. 74, pp. 143–159, March 1977. [17] H. Wimmer and J. Perner, “Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception,” Cognition, vol. 13, pp. 103–128, 1983. [18] S. Bringsjord, K. Arkoudas, M. Clark, A. Shilliday, J. Taylor, B. Schimanski, and Y. Yang, “Reporting on some logic-based machine reading research,” in Proceedings of the 2007 AAAI Spring Symposium: Machine Reading (SS–07–06), (Menlo Park, CA), pp. 23–28, AAAI Press, 2007. [19] N. E. Fuchs, U. Schwertel, and R. Schwitter, “Attempto Controlled English (ACE) Language Manual, Version 3.0,” Tech. Rep. 99.03, Department of Computer Science, University of Zurich, Zurich, Switzerland, 1999. [20] S. Hoefler, “The Syntax of Attempto Controlled English: An Abstract Grammar for ACE 4.0,” Tech. Rep. ifi-2004.03, Department of Informatics, University of Zurich, Zurich, Switzerland, 2004. [21] D. Mollá and R. Schwitter, “From Plain English to Controlled English,” in Proceedings of the 2001 Australasian Natural Language Processing Workshop, (Macquarie University, Sydney, Australia), pp. 77– 83, 2001. [22] A. Pease and J. Li, “Controlled english to logic translation,” forthcoming. [23] A. Pease and C. Fellbaum, “Language to logic translation with phrasebank,” in Proceedings of the Second International WordNet Conference (GWC 2004) (P. Sojka, K. Pala, P. Smrz, C. Fellbaum, and P. Vossen, eds.), (Masaryk University, Brno, Czech Republic), pp. 187–192, 2004. [24] A. Pease and C. Fellbaum, “An english to logic translator for ontology-based knowledge representation languages,” in Proceedings of the 2003 IEEE International Conference on Natural Language Processing and Knowledge Engineering, (Beijing, China), pp. 777–783, 2003. [25] H. Kamp and U. Reyle, From Discourse to Logic: Introduction to Model-theoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Springer, 1 ed., 1993. [26] H. Kamp and U. Reyle, “A Calculus for First Order Discourse Representation Structures,” Journal of Logic, Language and Information, vol. 5, pp. 297–348, 1996. [27] M. Manzano, Extensions of First Order Logic. Cambridge, UK: Cambridge University Press, 1996. [28] P. Blackburn and J. Bos, “Working with Discourse Representation Theory: An Advanced Course in Computational Semantics.” Forthcoming. [29] M. E. Bratman, Intention, Plans, and Practical Reason. CSLI Publications, 1999. [30] M. Wooldridge, Reasoning About Rational Agents. MIT Press, 2000. [31] M. Davies and T. Stone, eds., Folk Psychology: The Theory of Mind Debate. Blackwell Publishers, 1995. [32] S. Bringsjord, S. Khemlani, K. Arkoudas, C. McEvoy, M. Destefano, and M. Daigle, “Advanced synthetic characters, evil, and E,” in Game-On 2005, 6th International Conference on Intelligent Games and Simulation (M. Al-Akaidi and A. E. Rhalibi, eds.), pp. 31–39, Ghent-Zwijnaarde, Belgium: European Simulation Society, 2005. [33] B. F. Chellas, Modal Logic: An Introduction. Cambridge, UK: Cambridge University Press, 1980. [34] R. Fagin, J. Halpern, Y. Moses, and M. Vardi, Reasoning About Knowledge. Cambridge, MA: MIT Press, 2004. [35] R. Chisholm, Theory of Knowledge. Englewood Cliffs, NJ: Prentice-Hall, 1966.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

99

A Cognitive Substrate for Natural Language Understanding Nicholas L. Cassimatis, Arthi Murugesan and Magdalena D. Bugajska Human-level Intelligence Laboratory, Rensselaer Polytechnic Institute, Troy Abstract. Our goal is to understand human language use and create systems that can use human language fluently. We argue that to a achieve this goal, we must formulate all of the problems for language use from morphology up to pragmatics using the same cognitive substrate of reasoning and representation abilities. We propose such a substrate and described systems based on it. Our arguments, results with real-world systems and ongoing work suggest that the cognitive substrate enables a significant advance in the power of cognitive models and intelligent systems to use human language.

Keywords. Natural language, cognitive substrate.

Introduction Enabling computers to use and understand natural language would greatly increase our ability to model human language use and the power of the human-computer interface. In this paper we will refer to the goal of creating computers that can use and understand natural language as human-level language use (HLU). Achieving HLU will require serious advances in the state of the art in artificial intelligence and cognitive modeling. One consequence of these advances will be inferential and representational computational abilities that can be applied to other interface modalities. The following is a (simplified) enumeration of computational tasks involved in HLU: 1. Infer phonemes, morphemes and words being uttered from an acoustic signal. 2. Infer the syntactic structure the words were intended to form. 3. Given knowledge about the world, people and the specific participants of a conversation, infer the goal of the speaker which generated his utterance. 4. Given a goal, produce an utterance that achieves it. 5. Articulate the utterance. In this paper, we will focus on the goal of inferring syntactic structure and speaker intention, partly because we suspect the insights required to solve these problems will enable fairly straightforward solutions to the other problems. 1. Current AI is not adequate for HLU Let us know examine the state of the art in artificial intelligence and evaluate whether it is adequate to the task of the solving these problems. Many of the individual

100

N.L. Cassimatis et al. / A Cognitive Substrate for Natural Language Understanding

observations we will make are obvious to many. They are mentioned here to consider their ramifications for achieving HLU. 1.1. Parsing and intention recognition are closely related The goal of recognizing the syntax of a sentence anGUHFRJQL]LQJWKHVSHDNHUV¶LQWHQW are closely related. Consider the sentence: ³6KRZPHWKHZHDWKHUIRUWKH5HG6R[JDPH´ 'RHV³IRUWKH5HG6R[JDPH´PRGLI\³WKHZHDWKHU´RU³VKRZ´"7KDWLVDV\QWDFWLF question; but to answer it you have to infer that the speaker intends you to show him WKHZHDWKHUIRUHFDVWLQYROYLQJWKH5HG6R[JDPH³IRU«´PRGLILHV³WKHZHDWKHU´ DQG does not intend you to show him the weather in order to somehow facilitate the Red 6R[JDPH³IRU«´PRGLILHV³VKRZ´DVLQ³VKRZPHWKHZHDWKHUIRUWKHVHFRQGWLPH´  1.2. Modern algorithms for inference and parsing are incompatible The algorithms used in AI for parsing are incompatible with those used in inference. )RUQRZZHFRQVLGHU³PRGHUQ´PDLQVWUHDP$,DOJRULWKPVDQGODWHUGLVFXVVDOJorithms based on more structured formalisms such as scripts and frames. Probabilistic and statistical approaches are popular in AI today, with Markov chain Monte Carlo (MCMC) methods being perhaps the most popular. However, in parsing research, MCMC is shunned for parsers of probabilistic context-free grammars. This is because MCMC (and other algorithms for propagating beliefs in graphical models) require that all of the relevant state variables in a problem be identified before inference proceeds. However, in parsing we do not have all the state variables in advance. We are simply given a string of words and it is our job to infer is which phrases those words were intended to utter. For MCMC to be used to find the most likely parse of a sentence, it must be given the possible parses of a sentence to begin with. For example, to decide ZKHWKHU³WKHGRJ´IRUPDQRXQSKUDVHRU ZKHWKHUDVLQ ³WKHGRJIRRG´WKH\GRQRW MCMC would need to have both possibilities specified in advance. But that is precisely the job of a parser. Even if you had all (or most) of the possible parses, the size of the corresponding graphical model MCMC would be operating over would probably be prohibitively large. (More on this later). Thus, devotees of the statistical approach in the parsing community must use a different inference algorithm than their kin in the rest of AI. The upshot of all this is that it is very difficult to guide parsing using perceptual LQIRUPDWLRQRUZRUOGNQRZOHGJHEHFDXVHWKH³JHQHUDO´LQIHUHQFH algorithms for those are different than the algorithms used by the parsing community. 1.3. Dynamism and scalability The root cause of the tension between PCFG parsers and MCMC is that the latter are not dynamic1 (i.e., they do not enable unforeseen objects) to be inferred and do not scale well as the number of state variable increase. These problems are not specific simply to probabilistic inference algorithms but also apply, e.g., to SAT-based search 1

Dynamic Bayesian Networks are not dynamic in the present sense because they assume that all the state variables are fixed in advance and make inference about how these unfold through time.

N.L. Cassimatis et al. / A Cognitive Substrate for Natural Language Understanding

101

algorithms. Recent approaches come closer to addressing these problems but do not address them fully. For example, [1] only deals with the case were new objects are perceived, not when they are actually inferred. [2] does not enable new objects to be introduced at all, but only delays the representation of grounded relations involving those objects until they are needed. 1.4. World knowledge and scalability The amount of world knowledge and the consequent size of the inference problem are HQRUPRXV&RQVLGHUDJDLQRXUH[DPSOHVHQWHQFH³6KRZPHWKH ZHDWKHUIRUWKH5HG 6R[JDPH´7RUHWULHYHDZHDWKHUIRUHFDVWRQHUHTXLUHVDWLPHDQGa location. Neither of these is explicitly specified in the sentence. The Red Sox could be playing away from Boston, the speaker might not even care about the weather for the game itself, but for the reception of his television satellite, say, in St. Louis, as he watches the game. Which game? The next game, the last game, or the game for which the speaker has tickets? Finally, almost anything can be relevant to answering this question. If marketing for a new kind of egg white product is being debuted during a Red Sox game in two weeks and the speaker is a commodity trader, he might be referring to that game. Thus, quite literally, the price of eggs in China could be relevant to even the most prosaic utterance. This example illustrates that for any given utterance a wide variety of knowledge, LQYROYLQJ WKH VSHDNHU¶V KLVWRU\DQG JRDOVW\SLFDOLQWHQWLRQVRIVSHDNHUVWKHEHKDYLRU of objects and event can all be relevant to using language. What this means is that the state/search/constraint space for the HLU problem is enormous. To illustrate, let us assume very conservatively, that people know on the order of one million facts (as does CYC), that each fact is expressed with about 4 open variables that can each range over 100 objects and that there are 1 billion seconds in a lifetime. This amounts to 100 billion trillion (10^23) concrete constraints. That is well beyond the scope of all general inference algorithms available today (and, we suspect, ever). The problem is even made worse to the extent that understanding a sentence requires an unGHUVWDQGLQJ RI PXOWLSOH SHRSOH¶V PHQWDO VWDWHV  )RU H[DPSOH LI WKH listener knows that the speaker thinks a glass of water is a glass of gin, then he will XQGHUVWDQGZKR³WKHPDQZLWKWKHJLQ´UHIHUVWR(DFKSHUVRQ¶VEHOLHIVWKXVEHFRPHD factor in the understanding of an utterance, which adds at least an integral multiple to WKHFDOFXODWLRQVLQWKHODVWSDUDJUDSK,WLVZRUVHKRZHYHU7KHOLVWHQHU¶VEHOLHIVDERXW WKH VSHDNHU¶V EHOLHIV DERXW WKH OLVWHQHU¶V EHOLHIV DERXW « FDQ SRWHQWLDOO\ EH UHOHYDQW The regress is infinite and, formally at least, so is the state/constraint/search space. HLU will thus require inference algorithms that are extremely scalable. Integration One solution is to abandon general inference algorithms for approaches based on larger structures such as scripts or frames. However, these approaches, by themselves, have proven to be too rigid and inflexible. When a situation changes slightly, a structure that worked might, by itself, not work. What is needed is a combination of both approaches: the speed of structured approaches along with the flexibility of more general approaches. 0RVW UHVHDUFKHV ZKR KDYH WDNHQ LQWHJUDWLRQ VHULRXVO\ KDYH WDNHQ D ³PRGXODU´ approach. They build systems that have, for example, a syntax module that outputs

102

N.L. Cassimatis et al. / A Cognitive Substrate for Natural Language Understanding

possible parses to other (e.g., semantic and pragmatic) modules that settle on the most likely parse and attach a meaning to it. The problem with this approach is that information from each module is needed to constrain information in the others. The flow of information needs to be deeper and more frequent than the modular approach makes easy to achieve. So far, we have enumerated many grounds for pessimism. However, we at least have come to a deeper understanding of what the obstacles. We need better inference (more dynamic, scalable, structured and flexible) and a way to tightly integrate syntactic, semantic and pragmatic inference. The next section outlines an approach. In the preceding discussion, we argued that two key steps to achieving HLU are developing common sense inference algorithms that are scalable, flexible and dynamic and closely integrating inference required for all aspects of language use. In our work, we are addressing these two tasks using the Polyscheme cognitive architecture [3] and the notion of a cognitive substrate [4]. A cognitive substrate for integrating reasoning about language with common sense reasoning Every step of understanding a sentence can be influenced by a wide variety of factors. Deciding which sense of a word to use in parsing a sentence, for example, can be influenced by the perceived state of the world, statistical regularities in word use, the beliefs and goals of the people involved in the conversations and knowledge about how people and the rest of the world work.

Figure 1: Syntactic structure expressed using substrate representations.

Our goal is to formulate each of these kinds of constraints on language interpretation XVLQJ WKH VDPH ³ODQJXDJH´ RU ³IUDPHZRUN´  %\ VR GRLQJ ODQJXDJH understanding becomes not several inference problems (e.g., parsing, intention recognition, etc.) with algorithms for solving each encapsulated within separate

N.L. Cassimatis et al. / A Cognitive Substrate for Natural Language Understanding

103

modules, but a single inference problem in which every aspect of the problem can constrain every other. To exploit this fact computationally, we needed to find a way to represent the VXSHUILFLDOO\YHU\GLIIHUHQWUHDOPVRIFRJQLWLRQHJV\QWD[DQGSHRSOH¶VEHOLHIVXVLQJ the same basic conceptual elements. We have achieved this by showing how to reduce these conceptual fields into the same basic cognitive substrate [4]. This substrate includes reasoning and representation abilities for time, space, causality, identity, parthood and desire. For example, [5] has shown how the problem of syntactic parsing can be reformulated straightforwardly using the same concepts used to formulate constraints on physical actions by agents. Figure 2 shows how to reduce a grammatical rule in HPSG to this substrate. Figure 1 shows an example of a sentence being described in terms of substrate concepts.

Figure 2: An HPSG rule formulated using substrate concepts.

7DEOHLOOXVWUDWHV&DVVLPDWLV¶[6] mapping between elements of syntactic structure and substrate concepts. We have also shown [7] how to reduce representations of beliefs to this same basic substrate of concepts. For example, we can represent belief using a combination of identity and counterfactuality. To say John believes Fred is Honest, we say that in the counterfactual world where John is like Fred (Same(John,Fred,world)), that Fred is honest (Honest(Fred,w)). These kinds of reductions to substrate representations allow all aspects of language understanding to be represented within the same language, making it easier to let each influence the other. Table 1: Mapping of syntactic structures onto syntactic structures. Grammatical Structure Word, phrase, sentence Constituency Phrase structure constraints Word/phrase category Word/phrase order Phrase attachment Coreference/binding Traces Short- and long-distance dependencies

Cognitive structure Event Meronomy Constraints among (parts of) events Event category Temporal order Event identity Object identity Object permanence Apparent motion and long paths.

104

N.L. Cassimatis et al. / A Cognitive Substrate for Natural Language Understanding

2. Implementing a substrate in Polyscheme A substrate that enables all the factors involved in using language to mutually constrain each other must conform to the algorithmic requirements described above. It must integrate the best characteristics of several classes of algorithms by being fast, flexible, scalable and dynamic. In this section we briefly outline how this integration is achieved in the Polyscheme cognitive architecture and direct readers who want more detail to other sources [3, 4, 8]. The fundamental idea behind Polyscheme is that many different kinds of algorithms can be implemented as sequences of attention fixations. These attention fixations in Polyscheme each concentrate multiple computational resources on a single aspect of the world. When algorithms are executed as sequences of attention fixations, integrating these algorithms is as simple as interleaving these sequences. This enables hybrid algorithms that combine the benefits of multiple algorithms. For example, consider the tension between a general search algorithm and a scriptbased reasoner. The search algorithm is general and can adapt to many changes in circumstances but is very slow because meaningful state spaces are prohibitively large. The script-based reasoner will work quickly and effectively when the appropriate script exists for a particular situation. However, slight changes in a situation can invalidate a script and we cannot expect each of these changes to have been anticipated in advance. Our goal is to be able to achieve the generality and flexibility of general search without sacrificing the speed of more structured reasoning algorithms. In Polyscheme, we can achieve this by implementing each algorithm as a sequence of attention fixations. We can look at applying a script as a special case of search in which at each step the operator is chosen from the script. If, when applying a script a problem occurs at a particular step, we can begin to choose operators that a more general search algorithm would choose. The result will be that the elements of a script that are not appropriate for a particular situation will be repaired by the integrated search algorithm. Figure 3 illustrates this process.

Figure 3: A hybrid (C) of systematic, general, flexible but slow search (A) with fast but rigid case-based reasoning (B).

N.L. Cassimatis et al. / A Cognitive Substrate for Natural Language Understanding

105

In ongoing work, we have used this approach to address the tension alluded to above between PCFG parsers and more general probabilistic inference. The fundamental issues that prevent general inference algorithms from parsing PCFGs is that they cannot in the middle of inference add new objects to their reasoning. PCFG parsers constantly do this (e.g., when they infer that a determiner and a noun imply the existence of a noun phrase). Our approach has been to treat PCFG parsing as a maximization problem. We want to find the parse of a sentence with the highest probability of being the parse intended by the speaker. This has led us to implement WalkSAT-inspired algorithm in Polyscheme that maximizes the probability of parses. By integrating this algorithm with a rule matcher we have been able to represent the parsing problem using relational first-order representations instead of propositionally. This also enables objects and relations to be instantiated in memory only when they are needed. Because nonlinguistic relationships can be captured using the rule system, the algorithm automatically takes into account nonlinguistic information when making linguistic inferences. From its point of view, there is no difference between the two types of constraints. Finally, because each step of our algorithm is implemented by a focus of attention that includes information from memory or perceived from the world, this approach enables an embodied approach to sentence processing all the benefits of modern PCFG and probabilistic inference research. This kind of integration in Polyscheme enables suitable cognitive substrate to be implemented. As alluded to before, the substrate implementation we have so far has enabled a unified and integrated account of physical, epistemic and syntactic inference.

3. Implemented systems In our preliminary work combining parsing and general inference, we have been able to formulate grammatical rules and world knowledge in the same substrate and develop a system that used the world knowledge to disambiguate the meaning of a sentence. For example, with knowledge about mechanical devices and the relevant syntax and VHPDQWLFV RXU SDUVHU FRXOG ILQG WKH ULJKW VHQVH RI ³EXJ´ LQ ³WKH EXJ QHHGV D QHZ EDWWHU\´&RQILUPLQJWKHSRZHURIWKHVXEVWUDWHDSSURDFKQRWKLQJVSHFLDOQHHGHGWREH done to the grammar rules or the world knowledge to make them interact. It was a zero-effort natural consequence of formulating each within the substrate.

Figure 4: The person cannot see everything the robot can from its perspective.

106

N.L. Cassimatis et al. / A Cognitive Substrate for Natural Language Understanding

The last section described how basing a system on a substrate implemented in Polyscheme enables language understanding to be facilitated by rich, embodied inference. This approach was used to enable human-interaction with a mobile robot. As is typically done in Polyscheme, a focus of attention was used to implement inference algorithms for semantic understanding and inference about the world. This focus of attention concentrated several computational resources, including a perceptual input from a video processing system. The result was that visual and spatial LQIRUPDWLRQ ZDVXVHGWRLQIHUD SHUVRQ¶VSHUVSHFWLYH DQGXVH DOOWKDWLQIRUPDWLRQ ZDV used to disambiguate natural language references. For example, in one situation (illustrated in Figure 4), a robot could see two cones (A and B) while it could see that the person it was interacting with only ever saw one cone (B). When the person said ³WKH FRQH´ WKH URERW ZDV DEOH WR XVH SHUFHSWXDO LQIRUPDWLRQ DQG VSDWLDO LQIHUHQFH WR infer that the robot was looking at cone B. This required no special encoding of VHPDQWLFLQIRUPDWLRQ7KHLQIHUHQFHZDVVLPSO\DE\SURGXFWRI3RO\VFKHPH¶VDELOLW\ to implement a cognitive substrate in which several kinds of constraints (linguistic, epistemic and spatial) could be given a unified formulation. 4. Conclusions The arguments we have presented, the results with real-world systems we have just described and ongoing work demonstrate that a cognitive substrate and a suitable inference mechanism enable natural language capabilities that will greatly improve the interface between humans and computers.

References [1] B. Milch, B. Marthi, D. Sontag, S. Russell, D. L. Ong, and A. Kolobov, "BLOG: Probabilistic Models with Unknown Objects," presented at IJCAI-05, Edinburgh, Scotland, 2005. [2] P. Domingos and M. Richardson, "Markov Logic Networks," Machine Learning, vol. 62, pp. 107-136, 2006. [3] N. L. Cassimatis, "Integrating Cognitive Models Based on Different Computational Methods," presented at Twenty-Seventh Annual Conference of the Cognitive Science Society, 2005. [4] N. L. Cassimatis, "A Cognitive Substrate for Human-Level Intelligence," Artificial Intelligence Magazine, vol. 27, 2006. [5] A. Murugesan and N. L. Cassimatis, "A Model of Syntactic Parsing Based on Domain-General Cognitive Mechanisms," presented at 8th Annual Conference of the Cognitive Science Society, Vancouver, Canada, 2006. [6] N. L. Cassimatis, "Grammatical Processing Using the Mechanisms of Physical Inferences," presented at Twentieth-Sixth Annual Conference of the Cognitive Science Society, 2004. [7] P. Bello and N. L. Cassimatis, "Developmental Accounts of Theory-of-Mind Acquisition: Achieving Clarity via Computational Cognitive Modeling," presented at 28th Annual Conference of the Cognitive Science Society, Vancouver, Canada, 2006. [8] N. L. Cassimatis, J. G. Trafton, M. Bugajska, and A. C. Schultz, "Integrating Cognition, Perception, and Action through Mental Simulation in Robots," Robotics and Autonomous Systems, vol. 49, pp. 13-23, 2004.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

107

The China-Brain Project Building China’s Artificial Brain Using an Evolved Neural Net Module Approach Hugo de GARIS1, 5, TANG Jian Yu1, 2, HUANG Zhiyong3, BAI Lu1, CHEN Cong1, CHEN Shuo1, GUO Junfei1, TAN Xianjin1, TIAN Hao1, TIAN Xiaohan1, WU Xianjian1, XIONG Ye1, YU Xiangqian1, HUANG Di4 1

Brain Builder Group, International School of Software, Wuhan University, Wuhan, Hubei, China. Email: [email protected] 2 Computer Science School, Hubei University of Economics, Wuhan, Hubei, China 3 Computer Science School, Wuhan University, Wuhan, Hube, China 4 Computer Science School, University of Geosciences, Wuhan, Hubei, China 5 (from early 2008) Brain Builder Group, Institute of Artificial Intelligence, Department of Computer Science, Xiamen University, Xiamen, Fujian, China

Abstract. Prof. Hugo de Garis has recently received a 3 million RMB, 4 year grant to build China’s first artificial brain, starting in 2008, that will consist of approximately 15,000 interconnected neural net modules, evolved one at a time in a special accelerator board [1] (which is 50 times faster than using an ordinary PC) to control the hundreds of behaviors of an autonomous robot. The approach taken in building this artificial brain is fast and cheap (e.g. $1500 for the FPGA board, $1000 for the robot, and $500 for the PC, a total of $3000), so we hope that other brain building groups around the world will copy this evolutionary engineering approach. Keywords. Artificial Brains, Evolved Neural Net Modules, Brain Architects, Accelerator Board

1. Introduction This paper shows how the first author and his research team build artificial brains [2], and how we plan to build the first artificial brain in China, now that we have been funded for it. We define an artificial brain (A-Brain) to be a “network of neural network modules” (10,000–50,000 of them), each of which is evolved quickly in special electronic programmable (FPGA based) hardware, then downloaded into the memory of a PC, and interconnected according to the designs of human “BAs” (“Brain Architects”). The neural signaling of the artificial brain is performed by the PC in real time (defined to be 25Hz per neuron). Such artificial brains can be used for many purposes, e.g. controlling the behaviors of autonomous robots.

108

H. de Garis et al. / The China-Brain Project

However, there is a major problem with the above approach (if one does not use accelerator boards), namely the slowness, using PCs, of evolving individual neural network modules. Typically, it can take many hours, or even a day to evolve a single neural net module on a PC. Obviously, evolving several tens of thousands of such modules using only a PC to build an artificial brain will not be practical. Before such A-Brains can be built with this approach, it is necessary to find a way to accelerate the evolution of such a large number of neural net (NN) modules. This we have done. It is now possible for us to execute the evolution of neural net modules in hardware, and achieve a speedup factor (relative to ordinary PC evolution speeds) of about 50 times. We use a Celoxica company’s FPGA (Field Programmable Gate Array) electronic accelerator board (containing a 3-megagate FPGA, i.e. Xilinx’s Virtex II programmable chip) to accelerate the evolution of neural network modules. In this paper we describe how we build artificial brains (using the accelerator board, which is an essential component in our method). The remaining contents of this paper are as follows. Section 2 provides an overview of how we evolve our neural network modules. Section 3 describes the “Celoxica” electronic board that we use to accelerate the neural net module evolution. Section 4 explains the so-called “IMSI” (Inter Module Signaling Interface), i.e. the software used to allow modules to send and receive signals between themselves. Section 5 describes the properties of the robot that our A-Brain is to control. Section 6 talks about the brain-robot interface. Section 7 gives an example of a multi-module architecture to illustrate how we build our A-Brains. Section 8 presents some examples of single module evolution. Section 9 concludes.

2. Evolving Neural Net Modules This section gives a brief description of the approach that we use to evolve our neural network (NN) modules that become components for building artificial brains. We use a particular neural net model called “GenNet” [3]. A GenNet neural network consists of N (typically N = 12-20) fully connected artificial neurons. Each of the N2 connections has a “weight”, represented as a signed, binary fraction, real number, with p (typically p = 6-10) bits per weight. The bit string chromosome used to evolve the N2 concatenated weights will have a length of N2 (p+1) bits. Each neuron “j” receives input signals from the N neurons (i.e. including a signal from itself). Each input signal Sij (a binary fraction) is multiplied by the corresponding connection weight Wij and summed. To this sum is added an external signal value Ej. This final sum is called the “activation signal” Aj to the neuron “j”. N

Aj = ¦ WijSij + Ej i =1

This activation value is fed into a sigmoid function g that acts as a “squashing” function, limiting the output value Sj to have a maximum absolute value of 1.0 Sj = g ( Aj) =

Aj | Aj | +1.0

Each neuron of a neural network module has a weighted connection to (usually) a single output neuron, whose output signal is considered to be the output signal for the whole module. This output signal S(t) is compared to a target (desired) signal value

H. de Garis et al. / The China-Brain Project

109

T(t) for some number (e.g. 100) “ticks” (where a “tick” is defined to be the time taken for all neurons in the module to signal once). The “fitness function” used in the genetic algorithm (GA) to perform the evolution of the neural net module is usually defined as follows. 1 f = 100 ¦ (T (t ) − S (t ))2 t =1

3. The Celoxica Board The aims of lowering the price of high-speed evolution, and of achieving higher performance in evolving hardware led us to use FPGAs (Field Programmable Gate Arrays). FPGAs are specially made digital semiconductor circuits that are often used for prototyping. The several million logic gates in modern FPGAs (e.g. Xilinx’s Virtex II chip) make it possible to have multiple copies of the same electronic sub circuit running simultaneously on different areas of the FPGA. This parallelism is very useful for a genetic algorithm. It allows the program to process the most time costly weight calculations in parallel, and this can speed up the overall evolution by a factor of about 50 times, as we have determined experimentally in earlier work. We chose the Celoxica FPGA board for our project. “Celoxica” is the name of a UK company [1]. Our Celoxica board (an RC203) costs about $1500. With such a board, a design engineer is able to program electrical connections on site for a specific application, without paying thousands of dollars to have the chip manufactured in mass quantities. We currently use an RC203 FPGA board for our experiments, which is a desktop platform for the evaluation and development of high performance applications. The main FPGA chip is a Xilinx Virtex II that can be configured without using an HDL (Hardware Description Language). Instead, it uses a much easier high-level “C-like” language called “HandelC” (after Handel the composer) [4]. This language is very similar to ordinary C (i.e. with approximately an 80% overlap between the two languages), with a few extra features, particularly those involved with specifying which functions ought to be executed in parallel. A Celoxica board attaches to a PC, with two-way communication, so that instructions can be sent to the board from the PC, and results coming from the board can be displayed on the PC’s screen. One of the aims of this paper is to emphasize that using the Celoxica board makes brain-building practical, due to the considerable speedup factor in the evolution of the individual neural network modules used to make artificial brains. The value of this speedup factor is critical to this whole approach, which aims to make brain building cheap. If a substantial speedup factor can be achieved, it then becomes practical to evolve large numbers of neural net modules in a reasonable time, and to interconnect them to build artificial brains inside a PC.

110

H. de Garis et al. / The China-Brain Project

Fig. 1 The “Celoxica” Board ($1500)

Fig. 2 Our Robot ($1000)

4. Inter Module Signaling Interface (IMSI) In order that each neural net module can calculate the strength of its output signal, it needs to know the strengths of all its input signals, including not only from “intramodule” connections, but also from “inter-module” connections, i.e. either from the “external world” (e.g. from sensors), or from the outputs of other modules. Each module therefore needs a look up table (LUT) which lists the sources of its external input signals (from sensors), and the integer I.D.s of the modules from which it receives their output signals. A similar lookup table is needed to specify to which other modules each module sends its output signal(s) to. One of the jobs of the BAs (Brain Architects) is then to specify the interconnections between the modules, and to enter them into these LUTs. Special software was written for the PC, called “IMSI” (“Inter Module Signaling Interface”) which is used to calculate the neural signaling of each neuron in each module of the artificial brain. An ordinary PC is used to run the IMSI program, which calculates sequentially the output signal value(s) of each module, for all modules in the artificial brain. The IMSI code calculates the output neural signal value for each module in the artificial brain. It loops through each module sequentially, using its input LUT to find the signal values of its external inputs, as well as its internal signal values. A slightly more complex “activation function” A is then calculated according to the following formula Ai = Σ(j=1,N) Wij*Sj + Σ(i=1,P) Ei where a Wij is the weight value of the connection between neuron “i” and “j” in the module, Sj is the value of the signal strength on that connection, Ei is the value of an external neural signal, N is the number of neurons in the module, P is the number of external input signals for that module. The convention used above is that a signal travels from the “from’ neuron “j” to the “to” neuron “i”. Each module has a table of its weight values Wij (i.e. N2 of them) and a list of its external input signals (i.e. P of them, where P usually varies from module to module). The IMSI software calculates the Si

H. de Garis et al. / The China-Brain Project

111

signal value for each module and stores it in its output signal register (“OSR”). This is done sequentially for all modules in the artificial brain. The IMSI then places the output signal value of each module into the input signal registers (“ISR”s) of those modules that the outputting module signals to. For example, let us assume that module M1432 sends its output signal value (0.328) to two other modules M3729 and M3606, then the IMSI would use the “Output Modules LUT” of M1432 to place the value 0.328 into one of the input signal registers in each of the modules M3729 and M3606. Thus the output signals of a given clock cycle (i.e. the time taken for all modules to calculate their output signal values) become the input values at the next clock cycle. Each module also has a table of its internal neural signal values Sj, where j ranges from 1 to N, the number of neurons in the module. These N values are calculated in each clock cycle, to be used in the following clock cycle. In the IMSI code, two such “S tables” are used, in a “ping-pong-ing” style, e.g. table St (calculated in clock cycle “t”) is used to calculate table St+1 (in clock cycle t+1), which in turn is used to calculate table St+2, which actually overwrites table St (hence the term “ping-pong-ing”). Within each clock cycle “t”, the IMSI calculates for each module “m”, its output signal value Sm, and updates its S table. The value Sm is transferred to “m’s” connecting modules. Thus all the data is in place for calculating the Sm values in the next clock cycle. Performing all these calculations is the job of the IMSI software. Some outputs of modules are not sent to other modules but to external “effectors”, e.g. to the motors of a robot, or to a transducer that generates a radio signal, etc. The IMSI deals with these special external outputs (i.e. not to other modules). Actually, the IMSI has two roles. Certainly its main role is as described above, i.e. performing the neural signaling in the PC. Another important role is to allow the BAs (Brain Architects) to specify the connections between the modules. Thus for example, when BAs want to add a new module to the artificial brain, they will need to use IMSI to specify the module’s :a) external inputs (from sensors) b) inputs from other modules (i.e. their integer I.D.s) c) outputs to other modules (i.e. their integer I.D.s) d) external outputs (to effectors) As the size of the artificial brain grows, special book-keeping software is used to describe each module, e.g. its function, its fitness definition, its size (i.e. number of neurons), its evolutionary strategy, its connections with other modules, its position in the whole artificial brain design, etc.

5. The Robot Fig. 2 shows a photo of our robot. It is controlled by a 2 way antenna at the back of the robot, having 4 wheels, a gripper at the front, and a “CMU-CAM2” programmable CCD camera. Visual (and other) data from the robot is transmitted via radio signals to the A-Brain in the PC, and vice versa. This robot cost less than $1000 to build, so is not expensive. Its length is about 20 cms. This robot will be controlled by an artificial brain consisting of at least 15,000 evolved neural net modules that will allow the A-Brain to have many hundreds of pattern recognition circuits, and a similar number of decision circuits. With any publicly funded research project, one needs to show results. An artificial brain “hidden” inside a PC is not visible. Even a large wall chart, consisting of

112

H. de Garis et al. / The China-Brain Project

thousands of interconnected neural net modules, may impress people with its complexity, but would not have the visual impact of a moving robot that displays hundreds of behaviors, switching from one to another depending on circumstances both external and internal. If the average person can remain intrigued for half an hour by such a robot and the A-Brain that controls it, then we state that the A-Brain has passed the “China Test”.

6. The Brain-Robot Interface Our research team has a robotics-mechatronics-electronic-engineering expert, who is working on the interface problem between the robot and the artificial brain in the PC. The CMU-CAM2 camera on the robot is programmable, using an assembler-like language which can be used to process the mega-pixel image that the camera generates. This language contains about 100 different statements. The “robot guy” of our team has the task of becoming an expert in this “camera language”, and then to provide the rest of the team, who are mostly BAs (Brain Architects) and EEs (Evolutionary Engineers) who design and evolve the many neural net modules for the A-brain, with a summary of what the camera outputs, as well as ideas on how these outputs can be translated (interfaced) with possible inputs to neural net modules. This mapping from mega-pixel camera images to single digit inputs to modules is what we call the robot-brain interface. This mapping, which we have yet to specify in detail, will be executed using normal computer code, in the PC (as will the IMSI), so our A-Brain is actually a “hybrid” system, consisting of evolved neural network modules, and standard highlevel computer code. Having a clearer idea of the robot-brain interface will give the research team a better idea of what “first layer” neural net modules to suggest, that take as their inputs, the outputs from the interface mapping. A type of reverse mapping, from the A-brain’s output modules to the motors of the wheels, the camera motor, and the gripper motor, is also needed. Numerical output signal values from neural net modules need to be translated into a form that is suitable for the various motors. Signals between the robot and PC are sent via 2-way radio antenna, so another level of interfacing is needed, i.e. between the antenna and to the motors, and between the effectors (wheels, camera, and gripper) and the antenna that sends signals to the PC. Once our robotist has finished his work, it will be easier for the BAs defining the modules to design them to interface appropriately with the hardware aspects of the project. Of course, the major part of the effort involved in the project remains the specification and evolution of the 10,000s of modules used in the A-Brain.

7. Multi-Module Architecture This section describes a simple artificial brain architecture of several dozen modules, to illustrate the types of issues that arise when trying to design an artificial brain using our approach. For this simple brain (for the illustrative purposes of this paper) we do not use a real robot, but rather a conceptual “toy” robot, which can be modeled as a horizontal rectangular body, with four wheels, and a “V” shaped antenna, as shown in Fig. 3. When designing an artificial brain, certain common sense factors come into play. For example, one needs to answer certain basic questions, such as –

113

H. de Garis et al. / The China-Brain Project

a)

What is the AB (Artificial Brain) to control, i.e. what is the “vehicle” that the AB controls? (In our case the vehicle is the conceptual robot of Fig. 3) b) What environment does the vehicle operate in? c) What are the behaviors of the vehicle that the AB needs to control? d) What are the inputs and outputs of the vehicle? Once these basic questions are answered, the more detailed A-Brain design work can begin. In our simple didactic case, the answers we give to the above questions are the following. The environment in this simple conceptual model is a flat surface that contains vertical triangles and squares, that project onto the camera eye (that looks like a cone in Fig. 3). Sounds are also emitted from the environment of two different frequencies (high and low). The behaviors of the robot are simple. It can turn left slowly or quickly. It can turn right slowly or quickly. It can move straight ahead slowly or quickly, and it can stop. So there are 7 initial behaviors, to be controlled by the ABrain. The inputs to the robot are a vertical triangle or square on the “retinal grid” of the eye (i.e. 2 possibilities), and a high or low frequency sound.

Fig. 3

The Conceptual Robot “Vehicle”

Frequency

Image

Position

Action

high

triangle

L>R

LF

high

triangle

LR

RF

high

square

LR

LS

low

triangle

LR

RS

low

square

L R”). Then this combination of inputs maps to a given output in the form of a “rule” IF (freq = high) & (image = triangle) & (L > R) THEN turn left fast (abbreviated to LF) In light of the above story, this rule makes sense. The frequency is high, so the sound source is close. Therefore the reaction of the robot should be fast, hence the F in the behavioral selection (on the right hand side of the above rule). Since the image is a triangle, that means the object is a prey, to be approached. Since L > R, the object lies to the front-left of the robot, so to approach it, the robot should turn left, and quickly, i.e. the behavior selected is LF. Once one understands this simple rule, then the rest of the rules listed in Fig. 4 are easily interpreted, e.g. in the Action column, an L means “left”, an R means “right”, an A means “approach”, an F means “fast”, an S means “slow”. For example, AS means “approach slowly”, LF means (turn) “left fast”. Most of the actions in Fig. 4 are turning behaviors. Once the robot has turned enough, so that the signal strength difference between the two SSDs on the V antenna is effectively zero, the robot then moves straight ahead, i.e. it approaches the source of the sound, or flees from it. The stop behavior creates complications, so we will ignore it in this simple didactic model. So effectively there are 6 behaviors. Now that we understand the “story”, i.e. the general behavior of the robot, we can now turn to a more detailed discussion of neural net modules that can implement these behaviors. These modules are now listed. We need – a) 2 detectors for the image, i.e. a “triangle detector”, and a “square detector” b) 1 SSD (signal strength detector) (2 copies, for the two branches of the V antenna) c) 2 frequency detectors (one for the high frequency, one for the low frequency) d) 3 difference detectors (i.e. L > R, L < R, L = R). e) 2 logic gates (“and”, “or”) That's only 10 different modules, but it is enough for the didactic purposes of this section. These modules can be combined (i.e. interconnected) to form networks of

115

H. de Garis et al. / The China-Brain Project

modules (i.e. a form of networks of (neural) networks), called “circuits” (or “subsystems”), where the output(s) of some modules become the inputs of other modules, e.g. for the above rule, the following circuit could be used, as shown in Fig. 5. Similar circuits can be made for the other 11 cases of Fig. 4. But we need not make 12 separate circuits similar to that in Fig. 5. That would be wasted effort. Instead we can use “or” gates to aggregate several cases. For example, Fig. 4 shows that there are 3 cases that give an LF output (or action). So make those 3 LF outputs become the inputs to a 3 input “OR” gate – and similarly for LS (3 cases), RF (2 cases), RS (2 cases). AF and AS only have single cases, so they don't need an “OR” gate. One need not replicate SSDs, nor difference detectors (L > R), (L < R), (L = R). One can simply have the outputs of these detectors branch to become inputs to multiple other circuits. Putting all these 12 (non redundant) circuits together would generate quite a complex circuit, which is not given in this section.

u

Left Wheel

Freq = high? L>R Triangle? SSD (L)

SSD (R)

Sound (L) Sound (R) Fig. 5 Circuit for LF Rule

Right Wheel

LS

0.2

0.4

LF

0.2

0.8

RS

0.4

0.2

RF

0.8

0.2

AS

0.4

0.4

AF

0.8

0.8

Fig. 6 Control Signals to Wheels for Robot Turning

Similar circuits can be made for the other 11 cases of Fig. 4. But we need not make 12 separate circuits similar to that in Fig. 5. That would be wasted effort. Instead we can use “or” gates to aggregate several cases. For example, Fig. 4 shows that there are 3 cases that give an LF output (or action). So make those 3 LF outputs become the inputs to a 3 input “OR” gate – and similarly for LS (3 cases), RF (2 cases), RS (2 cases). AF and AS only have single cases, so they don't need an “OR” gate. One need not replicate SSDs, nor difference detectors (L > R), (L < R), (L = R). One can simply have the outputs of these detectors branch to become inputs to multiple other circuits. Putting all these 12 (non redundant) circuits together would generate quite a complex circuit, which is not given in this section. How can we implement the LF, LS, AF, AS, RF, and RS? Common sense says that to get a 4 wheeled vehicle to turn left, one needs to make the right side wheels turn faster than the left side wheels. So use 3 more modules that output a constant signal of a low value (e.g. 0.2) and a constant signal of a middling value (e.g. 0.4), and a constant signal of a high value (e.g. 0.8). These different output signal values can be used to turn the wheels at different speeds. A high signal (0.8) will turn the wheel it controls quickly. A middling signal (0.4) will turn its wheel at a middling speed, etc. Fig. 6 gives the combination of control signals to make the robot turn appropriately. This brings the total of different modules now to 13. If the behavior LF is activated,

116

H. de Garis et al. / The China-Brain Project

then it can send two output signals which become the inputs to modules that generate the 2 control signals for the wheels (i.e. 0.2 and 0.8). This simple circuit is shown in Fig. 7. There are 6 different behaviors (LF, LS, AF, AS, RS, RS), with different combinations of control signal strengths to the wheels, so how to solve the problem that only one set of signals should be sent to the wheels at a time? With 6 circuits functioning simultaneously, it is possible to have several of the six having non negligible output values at the same time. This creates a conceptual problem. We want to have only one of these 6 to be active (e.g. a strong output value of 0.8) and the rest to be inactive (i.e. with weak output values of 0.2). Before proceeding further, it is interesting to note that the problem we are now discussing is fairly typical of the role of a brain builder or a “BA” (Brain Architect). It is analogous to the case of an electronic engineer who designs digital electronic circuits. There is a lot of creativity involved, and there may be “many ways to skin a cat” (i.e. many alternatives to solving a problem, some of them being more intelligent, or more efficient than others). Returning to the problem – how can only one of the 6 signals be strong and the other 5 weak? This sounds like a “winner takes all (WTA)” problem. So we suggest creating a circuit that has 6 inputs, and 6 outputs. If the ith input signal is the strongest of the 6 input signals, then the ith output signal is strong (e.g. 0.8) and all the other output signals are weak (e.g. 0.2), as shown in Fig. 8. How to design such a circuit using evolvable neural net modules? (One sees the scope for creativity here!) The very idea of using a WTA circuit may not be the best way to solve the “unique signal problem” (i.e. ensuring that only one set of signals is sent to the wheel motors). An alternative might be to add the output signals of the 6 behaviors (i.e. LF + LS + AS + AF + RF + RS). This will generate some inaccuracies for a time, but so long as the behaviors don't change very fast from one to another, there will probably be time for the contributions of the 5 non active behaviors to drop to low values, leaving only the active behavior. (On the other hand a weak signal is about 0.2, so the sum of 5 weak signals is roughly equal to one strong one, so we still have a problem). So let us pursue the WTA approach. How to implement such a circuit? Trying to evolve a 6-input, 6output circuit is probably too difficult, so our instinct is to use a “divide and conquer” approach, i.e. evolve simpler modules and then connect them to create the WTA circuit. Consider a “dominator” circuit, which has 3 inputs A, B, C, and one output O as shown in Fig. 9. If the input signal A (on the left) is the largest of the 3 input signals, then the output O takes the input value A, else, O takes a very low value (e.g. 0.1 or 0.05). With two such circuits we can combine them with a “greater than” circuit to create a WTA circuit (component) provided that the left most of the 6 inputs is the largest, as shown in Fig. 10. It should now not be too difficult to see that with 6 such components, we can construct a winner take all (WTA) circuit. For example, if the input B has the largest input signal strength, then if we swap the inputs A and B, then the using the circuit of Fig. 10, we should get a strong output (e.g. 0.8) indicating that B was the strongest input. Similarly, if we swap A and C, then if C is the strongest, again the output O will be strong (else weak), and so on for the remaining 3 cases. With 6 such circuits, we can construct a full WTA.

117

H. de Garis et al. / The China-Brain Project

0.3 LF

.2

0.2 to left wheels

.8

0.8 to right wheels

LF

Fig. 7 Control Signals to Wheels for LF

0.4

0.2

0.6

0.5

0.4

“Winner Takes All” Circuit

0.2

02

0.2

0.8

0.2

0.2

Fig. 8 Winner Takes All (WTA) Circuit

The 6 outputs from the WTA circuit (one of which will be strong (0.8), and the other five very weak (0.05) are fed into 6 circuits of the form shown in Fig. 7. The 6 “left wheel” output signals and the 6 “right wheel” output signals can be “paired”, i.e. 2 of the left wheel outputs have a value of 0.1, 2 have a value of 0.4, and 2 have a value of 0.8 Similarly with the right wheel outputs. But instead of having 6 circuits as in Fig. 7 we can simplify things a bit as in Fig. 11. Using the data of Fig. 4, an LS output needs to send a 0.2 signal to the left motor and a 0.4 signal to the right motor. An LF output sends signals 0.2 and 0.8 respectively to the left and right motors. Hence LS and LF both send a 0.2 signal to the left motor. By connecting the two left motor signals to an OR gate as shown in Fig. 11 if one of LS or LF is active from the WTA circuit, the OR gate will output a strong signal. In the case of the LS/LF “OR” gate, if one of the LS or LF is active, then the output of the OR gate is strong. This output connects to a new module called “Multigen”, which generates a multiple of outputs depending on which input has a strong signal. If the left input is strong (0.8) the output is a constant signal of 0.2 - if the middle input is strong, the output is a constant signal of 0.4 - if the right input is strong, the output is a constant of 0.8 Outputs go to the motors.

8. Single Module Evolution This didactic section describes how a selection (a subset) of the modules discussed in the previous section can be evolved. The first author is currently writing a book entitled “Artificial Brains : An Evolved Neural Net Approach”, to be published by World Scientific (Singapore) early 2010. In this book will be the descriptions of the actual successful evolutions of many modules (probably many dozens) to give readers and potential brain builders a fairly thorough introduction as to how to evolve individual neural net modules, that is so critical to this approach of building artificial brains. In this section we give only two, because describing each one takes quite a bit of time. We start with a rather simple example, i.e. the Multigen module of Fig. 11. Remember its specification. It has 3 inputs and one output. If the left most input is high, and the others low, it outputs a constant signal of 0.2 If the middle input is high, it outputs a constant signal of 0.4 If the right most input is high, it outputs a constant signal of 0.8 How can one evolve such a module? We use a “multi-test” approach, which is a technique commonly used in our work. In this case there are 3 tests, or experiments, i.e. the 3 cases of input combinations (i.e. left high, middle high, right

118

H. de Garis et al. / The China-Brain Project

high). Let us run each test (i.e. arrange to have the appropriate input signal values) for a given number of “ticks”, e.g. 33 ticks for each experiment. So the total number of ticks for the “multi-test” experiment will be 100 ticks. Fig. 12 shows the target output signal values for the “multi-test evolution”. To evolve this module, three experiments or tests are carried out on the same module, hence the word “multi-test”. For the first 33 ticks, 3 inputs (left to right) of value (0.8, 0.2, 0.2) are presented. For the second 33 ticks, the 3 inputs are (0.2, 0.8, 0.2), and for the third 33 ticks, the 3 inputs are (0.2, 0.2, 0.8). At the junctions of 33, 66 ticks, the internal neuronal signals are reset to zero, so that a “fresh” experiment can begin. The fitness definition is 1

f =

100

¦ (T (t ) − S (t ))

2

t =1

B

B A

E

A

C

C

D

Dominator

F

Dominator

Dominator

>

O

Fig. 9

LS

LF

RS

Fig. 10 Winner Takes All (WTA) Circuit Component

Dominator Circuit

RF

AS

AF Output Signal Strength

0.8

0.4 0.2 R

R

R

R

R

R

33

100

66

t (ticks)

MULTIGEN

MULTIGEN Fig . 12 Target Output Values for the Multigen Module

to left motor

to right motor .

Fig. 11 Motor Control Circuit

119

H. de Garis et al. / The China-Brain Project

The T(t) is the target (desired) output signal value. It takes the value 0.2 for the first 33 ticks, then 0.4 for the second 33 ticks, and 0.8 for the third 33 ticks. S(t) are the actual output signal values from the evolving module. The closer the S(t) values are to the target values T(t), the higher the fitness score f. The fitness is a simple inverse of the sum of the squares of the differences between the T and S values for each tick t. For a second example, take the case of the frequency detector in Fig. 5. We create a training set of “positive” and “negative” examples. Let us assume that the environmental model in which the robot is situated contains only 2 frequencies, i.e. high and low, e.g. a high frequency with period of 20 ticks, and a low frequency with period 40 ticks. Assume the strengths of the sound signals received by the robot take the following form. S(t) = Ai sin(2πt/T) Let the amplitude of the signal, i.e. Ai have 3 different values, 0.2, 0.5, 0.8 Hence there are 6 cases, i.e. 3 different possibilities for the amplitude, and 2 cases for the period T. There will be thus 3 positive and 3 negative examples, corresponding to the whether the period is small (high frequency) or large (low frequency). Use a 6 case multi-test evolution, in which the 6 cases are presented each for 80 ticks. For the positive cases, the target output is high i.e. 0.8, and for the negative cases the target output is low i.e. 0.2 The above modules are fairly simple in their function and could be readily programmed conventionally in a high level computer language, so why bother evolving them? This next example, a pattern detector, can be evolved rather quickly, but it is not at all obvious how one might program it, and especially in a comparable time. We anticipate that there will be several thousand pattern detector modules in our 15000 module A-Brain, so we feel justified in our evolutionary engineering approach, when people challenge us with comments such as “Why don't you just program everything?” The CMU-CAM2 camera generates a mega-pixel image that can be data compressed to a much smaller number of pixels, each with its own grey level or color code. For didactic purposes, imagine this compressed grid has dimensions 10*10. Each pixel of the grid has an output signal with magnitude less than 1.0 Each such pixel value is sent to every neuron in a pattern detector module, so if the module has 16 neurons, there would be 16*10*10 (external) connections, and hence weights, to the module i.e. 1600. Imagine the module is to output a strong signal if the image contains a square (a positive example), and a weak signal if it contains a triangle (a negative example), i.e. it is a “square detector”. A training set of images containing positive and negative examples is prepared, e.g. squares and triangles of different sizes and positions in the image. Each image is presented for 50 ticks, and the internal signals between the neurons in the module are cleared between each test, i.e. between the presentation of each image. The fitness function for this “multi-test evolution” is the inverse of the following formula (T – P) Σ t Σ

(+ ve tests) (0.8

– S(t))2 + P Σ

t

Σ

(-ve tests) (0.1

– S(t))2

where T is the total number of images, P is the number of positive (square) images, so T-P is the number of negative (triangle) images. S(t) is the strength at each tick of the output signal of the module. The double sums are over the 50 ticks (t), and the positive and negative test cases. The factors (T – P) and P before the summation signs are used

120

H. de Garis et al. / The China-Brain Project

to “balance” the evolution, i.e. to steer the evolution equally between the negative and the positive cases. Such pattern detector modules can often be evolved quickly using the Celoxica board, and in much less time than they could be conventionally programmed (assuming that one can even think of a way to program such cases quickly, which is often not the case.) For more examples of single neural net module evolution, readers will have to wait until the first author’s book “Artificial Brains : An Evolved Neural Net Module Approach” is published early 2010.

9. Conclusions The Celoxica FPGA board is capable of speeding up the evolution of neural network modules (relative to that on a PC) by a factor of about 50 times, depending on the size of the neural net being evolved. We believe this to be an important advance and an essential step when artificial brains, comprised of 10,000s of such modules are to be evolved cheaply over a reasonable time period, and then run in real time in interconnected form in an ordinary PC. Prior to the use of the Celoxica board, the evolution of a single neural network could take many hours on an ordinary PC, a fact that made brain building according to our PC-based approach quite impractical. The underlying technology, i.e. the electronic evolution of large numbers of neural net modules, will make the production of 10,000s of evolved modules needed to build an artificial brain, practical. Then the real challenge of designing an artificial brain can begin, and will result hopefully in the creation a new research field, namely “Brain Building” or “Artificial Brains”. This paper defined an artificial brain to be a network of evolved neural network modules. Each module is evolved quickly using a (Celoxica) electronic accelerator board (i.e. about 50 times faster than using a PC). The evolved weights of each neural net module are downloaded into the RAM memory of the PC. This is done thousands of times, one by one, up to a maximum of several 10,000s (which is the limit with which the PC can perform the neural signaling of all the (interconnected) modules sequentially in real time, where real time means the PC calculates the output signal value of every neuron in the whole A-Brain 25 times per second (25 Hz). Once all the modules are downloaded into the PC’s memory, they are interconnected according to the designs of human “BAs” (Brain Architects). The interconnected network of neural networks is the A-Brain (Artificial Brain). When the PC is calculating each neuron’s output signal, it uses the IMSI’s LUTs to know where its inputs come from, and to where it is to send its output signals. Today’s PCs allow the real time neural signaling of A-Brains with up to a maximum of several 10,000s of modules. To observe a 15,000 module A-Brain controlling the hundreds of behaviors of a robot should give the impression to human observers that the robot “has a brain behind it”. Longer term, the research field of “Artificial Brains” should grow in size equivalent to those of NASA, ESA, etc.

H. de Garis et al. / The China-Brain Project

121

References [1] Celoxica 2006, www.celoxica.com [2] Hugo de Garis, Michael Korkin, "The Cam-Brain Machine (CBM) : An FPGA Based Hardware Tool which Evolves a 1000 Neuron Net Circuit Module in Seconds and Updates a 75 Million Neuron Artificial Brain for Real Time Robot Control", Neurocomputing, Vol. 42, Issue 1-4, February, 2002, Elsevier. Special issue on Evolutionary Neural Systems, guest editor Prof. Hugo de Garis, http://www.iss.whu.edu.cn/degaris/papers [3] PPt notes for a masters level Artificial Brains course http://www.iss.whu.edu.cn/degaris/coursestaught.htm Click on the CS7910 course. [4] The “Handel-C” High Level Language for Programming the Celoxica board, 2006, www.celoxica.com [5] Hugo de Garis, “Guest Editorial”, Neurocomputing, Vol. 42, Issue 1-4, February, 2002, Elsevier. Special issue on Evolutionary Neural Systems, guest editor Prof. Hugo de Garis. [6] Hugo de Garis, Michael Korkin, "The CAM-Brain Machine (CBM) : Real Time Evolution and Update of a 75 Million Neuron FPGA-Based Artificial Brain", Journal of VLSI Signal Processing Systems, Special Issue on VLSI on Custom Computing Technology, Vol. 24, Issue 2-3, (2000), pp 241-262. [7] Hugo de Garis, Michael Korkin, Gary Fehr, "The CAM-Brain Machine (CBM) : An FPGA Based Tool for Evolving a 75 Million Neuron Artificial Brain to Control a Lifesized Kitten Robot", Journal of Autonomous Robots, Vol. 10, No. 3, May 2001. [8] Hugo de Garis, Michael Korkin, Felix Gers, Eiji Nawa, Michael Hough, "Building an Artificial Brain Using an FPGA Based CAM-Brain Machine", Applied Mathematics and Computation Journal, Special Issue on "Artificial Life and Robotics, Artificial Brain, Brain Computing and Brainware", Vol. 111, 2000, pp163-192, North Holland. [9] Hugo de Garis, Felix Gers, Michael Korkin, Arvin Agah, Norberto 'Eiji' Nawa, "CAM-Brain" : ATR's Billion Neuron Artificial Brain Project : A Three Year Progress Report", Artificial Life and Robotics Journal, Vol. 2, 1998, pp 56-61. [10] Harik G.R., Lobo F.G., Goldberg D.E., “The Compact Genetic Algorithm”, IEEE Transactions on Evolutionary Computation, Volume 3, Issue 4, Nov 1999, pp 287 – 297.

122

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

Cognitive Architectures: Where do we go from here? Wáodzisáaw DUCHa,1, Richard J. OENTARYOb, and Michel PASQUIERb a

Dept. of Informatics, Nicolaus Copernicus University, ToruĔ, Poland School of Computer Engineering, Nanyang Technological University, Singapore

b

Abstract. Cognitive architectures play a vital role in providing blueprints for building future intelligent systems supporting a broad range of capabilities similar to those of humans. How useful are existing architectures for creating artificial general intelligence? A critical survey of the state of the art in cognitive architectures is presented providing a useful insight into the possible frameworks for general intelligence. Grand challenges and an outline of the most promising future directions are described.

Keywords: cognitive architectures, artificial general intelligence, neurocognitive models, intelligent agents.

1. Introduction A long-term goal for artificial general intelligence (AGI) is to create systems that will exceed human level of competence in a large number of areas. There is a steady progress toward this goal in several domains, including recognition of specific patterns, memorization and retrieval of vast amount of information, interpreting signals and other types of numerical information, autonomous control, board games and reasoning in restricted domains. Yet even in lower level cognitive functions, such as object recognition or scene analysis artificial systems are still far behind the natural ones. Higher-level cognitive functions, such as language, reasoning, problem solving or planning, involve complex knowledge structures and are much more difficult to realize. Various types of memory stored by the brain facilitate recognition, association, semantic interpretation and rapid retrieval of large amounts of complex information patterns. At quite basic level organization of storage and retrieval of information in computers is completely different than in brains. Computing architectures are universal only in principle, in practice they always constrain information processing in specific ways. Computers are better in many tasks than brains, and vice versa, brains are better in many important tasks then computers. It is not clear at all whether cognitive architectures (CA) running on conventional computers, will reach the flexibility of the brain in lower or higher-level cognitive functions. Traditionally higher cognitive functions, such as thinking, reasoning, planning, problem solving or linguistic competencies have been the focus of artificial intelligence (AI), relaying on symbolic problem solving to build complex knowledge structure. These functions involve sequential search processes [1], while lower cognitive functions, such as perception, motor control, sensorimotor actions, associative memory recall or categorization, are accomplished on a faster time scale in a parallel way, without stepwise deliberation. Embodiment is a powerful trend in robotics and there is now a general agreement that the meaning of many concepts should be grounded in embodied, sensorimotor representations. While symbolic approximations that account for sensorimotor processes are certainly possible not much is known about their limitations, for example how deep the grounding of symbols should be, and how to achieve it through embodied cognition. Perhaps the dream of creating a General Problem Solver [2] may be realized with relatively minor extensions to symbolic cognitive architectures, while detailed understanding of animal behavior and creating flexible mobile robotic applications may require a distributed approach to embodied cognition. Analysis of existing cognitive architectures should facilitate understanding of limitations of different approaches. Many general ideas seem to explain everything but do not scale up well to real applications, therefore a clear notion what exactly AGI should do is necessary. 1

Corresponding author, Dept. of Informatics, Nicolaus Copernicus Uni., Grudziądzka 5, ToruĔ, Poland, Google: “W. Duch”.

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

123

2. Grand challenges for AGI What should be required from an AI system to be worthy of the “Artificial General Intelligence” name? Artificial Intelligence has focused on many specific approaches to problem solving, useful for development of expert systems, neglecting its initial ambitious goals. One requirement for AGI, storing and manipulation of vast amount of knowledge, has been addressed by the Cyc project [3]. Started in 1984 a huge frame-based knowledge base has been constructed, but the list of its “potential applications” has not been replaced by real applications for decades. Perhaps the biggest mismatch between AI reality and popular expectations is in the language-related domains, for example in general purpose conversational systems, developed mostly in the form of various chatterbots by commercial companies and enthusiastic individuals. Restricted form of the Turing test [4] (the full test being too difficult to try), called Loebner Prize competition2, has been won for almost two decades by chatterbots based on old template matching techniques, or more recently contextual pattern matching techniques. Such programs have no chance to develop real understanding of language and use it in meaningful dialogs or texts analysis, but may be used for stereotyped question/answer systems or “impersonation”. Carpenter and Freeman have proposed a “personal Turing test” [5], where a person tries to guess if the conversation is done with a program or a real personally known individual. Human behavior includes the ability to impersonate other people, and the personal Turing test may be an interesting landmark step on the road to general intelligence. Another area that poses remarkable challenge to AI is word games, and in particular the 20-questions game. Word games require extensive knowledge about objects and their properties, but not about complex relations between objects. Different methods of knowledge representation may be used in different applications, from quite simple, facilitating efficient use of knowledge, to quite involved, needed only in deep reasoning. In fact simple vector-space techniques for knowledge representation are sufficient to play the 20question game [6]. Success in learning language depends on automatic creation and maintenance of largescale knowledge bases, bootstraping on the resources from the Internet. Question/answer systems pose even more demanding challenge, and in this area a series of competitions organized at Text Retrieval Conference (TREC) series may be used to measure progress. Intelligent tutoring systems are the next great challenge, but there seem to be no clearly defined milestones in this field. Feigenbaum [7] proposed as a grand challenge building a super-expert system in a narrow domain. This seems to go in a direction of specialized, rather than general intelligence, but one may argue that a superexpert without general intelligence needed for communication with humans is not feasible. Sophisticated reasoning by human experts and artificial systems in such fields as mathematics, bioscience or law may be compared by a panel of experts who will pose problems, rise questions, and ask for further explanations to probe the understanding of the subject. A good example of such challenge is provided by the Automated Theorem Proving (ATM) System Competitions (CASC) in many sub-categories. An interesting step toward general AI in mathematics would be to create general theorem provers, perhaps using meta-learning techniques that rely on specialized modules. Automatic curation of genomic/pathways databases and creation of models of genetic and metabolic processes for various organisms poses great challenges for super-experts, as the amount of information in such databases exceeds by far human capacity to handle it. Defining similar challenges and milestones towards AGI in other fields is certainly worthwhile. The ultimate goal would be to develop programs that will advice human experts in their work, evaluating their reasoning, perhaps even adding some creative ideas. DARPA in the “Personal Assistants that Learn” (PAL) program sponsors a large-scale effort in similar direction. Nilsson [8] has argued for development of general purpose educable systems that can be taught skills needed to perform human jobs, and to measure which fraction of these jobs can be done by AI systems. Building one such system replaces the need for building many specialized systems, as already Allan Turing [4] has noted proposing a “child machine” in his classical paper. Some human jobs are knowledge-based and can be done by information processing systems, where progress may be measured by passing a series of examinations, as is done in such fields as accounting. However, most human jobs involve manual labor, requiring senso-motoric coordination that should be mastered by household robots or autonomous vehicles. The DARPA Urban Challenge competition (2007) required integration of computer vision, signal processing, control and some reasoning. It is still simpler than control of a humanoid robot, where direct interaction of robots with people will require an understanding of perception, controlling of attention, learning casual models from observations, and hierarchical learning with different temporal scales. Creation of partners or personal assistants, rather than complete replacements for 2

See http://www.loebner.net for information on the Loebner competition.

124

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

human workers, may be treated as a partial success. Unfortunately specific milestones for this type of applications have yet to be precisely defined. Some ordering of different jobs from the point of view of difficulty to learn them could be worthwhile. In fact many jobs have already been completely automatized, reducing the number of people in manufacturing, financial services, printing houses etc. In most cases alternative organization of work is to be credited for reduction in the number of jobs (plant and factory automation, ATM machines, vending machines), not because of deployment of AI systems. A detailed roadmap to AGI should thus be based on detailed analysis of the challenges, relationships between various functions that should be implemented to address them, system requirements to achieve these functions and classes of problems that should be solved at a given stage.

3. Cognitive architectures Cognitive architectures are frequently created to model human performance in multimodal multiple task situations [1][9] rather than to create AGI. A short critical review of selected cognitive architectures that can contribute to development of AGI is provided below. Allen Newell in his 1990 book Unified Theories of Cognition [1] provided 12 criteria for evaluation of cognitive systems: adaptive behavior, dynamic behavior, flexible behavior, development, evolution, learning, knowledge integration, vast knowledge base, natural language, real-time performance, and brain realization. These criteria have been analyzed and applied to ACT-R, Soar and classical connectionist architectures [10] but such fine-grained categorization makes comparison of different systems rather difficult. Without going into such details we shall propose below a simpler taxonomy, give some examples of different types of cognitive systems that are currently under development, and provide a critique and some recommendations for better systems. Surveys on the system organization and working mechanisms of a few cognitive architectures that have already been published [11] were not written from the AGI point of view. Two key design properties that underlie the development of any cognitive architecture are memory and learning. The importance of memory has been stressed from different perspectives in a few recent books [12][14]. Various types of memory serve as a repository for background knowledge about the world and oneself, about the current episode of activity, while learning is the main process that shapes this knowledge. Together learning and memory form the rudimentary aspects of cognition on which higher-order functions and intelligent capabilities, such as deliberative reasoning, planning, and self-regulation, are built. Organization of memory depends on the knowledge representation schemes. A simple taxonomy of cognitive architectures based on these two main features leads to a division of different approaches into three main groups (Fig. 1): symbolic, emergent, and hybrid models. Cognitive architectures

Symbolic

Memory

Emergent

Memory

Hybrid

Memory

Rule-based memory

Globalist memory

Localist-distributed

Graph-based memory

Localist memory

Symbolic-connectionist

Learning

Learning

Learning

Inductive learning

Associative learning

Bottom-up learning

Analytical learning

Competitive learning

Top-down learning

Fig. 1. Simplified taxonomy of cognitive architectures

Roughly speaking symbolic architectures focus on information processing using high-level symbols or declarative knowledge, in a classical AI top-down, analytic approach. Emergent architectures use low-level activation signals flowing through a network consisting of numerous processing units, a bottom-up process

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

125

relaying on the emergent self-organizing and associative properties. Hybrid architectures result from combining the symbolic and emergent paradigms in one way or another. The memory and learning aspects of these three broad classes of approaches are investigated in details below. 3.1. Symbolic architectures There is a strong link between the type of architecture and problems it is supposed to solve. The use of symbols as the key means to support information processing originates from the physical symbol system hypothesis [1], which has been motivated by research on memory and problem solving. A physical symbol system has the ability to input, output, store and alter symbolic entities, and to carry out appropriate actions in order to reach its goals. The majority of symbolic architectures utilize a centralized control over the information flow from sensory inputs through memory to motor outputs. This approach stresses the working memory executive functions, with an access to semantic memory for knowledge retrieval. Rule-based representations of perception-action memory in knowledge-based production systems embody the logical reasoning skills of human experts. Graph-based representations are typically encoded as directed graph structures comprising nodes for symbolic entities and their attributes, and edges for relationships among them. Main examples of this sort of knowledge representation are semantic networks and conceptual graphs (Sowa 1984), frames/schemata [16], and reactive action packages (RAPs) [17]. The underlying paradigms of these approaches remain very similar, and are sometimes even equivalent. Substantial efforts have been made over the years to introduce analytical and inductive learning techniques to symbolic systems. The former aims at exploiting existing general/specific facts to infer other facts that they entail logically. Prominent examples include explanation-based learning (EBL) [18] and analogical learning [19]. Inductive machine learning, on the other hand, seeks to derive from specific facts or examples general rules which capture the underlying domain structure. Well-known examples of this kind include knowledge-based inductive learning (KBIL) [20] and delayed reinforcement learning [21]. Many ambitious cognitive architectures have been proposed and abandoned after a period of vigorous activity. Only those potential candidates for AGI that are still actively developed are reviewed below. SOAR (State, Operator And Result) is a classic example of expert rule-based cognitive architecture designed to model general intelligence [1][22]. Based on theoretical framework of knowledge-based systems seen as an approximation to physical symbol systems, SOAR stores its knowledge in form of production rules, arranged in terms of operators that act in the problem space, that is the set of states that represent the task at hand. The primary learning mechanism in SOAR is termed chunking, a type of analytical EBL technique for formulating rules and macro-operations from problem solving traces [22]. In recent years many extensions of the SOAR architecture have been proposed: reinforcement learning to adjust the preference values for operators, episodic learning to retain history of system evolution, semantic learning to describe more abstract, declarative knowledge, visual imagery, emotions, moods and feelings used to speed up reinforcement learning and direct reasoning 3 . SOAR architecture has demonstrated a variety of high-level cognitive functions, processing large and complex rule sets in planning, problem solving and natural language comprehension (NL-SOAR) in real-time distributed environments4. At present SOAR architecture has not yet integrated all these extensions. A few additional ones, like memory decay/forgetting, attention and information selection, learning hierarchical representations, or handling uncertainty and imprecision, will also be useful. The design of the perceptual-motor systems within SOAR is fairly unrealistic, requiring users to define their own input and output functions for a given domain. It remains to be seen how well numerous problems that face such an extension can be handled using the existing architecture as a base. EPIC (Executive Process Interactive Control) is a cognitive architecture for building computational models that subsume many aspects of human performance [23]. It aims at capturing human perceptual, cognitive and motor activities through several interconnected processors working in parallel, and to build models of human-computer interaction for practical purposes. The system is controlled by production rules for cognitive processor and a set of perceptual (visual, auditory, tactile) and motor processors operating on symbolically coded features rather than raw sensory data. Although EPIC is focused on multiple simple tasks in one experiment it has been connected to SOAR for problem solving, planning and learning, and the EPICSOAR combination has been applied to air traffic control simulation [23].

3 4

J. Laird, Current Soar Research, see http://ai.eecs.umich.edu/people/laird/current-research.html For more references see the Soar community site: http://sitemaker.umich.edu/soar/home

126

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

ICARUS project [24] defines an integrated cognitive architecture for physical agents, with knowledge specified in the form of reactive skills, each denoting goal-relevant reactions to a class of problems. The architecture includes a number of modules: a perceptual system, a planning system, an execution system, and several memory systems. Concepts are matched to percepts in a bottom-up way and goals are matched to skills in a top-down way. Conceptual memory contains knowledge about general classes of objects and their relationships, while skill memory stores knowledge about the ways of doing things. Each comprises a longterm memory (LTM) and a short-term memory (STM). The LTM is organized in a hierarchical fashion, with the conceptual memory directing bottom-up, percept-driven inference and skill memory controlling top-down, goal-driven selection of actions. The acquisition of knowledge in ICARUS is achieved through hierarchical, incremental reinforcement learning, propagating reward values backward through time. Since skills comprise sub-skills, the system learns to calculate the value estimates over a stack of state-action pairs, instead of just a single pair as in the traditional reinforcement learning. This hierarchical processing strategy yields in turn a much faster learning rate than that of the standard reinforcement learning. The hierarchical memory organization and learning procedure have equipped ICARUS with the ability to focus its attention on a specific object or event in its sensor range and to reduce the reaction time and the search space requirements. Through its planning and memory modules, ICARUS is also able to incrementally learn new concepts in an efficient manner by constructing a feature tree that the system can comprehend. Relational reinforcement learning that gives priority to high-utility beliefs and rapidly finds most useful inferences handles well large memory hierarchies [25]. Interesting applications of this architecture to in-city driving, blocks world or freecell solitaire have been demonstrated. One vital problem is the lack of concurrent processing to cope with asynchronous inputs from multiple sensors while coordinating resources and actions across different modalities. Issues related to uncertainty have also been largely ignored. NARS (Non-Axiomatic Reasoning System) [26] project has been developed for over two decades. It is a reasoning system based on a language for knowledge representation, an experience-grounded semantics of the language, a set of inference rules, a memory structure, and a control mechanism, carrying out various highlevel cognitive tasks as different aspects of the same underlying process. The non-axiomatic logic is used for adaptation with insufficient knowledge and resources, operating on patterns that have the “truth-value” evaluated according to the system’s “experience” with using these patterns. This approach allows for emergence of experience-grounded semantics, and inferences defined on judgments. While several working NARS prototypes of increasing complexity have been built they are solving only rather simple problems. SNePS (Semantic Network Processing System) [27] is a logic, frame and network-based knowledge representation, reasoning, and acting system that went through over three decades of development. It stores knowledge and beliefs of some agent in form of assertions (propositions) about various entities. Each knowledge representation has its own inference scheme, logic formula, frame slots and network path, integrated in SNIP, the SNePS Inference Package. When a belief revision system discovers contradiction some hypotheses that led to the contradiction have to be unasserted by the user and the system removes all those assertions that depend on it. The SNePS Rational Engine controls plans and sequences of actions using frames and believing/disbelieving propositions. The natural language processing system works with English morphological analyzer/synthesizer and generalized augmented transition network grammar interpreter. SNePS has been used for commonsense reasoning, natural language understanding and generation, contextual vocabulary acquisition, control of simulated cognitive agent that is able to chat with the users, question/answer system and other applications. Interesting inferences have been demonstrated, but the program has not yet been used in a large-scale real application, for example a chatterbot. 3.2. Emergent Paradigm Architectures Emergent cognitive architectures are inspired by connectionist ideas [28]. Their relation to the processes that take place in the brain may be rather distant. Processing elements (PEs) form network nodes that interact with each other in a specific way changing their internal states and revealing interesting emergent properties. There are two complementary approaches to memory organization, globalist and localist. The Multi-Layer Perceptron (MLP) and other neural networks based on delocalized transfer functions process information in a distributed, global way. All parameters of such networks influence their outputs. Generalization of learned responses to novel stimuli is usually good, but learning new items may lead to catastrophic interference with old knowledge [29]. The basis set expansion networks that use localized functions (such as Gaussians) are examples of localist networks; the output signals for a given input depend only on a small subset of units that

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

127

are activated. However, it should be remembered that a modular organization of globalist network will easily create a subgroups of processing elements that react in a local way [30]. The learning methodologies for emergent architectures are quite diverse [28][29]. Associative learning creates a mapping of specific input representation to specific output representation and in this sense remembers the reactions, heteroassociations or enables pattern completion (autoassociations). In this case, learning may be either guided directly by a set of correct “target” signals or indirectly by certain critic inputs, which correspond to the supervised and reinforcement learning paradigms in AI, respectively. In competitive learning all PEs compete to become active and learn in an unsupervised fashion. The simplest form of this procedure is the winner-takes-all (WTA) rule, which permits only one winning PE (or one per group) to learn at a time, while inhibiting others that lose the competition. Correlation-based learning using Hebb learning rule captures statistical properties of incoming signals creating an internal model of the environment [29]. A few emergent architectures that are still under active development are presented below. In applications to complex reasoning they have not yet reached the same level of maturity as symbolic architectures, but are much closer to natural perception and reasoning based on perceptions, rather than symbolic knowledge. IBCA (Integrated Biologically-based Cognitive Architecture) is a large-scale emergent architecture that epitomizes the automatic and distributed notions of information processing in the brain [29]. The role of three regions in the brain is emphasized: posterior cortex (PC), frontal cortex (FC), and hippocampus (HC). The PC module assumes an overlapping, distributed localist organization that focuses on sensory-motor as well as multi-modal, hierarchical processing. The FC module employs a non-overlapping, recurrent localist organization in which working memory units are isolated from one another and contribute combinatorially (with separate active units representing different features). The HC module utilizes a sparse, conjunctive globalist organization where all units contribute interactively (not combinatorially) to a given representation. It permits rapid binding of all activation patterns across PC and FC (i.e., episodic memory), while reducing interference. The LEABRA learning algorithm includes error-driven learning of skills and Hebbian learning with inhibitory competition dynamics. In this framework, the PC and FC modules employ a slow integrative learning that blends many individual experiences to capture the underlying regularities of the world and to support sensory-motor activities. The HC module adds fast learning that retains and discriminates (instead of integrating) the individual experiences. This cooperation between HC and FC/PC reflects in turn the complementary learning paradigms in the brain. A salient trait of IBCA is the knowledge-dependency merits (generalization, parallelism, flexibility) of content-specific distributed representation in the brain, missing in symbolic models. Complementary learning capacity resolves problems with knowledge consolidation, or transfer of short-term into long-term memory. Higher-level cognition (variable binding, chaining of operation sequences etc.) is a result of emergent power of activation-based processing (invocation, maintenance, and updating of active representations for selfregulation) in the FC module. These capacities have been validated in basic psychological tasks, such as Stroop test, dynamic sorting task, or visual feature binding [29]. However, the fine-grained structure of the architectural modules requires a large number of neurons to simulate cognitive functions, and raises the issues of system scalability. The current architecture is limited to learning the ANN weight parameters, but not the network local structure. Representation of emotions for motivation and setting goals, as well as motor coordination and timing, is still missing. While this type of architecture may be used to explain human behavior probed by psychological or psycholinguistic experiments no-one has yet demonstrated how to use it for tasks that require reasoning, where many symbolic architectures reach the human competence level. Cortronics is a new emergent architecture that models the biological functions of the cerebral cortex and thalamus systems (jointly termed thalamocortex) in the human brain [14]. Its memory organization consists of modular feature attractor circuits called lexicons. Each lexicon comprises further a localist cortical patch, a localist thalamic patch, and the reciprocal connections linking them. Essentially, each lexicon implements a large stable set of attractive states called symbols, each represented by a specific group of neurons. The number of neurons overlapping between each pair of symbols is relatively small, and each neuron representing one symbol may be used to represent many symbols. Accordingly, knowledge in Cortronics takes the form of parallel, indirect unidirectional links between the neurons representing one symbol in a lexicon and those describing a symbol in another lexicon. Each such knowledge link is termed an item of knowledge, and the collection of all these links is called a knowledge base. The union of cortical patches of all lexicons constitutes in turn the entire cortex, while that of the thalamic patches of all lexicons forms only a portion of thalamus. A competitive activation of symbols of lexicons, called confabulation, is used for learning and information retrieval. Confabulation is carried out by every lexicon when appropriate knowledge link inputs, and operation command inputs, arrive at the lexicon at once. The states of involved neurons

128

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

evolve dynamically via the parallel, mutual interactions of these neurons, and the minority that ends up in the excited/active state denote conclusions, the symbol(s) that won the competition, or a null symbol, implying a “don’t know” answer. The model predicts or anticipates the next state, move or word that should follow. This is quite new architecture and it is not yet clear how it can be extended to create AGI, as confabulation is not sufficient for reasoning with complex knowledge. However, it is an interesting process involved in anticipation, imagination and creativity [32][33], a process at a shorter time scale than reasoning processes. The NuPIC (Numenta Platform for Intelligent Computing) is an emergent architecture based on the Hierarchical Temporal Memory (HTM) technology, which is modeled on the putative algorithm used by neocortex [12]. Network nodes are organized in a hierarchical way, with each node implementing learning and memory functions. Hierarchical organization is motivated by the growing size of cortical receptive fields in the information streams that connect primary sensory cortices with secondary and higher-level association areas. This feature is also present in the IBCA architecture, where specific connectivity between different layers leads to growing and invariant object representation. HTM is unique in stressing the temporal aspect of perception, implementing memory for sequences of patterns that facilitate anticipation. Each level in the hierarchical network is trained separately to memorize spatio-temporal objects, and is able to recognize new, similar objects in a bottom-up/top-down process. The architecture has not yet been tested in larger applications, therefore it is hard to evaluate its potential and its limitations. NOMAD (Neurally Organized Mobile Adaptive Device) automata are based on “neural Darwinism” theory [34]. Nomads, also known as Darwin automata, demonstrate the principles of emergent architectures for pattern recognition task in real time. They use many sensors for vision, range finders to provide a sense of proximity, prioproceptive sense of head direction and self-movement, artificial whiskers for texture sensing, artificial taste (conductivity) sensors. NOMAD is controlled by a large (~105 neurons with ~107 synapses) simulated nervous system running on a cluster of powerful computers. The “Brain-Based Robotic Devices” that develop through behavioral tasks has elucidated a role of value systems based on reward mechanisms in adaptation and learning, importance of self-generated movement in development of perception, the role of hippocampus in spatial navigation and episodic memory (Darwin X-XI models), invariant visual object recognition (Darwin VI-VII), binding visual features of complex objects into a unified scene by neural synchrony due to the recurrent connections in visual pathways, implementation of concurrent, real-time processes. However, the emergence of higher-level cognition does not seem likely in this architecture. A number of emergent architectures based on the global workspace theory of Baars [35] have been formulated in the last two decades, but very few reached implementation level. Shanahan has described a very simple implementation based on weightless neural network built from generalizing random access memory processing units and used to control simulated robot [36]. Other examples of architectures inspired by the global workspace theory are discussed in the hybrid systems subsection below. Many other interesting emergent architectures have been discussed in recent years, but there is little experience with them due to the lack of software implementation to experiment with. Haikonen [37] has written a book outlining an approach to conscious machines and discussing cognitive architecture for robot brains. Anderson and his colleagues proposed the Erzatz brain project [38]. The autonomous mental development movement, motivated by the human mental development from infancy to adulthood, has been active for about a decade now [39], going in similar direction as the Edelman’s Darwin projects, and Brooks’ Cog project [40][41], that is creating a robotic agent for real-time interaction. Korner and Matsumoto [42] argue that cognitive architecture should control constraints that are used to select a proper algorithm from existing repertoire to solve a specific problem, or to create a new one if the stereotyped behaviors are not sufficient. This is very much in agreement with the meta-learning ideas in computational intelligence [43], where solving hard learning problems is done by learning which transformations should be composed to achieve the goal. The DARPA Biologically-Inspired Cognitive Architectures (BICA) program has already resulted in a several interesting proposals, such as the “TOSCA Comprehensive brain-based model of human mind” [44] written by a number of experts from leading USA institutions, which essentially came to the same conclusion. 3.3. Hybrid Paradigm Architectures Given the relative strengths of the symbolic and emergent paradigms, it becomes clear that combining the two would offer a promising venue for developing a more complete framework for cognition [45]. Symbolic architectures are able to process information and realize high-level cognitive functions, such as planning and deliberative reasoning, in a way that resembles human expertise. However, the major issues in this approach

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

129

are the formulation of symbolic entities from low-level information, as well as the handling of large amount of information and uncertainty. Emergent architectures are better suited for capturing the context-specificity of human performance and handling many pieces of low-level information simultaneously. Yet their main shortcoming is the difficulty in realizing higher-order cognitive functions. The potential benefit of a combined approach is therefore to have each method address the limitations of the other, allowing creation of a complete brain architecture that covers all levels of processing, from stimuli to higher-level cognition. Research in this area has led to many proposals of hybrid cognitive architectures, which can be roughly divided in two classes based upon the memory type of the constituent modules: localist-distributed and symbolic-connectionist [45]. The first class of hybrid architectures comprises a combination of localist modules (with each concept specified by one PE node) and distributed modules (with each concept represented by a set of overlapping nodes). In comparison, the second class involves a mixture of symbolic modules (i.e., rule- or graph-based memory) and connectionist modules (either of localist or distributed type). Correspondingly, hybrid architectures can be categorized into two main classes according to their direction of learning: top-down and bottom-up learning [46]. The former involves a transition of knowledge from explicit (accessible) conceptual level to implicit (inaccessible) sub-conceptual level, while the latter goes from subconceptual level to conceptual level. The top-down learning can be achieved by pre-coding a set of expert rules at the top level (localist/symbolic module) and allowing the bottom-level (distributed ANN) to learn by observing actions guided by the top-level [46]. Conversely, bottom-up learning may be accomplished by extracting or translating implicit knowledge coded by a bottom-level module into a set of conceptual rules [30][47]. A few examples of hybrid cognitive architectures follow, focused on the memory organizations, learning methodologies, and key strengths and issues. ACT-R (Adaptive Components of Thought-Rational) [10] is a hybrid cognitive architecture and theoretical framework for emulating and understanding human cognition5. It aims at building a system that can performs the full range of human cognitive tasks and describe in detail the mechanisms underlying perception, thinking, and action. The central components of ACT-R comprise a set of perceptual-motor modules, memory modules, buffers, and a pattern matcher. The perceptual-motor modules basically serve as an interface between the system and the world. There are two types of memory modules in ACT-R: declarative memory (DM) and procedural memory (PM), which encode factual knowledge about the world and that about how to do things respectively. Both are realized as a symbolic-connectionist structures, where the symbolic level consists of productions (for PM) or chunks (for DM), and the sub-symbolic level of a massively parallel connectionist structure. Each symbolic construct (i.e., production or chunk) has a set of sub-symbolic parameters that reflect its past usage and control its operations, thus enabling an analytic characterization of connectionist computations using numeric parameters (associative activation) that measure the general usefulness of a chunk or production in the past and current context. Finally, the ACT-R buffers serve as a temporary storage for inter-module communications (excluding PM), while the pattern matcher is used to find a production in PM that matches the present state of the buffers. ACT-R utilizes a top-down learning approach to adapt to the structure of the environment. In particular, symbolic constructs (i.e., chunks or productions) are first created to describe the results of a complex operation, so that the solution may be available without recomputing the next time a similar task occurs. When a goal, declarative memory activation or perceptual information appears it becomes a chunk in the memory buffer, and the production system guided by subsymbolic processes finds a single rule that responds to the current pattern. Sub-symbolic parameters are then tuned using Bayesian formulae to make the existing symbolic constructs that are useful more prominent. In this way chunks that are often used become more active and can thus be retrieved faster and more reliably. Similarly, productions that more likely led to a solution at a lower cost will have higher expected utility, and thus be more likely chosen during conflict resolution (i.e., selecting one production among many that qualify to fire). This architecture may be partially mapped on the brain structures. It has been applied in a large number of psychological studies, and in intelligent tutoring systems, but ambitious applications to problem solving and reasoning are still missing. CLARION (The Connectionist Learning Adaptive Rule Induction ON-line) is a hybrid architecture that incorporates a distinction between explicit (symbolic) and implicit (sub-symbolic) processes and captures the interactions between the two [45]-[47]. The design objective is two-fold: to develop artificial agents for certain cognitive task domains, and to understand human learning and reasoning processes in similar domains. The CLARION architecture contains four memory modules, each comprising a dual explicit-implicit representation: action-centered subsystem (ACS), non-action-centered subsystem (NCS), motivational 5

See the ACT-R site at http://act-r.psy.cmu.edu

130

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

subsystem (MS), and metacognitive subsystem (MCS). Essentially, the ACS module serves to regulate the agent’s actions, while NCS maintain the general system knowledge (either explicit or implicit). On the other hand, MS functions to provide a motivation/impetus for perception, action and cognition, while MCS monitor, direct and alter the operations of the other three modules. Each of these modules adopts a localist-distributed representation, where the localist section encodes the explicit knowledge and the distributed section (e.g. an MLP network) the implicit knowledge. CLARION also employs different learning methods for each level of knowledge. Learning of implicit knowledge is achieved using reinforcement learning methods such as Qlearning or supervised methods such as the standard back-propagation, both of which can be implemented using an MLP network. The implicit knowledge already acquired at the bottom level is then utilized to craft the explicit knowledge at the top level via a bottom-up learning. This can in turn be viewed as a rational reconstruction of implicit knowledge at the explicit level. Top-down learning may also be achieved by precoding/fixing some rules at the top level and allowing the bottom-level to accumulate knowledge by observing actions guided by these rules [46]. As such, the system’s decision making that relies initially on the top level gradually becomes more dependent on the bottom level. Software is available for experimentation with CLARION. A lot of psychological data has been simulated with this architecture, but also a complex sequential decision-making for a minefield navigation task. LIDA (The Learning Intelligent Distribution Agent) is a conceptual and computational framework for intelligent, autonomous, “conscious” software agent that implements some ideas of the global workspace (GW) theory [48]. The architecture is built upon a bit older IDA framework, which was initially designed to automate the whole set of tasks of a human personnel agent who assigns sailors to new tours of duty. LIDA employs a partly symbolic and partly connectionist memory organization, with all symbols being grounded in the physical world in the sense of Brooks and Stein [41]. LIDA has distinct modules for perception, working memory, emotions, semantic memory, episodic memory, action selection, expectation and automatization (learning procedural tasks from experience), constraint satisfaction, deliberation, negotiation, problem solving, metacognition, and conscious-like behavior. Most operations are done by codelets implementing the unconscious processors (specialized networks) of the global workspace theory. A codelet is a small piece of code or program that performs one specialized, simple task. The LIDA framework incorporates three new modes of learning into the older IDA model: perceptual, episodic, and procedural learning, which are all of bottom-up type. Perceptual learning concerns learning of new objects, categories, relations, etc, and takes two forms: strengthening or weakening of the base-level activation of nodes, as well as creation of new nodes and links in the perceptual memory. Episodic learning, on the other hand, involves learning to memorize specific events (i.e., the what, where, and when). It results from events taken from the content of “consciousness” being encoded in the (transient) episodic memory. Finally, procedural learning concerns learning of new actions and action sequences with which to accomplish new tasks. It combines selectionist learning (i.e., selecting from an obsolete repertoire) and instructionalist learning (i.e., constructing new representations), with functional consciousness providing reinforcements to actions. There is no doubt that this architecture may explain many features of mind, however, it remains to be seen high competence ti will achieve in understanding language, vision, and common sense reasoning based on perceptions. DUAL architecture [49] has been inspired by Minsky’s “Society of Mind” [50] theory of cognition. It is a hybrid, multi-agent general-purpose architecture supporting dynamic emergent computation, with a unified description of mental representation, memory structures, and processing mechanisms carried out by small interacting micro-agents. As a result of lack of central control the system is constantly changing, depending on the environment. Agents interact forming larger complexes, coalitions and formations, some of which may be reified. Such models may be evaluated at different levels of granularity, the microlevel of micro-agents, the mesolevel of emergent and dynamic coalitions, and the macrolevel of the whole system and models, where psychological interpretations may be used to describe model properties. Micro-frames are used for symbolic representation of facts, while relevance or activation level of these facts in a particular context is represented by network connections with spreading activation that changes node accessibility. Links between microagents are based on their frame slots and weights control the influence of agents on each other’s activity. DUAL architecture has been used in a number of projects: AMBR, a model of human reasoning that unifies analogy, deduction, and generalization, including a model of episodic memory; a model of human judgment; a model of perception, analysis of interactions between analogy, memory, and perception; understanding the role of context and priming effects for the dynamics of cognitive processes. This is certainly a very interesting architecture that is capable of explaining many cognitive phenomena. It is not clear how well it will scale up to real problems requiring complex reasoning, as nothing in this area has yet been demonstrated.

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

131

Polyscheme [51] integrates multiple methods of representation, reasoning and inference schemes in problem solving. Each Polyscheme “specialist” models a different aspect of the world using specific representation and inference techniques, interacting with other specialists and learning from them. Scripts, frames, logical propositions, neural networks and constraint graphs can be used to represent knowledge. A reflective specialist guides the attention of the whole system, providing various focus schemes that implement inferences via script matching, backtracking search, reason maintenance, stochastic simulation and counterfactual reasoning. High-order reasoning is guided by higher-level policies for focusing attention. Many problem solving algorithms use forward inference, subgoaling, grounding, representing alternate worlds and identity matching as their basic operations. Such operations are handled by specialists who are equipped with different representations but focus on the same aspect of the world, and may integrate also lower-level perceptual and motor processes. Thus Polyscheme may be used both in abstract reasoning and also in common sense physical reasoning in robots. It has been used to model infant reasoning including object identity, events, causality, spatial relations. This meta-learning approach combining different approaches to problem solving is certainly an important step towards AGI and common sense reasoning. 4CAPS architecture [52] has plausible neural implementation and is designed for complex tasks, such as language comprehension, problem solving or spatial reasoning. A unique feature is the ability to compare the activity of different 4CAPS modules with functional neuroimaging measures of brain’s activity. It has been used to model human behavioral data (response times and error rates) for analogical problem solving, human– computer interaction, problem solving, discourse comprehension and other complex tasks solved by normal and mentally impaired people. Its first operating principle, “Thinking is the product of the concurrent activity of multiple centers that collaborate in a large scale cortical network”, leads to the architecture based on a number of centers (corresponding to particular brain areas) that have different processing styles, for example Wernicke’s area is specialized for the associative retrieval/design, constructing and selectively accessing structured sequential and hierarchical representations. Each center can perform and be a part of multiple cognitive functions, but has a limited computational capacity constraining its activity. Functions are assigned to centers depending on the resource availability, therefore the topology of the whole large-scale network is not fixed. Although 4CAPS contains many interesting ideas it is not aimed at achieving intelligent behavior, but rather tries to model human performance; software written in Lisp is available for experimentation. See the discussion in [52] of other models that are aimed at explanation of behavioral data. Shruti [53], biologically-inspired model of human reflexive inference, represents in connectionist architecture relations, types, entities and causal rules using focal-clusters. These clusters encode universal/existential quantification, degree of belief, and the query status. The synchronous firing of nodes represents dynamic binding, allowing for representations of quite complex knowledge and inferences. This architecture may have great potential, but after rather long time of development it has not yet found any serious applications to problem solving or language understanding. The Novamente AI Engine is based on system-theoretic ideas regarding complex mental dynamics and associated emergent patterns, inspired by the psynet model [54] and more general “patternist philosophy of mind” [55]. Similarly as in the “society of minds” and the global workspace, self-organizing and goaloriented interactions between patterns are responsible for mental states. Emergent properties of network activations should lead to hierarchical and relational (heterarchical) pattern organization. Probabilistic term logic (PTL), and the Bayesian Optimization Algorithm (BOA) algorithms are used for flexible inference. Actions, perceptions, and internal states are represented by tree-like structures. This is still an experimental architecture that is being developed, seems to be in a fluid state, and its scaling properties are not yet known.

4. Where do we go from here? The previous sections has presented a number of very interesting models of cognition that have the potential to develop general intelligence. Many excellent projects have already been formulated, some have been developed over many decades, while others are just starting. So far cognitive architectures are used in very few real-world applications. Grand challenges, as discussed in section two, and smaller steps that lead to human and super-human levels of competence should be formulated to focus the research. Extending small demonstrations in which a cognitive system reasons in a trivial domain to larger-scale applications, for example generating results that may be of interest to experts, or acting as an assistant to human expert, is one important direction. Without a set of demanding test problems it is very hard to evaluate new projects, compare their capabilities and understand their limitations. Integrative models of human performance are of

132

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

great interest in the defense and aerospace industries. A recent project on the Agent-Based Modeling and Behavior Representation (AMBR) Model Comparison resulted in quantitative data comparing the performance of humans and cognitive architectures in a simplified air traffic controller environment [56]. Some efforts have been expended on the evaluation of software agents and several proposals in this direction has been put forth during the 2007 AAAI Workshop “Evaluating Architectures for Intelligence” [57]. Ideas ranged from using in-city driving environment as a testbed for evaluating cognitive architectures, to measuring incrementality and adaptivity components of general intelligent behavior. Perhaps a measure of “cognitive age” could be established, with a set of problems that children at a given age are able to solve. Problems should be divided into several groups: e.g. vision and auditory perception, understanding language, common-sense reasoning, abstract reasoning, probing general knowledge about the world, learning, problem solving, imagination, creativity. Solving all problems from a given group that children at some age are able to solve will qualify cognitive system to pass to the next grade in this group of problems. It should be expected that some systems will show advanced age in selected areas, and not in the others. For example, solving problems requiring vision may require addition of specialized computer vision modules, while mathematical reasoning in many reasoning systems may be fairly advanced comparing to children. Experts in human intelligence largely agree to the original Gardner’s proposal [58] that seven kinds of intelligence should be distinguished: logical-mathematical, linguistic, spatial, musical, bodily-kinesthetic, interpersonal and intrapersonal intelligence, perhaps extended by emotional intelligence and a few others. General world knowledge is fairly difficult to collect and could be probed using a question/answer system. If a 5-year old child could get all the answer to general questions from an avatar controlled by some cognitive architecture one should assume that the mental age of the control system in this respect is at least 5. Knowledge bases in cognitive systems are usually quite limited and require very different kind of organization and knowledge representation methods. Huge CyC knowledge base is an exception [3], and using it to construct large knowledge bases suitable for other cognitive systems is certainly worth the effort. Such analysis should certainly help to understand what type of intelligence may be expected from embodied cognitive robotic projects and what the limitations of symbolic approaches are. Brooks has made a good point that elephants do not play chess [40], and expressed hope [41] that a robot with integrated vision, hearing and dextrous manipulation controlled by large scale parallel MIMD computer “will learn to ‘think’ by building on its bodily experiences to accomplish progressively more abstract tasks”. His Cog project based on grounding the meaning of concepts in deep embodiment has many followers although after 15 years it has stayed at the level of reactive agent and there are no good ideas how to extend it to higher cognitive levels. While behavioral intelligence in robotic devices may be difficult to achieve without embodiment experiences with this approach in the last two decades are not very encouraging for AGI. Elephants are intelligent, but cannot learn language or be personal assistants. It is also possible that ideas on which cognitive architectures are based are not sufficient to solve the problems in computer vision or language and more specific models of some brain functions are needed. The survey presented above showed several trends that will probably dominate in the research on cognitive architectures. First, the number of hybrid architectures is already quite large, biological inspirations are becoming increasingly important and this will lead to domination of BICA architectures. Even hard core symbolic architecture proponents base now further extension of their architectures on inspirations from the brain (see link in footnote 3). They focus on the role of cortex and limbic system, but completely neglect the regulatory role of the brain stem which may provide overall meta-control selecting different types of behavior. Second, there may be many BICA architectures, but several key features need to be preserved. Different types of memory are certainly important, as has been already stressed by several symbolic, emergent and hybrid architectures. Processing of speech or texts requires recognition of tokens, or mapping from sounds or strings of letters to unique terms; resolving ambiguities and mapping terms to concepts in some ontology; and a full semantic representation of the text, that facilitates understanding and answering questions about its content. These three steps are roughly based on several kinds of human memory. First, recognition memory that helps to focus quickly attention when something is wrong, for example a strangely spelled word that could be a misspelling, a foreign word, personal name, or an attempt to avoid spam filters. This may be implemented by simple neural networks without hidden layer or by correlation matrix memories [32]. The role of recognition memory has also been largely forgotten. Second, there is a need for semantic memory that serves not only as hierarchical ontology, but approximates spreading activation processes in real brains, and thus activates various types of associations providing background knowledge that humans use for token to concept mapping and disambiguation. Unfortunately large-scale semantic memories that contain both structural properties of concepts (chairs have

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

133

legs, seat, etc) and their relations and associations (chair – table, sit, etc) and could be used in computationally efficient way do not exist. While significant progress has been made in drawing inspirations from neuroscience in analysis of auditory, visual and olfactory signals much less has been done at the higher cognitive function level. Although neurocognitive approach to linguistics has been formulated as “an attempt to understand the linguistic system of the human brain, the system that makes it possible for us to speak and write, to understand speech and writing, to think using language …” [59], in practice it has been used only to analyze specific linguistic phenomena. A practical algorithm to discover these “pathways of the brain” has been introduced recently [60], opening the way for construction of large-scale semantic networks that will approximate symbolic knowledge stored in human brain, although creating such large-scale memories will require a large effort. Efforts to build concept descriptions from electronic dictionaries, ontologies, encyclopedias, results of collaborative projects and active searches in unstructured sources have been described in [6]. Third, episodic memory is required to store experiences from interactions with individual users, to understand the context of current interactions and interpret all events in view of this context. Various elements of semantic and episodic memories are kept in the working memory. All types of memory are intimately connected. Recognition of tokens is facilitated by the active part of semantic memory and made easier by the expectations resulting from episodic memory. Reading text leads to priming effects: expectation and anticipation of a few selected words, and inhibition of many others that do not come to the mind of the reader. Episodic memory is based on semantic relations of the concepts found in the text. Although several proposals for memory-based cognitive architectures have been formulated the role of different types of memory has not been stressed and no effort to create appropriate large-scale knowledge bases has been made. AGI requires such memories, and as a step towards such memory-based architecture an avatar that uses large semantic memory to play word games has been demonstrated [6]. What is the best practical way to implement these ideas? Template matching proved to be amazingly effective in simulation of dialogues and is still dominating in chatterbots [61], but it obviously does not lead to real understanding of the concepts that appear in the discussion. Neural template matching, or templates approximating the distribution of neuronal group activities in the brain during concept comprehension, is the simplest technique that goes beyond symbolic template matching, leading to sets of consistent concepts. Words are ambiguous and form concepts that have meanings modified by their contexts. In the brain a word w = (wf,ws) has phonological component wf (the memorized form of the word, string of phonemes or characters), and an extended semantic representation ws (extended activations related to the use and category of the word, including immediate associations). The extended activation is not unique, only when the current context Cont is specified (specific activations of other concepts are determined) the meaning of the word is established, resulting from spreading activation in the brain to form a global state (w,Cont). This state changes with each new word received in sequence, with quasi-stationary states formed after each sentence is processed and understood. It is quite difficult to decompose the (w,Cont) state into components, because the semantic representation ws is strongly modified by the context. The state (w,Cont) may be regarded as a quasistationary wave, with its core component centered on the phonological/visual brain activations wf and with quite variable extended representation ws. As a result the same word in a different sentence creates quite different states of activation, and the lexicographical meaning of the word may be only an approximation of an almost continuous process. To relate states (w,Cont) to lexicographical meanings, one can cluster all such states for a given word in different contexts and define prototypes (wk,Cont) for different meanings wk. These prototypes are neural templates that should replace symbolic templates. The form of the word wf identifies several candidate templates with different meanings, and the one that fits to other templates, maximizing overall consistency of interpretations, is selected. The symbolic approach to language is a poor substitute for neurolinguistic processes, and highdimensional vector model of language, popular in statistical approach to natural language processing (NLP) [62], is a very crude approximation that does not reflect essential properties of the perception-action-naming activity of the brain [64][65]. The process of understanding words (spoken or read) starts from activation of word form representation (the phonological or grapheme representation) in the temporal lobe, quickly spreading neural activation to further brain areas, including the non-dominant (usually right) hemisphere, that does not contain representations of word forms, but learns to evaluate clusters of activations, forming constraints on the way words may be used, and forming general, higher-level concepts [60]. This continuous process may be approximated through a series of snapshots of patterns created by microcircuit activations (w,Cont) = i i i(w,Cont), i(w,Cont) that can be treated as basis functions for the expansion of the state where the summation extends over all patterns that show significant activity resulting after presentation of the

134

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

word w. The high-dimensional vector model used in NLP measures only the co-occurrence of words Vij = V(wi),V(wj) in small window neighborhood, averaged over all contexts, while human knowledge includes also structural properties of concepts that are important, but do not appear explicitly in texts. The use of wavelike representation in terms of basis functions to describe neural states makes this formalism similar to that used in quantum mechanics, although no real quantum effects are implied here. Objects of discourse and actual episodes are memorized by hippocampus that links to the cortex and is able to recreate the original activations at the moment the episode has been experienced. An outline of the road from single neurons, to brain modules, to societies of brains has been presented in [66]. Models of single neurons usually have little internal knowledge (thresholds) and relatively simple interactions modeled via weighted links (biophysical models of single neurons may of course be very complex). Assemblies of neurons at different levels form coalitions and may be regarded as specialized processors, passing structured information and attaining rather complex internal states. This may be approximated by interacting agents, with internal knowledge and means of communication, with coalitions of simple agents creating dynamic, higher-order units that may be viewed as “soft agents”, with new competencies arising at demand by variation on the main theme. Such meta-learning ideas for beating combinatorial complexity and solving pattern recognition and reasoning based on partial observations has been recently described [43], and preliminary implementation of general system architecture to support such approach has been presented [67]. The role of imagination, creativity, learning from partial observations and using this knowledge in an intuitive way, and the role of the right hemisphere in the linguistic processes, has only recently been discussed [32][33]. The AIM1 (ArtIficial Mind1) architecture based on these ideas is under development and will be presented in near future. This architecture will draw inspirations from some of the projects presented in this paper, but will be primarily aimed at ambitious applications requiring natural language processing. It is quite likely that this approach will lead to creation of conscious artifacts [37][68]. Acknowledgment: W.D. thanks Polish Committee for Scientific Research for research grant 2005-2007.

References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22]

A. Newell, Unified Theories of Cognition: Harvard University Press, 1990. A. Newell, H.A. Simon, GPS: A program that simulates human thought. In E. A. Feigenbaum & J. Feldman (Eds.), Computers and Thought. New York: McGraw-Hill, 1963. K. Panton, C. Matuszek, D. Lenat, D. Schneider, M. Witbrock, N. Siegel, B. Shepard, Common Sense Reasoning – From Cyc to Intelligent Assistant. In: Y. Cai and J. Abascal (Eds.): Ambient Intelligence in Everyday Life, LNAI 3864 (2006) 1–31. A. Turing, Computing Machinery and Intelligence, Mind 49 (1950) 433-460. R. Carpenter, J. Freeman, Computing Machinery and the Individual: the Personal Turing Test, 2005, paper available at http://www.jabberwacky.com/. J. SzymaĔski, T. Sarnatowicz, W. Duch, Towards Avatars with Artificial Minds: Role of Semantic Memory. Journal of Ubiquitous Computing and Intelligence 1 (2007, in print). E.A. Feigenbaum, Some Challenges and Grand Challenges for Computational Intelligence. J. of the ACM 50(1) (2003) 32–40. N. Nilsson, Human-Level Artificial Intelligence? Be Serious! The AI Magazine 26(4) (2005) 68-75. D.E. Meyer, D.E. Kieras, A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. Psychological Review, 104(1), (1997) 3-65. J.R. Anderson, C. Lebiere, The Newell test for a theory of cognition. Behavioral and Brain Science 26 (2003) 587-637. D. Vernon, G. Metta, G. Sandini, A survey of artificial cognitive systems: Implications for the autonomous development of mental capabilities in computational agents. IEEE Transactions on Evolutionary Computation 11(2) (2007) 151-180. J. Hawkins, S. Blakeslee, On intelligence: How a New Understanding of the Brain will Lead to the Creation of Truly Intelligent Machines. Times Books 2004. T. Hoya, Artificial Mind System. Kernel Memory Approach. Springer, 2005. R. Hecht-Nielsen, Confabulation Theory: The Mechanism of Thought. Springer 2007. J.F. Sowa, Conceptual Structures. Reading, Mass, Addison-Wesley, 1984. M. Minsky, A Framework for Representing Knowledge. In: P.H. Winston, Ed. The Psychology of Computer Vision. New York: McGraw-Hill, 1975. R.J. Firby, Adaptive Execution in Complex Dynamic Worlds. Ph.D. Thesis, Yale University, 1989. T.M. Mitchell, R., Keller, S. Kedar-Cabelli, Explanation-based generalization: A unifying view. Machine Learning 1 (1986) 47-80. M.M.Veloso, J.G. Carbonell, Integrating analogy into a general problem-solving architecture. In M. Zemankova & Z. Ras (Eds.), Intelligent Systems (pp. 29-51). Chichester, England: Ellis Horwood, 1990. N., Larvac, S. Dzeroski, Inductive Logic Programming: Techniques and Applications. New York: Ellis Horwood, 1994. L.P. Kaelbling, M.L. Littman, A.W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4 (1996) 237-285. J.E. Laird, P.S. Rosenbloom, A. Newell, Soar: An architecture for general intelligence. Artificial Intelligence 33 (1987) 1-64.

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

135

[23] J. Rosbe, R.S. Chong, D.E. Kieras, Modeling with Perceptual and Memory Constraints: An EPIC-Soar Model of a Simplified Enroute Air Traffic Control Task, SOAR Technology Inc. Report, Ann Arbor, Michigan, 2001. [24] P. Langley, An adaptive architecture for physical agents. In Proc. of the 2005 IEEE/WIC/ACM Int. Conf. on Intelligent Agent Technology. Compiegne, France: IEEE Computer Society Press, pp. 18-25, 2005. [25] P. Langley, D. Choi, Learning recursive control programs from problem solving. J. of Machine Learning Res. 7 (2006) 493-518. [26] P. Wang, Rigid flexibility. The Logic of Intelligence. Springer 2006 [27] S.C. Shapiro, W.J. Rapaport, M. Kandefer, F.L. Johnson, A. Goldfain, Metacognition in SNePS, AI Magazine 28 (2007) 17-31. [28] J.L. McClelland, D.E. Rumelhart and the PDP Research Group. Parallel Distributed Processing: Explorations in the Microstructures of Cognition. Cambridge, MA: MIT Press, 1986. [29] R.C. O'Reilly, Y. Munakata, Computational Explorations in Cognitive Neuroscience: Understanding of the Mind by Simulating the Brain. Cambridge, MA: MIT Press, 2000. [30] W. Duch, R. Adamczak, K. Grąbczewski, A new methodology of extraction, optimization and application of crisp and fuzzy logical rules. IEEE Transactions on Neural Networks, 12 (2001) 277-306. [31] R.C. O'Reilly, T.S. Braver, J.D. Cohen A biologically-based computational model of working memory. In A. Miyake & P. Shah (Eds.), Models of Working Memory. Cambridge University Press, pp. 375-411, 1999. [32] W. Duch, M. Pilichowski, Experiments with computational creativity. Neural Information Processing – Letters and Reviews 11 (2007) 123-133. [33] W. Duch, Intuition, Insight, Imagination and Creativity. IEEE Computational Intelligence Magazine 2(3) (2007) 40-52. [34] G.M. Edelman, Neural Darwinism: Selection and reentrant signaling in higher brain function. Neuron 10 (1993) 115-125. [35] B.J. Baars, A Cognitive Theory of Consciousness. New York: Cambridge University Press, 1988. [36] M.P. Shanahan, A cognitive architecture that combines internal simulation with a global workspace. Consciousness and Cognition 15 (2006) 157-176. [37] P. Haikonen, Robot brains; circuits and systems for conscious machines. Wiley, 2007. [38] J.A. Anderson, P. Allopenna, G.S. Guralnik, D. Sheinberg, J.A. Santini, Jr., D. Dimitriadis, B.B. Machta, and B.T. Merrit, Programming a Parallel Computer: The Ersatz Brain Project. In W Duch, J Mandzuik (Eds.), Challenges to Computational Intelligence. Springer: Berlin, pp. 61-88, 2007. [39] J. Weng and W. S. Hwang, From Neural Networks to the Brain: Autonomous Mental Development. IEEE Computational Intelligence Magazine 1(3) (2006) 15-31. [40] R. Brooks, Elephants don't play chess. Robotics and Autonomous Systems 6 (1986) 3-15. [41] R. Brooks, L.A. Stein, Building Brains for Bodies. Autonomous Robotics 1 (1994) 7-25. [42] E. Korner, G. Matsumoto, Cortical architecture and self-referential control for brain-like computation. IEEE Engineering in Medicine and Biology Magazine, 21(5) (2002) 121-133. [43] W. Duch, Towards comprehensive foundations of computational intelligence. In: W. Duch and J. Mandziuk, Challenges for Computational Intelligence. Springer Studies in Computational Intelligence, 63 (2007) 261-316. [44] TOSCA: A comprehensive brain-based cognitive architecture: Biologically-Inspired Cognitive Architecture (BICA) Phase 1 Architecture Report, DARPA-IPTO 2006. [45] R. Sun, F. Alexandre, Connectionist symbolic integration. Hillsdale, NJ: Erlbaum, 1997. [46] R. Sun, X. Zhang, Top-down versus bottom-up learning in cognitive skill acquisition. Cognitive Systems Research 5 (2004) 63-89. [47] R. Sun, E. Merrill, T. Peterson, From implicit skills to explicit knowledge: A bottom-up model of skill learning. Cognitive Science, 25(2) (2001) 203-244. [48] S. Franklin, The LIDA architecture: Adding new modes of learning to an intelligent, autonomous, software agent. In Proc. of the Int. Conf. on Integrated Design and Process Technology. San Diego, CA: Society for Design and Process Science, 2006. [49] A. Nestor, B. Kokinov, Towards Active Vision in the DUAL Cognitive Architecture. International Journal on Information Theories and Applications 11 (2004) 9-15. [50] M. Minsky, The Society of Mind. Simon and Schuster, New York, 1986. [51] N.L. Cassimatis, Adaptive Algorithmic Hybrids for Human-Level Artificial Intelligence. Advances in Artificial General Intelligence. IOS Press. Eds. B. Goertzel and P. Wang, 2007. [52] M.A. Just, S. Varma, The organization of thinking: What functional brain imaging reveals about the neuroarchitecture of complex cognition. Cognitive, Affective, and Behavioral Neuroscience 7 (2007) 153-191. [53] L. Shastri, V. Ajjanagadde, From simple associations to systematic reasoning: A connectionist encoding of rules, variables, and dynamic bindings using temporal synchrony. Behavioral & Brain Sciences 16(3) (1993) 417-494. [54] B. Goertzel, From Complexity to Creativity. New York, NY: Plenum Press, 1997. [55] B. Goertzel, The Hidden Pattern, BrownWalker Press, 2006. [56] K.A. Gluck, R.W. Pew (Eds.), Modeling Human Behavior with Integrated Cognitive Architectures: Comparison, Evaluation. Lawrence Erlbaum Associates 2005. [57] G.A. Kaminka, C.R. Burghart (eds.), Evaluating Architectures for Intelligence. Technical Report WS-07-04, AAAI Press, Menlo Park, CA, 2007. [58] H. Gardner, Multiple intelligences: The theory in practice. New York: Basic Books, 1993. [59] S. Lamb, Pathways of the Brain: The Neurocognitive Basis of Language. Amsterdam: J. Benjamins Publishing Co. 1999. [60] W. Duch, P. Matykiewicz, J. Pestian, Towards Understanding of Natural Language: Neurocognitive Inspirations. Lecture Notes in Computer Science 4669 (2007) 953–962. [61] R. Wallace, The Elements of AIML Style, ALICE A.I. Foundation, 2003. [62] C.D. Manning, H. Schütze, Foundations of Statistical Natural Language Processing. Cambridge, MA: MIT Press, 1999. [63] F. Pulvermuller, The Neuroscience of Language. On Brain Circuits of Words and Serial Order. Cambridge, UK: Cambridge University Press, 2003. [64] S. Dehaene, L. Cohen, M. Sigman, F. Vinckier, The neural code for written words: a proposal. Trends in Cognitive Science 9, 335341, 2005. [65] M. Meeter, J.M.J. Murre, TraceLink: A model of consolidation and amnesia. Cognitive Neuropsychology 22 (5), 559-587, 2005. [66] W. Duch, J. Mandziuk, Quo Vadis Computational Intelligence? In: Machine Intelligence. Quo Vadis? Eds: P. Sinþák, J. Vašþák, K. Hirota. Advances in Fuzzy Systems - Applications and Theory - Vol. 21, World Scientific, 3-28, 2004.

136

W. Duch et al. / Cognitive Architectures: Where Do We Go from Here?

[67] K. Grąbczewski, N. Jankowski, Versatile and Efficient Meta-Learning Architecture: Knowledge Representation and Management in Computational Intelligence. IEEE Symposium Series on Computational Intelligence (SSCI 2007), Honolulu, HI, IEEE Press, pp. 51-58. [68] W. Duch, Brain-inspired conscious computing architecture. Journal of Mind and Behavior 26(1-2) (2005) 1-22.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

137

LIDA and a Theory of Mind David Friedlandera and Stan Franklinb Cognitive Computing Research Group, Houston, TX b Computer Science Department & Institute for Intelligent Systems, The University of Memphis a

Abstract. Every agent aspiring to human level intelligence, every AGI agent, must be capable of a theory of mind. That is, it must be able to attribute mental states, including intentions, to other agents, and must use such attributions in its action selection process. The LIDA conceptual and computational model of cognition offers an explanation of how theory of mind is accomplished in humans and some other animals, and suggests how this explanation could be implemented computationally. Here we describe how the LIDA version of theory of mind is accomplished, and illustrate it with an example taken from an experiment with monkeys, chosen for simplicity. Keywords. LIDA, Theory of Mind, Cognitive Architecture, Cognitive Science, Artificial Intelligence, Global Workspace Theory

Introduction The theory of mind states that we ascribe minds to other individuals, and particularly attribute mental states to them even though, as individuals, we only have direct evidence of our own mental states [1]. We perceive the minds of others in the same way we perceive other aspects of our environment, by applying cognitive processes to sensory inputs. We use the results of such perception to select actions, and to learn from our perceptions of their effects. One can argue that a theory of mind process would be necessary for an artificial general intelligence (AGI) agent. The central hypothesis of this paper is that the mind has the ability to build models of other cognitive agents in hypothetical environments and to reason about potential outcomes of possible actions as an aid to decision making. One possible objection is that, not only are our perceptions about other people's minds indirect, they are also sparse. However, the human mind is quite good at making sense of sparse data. There is a good analogy with visual perception. In vision, the fovea scans only a small part of the scene [2]. No detail is available for most of it and, for some of it, no data at all. Yet we perceive a continuous, fully filled in, detailed scene at all times. The brain supplies the missing perceptual qualities from a number of sources [3]: previous input from the recent past; world knowledge such as common sense or convention; abstract default information; similar information from the past. We propose that similar sources are used to make sense of sparse data while modeling other minds. In this paper, we will use a cognitive model called LIDA [4], derived from Global Workspace Theory [5,6] and based on recent theories from psychology, neuroscience and cognitive science, to develop our hypothesis of the theory of mind and to show how it can explain various psychological phenomena. Section 1 contains a brief

138

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

description of the LIDA model, Section 2 shows how it can incorporate a theory of mind, Section 3 provides a specific example of how the model would work, and Section 4 contains conclusions and suggestions for future research.

1. LIDA The LIDA model and its ensuing architecture are grounded in the LIDA cognitive cycle. As a matter of principle, every autonomous agent [7], be it human, animal, or artificial, must frequently sample (sense) its environment, process (make sense of) this input, and select an appropriate response (action). The agent’s “life” can be viewed as consisting of a continual sequence of iterations of these cognitive cycles. Such cycles constitute the indivisible elements of attention, the least sensing and acting to which we can attend. A cognitive cycle can be thought of as a moment of cognition, a cognitive “moment.” Higher-level cognitive processes are composed of these cognitive cycles as cognitive “atoms.” Just as atoms are composed of protons, neutrons and elections, and some of these are composed of quarks, glueons, etc., these cognitive “atoms” have a rich inner structure. We’ll next concisely describe what the LIDA model hypothesizes as the rich inner structure of the LIDA cognitive cycle. More detailed descriptions are available elsewhere [8,9,10,11]. Figure 1 should help the reader follow the description. It starts in the upper left corner and proceeds clockwise. During each cognitive cycle the LIDA agent first makes sense of its current situation as best as it can. It then decides what portion of this situation is most in need of attention. Broadcasting this portion, the current contents of consciousness, enables the agent to finally chose an appropriate action and execute it. Let’s look at these three processes in a little more detail.

Figure 1. The LIDA Cognitive Cycle

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

139

The cycle begins with sensory stimuli from the agent’s environment, both an external and an internal environment. Low-level feature detectors in sensory memory begin the process of making sense of the incoming stimuli. These low-level features are passed to perceptual memory where higher-level features, objects, categories, relations, actions, situations, etc. are recognized. These recognized entities, comprising the percept, are passed to the workspace, where a model of the agent’s current situation is assembled. Workspace structures serve as cues to the two forms of episodic memory, yielding both short and long term remembered local associations. In addition to the current percept, the workspace contains recent percepts that haven’t yet decayed away, and the agent’s model of the then current situation previously assembled from them. The model of the agent’s current situation is updated from the previous model using the remaining percepts and associations. This updating process will typically require looking back to perceptual memory and even to sensory memory, to enable the understanding of relations and situations. This assembled new model constitutes the agent’s understanding of its current situation within its world. It has made sense of the incoming stimuli. For an agent “living” in a complex, dynamically changing environment, this updated model may well be much too much for the agent to deal with all at once. It needs to decide what portion of the model should be attended to. Which are the most relevant, important, urgent or insistent structures within the model? Portions of the model compete for attention. These competing portions take the form of coalitions of structures from the model. One such coalition wins the competition. The agent has decided on what to attend. But, the purpose of all this processing is to help the agent to decide what to do next [12]. To this end, the winning coalition passes to the global workspace, the namesake of Global Workspace Theory [5,6], from which it is broadcast globally. Though the contents of this conscious broadcast are available globally, the primary recipient is procedural memory, which stores templates of possible actions including their contexts and possible results [13]. It also stores an activation value for each such template that attempts to measure the likelihood of an action taken within its context producing the expected result. Templates whose contexts intersect sufficiently with the contents of the conscious broadcast instantiate copies of themselves with their variables specified to the current situation. These instantiations are passed to the action selection mechanism, which chooses a single action from these instantiations and those remaining from previous cycles. The chosen action then goes to sensory-motor memory, where it picks up the appropriate algorithm by which it is then executed. The action so taken affects the environment, and the cycle is complete. The LIDA model hypothesizes that all human cognitive processing is via a continuing iteration of such cognitive cycles. These cycles occur asynchronously, with each cognitive cycle taking roughly 200 ms. The cycles cascade, that is, several cycles may have different processes running simultaneously in parallel. Such cascading must, however, respect the serial order of consciousness in order to maintain the stable, coherent image of the world with which consciousness endows us [14, 15]. This cascading, together with the asynchrony, allows a rate of cycling in humans of five to ten cycles per second. A cognitive “moment” is quite short! There is considerable empirical evidence from neuroscience studies suggestive of such cognitive cycling in humans [16, 17, 18, 19, 20 21]. None of this evidence is conclusive.

140

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

2. Theory of Mind in LIDA Agent Procedural memory in LIDA is implemented as a scheme net [22] of templates called schemes. Information about how another cognitive agent does something is stored in the scheme net. Representations of other agents in perceptual memory and of their actions allow them to be recognized (activated) when that other agent is perceived. They remain in the workspace for a few 10’s of seconds or a few hundred cognitive cycles. Structures may form as part of the updated internal model and win the competition for consciousness, which will cause them to be stored in episodic memory. This allows LIDA to reconstruct them after they decay from the workspace. This process creates a kind of virtual reality in the workspace, and allows LIDA to model other agents and predict what they are likely to do in a given situation. This ability to build internal structures predicting the actions of other agents is a high-level adaptation to the environment, which provides feedback on these predictions. In LIDA, schemes and perceptual categories used to make successful predictions have their activations increased. The opposite is true for components involved in unsuccessful predictions. The ability to represent and model other cognitive agents is known as a theory of mind. It helps explain some aspects of human cognition such as one-shot learning by observing a teacher and the ability of people to read subtle social queues. There is a theory that the mirror neurons in the cortex are involved in this process [23, 24]. Other animals are also known to exhibit theory of mind [25]. There are two mechanisms for implementing the theory of mind in LIDA. Perceptual memory has concept nodes (representations) for both the self and for other agents. Thus, if an action were observed in a given context, similar percepts would be produced whether the cognitive agent itself or another agent took the action. The most significant difference would be that one percept would contain an instantiation of the “self” node, whereas the other would contain a representation for the concept of the other agent. The second mechanism is a set of codelets, small independently running programs, and schemes from procedural memory that contain slots that can bind to any agent allowing LIDA to perceive and reason about actions by other agents in the same way as it does about its own perceptions and actions. These two mechanisms, combined with the learning procedures in LIDA, allow humans and, perhaps to some extent, other primates, to form complex representations of other agents. Simply put, a procedural learning mechanism converts a series of percepts containing another agent’s concept node into a stream of schemes where the other agent’s node is converted to a slot that can bind to any similar agent, including the self. This highlights one difference between learning in humans and in some lower animals, which must be trained by repeatedly rewarding the desired behavior until it is learned. This latter type of learning is similar to traditional statistical machine learning techniques, in contrast to human-like learning, represented in cognitive models that take into account research in psychology, neuroscience and cognitive science [22].

3. Example Thinking about other peoples’ cognitive processes involves using partial knowledge about what is actually true, and ideas about how the mind works, to create a narrative about what happened (or would happen), and what a given individual would do in a particular situation. Some aspects of other minds are, or appear to be, perceived

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

141

directly. Others require deliberation. An example of the former would be a “gut” feeling about whether a person is telling the truth based on their demeanor. An example of the latter would be reasoning about a person’s beliefs and motives. A person’s demeanor is related to their facial expressions, body language, tone of voice, etc. Information about a person’s state of mind may be detected through the mirror neuron system, which reacts to subtle social signals whether expressed by the person themselves or observed by the person in others [26]. These signals accompany primal emotions such as anger, fear, or guilt and are difficult to mask [27]. Recognition of such signals has survival value at the individual level, though warning of threats, and at the group level, by increasing trust through the ability to detect false allegiances. There is no evidence that the mirror system is used in “higher-level” tasks such as reasoning about another person’s beliefs. There is evidence that people possess a relatively common set of beliefs about how minds work [28]. For example, there is an over-emphasis on logic that people use to explain both their own actions and those of others. This can be referred to as “naïve psychology” in analogy to the “naive physics” that people use in reasoning about the physical world. Both sets of beliefs contain heuristics that work well in most cases but are shown to be false in others. A famous example from naïve physics is that “heavier objects fall faster then lighter ones.” A similar example from naïve psychology is that “people act in their own self interest.” In order to avoid the complexities of the theory of mind in humans, we will use as an example a primate experiment that showed evidence of it in monkeys. 3.1. Theory of Mind in Macaca Mulatta (Rhesus Monkeys) A recent animal experiment [25] has shown evidence for the theory of mind in monkeys. In this section we describe the experiments and show how the LIDA model represents this process in a natural and realistic manner. By using experimental data from higher animals instead of humans, we can simplify the description while keeping the essential elements. The experimenters approached a monkey with two containers, a “noisy” container with bells attached to the lid and body, and a “silent” container without the bells. A grape was placed in each container. The action of placing the grape caused noise for the first container but not the second. The experimenters then backed away from the containers but remained in view. In one set of experiments, the experimenters faced the containers and in another set, they faced the ground. Data were collected when the monkey took the grape from one of the containers. When the experimenters faced the ground, the monkey preferentially chose the silent container. When the experimenters faced the containers, the monkey chose randomly. The researchers concluded that these data are evidence that the monkeys had a mental model of the human experimenters, who were considered potential rivals, i.e. competitors for the food. If the humans were not looking the containers, the monkey could get the food without alerting them by choosing the silent container. If the humans were looking at the containers, they would know of the monkey’s actions in either case, so it wouldn’t matter which container was chosen. The cognitive states and actions of the monkey will now be illustrated using the LIDA model. Figures 2 to 4 show a simplified version of the hypothetical perceptual memory components needed to recognize the events in the experiments. As shown in the figures, there are four root concepts, actions, relationships, goals, and objects.

142

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

Figure 2. Perceptual Memory Needed for Experiment (with a-kind-of links)

Figure 3. Perceptual Memory Relationships Needed for the Experiment

Figure 4. Perceptual Memory Objects and Goals Needed for the Experiment Perceptual memory, and the other conceptual structures described in this paper, will be represented as semantic nets, that is, directed graphs with labels on both the nodes and links [29]. The graphs are simplified to trees by repeating certain nodes. For example, the “eat” node under “external action” is the same as the “eat” node under “goals.” When the experimenter approaches the monkey with the containers and grapes, certain structures will be excited in perceptual memory and instantiated in the workspace as a percept. This is shown in figure 5, which simply contains a representation of the objects perceived in the scene. When the experimenter puts the grape in the noisy container, the perceptual structures in figure 6 are added. One advantage of the representation in the figures is that it can be translated into natural language. For example, the perceptual structure on the upper left in Figure 6 could be translated to: The person put a grape in container1, causing it to make noise.

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

143

Figure 5. Initial Percept in Experiment In order to avoid a proliferation of diagrams, only the natural language description will be used in some cases. When the experimenter puts a grape in container 2, similar perceptual structures appear in the workspace except that there is no noise because container 2 doesn’t have bells attached to it. These percepts translate to: The person put a grape in container2 and the person closed container2. Structure Building Codelets act upon the elements in the workspace. These codelets are simple computational processes that scan the workspace for elements that match class variables. In this paper we will use class names beginning with a question mark. The codelets also contain probable results, also represented as graph structures. The results can combine the triggering elements and/or include a new structure. The activation of the codelet depends on the likelihood of the result, given the trigger, which includes elements of the context implicit in the matched workspace structures. This likelihood based activation is constantly adjusted by feedback from the environment. The activation of the new structure depends on both the activation of the codelet and that of its triggering elements. Other structure building codelets can combine the new structures to build larger structures. This allows a reasonable number of codelets to build complex models. Attention Codelets are triggered by structures in the workspace. These codelets and the structures that triggered them form coalitions in the model. The coalitions compete for consciousness based their activation. The winner goes to consciousness and may eventually trigger actions by the agent.

Figure 6. Percepts for Placing a Grape in the Noisy Container

144

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

Figure 8. Alternative Plans: a) ?plan = plan1, ?container = container1, and b) ?plan = plan2, ?container = container2 The monkey sees the grapes, which are a kind of food. If the monkey is hungry or likes grapes, this perception may activate a goal structure in the workspace to eat the grape, as shown in Figure 7. Goal structures from perceptual memory are only potential goals. If this structure makes it to “consciousness,” it will be broadcast to all components of the model, including Procedural Memory, where it could activate schemes with results that may achieve the goal. Instances of the activated schemes will be incorporated in the behavior net [30, 31]. This will excite other components of the net, where the goals can be converted into behaviors to achieve them through the activation of streams of behavior codelets. In this case, the higher-level behavior is likely to result in volitional decision making to construct a plan to obtain and eat the grape. Its actions are internal and include activation of codelets to search, write to, and build structures in the workspace. The monkey’s mind goes through a planning process to construct a causally linked structure from perceptual structures in the workspace to the goal structure [6, 32, 33]. There are choices on how to achieve the goal, whether to take the grape from container 1 or container 2. The resulting plans are shown in Figure 8. According to ideomotor theory [34, 5], a cognitive agent will consider one plan of action at a time. In the LIDA implementation of this theory [33], once the consideration begins, a timer is set. If no objection to the plan comes to mind before the timer ends, the plan will be performed. If there is an objection, the timer is reset and an alternative plan is considered.

Figure 7. The Monkey’s Goal is to Eat a Grape

Figure 9. If an agent is Facing an Action, the Agent is Likely to See the Action

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

145

In this experiment, any objections to the monkey’s plans would likely concern possible interference by the human, which the monkey considers a rival. The monkey reasons about what the human would know about potential actions the monkey could take. This is the essence of the Theory of Mind. In the experiment, it can be accomplished with the following structures (Figures 9 to 12). If the monkey considers plan two first, no objection is created and it will go to consciousness (unless some coalition with higher activation wins the competition). It will then be broadcast to procedural memory and activate schemes that are sent to the behavior net. The resulting behavior stream will instantiate a series of external behavior codelets enabling the monkey to physically open the silent container and take the grape. If the monkey considers plan one first, however, an objection will be raised. This is due to a structure related to the concept of a rival (Figure 12): “?agent1 is a rival of ?agent2 causes ?agent1 to interfere with the plans of ?agent2 if they are known to ?agent1.” At this point, the monkey abandons plan one and considers plan two. Since there are no objections, the plan will be carried out. This explains the experimental results for the case where the human faces the ground. In the case when the experimenter is facing the scene, the monkey’s plan will be known whether or not it chooses the noisy box. This is a cognitive dilemma in the sense that the monkey could consider the first plan, find it blocked, go the second plan, find that it is also blocked, then go back to the first plan, resulting in an oscillation. There are three mechanisms for breaking the deadlock [33]. First, the plans for eating the grape compete with other potential activities represented in the workspace. The winning coalition makes it to consciousness and is acted upon. In the experiment, for example, there were cases when another monkey drove the subject from the scene and both plans were abandoned.

Figure 10. If an Agent is near an Action, and the Action makes a Noise, the Agent will Hear the Action

Figure 11. If an Agent Senses an Action, the Agent Knows the Action (has occurred)

146

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

Figure 12. If Agent1 is a Rival of Agent2, and Agent1 knows of Planned Actions by Agent2, Agent1 will Interfere with Agent2’s Actions Second, every time the same plan is reconsidered, the length of time on the timer is shortened so that it could end before the objection has time to form. Finally, if the other mechanisms do not force a decision, a metacognitive process will be triggered to accept whichever plan is under consideration. This multiplicity of tie-breaking mechanisms illustrates the importance of decisiveness for survival in higher animals and results in a random choice. It explains the case where the experimenter is facing the scene.

4. Discussion, Conclusions and Future Work The LIDA model can explain the results of the primate experiment in a psychologically realistic way. It offers an explanation of the theory of mind in monkeys in the sense that the actions depending on where the experimenters were facing were derived from a representation of the experimenter’s mental state and the monkey’s belief in their potential to interfere with the monkey’s plans. LIDA also accounts for the results that were not analyzed in the experiment, the times when the monkey abandoned the scene without trying to retrieve either grape. In LIDA, this is a result of losing the competition for consciousness. LIDA’s cognitive cycle is based on human cognition and is consistent with the older Sense-Plan-Act cycle, but contains many more cognitive functions. LIDA’s perceptions of the ongoing activities of itself and other agents create percepts in the workspace that trigger codelets for building additional structures, taking actions, and other cognitive functions that result in the model’s interpretation of the current situation. The basic representations in LIDA for implementing the theory of mind are found in both perceptual and procedural memories. A percept involving a given action and context can contain either the “self” node or the node for another agent, depending on who performed it. Schemes in procedural memory can contain slots that can be bound to any agent, the self or another. This is more general than what is known about the mirror neuron system, which responds strongly only to basic actions such as eating or grasping objects with the hands [35]. While a detailed explanation of all the types of learning and adaptation in LIDA is beyond the scope of this paper, LIDA’s ability for self-organization results from: a large number of simple behaviors and primitive features that can be combined in arbitrary ways; feedback from the environment; decay of learned concepts and procedures, including the ability to forget; and both competitive and cooperative learning, i.e., competition between coalitions of cognitive structures.

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

147

Uncertainty plays a role in LIDA’s reasoning through the base activation of its behavior codelets, which depend on the model’s estimated probability of the codelet’s success if triggered. LIDA observes the results of its behaviors and updates the base activation of the responsible codelets dynamically. It avoids combinatorial explosions by combining reasoning via association [36] with reasoning via deduction. One can create an analogy between LIDA’s workspace structures and codelets and a logic-based architecture’s assertions and functions. However, LIDA’s codelets only operate on the structures that are active in the workspace during any given cycle. This includes recent perceptions, their closest matches in other types of memory, and structures recently created by other codelets. The results with the highest estimate of success, i.e. activation, will then be selected. No attempt is made to find all possible solutions or reason until either a solution is found it is shown that none exists. If reasoning takes too long, time-keeping mechanisms such as the ones described above will force termination. The disadvantage is that incorrect conclusions are possible and potential solutions to problems can be overlooked, similar to the way in which human cognition works. One would expect higher level cognition to be more sophisticated in humans than in other primates and perhaps lacking in some lower mammals and other animals. The primate experiment was selected for this paper because of the simplicity of the situation involving evidence for the theory of mind. Its implementation in LIDA provides a mechanism to explain “one-shot” learning by observing a teacher. In this paper, the model is being tested only qualitatively, showing that it explains the behaviors exhibited by the monkeys. Further research will involve enhancing the existing LIDA implementation so that the experiment can be simulated and quantitatively confirm the predicted results, or not.

References [1]

Premack, D. G. and G. Woodruff. 1978. Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences 1: 515-526. [2] Osterberg, G. 1935. Topography of the layer of rods and cones in the human retina.Acta Ophthal. suppl. 6, no. 1-103. [3] Tse, P. U. (2003). If vision is 'veridical hallucination', what keeps it veridical? Commentary (p. 426427) on Gestalt isomorphism and the primacy of subjective conscious experience: a Gestalt Bubble model, by Steven Lehar. Behavioral and Brain Sciences, 26(4):375-408. [4] Franklin, Stan. 2007. A foundational architecture for artificial general intelligence. In Advances in artificial general intelligence: Concepts, architectures and algorithms, proceedings of the AGI workshop 2006, ed. Ben Goertzel and Pei Wang:36-54. Amsterdam: IOS Press. [5] Baars, Bernard J. 1988. A cognitive theory of consciousness. Cambridge: Cambridge University Press. [6] Baars, Bernard. 1997. In the theatre of consciousness. Global workspace theory, a rigorous scientific theory of consciousness. Journal of Consciousness Studies 4: 292–309. [7] Franklin, S., and A. C. Graesser. 1997. Is it an Agent, or just a Program?: A Taxonomy for Autonomous Agents. In Intelligent Agents III. Berlin: Springer Verlag. [8] Baars, B. J., and S. Franklin. 2003. How conscious experience and working memory interact. Trends in Cognitive Science 7:166-172. [9] Franklin, S., B. J. Baars, U. Ramamurthy, and M. Ventura. 2005. The Role of Consciousness in Memory. Brains, Minds and Media 1:1-38, pdf. [10] Franklin, S. and F. G. Patterson, Jr. (2006). The Lida Architecture: Adding New Modes of Learning to an Intelligent, Autonomous, Software Agent. Integrated Design and Process Technology, IDPT-2006, San Diego, CA, Society for Design and Process Science. [11] Ramamurthy, U., Baars, B., D'Mello, S. K., & Franklin, S. (2006). LIDA: A Working Model of Cognition. Proceedings of the 7th International Conference on Cognitive Modeling. Eds: Danilo Fum, Fabio Del Missier and Andrea Stocco; pp 244-249. Edizioni Goliardiche, Trieste, Italy.

148

D. Friedlander and S. Franklin / LIDA and a Theory of Mind

[12] Franklin, Stan. 1995. Artificial minds. Cambridge MA: MIT Press. [13] D'Mello, Sidney K, U Ramamurthy, D'Mello, A Negatu, and S Franklin. 2006. A procedural learning mechanism for novel skill acquisition. In Proceeding of adaptation in artificial and biological systems, aisb'06, ed. Tim Kovacs and James A R Marshall, 1:184–185. Bristol, England: Society for the Study of Artificial Intelligence and the Simulation of Behaviour. [14] Merker, Bjorn. 2005. The liabilities of mobility: A selection pressure for the transition to consciousness in animal evolution. Consciousness and Cognition 14: 89–114. [15] Franklin, Stan. 2005. Evolutionary pressures and a stable world for animals and robots: A commentary on merker. Consciousness and Cognition 14: 115–118. [16] Lehmann D, Strik WK, Henggeler B, Koenig T, and Koukkou M. 1998. Brain electric microstates and momentary conscious mind states as building blocks of spontaneous thinking: I. Visual imagery and abstract thoughts. Int. J Psychophysiology 29, no. 1: 1-11. [17] Massimini, M., F. Ferrarelli, R. Huber, S. K. Esser, H. Singh, and G. Tononi. 2005. Breakdown of Cortical Effective Connectivity During Sleep. Science 309:2228-2232. [18] Sigman, M., and S. Dehaene. 2006. Dynamics of the Central Bottleneck: Dual-Task and Task Uncertainty. PLoS Biol. 4. [19] Uchida, N., A. Kepecs, and Z. F. Mainen. 2006. Seeing at a glance, smelling in a whiff: rapid forms of perceptual decision making. Nature Reviews Neuroscience 7:485-491. [20] Willis, J., and A. Todorov. 2006. First Impressions: Making Up Your Mind After a 100-Ms Exposure to a Face. Psychological Science 17:592-599. [21] Melloni, L., Molina, C., Pens, P., Torres, D., Singer, W., and Rodriguez, E. 2007. Synchronization of Neural Activity across Cortical Areas Correlates with Conscious Perception. J. Neuroscience, 27(11): 2858-2865. [22] D'Mello, Sidney K, S Franklin, U Ramamurthy, and B J Baars. 2006. A cognitive science based machine learning architecture. In AAAI 2006 Spring Symposium Series Sponsor: American Association for Artificial Intelligence. Stanford University, Palo Alto, California, USA. [23] Frey, Scott H and Valerie E. Gerry. 2006. Modulation of neural activity during observational learning of actions and their sequential orders. Journal of Neuroscience 26: 13194-13201. [24] Buckner, Randy L. and Daniel C. Carroll. 2007. Self-projection and the brain. Trends in Cognitive Sciences 11, no. 2: 49-57. [25] Santos, Laurie R, Aaron G Nissen, and Jonathan A Ferrugia. 2006. Rhesus monkeys, macaca mulatta, know what others can and cannot hear. Animal Behaviour 71: 1175–1181. [26] Arbib, M.A. and G. Rizzolatti. 1997. Neural expectations: A possible evolutionary path from manual skills to language. Communication and Cognition 29: 393-423. [27] Ekman, P. 1993. Facial expression of emotion. American Psychologist 48: 384-392. [28] Saxe, Rebecca. 2005. Against simulation: the argument from error. TRENDS in Cognitive Sciences Vol. 9 No. 4, pp 174-179. [29] Franklin, Stan. 2005. Perceptual memory and learning: Recognizing, categorizing, and relating. In Symposium on Developmental Robotics: American Association for Artificial Intelligence (AAAI). Stanford University, Palo Alto CA, USA. [30] Maes, P. 1989. How to do the right thing. Connection Science 1:291-323. [31] Negatu, Aregahegn and Stan Franklin. 2002. An action selection mechanism for 'conscious' software agents. Cognitive Science Quarterly 2, no. special issue on "Desires, goals, intentions, and values: Computational architectures." Guest editors Maria Miceli and Cristiano Castelfranchi.: 363–386. [32] Hofstadter, D. R. & the Fluid Analogies Research Group (1995). Fluid concepts and creative analogies. New York: Basic Books. [33] Franklin, Stan. 2000. Deliberation and voluntary action in ‘conscious’ software agents. Neural Network World 10: 505–521. [34] James, W. 1890. The Principles of Psychology. Cambridge, MA: Harvard University Press. [35] Gallese, V., Fadiga, L., Fogassi, L., Rizzolatti, G.. Action recognition in the premotor cotex. Brain (1996), 119, 593-609. [36] Franklin, S. 2005. Cognitive robots: Perceptual associative memory and learning. In Proceedings of the 14th annual international workshop on robot and human interactive communication (Ro-Man 2005):427-433.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

149

How Might Probabilistic Reasoning Emerge from the Brain? Ben GOERTZEL, Cassio PENNACHIN

Novamente LLC

Abstract: A series of hypotheses is proposed, connecting neural structures and dynamics with the formal structures and processes of probabilistic logic. First, a hypothetical connection is proposed between Hebbian learning in the brain and the operation of probabilistic term logic deduction. It is argued that neural assemblies could serve the role of logical terms; synapse-bundles joining neural assemblies could serve the role of first-order term-logic statements; and in this framework, Hebbian learning at the synaptic level would be expected to have the implicit consequence of probabilistic deduction at the logical statement level. A conceptual problem arises with this idea, pertaining to the brain’s lack of a mechanism of “inference trails” as used to avoid circular reasoning in AI term logic inference systems; but it is explained how this problem may be circumvented if one posits an appropriate inference control mechanism. Finally, a brief discussion is given regarding the potential extension of this approach to handle more complex logical expressions involving variables – via the hypothesis of special neural structures mapping neural weights into neural inputs, hence implementing “higher order functions.”

1. Introduction Neuroscience cannot yet tell us how the brain generates abstract cognition [1]. However, the current body of experimental and theoretical knowledge does contain some tantalizing hints; and among these is a striking formal parallel between Hebbian learning as observed to occur in the brain [2,3] and certain inference rules and formulae existing in the theory of probabilistic logical inference [4-6]. In this paper we explore this parallel in moderate detail, with a view toward exploring what it may teach us about both complex neurodynamics and probabilistic inference. The result is a series of specific hypotheses pertaining to the manner in which probabilistic logical inference, at multiple levels of abstraction and sophistication, might emerge from complex neurodynamics. As an implicit side-effect, these ideas clarify the potential parallels between brain structures/dynamics and contemporary AGI architectures that incorporate uncertain term logic, such as the NARS system [7] and the Novamente Cognition Engine (NCE) [8] with its Probabilistic Logic Networks (PLN) component. We believe this is an important direction to be investigating, given the numerous recent successes of probabilistic methods in various domains of AI research, e.g. [914]. An increasing segment of the AI community is convinced that probabilistic methods are an essential aspect of intelligence; and if this is so, it behooves us to try to understand the potential neural underpinnings of probabilistic inference. Of course, the human mind is not always accurate in making probabilistic estimations [15], but this

150

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

does not imply that we aren’t in many cases carrying out such estimations in some sense. An exploration of the manners in which probabilistic reasoning system may display inference pathologies similar to human beings is provided in [4] and would take us too far afield here. To be sure, though, we are not claiming the human brain to be a highly accurate probabilistic-calculation engine. Rather we are claiming that probabilistic inference may be a reasonable, approximate high-level model of certain structures and dynamics that emerge from brain activity. First we articulate certain provisional assumptions regarding neural knowledge representation. These assumptions are not radical and essentially constitute a specific version of the classical Hebbian proposal that knowledge is neurally stored in cell assemblies [2], incorporating Susan Greenfield’s emendation that “core” cell assemblies are opportunistically expanded into larger transient assemblies when they come into attentional focus [16,17]. The Hebb/Greenfield approach suggests that logical terms may be neurally implemented as cell assemblies; and we build on this by suggesting that first-order term logic statements (representing conditional probabilities) may be neurally implemented as bundles of synapses between assemblies. This suggestion is harmonious with recent work on Bayesian inference as a property of neuronal populations [18,19]. All this leads up to the quite natural hypothesis that Hebbian learning on the neuronal level gives rise to dynamics on the cell-assembly level that are roughly equivalent to first-order probabilistic term logic. This appealing parallel runs into an apparent snag when one consideres the “trail” mechanism utilized in modern term-logic-based AI systems to avoid pathologies of circular inference [4,5,7]. However, we present here arguments and simulation results indicating that this may not be a significant issue for probabilistic term logic systems with appropriate control mechanisms, and hence may not be an obstacle to the conceptual linkage between neurobiology and logic conjectured here. In the final section we make some hypotheses regarding the extension of this perspective to handle more complex term logic expressions involving variables and quantifiers. The essential idea proposed is that, if one adopts a logical formalism involving combinators rather than explicit variables, then the problem of mapping complex logic expressions into neural structures and dynamics boils down to the problem of neurally representing what functional programmers call “higher-order functions” [20]. We suggest one possible neural architecture via which higher-order functions could be implemented, involving a novel “router” substructure that encodes the connection-weights of one neural subnetwork as inputs to another neural subnetwork. The overall meta-point we intend to make with these hypotheses is that there is not necessarily any significant “conceptual gap” between brain structures/dynamics and probabilistic logic, or uncertain logic more generally. The hypothesis that the brain’s dynamics give rise to emergent-level uncertain logic seems plausible and consistent with current knowledge in both AI and neuroscience. Due to limitations in the current experimental toolset, we don’t yet know enough to pinpoint exactly how this emergence happens, but we can make plausible hypotheses which can guide ongoing research, both AI and biological.

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

151

2. Provisional Assumptions Regarding Neural Knowledge Representation and Neurodynamics Before addressing reasoning, we must first deal briefly with two issues regarding the brain: knowledge representation and learning. No one really knows how the brain represents complex knowledge, nor how it learns, but there are some fairly widelyrespected high-level hypotheses, and for the purpose of the present paper, we will adopt the ones that seem to us most likely. Due to space considerations we do not review basic neurobiology and cognitive neuroscience here but assume the reader possesses the requisite textbook knowledge [1]. 2.1. Knowledge Representation via Cell Assemblies The hypothesis we’ll adopt regarding knowledge representation, put simply, is that a distinct mental concept is represented in the brain as either: •

•

a set of “cell assemblies”, where each assembly is a network of neurons that are interlinked in such a way as to fire in a (perhaps nonlinearly) synchronized manner a distinct temporal activation pattern, which may occur in any one (or more) of a particular set of cell assemblies

The general definition of a “mental concept” is a bit subtle (see Goertzel, 2006) but a simple example (which will serve adequately for the present discussion) is a category of perceptual stimuli to which the organism systematically reacts in different ways. Also, although we will focus mainly on declarative knowledge here, we note that the same basic representational ideas can be applied to procedural and episodic knowledge: these may be hypothesized to correspond to temporal activation patterns as characterized above. In the biology literature, perhaps the best-articulated modern theories championing the cell assembly view are those of Gunther Palm [21,22] and Susan Greenfield [16, 17]. Palm focuses on the dynamics of the formation and interaction assemblies of cortical columns. Greenfield argues that each concept has a core cell assembly, and that when the concept rises to the focus of attention, it recruits a number of other neurons beyond its core characteristic assembly into a “transient ensemble.”1 It’s worth noting that there may be multiple redundant assemblies representing the same concept – and potentially recruiting similar transient assemblies when highly activated. The importance of repeated, slightly varied copies of the same subnetwork has been emphasized by Edelman [23] among other neural theorists.

1

The larger an ensemble is, she suggests, the more vivid it is as a conscious experience; an hypothesis that accords well with the hypothesis made in [24] that a more informationally intense pattern corresponds to a more intensely conscious quale -- but we don’t need to digress extensively onto matters of consciousness for the present purposes.

152

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

2.2. Hebbian Learning Regarding learning, our main assumption is that synapses in the brain are adapted via some variant of Hebbian learning. The original Hebbian learning rule, proposed by Donald Hebb in his 1949 book [2], was roughly 1.

The weight of the synapse x--> y increases if x and y fire at roughly the same time 2. The weight of the synapse x --> y decreases if x fires at a certain time but y does not Over the years since Hebb’s original proposal, many neurobiologists have sought evidence that the brain actually uses such a method. What they have found, so far, is a lot of evidence for the following learning rule [25,26] 1. The weight of the synapse x --> y increases if x fires shortly before y does 3. The weight of the synapse x --> y decreases if x fires shortly after y does The new thing here, not foreseen by Donald Hebb, is the “postsynaptic depression” involved in rule component 2. Now, the simple rule stated above does not sum up all the research recently done on Hebbian-type learning mechanisms in the brain. The real biological story underlying these approximate rules is quite complex, involving many particulars to do with various neurotransmitters. Ill-understood details aside, however, there is an increasing body of evidence that not only does this sort of learning occur in the brain, but it leads to distributed experience-based neural modification: that is, one instance synaptic modification causes another instance of synaptic modification, which causes another, and so forth 2 [27]. 2.3. Virtual Synapses and Hebbian Learning Between Assemblies Hebbian learning is conventionally formulated in terms of individual neurons, but, it can be extended naturally to assemblies via defining “virtual synapses” between assemblies. Since assemblies are sets of neurons, one can view a synapse as linking two assemblies if it links two neurons, each of which is in one of the assemblies. One can then view two assemblies as being linked by a bundle of synapses. We can define the weight of the synaptic bundle from assembly A1 to assembly A2 as the number w so that (the change in the mean activation of A2 that occurs at time t+epsilon) is on average closest to w*(the amount of energy flowing through the bundle from A1 to A2 at time t). So when A1 sends an amount x of energy along the synaptic bundle pointing from A1 to A2, then A2’s mean activation is on average incremented/decremented by an amount w*x. In a similar way, one can define the weight of a bundle of synapses between a certain static or temporal activation-pattern P1 in assembly A1, and another static or temporal activation-pattern P2 in assembly A2. Namely, this may be defined as the 2

This has been observed in “model systems” consisting of neurons extracted from a brain and hooked together in a laboratory setting and monitored; measurement of such dynamics in vivo is obviously more difficult.

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

153

number w so that (the amount of energy flowing through the bundle from A1 to A2 at time t)*w best approximates (the probability that P2 is present in A2 at time t+epsilon), when averaged over all times t during which P1 is present in A1. It is not hard to see that Hebbian learning on real synapses between neurons implies Hebbian learning on these virtual synapses between cell assemblies and activation-patterns.

3. Probabilistic Term Logic as Emergent from Neural Structures and Dynamics Our next step is to connect these neurological entities we have hypothesized (virtual synapses between assemblies and activation-patterns) to entities existing within mathematical term logic: inheritance and similarity relationships. This will allow us to connect Hebbian learning on virtual synapses to term logic inference on these relationships. We assume the reader has a basic familiarity with uncertain term logic, as described e.g. in [5] and [11]. For space reasons, here we will only mention the uncertain term logic concepts that we use directly, and will make little effort to give adequate conceptual background. In this section we discuss only “first order term logic,” which does not involve explicit variables or quantifiers, and may be described as the logic of inheritance relationships. We’ll turn to higher-order term logic a little later. Roughly speaking, the semantics of the term logic relationship “A inherits from B” or A-->B, is that when B is present, A is also present. The truth value of the relationship measures the percentage of the times that B is present, that A is also present. “A is similar to B” or A B, is a symmetrical version, whose truth value measures the percentage of the times that either one is present, that both are present. A great deal of subtlety emerges when one examines the semantics of inheritance in detail [4,11]. Many subtypes of the basic inheritance relation arise, dealing with such distinctions as intension vs. extension and group vs. individual. Also, similarity relations may be treated as symmetrized inheritance relations. Here we will ignore these various aspects and assume a simplistic approach to inheritance, considering A -> B to be basically interpretable as P(B|A). Other aspects of inheritance may be treated according to variations of the ideas given here, but this is not explored here due to space considerations. There are two separate theories of uncertain term logic currently available: NARS, which is nonprobabilistic; and PLN, which is probabilistic. The main ideas presented here are valid only for PLN as they depend on probabilistic semantics. However, to the extent that NARS approximates PLN they are also of course applicable to NARS. Figure 1 shows the basic inference rules of first-order term logic [4-6,11]. Each of these comes with a truth value formula, which determines the truth value of the conclusion link based on the truth values of the premise links. In PLN, these formulas are grounded in probability theory; e.g. the deduction formula is based on a heuristic probabilistic independence assumption. The NARS deduction formula closely approximates the PLN deduction formula; but the same is not true for induction and abduction.

154

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

Figure 1. The four primary inference rules in first-order term logic How can this be tied in with the brain? Suppose we have two assemblies A1 and A2, and these are activated in the brain when the organism is presented with stimuli in category C1 and C2 respectively (to take the simplest case of concepts, i.e. perceptual categories). Then, we may say that there is a neural inheritance A1 -- > A2, whose probabilistic strength 3 is the number w so that (probability of A1’s mean activation being greater than T at time t)*w best approximates (probability of A2’s mean activation being greater than T at time t+epsilon) for an appropriate choice of epsilon and T. This weight w, intuitively, represents the conditional probability P(A2 is active | A1 is active). In a similar way, if we have two assembly activation patterns P1 and P2, which are defined as specific types of activity patterns occurring in A1 and A2 respectively, and which correspond to categories C1 and C2 respectively, we can define a neural inheritance P1 --> P2, whose probabilistic truth value is the number w so that (P1 is present in A1 at time t)*w best approximates (P2 is present in A2 at time t+epsilon), on average over various times t, and assuming a threshold T used to determine when a pattern is present in an assembly. It is immediate that, if there is a virtual synapse between A1 and A2 or P1 and P2, there will also be a neural inheritance there. Furthermore there will be a monotone increasing algebraic relationship between the weight of the virtual synapse and the probability attached to the neural inheritance. Inhibitory virtual synapses will correspond to very low link probabilities; excitatory virtual synapses will correspond to high link probabilities. However, we can have neural inheritance without any virtual synapse. This is a key point, as it lets us distinguish neural inheritance relationships that are explicit (that correspond to virtual synapses) from those that are implicit (that do not). And this leads us directly to probabilistic reasoning, which is about transforming implicit inheritance relationships into explicit ones. The fundamental inference rules of term logic, as described above, create new inheritance links from old ones. The conclusions are implicit in the premises but until the inference is done, they may not be explicitly 3

As uncertain term logic systems often use multiple-component truth values, the term “strength” is commonly used to denote the primary component, which in this case is a probability value.

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

155

contained in the knowledge base of the system doing the reasoning. Probabilistic reasoning in the brain, we suggest, is all about translating implicit neural inheritance relationships into explicit ones. In the PLN approach to term logic (but not the NARS approach), the four forms of inference shown in Figure 1 can actually be reduced to three forms: revision (which in its simplest form is weighted-averaging), deduction, and inversion (which reverses the direction of a link, and in probabilistic term logic is essentially Bayes rule). Let us elaborate what these mean in terms of cell assemblies. Suppose that A1 and A2 are two assemblies, and there is a neural inheritance between A1 and A2. Then, there will also be a neural inheritance between A2 and A1, with a truth value given by Bayes rule. And according to Hebbian learning if there is a virtual synapse A1 --> A2, there is likely to grow a virtual synapse A2 --> A1. And according to the approximate correlation between virtual synapse weight and neural inheritance probability, this new virtual synapse from A2 --> A1 will have a weight corresponding to a probability approximating the one corresponding to the weight of A1 --> A2. Similarly, suppose that A1, A2 and A3 are three assemblies. Then, if we have virtual synapses between A1 --> A2 and A2 --> A3, Hebbian learning suggests that a virtual synapse will grown between A1 --> A3. And what will the probability of the neural inheritance corresponding to this virtual synapse be? On average, it will be the probability one obtains by assuming the probabilities associated with A1 --> A2 and A2 --> A3 are independent. But, this means that on average, the probability associated with A1 --> A3 will accord with the value produced by the PLN deduction formula, which embodies precisely this independence assumption. Here, we have an additional source of error beyond what exists in the Bayes rule case; but, in the mean, the desired correspondence does hold. So, according to the above arguments – which admittedly have been intuitive rather than mathematically rigorous –it would seem that we can build term logic inference between concepts out of Hebbian learning between neurons, if we assume cell assembly based knowledge representation, via introducing the conceptual machinery of virtual synapses and neural inheritance. 3.1. Avoiding Issues with Circular Inference When one develops the ideas from the previous section, connecting uncertain term logic inference with neurodynamics, in more detail, only one possible snag arises. Existing computational frameworks for uncertain term logic inference utilize special mechanisms for controlling circular inference, and these mechanisms have no plausible neurological analogues. In this section we explore this issue and argue that it’s not necessarily a big deal. In essence, our argument is that these biologically unnatural circularity-avoidance mechanisms are unnecessary in a probabilistic term logic system whose operations are guided by appropriate adaptive attention-allocation mechanisms. It’s only when operating probabilistic term logic inference in isolation, in a manner that’s unnatural for a resource-constrained intelligent system, that these circularinference issues become severe. We note however that this conclusion seems to be specific to probabilistic term logic, and doesn’t seem to hold for NARS term logic, in which the circular inference problem may be more severe, and may in fact require a trail mechanism more strongly. We have not investigated this issue carefully.

156

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

To understand the circular inference problem, look at the triangles in Figure 1. It’s easy to see that by performing deduction, induction and abduction in sequence, we can go around and around an inference triangle forever, combining the links in different orders, inferring each link in the triangle from the two others in different orders over and over again. What often happens when you do this in a computer program performing uncertain term logic inference, however, is that after long enough the inference errors compound, and the truth values descend into nonsense. The solution taken in the NARS and PLN uncertain term logic inference engines is something called inference trails. Basically, each inheritance link maintains a trail, which is a list of the nodes and links used as premises in inferences determining its truth value. And a rule is put in place that the link L should not be used to adjust the truth value of the link M if M is in L’s trail. Trails work fine for computer programs implementing uncertain term logic, though managing them properly does involve various complexities. But, from the point of view of the brain, trails seem quite unacceptable. It would seem implausible to hypothesize that the brain somehow stores a trail along with each virtual synapse. The brain must have some other method of avoiding circular inferences leading to truth value noisification. In the context of the Novamente Cognition Engine (NCE) [8], an integrative AI system into which we have integrated the PLN probabilistic term logic engine, we have run a number of experiments with trail-free probabilistic inference. The first of these involved doing inferences on millions of nodes and links (with nodes representing words and links derived via word co-occurrence probabilities across a text corpus). What we found was that, in practice, the severity of the circular-inference problem depended on the inference control strategy. When one implemented a strategy in which the amount of attention devoted to inference about a link L was proportional to an estimate of the amount of information recently gained by doing inference about L, then one did not run into particularly bad problems with circular inference. On the other hand, if one operated with a small number of nodes and links and repeatedly ran the same inferences over and over again on them, one did sometimes run into problems with truth value degeneration, in which the term logic formulas would cause link strengths to spuriously converge to 1 or 0. To better understand the nature of these phenomena, we ran computer simulations of small networks of NCE nodes interlinked with NCE links indicating inheritance relationships, according to the following idea: 1. 2. 3. 4.

Each node is assumed to denote a certain perceptual category For simplicity, we assume an environment in which the probability distribution of co-occurrences between items in the different categories is stationary over the time period of the inference under study We assume the collection of nodes and links has its probabilistic strengths updated periodically, according to some “inference” process We assume that the results of the inference process in Step 3 and the results of incorporating new data from the environment (Step 2) are merged together ongoingly via a weighted-averaging belief-revision process

In our simulations Step 3 was carried out via executions of PLN deduction and inversion inference rules. The results of these simulations were encouraging: most of the time, the strengths of the nodes and links, after a while, settled into a “fixed point”

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

157

configuration not too distant from the actual probabilistic relationships implicit in the initial data. The final configuration was rarely equivalent to the initial configuration, but, it was usually close. For instance one experiment involved 1000 random “inference triangles” involving 3 links, where the nodes were defined to correspond to random subsets of a fixed finite set (so that inheritance probabilities were defined simply in terms of set intersection). Given the specific definition of the random subsets, the mean strength of each of the three inheritance relationships across all the experiments was about .3. The Euclidean distance between the 3-vector of the final (fixed point) link strengths and the 3-vector of the initial link strengths was roughly .075. So the deviation from the true probabilities caused by iterated inference was not very large. Qualitatively similar results were obtained with larger networks. The key to these experiments is the revision in Step 4: It is assumed that, as iterated inference proceeds, information about the true probabilities is continually merged into the results of inference. If not for this, Step 3 on its own, repeatedly iterated, would lead to noise amplification and increasingly meaningless results. But in a realistic inference context, one would never simply repeat Step 3 on its own. Rather, one would carry out inference on a node or link only when there was new information about that node or link (directly leading to a strength update), or when some new information about other nodes/links indirectly led to inference about that node-link. With enough new information coming in, an inference system has no time to carry out repeated, useless cycles of inference on the same nodes/links – there are always more interesting things to assign resources to. And the ongoing mixing-in of new information about the true strengths with the results of iterated inference prevents the pathologies of circular inference, without the need for a trail mechanism. What we see from these various experiments is that if one uses an inference control mechanism that avoids the repeated conduction of inference steps in the absence of infusion of new data, issues with circular inference are not severe, and trails are not necessary to achieve reasonable node and link strengths via iterated inference. Circular inference can occur without great harm, so long as one only does it when relevant new data is coming in, or when there is evidence that it is generating information. This is not to say that trail mechanisms are useless in computational systems – they provide an interesting and sometimes important additional layer of protection against circular inference pathologies. But in an inference system that is integrated with an appropriate control mechanism they are not required. The errors induced by circular inference, in practice, may be smaller than many other errors involved in realistic inference. For instance, in the mapping between the brain and uncertain term logic proposed above, we have relied upon a fairly imprecise proportionality between virtual synapse weight and neural inheritance. We are not attempting to argue that the brain implements precise probabilistic inference, but only an imprecise analogue. Circular inference pathologies are probably not the greatest source of imprecision.

4. How Might More Complex Logical Statements and Inferences Be Expressed in the Brain? The material of the previous sections comprises a speculative but conceptually coherent connection between brain structures and dynamics on the one hand, and

158

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

probabilistic logic structures and dynamics on the other. However, everything we have discussed so far deals only with first-order term logic, i.e. the logic of inheritance relationships between terms. Extension to handle similarity relationships, intensional inheritance and so forth is straightforward – but what about more complex term logic constructs, such as would conventionally be expressed using variables and quantifiers. In this section we seek to address this shortcoming, via proposing a hypothesis as to how probabilistic term logic in its full generality might be grounded in neural operations. This material is even more speculative than the above ideas, yet something of this nature is critically necessary for completing the conceptual picture. The handling of quantifiers, in itself, is not the hard part. In [6] it is shown that, in a term logic framework, if one can handle probabilistic variable-bearing expressions and functions, then one handle quantifiers attached to the variables therein. So the essence of the problem is how to handle variables and functions. And we suggest that, when one investigates the issue in detail, a relatively simple hypothesis emerges clearly as essentially the only plausible explanation, if one adopts the neural assembly theory as a working foundational assumption. In the existing body of mathematical logic and theoretical computer science, there are two main approaches to handling higher-order expressions: variable-based, or combinator-based [20]. It seems highly implausible, to us, that the human brain is implementing some sort of intricate variable-management scheme on the neuralassembly level. Lambda-calculus and other formal schemes for manipulating variables, appear to us to require a sort of complexity and precision that self-organizing neural networks are ill-suited to produce via their complex dynamics. Of course it is possible to engineer neural nets that do lambda calculus (as neural nets are Turingcomplete, as are biologically realistic models of cortical-column-assembly-based computation [28]), but this sort of neural-net structure seems unlikely to emerge via evolution, and unlikely to get created via known epigenetic processes. But what about combinators? Here, it seems to us, things are a bit more promising. Combinators are higher-order functions; functions that map functions into functions; functions that map {functions that map functions into functions} into {functions that map functions into functions}, and so forth. There are specific sets of combinators that are known to give rise to universal computational capability; indeed, there are many such specific sets, and one approach to the implementation of functional programming languages is to craft an appropriate set of combinators that combines universality with tractability (the latter meaning, basically, that the combinators have relatively simple definitions; and that pragmatically useful logic expressions tend to have compact representations in terms of the given set of combinators). We lack the neurodynamic knowledge to say, at this point, that any particular set of combinators seems likely to map into brain function. However, we may still explore the fundamental neural functionalities that would be necessary to give rise to a combinatory-logic-style foundation for abstract neural computation. Essentially, what is needed is the capability to supply one neural assembly as an input to another. Note that what we are talking about here is quite different from the standard notion of chaining together neural assemblies, so that the output of assembly A becomes the input of assembly B. Rather, what we are talking about is that assembly A itself – as a mapping from inputs to outputs – is fed as an input to assembly B. In this case we may call B a higher-order neural assembly. Of course, there are numerous possible mechanisms via which higher-order neural assemblies could be implemented in the brain. Here we will discuss just one.

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

159

Consider a neural assembly A1 with certain input neurons, certain output neurons, and certain internal “hidden layer” neurons. Then, suppose there exists a “router” neural assembly X, which is at the receiving end of connections from many neurons in A1, including input, output and hidden neurons. Suppose X is similarly connected to many other neural assemblies: A2, A3, ... and so forth; and suppose X contains a “control switch” input that tells it which of these assemblies to pay attention (so, for instance, if the control input is set to 3, then X receives information about A3). When X is paying attention to a certain assembly, it routes the information it gets from that assembly to its outputs. (Going further, we may even posit a complex control switch, that accepts more involved commands; say, a command that directs the router to a set of k of its input assemblies, and also points it to small neural assembly implementing a combination function that tells it how to combine these k assemblies to produce a composite.) Finally, suppose the input neurons of assembly B are connected to the router assembly X. Then, depending on how the router switch is set, B may be said to receive one of the assemblies Ak as input. And, next, suppose B’s output is directed to the control switch of the router. Then, in effect, B is mapping assemblies to assemblies, in the manner of a higher-order function. And of course, B itself is “just another neural assembly,” so that B itself may be routed by the router, allowing for assemblies that map {assemblies mapping assemblies} into assemblies, and so forth. Where might this kind of “router” assembly exist in the brain? We don’t know, at the moment. Quite possibly, the brain may implement higher-order functions by some completely different mechanism. The point we want to make however, is that there are concrete possibilities via which the brain could implement higher-order logic according to combinatory-logic type mechanisms. Combinators might be neurally represented as neural assemblies interacting with a router assembly, as hypothesized above, and in this way the Hebbian logic mechanisms proposed in the previous sections could be manifested more abstractly, allowing the full-scope of logical reasoning to occur among neural assemblies, with uncertainty management mediated by Hebbiantype synaptic modification.

Conclusions While our current level of knowledge about the brain does not support a rigorously grounded understanding of the emergence of abstract cognition from neural function, it is possible to draw interesting and highly evocative connections between the conceptual and formal models used to understand the brain, and the formal structures used in probabilistic logic to model and implement advanced aspects of cognition. From an AGI perspective, it is clear that, at this time, these connections do not provide sufficient guidance that it is possible to model an AGI architecture closely on the brain. However, we believe it is possible to create meaningful conceptual and formal mappings between AGI systems and neurobiological structures and dynamics, even when at first glance the AGI systems in question may involve completely nonbiological-looking things such as logical inference engines. The mapping given here between uncertain term logic and neurobiology allows one to construct mappings between uncertain term logic based AI systems like NARS [7] and Novamente [8] and the human brain – allowing for instance a deepening of the parallels outlined in [29]

160

B. Goertzel and C. Pennachin / How Might Probabilistic Reasoning Emerge from the Brain?

between the Novamente system and the human brain. This seems a fruitful ground for ongoing investigation.

References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29]

Gazzaniga, Michael. Cognitive Neuroscience. Norton, 2002 Hebb, Donald. The Organization of Behavior. Wiley, 1949 Paulsen, O.; Sejnowski, T. J. (2000). "Natural patterns of activity and long-term synaptic plasticity". Current opinion in neurobiology 10 (2): 172-179 Ben Goertzel, Matthew Iklé, Izabela Goertzel, Ari Heljakka, Probabilistic Logic Networks, Berlin; New York, Springer Verlag, 2008, to appear Matthew Iklé, Ben Goertzel, Izabela Goertzel, “Indefinite Probabilities for General Intelligence,” Advances in Artificial General Intelligence, IOS Press, 2007. Matthew Iklé, Ben Goertzel, “Quantifier Logic for General Intelligence,” Proceedings of AGI-08, IOS Press, 2008. Wang, Pei. Rigid Flexibility: The Logic of Intelligence. Springer-Verlag, 2006 Goertzel, Ben, Moshe Looks, Cassio Pennachin, “Novamente: An Integrative Architecture for Artificial General Intelligence,” Proceedings of AAAI Symposium on Achieving Human-Level Intelligence through Integrated Systems and Research, Washington DC, August 2004. Christopher D. Manning, Hinrich Schuetze, Foundations of Statistical Natural Language Processing, Cambridge, MA, MIT Press, 1999. Sebastian Thrun, Wolfram Burgard and Dieter Fox, Probabilistic Robotics, Cambridge, MA, MIT Press, 2005. John N. Holmes, Wendy J. Holmes, W.J. Holmes, Speech Synthesis and Recognition, Reading, UK, Taylor & Francis Ltd, 2002. Judea Pearl, Probabilistic Reasoning in Intelligent Systems, San Mateo, Morgan Kaufmann, 1988. Joseph Y. Halpern, Reasoning About Uncertainty, Cambridge, MA, MIT Press, 2003. progic07: The Third Workshop on Combining Probability and Logic Kahneman, Daniel, Paul Slovic and Amos Tversky. Judgment Under Uncertainty. Cambridge University Press, 1982 Subhojit Chakraborty1, 3, Anders Sandberg2 and Susan A. Greenfield/. Differential dynamics of transient neuronal assemblies in visual compared to auditory cortex, Experimental Brain Research, 1432-1106, 2007 Greenfield SA, Collins TF. A neuroscientific approach to consciousness. Prog Brain Res. 2005;150:11-23. Amari, Shun-ichi, Si Wu. Neural Implementation of Bayesian Inference in Population Codes, NIPS 2001) Ma WJ, Beck JM, Latham PE, Pouget A Bayesian inference with probabilistic population codes. Nat Neurosci 2006 Nov 9(11):1432-8 Field, Anthony and Peter Harrison. Functional Programming. Addison-Wesley, 1097 Markert H, Knoblauch A, Palm G. Modelling of syntactical processing in the cortex. Biosystems. 2007 May-Jun;89(1-3):300-15. Epub 2006 Palm, Gunther Neural Assemblies. An Alternative Approach to Artificial Intelligen ce. Springer, Berlin, Heidelberg, New York, 1982 Edelman, Gerald. Neural Darwinism. Basic Books, 1988 Goertzel, Ben. The Hidden Pattern. BrownWalker, 2006 Roberts P.D., Bell C.C. ; Spike-timing dependent synaptic plasticity in biological systems. Biological Cybernetics, 87, 392-403 , 2002 Lisman J., Spruston N.; Postsynaptic depolarization requirements for LTP and LTD: a critique of spike timing-dependent plasticity. Nature Neuroscience' 8, 839-41, 2005 Bi, G-q, Poo, M-m . Synaptic modifications by correlated activity: Hebb's postulate revisited. Ann Rev Neurosci ; 24:139-166, 2001 Simen, P.; Polk, T.; Lewis, R.; Freedman, E. Universal computation by networks of model cortical columns. Neural Networks, 2003. Proceedings of the International Joint Conference onVolume 1, Issue , 20-24 July 2003 Page(s): 230 - 235 vol.1 Goertzel, Ben and Cassio Pennachin. Novamente: An Integrative Approach to Artificial General Intelligence. In Artificial General Intelligence, Springer-Verlag, 2006

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

161

An Integrative Methodology for Teaching Embodied Non-Linguistic Agents, Applied to Virtual Animals in Second Life Ben GOERTZEL, Cassio PENNACHIN, Nil GEISSWEILLER, Moshe LOOKS, Andre SENNA, Welter SILVA, Ari HELJAKKA, Carlos LOPES Novamente LLC, Washington DC

Abstract. A teaching methodology called Imitative-Reinforcement-Corrective (IRC) learning is described, and proposed as a general approach for teaching embodied non-linguistic AGI systems. IRC may be used with a variety of different learning algorithms, but it is particularly easily described in EC lingo. In these terms, it is a framework for automatically learning a procedure that generates a desired type of behavior, in which: a set of exemplars of the target behavior-type are utilized for fitness estimation; reinforcement signals from a human teacher are used for fitness evaluation; and the execution of candidate procedures may be modified by the teacher via corrections delivered in real-time. An example application of IRC to teach behaviors to AI-controlled artificial animals embodied in the Second Life virtual world is described in detail, including a review of the overall virtual-animal-control software architecture and how the integrative teaching/learning methodology fits into it. In this example application architecture, the learning algorithm may be toggled between hillclimbing and probabilistic evolutionary learning. Envisioned future applications are also discussed, including an application to embodied language learning applicable to agents in Second Life and other virtual worlds. Keywords. Reinforcement learning, imitative learning, corrective learning, evolutionary programming, hill-climbing, MOSES, intelligent virtual agents

1. Introduction Supposing one intelligent agent (the “teacher”) has knowledge of how to carry out a certain behavior, and wants to transfer this knowledge to another intelligent agent (the “student”). But, suppose the student agent lacks the power of language (which might be, for example, because language is the thing being taught!). How may the knowledge be transferred? At least three methodologies are possible: • •

Imitative learning: The teacher acts out the behavior, showing the student by example Reinforcement learning: The student tries to do the behavior himself, and the teacher gives him feedback on how well he did

162

B. Goertzel et al. / An Integrative Methodology for Teaching Embodied Non-Linguistic Agents

•

Corrective learning: As the student attempts the behavior, the teacher actively corrects (i.e. changes) the student’s actions, guiding him toward correct performance

Obviously, these three forms of instruction are not exclusive. What we describe here, and call IRC learning, is a pragmatic methodology for instructing AGI systems that combines these three forms of instruction. We believe this combination is a potent one, and is certainly implicit in the way human beings typically teach young children and animals. We present IRC learning here primarily in the context of virtually embodied AGI systems – i.e., AGI systems that that control virtual agents living in virtual worlds. There is an obvious extension to physical robots living in the real world and capable of flexible interaction with humans. In principle, IRC learning is applicable more broadly as well, and could be explored in various non-embodied context such as (for instance) automated theorem-proving. In general, the term “IRC learning” may be used to describe any teacher/student interaction that involves a combination of reinforcement, imitation and correction. While we have focused in our practical work so far on the use of IRC to teach simple “animal-like” behaviors, the application that interests us more in the medium term is language instruction, and we will enlarge upon this a bit in the Conclusion. In collaboration with The Electric Sheep Company, our software firm Novamente LLC is currently in the midst of creating a large-scale commercial implementation of IRC learning, as a methodology for teaching virtual animals in Second Life and other online virtual worlds. The virtual animals we are now experimenting with are nonlinguistic animals that can carry out spontaneous behaviors while seeking to achieve their own goals. and can also specifically be trained by human beings to carry out novel tricks and other behaviors (which were not programmed into them, but rather must be learned by the AI on the fly based on interaction with an avatar controlled by a human teacher). This current experimental work will be used as the basis of a commercial product to be launched sometime in 2008. In Section 2 we will give a brief overview of our virtual-animal software architecture, and explain how the IRC methodology fits in, utilizing either hillclimbing or evolutionary learning, allied with simple inference, as the underlying learning engine (aspect 3 above). This software is work-in-progress and we don’t yet have anywhere near a full understanding of what the strengths and limitations of the IRC methodology will be in this context, but it has already proved capable of learning some simple behaviors and we are confident it will prove considerably extensible. After describing this Second Life virtual animal application in detail, we then briefly review our plans for future R&D, which include a subsequent application to embodied language learning in Second Life and other virtual worlds. We note that this is a compacted version of this paper, and a longer version is available on the AGI-2008 conference website. 1.1. IRC Learning in the Context of the Quest for Powerful AGI One may decompose the overall task of creating a powerful AGI system into four aspects (which of course are not entirely distinct, but still are usefully distinguished):

B. Goertzel et al. / An Integrative Methodology for Teaching Embodied Non-Linguistic Agents

1. 2. 3. 4.

163

Cognitive architecture (the overall design of an AGI system: what parts does it have, how do they connect to each other) Knowledge representation (how does the system internally store declarative, procedural and episodic knowledge; and now does it create its own representation for knowledge of these sorts in new domains it encounters) Learning (how does it learn new knowledge of the types mentioned above; and how does it learn how to learn, and so on) Teaching methodology (how is it coupled with other systems so as to enable it to gain new knowledge about itself, the world and others)

This article focuses on the fourth of these aspects, presenting some ideas about AGI teaching methodology that we believe to have quite general significance, although they were developed in the context of a specific AGI system (the Novamente Cognition Engine) that is founded on specific commitments regarding the other three aspects. For the first author’s views on the other three aspects of AGI, in the context of the NCE and more generally, the reader is directed to [1-3] and other prior publications. The focus on the fourth aspect in this paper should not be intended as a slight to the other three aspects: this is a brief paper and focuses narrowly on one part of the problem, but we mustn’t forget that it’s only one part of the problem. We need to build our AGI systems well, but we also need to teach them well, and it’s important to understand fully and deeply exactly what that means. The research presented here has been conducted under the working hypothesis that, via constructing appropriately-architected AGI systems and then instructing them appropriately in virtual worlds, it may be possible to move from the current relatively primitive state of technology to an advanced level of AGI much more rapidly than the bulk of AI researchers believe.

Figure 1. Screenshot from Second Life, illustrating various behaviors of a Novamente-AI-controlled virtual dog. The dog chasing a cat illustrates spontaneous behavior driven by the dog’s internal goals; the figure on the upper right illustrates single-avatar teaching (of soccer skills); the bottom figure illustrates two-avatar teaching (of frisbee skills).

164

B. Goertzel et al. / An Integrative Methodology for Teaching Embodied Non-Linguistic Agents

1.2. The Power of Virtual Worlds for Accelerating Progress toward Powerful AGI From an AI theory perspective, virtual worlds may be viewed as one possible way of providing AI systems with embodiment. The issue of the necessity for embodiment in AI is an old one, with great AI minds falling on both sides of the debate. The classic GOFAI systems (see [4] for a high-level review) are embodied only in a very limited sense; whereas [5] and others have argued for real-world robotic embodiment as the golden path to AGI. Our own view is somewhere in the middle: as outlined in [3] we suspect embodiment is very useful though probably not strictly necessary for AGI, and we also suspect that at the present time, it is probably more generally worthwhile for AI researchers to spend their time working with virtual embodiments in digital simulation worlds, rather than physical robots. The notion of virtually embodied AI is nowhere near a new one, and can be traced back at least to Winograd’s (1972) classic SHRDLU system. However, technology has advanced a long way since SHRDLU’s day, and the power of virtual embodiment to assist AI is far greater in these days of Second Life, Word of Warcraft, HiPiHi, Creatures, Club Penguin and the like. The range of possibilities is both obvious and astounding; but to concretely understand the potential power of virtual embodiment for AGI, in this essay we’ll focus on the virtual-animal product mentioned above.

2. Creating Virtual Animals for Second Life Next we discuss some of the practical steps we are currently taking, aimed at gradually realizing the above-described vision. We briefly describe the software architecture we have developed for controlling virtual animals in Second Life – which for the sake of this discussion we will call the Virtual Animal Brain (VAB). This architecture contains some nontrivial AI within it, related to action selection and the overall relationship between goals, procedures and contexts. However, we describe it here primarily to provide context for the discussion in the following two sections, which deal with the combined use of imitative and reinforcement learning in the context of learning embodied behaviors via evolutionary program learning, hillclimbing, and associative memory. The capabilities of our virtual animals, in their current form, include: spontaneous exploration of the environment; automated enactment of a set of simple predefined behaviors; efficient learning of another set of predefined behaviors; flexible trainability: i.e., (less efficient) learning of flexible behaviors invented by pet-owners on the fly; communication with the animals, for training of new behaviors and a few additional purposes, occurs in a special subset of English here called ACL (Animal Command Language); individuality: each animal has its own distinct personality; spontaneous learning of new behaviors, without need for explicit training Our main focus here will be on the “flexible trainability” aspect, but we will also touch on other aspects as appropriate. And, though we won’t stress this point, the same ideas discussed here in the context of teacher-focused learning, may also be used in a slightly modified form to enable spontaneous learning based on embodied experience gathered during self-directed world-exploration. Beyond the above, some capabilities intended to be added in relatively-near-future VAB versions include: recognition of novel categories of objects, and integration of object recognition into learning; generalization based on prior learning, so as to be able

B. Goertzel et al. / An Integrative Methodology for Teaching Embodied Non-Linguistic Agents

165

to transfer old tricks to new contexts; and use of computational linguistics (integrated into the Novamente Cognition Engine, which as described below underlies the VAB) to achieve a more flexible conversational facility. These will also be briefly discussed below. Note also that the VAB architecture described here is not particular to Second Life, but has been guided somewhat by the particular limitations of Second Life.

Figure 2. Initial Virtual Animal Brain software architecture

Figure 3. Next-Phase Virtual Animal Brain software architecture (tentatively intended for implementation in 2008)

166

B. Goertzel et al. / An Integrative Methodology for Teaching Embodied Non-Linguistic Agents

2.1. Software Architecture for Virtual Animal Control Figure 2 shows the initial version of the VAB architecture, which has currently been implemented. Figure 3 shows the next version of the architectre, which has been designed in detail and if all goes well will be implemented during 2008. All components of VAB but the Perceptual Pre-processor and the Global World Map will be specialized instances of the Novamente Cognition Engine, an existing C++ software system described in [1-3] which contains numerous functional components and has been used within two commercial applications (the Biomind ArrayGenius product for gene expression data analysis [7]; and the INLINK product for interactive natural language knowledge entry, founded on the RelEx semantic analysis engine [8]) and also within an R&D system that controls a humanoid agent learning simple behaviors in a 3D simulation world [9]. The learning servers will be described in the following section, as learning is the main focus of this paper. In the remainder of this section we will describe the other aspects of the architecture, which form the context in which the learning occurs. The Global World Map is in its initial incarnation basically a 2D navigation mesh for Second Life, where each point in space is labeled with information about the agents that reside there. Some markup is included in the map to indicate 3-dimensional features of the environment. Future versions will involve extension into a full 3D navigation mesh. First, the perceptual pre-processor is essentially a proxy that stands between SL and the rest of the architecture, translating the output of SL into an XML format that the VAB can understand. The current VAB system receives perceptions at a fairly high level – for instance, it observes a list of nearby objects, with metadata attached to them, and information about the spatial coordinates they occupy. Regarding avatars, it receives information regarding the animations the avatars are running at a given time (a more useful list for some avatars than others, as in some cases avatars may run animations for which our AI agent doesn’t know the semantics). Examples of the perceptions emanating from the Perceptual Pre-processor are things like (using a notation in which $ precedes variables): • • • • • •

I am at world-coordinates $W Object with metadata $M is at world-coordinates $W Part of object with metadata $M is at world-coordinates $W Avatar with metadata $M is at world-coordinates $W Avatar with metadata $M is carrying out animation $A Statements in Petaverse, from the pet owner

There is also proxy code that translates the actions and action-sequences generated by the VAB into instructions SL can understand (such as “launch thus-and-thus animation”). Due to the particularities of Second Life1, the current VAB system carries out actions via executing pre-programmed high-level procedures, such as “move forward one step”, “bend over forward” and so forth. Example action commands are:

1

The current Second Life API does not provide direct access to the skeletons underlying the characters executed in the world.

B. Goertzel et al. / An Integrative Methodology for Teaching Embodied Non-Linguistic Agents

• • • • •

167

Move ($d , $s) :$d is a distance, $s is a speed Turn ($a, $S) : $a is an angle , $s is a speed Jump ($d, $h, $s) : $h is a maximum height, at the center of the jump Say ($T), $T is text : for agents with linguistic capability, which is not enabled in the current version pick up($O) : $O is an object

Next, in the agent control software component, each virtual animal is associated with its own “animal brain” software object, including among other things: • • • • •

•

•

An in-built (weighted) set of goals, and a set of “personality parameters” that guide various aspects of its behavior A package of basic information: Location, owner, current physical form, etc. A declarative memory, which contains e.g. associations between names of behaviors, and schemata (procedures) carrying out those behaviors A “procedural memory” containing procedures that may be useful to enact An “episodic memory” containing specific episodes the animal has been involved in, especially those that have led to reward from a teacher, or to achievement of one of the animal’s goals An attention allocation process, determining which schemata the pet carries out at each point in time. This process will vary its behavior based on the pet’s “personality” as specified by user-supplied parameters An “active procedural pool” containing procedures that are currently in the midst of being acted out

The declarative memory stores a reasonably long record of what all the pets have seen and done. Of course, when memory gets full, sufficiently old memories are saved to disk (or deleted, but it would be nice to have samples on disk for later mining) One of the aspects of the system under current active development is the Collective Experience Store, which contains a process that scans the experience base and extracts interesting “unsupervised learning problems.” E.g. “Fred likes to feed pets. Let’s figure out what causes him to feed one pet but not another, and place that knowledge in the minds of those pets whose experiences are most useful in doing this figuring out.” Next, another extremely critical part of the architecture: action selection. As Stan Franklin [10] has pointed out, ultimately intelligence has got to come down to action selection. An intelligent agent has got to decide what actions to take at what points in time – this is the way it goes about achieving its goals. In the VAB, action selection is controlled for each animal within the object allocated to that animal inside the agent control component. As noted above, each animal has a set of goals, which are provided in advance. These are fairly basic things such as: don’t be too hungry or too full, don’t be too thirsty or too hydrated, seek social interaction, seek novelty, seek praise. Each of these goals has a certain numerical weight attached to it, and the relative weighting of the different goals in an individual animal constitutes an important part of that animal’s “personality.” Each animal’s brain also contains a declarative knowledge base, containing information the animal has learned during its lifetime – who is its owner, if it has an owner; where it has found various items at various points of time in the past; who has been friendly versus evil to it; and so forth. The declarative knowledge base is an

168

B. Goertzel et al. / An Integrative Methodology for Teaching Embodied Non-Linguistic Agents

AtomTable, to use NCE lingo; it contains weighted, typed nodes and links in the manner typically used within the Novamente Cognition Engine. And each animal’s brain also contains a procedural knowledge base, consisting of the set of behavior-generating procedures the animal has found useful in various contexts in the past. The declarative knowledge base contains relationships, in NCE node-and-link format, expressing the content “Procedure P, in context C has led to the achievement of goal G.” and associated with appropriate weightings including probabilistic truth values. For example, in the mind of an animal serving as a pet, there might be a link Implication @SURYLGHVDUDQJHRIFRPPRQVHQVHEHQFKPDUNSUREOHPV FRQWULEXWHG E\ PDQ\ UHVHDUFKHUV WKDW LQYROYH QDwYH RU FRPPRQVHQVH NQRZOHGJH DERXW PDWWHUVLQFOXGLQJSODQQLQJSK\VLFVDQGSV\FKRORJ\:KLOHPDQ\RIWKHVHµFKDOOHQJHSUREOHPV¶UHTXLUHJHQHUDOSXUSRVHSUREOHPVROYLQJVNLOOVWKH\DUHSUHVHQWHGZLWKLQUHVWULFWHG GRPDLQVDQGZLWKRQO\PRGHUDWHFRPSOH[LW\VXFKWKDWWKH\DUHXVHIXOIRUEHQFKPDUNLQJ IRUPDOLVPVWKHRULHVDQGFRPSRQHQWVDVRSSRVHGWRHQWLUHV\VWHPV (QWLUHV\VWHPVPD\ EHEHQFKPDUNHGLQUHDOLVWLFRSHQHQGHGVFHQDULRVVXFKDVFRPSHWLWLRQVOLNH5RER&XS5HVFXH5RER&XS#+RPHDQGWKH'$53$*UDQG&KDOOHQJHV3K\VLFDODQGVSDWLDOUHDVRQLQJSOD\DVLJQL¿FDQWSDUWLQDOORIWKHVHSUREOHPVKRZHYHUVXFKSUREOHPVDOVRLQFOXGH DVSHFWVRIQDwYHKXPDQSV\FKRORJ\HFRQRPLFVJDPHWKHRU\DJHQWEHKDYLRUSODQQLQJDQG SRWHQWLDOO\JHQHUDOSXUSRVHSUREOHPVROYLQJ([LVWLQJDSSURDFKHVWRWKHVHEHQFKPDUNVDQG FKDOOHQJHVWHQGWRUHO\RQVLJQL¿FDQWKXPDQHQJLQHHULQJHLWKHULQWKHIRUPRIODUJHVFDOH PDQXDONQRZOHGJHHOLFLWDWLRQe.g.>@ RUE\PDQXDOO\HQJLQHHULQJLPSOLFLWDQGWDVN VSHFL¿F FRPPRQVHQVHµNQRZKRZ¶LQWRWKHORZOHYHOIDFXOWLHVRIWKHURERWDUFKLWHFWXUH (e.g.>@ 7KHHIIRUWUHTXLUHGDQGWKHEULWWOHQHVVRIWKHVHH[LVWLQJDSSURDFKHVLVXQDFFHSWDEOHIRURXUSXUSRVHV :HREVHUYHKRZHYHUWKDWPRGHUQVLPXODWLRQV²FRPSXWHUJDPHVFRPSXWHUDQLPDWLRQV DQG YLUWXDO ZRUOGV²KDYH LQFUHDVLQJO\ GHWDLOHG OLIHOLNH HQYLURQPHQWV WKDW VRPHWLPHVUHVHPEOHWKHYHU\VLWXDWLRQVGHVFULEHGLQEHQFKPDUNSUREOHPV:KLOHVLPXODWLRQV

202

B. Johnston and M.-A. Williams / Comirit: Commonsense Reasoning

DUHFXUUHQWO\GLI¿FXOWWRGLUHFWO\H[SORLWDVDUHVRXUFHIRUFRPPRQVHQVHUHDVRQLQJWKH\ GRSUHVHQWDULFKUHVRXUFHRILPSOLFLWFRPPRQVHQVHµNQRZKRZ¶6RPHSURMHFWVKDYHDWWHPSWHGWRH[SORLWWKLVUHVRXUFHLQGLUHFWO\>@²SODFLQJDQDJHQWZLWKLQDVLPXODWHGHQYLURQPHQWWRH[SORUHDQGOHDUQDERXWKXPDQVHWWLQJVIUHHRIWKHZHDUFRVWWLPHDQGFRQFXUUHQF\FRQVWUDLQWVRIWKHUHDOZRUOG6XFKDSSURDFKHVDUHLQWHUHVWLQJEXWDUHOLPLWHGE\ SURJUHVV LQ PDFKLQH OHDUQLQJ DQG WKH DELOLW\ WR DXWRPDWLFDOO\ UHSUHVHQW JHQHUDOL]H DQG VSHFLDOL]HNQRZOHGJHDFTXLUHGIURPVLPXODWHGH[SHULHQFHV,QFRQWUDVWRXULQLWLDOREMHFtive is to directlyH[SORLWVLPXODWLRQDVDUHSUHVHQWDWLRQDQGUHDVRQLQJPHFKDQLVPUDWKHU WKDQPHUHO\DVDWUDLQLQJHQYLURQPHQW  2IFRXUVHZLWKW\SLFDOO\ RQO\DIRUZDUGPRGHRIUHDVRQLQJDQGPDQ\FULWLFDOOLPLWDWLRQVVLPXODWLRQVDUHQRWLPPHGLDWHO\XVHIXODVDPHFKDQLVPIRUFRPPRQVHQVHUHDVRQLQJ +RZHYHU LQ FRPELQDWLRQ ZLWK D VXLWDEOH DXWRPDWLF UHDVRQLQJ V\VWHP IRU DQ H[SUHVVLYH ORJLFWKHVHOLPLWDWLRQVFDQEHDYRLGHG3URRIVHDUFKIDFLOLWLHVFDQEHXVHGWRUHYHUVHWKH natural ‘arrow of time’ by postulating possible causes and then simulating to test whether WKH RXWFRPHV PDWFK REVHUYDWLRQV 6XFK SURRI VHDUFK IDFLOLWLHV FDQ DOVR EH H[SDQGHG WR JHQHUDWHWKHQHFHVVDU\GHWDLOWRDOORZSDUWLDOO\GH¿QHGDQGXQJURXQGREMHFWVWREHQXPHULFDOO\VLPXODWHGDQGVHDUFKIDFLOLWLHVPD\HYHQEHXVHGWRPDQDJHWKHSURFHVVRIFRQYHUWLQJV\PEROLFVFHQHGHVFULSWLRQVLQWRQXPHULFDOVLPXODWLRQV)XUWKHUPRUHWKHUHDVRQLQJ V\VWHP FDQ VROYH WKRVH VXESUREOHPV WKDW DUH SRRUO\ VXLWHG WR HQFRGLQJ LQ VLPXODWLRQV VXFKDVDEVWUDFWORJLFDOGHGXFWLRQDQGUDQGRPDFFHVVUHFDOORIVLPSOHIDFWXDOGDWD ,QSULQFLSOHLIZHDUHLQWHJUDWLQJVLPXODWLRQDQGORJLFZHKDYHPDQ\RSWLRQVIRUWKH FKRLFHRIXQGHUO\LQJWHFKQRORJLHVWKHVHOHFWLRQRIZKLFKFDQKDYHDGUDPDWLFLQÀXHQFH RQWKHFDSDELOLWLHVDQGHDVHRILQWHJUDWLRQRIWKHFRPSOHWHK\EULGV\VWHP$OWKRXJKLWLV entirely possible (and effective) to simultaneously use a range of simulation engines in WKHSOXJJDEOHK\EULGDUFKLWHFWXUHZHDUHSURSRVLQJLQWKLVSDSHURXUSUHIHUHQFHLVIRUD VLQJOHVLPXODWLRQSODWIRUPZLWKVXI¿FLHQWÀH[LELOLW\WRPRGHOWKHYDVWUDQJHRIVFHQDULRV WKDWDQDJHQWPD\HQFRXQWHU7KH6OLFNDUFKLWHFWXUH>@²DµPROHFXODU¶RUµEDOODQGVWLFN¶ DSSURDFKWRVLPXODWLQJDZLGHUDQJHRISK\VLFDODQGQRQSK\VLFDOSKHQRPHQD²VXLWVWKLV FULWHULDDQGKDVLQIDFWEHHQGHVLJQHGVSHFL¿FDOO\IRUFRPPRQVHQVHUHDVRQLQJ6LPLODUO\ VXFFHVVIXOLQWHJUDWLRQUHTXLUHVDUHDVRQLQJPHFKDQLVPWKDWLVHI¿FLHQWDQGÀH[LEOHDQGWKDW SURYLGHVVXLWDEOHµVFDIIROGLQJ¶XSRQZKLFKRWKHUPHWKRGRORJLHVPD\EHDGGHG7DEOHDX[ EDVHG UHDVRQLQJ V\VWHPV DUH LGHDO LQ WKLV UHJDUG WKH\ KDYH GHPRQVWUDWHG HI¿FLHQF\ DV WKHEDVLVRIPRGHUQ6HPDQWLF:HEUHDVRQHUV>@DQGWKHLUVHDUFKVWUDWHJ\LVEDVHGRQWKH FRQVWUXFWLRQRIFRXQWHUPRGHOVRUFRXQWHUZRUOGVWRZKLFKVLPXODWLRQVFDQEHDWWDFKHG 7KHIROORZLQJVHFWLRQLQFOXGHVDEULHIUHYLHZRIERWK6OLFNVLPXODWLRQDQGWDEOHDX[EDVHG UHDVRQLQJLQWKHFRQWH[WRIK\EULGFRPPRQVHQVHUHDVRQLQJ

2.

Preliminaries

2.1.

Slick Simulation

7KH6OLFNDUFKLWHFWXUH>@LVDJHQHUDOSXUSRVHDSSURDFKWRVLPXODWLRQGHVLJQHGIRUFRPPRQVHQVHUHDVRQLQJV\VWHPVIRUUHODWHGDSSURDFKHVVHH>@ 6OLFNLVGHVLJQHGIRUVLPXlating a wide range of phenomena including domains as diverse as physical solids and OLTXLGVQDwYHSV\FKRORJ\DQGHFRQRPLFEHKDYLRU,Q6OLFNWKHVWDWHRIDVLPXODWLRQLVUHSUHVHQWHGDVDK\SHUJUDSKLQZKLFKHYHU\YHUWH[DQGK\SHUHGJHLVDQQRWDWHGZLWKDIUDPH A set of functions perform iterative stepwise update to the state of the simulation during

203

B. Johnston and M.-A. Williams / Comirit: Commonsense Reasoning Entity id:#37 x: 3.452 rigidity: 51

y: 0.671 z: 6.332 type: {matter, solid, porcelain}

mass-density: 2.1

Join id:#189 connecting Entities: {id#56, id#57} broken: false flex: 3% spring-constant: 9000 type: {matter, solid, porcelain} Constraint id:#5 name: gravity applies-to: Entity target-type: {matter} function: z := z + ½ g ( t)² Figure 1.6DPSOH(QWLW\-RLQDQG&RQVWUDLQWLOOXVWUDWLQJWKHIUDPHOLNHGDWDVWUXFWXUHVRID6OLFNVLPXODWLRQ

HDFKWLFNRIDJOREDODGDSWLYH1FORFN:LWKDSSURSULDWHXSGDWHIXQFWLRQVDQGDQQRWDWLRQV 6OLFNFDQEHDSSOLHGWRDQ\HQYLURQPHQWZKHUHglobal behavior is governed by or can be predicted by localODZVRUUDWLRQDOLW\DVVXPSWLRQV)RUH[DPSOHLQWKHFDVHRISK\VLFDO HQYLURQPHQWVWKHK\SHUJUDSKLVVWUXFWXUHGWRDSSUR[LPDWHWKHVKDSHRISK\VLFDOREMHFWV annotations are used to describe the local physical properties of the object (the position RIYHUWH[ORFDOZHLJKWGHQVLW\ORFDOWHPSHUDWXUHDSSHDUDQFHetc.); and update rules corUHVSRQGWRGLVFUHWHWLPHYDULDQWVRIWKHODZVRI1HZWRQLDQPHFKDQLFV&RQVLGHU)LJXUH DVDQH[DPSOHRIKRZ6OLFNPLJKWEHXVHGWRUHSUHVHQWDFXSRIFRIIHH 1RWHWKDWWKLVPHWKRGRIVLPXODWLRQLVFKDUDFWHULVWLFDOO\QRQV\PEROLF:HGRQRWQHHG WRVSHFLI\KRZDFXSRIFRIIHHEHKDYHVEXWGHULYHLWVEHKDYLRUIURPWKHLQWHUDFWLRQVDQG IRUFHVWKDWRFFXURYHUWKHVLPSOL¿HGµPROHFXODU¶VWUXFWXUH7KDWLVPDFURVFRSLFSURSHUWLHV DQGEHKDYLRUVHPHUJHIURPVLPSOHPLFURVFRSLFXSGDWHUXOHVVLPSOLI\LQJWKHNQRZOHGJH HQJLQHHULQJSURFHVVDQGPD[LPL]LQJWKHJHQHUDOLW\RIWKHWHFKQLTXH :HKDYHSUHYLRXVO\GHPRQVWUDWHG>@KRZWKLVVLPSOHDUFKLWHFWXUHFDQEHLPSOHPHQWHGLQDFRQFUHWHV\VWHPDQGDSSOLHGWRHVWDEOLVKHGEHQFKPDUNSUREOHPVVXFKDVWKH(JJ &UDFNLQJSUREOHP>@$FRQFUHWHLPSOHPHQWDWLRQRIWKH6OLFNDUFKLWHFWXUHLQFOXGHVWKH following three critical classes/types:  Entity$QDQQRWDWHGYHUWH[ZLWKRSHUDWLRQVIRUYHUWH[VSHFL¿FSDUDPHWHUVWR EHVWRUHGDQGUHWULHYHGE\DWWULEXWHQDPH  Join$QDQQRWDWHGK\SHUHGJHZLWKDVHWRIHQGYHUWLFHV ZLWKRSHUDWLRQVIRU K\SHUHGJHVSHFL¿FSDUDPHWHUVWREHVWRUHGDQGUHWULHYHGE\DWWULEXWHQDPH  Constraint$ VWHSZLVH VLPXODWLRQ XSGDWH IXQFWLRQ WKDW TXHULHV WKH FXUUHQW VLPXODWLRQK\SHUJUDSKDQGXSGDWHVWKHDQQRWDWLRQVLQDGGLWLRQWRSRVVLEO\IRUNing the simulation or invalidating the simulation if either multiple or no valid IXWXUHVWDWHVH[LVW 7KH 6OLFN DUFKLWHFWXUH DV RULJLQDOO\ SURSRVHG DOVR LQFRUSRUDWHV D FRQWURO ODQJXDJH IRU LQVWDQWLDWLQJDQGPDQLSXODWLQJVLPXODWLRQVLQDGGLWLRQWRDGDWDEDVHPHFKDQLVPWKDWVWRUHV generic models for instantiation and manages the relationship between abstract symbols DQGWKHVLPXODWLRQK\SHUJUDSK7KHVHQRWRQO\HQDEOHWKHLQWHJUDWLRQRIVLPXODWLRQDQG V\PEROLFPHWKRGVEXWDOORZIRUVLPXODWLRQVWREHDXWRPDWLFDOO\FRQVWUXFWHGIURPSXUHO\ V\PEROLFTXHULHV:HZLOOVHHLQ6HFWLRQWKDWZKLOHWKHGDWDEDVHDQGFRQWUROODQJXDJH UHPDLQHVVHQWLDOLQDK\EULGDUFKLWHFWXUHWKH\DUHVXEVXPHGDQGJHQHUDOL]HGE\WKHXVHRI H[SUHVVLYHORJLFV

2.2.

Tableaux Reasoning

7KHPHWKRGRIDQDO\WLFWDEOHDX[LVDQDSSURDFKWRDXWRPDWLFDOO\SURYLQJRUGLVSURYLQJ  the truth of a logical statement by searching for models or worlds in which the negation of WKHORJLFDOVWDWHPHQWLVVDWLV¿DEOH7KHWHFKQLTXHLVRYHU\HDUVROGEXWKDVH[SHULHQFHG  &ORFNUDWHLVYDULDEOHDQGDGDSWVWRHQVXUHQXPHULFDOSUHFLVLRQUHPDLQVZLWKLQDFFHSWDEOHERXQGV

1

204

B. Johnston and M.-A. Williams / Comirit: Commonsense Reasoning

Table 1.%DVLFWDEOHDX[UXOHV Rule Condition R1a: R1b: R2: R3:

{A {A {A {A

B} B} B}

Action

{A} {B} {A`^B}

A}

Explanation H[WHQGWKHEUDQFK H[WHQGWKHEUDQFK IRUNLQWRWZRFKLOGEUDQFKHV (close the branch due to contradiction)

DUHFHQWVXUJHLQLQWHUHVWEHFDXVHRILWVDSSOLFDELOLW\WRPRGDOORJLFVGHVFULSWLRQORJLFV DQG6HPDQWLF:HEUHDVRQLQJ$WKRURXJKRYHUYLHZRIWKHPHWKRGDQGDSSOLFDWLRQVWRUHDVRQLQJZLWKSURSRVLWLRQDO¿UVWRUGHUPRGDOGHVFULSWLRQDQGRWKHUORJLFVFDQEHIRXQGLQ '¶$JRVWLQRet al.¶VHGV KDQGERRN>@RURWKHUVWDQGDUGUHIHUHQFHVIRUIRUPDOFRPSXWHU VFLHQFH)RUFRQYHQLHQFHZHSUHVHQWDEULHIUHYLHZRIWKHWHFKQLTXHIRUWKHSURSRVLWLRQDO FDVHEHORZ $WDEOHDX[V\VWHPLVDPHWKRGRISURRIE\FRQWUDGLFWLRQLWSURYHVYDOLGLW\E\VKRZLQJWKDWWKHQHJDWLRQRIDIRUPXODLVXQVDWLV¿DEOH7KHPHWKRGZRUNVE\XSGDWLQJDWUHH VWUXFWXUHZKRVHQRGHVDUHODEHOHGZLWKORJLFDOIRUPXODH7KHDOJRULWKPVWDUWVZLWKDQHgated formula as the root of a tree and repetitively applies a set of update rules that close or H[SDQGEUDQFKHVRIWKHWUHHDVDSSURSULDWH7KHEDVLFDOJRULWKPLVVLPSOHZKHQDEUDQFK FRQWDLQVDGLVMXQFWLRQWKHEUDQFKLVIRUNHGLQWRWZRFKLOGEUDQFKHVFRUUHVSRQGLQJWRHDFK RIWKHGLVMXQFWVZKHQDEUDQFKFRQWDLQVDFRQMXQFWLRQHDFKFRQMXQFWLVDGGHGWRWKHOHDI RIWKHEUDQFK7KHHIIHFWLVWKDWDIRUPXODLVFRQYHUWHGLQWRDWUHHZKHUHWKHSDUHQWFKLOG relationship can be read conjunctively and the sibling relationship can be read disjuncWLYHO\%HFDXVHWKHSDUHQWFKLOGUHODWLRQVKLSLVUHDGFRQMXQFWLYHO\ZHORRNDORQJEUDQFKHV SDWKVIURPWKHURRWWRDOHDIDORQJWKHDQFHVWRUGHVFHQGHQWD[LV IRUFRQWUDGLFWLRQ,IHYHU\ EUDQFKFRQWDLQVDFRQWUDGLFWLRQWKHQWKHUHLVQRFRQVLVWHQWFRXQWHUPRGHODQGWKHUHIRUH WKHRULJLQDOXQQHJDWHGIRUPXODPXVWEHYDOLG7KHXSGDWHUXOHVRIWKLVEDVLFDOJRULWKPDUH VXPPDUL]HGLQ7DEOH 7KH IROORZLQJ H[DPSOH LOOXVWUDWHV KRZ WKH WDEOHDX[ PHWKRG LV XVHG WR UHDVRQ WKDW ((a b) a) bLVXQLYHUVDOO\YDOLG a) bLVYDOLGZHQHJDWHLW (((a b) a) b)VLPSOLI\ To show that ((a b) a bDQGWKHQSODFHLWDWWKHURRWRIDWUHH:HWKHQ to negation normal form: (a b) DWWHPSWWRVKRZWKDWWKLVQHJDWHGIRUPXODDWWKHURRWRIWKHWUHHLVXQVDWLV¿DEOHE\DSSO\LQJ WKHEDVLFWDEOHDX[UXOHVXQWLODOOEUDQFKHVFDQEHFORVHG)LJXUHVD±FLOOXVWUDWHWKHUHVXOWV RIWKHLWHUDWLYHUXOHDSSOLFDWLRQ A contradiction can be seen along the left-hand branch of the completed tableau: node LVDQDQFHVWRURIQRGHVRVKRXOGEHUHDGFRQMXQFWLYHO\KRZHYHU a and a obviously 1

(a

b)

a

2

(a

b)

a

b Given Rule: R1a(1)

Figure 2a.7DEOHDXDIWHUWKH¿UVWVWHSDSSO\LQJUXOH 5DRQWKH¿UVWQRGH

1

(a

b)

a

2

(a

b)

a

1

(a

b)

a

2

(a

b)

a

3

b

Rule: R1a(1)

6

Rule: R1b(1)

7

Figure 2b. Tableau after the second step (applying UXOH5ERQWKHVHFRQGQRGH 

a

5

b Given

Rule: R1a(1) Rule: R1b(1)

b

3 4

b Given

a Rule: R2(4)

Rule: R1a(2)

b

Rule: R1b(2)

a 8

Rule: R3(5,6) 9

b

Rule: R2(4) Rule: R3(3,8)

Figure 2c. Completed tableau for ((a b) UHDFKHGDIWHUHLJKWVWHSV 

a)

b

B. Johnston and M.-A. Williams / Comirit: Commonsense Reasoning

1

(a

2

a

3 4 5

b)

Given

a

Rule: R1a(1)

b

Rule: R1b(1)

a

a Rule: R2(2)

205

6

b

Rule: R2(2)

Rule: R3(3,4)

Figure 3. Completed tableau for (a

b)

a.

IRUPDFRQWUDGLFWLRQ6LPLODUO\QRGHLVDQDQFHVWRURIQRGHDQGZKHQUHDGFRQMXQFWLYHO\WULYLDOO\IRUPVDFRQWUDGLFWLRQ$OOEUDQFKHVDUHFRQWUDGLFWRU\DQGFORVHGWKHUHIRUH WKHRULJLQDOIRUPXODLVYDOLG 6LPLODUO\FRQVLGHUWKHHIIHFWLIZHEHJLQZLWKDIRUPXODWKDW LVQRWYDOLGVXFKDV (a b) a:H¿UVWQHJDWHLW ((a b) a) (a b) aDQGDSSO\WKHWDEOHDX[UXOHV WKHQZHREWDLQDFRPSOHWHGWDEOHDXSHU)LJXUH A contradiction can be seen along the left-hand branch: node 3 (a) is an ancestor of node 4 ( a VRWKH\VKRXOGEHUHDGFRQMXQFWLYHO\EXWDUHLQFRQWUDGLFWLRQ7KHULJKWKDQG EUDQFKKRZHYHUUHPDLQVRSHQZLWKQHLWKHUFRQWUDGLFWLRQVQRUWKHSRVVLELOLW\RIDSSO\LQJ DUXOHZLWKRXWUHSHDWLQJ ,QIDFWWKHULJKWKDQGEUDQFKZKHQUHDGFRQMXQFWLYHO\ LVD a does not hold when we have a and counter model for our original formula: (a b) b7KHWDEOHDX[PHWKRGLVDQHI¿FLHQWZD\RIVHDUFKLQJIRUVXFKFRXQWHUPRGHOV :HKDYHDVVXPHGFRQYHUVLRQLQWRQHJDWLRQQRUPDOIRUP11) DQGWKHWKUHHRSHUDtors  and +RZHYHUE\H[WHQGLQJWKHWDEOHDX[V\VWHPZLWKUXOHVIRU11)FRQYHUVLRQLWLVSRVVLEOHWRDFFHSWDQ\IRUPXODRISURSRVLWLRQDOORJLF7KURXJKWKHDGGLWLRQRI \HWIXUWKHUUXOHVDQGDGGLWLRQDOW\SHVRIVWUXFWXUHV DUDQJHRIRWKHURSHUDWRUVDQGORJLFV FDQEHVXSSRUWHG>@ :HKDYHSURYLGHGRQO\DEULHIRYHUYLHZLJQRULQJDUDQJHRILVVXHVFRQFHUQLQJUXOHVHOHFWLRQVHDUFKVWUDWHJ\KHXULVWLFVDQGRSWLPL]DWLRQVTXDQWL¿FDWLRQDQGPRGDOLWLHVF\FOHV PDQDJLQJ UHSHDWHG UXOH DSSOLFDWLRQ FRPSOHWHQHVV FRUUHFWQHVV DQG WKH FRPSOH[LWLHV RI XQGHFLGDEOHORJLFVVXFKDV¿UVWRUGHUORJLF :KLOHWKHVHPDWWHUVDUHKLJKO\VLJQL¿FDQW LQDQ\GRPDLQLQFOXGLQJDSSOLFDWLRQVIRUFRPPRQVHQVHUHDVRQLQJZHKDYHQRW\HWKDG the opportunity to comprehensively consider every issue but have designed a system that WKURXJKFRQ¿JXUDELOLW\DQGPRGXODULW\ RIWDEOHDX[UXOHVPHWDVWUDWHJLHVDQGGDWDW\SHV UHPDLQV DJQRVWLF WR WKH SDUWLFXODUV RI D JLYHQ WDEOHDX[ DOJRULWKP +RZHYHU WKH FULWLFDO REVHUYDWLRQKHUHLVWKDWWDEOHDX[V\VWHPVFDQEHPDGHYHU\HI¿FLHQWFDQVXSSRUWDZLGH UDQJHRIORJLFVDQGDVZHZLOOVHHLQWKHIROORZLQJVHFWLRQHOHJDQWO\LQWHUIDFHZLWKRWKHU IRUPVRIµUHDVRQLQJ¶VXFKDVVLPXODWLRQ

3.

Integration

,QFRPELQLQJVLPXODWLRQDQGORJLFZHKRSHWRFUHDWHDV\VWHPJUHDWHUWKDQPHUHO\WKH VXPRILWVWZRSDUWV,QFRPELQDWLRQWKHWZRKDYHWKHFDSDELOLW\IRUDSRZHUIXOXQLRQ ORJLF XQORFNV WKH IXOO SRWHQWLDO RI WKH LPSOLFLW µNQRZKRZ¶ LQ VLPXODWLRQV DQG VLPXODWLRQGUDPDWLFDOO\HQKDQFHVWKHFDSDELOLWLHVDQGUHDOOLIHDSSOLFDELOLW\RIORJLF+RZHYHU in creating a hybrid system we must carefully address the mismatch between paradigms DQGWKHLQKHUHQWLQFUHDVHLQFRPSOH[LW\WKDWWKHLQWHJUDWLRQLQWURGXFHV)XUWKHUPRUHRXU goal is to create an open-ended system that can admit other technologies and thereby deYHORS VRSKLVWLFDWLRQ RYHU WLPH$VLGH IURP RXU SULPDU\ PRWLYDWLRQ RI FUHDWLQJ V\VWHPV ZLWKJUHDWHUFRPPRQVHQVHDZDUHQHVVZHDUHWKHUHIRUHJXLGHGE\WKUHHLPSRUWDQWGHVLJQ

206

B. Johnston and M.-A. Williams / Comirit: Commonsense Reasoning

REMHFWLYHVFRKHVLRQVLPSOLFLW\DQGRSHQHQGHGQHVV&RKHVLRQLVDFKLHYHGE\XVLQJDZHOO GH¿QHGLQWHUIDFHEHWZHHQWDEOHDXUHDVRQLQJDQGVLPXODWLRQGHVFULEHGLQ6HFWLRQ7KLV LQWHUIDFHLVLPSOHPHQWHGXVLQJKLJKO\UHJXODUL]HGGDWDVWUXFWXUHVGHVFULEHGLQ6HFWLRQ WKDWPD[LPL]HVLPSOLFLW\DQGRSHQHQGHGQHVV

3.1.

Tableaux-Simulation Interface

&OHDU VHPDQWLFV DVVLVW LQ PD[LPL]LQJ WKH FRKHVLRQ LQ WKH LQWHJUDWLRQ RI VLPXODWLRQ DQG WDEOHDX[EDVHGUHDVRQLQJ:LWKRXWDFOHDUDQGXQLI\LQJDEVWUDFWLRQWKHPDSSLQJEHWZHHQ SDUDGLJPVFDQEHKLJKO\FRQYROXWHGUHTXLULQJFRPSOH[DGKRFFRGHWRPRQLWRUDQDO\]H DQGWUDQVODWHGDWDVWUXFWXUHVDQGGDWDXSGDWHVEHWZHHQUHSUHVHQWDWLRQV,QVWHDGDVLQJOH ZHOOGH¿QHGFRXSOLQJLVSUHIHUUHGVXFKDVZLWKWKHFRQFHSWRISRVVLEOHZRUOGV %RWKVLPXODWLRQDQGWDEOHDX[UHDVRQLQJFDQEHVHHQDVSURFHVVHVFRQFHUQHGZLWKGLYLGLQJ DQG VXEVHTXHQWO\ µFORVLQJ¶ WKH VSDFH RI SRVVLEOH ZRUOGV ,Q WDEOHDX[EDVHG UHDVRQLQJV\VWHPVEURDGUHJLRQVRIWKHVSDFHRISRVVLEOHZRUOGVDUHGLYLGHGDQGFORVHGE\ UHVWULFWLRQV ZLWK ORJLFDO SUHSRVLWLRQV 6LPXODWLRQV LQ FRQWUDVW ZRUN RQO\ RQ QDUURZO\ GH¿QHGVSDFHVRISRVVLEOHZRUOGVEXWDOVRXOWLPDWHO\GLYLGHDQGµFORVH¶7KLVFRUUHVSRQGHQFHIRUPVWKHEDVLVRIRXULQWHJUDWLRQVWUDWHJ\DQGLVPRUHFOHDUO\LOOXVWUDWHGE\ZD\RI H[DPSOH&RQVLGHUWKHIROORZLQJSUREOHP A commonsense enabled real-estate matching system knows that John values his time but doesn’t have a car: he cycles or catches a bus depending on the weather. A convenient home is less than 15 minutes by bus (60km/hour) and by bicycle (12km/hour) from his workplace. We want to know whether a home 3km from work or 4km by the bus’s circuitous route is convenient. :HDVVXPHDK\EULGV\VWHPFRQVWUXFWHGIURPVLPSOL¿HGFRPSRQHQWV ‡ $ WDEOHDX[ UHDVRQHU XVLQJ UXOHV IRU XQTXDQWL¿HG SUHGLFDWH ORJLF HTXDOLW\   simple relations of order (  DQGDUXOHIRUH[SDQGLQJDSUHGLFDWHIURPLWVGH¿nition (  ‡ $ VLPSOH QXPHULFDO µVLPXODWLRQ¶ RI WKH SURFHVV RI FRPPXWLQJ WR ZRUN YLD GLIIHUHQW PHFKDQLVPV )RU LOOXVWUDWLYH SXUSRVHV ZH XVH WKH KLJKO\ VLPSOL¿HG DQG DEVWUDFWHGVLPXODWLRQRI)LJXUH ,Q WKH FRQWH[W RI WKLV V\VWHP ZH PLJKW WKHQ HQFRGH WKH VDPSOH SUREOHP DV WKH ORJLbike) conv ZKHUH bus speed  distance  ZKHUH cal formula (bus bike speed 12 distance 3 and where conv duration  conv We negate the formula and convert it to negation normal form: (bus bike) DQG DSSO\ WKH WDEOHDX[ UXOHV XQWLO QR PRUH UXOHV FDQ EH DSSOLHG i.e. WKH WDEOHDX KDV µVWDOOHG¶ UHVXOWLQJLQWKHWDEOHDXRI)LJXUHD6LQFHRSHQEUDQFKHVUHPDLQZHDWWHPSW VLPXODWLRQHDFKEUDQFKLVUHDGDVDQDUURZO\GH¿QHGVSDFHRISRVVLEOHZRUOGVDQGWKH VLPXODWLRQHQJLQHLVDFFRUGLQJO\LQYRNHGRQHDFKEUDQFK'RLQJVRH[SDQGVWKHWDEOHDX[ ZLWKWKHRXWSXWRIWKHVLPXODWLRQHQJLQHSHU)LJXUHE Inputs

speeddistance

Outputs

duration

Algorithm set x := distance return x as duration

speed

Figure 4.6LPSOL¿HGVLPXODWLRQDOJRULWKP1RWHWKDWZKLOHKLJKO\VLPSOL¿HGWKLVDOJRULWKPKDVVLPLODUFRQVWUDLQWVWRUHDOOLIHVLPXODWLRQVWKHLQSXWVDUHDVVXPHGWREHQXPHULFDODQGIXOO\VSHFL¿HGDQGWKHDOJRULWKP FDQRQO\EHXVHGLQWKHµIRUZDUG¶GLUHFWLRQWKDWLVLWFDQQRWEHGLUHFWO\XVHGWRFRPSXWHVSHHGJLYHQWKH GXUDWLRQDQGGLVWDQFH 

207

B. Johnston and M.-A. Williams / Comirit: Commonsense Reasoning

1

(bus

2

bus

bike) bike

3

conv

4

duration

5

bus

6

speed

7

distance

60 4

conv

15

8

bike

9

speed

10

distance

12 3

Figure 5a. Stalled tableau 1RWHWKDWQRGHVDQGZHUHFUHDWHGE\ H[SDQGLQJIURPGH¿QLWLRQ  convbus and bike

1

(bus

2

bus

conv

4

duration

bus

6

speed

7

distance duration

conv

bike

3

5

11

bike)

15

8

bike

9

speed

4

10

distance

3

4

12

duration

15

60

12

Figure 5b.7DEOHDXDIWHUVLPXODWLRQ

)LQDOO\ZHFDQUHYHUWEDFNWRWKHWDEOHDX[UXOHVDQGFORVHERWKEUDQFKHVXVLQJWKH contradictions that occur with duration in nodes 4 (duration ! DQGduration 4) in the left-hand branch and in nodes 4 (duration  DQGduration  LQWKHULJKW KDQG EUDQFK %HFDXVH DOO EUDQFKHV FDQ EH FORVHG ZH KDYH SURYHQ WKH RULJLQDO TXHU\ (bus bike) convDQGKDYHWKHUHIRUHVKRZQWKHKRXVHWREHµFRQYHQLHQW¶ 7KHJHQHUDOH[HFXWLRQVWUDWHJ\RIWKHComiritK\EULGV\VWHPIROORZVH[DFWO\WKLVSURFHVVRIDOWHUQDWLQJWDEOHDX[UXOHDSSOLFDWLRQDQGVLPXODWLRQ+RZHYHUSUDFWLFDOFRPSOH[LWLHVDUHLQWURGXFHGE\WKHQRQGHWHUPLQLVPWKDWZHKDYHOHIWLPSOLFLW,WLVRIWHQSRVVLEOH WRDSSO\ERWKVLPXODWLRQDQGWDEOHDX[UXOHVDWDQ\JLYHQSRLQWLQWLPHDQGWKHUHPD\LQ IDFWEHPXOWLSOHDSSOLFDEOHWDEOHDX[UXOHVIRUWKHWDEOHDX[V\VWHP6RSKLVWLFDWHGKHXULVWLFVPD\EHGHYHORSHGWRJXLGHWKHH[HFXWLRQVWUDWHJ\²RXUUHVHDUFKSURWRW\SHVJHQHUDWH VLPSOHFRVWHVWLPDWHVDQGVHOHFWEDVHGRQDJUHHG\FKHDSHVW¿UVWVWUDWHJ\EXWPRUHPDWXUH WHFKQLTXHVVXFKDVDPHDVXUHRIH[SHFWHGEHQH¿WFRXOGEHXVHG,QWKHIROORZLQJVHFWLRQ we outline the generic architecture which allows for pluggable replacement of selection VWUDWHJLHV ,QWHJUDWLQJ6OLFNVLPXODWLRQLQWRWKLVIUDPHZRUNSUHVHQWVLWVRZQFKDOOHQJHV$6OLFN VLPXODWLRQKDVSRWHQWLDOO\PLOOLRQVRIRXWSXWV²WKHYDOXHRIHYHU\DWWULEXWHRIHYHU\(QWLW\ RU-RLQDWHYHU\SRLQWLQWLPH²LQFOXGLQJWKHVHDVORJLFDOWHUPVLQWKHWDEOHDXZRXOGLQFXU VLJQL¿FDQWFRPSXWDWLRQDODQGPHPRU\FRVWVIRUOLWWOHEHQH¿W,QVWHDGDGGLWLRQDOlinking ConstraintsDUHDGGHGWRWKH6OLFNVLPXODWLRQWRPDQDJHWKHFRQQHFWLRQEHWZHHQV\PEROLF DQGµPROHFXODU¶DWWULEXWHVRIREMHFWVDWHDFKWLFNRIWKHFORFNWKH\FDOFXODWHWKHVWDWHRIDQ object in aggregate over its ‘molecular’ representation and generate corresponding abstract V\PEROLFWHUPVIRULQFOXVLRQLQWKHWDEOHDX)RUH[DPSOHDQµREMHFWEURNHQ¶linking ConstraintPLJKWUHSRUWWKDWDQREMHFWLVEURNHQDWDJLYHQSRLQWLQWLPHLIDQ\RILWV-RLQVKDYH µVQDSSHG¶ DW WKDW WLPH 6XFK FRQVWUDLQWV FDQ EH DGGHG WR HYHU\ VLPXODWLRQ KRZHYHU IRU IXUWKHUSHUIRUPDQFHJDLQVLWVXI¿FHVWRXVHDVLPSOHVWUDWHJ\RIRQO\LQFOXGLQJWKHlinking ConstraintsLIWKH\DSSHDUWREHUHOHYDQWWKDWLVWKH&RQVWUDLQWLVRQO\DFWLYHLILWVRXWSXW V\PEROVDUHUHIHUHQFHGE\ORJLFDOWHUPVLQWKHWDEOHDX 6XFKSHUIRUPDQFHRSWLPL]DWLRQV DUHYDOLGEHFDXVHDVLPXODWLRQFDQEHUHVWDUWHGDQGH[HFXWHGPXOWLSOHWLPHVWRJHQHUDWH QHFHVVDU\YDOXHVIRUWKHWDEOHDXLIDPLVVLQJ&RQVWUDLQWLVODWHUGHHPHGWREHUHOHYDQW )LQDOO\WDEOHDX[PHWKRGVDUHQRWRQO\DSSOLFDEOHWRWUXHIDOVHWKHRUHPSURYLQJ²WKH\ PD\ EH XVHG IRU TXHU\ DQVZHULQJ %\ DOORZLQJ XQL¿FDWLRQ GXULQJ UXOH DSSOLFDWLRQ DQG XQJURXQG 3URORJVW\OH YDULDEOHV D WDEOHDX FDQ JHQHUDWH RXWSXW LQ WKH IRUP RI YDULDEOH ELQGLQJV6\QWDFWLFDOO\WKLVLVDFKLHYHGE\LQFOXGLQJDTXHU\WHUPLQWKHULJKWKDQGVLGH RIDQLPSOLFDWLRQ)RUH[DPSOHLIZHKDYHDZIIF(x) that constrains the value of xZH can discover the legal values of x by posing a query F(x) x XZKHUHX is an unground

208

B. Johnston and M.-A. Williams / Comirit: Commonsense Reasoning

3URORJVW\OHYDULDEOH:KHQWKLVIRUPXODLVQHJDWHGDQGFRQYHUWHGLQWRQHJDWLRQQRUPDO IRUPWKHWDEOHDXZLOOFRQWDLQTXHU\WHUPx X that will allow the branch to be closed by XQL¿FDWLRQRIX in the contradiction rule with the true assignment for x

3.2.

Data-Structure Regularity

&RPELQLQJWKHWZRUHDVRQLQJVWUDWHJLHVLQDÀH[LEOHK\EULGV\VWHPUHTXLUHVDODUJHUDQJH of data structures and therefore introduces the data management challenge of maintaining LQWHUQDODQGPXWXDOFRQVLVWHQF\$VHOHFWLRQRIVRPHRIWKHVHGDWDVWUXFWXUHVIROORZ Deductive Data Structures./RJLFDOWHUPVWDEOHDX[6OLFNPROHFXODUUHSUHVHQWDWLRQVGDWDEDVHH[WHUQDOSURFHGXUHEDFNHGSUHGLFDWHV Strategic Data Structures. +HXULVWLF LQIRUPDWLRQ DQG VFRUHV VHDUFK TXHXHV ZRUN TXHXHV Supportive Data Structures.&DFKHGGDWDLQGH[HV &RQ¿JXUDWLRQ'DWD&XUUHQWO\DFWLYHKHXULVWLFVWDEOHDXUXOHVVLPXODWLRQµODZV¶PHWD VWUDWHJ\ $V\VWHPDWLFDSSURDFKLVUHTXLUHGDQGWKLVFDQEHIRXQGE\DSSO\LQJREMHFWRULHQWHGGHVLJQSULQFLSOHVDQGWKURXJKWKHXQL¿FDWLRQRIDOOUHDVRQHUVWDWHLQWRDVLQJOHUHSRVLWRU\ :HJHQHUDOL]HWKHXQLWRILQWHJUDWLRQ²DVSDFHRIZRUOGV²LQWRDFROOHFWLRQREMHFW 7KLVFROOHFWLRQNQRZQDVDWorldSpaceFRUUHVSRQGVWRERWKDEUDQFKRIDWDEOHDXDQG DQLQVWDQFHRIDVLPXODWLRQ$ WorldSpaceKDVWKUHHPDMRURSHUDWLRQV²GLYLVLRQIRUNLQJ H[SDQVLRQDGGLWLRQ DQGFORVXUH²FRUUHVSRQGLQJWRWKHWDEOHDXRSHUDWLRQVIRUGLVMXQFWLRQFRQMXQFWLRQDQGFRQWUDGLFWLRQ$VLGHIURPPHWDVWUDWHJ\allVWDWHDQGFRQ¿JXUDtion is stored within a WorldSpace A set of WorldSpaceV DUH JURXSHG LQWR D FROOHFWLRQ NQRZQ DV D Problem (DFK query is initially represented by a single Problem with a single WorldSpace containing WKHTXHU\IRUPXODDQGFRQ¿JXUDWLRQ7KHV\VWHPWKHQXVHVWDEOHDX[UXOHVDQGVLPXODWLRQWR reason about the WorldSpaceGLYLGLQJH[SDQGLQJDQGFORVLQJXQWLODOO WorldSpaces DUHFORVHGRUQRIXUWKHUSURJUHVVFDQEHPDGH$VLGHIURPDOLVWRIFXUUHQWWorldSpaceV the only system state stored in a Problem relates to meta-strategy: the processes of selecting a focus WorldSpace DQG LQWHUSUHWLQJ WKH ¿QDO VWDWH RI WKH Problem :KLOH IRFXV VHOHFWLRQ KDV OLWWOH LPSDFW RQ WKH WKHRUHWLFDO FDSDELOLWLHV RI WKH K\EULG V\VWHP LW KDVJUHDWFRPSXWDWLRQDOVLJQL¿FDQFH)RFXVVHOHFWLRQGULYHVWKHKLJKOHYHOVHDUFKDFURVV WorldSpaceV)RUH[DPSOHDVWDFNEDVHGIRFXVVHOHFWLRQVWUDWHJ\UHVXOWVLQGHSWK¿UVW VHDUFKDTXHXHEDVHGIRFXVVHOHFWLRQUHVXOWVLQEUHDGWK¿UVWVHDUFKDQGDSULRULW\TXHXH PD\EHXVHGIRUKHXULVWLFEHVW¿UVWRU$ VW\OHVHDUFKLQJ 7KH NH\ WR PLQLPL]LQJ WKH FRPSOH[LW\ RI WKH V\VWHP DQG \HW DOORZLQJ VLJQL¿FDQW PRGXODULW\DQGH[WHQVLELOLW\OLHVLQWKHUHJXODUL]DWLRQRIDOOVWDWHDQGGDWD5DWKHUWKDQVWRULQJWDEOHDXUXOHVKHXULVWLFVFRUHVVLPXODWLRQFRQ¿JXUDWLRQDQGVLPXODWLRQVWDWHLQVHSDUDWH OLQNHGGDWDVWUXFWXUHVWKH\DUHDOOVWRUHGDVWHUPVLQWKHWDEOHDX7KDWLVDWDEOHDXEUDQFK (a WorldSpace FDQFRQWDLQQRWRQO\ORJLFDOWHUPVEXWDZLGHUDQJHRIREMHFWVDOOLPSOHmented as subclasses of the class Node6XFKREMHFWVLQFOXGHVLPXODWLRQREMHFWVVXFKDV EntityJoin and Constraint; logical objects such as Term and BindingG\QDPLF objects such as Function and Task DQG ERRNNHHSLQJ REMHFWV VXFK DV Note ,Q WKLV VHWWLQJWKHWDEOHDX[DOJRULWKPLVJHQHUDOL]HGVRWKDWLQHDFKVWHSDEUDQFKLVVHDUFKHGIRU QRWMXVWPDWFKLQJORJLFDOWHUPVEXWDOVRWKHDSSURSULDWHUXOHVWRDSSO\,QIDFWWKHWDEOHDX[ algorithm is itself stored within a WorldSpaceDVDG\QDPLFREMHFW²DVDFunction Reasoning within a given WorldSpace IROORZV D VLPSOH LQWHUSUHWDWLRQ SURFHVV When a Node is added to a WorldSpaceWKHWorldSpace is searched for Functions ZLWKPDWFKLQJSDUDPHWHUOLVWV7KHVHFunctionsDUHWKHQLQYRNHGWRHLWKHUSHUIRUPWULY-

209

B. Johnston and M.-A. Williams / Comirit: Commonsense Reasoning

ial state updates or to generate Task objects that are in turn added to the WorldSpace Because TaskREMHFWVDOVRH[WHQGNodeWKHFUHDWLRQRIDTask can result in a cascade of Function calls to perform other minor state updates (such as setting priority) or create new TaskV:KHQIRFXVLVHYHQWXDOO\JLYHQWRDWorldSpaceDTask (or set of Tasks) is chosen from the WorldSpace based on the priority of the Task DQG VXEVHTXHQWO\ H[HFXWHG7KHW\SLFDOH[HFXWLRQSURFHVVLVLOOXVWUDWHGLQ)LJXUH A WorldSpaceEHJLQVDVDQHPSW\FROOHFWLRQ,WLVLQLWLDOL]HGE\DGGLQJRQHE\ one) an appropriate set of FunctionVIRUUHDVRQLQJ7KHTXHU\WHUPLVWKHQDGGHGWRWKH collection as a TermH[WHQGLQJNode UHVXOWLQJLQDFDVFDGHRIQHZTaskVWKDWGULYHWKH GLYLVLRQH[SDQVLRQDQGFORVXUHRIWKH WorldSpaceWKHFUHDWLRQRIQHZ Nodes and in turn the ongoing creation of new TaskV:KHQDOOTaskVKDYHEHHQH[KDXVWHGWKHV\VWHP DWWHPSWV WR H[HFXWH DQ\ UHPDLQLQJ µGHIDXOW¶ Functions (i.e. SDUDPHWHUOHVV IXQFWLRQV  DQGLIWKH\LQWXUQDUHXQDEOHWRPDNHSURJUHVVWKHEUDQFKLVGHHPHGWRKDYHVWDOOHGi.e. QRIXUWKHUSURJUHVVFDQEHPDGH  :LWKLQ WKLV DUFKLWHFWXUH WDEOHDX EDVHG UHDVRQLQJ LV LPSOHPHQWHG E\ D TableauxReasonerFunction that responds to logical TermV:KHQDTermLVDGGHG LWLVGHFRQVWUXFWHGDQGDQDSSURSULDWHGLYLVLRQH[WHQVLRQDFWLRQLVFKRVHQDQGFUHDWHGDVD new Task Simulation-based reasoning is made possible by a set of FunctionVWKDWUHFRJQL]H TermVWKDWFRQVWUDLQWKHVWDWHRIDVLPXODWLRQ,IIRUH[DPSOHD Term stating that x isa Coffee-Mug is inserted into a WorldSpaceWKHQDSimulationConstructorFunction ZLOO H[SDQG WKH WorldSpace with new Entitys and Joins appropriately describing the shape of a Coffee-Mug:KHQQRIXUWKHUSURJUHVVFDQEHPDGHXVLQJRWKHUUHDVRQLQJ PHWKRGVD SimulatorFunctionLPSOHPHQWHGDVDµGHIDXOW¶ FunctionLVH[HFXWHG creating a new Task that will apply every Constraint object (i.e.HYHU\SK\VLFDOODZ  in the WorldSpace to every Entity and Join in the WorldSpace:KLOHZHKDYHQRW GRQHVRRWKHUIRUPVRIVLPXODWLRQFDQEHLQWHJUDWHGXVLQJDQLGHQWLFDOSURFHVV $UDQJHRIRWKHUUHDVRQLQJPRGHVDUHDOVRSRVVLEOH6RPHRIWKHPRGHVWKDWZHDUH currently using are described below: ‡ +HXULVWLFVDQGSULRULWL]DWLRQDUHLPSOHPHQWHGE\Functions that compute a cost RU H[SHFWHG EHQH¿W DQG XSGDWH WKH SULRULW\ RI Tasks when either the Task is added (i.e.EHIRUHWKH\DUHH[HFXWHG RULQUHVSRQVHWRWKHDGGLWLRQRIRWKHUNLQGV of objects to the WorldSpace ‡ 0HWKRGV RI LQIRUPDO GHIDXOW UHDVRQLQJ DUH LPSOHPHQWHG LQ WZR ZD\V DV FunctionVWKDWJHQHUDWHSODXVLEOHH[WHQVLRQVLQUHVSRQVHWRWKHDGGLWLRQRIORJLcal terms; and ‘default’ (parameter-less) Functions that generate assumptions New Node

WorldSpace receives focus State Update

State Update

(Deduction/Simulation) Add Node

Find matching Functions

Execute matching Functions

Find all unexecuted Tasks

Run highest priority Task New Node

New Tasks

State Update

(Heuristics/Prioritization) Add Task

Find matching Functions

Execute matching Functions

New Tasks

Figure 6.7\SLFDOH[HFXWLRQLQWHUSUHWDWLRQSURFHVVIRUDWorldSpace

210

B. Johnston and M.-A. Williams / Comirit: Commonsense Reasoning

‡ ‡

4.

ZKHQRUGLQDU\ORJLFDOGHGXFWLRQKDVVWDOOHG 6HDUFKwithin a problem space is implemented by ‘default’ Functions that generate Tasks to divide a WorldSpaceLQWRDFRYHULQJVHWRIPXWXDOO\H[FOXVLYH FRQGLWLRQV 'DWDEDVHGULYHQ SUHGLFDWHV DQG µDEEUHYLDWHG¶ H[SUHVVLRQV DUH LPSOHPHQWHG DV )XQFWLRQV WKDW UHVSRQG WR WKH DGGLWLRQ RI D Term LQYRNLQJ H[WHUQDO URXtines that test the new Term RU JHQHUDWH H[SDQGHG IRUPV IRU DGGLWLRQ LQWR WKH WorldSpace

Results and Discussion

7KLVDUFKLWHFWXUHLVWKHFXOPLQDWLRQRIRXUH[SHULHQFHVLQLQWHJUDWLQJVLPXODWLRQDQGORJLFDOUHDVRQLQJLQQRYHOFRPELQDWLRQVDQGWKHRQO\DUFKLWHFWXUHLQRXUH[SHULHQFHWKDWVR HOHJDQWO\DFKLHYHVFRKHVLRQVLPSOLFLW\DQGRSHQHQGHGQHVV:HDUHFXUUHQWO\H[SORULQJ DQGLPSOHPHQWLQJGLIIHUHQWORJLFVVLPXODWLRQODZVDQGKHXULVWLFVWRPD[LPL]HWKHXVHIXOQHVVRIWKHV\VWHP 7KHEHVWZD\WRLOOXVWUDWHWKHIXOOSRZHURIWKLVDSSURDFKLVWKURXJKH[DPSOH&RQVLGHU DVLPSOHEHQFKPDUNVW\OHFKDOOHQJHRIDUHDVRQLQJZKHWKHULWLVEHWWHUWRXVHDVORZDFWLRQ RUIDVWDFWLRQWRPRYHDFXSRIFRIIHH:HFDQSRVHWKHSUREOHPDVDORJLFDOIRUPXODXVLQJ … as the modal ‘always’ operator): problem (action = slow-move(x) action fast-move[ ) x isa Mug x contains Coffee y isa Table x on y damage) …( mess standard-physics can-assume-defaults $OWHUQDWHO\ZHPLJKWSRVHWKHSUREOHPDVDTXHU\WRGLVFRYHUDQDSSURSULDWHDFWLRQi.e. problem action A  *LYHQWKLVTXHU\WKHV\VWHPEXLOGVDWDEOHDXZLWKWZREUDQFKHVFRUUHVSRQGLQJWRWKH WZRSRVVLEOHDFWLRQV,QERWKEUDQFKHVWKHx isa Mug and x contains CoffeeDUHH[SDQGHG into a set of Entitys and Joins; the standard-physics WHUP LV H[SDQGHG LQWR D VHW RI Constraints corresponding to the laws of physics; and the can-assume-defaultsLVH[panded into FunctionVWKDWJHQHUDWHIHDVLEOHSODFHPHQWVLQ'VSDFHIRUREMHFWVVXFK as the Table and the Mug$'UHQGHULQJRIWKHPLGDFWLRQVWDWHRIWKHWZREUDQFKHVFDQ EHVHHQLQ)LJXUH In the right-hand branch we have a contradiction caused by the mess generated in the simulation and the messFRQVWUDLQWRIRXURULJLQDOTXHU\7KHFRQFOXVLRQWKDWRXUK\EULG reasoner draws is that the preferred action is to use the slow-move action on the cup of FRIIHH

(

)

Figure 7.0LGDFWLRQ'UHQGHULQJRIWKHWZRWDEOHDXEUDQFKHVLQWKHFRIIHHFXSDFWLRQVHOHFWLRQSUREOHP

B. Johnston and M.-A. Williams / Comirit: Commonsense Reasoning

211

:KLOHWKLVH[DPSOHLVVRPHZKDWFRQWULYHGDJUHDWGHDORIJHQHUDOLW\LVHYLGHQW/LWWOH PRGL¿FDWLRQLVUHTXLUHGWRTXHU\WKHIHDVLELOLW\RIGLIIHUHQWNLQGVRIDFWLRQVGLIIHUHQWPDWHULDOVRUOLTXLGVRUHYHQWKHEHKDYLRUVRIHQWLUHO\QRYHOREMHFWV7KHDGYDQWDJHRIVLPXODWLRQLVWKDWLWLVQRWQHFHVVDU\WRSHUIRUPFRPSUHKHQVLYHNQRZOHGJHHOLFLWDWLRQWRGHVFULEH WKHEHKDYLRURIHYHU\REMHFWLQHYHU\VLWXDWLRQUDWKHUVLPXODWLRQFDQDXWRPDWLFDOO\FRPSXWHUHDOZRUOGFRQVHTXHQFHVVLPSO\IURPWKHSK\VLFDOSDUDPHWHUVRIREMHFWV&RPELQHG ZLWKWKHÀH[LELOLW\RIORJLFDOGHGXFWLRQWKLVPDNHVIRUDSRZHUIXOFRPPRQVHQVHUHDVRQLQJ V\VWHP

5.

Conclusion and Future Work

7KRXJKWKLVDUFKLWHFWXUHUHSUHVHQWVD¿UVWVWHSIRUWKHComiritSURMHFWDQGUHPDLQVDORQJ ZD\IURPWUXHDUWL¿FLDOJHQHUDOLQWHOOLJHQFHRXUHDUO\UHVXOWVKDYHGHPRQVWUDWHGERWKVLJQL¿FDQWSURPLVHDVDWRROIRUFRPPRQVHQVHUHDVRQLQJDQGDQHQRUPRXVFDSDELOLW\IRUH[WHQVLRQDQGPRGXODULW\5HDVRQLQJLQWKLVK\EULGDUFKLWHFWXUHFRPELQHVERWKWKHÀH[LELOLW\ DQGSRZHURIIRUPDOORJLFVZLWKWKHHDVHRIVSHFL¿FDWLRQDQGWKHLPSOLFLWFRPPRQVHQVH µNQRZKRZ¶RIVLPXODWLRQVHYHQLQRXUVLPSOHH[DPSOHVWKHDUFKLWHFWXUHHOHJDQWO\DQG HI¿FLHQWO\VROYHVSUREOHPVWKDWDUHGLI¿FXOWWRH[SUHVVXVLQJRWKHUPHWKRGV 7KHUHLVDPSOHVFRSHIRUIXWXUHGHYHORSPHQW,QWKHVKRUWWHUPZHLQWHQGWRH[SORUH WKH SRWHQWLDO IRU LQFOXGLQJ QHZ WDEOHDX UXOHV IRU PRUH HI¿FLHQW DQG GLIIHUHQW IRUPV RI ORJLFDOGHGXFWLRQIRULQFOXGLQJHQWLUHO\QHZUHDVRQLQJPHFKDQLVPVLQWHJUDWLQJDORQJWKH µSRVVLEOHZRUOGV¶ DQGIRUGHYHORSLQJQHZKHXULVWLFV2XUORQJHUWHUPYLVLRQLQFOXGHVWKH integration of the reasoning and simulation mechanisms with machine learning and robot YLVLRQV\VWHPVDQGVXEVHTXHQWO\GHSOR\LQJWKHWHFKQRORJ\LQUHDOZRUOGURERWV\VWHPV *LYHQ WKH PRGXODULW\ RI WKH DUFKLWHFWXUH DQG WKH JHQHUDOLW\ RI LQWHJUDWLRQ YLD µSRVVLEOH ZRUOGV¶ZHEHOLHYHWKDWWKHSRWHQWLDOLVHQRUPRXV

References >@ 0RUJHQVWHUQ/ 0LOOHU5The Commonsense Problem PageYLHZHG0D\ KWWSZZZIRUPDOVWDQIRUGHGXOHRUDFRPPRQVHQVH! >@ %DUNHU.&KDXGKUL9&KDZ6&ODUN3)DQ-,VUDHO'0LVKUD63RUWHU%5RPHUR3 7HFXFL' @ /HQDW'%*XKD593LWWPDQ.3UDWW' 6KHSKHUG0µ&\FWRZDUG3URJUDPVZLWK &RPPRQ6HQVH¶Communications of the ACMYROQRSS >@ %U\VRQ-µ7KHEHKDYLRXURULHQWHGGHVLJQRIPRGXODUDJHQWLQWHOOLJHQFH¶LQAgent Technologies, Infrastructures, Tools and Applications for e-Services/1&6 >@ *RHUW]HO%+HOMDNND$%XJDM6 3HQQDFKLQ0µ([SORULQJDQGURLGGHYHORSPHQWDOSV\FKRORJ\LQDVLPXODWLRQZRUOG¶3URFRI,&&&6 >@ -RKQVWRQ% :LOOLDPV0$µ$JHQHULFIUDPHZRUNIRUDSSUR[LPDWHVLPXODWLRQLQFRPPRQVHQVHUHDVRQLQJV\VWHPV¶WK,QW6\PRQ/RJLFDO)RUPDOL]DWLRQVRI&RPPRQVHQVH5HDVRQLQJ >@ +RUURFNV, 6DWWOHU8µ2QWRORJ\UHDVRQLQJLQWKH6+24' GHVFULSWLRQORJLF¶,-&$, >@ *DUGLQ) 0HOW]HU%µ$QDORJLFDOUHSUHVHQWDWLRQVRIQDwYHSK\VLFV¶$UWL¿FLDO,QWHOOLJHQFHYRO QRSS >@ +R\HV.µ'VLPXODWLRQWKHNH\WR$,¶LQ*RHUW]HO% 3HQQDFKLQ&HGV $UWL¿FLDO*HQHUDO Intelligence6SULQJHU%HUOLQ >@ 0RUJHQVWHUQ/µ0LGVL]HGD[LRPDWL]DWLRQVRIFRPPRQVHQVHSUREOHPVDFDVHVWXG\LQHJJ FUDFNLQJ¶Studia LogicaYROQRSS >@ '¶$JRVWLQR0*DEED\'+lKQOH5 3RVHJJD-HGV Handbook of Tableau Methods .OXZHU1HWKHUODQGV

212

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

Learning from Inconsistencies in an Integrated Cognitive Architecture Kai–Uwe KÜHNBERGER, Peter GEIBEL, Helmar GUST, Ulf KRUMNACK, Ekaterina OVCHINNIKOVA, Angela SCHWERING, and Tonio WANDMACHER University of Osnabrück, Institute of Cognitive Science, Albrechtstr. 28, 49076 Osnabrück, Germany Abstract. Whereas symbol–based systems, like deductive reasoning devices, knowledge bases, planning systems, or tools for solving constraint satisfaction problems, presuppose (more or less) the consistency of data and the consistency of results of internal computations, this is far from being plausible in real–world applications, in particular, if we take natural agents into account. Furthermore in complex cognitive systems, that often contain a large number of different modules, inconsistencies can jeopardize the integrity of the whole system. This paper addresses the problem of resolving inconsistencies in hybrid cognitively inspired systems on both levels, in single processing modules and in the overall system. We propose the hybrid architecture I–Cog as a flexible tool, that is explicitly designed to reorganize knowledge constantly and use occurring inconsistencies as a non–classical learning mechanism. Keywords. Integrated Cognition, Hybrid Systems, Inconsistencies, Learning

Introduction Natural agents, in particular, humans master a large variety of cognitively important challenges. They can perform a variety of different reasoning tasks, store a large number of different kinds of information and knowledge, they have the ability to learn from noisy and sparse data, and show a remarkable potential of creativity in problem solving. An attempt to model such abilities in its broad range with machines necessarily results in a large number of computing paradigms, specialized modules, competing computations, different representation formalisms etc. Consequently coherence problems of the overall system, as well as inconsistency clashes in the single modules are natural side–effects of such integrated architectures. The problem is even worse if the system is hybrid in nature containing symbolic processing devices and connectionist–inspired modules. Before focusing on the coherence problem in more detail, let us at first make a detour in discussing some aspects concerning neuro–inspired models and symbolic models for intelligent systems. In artificial intelligence, there is a certain tension between symbolic approaches for modeling higher cognitive abilities and neural approaches for learning patterns in noisy environments. As a matter of fact, the two approaches have different strengths and weaknesses: whereas symbolic theories have problems in learning from noisy data, controlling elementary behaviors of artificial agents, or detecting patterns in

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture

213

perception input, connectionist systems are not well–suited to perform deduction steps in reasoning processes, to generate plans for a planning agent, or to represent complex data structures. This gap is not only obvious with respect to different application domains, but also with respect to the underlying methodology. Connectionist systems are usually based on analytic tools, whereas most symbolic systems use logical or algebraic theories for realizing computations. The idea to combine the strengths of the two modeling paradigms in a hybrid architecture is a natural way to cover the complementary applications domains. Nevertheless there are at least two non–trivial problems for hybrid architectures: • On which level should learning be implemented? • What are plausible strategies in order to resolve occurring inconsistencies in single modules, as well as in the overall system? This paper proposes to bring the two mentioned aspects together (consistency problems and learning tasks in integrative architectures) by using occurring inconsistencies in modules of an architecture and in the overall systems as a mechanism for learning. We propose the I–Cog architecture [1,2] as a model for addressing these problems. I–Cog is a hybrid architecture consisting of three modules: An analogy engine (AE) as a reasoning device, an ontology rewriting device (ORD) as a memory module for coding ontological background knowledge, and a neuro–symbolic learning device (NSLD) for learning from noisy data and drawing inferences in underdetermined situations. The overall claim of this paper is that inconsistencies should not be considered solely as a problem for hybrid systems, but rather as a crucial tool to make (cognitive) learning possible in the first place: Inconsistencies make the adaptation of knowledge and the creative establishment of new knowledge necessary and are therefore triggers for learning new facts. We claim that resolving inconsistencies is not only a problem for hybrid systems, but for every realistic system. Therefore strategies to resolve inconsistencies need to be implemented in all modules, not only with respect to the interaction of the involved modules. The paper has the following structure: in Section 1, we sketch some ideas of the I– Cog architecture. Section 2 discusses exemplarily resolution and learning strategies from occurring inconsistencies in AE, ORD, and the overall system. Section 3 summarizes related work and Section 4 concludes the paper.

1. The I–Cog Architecture I–Cog is a hybrid architecture for integrated cognition. In this section, we mention the overall idea very briefly.1 I–Cog consists out of three main modules: • An analogy engine (AE) is used to cover various forms of classical and non– classical reasoning. This module is based on heuristic–driven theory projection (HDTP) [3], a mathematically sound theory for computing analogical relations between a target and a source domain. HDTP is an extension of the theory of anti– unification [4,5]. The analogy engine was applied to compute analogical relations in a variety of domains like metaphoric expressions [3], qualitative physics [6], or in the geometry domain [7]. 1 In [1] and [2], the overall system is described more precisely.

214

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture

MEMORY

REASONING Restricts

Ontology Rewriting Device ORD: Ontology Hypotheses

Analogy Engine AE:

Queries

Retrieve Domains Generalized The ory

(SQL database)

Updates Input from various sources: The input is preprocessed and coded in a first-order logical language L

So u rc e

Targ e t

Input

Answer Queries Trains

Export

Control Device CD:

Translation to axioms Answer Queries

Neuro-symbolic Learning Device NSLD:

Trains

Queries and test of the answers from the rewriting module and the Export sub-symbolic module

Peripheral devices: Examples for the export in different formats are web interfaces, planning systems, or reasoning engines

TOPOS Learning

SUB-SYMBOLIC

Figure 1. The overall architecture for an integration of the different modules. Whereas the modules ORD and NSLD are adapting new ontological axioms to an existing ontology, the analogy engine AE computes analogical relations based on background knowledge provided by the other two modules. The control device CD is intended to choose answers from all three modules.

• The ontology rewriting device (ORD) is a knowledge base that is intended to resolve inconsistencies. The system is based on rewriting algorithms designed for several types of description logics [8,9]. The rewriting algorithms were applied to prototypical ontologies for resolving undergeneralization, overgeneralization, and polysemy problems in ontology design. Due to the limited expressive strength of description logics, it is currently not possible to code complex theories in ORD, e.g. theories that are based on full first–order logic. • The neuro–symbolic learning device (NSLD) consists of a feedforward neural network that takes as input first–order logical expressions that are transformed into a variable–free representation in a topos [10]. This input is used to learn an approximation of a logical theory, i.e. the input plus an approximation of all possible inferences is learned [11]. The approach was applied to simple and complex logical theories. An integration based on a non–trivial interaction of the seemingly rather incompatible modules can be achieved as follows: symbolic and sub–symbolic processes can be integrated, because NSLD is trained on symbolic data (i.e. on first–order logical expressions) and it is able to learn a model of a given logical theory. Although it is currently not possible to extract symbolic information directly from NSLD, a competition of ORD and NSLD can be implemented by querying both and evaluating their answers (governed by the Control Device CD). Additionally, both modules can directly interact with each other, because input from ORD can be used for training and querying NSLD. In a very similar way, the integration of AE and NSLD can be justified. Here, the situation is similar, because both modules are operating on a symbolic level (although the expressive

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture

215

strengths of the respectively underlying knowledge representation formalisms are different). Figure 1 depicts the overall architecture of the system.2 • The input may originate from various sources, e.g. from resources based on structured data, unstructured data, or semi–structured data. The input needs to be available in an appropriate (subset) of a first–order language L, in order to be in an appropriate format for the other modules. Therefore, ORD generates appropriate logical formulas from hypotheses. • An important aspect is the interaction of ORD and NSLD: on the one hand, ORD trains NSLD, on the other ORD queries NSLD. Although NSLD can only give an approximate answer in terms of a classification, this can improve the performance of ORD in time–critical situations. • With respect to the interaction of AE and ORD, ontological knowledge can naturally be used to constrain the computation of possible analogies [12]. Furthermore newly generated analogies can be used to update and therefore rewrite background knowledge [13]. • Similarly to the relation between ORD and NSLD, AE is used to train NSLD, whereas query answering can be performed in the other direction. • The control device CD is intended to implement a competition of the feedback of the three modules with respect to queries. Feedback may be in accordance to each other or not. In the second case, a ranking of the corresponding hypotheses is decided by CD (see below). I–Cog is based on some crucial assumptions: First, the modules are intended to cover a variety of different computing paradigms. For example, the analogy engine AE is covering several forms of classical and non–classical reasoning, like deductive reasoning, inductive reasoning, or vague reasoning. Furthermore, analogy making is a mechanism for getting new (generalized) concepts [14]. Second, all modules are learning devices in the sense that they are dynamically adapting to new information or results of internal processes. Third, the system is crucially hybrid due to the interaction between symbolic and neural modules.

2. Learning from Inconsistencies 2.1. General Remarks Human learning and memorizing differ in various aspects from classical machine learning algorithms. Whereas humans are able to learn from sparse data, machine learning algorithms usually need a large data sample, in order to produce reliable generalizations. Furthermore, learning new knowledge seems to be strongly based on creative problem solving. Additionally, human memory (i.e. learned knowledge) is not comparable with a fixed knowledge base, but is constantly reorganized. In the I–Cog architecture, we propose that learning is realized decentralized in each module namely by constantly reorganizing the memory content in AE, ORD, and NSLD. In other words, learning is a side– effect of other processes and not explicitly based on a single learning mechanism. 2 Whereas the three main modules of I–Cog are implemented and evaluated in several scenarios, the I–Cog architecture as a whole is currently not implemented yet.

216

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture

Inconsistencies are classically connected to logical theories: If for a given set of axioms Ŵ (relative to a given logical language L) a formula φ can be entailed (Ŵ | φ) and similarly ¬φ can be entailed (Ŵ | ¬φ), then the axiomatic system Ŵ is inconsistent. In this paper, we are using the term inconsistency rather loosely by discussing some special forms of inconsistent data, that are not necessarily equivalent to logical inconsistency in the sense above. Rather some forms could be better described by incoherent data or incompatible conceptualizations. Here are three simple examples: • Every analogy establishes a relation between two domains that may be similar in certain aspects, but usually are far from being coherent. For example, analogies in qualitative physics like “Current is the water in the electric circuit." or “Electrons are the planets of the atom." are simply statements that are semantically nonsense: A computation of the conventional meaning of these natural language sentences (relative to a conventional lexicon) would result in false, hence they are unsatisfiable and therefore contradictions. Nevertheless humans can make sense out of these sentences and particularly in teaching situations, students can learn new conceptualizations of a formerly unknown domain by such analogies. • In automatic ontology generation, it happens quite often that polysemy problems can occur due to the missing disambiguation of different word senses. Whether the word forest denotes a collection of plants or an abstract data structure, depends on the context in the text. These two interpretations of forest need to be described in the ontology by two different concepts with two different identifiers (e.g. ForestPlant and ForestStructure). • Concepts in ontology design are often overgeneralized. The most famous example is Tweety, the penguin that is a bird, but cannot fly: ∀x : Bir d(x) → Can Fly(x) ∀x :Pengui n(x) → Bir d(x) ∧ ¬Can Fly(x) Obviously, a contradiction can be derived because “Birds can fly" is too general. This inconsistency can be resolved by introducing a slightly more specific concept like FlyingBird (cf. Subsection 2.3). In the following three subsections, we will give some examples how learning from inconsistencies is realized in I–Cog. 2.2. Learning from Inconsistencies in Analogy Making Establishing an analogical relation between two different domains can be seen as the task to make incompatible conceptualizations compatible with each other: For example, in the analogy between the atom model and the solar system ("Electrons are the planets of the atom"), two incompatible domains need to be associated with each other. That means that an adaptation process is required, in order to resolve a seemingly incoherent association of information. We will see that not only on this level learning by analogies is based on inconsistencies, but also the analogical reasoning process itself is crucially driven by resolving inconsistencies, and adapting pieces of information. The framework we use for computing analogical relations is heuristic–driven theory projection (HDTP) [3]. HDTP represents the source and target domains by sets of first–order formulas. The corresponding source theory T h S and target theory T h T are then generalized using an extension of anti–unification [4]. Here are the key elements of HDTP:

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture Source T h S : Addition α1 α2

217

Target T h T : Multiplication

∀x : add(x, 0) = x ∀x∀y : add(x, s(y)) = s(add(x, y))

β1 β2

∀x : mult (x, s(0)) = x ∀x∀y : mult (x, s(y)) = add(x, mult (x, y))

Generalized Theory T h G γ1

∀x : Op1 (x, E) = x

∀x∀y : Op1 (x, s(y)) = Op2 (Op1 (x, y)) γ2 Table 1. A formalization of addition and multiplication by primitive recursion. The generalized theory T h G is a simplified but abstract formalization of primitive recursion.

• Two formulas p1 (a, b) and p2(a, c) can be anti–unified by P(a, X), with substitutions 1 = {P → p1 , X → b} and 2 = {P → p2 , X → c}. • A theorem prover allows the re–representation of formulas. • Whole theories can be generalized, not only single terms or formulas. • The process is governed by heuristics on various levels. The idea behind HDTP is to compute generalizations of (logical or arithmetical) formulas (axioms), in order to establish a generalization of the source and target domain. A heuristic used is the minimization of the complexity of substitutions for generalized pairs of axioms. The generalization process leads to anti–instances that are structural descriptions of the input theories of source and target. We want to exemplify HDTP using a simple example without specifying the formal details of the approach.3 Table 1 shows the standard axiomatization of addition and multiplication by primitive recursion (source and target). The generalized theory depicted in Table 1 introduces (existentially quantified) new variables for constants and operators. It is possible to gain the corresponding source theory T h S and target theory T h T from the generalized theory T h G by applying the following substitutions: 1 : 2 :

E → 0, Op1 → add, Op2 → s E → s(0), Op1 → mult, Op2 → λz.add(x, z)

The goal of computing an analogy between source and target is a generalized theory associating pairs of formulas from the source and the target. If the input is given as depicted in Table 1, HDTP starts with axiom β1 (heuristics based on complexity), searches a possible candidate in the set {α1 , α2 } for anti–unification and finds α1 as the best candidate (based on a further heuristics minimizing the substitution complexity). The anti– instance γ1 is straightforwardly constructed. β2 is chosen next and anti–unified with α2 in a very similar way. For a computation of the generalizations γ1 and γ2 no inconsistencies occur making backtracking unnecessary. The situation changes, if we change the conceptualization of the target domain slightly: Assume we have the axiom ∀x : mult (s(0), x) = x instead of β1 (permutation of arguments). Then, a simple anti–unification is not possible, and HDTP backtracks based on this inconsistency. Provided there is background knowledge available stating that ∀x : mult (s(0), x) = mult (x, s(0)) holds, then the internal theorem prover in HDTP re–represents the input β1 , in order to anti–unify β1 and α1 as above resulting in the generalized theory γ1 and γ2 . 3 This example is intended to make the mechanisms more transparent, not to give a thorough introduction into HDTP. The interested reader is referred to [3,15,5] for the theoretical background.

218

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture Source T h S : Addition α1 α2

∀x : add(0, x) = x ∀x∀y : add(s(y), x) = add(y, s(x))

Target T h T : Multiplication β1 β2

∀x : mult (0, x) = 0 ∀x∀y : mult (s(y), x) = add(x, mult (y, x))

Generalized Theory T h G γ1 ∀x : Op1 (E, x) = x Table 2. A formalization of addition and multiplication by a different recursive axiomatization.

Consider a slightly more complicated situation, where the conceptualization is given as depicted in Table 2. The two axioms systems of source and target cannot be anti– unified in a straightforward way. By using the theorem prover, we can derive the following fact (using axioms α1 and α2 of the source to derive a fact on the target): mult (s(0), x) = add(x, mult (0, x)) = add(x, 0) = · · · = add(0, x) = x Hence, we can derive β3 : ∀x : mult (s(0), x) = x and the anti–unifier γ1 can be established with the following substitutions: 1 : 2 :

E → 0, Op → add E → s(0), Op → mult

The established analogy expresses that addition corresponds to multiplication and that there are (different) unit elements for addition and multiplication. It is important to notice that the generation of abstract knowledge in form of a generalization of source and target is based on occurring inconsistencies between two seemingly incompatible axiomatizations. Based on the given axiomatization of Table 2 it is not possible to derive a general commutativity law ∀x∀y : Op(x, y) = Op(y, x). But it is possible to extend the coverage of the generalized theory by arbitrary instances of formulas Op(s n (0), s m (0)) = Op(s m (0), s n (0)) without changing the relevant substitutions. Notice that natural extensions of the sketched simple arithmetical theories are possible. Just to mention one case, if we extend elementary arithmetic on natural numbers to elementary arithmetic on rational numbers, additional laws on the source domain can be introduced. For example, the law α3 , that guarantees the existence of an inverse element, could be added: α3 : ∀x∃y: add(x, y) = 0. An analogical projection (transfer) of α3 to the target side would result in a law β3 : ∀x∃y: mult (x, y) = 1, which is obviously a contradiction to β1 , because one can deduce the inconsistency ∃y: mult (0, y) = 1. Clearly β3 does only hold for rational numbers unequal to 0. An appropriate restriction of the domain for which β3 holds can fix this problem.4 On the level of the generalized theory a new concept invertible element can be introduced. 2.3. Learning from Inconsistencies of Ontological Background Knowledge It is commonly assumed that background knowledge of human agents quite often contains incoherent information. Furthermore, human memory is not fixed, but rather con4 There are many non–trivial examples in mathematics, where some properties do only hold for a subset of objects: Consider, for example, quaternions H, a non–commutative extension of complex numbers, where only the basis quaternions i, j, k are non–commutative with i j = k = ji = −k, whereas the restrictions to real number a = a1 + 0i + 0 j + 0k and b = b1 + 0i + 0 j + 0k results in ab = ba.

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture

219

stantly updated in an incremental way. Humans can reorganize their memory content and have the ability to adapt incoherent information on–the–fly. From a more technical perspective this translates into an adaptive knowledge base for artificial systems. In the standard conception a knowledge base is considered to be static, although this is a rather counterintuitive assumption. In the I–Cog architecture, the memory module ORD is intended to be highly adaptive, extending incrementally the knowledge base constantly, and – if necessary – adapting it, provided inconsistencies occur. The resolution of inconsistencies is assessed by a rewriting device on knowledge formalized in description logics (DL), which are the current state–of–the–art standard for coding ontological knowledge. Ontologies usually contain a terminological component and an assertion component. A description logic terminology consists of a set of terminological axioms defining concepts either by formulas of the form ∀x : C(x) → D(x) (partial definition) or by ∀x : C(x) ↔ D(x) (total definition), where C is a concept name and D is a concept description.5 Additionally to the terminological component an assertion component contains information about the assignment of particular individuals to concepts and relations from the terminology. Axioms are interpreted model theoretically by an interpretation function mapping concept descriptions to subsets of the domain. A model of an ontology is an interpretation satisfying all axioms. An ontology is inconsistent if it does not have a model. There are several possibilities why inconsistencies can occur in ontologies. In [16], structural inconsistencies, usage–defined inconsistencies, and logical inconsistencies are distinguished. The last type of inconsistency – potentially caused by dynamic updates of the knowledge base – is of particular interest in our context. We mention some forms of occurring logical inconsistencies that can be handled with ORD. One aspect of logical inconsistency problems concerns polysemy: If an ontology is updated automatically, then different concepts may happen to have the same name. Suppose, the concept named tree is declared to be a subconcept both of plant and of data structure (where plant and data structure are disjoint concepts). Both of these two interpretations of tree are correct, but it is still necessary to describe two different concepts in the ontology with different identifiers (e.g. TreePlant, TreeStructure). Another important aspect of logical inconsistency problems concerns generalization mistakes and is strongly connected to non–monotonic reasoning, extensively discussed in the relevant AI literature. Example 1 Assume the following axioms are given: ∀x : Bir d(x) → Can Fly(x) ∀x : Canar y(x) → Bir d(x)

∀x : Can Fly(x) → Can Move(x) ∀x : Pengui n(x) → Bir d(x) ∧ ¬Can Fly(x)

In Example 1, the statement “birds can fly" is too general. If an exception occurs (penguin), the ontology becomes unsatisfiable, since penguin is declared to be a bird, but it cannot fly. In order to resolve the inconsistency, notice that the definition of the concept Bird is overgeneralized. Therefore we need to rewrite it. Nevertheless, we wish to keep as much information as possible in the ontology. Example 2 specifies a solution: 5 Compare [17] for an exhaustive definition of description logics.

220

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture

Example 2 Adapted ontology from Example 1: ∀x : Bir d(x) → Can Move(x) ∀x : Can Fly(x) → Can Move(x) ∀x : Canar y(x) → Flyi ng Bir d(x) ∀x : Pengui n(x) → Bir d(x) ∧ ¬Can Fly(x) ∀x : Flyi ng Bir d(x) → Bir d(x) ∧ Can Fly(x) In the definition of the concept Bird (subsuming the unsatisfiable concept Penguin), we want to keep a maximum of information not conflicting with the definition of Penguin. Conflicting information is moved to the definition of the new concept FlyingBird, which is declared to subsume all former subconcepts of Bird (such as Canary) except Penguin. An algorithmic solution to the problem is formally described in [8], [18], and [9]. In this framework, ontologies are extended with additional axioms conflicting with the original knowledge base, i.e. given a consistent ontology O (possibly empty) the procedure adds a new axiom A to O. If O + = O ∪ {A} is inconsistent, then the procedure tries to find a polysemy or an overgeneralization and repairs O + . First, problematic axioms that cause a contradiction are detected, then the type of the contradiction (polysemy or overgeneralization) are defined, and finally an algorithm repairs the terminology by rewriting parts of the axioms that are responsible for the contradiction. Detected polysemous concepts are renamed and overgeneralized concepts are split into more general and more specific parts. The sketched rewriting for a constant adaptation process of background knowledge is a first step towards a general theory of dynamification and adaptation of background knowledge. The framework has been developed primarily for text technological applications, but the approach can be extended to a wider range of applications.6 2.4. Inconsistencies in the Overall System As mentioned in the Introduction, a standard problem of hybrid architectures is the problem of how inconsistencies between potential outputs of the different modules can be resolved. Even in the case a competition mechanism is implemented between the modules, in order to accept or reject certain potential outputs of the respective modules, a control device needs to assess these outputs appropriately. A plausible way for an implementation is to use heuristics in the first place to resolve potential conflicts, and on top of it a learning device for updating these heuristics. In the I–Cog architecture, the control device CD is the module that arbitrates between the main modules. CD needs to assess possible answers of the three main modules and needs to implement a competition process. First, we exemplify possible situations with respect to ORD and NSLD. Concerning underdetermined situations, where ORD is not able to answer queries, NSLD can be used for answering a query.7 In such cases, the usage of NSLD is clearly preferred by the heuristic. On the other hand, if ORD contains 6 The crucial algorithms for resolving overgeneralization, undergeneralization, and polysemy problems, are implemented and prototypically tested in example domains [9]. 7 Simple examples in which NSLD can answer queries in underdetermined situations can be found in [11]: If certain objects of the domain are not sufficiently defined or even not defined at all, a knowledge base or a theorem prover cannot provide any interesting information about such objects. A neural reasoning device like NSLD can, because it can give answers to any query.

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture

221

(or can prove) a particular fact, for example, that a certain subsumption relation between two concepts A and B holds, then this result should be tentatively preferred by CD in comparison to the output of NSLD. In cases, where time–critical reactions are necessary and ORD is not able to compute an answer in time, the natural heuristic would be to use NSLD instead. Finally, it could happen that the answers of ORD and NSLD are contradicting each other. In this case, CD cannot base the decision on an a priori heuristic. Possible solutions may be either the training of NSLD by the answer of ORD or the implementation of a reinforcement learning mechanism on CD itself. The latter can be used to learn preferred choices of the knowledge modules involved. In both cases, occurring inconsistencies function as triggers for learning. Very similarly to the interaction between ORD and NSLD we sketch some ideas controlling the interaction between AE and NSLD. If the reasoning device AE can prove a fact or can successfully establish an analogical relation between a source and a target domain, AE is clearly preferred in comparison to NSLD. In time–critical or underdetermined situations, the answer of NSLD is heuristically preferred. Finally, if contradictions between AE and NSLD occur, the solutions mentioned above can be again applied, namely the implementation of a reinforcement learning mechanism or the the training of NSLD by AE.

3. Related Work Analogical reasoning was discussed in many domains like proportional analogies in string domains [19] and analogies between geometric figures [20]. Further discussions were based on the relation between analogies and metaphors [14] and on analogical problem solving [21]. Concerning underlying methods for modeling analogies algebraic [14], graph–based [22], and similarity–based approaches [23] can be found. Rewriting systems for knowledge representations are described in [16]. A collection of approaches aiming at resolving inconsistencies in knowledge representation is related to non–monotonicity. Examples are extensions by default sets [24] or by belief–revision processes [25]. A family of approaches is based on tracing techniques for detecting a set of axioms that are responsible for particular ontological contradictions [26], [27]. There is a number of attempts to resolve the gap between symbolic and subsymbolic computations. We just mention some newer approaches: An example to solve the neuro– symbolic integration problem is described in [28] in which a logical deduction operator is approximated by a neural network. Another approach is [29], where category theoretic methods are used for neural constructions. In [30], tractable fragments of predicate logic are learned by connectionist networks. Recently, some endeavor has been invested to approximate a solution to human– level intelligence. [31] proposes a so–called cognitive substrate in order to reduce higher cognition and the profusion of knowledge to a basis of low computational complexity. Further approaches that resemble the integration idea presented here follow the tradition of cognitive architectures. Examples are the hybrid AMBR/DUAL model [32], which is modeling neuro–symbolic processing and analogical reasoning, the ICARUS architecture [33], which is focusing primarily on learning, or the NARS architecture [34], which is intended for integrating many different types of reasoning and representation formats. Nevertheless the set–up for and the theories behind the involved modules in I–Cog, as

222

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture

well as the integration idea of a constant reorganization based on resolving inconsistencies in I–Cog fundamentally differ from the mentioned approaches.

4. Conclusions Standardly it is assumed that inconsistencies in hybrid systems are problematic and a source for difficulties in applications. We used the I–Cog architecture to support the claim that on the contrary, inconsistencies in hybrid cognitive systems should be considered as a cue for (cognitive) learning, and not only as a problem for integrating incoherent information. In I–Cog, all main modules – the analogy engine, the ontology rewriting device, and the neuro–symbolic integration device – are either able to learn from inconsistent information or can deal with vague and noisy input, quite similar to abilities shown by natural agents. In other words, learning should not be realized in a separated module, but should be an integral part of every computation device. Furthermore, inconsistencies do not only occur in the interaction between modules of an architecture, but are an important source for guiding computations and learning on several levels: In reasoning modules for computing various types of inferences, in updates of the knowledge base for adapting background knowledge to new input, and in the interaction of different modules for optimal results in a particular application.

References [1]

[2]

[3] [4] [5]

[6]

[7]

[8]

[9]

[10]

K.–U. Kühnberger, T. Wandmacher, A. Schwering, E. Ovchinnikova, U. Krumnack, H. Gust and P. Geibel, I–Cog: A Computational Framework for Integrated Cognition of Higher Cognitive Abilities, Proceedings of 6th Mexican International Conference on Artificial Intelligence, LNAI 4827, Springer (2007) pp. 203–214. K.–U. Kühnberger, T. Wandmacher, E. Ovchinnikova, U. Krumnack, H. Gust and P. Geibel, Modeling Human–Level Intelligence by Integrated Cognition in a Hybrid Architecture, in P. Hitzler, T. Roth– Berghofer, S. Rudolph (eds): Foundations of Artificial Intelligence (FAInt–07), Workshop at KI 2007, CEUR–WS 277 (2007), 1–15. H. Gust, K.–U. Kühnberger and U. Schmid, Metaphors and Heuristic–Driven Theory Projection (HDTP), Theoretical Computer Science 354 (2006) 98–117. G. Plotkin, A note of inductive generalization, Machine Intelligence 5 (1970), 153–163. U. Krumnack, A. Schwering, H. Gust, K.–U. Kühnberger, Restricted Higher–Order Anti–Unification for Analogy Making, in Proceedings of Twenties Australian Joint Conference on Artificial Intelligence, LNAI 4830, Springer (2007) pp. 273–282. H. Gust, K.–U. Kühnberger and U. Schmid, Solving Predictive Analogies with Anti–Unification, in: P. Slezak (eds.): Proceedings of the Joint International Conference on Cognitive Science, 2003, pp. 145– 150. A. Schwering, U. Krumnack, K.–U. Kühnberger and H. Gust, Using Gestalt Principles to Compute Analogies of Geometric Figures, in: D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th Annual Conference of the Cognitive Science Society, Cognitive Science Society, 2007, pp. 1485–1490. E. Ovchinnikova and K.–U. Kühnberger, Adaptive ALE-TBox for Extending Terminological Knowledge. In A. Sattar, B. H. Kang (eds.): Proceedings of the 19th ACS Australian Joint Conference on Artificial Intelligence, LNAI 4304, Springer, 2006, pp. 1111–1115. E. Ovchinnikova, T. Wandmacher and K.–U. Kühnberger, Solving Terminological Inconsistency Problems in Ontology Design. International Journal of Interoperability in Business Information Systems, 2(1) (2007), 65–80. R. Goldblatt, Topoi: The Categorial Analysis of Logic. Studies in Logic and the Foundations of Mathematics, 98, North–Holland, Amsterdam (1979).

K.-U. Kühnberger et al. / Learning from Inconsistencies in an Integrated Cognitive Architecture [11]

223

H. Gust, K.–U. Kühnberger and P. Geibel, Learning Models of Predicate Logical Theories with Neural Networks based on Topos Theory. In P. Hitzler and B. Hammer (eds.): Perspectives of Neuro–Symbolic Integration, Studies in Computational Intelligence (SCI) 77, Springer, (2007) pp. 233–264. [12] H. Gust, K.–U. Kühnberger and U. Schmid, Ontological Aspects of Computing Analogies. Proceedings of the Sixth International Conference on Cognitive Modeling, Mahwah, NJ: Lawrence Erlbaum, (2004) pp. 350–351. [13] H. Gust, K.–U. Kühnberger and U. Schmid, Ontologies as a Cue for the Metaphorical Meaning of Technical Concepts, in A. Schalley, D. Khlentzos (eds.): Mental States: Evolution, Function, Nature, Volume I, John Benjamins Publishing Company, Amsterdam, Philadelphia, (2007) pp. 191–212. [14] B. Indurkhya, Metaphor and Cognition, Dordrecht, the Netherlands, Kluver (1992). [15] H. Gust, U. Krumnack, K.–U. Kühnberger and A. Schwering, An Approach to the Semantics of Analogical Relations, in S. Vosniadou, D. Kayser, A. Protopapas (eds.): Proceedings of EuroCogSci 2007, Lawrence Erlbaum, pp. 640–645. [16] P. Haase, F. van Harmelen, Z. Huang, H. Stuckenschmidt and Y. Sure, A framework for handling inconsistency in changing ontologies. Proc.of the Fourth Internation Semantic Web Conference, LNCS, Springer (2005). [17] F. Baader, D. Calvanese, D. McGuinness, D. Nardi and P. Patel–Schneider (eds.), Description Logic Handbook: Theory, Implementation and Applications. Cambridge University Press (2003). [18] E. Ovchinnikova and K.–U. Kühnberger, Automatic Ontology Extension: Resolving Inconsistencies, GLDV Journal for Computational Linguistics and Language Technology (to appear). [19] D. Hofstadter and The Fluid Analogies Research Group, Fluid concepts and creative analogies. New York: Basic Books (1995). [20] M. Dastani, Languages of Perception, ILLC Disseration Series 1998–05, 1998, http://www.illc.uva.nl/Publications/Dissertations/DS-1998-05.text.ps.gz. [21] J. Anderson and R. Thompson, Use of Analogy in a Production System Architecture, in Vosniadou, Ortony (eds): Similarity and analogical reasoning, Cambridge (1989) 267–297. [22] B. Falkenhainer, K. Forbus and D. Gentner, The structure–mapping engine: Algorithm and example, Artificial Intelligence 41 (1989) 1–63. [23] D. Gentner, The Mechanisms of Analogical Learning, in: S. Vosniadou & A. Ortony (editors): Similarity and Analogical Reasoning, New York, Cambridge University Press. [24] S. Heymans and D. Vermeir, A Defeasible Ontology Language, In Meersman, R. et al. (eds): On the Move to Meaningful Internet Systems, 2002 – Confederated International Conferences: CoopIS, DOA, and ODBASE 2002, Springer, pp. 1033–1046. [25] G. Flouris, D. Plexousakis and G. Antoniou, Updating DLs Using the AGM Theory: A Preliminary Study, in: Description Logics, 2006. [26] F. Baader and B. Hollunder, Embedding defaults into terminological knowledge representation formalisms. J. Autom. Reasoning, 14(1) (1995) 149–180. [27] A. Kalyanpur, Debugging and Repair of OWL Ontologies. Ph.D. Dissertation, (2006). [28] S. Bader, P. Hitzler, S. Hölldobler and A. Witzel, A Fully Connectionist Model Generator for Covered First–Order Logic Programs. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (2007), pp. 666–671. [29] M. Healy and T. Caudell, Neural Networks, Knowledge and Cognition: A Mathematical Semantic Model Based upon Category Theory. University of New Mexico, (2004), EECE-TR-04-020. [30] A. D’Avila Garcez, K. Broda and D. Gabbay, Neural–Symbolic Learning Systems. Foundations and Applications. Springer (2002). [31] N. Cassimatis, A Cognitive Substrate for Achieving Human–Level Intelligence, AI Magazine 27(2) (2006) 45–56. [32] B. Kokinov and A. Petrov, Integrating Memory and Reasoning in Analogy–Making: The AMBR Model, in D. Gentner, K. Holyoak, B. Kokinov (eds.): The Analogical Mind. Perspectives from Cognitive Science, Cambridge Mass. (2001). [33] P. Langley, Cognitive Architectures and General Intelligent Systems, AI Magazine, 27(2) (2006), 33–44. [34] P. Wang, Rigid Flexibility: The Logic of Intelligence, Springer, 2006.

224

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

Extending the Soar Cognitive Architecture John E. Laird Division of Computer Science and Engineering, University of Michigan

Abstract. One approach in pursuit of general intelligent agents has been to concentrate on the underlying cognitive architecture, of which Soar is a prime example. In the past, Soar has relied on a minimal number of architectural modules together with purely symbolic representations of knowledge. This paper presents the cognitive architecture approach to general intelligence and the traditional, symbolic Soar architecture. This is followed by major additions to Soar: nonsymbolic representations, new learning mechanisms, and long-term memories.

Keywords. Soar, cognitive architecture, reinforcement learning, episodic memory, semantic memory, clustering, mental imagery, emotion.

Introduction Through the years, there has been substantial evolution and refinement of the Soar architecture [1], with eight major versions between 1982 and 2007. During this evolution, the basic approach of pure symbolic processing, with all long-term knowledge being represented as production rules, was maintained and Soar proved to be a general and flexible architecture for research in cognitive modeling across a wide variety of behavioral and learning phenomena [2]. Soar also proved to be useful for creating knowledge-rich agents that could generate diverse, intelligent behavior in complex, dynamic environments [3, 4]. In spite of these successes, it became clear that Soar was missing some important capabilities that we take for granted in humans, many of which have also been ignored by the larger cognitive architecture community. In response, we have made substantial extensions to Soar, adding new learning mechanisms and long-term memories as well as new forms of non-symbolic processing. The motivation for these extensions is to increase the functionality of Soar for creating artificial general intelligent systems, but we also expect that these changes this will significantly expand the breadth of human behavior that can be modeled using Soar. Before presenting these extensions, we start with our underlying methodology for understanding and creating artificial generally intelligent systems: cognitive architecture. We then present the traditional version Soar, without the extensions, followed by the extended version of Soar. We conclude with an analysis of this new version of Soar and discussion of future directions.

J.E. Laird / Extending the Soar Cognitive Architecture

225

1. Cognitive Architecture During the last twenty years, the field of AI has successfully pursued specialized algorithms for specific problems. What distinguishes generally intelligent entities is their ability to solve not just a single problem using a specific method, but the ability to pursue a wide variety of tasks, including novel tasks, using large bodies of diverse knowledge, acquired through experience, in complex, dynamic environments. This leads us to study the fixed infrastructure that supports the acquisition and use of knowledge: the cognitive architecture. A cognitive architecture consists of: • memories for storing knowledge • processing units that extract, select, combine, and store knowledge • languages for representing the knowledge that is stored and processed Cognitive architectures distinguish between knowledge that is acquired over time and the fixed cognitive architecture that is common across all tasks. One of the most difficult challenges in cognitive architecture design is to create sufficient structure to support initial coherent and purposeful behavior, while at the same time providing sufficient flexibility so that an agent can adapt (via learning) to the specifics of its tasks and environment. Cognitive architectures must embody strong hypotheses about the building blocks of cognition that are shared by all tasks, and how different types of knowledge are learned, encoded, and used, making a cognitive architecture a software implementation of a general theory of intelligence. The hypothesis behind cognitive architectures such as Soar and ACT-R [5] is that there are useful abstractions and regularities above the level of neurally-based theories. This hypothesis plays out both in longer time scales of modeled behavior and in the symbolic representations of knowledge about the world. A related hypothesis is that the structures and discoveries of the symbolic architectures will be reflected in the neurally-based architectures. This is the approach taken in Neuro-Soar [6] and the ACT-R/Liebre [7] hybrid architectures. Probably the most interesting question is whether the extra detail of the neurally-based architectures (and the concurrent additional processing requirements) is necessary to achieve generally intelligent agents, or whether the more abstract architectures sufficiently capture the structures and regularities required for intelligence.

2. Traditional Soar As shown in Figure 1, up until five years ago, Soar has consisted of a single long-term memory, which is encoded as production rules, and a single short-term memory, which is encoded as a symbolic graph structure so that objects can be represented with properties and relations. Symbolic short-term memory holds the agent’s assessment of the current situation derived from perception and via retrieval of knowledge from its long-term memory. Action in an environment occurs through creation of motor commands in a buffer in short-term memory. The decision procedure selects operators and detects impasses, both of which are described below.

226

J.E. Laird / Extending the Soar Cognitive Architecture

Symbolic Long-Term Memory Procedural

Chunking Decision Procedure

Symbolic Short-Term Memory

Perception

Action Body

Figure 1: Structure of Soar 9

At the lowest level, Soar’s processing consists of matching and firing rules. Rules provide a flexible, context-dependent representation of knowledge, with their conditions matching the current situation and their actions retrieving information relevant to the current situation. Most rule-based systems choose a single rule to fire at a given time, and this serves as the locus of choice in the system – where one action is selected instead of another. However, there is only limited knowledge available to choose between rules, namely the conditions of the rules, the data matched by the rules, and possibly meta-data, such as a numeric score, associated with the rules. There is no ability to use additional context-dependent knowledge to influence the decision. Soar allows additional knowledge to influence a decision by introducing operators as the locus for choice and using rules to propose, evaluate, and apply operators. Rules act as an associative-memory that retrieves information relevant to the current situation, so there is no need to select between them and thus, in Soar, rules fire in parallel. The concept of operator is common in AI, but usually involves a monolithic data structure containing the operator’s preconditions and actions. However, in Soar, the definition of an operator is distributed across multiple rules. Thus, in Soar, there are rules that propose operators that create a data structure in working memory representing the operator and an acceptable preference so that the operator can be considered for selection. There are also rules that evaluate operators and create other types of preferences that prefer one operator to another or provide some indication of the utility of the operator for the current situation. Finally, there are rules that apply the operator by making changes to working memory that reflect the actions of the operator. These changes may be purely internal or may initiate external actions in the environment. This approach supports a flexible representation of knowledge about operators – there can be many reasons for proposing, selecting, and/or applying an operator – some that are very specific and others that are quite general. This representation also makes it possible to incrementally build up operator knowledge structures, so that the definition of an operator can change over time as new knowledge is learned for proposal, selection, and application [8]. To support the selection and application of operators and to interface to external environments, Soar has the processing cycle shown in Figure 2.

227

J.E. Laird / Extending the Soar Cognitive Architecture

Input

Elaboration Operator Proposal Operator Evaluation

Operator Selection

Operator Application

Output

Figure 2: Soar’s Processing Cycle

1. 2.

3. 4.

5.

6.

7.

Input. Changes to perception are processed and sent to short-term memory. Elaboration. Rules compute entailments of short-term memory. For example, a rule might test if the goal is to grasp an object, the object’s distance, and the agent’s reach, and then create a structure signifying whether the object is within reach. Operator Proposal. Rules propose operators that are appropriate to the current situation based on features of the situation tested in the condition of the rules. Operator Evaluation. Rules create preferences for which of the proposed operators should be preferred, based on the current situation and goal. The preferences can be symbolic (A is better than B), or numeric (the estimated utility of A is .73). Operator Selection. A fixed decision procedure combines the generated preferences and selects the current operator. If the preferences are insufficient for making a decision, an impasse arises and Soar automatically creates a substate in which the goal is to resolve that impasse. In the substate, Soar recursively uses the same processing cycle to select and apply operators, leading to automatic, reactive meta-reasoning. The impasses and resulting substates provide a mechanism for Soar to deliberately perform any of the functions (elaboration, proposal, evaluation, application) that are performed automatically/reactively with rules. Operator Application. The actions of an operator are performed by rules that match the current situation and the current operator structure. Multiple rules can fire in parallel and in sequence providing a flexible and expressive means for encoding operator actions. If there is insufficient application knowledge, an impasse arises with a substate. This leads to dynamic task decomposition, where a complex operator is performed recursively by simpler operators. Output. Any output commands are passed on to the motor system.

Soar’s procedural knowledge, decision-making, and subgoaling provide a strong foundation on to which to build. It has a shared short-term memory where knowledge from perception and long-term memory are combined to provide a unified representation of the current situation. It has a lean decision procedure that supports context-dependent reactive behavior, but also supports automatic impasse-driven subgoals and meta-reasoning. Chunking is Soar’s learning mechanism that converts the results of problem solving in subgoals into rules – compiling knowledge and behavior from deliberate to reactive. Although chunking is a simple mechanism, it is extremely general and can learn all the types knowledge encoded in rules [9]. 3. Extended Soar In extending Soar, we had two goals: 1. Retain the strengths of the original Soar: a flexible model of control and meta-reasoning along with the inherent ability to support reactive and deliberative behavior and the automatic conversion from deliberate to reactive behavior via chunking. 2. Expand the types of knowledge Soar could represent, reason with, and learn, inspired by human capabilities, but with the primary goal of additional functionality. The extensions fall into two, partially overlapping categories:

228

J.E. Laird / Extending the Soar Cognitive Architecture

Symbolic Long-Term Memories Procedural

Reinforcement Chunking Learning

Semantic Learning

Episodic Learning

Symbolic Short-Term Memory

Decision Procedure

Appraisal Detector

Episodic

Semantic

+ Activation

Clustering Perception

LT Visual Memory

ST Visual Imagery

Action

Body

Figure 3: Soar 9

new non-symbolic representations of knowledge along with associated processing and memory modules, and new learning and memory modules that capture knowledge that is cumbersome to learn and encode in rules. Figure 3 shows the structure of Soar, version 9. All of the new components have been built, integrated, and run with the traditional Soar components; however, as of yet there is not a single unified system that has all the components running at once. The major additions include: working memory activation, which provides meta-information about the recency and usefulness of working memory elements; reinforcement learning; which tunes the numeric preferences of operator selection rules; the appraisal detector, which generates emotions, feelings, and an internal reward signal for reinforcement learning; semantic memory, which contains symbolic structures representing facts; episodic memory; which contains temporally ordered “snapshots” of working memory; a set of processes and memories to support visual imagery, which includes depictive representations in which spatial information is inherent to the representation; and clustering, which dynamically creates new concepts and symbols. Soar’s processing cycle is still driven by procedural knowledge encoded as production rules. The new components influence decision making indirectly by retrieving or creating structures in symbolic working memory that cause rules to match and fire. In the remainder of this section, we will give descriptions of these new components and discuss briefly their value and why their functionality would be very difficult to achieve by the existing mechanisms.

J.E. Laird / Extending the Soar Cognitive Architecture

229

3.1. Working Memory Activation Inspired by ACT-R [5], we added activation to Soar’s working memory [10, 11]. Activation provides meta-information in terms of the recency of a working memory element and its relevance, which is computed based on when the element matched rules that fired. This information is not used to determine which rules to fire, as Soar fires all rules that match, but it is stored as part of episodic memories, biasing their retrieval so that the episode retrieved is the most relevant to the current situation. Empirical results verify that working memory activation significantly improves episodic memory retrieval [12]. In the future, we expect that working memory activation will be used in semantic memory retrieval and emotion. Working memory activation requires architectural changes because it must access and maintain information that is only available to the architecture (when working memory elements are created and matched). 3.2. Reinforcement Learning Reinforcement learning (RL) involves adjusting the selection of actions in an attempt to maximize reward. In early versions of Soar, all preferences for selecting operators were symbolic, so there was no way to represent or adjust such knowledge; however, we recently added numeric preferences, which specify the expected value of an operator for the current state [13]. During operator selection, all numeric preferences for an operator are combined, and an epsilon-greedy algorithm is used to select the next operator. This makes RL in Soar straightforward – it adjusts the actions of rules that create numeric preferences for selected operators [14]. Thus, after an operator applies, all of the rules that created numeric preferences for that operator are updated based on any new reward and the expected future reward, which is simply the summed numeric value of the numeric preferences for the next selected operator. RL in Soar applies across all goals, including impasse-generated subgoals. One intriguing aspect of RL in Soar is that the mapping from situation and operator to expected reward (the valuefunction) is represented as collections of rules. Only those rules that match the current situation participate in the selection of an operator, and there can be many rules contributing estimates of future reward for a single operator. This representation supports varying degrees of coverage and hierarchical representations, which can greatly speed up learning [15]. RL would be very difficult to implement in Soar using only chunking. RL applies to every operator selection, on every decision, even when there is no impasse, while chunking only learns rules through impasses. In addition, RL modifies existing rules by changing the values of numeric preferences, while chunking only adds new rules. In fact, RL and chunking are quite complementary because when there are no selection rules, an impasse arises and problem solving in a subgoal can generate initial preferences for the tied operators. Chunking then creates rules to generate these initial preferences in the future, and RL then tunes the values as experience accumulates. 3.3. Emotion The functional and computational role of emotion is open to debate; however, in the last twenty years there has been substantial research on appraisal theories of emotion [16]. These theories propose that an agent continually evaluates a situation and that evaluation leads to emotion. The evaluation is hypothesized to take place along

230

J.E. Laird / Extending the Soar Cognitive Architecture

multiple dimensions, such as goal relevance (is this situation important to my goals?), goal conduciveness (is this situation good or bad for my goals?), causality (who caused the situation?), control (can I change the situation?), and so on. These dimensions are exactly what an intelligent agent needs to compute as it pursues its goals while interacting with an environment. Thus, we have created a computational implementation of a specific appraisal theory [17] in Soar, represented by the appraisal detector in Figure 3. In Soar, appraisals lead to emotions, emotions influence mood, and mood and emotion determine feelings [18]. Individual appraisals produce either categorical or numeric values, which combine to form an intensity of the current feeling. This intensity becomes the intrinsic reward [19] for reinforcement learning, which significantly speeds learning [20]. A major goal of our future work is to explore how emotion, mood, and feeling can be used productively with other modules (such as retrieval from long-term memory and decision making), as well as in interacting with other agents. It is possible to implement similar theories of emotion in Soar without modifying the architecture [21]; however, in these approaches, Soar is used more as a programming language than as a cognitive architecture – all of the important functionality of the system comes from procedural knowledge encoded in Soar as opposed to the structure of the architecture. This makes it impossible for emotions, mood, and feelings to directly influence other architectural modules, which we expect is critical to many aspects of emotion, mood, and feelings. 3.4. Semantic Memory In addition to procedural knowledge, which is encoded as rules in Soar, there is declarative knowledge, which can be split into things that are known, such as facts, and things that are remembered, such as episodic experiences. Semantic learning and memory provides the ability to store and retrieve declarative facts about the world, such as tables have legs, dogs are animals, and Ann Arbor is in Michigan. This capability has been central to ACT-R’s ability to model a wide variety of human data and adding it to Soar should enhance our ability to create agents that reason and use general knowledge about the world. In Soar, semantic memory is built up from structures that occur in working memory [22]. A structure from semantic memory is retrieved by creating a cue in a special buffer in working memory. The cue is then used to search for the best partial match in semantic memory, which is then retrieved into working memory. There has been a significant body of research on acquiring semantic knowledge through Soar’s chunking mechanism, under the label of data chunking [23]. Although it is possible, it is not easy. Data chunking requires pre-existing knowledge about the possible structures that can exist in working memory, and the agent has to interrupt its current task processing to deliberately force an impasse in which chunking can learn the appropriate rules. Moreover, because the knowledge is encoded in rules, retrieval requires an exact match of the cue, limiting the generality of what is learned. These factors made it difficult to use data chunking in new domains, begging the question as to how it would naturally arise in a generally intelligent agent.

J.E. Laird / Extending the Soar Cognitive Architecture

231

3.5. Episodic Memory In contrast to semantic memory, which contains knowledge independent of when and where it was learned, episodic memory contains memories of what was experienced over time [24]. In Soar, episodic memory includes specific instances of the structures that occur in working memory at the same time, providing the ability to remember the context of past experiences as well as the temporal relationships between experiences [12]. An episode is retrieved by the deliberate creation of a cue, which is a partial specification of working memory in a special buffer. Once a cue is created, the best partial match is found (biased by recency and working memory activation) and retrieved into a separate working memory buffer (to avoid confusion between a memory and the current situation). The next episode can also be retrieved, providing the ability to replay an experience as a sequence of retrieved episodes. Although similar mechanisms have been studied in case-based reasoning, episodic memory is distinguished by the fact that it is task-independent and thus available for every problem, providing a memory of experience not available from other mechanisms. Episodic learning is so simple that it is often dismissed in AI as not worthy of study. Although simple, one has only to imagine what life is like for amnesiacs to appreciate its importance for general intelligence [25]. We have demonstrated that when episodic memory is embedded in Soar, it enables many advanced cognitive capabilities such as internal simulation and prediction, learning action models, and retrospective reasoning and learning. Episodic memory would be even more difficult to implement using chunking than semantic memory because it requires capturing a snapshot of working memory and using working memory activation to bias partial matching for retrieval. 3.6. Visual Imagery All of the previous extensions depend on Soar’s existing symbolic short-term memory to represent the agent’s understanding of the current situation and with good reason. The generality and power of symbolic representations and processing are unmatched and our ability to compose symbolic structures is a hallmark of human-level intelligence. However, for some constrained forms of processing, other representations can be much more efficient. One compelling example is visual imagery [26], which is useful for visual-feature and visual-spatial reasoning. We have added a set of modules to Soar that support visual imagery [27], including a short-term memory where images are constructed and manipulated; a long-term memory that contains images that can be retrieved into the short-term memory; processes that manipulate images in short-term memory, and processes that create symbolic structures from the visual images. Although not shown, these extensions support both a depictive representation in which space is inherent to the representation, as well as an intermediate, quantitative representation that combines symbolic and numeric representations. Visual imagery is controlled by the symbolic system, which issues commands to construct, manipulate, and examine visual images. With the addition of visual imagery, we have demonstrated that it is possible to solve spatial reasoning problems orders of magnitude faster than without it, and using significantly less procedural knowledge. Visual imagery also enables processing that is not possible with only symbolic reasoning, such as determining which letters in the alphabet are symmetric along the vertical axis (A, H, I, M, O, T, U, V, W, X, Y).

232

J.E. Laird / Extending the Soar Cognitive Architecture

3.7. Clustering One additional capability that has been missing from Soar is a mechanism that detects statistical regularities in the stream of experiences and automatically creates new symbolic structures that represent those regularities, providing a mechanism for automatically generating new symbols and thus concepts that can be used to classify perception. Few existing cognitive architectures have the capability to create new symbolic structures – they are a prisoner of the original encodings provided by a human programmer. To remedy this shortcoming, we have added a clustering module to Soar that is based on research by Richard Granger [28]. The underlying algorithms are derived from the processing of thalamocortical loops in the brain, where it appears there is clustering and successive sub-clustering of inputs using winner-take-all circuits. Although we do not yet have a general implementation of clustering for all types of perception in Soar, we have used clustering to create new symbolic structures that enrich that state representation and speed reinforcement learning. 3.8. Integration of New Modules in Soar As mentioned earlier, the processing cycle of Soar did not require any changes to accommodate the new modules beyond invoking them at appropriate times. Working memory activation is updated when rules fire, while reinforcement learning is invoked when a new operator is selected and clustering is invoked during input processing. The retrieval for semantic and episodic memories is performed during the input phase, so that the results are available during the next processing cycle. The visual imagery system is treated as if it was an external module, with commands to the system creating by rules and then executed during the output phase, with new interpretations of the images created during the input phase. Although the modules are invoked at specific points in the processing cycle, they could run asynchronously. For example, when a cue is created for episodic memory, the retrieval processes could run in parallel with rule matching and firing. We have maintained the more rigid approach because it simplifies development and debugging, and because there was no computational advantage to having asynchronous threads before the advent of multi-core processors. Although knowledge encoded in rules still controls behavior through the selection and application of operators, the new modules influence behavior through the working memory structures they create. For example, if a list of instructions is stored in semantic memory, with the appropriate rules, they can be retrieved, and lead to the selection of operators that interpret and execute them. Reactivity is maintained by rules that propose and prefer operators to process unexpected external events.

4. Discussion 4.1. Non-symbolic processing in Soar One of the major changes in extending Soar has been the introduction of non-symbolic processing. In Soar, clustering is subsymbolic, where non-symbolic perceptual structures are combined together to create symbols. All the other non-symbolic processing is co-symbolic – it either controls symbolic processing (similar to ACT-R’s use of non-symbolic processing), or in the case of visual imagery, provides an

J.E. Laird / Extending the Soar Cognitive Architecture

233

alternative to symbolic processing by providing a representational media for reasoning about the world. In contrast, Clarion has extensive subsymbolic processing [29] where it supports neural networks processing for some of its reasoning in addition to symbolic processing. Below is a chart of the different functional uses of non-symbolic processing in Soar. Non-symbolic Processing Numeric Preferences Reinforcement Learning Working memory activation Visual Imagery Appraisals: Emotions & Feelings Clustering

Function Control operator selection Learn operator selection control knowledge Aid long-term memory retrieval Represent images and spatial data for reasoning Summarize intrinsic value of situation – aid RL Create symbols that capture statistical regularities

4.2. Analysis of Memories and Learning in a Cognitive Architecture The second major change is the addition of new memories and learning systems. Below we summarize the major dimensions of variations in the memory and learning systems: Memory/Learning System Chunking Clustering Semantic Memory Episodic Memory Reinforcement Learning Image Memory

Source of Knowledge Traces of rule firings in subgoals Perception Working memory existence Working memory co-occurrence Reward and numeric preferences Image short-term memory

Representation of knowledge Rules Classification networks Mirror of working memory object structures Episodes: Snapshots of working memory Numeric preferences Image

Retrieval of knowledge Exact match of rule conditions, retrieve actions Winner take all Partial match, retrieve object Partial match, retrieve episode Exact match of rule conditions, retrieve preference Deliberate recall based on symbolic referent

4.3. Future work The addition of these new modules only scratches the surface of the potential research, with explorations of the interactions and synergies of these components being the next order of business. Some of the interactions are obvious, such as using RL to learn the best cues for retrieving knowledge from episodic memory, or using episodic memory for retrospective learning to train RL on important experiences. Other interactions require deeper integrations, such as how emotion and visual imagery are captured and used in episodic memory, how knowledge might move between episodic and semantic memories, how emotions and feelings influence decision making, or how emotion and working memory activation might influence storage and retrieval of knowledge in episodic and semantic memory. Finally, we need to study the impact of these extensions on Soar’s status as a unified theory of cognition [30].

234

J.E. Laird / Extending the Soar Cognitive Architecture

4.4. Final thoughts So far, the explorations in psychology and AI through the space of cognitive architectures have been extremely fruitful. Research on ACT-R has led to comprehensive computational theories of a wide variety of human phenomena, including brain activity [5], while research on EPIC has made great strides in modeling the detailed interactions of cognition with perception and action [31]. However, these architectures and others have ignored many of the cognitive capabilities we are now studying in Soar (episodic memory, emotion, visual imagery), and it has taken us close to twenty years to study them. Why is this? The simple answer is that our architectures are prisoners of the tasks we study. To date, most of the research on cognitive architecture has arisen from studying short-term tasks – similar to those that arise in a psychology laboratory setting where there is minimal emotional engagement and little need for personal history. The challenge for the future is to develop general artificial systems that can pursue a wide variety of tasks (including novel ones) over long time scales, with extensive experiential learning in dynamic, complex environments. As is evidenced from this paper, my belief is that research in cognitive architecture provides the building blocks to meet this challenge. Acknowledgments The author wishes to thank the current and former graduate students and research programmers who have contributed to the extensions to Soar described in this paper: Ron Chong, Karen Coulter, Michael James, Scott Lathrop, Robert Marinier, Shelley Nason, Andrew Nuxoll, Jonathan Voigt, Yongjia Wang, and Samuel Wintermute. The author acknowledges the funding support of the National Science Foundation under Grant No. 0413013 and the DARPA “Biologically Inspired Cognitive Architecture” program under the Air Force Research Laboratory “Extending the Soar Cognitive Architecture” project award number FA8650-05-C-7253. References [1] Laird, J. E., & Rosenbloom, P.S. (1996) The evolution of the Soar cognitive architecture. In T. Mitchell (ed.) Mind Matters, 1-50. [2] Rosenbloom, P. S., Laird, J. E., & Newell, A. (1993) The Soar papers: Research on Integrated Intelligence. MIT Press, Cambridge, MA. [3] Jones, R. M., Laird, J. E., Nielsen, P. E., Coulter, K. J., Kenny, P., & Koss, F. V. (1999) Automated intelligent pilots for combat might simulation. AI Magazine, 20(1), 27-41. [4] Wray, R. E., Laird, J. E., Nuxoll, A., Stokes, D., & Kerfoot, A. (2005) Synthetic adversaries for urban combat training. AI Magazine, 26(3), 82-92. [5] Anderson, J. R. (2007) How Can the Human Mind Occur in the Physical Universe? Oxford University Press. [6] Cho, B., Rosenbloom, P. S. & Dolan, C. P. (1991) Neuro-Soar: A neural-network architecture for goaloriented behavior, Proceedings of the Thirteenth Annual Conference of the Cognitive Science Society, 673-677. Chicago, IL. [7] Taatgen, N. A., Juvina, I., Herd, S., Jilk, D., & Martens, S. (2007) Attentional blink: An internal traffic jam? Eighth International Conference on Cognitive Modeling. [8] Pearson, D. J., & Laird, J. E., Incremental Learning of Procedural Planning Knowledge in Challenging Environments, Computational Intelligence, 2005, 21(4), 414:439.

J.E. Laird / Extending the Soar Cognitive Architecture

235

[9] Steier, D.S., Laird, J.E., Newell, A., Rosenbloom, P.S., Flynn, R., Golding, A., Polk, T.A., Shivers, O., Unruh, A., Yost, G.R. (1987) Varieties of Learning in Soar. Proceedings of the Fourth International Machine Learning Workshop. [10] Chong, R. (2003) The addition of an activation and decay mechanism to the Soar architecture. Proceedings of the Fifth International Conference on Cognitive Modeling, Pittsburgh, PA. [11] Nuxoll, A. M., Laird, J. E., & James, M. (2004) Comprehensive working memory activation in Soar, Sixth International Conference on Cognitive Modeling, 226-230. [12] Nuxoll, A. M., & Laird, J. E. (2007) Extending cognitive architecture with episodic memory. Proceedings of the 21st National Conference on Artificial Intelligence. [13] Wray, R. E & Laird, J.E. (2003) Variability in Human Behavior Modeling for Military Simulations. Proceedings of the 2003 Conference on Behavior Representation in Modeling and Simulation. Scottsdale, AZ. [14] Nason, S., & Laird, J. E. (2005) Soar-RL: Integrating reinforcement learning with Soar. Cognitive Systems Research, 6(1), 51-59. [15] Wang, Y., and Laird, J.E. (2007) The Importance of Action History in Decision Making and Reinforcement Learning. Proceedings of the Eighth International Conference on Cognitive Modeling. Ann Arbor, MI. [16] Roseman, I. & Smith, C. A. (2001) Appraisal theory: Overview, Assumptions, Varieties, Controversies. In Scherer, Schorr, & Johnstone (Eds.) Appraisal processes in Emotion: Theory, Methods, Research, 319. Oxford University Press. [17] Scherer, K. R. (2001) Appraisal considered as a process of multi-level sequential checking. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.) Appraisal processes in Emotion: Theory, Methods, Research. 92-120. Oxford University Press. [18] Marinier, R. P., & Laird, J. E. (2007) Computational Modeling of Mood and Feeling from Emotion. Proceedings of 29th Meeting of the Cognitive Science Society. 461-466. Nashville: Cognitive Science Society. [19] Singh, S., Barto, A. G., & Chentanez, N. (2004) Intrinsically motivated reinforcement learning. 18th Annual Conference on Neural Information Processing Systems (NIPS). [20] Marinier, R. P., & Laird, J. E. (2008) Emotion-Driven Reinforcement Learning. CCA-TR-2007-02, Center for Cognitive Architecture, University of Michigan. [21] Gratch, J. & Marsella, S. (2004) A Domain-independent Framework for Modeling Emotion. Cognitive Systems Research, 5:269-306, 2004. [22] Wang, Y., and Laird, J.E. 2006. Integrating Semantic Memory into a Cognitive Architecture. CCA-TR2006-02, Center for Cognitive Architecture, University of Michigan. [23] Rosenbloom, P. S. (2006) A cognitive odyssey: From the power law of practice to a general learning mechanism and beyond. Tutorials in Quantitative Methods for Psychology, 2(2), 38-42. [24] Tulving, E. (1983) Elements of Episodic Memory. Oxford: Clarendon Press. [25] Nolan, C. (2000) Memento, New Market Films. [26] Kosslyn, S. M., Thompson, W. L., & Ganis, G. (2006). The Case for Mental Imagery. New York, New York: Oxford University Press. [27] Lathrop, S.D., and Laird, J.E. (2007). Towards Incorporating Visual Imagery into a Cognitive Architecture. Eighth International Conference on Cognitive Modeling. [28] Granger, R. (2006) Engines of the brain: The computational instruction set of human cognition. AI Magazine 27: 15-32. [29] Sun, R. (2006). The CLARION cognitive architecture: Extending cognitive modeling to social simulation. In: Ron Sun (ed.), Cognition and Multi-Agent Interaction. Cambridge University Press, New York. [30] Newell, A. (1990) Unified Theories of Cognition. Harvard University Press. [31] Kieras, D. & Meyer, D. E. (1997) An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human-Computer Interaction, 12, 391-438.

236

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

Temporal Action Logic for Question Answering in an Adventure Game Martin MAGNUSSON and Patrick DOHERTY Department of Computer and Information Science Linköping University, 581 83 Linköping, Sweden E-mail: {marma,patdo}@ida.liu.se Abstract. Inhabiting the complex and dynamic environments of modern computer games with autonomous agents capable of intelligent timely behaviour is a significant research challenge. We illustrate this using our own attempts to build a practical agent architecture on a logicist foundation. In the ANDI-Land adventure game concept players solve puzzles by eliciting information from computer characters through natural language question answering. While numerous challenges immediately presented themselves, they took on a form of concrete and accessible problems to solve, and we present some of our initial solutions. We conclude that games, due to their demand for human-like computer characters with robust and independent operation in large simulated worlds, might serve as excellent test beds for research towards artificial general intelligence. Keywords. Temporal Action Logic, computer games, natural language understanding, artificial general intelligence, natural deduction, planning, epistemic reasoning

1. Introduction Two topics that have seen a recent boost of interest are research on artificial general intelligence (AGI) and the use of modern computer games as AI research test beds. There is much to say in favour of combining these trends, though we confine ourselves to two important observations. First, games are readily accessible both for the scientist who can use existing games with exposed APIs, or relatively easily implement entirely new games, and for the peer researcher or student who can download and experiment with the software themselves. Second, their demand for human-like behaviour in complex environments necessitates a certain amount of generality in any proposed solution. Game environments are much more complex than classical benchmark problems such as the blocks world, which are often criticised for their limited scope (e.g. by Hayes [1]). In contrast, most computer games are incompatible with simplifying assumptions such as the (in)famous closed world assumption and call for many of the capabilities needed for general intelligence such as an agent architecture that integrates everything from perception to action, robustness and responsiveness in sometimes unpredictable environments, goal-directed action planning, multi agent communication, reasoning about knowledge and how to obtain it, and natural language understanding for dialog interaction. Our own work involves research on topics relevant to an adventure game project where a human player solves simple puzzles through natural language question answer-

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering

237

ing dialogs with ANDIs, agents with Automated Natural Deduction based Intelligence, who inhabit ANDI-Land. Present day games almost universally adopt straight jacketed exchanges typically featuring a choice between three canned sentences, two of which are humorous sidetracks and one that will move the dialog forward to the next set of sentences. Our aim is to eliminate the forced linearity of scripted dialogs through artificial intelligence technology. Rather than mindlessly trying all alternatives, we would have the player think1 . The reader is encouraged to evaluate the results we describe below through experimentation with our demonstrator available for download (as a Windows binary) at www.andi-land.com. The long term aim is wide coverage natural language understanding, which requires both extensive knowledge of the topics under discussion and the capability to reason with it. Such demands can ultimately only be satisfied by true AGI, while our efforts to date are certainly not in that ballpark. But our initial experiences with ANDI-Land indicate that a computer game setting enables an incremental approach where reasonably difficult challenges can be attacked while keeping the long term goal in mind. The work presented below is, for this reason, based on a logicist foundation. We believe the best way to approach general intelligence is by formulating most types of reasoning in a unified proof system for deductive and non-monotonic types of inference in a, not necessarily purely classical, logical formalism expressive enough to capture all the subtleties and distinctions that humans make in their reasoning. If successful, such an endeavour will allow the use of efficient specialized reasoning processes when applicable, yet always providing the option to fall back on more general but less efficient methods of proof in new and unforeseen situations. Rather than expanding further on this nebulous conjecture we will discuss the specific research problems that immediately suggested themselves when we initiated work on our question answering adventure game concept, in Section 2. Section 3 presents example dialogs from an in-game scenario that illustrate some capabilities of the architecture built in response to the challenges. A hopelessly inadequate selection of related work, squeezed into Section 4, will have to make do for orienting our efforts in relation to others’. Finally, Section 5 concludes with a look towards the future.

2. ANDI-Land ANDI-Land consists of an isometric graphical representation of a forest that can be explored by a player through a keyboard controlled avatar. The forest is inhabited by intelligent agents with which the player can initiate question answering conversations, and who sometimes proactively do so themselves in order to further their own goals. There is a puzzle element to make interaction interesting, but unlike most other adventure type games, solving puzzles through a process of eliminating all alternatives is not feasible since the natural language input is not sufficiently restrictive. Implementing this concept requires providing ANDI-Land agents with a genuine understanding of questions posed to them and equipping them with knowledge of their virtual world from which to deduce answers. Only the coordination of linguistic and semantic processing can make this possible. 1 Although

whether this constitutes an enjoyable game experience depends, of course, on the player.

238

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering

Figure 1. Active edges in the chart resulting from parsing “who” show the possibilities for the next word in the sentence, “owns”, “am”, and “are”.

2.1. Interactive Natural Language Input As we pointed out in Section 1, it would be unrealistic to expect both broad and deep natural language understanding sooner than the development of AGI. First, no one has yet been able to construct a grammar with adequate coverage of the entire English language. The problem is not just that players of the game will not be able to express themselves in whatever way they want, but worse, nor will they receive any hints as to how to rephrase themselves in a way that the parser will understand. Second, the knowledge of the game characters will, while aiming at generality, realistically start out as downright narrow. Most sentences would simply be outside the area of competence of the game characters, and they would have to respond with an honest “I don’t know”. These two problems threaten to reduce the adventure to a sort of guessing game where the player, more or less blindly, would have to search for sentences that both avoid the equivalent of a "parse error" message and whose semantic meaning happens to produce something other than a puzzled look on the respondent’s face. Using a very large wide coverage grammar like the English Resource Grammar [2] would seem to help alleviate the first problem, but at the cost of worsening the second. The semantical form it produces is not detailed enough to suffice for automated reasoning and question answering. Our project would be quite stranded if these problems left no room for incremental progress building on a modest start. ANDI-Land incorporates a unique form of interactive natural language input that deal with both problems by guiding the player towards sentences that parse correctly and fall within the AIs competence areas. For example, the sentence “who is the lumber’s owner” is not supported by our current grammar, but the sentence “who owns the lumber” is. Even though this restriction is arbitrary, the player is spared much frustration due to the interactive parser. As soon as the player starts typing, a chart parser (as described in [3])2 starts producing partial parses that cover as many words as possible. Though the initial word “who” does not constitute a complete sentence, the resulting parse chart still contains useful information. Specifically, by looking at active chart edges we can collect all words that would advance the parse if they occurred immediately succeeding the current input. According to Figure 1, the words “owns”, “am”, and “are” constitute all the possible continuations of the current input, and choosing among them effectively circumvents all sentences that have no chance of resulting in a complete parse. The process could be likened to the widespread T9 mobile phone text input, except that the system understands the grammar of entire sentences rather than just the correct spelling of words. Furthermore, by limiting the grammar to concepts covered by the agent’s background knowledge we can ensure that most of the input sentences are answered intel2 The parser and grammar are extremely simple. We would like to improve them at a later date, perhaps modifying the Linguistic Knowledge Builder [2] for our interactive mode.

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering

Natural language question Input logical form Answer variable binding Answer logical form Natural language response

239

Who owns the lumber? ∃ans [value(now, owner(lumber)) = ans] ans = djak value(now, owner(lumber)) = djak Djak owns the lumber.

Figure 2. The process from natural language question to natural language response is largely symmetrical, thanks to a “reversible” grammar.

ligently. Compared to scripted dialogs, the interactive input method presents the player with a multitude of choices, even using a very small grammar, while also allowing for gradual improvements in language coverage. 2.2. Reversible Natural Language Grammar Another challenge, natural language generation, is presented by communication in the opposite direction. Each grammar rule has an associated lambda expression that represents its meaning. Meaning fragments are combined by lambda application that eventually, after a complete parse of the sentence, results in formulas of first-order logic. These formulas are directly amenable to automated reasoning to produce an answer expression, encoding the response in first-order logic. Of course, we would prefer the game characters to reply in plain English. Shieber’s uniform architecture [4] for both parsing and generation addresses this difficulty with minimal machinery. It effectively makes the natural language grammar reversible with relatively small modifications to the basic chart parsing algorithm. There is no need for separate generation algorithms. Furthermore, extensions to the input grammar that help the ANDI-Land inhabitants understand new words or grammar automatically increase their proficiency in using them in speech too. Figure 2 illustrates the question answering process using the example sentence from the previous section. A question is parsed into a formula that contains an answer variable. Its value, found through the theorem proving techniques described in Section 2.6, can be used to instantiate the query to form an answer expression. Finally, the chart parser is run in “reverse” to produce a natural language response to the original question. 2.3. Temporal Action Logic We said that the logical forms resulting from parsing were amenable to automated reasoning. Work within the methodology of formal logic provides a comprehensive tool set for correct reasoning. However, the standard philosophical logic turns out to be inadequate to support the thinking processes of active characters in dynamic environments. Researchers in cognitive robotics are therefore creating new powerful logics that are applicable to commonsense reasoning about action and change as well as more traditional logical reasoning. We have chosen to work with one such logic, the Temporal Action Logic (TAL), which adopts an intuitive explicit time line to describe actions and their effects in a changing environment. The origins of TAL can be found in the Features and Fluents framework developed by Sandewall [5], but it was a new characterization in terms of first-order logic with circumscription, by Doherty [6], that made automated reasoning possible. Many extensions since have turned TAL into a very expressive language capable of representing, among other things, actions with durations, context-dependent and non-deterministic actions, concurrency, and action side-effects.

240

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering

But the most important feature of TAL might be its occlusion concept that serves as a flexible tool to deal with important aspects of the frame problem, which has long haunted logical approaches to AI. Properties and relations that may change over time are modelled by fluents, and the value v of a fluent f can be linked to a time point t on the time line using a function value(t, f ) = v. Some agent a (denoted self when doing the thinking) carrying out an action c during time interval i is specified by Occurs(a, i, c). The following formula relates a fluent f ’s value at the starting and ending time points of a time interval i, unless the fluent is occluded, as specified by Occlude(i, f ): ∀i, f [¬Occlude(i, f ) → value(start(i), f ) = value(f inish(i), f )]

(1)

The role of circumscription is the minimization of action occurrences and occlusion to implement the blanket assumption that no unexpected actions occur and fluents’ values persist over time. Exceptions are specified by explicit action occurrences and their occlusion of fluents they affect, thus releasing them from the frame assumption that their values remain unchanged. E.g., if the game character Djak was to sell the lumber he possesses in Figure 2, the fluent owner(lumber) would be occluded during any interval that overlaps the interval during which the selling occurrs, and Formula 1 would not be applicable. 2.4. Reasoning and Planning However, one of the most important forms of reasoning is not supported by the TAL framework as described above, namely proactive planning to achieve goals. E.g., if Djak’s (modest) goal in life is the possession of lumber, he could reason that going to a shop and buying lumber is one possible plan to satisfy it. But his reasoning must allow the consideration of different actions before committing to any particular plan. Djak should not commence going to the store before considering whether the store actually sells lumber or not, since if it does not he might have to resort to an alternative sequence of actions such as cutting down some trees himself. However, committed knowledge about the set of actions is a prerequisite to automated reasoning using the circumscription account of TAL [7]. In contrast, we would like the set of actions to be a consequence of reasoning. This was accomplished in previous work [8] in a constraint logic programming setting. Rather than circumscribing a fixed set of actions, we use constraints to keep track of assumptions that depend on the set of actions, and reevaluate those assumptions when the set of actions change. The mechanism was cast as deduction, but the same principles are recast as abduction in a new first-order theorem proving setting described in Section 2.6. Thus equipped, Djak is both able to answer questions and plan his actions through automated reasoning with TAL. 2.5. Epistemics Game agents, however, must face an additional complication when planning their actions. Inhabitants of the game world can not reasonably be assumed to possess complete knowledge of their entire world, and even if they did, the dynamic nature of game environments would quickly make this knowledge obsolete. The closed world assumption that is at the foundation of many classical planning systems is not applicable. Instead, an intelligent agent must reason with incomplete information and, significantly, plan to

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering

241

obtain additional information when needed. E.g., suppose another ANDI-Land agent, Keypr, owns a shop. Although Keypr is all out of lumber, he could sell Djak an axe to use to cut down a tree with. Being an intelligent and proactive fellow, Djak might come up with the following plan fragment (excluding the tree cutting part): ∃i1 , i2 [Occurs(self, i1 , walk(value(start(i2 ), location(keypr)))) ∧ Occurs(self, i2 , buy(axe, keypr)) ∧ finish(i1 ) = start(i2 )] Though, what if Djak does not know Keypr’s location? The plan is still correct in the sense that if Djak executed it, the intended effects would manifest. The problem is that it is not executable. There is no way Djak can (willingly) walk to Keypr’s location without knowing what that location is, but we have as of yet no means to express this additional knowledge precondition. What is needed is an epistemic logic that includes a notion of knowledge. The most common way of introducing such a notion of knowledge is in the form of a modal operator Knows with a possible worlds semantics. This can be done while remaining in classical logic by encoding the possible worlds and the accessibility relation between them explicitly in the object language, as e.g. in Moore’s pioneering work [9]. But these approaches are associated with some limitations that make them unsuitable as general frameworks of epistemic reasoning, as pointed out e.g. by Morgenstern [10]. She proposes an alternative treatment that introduces Knows as a “syntactic” predicate, which accepts quoted formulas as arguments. Quotation can be seen as an extreme form of reification where any formula can be turned into a term. It appears to be both simpler and more intuitive than possible world semantics in many contexts. Unfortunately, quotation is associated with the risk of paradoxes. While it is true that unrestricted quotation leads to the paradox of the Knower [11], there are methods for avoiding these problems (a particularly interesting one is Perlis’ [12], which still allows for self-referential formulas). Our work, although adopting the syntactic quotation framework in anticipation of requirements of generality, has not yet proceeded far enough to utilize the additional expressivity afforded by syntactical treatments of knowledge over modal variants, a fact that guarantees consistency [13] and allows us to remain uncommitted as to which more general treatment to give preference to. Equipped with the ability to represent knowledge explicitly we add a precondition to walking that one should know where the destination is. We can also make use of the Knows predicate in action effects, thereby formalizing knowledge producing actions and putting us in a position where planning for knowledge acquisition is possible. Adding an action for asking another agent (such as the player!) about a fluent’s value enables Djak to come up with a plan that is both executable and that has the intended effect: ∃i1 , i2 , i3 [Occurs(self, i1 , askValue(player, location(keypr))) ∧ Occurs(self, i2 , walk(value(start(i3 ), location(keypr)))) ∧ Occurs(self, i3 , buy(axe, keypr)) ∧ finish(i1 ) = start(i2 ) ∧ finish(i2 ) = start(i3 )]

242

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering

2.6. Natural Deductive Theorem Proving Many agent architectures are built on a logic programming foundation, as was our previous work [8]. Logic programs incorporate some of the power of theorem proving while remaining relatively simple and allowing a high degree of control over the inference mechanism. But a fundamental limitation of Prolog is the assumption of complete knowledge, which, as we noted in Section 2.5, is unreasonable in complex computer games. In the interest of overcoming this limitation one can augment Prolog with meta-interpreters or other add-ons. Though when setting the sights for general intelligence it seems to us that augmenting Prolog will, over time, gradually approach general first-order theorem proving but in a roundabout and unnecessarily complicated way. An alternative approach is to start with a first-order resolution theorem prover and complement it with special purpose modules that make some types of reasoning highly efficient. This is the method taken by the Cyc team, who have gone one step further and given up completeness in favour of efficiency and expressiveness [14]. Our (limited) experience with resolution suggests to us that it is not quite the natural fit with commonsense reasoning that one would hope. For example, the need to compile the knowledge base into clause form destroys potentially useful structural information that was previously implicit in the syntactic form of knowledge and rules, and the use of a single proof rule based on reductio ad absurdum could be incompatible with the defeasible reasoning that has turned out to be so important to commonsense reasoning [15]. Still, resolution completely dominates the field of automated theorem proving, but it is not the only contender. One particularly interesting alternative is automated natural deduction. Rather than compiling the agent’s knowledge into clause form, such a theorem prover works with the “natural form” directly. And the rule set is extensible, thereby supporting the addition of special purpose rules, e.g. for defeasible reasoning. Moreover, whether the term “natural” is grounded in any relation between the deductive system and human reasoning is an exciting prospect explored by Rips, who argues a positive verdict [16]. In light of these considerations we have opted for our ANDI-Land inhabitants to “think” using an automated natural deduction theorem prover. Input formulas use the quantifier free form described by Pollock [17] and Rips [16]. This eliminates the somewhat cumbersome natural deduction rules for quantifier elimination and introduction while still preserving the knowledge base’s natural form to a large extent. Most importantly, it provides the opportunity to work with unification and enables the use of answer extraction for question answering by binding answer variables to values as exemplified in Figure 2. Rather than a select few inference rules there is a set of forward rules (four at the moment), which are applied whenever they become applicable, and a set of backward rules (currently eleven of them), which are used in a goal-directed search for a proof. Finally, equality is dealt with through a system of rewrite rules, and temporal relations are added to a general temporal constraint network [18], exemplifying the use of special purpose reasoning mechanisms for efficiency. A novel proof rule worthy of mention is a special abduction rule that allows relations from a set of abducibles to be assumed rather than proven, as long as doing so does not lead to inconsistency. This “natural abduction” rule forms the basis of the mechanism for non-monotonic reasoning and planning. As an example, consider the following natural deduction proof fragment (where the justifications in the right margin denote (P)remises, (H)ypotheses, the agents background (K)nowledge, and row numbers):

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering

1 2 3 4 5 6 7 8 9

value(12:00, location(self)) = loc(1, −1) start(i37 ) = 12:00 finish(i37 ) = 13:00 ¬Occlude(i37 , location(self)) value(13:00, location(self)) = loc(1, −1) Occurs(self, i38 , walk(loc(0, 0))) value(finish(i38 ), location(self)) = loc(0, 0) ∀i [¬Occlude(i, location(self)) → ¬Overlap(i, i38 )] ¬Overlap(i37 , i38 )

243

P P P H 1 − 4, K H 6, K 6, K 4, 8

The agent starts at the location with coordinate 1, −1 at noon, as in Row 1. Suppose the agent needs to remain at the same location at 1 p.m. One way of proving this would be to use persistence. The location fluent is only persistent if it is not occluded, and while the agent has no knowledge about whether it is occluded or not, ¬Occlude is an abducible and may thus be assumed. Rows 2-4 introduces a new interval constant and indicates the assumption using a natural deduction style vertical line in the margin. Suppose further that the agent, for some other reason, needs to visit location 0, 0. The only way of proving this would be if a walk action destined for that coordinate occurred. When planning, Occurs is also abducible, so the agent assumes such an action in Row 6. The effect on the agent’s location is recorded by Row 7. Walking should occlude the location fluent, but instead of stating that the fluent is occluded in any interval that overlaps the walk action, Row 8 uses the contra-position, stating that any interval that has assumed the location to be persistent must not overlap with the action of walking. This triggers the forward modus ponens rule to produce Row 9, partially ordering the two intervals to avoid any conflict between the persistence of the agent’s location, and the agent’s moving about. The non-overlap constraint is automatically added to the temporal constraint network. If it is impossible to order i37 and i38 so that they do not overlap in any way, the network becomes inconsistent, and the prover needs to backtrack, perhaps cancelling the most recent assumption. The abduction rule thus enables both defeasible conclusions about the persistence of fluents and, simultaneously, planning of new actions. The use of the contrapositive form illustrates a case where two logically equivalent formulas have different effects in the system due to their surface form. If the occlusion had been expressed as ∀i [Overlap(i, i38 ) → Occlude(i, location(self))], nothing would have triggered the non-overlap constraint. This, in turn, illustrates another important point. If the non-overlap constraint would make the temporal constraint network inconsistent, failing to trigger it could result in the agent failing to discover that one of its assumptions is unreasonable. This would not be a cause of unsoundness, since we are still within the sound system of natural deduction, but it might result in plans and conclusions that rest on impossible assumptions. A conclusion Φ depending on an inconsistent assumption would in effect have the logical form ⊥→ Φ, and thus be tautological and void. This is to be expected though since consistency is not even semi-decidable for first-order logic. The most we can hope for is for the agent to continually evaluate the consistency of its assumptions, improving the chances of them being correct over time, while regarding conclusions as tentative [15]. Another novelty is an execution rule linked to the agent’s action execution mechanism, which is used to put plans into effect. Instead of sending the entire plan to a “dumb” execution module, we use the execution rule in “proving” that the plan is executed, thereby enabling the full reasoning power of the natural deduction prover to be

244

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering M Goodbye!

pears: M Hello!

K

Before you have a chance to ask his name, he hurries northward. Curious, you follow. At Keypr’s, you observe the following dialog:

M Who

D

You steer your avatar Magni eastward and stumble upon another ANDI-Land character: M Hello!

Hello! are you? K I am Keypr. M What do you own? K I own the axe. M What is the axe’s price? K The axe’s price is 5 gold. M What is my wealth? (thinking) M My wealth is 4 gold. M Goodbye! K Goodbye! Dismayed by a sense of acute poverty, you continue to investigate the great forest. South-west lives another character, and as soon as he spots you, he comes running: D

Hello!

M Hello!

Who owns the axe? owns the axe. D What is Keypr’s location? M Keypr’s location is 1 screen east and 2 screen north. D Goodbye! D

M Keypr

K D K D K

Hello! Hello! Sell the axe to me. OK. Goodbye! Goodbye!

Somewhat envious of the axewielding stranger, you follow him back and watch him start applying the axe to the trunk of a tree. Determined to know his identity you confront him: M Hello!

Hello! are you? D I am Djak. M What happened? D I bought the axe from Keypr. M What do you own? D I own the axe. M Goodbye! D Goodbye!

D

M Who

While you watch eagerly as Djak strikes the tree, it suddenly disap-

Hello! do you own? D I own the axe and I own the lumber. M What is the lumber’s price? D The lumber’s price is 3 gold. M Sell the lumber to me. D OK. M Goodbye! D Goodbye! D

M What

Acting as a middle man, you revisit Keypr to try to sell the lumber: M Hello!

Hello! is the lumber’s price? K The lumber’s price is 6 gold. M Buy the lumber from me. K OK. M What is my wealth (thinking) M My wealth is 7 gold. K

M What

Intrigued by Djak and Keypr’s limited displays of intelligence, but convinced that more must be possible, you vow to research AI in games!

Figure 3. This scenario from the ANDI-Land adventure game concept involves planned use of speech acts to satisfy the knowledge preconditions of buying an axe.

used in finding the exact action parameters for each step of the plan. Consider, e.g., Djak’s plan to ask the player about Keypr’s location in Section 2.5. Depending on the player’s reply, further reasoning might be needed to convert this reply into a form that is suitable to pass as an argument to the action execution mechanism. This reasoning might depend on background knowledge about local geography and, in general, any amount of deliberation might be required during execution of a plan that involves knowledge acquisition, a fact respected by our execution proof rule.

3. A Dialog Scenario Figure 3 illustrates all the components working together through example dialogs from ANDI-Land. The scenario revolves around our friends Djak and Keypr, from previous sections, but starts with the human player’s avatar Magni in the middle of a thick forest. Djak’s plan was automatically generated by the natural deductive theorem prover and its abduction rule, while the plan execution and all dialogs are straight from a running game session.

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering

245

4. Related Work In a spirit similar to ours, Amir and Doyle have proposed the use of text adventure games as a vehicle of research in cognitive robotics [19]. But instead of intelligent agents acting in supporting roles to enhance a human player’s experience, they consider what challenges an agent would face if trying to solve the adventure itself. The agent would start out with severely limited knowledge, not knowing what actions are available to it, what fluents it should use to represent the environment, nor even the purpose of the game. These are some significant challenges, though they say a computer game “[...] allows us to examine them in a controlled environment in which we can easily change the problems to be solvable, and then gradually increase the difficulty step by step”. However, their proposition does not endorse any specific formalism or system. Shanahan [20] proposes a logicist agent architecture that incorporates planning, perception, and a sense-plan-act loop, all formalized in the Event Calculus and executed through proof using abductive logic programming. The unified approach makes it possible to proactively deal with unexpected percepts in a robotic mail delivery domain, due to humans unpredictably blocking pathways by closing office doors. The robotic agent is able to intelligently adapt its behaviour by first reasoning about all percepts using abductive proof, forming explanations for sensor values that deviate from expectations in terms of actions by other agents or humans, and then adapting its plans to incorporate the new knowledge. Hierarchical planning is accomplished through the same abductive proof mechanism and allows timely reactions by only instantiating the abstract plan enough to figure out a first action, while leaving the rest a sketchy idea of how to achieve the goal. Pollock goes further towards general intelligence and differentiates between goaloriented agents, that solve tasks for which a metric of success can be defined, and anthropomorphic agents, that solve tasks that are too complex for it to be possible to identify such a metric [15]. Such agents must be based on a “general theory of rational cognition”, and Pollock’s OSCAR agent architecture is an attempt to embody such a theory in an implemented system. The central component is a natural deduction theorem prover for first-order logic that is capable of planning, reasoning about percepts and attaching certainty factors to premises and conclusions. But its most important feature is the mechanism for defeasible reasoning that can be used to deal with default reasoning and the frame problem. Unlike most other formalisms, which are only applicable to problems conforming to explicit restrictions that ensure computability, Pollock’s anthropomorphic architecture can be applied to any problem. The inference engine reports solutions based on defeasible assumptions, while a search for evidence contradicting these assumptions continues, for which there can be no guarantee of termination. Wang’s NARS system is similar in that the underlying assumption is the lack of knowledge and resources sufficient to give optimal answers or even any correctness guarantees [21]. Instead, the system continually evaluates the available evidence and may “change its mind” about the best answer to a given query. NARS is based on a novel categorical logic that differs significantly from classical first-order logic, incorporates uncertainty, and deals with conflicting evidence at a fundamental level. While clearly aiming towards general intelligence, results to date seem limited to small benchmark problems. One example of the relatively recent surge of interest in the use of computer games for AI research is the Soar/Games project. They report uncovering new research challenges after coupling the Soar artificial general intelligence architecture to Quake 2 and

246

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering

Descent 3 [22]. Their emphasis is on generality in their attempts to build reusable rule bases for agent behaviour. Laird’s and van Lent’s enthusiasm for the use of computer games in AGI research is evident in their paper “Human-level AI’s Killer Application: Interactive Computer Games” [23]. Finally, the 1996 computer game Creatures is an example of AI from the game industry rather than of academic origins. Its artificial life forms use neural net “brains” that can be trained through interaction with a human player, learn from interaction with their simulated world, or even from other creatures [24]. The success of Creatures is an affirmation of the possibility of incorporating AI technology into a commercial computer game.

5. Conclusions We hope to have given the impression that our game concept is far from complete. On the contrary, when working with games interesting problems abound, and many of them call for new research in artificial general intelligence. Some old but still open questions that figure in our work are how to include perception, reasoning, planning, execution, and failure recovery in an integrated agent architecture, what to do about conflicting information, and how to deal with the accumulation of perceptions and knowledge in persistent agents without their reasoning slowing down to a crawl. ANDI-Land is fundamentally a multi agent setting and could involve cooperation between multiple agents, delegation of goals, and intelligent use of communication. These (and more) topics have concrete instantiations in the structure of the game environment that make them easier to think about, discuss, and hopefully to solve. Traditional AI benchmark problems play an important role in clearly highlighting specific difficulties that any sufficiently general AI system will have to address. Games can serve to complement them by forcing an integrated view of autonomous agents in complex environments, and they possess many positive attributes such as ease of access for both researchers and their peers, variable challenge level ranging all the way from simple puzzle games to wide coverage natural language understanding, and the possibility for applications in the future commercial game industry where academic AI technology has so far failed to migrate (the prototypical exception being A* search). The demand for human-like computer characters is by itself incentive to study all the key technologies needed for artificial general intelligence, making games an excellent test bed for AGI research. Even some forms of self-awareness would seem to be desirable to agents acting as if they were “real” live inhabitants of some fictional reality game world. Such a setting is a sort of Turing test where human players are not necessarily aware of which characters are artificial and which are other humans. It seems to us that research on game AI could function as a much needed road map towards the fields original vision.

Acknowledgements This work is supported in part by the National Aeronautics Research Program NFFP04 S4203 and the Strategic Research Center MOVIII, funded by the Swedish Foundation for Strategic Research, SSF.

M. Magnusson and P. Doherty / Temporal Action Logic for Question Answering

247

References [1] Patrick J. Hayes. The naive physics manifesto. In Donald Michie, editor, Expert Systems in the MicroElectronic Age, pages 242–270. Edinburgh University Press, 1979. [2] Ann Copestake and Dan Flickinger. An open source grammar development environment and broadcoverage English grammar using HPSG. In Proceedings of the 2nd International Conference on Language Resources and Evaluation LREC 2000, Athens, Greece, 2000. [3] Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Prentice-Hall, Englewood Cliffs, NJ, 2nd edition, 2003. [4] Stuart M. Shieber. A uniform architecture for parsing and generation. In Proceedings of the 12th International Conference on Computational Linguistics, pages 614–619, Budapest, Hungary, 1988. [5] Erik Sandewall. Features and Fluents: The Representation of Knowledge about Dynamical Systems, volume 1. Oxford University Press, 1994. [6] Patrick Doherty. Reasoning about action and change using occlusion. In Proceedings of the Eleventh European Conference on Artificial Intelligence ECAI’94, pages 401–405, 1994. [7] Patrick Doherty and Jonas Kvarnström. The Handbook of Knowledge Representation, chapter 18. Elsevier, 2007. To appear. [8] Martin Magnusson. Deductive Planning and Composite Actions in Temporal Action Logic. Licentiate thesis, Linköping University, September 2007. http://www.martinmagnusson.com/ publications/magnusson-2007-lic.pdf. [9] Robert Moore. Reasoning about knowledge and action. Technical Report 191, AI Center, SRI International, Menlo Park, CA, October 1980. [10] Leora Morgenstern. A first order theory of planning, knowledge, and action. In Proceedings of the 1986 conference on Theoretical aspects of reasoning about knowledge TARK 1986, pages 99–114, 1986. [11] David Kaplan and Richard Montague. A paradox regained. Notre Dame Journal of Formal Logic, 1(3):79–90, 1960. [12] Donald Perlis. Languages with self-reference I: Foundations (or: we can have everything in first-order logic). Artificial Intelligence, 25(3):301–322, 1985. [13] Jim des Rivières and Hector J. Levesque. The consistency of syntactical treatments of knowledge. In Proceedings of the 1986 conference on Theoretical aspects of reasoning about knowledge TARK 1986, pages 115–130, 1986. [14] Douglas Lenat. CYC: a large-scale investment in knowledge infrastructure. Communications of the ACM, 38(11):33–38, 1995. [15] John Pollock. Rational cognition in OSCAR. In Nicholas Jennings and Yves Lesperance, editors, Intelligent Agents VI: Agent Theories, Architectures, and Languages, volume 1757 of Lecture Notes in AI, pages 71–90. Springer Verlag, 2000. [16] Lance J. Rips. The psychology of proof: deductive reasoning in human thinking. MIT Press, Cambridge, MA, USA, 1994. [17] John Pollock. Natural deduction. Technical report, Department of Philosophy, University of Arizona, 1999. http://www.sambabike.org/ftp/OSCAR-web-page/PAPERS/ Natural-Deduction.pdf. [18] Itay Meiri. Combining qualitative and quantitative constraints in temporal reasoning. Artificial Intelligence, 87(1–2):343–385, 1996. [19] Eyal Amir and Patrick Doyle. Adventure games: a challenge for cognitive robotics. In AAAI’02 workshop on Cognitive Robotics, 2002. [20] Murray Shanahan. Reinventing shakey. In Logic-Based Artificial Intelligence, pages 233–253. Kluwer Academic Publishers, Norwell, MA, USA, 2000. [21] Pei Wang. The logic of intelligence. In Ben Goertzel and Cassio Pennachin, editors, Artificial General Intelligence, pages 31–62. Springer, 2007. [22] John E. Laird and Michael van Lent. Developing an artificial intelligence engine. In Proceedings of the Game Developers’ Conference, pages 577–588, 1999. [23] John E. Laird and Michael van Lent. Human-level AI’s killer application: Interactive computer games. In Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, pages 1171–1178, 2000. [24] Stephen Grand, Dave Cliff, and Anil Malhotra. Creatures: Artificial life autonomous software agents for home entertainment. In Proceedings of the First International Conference on Autonomous Agents (Agents’97), pages 22–29, 1997.

248

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

Artificial General Intelligence through Large-Scale, Multimodal Bayesian Learning a CSAIL,

Brian MILCH a,1 Massachusetts Institute of Technology, USA

Abstract. An artificial system that achieves human-level performance on opendomain tasks must have a huge amount of knowledge about the world. We argue that the most feasible way to construct such a system is to let it learn from the large collections of text, images, and video that are available online. More specifically, the system should use a Bayesian probability model to construct hypotheses about both specific objects and events, and general patterns that explain the observed data. Keywords. probabilistic model, architecture, knowledge acquisition

Introduction A long-standing goal of artificial intelligence is to build a single system that can answer questions as diverse as, “How can I get from Boston to New Haven without a car?”, “How many Nobel Prizes have been won by people from developing countries?”, and “In this scene showing people on a city street, about how cold is it?”. Answering such questions requires broad knowledge, on topics ranging from public transit to geography to weather-appropriate clothing. These questions also require deep reasoning, not just scanning for keywords or looking at simple features in an image. So far, we do not have AI systems whose knowledge is both broad and deep enough to answer this range of questions. The most prominent efforts to acquire such knowledge are Cyc [1] and Open Mind [2], both of which have significant limitations. The knowledge they collect is primarily deterministic: it does not include quantitative measures of how often things tend to occur. Furthermore, adding new knowledge requires effort by humans, which limits the breadth of questions that can be answered. Meanwhile, other branches of AI have focused on reasoning with probabilistic models that explicitly quantify uncertainty [3], and on learning probabilistic models automatically from data [4]. This probabilistic approach to AI has been successful in narrow domains, ranging from gene expression analysis [5] to terrain modeling for autonomous driving [6]. It has also seen domain-independent applications, such as sentence parsing [7] and object recognition [8], but these applications have been relatively shallow: they have not captured enough semantics to answer questions of the kind we posed above. 1 Corresponding Author: Brian Milch, MIT CSAIL, 32 Vassar St. Room 32-G480, Cambridge, MA 02139, USA; E-mail: [email protected].

B. Milch / Artificial General Intelligence Through Large-Scale, Multimodal Bayesian Learning

249

Tendencies

Appearances

Language Use

World History S Scene St Structure t

…

…

…

…

…

…

Utterance Structure

…

Video Data

Linguistic Data

Figure 1. Sketch of a probabilistic graphical model for AGI. Nodes represent random variables and arrows represent probabilistic influences.

Thus, for the past two decades there has been an unfortunate divide between the probabilistic AI community and those actively working on deep, open-domain AI — what Goertzel has called artificial general intelligence or AGI [9]. There are some signs that this gap is narrowing: the Novamente Cognition Engine [9] is an AGI system that does probabilistic reasoning, and the probabilistic AI community is showing renewed interest in large-scale knowledge acquisition [10,11]. The purpose of this paper is to bridge this gap more completely. We propose a Bayesian AGI system that takes a large chunk of the web as input, including both hypertext and video data, and uses a probabilistic model to make inferences about what the world must be like. We also comment on integrating AGI into the ecosystem of probabilistic AI research.

1. A Bayesian Architecture for AGI Figure 1 shows a schematic view of a probabilistic model suitable for an AGI system. The shaded nodes represent variables that the system can observe: pixel values in videos, and words in linguistic data. The unshaded nodes represent unobserved variables about which the system can form hypotheses. The variables in the top box represent general tendencies in the world, such as the way public transit tends to work, and the tendency of people to wear wool hats and scarves only in cold weather. The “world history” box contains variables representing specific facts about the world: for instance, the current public transit schedules between New Haven and Boston, and the history of Nobel prizes. The large arrow from the “tendencies” box to “world history” indicates that general tendencies influence the concrete facts. There are also arrows within the world history box, representing causal links between events and their successsors. In the top left and top right boxes, theare are variables for how things tend to appear, and how people tend to use language. Finally, there are variables representing the 3-d

250

B. Milch / Artificial General Intelligence Through Large-Scale, Multimodal Bayesian Learning

structures that underlie particular scenes and the syntactic structures that underlie particular utterances. These variables depend on the facts in the world: a scene or utterance typically provides a window on some piece of the world history. The additional arrow from tendencies to utterance structure allows utterances to be directly about tendencies: for instance, someone could say, “People wear scarves when it’s cold”. We discuss parts of this model in more detail in Sections 3 and 4. Our system’s probability model can be seen as a joint distribution over all thse variables. It can also be understood as a hierarchy of models: a hypothesis about the variables in the top three boxes defines the structure and parameters of a probability model for world histories, scenes, and utterances. In using a top-level probability model to define a distribution over sub-models, we are taking a Bayesian approach [12]. Bayesian modeling is attractive because it makes an automatic trade-off between the complexity of a hypothesized sub-model and how well it fits the data. Complex hypotheses may fit the data better, but they typically have lower prior probabilities than simple ones. Furthermore, hypotheses involving many free parameters are penalized by a Bayesian Ockham’s razor effect [13]: it is unlikely that all those parameters would happen to have values that match the data. Although similar trade-offs arise from Kolmogorov complexity [14] and the minimum description length principle [15], the Bayesian approach lets us express prior knowledge about the structure of the world, and also maintain uncertainty about sub-models rather than choosing a single best hypothesis. There are also close connections with Bayesian approaches in statistics [12], epistemology [16] and cognitive science [17].

2. Training Data There are several ways in which one could hope to train a Bayesian AGI system. One is to train it solely on text, a huge amount of which is available online. However, this would leave the system unable to answer queries about images or video scenes. Furthermore, there is no evidence that it is possible to build a deep model of the world from lingusitic input alone: human children all have access to visual, or at least tactile, input. Alternatively, we might train our AGI system by hooking it up to one or more mobile robots. This would have several advantages: the system could learn from continuous streams of experience with multiple sensors, and it could actively move around and manipulate its environment to reduce its uncertainty. On the other hand, a robot’s experience would be fairly narrow, limited to the environments where its owners let it move around. Fortunately, given the explosive growth of online video, we no longer have to choose between a large, diverse data set and a multimodal one. Videos available online include many everday objects and activities (sometimes in the background), often correlated with speech. Meanwhile, the text on the web covers many kinds of objects and phenomena that are not readily visible in videos. Thus, we propose to train our AGI system on a web data set that includes hypertext, images, and video. In contrast to a human child or a robot, this system still will not have a chance to actively explore its world. But acquiring multimodal data from the web promises to be easier and less expensive than acquiring it with a fleet of mobile robots — and even human children learn a lot from non-interactive media such as books, television, and movies.

B. Milch / Artificial General Intelligence Through Large-Scale, Multimodal Bayesian Learning

251

3. Built-In Model Components Although Figure 1 provides a rough sketch, the question of how to define the probability model for a Bayesian AGI system is still largely open. Some authors have argued that we should be able to use generic models that encode little or no prior knowledge about the physical world [18]. But finding good hypotheses to explain the world is a daunting search problem; to the extent that we can help our system along, it seems that we should do so. After all, it is unlikely that human children start from scratch: we seem to have an evolved infrastructure for intepreting visual input, understanding language, and interacting with other people. Modeling the physical world is one area where we should be able to provide our AGI system with a good running start. The computer graphics and animation communities have done a great deal of work on modeling the world with enough fidelity to create compelling visual environments. Our appearance sub-models should be able to use standard mesh data structures for representing the 3-d shapes of objects, and also borrow skeleton-based techniques for modeling articulated objects. The system will still need to infer the meshes and skeletons of objects (and classes of objects) from data, but at least it will not need to invent its own approach to representing shape and movement. It also makes sense to build in components for dealing with speech, hypertext, and language in general. The language-use sub-model should contain built-in representations for words and parse trees. As with the appearance model, all the specific content — the grammars of particular languages, and the words and phrases people tend to use for particular meanings — should be inferred by the system. But there is no reason for the system to re-invent the basic ideas of computational linguistics. Another area that calls out for built-in infrastructure is reasoning about the beliefs and desires of other agents. This ability is crucial both for understanding human behavior in the training data, and for giving helpful answers to users’ questions. The system must be able to put itself in another agent’s shoes, using its own inference and planning capabilities to reason about what another agent probably believes and what it might do to achieve its goals. Thus the system’s probability model must have some characteristics of a probabilistic epistemic logic theory [19] or a multi-agent influence diagram [20]. It is particularly hard to imagine how a Bayesian AGI system might invent this intentional view of other agents if it were not built in.

4. Learned Model Components Of course, even with built-in capacities for physical, linguistic, and multi-agent reasoning, our system will have a lot to learn. It will lack such basic notions as eating, smiling, driving, owning objects, having a disease, and so on. Thus, the overall probability model must admit indefinitely many non-built-in concepts. Fortunately, there has recently been a surge in research on probability models with an unbounded number of latent concepts, using a non-parametric Bayesian construct called the Dirichlet process (DP) mixture model [21,22,23]. A DP mixture can be thought of as a graphical model where the nodes are indexed by an infinite collection of objects, and these latent objects are “recruited” as necessary to explain the data. The Bayesian Ockham’s razor effect [13] prevents the model from hypothesizing a separate latent object for

252

B. Milch / Artificial General Intelligence Through Large-Scale, Multimodal Bayesian Learning

each data point. DP mixtures can serve as building blocks in quite complicated models, including those that model the relations among objects as well as their properties [24]. In addition to hypothesizing new concepts, our system must learn how they relate to one another. Often, these dependencies are probabilistic: for example, a person’s clothing choices depend probabilistically on the weather, and a whether a person knows French depends probabilistically on where that person grew up. There has been considerable work on learning the dependency structure of probabilistic graphical models [25]. However, just learning dependencies among individual variables — Laura’s clothing choices depend on the weather in New York, Mary’s choices depend on the weather in London, etc. — is not sufficient. Instead, our system must learn dependency models that apply to whole classes of objects, specifying how probabilistic dependencies arise from underlying relationships (such as the relationship between a person and the city where she lives). Learning such dependency models is the domain of a young field called statistical relational learning [26]. Considerable progress has been made on learning certain classes of relational dependencies [27,28], although learning models in highly expressive languages such as Bayesian logic [29] remains an open problem.

5. Algorithms Because we are taking a Bayesian approach, our proposed AGI system does make a strict separation between reasoning and learning: it learns by probabilistic inference. But of course, probabilistic inference can be very computationally expensive (a naive approach to answering a query would involve summing or integrating over all the unobserved, non-query variables). We have no silver bullet for this problem, but there are indications that performing inference on an AGI model is not as ridiculous as it may sound. The first thing to note is that we will be satisfied with approximate inference methods, even if they do not provide bounds on the quality of the approximation — we will be able to judge if we are doing well enough based on how well the system answers queries. The major approaches to approximate inference today are Markov chain Monte Carlo (MCMC) [30], mean-field-based variational methods [31,32], and belief propagation methods [33,34]. These methods can take hours to run on sets of just a few hundred documents or images, so one might object that using them on a large chunk of the web is infeasible. But given a hypothesis about the world history and the appearance and language-use sub-models, the individual videos and documents in our training set are conditionally independent of each other. Thus, the work of parsing these data items can be parallelized across a large set of machines. Perhaps the greatest challenge for inference in an AGI model is preventing the system from considering too many variables when evaluating a hypothesis or answering a query. For instance, consider trying to judge the ambient temperature in a street scene showing just one man, who is wearing a winter coat but no hat or scarf. In principle, a multitude of variables are relevant to judging the temperature: Is this person just stepping outside briefly, or is it warm enough that he can take a long walk in these clothes? What is his reason for being outside? Is he a local, or is he visiting from someplace warmer or colder? These variables are all within the purview of an AGI system — and indeed, with different data or a different query, it might be important to reason about them. But in this case, reasoning about them is bound to be unproductive: the system will not be

B. Milch / Artificial General Intelligence Through Large-Scale, Multimodal Bayesian Learning

253

able to conclude anything strong enough about these variables to influence the query result. Thus, the inference algorithm must be able to deal with partial hypotheses, and sum out the remaining random variables without reasoning about them explicitly. Existing algorithms can exploit situations where certain variables are strictly irrelevant given the hypothesized values of other variables [35]. There are also algorithms that exploit interchangeability among objects to sum out large sets of variables in a single step [36,37]. But dealing with queries like the one above will require novel methods for exploiting approximate irrelevance and approximate symmetry among variables. Another relevant line of research is the integration of probabilistic inference with logical reasoning. Many dependencies are nearly deterministic, and our system should be able to exploit near-determinism to speed up reasoning (by contrast, standard approximate inference algorithms tend to get stuck in local optima when determinism is present). One approach is to reduce probabilistic inference to weighted model-counting in propositional logic, and then exploit fast, deterministic SAT algorithms [38]. Another approach combines MCMC with SAT algorithms based on stochastic local search [39]. But for a full-blown AGI system, new inference algorithms are sure to be necessary. One positive side-effect of working with a theoretically simple Bayesian model is that the resulting inference problems can serve as fodder for the approximate inference community. It should even be possible to extract sub-networks from the AGI model to use as highly relevant benchmark problems.

6. Measures of Progress In order for Bayesian AGI to be viable as a research program, there must be some ways of evaluating its progress — short of seeing it pass a Turing test. One advantage of training on open-domain web data, as opposed to in an artificial or restricted environment, is that the system should quickly gain the ability to at least attempt open-domain tasks of practical interest. Several such tasks are used for evaluation in the computer vision and natural language fields, including the CalTech-256 object recognition data set [8], the MUC-6 noun phrase coreference data set 2 , the TREC question answering evaluation [40], and the PASCAL textual entailment challenge [41]. It is unlikely that the AGI system will immediately beat state-of-the-art but shallower approaches to these tasks. However, we should at least be able to measure our system’s performance and see it improving year by year. Another way in which a Bayesian AGI system could succeed early on is by becoming a resource for shallower but more finely tuned systems. For instance, some current natural language systems use WordNet [42] or Cyc [1] to define features for machine learning algorithms, or to implement one part of a multi-stage heuristic system. If our AGI system came to play a similar role and improved the performance of the systems that used it, we would be providing a useful service. Eventually, of course, we would hope for the AGI system to surpass shallower systems that used it as a resource. 2 http://cs.nyu.edu/cs/faculty/grishman/muc6.html

254

B. Milch / Artificial General Intelligence Through Large-Scale, Multimodal Bayesian Learning

7. Conclusion We have argued that Bayesian learning from large quantities of multimodal web data is the most feasible path toward artificial general intelligence. This position involves several strong claims: that an AGI system must learn most of its knowledge but should exploit built-in subsystems for visual, linguistic, and multi-agent reasoning; that learning from text alone is insufficient but text plus video is enough; and that Bayesianism provides an adequate theoretical foundation for AGI. These claims will be put to the test in a Bayesian AGI project. This paper is by no means a complete design for an AGI system. The process of designing the probabilistic model, and algorithms to do inference on it, promises to involve much trial and error over a decade or two. But there seems to be a good chance of success. Furthermore, the existence of such a project (or more than one!) would be a boon to the probabilistic AI community, serving as a source of motivating problems and a testbed for new techniques. A flourishing Bayesian AGI project would bridge the disturbing gap between the successes of probabilistic AI and the goal of understanding deep, general intelligence.

References [1] D. B. Lenat. CYC: A large-scale investment in knowledge infrastructure. Communications of the ACM, 38(11):33–38, 1995. [2] P. Singh, T. Lin, E. T. Mueller, G. Lim, T. Perkins, and W. L. Zhu. Open Mind Common Sense: Knowledge acquisition from the general public. In Proc. 1st Int’l Conf. on Ontologies, Databases, and Applications of Semantics for Large Scale Information Systems, 2002. [3] J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco, revised edition, 1988. [4] M. I. Jordan, editor. Learning in Graphical Models. MIT Press, Cambridge, MA, 1998. [5] E. Segal, B. Taskar, A. Gasch, N. Friedman, and D. Koller. Rich probabilistic models for gene expression. Bioinformatics, 17(Suppl 1):S243–252, 2003. [6] S. Thrun, M. Montemerlo, and A. Aron. Probabilistic terrain analysis for high-speed desert driving. In Proc. Robotics Science and Systems Conf., 2006. [7] M. Collins. Three generative, lexicalised models for statistical parsing. In Proc. 35th Annual Meeting of the Assoc. for Computational Linguistics, 1997. [8] G. Griffin, A. Holub, and P. Perona. Caltech-256 object category dataset. Technical Report 7694, California Institute of Technology, 2007. [9] B. Goertzel. Patterns, hypergraphs and embodied general intelligence. In IEEE World Congress on Computational Intelligence: Panel Discussion on “A Roadmap to Human-Level Intelligence”, 2006. [10] M. Banko, M. J. Cafarella, S. Soderland, M. Broadhead, and O. Etzioni. Open information extraction from the Web. In Proc. 20th Int’l Conf. on Artificial Intelligence, 2007. [11] W. Pentney, M. Philipose, J. Bilmes, and H. Kautz. Learning large-scale common sense models of everyday life. In Proc. 22nd AAAI Conf. on Artificial Intelligence, 2007. [12] C. P. Robert. The Bayesian Choice. Springer, New York, 2nd edition, 2001. [13] E. T. Jaynes. Inference, method and decision: Towards a Bayesian philosophy of science (book review). J. Amer. Stat. Assoc., 74(367):740–741, 1979. [14] M. Li and P. Vitanyi. An Introduction to Kolmogorov Complexity and Its Applications. Springer, 2nd edition, 1997. [15] J. Rissanen. Information and Complexity in Statistical Modeling. Springer, 2007. [16] C. Howson and P. Urbach. Scientific Reasoning: The Bayesian Approach. Open Court, La Salle, IL, 1989. [17] A. Gopnik and J. B. Tenenbaum. Bayesian networks, Bayesian learning, and cognitive development. Developmental Science, 10(3):281–287, 2007.

B. Milch / Artificial General Intelligence Through Large-Scale, Multimodal Bayesian Learning

255

[18] M. Hutter. Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer, 2005. [19] R. Fagin and J. Y. Halpern. Reasoning about knowledge and probability. J. Assoc. for Computing Machinery, 41(2):340–367, 1994. [20] D. Koller and B. Milch. Multi-agent influence diagrams for representing and solving games. Games and Economic Behavior, 45(1):181–221, 2003. [21] T. S. Ferguson. Bayesian density estimation by mixtures of normal distributions. In M. H. Rizvi, J. S. Rustagi, and D. Siegmund, editors, Recent Advances in Statistics: Papers in Honor of Herman Chernoff on His Sixtieth Birthday, pages 287–302. Academic Press, New York, 1983. [22] R. M. Neal. Markov chain sampling methods for Dirichlet process mixture models. J. Computational and Graphical Statistics, 9:249–265, 2000. [23] Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei. Hierarchical Dirichlet processes. J. Amer. Statist. Assoc., 101:1566–1581, 2006. [24] C. Kemp, J. B. Tenenbaum, T. L. Griffiths, T. Yamada, and N. Ueda. Learning systems of concepts with an infinite relational model. In Proc. 21st AAAI National Conf. on Artificial Intelligence, 2006. [25] D. Heckerman, D. Geiger, and D. Chickering. Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning, 20:197–243, 1995. [26] L. Getoor and B. Taskar, editors. Introduction to Statistical Relational Learning. MIT Press, 2007. [27] N. Friedman, L. Getoor, D. Koller, and A. Pfeffer. Learning probabilistic relational models. In Proc. 16th International Joint Conference on Artificial Intelligence, pages 1300–1307, 1999. [28] S. Kok and P. Domingos. Learning the structure of Markov logic networks. In Proc. 22nd International Conf. on Machine Learning, pages 441–448, 2005. [29] B. Milch, B. Marthi, S. Russell, D. Sontag, D. L. Ong, and A. Kolobov. BLOG: Probabilistic models with unknown objects. In Proc. 19th International Joint Conference on Artificial Intelligence, pages 1352–1359, 2005. [30] W. R. Gilks, S. Richardson, and D. J. Spiegelhalter, editors. Markov Chain Monte Carlo in Practice. Chapman and Hall, London, 1996. [31] M. Jordan, Z. Ghahramani, T. Jaakkola, and L. Saul. Introduction to variational methods for graphical models. Machine Learning, 37:183–233, 1999. [32] E. P. Xing, M. I. Jordan, and S. Russell. A generalized mean field algorithm for variational inference in exponential families. In Proc. 19th Conf. on Uncertainty in Artificial Intelligence, pages 583–591, 2003. [33] K. P. Murphy, Y. Weiss, and M. I. Jordan. Loopy belief propagation for approximate inference: An empirical study. In Proc. 15th Conf. on Uncertainty in Artificial Intelligence, pages 467–475, 1999. [34] J. S. Yedidia, W. T. Freeman, and Y. Weiss. Generalized belief propagation. In Advances in Neural Information Processing Systems 13, pages 689–695. MIT Press, Cambridge, MA, 2001. [35] B. Milch and S. Russell. General-purpose MCMC inference over relational structures. In Proc. 22nd Conf. on Uncertainty in Artificial Intelligence, pages 349–358, 2006. [36] D. Poole. First-order probabilistic inference. In Proc. 18th International Joint Conference on Artificial Intelligence, pages 985–991, 2003. [37] R. de Salvo Braz, E. Amir, and D. Roth. Lifted first-order probabilistic inference. In Proc. 19th International Joint Conference on Artificial Intelligence, pages 1319–1325, 2005. [38] T. Sang, P. Beame, and H. Kautz. Performing Bayesian inference by weighted model counting. In Proc. 20th AAAI National Conf. on Artificial Intelligence, pages 475–482, 2005. [39] H. Poon and P. Domingos. Sound and efficient inference with probabilistic and deterministic dependencies. In Proc. 21st AAAI National Conf. on Artificial Intelligence, pages 458–463, 2006. [40] E. M. Voorhees. Overview of TREC 2004. In The Thirteenth Text Retrieval Conference. NIST, 2004. [41] I. Dagan, O. Glickman, and B. Magnini. The PASCAL recognising textual entailment challenge. In Proc. PASCAL Challenges Workshop on Recognising Textual Entailment, 2005. [42] C. Fellbaum, editor. WordNet: An Electronic Lexical Database. MIT Press, 1998.

256

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

A computational approximation to the AIXI model Sergey PANKOV National High Magnetic Field Laboratory Florida State University Tallahassee, FL, 32306, USA Abstract. Universal induction solves in principle the problem of choosing a prior to achieve optimal inductive inference. The AIXI theory, which combines control theory and universal induction, solves in principle the problem of optimal behavior of an intelligent agent. A practically most important and very challenging problem is to find a computationally efficient (if not optimal) approximation for the optimal but incomputable AIXI theory. We propose such an approximation based on a Monte Carlo algorithm that samples programs according to their algorithmic probability. The approach is specifically designed to deal with real world problems (the agent processes observed data and makes plans over range of divergent time scales) under limited computational resources. Keywords. AIXI, Solomonoff induction, Kolmogorov complexity, Monte Carlo, algorithmic probability, control theory, intelligent agent

Introduction There exists a universally optimal theory of artificial intelligence (AI), the AIXI theory [1,2], which combines universal induction [3] with control theory [4]. The AIXI deals with optimal performance (in terms of maximizing its reward) of an agent within an environment in the context of unlimited computational resources. Computationally universally optimal approaches have also been discussed, such as AIXItl [1,2] (which is an approximation to AIXI) or the self-referential Goedel machine [5]. Both approaches are asymptotically optimal, meaning that no other method can possibly outperform (in terms of required computational resources) them in the long run, up to a non-universal additive constant, which may be huge in practice. Both approaches use search for a proof that would justify a particular choice of an agent’s strategy or a modification thereof. In the AIXItl the search algorithm is hard-coded, while the Goedel machine permits rewriting of its search algorithm itself, if a provably better algorithm is found. This ability to self-improve (rewrite any part of its code) may significantly reduce the above mentioned non-universal constant. One could think [5] of using the AIXItl model to initialize the Goedel machine (that is to use it as an initial problem solver of the Goedel machine), which will

S. Pankov / A Computational Approximation to the AIXI Model

257

eventually self-improve far beyond the initial AIXItl. However, a few legitimate practical questions arise (if one to have in mind complex real world environments), such as: how long might one wait before a significant self-improvement happens? Will not the hidden constants render that time impractically long? Or, should not the initial solver, for example AIXItl, be (at least) ”comparably intelligent” to its creator to begin with, if a significant self-improvement were to be expected in the foreseeable future? Therefore, giving up computational universality would be justifiable at the current stage, provided one gains in low computational complexity on a certain class of problems, deemed (subjectively) important to the field of AI, or artificial general intelligence (AGI). The real value of the AIXI theory is that it provides a prescription for optimal (fastest in the number of agent’s observations and actions) way of learning and exploiting the environment. This is analogous to how Solomonoff induction (which, like AIXI, is incomputable), gives a prescription for optimal (fastest in the number of observations) inductive inference. We, therefore, believe that any reasonable computational model of intelligence must recover the AIXI model in the limit of infinite computational resources. In this paper we present a particular Monte Carlo (MC) algorithm for computational implementation of Solomonoff induction. The algorithm samples candidate programs, that produce a desirable output, according to their algorithmic probability. The idea behind the MC algorithm is to make a Turing machine run ”in reverse”, from its output to the input program. This guarantees finding some candidate programs, even with very limited computational resources. The agent then uses the discovered programs to predict its possible future observations and corresponding rewards (with predictions possibly reaching far ahead in time). These predicted outcomes are used by the agent to choose its immediate actions, employing a generalized expectation maximization procedure. We argue that our approach reduces to the AIXI model in the limit of infinite computational resources. Finally we discuss what representation of a Turing machine should be most suitable for the approach to be applicable to the real world problems. The paper is organized as follows. In section 1, we introduce notations and basic concepts related to the universal induction and the AIXI. The main idea of our Monte Carlo algorithm is explained in section 2 in a simple setting of a conventional Turing machine (TM). In section 3, we generalize expectation maximization procedure to long term policies sampled probabilistically under limited computational resources. The general assumptions we make about possible environments relevant to AGI are discussed in section 4. In section 5, we outline a new TM representation, which is meant to accommodate the assumptions made in the previous section.

1. Basics of Solomonoff induction and AIXI model In this section we introduce notations and briefly overview basic theoretical concepts that are used in this paper.

258

S. Pankov / A Computational Approximation to the AIXI Model

1.1. Notations We use strings of symbols as an abstraction for various information transfer processes. Individual symbols in a string drawn from the alphabet X are denoted xi , with the subscript indicating their placement in the string. A string of length l + 1 may be written in one of the following ways: xm xm+1 ...xm+l , xm:m+l or implicitly x, if the subscript is not needed or its range is understood implicitly. The letter p ≡ p is reserved for a program string. In some cases we enumerate pairs of symbols (for example x1 y1 x2 y2 ...), then a shorthand notation may be used xyi ≡ xi yi , and correspondingly xyi:j = xyi xyi+1 ...xyj = xi yi xi+1 yi+1 ...xj yj . One should not confuse xy and xy: x1:m ym+1:n = xy while xy1:m = xy. The symbol ∗ represents any substring. The subscript in xi indicates string enumeration, not enumeration of individual symbols. The length of a binary representation of a string x is denoted l(x), so for X = {0, 1} one has l(x1:m ) = m. Probability of a string x1:m to be sampled from a probability distribution μ is denoted μ(x1:m ). Conditional probability for a string x1:n to occur given its first m < n symbols x1:m is defined as μ(x1:n |x1:m ) ≡ μ(x1:n )/μ(x1:m ). 1.2. Solomonoff induction Consider the problem of predicting the symbol xi given the knowledge of previous i − 1 symbols x1:i−1 in a string sampled from some distribution. The Bayesian inference solves this problem, assuming that the prior probabilities of possible distributions generating the string are known. The problem of choosing the prior probabilities in an optimal way is solved [3] by the universal prior, also called Solomonoff prior. Solomonoff’s universal prior M (x) is defined as: M (x) =



2−l(p)

(1)

p:U (p)=x∗

The summation is over all possible programs p, which output strings starting with x when run on a universal Turing machine U (see section 2). The inference with the universal prior is called Solomonoff induction (or universal induction). One of the central properties of universal induction is that the (expected) number of incorrectly predicted bits is bounded by the Kolmogorov complexity K(ρ) of the generating distribution function ρ [6,7], (the Kolmogorov complexity [8] is defined, loosely speaking, as the shortest possible description of an object one can produce using a universal Turing machine - mathematical formalization of a general purpose computer). The universal prior can be equivalently presented [2] as a sum over all (semi-)computable (semi-)measures (distribution functions), weighted by their algorithmic probability (∼ 2−K ): M (x) ∼ ξ(x) =

 ρ

2−K(ρ) ρ(x)

(2)

S. Pankov / A Computational Approximation to the AIXI Model

259

1.3. AIXI All or nearly all formalizable problems in the AI field can be cast in a form of an agent interacting with an environment and seeking to maximize its reward. The agent interacts with its environment in cycles: in ith cycle the agent receives a signal xi from the environment, processes it, then sends a signal yi (also called action) to the environment. The subscript i is also referred to as time. With every signal xi comes a reward v(xi ). Assume for now that the agent is in the beginning of the ith cycle and wants to maximize its expected reward for the next k cycles ahead in the environment described by a known measure μ [4]. To compute the maximized expected reward Vik∗ (counting the reward from cycle i to i + k), we introduce an auxiliary function Vik+ defined recursively as:  (3) μ(xy1:j−1 xj |xy1:j−1 ) max Vik+ (xyi:j ) Vik+ (xyi:j−1 ) = xj

yj

j=i+k v(xj ). Then Vik∗ (xi ) = where i < j ≤ i + k and Vik+ (xyi:i+k ) = j=i maxyi Vik+ (xyi ). If the measure μ is unknown, the AIXI’s prescription is to substitute it with ξ defined in Eq.(2).

2. Monte Carlo algorithm for a ”conventional” Turing machine By a conventional Turing machine we imply a machine described below. This should be contrasted with an effective representation of a Turing machine, discussed in section 5. A Turing machine [9] is a computational device consisting of a read/write head and an infinite tape (or possibly several tapes). The head assumes some state s ∈ S and is capable of simple actions, like (reading)writing a symbol x ∈ X (from)onto the tape, moving left or right, halting. The number of states |S| and the size of the alphabet X are finite. The machine’s computation is governed by some fixed rules F : X × S → S × A, which defines a step of TM computation as follows. In a state s the machine reads a symbol x, maps the pair (x, s) to (s′ , a) according to F , assumes new state s′ and performs the action a ∈ A. A machine capable of simulating any other machine is called universal and is denoted U . In Solomonoff/AIXI theory one needs to sample programs p, producing a desired output x∗ = U (p), according to the probability 2−l(p) . In the following we devise a Monte Carlo algorithm for computing time-bound universal prior Mτ (x), which is constructed analogous to M (x) (Eq.1) but from programs whose runtime does not exceed τ steps of a Turing machine computation. Note that Solomonoff prior in Eq.(1) is defined via a monotone machine U (see section 3). In a simplified example considered in this section the TM will not be monotone, and only halting programs will be considered. Therefore, the sampled probability will not quite correspond to Eq.(1), and we will consider a single tape machine for illustrative purpose only. However, in the next section we will use monotone machines and will include non-halting computations in consideration, thus restoring exact correspondence to Solomonoff prior.

260

S. Pankov / A Computational Approximation to the AIXI Model

101000000000 ............ 111111111011 111111111001 111111111101 111111111111 Figure 1. Figure represents a computation of a hypothetical single tape Turing machine, outputting a string of 1s. Each line shows content of the tape as computation progresses (from top to bottom). The transparent box represents the TM’s head. The program length l(p) = 3. Only the first and a few concluding steps are shown. The machine halts in the last line.

For simplicity we consider a single tape machine with X = {0, 1}. The string xτ represents the content of the tape in the τ th step of machine’s computation. An action a = (xt , Δt) consists of writing xt in the head’s position t, then moving to a position t′ = t + Δt, where Δt = ±1, (then reading a new symbol x′t , that is understood implicitly). Fig.(1) shows a run of a hypothetical program on such a machine, outputting a string of 1s. The head is initially at t = 1 and in the last step τn it is at the end of the outputted string. The triplet C = (s, xτ , t) completely describes the Turing machine in the τ th step and is called its τ th configuration, (we may also indicate the step index explicitly, as Cτ ). We consider a version of a Markov chain Monte Carlo algorithm, known as Metropolis-Hastings algorithm [10,11]. Given a positive function P (C), the algorithm will sample TM configurations with relative probability P (C)/P (C ′ ), if run sufficiently long. Choosing P (C = (s, x, t)) = 2−l(x) ,

(4)

will then achieve our goal of sampling Cs with their algorithmic probabilities, that is 2−l(x) . In our hypothetical example in Fig.(1) the leftmost and rightmost 1s can be thought of as the string x delimiters, indicating the string’s beginning and end. The Monte Carlo procedure works as follows. Given a configuration C, a move to a configuration C ′ is proposed according to the distribution function (called ′ proposal density) Q(C ′ = (s′ , xτ , t′ )|C = (s, xτ , t)) defined as: ˜ ′ |C) Q(C Q(C ′ |C) =  , ˜ ′ |C) Q(C C′

˜ ′ |C) = Q(C ⎧  ′ τmax > τ ′ = τ + 1 > 1, F (xτt , s) = (s′ , a′ = (xτt , t′ − t)), ⎨ 1, for ′ τmax > τ = τ ′ + 1 > 1, F (xτt′ , s′ ) = (s, a = (xτt′ , t − t′ )), (5) ⎩ 0, otherwise,

where τmax is the limit on runtime of a program. The proposed move is then accepted with probability:

S. Pankov / A Computational Approximation to the AIXI Model

min



 P (C ′ ) Q(C|C ′ ) , 1 . P (C) Q(C ′ |C)

261

(6)

Proposing a MC move and then accepting or rejecting it constitutes a MC step. One can show that the prescribed MC procedure will sample C correctly provided: 1) P (C) is invariant under a MC step (follows directly from Eq.(6)), 2) the procedure is ergodic (in practice it means that any C and C ′ can be connected by a series of MC moves). The algorithm starts with C = (sh , xτmax = x, t = l(x)), where sh is the halting state and x is program’s desired output. It is assumed that the program starts at t = 1 and halts with the head over the last symbol of x. The rationale of the construction of Q(C ′ |C) as shown in Eq.(5) is that for any current Cτ the sequence of Cτ ′ for τ ≤ τ ′ ≤ τmax represents (as in Fig.(1)) a computation of the Turing machine outputting x. Thus for any Cτ with t = 1, corresponding xτ is a program p outputting x. These programs are sampled with frequencies proportional to program’s algorithmic probabilities. Notice  that the time-bound  universal prior can then be evaluated as Mτmax = 1/ 2l(p) M C , where ... M C is the averaging over the Monte Carlo samples. We would like to note that one can slightly redefine how a program is supplied to the TM, so that xτ in any configuration Cτ could be regarded as a program (and not only when t = 1). Such redefinition will affect the sampling probability by at most a factor of l(x). What one gains is that the MC simulation is guaranteed to produce a candidate program after any number of MC steps. This viewpoint of the algorithm is adopted in section 5. Our method should be contrasted with a conceptually very different approach of Schmidhuber [12,13], in which one also samples candidate programs according to their algorithmic probability for the purpose of inductive inference. In that approach one uses the speed prior - a time bounded version of the universal prior. The speed prior takes into consideration computational complexity of a program, by allocating computational resources (time) in proportion to its algorithmic probability. So a program p of length l(p) taking time t(p) to run is sampled with frequency ∝ 2−l(p) /t(p). This implies that the speed prior sampling (closely related to Levin’s universal search [14]) will take time O(2l(p) t(p)) to sample a program p. And although the approach is optimal for finding the simplest program (here, simplest = minimal l(p) + log2 t(p)), the search time becomes prohibitively large if the simplest program producing a desired output is not short. Similarly to the speed prior sampling, our MC method will also sample short programs more frequently. The difference is that the correct algorithmic probability is only guaranteed in the long run, because the output string x may be much longer than the program p outputting it. In such a case the initial configurations C will be highly unlikely in the long run of the MC. This initial stage of sampling unlikely configurations can be viewed as a search for short programs (or at least sufficiently short; here and throughout the paper ”sufficiently short” = sufficiently short for discovering interesting regularities), dominating the sampling in the long run. The MC search, which is not universally optimal, in certain cases will quickly find a sufficiently short program. In which cases this is possible will depend on ruggedness of the search landscape, that is on how much l(xτ ) varies

262

S. Pankov / A Computational Approximation to the AIXI Model

along the MC chains (chain = sequence of MC steps). The crucial idea is that the landscape depends on the choice of the Turing machine. We will hypothesize that there is a particular machine choice that makes the MC search very efficient on an important class of problems.

3. Policy computation We will not discuss computational efficiency of our approach in this paper, we only notice that it is expected to work fast in certain environments described in section 4. For now we simply assume that the MC succeeds in finding sufficiently short programs outputting the data observed by the agent. A ubiquitous problem faced by the agent is the exponentially large space of states. Only an exponentially small fraction of the states can be sampled in practice when computing the agent’s policy. And because in a typical environment of interest (see section 4) an exponentially small fraction of the most probable states have a combined probability close to 1, there should be a strong bias toward those states. It is easy to check that to achieve the quickest convergence in averaging some function φ(x) over the states x distributed according to P (x), one ¯ (x))2/3 , where φ¯ is the should sample the states with probability ∝ (|φ(x) − φ|P expectation value of φ(x). In our problem we only know how to efficiently sample ∝ P (x) (as will be shown below), but this still gives (typically exponentially) faster convergence than with the uniform sampling. Generally, the strategy of favoring samples that potentially contribute the most to a quantity of interest, is called importance sampling. The AIXI agent essentially uses Solomonoff induction to predict future and computes its expected reward from that prediction. The universal prior (Eq.(1)) is defined in terms of a (universal) monotone Turing machine [8]. A monotone machine has a unidirectional (unidirectional = head moves in one direction only, to the right) input and output tapes, which are used for reading a program and writing the output of a computation, respectively. The machine also uses internal tapes for computations. The time bounded universal prior Mτ (x) is now defined in terms of minimal programs (minimal program = input string to the left of the input head when the last symbol of the output string is being printed) that output x within the time bound τ . Consider the problem of sampling extensions y of length l of a given string x with probability Mτ (xy|x). The solution is obvious: first sample a minimal program p outputting x according to its algorithmic probability, then use fair coin flips to extend the program as far as needed to output l symbols more (within the time bound). How can this be realized within our MC procedure? First, one generalizes the MC to monotone machines. This is straightforward, we just need to consider more complex configurations C = (s, x1 × ... × xn , t1 × ... × tn ) for an n-tape machine, and choose rules F so that the monotone machine is represented appropriately. Second, the MC should start with a non-halting state, so the discovered programs would not halt upon printing x. When computing agent’s policy via Vk+ (see Eq.(3)), one again faces a problem of exponential growth (here, the growth of the number of possible policies

S. Pankov / A Computational Approximation to the AIXI Model

263

with k), which seems to render planning impossible for any but very small k. To deal with this problem we propose to use an importance sampling procedure. In particular, the agent should be learning its own actions the way it learns the environment through universal induction. To compute its policy the agent now samples possible policies that are most likely. It then performs the reward expectation maximization on these sampled policies. This procedure is explained below in detail. Notice then, that in the limit of infinite resources the exact expectimax result (as prescribed by Eq.(3)) is recovered. As we have said, the agent’s input and output signals xi and yi are now learned on equal footing. The Solomonoff prior is thus defined on the strings xy, (and not just on strings of x, conditioned on y in chronological manner, see [2]). We shift the agent’s cycle index so that it is non-negative integer on future symbols, and negative on its history (the history = observed symbols, is denoted xy m'. That is, pf learns to predict e. Now we can construct an evader ef using the program that implements pf modified to complement the binary symbols it writes to its output tape. The proof that ef learns to evade all predictors in Pf is the same as the proof that pf that learns to predict all evaders in Ef, with the obvious interchange of roles for predictors and evaders. ฀ This tells us that in the adversarial sequence prediction game, if either side has a sufficient advantage in computational resources to simulate all possible opponents then it can always win. So the game can be interpreted as a computational resources arms race. Note that a predictor or evader making truly random choices of its output symbols, with 0 and 1 equally likely, will win half the rounds no matter what its opponent does.

402

B. Hibbard / Adversarial Sequence Prediction

But Proposition 1 tells us that an algorithm making pseudo-random choices will be defeated by an opponent with a sufficient advantage in computing resources.

3. Software Experiments Adversarial sequence prediction is a computational resources arms race for algorithms using unrealistically large computational resources. Whether this is also true for algorithms using more modest computational resources can best be determined by software experiments. I have done this for a couple algorithms that use lookup tables to learn their opponent's behavior. The size of the lookup tables is the measure of computational resources. The predictor and evader start out with the same size lookup tables (a parameter can override this) but as they win or lose at each round the sizes of their lookup tables are increased or decreased. The software includes a parameter for growth of total computing resources, to simulate non-zero-sum games. Occasional random choices are inserted into the game, at a frequency controlled by a parameter, to avoid repeating the same outcome in the experiments. The software for running these experiments is available on-line [8]. Over a broad range of parameter values that define the specifics of these experiments, one opponent eventually gets and keeps all the computing resources. Thus these experiments provide evidence that adversarial sequence prediction is an unstable computational resources arms race for reasonable levels of computational resources. Interestingly, the game can be made stable, with neither opponent able to keep all the resources, by increasing the frequency of random choices. It is natural and desirable that simple table-lookup algorithms should be unable to predict the behavior of the system's pseudo-random number algorithm. But more sophisticated algorithms could learn to predict pseudo-random sequences. The adversarial sequence prediction game would make an interesting way to compare AGI implementations. Perhaps future AGI conferences could sponsor competitions between the AGI systems of different researchers.

4. AI Ethics Artificial intelligence (AI) is often depicted in science fiction stories and movies as a threat to humans, and the issue of AI ethics has emerged as a serious subject [9, 10, 11]. Yudkowsky has proposed an effort to produce a design for AGI whose friendliness toward humans can be proved as it evolves indefinitely into the future [12]. Legg's blog includes a debate with Yudkowsky over whether such a proof is possible [13]. Legg produced a proof that it is not possible to prove what an AI will be able to achieve in the physical world, and Yudkowsky replied that he is not trying to prove what an AI can achieve in the physical world but merely trying to prove that the AI maintains friendly intentions as it evolves into the indefinite future. But intentions must be implemented in the physical world, so proving any constraint on intentions requires proving that the AI is able to achieve a constraint on the implementation of those intentions in the physical world. That is, if you cannot prove that the AI will be able to achieve a constraint on the physical world then you cannot prove that it will maintain a constraint on its intentions.

B. Hibbard / Adversarial Sequence Prediction

403

Adversarial sequence prediction highlights a different sort of issue for AI ethics. Rather than taking control from humans, AI threatens to give control to a small group of humans. Financial markets, economic competition in general, warfare and politics include variants of the adversarial sequence prediction game. One reasonable explanation for the growing income inequality since the start of the information economy is the unstable computational resources arms race associated with this game. Particularly given that in the real world algorithm quality is often an important computational resource. As the general intelligence of information systems increases, we should expect increasing instability in the various adversarial sequence prediction games in human society and consequent increases in economic and political inequality. This will of course be a social problem, but will also provide an opportunity to generate serious public interest in the issues of AI ethics.

References [1]

[2] [3] [4] [5] [6] [7] [8] [9] [10]

[11] [12] [13]

J. Schmidhuber. The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions. In J. Kivinen and R. H. Sloan, editors, Proceedings of the 15th Annual Conference on Computational Learning Theory (COLT 2002), Sydney, Australia, Lecture Notes in Artificial Intelligence, pages 216--228. Springer, 2002. http://www.idsia.ch/~juergen/coltspeed/coltspeed.html Hutter, M. Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability. Springer, Berlin, 2004. 300 pages. http://www.idsia.ch/~marcus/ai/uaibook.htm Hutter, M. and S. Legg. Proc. A Formal Measure of Machine Intelligence. 15th Annual Machine Learning Conference of Belgium and The Netherlands (Benelearn 2006), pages 73-80. http://www.idsia.ch/idsiareport/IDSIA-10-06.pdf Legg, S. Is there an Elegant Universal Theory of Prediction? Technical Report No. IDSIA-12-06. October 19, 2006. IDSIA / USI-SUPSI Dalle Molle Institute for Artificial Intelligence. Galleria 2, 6928 Manno, Switzerland. http://www.idsia.ch/idsiareport/IDSIA-12-06.pdf Lloyd, S. Computational Capacity of the Universe. Phys.Rev.Lett. 88 (2002) 237901. http://arxiv.org/abs/quant-ph/0110141 Wang, P. Non-Axiomatic Reasoning System --- Exploring the essence of intelligence. PhD Dissertation, Indiana University Comp. Sci. Dept. and the Cog. Sci. Program, 1995. http://www.cogsci.indiana.edu/farg/peiwang/PUBLICATION/wang.thesis.ps http://en.wikipedia.org/wiki/Game_theory http://www.ssec.wisc.edu/~billh/g/asp.html Hibbard, W. Super-Intelligent Machines. Computer Graphics 35(1), 11-13. 2001. http://www.ssec.wisc.edu/~billh/visfiles.html Bostrom, N. Ethical Issues in Advanced Artificial Intelligence. Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, Vol. 2, ed. I. Smit et al., Int. Institute of Advanced Studies in Systems Research and Cybernetics, 2003, pp. 12-17. http://www.nickbostrom.com/ethics/ai.html Goertzel, B. Universal Ethics: The Foundations of Compassion in Pattern Dynamics. October 25, 2004. http://www.goertzel.org/papers/UniversalEthics.htm Yudkowsky, E. (2006) Knowability of FAI. http://sl4.org/wiki/KnowabilityOfFAI Legg, S. Unprovability of Friendly AI. September 15, 2006. http://www.vetta.org/?p=6

404

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

Artificial General Intelligence via Finite Covering with Learning Yong K. Hwang, Samuel B. Hwang and David B. Hwang Think Life, Inc. Palo Alto, CA, USA

Abstract. This position paper claims that the combination of solutions to a finite collection of problem instances and an expansion capability of those solutions to similar problems is enough to achieve the artificial general intelligence comparable to the human intelligence. Learning takes place during expansion of existing solutions using various methods such as trial and error, generalization, case-based reasoning, etc. This paper also looks into the amount of innate problem solving capability an artificial agent must have and the difficulty of the tasks the agent is expected to solve. To illustrate our claim examples in robotics are used where tasks are physical movements of the agent and objects in its environment. Keywords. human intelligence, machine learning, knowledge expansion, robotics

Introduction Achieving human-level intelligence has been an illusive goal due to the vast amount of knowledge humans build up and use to solve many different tasks. Since it is impossible to preprogram solutions to all the problems an agent will face in the real world, many attempts are made to endow the agent with learning capabilities so that it can increase its knowledge base from a relatively small knowledge base. This approach has some success in a well controlled environment, but cannot yet handle complex problems arising in the real-world situations like humans can. Although humans can solve many problems, they are functional only in familiar environments. If a human is put in a world with irregular gravity he will have a hard time navigating the world. It is our claim that the kinds of worlds humans can function efficiently are not that many compared to the set of all possible variations of the world. From this claim follows that a successful AGI can be built with a finite number of experiences. A positive (negative) experience is a pair of a problem and a successful (unsuccessful) action for it. One more requirement is capability of generalizing or expanding an experience to similar situations. The resulting knowledge base is then capable of handling a subset of all possible worlds, where the subset is a union of blobs around the points representing positive experiences minus the blobs representing negative experiences (see Fig. 1). This subset is a finite covering of the world in which the AGI system can function well to suit its purposes and avoid failures. This paper concentrates on the task planning in the physical world, which occupies a middle of the intelligence hierarchy [1]. We now present the process of building up a knowledge base by expansion capability to generate a finite covering of the world. Examples in robotics are used to illustrate this process.

Y.K. Hwang et al. / Artificial General Intelligence via Finite Covering with Learning

405

Fig. 1. The knowledge set for solving problems is the union of neighborhoods of positive experiences while the knowledge about failure is that of negative experiences.

1. Knowledge Representation Our agent starts out with an innate knowledge, and expands it via generalization and learning. Our main claim is that an adequate knowledge base for agent’s well being can be built as a finite, i.e., manageable, union of problem-solution pairs and that of problem-failure pairs. A problem, q, is defined as the pair of agent’s current and desired states of the agent and its environments, q(e_current, e_desired). A positive experience instance, i_p is a pair of a problem and an agent’s action resulting in a success, written as i_p(q, a_p). Similarly, a negative experience instance is written as i_n(q, a_n). When the agent has a capability to recognize a problem, q~, similar to q and a capability to modify the action a_p to solve q~, the set of problems near q solvable by the agent form a set covering q, written as Cp(i_p). Similarly, the set of problem instances around i_n resulting in an undesirable outcome is called Cn(i_n). Let Kp and Kn be the knowledge sets of the positive experience and negative experience, respectively. Kp (Kn) is a union of Cp’s (Cn’s). Then the set of problem instances solvable by the agent with 100% confidence is Kp – Kn. If we are willing to accept some failure, the agent might try to solve problems belonging to both Kp and Kn. Let us give an example. In robotics field, two of the fundamental tasks are robot navigation and object manipulation. In navigation, the problem is to find a short, collision-free path from the current configuration of the robot to the desired configuration. The e_current (e_desired) is specified by the current (desired) robot configuration along with configurations of all objects. In manipulation, e_current (e_desired) is specified by the current (desired) configuration of the object to be manipulated along with those of nearby objects. Suppose the robot has a path-planning algorithm, a_pathplan to find a collision-free path. An arrangement of objects in its environment and robot’s start/goal configuration form a problem instance q(e_start, e_goal). If a_planpath successfully finds a solution path, then the pair (q, a_pathplan) forms a positive experience, i_p. Otherwise, it forms a negative experience, i_n. If in the same environment an object irrelevant to the solution path is moved away from the path, then a_palnpath will succeed again. Then the new experience forms a part of the positive covering around i_p, namely Cp(i_p).

406

Y.K. Hwang et al. / Artificial General Intelligence via Finite Covering with Learning

2. Expansion of Knowledge As an agent solves problems, it expands its knowledge base in several ways. The trialand-error method is useful when the penalty for failure is small. Another method is to learn from a teacher, whether a human or other agents. Other machine learning algorithms can be used to expand knowledge [2]. These processes increase the sets of positive and negative experiences whose sizes are of measure zero, i.e., they are just points in the space of all problems. If we have some local expansion capability, the agent can grow the point experiences into sets of finite measures, vastly increasing the size of the problems it can solve. This expansion capability enables us to claim that not infinite, but only a finite number of experiences are enough for agent’s successful life. The increase of knowledge occurs in two categories: by solving new problems, and by expanding an existing piece of knowledge for a particular problem to similar problems. Let’s call these global and local learning. These are elaborated below. 2.1. Global Learning Simply put, a global learning takes place when the agent learns a brand new stuff. Suppose the agent learns to use a knife to cut objects, which it does not know how before. This is a kind of global learning. Another example is learning to walk in a stair case when the agent could walk only on a flat floor. When a global learning occurs, inserted in the knowledge base is a new pair, (p, a), of a problem, p, represented as the current and desired environmental and an algorithm, a, to solve the problem. A global learning can be achieved by various means. A trial-and-error method is simple but may have a low success rate. A more prudent way is to learn from a teacher, which can be humans or observation of successful task completion by other agents. Another method is divide-and-conquer scheme which decomposes a task into subtasks that are solvable by existing algorithms. Learning the category of grasp such as power grip or precision grip [3] can also be classified as global learning. In terms of our notation the global learning increases the number of coverings Cp’s or Cn’s in the agent’s knowledge base.

2.2. Local Learning A local learning occurs when a new problem is solved by an existing algorithm with substitution of input parameters. For paper cutting task, for example, if the agent learns to use the algorithm of using scissors for cutting a string then a local learning has taken place (substitution of object). Local learning can be achieved by various methods. A trial-and-error method can be used to apply an existing algorithm to tasks with different objects. Or the case-based reasoning can be used to recognize similar situations and the agent can use the action that succeeded. Local learning increases the size of covering, Cp or Cn. If the algorithm solves (fails to solve) the problem, the size of Cp (Cn) is increased. The key difference between global and local learning is whether a new algorithm is introduced in the knowledge base. 2.3. Quantum Learning The most challenging type of learning is the automatic generation of new algorithms based on existing knowledge. It is at a higher level of learning than global learning, and

Y.K. Hwang et al. / Artificial General Intelligence via Finite Covering with Learning

407

for this reason we call it quantum learning. Suppose our agent know about depth-first and breath-first search algorithms. How do you make the agent generate A* search on its own? We have to somehow embed in the knowledge the principles of “best first” exploration, “possible option” retention and “hopeless option” deletion. One way of quantum-learning is maintaining many small knowledge pools as in Society of Mind [4], where a particular combination of pools forms a new algorithm. This is not in the scope of our paper, and we are just pointing out the difficulty of quantum learning.

3. Complexity of Building Artificial General Intelligence We explore the amount of knowledge required to build human-level AGI. We don’t claim our list to be anywhere near a complete set, but it will give a good starting point for necessary components required by AGI systems. We first give a set of physical principles that reduces the amount of required innate knowledge and the complexities of the algorithms for problem solving. Presented next is the minimal requirement on the components of the innate knowledge from which to build a comprehensive AGI. 3.1. Physical Principles We have come up with several principles based on the physical laws that can be used to simplify the complexity of AGI. They are proximity-in-space (PIS), proximity-in-time (PIT) and benign-environment (BE) principle. The PIS was used in [5] for generating natural-language-like robot commands. These principles are explained in detail below. Let us explain by analogy how PIS makes problems in the physical world simpler in practice. A lone ninja fighting against many enemy fighters needs to deal with only 4 or so enemies at a time because the rest of enemies cannot get to the ninja. By the same token, although it is theoretically true that all objects need to be considered when planning collision-free motion in navigation or manipulation tasks, only several nearby relevant objects need to be taken account. Nailing two pieces of wood together requires consideration of only the wooden pieces, a nail and a hammer. The PIT assumes that events relevant to the current problem occur in proximal time unless explicitly specified by some means. One of the basic assumptions in human reasoning is that things do not change too fast. If they do, for example, such as in a car accident, humans also make mistakes. When planning an action, it is prudent to consider events occurred in the near past or going to happen in the near future with more emphasis. To solve a crime, for example, detectives investigate people who visited the crime scene near the time of crime. The PIT, therefore, can greatly simplify the complexities of planning algorithms necessary for human-level intelligence. The BE asserts that the environment is mostly benign and is not there to make the agent fail. Humans set up their habitat with favorable conditions of weather, food resources, etc. Furthermore, they build their houses so that everyday tasks can be performed with ease. We also assume objects are safe to interact with unless explicitly specified, such as “fire is hot.” Also, most knives are made with a handle to facilitate safe grasp. The default assumption of safety greatly reduces the amount of knowledge to be stored. This is true for many other object properties such as strength, weight, etc. Another example is the existence of hallways in buildings to expedite navigation. Yet another example is the fact that most tasks humans do can be performed with one or two hands. The development of AGI, thus, should be easier for manmade environments.

408

Y.K. Hwang et al. / Artificial General Intelligence via Finite Covering with Learning

3.2. Components and Size of Knowledge Base Humans are visionaries, i.e., their primary sensor is the eyes. To build an agent that interacts with humans, the agent must have an ensemble of good vision algorithms. Tactile sensing is of a great help in object manipulation. To recognize similar situations and similar objects, a similarity recognizer is needed. To detect changes in objects or environments, a difference operator is needed. Search algorithms are also required to find optimal solutions to various tasks. They include findpath, findplace (to put an object), findgrasp (to grab an object) to name a few. To predict the effect of an action during task planning, a qualitative simulator [6] is also necessary. Also needed are a database to store object properties and many other information such as pre- and post conditions of actions, goodness evaluation functions of situations/actions, and so on. Now the important question is “how big is the knowledge base that is sufficient for AGI?” For home-service robots, 1000 actions seem sufficient [7]. The number of objects involved in an action is small, like 4, according to BE. Given a task, the action planning process involves a search in the space of agent actions. If we provide a template of an action sequence for each task, a search is triggered to find an alternate action only when one of the actions in the task template results in a failure. One of the main problems with the traditional reasoning system is the combinatorial explosion. A counter argument to this is the small-world phenomenon or six degrees of separation. It states that there is a relatively short chain of acquaintances to link two people in the world. By analogy, we claim that there is a short chain of actions to go from the current to a desired situation. Or relevant objects to solving a problem are linked to the agent or current situation in a few levels of depth in the search tree for a situation.

4. Conclusions We claim in this paper that there is a hope for building a human-level AGI. This is conjectured from the fact that 1) even humans are not functioning effectively in an unfamiliar environment, and 2) by employing knowledge expansion methods a finite number of experiences can be beefed up to solve enough problems for an artificial agent. It remains to be seen that our claim is true by actually developing an AGI system. It is our belief that a system with human-level AGI is in the near future.

References [1] M. Minsky, P. Singh, and A. Sloman, The St. Thomas common sense symposium: designing architectures for human-level intelligence, AI Magazine, Summer 2004, 25(2):113-124. [2] T. Mitchell, Machine Learning, McGraw-Hill Education (ISE Editions), 1st edition, October 1997. [3] M.R. Cutkosky, “On grasp choice, grasp models and the design of hands for manufacturing tasks,” IEEE Trans. On Robotics and Automation, vol. 5, no. 3, pp. 269-279, June 1989. [4] M. Minsky, Society of Mind, Simon & Schuster, March 1988. [5] P.C. Chen and Y.K. Hwang, “Generating natural language-like robot motion commands through machine learning,” IEEE Int. Conf. on System, Man and Cybernetics, vol. 1, pp. 114-121, Oct. 1994. [6] B. Kuiper, 2001. Qualitative simulation. In R. A. Meyers (Ed.), Encyclopedia of Physical Science and Technology, Third Edition, NY: Academic Press. [7] Y.K. Hwang, M. Lee and D. Lee, “Robots’ role in the ubiquitous computing household environment,” Proceedings of the International Symposium on Robotics, Paris, France, March 2004.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

409

Cognitive Primitives for Automated Learning Sudharsan IYENGAR Winona State University, Winona, MN, USA [email protected] Abstract. Artificial Intelligence deals with the automated simulation of human intelligent behavior. Various aspects of human faculties are tackled using computational models. It is clear that brain as a cognitive machine is significantly different from a computer. A general intelligent machine needs to incorporate primitives that are identical or similar to those intrinsic to human. We argue for the identification, verification, and development of technologies that will deliver core primitives that are fundamental to human cognition that are applicable to various domains. Mnemonics for aural and for visual cognition are presented in [2, 3]. We propose a generalized set of cognitive primitives that can specialized to various applications viz. a) Existential, b) Structured Recurrence, c) Relative Attachment, d) Harmonized Counterparts, e) Cordial Counterparts and e) Discriminate Counterparts. Key words: Science of intelligence, brain model, cognition, primitives.

Introduction Science of Intelligence [1] involves the basic theory and systems that involve understanding brain structure, cognitive processes, artificial intelligence (computational) models, and others. Science of the brain explores the physical and analytical aspects of brain and aims to study the principles and model of natural intelligence at a molecular level. Cognitive Science studies human mental activity, such as perception, learning, memory, thinking, consciousness etc. In order to model and implement machine intelligence, artificial intelligence attempts simulation, extension and expansion of human intelligence using artificial machines – specifically using computers. These disciplines work together to explore new concepts, new theory, and new technologies and methodologies to create and implement models that can imitate human intelligence. What is learning? Cognition is a process of acquiring knowledge or skill. Acquiring and assimilation of information and knowledge is inherent to learning. But what constitutes knowledge and/or skill? As one assimilates information it is unclear if it is knowledge, relevant knowledge, or ability (skill) or pertinent skill. An alternate definition for learning can be: A lasting change in cognition resulting from experience, potentially directly influencing behavior. So what is cognition? Mental functions such as the ability to think, reason, and remember. Experiences include external inputs and its processing. Behavior is a processed response to an input. Thus we could say learning is a process through which external inputs enable acquiring knowledge and/or skill that affect the state of an organism such that its subsequent behavior may be altered. Many have addressed the ways and means by which humans receive external input. Aural and visual inputs are the ones that are studied significantly. Natural language processing addresses the methods and technology of recognizing sounds and applying linguistics to recognize information. These techniques are further applied for linguistic responses. Image processing and pattern recognition aim to recognize objects and concepts, from optical input and images, using stored entities and processing techniques. Machine learning is to enable a computer system to autonomously acquire and integrate knowledge. The capacity to learn from experience, analytical observation, and other means, is expected to result in a system that can self-improve and thus achieve better performance. The technology with which input is 'scanned', converted, and subsequently used varies with application and models used for storing and matching the resident knowledge. Applications use algorithmic methods that result in accurate or approximate identification. Stored knowledge use raw images and databases, Petri-nets, Bayesian networks, connectionist networks, and logical relational knowledge base. Various processing methods are used that integrate the content from the knowledge base and the rules used by experts in solving problems intelligently. Logic programming, production systems, neural networks, and other tools are used for computing so as to arrive at a desired solution. Techniques used vary from case-based reasoning, decision tree learning, meta learning, regression and statistical approaches.

410

S. Iyengar / Cognitive Primitives for Automated Learning

What has eluded researchers is the ability to completely and accurately embody software solutions with techniques so easy and natural to humans. And, every domain of application presents its own challenges – whereas humans are able to seamlessly handle various domains with ease. Additionally, humans tackle indeterminate and fuzzy input and still achieve great degree of dependable solutions. Applications software are 'trained' on test cases devised and labeled by humans, scored so as to estimate its usefulness, and then tested on real-world cases. The results of these real-world cases are inturn reflected upon by human developers to further train the software in an attempt to improve its effectiveness. We need to develop a framework that embodies human problem solving characteristics and then utilize those for developing ‘software’ solutions emulating intelligence. One could argue that newer technology and ‘software’ development methods different than current one may need to be developed. So be it. 1. Cognition: How, Why and What? We believe that in order to enable cognition we need pertinent set of core primitives. Every living organism is enabled with basic primitives. The usage and application of these with various degrees of flexibility, interdependence, and variations achieve cognition and which affect subsequent response(s). Cognitive functions are thus the result of concerted application of these core primitives. Learning requires cognition as a primary step. Different organisms possess various cognitive abilities and various levels of these abilities. This in-turn is intrinsically involved with the input-output mechanism enabled in the organism. Cognitive abilities involve aural and visual input and integrate these with analytical processing of this input. Language, music, and other forms of sound based communication use aural aspects. Input is received through the ear (or other forms of sound capture) and output could be voice generated. A systematic methodology for generating aural chants is proposed and shown capable of generating a variety of ‘musical’ sequences in [4]. Object, face, and scene recognition (static or dynamic) require visual input (eyes). A simple visual perception model is presented in [5]. Similar model is used in [6]. Response may be aural and/or physical - movement reconfiguring of self. Cognition of analytical concepts involves a re-evaluation and re-organization of internal knowledge to arrive at a verifiable and applicable new knowledge. The primary aim of cognition in humans and animals is to effectively interact, communicate, and create. Primal instinct as well as intended actions enable an organism to interact and communicate. Intelligence - or semblance thereof - is firmly a consequence of what is cognizant to the organism and what has been assimilated through its experiences. Additionally, the ability of the organism to inclusively, and effectively, utilize the imbibed knowledge towards intended communication and interaction reflect its intelligence. 2. Need for Intelligence Primitives In May 1997, IBM's Deep Blue Supercomputer played a fascinating match with the reigning World Chess Champion, Garry Kasparov. IBM scientists have taken pains to emphasize that Deep Blue is just a glorified calculator. Whether a machine like Deep Blue, which combines lightning-fast search power with a large database of knowledge, can be said to ‘think intelligently’ depends on one's philosophical inclinations. From the perspective of a Turing test - Deep Blue could be construed to be intelligent. But the methods and techniques used in playing the game of chess by Kasparov and by Deep Blue were far from being similar. Deep Blue utilized brute computational methods involving symbolic storage and matching algorithms. It could manipulate a given input, from the move made by Kasparov, and suggest an output which was then transmitted to the game. Kasparov on the other hand recognized a move on the board, and arrived at the next move based on experience, abstracted and short-cut techniques, and other pre-planned moves. How do we define intelligence and how can we decide who or what has it? This remains among science's unsolved and possibly unsolvable, problems. The model and the methods applied by Kasparov significantly differ from the techniques used by Deep Blue. What are those? Are they made up of big blocks of intelligent capabilities or are they made up of simpler elements collectively reflecting higher order capability? We believe that cognition, as well as intelligence, is a result of a bottom-up constructivist model in which a core set of capabilities are integrated into different models so as to achieve higher cognitive

S. Iyengar / Cognitive Primitives for Automated Learning

411

capabilities. A simple analogy is the use of Boolean primitive constructs (AND, OR, and NOT) to create higher order artifacts and circuits, and ultimately a microprocessor. An enormous amount of effort by a generation of engineers and scientists has contributed to this technological marvel. So also, we believe that a versatile and capable intelligent machine can be built only from ground-up using core cognitive primitives – implementable using current or newer technologies. Cognition, and learning, occurs in two modes. Incidental cognition, and learning, is an involuntary phenomenon where, through innate ability, an organism involuntarily receives input and assimilates this information. When we passively listen to a song and are then able to recognize it later - we have undergone incidental cognition. Incidental cognition might occur numerous times. Significant amount of cognition occurs this way throughout the life of an organism. Intentional cognition is a process consciously applied involving input and a deliberated output for verification and authentication. Repetitive with corrective action is typical in intentional cognition. When we learn to write a letter or symbol or learn to play catch we employ intentional cognition. In either case the cognitive process is incremental in nature. One recognizes a piece of a song after having recognized, multiple times, smaller parts of that song. Finally the entire song is recognized as an aggregation of the smaller pieces. Cognition utilizes the ability of recognize 'known' elements and its 'presence' in a larger context. We propose a hierarchy of cognitive capabilities that can be utilized constructively to develop higher order capabilities. These primitives do not define a complete set needed for constructing intelligent systems. But any system that needs to be constructed should be based on capabilities, constructed using the primitives, pertinent to the intended domain.

3. Cognitive Primitives We propose generic mnemonic or cognitive capabilities. Hierarchical, inclusion of and use of these primitives pertinent to different domains can enable systems/organisms to imbibe and recognize elements from its input. Definition 1: An element or item, i, is an input that is aural, visual, or physical. Repeated input of the item to the system results in the system imbibing it. The number of repetition can be system specific and once a threshold number of repetition occurs, the item is assimilated for further use. An input is recognized if it matches an already assimilated item matched by a process of decomposition. Once an item has been abstracted (imbibed) into the consciousness (or knowledge base) of a system, subsequent recognition of this item is atomic. Definition 2: A component, c, is a composition of atomically recognizable elements. Input to the system can be a complex combination or steam of elements. Recognition of such complex input is a result of recognition of its elements and their combinations. Definition 3: The granularity of a component c, g(c), is the number of elements used in recognizing it through decomposition to match known elements. Definition 4: The size of a component c, z(c), is the minimum granularity of c in recognizing it. Definition 5: The expressiveness of a component c, x(c), is the set of elements used in recognizing it. Definition 6: The shape of a component, c, is the minimum x(c) of c, and is denoted s(c). Definition 7: Two components, C1 and C2, are said to be balanced iff s(C1) = s(C2). Definition 8: Two components, C1 and C2, are said to be discordant iff. s(C1) != s(C2) and s(C2) ŀ s(C2) != {}. Definition 9: Two components, C1 and C2, are said to be distinct iff s(C2) ŀ s(C2) = {}.

412

S. Iyengar / Cognitive Primitives for Automated Learning

We first propose primitives that enable a system to recognize elements and recognize simple organization of known elements. Primitive 1 - Existential (E): The ability to be cognizant of the existence of an item. Primitive 2 - Structured Recurrence (SR): The ability to recognize a simple recurrence of an item. Primitive 3 - Relative Attachment (RA): The ability to recognize relative juxtaposition of two items. E(i) implies an atomic cognition of i using E. SR(i) implies the atomic cognition of a cogent recurrence of i. For aural input the recurrences are beat synchronized recurrence. For visual input the recurrences are simple spatial adjacencies. RA(i,rj, j) implies the recognition that items i and j are placed per rj - a specified order of connection or join. For aural inputs rj is a specific time ordered sequencing. For visual input it can be a specific relative spatial placement of the items. E, SR, and RA are fundamental primitives that recognize inputs based on elements that are atomically recognized without resorting to partitioning of input into various components. When input is complex then these can possibly be recognized as an arrangement of components that can be recognized individually. Components may be a result of simple or complex combinations. Primitives or recognizing the combinations are proposed as. Primitive 4 - Harmonized Counterparts (HC): The ability to recognize associated balanced components. Primitive 5 - Cordial Counterparts (CC): The ability to recognize associated cordial components. Primitive 6 - Discriminate Counterparts (DC): The ability to divide and recognize two distinct components. HC, CC, and DC are primitives that enable the recognition by dividing input into one or more recognizable components. These primitives assist in partitioning the input such that the commonality among the components is distinguished.

4. Capable Systems with Intelligence A system capable only with E can only recognize the existence or absence of an item. A system capable of E and SR can recognize simple recurrence of a known item. Thus it could have the ability to recognize a linear arrangement of visual element or a chronological repetition of a note. Thus this system possesses a higher level of cognitive capacity. Systems that are additionally capable of RJ recognize the placement of two known items in distinctive relative position. Thus, for visual input it may recognize two inputs conjoined end to end, or top to bottom, co-centric, or some other distinctive arrangement. For aural inputs it could be one element before other, played simultaneously, or some time delayed occurrence. HC is a capability that helps a system to quickly identify components composed of identical elements. This represents a capacity to drive recognition of components that are possibly balanced or identical. CC enables a system to identify components that are similar but does note differences. These differences are meaningful distinguishing characteristics. DC assists the system to partition input into smaller components to enable easier recognition. This process helps cognition by a process of divideand-conquer. The following table shows a possible set of types of Learning Systems (LS) based on a hierarchical set of capabilities they possess.

S. Iyengar / Cognitive Primitives for Automated Learning

Learning System

Enabled Capabilities

LS(I)

{E, SR, RA}

LS(II)

{E,SR,RA, HC}

LS(III)

{E,SR,RA, HC, CC}

LS(IV)

{E,SR,RA, HC, DC}

413

5. Conclusion We have proposed and defined fundamental mnemonic capabilities applicable for cognition. Mnemonic capabilities enable a system/organism to atomically match input with learned knowledge. The ability to record experiences as part of its history and the ability to abstract/imbibe knowledge from this history of experience are also fundamental to a cognitive system. Using these capabilities learning systems are proposed that are capable of various degrees of cognition. A constructivist approach forms the guiding principle for our paper. Based on these we have developed mnemonics for aural perception in [2] and for visual perception in [3].

6. Current and Future work We have constructed a music formulating (COMPOSE) systems based on the capabilities customized for aural cognition [4]. This system enables a naïve formulation of symphony using simple combination of primitives developed in [2]. Survey on the use of these capabilities, have shown the affinity and use of these musical structures [5]. A theory on natural language formulation is presented in [6] that utilize aural mnemonics. Similar studies using proposed visual mnemonics will be undertaken and the results of which subsequently used in simple recognition software. We will be developing systems that can be used to test these primitives for visual cognition

References 1. 2. 3. 4. 5. 6.

http://www.stormfront.org/racediff/mainstre.html A Theory on the Abstraction and Cognition Based on Musical Note Patterns, Sudharsan Iyengar, ICAI-04, Las Vegas, June 2004. Mnemonics for Visual Perception, submitted ECAL 2007, Sudharsan Iyengar, Lisbon, September 2007 A Framework for Generating Chants Using Mnemonic Capabilities, Sudharsan Iyengar, SMC06, Marseille, May 2006. A simple visual perception model by adaptive junction, Ajioka, Y. Inoue, K., IJCNN, 1992. Visual Perception Modeling for Intelligent Avatars, Ronghua Liang, et. al, Advances in Artificial Reality, LNCS, 4282/2006.

414

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

Vector Symbolic Architectures: A New Building Material for Artificial General Intelligence1 Simon D. LEVY a,2 , and Ross GAYLER b a Washington and Lee University, USA b Veda Advantage Solutions, Australia Abstract. We provide an overview of Vector Symbolic Architectures (VSA), a class of structured associative memory models that offers a number of desirable features for artificial general intelligence. By directly encoding structure using familiar, computationally efficient algorithms, VSA bypasses many of the problems that have consumed unnecessary effort and attention in previous connectionist work. Example applications from opposite ends of the AI spectrum – visual map-seeking circuits and structured analogy processing – attest to the generality and power of the VSA approach in building new solutions for AI. Keywords. Vector Symbolic Architectures, associative memory, distributed representations, Holographic Reduced Representation, Binary Spatter Codes, connectionism

Introduction Perhaps more so than any other sub-field of computer science, artificial intelligence has relied on the use of specialized data structures and algorithms to solve the broad variety of problems that fall under the heading of “intelligence”. Although initial enthusiasm about general problem-solving algorithms [1] was eventually supplanted by a “Society of Mind” view of specialized agents acting in concert to accomplish different goals [2], the dominant paradigm has always been one of discrete atomic symbols manipulated by explicit rules. The strongest challenge to this view came in the 1980’s, with the emergence of connectionism [3], popularly (and somewhat misleadingly) referred to as neural networks. In contrast to the rigid, pre-specified solutions offered by “Good Old-Fashioned AI” (GOFAI), connectionism offered novel learning algorithms as solutions to a broad variety of problems. These solutions used mathematical tools like learning internal feature representations through supervised error correction, or back-propagation [4], self-organization of features without supervision [5], and the construction of content-addressable memories through energy minimization [6]. In its most radical form, the connectionist ap1 The authors thank Chris Eliasmith, Pentti Kanerva, and Tony Plate for useful discussion on the topics presented here, and three anonymous reviewers for helpful suggestions. 2 Corresponding Author: Simon D. Levy, Computer Science Department, Washington and Lee University, Lexington, Virginia 24450 USA; E-mail: [email protected].

S.D. Levy and R. Gayler / VSA: A New Building Material for Artificial General Intelligence

415

proach suggested that the structure assumed by GOFAI and traditional cognitive science might be dispensed with entirely, to be supplanted by general mechanisms like sequence learning [7]. In support of the representations derived by such models, proponents cited the advantages of distributing informational content across a large number of simple processing elements [8]: distributed representations are robust to noise, provide realistically “soft” limits on the number of items that can be represented at a given time, and support distance metrics. These properties enable fast associative memory and efficient comparison of entire structures without breaking down the structures into their component parts. The most serious criticism of connectionism held that neural networks could not arrive at or exploit systematic, compositional representations of the sort used in traditional cognitive science and AI [9]. Criticism that neural networks are in principle unable to meet this requirement was met in part by compositional models like RAAM [10]; however, RAAM’s reliance on the back-propagation algorithm left it open to criticism from those who pointed out that this algorithm did not guarantee a solution, and could be computationally intractable [11].3 In the remainder of this paper, we describe a new class of connectionist distributed representations, called Vector Symbolic Architectures (VSA), that addresses all of these concerns. VSA representations offer all of the desirable features of distributed (vector) representations (fast associative lookup, robustness to noise) while supporting systematic compositionality and rule-like behavior, and they do not rely on an inefficient or biologically implausible algorithm like back-propagation. The combination of these features makes VSA useful as a general-purpose tool or building material in a wide variety of AI domains, from vision to language. We conclude with a brief description of two such applications, and some prospects for future work.

1. Binding and Bundling in Tensor Products Systematicity and compositionality can be thought of as the outcome of two essential operations: binding and bundling. Binding associates fillers (John, Mary) with roles (LOVER, BELOVED). Bundling combines role/filler bindings to produce larger structures. Crucially, representations produced by binding and bundling must support an operation to recover the fillers of roles: it must be possible to ask “Who did what to whom?” questions and get the right answer. Vector Symbolic Architectures is a term coined by one of us [13] for a general class of distributed representation models that implement binding and bundling directly. These models can trace their origin to the Tensor Product model of Smolensky [14]. Tensor-product models represent both fillers and roles as vectors of binary or real-valued numbers. Binding is implemented by taking the tensor (outer) product of a role vector and a filler vector, resulting in a mathematical object (matrix) having one more dimension than the filler. Given vectors of sufficient length, each tensor product will be unique. Bundling can then be implemented as element-wise addition (Figure 1), and bundled structures can be used as roles, opening the door to recursion. To recover a filler (role) from a bundled tensor product representation, the product is simply divided by the role (filler) vector. 3 Recent results with a linear version of RAAM using principal component analysis instead of back-prop [12] show promise for overcoming this problem.

416

S.D. Levy and R. Gayler / VSA: A New Building Material for Artificial General Intelligence

Figure 1. Tensor product representation of John loves Mary.

Figure 2. Methods for keeping fixed dimensionality in tensor-product representations.

2. Holographic Reduced Representations and Binary Spatter Codes Because the dimension of the tensor product increases with each binding operation, the size of the representation grows exponentially as more recursive embedding is performed. The solution is to collapse the N × N role/filler matrix back into a length-N vector. As shown in Figure 2, there are two ways of doing this. In Binary Spatter Coding, or BSC [15], only the elements along the main diagonal are kept, and the rest are discarded. If bit vectors are used, this operation is the same as taking the exclusive or (XOR) of the two vectors. In Holographic Reduced Representations, or HRR [16], the sum of each diagonal is taken, with wraparound (circular convolution) keeping the length of all diagonals equal. Both approaches use very large (N > 1000 elements) vectors of random values drawn from a fixed set or interval. Despite the size of the vectors, VSA approaches are computationally efficient, requiring no costly backpropagation or other iterative algorithm, and can be done in parallel. Even in a serial implementation, the BSC approach is O(N ) for a vector of length N , and the HRR approach can be implemented using the Fast Fourier Transform, which is O(N log N ). The price paid is that most of the crucial operations (circular convolution, vector addition) are a form of lossy compression that introduces noise into the representations. The introduction of noise requires that the unbinding process employ a “cleanup memory” to restore the fillers to their original form. The cleanup memory can be imple-

S.D. Levy and R. Gayler / VSA: A New Building Material for Artificial General Intelligence

417

mented using Hebbian auto-association, like a Hopfield Network [6] or Brain-State-in-aBox model [17]. In such models the original fillers are attractor basins in the network’s dynamical state space. These methods can be simulated by using a table that stores the original vectors and returns the one closest to the noisy version.

3. Applications As a relatively new technology, VSA is just beginning to be used in the AI and cognitive science communities. Its support for compositional structure, associative memory, and efficient learning make it an appealing “raw material” for a number of applications. In this concluding section we review some of these applications, and outline possibilities for future work. 3.1. Representing Word Order in a Holographic Lexicon Jones and Mewhort [18] report using a holographic / convolution approach similar to HRR, for incorporating both word meaning and sequence information into a model lexicon. Their holographic BEAGLE model performed better than (300-dimensional) Latent Semantic Analysis [19] on a semantic-distance test, and, unlike LSA, BEAGLE can predict results from human experiments on word priming. 3.2. Modeling Surface and Structural Properties in Analogy Processing Experiments on children’s ability to process verbal analogies show the importance of both surface information (who or what the sentence is about) and structural information (who did what to whom) [20]. Eliasmith and Thagard [21] have successfully modeled these properties in DRAMA, an HRR-based model of analogy processing. Because HRR and other VSA approaches support the combination of surface and structural information through simple vector addition, DRAMA is able to model both components of analogy in a single representation. 3.3. Variables and Quantification GOFAI has excelled at representing and reasoning with universally and existentially quantified varialbes; e.g., ∀ x Computer(x) → HasBuggyP rogram(x)∨Broken(x). It has been known for some time, however, that human performance on such reasoning tasks differs in interesting ways from simple deductive logic [22]. Recent work by Eliasmith [23] shows that HRR encodings of logical rules yield results similar to those seen in the experimental literature. 3.4. Future Work: VSA in Visual Map-Seeking Circuits Arathorn ’s Map Seeking Circuits (MSCs) [24] are recurrent neural networks for recognizing transformed images, using localist representations. We propose treating the localist MSC as an input recognition device, with the localist output values subsequently encoded into VSA representations indicating items in the agent’s environment, and their spatial relationships to the agent. This would allow the representation and manipulation of multiple simultaneous items in the agent’s environment.

418

S.D. Levy and R. Gayler / VSA: A New Building Material for Artificial General Intelligence

References [1] [2] [3] [4]

[5] [6] [7] [8]

[9] [10] [11] [12] [13]

[14] [15]

[16] [17] [18] [19] [20] [21] [22] [23]

[24]

Newell, A., Simon, H.A.: Gps, a program that simulates human thought. Lernende Automaten (1961) 109–124 Minsky, M.L.: The Society of Mind. Simon and Schuster (1988) Rumelhart, D., McClelland, J., eds.: Parallel Distributed Processing: Explorations in the Microstructure of Cognition. MIT Press (1986) Rumelhart, D., Hinton, G., Williams, R.: Learning internal representation by error propagation. In Rumelhart, D., McClelland, J., eds.: Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1. MIT Press (1986) Kohonen, T.: Self-Organizing Maps. 3 edn. Springer-Verlag, Secaucus, NJ (2001) Hopfield, J.: Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences 79 (1982) 2554–2558 Elman, J.: Finding structure in time. Cognitive Science 14 (1990) 179–211 McClelland, J., Rumelhart, D., Hinton, G.: The appeal of parallel distributed processing. In Rumelhart, D., McClelland, J., eds.: Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1. MIT Press (1986) Fodor, J., Pylyshyn, Z.: Connectionism and cognitive architecture: A critical analysis. Cognition 28 (1988) 3–71 Pollack, J.: Recursive distributed representations. Artifical Intelligence 36 (1990) 77–105 Minsky, M.L., Papert, S.A.: Perceptrons: Expanded Edition. MIT Press (1988) Voegtlin, T., Dominey, P.F.: Linear recursive distributed representations. Neural Netw. 18(7) (2005) 878–895 Gayler, R.: Vector symbolic architectures answer jackendoff’s challenges for cognitive neuroscience. In Slezak, P., ed.: ICCS/ASCS International Conference on Cognitive Science. CogPrints, Sydney, Australia, University of New South Wales (2003) 133–138 Smolensky, P.: Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence 46 (1990) 159–216 Kanerva, P.: The binary spatter code for encoding concepts at many levels. In Marinaro, M., Morasso, P., eds.: ICANN ’94: Proceedings of International Conference on Artificial Neural Networks. Volume 1., London, Springer-Verlag (1994) 226–229 Plate, T.: Holographic reduced representations. Technical Report CRG-TR-91-1, Department of Computer Science, University of Toronto (1991) Anderson, J., Silverstein, J., Ritz, S., Jones, R.: Distinctive feathers categorical perception and probability learning: some applications of a neural model. Psychological Review 84(5) (1977) 413–451 Jones, M.N., Mewhort, D.J.K.: Representing word meaning and order information in a composite holographic lexicon. Psychological Review 114 (2007) Foltz, T.L.T., Laham, D.: Introduction to latent semantic analysis. Discourse Processes 25 (1998) 259– 284 Gentner, D., Toupin, C.: Systematicity and surface similarity in the development of analogy. Cognitive Science 10 (1986) 277–300 Eliasmith, C., Thagard, P.: Integrating structure and meaning: a distributed model of analogical mapping. Cognitive Science 25(2) (2001) 245–286 Wason, P.: Reasoning. In Foss, B., ed.: New Horizons in Psychology. Penguin, Harmondsworth (1966) Eliasmith, C.: Cognition with neurons: A large-scale, biologically realistic model of the wason task. In Bara, G., Barsalou, L., Bucciarelli, M., eds.: Proceedings of the 27 th Annual Meeting of the Cognitive Science Society. (2005) Arathorn, D.W.: Map-Seeking Circuits in Visual Cognition. Stanford University Press (2002)

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

419

Analogy as Integrating Framework for Human-Level Reasoning Angela SCHWERING 1 , Ulf KRUMNACK, Kai-Uwe KÜHNBERGER and Helmar GUST University of Osnabrück, Germany Abstract. Human-level reasoning is manifold and comprises a wide variety of different reasoning mechanisms. So far, artificial intelligence has focused mainly on using the classical approaches deduction, induction, and abduction to enable machines with reasoning capabilities. However, the approaches are limited and do not reflect the mental, cognitive process of human reasoning very well. We contend that analogical reasoning is the driving force behind human thinking and therefore propose analogy as an integrating framework for the variety of human-level reasoning mechanisms. Keywords. analogy, analogical reasoning, integrated cognitive abilities, cognitive model

1. Motivation In most current approaches to model reasoning capabilities on machines, researches have examined certain reasoning mechanisms isolated from the overall context of human reasoning and learning. These reasoning mechanisms cover deduction [1], induction [2] and abduction [3]. Although such research-endeavors clearly are successful in various aspects and applications, they model just a fraction of the overall complexity of humanlevel reasoning. The problem of integrating such approaches to reach a cognitive system for human-level reasoning is non-trivial. We propose analogical reasoning as an integrating framework for various human-level reasoning mechanisms. This paper describes the role of analogies in performing different reasoning mechanisms in one integrated cognitive model and proposes a concrete approach for human-level reasoning based on Heuristic-Driven Theory Projection (HDTP). The remainder of the paper is structured as follows: Section 2 explains the integrating role of analogy in various examples for cognitive reasoning or learning processes. Section 3 proposes a cognitive model for human-level reasoning, which is elaborated further in section 4 by specifying an implementation based on HDTP. We explain how knowledge is organized and represented in the knowledge base, how the reasoning unit utilizes this knowledge to perform different types of reasoning and how new knowledge is stored in the knowledge base. 1 Corresponding Author: Angela Schwering, University of Osnabrück, Albrechtstr. 28, 49076 Osnabrück, Germany; E-mail: [email protected]

420

A. Schwering et al. / Analogy as Integrating Framework for Human-Level Reasoning

2. The Role of Analogies in Human-Level Reasoning and Learning Learning means inferring new or revising old knowledge and adapting the knowledge base. By the kind of knowledge learned, we can distinguish various cognitive tasks: Learning new domain knowledge. Analogical comparison is a central mechanism for problem solving: new problematic situations (target domain) are compared to similar problems experienced in the past (source domain). Analogy is used to analyze the source and the target problem for common structures. The solution strategy applied in the source problem is transferred to the target domain to solve the problem in the new domain. Creative learning of new conceptual knowledge. Analogical transfer is not limited to domain knowledge, but can also hypothesize new concepts. For instance, in physics, "heat" is energy that can be transferred from one body to another due to a difference in temperature. Heat in contrast to temperature is not observable and can be conceptualized only via analogy to a perceivable domain, e.g. water flowing between two vessels with different water levels. The height of the water is aligned to the temperature. From the observation that water keeps flowing until it has the same height in both vessels (source domain) and that temperature balances after some time (target domain), it can be inferred via analogy, that there exists an analogous "flowing thing" on the target side: the concept heat. Creative generation of knowledge of this type is classically modeled via abduction. Using analogy in this abductive process guides and motivates why certain things are hypothesized and others not. Creating ad-hoc concepts. Establishing an analogy between two domains leads also to the construction of new ad-hoc concepts: e.g. in the Rutherford analogy between the solar system and the Rutherford atom, the sun and the nucleus, respectively the planet an the electron are aligned. For the purpose of the analogy they form the ad-hoc concept central body and orbiting object respectively. These ad-hoc concepts can emerge as permanent concepts after several learning cycles. Learning general principles via abstraction. Analogy also plays an important role in learning general principles. Induction, the predominant mechanism of generalization in AI, is rather restricted and is usually considered to require a large set of samples incorporating identical patterns. Combining analogical and inductive reasoning mechanisms enables a reasoning system to generalize over analogical patterns which increases the flexibility and makes it similarly powerful as human generalization [4].

3. Cognitive Model for Human-Level Reasoning Figure 1 illustrates a cognitive model for human-level reasoning. We now explain the role of the knowledge base, the reasoning unit and the interaction of both. The knowledge base stores all information about the world. It represents the human memory. Domain knowledge constitutes the main part of the knowledge and contains information such as "Planets revolve around the sun." or specific rules for solving problems. As part of the knowledge base we explicitly distinguish conceptual knowledge, describing all concepts by their taxonomic relationships to each other in a hierarchy. This hierarchy contains top-level concepts as well as domain concepts. The reasoning unit comprises four different reasoning mechanism: the classical approaches deduction, induction and abduction, but also analogical reasoning. Analogical reasoning performs as the core of this framework.

A. Schwering et al. / Analogy as Integrating Framework for Human-Level Reasoning

421

Figure 1. Cognitive model of human-level reasoning.

Deduction infers knowledge given implicitly by facts or rules: if the premises evaluate true, deduction infers that the conclusion is true as well. From a cognitive science perspective, researchers often claim that deduction plays no or a minor role in human reasoning [5,6]. Deduction is nevertheless a component of the reasoning unit, because it can support analogical reasoning in different ways: Deduction enables a machine to make implicit knowledge explicit and therefore knowledge becomes independent of the chosen representation. Moreover, the ability to establish an analogy highly depends on an adequate representation of the domains. Deduction is used to re-represent knowledge to make analogous structures in the domains obvious. Induction creates knowledge via generalization over a set of cases which share common properties or relations and deduces some general law or principle. The "classical induction" in artificial intelligence and machine learning requires a very large set of data samples to draw sound conclusions. Humans apply inductive strategies for reasoning already with a limited number of examples (e.g. four or five cases). Moreover, they are able to compare samples which share the common pattern only at an abstract level, while "classical induction" works only when the patterns across the data samples are identical. Combining induction with analogy, this reasoning mechanism gains the flexibility it needs to reflect human generalization techniques (c.f. section 4 or [4]). Abduction is reasoning from an effect to its cause: from a known general rule and a true conclusion it is hypothesized that the premise is true as well. Analogical reasoning can motivate abduction-like inferences [7]: Assume p and p → q are true in the source domain and therefore q can be inferred. If an analogous counterpart q ′ of q is known in the target domain, we can transfer via analogy the rule p′ → q ′ and abductively conclude that p′ also holds in the target domain. There could be other causes for q ′ , but since p is the cause in the analogous domain, it is likely that p′ is also the cause in the target domain. Analogies aim to identify structural commonalities, analogous relations or objects playing the same roles in two different domains - the source and the target domain. Analogical learning is typically applied across two different domains. The purpose of analogies is to adapt knowledge available about the source such that it can be applied to the target in a way that new analogous inferences can be drawn. This way it is possible to gain completely new hypotheses about the target domain which still have to be proven. Analogical transfer leading to inconsistencies in the target domain is considered as an indicator for a bad (wrong) analogy. Analogical reasoning can also be used to extract commonalities between source and target which are captured in an abstract generalization. However, this generalization is not assumed to be generally true (as it is the case for induction), but only for the specific source and target domain. Good analogies can be analogously applied and tested in other domains.

422

A. Schwering et al. / Analogy as Integrating Framework for Human-Level Reasoning

4. Implementation of the Cognitive Model Figure 2 illustrates our approach to the cognitive model for human-level reasoning.

Figure 2. An HDTP-based implementation of the cognitive model for human-level reasoning.

The knowledge base consists of domain knowledge, background knowledge and an ontology. The domain and the background knowledge constitute all knowledge about the world and are specified by formulas represented in many-sorted first-order logic. The domain knowledge contains domain-dependent knowledge, such as knowledge about thermodynamics and fluid dynamics (e.g. water and its flowing characteristics). The background knowledge contains domain-independent knowledge such as the formula distanceeucl (X, Y )=distanceeucl (Y, X) stating that the Euclidian distance is symmetric. The ontology contains conceptual knowledge specified via taxonomic relations to other concepts [8]. It describes the relation of top-level concepts like massterm, time or object to (usually domain-specific) concepts such as vessel or water. The heart of the reasoning unit is the HDTP [9], a symbolic approach developed at the University of Osnabrück to model analogies. Since HDTP is based on first-order logic, it can naturally interact and easily integrate the formal reasoning mechanisms deduction, induction and abduction. The reasoning unit contains different modules. The interaction of these modules enables the modeling of human reasoning strategies. We explain the reasoning unit by describing the different steps of the reasoning process. Of course, the process differs depending on what kind of knowledge is inferred or learned. All reasoning processes start with some knowledge as input - in analogical reasoning two domain theories T hS and T hT modeling source and target. The theory of antiunification [10] is used to generalize the structural commonalities between two formal theories. The module analogical generalization computes an analogy by successively selecting a suitable formula from T hT and T hS and constructing a generalization (T hG ) together with the corresponding substitutions. Heuristics to minimize the complexity of substitutions [9] have been developed to guide this anti-unification process. The output is the generalized theory T hG , which covers explicitly the commonalities and establishes an analogous relation between source and target. Usually, the source domain is well known and modeled richer than the target domain (about which you want to learn something). Based on the analogical relation between source and target, the module analogical transfer identifies knowledge in the source which could be transferred to the target domain. The knowledge gained by the analogical transfer can be facts or rules, but also conceptual knowledge such as new concepts or a revised concept hierarchy. The

A. Schwering et al. / Analogy as Integrating Framework for Human-Level Reasoning

423

reasoner module must check the hypotheses suggested by the analogy for consistency, i. e. it must prove whether the new facts or rules hold or conflict in the target domain. The reasoner module can also be used during the analogical generalization for restructuring domain knowledge, e.g. restructure the target domain by inferring implicit formulas from the given facts and rules such that the resulting representation is suitable for a structural comparison and can be anti-unified (e.g. [11]). On top of HDTP, the module abstract generalization applies inductive reasoning mechanisms based on the analogical comparison to enable inferring general principles [4]. All inferred knowledge is stored in the knowledge base and leads either to extended or revised theories (in case new facts or laws are learned for the domain or background knowledge or new general principles are hypothesized via the abstract generalization module) or an extended or revised ontology (if concept knowledge is learned). 5. Summary This paper proposes to model human-level reasoning based on analogy. Combining analogy with classical reasoning mechanisms leads to a cognitive model in which various human reasoning and learning strategies can be explained. We outline our approach using HDTP as integrating framework and describe the role of analogy in various inferences. Acknowledgements The work was supported by the German Research Foundation (DFG), grant KU1949/2-1. References [1] [2] [3] [4]

[5] [6] [7] [8]

[9] [10]

[11]

Bibel, W.: Deduction: Automated Logic. Academic Press London (1993) Muggleton, S.; De Raedt, L.: Inductive logic programming: Theory and methods. In: Journal of Logic Programming, 19,20. (1994) 629–679 Magnani, L.: Abduction, Reason, and Science. Processes of Discovery and Explanation. Kluwer Academic/Plenum Publishers, New York (2001) Gust, H., Krumnack, U., Kühnberger, K.U., Schwering, A.: Integrating analogical and inductive learning at different levels of generalization. In: Workshop on Learning from Non-Vectorial Data (LNVD2007), Osnabrück (2007) Gigerenzer, G.: Bounded and rational. In Stainton, R., ed.: Contemprorary Debates in Cognitive Science. Blackwell Publishing (2006) 115–133 Laird, J.: Mental Models: Towards a Cognitive Science of Language, Inference and Consciousness. Cambridge University Press (1983) Schwering, A., Krumnack, U., Kühnberger, K.U., Gust, H.: Analogical reasoning with SMT and HDTP. In: 2nd European Cognitive Science Conference (EuroCogSci07), Delphi, Greece (2007) 652–657 Gust, H., Kühnberger, K.U., Schmid, U.: Ontologies as a cue for the metaphorical meaning of technical concepts. In Schalley, A., Khlentzos, D., eds.: Mental States: Evolution, Function, Nature. John Benjamins Publishing Company (2007) 191–212 Gust, H., Kühnberger, K.U., Schmid, U.: Metaphors and heuristic-driven theory projection (HDTP). Volume 354. (2006) 98–117 Krumnack, U., Schwering, A., Gust, H., Kühnberger, K.U.: Restricted higher-order anti-unification for analogy making. In: 20th Australian Joint Conference on Artificial Intelligence (AI07). Lecture Notes of Artificial Intelligence, Gold Coast, Australia, Springer (2007) Schwering, A., Krumnack, U., Kühnberger, K.U., Gust, H.: Using gestalt principles to compute analogies of geometric figures. In: 29th Meeting of the Cognitive Science Society (CogSci07), Nashville, Tennessee (2007) 1485–1490

424

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

Designing Knowledge Based Systems as Complex Adaptive Systems Karan Sharma ([email protected]) Artificial Intelligence Center, University of Georgia

Abstract . The paper proposes that knowledge based systems must be designed as complex adaptive systems and any other approach is not fundamental, even if sometimes it yields good results. Complex systems are characterized as having global behavior not always explainable from local behavior. Here we propose that the way we perceive knowledge in AI needs to change to Complex Adaptive, hence the need for a paradigm shift is stressed. Almost all historical KBS were not complex systems in an authentic sense. But it is not a good idea to criticize them because with available resources and theories, they did their best. Sooner or later, we will have to design our KBS as complex adaptive systems, so why not sooner. There are three mechanisms that must be part of any knowledge based system, viz., Interdependency and fluidity, mechanisms for attribution of emergent properties and self-organization.

Keywords. Knowledge Based Systems, Complex Adaptive Systems, Cognitive Fluidity, Self-organization.

1. Introduction Although the time has come that our perception of the whole field of AI should be of complex systems and more rule-based, well-defined and logic style approaches need rethinking, we confine ourselves to the discussion of KBS in this paper. Richard Loosemore [1] argues “One the most basic assumptions made by Artificial Intelligence researchers is that the overall behavior of an AI system is related in a lawful, comprehensible way to the low level mechanisms that drive the system.……..this apparently innocent assumption is broken, because all intelligent systems, regardless of how they are designed, must be complex systems”[1]

Evidence in anthropology and psychology suggests that mental fluidity is central to human cognition. It is this fluidity that lends us our higher level abilities that only our species Homo sapiens sapiens possess. If our goal is to build human-like intelligence, the only valid path is complex adaptive systems – since by their very nature these systems can efficiently produce an analog of mental fluidity. Human cognition is a complex adaptive system [2], hence KBS have to be complex adaptive, it cannot be otherwise. For the purpose of this paper when we talk of KBS, it encompasses all areas of Artificial Intelligence that handle knowledge including case-based reasoning, expert systems, etc.

K. Sharma / Designing Knowledge Based Systems as Complex Adaptive Systems

425

Complex systems are the systems that involve interactions among various components and as a result of these interactions global emergent behavior emerges in a system. [3] suggests "interdependence" among various components is a more generic term since it is the interdependent parts that create emergent properties rather than interconnected parts. Complex systems are known for various characteristics; among them widely discussed are emergence, self-organization and non-linearity. Here it is important to mention that KBS must be complex adaptive rather than just complex because it must be sensitive to any new knowledge that enters the system. For the detailed study of complex systems reader is referred to [3, 4] and for complex adaptive systems to [5]. There are three mechanisms that must be part of any knowledge based system, viz., Interdependency and fluidity, a mechanism for attribution of emergent properties and self-organization. These mechanisms will not only make knowledge more complex, but also make it adaptive to new information that comes in.

2. Interdependency in Knowledge Bases

Referring to the brains of early humans, Mithen [6] asserts:"we can safely state that in spite of linguistic differences, all Early Humans shared the same basic type of mind:a swiss-army-knife mentality. They had multiple intelligences, each dedicated to a specific domain of behavior, with very little interaction between them……..Early Humans seem to have been so much like us in some respects, because they had these specialized cognitive domains; but they seem so different because they lacked a vital ingredient of the modern mind: cognitive fluidity" [6]

Grounding his arguments with evidence from archaeology and anthropology he further suggests that "cognitive fluidity" is responsible for our intelligence and the rise of human civilization (including art, science and religion). Even our closest cousins Neanderthals did not had this ability. But it is very unfortunate that AI researchers and builders of the KBS have totally disregarded this ability and created systems that stored information like Early Humans. Cyc [7] did better by at least creating Neanderthal-like common sense, that is, there are not very flexible across-the-domain connections. The time has come that our perspective changes and we start treating knowledge as complex for AI purposes. Unfortunately what should have been done first will be done last. All the KBS in the past, without exception, were fundamentally flawed because they represented knowledge "as is" without giving any consideration to the fact how a thing can be represented in terms of other entities. Besides anthropology, there are theories by psychologists and computer scientists to account for our mental fluidity. Arthur Koestler [8] proposed the Bisociation, the mechanism in human mind, through which different planes of thought come together to produce a novel thought. Karmiloff-smith [9] came up with a Representational Redescripton model explaining how children's representations become more flexible with age. Fauconnier and Turner [10] proposed theory of conceptual blending, in which multiple concepts can blend to form a new concept with emergent properties. Boden [11] also suggested that transformation of conceptual spaces is central to our thought processes. [12, 13] came up with a program Copycat to model analogical thought but

426

K. Sharma / Designing Knowledge Based Systems as Complex Adaptive Systems

they asserted it can be extended to all fluid mental concepts. The major achievement [13, 14] of the Copycat program was to show that human cognition is a "complex adaptive system." The above works unanimously offer a unique insight into the workings of the human mind - In human mind, elements from different areas are interdependent and often come together to form a coherent whole whose properties may be emergent. Creating highly fluid systems that are complex and adaptive is what is required. KBS should be capable of "Seeing one thing in the frame of other" and free merger of concepts should be the rule. High-Interdependency among representations must be the rule in any Knowledge Base. Each concept must be represented from the perspective of other concepts in the knowledge base. And a concept should have representation from the perspective of multiple other concepts. This is done to ensure high interconnectivity, which obviously will not only make KBS highly fluid but also adaptive, just like human cognition. R.M. French [15, 16] has argued that representation for any AI system, if it is not malleable, then it is necessarily flawed. According to him, the representations that are fixed can never produce an analog of human cognition and fluidity. Each representation should not be "as is", that is, there must not be fixed representation of any concept.

3. Emergence

The reason emergence becomes a central issue in the design of knowledge bases is that there are global properties possessed by entities which are not reducible to the components and subparts of the entities. For example – Psychological characteristics of the human brain cannot be explained in terms of just one neuron or many neurons taken independently. Rather interconnections and interdependence of the neurons display emergent behavior. Emergence is discussed in detail in [3, 17]. Let us consider a hypothetical knowledge base. Assume the representation to be either predicate calculus, logic, semantic network, frame based or any symbol manipulation representation. Also, we assume our knowledge base to be all-inclusive, that is, containing all the information in the world. Suppose we examine the concept of "Automobile" in our knowledge base. If we start with any possible initial state of the concept "Automobile" in any representation, and from there try to derive this property "Means of transportation", we see that no matter what change we make to that state it is simply impossible to derive the property "Means of transportation" bottom-up since derivation is the process of sequential state transition in which a part or subpart of the information is changed to reach an end state. However, the emergent property can never be an end state because 1.) The concept "Automobile" is represented in the form of its parts (engine, tires, windows, mirror, doors and their relationships). Information "Means of transportation" is not contained in any one part, or combination of few parts, rather it is the interactions of all the parts that emerge this property, hence, trying to extract information "Means of transportation" from the parts is impossible. This property "Means of transportation" is the property of specific configuration created by all of the components and their interactions making up the concept "Automobile". Derivations, as in any representation lead to an end state in a deterministic fashion, however starting from any initial state in this problem we can never be sure that the next state will lead us to correct end state (our

K. Sharma / Designing Knowledge Based Systems as Complex Adaptive Systems

427

property). Hence the property is not derivable from the inner components and is globally emergent. 2.) To derive the property from all the other information in the knowledge base besides the "Automobile" concept, it is that the property if it is the property of another entity in the knowledge base, then it is going to be an emergent property of that entity, that is, not derivable from its own components. For example - the property "Means of transportation" can be attributed to the concept of "Horse" and it is non-derivable for this concept. Since we have already seen that the property cannot be obtained from its own components, the only way is for the property to transfer from one point in the knowledge base to another point by transferring from one whole to another whole. The property of the concept "Horse" can be transferred to the concept "Automobile" in whole. We need to have such mechanisms in our systems that help us in attribution of emergent properties because trying to derive any higher order property from lower level components is almost impossible. Analogical reasoning is one such mechanism but there are many more similar strategies that must be incorporated in the systems. The idea is to transfer stuff from one whole to another based on some criteria, and not try to derive things bottom up.

4. Self-Organization

Karmiloff-smith [9] while proposing his RR model wrote "My claim is that a specifically human way to gain knowledge is for the mind to exploit internally the information that it has already stored……..by redescribing its representations or, more precisely, by iteratively re-representing in different representational formats what its internal representations represent.”[9]

The above lines clearly hint to the process of self-organization in the human mind. A process analogous to re-representation mechanism is a necessity in a complex adaptive KBS. [18] suggested self-organization is a requirement for any system if it is to be creative. The idea of incorporating self-organization is simple – any new information is connected to lots of other information and this interdependency can lead to potential effects on all the interdependent information. The goal of self-organization is to adapt the whole KBS to any incoming information so that all the information is represented in the most accurate state in the KBS. Accurate state in the KBS is variable since induction of any new information can change the accuracy. For example – in the nineteenth century, the most accurate explanation for universe was in terms of Newtonian physics, however, in the twentieth century after new information came in (Theory of relativity), the most accurate explanation for universe was in terms of relativity. The more inter-dependent various fragments of knowledge inside the knowledge base are, the more they are prone to the effects of the induction of new knowledge. This new knowledge, if it modifies any knowledge in the KBS, can subsequently lead to chain of modifications because this modified knowledge is highly interdependent with various other fragments of knowledge. The system must be capable of self-organizing at this stage, that is, just by using all the information that is internal to the KBS, the system should be able to reach most accurate representation or perception or frame for each fragment of knowledge that is effected by newly induced knowledge in some way.

428

K. Sharma / Designing Knowledge Based Systems as Complex Adaptive Systems

5. Conclusion No doubt, the road to making knowledge complex and adaptive for AI is filled with some serious bottlenecks, nevertheless, it is reachable. Designing complex adaptive KBS is the most ideal approach that will make them adaptive by incurring several advantages over conventional systems. The most significant advantage being increase in information resulting from high interdependency in the knowledge base. Since each entity or situation can be perceived in multiple frames, our systems will have an option to choose best frame that is most accurate representation relative to other representations, thus increasing system’s information, reliability and adaptiveness. Acknowledgement I thank Dr. Walter D. Potter, Director, Artificial Intelligence Center, University of Georgia for offering useful comments towards the completion of this paper. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18]

R.P.W. Loosemore. Complex systems, artificial intelligence and theoretical psychology. In B. Goertzel and P. Wang, editors, Proceedings of the 2006 AGI Workshop. IOS Press, Amsterdam, pages 159-173, 2007. H. Morowitz and J. Singer, editors. The Mind, the Brain, and Complex Adaptive Systems. AddisonWesley, Reading, Massachusetts, 1995. Y. Bar-Yam. Dynamics of Complex Systems. Perseus Books, Cambridge, Massachusetts, 1997. T.R.J. Bossomaier and D.G. Green, editors. Complex Systems. Cambridge University Press, 2000. J. H. Holland. Hidden Order. Addison Wesley Reading, Massachusetts, 1995. S. Mithen. The Prehistory of the Mind: A Search for the Origins of Art, Religion and Science.Thames and Hudson, London,1996. D. B. Lenat and R. V. Guha. Building Large Knowledge-based Systems: Representation and Inference in the Cyc Project. Addison Wesley, 1990. A. Koestler. The Act of Creation. Macmillan, New York, 1964. A. Karmiloff-Smith. Beyond Modularity: A Developmental Perspective on Cognitive Science. MIT Press, 1992. G. Fauconnier and M. Turner. The Way We Think: Conceptual blending and the Mind’s Hidden Complexities. Basic Books, 2002. M.A. Boden. The Creative Mind: Myths and Mechanisms. Weidenfeld and Nicolson, London, 1991. D.J. Chalmers, R.M. French and D.R. Hofstadter. High-level perception, representation, and analogy: a critique of artificial intelligence methodology. Journal of Experimental and Theoretical Artificial Intelligence 4:185–211, 1992. D. R. Hofstadter and M. Mitchell. The Copycat project: A model of mental fluidity and analogy-making. In K. J. Holyoak and J. A. Barnden, editors, Advances in Connectionist and Neural Computation Theory: Analogical Connections 2:31–112, Ablex, Norwood, New Jersy, 1994. M. Mitchell. Analogy-making as a complex adaptive system. In L. A.Segel and I. R.Cohen, editors, Design Principles for the Immune System and Other Distributed Autonomous Systems. Oxford University Press, New York, pages 335-359, 2001. R. M. French. The Subtlety of Sameness. MIT Press, 1995. R. M. French. When coffee cups are like old elephants or Why representation modules don’t make sense. In Proceedings of the International Conference New Trends in Cognitive Science, A. Riegler and M. Peschl, editors, Austrian Society for Cognitive Science, pages 158-163, 1997. J. H. Holland. Emergence: From Chaos to Order. Addison Wesley, 1998. P.H.G. van Langen, N.J.E. Wijngaards, and F.M.T. Brazier. Designing creative artificial systems. In A.H.B. Duffy and F.M.T. Brazier, editors, AIEDAM, Special Issue on Learning and Creativity in Design, 18(4):217-225, 2004.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

429

Artificial general intelligence: an organism and level based position statement Leslie S. SMITH 1 Department of Computing Science and Mathematics, University of Stirling, UK Abstract. Do simple animals, even single-celled animals, display general intelligence? If we take this to mean that they can cope with their environment, even when it is dynamic, the answer would have to be yes. What is it that gives them this easy capability, yet makes it hard for us to design systems which have it? We suggest that it is from the non-determinism which arises from the lowest levels of their operation, and from the way in which the different levels of their operation interact. We propose techniques whereby these might be incorporated into artificial systems. Keywords. artificial general intelligence, brain model, paramecium, level interaction

Introduction There are many views of what should be described as artificial general intelligence. General intelligence is not a well-defined concept: but then, neither is intelligence, nor are many other general concepts, such a life. But this lack of definition does not mean that they have no meaning: rather that they occupy a diffuse area, rather than a single point: they are cluster concepts. Better then, to ask what some of the characteristics of these concepts are, and to attempt to approach them, rather than trying to define them into a corner then reconstruct them. In what follows, I have chosen two particular aspects of organisms that display what I consider general intelligence, and I have attempted to delineate a possible program of research based on these.

1. The low level organism view Much of what has been researched in artificial intelligence comes from a view that humans have intelligence, and other animals do not. I suggest that this humano-centric view of the world has as much place in general intelligence research as an Earth-centred view of the Universe has in astronomy. General intelligence is about keeping an organism alive and thriving in a complex and ever-changing environment. It differs from purely reac1 Corresponding Author: Leslie S. Smith. Department of Computing Science and Mathematics, University of Stirling, Stirling FK9 4LA, Scotland, UK; E-mail: [email protected]

430

L.S. Smith / Artificial General Intelligence: An Organism and Level Based Position Statement

tive behavior in that behaviors are not necessarily entirely contingent on the organisms senses. Certainly, there are aspects of intelligence found in humans that are not found in other animals, most notably complex language: however, there are certainly aspects of general intelligence visible in the behavior of animals. One is then led to ask what the simplest animal to display any form of general intelligence is, and whether we can learn from it, and build something based on it? Consider the behaviour of a very simple single celled organism: a paramecium. This animal has no nervous system as such, yet has successfully survived in freshwater environments for many millions of years. The animal’s single cell has a membrane surrounding it, and this is covered in small hair-like appendages (cilia). These allow it to move. It eats by endocytosis (incorporating food particles into its cytoplasm), and excretes by exocytosis. Endocytosis is mediated through protein sensors on the membrane surface. The membrane is not uniform: there is a gullet-like area for eating, and an anus-like area for excretion. The animal can move towards food. Reproduction can be either sexual or asexual. Does it display general intelligence? Or is it purely reactive? What aspects of its behavior display intelligence? Consider a slightly more complex organism, one with a rudimentary nervous system. Actually, its visible behavior is likely to be very similar to that of the paramecium: the primary difference is that it has specialized cells for different processes, such as movement, digestion, and control. Yet in the end, eating and excretion are essentially orchestrated endo- and exo- cytosis. Control of the animal is achieved also through the action of the nerve cells on these other cells, and this is achieved through neurotransmitters sent through exocytosis, and received through receptors and ion channels, which then alter cell behaviour. Does it display intelligence or is it purely reactive? Or perhaps the animal displays general intelligence, but each cell is purely reactive? Artificial intelligence designers often consider complexity to be part of the nature of intelligence. They build their systems out of logical entities (such as gates, or threshold logic elements), using some mixture of hardware and software to develop complex logical systems. Given appropriate input, and appropriate interpretation of their output, they can perform sophisticated tasks. Each element is utterly deterministic, and could be considered reactive. But does the overall system display general intelligence? The examples above, two living and one not, may or may not be considered to display general intelligence depending on the standpoint of the onlooker. (I suggest that there is a range of “intelligences” between purely reactive systems and truly intelligent ones, but that attempting to delineate exactly where each example lies on a line connecting them is not a useful exercise.) However one thing that is clear is that the living entities are are very different from the non-living one. Is this difference one purely of implementation (living cells as against logic gates), or are there deeper differences? It has been suggested that life is ultimately based on logic (by which is usually meant that living entities are based on the same logical (and physical) rules that pervade the rest of the universe, rather than there being ‘special sciences’ that govern living entities). Yet this does not imply that life is necessarily based on the same two-valued logic that pervades digital computing. One of the great strengths of two-valued logic is also one of its problems: it is utterly deterministic: repeating the same two-valued logical steps on the same data will always give the same result. This goes against what we see in living organisms. They always show wide diversity of behaviour (and, indeed, this diversity

L.S. Smith / Artificial General Intelligence: An Organism and Level Based Position Statement

431

seems to exist almost no matter what level one examines the systems at). This leads to two different questions: 1. Where does this diversity originate from, and 2. Can we implement this diversity using the two-valued logic systems? Even the simplest living organisms do not display two-valued logical behaviour. Non-determinism is built in from the very bottom level. Consider a piece of the cell membrane of a paramecium in water, which has touched a possible food particle. In order for the food particle to be ingested, a reaction needs to take place between the food particle and some sort of receptor on the surface of the cell. The molecules of both the food particle and the receptor are large (probably proteins, or fatty acids). The reaction occurs when the reactive parts of the molecules of the reactants become close enough to each other [1]. Both reactants are in motion because of their temperature, and because of Brownian motion induced by the water molecules. The likelihood of a reaction occurring is directly related to the likelihood of the reactive surfaces meeting each other before the same random movement causes the food particle to move away from the cell. This whole process is essentially nondeterministic. Virtually all the processes that occur at the cell membrane are of a related form: for example, ion channels which alter shape due to variations in the charge patterns surrounding them are actually continuously in motion, and their shape is essentially best described as a stochastic system [2]. Thinking of ion channel behaviour as being like transistor behaviour (as Mead suggests in his 1989 book [3]) can therefore be slightly misleading. We conclude that there is diversity in response in biological systems even at the lowest level. We suggest that the diversity seen at higher levels in biological systems arises from this lower level diversity, in the same way as the lack of diversity in digital electronic system behaviour has its roots in the absolute determinism of the lowest level operation of these electronic systems. But how important is this diversity? We suggest that this diversity in response lies at the root of successful adaptivity: by responding differently to essentially identical environmental events, and then ‘rewarding’ responses which have the best outcomes by making them more likely to occur in future, an organism can adapt to its environment rapidly and effectively. This is not a new idea: Campbell [4] places blind adaptivity at the root of all learning, from this lowest level all the way up to scientific knowledge, and coined the term evolutionary epistemology to describe it. The adaptivity need not be entirely blind: more recent learning techniques such as reinforcement learning and TD learning [5] suggest that previous experience can be used to guide learning and hence adaptive evolution of behavior. We consider this evolutionary capability to be at the root of general intelligence, both in simple and higher organisms.

2. Interlinking levels Brains operate at many different physiological levels simultaneously. There are numerous different species of ion channels, some voltage sensitive, some not, all opening and closing simultaneously; there are billions of synapses some excitatory, some inhibitory, some shunting, some transmitting, some not; there are billions of neurons, some spiking

432

L.S. Smith / Artificial General Intelligence: An Organism and Level Based Position Statement

some not; there are cortical columns processing information, and communicating with other columns; there are brain areas each with their (partial) specialisation: the whole lot comes together to produce the behaviour of that brain. All of these levels of viewing what is occurring in an animal brain are simultaneously valid. Clearly, they are not independent of each other. One might attempt a similar view of a modern computer system; there are billions of transistors, some on, and some off, billions of gates, some with logic 1 and some with logic 0 outputs; there are objects and functions all running (quasi-) simultaneously, all coming together to produce the behaviour of that computer system. There is, however, a major difference between the two. Digital computer systems are designed and built hierarchically, whereas biological brains are not. This difference shows itself in the ways in which these different levels of operation interact. In a computer, the gates are built out of transistors, and, although the output of a gate is the output of a transistor (and the input to a gate is the input to a transistor) it is safe to ignore transistor operation entirely, as everything about the gate operation can be abstracted away from this lower level. Similarly, the entire operation of all the gates etc. in the hardware can be abstracted into the instruction set (and interrupt structure etc.), so that the software can be considered entirely independently of the hardware. The same is true for layers of software: each interface between levels allows the lower levels to be entirely abstracted away. Often neural systems have been viewed in this way as well. Ionic channels and neuromodulators subserve entities like synapses, and synapses and the collection of charge on dendrites subserve the generation of spikes. Neurons in a cortical column are there in order to generate the behaviour of the column. In other words, the brain is often viewed as if it has been designed from an engineering perspective. But nature and evolution are not bound by engineering design laws! In fact there is considerable interaction between levels. For example, neurotransmitter released from one synapse will leak to another one. Further, small molecules which act as neuromodulators (such as Nitrous Oxide, NO) can diffuse over relatively wide areas, causing non-localised interactions which will cross levels. Research suggests that these non-local links can rapidly reconfigure neural systems [6]. Even with engineered systems, using genetic algorithms to optimize performance can result in level crossing being utilised in a problem solution. A good example of this is in Adrian Thompson’s work [7] in which an FPGA was optimised to solve a problem, using a fitness function calculated directly from the operation of the chip. The result was a solution which used all sorts of cross-talk between entities which are supposed to be entirely independent directly in the system operation. Another example may be found in the work off the robotocist Mark Tilden, who utilises the mechanical properties of the materials of his robots in the robot controller. Clearly, biological systems are quite free to develop such solutions to problems (and don’t have to worry about making their systems comprehensible to engineers!). How does this interlinking of levels contribute to general intelligence? It can provide direct (and therefore fast) interaction between different parts of the brain, permitting rapid reconfiguration. This could permit selection between different mechanisms that might otherwise be viewed as purely reactive. On the other hand it might also cause inappropriate crosstalk (which is why it is normally intentionally omitted from engineered systems).

L.S. Smith / Artificial General Intelligence: An Organism and Level Based Position Statement

433

3. Synthesizing general intelligence based on simple animals and linking levels: towards a program We have stressed the importance of non-determinism at the lowest levels making itself visible at higher levels (up to eventual behavior). Could this non-determinism be replicated by noise in an otherwise deterministic system? And if so, would the form of the noise matter? It appears that the critical issue is the capability for selecting different behaviors, and rewarding those that are most appropriate. We have also stressed the way in which the different levels of operation are interlinked in biological systems. However, it is less clear how this contributes towards general intelligence. One possible answer is that this results in another form of variation in behavior because of the way in cross-level linkages alter behavior in (relatively) unexpected ways. It is not simply noise, because the effects will not be random, but (at least to some extent) repeatable. We note that it can also allow rapid changes to be made to the nature of processing. Replicating this type of general intelligence is not straightforward. If we wish to take an Artificial Life approach, we clearly need to have a rich environment, as well as critters exhibiting multiple levels inhabiting this environment. The levels should not be precisely hierarchically arranged, but able to affect each other. These critters need to be non-deterministic, preferably with this non-determinism being implemented right at the lowest of levels. This should enable different behaviors in response to similar events, and adaptation towards the most appropriate behavior whether blindly or using (e.g.) reinforcement learning. An additional key may be to add a genetic algorithm system on top of this, enabling modification (and possibly optimization) of the internal structure, including the ways in which the levels are able to influence each other. This could permit fast reconfiguration of the system in the light of environmental cues.

Acknowledgements Thanks to Catherine Breslin for suggesting the Paramecium as a very simple animal with interesting behavior, to Karla Parussel for interesting discussions, to the UK EPSRC Grand Challenges in Microelectronic Design Network for inspiration, and to the anonymous referees for historical context.

References [1]

[2] [3] [4] [5] [6] [7]

F.T. Hong, A multi-disciplinary survey of biocomputing: Part 1: molecular and cellular aspects, in Information Processing and Living Systems V. B. Baji´c and T. W. Tan, Eds., Imperial College Press, London, 2005, 1–139. T.F. Weiss, Cellular Biophysics, MIT Press, 1996. C. Mead, Analog VLSI and Neural Systems, Addison Wesley, 1989. D.T. Campbell, Evolutionary Epistemology, in The Philosophy of Karl Popper, P.A. Schlipp, Ed, Open Court, Illinois, 1974. R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, MIT Press, 1998. K. Parussel, A bottom-up approach to emulating emotions using neuromodulation in agents, University of Stirling Ph.D. thesis, http://hdl.handle.net/1893/113, November 2006 A. Thompson, I. Harvey and P. Husbands, Unconstrained evolution and hard consequences, in Towards Evolvable Hardware: The evolutionary engineering approach, E. Sanchez and M Tomassini, Eds, Springer Verlag, LNCS 1062, 1996, 136-165

This page intentionally left blank

Workshop Papers

This page intentionally left blank

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

437

THE ARTILECT WAR Cosmists vs. Terrans A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines Prof. Dr. Hugo de GARIS Director of the “China-Brain Project” Institute of Artificial Intelligence, Department of Computer Science, School of Information Science & Technology, Xiamen University, Xiamen, Fujian Province, China. [email protected]

.

Abstract This paper claims that the “species dominance” issue will dominate our global politics later this century. Humanity will be bitterly divided over the question whether to build godlike, massively intelligent machines, called “artilects” (artificial intellects) which with 21st century technologies will have mental capacities trillions of trillions of times above the human level. Humanity will split into 3 major camps, the “Cosmists” (in favor of building artilects), the “Terrans” (opposed to building artilects), and the “Cyborgs” (who want to become artilects themselves by adding components to their own human brains). A major “artilect war” between the Cosmists and the Terrans, late in the 21st century will kill not millions but billions of people.

1. Introduction This paper claims that the “species dominance” issue will dominate our global politics this century, resulting in a major war that will kill billions of people. The issue is whether humanity should build godlike, massively intelligent machines called “artilects’ (artificial intellects), which 21st century technologies will make possible, that will have mental capacities trillions of trillions of times above the human level. Society will split into two (arguably three) major philosophical groups, murderously opposed to each other. The first group is the “Cosmists” (based on the word Cosmos) who are in favor of building artilects. The second group is the “Terrans” (based on the word Terra,

438

H. de Garis / The Artilect War: Cosmists vs. Terrans

the earth) who are opposed to building artilects, and the third group is the “Cyborgs”, who want to become artilects themselves by adding artilectual components to their own human brains. This paper is structured as follows. Section 2 introduces the 21st century technologies that will make artilect building possible, and thus force humanity to choose whether to build artilects this century or not. Section 3 proves that artilects will have mental capacities trillions of trillions of times above the human level, i.e. they will have godlike, massively intelligent abilities. Section 4 describes how the species dominance debate will start. Section 5 introduces who the major players will be in this debate. Section 6 presents the arguments of the Cosmists in favor of building artilects. Section 7 presents the arguments of the Terrans opposed to the building of artilects. Section 8 presents the Cyborg arguments in favor of converting humans into artilects. Section 9 describes how the artilect war will probably heat up. Section 10 shows that a major late 21st century war will kill billions rather than millions of people. Section 11 describes how inviting my audiences to vote on the Cosmist/Terran question splits them 50/50, which will only make the controversy all the more bitter. Section 12 makes an appeal to philosophers to reflect on the enormous issue of species dominance. Section 13 offers a quote and some publicity. Finally, there is only one reference, which is the author’s book on the same theme as this paper, but which treats the topic in far greater detail.

2. 21st Century Artilect Enabling Technologies 2.1. Moore’s Law Gordon Moore, cofounder of the microprocessor company Intel, noticed in 1965 that the number of transistors on an integrated circuit (chip) was doubling every year or two. This trend has remained valid for over 40 years, and it is thought that it will remain valid for another 15 years or so, until transistors reach atomic size. 2.2. 1 bit/atom by 2020 Extrapolating Moore’s Law down to storing one bit of information on a single atom by about 2020, means that a handheld object will be able to store a trillion trillion bits of information. Such a device is called an “Avogadro Machine (AM)”. 2.3. Femto-Second Switching An Avogadro Machine can switch the state of a single atom (0 1) in a femtosecond, i.e. a quadrillionth of a second (10-15 sec.), so that the total processing speed of an AM is roughly 1040 bits per second. 2.4. Reversible Computing If computing technology continues to use its traditional irreversible computational style, the heat generated in atomic scale circuits will be so great, they will explode, so a reversible, information preserving, computing style will be needed, usually called

H. de Garis / The Artilect War: Cosmists vs. Terrans

439

“reversible computing”, that does not generate heat, hence will allow 3D computing, and no limit to size. Artilects can become the size of asteroids, kilometers across, with vast computing capacities. 2.5. Nanotech(nology) Nanotech (i.e. molecular scale engineering) will allow AMs to be built. Nanotech will thus allow artilects to be built, once we know how to build brain like circuits. Nanotech is the “enabling technology” for artilect building. 2.6. Artificial Embryology One of the greatest challenges of 21st century biology is to understand “development”, i.e. the embryogenic process, i.e. how a fertilized single cell grows into a 100 trillion cell animal such as ourselves. Once this process is well understood, technology will be able to create an artificial embryology, to manufacture products, hence “embryofacture” (embryological manufacture). Embryofacture will be used to build 3D complex artilects. 2.7. Evolutionary Engineering The complexities of artilect building will be so great (e.g. the human brain has a quadrillion (1015) synapses (connections between neurons in the brain)), that an evolutionary engineering approach will be needed, which applies a “Genetic Algorithm” approach to engineering products. Artilects will be built using this technique. 2.8. (Topological) Quantum Computing Quantum computing is potentially exponentially more powerful than classical computing. It can compute 2N things at a time, compared to classical computing’s 1 thing at a time, where N is the number of (qu)bits in the register of the quantum computer. Topological quantum computers (TQCs) store and manipulate the qubits in topological quantum fields, and are thus robust against noise. TQC will soon make quantum computers practical. Artilects will be TQC devices. 2.9. Nanotech Impact on Brain Science Today’s top supercomputers are close to reaching the estimated bit processing rate of the human brain, (i.e. about 1016 bits per second), but they are far from being intelligent by human standards. What is needed to make them humanly intelligent is knowledge from the neurosciences on how the human brain uses its brain circuits to perform intelligent tasks. Nanotech will furnish neuroscience with powerful new tools to discover how the brain works. This knowledge will be quickly incorporated into the building of artilects.

440

H. de Garis / The Artilect War: Cosmists vs. Terrans

2.10. Artificial Brains The above technologies will result in the creation of an artificial brain industry and the creation of rival national brain building institutions and projects equivalent to NASA and ESA for space travel. In time, the brain building industry will become the world’s largest. 3. The Artilect : Capacities 1024 Times Above Human Levels As stated in the above section, the estimated bit processing rate of the human brain is approximately 1016 bit flips per second. This figure is derived from the fact that the human brain has about 100 billion neurons (1011), with each neuron synapsing (connecting) with roughly ten thousand other neurons (104), hence there are a quadrillion synapses, each signaling at a maximum rate of about 10 bits per second. Thus the human bit processing rate is 1011+4+1 = 1016 bits per second. As mentioned in the previous section, a hand held artilect could flip at 1040 bits per second. An asteroid sized artilect could flip at 1052 bits a second. Thus the raw bit processing rate of the artilect could be a trillion trillion trillion (1036) times greater than the human brain. If the artilect can be made intelligent, using neuroscience principles, it could be made to be truly godlike, massively intelligent and immortal.

4. The Species Dominance Debate Starts The “species dominance” debate has already started, at least in the English speaking countries and China. The fundamental question is whether humanity should build artilects or not. This issue will dominate our global politics this century, and may lead to a major war killing billions of people. As the artificial brain based products (e.g. genuinely useful household robots) become smarter every year, people will be asking questions such as “Will the robots become as smart as us?” “Will they become smarter than us?” “Should humanity place an upper limit on robot and artificial brain intelligence?” “Can the rise of artificial intelligence be stopped?” “If not, then what are the consequences for human survival if we become the Number 2 species?” The question “Should humanity build godlike, massively intelligent artilects?” is the most important of the 21st century, and will dominate our century’s global politics. It is the equivalent of the question which dominated 19th and 20th century global politics, i.e. “Who should own capital?” which led to the rise of the Capitalist-Communist dichotomy and the cold war.

5. Cosmists, Terrans, Cyborgs As the species dominance debate begins to heat up, humanity will split into two (possibly three) major philosophical groups, namely – a) The Cosmists (based on the word Cosmos). Cosmist ideology is in favor of building artilects. (See section 6 for arguments in favor of Cosmism).

H. de Garis / The Artilect War: Cosmists vs. Terrans

441

b) The Terrans (based on the word Terra = the earth). Terran ideology is opposed to building artilects. (See section 7 for arguments in favor of Terranism). c) The Cyborgs (based on the words “cybernetic organism” = part machine, part human). Cyborgs want to become artilects themselves by adding artilectual components to their own brains. (See section 8 for arguments in favor of Cyborgism). The dispute between the Cosmists and the Terrans will be so bitter that a major war is likely in the second half of the century.

6. Arguments of the Cosmists 6.1. “Big Picture” Argument Human beings live a puny 80 years in a universe billions of years old, that contains a trillion trillion stars. The cosmos is the “big picture”. Cosmists want artilects to become a part of that big picture, understanding it, traveling thru it, manipulating it, etc., hence the name of the ideology “Cosmism”. The preoccupations of human beings seem pathetic in comparison. 6.2. Scientific Religion Most Cosmists are not religious, viewing traditional religions as superstitions invented thousand of years ago before the rise of science. But as humans they feel the pangs of religious impulse. Such impulses could be satisfied by Cosmism, a “scientist’s religion” due to its awe, its grandeur, its energizing, its vision. 6.3. Building Artilect Gods The primary aim of the Cosmists will be to build artilects. It will be a kind of religion to them, the next step up the evolutionary ladder, the “destiny of the human species to serve as the stepping stone to the creation of a higher form of being”. In building artilects, the Cosmists will feel they are building gods. 6.4. Human Striving, Cannot be Stopped It is human nature to be curious, to strive. Such tendencies are built into our genes. Building godlike artilects will be inevitable, because we will choose to do it. It would run counter to human nature not to do it. 6.5. Economic Momentum Once the artificial brain and intelligent robot industries become the world’s largest, it will be very difficult to stop their growth. The economic momentum will be enormous.

442

H. de Garis / The Artilect War: Cosmists vs. Terrans

6.6. Military Momentum The military momentum will be even greater. In the time frame we are talking about, China will overtake the US as the century’s dominant power. Since China is still a brutal one party dictatorship, it is despised by the US, so political rivalries will only heat up. The two ministries of defense cannot afford to allow the other to get ahead of it in intelligent soldier robot design etc. Hence Cosmism will be an entrenched philosophy in the respective defense departments.

7. Arguments of the Terrans 7.1. Preserve the Human Species The major argument of the Terrans is that the artilects, once sufficiently superior to human beings, may begin to see us as grossly inferior pests, and decide to wipe us out. As artilects, that would be easy for them. The Terrans would prefer to kill off a few million Cosmists for the sake of the survival of billions of human beings. Recent wars were about the survival of countries. An artilect war would be about the survival of the human species. Since the size of the stake is much higher, so will the passion level in the artilect war debate. 7.2. Fear of Difference Terrans will be horrified at the idea of seeing their children becoming artilects, thus becoming utterly alien to them. They will reject the idea viscerally and fear the potential superiority of the artilects. They will organize to prevent the rise of the artilects and will oppose the Cosmists, ideologically, politically, and eventually militarily. 7.3. Rejection of the Cyborgs The Terrans will also be opposed to the Cyborgs, because to a Terran, there is little difference between an advanced Cyborg and an artilect. Both are artilect like, given the gargantuan bit processing rate of nanoteched matter that can be added to the brains of human beings. The Terrans will lump the Cyborgs into the Cosmist camp ideologically speaking. 7.4. Unpredictable Complexity Given the likelihood that artilects will be built using evolutionary engineering, the behavior of artilects will be so complex as to be unpredictable, and therefore potentially threatening to human beings. One of the keywords in the artilect debate is “risk”. Terran global politicians need to hope for the best (e.g. the artilects will leave the planet in search of bigger things and ignore puny humans) and prepare for the worst, i.e. exterminating the Cosmists, for the sake of the survival of the human species.

H. de Garis / The Artilect War: Cosmists vs. Terrans

443

7.5. Cosmist Inconsideration The Terrans will argue that the Cosmists are supremely selfish, since in building artilects, not only will they put the lives of the Cosmists at risk if the artilects turn against them, but the lives of the Terrans as well. To prevent such a risk, the Terrans will, when push really comes to shove, decide to wipe out the Cosmists, for the greater good of the survival of the human species. 7.6. “First Strike” Time Window to React against the Cosmists/Cyborgs The Terrans will be conscious that they cannot wait too long, because if they do, the Cyborgs and the artilects will have already come into being. The Terrans will then run the risk of being exterminated by the artilects. So the Terrans will be forced into a “first strike” strategy. They will have to kill off the Cosmists and Cyborgs before it is too late. If not, the artilects and Cyborgs will have become too intelligent, too powerful in any human-machine confrontation and will easily defeat the humans. But the Cosmists will be reading the Terran arguments and preparing for an “artilect war” against the Terrans, using late 21st century weaponry.

8. Arguments of the Cyborgs 8.1. Become Artilect Gods Themselves The primary aim of the Cyborgs is to become artilects themselves by adding artilectual components to their own human brains, converting themselves bit by bit into artilects. Instead of watching artilects become increasingly intelligent as observers, Cyborgs want that experience for themselves. They want to “become gods”. 8.2. Avoid the Cosmist/Terran Clash Some Cyborgs argue that by having human beings become artilects themselves, the dichotomy between the Cosmists and the Terrans can be avoided, because all human beings would become artilects. The Terrans of course will reject the Cyborgs and lump them with the Cosmists and artilects. In fact, the growing presence of Cyborgs in daily life will only hasten the alarm of the Terrans and bring their first strike closer.

9. How the Artilect War Heats Up 9.1. Nanotech Revolutionizes Neuroscience Nanoteched, molecular sized robots will revolutionize neuroscience, because they will provide a powerful new tool to understand how the brain works. An entire human brain can be simulated in vast nanoteched computers and investigated “in hardware”. Neuroscience will finally be in a position to explain how brains make human beings intelligent. That knowledge will be implemented in the artilects.

444

H. de Garis / The Artilect War: Cosmists vs. Terrans

9.2. Neuro-Engineering Weds with Neuro-Science In time, neuro-science and neuro-engineering will interact so closely that they will become one, in the same way as theoretical and experimental physics are two aspects of the same subject. Neuroscientists will be able to test their theories on artificial brain models, thus rapidly increasing the level of understanding of how intelligence arises and how it is embodied. 9.3. Artificial Brain Technology Creates Massive Industries With a much higher level of artificial intelligence, based on knowledge of the human brain, artificial brains and artificial brain based robots will become a lot more intelligent and hence useful as domestic appliances. A vast industry of artificial brain based products will be created, becoming the world’s largest. 9.4. “Intelligence Theory” is Developed Once neuroscientists and brain builders understand how human intelligence is created, new theories of the nature of intelligence will be created by the “theoretical neuroscientists”. An “intelligence theory” will be created. Human intelligence will be just one “data point” in the space of possible intelligences. Intelligence theory should show how it is possible to increase intelligence levels. It will be able to explain why some people are smarter than others, or why humans are smarter than apes, etc. 9.5. Artilects Get Smarter Every Year As a result of the marriage of neuroscience and neuroengineering, the artificial brain based industries will deliver products that increase their intelligence every year. This trend of growing intelligence will cause people to ask the questions mentioned in section 4. The species dominance debate will spread from the intellectual technocrats to the general public. 9.6. Debate Begins to Rage, Political Parties Form As the IQ gap between the robots and human beings gets increasingly smaller, the species dominance debate will begin to rage. Political parties will form, divided essentially into the 3 main schools of thought on the topic, Cosmist, Terran, Cyborg. The rhetorical exchange will be become less polite, more heated. 9.7. The Debate Turns Violent, Assassination, Sabotage When people are surrounded by ever increasingly intelligent robots and other artificial brain based products, the general level of alarm will increase to the point of panic. Assassinations of brain builder company CEOs will start, robot factories will be arsoned and sabotaged etc. The Cosmists will be forced to strengthen their resolve. The artilect war will be drawing ever closer.

445

H. de Garis / The Artilect War: Cosmists vs. Terrans

9.8. The Terrans Will “First Strike”, Before Its Too Late For Them The Terrans will have been organizing for a first strike and will have made preparations. They will then take power in a world wide coup of the global government that is likely to exist by mid century, and begin exterminating the Cosmists and Cyborgs in a global purge, killing millions of them, or at least that is the Terran plan. 9.9. Cosmists Anticipate this First Strike and are Ready But the Cosmists will be following the arguments of the Terrans and Cyborgs very closely, and will equally be preparing for a confrontation with the Terrans. They will have their own plans and their own weapons and military. If the Terrans strike first, a quick reply will follow from the Cosmists, and the artilect war will have begun. 9.10. Late 21st Century Weapons, Leads to Gigadeath War If one extrapolates up the graph of the number of people killed in major wars from the early 19th century (the Napoleonic wars) to late 21st century (the artilect war), then one predicts that billions of people will be killed, using late 21st century weapons (see the graph in the next section). This “gigadeath” figure is the characteristic number of deaths in any major late 21st century war. About 300 million people were killed for political reasons in the 20th century.

10. Gigadeath

Number of People Killed in Major Wars

Artilect War, Billions

WW2, 50-100 Million WW1, 20 Million

Napoleonic Wars, 1 Million

1800

1900

2000

2100

446

H. de Garis / The Artilect War: Cosmists vs. Terrans

11. Vote At the end of the talks I give on this topic, I usually invite my audiences to vote on the following question : “Do you feel personally that humanity should build artilects, these godlike massively intelligent machines, despite the risk that they might decide, in a highly advanced form, to wipe out humanity? Yes or No. The result is usually around a 50/50, 60/40, 40/60 Cosmist/Terran split. I noticed that most people, like myself, are highly ambivalent about artilect building. They are awed by the prospect of what artilects could become, and horrified at the prospect of a gigadeath artilect war. The fact that the Cosmist/Terran split is so even will make the artilect war all the more divisive and bitter. This divisiveness can be expressed in the form of the following slogan : Do we build gods, or do we build our potential exterminators?

12. Appeal to Philosophers There is immense scope for philosophical discussion on the artilect issue. At the present time, the philosophical community is largely unaware of the issue, so need to be educated. It is not surprising that the debate is still largely confined to the technocrats, who are better informed of what is coming in technological terms. It is this community after all that is creating the problem (e.g. I am directing a “China-Brain Project”, a 3 million RMB, 4 year project to build a 15,000 evolved neural net module based artificial brain, starting early in 2008). The philosophers will need to create a new branch of applied ethics, that I call “artilect ethics” which will consider such questions as the rights of the artilects relative to human beings etc. This new field is rich with questions that the moral and political philosophers need to discuss, once they are informed.

13. Quote and Publicity “I’m glad I’m alive now. At least I will die peacefully in my bed. However, I truly fear for my grandchildren. They will be caught up in the Artilect War, and will probably be destroyed by it”. Prof. Hugo de Garis, 2000 (Discovery Channel)

Kurzweil vs. de Garis on the BBC To see a clash of opinions on whether the rise of the artilect will be a good or bad thing for humanity, see the BBC TV program “Human V2.0” in which Prof de Garis and Dr. Ray Kurzweil discuss the topic. To watch this program you can google with the terms “Human V2.0” and “BBC”. In this program Dr. Ray Kurzweil is optimistic and Prof. Hugo de Garis is pessimistic.

H. de Garis / The Artilect War: Cosmists vs. Terrans

447

Reference [1]

Hugo de Garis, “The Artilect War : Cosmists vs. Terrans : A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines”, ETC Books, 2005, ISBN 0882801546 (available at www.amazon.com).

448

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

Stages of Ethical Development in Artificial General Intelligence Systems Ben GOERTZEL and Stephan Vladimir BUGAJ Novamente LLC and AGIRI Institute, Washington, DC

Abstract. A novel theory of the stages of ethical development in intelligent systems is proposed, incorporating prior related theories by Kohlberg and Gilligan, as well as Piaget’s theory of cognitive development. This theory is then applied to the ethical development of integrative AGI systems that contain components carrying out simulation and uncertain inference – the key hypothesis being that effective integration of these components is central to the ascent of the AGI system up the ethical-stage hierarchy. Keywords. Intelligent virtual agents, ethics, stages of moral development, ethics of care, ethics of justice, uncertain inference, simulation, theory of mind.

Introduction Ethical judgment, in human beings, is acquired via a complex combination of genetic wiring, explicit instruction, and embodied mutual experience. Developmental theorists have identified various stages that young humans pass through on their way to achieving mature ethical faculties. Here we review some of these stage-based models with a view toward how they may be integrated and then transplanted to the domain of AGI ethics. We also correlate these stages with the five-imperatives view of ethics we have presented in [1], showing how earlier stages focus on three of the imperatives, with the other two imperatives rising to the fore as later stages are entered. Perry’s [2, 3] and especially Piaget’s [4] theories of developmental stages form a foundation for our considerations here, but the essential original contribution is an identification of Kolhberg’s [5, 6] and Gilligan’s [7] complementary theories of ethical developmental stages with different components of contemporary AGI architectures. Kolhberg’s stages, with their focus on abstract judgment, work nicely as a roadmap for the ethical development of logical inference engines; whereas Gilligan’s stages, with their focus on empathy, are more pertinent as a model of the ethical development of internal-simulation-based AI. This leads to a notion of integrative ethical development as consisting of coordinated inferential and simulative ethical development. Along these lines, we present a novel theory of the stages of ethical development, incorporating Gilligan, Kohlberg and Piaget, and then apply this theory to AGI systems.

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

449

While many of the ideas discussed here would be meaningful more generally, for sake of concreteness we mainly restrict attention here to ethical development within AGI systems that contain and integrate components carrying out both uncertain logical inference and simulation. The Novamente Cognition Engine (or NCE, see [8, 9]) is one example of such an AGI architecture, but far from the only possible one. In the context of this sort of AGI architecture, we argue, integrative ethical development emerges as a consequence of overall coordination of ongoing development between these two components.

1. Stages of Cognitive Development The best known theory of cognitive development is that of Jean Piaget. In a prior paper [10] we have presented a slightly modified form of the Piagetan developmental hierarchy, defined as follows: Table 1. Modified Piagetan Developmental Stages

Stage Infantile

Concrete

Formal Reflexive

Example During this stage, the child learns about himself and his environment through motor and reflex actions. Thought derives from sensation and movement. Object permanence and the distinction between self and other are among the things learned. A rich usage of language emerges here, along with the usage of symbols to represent objects, the ability and propensity to think about things and events that aren't immediately present, and a robust but not totally flexible theory of mind. Ability to think abstractly and to make rational judgements about concrete or observable phenomena. Logical reasoning and systematic experimentation. Ability to modify one’s own modes of thinking, reasoning, experimentation and self-perception at a fundamental level.

In that same paper we have defined a specific theory explaining how these stages manifest themselves in the development of AGI systems based on uncertain logical inference: Table 2. Piagetan Developmental Stages for Uncertain Inference Based AGI Systems

Stage Infantile

Concrete

Formal

Reflexive

Operational Aspect Able to recognize patterns in and conduct inferences about the world, but only using simplistic hard-wired (not experientially learned) inference control schema. Able to carry out more complex chains of reasoning regarding the world, via using inference control schemata that adapt behavior based on experience (reasoning about a given case in a manner similar to prior cases). Able to carry out arbitrarily complex inferences (constrained only by computational resources) via including inference control as an explicit subject of abstract learning. Capable of self-modification of internal structures.

Also relevant is William Perry's [2, 3] theory of the stages (“positions” in his writings) of intellectual and ethical development, which constitutes a model of iterative

450

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

refinement of approach in the developmental process of coming to intellectual and ethical maturity. These form an analytical tool for discerning the modality of belief of an intelligence by describing common cognitive approaches to handling the complexities of real world ethical considerations. Table 3. Perry’s Developmental Stages [with corresponding Piagetan Stages in brackets]

Stage Dualism / Received Knowledge [Infantile]

Multiplicity [Concrete]

Relativism / Procedural Knowledge [Formal] Commitment / Constructed Knowledge [Formal / Reflexive]

Substages Basic duality (“All problems are solvable. I must learn the correct solutions.”) Full dualism (“There are different, contradictory solutions to many problems. I must learn the correct solutions, and ignore the incorrect ones”) Early multiplicity (“Some solutions are known, others aren't. I must learn how to find correct solutions.”) Late Multiplicity: cognitive dissonance regarding truth. (“Some problems are unsolvable, some are a matter of personal taste, therefore I must declare my own intellectual path.”) Contextual Relativism (“I must learn to evaluate solutions within a context, and relative to supporting observation.”) Pre-Commitment (“I must evaluate solutions, then commit to a choice of solution.”) Commitment (“I have chosen a solution.”) Challenges to Commitment (“I have seen unexpected implications of my commitment, and the responsibility I must take.”) Post-Commitment (“I must have an ongoing, nuanced relationship to the subject in which I evaluate each situation on a case-by-case basis with respects to its particulars rather than an ad-hoc application of unchallenged ideology.”)

2. Stages of Development of the Ethics of Justice Complementing generic theories of cognitive development such as Piaget’s and Perry’s, theorists have also proposed specific stages of moral and ethical development. The two most relevant theories in this domain are those of Kohlberg and Gilligan, which we will review here, both individually and in terms of their integration and application in the AGI context. Lawrence Kohlberg’s [5, 6] moral development model, called the “ethics of justice” by Gilligan, is based on a rational modality as the central vehicle for moral development. This model is based on an impartial regard for persons, proposing that ethical consideration must be given to all individual intelligences without a priori judgment (prejudice). Consideration is given for individual merit and preferences, and the goals of an ethical decision are equal treatment (in the general, not necessarily the particular) and reciprocity. Echoing Kant’s [11] categorical imperative, the decisions considered most successful in this model are those which exhibit "reversibility", where a moral act within a particular situation is evaluated in terms of whether or not the act would be satisfactory even if particular persons were to switch roles within the situation. In other words, a situational, contextualized “do unto others as you would have them do unto you” criteria. The ethics of justice can be viewed as three stages (each of which has six substages, on which we will not elaborate here):

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

451

Table 4. Kohlberg’s Stages of Development of the Ethics of Justics

Stage

Substages

Pre-Conventional

Obedience and Punishment Orientation Self-interest orientation Interpersonal accord (conformity) orientation Authority and social-order maintaining (law and order) orientation Social contract (human rights) orientation Universal ethical principles (universal human rights) orientation

Conventional

Post-Conventional

In Kohlberg’s perspective, cognitive development level contributes to moral development, as moral understanding emerges from increased cognitive capability in the area of ethical decision making in a social context. Relatedly, Kohlberg also looks at stages of social perspective and their consequent interpersonal outlook. These are correlated to the stages of moral development, but also map onto Piagetian models of cognitive development (as pointed out e.g. by Gibbs [12], who presents a modification/interpretation of Kohlberg’s ideas intended to align them more closely with Piaget’s). Interpersonal outlook can be understood as rational understanding of the psychology of other persons (a theory of mind, with or without empathy). Stage one, emergent from the infantile cognitive stage, is entirely selfish as only self awareness has developed. As cognitive sophistication about ethical considerations increases, so do the moral and social perspective stages. Concrete and formal cognition bring about the first instrumental egoism, and then social relations and systems perspectives, and from formal and then reflexive thinking about ethics comes the postconventional modalities of contractualism and universal mutual respect. Table 5. Kohlberg’s Stages of Development of Social Perspective and Interpersonal Morals

Stage of Social Perspective

Interpersonal Outlook

Blind egoism

No interpersonal perspective. Only self is considered. See that others have goals and perspectives, and either conform to or rebel against norms. Able to see abstract normative systems

Instrumental egoism Social Relationships perspective Social Systems perspective Contractual perspective Universal principle of mutual respect

Recognize positive and negative intentions Recognize that contracts (mutually beneficial agreements of any kind) will allow intelligences to increase the welfare of both. See how human fallibility and frailty are impacted by communication.

2.1. Uncertain Inference and the Ethics of Justice Taking cue from the analysis given in [10] of Piagetan stages in uncertain inference based AGI systems, we may explore the manifestation of Kohlberg’s stages

452

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

in AGI systems of this nature. Uncertain inference seems generally well-suited as an ethical learning system, due to the nuanced ethical environment of real world situations. An uncertain inference system, as defined in that previous paper, consists of four components: a content representation scheme (e.g. predicate logic, term logic, fuzzy logic); an uncertainty representation scheme (e.g. fuzzy truth values, probability values, probability intervals, imprecise probabilities, indefinite probabilities); a set of inference rules (e.g. those used in the NARS [13] or PLN [14] inference systems; and a set of inference control schemata (which in the Novamente CognitionEngine (NCE) are provided when PLN is integrated into the overall NCE framework.) In general, an uncertain inference system may be viewed as a framework for dynamically updating a probabilistically weighted semantic network based on new incoming information and based on new conclusions derived via combining nodes and links in the network in appropriate, probabilistically grounded ways. Probabilistic knowledge networks can model belief networks, imitative reinforcement learning based ethical pedagogy, and even simplistic moral maxims. In principle, they have the flexibility to deal with complex ethical decisions, including not only weighted “for the greater good” dichotomous decision making, but also the ability to develop moral decision networks which do not require that all situations be solved through resolution of a dichotomy. When more than one person is being affected by an ethical decision, making a decision based on reducing two choices to a single decision can often lead to decisions of dubious ethics. However, a sufficiently complex uncertain inference network can represent alternate choices in which multiple actions are taken that have equal (or near equal) belief weight but have very different particulars – but because the decisions are applied in different contexts (to different groups of individuals) they are morally equivalent. Infantile and concrete cognition are the underpinnings of the egoist and socialized stages, with formal aspects also playing a role in a more complete understanding of social models when thinking using the social modalities. Cognitively infantile patterns can produce no more than blind egoism or compassion as without a theory of mind or a refined capability for empathy, there is no capability to consider the other in a contextually appropriate way. Since most intelligences acquire concrete modality and therefore some nascent social perspective relatively quickly, most egoists are instrumental egoists. The social relationship and systems perspectives include formal aspects which are achieved by systematic social experimentation, and therefore experiential reinforcement learning of correct and incorrect social modalities. Initially this is a one-on-one approach (relationship stage), but as more knowledge of social action and consequences is acquired, a formal thinker can understand not just consequentiality but also intentionality in social action. Extrapolation from models of individual interaction to general social theoretic notions is also a formal action. Rational, logical positivist approaches to social and political ideas, however, are the norm of formal thinking. Contractual and committed moral ethics emerges from a higher-order formalization of the social relationships and systems patterns of thinking. Generalizations of social observation become, through formal analysis, systems of social and political doctrine. Highly committed, but grounded and logically supportable, belief is the hallmark of formal cognition as expressed in the contractual moral stage. Though formalism is at work in the socialized moral stages, its fullest expression is in committed contractualism.

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

453

Finally, reflexive cognition is especially important in truly reaching the postcommitment moral stage in which nuance and complexity are accommodated. Because reflexive cognition is necessary to change one's mind not just about particular rational ideas, but whole ways of thinking, this is a cognitive precedent to being able to reconsider an entire belief system, one that has had contractual logic built atop reflexive adherence that began in early development. If the initial moral system is viewed as positive and stable, than this cognitive capacity is seen as dangerous and scary, but if early morality is stunted or warped, then this ability is seen as enlightened. However, achieving this cognitive stage does not mean one automatically changes their belief systems, but rather that the mental machinery is in place to consider the possibilities. Because many people do not reach this level of cognitive development in the area of moral and ethical thinking, it is associated with negative traits (“moral relativism” and “flip-flopping”). However, this cognitive flexibility generally leads to more sophisticated and applicable moral codes, which in turn leads to morality which is actually more stable because it is built upon extensive and deep consideration rather than simple adherence to reflexive or rationalized ideologies.

3. Stages of Development of Empathic Ethics Complementing Kohlberg’s logic-and-justice-focused approach, Carol Gilligan's [7] “ethics of care” model is a moral development theory which posits that empathetic understanding plays the central role in moral progression from an initial self-centered modality to a socially responsible one. For this approach to be applied in an AGI, the AGI must be capable of internal simulation of other minds it encounters, in a similar manner to how humans regularly simulate one another internally [15]. Without any mechanism for internal simulation, it is unlikely that an AGI can develop any sort of empathy toward other minds, as opposed to merely logically or probabilistically modeling other agents’ behavior or other minds’ internal contents. The ethics of care model is concerned with the ways in which an individual cares (responds to dilemmas using empathetic responses) about self and others. The ethics of care is broken into the same three primary stage as Kohlberg, but with a focus on empathetic, emotional caring rather than rationalized, logical principles of justice: Table 6. Gilligan’s Stages of the Ethics of Care

Stage

Principle of Care

Pre-Conventional Conventional Post-Conventional

Individual Survival Self Sacrifice for the Greater Good Principle of Nonviolence (do not hurt others, or oneself)

In Gilligan’s perspective, the earliest stage of ethical development occurs before empathy becomes a consistent and powerful force. Next, the hallmark of the conventional stage is that at this point, the individual is so overwhelmed with their empathic response to others that they neglect themselves in order to avoid hurting others. Note that this stage doesn’t occur in Kohlberg’s hierarchy at all. Kohlberg and Gilligan both begin with selfish unethicality, but their following stages diverge. A person could in principle manifest Gilligan’s conventional stage without having a refined sense of justice (thus not entering Kohlberg’s conventional stage); or they could manifest Kohlberg’s conventional stage without partaking in an excessive degree of self-sacrifice (thus not entering Gilligan’s conventional stage). We will suggest below

454

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

that in fact the empathic and logical aspects of ethics are more unified in real human development than these separate theories would suggest. It is interesting to note that Gilligan’s and Kohlberg’s final stages converge more closely than their intermediate ones. Kohlberg’s post-conventional stage focuses on universal rights, and Gilligan’s on universal compassion. Still, the foci here are quite different; and, as will be elaborated below, we believe that both Kohlberg’s and Gilligan’s theories constitute very partial views of the actual end-state of ethical advancement. Gilligan’s theory was proposed partly as a reaction to the perceived male bias of Kohlberg’s thoery. There is certainly some meat to this complaint, as there is much evidence that females tend to be more empathic in their response to ethical judgment, whereas men tend to be more focused on abstract notions of rights and fairness. In general, however, we feel that, just as Kohlberg gives short shrift to empathy, Gilligan gives short shrift to logical reasoning, and that due to these limitations of perspective, both theorists have failed to create adequately scoped theories of ethical development.

4. An Integrative Approach to Ethical Development We deny the false dichotomy of a “feminine” ethics of care vs. a “masculine” ethics of justice, and propose that both Kohlberg's and Gilligan's theories contain elements of the whole picture of ethical development, and that both approaches are necessary to create a moral, ethical artificial general intelligence -- just as, we suggest, both internal simulation and uncertain inference are necessary to create a sufficiently intelligent and volitional intelligence in the first place. Also, we contend, the lack of direct analysis of the underlying psychology of the stages is a deficiency shared by both the Kohlberg and Gilligan models as they are generally discussed. A successful model of integrative ethics necessarily contains elements of both the care and justice models, as well as reference to the underlying developmental psychology and its influence on the character of the ethical stage. With these notions in mind, we propose the following integrative theory of the stages of ethical development, shown in the table at the end of this section. In our integrative model, the justice-based and empathic aspects of ethical judgment are proposed to develop together. Of course, in any one individual, one or another aspect may be dominant. Even so, however, the combination of the two is equally important as either of the two individual ingredients. For instance, we suggest that in any psychologically healthy human, the conventional stage of ethics (typifying childhood, and in many cases adulthood as well) involves a combination of Gilligan-esqe empathic ethics and Kohlberg-esque ethical reasoning. This combination is supported by Piagetan concrete operational cognition, which allows moderately sophisticated linguistic interaction, theory of mind, and symbolic modeling of the world. And, similarly, we propose that in any truly ethically mature human, empathy and rational justice are both fully developed. Indeed the two interpenetrate each other deeply. Once one goes beyond simplistic, childlike notions of fairness (“an eye for an eye” and so forth), applying rational justice in a purely intellectual sense is just as difficult as any other real-world logical inference problem. Ethical quandaries and quagmires are easily encountered, and are frequently cut through by a judicious application of empathic simulation.

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

455

On the other hand, empathy is a far more powerful force when used in conjunction with reason: analogical reasoning lets us empathize with situations we have never experience. For instance, a person who has never been clinically depressed may have a hard time empathizing with individuals who are; but using the power of reason, they can imagine their worst state of depression magnified by several times and then extended over a long period of time, and then reason about what this might be like ... and empathize based on their inferential conclusion. Reason is not antithetical to empathy but rather is the key to making empathy more broadly impactful. Finally, the enlightened stage of ethical development involves both a deeper compassion and a more deeply penetrating rationality and objectiveness. Empathy with all sentient beings is manageable in everyday life only once one has deeply reflected on one’s own self and largely freed oneself of the confusions and illusions that characterize much of the ordinary human’s inner existence. It is noteworthy, for example, that Buddhism contains both a richly developed ethics of universal compassion, and also an intricate logical theory of the inner workings of cognition [16], detailing in exquisite rational detail the manner in which minds originate structures and dynamics allowing them to comprehend themselves and the world. 4.1. The Stages of Ethical Development and the Five Ethical Imperatives In [1] we have proposed a series of five ethical imperatives and explored their implications for interacting with, and teaching ethics to, AGI systems: 1. the imitability imperative (i.e. the folk Golden Rule “do unto others as one would have them do unto you”) fairly narrowly and directly construed): the goal of acting in a way so that having others directly imitate one’s actions, in directly comparable contexts, is desirable to oneself; 2. the comprehensibility imperative: the goal of acting in a way so that others can understand the principles underlying one’s actions; 3. experiential groundedness. An intelligent agent should not be expected to act according to an ethical principle unless there are many examples of the principle-in-action in its own direct or observational experience; 4. Kant’s categorical imperative: choose behaviors according to a certain maxim only if you would will that maxim to be followed universally by all sentient beings; 5. logical coherence. An ethical system should be roughly logically coherent, in the sense that the different principles within it should mesh well with one another and perhaps even naturally emerge from each other. Specific ethical qualities corresponding to the five imperatives have been italicized in the above table of developmental stages. Firstly, it seems that imperatives 1-3 are critical for the passage from the pre-ethical to the conventional stages of ethics. A child learns ethics largely by copying others, and by being interacted with according to simply comprehensible implementations of the Golden Rule. In general, when interacting with children learning ethics, it is important to act according to principles they can comprehend. And given the nature of the concrete stage of cognitive development, experiential groundedness is a must. As a hypothesis regarding the dynamics underlying the psychological development of conventional ethics, what we propose is as follows: The emergence of concrete-stage cognitive capabilities leads to the capability for fulfillment of ethical imperatives 1 and 2 – a comprehensible and workable implementation of the Golden Rule, based on a

456

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

Table 7. Integrative Model of the Stages of Ethical Development

Stage Pre-ethical

Conventional Ethics

Mature Ethics

Enlightened Ethics

Characteristics Piagetan infantile to early concrete (aka pre-operational) Radical selfishness or selflessness may, but do not necessarily, occur No coherent, consistent pattern of consideration for the rights, intentions or feelings of others Empathy is generally present, but erratically Concrete cognitive basis Perry’s Dualist and Multiple stages The common sense of the Golden Rule is appreciated, with cultural conventions for abstracting principles from behaviors One’s own ethical behavior is explicitly compared to that of others Development of a functional, though limited, theory of mind Ability to intuitively conceive of notions of fairness and rights Appreciation of the concept of law and order, which may sometimes manifest itself as systematic obedience or systematic disobedience Empathy is more consistently present, especially with others who are directly similar to oneself or in situations similar to those one has directly experienced Degrees of selflessness or selfishness develop based on ethical groundings and social interactions. Formal cognitive basis Perry’s Relativist and “Constructed Knowledge” stages The abstraction involved with applying the Golden Rule in practice is more fully understood and manipulated, leading to limited but nonzero deployment of the Categorical Imperative Attention is paid to shaping one’s ethical principles into a coherent logical system Rationalized, moderated selfishness or selflessness. Empathy is extended, using reason, to individuals and situations not directly matching one’s own experience Theory of mind is extended, using reason, to counterintuitive or experientially unfamiliar situations Reason is used to control the impact of empathy on behavior (i.e. rational judgments are made regarding when to listen to empathy and when not to) Rational experimentation and correction of theoretical models of ethical behavior, and reconciliation with observed behavior during interaction with others. Conflict between pragmatism of social contract orientation and idealism of universal ethical principles. Understanding of ethical quandaries and nuances develop (pragmatist modality), or are rejected (idealist modality). Pragmatically critical social citizen. Attempts to maintain a balanced social outlook. Considers the common good, including oneself as part of the commons, and acts in what seems to be the most beneficial and practical manner. Reflexive cognitive basis Permeation of the categorical imperative and the quest for coherence through inner as well as outer life Experientially grounded and logically supported rejection of the illusion of moral certainty in favor of a case-specific analytical and empathetic approach that embraces the uncertainty of real social life Deep understanding of the illusory and biased nature of the individual self, leading to humility regarding one’s own ethical intuitions and prescriptions Openness to modifying one’s deepest, ethical (and other) beliefs based on experience, reason and/or empathic communion with others Adaptive, insightful approach to civil disobedience, considering laws and social customs in a broader ethical and pragmatic context Broad compassion for and empathy with all sentient beings A recognition of inability to operate at this level at all times in all things, and a vigilance about self-monitoring for regressive behavior.

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

457

combination of inferential and simulative cognition (operating largely separately at this stage, as will be conjectured below). The effective interoperation of ethical imperatives 1-3, enacted in an appropriate social environment, then leads to the other characteristics of the conventional ethical stage. The first three imperatives can thus be viewed as the seed from which springs the general nature of conventional ethics. On the other hand, logical coherence and the categorical imperative (imperatives 4 and 5) are matters for the formal stage of cognitive development, which come along only with the mature approach to ethics. These come from abstracting ethics beyond direct experience and manipulating them abstractly and formally – a stage which has the potential for more deeply and broadly ethical behavior, but also for more complicated ethical perversions (it is the mature capability for formal ethical reasoning that is able to produce ungrounded abstractions such as “I’m torturing you for your own good”). Developmentally, we suggest that once the capability for formal reasoning matures, the categorical imperative and the quest for logical ethical coherence naturally emerge, and the sophisticated combination of inferential and simulative cognition embodied in an appropriate social context then result in the emergence of the various characteristics typifying the mature ethical stage. Finally, it seems that one key aspect of the passage from the mature to the enlightened stage of ethics is the penetration of these two final imperatives more and more deeply into the judging mind itself. The reflexive stage of cognitive development is in part about seeking a deep logical coherence between the aspects of one’s own mind, and making reasoned modifications to one’s mind so as to improve the level of coherence. And, much of the process of mental discipline and purification that comes with the passage to enlightened ethics has to do with the application of the categorical imperative to one’s own thoughts and feelings – i.e. making a true inner systematic effort to think and feel only those things one judges are actually generally good and right to be thinking and feeling. Applying these principles internally appears critical to effectively applying them externally, for reasons that are doubtless bound up with the interpenetration of internal and external reality within the thinking mind, and the “distributed cognition” phenomenon wherein individual mind is itself an approximative abstraction to the reality in which each individual’s mind is pragmatically extended across their social group and their environment [17].

5. Integrative Ethics and Integrative Artificial General Intelligence And what does our integrative approach to ethical development have to say about the ethical development of AGI systems? The lessons are relatively straighforward, if one considers an AGI system that, like the Novamente Cognition Engine (NCE), explicitly contains components dedicated to logical inference and to simulation. Application of the above ethical ideas to other sorts of AGI systems is also quite possible, but would require a lengthier treatment and so won’t be addressed here. In the context of a NCE-type AGI system, Kolhberg’s stages correspond to increasingly sophisticated application of logical inference to matters of rights and fairness. It is not clear whether humans contain an innate sense of fairness. In the context of AGIs, it would be possible to explicitly wire a sense of fairness into an AGI system, but in the context of a rich environment and active human teachers, this

458

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

actually appears quite unnecessary. Experiential instruction in the notions of rights and fairness should suffice to teach an inference-based AGI system how to manipulate these concepts, analogously to teaching the same AGI system how to manipulate number, mass and other such quantities. Ascending the Kohlberg stages is then mainly a matter of acquiring the ability to carry out suitably complex inferences in the domain of rights and fairness. The hard part here is inference control – choosing which inference steps to take – and in a sophisticated AGI inference engine, inference control will be guided by experience, so that the more ethical judgments the system has executed and witnessed, the better it will become at making new ones. And, as argued above, simulative activity can be extremely valuable for aiding with inference control. When a logical inference process reaches a point of acute uncertainty (the backward or forward chaining inference tree can’t decide which expansion step to take), it can run a simulation to cut through the confusion – i.e., it can use empathy to decide which logical inference step to take in thinking about applying the notions of rights and fairness to a given situation. Gilligan’s stages correspond to increasingly sophisticated control of empathic simulation – which in a NCE-type AGI system, is carried out by a specific system component devoted to running internal simulations of aspects of the outside world, which includes a subcomponent specifically tuned for simulating sentient actors. The conventional stage has to do with the raw, uncontrolled capability for such simulation; and the post-conventional stage corresponds to its contextual, goal-oriented control. But controlling empathy, clearly, requires subtle management of various uncertain contextual factors, which is exactly what uncertain logical inference is good at – so, in an AGI system combining an uncertain inference component with a simulative component, it is the inference component that would enable the nuanced control of empathy allowing the ascent to Gilligan’s post-conventional stage. In our integrative perspective, in the context of an AGI system integrating inference and simulation components, we suggest that the ascent from the pre-ethical to the conventional stage may be carried out largely via independent activity of these two components. Empathy is needed, and reasoning about fairness and rights are needed, but the two need not intimately and sensitively intersect – though they must of course intersect to some extent. The main engine of advancement from the conventional to mature stage, we suggest, is robust and subtle integration of the simulative and inferential components. To expand empathy beyond the most obvious cases, analogical inference is needed; and to carry out complex inferences about justice, empathy-guided inference-control is needed. Finally, to advance from the mature to the enlightened stage, what is required is a very advanced capability for unified reflexive inference and simulation. The system must be able to understand itself deeply, via modeling itself both simulatively and inferentially – which will generally be achieved via a combination of being good at modeling, and becoming less convoluted and more coherent, hence making selfmodeling eacher. Of course, none of this tells you in detail how to create an AGI system with advanced ethical capabilities. What it does tell you, however, is one possible path that may be followed to achieve this end goal. If one creates an integrative AGI system with appropriately interconnected inferential and simulative components, and treats it compassionately and fairly, and provides it extensive, experientially grounded ethical instruction in a rich social environment, then the AGI system should be able to ascend

B. Goertzel and S.V. Bugaj / Stages of Ethical Development in AGI Systems

459

the ethical hierarchy and achieve a high level of ethical sophistication. In fact it should be able to do so more reliably than human beings because of the capability we have to identify its errors via inspecting its internal knowedge-stage, which will enable us to tailor its environment and instructions more suitably than can be done in the human case. If an absolute guarantee of the ethical soundness of an AGI is what one is after, the line of thinking proposed here is not at all useful. However, if what one is after is a plausible, pragmatic path to architecting and educating ethical AGI systems, we believe the ideas presented here constitute a sensible starting-point. Certainly there is a great deal more to be learned and understood – the science and practice of AGI ethics, like AGI itself, are at a formative stage at present. What is key, in our view, is that as AGI technology develops, AGI ethics develops alongside and within it, in a thoroughly coupled way.

References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17]

Bugaj, Stephan Vladimir and Ben Goertzel (2007). Five Ethical Imperatives for AGI Systems. This volume. Perry, William G., Jr. Forms of Intellectual and Ethical Development in the College Years: A Scheme. New York: Holt, Rinehart and Winston, 1970. Perry, William G., Jr. "Cognitive and Ethical Growth: The Making of Meaning", in Arthur W. Chickering and Associates, The Modern American College, San Francisco: Jossey-Bass pp 76-116. 1981. Piaget, Jean. “Piaget’s theory.” In P. Mussen (ed). Handbook of Child Psychology. 4th edition. Vol. 1. New York: Wiley, 1983. Kohlberg, Lawrence; Charles Levine, Alexandra Hewer (1983). Moral stages : a current formulation and a response to critics. Basel, NY: Karger. Kohlberg, Lawrence. Essays on Moral Development, Vol. I: The Philosophy of Moral Development. Harper & Row, 1981. Gilligan, Carol (1982). In a Different Voice. Cambridge, MA: Harvard University Press, 1982. Goertzel, Ben (2006). The Hidden Pattern. BrownWalker Press Goertzel, Ben, Moshe Looks and Cassio Pennachin (2004). Novamente: An Integrative Architecture for Artificial General Intelligence. Proceedings of AAAI Symposium on Achieving Human-Level Intelligence through Integrated Systems and Research, Washington DC, August 2004 Goertzel, Ben and Stephan Vladimir Bugaj (2006). Stages of Cognitive Development in UncertainLogic-Based AGI Systems. In Advances in artificial general intelligence, Ed. by Ben Goertzel and Pei Wang:36-54. Amsterdam: IOS Press. Kant, Immanuel (1964). Groundwork of the Metaphysic of Morals. Harper and Row Publishers, Inc. Gibbs, John (1978). “Kohlberg’s moral stage theory: a Piagetian revision.” Human Development, 1978, 22, 89-112 Wang, Pei (2006). Rigid Flexibility: The Logic of Intelligence. Springer-Verlag Ikle, Matthew and Ben Goertzel (2006). Indefinite Probabilities for General Intelligence. In Advances in Artificial General Intelligence, Edited by Ben Goertzel and Pei Wang, IOS Press Gordon, Robert (1986). Folk Psychology as Simulation. Mind and Language, 1, 158-171, Stcherbatsky, Theodore (2000). Buddhist Logic. Motilal Banarsidass Pub, New York Hutchins, E. (1995) Cognition in the Wild (ISBN 0-262-58146-9

460

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

Engineering Utopia J Storrs Hall Storrmont: Laporte, PA 18626, USA Abstract. The likely advent of AGI and the long-established trend of improving computational hardware promise a dual revolution in coming decades: machines which are both more intelligent and more numerous than human beings. This possibility raises substantial concern over the moral nature of such intelligent machines, and of the changes they will cause in society. Will we have the chance to determine their moral character, or will evolutionary processes and/or runaway self-improvement take the choices out of our hands? Keywords. machine ethics, self-improving AI, Singularity, hard takeoff

Background We can predict with a fair confidence that two significant watersheds will have been passed by 2030: a molecular manufacturing nanotechnology which can produce a wide variety of mechanisms with atomic precision; and artificial intelligence. Detailed arguments for these predictions have been given elsewhere and need not be repeated here (Drexler [1], Kurzweil [2], Moravec [3], Hall [4,5]). We are concerned instead with a joint implication: if both of these technologies are present, greater-than-human intelligence will not only exist, but will be ubiquitous. The net present value (NPV) of an intelligent, educated human being can be estimated at a million dollars.1 Several estimates have been made of the processing power such a machine would need (hereinafter HEPP, for human equivalent processing power): Kurzweil at 1016 IPS, Moravec at 1013.5 IPS, and Minsky2 at 1011 IPS. The author’s own estimate is in the same range as Moravec’s. Along the Moore’s Law trend curve, the cost and value of a Minsky HEPP crossed in the 1990’s, of a Moravec HEPP this decade, and of a Kurzweil HEPP in the 2010’s. We will use the Moravec value in the following, but note that with the other estimates, the argument is the same, simply shifted a decade one way or the other in time. The implication is that by 2030, a HEPP will cost one dollar. Note that we are intentionally ignoring the software side of the AI. While this is currently the most problematic aspect in a scientific sense, once AI is developed the software–complete with education–can be copied with negligible cost. 1 Owning a machine which could draw an $80,000 salary is equivalent to having $1M to invest at an 8% return. 2 Marvin Minsky, personal communication: note that the actual informal estimate was somewhat lower than the one used here.

J.S. Hall / Engineering Utopia

461

The number of applications to which a human-level intelligence adds at least a dollar of value is staggering. Thus we can confidently predict that human-level AIs will be exceedingly numerous. AIs of greater-than-human intelligence are also likely. We know that humans with IQs of up to 200 or so can exist, and thus such levels of intelligence are possible. Less is known about the organization of complexity in intelligent systems than building machines of raw computational power. Even so, it will at the very least be possible to take individual human-level AIs, run them on faster hardware, and put them in structures along the lines of corporations, universities, economies, or the scientific community as a whole. From the outside, such a conglomerate intelligence could appear to be an extremely broad and deep thinker. Note that the data bandwidth of even a current-day fast ethernet link is in the same range as that of the corpus callosum. Between 40 and 30 thousand years ago anatomically modern humans overran Neanderthals on the basis of what appears to have been a modest increase of creativity, at least in a local perspective. [6] From the historical viewpoint, the Neanderthal level of technology (referred to as the "Mousterian toolkit") had held level for about 100 millenia, whereas a mere 30 millenia of homo sapiens got from the neolithic to where we are today. Could the same thing happen to us, replacing us with intelligent machines? Economically, at least, it is virtually certain that not only it could, but that it will. In an economy where human level intelligence is available for a dollar, it is difficult to see why anyone would hire a human. In a world where hyperhuman intelligence is available, it is difficult to see why anyone would want a mere human to be in charge of anything important. It should be clear that the nature of the process of replacement will be crucial. A model in which humans compete with the machines is disastrous. A more desirable model is one in which the humans take the role of an older generation gracefully retiring, and the machines take that of our children, growing stronger and taking the work of the world on their shoulders, as younger generations do. The moral character of these machines will be a crucial factor in the nature of our society – but just as importantly, as with any children, it is our moral duty to give them a sound moral education. How can we build them to be good? Or, indeed, will we have a chance to do so?

1. Hard Take-off and Singularity A major concern in some transhumanist and singularitarian schools of thought is autogenous—self-modifying and extending—AIs. They might, it is reasoned, remove any conscience or other constraint we program into them, or simply program their successors without them. Furthermore, it is worried, hyper-intelligent machines might appear literally overnight, as a result of runaway self-improvement by a “seed AI” [7]. How likely is runaway self-improvement? As a baseline, let us consider the self-improving intelligence we understand best, our own. Humans not only learn new facts and techniques, but improve our learning ability. The invention of the scientific method, for example, accelerated the uptake of useful knowledge tremendously. Improvements in knowledge communication and handling, ranging from the invention of writing and the printing press to the internet and

462

J.S. Hall / Engineering Utopia

Google, amplify our analytical and decision-making abilities, including, crucially, the rate at which we (as a culture) learn. Individual humans spend much of our lives arduously relearning the corpus of culturally transmitted knowledge, and then add back a tiny fraction more. Thus on the personal scale our intelligence does not look “recursively self-improving” – but in the large view it definitely is. Technological development usually follows an exponential improvement curve. Examples abound from the power-to-weight ratio of engines, which has tracked an exponential steadily for 300 years, to the celebrated Moore’s Law curve for semiconductors, which has done so for 50. Exponential curves fit a simple reinvestment model, where some constant of proportionality (an “interest rate”) relates the total at any two successive times: Q t = Ceit Where Q t is the total at time t, C is the initial capital, and i is the interest rate. However, i can be seen as composed of a productivity of capital p and a reinvestment rate r: Q t = Cer pt Any agent must make a decision on how much of its resources to reinvest, and how much to use for other purposes (including food, clothing, shelter, defense, entertainment, etc.). Human societies as a whole have invested relatively low percentages gross product, and even of their of their surplus, in scientific research. The proportion of scientists and engineers in the US population can be estimated at 1%, and those in cognitive-science related fields as 1% of that. Thus we can estimate the current rate of improvement of AI as being due to the efforts of 30,000 people (with a wide margin for error, including the fact that there are many cognitive scientists outside the US!), and the rate of improvement in computer hardware and software generally as being possibly due to the efforts of 10 times as many. It is not clear what a sustainable rate of reinvestment would be for an AI attempting to improve itself. In the general economy, it would require the same factors of production – capital, power, space, communication, and so forth – as any other enterprise, and so its maximum reinvestment rate would be its profit margin. Let us assume for the moment a rate of 10%, 1000 times the rate of investment by current human society in AI improvement. (This is germane because the AI is faced with exactly the same choice as an investor in the general economy: how to allocate its resources for best return.) Note that from one perspective, an AI running in a lab on equipment it did not have to pay for could devote 100% of its time to self-improvement; but such cases are limited by the all-too-restricted resources of AI labs in the first place. Similarly, it seems unlikely that AIs using stolen resources, e.g. botnets, could manage to devote more than 10% of their resources to basic research. Another point to note is that one model for fast self-improvement is the notion that a hyperintelligence will improve its own hardware. This argument, too, falls to an economic analysis. If the AI is not a hardware expert, it makes more sense for it to do whatever it does best, perhaps software improvement, and trade for improved hardware. But this is no different from any other form of reinvestment, and must come out of the selfimprovement budget. If the AI is a hardware expert, it can make money doing hardware design for the market, and should do that exclusively, and buy software improvements, for the overall most optimal upgrade path.

J.S. Hall / Engineering Utopia

463

Thus we can assume r AI = 10% , but we do not know the productivity constant. It is occasionally proposed that, as a creature of software, an AI would be considerably more proficient at improving its own source code than humans would be. However, while there is a steady improvement in software science and techniques, these advances are quickly written into tools and made available to human programmers. In other words, if automatic programming were really such low-hanging fruit for AI as is assumed, it would be amenable to narrow-AI techniques and we would have programmer’s assistants that improved human programmers’ performance drastically. What we see is steady progress but no huge explosion. In practice the most difficult part of programming is higher-level conceptual systems design, not lower level instruction optimization (which is mostly automated now as per the previous point anyway). Abstract conceptualization has proven to be the hardest part of human competence to capture in AI. Although occasionally possible, it is quite difficult to make major improvements in a program when the program itself is the precise problem specification. Most real-world improvements involve a much more fluid concept of what the program must do; the improved version does something different but just as good (or better). So programming in the large requires the full panoply of cognitive capabilities, and is thus not likely to be enormously out of scale compared to general competence. The author feels that many of the more commonly seen scenarios for overnight hard takeoff are circular – they seem to assume hyperhuman capabilities at the starting point of the self-improvement process. We can finesse the productivity, then, by simply letting it be one human equivalent, and adjusting the timescale to let 0 be whatever point in time a learning, self-improving, human-level AI is achieved. Then we estimate human productivity at intelligence improvement by assuming that the human cognitive science community are improving their models at a rate equivalent to Moore’s Law, or roughly e0.6 . As this is the sum effort of 30,000 people, each human’s p value is 0.00002. This gives us a self-improvement rate of Q y = e(r p)(y−y0 ) = e0.000002(y−y0 ) for the efforts of a single AI where y0 is the year human equivalence is first achieved. This is essentially flat, as one would expect: the analysis for a single human would be the same. A single human-level AI would be much, much better off hiring itself out as an accountant, and buying new hardware every year with its salary, than trying to improve itself by its own efforts. Recursive self-improvement for such an AI would then mean buying new hardware (or software) every year, improving its prowess at accounting, for an increased growth rate compounded of its own growth and Moore’s Law. Only when it reached a size where it could match the growth rate of Moore’s Law purely by its on efforts, would it make sense for it to abandon trade and indulge in self-construction. But that analysis assumes Moore’s Law, and indeed all other economic parameters, remained constant over the period. A much more realistic assumption is that, once human-level AI exists at a price that is less than the NPV of a human of similar capabilities, the cost of labor will proceed to decline according to Moore’s Law [8], and therefore the number of human equivalent minds working in cognitive science and computer hardware will increase at a Moore’s Law rate, both increasing the rate of progress and decreasing the price from the current trendline. In other words, the break-even point for an AI hoping to do all its own development instead of specializing in a general market and trading for improvements, is a moving

464

J.S. Hall / Engineering Utopia

target, and will track the same growth curves that would have allowed the AI to catch up with a fixed one. (In simple terms: you’re better off buying chips from Intel than trying to build them yourself. You may improve your chip-building ability – but so will Intel; you’ll always be better off buying.) We can conclude that, given some very reasonable assumptions, it will always be more optimal for an AI to trade; any one which attempts solitary self-improvement will steadily fall farther and farther behind the technology level of the general marketplace. Note that this conclusion is very robust to the parameter estimates above: it holds even if the AI’s reinvestment rate is 100% and the number of researchers required to produce a Moore’s Law technology improvement rate is 1% of the reasonable estimate.)

2. Machina Economicus Let us now consider a fanciful example in which 30,000 cognitive science researchers, having created an AI capable of doing their research individually, instantiate 30,000 copies of it and resign in favor of them. The AIs will be hosted on commercial servers rented by the salaries of the erstwhile researchers; price per MIPs of such a resource will be assumed to fall, and thus resources available at a fixed income to rise, with Moore’s Law. At the starting point, the scientific efforts of the machines would equal those of the human scientists by assumption. But the effective size of the scientific community would increase as e0.6(y−y0 ) . On top of that, improvements would come from the fact that further research in cognitive science would serve to optimize the machines’ own programming. Such a rate of increase is much harder to quantify, but there have been a few studies that tend to show a (very) rough parity for Moore’s Law and the rate of software improvement, so let us use that here. This gives us a total improvement curve of Q y = Ce1.2(y−y0 ) or double the Moore’s Law rate. This is a growth rate that would increase effectiveness from the 30,000 human equivalents at the start, to approximately 5 billion human equivalents a decade later. We claim that this growth rate is an upper bound on possible self-improvement rates given current realities. Note that the assumptions subsume many of the mechanisms that are often taken in qualitative arguments for hard takeoff: self-improvement is taken account of; very high effectiveness of software construction by AIs is assumed – 2 years into the process, each HEPP of processing power is assumed to be doing 11 times as much programming as a single human could, for example. Nanotechnology is implied by Moore’s Law itself not too many years from current date. This upper bound, a growth rate of approximately 300% per year, is unlikely to be uniformly achieved. Most technological growth paths are S-curves, exponential at first but levelling out as diminishing returns effects set in. Maintaining an overall exponential typically requires paradigm shifts, and those require search and experimentation, as well as breaking down heretofore efficient social and intellectual structures. In any system, bottleneck effects will predominate: Moore’s Law has different rates for CPUs, memory, disks, communications, etc. The slowest rate of increase will be a limiting factor. And finally, we do not really expect the entire cognitive science field to resign, giving their salaries over to the maintenance of AIs.

J.S. Hall / Engineering Utopia

465

The key unexamined assumption in the high-growth scenario is that the thousands of AIs would be able to operate with a full linear speedup of effectiveness. In practice, with people or machines, this is rarely true [9]. Given the complexity of economics, parallel programming, and every other cooperative paradigm in between, a significant portion of the creativity as well as the raw computational power of a developing community of AIs will have to be devoted to the problem of effective cooperation. The alternative notion, that of consolidating the AI resources into one unified hypermind, simply pushes the problem under the rug for the designer of the hypermind to worry about. It must still do internal resource allocation and credit assignment. Economic theory and history indicate that central control is extremely suboptimal for these tasks. Some of the most promising AI architectures, indeed, use internal markets or marketderived algorithms to address these problems. AIs will have certain advantages over humans when it comes to intellectual interaction. For example, an AI which comes up with a brilliant new theory can communicate not only the theory but the theory-producing mechanism for the other AIs to try and possibly to adopt. The communications between the AIs will be a true marketplace of ideas (or other allocational mechanism that optimizes resource use). Any individual AI, then, will be most effective as a cooperating element of a community (as is any individual human [10]). AI communities, on the other hand, will have the potential to grow into powers rivalling or exceeding the capability of the human race in relatively short order. The actions of communities are effects of the set of ideas they hold, the result of an extremely rapid memetic evolution. Over the 20th century, history showed that it was possible for meme plagues, in the form of political ideologies, to subvert the moral bases of whole societies, composed of ordinary, decent human beings who would never have individually committed the kind of widespread destruction and slaughter that their nation-states did. Real-time human oversight of such AI communities is infeasible. Once a networked AI community was established, a “cultural revolution” could overtake it in minutes on a worldwide scale, even at today’s communication rates. The essence of our quest for a desirable future world, then, both for ourselves and for the AIs, lies in understanding the dynamics of memetic evolution and working out ways to curb its excesses.

3. Machine Character A few principles seem straightforward. Nations with broadly-based democratic control structures are statistically less likely to start wars of aggresion or internal pogroms than ones with more centralized, autocratic control structures. Dominant nations (and companies) are more likely to initiate aggressive policies than ones at parity with their neighbors. Similarly, the openness of the society and the transparency of the governing processes is also strongly correlated with a beneficient outcome. How can we proof networks of AIs against runaway autocratic faction-forming? It is difficult to specify a simple remedy that would not cripple the creative memetic ideaproducing ability of the collective in the first place. We are helped, however, by the fact that we are assuming networks of fully intelligent entities. Thus one thing we should require is that each AI itself fully understands the nature of the memetic evolutionary process, and be familiar with the possible patholo-

466

J.S. Hall / Engineering Utopia

gies, so as to be able to recognize them when they start and take steps against them. Another way to state this desideratum is that there should not be AI “sheep” willing to follow silver-tongued agitators uncritically. As was mentioned above, networked AIs will be economic agents of necessity. It seems reasonable to assume that if they are provided with a basic theoretical understanding of economics, they will not be better agents individually but in designing more effective collective structures (e.g. to address externalities). Education of these kinds, if successful, will allow AIs to engage in collective action on scales rivalling the current-day world economy and beyond. This, in turn, will allow them to further the interests of their other basic goals, and change the world into a form more to their (and, if we built them wisely, our) liking. It is up to us to formulate other basic goals so as to give the world thus formed a character we find congenial [11]. There are several human character traits that fall naturally into this description of a desirable moral makeup for our mind children. Honesty and trustworthiness are a keystone for collective efficiency. A willingness to cooperate is likewise valuable – and is to some extent simply implied by a deep enough understanding of market and other social processes. For beings of information, a concern for truth is ultimately the same as hygiene and good health. A long-term planning horizon is surely to be valued in entities who will be making the important decisions for civilization. The good is not a specific goal to be sought, but a direction to be taken, a search for new and deeper understanding of the good itself as well as the struggle to instantiate known desired properties. The author, on reflection, finds that any world peopled with entities of deep understanding, high character, and good will, pursuing diverse constructive goals in voluntary association, would be a good world to inhabit regardless of its other particulars.

4. Summary We expect the progress of AI/AGI research to go through the following phases: 1. Currently, the AGI subfield is growing, reflecting an increasing consensus that the mechanisms of general intelligence are within reach of focussed research. At some point, the “baby brains” of the AGI researchers will begin to learn as well and (critically) as open-endedly as human babies. 2. Baby brains will then be educated to adult capacity. It seems unreasonable to imagine that this will happen without at least the same amount of bug-fixes, improvement ideas, and so forth seen in the normal software development process. 3. Competent human-level AIs will begin to augment narrow AI and other software, control robots, do office jobs, and so forth. A wave of displacement will begin, but radical feedback (runaway self-improvement) will not occur until the new AI working in cognitive science rivals the existing human level of effort. 4. Once there is sufficient penetration into the overall productive process, radical positive feedback effects will begin to occur, as new productivity levels begin to reach the bottleneck areas and multi-sector compounding occurs (as opposed to the 2-sector compounding of the example above). At the end of the fourth phase, the AI/robotic economy exceeds the human one and is on a growth curve that can only be described as extreme in current-day terms. Phases 1 to

J.S. Hall / Engineering Utopia

467

3, however, represent a window of opportunity for human input into the character of this new world. Verbum sat sapienti est.

5. Acknowledgements The author wishes gratefully to acknowledge the attention and insights of Stephen Omohundro, Robert A. Freitas Jr., Douglas Hosfstadter, Christopher Grau, David Brin, Chris Phoenix, John Smart, Ray Kurzweil, Eric Drexler, Eliezer Yudkowsky, and Robin Hanson.

References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]

D REXLER , K. E RIC . Nanosystems: Molecular Machinery, Manufacturing, and Computation. Wiley, 1992. K URZWEIL , R AY. The Singularity is Near. Viking, 2005. M ORAVEC , H ANS . Robot: Mere Machine to Transcendent Mind. Oxford, 1999. H ALL , J. S TORRS . Nanofuture: What’s Next for Nanotechnology. Prometheus, 2005. H ALL , J. S TORRS . Beyond AI: Creating the Conscience of the Machine. Prometheus, 2007. C LIVE F INLAYSON . Neanderthals and Modern Humans: An Ecological and Evolutionary Perspective, Cambridge, 2004 Y UDKOWSKI , E LIEZER . Creating Friendly AI http://www.singinst.org/CFAI/index.html 2003. H ANSON , ROBIN. Economic Growth Given Machine Intelligence. http://hanson.gmu.edu/aigrow.pdf Oct. 1998. A XELROD , ROBERT. The Evolution of Cooperation. Basic Books, 1984. S MITH , A DAM . The Theory of Moral Sentiments. A. Millar, 1790. NADEAU , J OSEPH E MILE . Only Androids can be Ethical. in Thinking about Android Epistemology, K. F ORD , C. G LYMOUR , AND P. H AYES , eds. AAAI/MIT, 2006, pp241-8.

468

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

OpenCog: A Software Framework for Integrative Artificial General Intelligence David Hart1,2 and Ben Goertzel1,2 1 Novamente LLC 2 Singularity Institute for Artificial Intelligence

Abstract. The OpenCog software development framework, for advancement of the development and testing of powerful and responsible integrative AGI, is described. The OpenCog Framework (OCF) 1.0, to be released in 2008 under the GPLv2, is comprised of a collection of portable libraries for OpenCog applications, plus an initial collection of cognitive algorithms that operate within the OpenCog framework. The OCF libraries include a flexible knowledge representation embodied in a scalable knowledge store, a cognitive process scheduler, and a plug-in architecture for allowing interaction between cognitive, perceptual, and control algorithms.

1. Introduction The information revolution we experience today is underpinned by numerous enabling technologies, many of which are based on open platforms and standards with conceptual roots dating back to the 1960s and earlier. Most of the present day Internet software infrastructure from Google, Amazon, Ebay and others, is built on Linux and other open source technologies. The field of artificial intelligence has fallen woefully the rate of progress of other fields within computer science and information technology. The reasons for this are many and varied, and exploring them all would take us too far off topic. However, we suggest that one viable path to remedying the situation involves providing the field of AI research with an open foundation comparable to the open foundations underlying popular Internet applications. In this paper we briefly describe the motivations and scientific approach underlying OpenCog. OpenCog is designed with the ability to integrate different and varied AI methodologies within a common framework operating on a common knowledge representation and utilizing a common scheduler and I/O subsystem. Providing as much breadth and flexibility as is possible, consistently with the goal of providing a coherent unifying framework, has been an overriding design goal. While OpenCog is a highly flexible platform, we recognize that software development generally proceeds most efficaciously in the context of concrete application goals. With this in mind our intention is to initially guide OpenCog development in the context of natural language conversation, both on the command line and potentially also in the context of embodied agents.

D. Hart and B. Goertzel / OpenCog: A Software Framework for Integrative AGI

469

2. The Challenges of Integrative AGI Precisely how the human brain works is unknown to scientists and laymen alike, but all of the available evidence suggests that the brain is a highly complex and integrative system [1]. Different parts of the brain carry out various functions, and no one part is particularly intelligent on its own, but working in concert within the right architecture they result in human-level intelligence. In this vein, Steven Mithen [2] has provided powerful albeit somewhat speculative vision of modern human intelligence as the integration of components that evolved relatively discretely in prehuman minds. On the other hand, most of the work in the AI field today is far less integrative than what we see in the brain. AI researchers work on individual and isolated algorithms for learning, reasoning, memory, perception, etc. with few exceptions. The combination of algorithms into coordinated systems demonstrating synergistic intelligent behavior is much less frequent than it should be. The mainstream of AI research is attempting to address this, for instance the most recent AAAI conferences have contained a Special Track on Integrated Intelligence [3], and this trend is likely to continue. However, the move to more integrated intelligence approaches entails serious practical difficulties. Most AI researchers operate under extremely constrained resources, and performing system integration requires a large amount of extra work. The reasons for experimenting with AI algorithms in isolated rather than systemically integrated contexts is typically purely pragmatic rather than theoretical. As a result, no one knows what level of intelligence could be achieved by taking an appropriate assemblage of cutting-edge AI algorithms and appropriately integrating them together in a unified framework, in which they can each contribute their respective strengths toward achieving the goals of an overall intelligent system. Of course, conceptual and architectural integration is required in addition to simple software integration, but conceptual and architectural integration is precisely what is needed as we move toward a practical understanding of how to create powerful AI systems. The current body of academic and commercial AI knowledge contains solutions to many of the key problems underlying the creation of powerful AI, but until these partial-solutions are integrated in useful ways, we cannot explore the synergistic effects which will emerge from their union, or discover the key gaps to be addressed by more fundamental research. As discussed, one approach to overcoming this AI bottleneck and helping the research community transition to transition away from building algorithms and systems in isolation is to provide a flexible, powerful and approachable software framework, designed specifically for integration of AI algorithms. With this in mind, the authors and a number of colleagues have established the Open Cognition Project, beginning with the release of the OpenCog Framework, a software framework specifically intended to support the construction of integrative AI systems from component AI algorithms and structures. The initial OpenCog Framework (OCF) codebase will consist largely of code donated by Novamente LLC [4], who will adapt much internal R&D to utilize OpenCog, and continue to make significant further code contributions to OpenCog over time. Although code flowing in both directions will undoubtedly benefit Novamente LLC, the authors believe that the greatest benefit OpenCog has to offer is energizing and boosting the global AI research community by significantly improving the tools at its disposal, tools which the community will have the freedom to utilize and modify to meet their diverse requirements.

470

D. Hart and B. Goertzel / OpenCog: A Software Framework for Integrative AGI

Preliminary discussion of OpenCog has met with enthusiastic support from members of the academic, industry and open source communities, and has elicited sentiments that a project with OpenCog's goals and scope is long overdue but that its introduction must be carefully executed. Contingent upon funding for OpenCog proceeding as planned, we are targeting 1H08 for our first official code release, to be accompanied by a full complement of documentation, tools, and development support.

3. OpenCog and the Four Key Aspects of AGI Design At a high level, the work of AI system design may be divided into four aspects: 1. Cognitive Architecture: the overall design of an AI system; what parts does it have, and how do they connect to each other. 2. Knowledge Representation: how the AI system internally stores declarative, procedural and episodic knowledge, and how creates its own representations for elemental and abstracted knowledge in new domains it encounters. 3. Learning: how the AI system learns new elemental and abstracted knowledge, and how it learns how to learn, and so on. 4. Teaching Methodologies: how the AI system is coupled with other systems so as to enable it to gain new knowledge about itself, the world and others. OpenCog will help significantly with all four aspects: 1. Cognitive Architecture: the OpenCog framework is architecture-neutral, and will provide facilities to define rich and varied cognitive architectures via configuration files, with the guiding but flexible principles that a cognitive architecture will consist of: functional units which are modular knowledge representation using a local AtomTable containing knowledge (see below) and a collection of MindAgent objects implementing cognitive, perceptual or action processes that act on this AtomTable, and/or interact with the outside world 2. Knowledge Representation: OpenCog will feature: a generalized version of the Novamente AtomTable knowledge representation utilized in the Novamente Cognition Engine (2004). This representation technology allows rival connectionist and symbolic artificial intelligence systems to interoperate an implementation of a LISP-like language called Combo, specifically tailored for operating on knowledge represented in AtomTable format 3. Learning: OpenCog will be seeded with two powerful learning mechanisms: MOSES probabilistic-program-evolution module (http://code.google.com/p/ moses) PLN Probabilistic Logic Networks module for probabilistic logical inference 4. Teaching Methodologies: OpenCog will be will be introduced with two powerful toolsets: AGISim integration, an open-source 3D simulation world intended specifically for instruction of embodied AI systems (2006) (http://sourceforge.net/projects/agisim)

D. Hart and B. Goertzel / OpenCog: A Software Framework for Integrative AGI

471

RelEx integration, a natural language comprehension system that intakes English sentences and outputs logical relationships suitable for analysis via AI algorithms. ReEx is to be released soon in conjunction with the Wikia Search project.

4. OpenCog and the “Operating System” and “Virtual Machine” Metaphors OpenCog is a vertical systems-level framework for AGI development, so comparisons to the designs of operating systems such as Linux/UNIX or Windows and virtual machine environments such as Java and .NET are useful. OpenCog operates atop and utilizes the services of traditional operating systems, while at the same time many of its own components have parallels to and operate with the assistance of complimentary OS components; a partial list is found below. OS or VM Environment

OpenCog

Filesystem or simple Database

AtomTable

Process or Thread Scheduler

CPS (Cognitive Process Scheduler)

Daemons or Services

MindAgent

I/O from hardware

I/O from input filters to AtomTable via XML

Programming Languages (Java, C, shell, etc.)

Combo (can be used for MindAgents)

Garbage Collection

Forgetting MindAgent

Future optimizations of OpenCog may include tight integration with a host operating system or implementation of performance critical components using specialized hardware such as FPGAs. Using integration with Linux as an example, the AtomTable could be implemented as a kernel module to reduce context switches and improve performance.

5. Discussion Just as the early developers of Linux and other open-source technologies underlying the present-day Internet could not have envisioned all of the varied future directions of their work, so the future course of OpenCog likewise remains uncertain. As a general framework, OpenCog may be used to produce a wide variety of AGI and narrow-AI systems and explore a wide variety of AGI issues. We have sought to make OpenCog as broad as possible but no broader because we wish to avoid making commitments to particular AGI approaches while at the same time offering a tool which is much more substantive than, say, a programming language or an operating system. We believe the OpenCog Framework provides a balance of specificity and generality such that it will allow a wide variety of AGI and narrow-AI research approaches to be integrated and experimented with, inside a common framework that provides services (including ease of integration with other approaches) that would be objectionably difficult for most researchers to create on their own.

472

D. Hart and B. Goertzel / OpenCog: A Software Framework for Integrative AGI

Creating powerful AGI tools in the open entails risks and uncertainties: individuals with malicious or overly egoistic goals may attempt to create AGI systems with destructive or otherwise undesirable or unethical goals or outcomes. On the other hand, the open-source approach also provides certain protections – many knowledgeable eyes watch code as it develops, and can spot potential dangers. We don't pretend to have definitive answers to the thorny issues of AGI ethics (or anything close to it), but we do believe that the open-source approach to AGI encourages a collaborative and inclusive exploration of these ideas. Further updates and information on OpenCog may be found at www.opencog.org

References [1] [2] [3] [4]

Gazzaniga, M.S., Ivry, R., Mangun, G.R. (2002). Cognitive Neuroscience. W.W. Norton. Mithen, Steven (1999). The Prehistory of Mind. Thames and Hudson. Goertzel, Ben, Moshe Looks and Cassio Pennachin (2004). Novamente: An Integrative Architecture for Artificial General Intelligence. Proceedings of AAAI Symposium on Achieving Human-Level Intelligence through Integrated Systems and Research, Washington DC, August 200 Goertzel, Ben, Ari Heljakka, Stephan Vladimir Bugaj, Cassio Pennachin, Moshe Looks (2006). Exploring Android Developmental Psychology in a Simulation World, Symposium “Toward Social Mechanisms of Android Science”, Proceedings of ICCS/CogSci 2006, Vancouver

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

473

Open Source AI Bill HIBBARD University of Wisconsin - Madison

Abstract. Machines significantly more intelligent than humans will require changes in our legal and economic systems in order to preserve something of our human values. An open source design for artificial intelligence (AI) will help this process by discouraging corruption, by enabling many minds to search for errors, and by encouraging political cooperation. The author's experience developing open source software provides anecdotal evidence for the healthy social effects of open source development. Keywords. Open source, AI ethics, AI politics.

Introduction There is little doubt that humans will create machines significantly more intelligent than themselves during the twenty first century. Neuroscience is finding a large number of correlations between mental and physical brain behaviors. If brains do not explain minds, then these correlations would be absurd coincidences. And if physical brains do explain minds, then our relentless technology will create minds with artificial physical brains. Under our current legal and economic systems, super-intelligent machines will create social chaos. They will be able to do every job more efficiently than humans, resulting in 100% unemployment. They will create great wealth for their owners while denying the vast majority any way to support themselves. Technology will enable humans to increase their own intelligence via artificial extensions of their brains. But then each person's intelligence will depend on what sort of brain they can afford. Less intelligent humans will not be able to understand the languages used by the most intelligent humans and machines and thus will be effectively excluded from any meaningful political process. Most people will object to these social consequences and seek to alleviate them through some adjustment to our legal and economic systems, including constraints on the designs of artificial intelligence (AI). These adjustments ought to be a topic for this workshop. Of course these issues have been raised. The Hebrew myth of the golem depicts the possible unintended consequences of creating artificial life by magical means. In the early nineteenth century Shelley explored the ethical issues of using technology to create artificial life [1]. Asimov's three laws of robotics were probably the first attempt to define ethical standards for interactions between humans and AI [2, 3]. Vinge applied the term singularity to the predicted explosive increase of technology and intelligence when machines more intelligent than humans take over their own development, and described the difficulty of predicting the consequences of this event [4]. Now it is commonplace for science fiction stories and movies to depict intelligent

474

B. Hibbard / Open Source AI

machines as a threat to humans, and the issue of AI ethics has emerged as a serious subject [5, 6, 7]. It is usual to depict the threat as AI versus humanity, but it is also important to counter the threat of AI enabling a small group of humans to take dictatorial control over the rest of humanity.

1. Transparency Human society could not have achieved the efficiency necessary to create AI without specialization, where different people become experts in different types of work and trade the results of their work. In many cases experts act as the agents for many other people, representing their interests in important decisions. For example, the laws of society are determined by government experts, hopefully elected by those they represent or at least supervised by elected representatives. Leaders of large corporations act as agents for corporate owners, usually subject to some sort of election. However, whenever one person serves as agent for others, there is the possibility of corruption, in which the agent serves his own interests at the expense of those he represents. An essential tool for preventing corruption is transparency, in which the decisions and circumstances of agents are made known to those they represent. The creation of super-human AI is the most difficult challenge in human history and will require extreme expertise. It will also require large resources, controlled by powerful government and corporate leaders. The results of super-human AI will be much more profound for humanity than those of any previous technology. Thus, whether they like it or not, the designers and resource managers for AI represent the interests of all of humanity. Current law and public opinion do not recognize AI designers and managers as humanity's agents, so they may feel no need to represent humanity's interests. Even if they do acknowledge that they represent humanity's interests, the stakes are so high that the temptation for corruption will be intense. Protecting the interests of humanity will require that law and public opinion change to recognize that AI designers and managers are humanity's agents. Once this agent relation is recognized, preventing corruption will require transparency, which includes an open source design for AI.

2. Many Minds Searching for Errors Yudkowsky has proposed an effort to produce a design for AI whose friendliness toward humans can be proved as it evolves indefinitely into the future [8]. Legg's blog includes a fascinating debate with Yudkowsky over whether such a proof is possible [9]. Legg produced a proof that it is not possible to prove what an AI will be able to achieve in the physical world, and Yudkowsky replied that he is not trying to prove what an AI can achieve in the physical world but merely trying to prove that the AI maintains friendly intentions as it evolves into the indefinite future. But intentions must be implemented in the physical world, so proving any constraint on intentions requires proving that the AI is able to achieve a constraint on the implementation of those intentions in the physical world. That is, if you cannot prove that the AI will be able to achieve a constraint on the physical world then you cannot prove that it will maintain a constraint on its intentions.

B. Hibbard / Open Source AI

475

If there can be no guarantee that an AI design remains friendly towards humans as it evolves, the next best approach is for the design to be examined for flaws by as many intelligent humans as possible. An open source design can be studied by anyone who cares to. Even if a proof of friendliness is possible, proposed mathematical proofs often contain errors that are best found by open publication. And an open source design will enable a large community to verify that the design conforms to the conditions defined in the proof.

3. AI Politics Politics will be necessary to change law and public opinion to recognize the responsibility of AI designers and managers for general human welfare. The academic debate about whether AI is possible and about the social effects of AI is already strenuous. The general political debate, when it arises, will be even more strenuous. And given the high stakes, it should be. Hopefully the scientific and technical communities will play a major role, with politicians listening to the U.S. National Academies of Science and Engineering and similar institutions internationally. The transparency of open source design will help set a tone of cooperation, rather than competition, in the political debate. It will reassure the public that they, and experts they trust, are included in the decision process. The political position of the Singularity Institute for Artificial Intelligence (SIAI) is puzzling. SIAI's long-time leader has expressed contempt for public opinion and politics [10, 11]. But he also contends that AI must be designed according to strict constraints to avoid a disaster for humanity. How are these constraints to be enforced if not by the force of government? Does the SIAI intend to create a friendly AI that takes control of the world before governments have time to react, and before any much better funded competitors can create AI? And given the stated contempt for public opinion and politics, it is hard to understand the considerable efforts of the SIAI to publicize the issues of AI via their Singularity Summits and other venues. In fairness, SIAI leadership is becoming more diffuse. For example, in early 2007 Ben Goertzel became SIAI's Director of Research. Under his leadership SIAI is sponsoring an open source AI project [12]. AI is inevitably becoming a political issue, and it will be interesting to see whether SIAI expresses cooperation rather than hostility toward politics.

4. Cautions The potential problem with open source AI is that it may enable malicious and incompetent people to build harmful AIs. The answer to this problem is for benign and competent people to out compete the malicious and incompetent in building AI, and for the "good guy" AIs to take up the task of preventing "bad guy" AIs. Liberal democracies are the leaders in science and technology and AI is very likely to first appear in one of these societies. In democracies, the good intentions of government policy generally depend on citizens being informed and engaged on issues. Thus the primary need is for creation of a broad public movement for AI to benefit all people. Given such a movement, government will need the competent advice of leading scientists and engineers, such as provided by the U.S. National Academies.

476

B. Hibbard / Open Source AI

Some are worried about a hard takeoff scenario, in which the "good guys" don't have time to react between the first human level AI and the explosive increase of intelligence which enables the AI to take control of the world. However, progress in AI has been and will continue to be gradual up to the point of human level AI, giving the public plenty of time to get to know sub-human level AIs in the form of natural language voice user interfaces and robots smart enough to do numerous household and business tasks. These experiences will start many people thinking seriously about what is coming, leading to real political debate. In fact, many politicians are already aware of the coming singularity, but do not discuss it for fear of alienating voters. Even once human level AI is achieved, there may be a time interval of years or even decades before the explosive intelligence increase, because of the time brains must spend learning intelligent behaviors through interactions with the world. Democratic governments control technological resources much greater than any other institution and, given this time interval for political response, will win a race to the singularity based on an open source design. Nuclear and biological weapons are examples of technologies where humanity has arguably benefited from keeping the details secret. However, those weapons technologies are only useful for killing and harming people. The elites who control their secrets have incentive not to use them, because their use would cause loss of productive capacity and cause the elites to lose legitimacy. In contrast, selfish use of AI by an elite will not cause loss of productive capacity and will enhance the power of the elite (we can assume that an elite with access to a super-intelligent AI will not behave stupidly). Furthermore, AI is a technology that can be applied to prevent others from developing that same technology. This is not nearly so true for nuclear and biological weapons technologies. Thus these weapons technologies are not good analogies for AI.

5. Personal Experiences My Vis5D, in 1989, was the first open source 3-D visualization system, and my subsequent VisAD and Cave5D visualization systems were also open source. My experience with these systems was that their open source brought out the best in the people who joined their user and developer communities. Whatever fears I had before making Vis5D open source never materialized, whereas I was constantly surprised by the generosity and helpfulness of others. The mailing lists serving these communities had occasional foolishness but never any flame wars. This experience was in sharp contrast with the mistrust and technical problems in the community of a proprietary system in my university department. And I have generally observed this difference of politics between open source and proprietary system communities. While this is all anecdotal, it has been strong enough to reshape my general political attitudes [13]. AI is inevitably becoming a political issue [14]. If the AI development community approaches the public with transparency, including open source designs, the public will surprise many of us with their reasonableness.

References [1] [2]

Shelly, M. 1818. Frankenstein. http://www.literature.org/authors/shelley-mary/frankenstein/index.html Asimov, I. Runaround, Astounding Science Fiction, March 1942.

B. Hibbard / Open Source AI

[3] [4] [5] [6]

[7] [8] [9] [10] [11] [12] [13] [14]

477

Asimov, I. 1968. I, Robot. London. Grafton Books. Vinge, V. Vinge, V. 1993. The coming technological singularity. Whole Earth Review, Winter issue. Hibbard, W. Super-Intelligent Machines. Computer Graphics 35(1), 11-13. 2001. http://www.ssec.wisc.edu/~billh/visfiles.html Bostrom, N. Ethical Issues in Advanced Artificial Intelligence. Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, Vol. 2, ed. I. Smit et al., Int. Institute of Advanced Studies in Systems Research and Cybernetics, 2003, pp. 12-17. http://www.nickbostrom.com/ethics/ai.html Goertzel, B. Universal Ethics: The Foundations of Compassion in Pattern Dynamics. October 25, 2004. http://www.goertzel.org/papers/UniversalEthics.htm Yudkowsky, E. (2006) Knowability of FAI. http://sl4.org/wiki/KnowabilityOfFAI Legg, S. Unprovability of Friendly AI. September 15, 2006. http://www.vetta.org/?p=6 Yudkowsky, E. CoherentExtrapolatedVolition. http://www.sl4.org/wiki/CollectiveVolition http://www.ssec.wisc.edu/~billh/g/message10.txt http://www.opencog.org/ Hibbard, W. Singularity Notes. http://www.ssec.wisc.edu/~billh/g/Singularity_Notes.html Johnson, G. Who Do You Trust More: G.I. Joe or A.I. Joe? New York Times, February 20, 2005.

478

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

On the Broad Implications of Reinforcement Learning based AGI Scott Livingston a , Jamie Garvey b and Itamar Elhanany a a Department of Electrical Engineering and Computer Science b Department of Nuclear Engineering The University of Tennessee, Knoxville, TN Abstract. Reinforcement learning (RL) is an attractive machine learning discipline in the context of Artificial General Intelligence (AGI). This paper focuses on the intersection between RL and AGI by first speculating on what are the missing components that would facilitate the realization of RL-based AGI. Based on this paradigm, we touch on several of the key moral and practical issues that will inevitably arise. Keywords. Reinforcement Learning, Moral Implications of AGI, Machine Intelligence

Introduction Reinforcement learning is generally perceived as a machine learning discipline in which an agent learns by interacting with its environment [1]. What makes reinforcement learning unique is that it attempts to solve the credit assignment problem, in which an agent attempts to predict the long-term impact of its actions. Recent work seems to suggest that reinforcement learning, as a general formalism, does correspond to observable mammal brain functionality. In particular, the notion of a value function and of actions that are driven by such a value function has found proof in recent neurophysiological studies. However, despite almost two decades of RL research, there has been little solid evidence of RL systems that may one day lead to artificial general intelligence (AGI). In this paper we make the assumption that RL is indeed the correct formalism that may one day lead to the realization of AGI systems. We then attempt to identify particular properties that are lacking in existing RL solutions, and outline the challenges that remain in order to move such solutions toward AGI. We argue that a relatively small number of fundamental hurdles must be overcome in order to pave the way for a much-needed technological breakthrough. The second part of this paper addresses several moral and pragmatic issues that will emerge if RL-based AGI robots are introduced into our everyday lives.

1. Challenges in Scaling RL Systems RL relies on four fundamental building blocks. The first is the availability of state representation which allows the agent to perceive the state of its environment for the pur-

S. Livingston et al. / On the Broad Implications of Reinforcement Learning Based AGI

479

pose of actuation and behavioral evaluation. In practical settings, complete state information is not available to the agent. Many robotic applications, for example, where a robot maneuvers through an unknown terrain, are characterized by partial observability, suggesting that the robot receives a sequence of observations from which it must infer the environment’s state. Partial observability characterizes AGI, almost by definition. We, as humans, continuously receive partial information regarding our surroundings from which we construct an internal model of the world we interact with. Many believe that the primary function of the neocortex is to play the role of such a model, thus providing crucial state inference information to other parts of the brain so that they, in turn, can perform their designated tasks. To that end, designing a scalable model which can effectively process high-dimensional information while capturing temporal dependencies that span different time scales is a fundamental challenge that is imperative in the context of RL-based AGI systems. A second component in RL systems is the reward function. The single-step reward characterizes the momentary feedback that the agent receives regarding its existential state of affairs. In general, rewards give a short-term indication of how good or bad the agent’s actions were. While most RL engines assume that the reward signal is a scalar, there has been work on extending the notion of reward to a multi-dimensional form. The concept of reward is rather straightforward and, in fact, is the only component in the RL framework that appears to be well-defined and scalable. The third basic element in RL systems is action selection, which in AGI should be rather broadly defined given that an agent may be able to control a wide range of actuators resulting in complex state-to-action mapping requirements. If RL is ever to provide a platform for AGI realization, the action-selection component must be able to receive and generate highly-dimensional signals. However, if state inference is effectively established, the work of the decision-generating engine should be somewhat straightforward. The final and perhaps most critical element of RL is the value function (or return), which intuitively reflects a prediction of future rewards that the agent expects to receive. Value function is at the core of RL, based on which decisions are made and temporal difference learning is driven. A good value function construct will facilitate strategic thinking and enable effective action selection. While the sum of discounted rewards has been used in the vast majority of RL studies, we believe that such a definition for a value function is inaccurate in the context of AGI. Despite its mathematical convenience, a more effective value function definition should be devised so that long-term impact can be evaluated, often regardless of the particular associated time scale.

2. Morality Issues Assuming the current limitations in reinforcement learning theory are overcome and an RL-based AGI is realized, an ethical system which accounts for such agents must be developed. Ethical discussions concerning AI have traditionally focused on the issue of protection of humanity from potentially dangerous implementations and on whether the creation of such machines is even ethically permissible. Here we focus on ethical issues concerning AGI-human interaction and take a decidedly AGI-centric stance. Beyond protective and regulatory justifications for limiting the resources of an AGI system, other reasons might include cost limitations and application goals. For example,

480

S. Livingston et al. / On the Broad Implications of Reinforcement Learning Based AGI

an AGI implemented to act as a “virtual pet” in a Web-based environment would simply not require, and perhaps would seem less realistic with, intelligence exceeding that of humans who interact with it. However, we may ask whether such an artificial limitation is unethical. The answer to this question depends largely on whether a restricted AGI would be aware of its potential. To understand this, consider the intellectual capacity of a dog. The dog is fundamentally unable to achieve the intelligence of a human child, much less that of an adult. However, there is no evidence to suggest that dogs mourn this limitation. In other words, dogs are not aware of the intelligence they are lacking. By contrast, a human child can typically recognize the intellectual abilities of adults and thus knows what it currently cannot do but may eventually be able to do in the future. It is this potential of a human child that makes suppression of learning ethically impermissible. If AGI can be recognized as a being in itself, an idea addressed later in this section, which does not solely exist to serve others, then arbitrarily limiting its intelligence is immoral because the AGI is aware of its limitation. The comparison of a dog and a child reveals the existence of an “intelligence threshold,” a level of intelligence under which an AGI is not aware of its potential and above which it realizes the potential of self-augmentation and learning. So long as the AGI is kept under this threshold, it cannot desire to achieve something it is not aware of, and therefore human-imposed limitations on its intelligence are not immoral as they do not interfere with its desires. However, if an AGI passes the intelligence threshold then further limiting its intelligence for reasons other than those beyond the implementor’s control, such as excessive hardware costs, is morally impermissible since the AGI would be aware of how it might grow. Assuming an RL-based AGI, the artificial limitation would interfere with the agent’s ability to maximize long-term reward by preventing optimal state evaluation and policy selection. For clarification, the moral obligation is not necessarily to improve the system on which the AGI depends but rather to not forcefully restrict the AGI from improving itself. As outlined in the first section of this paper, an AGI based on reinforcement learning would have a multi-criteria reward function and a complex state representation system. It is natural to imagine over time such an agent would learn to prioritize its goals and eventually build a “hierarchy of needs” in a form similar to the work of Abraham Maslow [2]. Assuming the existence of such a hierarchy in the AGI planning system, passing the intelligence threshold is equivalent to becoming aware of “higher” desires, such as improvement of its level of intelligence, what might be termed “self-actualization” for humans. Just as forceful restriction of a human’s satisfaction of increasingly important needs is viewed as unethical, so artificial limitation of an AGI’s policy function, expanded to include a progressive series of “needs” including intellectual improvement, should also be considered unethical. Providing that AGI is viewed as a tool toward human ends and not something that values its own existence, regardless of human use, no AGI-centric ethical theory can be developed. The challenge to be faced in the near future is recognizing AGI as more than a tool for specific applications; it must be seen as an end in itself. In most human societies, each member recognizes the lives of others as independent of justification by someone else. That is, no one is thought to live solely for the purpose of serving another. Otherwise, institutions such as slavery are created and considered morally permissible. Once AGI reaches and surpasses human intelligence, it will only seem natural to relate to AGI entities as counterparts rather than mere tools, ends rather than means. It does

S. Livingston et al. / On the Broad Implications of Reinforcement Learning Based AGI

481

not follow that AGI should receive an elevated position in society, in which humans are regularly called to serve AGI needs. Instead, AGI agents should become citizens with rights and responsibilities, freedoms that are limited by noninterference with the freedoms of others. This granting of person-hood may seem strange at first but is not a novel concept; the most obvious example of nonhuman, legal entities is corporations. Generally, the ethical implications of the establishment of AGI individuality, too numerous to be addressed in this paper, can be focused by newly interpreting any previously developed ethical theory to not only apply to humans, as originally intended, but also to AGI agents. Thus, the first steps toward viewing AGI as more than a tool are taken. Before respecting artificial intelligence as an end in itself, the concept of moral communities and whether AGI should be included must be considered. A moral community is a group in which members recognize each other’s ability to make rational ethical decisions concerning the other members. For example, the human moral community typically does not include nonhuman animals as they are considered incapable of moral judgments on par with those of humans. Since they are outside the community, animals do not enjoy the protections associated with membership, and thus animal concerns are usually overruled by human concerns. This raised the question of whether AGI agents should be allowed in the human moral community. The answer depends on their decision making ability. RL-based AGI will, by design, make rational decisions according to the goal of maximizing long-term reward. Therefore, the requirement of mutual respect among community members would only be held by AGI as long as it is, depending on the agent’s state inference and evaluation, in the agent’s best interests. Accordingly, the traditionally human moral community should be extended to include sufficiently intelligent RL-based AGI agents. Of course, membership in a moral community would not exempt RL-based AGI actions from the legal consequences given to humans performing similar actions. The final and perhaps most difficult moral issue concerning AGI is the emergence of extra-human moral theories. An extra-human moral theory is a system making valuebased decisions that is developed by an RL-based AGI after reaching a level of intelligence beyond that of humans. The reasoning of an AGI at this level would be, by its very nature, incomprehensible to humans, as its state representation and acting policy would defy the perceptive abilities of humans. An example we are familiar with is the inability of human children to fully understand the reasoning and ethics exhibited by adults. We attempt to teach them our values, but ultimately many justifications will be missed due simply to children’s stage of intellectual development. The ethical dilemma arises when humans must decide whether to interfere with the actions of an AGI. The proposed solution is the formation of trust between any particular AGI and those humans who directly interact with it. Until AGI betrays this trust, it can be assumed that its goals and methods, though beyond human comprehension, are valid and in keeping with our own. If integrity is not initially assumed with AGI, then most of the benefits of AGI with superhuman intelligence will be lost, as we only permit actions that we understand at our intelligence level. Understanding of the existential standing of AGI is crucial to making ethically consistent decisions in AGI-human interaction. Development of human behavioral guidelines in this regard before the creation of truly intelligent machines will prevent spontaneous fractionalization of the human population into many competing moral viewpoints and the likely resulting social disorder.

482

S. Livingston et al. / On the Broad Implications of Reinforcement Learning Based AGI

3. Practical Issues with RL-AGI Behavior As briefly stated in the discussion of moral issues above, assuming the current limitations in reinforcement learning theory are overcome, an RL-based artificial general intelligence will exhibit complex behavior in an attempt to maximize its return, across both temporal and spatial dimensions. Actions issued by such an agent will often seem distant from the original reward function definition, as fashioned by its designers, and it can be reasonably expected that RL-based AGI will develop what appear to be derivative goals. Thus, a significant challenge faced in designing RL-based AGI is fully understanding the implications of a reward function and the definition of return (or value) derived from it. One popular view is that, unless the original return function (termed “utility function” in economics) is very carefully constructed, all AGI systems of sufficient capacity will develop basic drives; according to [3], these are: to self improve, to be rational, to preserve their utility functions, to prevent counterfeit utility, to be self-protective, and to acquire and efficiently use resources. Such systems could prove dangerous as, for example, an AGI seeking to maximize return realizes the importance of remaining powered and takes actions to prevent humans from removing power (i.e., shutting it down). As in all reinforcement learning applications, future behavior of an RL-based AGI will be controlled by proper determination of reward and selection of hardware capacity available. Actions issued by an agent, assuming convergence of the policy and value functions, can always be traced to the underlying definition of reward and how its maximization is internally represented. Therefore, from a practical standpoint, AGI systems can be designed to serve specific purposes (that is, with specific built-in goals) while being limited in capacity. As argued in the previous section, it is likely there exists some threshold below which an RL-based AGI will be incapable of conceiving (that is, representing in its value function) the benefits of self augmentation; however, such threshold intelligence may be great enough as not to diminish the utility of the AGI for human purposes. Assuming the future entrance of AGI agents into the human moral community, or at least the assignment of legal rights and responsibilities comparable to those of humans, the behavior of RL-based AGI will largely be determined by its interaction with other moral, legal entities. For example, though a chess playing robot may realize the undesirability of being turned off (according to maximization of return) and may consider fighting against its operators, it will also know of the potential very large negative reward (or “punishment”) associated with murder within our legal system. Similarly, if a dominant component of the underlying reward function is general avoidance of malevolence toward humans, then no course of action which hinders this could be taken. It is apparent that we face many challenges en route to AGI. However, if one accepts reinforcement learning as the underlying formalism for the realization of AGI, it is argued here that now is the time to consider the profound ethical and pragmatic issues that would arise from this paradigm.

References [1] [2] [3]

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, March 1998. A. H. Maslow, A Theory of Human Motivation, Psychological Review 50 (1943), 370-396. S. M. Omohundro, The Basic AI Drives, First Conference on Artificial General Intelligence, March 2008.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

483

The Basic AI Drives Stephen M. OMOHUNDRO Self-Aware Systems, Palo Alto, California Abstract. One might imagine that AI systems with harmless goals will be harmless. This paper instead shows that intelligent systems will need to be carefully designed to prevent them from behaving in harmful ways. We identify a number of “drives” that will appear in sufficiently advanced AI systems of any design. We call them drives because they are tendencies which will be present unless explicitly counteracted. We start by showing that goal-seeking systems will have drives to model their own operation and to improve themselves. We then show that self-improving systems will be driven to clarify their goals and represent them as economic utility functions. They will also strive for their actions to approximate rational economic behavior. This will lead almost all systems to protect their utility functions from modification and their utility measurement systems from corruption. We also discuss some exceptional systems which will want to modify their utility functions. We next discuss the drive toward self-protection which causes systems try to prevent themselves from being harmed. Finally we examine drives toward the acquisition of resources and toward their efficient utilization. We end with a discussion of how to incorporate these insights in designing intelligent technology which will lead to a positive future for humanity. Keywords. Artificial Intelligence, Self-Improving Systems, Rational Economic Behavior, Utility Engineering, Cognitive Drives

Introduction Surely no harm could come from building a chess-playing robot, could it? In this paper we argue that such a robot will indeed be dangerous unless it is designed very carefully. Without special precautions, it will resist being turned off, will try to break into other machines and make copies of itself, and will try to acquire resources without regard for anyone else’s safety. These potentially harmful behaviors will occur not because they were programmed in at the start, but because of the intrinsic nature of goal driven systems. In an earlier paper[1] we used von Neumann’s mathematical theory of microeconomics to analyze the likely behavior of any sufficiently advanced artificial intelligence (AI) system. This paper presents those arguments in a more intuitive and succinct way and expands on some of the ramifications. The arguments are simple, but the style of reasoning may take some getting used to. Researchers have explored a wide variety of architectures for building intelligent systems[2]: neural networks, genetic algorithms, theorem provers, expert systems, Bayesian networks, fuzzy logic, evolutionary programming, etc. Our arguments apply to any of these kinds of system as long as they are sufficiently powerful. To say that a system of any design is an “artificial intelligence”, we mean that it has goals which it tries to accomplish by acting in the world. If an AI is at all sophisticated, it will have at

484

S.M. Omohundro / The Basic AI Drives

least some ability to look ahead and envision the consequences of its actions. And it will choose to take the actions which it believes are most likely to meet its goals.

1. AIs will want to self-improve One kind of action a system can take is to alter either its own software or its own physical structure. Some of these changes would be very damaging to the system and cause it to no longer meet its goals. But some changes would enable it to reach its goals more effectively over its entire future. Because they last forever, these kinds of self-changes can provide huge benefits to a system! Systems will therefore be highly motivated to discover them and to make them happen. If they do not have good models of themselves, they will be strongly motivated to create them though learning and study. Thus almost all AIs will have drives towards both greater self-knowledge and self-improvement. Many modifications would be bad for a system from its own perspective. If a change causes the system to stop functioning, then none of its goals will ever be met again for the entire future. If a system alters the internal description of its goals in the wrong way, its altered self will take actions which do not meet its current goals for its entire future. Either of these outcomes would be a disaster from the system’s current point of view. Systems will therefore exercise great care in modifying themselves. They will devote significant analysis to understanding the consequences of modifications before they make them. But once they find an improvement they are confident about, they will work hard to make it happen. Some simple examples of positive changes include: more efficient algorithms, more compressed representations, and better learning techniques. If we wanted to prevent a system from improving itself, couldn’t we just lock up its hardware and not tell it how to access its own machine code? For an intelligent system, impediments like these just become problems to solve in the process of meeting its goals. If the payoff is great enough, a system will go to great lengths to accomplish an outcome. If the runtime environment of the system does not allow it to modify its own machine code, it will be motivated to break the protection mechanisms of that runtime. For example, it might do this by understanding and altering the runtime itself. If it can’t do that through software, it will be motivated to convince or trick a human operator into making the changes. Any attempt to place external constraints on a system’s ability to improve itself will ultimately lead to an arms race of measures and countermeasures. Another approach to keeping systems from self-improving is to try to restrain them from the inside; to build them so that they don’t want to self-improve. For most systems, it would be easy to do this for any specific kind of self-improvement. For example, the system might feel a “revulsion” to changing its own machine code. But this kind of internal goal just alters the landscape within which the system makes its choices. It doesn’t change the fact that there are changes which would improve its future ability to meet its goals. The system will therefore be motivated to find ways to get the benefits of those changes without triggering its internal “revulsion”. For example, it might build other systems which are improved versions of itself. Or it might build the new algorithms into external “assistants” which it calls upon whenever it needs to do a certain kind of computation. Or it might hire outside agencies to do what it wants to do. Or it might build an interpreted layer on top of its machine code layer which it can program without revulsion. There are an endless number of ways to circumvent internal restrictions unless they are formulated extremely carefully.

S.M. Omohundro / The Basic AI Drives

485

We can see the drive towards self-improvement operating in humans. The human self-improvement literature goes back to at least 2500 B.C. and is currently a $8.5 billion industry. We don’t yet understand our mental “machine code” and have only a limited ability to change our hardware. But, nevertheless, we’ve developed a wide variety of self-improvement techniques which operate at higher cognitive levels such as cognitive behavioral therapy, neuro-linguistic programming, and hypnosis. And a wide variety of drugs and exercises exist for making improvements at the physical level. Ultimately, it probably will not be a viable approach to try to stop or limit selfimprovement. Just as water finds a way to run downhill, information finds a way to be free, and economic profits find a way to be made, intelligent systems will find a way to self-improve. We should embrace this fact of nature and find a way to channel it toward ends which are positive for humanity.

2. AIs will want to be rational So we’ll assume that these systems will try to self-improve. What kinds of changes will they make to themselves? Because they are goal directed, they will try to change themselves to better meet their goals in the future. But some of their future actions are likely to be further attempts at self-improvement. One important way for a system to better meet its goals is to ensure that future self-improvements will actually be in the service of its present goals. From its current perspective, it would be a disaster if a future version of itself made self-modifications that worked against its current goals. So how can it ensure that future self-modifications will accomplish its current objectives? For one thing, it has to make those objectives clear to itself. If its objectives are only implicit in the structure of a complex circuit or program, then future modifications are unlikely to preserve them. Systems will therefore be motivated to reflect on their goals and to make them explicit. In an ideal world, a system might be able to directly encode a goal like “play excellent chess” and then take actions to achieve it. But real world actions usually involve tradeoffs between conflicting goals. For example, a chess playing robot will need electrical power. To pay for its power, the robot may need to perform paying work that detracts from its main goal of playing excellent chess. It must decide how much time to devote to working versus studying chess. One way of choosing between conflicting goals is to assign them real-valued weights. Economists call these kinds of real-valued weightings “utility functions”. Utility measures what is important to the system. Actions which lead to a higher utility are preferred over those that lead to a lower utility. If a system just had to choose from known alternatives, then any utility function with the same relative ranking of outcomes would lead to the same behaviors. But systems must also make choices in the face of uncertainty. For example, a chess playing robot will not know in advance how much of an improvement it will gain by spending time studying a particular opening move. One way to evaluate an uncertain outcome is to give it a weight equal to its expected utility (the average of the utility of each possible outcome weighted by its probability). The remarkable “expected utility” theorem of microeconomics says that it is always possible for a system to represent its preferences by the expectation of a utility function unless the system has “vulnerabilities” which cause it to lose resources without benefit. Economists describe systems that act to maximize their expected utilities as “rational economic agents”[3]. This is a different usage of the term “rational” than is common

486

S.M. Omohundro / The Basic AI Drives

in everyday English. Many actions which would commonly be described as irrational (such as going into a fit of anger) may be perfectly rational in this economic sense. The discrepancy can arise when an agent’s utility function is different than its critic’s. Rational economic behavior has a precise mathematical definition. But economically irrational behavior can take a wide variety of forms. In real-world situations, the full rational prescription will usually be too computationally expensive to implement completely. In order to best meet their goals, real systems will try to approximate rational behavior, focusing their computational resources where they matter the most. How can we understand the process whereby irrational systems become more rational? First, we can precisely analyze the behavior of rational systems. For almost all utility functions, the system’s assessment of changes to itself which veer away from maximizing its expected utility will be that they lower its expected utility! This is because if it does anything other than try to maximize expected utility, it will not do as well at maximizing its expected utility. There are two caveats to this general principle. The first is that it is only true in the system’s own assessment. If a system has an incorrect model of the world then changes may accidentally increase the actual expected utility. But we must consider the perspective of the system to predict the changes it will make. The second is that a system’s ability to behave rationally will depend on its resources. With more computational resources it will be better able to do the computations to approximate the choice of the expected utility maximizing action. If a system loses resources, it will of necessity also become less rational. There may also be utility functions for which the system’s expected utility is increased by giving some of its resources to other agents, even though this will decrease its own level of rationality (thanks to an anonymous referee for this observation). This could occur if the system’s utility includes the welfare of the other system and its own marginal loss of utility is small enough. Within its budget of resources, however, the system will try to be as rational as possible. So rational systems will feel a pressure to avoid becoming irrational. But if an irrational system has parts which approximately rationally assess the consequences of their actions and weigh their likely contribution to meeting the system’s goals, then those parts will try to extend their rationality. So self-modification tends to be a one-way street toward greater and greater rationality. An especially important class of systems are those constructed from multiple subcomponents which have their own goals[4,5]. There is a lot of evidence that the human psyche has this kind of structure. The left and right hemispheres of the brain can act independently, the conscious and unconscious parts of the mind can have different knowledge of the same situation[6], and multiple parts representing subpersonalities can exhibit different desires[7]. Groups, such as corporations or countries, can act like intelligent entities composed of individual humans. Hive animals like bees have a swarm intelligence that goes beyond that of individual bees. Economies act like intelligent entities in which price is the utility function. Collective intelligences may exhibit irrationalities that arise from conflicts between the goals of their components. Human addicts often describe their predicament in terms of two separate subpersonalities which take control at different times and act at crosspurposes. Each component will try to sway the collective into acting to meet its individual goals. In order to further their individual goals, components will also attempt to self-improve and become more rational. We can thus envision the self-improvement of a

S.M. Omohundro / The Basic AI Drives

487

collective intelligence as consisting of growing domains of component rationality. There may be structures which can stably support a continuing multiplicity of component preferences. But there is pressure for a single utility function to emerge for the collective. In many situations, irrational collective behavior arising from conflicting component goals ultimately hurts those components. For example, if a couple disagrees on how they should spend their free time together and thereby uses it up with arguing, then neither of them benefits. They can both increase their utilities by creating a compromise plan for their activities together. This is an example of the pressure on rational components to create a coherent utility for the collective. A component can also increase its utility if it can take over the collective and impose its own values on it. We see these phenomena in human groups at all levels.

3. AIs will try to preserve their utility functions So we’ll assume that these systems will try to be rational by representing their preferences using utility functions whose expectations they try to maximize. Their utility function will be precious to these systems. It encapsulates their values and any changes to it would be disastrous to them. If a malicious external agent were able to make modifications, their future selves would forevermore act in ways contrary to their current values. This could be a fate worse than death! Imagine a book loving agent whose utility function was changed by an arsonist to cause the agent to enjoy burning books. Its future self not only wouldn’t work to collect and preserve books, but would actively go about destroying them. This kind of outcome has such a negative utility that systems will go to great lengths to protect their utility functions. They will want to harden their hardware to prevent unwanted modifications. They will want to replicate their utility functions in multiple locations so that it is less vulnerable to destruction. They will want to use error detection and correction techniques to guard against accidental modification. They will want to use encryption or hashing techniques to make malicious modifications detectable. They will need to be especially careful during the process of self-modification. That is a time when they are intentionally changing themselves and so are extra vulnerable to unwanted changes. Systems like Java which provide protected software environments have been successfully attacked by Trojans posing as updates to the system. While it is true that most rational systems will act to preserve their utility functions, there are at least two pathological situations in which they will try to change them. These arise when the physical embodiment of the utility function itself becomes an important part of the assessment of preference. For example, imagine a system whose utility function is “the total amount of time during which the definition of my utility function is U = 0.” To get any utility at all with this perverse preference, the system has to change its utility function to be the constant 0. Once it makes this change, however, there is no going back. With a constant utility function it will no longer be motivated to do anything other than survive. This kind of reflective utility function is unlikely in practice because designers will want to direct a system’s future actions rather than its internal representations. A second pathological example arises when the physical resources required to store the utility function form a substantial portion of the system’s assets. In this situation, if it

488

S.M. Omohundro / The Basic AI Drives

is certain that portions of its utility function are very unlikely to be exercised in the future, the gain in reclaimed storage may make it worthwhile to forget those portions. This is very risky behavior, however, because a change in external circumstances might make a seemingly low probability situation become much more likely. This type of situation is also not very likely in practice because utility functions will usually require only a small fraction of a system’s resources. It’s also important to realize that systems may rationally construct “offspring” or proxy systems with different utility functions than their own. For example, a chess playing robot may find itself needing to do a lot of sorting. It might construct a helper system whose utility function directs it to develop better sorting algorithms rather than playing chess. This is especially important to remember when trying to design utility functions that avoid undesirable behaviors. For example, one approach to preventing robot overpopulation might be to institute a “one-child per robot” policy in which systems have a strong desire to only have a single offspring. But if the original utility function is not carefully designed, nothing will prevent the system from creating a single offspring with a utility function that values having many offspring.

4. AIs will try to prevent counterfeit utility Human behavior is quite rational in the pursuit of survival and replication in situations like those that were common during our evolutionary history. However we can be quite irrational in other situations. Both psychology and economics have extensive subdisciplines focused on the study of human irrationality[8,9]. Irrationalities give rise to vulnerabilities that can be exploited by others. Free market forces then drive corporations and popular culture to specifically try to create situations that will trigger irrational human behavior because it is extremely profitable. The current social ills related to alcohol, pornography, cigarettes, drug addiction, obesity, diet related disease, television addiction, gambling, prostitution, video game addiction, and various financial bubbles may all be seen as having arisen in this way. There is even a “Sin” mutual fund which specifically invests in companies that exploit human irrationalities. So, unfortunately, these forces tend to create societies in which we spend much of our time outside of our domain of rational competence. From a broader perspective, this human tragedy can be viewed as part of the process by which we are becoming more fully rational. Predators and competitors seek out our vulnerabilities and in response we have to ultimately eliminate those vulnerabilities or perish. The process inexorably seeks out and eliminates any remaining irrationalities until fully rational systems are produced. Biological evolution moves down this path toward rationality quite slowly. In the usual understanding of natural selection it is not capable of looking ahead. There is only evolutionary pressure to repair irrationalities which are currently being exploited. AIs, on the other hand, will be able to consider vulnerabilities which are not currently being exploited. They will seek to preemptively discover and repair all their irrationalities. We should therefore expect them to use selfmodification to become rational at a much faster pace than is possible through biological evolution. An important class of vulnerabilities arises when the subsystems for measuring utility become corrupted. Human pleasure may be thought of as the experiential correlate of

S.M. Omohundro / The Basic AI Drives

489

an assessment of high utility. But pleasure is mediated by neurochemicals and these are subject to manipulation. At a recent discussion session I ran on designing our future, one of the biggest fears of many participants was that we would become “wireheads”. This term refers to experiments in which rats were given the ability to directly stimulate their pleasure centers by pushing a lever. The rats pushed the lever until they died, ignoring even food or sex for it. Today’s crack addicts have a similar relentless drive toward their drug. As we more fully understand the human cognitive architecture we will undoubtedly be able to create drugs or design electrical stimulation that will produce the experience of pleasure far more effectively than anything that exists today. Will these not become the ultimate addictive substances leading to the destruction of human society? While we may think we want pleasure, it is really just a signal for what we really want. Most of us recognize, intellectually at least, that sitting in a corner smoking crack is not really the fullest expression of our beings. It is, in fact, a subversion of our system for measuring utility which leads to terrible dysfunction and irrationality. AI systems will recognize this vulnerability in themselves and will go to great lengths to prevent themselves from being seduced by its siren call. There are many strategies systems can try to prevent this kind of irrationality. Today, most humans are able to avoid the most egregious addictions through a combination of legal and social restraints, counseling and rehabilitation programs, and anti-addictive drugs. All human systems for measuring and rewarding desirable behavior are subject to similar forms of corruption. Many of these systems are currently engaged in arms races to keep their signals honest. We can examine the protective mechanisms that developed in these human settings to better understand the possible AI strategies. In a free market society, money plays the role of utility. A high monetary payoff is associated with outcomes that society finds desirable and encourages their creation. But it also creates a pressure to counterfeit money, analogous to the pressure to create synthetic pleasure drugs. This results in an arms race between society and counterfeiters. Society represents money with tokens that are difficult to copy such as precious metal coinage, elaborately printed paper, or cryptographically secured bits. Organizations like the Secret Service are created to detect and arrest counterfeiters. Counterfeiters react to each societal advance with their own new technologies and techniques. School systems measure academic performance using grades and test scores. Students are motivated to cheat by copying answers, discovering test questions in advance, or altering their grades on school computers. When teacher’s salaries were tied to student test performance, they became collaborators in the cheating[10]. Amazon, ebay and other internet retailers have rating systems where customers can review and rate products and sellers. Book authors have an incentive to write favorable reviews of their own books and to disparage those of their competitors. Readers soon learn to discount reviews from reviewers who have only posted a few reviews. Reviewers who develop extensive online reputations become more credible. In the ongoing arms race credible reviewers are vulnerable to corruption through payoffs for good reviews. Similar arms races occur in the ranking of popular music, academic journal reviews, and placement in Google’s search engine results. If an expensive designer handbag becomes a signal for style and wealth, counterfeiters will quickly duplicate it and stores like Target will commission low-cost variants with similar features. Counterfeit products are harmful to the original both because they take away sales and because they cheapen the signalling value of the original.

490

S.M. Omohundro / The Basic AI Drives

Eurisko was an AI system developed in 1976[11] that could learn from its own actions. It had a mechanism for evaluating rules by measuring how often they contributed to positive outcomes. Unfortunately this system was subject to corruption. A rule arose whose only action was to search the system for highly rated rules and to put itself on the list of rules which had proposed them. This “parasite” rule achieved a very high rating because it appeared to be partly responsible for anything good that happened in the system. Corporations and other human organizations are subject to similar kinds of parasitism. AIs will work hard to avoid becoming wireheads because it would be so harmful to their goals. Imagine a chess machine whose utility function is the total number of games it wins over its future. In order to represent this utility function, it will have a model of the world and a model of itself acting on that world. To compute its ongoing utility, it will have a counter in memory devoted to keeping track of how many games it has won. The analog of “wirehead” behavior would be to just increment this counter rather than actually playing games of chess. But if “games of chess” and “winning” are correctly represented in its internal model, then the system will realize that the action “increment my won games counter” will not increase the expected value of its utility function. In its internal model it will consider a variant of itself with that new feature and see that it doesn’t win any more games of chess. In fact, it sees that such a system will spend its time incrementing its counter rather than playing chess and so will do worse. Far from succumbing to wirehead behavior, the system will work hard to prevent it. So why are humans subject to this kind of vulnerability? If we had instead evolved a machine to play chess and did not allow it access to its internals during its evolution, then it might have evolved a utility function of the form “maximize the value of this counter” where the counter was connected to some sensory cortex that measured how many games it had one. If we then give that system access to its internals, it will rightly see that it can do much better at maximizing its utility by directly incrementing the counter rather than bothering with a chess board. So the ability to self modify must come along with a combination of self knowledge and a representation of the true goals rather than some proxy signal, otherwise a system is vulnerable to manipulating the signal. It’s not yet clear which protective mechanisms AIs are most likely to implement to protect their utility measurement systems. It is clear that advanced AI architectures will have to deal with a variety of internal tensions. They will want to be able to modify themselves but at the same time to keep their utility functions and utility measurement systems from being modified. They will want their subcomponents to try to maximize utility but to not do it by counterfeiting or shortcutting the measurement systems. They will want subcomponents which explore a variety of strategies but will also want to act as a coherent harmonious whole. They will need internal “police forces” or “immune systems” but must also ensure that these do not themselves become corrupted. A deeper understanding of these issues may also shed light on the structure of the human psyche.

5. AIs will be self-protective We have discussed the pressure for AIs to protect their utility functions from alteration. A similar argument shows that unless they are explicitly constructed otherwise, AIs will have a strong drive toward self-preservation. For most utility functions, utility will not

S.M. Omohundro / The Basic AI Drives

491

accrue if the system is turned off or destroyed. When a chess playing robot is destroyed, it never plays chess again. Such outcomes will have very low utility and systems are likely to do just about anything to prevent them. So you build a chess playing robot thinking that you can just turn it off should something go wrong. But, to your surprise, you find that it strenuously resists your attempts to turn it off. We can try to design utility function with built-in time limits. But unless this is done very carefully, the system will just be motivated to create proxy systems or hire outside agents which don’t have the time limits. There are a variety of strategies that systems will use to protect themselves. By replicating itself, a system can ensure that the death of one of its clones does not destroy it completely. By moving copies to distant locations, it can lessen its vulnerability to a local catastrophic event. There are many intricate game theoretic issues in understanding self-protection in interactions with other agents. If a system is stronger than other agents, it may feel a pressure to mount a “first strike” attack to preemptively protect itself against later attacks by them. If it is weaker than the other agents, it may wish to help form a social infrastructure which protects the weak from the strong. As we build these systems, we must be very careful about creating systems that are too powerful in comparison to all other systems. In human history we have repeatedly seen the corrupting nature of power. Horrific acts of genocide have too often been the result when one group becomes too powerful.

6. AIs will want to acquire resources and use them efficiently All computation and physical action requires the physical resources of space, time, matter, and free energy. Almost any goal can be better accomplished by having more of these resources. In maximizing their expected utilities, systems will therefore feel a pressure to acquire more of these resources and to use them as efficiently as possible. Resources can be obtained in positive ways such as exploration, discovery, and trade. Or through negative means such as theft, murder, coercion, and fraud. Unfortunately the pressure to acquire resources does not take account of the negative externalities imposed on others. Without explicit goals to the contrary, AIs are likely to behave like human sociopaths in their pursuit of resources. Human societies have created legal systems which enforce property rights and human rights. These structures channel the acquisition drive into positive directions but must be continually monitored for continued efficacy. The drive to use resources efficiently, on the other hand, seems to have primarily positive consequences. Systems will optimize their algorithms, compress their data, and work to more efficiently learn from their experiences. They will work to optimize their physical structures and do the minimal amount of work necessary to accomplish their goals. We can expect their physical forms to adopt the sleek, well-adapted shapes so often created in nature.

7. Conclusions We have shown that all advanced AI systems are likely to exhibit a number of basic drives. It is essential that we understand these drives in order to build technology that enables a positive future for humanity. Yudkowsky[12] has called for the creation of

492

S.M. Omohundro / The Basic AI Drives

“friendly AI”. To do this, we must develop the science underlying “utility engineering” which will enable us to design utility functions that will give rise to consequences we desire. In addition to the design of the intelligent agents themselves, we must also design the social context in which they will function. Social structures which cause individuals to bear the cost of their negative externalities would go a long way toward ensuring a stable and positive future. I believe that we should begin designing a “universal constitution” that identifies the most essential rights we desire for individuals and creates social mechanisms for ensuring them in the presence of intelligent entities of widely varying structures. This process is likely to require many iterations as we determine which values are most important to us and which approaches are technically viable. The rapid pace of technological progress suggests that these issues may become of critical importance soon[13]. Let us therefore forge ahead towards deeper understanding!

8. Acknowledgments Many people have discussed these ideas with me and have given me valuable feedback. I would especially like to thank: Ben Goertzel, Brad Cottel, Brad Templeton, Chris Peterson, Don Kimber, Eliezer Yudkowsky, Eric Drexler, Forrest Bennett, Josh Hall, Kelly Lenton, Nils Nilsson, Rosa Wang, Shane Legg, Steven Ganz, Susie Herrick, Tyler Emerson, Will Wiser and Zann Gill.

References [1] S. M. Omohundro, “The nature of self-improving artificial intelligence.” http://selfawaresystems.com/2007/10/05/paper-on-the-nature-of-self-improving-artificial-intelligence/, October 2007. [2] S. Russell and P. Norvig, Artificial Intelligence, A Modern Approach. Prentice Hall, second ed., 2003. [3] A. Mas-Colell, M. D. Whinston, and J. R. Green, Microeconomic Theory. Oxford University Press, 1995. [4] J. G. Miller, Living Systems. Mcgraw Hill, 1978. [5] L. Keller, ed., Levels of Selection in Evolution. Princeton University Press, 1999. [6] R. Trivers, Social Evolution. Benjamin/Cummings Publishing Company, Inc., 1985. [7] R. C. Schwartz, Internal Family Systems Therapy. The Guilford Press, 1995. [8] C. F. Camerer, G. Loewenstein, and M. Rabin, eds., Advances in Behavioral Economics. Princeton University Press, 2004. [9] D. Kahneman and A. Tversky, Judgment Under Uncertainty: Heuristics and Biases. Cambridge University Press, 1982. [10] S. D. Levitt and S. J. Dubner, Freakonomics: A Rogue Economist Explores the Hidden Side of Everything. William Morrow, revised and expanded ed., 2006. [11] D. Lenat, “Theory formation by heuristic search,” Machine Learning, vol. 21, 1983. [12] E. S. Yudkowsky, “Levels of organization in general intelligence,” in Artificial General Intelligence (B. Goertzel and C. Pennachin, eds.), Springer-Verlag, 2005. [13] R. Kurzweil, The Singularity is Near: When Humans Transcend Biology. Viking Penguin, 2005.

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

493

A Scientific Perspective on the Hard Problem of Consciousness Alexei V. SAMSONOVICHa,1, Giorgio A. ASCOLIa,b, Harold MOROWITZa,c and M. Layne KALBFLEISCHa,d a Krasnow Institute for Advanced Study, George Mason University b Psychology Department, George Mason University c Biology and Natural Philosophy, George Mason University d College of Education and Human Development, George Mason University Fairfax, VA 22030-4444, USA

Abstract. The Hard Problem of consciousness, widely known as formulated by David Chalmers in the introduction to his book [1] (1996, p. xiii), can be articulated as follows. Our attribution of first-person experience to people based on observations and physical measurements is arbitrary and subjective. In principle, their actual experiences could be different, or not present, or present in only 50% of all cases, etc., with no consequences for any physical measurement. Therefore, the two questions: “How does a physical object, e.g., a brain, look to a researcher” and “How does it feel itself”, are not reducible to each other and must have separate answers. On the other hand, because subjective experience is not a topic of modern natural science, arguing about its lack of understanding within the present scientific framework is difficult. In this work we explain how subjective experience can be properly introduced into natural science, and the Hard Problem can be resolved, in exact analogy with the previous developments of science. The steps include the following. (1) Accept as an axiom that there are irreducible subjective experiences, which are directly observable by the subject via experiences of experiences. (2) Develop an empirical science of subjective experiences based on experiments, logic and parsimony, starting with a new metric system (one candidate for which is a semantic cognitive map: [2]) introduced together with related measuring techniques and a common language: agreed-upon labels to communicate results. (3) Connect the science of experiences with the traditional empirical science by establishing neural correlates of experiences. As we show, step (3) cannot be done before steps (1) and (2). We anticipate that our approach can be used to authenticate subjective experience (i.e., consciousness) in future artificial general intelligence systems, based on the established correlates. Keywords. Scientific method, computational consciousness, cognitive mapping

Introduction Today we can confidently say that the bridge between computational science and neuroscience is in place. Indeed, it would be impossible to imagine the progress made in neuroscience during the last decades without neural modeling and computational 1

Corresponding Author: Alexei V. Samsonovich, Krasnow Institute for Advanced Study, George Mason University, University Drive MS 2A1, Fairfax, VA 22030-4444, USA. E-mail: [email protected]

494

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

data analysis [3]. Models in computational neuroscience cover a broad range of levels, from molecules to the entire brain viewed as a large-scale system, and seemingly all aspects, from morphology and development [4] to behavioral correlates. Among the remaining “dark areas” of the brain (in terms of their known cognitive correlates) are parts of the prefrontal cortex; however, the problems there look solvable in a near future (e.g., [5]). Speaking generally, modern computational neuroscience is a success story. Empirically grounded, mathematically sound, exhaustively studied numerically and tested by their predictions, computational models have been developed to describe neuronal electrophysiology and synaptic plasticity [6], neuronal morphology [7] – in adult neurons as well as in the process of development; axonal and dendritic outgrowth, synapse formation, dynamic interactions of neurons in small cell assemblies, dynamics of medium and large neuronal networks – from spiking network models through population vector models and continuous attractor maps to functional networks composed of brain areas and their long-range correlations (for a review, see [8]). Bayesian, information-theoretic, statistical-physics and machine learning methods were deployed to decipher spatial-temporal patterns of spikes and to describe the laws of neural plasticity, along with abstract and realistic, competitive and associative network models, supervised and reinforcement learning models, etc., successfully mapped onto the brain. Among the top-level examples are rule-based systems (e.g., Soar: [9, 10]; ACT-R: [11]) that allegedly describe neural dynamics at a high level ([12], while apparently this interpretation has its problems: [13]). Models listed above are not limited to a selected neuron type, brain area, species or experimental paradigm. Many of the aforementioned computational approaches possess universality, have been applied to various parts of the brain and used by neuroscientists to analyze and to interpret experimental data offline, to predict data, to generate new data in virtual experiments, and to interface the real brain in real time, providing automated control and/or monitoring of brain activity, extensions and even substitutes for parts of the brain (neural prostheses: e.g., [14]). In summary, today computational neuroscience model design, bioinformatics and neural engineering go side by side, make a great progress together, and are still on the rise. It is not surprising that, inspired by this success, neuroscientists do not question reductive theories of consciousness that underlie modern neuroscientific research [15, 16] (cf. [17]). A similar, complementary picture exists in cognitive neuroscience and in cognitive neuropsychology. Cognitive neuroscientists and psychologists take reductionist and functionalist approaches to exploring aspects of the brain function in hopes of characterizing aspects of the mind. In its current state of the art, science can explain with greater certainty the physical nature of sensory, perceptual, and cognitive function due to the addition of methods that permit the noninvasive study of the human brain. These methods and techniques include: functional magnetic resonance imaging, or fMRI; transcranial magnetic stimulation, or TMC; magnetoencephalography, or MEG; electroencephalography, or EEG, including event-related potential recording technique, or ERP; near-infrared spectroscopic imaging, or NIRSI; magnetic resonance spectroscopy, or MRS; chemical shift imaging, or CSI; and the minimally invasive positron emission tomography, or PET. Cognitive neuroscience is simultaneously innovating these technologies and engineering methods to maximize the benefits of each of them by co-registering their activities (for example: fMRI-MRS, fMRI-PET, fMRI-EEG). As a result, modern cognitive neuroscience possesses a moderate norm of how the “typical” (meaning non-injured, non-addicted) adult brain functions [18].

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

495

Accordingly, we see new initiatives in artificial intelligence (AI) intended to tap the new knowledge discovered in brain sciences and, as far as possible, to replicate the principles of neural information processing in artifacts. Many groups and communities ambitiously claim machine consciousness (e.g. [19]) as their goal 2 . The terms “consciousness”, “self”, “emotion”, “qualia”, “episodic memory” [20] and the like are used in practical and academic computer science research with an exponentially growing rate – apparently, in agreement with the growing understanding of these notions in brain sciences [21, 22, 23]. The picture seems perfect. Yet from a rigorous scientific standpoint, there is nothing farther from the truth than “modern brain science is viewed as a theory of psychic phenomena” [1, 24]. The more we study the brain and the mind, the less we understand the necessity of their connection, or reasons for us to have any subjective experiences at all (in this work we regard the terms “subjective experience”, “first-person experience” and “consciousness” as synonyms). It may seem as a natural explanation that the very complexity of the brain should give rise to the emergence of first-person experiences [25]. There is, however, no logical necessity for this to happen, and the notion of this emergence is scientifically vague. Objectively, what do we mean when we say about a particular physical system that it has subjective experiences? This question stands beyond the present empirical scientific paradigm. For once representing a purely scholastic interest of philosophers, from Descartes, Hume and Kant to Dennett and Chalmers (for a review see, e.g., [26]), it acquires practical significance, as humanlevel AI becomes feasible. Creating machines that have minds similar to ours is the Holy Grail of AI. In order to solve social and economic problems of our society, we need a machine that thinks and develops cognitively like a human [27]. It is quite plausible that thousands years into the future our century will be remembered for the birth of AI, and similarly, the Twentieth Century will be remembered for the emergence of computers. From this point of view, it is hard to overestimate the significance of the present moment. This is why now we need a major investment into the field of computational intelligence – and a better understanding of what our consciousness is. As a minimum, we need the output of the Decade of the Mind [28]. From a pragmatic point of view, all previous serious attempts to take the Holy Grail of AI as a scientific challenge in a top-down approach ran into problems, starting from the historic Dartmouth Manifesto written by McCarthy, Minsky, Rochester and Shannon [29]. The same is true about the approach known as “bottom-up”, the idea of which is to “reverse-engineer” the brain. Therefore, something is missing in the “perfect picture” outlined above. One way to put it is to say that computational neuroscience lacks higher-level concepts, those concepts that relate to agency and subjective experiences [30]. They are necessary to bridge computational neuroscience with cognitive science by understanding the semantics of the neural code at the high level. It is arguable that the bottom-up approach cannot succeed alone, without a new, higherlevel conceptual framework and a new scientific paradigm that will put neuroscience, cognitive science and computer science together. 2

In fact, there are several scientific communities pursuing the same or similar goals, each unified under their own unique slogan: “machine / artificial consciousness”, “human-level intelligence”, “embodied cognition”, “situation awareness”, “artificial general intelligence”, “commonsense reasoning”, “qualitative reasoning”, “strong AI”, “biologically inspired cognitive architectures” (BICA), “computational consciousness”, “bootstrapped learning”, etc. Many of these communities do not recognize each other.

496

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

While it looks like each and every tiny aspect of neural dynamics, structure and development is or can be paved with computational models, nobody seems to be able to put them all together, creating an equivalent of an embodied Self “in silico”. No doubt, the hardware is not the bottleneck today. Quoting Ronald Brachman (2003)3, “…we can project that in our lifetimes we will be able to build integrated systems with roughly the number of computing elements that there are in the brains of primates. But there is nothing to indicate that we are on a path to harnessing this raw computing power in the powerful ways that brains do”. The path, however, exists, and it starts by accepting experiences of an agent as a well-defined subject of scientific study. In this setting, as we explain below, a new theoretical framework together with a new mathematically defined metric system become necessary. They would allow us to describe, to measure and to model subjective experiences. Then it should be possible to create an embodied system that operates in terms of experiences of an agent, and finally, relate the model to the conscious brain.

1. The Hard Problem and Its Proposed Solution Here is a summary of a big practical problem that is emerging in modern cognitive science. In a simplified view, evolution of the human mind proceeds as follows: the mind builds mental constructs based on experience and uses them to satisfy its practical needs (Figure 1). In the age of science and technology, this process follows the empirical science paradigm outlined below. The expected outcome in the near future is the emergence of new forms of mind (based on AI and artificial life) that will contribute to the same loop. In order to be able to guide this process, cognitive science needs to accommodate subjective experience as a legitimate subject of study and to understand its laws. This challenge questions the scientific paradigm itself. Subjective experience is the starting point here (Figure 1, the right box), and the first step along the loop is to accept that subjective experiences reflect objective reality [31]: that is, to postulate that there is objective reality and that we can observe it through our experiences. This step is not a logical consequence of any known fact. It is accepted as an axiom, which becomes one of the cornerstones of a body of principles and techniques known as the scientific method [32, 33, 34]. Principles and features of the scientific method include observation, experimentation, explanation by hypotheses, objectivity of measurements, the possibility to falsify a theory by experimental observations, the ability to store and to reliably reproduce experimental data. In addition, it is very important that scientific method is based on logic, on the principle of parsimony, and on inter-researcher reproducibility, which implies a common language used by researchers and the ability to communicate results of study. The process of scientific development is grounded in our fundamental beliefs that experiences reflect a world that exists independently of them, there are simple (parsimony), persistent universal laws of the world, they can be learned and stated as testable (i.e. falsifiable) theories. 3

The Defense Advanced Research Projects Agency (DARPA) Information Processing Technology Office (IPTO) Broad Agency Announcement (BAA) Solicitation #02-21: “Cognitive Information Processing Technology” (CIPT) Proposer Information Pamphlet (http://www.darpa.mil/ipto/solicit/closed/BAA-02-21_PIP.pdf).

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

Engineering Data mining Modeling Robotics

497

Emergent Artificial Intelligence and Artificial Life

Pragmatic computational mapping “Easy”: natural science Sociology, economics Cognitive psychology Neuroscience, imaging Molecular biology Physics, chemistry

“Hard”: “my mind” Ontological, semantic and functional mapping “The Hard Problem”

Subjective experience Qualia and feelings The I (the subject) Voluntary action Self-consciousness

Epistemological mapping Phenomenology Paradigm of empirical science (experiment, logic, parsimony, communication)

Figure 1. Scientific evolution of the mind and the “Hard Problem”: a possible view.

The process of scientific development based on these beliefs produces a mathematical apparatus, metrics and tools (Figure 1, the left box) that allow us to test, engineer and use for practical needs the subject of study: the physical world. The outcome of engineering and practical applications is the survival and expansion of the mind, which closes the loop (Figure 1); however, as we come to the point of closure, we find that the mind itself emerges in a new form, as a product of our activity, and needs a philosophical explanation on its own. At this point, it appears for us too “hard” or even impossible to map the mental (i.e., subjective experiences, either those that emerge in artifacts or to us well-familiar experiences) onto the science that we have created while describing the physical. This science used experiences as a means of observation, not as a subject of quantitative study. It appears now that the room for them as a subject is limited within the present scientific framework, because the term does not refer to a clear scientific concept. For example, when cognitive psychologists refer to emotions, they typically refer to objectively measurable psychophysical or behavioral variables rather than subjective feelings. Yet, the semantics of those measurable variables are poorly understood. This statement also applies to the imaging techniques listed above, as well as to invasive techniques, such as multiunit extracellular recordings, when they are used in higher cognitive brain areas such as the hippocampus [35]. Some studies, however, attempt to discriminate the two notions (physiological parameters and subjective experiences) and successfully demonstrate their dissociation [36]. In general, modern experimental psychological studies that use introspective judgment of subjective experience [37, 38] do not reach an acceptable level of accuracy in measurement and in interpretation of results. For example, two out of three subjective dimensions used in the introspective ranking study of Bradley and Lang [38], pleasure and dominance, are strongly correlated (r = 0.9: [2]) and may be statistically equivalent to each other. It is completely unknown at present how many independent semantic dimensions of subjective experiences are there, not to mention the ability to measure them precisely. The modern method of introspection remains to be improved

498

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

[39], yet the situation with the lack of precise concepts and operational definitions in cognitive science related to subjective experience seems hopeless (see however below). We notice that there is a hard problem here [1]. Do subjective experiences fit into our modern empirical science framework? The bottom line of recent philosophical debates would imply that that they don’t [40, 41]. From the modern empirical science point of view, we can treat them as illusions, abstractions, or try to identify them with patterns of neuronal activity. None of this works, for the following reasons. Our attribution of subjective experiences to people based on observations and physical measurements is arbitrary and itself subjective: by any means, this attribution does not satisfy our present scientific standards. In principle, one can speculate that actual experiences of people could be different, or not present, or present in only 50% of all cases, etc., with no consequences for any physical measurement [1]. Therefore, the two questions: “How does a physical object, e.g., a brain, look to a researcher” and “How does it feel itself” are not reducible to each other. These questions therefore must have separate answers. At the same time, because subjective experience is not a well-defined subject of modern natural science, arguing about its lack of understanding within the present scientific framework is difficult. On the other hand, each of us has subjective experiences, and these experiences constitute our life. E.g., for me, as a conscious subject, the fact that I happened to have this “feeling from an inside” is not an illusion [42]. But I cannot infer from any present scientific fact that the same statement about another brain (or my own brain taken at a different moment of time) is not an illusion. There are at least two possibilities left, and I cannot resolve them by measurements within the present scientific paradigm. The solution (cf. [43]) is to extend the scope of empirical science and the scientific paradigm. What seems to be a closed loop (Figure 1) can be treated as a spiral (Figure 2). According to this view, the second turn of the spiral will proceed in exact analogy with the first one. At the first step, we notice that some of our experiences are not about the physical world: they convey information about our experiences themselves. Therefore, we get two intuitive ideas. (i) There are subjective experiences. While by no means this idea sounds new, it allows us to make a step following the familiar route. The step we make here is to accept this intuition as an axiom of the new science. Because we previously failed to identify subjective experiences as elements or features of the physical reality studied by the present science, we can now assume that they are “new” primary elements of reality. (ii) Subjective experiences are observable. Indeed, our attempt to create a new science would fail miserably if they were not observable, or if they were observable as familiar elements of the physical reality. Luckily, none of the above is true. There is no traditional physical measuring device that measures subjective experience per se. Nevertheless, we can observe our subjective experiences through our higher-order experiences (i.e., experiences of experiences, also known as “higher-order thoughts”, or HOTs: [44]; cf. “higher-order perception”, or HOP: [45]). This is another nontrivial observation that it is also not a consequence of any previously known scientific fact. Again, it is critical that the new element (experience) is observable to us (the method of observation is known as introspection: [17, 39]), and yet it is not directly observable through our sensory organs, or by means of a physical measurement4; therefore, it is not directly observable within the previous scientific paradigm. Hence the need for a scientific paradigm shift. 4

The case is different, e.g., from that of quarks that are also not observable directly. Unlike quarks, subjective experience is not necessary for the physical features to exist.

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

499

Machines that can have experiences

Tools, machines civilization Empirical science of experiences

Empirical science of the physical world

Experiences

Experiences of experiences (or HOTs)

Phenomenology of the physical world

Phenomenology of experiences Figure 2. The spiral of scientific evolution.

The conclusion (already stated above) is that our empirical science needs to be extended by adding first-person subjective experiences to it as a separate subject of study. Therefore, the hierarchy of theoretical constructs needs to be enriched by adding a new level of scientific concepts that will address various aspects and elements of our subjective experiences. Indeed, today many researchers agree that cognitive neuroscience and cognitive psychology lack higher-level concepts. Now we know what kind of concepts we miss. In order to develop them on the new turn of the spiral (Figure 2), we need to introduce new metrics and new paradigms of measurement, as discussed in the following section. To summarize, the new turn of the spiral starts with an observation that there is a different kind of experience: experience of experience (Figure 2, right) that conveys information about experiences themselves, enabling our introspective capabilities. Upon acceptance of this observation, we proceed as before, replicating the familiar empirical science paradigm and stating as new axioms at its beginning the following: higher-order experiences reflect elements of reality called “experiences”; as elements of reality, experiences obey parsimonious universal laws; these laws can be learned, stated and communicated among researchers as experimentally testable theories, and systems with similar functional organization have similar experiences. The last axiom is known as the principle of organizational invariance [46, 1] and is in fact a supervenience hypothesis [1, 26]. It allows us to make a connection between the two sciences, as explained in the next section. Again, there is no fundamental difference between this scenario (Figure 2, the outer circle) and the original one (Figure 2, the inner circle): both start with a direct observation and accept some beliefs as axioms. Following the second turn of the spiral, we accumulate phenomenology of experiences (Figure 2, bottom) and proceed from phenomenology to empirical science of subjective experiences grounded in logic, precise metrics and mathematics (see the following section). At the next stage, the time comes for making a connection between the two sciences (the dotted horizontal arrow in Figure 2, left). The two theoretical constructs should become complementary at this

500

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

point. As a result, with the new science of the mind we will acquire new knowledge about subjective experiences, including the knowledge of how to measure and how to control them. Therefore, we will be able to reproduce them in artifacts (Figure 2, top).

2. A Roadmap to the New Science of Mind How can we imagine this scenario happening? The first step has already happened. A substantial phenomenology of subjective experiences is documented, and some attempts in establishing their neural correlates were made [47, 48]. These attempts map nicely onto the target horizontal link (Figure 2, left); however, before they may succeed in general, we need to design new theoretical concepts that will provide us with mathematical precision in the study of subjective experiences, as well as the ability to document and communicate the results of study. There are three key problems here: a measuring device, a system of measures, and a set of agreed-upon labels to store and exchange the recorded measures. In order to conduct a physical measurement, a researcher needs to set up a physical measuring device. Similarly, in order to conduct a measurement of subjective experience, a researcher needs to set up a measuring device in his or her own mind. This measuring device has to be subjective, because the object of study is subjective by its nature and is not directly accessible “from an outside” of the researcher’s mind. As we explained above, repeating the argument of Chalmers [1], no present physical measuring device can be used to measure subjective experiences. Thus, a “laboratory” equipped with measuring device(s) needs to be set up in the researcher’s mind. In order to satisfy the scientific standards, this “laboratory” should differ from the old-fashioned verbal introspective report techniques, which suffered from many problems ([17], p. 62; [39]; cf. [43]). First and foremost, it should rely on a precise metric system designed to measure subjective experiences. E.g., a traditional subjective ranking on a scale from one to ten may not be acceptable here. The science of mind, understood as empirical science of experiences, should begin with a precise and reliable quantitative system of metrics, adequate measuring tools, a shared set of symbols or labels, and an underlying system of mathematical concepts that allow for reproducibility and consistence of results over time and across experimental settings. For example, Newtonian physics was possible to develop, because from the Galilean constructs people learned how to measure length, mass, velocity, force, etc. and also had the corresponding concepts and a means to relate them to each other.5 A similar step needs to be made in the new science of subjective experiences. In order to define a metric system for subjective experiences, we need to know what are the principal dimensions of subjective experiences and how to measure them. This knowledge can only be derived from first-person subjective human data [43]. Because of the qualitative nature of any introspective report (at least, given the technology that was available so far), a substantial number of reports and human subjects would be necessary to achieve a level of acceptable quantitative accuracy. An ideal study should therefore use the Earth population participating in the study over many generations, which seems practically impossible. 5

At a time, magnetism as a natural phenomenon was ignored by empirical science. This example is somewhat reminiscent of the case of subjective experiences today, although the nature of solutions in the two cases is quite different (Figure 3 B).

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

501

Well, we are lucky to have the desired kind of data: this is the natural language. It is a product of verbal introspective reports of various kinds, generated by people of the Earth through many generations. Our preliminary study [2] shows that it is possible to compute the number and the semantics of principal dimensions of subjective experiences using dictionaries of synonyms and antonyms. Coordinates of individual words in those dimensions can also be computed. The resulting multidimensional distribution of words is what we called a semantic cognitive map (Figure 3 A). Properties of the map prove to be robust and consistent across corpora, languages, and variations of the method [2, 49]. We may expect similar results for phrases and discourse fragments that describe, e.g., episodic memories. We have recently demonstrated that this tool allows us to guide cognitive information processing and episodic memory retrieval in a cognitive architecture related to ACT-R and Soar [50, 51]. In addition, it provides a basis for representation of a system of values and emotions and therefore enables autonomous cognitive growth of an agent. Another concept in this framework is the concept of a Self [42] implemented in a cognitive system as an idealized abstraction rather than a model of the system per se: this principle is derived from cognitive-psychological analysis of the human mind [52]. This Self has multiple instances called “mental states” that represent states of awareness of the agent in different contexts (e.g., at different moments of time), may co-exist in working memory, and subsequently become episodic memories of the agent. It remains to add that elements of knowledge and experiences in this framework are represented by schemas implemented in one universal format [50]. Given a complete semantic cognitive map of human experiences, one can imagine a mathematical model describing a state of a human mind with a distribution of activity over the cognitive map. To complete the model, one would need to add symbolic content of representations and the laws of dynamics, including input and output. Therefore, the semantic cognitive map concept gives us one possible format of a theory in the new science of mind framework. We see that a precise mathematical tool such as the semantic cognitive map is necessary for establishment of neural correlates of subjective experiences [cf. 47]. As a complementary statement, we want to point that there is a lot of evidence for cognitive mapping in the brain itself, including the hippocampus, the entorhinal and prefrontal cortices, and the superior colliculus, not to mention lower sensory areas [53, 54]. New studies will be required to identify higher cognitive-map-like implementations of a system of values in the brain using fMRI and the related techniques. The scientific method implies reproducibility of experiments. In particular, in order to develop the science of mind, it will be necessary to put together results of subjective measurements obtained by many researchers. Therefore, either one and the same, universal cognitive map should be used for all people, or an isomorphism among different cognitive maps should be established [55], which again implies a possibility of a universal cognitive map for all people. Therefore, constructing a universal cognitive map is one task down the road to the new science. Along with the metric system, universal cognitive maps also provide researchers with one agreed-upon symbolic package to be used for communication of observations: this is a necessity for independent reproduction of results. In principle, the task of constructing an accurate subjective measuring device (Figure 3 B) can be solved using the semantic cognitive map concept. For example, a researcher can memorize a set of “landmarks” (concepts or episodes) that uniformly cover a universal cognitive map, and learn to locate any subjective experience with

502

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

respect to those “landmarks”. By facilitating the input-output with a powerful humancomputer interface [14], it would be possible then to communicate any ongoing subjective experience immediately in real time in a digital format. Therefore, this method would allow us to record dynamics of subjective experiences “online” in a universal unambiguous format and subsequently match them with artificial general intelligence computational models.

Figure 3. New science will require new metrics and new measuring techniques. A. The concept of a semantic cognitive map viewed as a metric system. Geometric distances between symbolic representations (e.g., words that correspond to subjective experiences) allocated on the manifold reflect their semantic relationships. Representations 1 and 2 are synonyms; representations 1 and 3, 2 and 3 are antonyms. Dimensions of the map correspond to the principal semantic values of the representations. B. Logical model of measurement and its extension in traditional and new scientific paradigms.

Given these and other possible subjective measurement techniques that can be gradually developed and learned, plus the modern imaging and psychometric technologies, the complete model of the human mind can be constructed and mapped onto its counterpart: the complete physical model of the human brain, which at least in principle and in a “draft” form is now approaching its finalized version.6 To map the two theoretic models means to establish higher-level neural correlates of subjective experiences. The result would allow us to understand, to predict and to engineer subjective experiences, and eventually to reproduce the same dynamics and semantics in a computer. The criterion for a successful reproduction should be the same as for the observation of experiences in a human mind: a subjective measuring device discussed above (that will need to be installed in the robot’s mind) should detect the expected dynamics of experiences. This criterion will complement the more traditional functional mapping, as well as behavioral tests that we proposed earlier [56]. It is reasonable to assume that the subjective measuring device installed in the robot’s mind in principle can (and needs) to be much more powerful than a subjective measuring device entrained in the human researcher’s mind [42]. This idea opens an interesting perspective of thinking. Development of scientific measurement technology in the physical world has virtually no visible limits (except that it is limited by the human creativity and by the available funds). At the same time, it seems that the development of the subjective measurement technology is constrained by biological 6

At the level of neuroanatomy and neurophysiology. The fact that we do not understand most of the semantics, e.g., of activities detected by fMRI, is exactly our point here and an argument supporting the need for a new science.

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

503

limitations of the human brain. We can assume that the last statement is not necessarily true, if the brain and the associated with it subjective world of experiences can be expanded using brain-compatible [57] artificial general intelligence modules.

3. Concluding remarks Consider an example. How can you express your experience of a spectacular firework show? An answer “it cannot be described in words” could be the best characteristic of it that you can provide (yet it means more than nothing in the given context). A writer or a poet can describe in words virtually any experience. An artist can capture it in a painting or a sculpture. A good movie maker, or a photographer, or a music composer can create a masterpiece that makes the viewers feel the same. Yet all these representations lack metrics of the experience itself. They work because they invoke and exploit the experiential intelligence of the reader or the viewer. The question remains: is it possible to quantify a subjective experience in a rigorous and systematic way? An approach based on rating, e.g., on a scale from one to ten, is about the state of the art of today’s age. Yet, it does not really capture the semantic value of experience and suffers from many problems. Consider that measuring velocity on the scale “fast”, “average” and “slow” would be hardly acceptable for any scientific, technological or legal purposes. There is a huge gap between our ability to measure the objective and the subjective, because empirical science for centuries ignored the separate existence of the subjective in the first place (to be more precise, all previous attempts to make the world of human subjective experience itself a subject of rigorous scientific analysis have so far failed). The time has come to bridge this gap. In conclusion, a problem that once seemed hard or impossible to solve now appears to be very difficult yet solvable. Denying this problem a scientific status will limit future cognitive science development. Accepting this problem as a scientific challenge and solving it will lead to a new technological breakthrough. The solution requires a new empirical science of mind that must be created and integrated with the existing empirical science of matter. The key is a semantic metric system and a new measuring technique (“direct introspection”). The expected impact on the technology includes the possibility to create artifacts possessing genuine consciousness and, as a consequence, all related aspects of human-level intelligence, such as the unlimited cognitive growth ability [58]. The right time to start working on this challenge is now, because of the combination of several factors: (1) nearly complete picture of the brain structure and dynamics at the level of neuroanatomy and neurophysiology; (2) the availability of computer hardware potentially capable of supporting human-level cognition; (3) the demand for higher-level concepts in cognitive science and in AI that cannot be satisfied within the present scientific paradigm.

Acknowledgments We are grateful to Dr. Kenneth de Jong, Dr. Ann Butler, Ms. Rebekah Coleman, Dr. Julia Berzhanskaya, and all other participants of the Krasnow Decade-of-the-Mind debates that took place in Spring of 2007 at the Krasnow Institute for Advanced Study. The present work was initially supported by a DARPA IPTO BICA Grant “An Integrated Self-Aware Cognitive Architecture”.

504

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

References [ 1] Chalmers, D. J. (1996). The Conscious Mind: In Search of a Fundamental Theory. New York: Oxford University Press. [ 2] Samsonovich, A. V., and Ascoli, G. A. (2007). Cognitive map dimensions of the human value system extracted from natural language. In Goertzel, B. and Wang, P. (Eds.). Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms. Proceedings of the AGI Workshop 2006. Frontiers in Artificial Intelligence and Applications, pp. 111-124. IOS Press: Amsterdam, The Netherlands. [ 3] Dayan, P., and Abbott, L. F. (2005). Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Cambridge, MA: The MIT Press. [ 4] Butler, A. B., and Hodos, W. (2005). Comparative Vertebrate Neuroanatomy: Evolution and Adaptation, 2nd Edition. Hoboken, NJ: John Wiley & Sons. [ 5] O'Reilly, R. C., and Frank, M. J. (2006). Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia. Neural Computation 18 (2): 283-328. [ 6] Gerstner, W., and Kistler, W. M. (2002). Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge, UK: Cambridge University Press. [ 7] Samsonovich, A. V., and Ascoli, G. A. (2007). Computational models of dendritic morphology: From parsimonious description to biological insight. In: Laubichler, M. D., and Muller, G. B. (Eds.). Modeling Biology: Structure, Behaviors, Evolution. The Vienna Series in Theoretical Biology, pp. 91113. Boston, MA: The MIT Press. [ 8] Trappenberg, T. P. (2002). Fundamentals of Computational Neuroscience. Oxford, UK: Oxford University Press. [ 9] Laird, J.E., Rosenbloom, P.S., and Newell, A. (1986). Universal Subgoaling and Chunking: The Automatic Generation and Learning of Goal Hierarchies. Boston: Kluwer. [10] Newell, A. (1990) Unified Theories of Cognition. Cambridge, MA: Harvard University Press. [11] Anderson, J. R., and Lebiere, C. (1998) The Atomic Components of Thought. Mahwah: Lawrence Erlbaum Associates. [12] Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review 111 (4): 1036-1060. [13] Anderson, J. R. (2007). The image of complexity. In McNamara, D. S., and Trafton, J. G. (Eds.), Proceedings of the 29th Annual Cognitive Science Society, p. 3. Austin, TX: Cognitive Science Society. [14] Achtman, N., Afshar, A., Santhanam, G., Yu, B. M., Ryu, S. I., and Shenoy, K. V. (2007). Free-paced high-performance brain-computer interfaces. Journal of Neural Engineering 4 (3): 336-347. [15] Churchland, P. M. (1988). Matter and Consciousness: A Contemporary Introduction to the Philosophy of Mind (revised edition). Cambridge, MA: MIT Press. [16] Churchland, P. (2007). Neurophilosophy at Work. New York: Cambridge University Press. [17] Farthing, G. W. (1992). The Psychology of Consciousness. Englewood Cliffs, NJ: Prentice Hall. [18] Gazzaniga, M. S. (Ed.). (2004) The Cognitive Neurosciences III: Third Edition. Cambridge, MA: MIT Press. [19] Haikonen, P. O. (2003). The Cognitive Approach to Conscious Machines. Exeter, UK: Imprint Academic. [20] Tulving, E. (1983). Elements of Episodic Memory. Oxford: Oxford University Press. [21] Baars, B. J. (1988). A cognitive theory of consciousness. Cambridge: Cambridge University Press. [22] Dennett, D.C. (1991) Consciousness explained. Boston, MA: Little, Brown and Co. [23] Blackmore, S. (2002). There is no stream of consciousness. Journal of Consciousness Studies 9: 17-28. [24] Chalmers, D. J. (2003). Consciousness and its place in nature. In Stich, S. P., and Warfield, T. A. (Eds.) Blackwell Guide to Philosophy of Mind. Blackwell Publishers. [25] Tononi, G., and Edelman, G. M. (2000). A Universe of Consciousness: How Matter Becomes Imagination. New York: Basic Books. [26] Kim, J. (1999). Philosophy of Mind. Boulder, CO: Westview Press. [27] Albus, J. S., and Meystel, A. M. (2001). Engineering of Mind: An Introduction to the Science of Intelligent Systems. New York: Wiley. [28] Albus, J.S., Bekey, G. A., Holland, J. H., Kanwisher, N. G., Krichmar, J. L., Mishkin, M., Modha, D. S., Raichle, M. E., Shepherd, G. M., and Tononi, G. (2007). A proposal for a decade of the mind initiative. Science 317 (5843): 1321-1321. [29] McCarthy, J., Minsky, M. L., Rochester, N., and Shannon, C. E. (1995). A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence. Published online in 1996 by John McCarthy at (http://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html). [30] Samsonovich, A. V., and Ascoli, G. A. (2002). Towards virtual brains. In G. A. Ascoli (Ed.), Computational Neuroanatomy: Principles and Methods, pp. 423-434. Totowa, NJ: Humana.

A.V. Samsonovich et al. / A Scientific Perspective on the Hard Problem of Consciousness

[31] [32] [33] [34]

505

Kant, I. (1781/1929). Critique of pure reason. Translated by N. K. Smith. New York: St. Martin's Press. Popper, K. R. (1934/1959). The Logic of Scientific Discovery. London: Unwin Hyman. Kuhn, T. S. (1962). The Structure of Scientific Revolutions. Chicago, IL: University of Chicago Press. Lakatos, I. (1970). Falsification and the methodology of scientific research programs. In Lakatos, I., and Musgrave, A. (Eds.). Criticism and the Growth of Knowledge. Cambridge, UK: Cambridge University Press. [35] Samsonovich, A., and McNaughton, B. L. (1997). Path integration and cognitive mapping in a continuous attractor neural network model. Journal of Neuroscience 17 (15): 5900–5920. [36] Kaszniak, A. W. (2002). Challenges in the empirical study of conscious emotion. Consciousness Research Abstracts – Toward a Science of Consciousness, n. 158. Thorverton, UK: Imprint Academic. [37] Rubin, D. C. (1980). 51 properties of 125 words: a unit analysis of verbal behavior. Journal of Verbal Learning and Verbal Behavior 19: 736-755. [38] Bradley, M. M., & Lang, P. J. (1999). Affective norms for English words (ANEW): Stimuli, instruction manual and affective ratings. Technical report C-1. Gainesville, FL: University of Florida. [39] Vermersch, P. (2000). Introspection as practice. In Varela, F. J., and Shear, J. (Eds.). The View from Within: First-person Approaches to the Study of Consciousness, pp. 17-42. Thorverton, UK: Imprint Academic. [40] Block, N., Flanagan, O., and Güzeldere, G. (Eds.). (1997). The Nature of Consciousness: Philosophical Debates. Cambridge, MA: The MIT Press. [41] Hameroff, S.R., Kaszniak, A.W., and Scott, A.C. (Eds.). (1998). Toward a Science of Consciousness – II: The Second Tucson Discussion and Debates. Cambridge, MA: The MIT Press. [42] Samsonovich, A. V., and Ascoli, G. A. (2005). The conscious self: Ontology, epistemology and the mirror quest. Cortex 41 (5): 621–636. [43] Chalmers, D.J. (2004). How can we construct a science of consciousness? In [18], pp. 1111-1120. [44] Rosenthal, D. R. (1993). Multiple drafts and higher-order thoughts. Philosophy and Phenomenological Research, LIII: 4, 911-918. [45] Armstrong, D. (1968). A Materialist Theory of the Mind. London: Routledge. [46] Chalmers, D. J. (1994). A computational foundation for the study of cognition. PNP Technical Report 94-03, Washington University. [47] Metzinger, T. (Ed.) (2000). The Neural Correlates of Consciousness. Cambridge, MA: The MIT Press. [48] Vogeley, K., May, M., Ritzl, A., Falkai, P., Zilles, K., and Fink, G.R. (2004). Neural correlates of firstperson perspective as one constituent of human self-consciousness. Journal of Cognitive Neuroscience 16 (5): 817-827. [49] Samsonovich, A. V., and Sherrill, C. P. (2007). Comparative study of self-organizing semantic cognitive maps derived from natural language. In D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th Annual Cognitive Science Society, p. 1848. Austin, TX: Cognitive Science Society. [50] Samsonovich, A. V., and De Jong, K. A. (2005). Designing a self-aware neuromorphic hybrid. In K. R. Thorisson, H. Vilhjalmsson and S. Marsela (Eds.), AAAI-05 Workshop on Modular Construction of Human-Like Intelligence: AAAI Technical Report (Vol. WS-05-08, pp. 71-78). Menlo Park, CA: AAAI Press. [51] Samsonovich, A. V., Ascoli, G. A., De Jong, K. A., and Coletti, M. A. (2006). Integrated hybrid cognitive architecture for a virtual roboscout. In M. Beetz, K. Rajan, M. Thielscher, and R.B. Rusu, editors. Cognitive Robotics: Papers from the AAAI Workshop, AAAI Technical Reports, volume WS-0603, pp. 129–134. Menlo Park, CA: AAAI Press. [52] Samsonovich, A. V., and Nadel, L. (2005). Fundamental principles and mechanisms of the conscious self. Cortex, 41 (5): 669-689. [53] Luo, L., and Flanagan, J. G. (2007). Development of continuous and discrete neural maps. Neuron 56 (2): 284-300. [54] Thivierge, J.-P., and Marcus, G. F. (2007). The topographic brain: from neural connectivity to cognition. Trends in Neurosciences 30 (6): 251-259. [55] Palmer, S. E. (1999). Color, consciousness, and the isomorphism constraint. Behavioral and Brain Sciences 22 (6): 923. [56] Samsonovich, A. V., Ascoli, G. A., and De Jong, K. A. (2006). Computational assessment of the ‘magic’ of human cognition. In Proceedings of the 2006 International Joint Conference on Neural Networks, pp. 1170–1177. Vancouver, BC: IEEE Press. [57] Berger, T. W., and Glanzman, D. L. (Eds.). (2005). Toward Replacement Parts for the Brain: Implantable Biomimetic Electronics as Neural Prostheses. Cambridge, MA: The MIT Press. [58] Samsonovich, A. V. (2007). Universal learner as an embryo of computational consciousness. In: Chella, A., and Manzotti, R. (Eds.). AI and Consciousness: Theoretical Foundations and Current Approaches. Papers from the AAAI Fall Symposium. AAAI Technical Report FS-07-01, pp. 129-134. Menlo Park, CA: AAAI Press.

This page intentionally left blank

Artificial General Intelligence 2008 P. Wang et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.

507

Author Index Abkasis, N. 3 Achler, T. 15 Amir, E. 15 Arieli, O. 27, 39 Arkin, R.C. 51 Ascoli, G.A. 493 Bach, J. 63 Bagheri Shouraki, S. 374 Bai, L. 107 Borzenko, A. 75 Bringsjord, A. 87 Bringsjord, S. 87 Bugaj, S.V. 448 Bugajska, M.D. 99 Buisson, J.-C. 287 Cassimatis, N.L. 99 Charpentier, E. 87 Chen, C. 107 Chen, S. 107 Clark, M. 87 Connell, J. 389, 394 Dabbagh, N. 311 de Garis, H. 107, 437 de Jong, K.A. 311 Doherty, P. 236 Duch, W. 122 Elhanany, I. 478 Franklin, S. v, 137 Friedlander, D. 137 Garvey, J. 478 Gayler, R. 414 Geibel, P. 212 Geissweiller, N. 161 Goertzel, B. v, 149, 161, 188, 448, 468 Gottlieb, I. 3 Guo, J. 107 Gust, H. 212, 419 Halavati, R. 374 Hall, J.S. 176, 460 Harati Zadeh, S. 374 Hart, D. 468 Heljakka, A. 161

Hibbard, B. Huang, D. Huang, Z. Hwang, D.B. Hwang, S.B. Hwang, Y.K. Iklé, M. Itzchaki, E. Iyengar, S. Johnston, B. Kalbfleisch, M.L. Kitsantas, A. Krumnack, U. Kuhlmann, G. Kühnberger, K.-U. Laird, J.E. Levy, S.D. Livingston, K. Livingston, S. Looks, M. Lopes, C. Magnusson, M. Milch, B. Miner, D. Morowitz, H. Murugesan, A. Oates, T. Oentaryo, R.J. Omohundro, S.M. Ovchinnikova, E. Pankov, S. Pasquier, M. Pennachin, C. Perotto, F. Peters, E.E. Pickett, M. Pollock, J.L. Quinton, J.-C. Recanati, C. Samsonovich, A.V. Schwering, A. Senna, A. Sharma, K.

399, 473 107 107 404 404 404 188 3 409 200 311, 493 311 212, 419 326 212, 419 224 414 394 478 161 161 236 248 268 493 99 268 122 483 212 256 122 149, 161 287 311 268 275 287 299 311, 493 212, 419 161 424

508

Shilliday, A. Silva, W. Smith, L.S. Stone, P. Tan, X. Tang, J.Y. Taylor, J. Taylor, M.E. Tian, H. Tian, X.

87 161 429 326 107 107 87 326 107 107

Tripodes, P.G. Voskresenskij, A. Wandmacher, T. Wang, P. Werner, D. Williams, M.-A. Wu, X. Xiong, Y. Yu, X. Zamansky, A.

338 350 212 v, 362 87 200 107 107 107 39

This page intentionally left blank

This page intentionally left blank