Ai: The Tumultuous History Of The Search For Artificial Intelligence [1 ed.] 0465029973, 9780465029976

Citation preview

"Superbly written and utterly engrossing."

— Wall

Street Journal

TUMULTUOUS HISTORV OF THE

2

ARTIFICIAL INTF

DANIEL "A

superb history ...

it

was

like

C

R

going back to see

it

2

E all

V

FOR

H E

NCE

I

with other eyes."

— Marvin Minsky

I

/ft

i

7- u~

-f

\y*> "?

•»

THE

TUMULTUOUS HISTORY

OF THE SEARCH FOR

ARTIFICIAL INTELLIGENCE

DANIEL CREVIER

BasicBooks A Division of WzvperGoWmsPublishers

Copyright

©

1993 by Daniel Crevier.

Published by BasicBooks,

A

Division of HarperCollins Publishers, Inc.

All rights reserved. Printed in the United States of America.

of

this

book may be reproduced

in

written permission except in the case of brief quotations in critical articles

No

part

any manner whatsoever without

embodied

and reviews. For information, address BasicBooks,

10 East 53rd Street,

New

York,

NY

10022-5299.

Designed by Ellen Levine

Library of Congress Cataioging-in-Publication Data Crevier, Daniel,

AI

:

1947-

the tumultuous history of the search for

artificial

intelligence / Daniel Crevier. p.

cm.

Includes bibliographical references and index.

ISBN 0-465-02997-3

ISBN 0-465-00104-1 1.

Artificial intelligence

(cloth)

(paper)

—

History.

I.

Title.

Q335.C66 1993 006.3'09— dc20

91-55461 CIP

94 95 96 97

CC/HC

987654321

To

Celine

Contents

ix

Preface

Acknowledgments Introduction:

Human 1

xiii

Probing the Mystery of

Intelligence

1

Engineering Intelligence: Computers and

Programming

9

2

The

3

The Dawn of

Golden Years: 1956-63

51

4

The Conquest of Micro Worlds: 1963-70

73

5

Clouds on the AI Horizon

108

6

The Tree of Knowledge

145

7

Coming of Age

163

8

The

197

9

Game

First

AI Program: Defining the

the Field

Rollercoaster of the 1980s Playing:

Checkmate

for

Machines?

26

217

Ill

CONTENTS

1

Souls of Silicon

237

1 1

How Many

281

12

The

Bulldozers for an Ant Colony?

Silicon Challengers in

Our

Future

312

Notes

342

Index

369

Preface

\\ ot too long ago,

ll

if

you walked into the computer room of MIT's

Artificial Intelligence

Laboratory, the

winding yellow brick path painted on the

first

thing you noticed was a

floor.

At the end of this

path,

dangling from the ceiling above a large computer, was a resplendent

rainbow. In case you missed the

first

two

references, a poster of Judy

Garland as Dorothy was taped to the computer's side and the computer

had been given the nickname Oz.

itself

In

reality,

Oz was

nothing more sophisticated than a mainframe

computer controlling earlier traveler

a

MIT's

network of smaller computers. But

just as

over the rainbow had hoped that a wizard named

might be able to fashion ers at

1

an

Oz

a brain for a straw-stuffed friend, the research-

Artificial Intelligence

Lab hoped

that their

Oz

might be

used to create computer-generated intelligence.

From

its

inception the Artificial Intelligence Laboratory at

occupied most of a a section

I

building in

what is known

as

MIT

has

Technology Square,

of campus a short jog away from the heart of the Massachu-

setts Institute

that

tall

of Technology on Mass Avenue.

spent far too

many

It

was on

this

campus

days and nights in the early 1970s busily

assembling a Ph.D. thesis on a topic related to AI, but different enough to keep

me away from

the lab

observed the goings on over

at

itself.

While doing

Tech Square with

my own

work,

a perplexed

I

and

— x

PIEFACE

somewhat envious out about some

For every so often information would leak

interest.

new and

exotic breakthrough that suggested that

where the computer world's future was being created and being loosed on the

demonstration

vividly a

TV camera,

us.

so that those of us interested in a

a

demonstration of the piling

image of a model block structure

to the camera. In another demonstration, an operator conversed

with a computer in English, ordering tion displayed

about

first

computer manipulate the arm of a robot

children's blocks, building a mirror

shown

remember

I

which the AI computer was hooked up to

in

future could observe the

up

of

rest

followed these developments with intense interest.

I

AI was

tested before

its

on

it

to manipulate a block construc-

and even had the machine answer questions

a screen,

Not knowing how to evaluate all this, many of us whether we had just witnessed the dawning coming age of AI or a bit of theater.

motivations.

in attendance weren't sure

experiment of the

The answer was not long the

1

960s and early

able to create,

would not be

it

1

coming. While these AI experiments of

in

970s were fun to watch and probably even enjoy-

soon became

lems in restricted

areas.

main sponsors of

early

clear that the techniques they

from dealing with

useful apart

Not AI

surprisingly, the U.S. military,

was

research,

employed

carefully simplified prob-

also

one of the

one of the

first

to have

second thoughts. Tech Square, and other centers that had sprung up

around the world with high hopes of

early success,

selves fighting for their very survival.

Human

was, was not about to yield

In the early 1980s, after

work

I

its

had

in Al-related research,

left

it

MIT

to return

home

to

Canada

to

read about the next breakthrough

I

years away, the idea that

whatever

secrets quickly.

expert systems. While broad-based

many

soon found them-

intelligence,

artificial intelligence

one could create systems

the decision-making processes of

human

might

still

be

that captured

experts, even if those systems

operated only on narrowly focused tasks, suddenly seemed on the

horizon sort

or, at

most,

of nonhuman

without

guilt

just

over

idiot savant,

or shame.

It

humans and machines, or

it.

The

expert system was promoted as a

but one that could be exploited for profit

could be tailored to diagnose the ailments of to select the optimal site for oil exploration

or the best wine for dinner.

The prospects were

exciting

and appeared

boundless. Bankrolled primarily by a great deal of private capital, expert

systems

moved out of

the laboratory and into the marketplace, where

si

PREFACE

many of

the

new companies

have always been part of a community that has a

artificial intelligence

common

foundered. But those people involved in

which numerous approaches

ideal, in respect to

rival

The

other for attention, funding, and the chance to succeed.

expert systems rekindled interest in the rival technology of

webs of interconnected,

neural networks, those

each

failure

specially

of

artificial

designed

processors that could be "trained" (through the application of varying degrees of voltage) to respond

At

that point,

on

own

their

began gathering material for

I

following years, which

I

Al-related business,

we may soon have

that

is

and running

my own

a

computer for

a

world

taking hold of the marketplace with litde noise or

Meanwhile, large AI projects

computers with

field,

A second generation of expert systems, this time more

savant than idiot, fanfare.

book. Over the

observed the next stage of the rollercoaster ride

I

now seems

chess champion.

this

spent renewing contacts, attending conferences,

interviewing the founders and stars of the

begin. It

to external stimuli.

common

worldly wise machines

CyC and SOAR seek to endow

like

sense in the not-too-distant future; such

may

acquire knowledge and refine their reason-

power by themselves, much as humans do, but with the advantages attendant on greater computer speed and memory.

ing

The

story of

naysayers,

AI

consists of successes

new hardware and

lots

and

failures, visionaries

of programs.

It is also

and

the story of a

how humans think. means uncovering the

slow but steady acquisition of knowledge about After

all,

the pursuit of

artificial intelligence

unbelievably complex layers of our

own

thought processes. The quest some crucial philosophical ishumans if researchers in artifi-

for artificial intelligence therefore raises sues.

What will be

cial intelligence

the consequences for

succeed and force us to share the world with

we

smarter than ourselves? Are

of the species that will replace to

make not only business and

legal,

and moral choices?

scientific decisions for us,

raised these

I

the events

I will relate

happened

emphasis stems only in part from

where ing

I

studied and keep so

AI research occurred

in

many

now

social,

in the

may

my own

friends:

bias

it is

surprise you.

United

States.

This

toward the country

a fact that

most pioneer-

America, probably because of the overbear-

ing interest of the U.S. military. are

but also

and similar questions with the

leaders of AI: their answers (and your conclusions)

Most of

entities

new Renaissance or the creation us? And should we rely on these creations facing a

My apologies

catching up: future accounts of

AI

to Japan will say

and Europe,

who

more about them!

Acknowledgments

am I me

deeply grateful to those participants in the history of AI

and help

their time

Particular thanks

in preparing this account

must go

to

of

who

lent

their activities.

Marvin Minsky, Herbert Simon, Allen

Newell, Gerald Sussman, Daniel Dennett, Joseph Weizenbaum, Ber-

John McClelland, Hans MoraDavid Waltz, Patrick Winston, Guy Lapalme, and

thold Horn, Randall Davis, Carl Hewitt, vec, Raj Reddy,

Marion

Finley,

Roger Schank,

who

granted

also read

me

interviews. Several of them, as well as

and commented on relevant sections of the

manuscript.

Many

thanks also to those friends and relatives

who

reviewed the

manuscript and supplied their comments: Simon Pare, Jacques Martineau, Francis Dupuis, Marie-Josee Rousseau, Laurens R. Schwartz,

Dowey, and, of course, Paul-Henri Crevier, my father, who helped out on the neurology and philosophy. Basic Books's Susan Rabiner must be acknowledged for her prodding help throughout; and Phoebe Hoss, for her enlightened revision of the Claire

manuscript.

Many

thanks also to those publishers and authors

who

granted

me

permission to quote from their copyrighted material: Academic Press, the I

American Council on Education, and Patrick Winston. could not have completed the project without the understanding

xlv

acknowledgments

attitude

toward book writing of my employer, the Ecole de Technologie

Superieure, which final

I

thank for providing time and secretarial help

months. Jocelyne

Hall, in particular,

bringing the illustrations into

My

final

deepest gratitude goes to

tience, love,

in

shape.

my

wife, Celine, for her unfailing pa-

and support throughout the long years

the project through.

in the

was of invaluable help

it

has taken to carry

Introduction

PROBING THE MYSTERY

HUMAN INTELLIGENCE

OF

It is not

my aim

or shock you

to surprise

now

to say that there are

Moreover, their ability to do these things future the

—

—

but the simplest way I can summarise

in the world machines that think, that learn is

going

and

—

to increase rapidly until

is

that create. in

a

visible

the range ofproblems they can handle will be coextensive with the range to which

human mind has

— Herbert Simon, 1957

been applied.

driven by some invisible hand, humans have always yearned to Asunderstand what makes them think, and be, and have tried to if

feel,

re-create that interior silicon chips

Long before

life artificially.

of the modern

digital

computer, mythologies and

the

vacuum

computer, long before the

literature

tubes and

first

mate the inanimate, from Pygmalion's attempt to bring to perfecdy sculpted Galatea to Gepetto's desire that the

Pinocchio be a

An

early

the royal city of

until the

in the

Napata

machine" was the in ancient

eligible heirs

god made known

grab the successor.

A priest,

the

life

wooden puppet

real boy.

"man

demise of a pharaoh,

Amon

analog

recorded a timeless need to ani-

Amon

statue

of the god

Egypt around 800

b.c.

Amon Upon

in

the

were marched past the statue of

his choice

by extending

his

arm

then "delivered" a consecrating speech.

to

1

of course, controlled the statue with levers and uttered the

I

2

A

sacred words through an opening on the back of the statue. Although likely that

those

who

and onlookers

alike,

knew

it is

took part

procedure was taken seriously: statue system

added up

to

of theater was being put on, the

in the

more than

would-be pharaohs

in the process,

that a bit

Egyptian mind, the priest-cum-

the

sum of its

and embodied

parts,

the god.

The

more sophisticated self-activated automata, atwork of the divine blacksmith Hephaestus. Among

Iliad describes

tributing

them

to the

the golden female attendants were gangly walking tripods, forerunners

of today's many-legged

how

field robots.

In the

molded

The truvius

such

likely inspiration for

tells

city

literary

automata were

of Alexandria. The

us that between the third and

and amusing things of

Icarus,

first

kinds

all

.

.

.

ones being

real

Roman

architect Vi-

centuries B.C., a school

of early engineers founded by Ctesibius "conceived vices

also recounts

woe of his son

which patrolled the shores of Crete. 2

the copper giant Talos,

engineered in the Greek

Homer

Iliad,

Daedalus, wing maker extraordinaire to the

.

.

.

automatic de-

ravens singing through

contrivances using the force of water, and figures that drank and

moved

3

about." Similarly, Hero, author of a treatise called Automata? treated his readers to a description of a steam-propelled carousel;

it

now

stands as

the first-known description of a steam engine!

In Europe, the late Middle Ages and the Renaissance saw a resur-

gence of automata. Roger Bacon reportedly spent seven years constructing talking figures. 5

automaton

in the

To honor

shape of a

Louis XII, Leonardo da Vinci built an

lion.

During the sixteenth and seventeenth

and French designers, such as Gio Battista Aleotti and Salomon de Caus, elaborated on designs of the Alexandrian school:

centuries, Italian

gardens and grottoes resonated with the songs of flutists,

pepped up

aristocratic receptions.

automaton

that

while

fully

artificial

birds and

animated nymphs, dragons, and satyrs

mechanical

6

Rene Descartes

built in

1649 an

he called "my daughter Francine." But on one of Des-

cartes' trips, a superstitious ship captain

happened

containing Francine and, frightened by her

lifelike

to

open the case

movements, threw

case and contents overboard.

By

the eighteenth century, the French artisan Jacques de

Vaucanson

could assemble his celebrated duck, which provided an almost complete imitation of its model.

One

prospectus described

it

as

an

of golden Copper that Drinks, Eats, Croaks, Splashes Digests as a live duck." Life-sized, the animal rested

on

"artificial

in

duck

water and

a waist-high case

INTRODUCTION: PROBING THE MYSTERY OF HUMAN INTELLIGENCE containing a

drum engraved with cogs and

passing through

the duck's legs caused

grammed in the Memory (ROM)

device.

Although

we would

cogs: today

artisans like Pierre

had reached

call

grooves.

3

Control rods

move in a manner prothe drum a Read Only

and Louis Jaquet-Droz kept producing

automata for the pleasure of the rich cal arts

to

it

7

their limits.

late into the century, the

The next

mechani-

leap required the automatic

switch, or electromechanical relay. This device contains an iron kernel

which can change positions under the influence of a magnetic erated by a current fed to the relay.

Depending on the

field

gen-

polarity

of

the current, the kernel opens or closes electrical contacts to motors, lights,

or even other relays. Interconnected relays can be used to build

mechanisms more complex than those allowed by interlocking gears and cams.

At the turn of the century, the Spaniard Leonardo Torres y Quevedo built a relay-activated automaton that played end-games in chess. The philosophers Leibnitz and Pascal had constructed mechanical computing devices centuries before, but those were perceived as soulless contraptions intended to alleviate the drudgery of addition 7

De

and subtraction.

Vaucanson's duck was a superb emulation of a living creature. But

a device that

would play chess

completely different:

it

against

seemed

to have the ability to think.

In early chapters of this book,

vacuum tubes and

transistors,

human opponents was something

I

shall describe

how relays

which formed the

computer, the invention that made

evolved into

basis for the digital

artificial intelligence

possible. Digital

computers are simply devices for manipulating discrete pieces of information,

of

initially

taken to correspond to numbers.

artificial intelligence

was

The

insight at the root

that these "bits" could just as well stand as

symbols for concepts that the machines would combine by the rules

strict

of logic or the looser associations of psychology. European philos-

ophers and mathematicians had been probing the issues involved in representing the world through abstract concepts for millennia; and they

had been asking

in the process

fundamental questions about the nature

of mind and thought. The American pioneers of AI,

and

later in

at first unwittingly

enlightened fashion, began to tap the insights of these

humanist predecessors. Expressions such

as "experimental epistemol-

ogy" and "applied ontology" then started to describe systematic, downto-earth research projects in

Emerging

as

it

computer

does from

many

science.

fields

—

philosophy, mathematics,

4

\|

psychology, even neurology

about

human

—

intelligence raises basic issues

artificial

memory,

intelligence,

mind/body problem, the

the

origins

of language, symbolic reasoning, information processing, and so

— from base metal — seeking

AI researchers

are

who

alchemists of old

like the

to create thinking

forth.

sought to create gold

machines from

infinitisi-

mally small bits of silicon oxyde.

The

birth

of AI was

tied to the efforts

trical

engineers, psychologists, and even

became

these figures never really ideas,

developed

mining the tions as

of a variety of

talented, in-

well-educated budding mathematicians, elec-

tellectually self-confident,

of the

early direction

political scientist.

of AI research per

a part

in other contexts,

AI researchers and then

one

were enormously

se; yet their

influential in deter-

Others made seminal contribu-

field.

left

Some of

the field to pursue other work.

few, there at the creation, remain there to this day.

The

A

various origins

of the creators of AI and the enormous influence of their work explains

some of

the colorful aspect of their pronouncements, and the roller-

coaster evolution of their trated in 1989,

even the

field.

As

the hoopla over cold fusion

of physics

staid science

is

not

illus-

immune

to

exaggeration and false claims. Yet excesses of optimism seem to occur

with particular frequency in AI. There are several reasons that AI

workers were, and often

still

are,

more

likely to

make exaggerated

claims

than their colleagues of other disciplines. First, there its

were plausible reasons

in AI's early years for believing in

rapid progress, and these induced early researchers to display the

excessive optimism that in using

came

to characterize the field. Early progresses

computers for arithmetic were

truly breathtaking. In a

few

years,

technology went from cranky mechanical calculators to machines that could perform thousands of operations per second.

It

was thus not

unreasonable to expect similar progress in using computers as manipulators

of symbols to imitate human reasoning.

One

misconception further enhanced

this temptation. Psychological

experiments in the 1950s and 1960s pointed to an oversimplified picture

of the mind. The Carnegie Mellon researcher and AI pioneer Herbert its

way around complex

obstacles, saying in effect that complexity lay in the

environment and not

Simon compared in the

mind

itself.

it

8

to a dim-witted ant

There

between ant and human

is

winding

truth in this statement, but bridging the gap

still

requires a giant leap in complexity. Yet, in

the postwar decades, controlled studies

reason showed their basic limitations.

on our ability to remember and The next time you look up a

INTRODUCTION: PROBING THE MYSTERY OF HUMAN INTELLIGENCE seven-digit

phone number,

seconds before

try thinking

dialing. If you're like

forget the number.

else for a

this will

few

make you

We can't keep more than five to nine items at a time

our short-term memory; and

in

of something

most people,

5

Our long-term memory

soon

as

as

we look

away, they vanish.

has an almost unlimited capacity, but

very slowly. Transferring an item from short-term to long-term takes several seconds.

(It

takes

me

it

learns

memory

about two minutes to learn by heart

phone number.) When we rate alternatives in any complicated problem, like a number puzzle, we need pencil and paper to make up for these deficiencies of our memory. the seven digits of a

Early

AI

from these

researchers reasoned that their computers did not suffer limitations.

Even

in those days, the

machines had memories

with capacities of thousands of items and access times of microseconds.

They could shuffle data much faster than fumbling, pencil-pushing humans can. Computers, thought Simon and his colleagues, should be able to take advantage of these capabilities to overtake humans: it was only a matter of a few years before suitable programming would let them do

it.

These researchers had not

realized that, in activities other than

purely logical thought, our minds function

puter yet devised.

They

much faster we are

are so fast, in fact, that

than any com-

not even con-

scious of their work. Pattern recognition and association

make up

the

core of our thought. These activities involve millions of operations carried in parallel, outside the field to hit a wall after inability to

of our consciousness. If AI appeared

earning a few quick victories,

it

did so owing to

its

emulate these processes.

Already deluded by

false expectations, early

further onto the path of exaggerated claims

by

a

AI workers were drawn myriad of other

One was

the recent and sudden emergence of

discipline.

Like

all

new frontiers, AI

AI

as

factors.

an identifiable

attracted a particular kind of person:

of academic security in the new

one

willing to face the lack

live

with the haphazard financing of the early years. So novel were the

insights offered

by the new technology that

early researchers

field

and to

looked

like

As some branches of deeply affecting modern philoso-

elephants in the well-tended flower beds of conventional sciences.

we

shall see,

AI brought about major

psychology and mathematics. phy.

To one

It is

also

revisions in

used to bringing forth such innovations, moderation

is

a

hard virtue to learn. I

have already mentioned that AI

is

a multidisciplinary science.

As

Herbert Simon told me: "AI has had problems from the beginning.

It

I

6 is

A

a

new

field in

which people came from many

know where many people imported them. And we still has meant they didn't always

norms of wheels

responsibility,

the time." 9

all

One

haven't established a set of

of Herbert Simon's best-known students, his

illustrates the incompatibility

Ph.D.

engineering graduate, Feigenbaum was

and worked on

registered at the business school,

ously belonged to psychology: the modeling of fields

computer

from which AI researchers emerge nowadays (psychology,

its

they are often at odds with each other.

common

own

accepted methodology, and

The

different branches

language, values, or standards of achievements.

discipline acts as a

moderator on other

fields

and

The need

it

of AI lack

A

uniform

of science and enables

AI

research communities to police themselves. fluence,

a subject that previ-

human memory. Each

science, linguistics, physics, philosophy, mathematics, neurol-

ogy, or electrical engineering) has

a

of disciplines that well

in a tangle

of AI with the academic structures of the

An

1950s and early 1960s.

of the

came from, because

of referencing, that keeps us from reinventing

Edward Feigenbaum, earned late

That

different directions.

things

their

lacks that sobering in-

shows.

to attract research

in researchers, especially in a

money can

young

space research, and astronomy,

also induce careless behavior

science. Like nuclear physics, aero-

AI

gets

its

funds from government

AI researchers found way they could channel money away from these traditional was to proclaim their merits, and the louder the better. Even

sources. Davids against Goliaths, the

young

early

that the only disciplines

AI

today,

researchers have a vested interest in AI's appearing solid and

confirmed. Public discussion of the failures and are against their interest.

predictions and

Simon

They now do

empty promises

likes to say that

AI

is

difficulties

of the

field

however, that rash

realize,

are not to their advantage. Herbert

"hankering for respectability."

It is

perhaps

symptomatic that many younger researchers do not repeat the errors of their elders in this respect. "I refuse to

Horn

thold

researcher

was

told me.

He compared

make

predictions,"

his attitude to that

MIT's Berof an older

who

in charge

of the AI conference

in

Boston ten years ago. There

were reporters swarming around, and he was saying things years

up

from

now

we'll

the things that

like

"Five

have robots going around your house picking

you dropped on the

floor."

I

dragged him into a

corner and told him, "Don't make these predictions! People have

INTRODUCTION: PROBING THE MYSTERY OF HUNAN INTELLIGENCE

done time

before and gotten into trouble. You're underestimating the

this

He said, "I don't were after my retirement

will take."

it

chosen

and people

retired

7

care.

date!"

come back and

will

Notice that

all

the dates I've

said, "Well, I

I

me why

ask

won't be

they don't have

10 robots picking up socks in their bedrooms!"

A

AI

further spur to boastfulness about

much of

that reporters can explain

is

The

in words understandable to anyone.

it

and

final goals

A commore imme-

everyday successes of AI are close to our everyday concerns. puter that beats chess champions or diagnoses diseases has diate

impact than the discovery of another elementary particle or prog-

ress in gene- splicing technology. a temptation

is

hard to

been above making a

resist,

little

For

a reclusive scientist, the limelight

and some

more of

have probably not

scientists

their discoveries to get front-page

coverage. Finally, fast-evolving fields like

AI

are

more

subject than others to a

perennial problem in technological forecasting. In

all

domains, research-

ers invariably overestimate the short-term potential

the speed of

its

One

progress.

result

is

more vulnerable than other high-technology grams

that, like aerospace,

of

their

cost overruns, to

work and

which AI

is

projects. Contrary to pro-

AI

involve national prestige or security,

cannot overcome errors of judgment through sheer budgetary excesses.

Often AI researchers helplessly watch a reputation for not delivering

their finances

Let's not forget, though, that the other

technological forecasting

onlookers

at Kitty

Hawk

is

most common mistake

to underestimate long-term achievements.

never imagined today's

airliners.

never thought of Nagasaki. Believing today in the

be

like deciding, after the

run dry, and gain

what they promise.

Vanguard

in

The

Marie Curie

of AI would

failure

flops in the 1950s, that space travel

was impossible. I shall, in

forecast.

the a

first

probing the past of AI research,

Having recounted the

four chapters,

I shall

origins

I

try to arrive at a better

early

golden years of AI in

demonstrate in chapters 5 to 8

few amazing successes, AI has not so

promised.

and

shall investigate the

far delivered

that, despite

what the pioneers

reasons for this state of

affairs

and

examine more recent developments to see whether there are reasons to believe that early promises might be fulfilled. I shall first

establish the conviction

To answer

this question,

of most modern philosophers that

our minds are essentially a product of the complex physical processes

.

8 in

Al

our brains.

Do

I

shall

then ask,

How

is

thought generated

our machines accomplish anything similar to

gree? This will be the subject of chapters 9 to

that,

in the brain?

and to what de-

1 1

Recent research and the trends of the past decades indicate that

machines

just as clever as

too-distant future.

Such

a

humanity's self-esteem, to and, indeed, to

its

concluding chapter.

human

beings

development its

meaning

very survival.

I

may indeed emerge will raise

in the overall

shall

in a not-

deep challenges to

scheme of

things,

examine these questions

in a

1

ENGINEERING

INTELLIGENCE: COMPUTERS

AND PROGRAMMING

I .

about

believe that in .

to

.

make them play

fifty

years' time

the imitation

it

game

will be possible to

so well that

programme computers

an average

interrogator will not

have more than 70 per cent chance of making the right identification after five minutes

— Alan

of questioning.

A

definition

of

artificial intelligence

this art is that

accepted by

of MIT's Marvin Minsky: "AI

is

many

Turing, 1950

practitioners

of

the science of making

machines do things that would require intelligence

if

done by men."

1

The machines involved are usually digital computers, and they can be "made" to do things by programming them in certain ways. "Computers" and "programming" will make up the two themes of this chapter. I shall first review how computers came about, and provide an overview of

how

they work. Next,

programming,

early philosophers

embodiment

I

shall

logic, calculation,

in

examine the relationships between

and thought, and

how

the inquiries of

and logicians into the nature of thought found

contemporary computer programs.

their

10

Al

COMPUTERS Early Devices It

can be argued that computing devices emerged

much

the

same reason

have only ten fingers and ten

and

larger

in

our century for

we

that the abacus did centuries earlier: because

and computations involving

toes,

numbers required devices

larger

that can handle these greater

sums

with better accuracy and speed. Stone-age calendars are the external

mechanisms

evidence of

first

on

this desire to rely

to alleviate mental burdens. Rather than tediously

counting the days to crop-planting time, our prehistoric ancestors used for alarm clocks the coincidence

of

celestial

bodies with lines of sight

defined by carefully positioned ground markers. In the Orient, the

abacus helped out in numerical calculations even before the invention

of long-hand arithmetic.

With technology showing

its

metallic head, the lack of extensive

number-crunching power became a serious

on

tion tables sent ships crashing

imprecise calculations, tumbled down.

manual

liability.

coastlines;

Error-filled naviga-

and bridges,

To weed

calculations, several inventors, including Leibnitz

tried their

hands

at building

and Pascal, 2

mechanical calculators. They met with

limited success, partly because of the cost of their complex, devices,

and

partly because

of their specialized natures:

could perform only one basic arithmetic operation a sequence

upon

built

out mistakes from

hand-made

early calculators

at a time.

Performing

of operations involving several numbers involved many

lengthy manipulations by the users. In other words, one could not

program these machines to perform several operations.

Oddly enough, with numbers.

the

first truly

programmable device had nothing

The Frenchman Joseph-Marie Jacquard invented

1805 to drive looms: removable punched cards

weave

different patterns.

Some

let

the

do

to it

in

same machine

forty years later, the British inventor

Charles Babbage picked up the idea of punched cards to feed instructions to his ill-fated "analytical engine." This steam-driven contraption

would have contained, had

it

ever been built to specifications,

elements of a modern computer, including a unit

(I'll

define these

words

memory and

all

the

processing

shortly).

Babbage teamed up with Augusta Ada, countess of Lovelace and

ENGINEERING INTELLIGENCE: COMPUTERS AND PROGRAMMING daughter of the poet Lord Byron. She

is

computer programming, the science of tions to

perform on what pieces of

11

often credited with inventing

computer what opera-

telling a

data. Unfortunately,

Ada never

had a chance to run her programs because Babbage's grand ideas crashed against the same technological limit that had put a cap on the progress of automata a century

earlier.

Nineteenth-century mechanics

couldn't produce sufficiently accurate parts, and the analytical en-

still

gine was never completed. Babbage and Lovelace later tried to recoup their losses

by inventing

a chess-playing

machine and

machine. They even devised a system for winning

which sions. tary,

in fact forced the countess to

Nowadays

the

perpetuates the

By

1

memory of

on two occa-

they nevertheless

made

filled a

mili-

the countess of Lovelace.

890, hand-driven mechanical calculators

one, would have cost

at the race track,

jewels

computer language Ada, favored by the U.S.

Babbage's visionary device sive,

pawn her

a ticktacktoe

much more modest than

their appearance.

crying need.

more than $100,000 in

Clumsy and expen-

The Ohdner calculator, today's money and took

for

ten

minutes to perform a multiplication! In the same year, the American

Herman

a tabulating maon punched cards. Hollerith's Tabulating Machine Company eventually merged into a conglomerate that became IBM. Such machines were called "digital calculators," because they represented numbers by cogged wheels or related devices, which could take only a fixed number of positions corresponding to the digits in the numbers they denoted. For example, if a wheel had ten possible positions, it could represent the digits from to 9. Even then, though, the cheapest and most efficient way to speed up a calculation

Hollerith invented for the U.S.

chine that processed census data fed to

government

it

remained the abacus. For technical work, the alternative

when

it

didn't matter

slide rule

offered an

whether the answer was off by

a

few

percentage points.

Claude Shannon and His Switches In the cal.

first

third

of this century, calculators remained

essentially

mechani-

In 1 93 1 Vannevar Bush at MIT brought to its pinnacle the technology ,

of computing through mechanical devices. His

much more

than add or multiply numbers:

it

differential analyzer did

actually solved differential

equations, using rotating shafts as integrating devices. (Bush's machine

was an

analog calculator: the angular position

of a shaft could take any

12

Al

value,

and the machine's accuracy was limited only by how precisely one

could manufacture the shafts and measure their positions. By contrast, the accuracy of a

how many

depends on

digital calculator

[cogged

digits

wheels] are used to represent numbers.)

Bush hired

In 1938,

Shannon, to run the

a

twenty-two-year-old research assistant, Claude 3

differential analyzer.

Even though

the heart of the

device was mechanical and analogical, a complicated digital circuit using

electromechanical relays controlled

Couldn't one make the circuit

it.

This fact set Shannon thinking.

complicated than

less

it

was?

And how

about using the relays themselves for computing instead of the spinning disks?

These considerations

Shannon

led

to

show

that

one could

build,

using only interconnected switches that could turn each other on or a digital calculating

off,

machine performing any imaginable operation on

numbers. Further, Shannon's theory showed how, by examining the nature of the operations to perform, one could arrive at a configuration

of switches that embodied of operating much digital

The

this operation.

faster than the

switches had the potential

cogged wheels previously used

in

machines. Since, however, they could only take two positions (on

or off), only two values, taken to be

and

1,

were

representing numbers in the machines. This

available for the digits

why computers

is

started

using binary arithmetic.

And

so

it

was

that in the late 1930s, electromechanical relay switches

started replacing gears

and cogs

World War, Howard Aiken

at

in calculators. Just before the

Harvard

human

being.

soon replaced electromechanical

relays.

calculate twelve times faster than a transistors

driven switches embodied no faster than relays,

moving

tubes and containing no

World War.

It

parts

moving

Vacuum These

could

tubes and

electronically

and could operate much

which were limited by how

could change positions. Thus the

Second

built a calculator that

fast their iron kernels

electronic

computer, based on vacuum

parts,

appeared during the Second

was invented not once, but three

times: in

Germany,

airplane design; in the United States, for calculating artillery tables; in

England, for breaking

German

secret codes.

for

and

The American machine much power as a

contained 18,000 tubes, weighed 30 tons, used as locomotive, and would have

filled a

tions per minute, the Electronic

tor

(ENIAC) was

competition.

a

ballroom. But at 20,000 multiplica-

Numerical Integrator

thousand times

faster

than

its

And

Calcula-

relay-operated

13

ENGINEERING INTELLIGENCE: COMPUTERS AND PROGRAMMING

Von Neumann and

His Architecture

ENIAC

to uses other than war, a

When

put

scientists

major design

To change the sequence of operations per(what we now call the "program"), engineers had the data on formed to rewire hundreds of connections in the machine. John von Neumann

weakness became

is

clear.

way to

usually credited for pointing the

a better

computer architecture

in 1945.

Names can

mislead: as

German. 4 Born

Chopin wasn't French, von Neumann wasn't

Budapest

in

family

moved

mann

could rightly count

game

in 1903, this scion

of an upper-class Jewish

to Princeton in the 1930s. In the mid-1 940s,

among

von Neu-

his contributions the invention

of

theory; the theory of automata (which discusses the possibility that

machines might be able to reproduce themselves); and, in the

field

of

hydrodynamics, calculations of shock-wave propagation, which were used during the Manhattan Project to help trigger and control the chain reaction of a nuclear explosion in

its

early phases.

He was

also the author

of a celebrated essay on the mathematics of quantum theory, which was said to

have inspired Alan Turing

(whom

I shall

come a mathematician. 5 Remembering the difficulty he had had

discuss shortly) to be-

earlier

using a mechanical

desk machine to calculate shock-wave propagation, von Neumann, hearing of the platform,

ENIAC

became

what

was

one could to

do

store the sequence

in the

same

also the first

of instructions

one to use the term memory for

computer: Babbage had called

He

telling the

used to hold the data. (Von

circuitry

it

a train

answer to the problem of changing the

computer's instructions appears in retrospect very simple. that

on

project in a casual conversation

fascinated. His

the "store.")

The

realized

machine

Neumann

this part

so-called

of the

von Neu-

mann computer the Second

The

architecture, embodied in virtually all computers World War, breaks a computer into two parts.

since

(CPU) operates on the data items

to be

central processing unit

manipulated. These data items (numbers or symbols) are stored in the part of the computer. The memvon Neumann machine operates in well-defined cycles: Fetch the first instruction from memory. Fetch the data item to operate upon from another part of memory. Perform the operation. Fetch the next instruction from memory, and so on.

memory, which makes up the second ory also contains the program.

A

14

Al

The

Electronic Discrete Variable

Computer (EDVAC)

first

embod-

RAND

Corporation's

JOHNNIAC, so named in honor of John von Neumann.

Following von

ied this architecture.

Neumann's penchant

It

was followed by the

for puns, the creators

of another machine couldn't

new machine Mathematical Analyzer, Numerical Inte(MANIAC). von Neumann's colleagues did not enjoy his Old World charm

resist calling their

grator

And

Alas,

Calculator

and

jokes, or

war.

He

endure

his impossible driving habits, for

long after the

died in 1957 of cancer, perhaps induced by radiation exposure

during the Manhattan Project.

COGNITION AS COMPUTATION ror

several years following their invention,

computers were generally

perceived as devices for manipulating numbers and straightforward items of data such as names in a telephone directory. However,

became

clear to

some of

switch positions inside the machines could take particular, they

it

soon

the computers' inventors and users that the

on other meanings. In

could stand for symbols representing concepts more

abstract than straightforward data. If the

these symbols as specified in

its

computer then manipulated

program, perhaps then

it

could be said

to "think." This concept of cognition as computation had been the subject

of much debate throughout the history of philosophy and mathematics.

Could one represent

Could thought

result

all

things under the sun through a set of symbols?

from the manipulation of these symbols accord-

ing to a set of predefined rules?

symbols be? As we

AI

shall see,

And

if so,

what should the

rules

and

such questions found their echoes in early

efforts.

Early Attempts to Formalize Thought The

thirteenth-century Spanish missionary, philosopher, and theologian

Ramon

Lull

artificially

is

often credited with making the

first

systematic effort at

generating ideas by mechanical means. Lull's method, crude by

today's standards, simply consisted in

randomly combining concepts

through an instrument called a "Zairja," which the missionary had

brought back from his travels in the Orient. A Zairja consisted of a circular

15

ENGINEERING INTELLIGENCE: COMPUTERS AND PROGRAMMING

with concentric disks on which appeared letters and philosophi-

slide rule cal

symbols. 6

The combinations obtained by spinning the

to provide metaphysical insights. Rechristening it without

the

Ars Magna (Great

disks

were

said

undue modesty

Art), Lull generalized the Zairja to a host

of other

beyond metaphysics and turned the instrument into the Middle

fields

Ages equivalent of a computer for blending books dealing with various applications of

wrote dozens of

ideas. Lull

Great Art, ranging from

his

morals to medicine and astrology. For every subject the method was the same: identify basic concepts; then combine them mechanically with

themselves or ideas pertaining to a related

field.

Yet merely generating random combinations was only a small step toward mechanizing thought: to interpret

one

also required systematic

first

means

and evaluate the combinations. In the seventeenth century,

the diplomat, mathematician,

and philosopher Gottfried- Wilhelm Leib-

nitz suggested the possibility

of a

lus, to

calculus ratiocinator,

or reasoning calcu-

achieve this goal. Apparently following Lull's lead, 7 Leibnitz

number 8 and

hoped

to assign to every concept a

issues

by formally manipulating these numbers. The diplomat

nitz

foresaw such an instrument as a

common

to resolve the thorniest

language

among

in Leib-

nations.

Leibnitz never achieved his objective of completely formalizing

thought and in time became keenly aware of the taking.

of

all

One major

concepts: "There

no

contains

difficulty

stumbling block, he noted, lay in the

relations

other things or even

is

no term so absolute or detached

and of which

all

a perfect analysis

other things." 9 Three centuries

researchers, trying to carve

of the under-

interconnectedness

up

reality into

"micro worlds," would also founder on

this

that

it

does not lead to later,

modern AI

convenient niches called very issue.

Boole and the "Laws of Thought" The

first

recognizable glimmerings of the logic that would later be

implemented into computers emerged from the work of

a self-taught

Englishman, the son of a shoemaker, named George Boole. Boole eventually century.

became one of the most

To

influential thinkers

of the nineteenth

help support his parents, he became an elementary school-

work occurred before and after the room he plowed through advanced mono-

teacher at age sixteen. But his real

school day

when

alone in his

graphs in mathematics, learning them with the same thoroughness that

had

earlier

marked

his

mastery of Greek and Latin.

A few years later, he

16

Al

was publishing

mathematical journals. By the time he was thirty-four,

in

even though he did not have

a university degree,

Boole was appointed

professor of mathematics of the newly founded Queen's College at Cork in Ireland.

There he attempted nothing

than a mathematical formu-

less

He started by investihow one could combine classes and subclasses of objects, and then how such classes intersected with other classes. Boole showed how one could draw useful conclusions from such analysis. He assigned of the fundamental processes of reasoning.

lation

gating

symbols to the operations of combining either (which he

elements of two sets

all

named "union"), elements belonging

to

both

sets ("intersec-

tion"), or objects falling outside a given set ("complement"). In this

way, operations on sets could be represented in a crisp shorthand. For

example,

A u B

meant "the union of

sets

A

and B." Using these

symbols, Boole could analyse and simplify complicated operations volving

many

sets,

much

as his fellow

in-

mathematicians could manipulate

ordinary algebraic equations. Boole formulated simple and well-defined

laws to perform these simplifications (see figure

1.1).

854 book, The Laws of Thought, Boole stated that these principles were fundamental descriptions of thought. In the tide of his celebrated

He was

pardy

right.

oranges, one should

or

of

class)

not

all.

After

know

fruits. It also

Further, not

some form of

sugar

all

all,

that

1

to talk intelligendy about apples

both belong to the wider

helps to

know

that

some

set (or category

apples are red, but

red fruits are apples, although they

when

ripe,

and

and so on. For the

all

time,

first

contain

Boolean

algebra enabled a rigorous and quasimechanical manipulation of categories,

an

activity basic to

human

thinking.

Boole could claim universality for Replace the concept of true or false.

That

is,

sets

by

his laws in yet

another way. 10

logical propositions that

instead of the set of

sentence "Boole was born in 1815," which

is

all

can be either

apples, consider the

true. Further, replace the

operators union, intersection, and complement by the logical operators

OR, AND, and NOT. These can combine

logical propositions to

form

other propositions. For example, the combined proposition "Boole was

born

in

1815

AND

he died

in

1816"

propositions "Boole was born in 1815

was born

in

1815

is false.

On

the other hand, the

OR he died in

AND he did NOT die in

1816" and "Boole

1816" are both

true. It turns

out that these operators and propositions combine together in a manner exactly analogous to the set-theoretic operators union, intersection,

complement. Thus, assumed Boole,

if

the

and

mind works according

to

A 17

ENGINEERING INTELLIGENCE: COMPUTERS AND PROGRAMMING

FIGURE The Here

1.1

Postulates for Boole's

Laws of Thought

are Boole's laws as they apply to logical propositions

case, the objects studied, basic operations,

and

set theory. In either

and identity elements are

as follows:

Logic

Sets

logical propositions

Objects Studied

collections of objects

b)

(a,

(A,B)

AND:

Operations

O:

equivalent to the

intersection

preposition "and"

Elements

Identity

In these two fields of study,

it

OR: "or"

U: union

NOT:

~"

"not"

complement

:

1:

true

I:

0:

false

(J):

universal set

empty

set

can be shown by inspection that the following basic

facts, called postulates, are true.

Operations are

a

commutative.

a

There are identity elements

a

for the

two operations.

Each operation

AND b = b AND OR b = b OR a OR AND

a

distributes

a

over the other.

(a

= 1

AOB

a

A

a

=

OR (b AND c) = OR b) AND (a OR

AND (b OR c) AND b) OR (a AND c)

=

a

a

AND (NOT

=

a

OR (NOT

From

these basic facts, and others that follow

logical expressions, or sentences

about

manipulate algebraic expressions.

One

a)

=

a)

sets, in

U (B n Q = U B) n (A U Q a n (B u Q = (a n b) u (a n Q

could do

which elements

and multiplications,

is

+

AU""A

much

it is

is, it is

=

(|)

I

possible to manipulate

same way as one would more easily, in fact, because

the

this a little

sets.

More

specifically,

numbers and the operations

not a Boolean algebra because addition

over multiplication. That {a

are the real

""A =

API

1

ordinary algebra offers less freedom than does logic or algebra, in

A

(A

from them, very

A c)

(a

complement.

=

H UA

Am = a

a

a

Each element has

= B B = B

U U

A

not generally true that a

+

(b

not distributive

is

x

ordinary

are additions

c)

=

{a

+

b)

x

,).

these laws,

it

performs

logical operations in the

same way

it

manipulates

sets (see figure 1.1).

In

fact,

Boole had

laid the

foundation for analyzing thought in more

ways than even he had foreseen. Ninety years

after their publication,

18

Al

Boole's ideas supplied the basis for Claude Shannon's analysis of switchI have described, makes up the modern computers. Shannon's intuitive

ing circuits, which, as

theoretical founda-

tion for

leap

all

was

to realize

that switches resembled logical propositions in that they could take only

two true

and

positions, open

If

closed.

one took these positions

to stand for

and false, one could then analyze combinations of switches with the

same mathematical machinery illustrate this point,

sitions

and

their

I

Boole had used for propositions.

that

To

have drawn up examples of simple Boolean propo-

embodiments

in switching circuits in figure 1.2.

Reasoning Calculuses, or the Fundamental Impotence of Logic Yet Boole's laws did not shape up to

a

complete calculus for reasoning.

"Laws of

Essential elements were missing. Boolean algebra, as his

Thought"

are

now known,

could not serve as a complete generic tool

for expressing logical sentences because of

you assign true or

false values to basic

its

lack of flexibility

7

propositions such as "I

It lets

.

own my

house" or "Mary owns her house," but cannot express statements such as

"Every house has an owner." Boole's formalism prevents the creation

and manipulation of statements about general or Further, each Boolean proposition totally

beyond

ability to

reach.

A

false logical propositions,

sentences that could be true or

Then

came up with such

and combine these into

system in 1879. 11

mathematics

at the

on Boole's system by introducing

the

contains arguments that are not logical variables: the predicate

true ifj really

The

insides

A predicate is a logical entity with a true-false value.

OWNS(x,j,) could mean

false.

its

require the

The German mathematician Gott-

thirty-one, Frege, an assistant professor of

concept of predicates. it

false.

a

University of Jena, improved

But

an unbreakable atom,

more powerful formalism would

define basic elements (such as house and owner) that are not

themselves true or

lob Frege

is

indefinite objects.

owns

x,

that person j

x and j,

but

owns house

x.

OWNS has value

in themselves, are neither true

nor

A further refinement of Frege's system introduced two quantifiers. universal quantifier

logical proposition

is

V

x,

which means "For

true for

quantifier 3 j ("There exists

all

all

x," denotes that a

values of variable x.

aj such

that")

means

The

that at least

existential

one value

FIGURE

1.2

Three Switching Networks and Their Corresponding Boolean Propositions

Switch Switch

k

a:

If

you

Switch

are driving

and there

is

Current: then you

Switch

a stop sign

must stop

If

a:

you

and there

b:

Switch

c:

are driving is

a traffic light

or a stop sign

Current: then you must stop

W

(*)

Switch Switch

The two

d:

b:

If the light

you see

is

then the cross-street light

green

is

not green

switches are connected by the Boolean operator

as a string:

when

b

is

closed,

d

is

NOT,

here portrayed

forced open and vice versa. In computers,

electronic connections replace strings.

20

||

of j

which the proposition

exists for

that follows

is

true.

x 3 y OWNSfojj ("For all x there exists OWNSfoj/*) means "All houses have an owner."* sentence V

So powerful was mathematics,

this idea that for the first

became

it

a

y

Thus, the such that

time in the history of

possible to prove general theorems by simply

applying typographical rules to sets of predefined symbols.

was

It

still

necessary to think in order to decide which rules to apply, but the written

proof required no intermediate reasoning expressed

in natural language.

In that sense, Frege had finally realized a true reasoning calculus.

But

in

language.

another sense, Frege had

still

The word predicate stems from

"to proclaim." Thus, a predicate

proclaiming to be what

its

is

not

fully freed

reason from

the Latin predicare, which

means

nothing but a mnemonic label

user defined, through the use of language,

beforehand. Hence, the meaning of an argument in formal logic entirely in the

mind of

lies

the beholder. Because of this subjective bias,

Frege emphasized that his reasoning calculus worked well only in very restricted

domains. 12 Even so,

many

early efforts at

programming com-

puters for general-purpose intelligent behavior relied in great measure

on

this calculus, steering

begun

to

emerge only

Some of

AI

from which

into an equivocal path

it

has

recently.

the problems inherent to reasoning calculuses were

first

noted by the British mathematician Bertrand Russell. These observations, while disconcerting to those

hoping to

reason-

finally create a true

ing calculus, provided unexpected insights into the nonlogical nature of

thought and consciousness. In June 1902, Russell wrote to Frege to disclose his discovery of a contradiction in the metic.

The

gist

latter's

theory about the fundamental laws of arith-

of Russell's argument was

with two central catalogs: catalog selves,

and catalog B

lists all

assume that the books central catalogs

question

is,

On

A

A

books

that

do

books

Consider

a library

that refer to

them-

not. (For simplicity, let us

in the library are also catalogs, thus

and B catalogs of

which

as follows.

lists all

catalogs, or sets

central catalog should

we

list

of

making

sets.)

The

B? Either choice

*Actually, Frege used a different notation for the universal and existential quantifiers.

notations V and 3, as well as many other mathematical symbols in use today, were introduced by the Italian mathematician Giuseppe Peano in the late nineteenth century, and later finalized by Bertrand Russell and Alfred North Whitehead in Prinapia Math-

The

ematica (1910).

21

ENGINEERING INTELLIGENCE: COMPUTERS AND PROGRAMMING sounds wrong. to

A

We

we

but

itself,

few years

cannot

also

later,

list

cannot

B

list

in catalog

B

overcame the problems

raised

B does

A, because

because then

Russell and his associate Alfred

wrote a three-volume work entitled thors

B

in

it

North Whitehead

Principia Mathematica.

by

classes

not refer

refers to itself.*

In

it,

the au-

of classes through what

they called the "theory of types." Individuals, sets (or classes), and

of classes belong to different

classes

belong to a

belong to a

class,

class

and

logical types.

of

a class to a class

of objects and, in

particular,

classes,

individual can

but a class cannot

cannot belong to

in effect outlawing self-reference in their reasoning

and Whitehead managed to avoid the

An

logical traps

about

By

itself.

sets, Russell

on which

Frege's

work had foundered. The prospects for completely formalizing mathematics appeared excellent, until in 1931 a paper by an unknown twenty- five-year-old Austrian

mathematician brought

optimism to an end. Entitled

this

"On

Formally Undecidable Propositions in Principia Mathematica and Related Systems I," It

13

the article shattered Russell and Whitehead's system.

demonstrated that using their very axioms and notation, one could

state true

theorems that no amount of manipulation would ever prove.

Kurt Godel, the paper's author, went even further and claimed that every consistent logical system would suffer from a similar weakness.

proved

this result

up and acquire

Through

logical

about,

Godel

over, he

a level

a clever

tween

by coaxing the

logical

of meaning

formalism in

its

authors had never foreseen.

encoding scheme establishing a correspondence be-

symbols and the very numbers they were supposed to built logical sentences that referred to themselves.

showed

and consistent

He

Principia to stand

that

one can always encode, in any

logical system, a sentence that

sufficiently

talk

More-

powerful

means: "This sentence

cannot be proved using the system's formalism." The intriguing result

of such a construction

is

that the sentence has to be true!

To

see why,

remember that the logical system in question is assumed consistent, which means that it does not allow us to prove false statements. But first

suppose

now

that the sentence

proved," being false means that

is it

false.

Since

it

can, in fact, be proved. If

proved, our system would not be consistent, since

*The

original (and entirely equivalent) formulation

the set of

all

sets

which

are not

says "I cannot be

members of

it

of Russell's paradox

themselves.

Is

R

it

can be

would enable us

a

is

to

"Consider R,

member of

itself?"

22

41

prove a

false statement.

Thus, the sentence has to be

that

demonstration

really

its

beyond the

is

true,

which means

of our

capabilities

logical

system.

As

the philosopher

J.

R. Lucas pointed out in 1961, 14 an even

surprising fact about this result

of the sentence, but the

truth

is

human

that

logical

system cannot.

We

realize

by reflecting upon the meaning of the sentence and deducing consequences. As

I

pointed out

more

reasoning recognizes the

its

its

truth

obvious

the logical system cannot recog-

earlier,

nize the truth of the sentence since the symbols in the sentence have

meaning

for

no

it.

Alan Turing and His "Machine" The

British

mathematician Alan Turing came to conclusions similar to

Godel's, but in an entirely different manner. For the

approach brought together, into

first

time, Turing's

a reasoning calculus, the theoretical

investigations with the builders of automata's hands-on yearnings to create

life.

Born

in 1912,

strange mixture of blunt,

clumsy

man

Alan Turing remained throughout

his adult life a

boy genius and bemused professor. In appearance with

little

a

care for social graces, Turing disconcerted

auditors with his high, stammering voice and nervous crowing laugh.

He

way of making strange screeching sounds when lost in thought, his mind almost audibly churning away at concepts. He was also renowned for his absent-mindedness. His younger colleague Donald Michie recalls how Turing, fearing a German invasion during the Second World War, tried to provide against the confiscation of his bank ac-

had

a

count: converting his savings into silver bullion, he buried

woods of Buckinghamshire, only

to lose track

Turing had a knack for solving

at a

befuddle engineers for days. time

visit to

A

typical

glance problems that tended to

example occurred during a war-

when he won over AT&T Bell Laborafiguring out how many combinations a

the United States,

tory personnel by instantly

special voice-encoding device provided a Bell Lab's technician to arrive

Mathematicians, computer ing for

in the

it

of the spot forever. 15 But

at.

—

a result

it

had taken a week for

16

scientists,

and AI researchers revere Tur-

two major ideas he had: the Turing machine and the Turing

A Turing

machine

abstract device

is

not, in fact, a physical

mechanism;

rather,

test.

it is

which enjoys many of the properties of a modern

an

digital

23

ENGINEERING INTELLIGENCE: COMPUTERS AND PROGRAMMING

computer. "Imagine," said Turing in substance, "a reading and writing

head which processes a tape of divided into

which

on

the reading head could be in any states defined

the tape.

or a

1

.

how

the machine

For example,

state

number of

would

tape

would be

Like a typewriter,

lower- or uppercase letters depending

will print

in,

it is

These

The

infinite length."

squares, each containing a

little

on which mode

"states

of mind."

symbol

react to a given

number 73 might correspond

to the

statements:

change to

If the square contains a 0, to the left

on

If the square contains a 1,

square by a

0,

state 32,

and move one square

the tape.

change to

state 57, replace the

and move one square to the

1

in the

right.

number 32 and 57, in turn, might correspond to other statements similar to these. In modern parlance, we would call the sequence of states controlling the head a "program." The initial writing on the tape would be the "data" on which the program acts. Turing showed that these elementary steps could be used to write a States

program performing any sequence of well-defined operations. For example,

if

the l's and 0's

on

the tape represent binary numbers, one could

write programs to extract their square roots, to divide them, or to

combine them

in

any manner imaginable. The idea that any systematic

procedure for operating on numbers could be encoded as a sequence

of elementary, machinelike operations has since become

"Church-Turing

thesis."

known

as the

(The American mathematician Alonzo Church

independently reached conclusions similar to Turing's.) Turing even

demonstrated the existence of one machine that could mimic the operation of any other of his machines: this he called the "universal machine."

One

could aptly describe modern

digital

computers

as practical

embodi-

ments of universal Turing machines. Like Godel, Turing noted his abstract

(in his case, in

regard to the capabilities of

machines) that there exist certain kinds of calculations,

which sometimes appear

trivial

to

humans,

that

no Turing machine can more defeat

ever perform. Although this result could be viewed as one

means of dealing with the world, Turing himself did be reason enough to doubt the possibility of making

for pure logic as a

not believe

computers

it

to

think.

When computers had become a reality in

1950, Turing

24 discussed this question in a celebrated paper entided

"Computing Ma-

chinery and Intelligence":

[T]his [weakness Is this

of Turing machinesl gives us

feeling illusory? It

much importance

is

should be attached to

it.

of superiority. do not think too

a certain feeling

no doubt quite genuine, but

I

We too often give wrong answers

to questions ourselves to be justified in being very pleased at such evidence

on the parts of the machines. Further, our superiority can only on such an occasion in relation to the one machine over which we have scored our petty triumph. There would be no question of tnumphing of

fallibility

be

felt

simultaneously over

all

machines.' 7

Mathematical arguments, claimed Turing in the same paper, are no

He

help in deciding whether a machine can think.

real

argued that the

question could be settled only experimentally, and proposed the following test to this

effect.

you might put

to

Suppose

just as a

it

communicating through tell

a

computer could answer any question

human would.

a terminal with

In

fact,

two hidden

suppose you were

parties

by questioning them which was human and which was

Wouldn't you then have to grant the computer

and couldn't a

computer.

this evasive quality

we

call intelligence?

This procedure, which has the advantage of neady sidestepping the

thorny issue of defining intelligence, has become test."*

come

known

as the

"Turing

Turing firmly believed that thinking machines would one day

about.

He

would "play the

predicted in his paper that, by the year 2000, a machine imitation

game

so well that an average interrogator will

not have more than 70 percent chance of making the right identification after five

minutes of questioning." 18

such an early date; but as off

we

by more than twenty-five

Alan Turing

will

well have been in cracking

years of the war, while

people would

still

agree with

Turing may not have been

years.

down

probably go

about thinking machines. Yet in

may

Few

shall see later,

working

in history for his seminal ideas

his lifetime, his

German at

most important work

intelligence codes. In the early

Bletchley Park, a suburb of London,

Turing designed a machine called the "Bombe," which explored the *In fact, the procedure as Turing defined it was a little more elaborate: the computer was supposed to pretend it was a woman, the other party being a real woman trying to convince you of her identity. If the computer could fool you as often as a man would in

its

position, then

it

passed the

the simplified procedure

I

test.

Nowadays

have described.

the term Turing

test

usually refers to

ENGINEERING INTELLIGENCE: COMPUTERS AND PROGRAMMING possible combinations generated by the

Enigma. The

Bombe was

25

German encoding machine

a special-purpose calculating

machine based

on electromechanical relays. Turing's efforts eventually led to the development of Colossus, a machine many consider the first electronic computer. Colossus, which relied on vacuum tubes rather than on relays, laid German communications bare to British eavesdropping. The British could now direct transatlantic supply ships to steer away from German U-boats, which

made

possible the buildup leading to the

Normandy

landing.

Turing did not reap any reward from the society he had helped so generously during individualistic,

its

time of need. Atheist, homosexual, and fiercely

he did not

fit

into the conformist British society of the

day, nor even into the organizations in

which he worked. After the war,

unable to deal with politics and bureaucracy, Turing Physical Laboratory

where he had participated

chine called the Automatic

left

the National

in the design

Computing Engine,

in

many ways

of a mathe suc-

cessor to Colossus. Prosecuted for his homosexuality, he was convicted in cal

1953 of "gross indecency" and sentenced to a one-year pharmaceutitreatment tantamount to chemical castration.

Turing ended his

life

by eating an apple dipped

On

7 June 1954, Alan

in cyanide.

2 THE FIRST Al PROGRAM: DEFINING THE FIELD

Every aspect of learning or any otherfeature of intelligence can in principle be described that

a machine can be made

to

simulate

— Organizers of Symbol-processing AI

as

we know it

tual field in the years following the

to 1956. First,

Three

critical

AI had

intelligent

the

Dartmouth conference, 1956

today defined

this

processes could be emulated

new

digital

who

in

Al-computer work to

The

is,

neurons.

The

more

efficiendy

by the

did not develop a fascina-

machines either quickly

intelligence or stayed in the parallel field

to develop: that

efforts at replicat-

artificial

approach when they decided that

emerging computer technology. Those

mass had

intellec-

events had to occur during this period.

AI broke away from

tion with the

an

— from 1945

to be tied to the computer. Early investigations about

machines centered on feedback theory and

human thought

critical

itself as

Second World War

ing the workings of the brain in networks of

pioneers of

so precisely

it.

lost interest in artificial

of neural networks. Second,

enough people had

create an intellectual

a

to start to dabble

community

for such ideas.

kernel of this group was formed by Marvin Minsky, John McCarthy,

Allen Newell, Herbert Simon, and their students. Third and most important, these individuals

had to find each

other. This gathering process

THE FIRST

27

PROGRAM: DEFINING THE FIELD

AI

started with the

emergence of two independent, informal groups around

Boston and Pittsburgh, and culminated in the 1956 Dartmouth conference, where the first AI program was presented and discussed.

POSTWAR EFFORTS In the period immediately following the Second World War, the study

of

intelligent

machines blended

disciplines: artificial

cessing in digital computers.

human

The fundamental

brain, feedback systems,

and

this period,

it

fields that later

congealed into distinct

neural networks, control theory, and symbol pro-

and

belonged to the

digital

first

differences

between the

computers were not

clear in

generation of researchers to bring

out the disparities between them.

Norbert Wiener and Feedback Theory One of

the

Americans

first

common

to observe

points between the

mind and engineered devices was the MIT professor of engineering and mathematics Norbert Wiener. The embodiment of the distracted genius (pictorial caricatures of his cigar-toting rotund figure still hang proudly today on the walls of the main hallway of MIT's Building 10), 1

Wiener was MIT's

star

sometimes also the

institute's chief

died in 1964). difficult to

A

performer as all-round

intellectual gadfly,

embarrassment, for

and

thirty years (he

speaker of several languages, he was

known

to be

follow in any of them.

Prior to joining the mathematics department of MIT shortly after the

Second World War, Wiener received formed postdoctoral work

in

Ph.D. from Harvard and per-

his

England, where he managed to displease

such eminent mathematicians as Bertrand Russell and David Hilbert. 2 Seeing himself as too broad an a single field

intellect,

however, to confine himself to

of study, Wiener wandered in what he called the "frontier

areas" between disciplines. While roaming along the borders of engi-

neering and biology, Wiener created the science of cybernetics.

Feedback

is

a

well-known mechanism

in biology.

Warm-bodied

ani-

mals keep themselves within a certain temperature range through biological

feedback mechanisms; predatory animals adjust their move-

28

Al

ments

stat:

catching their prey through scent and visual feedback mech-

in

The feedback system we

anisms.

most

are

familiar with

is

by assessing the actual room temperature, comparing temperature, and then responding

on or

ditioner) either

it

to a desired

the heater (or air con-

whether the existing tempera-

value.

The word feedback

describes

the process returns (feeds back) the result of the control action

compensating mechanism. Cybernetics

(the temperature) to the

was

science of control) ics

— by turning

off according to

below or above the desired

ture lies

how

the thermo-

achieves a constant temperature in any enclosed environment

it

(the

mathematical theory of feedback. 3 Cybernet-

why feedback mechanisms,

explained

teracting

a

complex

especially in

in-

sometimes become unstable. This breakthrough

systems,

allowed Wiener, with

the

of others, to develop procedures

help

used during the Second World

War

to stabilize radar-controlled anti-

aircraft guns.

But more important to our feedback theory

is

story,

Wiener recognized

that central to

the idea of information. In essence, feedback

mech-

anisms are information-processing devices: they receive information

and then make gent behavior

a decision

is

based on

it.

Wiener speculated

that

all intelli-

the consequence of feedback mechanisms; perhaps by

definition, intelligence

is

the

outcome of receiving and processing

infor-

mation.

Nowadays

this

notion appears obvious. Yet

it

was then

a

major

departure from accepted ideas, notably from Sigmund Freud's theory

7

that the

mind

essentially manipulates biological energies,

repress only at the risk of seeing

This paradigm

become gence.

shift

away from energy

the underpinning of

It

them emerge again

all

in

which one can

harmful disguises.

to information processing

subsequent work in

would

artificial intelli-

deeply affected psychology as well, marking the beginning of

the fruitful but uneasy relationship this discipline

would maintain with

information sciences for the rest of the century.

Weiner himself, however, never

really

computers. "I could never get him to

talk

developed

commented Marvin Minsky, another seminal "and so

it is

a strong interest in

about computers very much," figure in early

AI

research,

not surprising that Wiener never made any other significant

contributions to AI." 4 Nonetheless, his cybernetic theory influenced

many

generations of early

AI

researchers.

THE FIRST

A

I

29

PROGRAM: DEFINING THE FIELD

Neural Networks: McCulloch, and Hebb Among

who

the researchers

incorporated cybernetics into their early

who

theories of intelligence were those brain's workings. ual

They planned

neurons with

who

Pitts,

tried to

model the

components. Contrary to

electrical

of the

detail

by simulating individ-

to accomplish this

later researchers,

concentrated on experimental simulations, early neural net workers

how

attempted mathematical analyses of

networks of such neurons

would behave. Warren McCulloch and Walter most

truly colorful figures to

figure

was Donald Hebb, who

work

later

were two of the

Pitts

in this field.

Another

influential

provided more theoretical insight by

considering biological neurons.

Born

into a family of lawyers, doctors, engineers,

Warren McCulloch was

and theologians, 5

destined for the ministry. In the

initially

fall

1917, he entered Haverford College and was soon, according to his

account, called in by the Quaker philosopher Rufus Jones, "

'What

thee going to be?'

is

thee going to do?'

question it,

and

a

I

would

man,

And

like to

that he

And

again

may know

a

his

a

number,

number?' lives.'

fifteen

and spent time

knew only ests,

as Bert.

in a

"When

is

He

that a

smiled and said, 'Friend,

that

Pitts.

The most

home

he ran away from

Pitts read a

book

that

had

office for

"Carnap was amused, because when he

said

what

it

he [emphasis added]

Russell."

meant was

that

just

been

name of

an explana-

something wasn't

was nonsense. So he

newly published book to where young it

he

not only read the book but, upon discovering

clear,

his

age of

man

Bert detected the boy's [mathematical] inter-

tion.

and sure enough,

salient

at the

Chicago park where he met an older

something he found "unclear," went to Carnap's

opened up

one

answer to the second part of that question

he suggested that young

Pitts

is

is

man may know

published by a professor at the University of Chicago by the

Rudolf Carnap."

asked:

"6

with the help of the mathematical prodigy Walter feature of Pitts's childhood

who

don't know.' 'And what

'I

don't know; but there

'I

What is

thee will be busy as long as thee

McCulloch formulated

said,

said,

I

answer:

I

of

own

Pitts

was nonsense. Bert turned out

was pointing,

to be Bertrand

7

In 1943, three years after this encounter,

McCulloch and

Pitts tried

30

Al

to explain the workings of the human brain by coming up with a mechanism by which networks of interconnected cells could perform 8 logical operations. They started out by asking themselves what could be

considered a "least psychic event," and realized that such a fundamental event could be the result of an all-or-nothing impulse by a nerve

Perhaps

it

was

of the single nerve

at the level

cell,

by

its

cell.

release or failure

humans make true/false decisions. The Pitts-McCulloch paper on neural networks relied heavily on the idea of feedback loops (which they called "circles") to reach some of their conclusions. They pointed out that the loop "senses, to brain, to to release an impulse, that

muscles" can result

between

difference state

purposive behavior

in

the muscles reduce the

if

by the senses and

a condition perceived

of the world. Likewise, they defined memory

ing in closed paths of neurons. Every

remembrance was, according

them, the reactivation of a trace of one such signal in fact that a reactivation

tion occurred time.

We

what

order.

of

remember

that conscious decisions about the truth

occur

at a level

much

and thorough mathematical

by transmitting or

cells.

analysis

how

laid the

a

mechanism

McCulloch and it

managed

could perform logical

like the brain

foundation of what

Pitts's

to present

of how interconnected ceDs,

failing to transmit impulses,

thus,

of

higher than that of the single

contribution was important, nonetheless, because

These ideas

to

The

does not tell when the original activawhy our memories are so indefinite as to but not always when they happened, and in

neuron, probably involving millions of brain

—

closed path.

a trace

events,

Nowadays we know

operations

its

might explain

logical propositions

a valid

a desired

as signals reverberat-

is

today

might compute.

known

as "artificial

neural net theory." Pitts

the

and McCulloch were

also able to

computing powers of both

draw

artificial

striking parallels

between

neural networks and Turing

machines. These comparisons, unfortunately, gave the

false

impression

work like digital computers. They don't, and it took many AI research away from the dead-end path inspired by this draw

that our brains

years to

misconception.

The mention of feedback loops in the papers both of Pitts and is no accident: they and their co-workers knew each other. When a small group of scientists grow interested in a

McCulloch and of Wiener

problem, they often form a club to chat about their favorite subject. In this case, the club

they formed in the late

1

940s was

named

the Teleo-

— THE FIRST

31

PROGRAM: DEFINING THE FIELD

AI

same time their interest in goal-oriented means "end" in Greek), McCulloch's taste for

logical Society, reflecting at the

behavior in nature

pompous

{telos

terminology, and Wiener's considerable ego (the club's

stemming from the

title

of

his seminal

name

paper published in 1943 9 ). The

group enjoyed the lively company of yet another first-rate intellectual John von Neumann, who had emigrated to Princeton before the war. A group like the Teleological Society was started in England. Its

members called themselves the Ratio Club because "they liked the [way 10 Alan the word ratio] combined reasoning, relations and numbers." Turing attended some of the meetings, but did not play a central role in the club's activities. Other members included the philosopher Donald Mackay; Turing's colleague Jack Good; the biologist John Pringle; Albert M. Uttley, whose fertile imagination provided the club's multireferencing name; and the neurosurgeon John Bates. Another member was the brain physiologist Grey Walter, inventor of the first cybernetic "turtle."

A

wheeled

illustration

dome-shaped device navigated fed from an electrical outlet

and improved on by tories into the

The

British

McCulloch

of the power of feedback loops, its

way around

when

its

batteries ran low.

later researchers, the turtle

mobile

field robot.

"visited the British, argued with

banter seems to have

Alan Turing was 12

Activity,"

"A

come out of

less

The reasons

speculate. In

Widely copied

evolved in other labora-

and American groups were aware of each other. Warren

they returned the favor"; 11 but nothing

tan."

this

Walter's laboratory and

impressed and considered McCulloch "a charla-

Logical Calculus for the Ideas

in

than discussion and

these interchanges.

for Turing's aversion are not clear, but

McCulloch and

were equivalent

and delighted them, and

much more

Pitts

had argued that

computing power

Immanent

in

their neural

to Turing machines. 13

one can

Nervous networks

As Seymour

Papert points out in his introduction to a collection of McCulloch's papers, 14 this claim like

numbers. the

was exaggerated: McCulloch-Pitts nets

Turing machines, but correspond to a narrower It

would have been

in character for

class

are indeed

of calculable

Turing to take offense

in

bevue.

In 1949, six years after McCulloch and Pitts had

shown how

neural

networks could compute, the McGill University physiologist Donald O.

Hebb

suggested

how they could learn. 15 He proposed the idea that brain we learn different tasks, and that specific new

connections change as

neural structures account for knowledge. Hebb's ingenious proposal

32

Al

dealt with the conductivity

rons.

He

of synapses, or connections between neu-

postulated that the repeated activation of one neuron by

another through a particular synapse increased

its

conductivity. This

change would make further activations more

likely

formation of tightly connected paths of neurons

in

and induce the

an otherwise loosely

connected structure.

DUO

THE CAMBRIDGE Although Wiener, McCulloch,

Pitts,

and Hebbs belong to the genera-

tion preceding the actual founders of AI, the latter

neural nets and cybernetics. Marvin Minsky and

two key

were educated

in

John McCarthy were

of the new generation.

figures

Marvin Minsky At

the

same time

as

Donald Hebb, an unusual Harvard undergraduate

named Marvin Minsky was independently coming,

in a

roundabout way,

to conclusions similar to his. Minsky's physicist friend says that "it

was "I

—

was not

or, perhaps,

entirely clear

what

it

Jeremy Bernstein what (Minsky's) major academic field

wasn't." 16

Minsky himself recalled elsewhere:

wandered around the university and walked into people's

laboratories

know anything about the social life of the undergraduates, but I knew when the department teas were, and I'd go and eat cookies and ask the scientists what they did. And and asked them what they

they'd

tell

me." 17 They more than told him: they gave him labs of

own, three of them! tal

work

A nominal physics

student,

him use

a

He

Minsky had attached

animal's claw, the crayfish picked

the claws.

when Minsky

When

hung around

grew

roomful of equipment, where he became an expert

influence of electrodes

it

also

and talked a zoology professor, John Welsh, into

the neurophysiology of crayfish (a small fresh-water lobster).

released

his

Minsky did experimen-

in physical optics in the physics department.

interested in neurology letting

did. I didn't

up

Under

in

the

to individual nerves of the

a pencil,

waved

it

around, and

excited the fibers that inhibited the closing of

he wasn't doing physics or dissecting

crayfish,

Minsky

the psychology laboratory, where he was able to sample a

cross-section of psychology as

it

existed in the late 1940s.

At one end

THE FIRST

of the lab was the behaviorist camp of B.

who

33

PROGRAM: DEFINING THE FIELD

Al

F.

Skinner and his followers,

then held sway over most psychological research in the United

States.

Born

1898 with the publication of the American educator and

in

psychologist

Edward Lee Thorndike's book Animal Intelligence, behavior-

human psychology of

ism was a brutal transposition to

Pavlovian ex-

periments conducted on animals in which one would, for example, flash a bright light at a cat

would

the animal

whenever one fed the animal. After

several days,

response to the flashing light even in the

salivate in

absence of food. Pavlov called the light "stimulus" and the salivation "response." For behaviorists,

all

ply reflexes triggered by a higher

actions, thoughts, or desires

were sim-

The only

difference

form of

stimulus.

between animals and humans was that humans were able to react to

more complex was

sets

of

stimuli, called "situations."

just a device for associating situations

point in examining

it,

as

Given

with responses, there

had been done by the

earlier

mind was no

that the

methodology

that

used introspection to study the mind. The AI pioneer Herbert Simon said later,

nal

—

word like 'mind' in a psychology jourmouth washed out with soap!" 18 For extremists like mind did not even exist. One could study the act of

"You

couldn't use a

you'd get your

B. F. Skinner,

memory

remembering, but to investigate discipline. Ironically,

when

in the 1940s

itself

transgressed scientific

and 1950s engineers

started

building machines that played checkers, proved mathematical theorems,

and

also contained a device they called a

"memory," the engineers

discussed the "minds" of their machines in as

wanted

to.

much

detail as they

Nevertheless, Minsky liked Skinner very much, and spent

some time helping the psychologist design equipment for his experiments. As I shall show, Skinner's ideas about reinforcement learning also later inspired Minsky to build a neural net machine. Minsky

didn't think

much of

the physiological psychologists at the

other end of the Harvard psychology lab. This group tried to understand little

parts of the nervous systems, such as the sensitivity of the ear,

without relating them to the

rest.

In the middle of the laboratory, however, were young assistant professors who came much closer to Minsky's own way of thinking. Among them was George Miller, who attempted to model the mind through

mathematics.

A

later, in 1956, Miller became famous with the on short-term memory. Entitled "The Magical article shed a critical light on our reasoning pro-

few years

publication of an essay

Number

Seven," this

34

Al

We

19

cesses.

suffer,

claimed Miller, from an

more than seven items of information limitation, Miller

emphasized the active

inability to

of the mind

role

of information: gone was the behaviorist model of

With

association mechanism.

found

"I

me

told

Yet

make

I

As soon

a learning machine?'

let his

grades

fall.

memorable undergraduate

as "

I

didn't allow for a thesis.

saw

that

started to think

I

side,

boost his average, he decided to write

thesis.

For

this

a

he had to switch over to the in the physics

department

That presented no problem: Minsky had

also

sampled the Harvard course curriculum and earned enough

liberally

a

Minsky

music composition courses on the

To

mathematics department, since regulations

math

to study learning.

department meetings and hopping between too many labora-

tories, in addition to taking

he had

as a processor

a purely passive

Minsky was facing more down-to-earth problems. Eating

in 1949, at

mind

works of Warren McCulloch and the

years later. "It had the

could

cookies

in

this thing called the Bulletin of Mathematical Biophysics"

great pioneers of the 1940's. ...

'How

Minsky decided

Miller,

keep

pointing out this

at a time. In

credits to qualify as a

Under the somewhat

proving that

major

in that field.

direction of the mathematician esoteric paper at

with three of

each

its

on

Andrew Gleason, he wrote

the subject of topology.

moment there is on

It

involved

the surface of the earth a square

four corners at the same temperature. Mathematicians

don't judge results by their practicality.

It

was the elegance of Minsky's

demonstration that impressed Gleason. "You are a mathematician," he said

on reading

the thesis, and urged

Minsky

to register for his

Ph.D.

at

the prestigious Princeton mathematics department.

Minsky followed

this advice

and

blissfully

discovered that Princeton

wouldn't cause him any problem with grades. "Once,

my

transcript,"

A's

— many of them

he

said.

"Instead of the usual grades,

in courses

I

mathematician or one wasn't, and

one

actually

it

work with George

Miller,

and

still

Pitts

at

that either

(the

mathe-

one was

a

how much mathemat-

in the study

through his undergradu-

somewhat under

the influence of

Minsky approached another graduate student

with an idea for putting theory into practice.

electronics.

took a look

the grades were

knew." 20

Skinner's behaviorism,

were not

felt

didn't matter

Heavily influenced by McCulloch and ate

I

had never taken. Lefschetz

matics department's director at Princeton)

ics

all

of the brain but

Dean Edmonds's interests new science of

in the relatively

"Instead of studying neural networks in the abstract,"

THE FIRST

Minsky thought did

George

35

PROGRAM: DEFINING THE FIELD

Al

to himself,

Miller,

who

"why not

Edmonds

build one?"

agreed, as

obtained a two-thousand-dollar grant from the

Office of Naval Research.

During the summer of 1951, Minsky and Edmonds returned to Harvard and assembled the

vacuum

first

neural net machine from three hundred

The

tubes and a surplus automatic pilot from a B-24 bomber.

assemblage (which they called the Snare) consisted of a network of forty artificial

neurons that simulated the brain of a

way

rat learning its

through a maze. Each neuron corresponded to a position in the maze and,

when

fired,

it

open

the choices

showed

that the "rat"

knew

itself to

be

at this

point

Other neurons connected to the activated one represented

in the maze.

of these neurons

to the "rat" (for example, to fired

the activated neuron:

go

left

or

right).

Which

depended on the strength of their connections

was the automatic

it

connections. Instead of

moving the

to

pilot's role to adjust these

elevators or ailerons of an airplane,

the automatic pilot turned the knobs that set the strengths of the

connections. When, by chance, the "rat" made a sequence of good moves and found its way out of the maze, the connections corresponding to these moves were strengthened. In this way, the "rat" gradually learned its way through the maze. Two thousand dollars went a long way in 1951, but certainly not as far as allowing custom-designed parts. Minsky and Edmonds had to make do with whatever surplus gear they could scrounge up, and the world's

first artificial

ing elegance.

To

neural net wasn't exactly the epitome of engineer-

As Minsky

recalled to

me:

you had to reward the neurons which had

train [the system],

recently.

So each neuron had

a timing circuit that

say, five

seconds after

The

that

went

a shaft

it

fired.

circuit

rolling in, with a chain drive

potentiometers. So

if

the neuron

had

stay

fired

on

for,

operated a magnetic clutch

to the potentiometer. If you turned

was

would

on

this big

going to

fired three

all

motor, then

forty

of these

seconds ago and you

switched on the motor, then the potentiometer would slowly turn for

two seconds. Since the procedure resembled the stimulus-reward techniques of

behaviorism, Minsky tried talking to Skinner about the machine. psychologist wasn't interested.

It

soon became

learning techniques were not leading

The

clear that Skinnerian

Minsky anywhere: they provided

36

||

no way

tor the

machine

to reason

about what

it

was doing, and thus

formulate a plan.

Returning to Princeton

at the

problems the subject of brains

much

end of the summer, Minsky made these

larger" than the

Harvard machine, Minsky told me. "[They

had] sensors that turned

on intermediate

feedback that could

it

let

ideas are just

sections and different kinds of

do some planning ahead. There was some network can control another. Some of these

how a now being rediscovered

discussion about

which "described

his doctoral dissertation,

as the connectionists start to think

about multimode networks."

who w ondered whether this work was really mathematics, von Neumann, also a member of Minsky's dissertation committee, replied, "If it isn't now it will be someday let's

To

a

doubting department head

r

—

encourage

it."

21

Minsky obtained

Ph.D.

his

neural network with

in 1954,

convinced that a sufficiendy large

enough memory loops

require thousands or millions of neurons. that large a network,

would

to actually reason

He knew

he couldn't build

and looked for other ways to get machines to

Dean Edmonds, who had never been impressed with the Harvard machine, went on to become a professor of physics at Boston think.

University.

Meanwhile, Andrew Gleason wanted Minsky back

at

On the

Harvard.

recommendations of von Neumann, Norbert Wiener, and Claude Shannon, Gleason had his former protege accepted as a junior fellow. As

such Minsky could proceed with his research in complete freedom for three years. His only obligation

was to dine with the other junior fellows

on Monday evenings. Since Minsky was then reconsidering his interest in

artificial

networks, he temporarily directed his mental energies into the optics and, in 1955, invented

and patented the

first

neural

field

of

"confocal scan-

ning microscope." 22 This device imaged interconnections of neurons in

much

greater detail than the Golgi staining process (see chapter

11). Strangely, the

instrument was ignored by microscope manufac-

turers until the late 1980s, but there are

thousand of them, priced Unfortunately for him,

at

this

now on

commercial success happened long

the patent expired. It

was around 1955,

as

the order of one

around SI 00,000 each, Minsky told me.

Minsky

recalled

it:

after

THE FIRST that

met

I

young man named Ray Solomonoff who was working on

a

an abstract theory of deductive inference. learning machine

decided in

this

.

.

.

built a piece

With

make

I

.

.

He had worked on

a

was so impressed

I

I

[this

of hardware and hoped

new] approach, you

what kind of inferences you wanted

would

.

was pretty formal.*

that

was much more productive than the neural net system,

which you

right thing.

37

PROGRAM: DEFINING THE FIELD

Al

a

to

tried to

it

would do the

make

theories of

make, and then asked,

machine do exactly that?"

It

was

"How

a different line

of

thought.

Following this encounter, it gradually dawned on Minsky that there was a difference

what

it

between understanding how the brain is

does.

As

starting to offer a

it

turned out,

way

digital

built

computers were

to explore this last path. It

and finding out just

about then

was becoming possible

for scientists to describe to a computer, in a symbolic way,

what they

thought the mind did, and have the machine behave in exacdy

this

manner.

John McCarthy John McCarthy 23 was born to a Lithuanian Jewish mother and an Irish Catholic father who took up Marxism and fought for it as a union leader. Opposition to conventional ideas ran deep in the McCarthy family: John was thrown out of Cal Tech for refusing to attend physical education classes. Later, the U.S.

Army

communist. Patrick also

dismissed his brother Patrick for being a

lost a post office job for refusing to sign a

loyalty oath. In 1945, though, after John's

didn't have any choice but to put

brush with Cal Tech, the army

up with

his

own communist

bias

because he had already been drafted. Fortunately for John McCarthy, the

war promptly ended; and

Tech.

He

thereafter,

as a veteran, John

was able

to return to Cal

obtained his bachelor degree in mathematics in 1948. Shortly

he moved to Princeton for graduate work. Having read about

von Neumann's research on

finite

automata, he began to explore their

possibilities as intelligent agents. After graduation,

he spent

a

summer

with Claude Shannon editing a collection of papers on the subject. 24 In *SolomonofFs theory,

later independently rediscovered by other researchers, today "algorithmic probability theory."

is

called

38 his

||

own

contribution to the volume, McCarthy discussed the possibility

of making

a

Turing machine behave intelligendy. This

artificial intelligence

two

first effort

at

McCarthv

did not turn out very well, and taught

lessons.

Turing machines did not provide the right medium for estab-

First,

how machines

of

lishing a theory

problem was one of

could imitate

human

behavior.

The

small changes in machine structure

sensitivity:

brought about enormous changes

and vice

in behavior,

versa. Intuitively

small changes in humanlike behavior required verv large changes in

machine structure

name "automata

to account for them. Second, the

studies" wasn't right for the kind of investigations

McCarthv had

in

mind: most of the papers he and Shannon received had nothing to do with the reproduction of

human

intelligence.

A

catchier appellation

was

required.

A summer on

ests

ers

digital

spent working

at

IBM

in

1955 focused McCarthy's

inter-

computers. Hands-on work convinced him that comput-

provided the tool for actually building

artificial intelligences,

Turing machines and automata theory onlv allowed him to studv

while intelli-

gence in the abstract. Since then, McCarthy has never stopped looking for ways to

mind

into computers. Fellow researchers

in the history

His

of AI and agree that McCarthy's

ability to dive into the

tion

is

legendary.

tends to

now

inflict

grant

own mind

intellectual standards,

is

apparendy

say.

his dissatisfaction

isn't a

explaining

some of them he John

to

product

up when

Those standards account list

of publica-

hard for John to communicate well

"It's

The ideas are make some compromise in

graduate student in his areas.

there, but the willingness or the ability to

you're talking with

them

with other people's work. His former stu-

dent Hans Moravec told me: with anybody that

a

to shut

for McCarthy's successes, but also for his relatively short

and

also special.

the long silences he

is

which command him

he can't think of anything worthwhile to

tions

is

his partners in a conversation, leaving

wonder how they may have offended him. This of his high

embody

a special place

depths of a problem through sheer concentra-

Another McCarthy trademark

upon

him

it

thinks should be obvious are missing. If takes

skill

to learn to

communicate with

him" Yet McCarthv

is

far

from being a

loner. In fact, his social

heavilv influences his opinions and

way of

inconvenience in the early 1950s,

when he

life.

environment

That could have

led to

followed his parents and

THE FIRST

Al

39

PROGRAM: DEFINING THE FIELD

brother into communist militancy. Fortunately, his relative obscurity as an assistant professor protected

him from

McCarthy

the wrath of another

then witch-hunting from Washington. In the 1960s, John McCarthy grew

donned

his hair,

a

headband, and joined the counterculture movement.

socially conscious,

Still

he became one of the

first

crusaders against the

misuse of information possible through government and corporate com-

became

puter data banks. Years before the Privacy Act of 1974

McCarthy's proposal, which he called the

"Bill

September 1966 issue of Scientific American. With the 1970s,

McCarthy took

his

law,

of Rights," appeared in the his

second wife, Vera,

cue from the "me" generation:

his

in

"own

thing" was to court danger through parachute jumping and alpinism.

Unfortunately, this nist in the reach

way of life was more dangerous than being a commu-

of Joseph McCarthy: Vera died in

a climbing accident

during an all-women ascent of Annapurna. In the 1980s, John McCarthy

donned his

three-piece suits and voiced conservative opinions.

He

decided

1966 proposals for computer privacy were a mistake, since merely

possessing information causes no harm.

should concern

About

abouts.

itself

The

law,

McCarthy now

with the usage of information, not with

research financing, McCarthy's philosophy

opposite of equality: he summarizes his position as higher," better.

its

thinks,

where-

is

the direct

"Make

the peaks

meaning that one should make the best research

institutions

still

And the former communist opposed Edward Fredkin's idea of an

international

AI

laboratory: his reason: the Soviet

unfair advantage of it.

During the 1955-56 academic fellow,

and McCarthy taught

Hampshire. The idea of

"They had

gleam in

a

Union would

take an

26

at

year,

Marvin Minsky was

a

Harvard

nearby Dartmouth College, in

intelligent

New

machines fascinated them both:

their eye!" recalls their colleague

Herbert Simon. 27

They were becoming aware of the work of other researchers in the field and wished to bring them together. For this, they enlisted the help of two senior

One was first

scientists

who were

also interested in the subject.

Nathaniel Rochester of IBM, designer of the

IBM

general-purpose, mass-produced electronic computer.

had met him

in

connection with IBM's

the help of three colleagues keepsie,

New York,

from the

gift

701, the

McCarthy

of a computer to MIT. With

IBM research laboratory in Pough-

Rochester was then simulating neural networks on

IBM's new 704 computer. 28 More

precisely,

he was programming the

machine to solve the numerical equations describing

a large-scale neural

network. This use of the computer was quite different from the symbol

40

Al

processing, "top-down" approach in which Minsky and McCarthy were Intrigued, Rochester

interested.

hoped

approach might help computers exhibit

which was

main

his

The other

Minsky and McCarthy's

that

originality in

problem

solving,

interest.

was Claude Shannon. Both Minsky and him during the summer of 1953 at Bell Labs.

helpful elder

McCarthy had worked

for

Co-editing the book on automata theory with intelligence had furthered

Shannon's interest

And

so

was

it

younger man's work.

in the

of Rochester and Shannon,

that with the backing

McCarthy and Minsky persuaded the $7,500 cost of a

the Rockefeller Foundation to cover

summer workshop on

two-month meeting, held

in

thinking machines.

The

1956 on the Dartmouth campus under

McCarthy's auspices, brought together the few researchers then active in the field.

29

In addition to the four organizers

Rochester

showed Minsky

— of

up.

this

other

six

participants

One was Ray SolomonofT of MIT, who had

among would-be AI intellectual

a tendency that

was already becoming

researchers: that of giving the

problems to machines

intelligence to the world.

converted

of view. During the conference,

to the symbol-processing point

SolomonofT pleaded against and

— McCarthy, Minsky, Shannon, and

Dartmouth conference,

in

clear

most complicated

an effort to demonstrate their

SolomonofT proposed

that using easier prob-

lems would simplify the analysis of the mental processes involved. The point was well taken for yet another reason probably

SolomonofT nizing a

at the time: the

human

computer. The

of

problems that we find

face) are often the thorniest

ability to tackle intellectual

unknown

to

easiest (like recog-

ones to program into a

problems

is

a

poor

definition

intelligence.

Also present was Oliver Selfridge,

Wiener who had proofread the Cybernetics in 1948.

galleys

a

former assistant of Norbert

of the

In 1956, Selfridge was

at

first

MIT's Lincoln Laboratory

working on pattern recognition by machine.

programming computers translate his

edition of the latter's

He had

already started

to recognize letters of the alphabet

Morse code. 30 Shortly

and to

after the conference, Selfridge delivered

most important contribution

to the field

—

a forerunner

of expert

systems called Pandemonium. 31 Instead of a dignified sequence of state-

ments, Selfridge believed an AI program should look capital

a

of Hell: a screaming chorus of demons,

master decision-making demon. Each

all

like Milton's

yelling their wishes to

demon was

a short sequence

of

THE FIRST

Al

41

PROGRAM: DEFINING THE FIELD

statements looking for a particular configuration in the data (say, a vertical stroke in a printed character, or a specific

The master demon made

conjunction of symp-

its

decision by integrating

the lower-level decisions of each of the small

demons. For technical

toms

in a patient).

reasons, the idea did not catch

accounted for

Two

minor

much of

on

for fifteen years but,

participants at the conference

Arthur Samuel.

A

when

it

did,

the success of expert systems.

were Trenchard More and

graduate student at Princeton,

More was

writing a

on ways of proving theorems using a technique called natural deduction. Samuel, then at IBM, was investigating how computers could be made to learn by teaching them to play checkers. His efforts had a thesis

significant

impact

later.

THE FIRST Al two

1 he last

PROGRAM

participants in the

Dartmouth conference, Herbert Simon

and Allen Newell, had an immediate impact on AFs debut. Together, they had already written a computer program that, they claimed,

showed

computers could think.

that

Herbert Simon Herbert Simon was then explain his training,

In

1

the

he had started

948, he had

body

made

and

his

interest in

background did not

AL 32 A

his career researching

year,

which established

a

book

his reputation as

organizations.

Soon

by

municipal administrations.

a brief venture into civil service as a

he published

readily

political scientist

that administered the Marshall Plan after the

War. That same

human

thirty-six,

competence or even

member of

Second World

entitled Administrative Behavior,

an expert in the functioning of

thereafter,

Simon helped found Carnegie

Tech's Graduate School of Industrial Administration. (Carnegie Tech

now

is

Carnegie Mellon University.) In 1956, Simon was a professor of

industrial administration there.

Many would cracies

—

lies at

claim that Simon's specialty in those years

— bureau-

the exact opposite of intelligence. Yet his groundbreak-

him to guess at several common points in the workings of both, and more recent findings by others have confirmed his intuition.

ing

work

led

42

Al

He had

long been fascinated with

how

people make decisions; and his

conclusions, though contrary to conventional economic theory, offered

an open window into the workings of the

human mind.

Before Simon, economists believed (and to a certain extent

do) that

still

companies and even individuals making economic decisions behaved with perfect rationality and omniscience. Before reaching an investment or policy decision, a alternatives

The same kind of refrigerator.

company was assumed

to consider

and choose the one that brought about the

was expected of

care

a

all

possible

largest benefits.

consumer shopping

for a

These assumptions enormously simplified the mathematical

of economic systems because they assumed economic agents

analysis

were always trying to maximize or minimize some function, such

as profit

or cost. Since well-known mathematical tools existed for finding a function's trough or crest,

economic behavior then became

problem. But Simon pointed out, in

do not always work

these basic assumptions First,

no one looks

his theory

at all

an appliance, most people

of the

will

of Bounded

in practice.

shopping for

decide roughly what they are willing to

pay, look at a couple of models,

and pick the one that comes closest to

Companies do the same: year-end budgeting

their requirements.

why

33

When

alternatives.

a tractable

Rationality,

exer-

few simu-

cises typically exhaust the executives involved after only a

lation runs.

Simon

realized that alternatives

a golden plate, and that there

is

do not come

to a decision

maker on

a cost associated with finding

and

evaluating each possibility. For this reason, a decision process really consists in a search through a finite better.

number of

options, the fewer the

Rather than optimizing some function of

all

options, like the

associated profit, the bureaucrat in each of us picks the that

meets a pre-set acceptance

criterion.

Simon

first

choice

called this behavior

"satisficing."

The inability

to

overcome the cost of searching through many

tives limits us as decision will

confirm, this weakness does not necessarily

in

our minds. Simon

attributes

of a mental

both people and organizations have

difficulty

coming up

with original solutions to problems. In

shows

lie

more of the

discovered another weakness that has restriction:

alterna-

makers. As any sore-footed appliance shopper

itself in the

In recent years, a

organizations, this inaptitude

existence of the rule book, or

more

name of "corporate

management manual.

abstract manifestation of

culture."

it

has received the

Such observations led Simon to speculate

THE FIRST that the

43

PROGRAM: DEFINING THE FIELD

Al

mind mostly functions by applying approximate or cookbook became the basis for "heuristic," or

solutions to problems. This idea

"rule-bound," programming.

Simon had noted

Finally,

that

members of organizations

identify with

subgoals rather than with global aims. For example, a company's advertising

department

strives to

produce the

flashiest

whether or not they increase the company's

profits.

ad campaigns

it

can,

Only supervision by

top management can reconcile the two goals of profits and glamorous

Thus, organizations in

advertising.

growth or

survival)

effect achieve global goals (here,

by breaking them up into smaller aims

(like advertis-

ing and profits), which different departments pursue in a coordinated

manner. This "goal, subgoal" strategy

is

now

a key concept

In retrospect, Simon's transition from economics to

of

AL

artificial intelli-

gence appears almost natural. As he told me: "I was always interested

of intelligence.

in the nature

intelligent-like

Any word of some mechanism that behaved

was something

I

pricked up

my

ears at."

Allen Newell Simon

Dartmouth conference with his younger colleague, The son of a radiology professor at Stanford University, Newell had grown up in San Francisco. He graduated in physics from arrived at the

Allen Newell.

Stanford, where he took courses from the mathematician Artificial intelligence

for the rules

owes

to Polya the

of thumb people apply

word

heuristic,

George Polya.

which he coined

in everyday reasoning. In 1945,

Polya showed the problem- solving power of heuristics in an influential

book

called

How

to Solve It.

34

After Stanford, Newell spent a year in Princeton's graduate school of

mathematics, but did not find

had (he entered Princeton

Minsky by

a couple

of

it

as congenial as

at the

same time

as

Minsky and McCarthy McCarthy, but missed

Contrary to them, Newell decided he

years).

wasn't a mathematician, and dropped out of graduate school.

He prefer-

work where he would have concrete problems to solve. It was then 1950, and the RAND Corporation of Santa Monica offered bright young

red

scientists

with a practical bent the opportunity to prove themselves.

Newell was

set to

work on

a project

aimed

at

modeling a regional

air-defense center. Part of the activity consisted in producing aerial

through a computer-driven printer. Simon, schedule to consult for

RAND,

happened

who found

maps

time in his hectic

to see the printer in activity.

44

Al

To anyone familiar with the drawing programs available on today's home computers, this would have been a perfecdy trivial sight. But in the early 1950s,

Simon found

it

an eye-opener: the dots and characters

making up the maps weren't numbers. He saw them computer was manipulating them! From there ers could just as well simulate

Simon made

leap, but

it

thought

still

without flinching.

started holding informal discussions

on

as symbols,

to deciding that

and

required a major conceptual

From

then on, he and Newell

the subject.

For Newell, however, the lightning bolt of conversion occurred

when

1954,

recognition ideas

Oliver Selfridge visited

work

come about as "It

that

all

The

a result

happened

in

First Al

RAND

in

to describe the pattern-

would soon lead him

hooked Newell with

a

comput-

Pandemonium. These

to

the realization that a

complex process could

of the interactions of many simpler subprocesses.

one afternoon," he

says.

35

Program

With the help of J. C. Shaw, an actuary turned computer programmer at

RAND, Newell and Simon started in the fall of 1955 the development

of what

now

is

Theorist.

considered the

Simon

recalled

it

for

intelligence program: Logic

first artificial

me

as follows:

That autumn, we had considered three tasks for the program. Our original intention

was

to start out with chess,

considered doing geometry.

We

and then we also

thought, probably wrongly, that in

both of these there were perceptual problems, important

performed the logic, for

[Russell

would be hard

no deeper reason than

and Whitehead's]

for an efficient

humans, by

was

tasks, that

that

Principia at

to deal with.

selective heuristics,

home.

found the

humans to

had the two volumes of

I

.

.

.

We

way of proving theorems.

certainly in the center

as

So we went

We were not looking at

how

next.

That

were looking

right thing to

of my mind: what heuristic

.

do .

.

would kick

out the right theorem to use rather than searching forever.

As Simon, Newell, and Shaw had reduced

realized,

theorem proving can be

to a selective search. It helps to represent the search graphically

as a treelike structure, called the "search tree."

The

starting point, or root

is the initial hypothesis, on which the rules of logic allow a certain number of elementary manipulations, which yield slightly modified ver-

node,

THE FIRST

Al

45

PROGRAM: DEFINING THE FIELD

sions of the hypothesis.

Each possible manipulation corresponds

to a

branch out of the root node. Applying other manipulations to each of these results gives the next generation of branches, and so on.

where down the manipulations,

tree, after the application

lies

Some-

of an unknown number of

the desired conclusion: the

problem

is

to find a path

leading to this result.

To

find this path, Logic Theorist explored the tree in a goal-oriented

manner. Starting

at the

root node,

applied appropriate rules of thumb,

it

or heuristics. These rules allowed

to

it

among

select,

branches leaving the node, the one that was most

the goal. Following this branch, Logic Theorist reached a

which

it

possible

toward

new node

at

applied the heuristics again. Searching, goal-oriented behavior,

rule-bound decisions rationality theory

Programming

— most of

— were

before coding

it

AL

any computer, was laborious in 1955; and

Allen [Newell] and

I

session:

wrote out the

(subroutines) in English

program Simon de-

easier to hand-simulate the

it

into the machine. In his autobiography,

one such simulation

bounded-

the basic ideas of Simon's

thus carried over to

a computer,

the trio of researchers found

scribes

all

likely to lead

components of the program

rules for the

on index

cards,

and

also

contents of the memories (the axioms of logic). Industrial Administration] building

on

At

made up

cards for the

the [Graduate School of

a dark winter evening in January

we assembled my wife and three children together with some graduate students. To each member of the group, we gave one of the cards, so that 1956,

each person became, in puter program

—

effect, a

component of the [Logic Theorist] comperformed some special function, or a

a subroutine that

component of its memory.

It

was the

task of each participant to execute his

or her subroutine, or to provide the contents of his or her memory, whenever called by the routine at the next level above that was then in control.

So we were able to simulate the behavior of [Logic Theorist] with a

computer constructed of human components. Here was nature imitating imitating nature.

.

.

.

Our

children were then nine, eleven, and thirteen.

art

The

occasion remains vivid in their memories. 36

The

actual implementation

of Logic Theorist on a computer

at

RAND did not occur before the summer of 1956. Well before that, the hand simulations demonstrated the program's soundness satisfaction. class

He

thus proceeded to

of the new year

at

make an announcement

to Simon's

to his first

Carnegie Mellon: the future Nobel laureate

46

vi

boasted of nothing

less

than inventing an intelligent program over the

Christmas vacation. Simon

where he reports

is

no more abashed

this success as follows:

autobiography,

in his

"[W]e invented

computer

a

program capable of thinking non-numerically, and thereby solved the venerable mind/body problem, explaining

how

a

system composed of

matter can have the properties of mind." 37 Boastful as he sounds,

whom

Daniel Dennett, to that Logic Theorist

by

Simon may have I

The philosopher

a point.

read this quotation, scoffed

itself solves

the

at the

thought

mind-body problem. Dennett,

however, went on to argue that AI as a whole may very well hold the key to

Logic Theorist in any case kicked off an unending

this mystery.

debate about

its

first

fifty-two

As

philosophical implications.

point, Logic Theorist

was eventually able

theorems

in chapter 2

confirm Simon's

if to

of Russell and Whitehead's

Mathematica. Logic Theorist's proof for

of the

to prove thirty-eight

one theorem (number

Principia

2.85)

was

even more elegant than the one derived by Russell and Whitehead:

Simon In

delighted Russell by informing

deriving this

him of

another way: they had not explicidy instructed the structure of the that

programs can

Theorem

this success.

proof, the program had surprised

authors in

to find the proof. Yet,

do so anyway, thereby showing times do more than their programmers tell them.

program caused at

it

38

its

it

to

2.85 also provided an amusing footnote to the history of AI:

Newell and Simon submitted the new proof for publication journal of Symbolic Logic, listing the

program

missing the implications, the editor turned

down

the paper

it was no accomplishment outmoded system of Principia. 39

prove

a

grounds that

"Beads"

of

to

of

a

it

on

theorem

a

in the

long time

had to await the development by Newell, Simon, and Shaw

computer-programming language with enough power and

bility.

the

Memory

The computer implementation of Logic Theorist took because

to the

as a co-author. Entirely

flexi-

This language was called IPL (for Information Processing Lan-

guage) and incorporated another invention of the

more important than Logic

Theorist: the

trio that is

list-processing

perhaps

technique for

programming.

IPL which

differed

IBM

from other

released a

first

high-level languages like

FORTRAN,

of

version shortly before Newell, Simon, and

THE FIRST

Al

Shaw developed IPL. Short was aimed

computing

First,

you have pro-

algebraic formulas. If

BASIC

language,

FORTRAN.

with

Shaw were unhappy with languages

Newell, Simon, and

ing,

eased the description of numeri-

It

common

in

FORTRAN

FORmula TRANslation,

microcomputer, you may have used the

a

which has many features

TRAN

for

and engineers.

at scientists

cal operations, like

grammed

47

PROGRAM: DEFINING THE FIELD

like

FOR-

because they didn't model two important features of the mind.

they assumed that thought consists in constantly creating, chang-

and destroying interacting symbol

TRAN

and BASIC cannot do

numbers or symbols used symbols

Languages

require that

until they

effect reserves a region

come

into use.

Having

symbol

will do,

structures. Say,

is

we

give

impossible to a

program could do

this,

ahead of time for

its

of

in a

of memory

in

advance

and prevents the program from creating new it

information about individual items

named JOHN, JIM, and MARY. Generating a new

PEOPLE

FOR-

arrays

to include this

know

statement in the program forces the programmer to

what the program

like

all

program be defined beforehand

in a

program statement. This statement in to store these

structures.

They

that.

FORTRAN

however, because

symbol

Simon, and Shaw wanted to model was human memory. In our minds, each idea

them

program.

The second

called

An IPL

memory space

feature Newell,

the associative character of

or remembrance can lead to

and these

it,

array for

BASIC

didn't reserve

it

structures.

other symbols that are linked to

or

can be acquired

links

through learning.

The

trio

was able to incorporate into

to associate

technique, which Herbert

The

a

computer language the

basic idea

is that,

Simon has described

whenever

a piece

(associated) piece like a

of information. In

as follows:

of information

additional information should be stored with

organized

ability

and modify symbol structures through the list-processing

this

it

way

telling

is

stored in

where

the entire

memory,

to find the next

memory

could be

long string of beads, but with the individual beads of the

string stored in arbitrary locations.

"Nextness" was not determined by

physical propinquity but by an address, or pointer, stored with each item,

showing where the associated item was to a string or omitted

from

located.

a string simply

Then

a

bead could be added

by changing

a pair

of addresses,

without disturbing the rest of the memory. 40

Simon told me that he and his colleagues took their cue from early drum machines, the ancestors of today's hard-disk memory-storage

48

||

For technical reasons

devices.

machines already used

Simon discussed

who

McCarthy, chapter

a technique similar to

this

it

in his

modeling the mind, drum list

processing. Newell and

Dartmouth with John own AI language called LISP (see

extensively

idea

used

later

irrelevant to

at

3).

The Dartmouth Conference of 1956 At Dartmouth, Newell and Simon were, working AI program,

remembers them

as

being a

little

all.

it

Neither AI [Newell] nor

under such circumstances." That

moments

state

the following September,

workshop

at a

meeting of the

Minsky

this reason,

standoffish during the conference.

"We were

Herbert Simon confirmed to me:

about

as the only participants with a

ahead of the others. For

far

I

are

of

when

Institute

probably

known

for

affairs also

the time

gave

came

arrogant

fairly

much modesty rise to tense

on

to report

of Radio Engineers held

at

the

MIT.

Simon

Since they were the only ones with concrete results, Newell and

challenged McCarthy's right to report alone. They finally settled by talks: McCarthy summarized the meeting in Simon expounded on Logic Theorist. 41 The organizers of the Dartmouth conference had hoped

having two separate

general,

while Newell and

emergence of

where

it

a

common

was going. As

feeling

on both where

the discipline

a starting point for discussion, they

for the

was and

had proposed

the following statement: "Even* aspect of learning or any other feature

of intelligence can

in principle

be so precisely described that

a

machine

made to simulate it" This belief has remained the cornerstone of most AI work until today. It later became known as the "physical symbol system hypothesis." The basic idea: our minds do not have can be

direct access to the world.

tation

of

it,

We can

which corresponds

operate only on an internal represen-

to a collection

of symbol structures.

These structures can take the form of any physical

pattern.

They can

consist of arrays of electronic switches inside a digital computer, or

meshes of (brain or

firing

neurons

in a biological brain.

An

intelligent

into other constructions.

Thought

consists

of expanding symbol

tures, breaking them up and reforming them, destroying

creating

new

svmbois. it,

system

computer) can operate on these structures to transform them

ones. Intelligence

It exists in a

transcends

it,

is

struc-

some and

thus nothing but the ability to process

realm different from the hardware that supports

and can take different physical forms.

THE FIRST

49

PROGRAM: DEFINING THE FIELD

Al

The Dartmouth conference was deeply disappointed

many ways

in

inconclusive and

John McCarthy. For one periods of time, which precluded regu-

principal organizer,

its

came for different The problem was perhaps that not all participants agreed on the conference format. Herbert Simon recalled to me that "they were going to have a kind of floating crap game all summer with people sitting thing, people

meetings.

lar

with each other, thinking and so on.

And

[Newell and

programming the Logic Theorist, so we agreed we'd spend at

were busy

I]

[only] a

week

Dartmouth."

No

consensus emerged on what the

and most of the participants

Simon described They didn't want them:

we

we

it

to me: "It

to hear

was or where

field

later persisted in their

from

was going us,

already

had done the

first

much

'Not Invented Here' sign

is

it

was going,

approaches.

and we sure didn't want to hear from it

was

ironic because

example of what they were

attention to

But

it.

that's

after;

recalls that

versely induced a false sense of achievement. 42 Contrary to participants believed, their understanding of the theories

elicit

half a

still

quite incomplete. Further,

the worldwide interest they

dozen people

remained

AI

it

per-

what the

of symbolic

research did not

had expected. "Dartmouth got only

active that weren't before,"

a very small

and

not unusual. The

up almost everywhere, you know."

There was, nevertheless, enthusiasm; but Minsky

manipulation was

As

off into different directions.

had something to show them! ... In a way,

second, they didn't pay

own

group for quite

a

Simon

few years

told

me. "[We]

Minsky

after that,"

confirmed to me, "because most people thought AI was impossible. [In a sense] that

was the pleasant part of

have to worry about publishing

Yet the conference the

is

it

the

it:

you got an

if

idea,

you didn't

same week!"

generally recognized as the official birth date of

new science of artificial intelligence. One reason is perhaps

participants in the meeting crystallize the

Simon and

group, gave a sense that

say,

that

had never met each other before. got an idea, I'd

if I

'What do you think of this?'

.

.

.

call

Looking back,

most

"It did

Herb was

that

the start of the community,"

Marvin Minsky told me. Dartmouth indeed defined the AI establishment: for almost two decades afterward, all significant AI advances were made by the original group members or their students. We can surmise that if the brilliance of the conference participants accounts for

much of this

then the preference of agencies to fund recognized ligible factor either.

state

elites is

of affairs,

not a neg-

50

Al

The other AI was his

claim of the Dartmouth conference for being the cradle of

the christening of the

new

discipline.

McCarthy, remembering

disappointment with the automata-theory papers edited with Shan-

non, was looking for an accurate and catchy name. Overcoming the resistance of

some

participants (Samuel felt that "artificial"

phony, and Newell and Simon persisted

sounded

work "complex information processing" for years afterward), McCarthy persuaded the majority to go for "artificial intelligence." He lays no claim to having coined the phrase, and admits it may have been used casually beforehand. Yet nobody denies him the achievement of getting it widely accepted. this

To

label a discipline

is

in calling their

to define

its

boundaries and identity:

accomplishment belongs to John McCarthy.

3 THE

DAWN

OF THE

GOLDEN YEARS: 1956-63

After Dartmouth,

AI, for better or for worse, was

intellectual inquiry. In

many ways

it

now

a field

was no more unified than

it

of

had

been before 1956 but, perhaps because of the continuing exchange of

Dartmouth, AI

ideas initiated at It is

probably not

started progressing in leaps

much of an

and bounds.

exaggeration to suggest that the later

advances made in AI consist largely of elaborations and implementations

of ideas

first

formulated in the decade following Dartmouth.

During those years the main centers of AI research were Carnegie Mellon (then Carnegie Tech), a lesser extent, Stanford

AI

MIT

and

its

Lincoln Laboratory, and, to

and IBM.

researchers centered their

work around two main themes.

First,

they wanted to limit the breadth of searches in trial-and-error problems; the Logic Theorist,

were

results

of

Geometry Theorem Prover, and SAINT programs Next, they were hoping and trying to make

this effort.

computers learn by themselves; chess, checkers,

their attempts in this direction

and pattern recognition programs.

were the

52

A

MODELING HUMAN COGNITION

AT

CARNEGIE TECH As proud some new

Newell and Simon were of

as

humans

solving behavior in

way Logic Theorist all

that disturbing.

Logic Theorist program,

their

—

coming out of psychology

research

did.

— suggested

Of course,

that

in 1954 on problemhumans do not reason the

signing a flying machine, they

first

should not have been

in itself this

For humans to make the

final

breakthrough in de-

had to accept that such machines do

not necessarily have to be modeled on the way birds thinking machine have to think the if

humans can

mouth

why

fly.

Why

should a

think? But then again,

ignore totally the design of the only success-

system around?

ful intelligent

And

reason,

way humans

so contrary to the goals of

most other

conference, Alan Newell and

participants at the Dart-

Herb Simon's attempts

at

program

design soon shifted away from trying to exploit the capabilities of

computers to trying

to simulate the

human

So

cognitive processes.

productive would this approach turn out to be that they never again deviated from

What

set

it.

them on

had presented

this

path was some ground-breaking research by

Moore and

psychologists O. K.

test subjects

S.

B. Anderson.

1

Moore and Anderson

with a series of puzzles and logical problems

of the kind Logic Theorist solved, and had asked

their subjects to "think

aloud" while working on the problems. Following Moore's and Anderson's lead, Newell and

Simon

carried out similar experiments that

proved "fabulously interesting" to the Carnegie Tech researchers. Simon

remembered

We

it

for

me

as follows:

started looking at this

human

data,

and asking "Are these people

behaving like the Logic Theorist?" The answer was no. Then we asked,

"How

are they behaving?"

And we

extracted the ideas for our [next

program] General Problem Solver right out of human protocols.

What

separates

GPS

from Logic Theorist and

mention of a

The whole

[particular] task.

a completely task

task-specific

a set

is

independent manner.

structure

You

just

.

.

.

GPS we learned

that with

of heuristics which had

to extract out an organization

.

.

.

in

it

no

was encoded

had to plug

in

in the

components, so to speak, into the slots, and it worked! The

53

THE DAWN OF THE GOLDEN YEARS: 1956-63

word general refers

we had

specifically to the fact that

segregated the

2 task-dependent and the task-independent parts.

The

first

run of Newell and Simon's

then they had given a

name

GPS

occurred in 1957. 3 By

method: "means-

to this task-independent

ends analysis." In a sense, means-ends analysis

back principle carried to a higher

level

is

Wiener's feed-

just

of abstraction. Like any good

feedback mechanism, means-ends analysis worked by detecting ences between a desired goal and the actual state of

Where means-ends

reducing these differences.

concept was in

this basic

variations react

— not

when

its

just one, as a

analysis

ability to react to a

differ-

and then

affairs,

improved upon

wide spectrum of

thermostat might, but

much

humans

as

they are given a variety of slightly different problems to

solve.

Consider, for example,

how you would program GPS

to solve the

problem of the monkey faced with the perennial too-high banana. The animal, alone in a

room

dangling out of its reach.

containing a single chair, Is

the

monkey

(or

GPS)

grab a banana

tries to

clever

enough

to

move

the chair to the banana and climb up?

To

explain the problem to

spatial coordinates,

GPS,

the

programmer

first

gave

banana, and the chair. differences in positions

the set of predefined actions or operators was:

"jump," and "move

chair,"

GPS

a set

of

The programmer instructed GPS to calculate among these elements. He also informed GPS

of certain actions to perform to reduce these differences. In

itself as the

it

which described the positions of the monkey, the

self." (We'll

"move

this case,

chair," "climb

up

assume the program thought of

monkey.)

some of the actions unless certain conditions programmer also informed GPS. For example, it was of no use to have the monkey "climb up chair" unless the monkey had already "moved self and "moved chair" underneath the banana. likewise, the monkey could not move the chair unless both chair and monkey were first brought together that is, to the same set could not perform

existed beforehand

—

as the

—

of coordinates. All

this

information was given in a standardized form

called the "difference table."

To

explain a

new problem

needed only to construct an appropriate difference of the

rest

through means-ends

table.

to

GPS, one

GPS

took care

analysis.

In the monkey's case, GPS applied the actions "move self to chair," "move chair to banana," and "climb upon chair." This sequence of actions

54

Al

reduced the three kinds of differences identified during the problem

and thus allowed the problem to be solved.

analysis to zero,

Furthermore, the preconditions associated with two of the actions led

GPS

two subgoals accessory

to identify

to

subproblems was

feature

a characteristic

main

its

chair" and "get to the chair." This breaking

"move

goal:

the

up of the problem into

of means-ends

analysis. It

stemmed direcdy from Simon's observations on the workings of organizations. Also, when GPS abandoned jumping and tried another approach instead,

(Does

strategy

it

applied "backtracking," another basic tool of AI. This

work?

this

computers can

at

If not, try

times generate

something

explains

else)

more knowledge than

their

how

program-

mers put into them.

GPS,

forms, remained a part of Simon and NewelTs

in various

GPS

was

G. Ernst,

who

research from 1957 to 1968. dissertation

GPS

by

a student,

also the subject

adapted

of

a doctoral

4 to several problems.

it

learned to solve various puzzles, performed symbolic integration,

and broke secret codes.

Other experiments on modeling human cognition Carnegie Tech in the studied

human

syllables: his

And Memorizer) new

ory in a

light.

5

(I

also

went on

program

model of how people

EPAM

(for

induced psychologists to look shall say

more about

Elementary Perat

EPAM

human mem-

in chapter 10.)

Another student, Robert K. Lindsay, studied verbal behavior level: his

at

1950s and early 1960s. Edward Feigenbaum

learning by building a working

memorize nonsense ceiver

late

SAD SAM program

at a

deeper

parsed sentences in ordinary English and

them information about family trees. 6 Given the sentences "Jim is John's brother" and "Jim's mother is Maty," the machine would start building the internal equivalent of a genealogical tree. This tree implicitly contained the information that Mar)- was also the mother extracted from

of John.

The computer running

SAD SAM may

well have been the

machine to show the glimmerings of understanding For

us, as

humans,

to understand

information to other facts that usually lets us

we

is

to be able to relate

already

know, and

to

draw conclusions we have not

first

human sense. a new piece of

in the

do so

yet

in a

manner

been given. The

richer the

network of connections, the deeper the understanding.

SAM was

a first step in this direction.

SAD

55

THE DAWN OF THE GOLDEN YEARS: 1956-63

MACHINE-BASED INTELLIGENCE While

Tech contingent from the Dartmouth conference work on human cognitive processes, most of the other partici-

the Carnegie

did their

pants were the

still

human

under the impression that extensive knowledge of

brain works

was not necessary for

how

their original goal

of

developing machine-based intelligence. Their programs reflect what

would become the

traditional

AI approach.

The Geometry Theorem Prover at IBM IBM became

In the late 1950s

seriously involved in the

computer

business. Since artificial intelligence at the time looked like a natural

company allowed such work to IBM, for example, was impressed by

extension of computer research, the

proceed. Nathaniel Rochester of

some simple a simulation

theorems.

results

Marvin Minsky had gotten from manually running

of a program he had written to prove high school geometry

He

decided to try out the idea on IBM's newest machine, the

young

704 model, and entrusted the job to Herbert Gelernter,

a

with a Ph.D. in physics and earth-shaking enthusiasm.

The road from

manual simulation

much

longer and

to a running program,

much more arduous

Gelernter close to three years of

work

recruit

however, would prove to be

than anyone expected. to write

It

took

and debug the twenty

thousand individual instructions making up Geometry Theorem Prover. Before

that,

he had

endowed with both

virtually to invent a

Shaw's IPL, and the ease

FORTRAN

new programming

language

power of Newell and of programming afforded by IBM's new

the symbol-manipulating

language for scientific computations.

The Geometry Theorem Prover worked backward. 7 One first scribed to it the theorem to prove. Much as a human being does, program then

started to build a chain

of intermediate

dethe

results leading

back to known theorems or axioms. But what makes Gelernter's work so interesting and important

is

that to figure out

back to the axioms, the program looked

at a

what

steps might lead

drawing. Since computers

couldn't yet "see" through television cameras, Gelenter had to enter a representational figure as a series of point coordinates cards.

on punched

Using these coordinates, the program was able to extract the same

kind of information a

human does when

looking

at a representational

56

M Which

figure:

right angles?

With tried to

sides are equal or parallel to each other?

information, the program pruned

this

problems of

search

tree: that is,

it

something that humans do unconsciously with

this kind.

made

This pruning

all

the difference: 8 to derive a two-step proof in the

mode, the program had

1,000 x 1,000 x 1,000

program reduced

choose between 1,000 x 1,000 (one

to

million) possible combinations.

the

its

demonstrate formally only those properties that appeared to be

true in the drawing,

blind

Are there any

Are some angles equal to each other?

For a three-step theorem, there would be

(a billion)

this

choices.

By "looking"

at the "figure,"

unmanageable quantity to 25 choices for the

two-step problem, and to 125 for three steps (see figure

3.1).

The Geometry program could eventually prove theorems involving up to ten steps. More important, however, GTP was the first demonstraFIGURE Example of

a

3.1

Proof Found by the Geometry Theorem Prover

GOAL BD = EC

BDA&

DBA& CEM

ADB& MEC

BDM&

CEM congruent?

congruent?

congruent?

congruent?

AB AC? No

matching

premise

* # Tp Tr # Tr

<

PT* ^Ti

"

5 Tr M

*»

in

Tl Ti IT Tl

#

ofio qTi Tr Tr Tr °pt Tr Tr Tr hp Tr Tr

"43"~

ff

«r

ei

1

I

°

I

I

I

1

l°

°|

l

I

I

I

'IT 'M m

to

ib" =H= =y£ -H£

Tr tf Tr Tr

i

"^

i

*

aft: =& ft Tr Tr d£ Tr Tr Tr

3t Tr =tt tr Tr it

zt£ "ott

ft Tr it Tr a£ Tr

Tr -i Tr Tr Tr Tr Tr Tp Tp Tr Tr ilt Tr~

8 a IS

is 3S

3

8

.sy

"^TT"

Figure 9.1 (continued)

To

on one page,

help keep the drawing

have taken symmetries into account.

I

You

can reduce

missing ricktacktoe grids to one of those appearing in the figure by appropriate rotations.

of grids corresponding to the

by crosses) are

To

laid

out horizontally.

how computers

illustrate

aim

scoring, function. Its

You

to winning.

first level (first

is

To

save space,

game

search a

I

tree,

The

sets

naughts) and second level (countermove

have

laid

out the third level

vertically.

have devised the following evaluation, or

I

any stage of the game,

to measure, at

can calculate

move by

how

close the naughts side

is

value for any of the grids as follows:

its

— — For each way of completing Start at zero.

a straight line

of naughts by adding one naught to the

grid,

add

2 to the score.

— For each way of completing add

I

of naughts by adding two naughts to the

a straight line

have calculated the scores for each of the third-level grids

by the

For the second and

grids.

values of the scoring function.

appearing by

To

see

of the

some of

how

tree.

in the figure: they are the

Naughts can use them

move. The

to plan their first

naughts can plan their

move,

first

let

E on

us start at grid

(as-yet) hypothetical situation

the rightmost branch

where naughts have taken

(6)

occurs for naughts playing the lower-right corner (grid A). Naughts would

therefore take that position in an actual game. play the middle of a

row on second

level,

I

therefore assign a value of 6 to grid E. This that 6

the

is

largest

the third level (grids

grid (grid

The

on second

second

now

F

to

I).

for naughts

will

They will

is

to

level.

will

and

Naughts a

row or

will lead

to a

say: "If crosses

third level."

E

(grids

naughts would achieve

that the cross-player

know

Assuming

this

on the

is

at

Naughts

A

will

Note

to D). Similarly,

most only

also clever

that if they play a corner

a 5-score

and

will

on second

conduct

6.

Obviously then, crosses

a

level (grid J),

next move. If crosses play the middle of a

five in the

achieve the higher score of

will play a

row

corner on

worst-case play by crosses, naughts can thus assign a score to taking

first level (grid

the first-level score should be.

E

on

thus assign this value to the corresponding second-level

assume

be able only to achieve a

naughts

the center position

(grids

six

the value appearing by this grid in the figure.

level (grid J),

similar analysis. Crosses will therefore

(grid E),

is

naughts can

stage,

J).

trick

naughts

At the planning

can always achieve a

of the values associated with the daughters of grid

crosses played a corner

on

Naughts must now

second move. Excluding symmetries, there are only four choices. The largest

their

scoring function

letters

the grids are labels to clarify these explanations.

This grid corresponds to the

upon

numbers

the scores appearing are the so-called backed-up

first levels,

the center position, and crosses have just played the middle of the left column.

decide

if

grid,

to the score.

1

Note

that 5

K). Since is

it

leads to a 5 in the third level, this

is

what

the smaller of the scores of the daughters of grid

K

J).

will

then carry out a similar analysis for other

a corner: grids

Q

only to a 3 and a

and W). They 4.

will

first-level

discover that

The winning opening move

is

if

moves

(either the

middle of

crosses play correctly, these

moves

therefore to play center, which leads

5.

How

can

we summarize what we have done? Start at the deepest level laid out in the tree, and move up one level. If it's your move, assign the maximum score

calculate the scoring functions,

of the deeper positions to the parent position.

If

it's

your opponent's move, assign the minimum

score of the deeper positions to the parent. Back up along the tree in this the present position.

The

procedure's name, minimax, shows that

maximizes along the

tree.

In principle, one can use this strategy to back

analysis.

it

way

alternately

until

you reach

minimizes and

up from any depth of

226 the

Al

most promising moves

for further evaluation.

program contained information on chess

blatt's

A

section of Green-

how

and

strategy

to

generate moves.

MacHack's

by the U.S. Chess Federation, stood

rating, as evaluated

between 1,400 and 1,500' 2

—

tantalizingly close to the 1,537

mean

rating

of UCSF members. Greenblatt's program inspired many AI researchers to take

up computer

interest in AI.

chess. Conversely, chess experts also took

Among them was Hans

correspondence chess championship years

one of the top twelve players

in

Berliner,

who won

1968 and was for some

in the

United

States.

up an

the world fifteen

His fascination

with computers and chess led him to quit industry to seek higher education in the

Among

field

of

artificial intelligence.

other successes, Greenblatt had the pleasure of seeing his

program trounce Hubert Dreyfus, 13 who had recendy put out

RAND paper against AI

olic

14

and

insisted that

his vitri-

no chess program could

play even amateur chess.

By 1970, there were enough chess programs for the Association of Computing Machinery to schedule a tournament. It drew six entries and made up one of the main attractions of the association's annual meeting. Seeing

ACM organizers decided to hold tournaments annually:

this,

next year there were eight entries. Other countries followed

suit;

the

and by

1974, contenders for a world chess championship convened in Stock-

holm under the

auspices of the International Federation for Information

Processing (IFIPS). Kaissa, the Russian program that had

match

against Stanford,

won

won

the mail

the tide.

Meanwhile, computer programs started actively participating

man battle

in hu-

tournaments, and climbed their way up chess ratings. The

of wits between humans and

competitive

champions

spirit,

A Scottish international master with a penchant

David Levy, threw the

celebrated 1968 wager, Levy bet that

chess for ten years.

this

bets and prizes for programs that could defeat chess

proliferated.

for computers,

first

minds was on. Spurring

artificial

first

challenge.

In a

now-

no computer could beat him

The AI researchers John McCarthy, then

and Donald Michie, of the University of Edinburgh, took him up on for

£250

each. In 1971,

Seymour

of the University of California amounts. Levy collected bition in Toronto.

He

in

Papert, of at

at

at Stanford, it,

MIT, and Ed Kozdrowicki,

Davis, raised the ante by similar

August 1978

at the

Canadian National Exhi-

held up for humankind by trouncing the world

computer champion Chess

4.5, a

program created

at

Northwestern

227

GAME PLAYING: CHECKMATE FOR MACHINES? University. ers

Levy won the match 3V4

would not reach the

He

years.

1

and estimated that comput-

V£,

computer champion of the day, Levy again

Emboldened, he issued

handily.

a challenge to the

world

he was ready to wager £100,000 that no computer would

he would appoint, for ten

a chess player

him up on large

years.

So

far

at large:

defeat him, or

no one has taken

15 it.

AI

In general though, impoverished

wager

many more

thus renewed the bet for another six years. In a 1984 rematch

against Cray Blitz, the world

won

to

international grand-master level for

researchers are unwilling to

sums of their own money on the

of their programs.

abilities

however, do get their attention. In 1979, a Dutch software

Prizes,

company, Volmac, offered $50,000 to the author of any program could beat the former world champion

The

Max Euwe by Edward

prize remained unclaimed. In 1979 also,

MIT

inventor and

first

January 1984. 16

Fredkin, a wealthy

up three prizes with no time

professor, set

them: $5,000 for the

1

that

limit

on

chess program to earn a master level in

tournament play against humans; an intermediate prize of $10,000 for the

first

program

to reach the level

of international grand master; more

significandy, for $100,000 the authors

world champion. Fredkin thinks, puter chess

a kind

is

fly

by the

to beat the that

com-

he

prizes,

cites the

one

that got

more of them." 17

Spurred by the competitive tracted

program

solo across the Adantic in 1927: "Prizes are a wonderful

thing and there should be

higher in the

first

of benchmark for progress in AI research. As an

example of the positive impact of Lindbergh to

of the

most AI researchers do,

as

prizes,

human

AI

spirit

of the tournaments, and

later at-

researchers designed programs that scored ever

chess ratings. Their steady progression

is

depicted

in figure 9.2.

The undisputed computer champion of 3.0.

Written

at

Northwestern University,

model of Greenblatt's MacHack and programs did not stay at the top

the early 1970s was Chess this

program followed the

tried to simulate

for long: they

human

play.

Such

had one crippling

weakness. Despite the competence of most of their moves, they were likely to

make

outright blunders. This

either obvious or

embodied rules

happened when they overlooked

subde plays defying the general principles of chess

in their rules or evaluation functions. Conversely, these very

would

at

times lead

them

to

dumb maneuvers

that cost

them

a

game. The authors of these so-called selective search programs have

been able to formalize the principles that guide human playing only to

228 FIGURE

9.2

Computer Chess Tournament-Winning Programs in Their Times of Glory

Representative Ratings of

U.S.

1

Chess

Federation

2

Titles

World Champion's Rating

2800 2700 2600

_

•

senior _

/

master 2500

_

2400

2300

_

master

_

expert

2200

2100

y

+ x

nf

2000

Deep Thought Hitech Belle

1900

1800.

/

A

_

"

/ Mean rating, USCF members 153^

Jl

a

Chess

A

MacHack

x.x

:

1700

_

1600..

1500

_

c

^^i

1400 1300 1200

_

D

These

ratings,

show how ..

1100 _ 1000

E

on

the average against

human

"opponents. The scale s such that for a difference of 200 points between players, the o ne with the higher rating will win 75% of the time. An extrapola tion of the curve reaches the estimated rating of world champ ion Boris Kasparov (2800) in 1993. I

65

awardec by the United States Chess Federation,

well the pr< Dgrams did

70

I

75

r80

1

85

1

—

90

-

i

95

100

Yeai

229

GAME PLAYING: CHECKMATE FOR MACHINES?

an extent. They have captured some aspect of competent play, but

beyond

excellence remains

their grasp.

In 1973, the authors of Chess 3.0, Davis Slate and Larry Adkin,

weakness of

realizing this

around the so-called brute force approach. Relying on procedures and ever faster computers, the

game tree misnomer

to

new

program

their early strategy, redesigned their

efficient search

new program

searched the

depths. In fact, the "brute force" appellation

since the technique

relies

still

on

is

a

subtle strategies to reduce

it is possible to prune the game tree (that number of moves considered) by means that do not require any knowledge of the game itself. One of these strategies, called "alphabeta pruning," consists of abandoning the investigation of a move when

the search effort. Surprisingly,

is,

limit the

the

opponent can respond with

sponse to a

move

countermove better than the best

a

re-

already examined.

In the ticktacktoe example of figure 9.1, alpha-beta pruning would

work

as follows:

Assume

the search proceeds

from

right to

We,

left.

the

naughts player, have already examined the rightmost branch and assigned a value of 5 to playing center

K in

corresponds to grid

the figure).

value of playing the middle of a

Proceeding from right to

on

We

the

first

move

(this situation

then proceed to examine the

row on the first move (grid Q). we would immediately discover

left at level 2,

that for crosses to play center results in a scoring function

Consider

now that

the scoring function at level

smallest of the level 2 scores and, thus, can be

already

worth

know

(grid

K)

1

will

no

it

is

considering, and there

is

no point

thus a better

move

(grid P).

larger than 3.

on

that since playing center

5 for naughts,

of 3

correspond to the

the

first

than the one

Yet we

move is we are

to rating the other daughters of grid

Q. Alpha-beta pruning has saved us about 75 percent of the work rating the branch. If (say,

from

we had

left to right),

in

explored the branches in a different order

the search

would have taken

Modern make the

longer.

chess programs have ways of optimizing the search order to

most of alpha-beta pruning. One of them is the "killer heuristic": the computer gives precedence to investigating opponent responses that killed,

or refuted, other

the example, the killer

moves

the computer has already considered. In

move was

for crosses to play center (grid P). In

investigating the leftmost branch of the tree, the

computer would look

for this opponent's

move

first (grid

play also leads to a

weak

position for naughts, and immediately assign

a

weak

V). It

would then discover

rating to the entire branch following grid

W.

that this

230

\i

An

obvious

pruning consists

strategy' for tree

move

duplicate boards, as follows: If a

in

leads to the

keeping track of

same

situation as a

combination of moves already examined, use the corresponding known value for the scoring function. If the previous situation

through this

a

sequence of moves, to the present one, we are

move on

branch "draw," and

One problem

one

is

that led,

Label

in a loop.

to another branch.

with straightforward searches

which enables knowledgeable opponents

to

is

the "horizon effect,"

sandbag the computer by

dangling a juicy capture leading to a deathly trap, lying just beyond the

machine's search horizon. Even in the absence of traps,

computer

lead the

For

ate advantage.

and hungry for

zon

effect

to give

up an eventual

this reason,

material.

The

could

this effect

large gain for a small

immedi-

computer chess often looks impatient

chess programmers' response to the hori-

was the "quiescence search": they

let

the machine search to

deeper levels for moves leading to the capture of pieces.

These not-so-brutish search methods paid

who were

off for the

Northwestern

able to recapture the world and

North

American computer chess championships, which they had both

lost in

University researchers,

1974.

A

them

free use

unique arrangement with Control Data Corporation allowed

of the mighty Cyber

series

of computers during tourna-

ments and exhibitions. The Cyber's power allowed the Northwestern team to hold on to the annual North American championship and to the world championship grams, such as Duchess from

until 1980.

Duke

until

1

977

Other participating pro-

University, also ran

on very

large

machines. According to one participant, the value of the computers used

by the sixteen entrants to the 1977 Toronto tournament exceeded SI 00 18

The Northwestern program could rate about 3,600 chess moves per second. The resulting high level of play made it the first program to gain the tide of "expert," awarded to tournament players million.

who

attain a U.S.

Chess Federation rating of 2,000.

Chess 4.7 eventually

The

researchers

lost

both

its

tides to a

hardware version of itself.

Ken Thompson and Joe Condon of Bell

implemented algorithms

similar to those

into special-purpose silicon chips.

The

result

was

a dedicated, portable

chess computer that could beat any other machine in sight

connected to any telephone pretty-

much had

in sight, since the bulkier

had

called their

—

or, rather,

machines

it

battled

The portability* of Belle (as machine) was at times a source of

to stay in their laboratories.

Bell Lab's researchers

Laboratories

of the Northwestern program

231

GAME PLAYING: CHECKMATE FOR MACHINES? mishaps.

As

a plane for a

Ken Thompson were supposed

the machine and

Moscow

someone had

Labs' security department, claiming that

computers. They were trying to ship

their

Customs was

Meanwhile,

Thompson was

computer was not

the

it

Moscow. "Not

to

its

loss

remained

that

lost in

two weeks and could not

On another trip, this time to the 1983 New York, Belle had a car accident. Many

participate in the tournament.

people blamed

Moscow, unaware

in the plane's cargo hold. Belle

world chess tournament

in

19

of the world

tide to the resulting "electronic

A few weeks before, in August 1983, Belle had neverthe-

concussion." 20

master

less raised its chess rating to the

level. It

was the

ever to earn this tide, which requires a rating of 2,200 eration scale. It also earned Belle the

could then look

New

to worry,"

of high technology to the

flying to

the bowels of the federal administration for

In

one of

stolen

"our Exodus team has seized the computer." (Exodus

said,

a special project to prevent illegal exports

Soviets.)

to board

tournament, the U.S. Customs Service called Bell

at

first

first

on

computer

the U.S. Fed-

Fredkin prize of $5,000. Belle

150,000 chess positions per second.

lost to Cray Blitz, a 28,000-line-long comon a general purpose Cray XMP-48 supercomNo simple minded brute, the program had required 32,000 hours authors: Bob Hyatt and Albert Gower, from the University of

York, Belle had

puter program that ran puter.

of

its

Southern Mississippi, and Harry Nelson, of Lawrence Livermore Laboratory.

twice:

21

Cray

it

Blitz

won

was the only chess program

New York

the 1983

to reap the

world

twenty-one other teams from eight countries and, three years successfully defended also

its title

in

Cologne, Germany. Cray

Blitz,

marked the swan's song of general-purpose computers

championships. After

demise,

its

cated machines like Belle

Cray

was

a

in chess

were, so to speak, to the trade born.

North American chess champion-

machine conceived by none other than

Berliner of Carnegie Mellon University.

tion, Hitech, celebrated the

later,

though,

world championships went to dedi-

Blitz first bit the dust at the

ship of 1985. Its victor

Hans

who

all

title

championship against Belle and

days

when

CMU

The computer's

designa-

went under the name of

Carnegie Tech. Berliner's creation combined the innovative elements of Belle

and Cray

Blitz: its

positions per second, yet

dedicated circuits it

did so in a

let it

manner

process 175,000 chess

that could take elaborate

chess knowledge into account. Berliner's experience as world mail chess

champion provided the source

for

most of

this savvy. In contrast to

232

U

the $14,000,000

machine required to run Cray

Blitz, the

Hitech system

consisted of a special breadbox-sized contraption called the Searcher,

connected to a $20,000 Sun microprocessor. The Searcher contained sixty-four dedicated processor chips,

one for each square of the chess

board, which the processors monitored. Before each move, the processor detected

all

moves

on

that could land a piece

square, and analyzed

its

them. Meanwhile, other chips were doing the same thing for their squares,

which considerably increased the machine's speed. Yet Hitech

on hardware

didn't entirely rely

to select

its

moves: the sixty-four

processors fed summaries of their observations to a program called

much of

Oracle, which contained

cided

upon which move

to

Berliner's chess

make. In

knowledge and de-

way, Hitech managed an

this

average look ahead of eight levels but could, on occasion, pursue interesting

moves

of fourteen

to a depth

chess rating to well over 2,300, which

levels.

made

22

In 1986, Hitech raised

it

the world's

first

its

computer

international master.

The next quantum

leap in

computer chess

also

came from Carnegie

Mellon, but involved different people. Aside from any personal the

new machine probably

disappointed Berliner.

endowing them

believe in enhancing chess computers' performances by

with humanlike chess knowledge. They were a team of students

who knew little

creation, relied

about chess. Deep Thought,

CMU

graduate

as they called their

on speed and clever search methods. It made absolutely no

pretense at imitating

human

play.

And

deeply did the machine probe:

could analyze 700,000 chess moves per second Hitech) and project

of play. 24 In 1987,

many

rivalries,

designers did not

Its

1

23

(four times as

many

it

as

20 moves ahead along the most promising lines

5 to

a prototype

of Deep Thought, containing only half as

dedicated computer chips as the

final version,

won

the

North

American Computer Chess championship. In 1988, Deep Thought raised chess rating over the benchmark of 2,500. This

its

computer international grand master and

qualified

its

made

intermediate Fredkin prize. 25 Being a better chess player than

two hundred people league as the

in the world,

Deep Thought was now

And, indeed, the reigning champion an

official

the afternoon of 22

Boris

the

first

all

but some

in the

same

human world champion.

in history, to accept

On

it

designers for the

felt

compelled, for the

challenge from a

October 1989,

a

first

time

nonhuman opponent.

two-game match opposed

Kasparov and Deep Thought in the New York Academy of Art. 26

The computer and

the

human champion

faced off in front of an audi-

233

GAME PLAYING: CHECKMATE FOR MACHINES? ence of hundreds of reporters and aficionados, most of them in a to see the upstart

machine put

Kasparov won both games

in

its

place.

Meeting

mood

their expectations,

much

handily. This defeat did not

disappoint

Even though they had hoped to at least draw one game, their underdog status put them in a no-loss situation. Kasparov, for his part, was clearly taking no chances. He had prepared for the match by studying fifty of his opponent's previous games and the machine's designers.

avoided the daring moves that constitute his trademark. Elated as the audience was at Kasparov's victory,

members could not shake put

off a feeling

its

knowledgeable

of doom. As one commentator

"In the rapidly evolving relationship between people and their

it:

machines,

match

this

is

— one

an acknowledgement of

new, and inherently

a

on the board." 27 Said the match's organizer, Shelby Lyman, "The real drama here is that Gary is facing his fate." 28 Kasparov thought he could beat computers "perhaps 29 Lyman gave him five to ten years; and to the end of the century." short-lived state

Berliner, four.

The

of

essential parity

history of

computer chess supports

Kasparov's chess rating, the highest in the world,

is

Berliner's view.

about 2,800. The

curve in figure 9.2 portrays the steady climb of computers through

human

chess ratings.

A

straight line extrapolation

of

it

reaches Kas-

parov's level by 1993.

Which computer will beat the world champion and win the $100,000 It may very well be a direct descendant of Deep Thought.

Fredkin prize?

IBM

hired

writing,

most of the machine's design team, who

working on a new version of Deep Thought

Yorktown Heights thousand times

laboratory.

The new system

faster than the current

chess positions per second. Kasparov

have to challenge

it

to protect the

will

ones and look is

of

this

company's

deploy chips one at

roughly a billion

willing to take

human

are, as

at the

it

race," said the

on: "I

would

champion. 30

THE IMPLICATIONS OF

COMPUTER CHESS I he fun sight

of

and challenge of computer chess should not make one lose its

deeper implications. The computer's performance

penetrating questions that

from other potential

may help

intelligences,

us understand

how our minds

raises differ

and help us forecast the future of AI.

234

Al

Are Chess-Playing Computers Intelligent? Deep Thought plays chess better than 99.9 percent of human players. Does that performance make it intelligent? To this question, there are no simple answers. have already said that Deep Thought and other winning chess

I

programs and machines do not play chess minutes allowed per

move

in

tournament

humans do. In the three Deep Thought considers of the Dutch psychologist

as

play,

126 million moves. By contrast, the studies

human master

Adrian de Groot show that

moves per

average of 1.76

play.

31

ponder only an

players

They use up most of

their three

minutes verifying that these one or two moves are indeed the judicious ones.

As Herbert Simon noted, expert

players have an instantaneous

understanding of chess positions and a compelling sense of the winning

move. Simon believes

that chess masters are familiar with thousands

of

patterns involving small groups of pieces in certain relationships to each other.

Simon

a desirable

move

Each chunk would suggest

these patterns "chunks."

calls

or strategy, which would cut the need for extensive

simulations of later moves. 32 tion of chunking to

Hans

computer

According to the AI

human and machine

Hubert Dreyfus, the difference between

critic

play

is

Berliner has investigated the applica-

chess.

even more

basic.

Expert chess players,

claims Dreyfus, respond to whole board positions, not

chunks.

games to rate

An

33

expert player practicing fast play, or earning out forty

at a time, is

not simply showing

board positions instandy and

off.

He

is

intuit the

developing his

winning move

given situation. Topflight masters can play a very strong given only five seconds per move. Gary Kasparov said that the intuitive chess player

into account

component

when planning

a

somehow

move. While

takes

is

ability

in any

game when

reported to have

all

resulting

games

certainly exaggerated, this

claim agrees with Dreyfus's assessment that our major intellectual leaps are ^rational.

Some,

like the

computer-trouncing chess master David Levy, offer a

negative view of the intelligence of chess-playing computers. Tree searching, .

.

.

Levy

says,

"produces a kind of monkey/typewriter

[The computer] appears to play moderately well, whereas

playing very

weak chess so much

of the time that

its

situation.

it is

actually

best results

resemble the moves of strong players." 34 The fact remains, however, that

computers play games and win them. "Intelligence," says Edward

235

GAME PLAYING: CHECKMATE FOR MACHINES? Fredkin, tainly

having a problem and solving

"is

do

it."

35

Chess machines

cer-

that.

know human beings can reason about a large variety of subjects. Yet aren't human chess champions somewhat limited in their world views also? General-purpose game programs could, in any case, learn games other than chess. It took only a few weeks to train Deep Thought to play grand-master chess, a lifetime endeavor for any human being. Programs similar to Deep Thought could master other games, like Holders of Levy's opinion could also argue that chess computers

nothing but chess, while

intelligent

checkers, in a snap.

In the opinion of many researchers,

computers are tors

now

puters

intelligent,

reasonable to claim that chess

it is

but in a way different from

us.

Many investiga-

believe in a knowledge-search continuum, within

make up

searching.

for a lack of chess

Much

as airplanes

knowledge by an

do not

by flapping

fly

which com-

ability to

do more

their wings, chess

computers do not imitate human thought processes to win games.

Computer Chess and This pattern chines

may

brains. If

rely

such

is

AI developments. When ma-

well reappear in future

do perform mental

own, they may

Al

on

tasks

of a depth and diversity similar to our

principles different

from the ones

and

the case, interesting

that

might appear, the machines would be foreign to us inside times, this difference

govern our

familiar as their behavior

— and some-

would show.

Chess computers already provide us with some inklings of these differences.

called

A

popular distraction during AI meetings, which could be

"Turing chess," consists of pitting

human

chess players against

hidden opponents, some of them machines, some of them humans. The

and players, comes from guessing which opponents Even when humans and machines play at the same strength, knowledgeable observers can usually tell them apart from play styles: computer chess often looks ugly and inelegant. The computer, says Dave Slate of Northwestern University, "is like a shark sitting fun, for audience

are machines.

around.

It's

not very bright, but once

there and goes in all

it

munch, munch, crunch.

your armor, then you suddenly find your nicely

strategies,

laid plans

go

astray."

... If this

36

computers also often drag

gets a taste

of blood,

you allow any

thing

coming

slight

right

chink

after you,

Unencumbered by

their

it's

opponents into

and

traditional

situations

236 human

||

"They

players find bizarre.

designer,

are always original," says Belle's

Ken Thompson. "They're not

in the past."

enslaved by what's been done

37

Computer chess might model

future

AI progress

in

yet another

manner: success came about because of hardware advances; computers

human champions at chess before they achieved a certain The fact that Deep Thought could in 1 989 examine roughly two hundred times as many positions per second as the compara-

did not equal

processing power.

tively pedestrian

similar,

shows

Chess 4.7 could

that speed

is

in 1978,

even though

their software

root of better performance.

at the

As

discuss later, AI's present failure at modeling certain aspects of

thought, such as

common

sense, stems in part

their

may overcome them

machines overcome the speed bottleneck.

human mind in many people

This demonstration that machines can equal the

games

playing certain

oppose

forever

"They

I

will

put into them. Computers

dumb

in situations

that

of AI. "Computers can do only what they

are told," this objection goes. their creators

argument

also demolishes an

to the final success

never produce more than what

will

never create and

Deep Thought know

trying to

win

Thought

designer,

program

that can

little

against their machine.

do something

is

remain

"The

fascination," says the

can't do."

I

it

38

The

Deep

writing a computer

During computer chess

hard to

tell

of strategy or

a subtle piece

regularly.

chess and would never dream of

Thomas Anantharanam, "was

competitions, such programmers find

computer move

will

unforeseen by their programmers." Yet, as

mentioned, Samuel's checker program used to beat him

creators of

will

human

from the weakness of our

present hardware (see chapter 11). Researchers

when, and only when,

I

was

a

whether

a

puzzling

downright blunder.

Yet, in one significant way, computer chess failed the expectations of early researchers:

human mind. By

it

failed to

meet

hope of learning about the

their

getting computers to play chess, the reasoning had

gone, investigators could discover

how

times labeled the "drosophilia" of

artificial intelligence,

would play the

role in developing

AI

research. This did not happen. Carried

people think and decide. Some-

that fruit

flies

away by sheer competition, many

researchers started designing programs that

won games

ing like people. That

them

it

nevertheless led

processes different from our science. In the next chapter,

own I'll

is

computer chess

played in genetic

an

illustration

examine

how

instead of think-

to discover thinking

of the serendipity of

other approaches have

enabled AI researchers to model their programs on the

human mind.

SOULS OF SILICON

Long

before

we became concerned with understanding how we work, our

already constrained the architecture of our brains. However,

machines as we wish, and provide them with better ways of their own

activities

consciousness than

— and

we

this

to

keep and examine records

— Marvin Minsky

The significant forward leaps AI made during the Much more

common

industrial debacle

of the

in the eye the

sense and attacked

late

1

980s did not make

than an era of commercial ventures, this

was the decade when AI looked computers with

had

means that machines are potentially capable offar more

are.

the front pages.

evolution

we can design our new

it

problem of endowing

on

a

grand

scale.

The

1980s painfully emphasized the urgency of

Making computers see, understand the spoken word, and communicate in English became a priority. As a result, approaches more

this task.

basic gets

and daring than those permitted by the comparatively puny budand primitive computers of

for the

decades produced AI projects some of them claiming to account

earlier

involving person-centuries of effort,

whole of human cognition. These

new light on

the nature of awareness,

seriously the possibility that their

original

approaches threw a

and researchers

started to consider

machines might some day wake up to

conscious thought and feelings. Interesting debates followed a philoso-

238

41

phcr's objection to the

emergence of what amounts to

made of

assemblages of electronic chips

mere

COMMON-SENSE QUESTION

THE

with the common-sense question. Needless to say, the

It all started

various chapels into which the

AI community had subdivided

Some of

dressed the problem in different ways.

doggedly depended on with

a soul in

silicon metal.

logic.

all

ad-

these approaches

still

Others, as we'll see, would have no truck

it.

Logical and Not-So-Logical Solutions Recognizing that ability to

the neat, or logic-oriented, vestigating 5):

How

there are

To

common

element of

a crucial

sense resides in the

change one's mind when circumstances require,

community

ways out of McCarthy's

can one

tell

a

computer

good reasons why

it

in

AI research

now

problem

qualification

something

that

is

of

a large part

is

true,

busy

in-

(see chapter

except

when

should not be?

achieve this discernment, the neats discovered they had to revamp

their logical calculus, lifted

1910 edition of

almost intact from Russell and Whitehead's

They

Principia Mathematica.

Order Predicate Calculus (FOPC,

if

you

are

outfitted

on

good old

First

familiar terms with

it)

with extensions called "default," "nonmonotonic," or "modal" logic to deal with exceptions.

John McCarthy tially infinite

calls his

own

technique for dealing with the poten-

exceptions to a general rule "circumscription." His idea 1

is

to circumscribe, or delimit, the set of possible exceptions to a statement.

In the river-crossing example (see chapter isolate in the

If

none

exist in a given situation,

can cross the

As

then

a result of these efforts, the

knowing system

a

would be

of possible obstacles. safe to infer that

one

new

generation of expert systems

is

monitoring mechanism called a Truth Maintenance

TMS. Returning

that Charlie

will

it

circumscription would

set

river.

equipped with System, or

8),

knowledge base, or minimize, the

is

to Minsky's

a duck,

duck challenge (chapter

and that ducks normally

conclude that Charlie

flies.

Upon

fly,

8),

the expert

learning of poor Charlie's

239

SOULS OF SILICON passing away, the

TMS

normally undo

will

this inference.

The amount

of computation required tends, however to grow exponentially with the size of the knowledge base, and the millions of facts needed for true

common-sense reasoning would

still

snow

Other

A

to logic.

prominent example

mind of former Roger Schank

is

secretary of state Cyrus

in 1980, she

Vance

was among the

first is

form of

directiy as the experts do, in the

Confronted with

as a doctoral project for

to use a technique called

to stop painstakingly trying

new

a

a series

then search situation,

its

case, such as a set

knowledge banks for

of symptoms in a

computer would

a similar case, adapt

to the

it

new

and conclude accordingly.

David Waltz, of constraint propagation fame 4), is

now

spinoff of

Inc., a late-1980s vintage

MIT's AI Lab. Waltz, who

Thinking Machines,

It consists

in the blocks

world

(see

pursuing a goal similar to Kolodner's with the super-

computers of Thinking Machines,

at

it

of well-documented

patient or the salient points of a legal argument, the

chapter

less

the knowledge of experts into rules and, instead, record

distill

cases.

TMS. owe

Janet Kolodner: after modeling the

"case-based reasoning." 2 Kolodner's idea to

a logic-based

investigators are exploring reasoning techniques that

calls his

is

commercial

director of information systems

approach "memory-based reasoning." 3

of feeding each one of the possibly sixteen thousand different

processors of his company's Connection Machine with memories of a

When a new situation comes up, the Connec-

recorded case or situation. tion

Machine compares

it

to

and uses

it

to define a possible solution. Janet

closest match,

all its

records simultaneously, identifies the

has recently spent a sabbatical at Thinking Machines on a

Kolodner first try at

implementing case-based reasoning into the Connection Machine: the results are encouraging.

However, the most dramatic AI project aimed

at instilling

common

sense into computers, and certainly the best-funded one, owes logic. It

is

Douglas Lenat's Cyc

An Ontology

for

effort at

Common

little

to

MCC.

Sense: Cyc

his quest for ways to make computers learn (see chapter met with some spectacular successes. His Automated Mathematician program learned by discovering mathematical concepts; and its succes-

Lenat had, in 7),

sor,

EURISKO,

as playing

learned by inventing new heuristics for such activities games and designing computer chips. In 1984, Lenat was

240

||

MCC,

approached by

a

consortium that made up America's answer to

the Japanese Fifth Generation Project. Short for Microelectronics and

Computer Technology Corporation,

MCC was chartered by such heavy-

weights as the Digital Equipment Corporation, Control Data Corporation,

Kodak, and National Cash Register

And

decade-sized projects.

began an

MCC

so

it

was

to carry out large, high-risk

September 1988, Lenat

that in

research report as follows: "I

would

like to

surprisingly compact, powerful, elegant set of reasoning

form

a set

mon

sense reasoning

of

first

Although such

principles

—

which explain

a sort

a discovery

creativity,

present a

methods

of 'Maxwell Equations'* of thought."

would have been

a logical

outcome of Le-

nat's previous

work

into the world

and make sense of what they saw, he continued:

very

much

in

that

humor, and com-

programming computers

to figuratively

to present [those reasoning methods], but, sadly,

believe they exist. So, instead, this paper will

tell

go out

"I'd like I

don't

you about Cyc, the

massive knowledge base project that we've been working on

at

MCC for

the last four years." 4

Stemming from an admission of defeat, Cyc is

(short for encyclopedia)

a $25-million research project that will last for

two person-centuries.

Lenat had become convinced that no amount of finessing and fancy

footwork would ever facts as

let a

"Nothing can be

in

machine discover by

two places

at

itself

such elementary

once," or "Animals don't

like

The most that we need

pain," and "People live for a single solid interval of time." 5 salient discovery in

to

know

a colossal

in the world.

new

AI since the Dartmouth conference is number of these common-sense assertions

Lenat convinced

discipline

stemmed from

need to encode

this

his

MCC

to get

by

sponsors that the woes of the

repeatedly trying to wriggle out of the

knowledge manually, tedious

fact after painful

assertion, in machine-usable form.

new land without long And thus in 1 984, they embarked

"Fifteenth century explorers couldn't discover

voyages," claimed Lenat and his team. 6

on

own

their

excursion over the ocean of knowledge: during a ten-year

period, they were to encode about ten million assertions like the ones in

the previous paragraph into a gigantic knowledge base a billion bytes

Derived

in the nineteenth century

will

allow

by Scottish physicist James Clark Maxwell, the all of electromagnetism in a few crisp lines

equations Lenat refers to explain virtually

of algebra.

amounting to over

of information. Ad-hoc inferencing mechanisms

241

SOILS OF SILICON a

computer to reason from

language. Like an iceberg,

and much

evident" facts

knowledge and understand natural

Concise Columbia Encyclopedia. Below the loom the behemoth collection of "selfthat children know when they enter grade school, and that

to the contents of the waterline,

that

Cyc will have a small visible part corresponding

one-volume

larger, will

are never included in reference books. Lenat's first challenge

impose

a

workable structure onto

To work out an adequate universe,

set

this

amorphous jumble of knowledge.

of ontological categories for carving up the

MCC researchers started by lifting pairs of sentences at random

from newspapers, encyclopedias, and magazine

grammed the

was to

into

Cyc the basic concepts inherent

program could "understand"

articles.

They then pro-

in each sentence, so that

meanings. 7

their

The first two sentences took Lenat's team three months to code. They were: "Napoleon died on St. Helena. Wellington was saddened." Through a complex hierarchy of interlocking frames, the researchers were able to impart to Cyc the knowledge that Napoleon was a person.

To

Cyc, a person

is

a

member of a

make up

Agent. Persons

a subset

denoted by the frame Individual-

set

of the larger category of Composite-

Objects: those objects that have a physical extent (mass) and an intangible extent (such as a mind).

The

set

CompositeObject

IndividualAgent states that the habit of dying. Death, in turn, as

members of this is

a subset

one of its properties TemporalExtent

this

means

that

when a person is

to Wellington, other properties

set

also comprises

A slot in the frame

such things as books, which have mass and meaning.

have the unfortunate

of the frame Event, which has

(indefinite, in the case

dead, he or she stays dead).

of death:

Going back

of IndividualAgents are those of har-

boring beliefs and emotions. Sadness

is

an element of the

set

Emotion

and frequently accompanies death. Invoking further combinations of frames, the researchers could also convey to

doing

battles

is

Cyc

that if the time

of

long over, even the death of one's archenemy can

sadden. This involved talking about war, France, and England (which are CollectiveAgents),

and news media (Wellington probably learned

about Napoleon's death through the newspapers). islands

was needed

to explain St. Helena,

out the nature of land,

For

a while

it

sea,

which

A

description of

in turn required spelling

and water.

appeared to Lenat's team that they would be stumped

by the philosophical objections of Hubert Dreyfus: every sentence required the definition of a categories,

and

reality

new and

seemed

arbitrarily

long chain of related

to branch out into an

infinitely large

242

Al

number of unrelated a stage that

Lenat

concepts. In September 1987, though, Cyc reached

calls

define systematically

"semantic convergence":

new concepts

it

became possible

researchers (or "Cyclists," as Lenat dubs them) could by then enter

knowledge

in

Cyc by locating

modifying the copy. As a

similar

faster than originally estimated.

1990s,

we can

transition

would

from

answering Cyc's questions about

Lenat

that

difficult

entry of assertions

texts; the role

of humans

of brain surgeons to

tutors,

sentences and passages." 9

upbeat about the future of Cyc, and believes that

is

new

and slighdy

Lenat hopes that "around the mid-

by reading on-line

transition

it,

knowledge could be entered

more and more from manual

to (semi-) automated entry in the project

knowledge, copying

result, said Lenat, 8

to

terms of other concepts. Cyc

in

it

has a

good chance of serving "as the foundation for the first true AI agent. No one in 2015 would dream of buying a machine without common sense, and more than anyone today would buy a personal computer that .

.

.

couldn't run spreadsheets

[or]

The opinions of other AI

When

I

researchers about Cyc vary considerably.

discussed Cyc with him, Marvin Minsky was enthusiastic about

the project.

He

agreed that Lenat was on the right path, and regretted

were no other projects

that there

Minsky

word processing programs." 10

travels

Most other

from time

I

MCC

facility in

Austin, Texas.

Cyc

going to work,

researchers just don't believe that

but can't help being fascinated with

on Cyc

Cyc. Himself a part-time Cyclist,

like

to time to the

it.

is

Witness a technical presentation

attended at the 1990 American Association for Artificial In-

meeting in Boston: it was scheduled in an otherwise-dull on knowledge representation which attracted few listeners. Yet the AAAI organizers are sticklers for punctuality, and everybody knew that the talk about Cyc would start at 3:15 p.m. sharp. Just as Cyc's co-director, Ramanathan V. Guha, was climbing the podium, the small auditorium suddenly filled to capacity with about two hundred deserters telligence

session

from other simultaneous presentations. After

listening to

using up the question period to throw several barbs representing default knowledge, they prompdy

at

filed

Cyc's

Guha and method

out of the

for

room

again.

The problem Lenat and

is

that

Cyc

is

the quintessential scruffy

Company make no bones about

of the AI community," said Lenat,

".

agonizing over the myriad subtleties

.

.

it.

"We

in that

of, for

AI

differ

we have

project,

and

from the

rest

refrained

from

example, the different for-

malisms for representing pieces of time; instead,

we have

concentrated

243

SOULS OF SILICON

on using

we have

the formalisms

for actually encoding information

about various domains." 12 What he meant was that instead of worrying about

why Cyc

AI people

couldn't be done, they just

went ahead and did

are spending their research lives creating

"Most

it.

bumps on

logs,"

Lenat said elsewhere. 13

Hans Moravec, who was

me

said to

that

a graduate student with Lenat at Stanford,

Cyc has been labeled

in the grapevine as a "half-serious

attempt." 14 Randall Davis, another of Lenat's fellow graduate students,

doesn't grant Lenat substantial

more than an "outside chance" of capturing

Newell both pointed out to tion

a

body of common-sense knowledge. 15 David Waltz and Allen

mechanisms

in Cyc:

related concepts in the

me

16,17

the ad-hoc character of the representa-

as a result,

Cyclists

same way, which could make

them

the knowledge base to associate rect the situation

two

may not encode it

impossible for

afterward. Lenat intends to cor-

by having learning programs crawl over the knowledge

base at night and set

it

but Newell was pessimistic about the

right,

outcome.

But the of

of the

attitude

total rejection, as

Cyc

is

It is a

toward Cyc

artificial intelligentsia

Randall Davis

summed

it

up

for me:

moon

or not.

Some-

is

going to make

come out of it.

whose overarching goal

is

.

.

.

it

to the

There are people

thing to work.

and

.

.

.

is

causality for millennia,

and they are

still

arguing about

attitude that until they get the

answer

we're not going to be able to build an intelligent system.

look around and see that the planet systems [ourselves],

if

we

just build

it

who

and so in a

learn a lot!" This

to get the

Philosophers have been dealing in theories of time,

you can take the

space, causality,

in the field

to get the theory right. That's important,

but there have to be people whose fundamental goal

space,

not one

vast experiment in absolutely hard-core empirical AI.

not a rocket ship that

thing important will

Now

is

is

is

it.

right,

Or you

can

populated by semi-intelligent

have only the barest theory about time,

forth.

You

can

justify

way that is good enough what Lenat

is

doing.

Cyc by

saying:

"Maybe

for the time being, we'll

244

Al

AND PSYCHOLOGY

Al

Where Cyc embodies humanistic flavor.

From

early on,

a

much more

Al researchers have maintained

tions with their psychologist colleagues, results.

com-

the pragmatic, blue-collar approach to the

mon-sense problem, other avenues of investigation have

rela-

sometimes with productive

The examination of human thought processes

occasionally

helped scientists design replicas of them into machines. Conversely, findings

made without

human cognition have helped psyhuman mind better. A science of universal regrouping both people and machines, may well be in the reference to

chologists understand the

psychology,

making. finally

It

might help us answer such questions

as:

When

machines

think and manipulate their environment, will our minds and theirs

common? Will our basic

have anything in

ground with

drives and purposes share any

theirs?

Modeling Human Problem Solving We

saw

in chapter 3

how NeweU and Simon implemented

into their

General Problem Solver reasoning techniques abstracted from tape recordings of actual problem-solving efforts by

Another

early

ceiver rize

later

His program

And Memorizer)

meaningless

subjects.

example of psychological modeling was the work of

Edward Feigenbaum, who (see chapter 6).

human

pioneered commercial expert systems

EPAM

(an

acronym

for

Elementary Per-

mechanism by which we memoBy so doing, Feigenbaum hoped to unearth

simulated the

syllables.

18

principles applicable to the entire learning process. Perhaps

the influence of behaviorism (see chapter

phenomenon

2),

the

first

member of

under

Feigenbaum studied the

as a stimulus-response situation.

sented a subject with pairs of monosyllables.

still

An

experimenter pre-

The experimenter

stated

the pair (the stimulus), and the subject tried to

answer with the second

member

(the response). Behaviorist theories did

not properly account for some of the phenomena that occurred in such experiments, like oscillation: a subject learned a sequence of syllables correctly,

and then forgot them and learned them again many times.

Neither could behaviorists explain

why learning new associations made phenomenon called "retroactive

the subject forget previous ones, a inhibition."

245

SOILS OF SILICON Explaining these

phenomena through

procedure a behaviorist

a

would never contemplate, Feigenbaum postulated a complex information structure in the subject's mind and proceeded to build a computer

model of it. This

structure, called a "discrimination tree,"

learning process.

assumed

Feigenbaum worked out of

that subjects didn't bother

instead, extracted salient features

and

letters)

tion used

built

no more

already learned.

assumptions

of the

he

entire syllables but,

syllables (for

example, the

features.

The

first

discrimina-

features than those required to associate the pairs

Feigenbaum found out

in his

a very simple idea:

remembering

up associations between these

grew with the

implementing these

that by

program, he could reproduce the inexplicable phe-

nomena. For example, suppose the program had learned to associate the

JIR-DAX by

syllable pair

starting with/,

new

I

simply remembering "any time

must respond with DAX." Then,

pair JUK-PIB, the

program

suffered

from

if

I

retroactive inhibition for

the following reason: Since the discrimination rule could

JIR and

more

JUK

apart, the

program had

syllable presentation

syllable

to develop a

were needed. During

was "forgotten" because access

to

it

see a syllable

presented with the

new

no longer one. For

this time, the

was

tell

this,

response

lost.

Generalization common ground

Yet another

between psychology and AI concerns the

problem of generalization, about which psychologists have always

How

speculated.

do we manage

to extract global concepts out

of the

we constantly face? How do we learn about furniture and plants, when we see only individual chairs or trees? On a more abstract level, how do we identify a given instance of interpersonal behavior for example, a conflict or a betrayal? AI made a direct use torrent of specific instances

—

of psychology's response to these questions.

We

make

turing our

sense of the world,

knowledge

in the

some

psychologists assumed, by struc-

form of prototypes, or schemata. For

example, the prototype for a chair was an information structure containing a generalized description of statement:

"A

chair

is

it.

something to

It sit

somehow corresponded

a seat,

and

cepts.

Although the philosopher Immanuel Kant

a back." Similar descriptions existed for

schema in his Critique of Pure Reason, published in in the

realm of philosophy

to the

on, usually including four legs,

1

more first

abstract con-

used the word

787, the idea remained

until the twentieth century,

when

the

new

246

41

science of psychology took

up.

it

19

Sir Frederick

psychologist, used schemata in his 1932 studies scribe

how we

remember oudine

first

For example, of

a

it

which

are

common

The Swiss

in his theory

its

One remembers

schema and the

typi-

a given story as a

particular traits that

on

psychologist Jean Piaget elaborated

made

stand

it

the idea and used

of mental development. 21 The German philosopher and

Max Wertheimer

psychologist

we

to several stories.

corresponds to one schema ("Valiant prince

Western make up another.

superposition of out.

features,

a fairy tale

to de-

essence of a story. 20 According to Bartlett,

princess from bewitchment"). The main sequence of events

frees cal

recall the

Bartlett, a British

on remembering

incorporated schemata into the Gestalt

theory of perception. 22

The concept remained vague, however, and experimental psychologists could never make any concrete use of it. When Marvin Minsky introduced schemata into AI in his 1975 paper "A Framework for Representing Knowledge," he called them "frames" in order to freshen up the tion.

23

As we saw

income

tax form,

its

some

generic frame,

slightly stale psychological

earlier (chapter 7), a

main

Minskyan frame looks of

feature being a set

slots are

slots to

fill

no-

like in.

an

In a

empty, and others contain default entries

corresponding to usual or necessary characteristics of a concept. For example, the frame for chair has

number of legs holds the number tion

Another

to sit on.

more

slot

4.

shows

slots labeled seat

The

back.

The slot men-

that a chair

is

a specific case

of a

general frame called piece offurniture. In the generic case, other

— remain empty

slots

— such

fills

them with appropriate values when applying

cific

and

slot purpose contains the

as

color, weight, material, height,

position

7 .

One

the frame to a spe-

instance of a chair.

In AI programs, frames serve several purposes.

One

can use them to

recognize objects or concepts ("It has four legs, a seat, and a back: what is it?").

are. They can help reach Look for something with four legs, a seat, Frames allow AI programs to make inferences ("It's a

Frames can keep track of where objects

goals ("Need to rest your legs?

and

a back").

chair,

but

I

can see only two

legs: the table

corner probably hides the

other two"). Schank's scripts (chapter 7) are frames designed for the express purpose of inferring what stories leave untold ("If John ordered a

hamburger

in a restaurant, he probably ate

it,

even

if

the story doesn't

say so.")

Not by

coincidence, studying the role of prototypes in intelligent

behavior has developed into a growing

new

field

of psychology.

Categori-

247

SOULS OF SILICON Ration

and

prototypicality effects are

now new

research avenues in this

science.

Tools for Psychological Investigation The

influence of AI

on psychology extended beyond providing general workings of intelligence. AI has also influenced the

about the

insights

everyday practice of psychological investigation. Because of AI, psychologists

now have new instruments

for prying

open the

secrets

of the

mind.

The

first

one

is

a tool for expressing

many of

their theories in un-

ambiguous terms: the computer program. Formulating precisely that

it

will

run on a computer forces crystal

puters tolerate neither

a hypothesis so

clarity

on

it.

Com-

hand waving nor fuzzy thinking and keep you

honest. Unrealized, implicit, or unspoken assumptions are mercilessly

weeded

out.

Weaknesses of construction appear

as if under a

magnifying

glass.

Second, running theories on a computer allows psychologists to

The computer has a substantial edge over the As MIT's Patrick Winston facetiously pointed out to Pamela McCorduck, a computer requires little care and feeding and does not bite. 24 As a more considerable advantage, computers give psycholoevaluate

them

laboratory

better.

rat.

gists a capability

isolate

up

so far restricted to the hard sciences: the ability to

phenomena.

their

Physicists

and chemists have always been able to

experiments so as to study only one factor

to investigate gravity? Plot the impact speeds their falling heights; see

of objects

whether heavier objects

ers,

one could hardly apply

similar

Keep

it

methods

Yet

all

this isolation

other mental

of factors

memorization of meaningless

is

as a function

of

You want vacuum. You think

constant! Before

comput-

in psychology. It

tremely difficult to isolate a particular aspect of a

independently of

set

You want

fall faster.

to cut air friction? Carry out the experiments in a

temperature might have an influence?

at a time.

was ex-

mind and study

it

activities.

precisely

syllables.

what Feigenbaum did

He

riations in a hypothesized discrimination

could study

for the

how minute

va-

procedure affected learning.

He never had to worry about subject fatigue or random fluctuations. The writer Pamela McCorduck aptly captured the advantage by calling AI programs "parts of intelligent behavior cultured in silicon." 25

248

Al

Cognitive Science Through ever-increasing intermingling, intelligence

which the two a discipline

from other

the boundaries between

and psychology have grown fuzzy. The blurred of

fields its

blend into each other

is

science."

as

With help

(anthropology, linguistics, philosophy, and neuro-

science), cognitive science aims to explore the nature

The Sloan Foundation favored

the mind.

along

even starting to emerge

own, under the name "cognitive

fields

artificial

line

and functioning of

the emergence of cognitive

science through a multimillion-dollar financing effort in the 1980s.

The main

cognitive science research centers are

Diego. Others exist

MIT,

Berkeley, and San

Carnegie Mellon, and the University of

at Stanford,

Pennsylvania.

How science,

does one distinguish between the disciplines of AI, cognitive

and psychology? At the

here

fields,

is

a try at telling

risk

them

of offending researchers

apart.

building thinking machines, while psychology studies

and

feel.

Cognitive psychology

tive science is a

in

Roughly speaking, AI

tries to learn

how

how

about

people act

people think. Cogni-

meeting ground between AI and psychology.

These differences

in goals

frequendy correspond to differences

methodology and philosophy, often with misunderstanding, and even contempt between workers gists,

three

all

is

in different disciplines.

in

distrust,

Psycholo-

for example, mostly interpret the workings of the mind through

meaning. AI workers, instead, look to the mechanism of the process

The

involved as an explanation.

psychologist Sherry Turkle of

gives the example of a typical Freudian

opens a meeting by declaring

would read

feelings

meeting to be over.

it

closed.

An AI

may proceed

symbol back,

which

chair's

hidden wish for the

worker, on the other hand,

in the

may remark

that

their opposites in similar ways:

same manner. Minute

like flipping the sign bit

MIT

a chairperson

Into this lapse, psychologists

of uneasiness and the

computers often encode meanings and the brain

26

slip, in

errors in reading the

from positive

to negative,

may

turn "open" into "closed."

The

psychologists Noel Sharkey from Stanford and Rolf Pfeifer

from Zurich University point out another

difference:

AI and psychol-

ogy section cognition differendy. 27 Psychologists investigate long, horizontal sections of cognition. Sharkey and Pfeifer cite as a typical exlogogen model or word recognition. word of our vocabulary corresponds in

ample the psychologist Morton's

Morton postulated

that each

249

SOILS OF SILICON our mind to a specific word detector, or structure of these detectors,

of word perception

in several contexts

experimental effects from

model applied both cases to

it.

to written

why we

An AI

work, since

logogen.

Morton applied them

many

Elaborating on the to the

phenomenon

and explained a wide range of

different sources.

For example, the

and spoken word perception:

it

explained in

can recognize a word faster after a recent exposure

worker might complain about the lack of generality of it

concentrates

on only one

this

part of our understanding

processes: the word.

For an AI researcher, generality involves processing section of cognition that

would look very

a deep, vertical

thin to a psychologist.

For

example, Roger Schank's story-understanding programs involve everything from parsing sentences to explaining the behaviors described. Yet a psychologist

may

find this

stories in a limited field

AI workers and

work too

restrictive

because

it

handles only

of understanding.

psychologists frequently emerge from different back-

grounds and pursue different

They often deeply

goals.

mistrust each

work and apply adjectives like "naive," "slipshod," and "irrelevant" to work performed in the other discipline. Psychologists often decry AI workers as amateur psychologists, and vice versa. Cognitive other's

scientists try to reconcile

of both

opposing viewpoints, often to the disapproval

sides.

Despite these differences, an uneasy alliance has formed between AI

and psychology

in specific fields.

Researchers recognize that work on

sensory perception can be good experimental psychology and good AI.

They can match computer program ior

results

of subjects with such accuracy that

many

instances, researchers even

know

neuroanatomical structures in enough modeling. Higher-level cognitive

and the experimental behav-

litde

argument

is

possible. In

the workings of the pertinent

detail to

permit specific hardware

activities like

conscious thought or

language processing, on the other hand, are too remote from experimental test

to allow such research. It

is

there that

AI may have

to precede

psychology. Hypotheses formed through computer modeling could

then enable the design of experiments specifically aimed

Is If

the

at testing

them.

Human Mind Unique?

computers can model our mental

activity in different

proved that our minds are not unique? Could processes

ways, have

we

radically differ-

250

Al

ent from the ones in our brains lead to cognition? Such speculations led cognitive scientists to investigate the question of if

minds can

apply to

all

differ

of them, whatever

mental properties aliens

from each other,

from other

common stars?

to

The

their

mind

in general.

Even

are there general principles that

make up?

Is

it

possible to extract

humans, thinking computers, or even

investigation of this question has barely

begun, and scientists can offer few conclusions. However, the very questions they are asking expose the flavor of the research.

The more

abstract question deals with the continuity of the space of

minds. Can one say that the set of

possible

all

minds forms

a

smooth

slope? Is there an unbroken rise starting at bacteria and thermostats and

leading to the

human mind

The sketchy results From what we see, the path to

or intelligent computers?

available point to a negative answer.

higher minds involves long stretches of cliffs.

In the 1950s, John

von Neumann

ground broken by soaring

flat

identified

one of these

cliffs

and

Beyond this level of organization, he 28 said, a being can make another one more complex than itself. Yet another step-increase in mind power is the phenomenon of cognitive called

it

the "complexity barrier."

convergence, which Douglas Lenat claims to have achieved with the Cyc project. Cyc's designers

now

have

fed into

be able to explain most new entries

Other questions cognitive gence inherited?

How much

in

it

enough

terms of

scientists ask are:

can one develop

basic concepts to

earlier ones.

To what it

extent

after birth?

answer stems from the position that intelligence

is

29 is intelli-

One

partial

mostly knowledge:

the very act of reasoning requires the application of techniques and steps to learn. Some speculate that the structure of the somehow determines what we can learn, and experience determines what we do learn. If this is the case, we would mosdy inherit intelligence. Others claim that we are all born with the same basic abilities. The specially bright people are those who, accidentally or that

humans have had

brain

otherwise, learn in their youth ways of making their learning

more

efficient.

Yet another all

issue concerns

AI programs,

domain

limiting their fields

limitation. This

of competence to narrow domains.

For example, an expert system for

electric

nothing about diesel engines or railroad

workers claim that

Some

in practice

point out that

we even

weakness plagues

cars.

locomotive repair knows

Perhaps

in self-defense,

AI

people are also severely domain-restricted. label each other according to

our domain

restrictions, or specialties: salesman, accountant, lawyer, cook,

and so

251

SOILS OF SILICON on.

From making

observation to claiming domain restriction as a

this

general property of intelligence

is

and many have taken

a small step,

it.

Were they right? The future will tell. Anyone who has had a hand at programming computers knows about program loops. They occur when a weakness in programming logic causes the computer to repeat the same steps endlessly. AI programs are not it

immune

to loops,

and debugging one

from going into ever wider loops.

First,

is

often a process of keeping

one

identifies trivial errors that

induce the repetition of short sequences of instructions: these are easy to

One

fix.

then discovers that sometimes a subtle conjunction of

cumstances makes the program jump back ecuting

many

instructions.

The reasons

for this behavior often

of the program. Devising additional program

in the structure

detect offending circumstances and prevent looping

may be

We

know

all

people

managed

to

who

it

is

so

common

that

classical diagnosis

of

a

woman who

marry not one, not two, but three cancerous husbands

she attended on their death beds. 30 Granted the pathological

program loops? Often the

common

occurrences be

cycles last several years. Their victims

well act out of healthy and natural inclinations. Since one's basic

disposition remains the same,

same kinds of people a

they occur

compulsive acting out of unconscious

as a

made one such

character of such extreme cases, couldn't the

may

tests to

endlessly repeat the same, often self-

of behavior. This phenomenon

psychologists interpret wishes. Freud

just

when

deep

lie

major undertaking.

a

destructive, pattern

whom

cir-

to an earlier step after ex-

little

it

sooner or

will

situations. Like the

one

in the

—

may simply lack an appropriate loop-detection mechanism like man on one's shoulder, for example, who at the beginning of a

new loop would

detect analogies with the present situation and

equivalents in past cycles. But since each loop

the one before, and similar situations several years,

there

later place

computer program above, these

you

might take

it

are!

activities tinuities,

attributes

all

it

again!

that such monitoring

minds, natural or

Invoking such a unsatisfying.

little

its

from

may occur only at intervals of little man indeed to say, "Look, Stop it!" As we shall soon see,

make up the major part of our mental domain restriction, and the tendency of

slightly different

a very clever

You're doing

Marvin Minsky surmises

is

man

and error-prevention

processes. Like disconto loop, they could be

artificial.

to explain mental processes

For what accounts for the thought processes of

is

deeply

this little

2

M

52

,.

Of

.

.,-

r:iir

7"

'i.i.

" :

i_

253

SOILS OF SILICON

(author of the MicroPlanner language, itself an early experiment in

decomposition), recalled

for me:

it

more

Kristen Nygaard and Ole-Johan Dahl* were inspired to develop

modular ways to program guages

FORTRAN.

like

.

.

simulation than existing If

.

ways using

you were going to simulate

lan-

a car wash,

you needed an array of variables over here to keep track of which

cars

had been washed or not, you'd have an x-y-^ coordinate array over there that kept track of

and

what the positions of the automobiles were,

piece of code to put

this spaghetti

together.

it all

So they

said,

"Hey, we've got objects out there: automobiles and car-washing stations. bile

So

let's

have each object keep track of itself. Let an automo-

keep track of whether

it's

clean or dirty, of

what

its

position

is,

Each washing station will keep track of how much soap and water it has." ... [In this way], they were able to do

of how long

it is.

simulations in beautiful, elegant fashion. 35

Special-purpose programming languages like

SIMULA

and Small-

Talk appeared, which implemented these concepts. Their basic philoso-

phy was

for the

programmer

to describe not

using computer data structures, but

who

anisms built into the languages then

let

what the objects did by

they were; specialized mech-

these creatures interact as their

natures dictated.

The advent of computer networks provided

the clinching motivation

for developing software-enabling multiagent interactions. Instead

concentrating

all

their data processing in a single

corporations found ers,

and

locally.

later

it

more

practical in the

1

of

mainframe, large

970s to have minicomput-

microcomputers, perform whatever operations they could

The machines then

transmitted only the required information to

each other. Pretty soon they themselves started gossiping over telecom-

munication mation,

lines like

unquenchable busybodies:

air traffic

control infor-

and electronic fund

transfers

now make up

airline reservations,

a sizable fraction

of long-distance communications. Protocols such

Message-Passing allow the orderly and error-free cooperation of processors. Carl Hewitt formalized their analysis in a

computer science

By

called

mind was

*Inventors of the language

new branch of

"Actor Theory."

coincidence, at about the

ing minds within a

as

many

same

time, the idea of multiple cooperat-

also gaining

SIMULA

at the

ground

in

psychology

— owing

Norwegian Computing Center

in Oslo.

254

Al

from Freud's model of the brain

in part to psychologists' shift

of Norbert Wiener tion,

of cybernetics

in his theory

(see chapter 2).

he and psychologists after him believed, better describes the

of thought than energy does, and

this

is

stuff

what the brain receives from

outside. Information, contrary to energy, does not build fact,

to that

Informa-

up pressures. In

the law of conservation does not even apply to information:

necessary, tion than

one can destroy

it

was

gists,

releases.

The

if

and the brain can receive more informa-

data,

brain's

main

concluded these psycholo-

drive,

and form

to organize information into coherent objects

between them. Thus was born the psychology of internal-

relationships

object relations.

Freud, to his credit, had already described a process by which "take in" people to form inner objects.

phenomenon

pathological

—

He

first

internal-object formation as a part of

we

as a

Freud came to think of

normal development. Later theo-

the Austrian psychoanalyst Melanie Klein, widened this view,

up

setting

it

the reproaches of an internalized father,

for instance, leading to melancholia. Eventually,

rists, like

conceived of

as a base for

mental

life

the relationships

we have

with our

inner images of people as entities within the mind. 36 In the words of the psychologist

Thomas Ogden,

these agents were "microminds

scious suborganizations of the ego capable of generating

experience,

i.e.

.

.

.

uncon-

meaning and

capable of thought, feeling and perception." 37

though many psychologists do not accept

this

view

in

its

entirety,

Even most

admit that one does build up one's personality by identifying with other people or fragments of their personalities.

A Society

of

Mind

In 1986, Minsky published The

Society ofMind, in

cept of subminds into a sweeping attempt ligence.

scruffy

The

which he

at explaining

built the

con-

human

intel-

38

In a sense, The Society of Mind represents the full blooming of the movement away from pure logic as an explanation of the mind.

scruffies

a gadget,

hold

that, in the

words of Daniel Dennett, the mind

an object that one should not expect to be governed by deep

mathematical laws, but nevertheless a designed object, analyzable tional terms:

.

.

.

.

.

The mix of will

in func-

ends and means, costs and benefits, elegant solutions on the

one hand, and on the other, shortcuts, .

is

be discernible

in the

jury rigs,

and cheap ad hoc

Rube Goldberg found elsewhere mind as well." 39

elegance and

fixes.

in nature

255

SOILS OF SILICON

own mix of elegance and ad-hoc fixes is reminiscent of the hive-mind theme of science fiction. Our minds, claims Minsky, are made Minsky's

up of

a billion entities,

dumb and know

which he

calls

"agents." Individual agents are

only one function. They constantly monitor inflow

from the senses or

produced by other agents. They perform

signals

whatever action they are capable of upon recognizing

Mind

tion signals. conflicting,

results

their

own

from the simultaneous and often

and disorderly action of agents. Structured

activa-

tangled,

as a loose hierar-

make up specialized systems called "services." A high-level may be capable of quite advanced tasks, and terms usually

chy, agents service

reserved for a whole person might apply to

it.

In an example strikingly suggestive of the block-manipulating robot in the

Micro Worlds

ate in the this

mind of

hierarchy

(call it

nate functions:

Minsky describes how agents might oper-

project,

The

a child building block towers.

BUILDER) knows

only

how

highest agent in

to call three subordi-

BEGIN decides where to place the tower, ADD piles up

the blocks to build

it,

and

END

decides whether the tower

enough. Each one of these functions depends on

its

own

high

is

subordinates:

FIND a new block, GET it, and PUT FIND, GET, and PUT, in turn, activate their own suband so on down to the agents that eventually control eye move-

those of ADD, for example, will it

into place.

agents,

ments or

Some

activate the muscles causing the child to pick

find this

ill-defined tissue

clear that

model too formal

of interlocking meshes of neurons

one can

up the blocks.

a description for the diffuse in the brain. It

isolate individual neural structures

is

and not

corresponding to

Minsky's agents. Yet the model provides a surprisingly faithful account

of many mental phenomena. Here questions

it

a brief sampling

of some of the

answers:

Why do babies all

is

seen a baby

suffer

move

in

sudden and

drastic changes in

humor?

We have

seconds from a contented smile to tears of rage

or hunger. Answer: early minds consist of few independent services

geared to satisfy our basic needs. other,

services

make up

When

control passes from one to the

mood

result.

More complex,

the adult mind:

mood

shifts are less drastic

sudden changes

in

tightly linked

and take

longer.

How does

the

mind grow?

How do we learn?

agents and services and include

when

a structure

crowded nearby

becomes too

tissue.

them

Answer: we form new

in existing hierarchies.

large,

it

However,

can no longer expand into

In addition, other services

come

to

depend on

256 it,

Al

and further changes might disturb

When

brain.

explains

it

is

why we

their activities.

new

ready, transfer control to the

learn

new

The

solution?

structure. This

bursts of rapid progress separated

skills in

A

by stretches of slow development, or "learning plateaus." corresponds to the construction and testing of bursts of progress occur

How

when

do remembering,

new

a

recall,

structure

in several services.

when we have

a

These "knowledge

line associated

new

plateau

structure.

The

suddenly switched on.

specific aspects

lines," as

Minsky

new experience. For example, our

might activate the agents for

knowledge

is

a

and associations occur? Some agents

between other agents, activating

act as links

Copy

improvements into another part of the

the structure with any required

first

sight

round, fist-si^ed, smooth-skinned,

new concept of apple

with the

of a concept

calls

them,

arise

of an apple

and

will

red.

The

afterward

link, framelike, these properties together.

Like Freud's, Minsky's model of the mind assigns an important role to unconscious

mechanisms. In Minsky's view, avoiding mistakes

new

important as learning "suppressors," ated

when we

come

first

skills.

into being

is

as

Special agents, called "censors" and

when we

blunder.

The suppressor

cre-

put our fingers in a flame detects our intention to do

so the next time around and prevents the action. With time, this suppressor evolves into a censor, which prevents us from even thinking of

touching a flame. Censors speed up our mental processes: to reach an object

on the other

side

of

a flame,

option of going straight for

we

don't lose time considering the

it.

Censors are exceptional agents for two reasons. definition invisible. Faithful to their purpose of

our consciousness, they stay carefully outside in the sense

First,

they remain by

unburdening the

it.

field

Second, they are

of

large,

of requiring much knowledge and processing power. For

example, the censor against touching flames should catch than our intention of touching a flame:

it

much more

must detect the many more

circumstances that might lead us to even think of touching a flame. This

watchdog might give

activity requires a close rise to

monitoring of the many agents that

such thoughts. If

new

circumstances occur that lead

us to think about touching a flame, the censor must learn to recognize

them

also. Invisible, large,

and growing, censors

are the black holes

of

thought.

Unlike Minsky's previous contributions about Perceptrons and frame theory, the Society of

Some thought Minsky

Mind was

—

as

was

given a cold shoulder in AI

his right as

circles.

one of the originators of the

257

SOULS OF SILICON idea

— had simply renamed and repackaged

object-oriented program-

ming. 40 As Minsky pointed out to me, though, there difference

between the two concepts:

which objects-oriented programs called

handle through a protocol

typically

"message passing":

Message passing

basically a foolproof logical

is

that nothing gets lost. If a

up.

a fundamental

is

has to do with communication,

it

However,

message

of Mind

a Society

is

will tolerate errors

agents acting as managers have their

may

express progress, and

way of making through

own knowledge

decide that

sure

not accepted, the system locks heuristics:

how you

about

some other agency

having a

is

bad day and should be made to go away.

Another of the Society of Mind's problems has to do with Minsky's of leaving plenty of work to do for those

strategy

of

his

bandwagons:

applied

it

to the Society

it

It's

too

soft.

a serious proposal," Allen

.

.

The problem

.

climb onto one

when he

of Mind. This time, potential followers thought

do not consider

they just didn't have enough to go on. "I

of Mind to be

who

did marvels for frames but backfired

is

that

the Society

Newell told me.

Marvin wants to

talk

about his

agents in metaphoric language, so you are never able to feel limited they are or whether they are really a in there with

all

Many

critics feel that

interaction of ior.

human. There

the apparatus of a

in that design proposal to enable

many

how

whole bunch of litde kids

you to pin

it

isn't

any constraint

down.

Minsky does not adequately explain how the

simple components results in very complex behav-

Terry Winograd accused Minsky of engaging in "sleight of hand by

changing from 'dumb' agents to in natural language at the point

Minsky, for his part,

is

'intelligent'

homunculi communicating

of Wrecker versus Builder

leery about

becoming more

in a child."

specific.

When

41

I

asked him whether there were ongoing efforts to implement the Society

of Mind into hardware, he answered, "No, not anymore. People want to but they

keep asking

decisions like that.

I

me how

to

do

it.

I'm unable to make design

might mislead them."

Patrick Winston, Minsky's successor at the head of MIT's tory,

AI Labora-

conceded the theory's incompleteness but cautioned against hasty

judgments:

258

Al

Marvin's Society of ideas. I've

of gold

lots

of it it is

is

Mind

not a single idea, but a vast potpourri of

is

sometimes compared in

it,

exactly in

gem form some of it

fool's gold,

it

diamond or gold mine. There's

to a

and there are diamonds and there yet, half

of

it is

hidden

are duds.

in rocks,

None

some of

too low-grade to mine for a long time.

is

There's material for lots of Ph.D. theses there. So as time goes on the Society of

Mind

mark than

it is

become

will

many

in

respects even

more of

a land-

today. 42

Unified Theories of Cognition So

far in this

chapter

may have

I

given the impression that researchers

have definitely abandoned the dream of accounting for mind through a handful of simple mechanisms, the "Maxwell's equations of thought"

upon which Doug Lenat has turned

his back.

Even though

the logicians

have attacked the common-sense problem with renewed and more powerful logical techniques, can account for

all

it is

of thought.

not

at all clear that a single

Scruffies,

kind of logic

through the Cyc project, have

logic down into many special-purpose inferencing mechwhose main purpose is to sift through the mountain of handcoded knowledge where the real power of Cyc will reside. Marvin Minsky's Society of Mind makes up yet another ^//-integrated effort to

in fact

broken

anisms,

explain mind, relying as

does on the sheer proliferation of interacting

it

special-purpose functions.

Other schools of AI researchers original goal in

all its

purity

are,

and finding

in

however,

still

pursuing the

psychology the basis for the

powerful, general-purpose mechanisms with which they hope to explain thought.

The 1980s saw

the emergence of theories with the colorful

names of Act* (pronounced "Act

much

in

common

accounting for

all

in

is

.

.

.

his

and Prodigy, which have

of cognition. Indeed, Allen Newell, showing no sign

of the cancer that would

my chair in

Star"), Soar,

methodology and share the ambitious goal of

kill

him

a year later,

all

but swept

me out of my stuff

enthusiasm for Soar: "In Herbert [Simon] 's and

always the concern for

artificial intelligence

.

.

.

right there with

the concern with cognitive psychology. Large parts of the rest of the field believe that this is exactly the

to I

know whether

wrong way

to look at

it:

you're being a psychologist or an engineer.

AI

you ought

Herb and

have always taken the view that maximal confusion between those

the

way

to

make

progress."

is

259

SOILS OF SILICON

roots of Soar go back to the 1960s and the shift from search to

The

knowledge that Newell and

solution to the

his followers

never quite bought. Although

power of knowledge, they don't

they believe in the

see

as a final

it

problem of intelligence. Further, they point

out, dealing

with mountains of knowledge complicates the search problem by making

it

put

harder to identify the pertinent pieces of knowledge.

"There were attempts to move out of search, to say

it:

knowledge now!' but find

As Newell

—

lo

and behold

as

soon

one deals with

—

the search efforts continue."

as

difficult

'It's

all

problems, you'll

Throughout the 1 960s, Newell and Simon continued the experiments had led them to discover means-ends

that

analysis late in the previous

decade: they conducted psychological tests to find out

how people

solve

embody their reasoning techniques in software. much by asking people to prove theorems in logic and later studying how other subjects played chess. The crucial discovery occurred, however, when Newell and Simon asked their subproblems, and tried to

The experimenters

learned

jects to identify digits

known

corresponding to the different

as "cryptarithmetic

letters in

puzzles

problems":

SEND

MORE MONEY

+

Newell and Simon observed that different

numbers more or

less at

their subjects started

random:

this

by trying out

corresponded to pure

search behavior. After a while, the subjects discovered shortcuts that

speeded up result

their searches.

For example, they would

of the addition had more

then the

first digit

M stands for

1

of the

these

efficient use

had

in this problem).

puzzles, Newell and

more of

result

little

digits

to

be the number

Learning

how

fact,

Newell and Simon discovered that the

moved over

time-consuming searches,

a

They would which became shorter

they traded search for knowledge.

more knowledge. Experienced problem at all

(thus, the letter

Simon discovered, simply consisted of acquiring how to make

of them. In

any searching

1

to solve cryptarithmetic

pieces of knowledge and finding out

behavior of their subjects gradually

quired

realize that if the

than the numbers being added,

and arrived

continuum start off

in

which

with pure,

as the subjects ac-

solvers hardly

performed

at a solution quickly by the routine

260

II

application of knowledge. Newell and Simons's key discovery pertained to

how

One

the subjects stored their different pieces of knowledge.

could give a detailed account of a subject's performance, they found out,

by assuming that he or she kept available knowledge fragments long-term memory, more or

of IF

.

.

THEN

.

rules (as

independent of each other,

less

formulated the knowledge

I

reaching the solution for M).

was

It

in the

I

in

form

applied in

precisely this observation that led

Edward Feigenbaum, a former student of Newell and Simon, to implement IF THEN rules into the first expert system over at Stanford .

.

.

(see chapter 6).

At the beginning of the 19"0s, Newell and Simon had analyzed thinking into two basic

abilities:

the

ability*

and the

different solutions to problems;

pertinent fragments of knowledge as IF

to search

—

that

.

.

THEN

.

is,

and

ability to store

rules, in

out

try

retrieve

order to

speed up the searches. In 1972* Newell and Simon published their findings about the strong role of production systems (another sets

of IF

.

.

.

THEN

rules) in a

name for Human

one-thousand-page book called

Problem Solving which had taken fourteen years to write. ("I'm glad

Simon confided

didn't have to earn a living as a writer!" Herbert

upon remembering different interests,

that

was

it

would become

in a fairly

this period.)

Soar.

left to

As

43

From

Newell to

I

me

to

then on, as Simon pursued refine the theory

often happens in science, Soar

roundabout manner. "The Soar

effort

is

of cognition

came about

an outgrowth of

another project called the Instructible Production System," Newell told me.

That's one of the few efforts I've been associated with that turned into an out-an-out total failure. tion systems

would have

programmed.

\\"e

The

idea

to be educated

w as r

that very large produc-

from the outside rather than

addressed the issue of

how

to build a production

system which could be instructed without knowing in

was

in

grated,

it.

it

The

project never

went anywhere

at all

detail

— but

as

it

all

that

disinte-

gave birth to some major achievements.

These include the OPS-5 production system language; John McDermott's

XCON,

the

first

commercial expert system; and Soar, which

Newell assembled with two graduate students, John Laird and Paul

Rosenbloom. '"What was missing

was an organization

in the Instructible

Production System

for putting tasks together," continued Newell.

261

SOULS OF SILICON

We needed to identify knowledge relevant to particular situations, and things like implementing operators and

do

was when we

functions. This

to be central in

represent

implemented

it's

Soar:

it

was the

this

through the principle of universal sub-

when you run out of

a device for recognizing

providing a

new

As

example of subgoaling, suppose Soar

a simple

memory would moves

like

tell it

what moves

the game, though, Soar

allowed moves, without

up

Soar

new

as a

available.

the

full

.

.

are legal

.

is

playing ticktack-

THEN rules within Soar's

and perhaps suggest useful

may have its

to

some

stage in

choose between two or more

rules expressing a preference for

rises to this situation, called a "tie

goal the selection of the best

any

move

impasse," by setting

move among

the alternatives

This selection becomes a separate subproblem, upon which

problem-solving power of Soar can be brought to bear.

"You "They

A set of IF

blocking the opponent whenever possible. At

in particular.

gas and

opportunity for more knowledge.

human opponent.

toe against a

we could own problem

realization that

these other tasks simply as searches in their

all

We

spaces. goaling:

making

computing evaluation

discovered the gimmick that turned out

see, that's

the magic of production systems," Newell said.

are self-selecting systems, in

which the

rules themselves say, 'I'm

relevant to that situation.' " Soar contains standard rules for resolving

come

impasses, which as is,

out

situation.

soon

as a situation has

tie

been tagged

—

would tell Soar to look ahead that moves and see which leads to the most desirable This strategy would remain the same whether Soar were

an impasse. In try

into play as

all

this case, the rules

the

playing chess, checkers, or ticktacktoe: therefore, universal subgoaling potentially allows Soar to play

any game.

Universal subgoaling was the subject of John Laird's doctoral dissertation;

44

Paul Rosenbloom's thesis

mechanism chologist

for learning called

George

initially

concerned not Soar, but the

chunking suggested by the Harvard psy-

Miller during the 1950s in his

famous paper on short-

term memory. 45 Chunking consists of tying existing notions into a new bundle that

itself

we chunk seven In his thesis,

becomes

digits

a single notion:

we do

it

for

example when

under the heading of a person's phone number.

Rosenbloom was

able to provide substantial evidence for

the presence of chunking at the heart of learning. 46

He skill

did

it

as follows.

— whether

it's

We all know that the only way to improve a new

typing, skating, or speaking a

new

language

—

is

262

Al

through practice. Since the 1960s, experimental psychologists had been

which we perform the new task some fractional power (say, the square root) of the number of times we've done it before. They called this phenomenon the "power able to determine that the speed with

increases as

law of practice" but were if

at a loss to explain

learning occurs by chunking, and if

constant

rate,

then

improve

skill will

it.

Rosenbloom showed

we perform

in a

manner

this

chunking

dictated by the

that at a

power

law of practice.

When

these results were

somewhat

in,

Newell, Laird, and Rosenbloom realized,

to their surprise, that Soar provided a ready-made

was only necessary

for incorporating chunking. It

As

I

indicated, these

happened only

of pertinent knowledge was

available. In the ticktacktoe

when none of its

reached an impasse

where no

in situations

rules could

mechanism

to exploit impasses.

tell it

direct piece

example, Soar

which move to

make. Even more to the point, Soar already had the means for discovering the missing piece of knowledge. After evaluating the available ticktacktoe moves, for example,

it

knew which one was

for Soar to have the ability to learn, this

knowledge by means of

new IF

a

described the situation that had given

and prescribed the action taken

best.

Thus,

order

in

had to be enabled to remember

it

.

.

rise to

as a result

.

THEN

This rule

rule.

the impasse (the IF part),

of the ensuing search

in the

THEN part. For example, a new ticktacktoe rule might say: "IF you can play either a corner or the center, THEN take the center." In later know which move

games, Soar would

to make,

and no impasse would

occur.

Newell recalled to

I

couldn't believe

still

how

easy

it

turned out to be; as he

me:

was out of town when John and Paul decided

learning for one

discovered

how

little

to

task.

make

it

And when

general.

to try

and program-in

they implemented

So when

I

came back

it,

after

they

two

or three days, chunking was in and working in a general fashion

through that tight

all

of Soar.

had

a

few breakthroughs It

in

my

career, but

brought everything into a very

knot in which problem spaces, production systems, impassing,

and chunking are a

I've

was the most dramatic one. just

one

ball

complete cognitive engine.

of wax. All of

a

sudden Soar became

263

SOULS OF SILICON

These events happened

in January 1984. Since then,

into a multidisciplinary research

dred researchers on both sides of

same

has proposed

tide,

Maxwell

in his equations explained

all

of cognition." 47

of electromagnetism

four quantities (charge, current, electric and magnetic

a

hun-

book of the

the Atlantic. Newell, in a

as a "unified theory

it

Soar has evolved

program involving more than

in

Much

as

terms of

fields),

Newell

hopes to eventually account for the whole of human cognition through the four fundamental

Among

mechanisms implemented

Anderson's Act*

48

and a system

called Prodigy

in Soar.

most notable

John by Steven Minton, Jaime

other architectures similar to Soar,

are

Carbonell, and others. 49 Both Act* and Prodigy posit a learning mecha-

nism

which

similar to Soar's,

called "explanation-based learning."

is

Contrary to Soar, which can learn only from

from

also learn

its

mistakes. Since

its

Anderson

successes, Prodigy can is

a Carnegie

Mellon

psychologist, and Carbonell one of Newell's colleagues at that university's

computer science department, these projects obviously sprouted

off the

same

chapel of

intellectual branch.

AI

Together, they

make up

a substantial

research.

The Question of Awareness: Could Machines Love? Whether through the worldly knowledge of architectures

future

modeled on our own minds,

may comprehend

own. This

raises the question

attributes with us: Will feel pain, love,

On

the world in a

I

whether they

is

to

compare

its

it

our

other mental

as

one

studies

inability to define

to establish

its

presence in

human

behavior.

when we succeed in embodying

in machines, since relatively simple logic or association

mechanisms can make intelligence.

way

human

behavior with

Further, intelligence seems to disappear aspects of

will share

one can study emotions

have already discussed the

system

artificial

machines of the totally unlike

and anger?

intelligence in absolute terms: the only

an

manner not

such machines be aware of themselves? Will they

a philosophical level,

intelligence.

a Cyc-like data base or

intelligent

There

are

a system

behave

two ways

as if

it

did contain a piece of

to consider this

stems from the position that intelligence

is

phenomenon. One from

in essence different

264

M

matter, a kind of immaterial fluid breathed into a being.

consequence of

this position

is

to dismiss as pure

successes at imitating intelligence. If intelligence

is

The

of

life

plex

intelligence. In this case,

we

with mechanical dolls,

mechanism

The other

much

as

we

partial

in essence different

from matter, and we don't put any into our mechanism, then be imitating

logical

mimicry our

can onlv

it

can't imitate

aspects

all

probably couldn't build a more com-

to imitate the entire mind.

interpretation of the seeming disappearance of intelligence

takes a reverse view of the matter. The fact that one can build pieces of mind out of nonmind substance like computer circuitry may just show that mind naturally emerges out of properly organized matter. Indeed, it is

a

widespread property

in nature that

complex phenomena emerge

out of simple interactions and components. The simple forces between ice molecules, for instance,

induce an

infinite variety

hibit properties totally absent in individual

doesn't

make

of

intricate

components. For example,

it

sense to talk about the temperature, entropy, and pressure

of an individual molecule, yet these properties emerge out of the tive

snow-

groups of interacting components often ex-

flake structures. Further,

collec-

behavior of large numbers of gas molecules. This interpretation

is

much more encouraging

for the future of AI; and, indeed,

researchers subscribe to

Many of them claim that one good reason we is that we all carry one in our skulls

can build

a thinking

it.

most AI

machine

hence, the "meat machine" quality of the brain Marvin Minsky

is

fond

of invoking. Is the brain truly a

one's immortal soul physical body?

A

machine? Aren't one's conscience, one's thoughts, if it

comes

consensus

is

to that, essentially different

building

among

scientists

from one's

and philoso-

phers that they are not. Most of these experts subscribe to the opinion that the

mind

is

essentially

an emanation of the brain. In other words,

physical processes occurring in the brain explain our mental processes

Many arguments justify this opinion. One is anatomical evidence for the association of mind and brain. Damage to the brain disturbs our thinking. Damage to specific areas in their entirety.

even gives

rise to specific

correspond to abstract

kinds of disorder. Particular brain structures

abilities,

such as language. For example, damage

to either Broca's or Wernike's area,

both located below the

can produce permanent and specific language

problem of our drug-abusing society

disabilities.

left

And

temple,

isn't

one

precisely that specific chemical

changes in the brain correlate with changes of

moods and emotions?

265

SOULS OF SILICON

Another argument for mind-brain association stems from the history

We

of science.

have not, so

discovered anything in nature which

far,

physical law cannot explain. It

would be surprising

exception. If fact, the entire history

phenomena

of science

if

the

mind were an

one of explaining

is

previously thought to be governed by forces extraneous to

Wind, lightning, volcanoes, and earthquakes are no longer expressions of the whims of gods, but strictly material phenomena governed by natural law. We can now make mathematical the physical universe.

models of the evolution of the universe, bang.

The laws of physics and

all

way to

the

the primordial big

chemistry, together with the

of natural evolution, explain the appearance of

life

on

mechanisms

earth.

They

also

appear to account for the evolution of humans.

humans

In a parallel phenomenon, science has gradually displaced

and

their physical

environment from

their special status in the world.

Copernicus started the movement by moving the earth from

human body from

its

pedestal and proved

ancestors. Biochemistry

natural

showed

its

lineage with

human mind, showed

that

was

it

at least

even our minds have purely material If such

is

amenable to

last step in this revolution: its

the case, shouldn't

it

central

nonhuman

that the life processes in this

phenomena. Freud, without proving the natural

would be the

its

Darwin displaced the

position to being a satellite of the sun. Later

body

are

origin of the

scientific study.

success

would show

AI

that

origins.

be possible to re-create, out of inert

matter, beings with not only thought but also awareness, feelings,

and

emotions? Although the opinions of philosophers and AI researchers cover the usual spectrum of diverging views, find any creditable expert

I

have been hard put to

who would answer

a clearcut no to this Most responded with a qualified yes; and in the past decade, most debates on the subject did not oppose the partisans of yes and no answers. The action centered on the yes part of the spectrum, with

question.

factions arguing over

whether they should qualify

their positive answers,

and how.

The

closest approximation to an

Berkeley's

Hubert Dreyfus,

machines cannot be made

who

uncompromising no comes from

simply claims that truly intelligent

in the first place.

And

even on that point,

Dreyfus's objections stop at symbol-manipulating machines: he

committal about machines based on

non-

neural nets, presumably

way our brains are built. 50 somewhat less hard line, MIT's Joseph Weizenbaum, author

closer to the

Taking a

artificial

is

266

||

of the anti-AI book Computer Power and Human Reason,^ summarized

his

position for me:

I

don't see any

that

way

why

can't understand

develop in

be

of

to put a limitation to the degree

machine] could acquire. The only qualification

[a

way

this

resisted,

it's

will

intelligence

make, and

I

that the intelligence that will

is

always be alien to

at least as different as the intelligence

human

I

human

intelligence. It will

of a dolphin

is

to that of a

being. 52

Weizenbaum

points out that dolphins are different

comparison leaves open the

— not

of creating

possibility

unfeeling. This

machine with

a

As he himself said, "I don't understand what [my position] takes away from any ambition that the AI people might have." For an all-out yes, consider Carnegie Mellon's Hans Moravec, who feelings.

we

believes that early in the next century

generations of gradually

more

He

told me.

be served by successive

intelligent robots.

"On

chines as anything but unfeeling aliens. in general will

will

He

views these ma-

the contrary,

I

think robots

be quite emotional about being nice people," Moravec

explained this character

trait as

the result of evolutionary

forces:

Imagine these robots being made in a factor} and the main purpose 7

,

of the factory, which robots to

sell well.

companies, which response that

when you person

it's

is

the reproductive unit of the robots,

will

then build more factories.

bring one into your house, there for, and that

it

a positive conditioning if

sitive

feel

it

will

it

understand that you're the

it.

It

should

is

You that

somehow

its

internal

at least

estimate

make-up]

does something that makes you happy.

about

its

actions. It will try to please

manner because

reinforcement.

Moravec's point

customer

it is

had better keep you happy, or

induce you to buy another one of

apparently selfless

Then

mostly shape the character of these robots. So

will

how you

to cause

best-selling robots will bring in profits to their

whether you're happy or not, and receive [from

will care

is

The

it

will get a thrill

you

out of

in

this

It

an

po-

can interpret that as a kind of love.

emotions are

just devices

for channeling

behavior in a direction beneficial to the survival of one's species. The

most

basic emotions are love,

which

stirs

us toward certain goals, and

267

SOILS OF SILICON

which keeps us out of harm's way. Moravec believes robots

fear,

experience fear also. for his future

Tongue

will erase

may

and

run it

down

a

will

minor emergency

try desperate

to nothing, because then

will forget all sorts

consequences. So

terrible, terrible it

he imagined

household robot:

It can't let its batteries

memories

in cheek,

if it gets

measures to obtain

all

its

of important things with locked out of the house,

a recharge

somewhere.

Its

emergency modules would come into

play, it would express agitation, humans can recognize. It would go them to use their plug saying, "Please!

or even panic, with signals that to the neighbors

Please!

I

need

and ask

this!

It's

so important,

it's

such a small cost! We'll

his

robot carts calibrate their

reimburse you!"

He

also sees in the

maneuvers by which

young animals: such on which will depend the life-saving Marvin Minsky has, in The Society of Mind, of emotions in which he explains them as

vision sensors the forebears of playful behavior in activities serve to

fights or flights

sharpen the

of adult

life.

carried out a deeper analysis

skills

special-purpose cognitive devices. Fear, anger, and pleasure act as short-

term attention focusers. For longer time spans, "liking" holds us to

we ought to underdown our universe." 53 Minsky even sees a clear cognitive role for humor and laughter, which play "a possible essential function in how we learn": "When we learn in a serious context, the result is to change connections among ordinary agents. But when we learn in a humorous context, the principal result our choice: "Liking's job stand

is

to

sors."

its

is

shutting off alternatives;

role since, unconstrained,

it

narrows

change the connections that involve our censors and suppres54

Freud had already understood the

role

of censors and suppressors

as

inhibitor)7 agents responsible for preventing harmful or socially unaccept-

able behavior, but Minsky extends their action to the cognitive realm: in common-sense reasoning, censors and suppressors must recognize trains of thought that lead to absurdity or infinite recursion, and make us avoid

them in the as

touchy as

future. sex.

This

is

why, to our subconscious, the absurd

Absurd and sexy jokes are funny because

surprise us with forbidden outcomes,

during which censors against these

and laughter

new approaches

is

is

almost

their punchlines

the

mechanism

are built. "In order to

construct or improve a censor," Minsky explains, "you must retain your

268

Al

records of the recent states of thought. This takes

some

are fully occupied." 55

mind

For Minsky, laughter's function

the censor- formation process. If

next time around, and the joke

To

all

and

this,

think the censored

which your short-term memories

on these memories and ensure

attention focused

nomena,

made you

that

time, during

all

works

to

is

keep your

that nothing interrupts

well, surprise

is

avoided the

no longer funny.

is

one might object

emotions are biochemical phe-

that

that mood-altering drugs can induce depression or ecstasy,

have demonstrated that emotions are related to

that biochemists

minute variations of neurotransmitters

in the brain.

How,

then, can a

machine made of transistors and wires feel anything like an emotion? The answer is that neurotransmitters are just handy devices for inducing special electrical behavior in the brain. As their name implies, neurotransmitters are chemicals that selectively alter the conductivities

of synapses, the connections between neurons. Relaxants function by switching activity

make you

on whatever brain

of brain

Dedicated hardware or software con-

feel relaxed.

could achieve similar effects in machines:

trols

Valium

like

circuits or patterns

meant by switching on an "emergency module"

this is

what Moravec

to induce panic in his

discharging robot. "All right," the skeptic might say at this point,

can make a machine behave,

when observed from

acting in an emotional manner.

I

will

"I'll

believe that

outside, as if

it

you

were

even admit that such behavior

is

Howwhen Moravec's home robot

necessary for a machine's intelligent interaction with the world. ever,

I

absolutely refuse to admit that

pampers you,

feels

it

business assigning late

human

many

love or even affection.

states to

We

have no

machines that cleverly simu-

is

known

to the philosophically inclined artificial intelli-

"weak AI," and they have fought

private

strong

like

behavior modes."

This position gentsia as

anything

human mental

and public debates over the

AI hold

that intelligent

believers in "strong

last

AI"

machines can be imbued with

awareness, consciousness, and true feelings.

in

two decades. Adepts of

The

issue

may remain

self-

for-

ever uncertain since, contrary to behavior, internal states of mind cannot

be assessed objectively.

Although

at first the

one could argue

reasonable reaction seems to be

that for

"Who

cares?"

moral reasons such questions should be ad-

dressed and resolved before

we

example, what should be the

ever build truly intelligent machines. For fate

of your future loving but battered

269

SOULS OF SILICON robot nanny? Wouldn't

ponder such

it

mass-produce robot brains and wire them

Wouldn't a robot personality

vations.

unhappy

quite

The most

in repetitive factory

no

ness are

classic

perform

as a tour guide

be

AI came from

the Berkeley philos-

claims that computer simulations of aware-

made anyone

wet.

Room"

1980 "Chinese

He

drove the point

home

Chinese. Searle, in a closed

who

is

totally

room and

priate responses,

all

(a

test in

ignorant of the Chinese language, would

receive the judge's questions as Chinese ideo-

grams on pieces of paper. language instructions

in his

paper, 56 which described a thought

experiment in which he simulated a computer passing Turing's

sit

fact,

with the same basic moti-

all

to

should, in

turn out to be cruel to

closer to the real thing than their simulations of thunder-

storms, which never

now

who

Searle,

fit

may

work?

vivid attack against strong

opher John R.

We

be murder to junk her?

it

issues at the design stage:

He would

"program")

in terms

then consult a set of English-

telling

him how

of Chinese symbols

to

compose appro-

unintelligible to him.

After working out his answers in this way, he would hand them out as

ideograms on paper.

Now,

to

the room, the being inside nese. Searle, however,

it

one observing the procedure from outside

would

clearly

appear to understand Chi-

would have been merely following formal

he would have remained

totally ignorant

rules:

of the meaning of his answers

and unaware of the mental processes involved

in

working them

out.

He

concluded that mere symbol manipulations, even were they to generate outwardly intelligent behavior, could not induce awareness in the mech-

anism performing them. Searle's

opponents answered by claiming that

indeed emerge from these

activities,

the system consisting of the

processing the

man

activities.

in the

to point out,

create a

new

practice,

it

but that

it

a

new awareness might

would be associated with

room, the program, and the man's symbol-

Since this consciousness would be external to him,

room would remain unaware of it. As does seem a

bit ridiculous to

Searle

was quick

argue that a person could

consciousness by merely shuffling around bits of paper. In

how

however, the instructions explaining

to generate the an-

swering ideograms would amount to a million pages of printed

which

is

the quantity of knowledge a full-fledged

text,

common-sense data

base would contain. Further, in order to carry out the simulation at a speed approaching real time, the

to

perform long-hand calculations

poor

mug

in the

at the rate

room would have

of one hundred

billion

270

Al

operations every second, the speed at which our brains process infor-

mation for

wind of

us.

activity

Outside of

its

sheer physical impossibility, this whirl-

makes the appearance of an awareness somewhat

less

preposterous.

My

favorite

Room"

counterargument to the "Chinese

paper was

offered by the philosophers Paul and Patricia Churchland in 1990. 57

Suppose

that, instead

of trying to show that symbol manipulation and

man

awareness are unrelated, the onstrate, falsely, that light

He

with each other.

pumping

in the

could achieve

magnet up and down

a

room were attempting

at

this

catch here

is

dem-

by darkening the room and

arm's length, thereby generating

electromagnetic waves of a very low frequency.

permeating the room would

waves of

to

and electromagnetic waves have nothing to do

let

him

The

pitch-darkness

simply that to generate any light (that

a frequency perceptible to the eye), the

is,

still

The

infer the desired conclusion.

electromagnetic

man would

have to

speed up his pumping rate by about fifteen orders of magnitude

— or

about the same factor by which he would have to accelerate

just

paper shuffling in order to generate an observable awareness

his

in the

Chinese Room.

What of Searle's claim about the difference between simulations and What difference is there between a computer performing operations that appear to endow it with awareness, and the same comthe real thing?

puter simulating a storm in a weather-forecasting center? Is the computer's awareness any

more

real

computer's intelligence certainly

around raindrops, surely isn't doing.

air

the "essence" of

one

really

However

The "essence" of a storm

the "essence" of intelligence precisely

is

is

this

the Chinese

to

is

what the computer does.

awareness also the manipulation

knows, but

more than

is.

to whirl

molecules, and lightning, which the computer

information, and this

late

than the storm? Well, for one thing the

manipu-

Now is No

of information?

argument doesn't show us otherwise, any

Room

does.

Indeed, neither Searle nor other proponents of weak AI have pro-

vided a definition of the nature of awareness that would satisfy anyone.

For by a

their position begs the question: If the manipulation digital

computer cannot generate

take to do it? phenomenon: mental

Searle himself answers that "cognition

it

cesses. This it

states

of information

true awareness, then

and processes

are caused

is

what does

a biological

by brain pro-

does not imply that only a biological system could think, but

does imply that any alternative system, whether made of

silicon,

beer

271

SOULS OF SILICON

cans or whatever, would have to have the relevant causal capacities 58 equivalent to those of brains."

consist of

What these

mysterious causal capacities

exactly, Searle refuses to specify, except to

claim that parallel computers or

add the further

neural-net systems

artificial

would not

possess them either.

On

an argument proposed by the philosopher Zenon

this point,

Pylyshyn raises puzzling questions on the boundary between biological

and electronic systems. In

by electronic

this

thought experiment, Pylyshyn supposed

more and more of her brain cells are replaced components with identical input-output functions, until

that as a person

is

talking,

the entire brain consists of integrated circuit chips. In

person would keep on acting in is

right,

she

would

at

just the

some point have

"meaning" anything with her words.

all

likelihood, the

same way except

lost her

that if Searle

awareness and stopped

Somehow that doesn't sound

quite

right."

Daniel Dennett drew yet another argument for strong AI from the theory of evolution: causal

if

systems with and without Searle's mysterious

powers for awareness

can't

be told apart by

their behavior, then

they have exactly the same survival value. In this case, evolution would

have absolutely no incentive for developing such a superfluous mecha-

how did it come about and maintain some chance mutation had robbed our ancestors of awareness, we would be acting exactly as we are now, claiming to be

nism

itself?

as "true" awareness. If so,

Indeed,

aware, except

if

we wouldn't

be.

60

Wouldn't that be downright

Assuming, then, that one can define "true" awareness

silly?

in terms

of

pure functionality, what are the specific functions a system should

embody these

in order to qualify? If the

proponents of strong AI are

would be the same functions

Turing's

test.

that

would

let

a

right,

machine pass

But can some of the philosophical steam generated

in the

weak-strong AI debate allow us to point out some feature of those

mechanisms? In other words, can philosophers provide engineers with design guidelines for conscious machines?

Perhaps they can. As I've hinted, philosophers draw a strong link

between awareness and the

John

Searle rests his

ability to give

whole case on the

meaning

fact that

to symbols. Indeed,

symbols manipulated

meaning only when interpreted by humans. In Searle's parlance, symbol-manipulating computers possess syntax but no semantics. Hans Moravec, for his part, was trained as an engineer, and

by a computer acquire

a

his bottom-line conclusions

on machine awareness

are entirely opposite

272

Al

to Searle's.

Yet Moravec agrees with Searle on

'Today's

this point.

humans to front for them," Moravec admitme. "They need somebody to tell them what's in the world, and on what the programs say. In essence, to give meaning to the

reasoning programs require ted to to act

abstract symbols the

programs manipulate." Moravec however sees

a

straightforward engineering solution to this philosophical quagmire: "If

we

could graft a robot to a reasoning program,

person to provide the meaning anymore:

Moravec

physical world."

ence to the world

is

also believes that

we wouldn't need

a

would come from the some kind of sensory referit

required for a machine to pass Turing's

test:

Human communication is only language on the surface. What's below the language are these perceptual models of the world containing

mythical allusions and pictures and emotions. There bal

machinery

in

our heads.

probing for these things, asking,

"Here

is

a situation,

how

is

much

nonver-

A really insightful Turing judge would be

does

it

"How do strike

no more compact way of encoding analogous to the actual structures

you

you?"

that

feel

And

I

about

this,"

believe there

or is

machinery than something

we have in our brains, including our

brain stems and limbic systems.

And, indeed, since the mid-1980s, for reasons not unrelated issue raised

by Moravec, an

has concerned

itself with

to the

of the AI research world

influential faction

coupling their programs ever more closely with

the physical world. Moravec belongs to this school of thought, and Rodney Brooks of MIT is its most colorful figure. He, in fact, takes the

extreme position that reasoning that

mechanisms akin

suffice to explain

it.

to those

is

not necessary for intelligence, and

of reactive or inborn behavior

His tiny six-legged robots, which mimic

behavior of insects, are Brook's

first

in animals

much of the

61 step in proving his point of view.

In respect to design rules, philosophers also point to self-consciousness as

another crucial component of awareness. Daniel Dennett notes,

though, that "self-awareness can simplest, crudest notion

mean

several things. If

of self-consciousness,

the sort of self-consciousness that a lobster has:

something, but

between

itself

it

never

and the

eats itself. It

rest

I

When it's

some way of

has

of the world, and

it

you take the

suppose that would be hungry,

it

eats

distinguishing

has a rather special regard

for itself." 62

That kind of bodily awareness has been

instilled into

robots from the

273

SOULS OF SILICON beginning. For example, Marvin Minsky's well

had to have

blocks

hand

it,

in order for the

was manipulating.

it

in the image,

whether

it

would move

was

really

it

When

hand-eye robot pretty

it

hand from the

the

tell

the robot thought

had

it

identified

its

slowly in front of the camera to see

chapter

itself (see

first

camera to

4).

A higher level of self-consciousness corresponds to introspection: the some of

ability to inspect

own

one's

mental

Most AI programs recommen-

states.

require this ability in various degrees, if only to justify their

dations to their users. All medical expert systems, starting with (see chapter 6)

have had the

certain treatment: for this, they

by retracing

SHRDLU

could explain

exhausting a long

me

asked

chapter

they prescribe a

have to examine

their

own

motivations

a further example, Terry

why

Winograd's

it

had manipulated

of intermediate objectives,

certain blocks: after

gave "Because you

it

to" as the ultimate justification to a sequence of

4).

Marvin Minsky, for one, turns the

we

out that

list

why

As

their reasoning.

MYCIN

ability to explain

table

moves

(see

around and points

could design machines better equipped than our brains to

monitor themselves, thus making them more conscious than we

are.

63

Perhaps the major disappointment of AI research to those of us schooled in traditional Western values was to evacuate the substance out

of

intelligence: if

source of

its

you take apart an AI program and

cleverness, you'll see

it

locking subprocesses, in themselves

try to trace the

disappearing into a all

quite

trivial.

maze of inter-

Recent forays into

cognitive architectures indicate that our consciousness and sense of identity

may

When

came

I

well participate in the

same disappointing evanescence.

across the following paragraphs in The Society of Mind,

couldn't help but feel that Marvin

Minsky was

letting

sections appear far apart in the book, but together left-right

punches against consciousness and

we

I

me down. These

make up

devastating

identity:

little more than menu lists that on mental screen displays that other systems use. It is very much like the way the players of computer games use symbols to invoke the processes inside their complicated game machines without the

[W]hat

flash,

call

"consciousness" consists of

from time

slightest

to time,

understanding of

[Ojur brains appear to

how

make

they work. 64

us seek to represent dependencies. Whatever

happens, where or when, we're prone to wonder .

.

.

But what

if

who

or what's responsible.

those same tendencies should lead us to imagine things and

274

Al

causes that do not exist?

hand

see their

Then

we'll invent false

gods and superstitions and

every chance coincidence. Indeed, perhaps the strange

in

word "I"

—

If you're

compelled to find some cause that causes everything you do

as used in "1just

had a good idea"

then, that something needs a name.

You

—

reflects the selfsame tendency.

call

"me. "

it

I

call

it

you.

— why, '* 5

Consciousness as an arcade player and the "I" as a figure of speech,

Minsky wouldn't give

indeed! In person,

whether he thought the It's

a

self

complicated set of

involved in

it.

One

is

about two years old

person. ...

By

concepts of

self.

me

it

any reprieve.

illusions, there are

many

complex

can have emotions, dismayed

body, and

explain

particle,

why the

the child a

it is

as illusory, superficial explana-

me

who

believes that machines

by agreeing with Minsky:

the "I" the Center of Narrative Gravity.

of gravity was

when

to be accounted for in a convenient way.

center of gravity of this object

not a

different processes

usually gets this wonderful idea that

Daniel Dennett, the very same philosopher

it's

asked

of correcdy recognizing that

at first the I is a

But Minsky considers these concepts

call

I

the time you're adult, you have a dozen different

tions of processes too

I

When

he answered:

illusion,

just the inference

people are also objects. So is

an

is

stapler

on

Now

his desk]. It's

consider the

not an atom,

an abstract point in space. But you can use

it's

stapler tipped still

[a

back when

over a point

I tilted it:

in the

it

to

because the center

supporting base.

Now,

are

centers of gravity real? In one sense they are, in another sense they aren't.

That

is,

the center of mass of an object

just a

is

very convenient

way of organizing what would otherwise be hopelessly messy data, but where would we be without it! Now the self, the "I," is the center of narrative gravity for [our discourse about human beings]. And, boy, does

it

We

help!

human body flinging about, talking: it just so much [random] motion, if we of center of narrative gravity, that agent who is see a

would be incomprehensible, couldn't posit a sort

body

responsible for these words the think this agent

is

the brain which

is

in the

agent

the

Oval Office is

a

is

uttering. It's a mistake to

a point in the brain. There's self,

and

it's

in the brain.

a mistake to

not any one thing

in

look for the President

But the idea of there being

wonderful way of organizing psychology.

a single

275

StCLS IF SILICM

True enough,

I reflected,

through

myriad mechanisms,

its

constructive

upward

ments blend into a

direction?

single

why not

As

symphony, couldn't the

Mil

put the question to

try to trace

s

it

the separate notes of

emerging from the concerted

entity I

but being useful doesn't necessarily make a

of speech wrong. Instead of trying to follow the

figure

activities

down

self

more

in the

many

instru-

self truly exist as

an

of a multitude of agents?

Minsky and to Tuft's Dennett. Minsky

wasn't encouraging:

That's what :

many people believe, and I don't think so. I think the self we make it it's not an emergent; it's an

doesn't emerge. In fact,

afterward construction. In general, emergents don't produce anything that clever. ... If

to

like

it,

it

were an emergent, you couldn't attach properties

"I'm handsome." Because to do so you need a small

representation, a

little

symbolic "I" that you can attach properties

to.

You can't attach them to a vague emergent: it doesn't have any hooks. There

is

beehive

you

something generally wrong with the idea of emergence: starts to

swarm

can't attach to

it.

in a certain direction, that's

It can't store

if

a

an emergent, but

any knowledge because

it

doesn't

i solution to a (inferential equation.

actually

-.hough Daniel Dennett was willing to label our feeling that there's "somebody home" in our heads as an emergent phenomenon, he hastily qualified his opinion:

Some people

use the term emtrgna in a mystical way. Emergent

properties are supposed to be very mysterious that

you

Coast Emergence."]

pedestrian, but I think

of description. In non:

special properties

can't explain with science. [Dennett prfvatefy calls this

"Woo Woo West more

and

many

mean

useful, sense

this sense, a traffic

different

into their cars,

more

I

jam

view

m gemf in the much

ej

/

of a convenient level

an emergent phenome-

is

people make semi-independent decisions to get

and suddenly you get the

"traffic

jam" phenomenon.

And you really want to talk about it at that leveL Don't try to "reduce" talk

about

traffic

jams to talk about behaviors of just individual

motorists, even though that's all they really are. A traffic jam is just one god-awful combinanon of individual motorist behaviors. But it is

also

emergent in the sense that there are

jams which are best described

regularities

at the "traffic

about

jam" leveL

traffic

276 As

it

k\

turned out, then, what Dennett means bv calling the self an

"emergent entity"

is

concept of

a convenient figure

self

is

fundamental agreement with Minskv after

in

of speech

—

the

all:

so convenient as to

be indispensable.

On

this question, I

Somehow,

found more solace

had expected to meet the

I

jreen Tufts

campus and MITs

less

in Pittsburgh than in Boston.

down-to-earth attitude in the

lofty hallwavs, the latter

once de-

scribed as a cross between the Pentagon and the Vatican. 6* Yet

surprised and

somewhat comforted by the

attitude

of researchers

I

was

in Car-

Andrew Carnegie's utilitarian former Carnegie they didn't make any case for teducing human awar.

negie Mellon University,

Tech. At

least,

to a figure of speech.

put the question to the psychologist fames

I

McClelland who, with a group of San Diego colleagues in the mid1980s, resurrected neural-net research by publishing a two- volume col-

of groundbreaking papers called

lection

Parallel Distributed Processing^

PDP

showed how, by extending Rosenblatt's Perceptrons to manylayered structures, they could be made to overcome most of the failings Minsk}- and Papert had pointed out in their 196" book." Bearded and relaxed,

John McClelland

reflection

when

a slow speaker.

is

He

enters a state of deep

a question requires consideration.

Eyes closed, he tends

to hold his head with both hands and, slowlv intoning, appears painfully to force his answer through as if hauling

it

out of a deep well.

He

discussed the Society of Mind's relevance to neural networks:

Minsky's notion that the

I is

an

illusion misses the point that there

a fundamental coherence to the mental state. There are

and

fifty

United that.

million people

States,

So

I

going about their daily business in the

and you don't get a sense of

think the Society of

might be that the ent.

all

states

is

two hundred

Mind

is

a collective

mind out of

missing something, which

of a neural network are

actually quite coher-

When you have large numbers of simple computational elements

that are massively interconnected with each other, they can't just

about doing whatever they basically

governed by what

So the human mind

is

damn all

well please.

the others are doing at the

much more

population of a country.*

go

What each one does same

is

time.

coherent than the distributed

9

Over in Carnegie Mellon's computer science department, Allen Nemade similar remarks. I tried out on him a notion of mine that one

well

277

SOULS OF SILICON

could draw parallels between Soar and Minsky's Society of Mind.

Weren't Soar's complicated IF

Minskyan agents,

all

all

his

.

.

THEN rules somewhat equivalent to

own, which

own

specialized ways?

wrong," Newell answered

"I think you're just plain

manner

.

interacting in their

found disconcerting

I

after

vehement

in a

McClelland 's slow

pronouncements.

A

production

Minskyan

—

agent.

a rule, if

you

will

Minskyan agents

—

much

is

are in fact

too simple to be a

little

homunculi

by

all

themselves. Also, productions don't direcdy communicate with each other: they just

could

talk to

all

look

another

at the

common

perceive in this

same

data.

by making

is

a

The only way

a

production

change that the other could of

data: this places a lot

on

restrictions

communication between productions. So the Society of Mind and Soar are profoundly different in two ways. agents

is

much

larger in the Society

agents to communicate with each other absolutely unclear to

First, the grain size

of Mind. Second, the is

much

of the

ability

smaller in Soar.

of

It's

me how you can produce in that Society of Mind

any sort of integration.

The paradigm on Soar tion rate high little

enough so

is

much more how you get

that

agents knowing separate different things.

communica-

A community of scien-

with telephones in their hands, cannot produce an integrated

tists, all

intelligence. tive to

the

you don't have the problem of separate

The

available rate

of communication between them

what they've each got

problem of producing beings

in their like

memories

too small.

is

you and me, which

rela-

The

are highly

integrated, requires an architecture that's very responsive to that I believe that Marvin is on the wrong side wrong region of the communication space with

communication problem. of this chasm,

in the

respect to the integration issue. Marvin's view

of ways

in

which we

certainly right, but

I

is

that there are a lot

are dis-integrated, unintegrated.

think that he just

accounting for the huge amount of integration that

don't have any objections to being

I

ness

more than an

tions

marching

for

it.

shown wrong.

illusion required the victory

in lockstep

And

he

is

doesn't even come close to

we

If

have.

making aware-

of an army of produc-

over a rabble of individualist agents,

I

am

all

278

Al

AND RELIGION

Al

Most of

us are perfectly willing to accept scientific explanations in

matters physical, yet

One

material origin for our minds.

human

erosion of

all

uncomfortable with the idea of an essentially

feel

reason, as I've said,

is

the potential

values inherent in this belief, which led

some

re-

searchers to dismiss consciousness and feelings as illusions. Another of

our discomforts with the concept religious beliefs.

arises

from

its

potential conflict with

Doesn't the materialistic view of the mind contradict

the existence of an immortal soul, different in substance

Perhaps, but as

show,

shall

I

this

from the body?

opinion doesn't clash with Western

religions.

These

verses, for example,

that Judeo-Christian tradition

ation of

will

Of those who

lie

life,

prophesied as

noise, a

come

to

in the dust, for

lie

everlasting

I

is

not inconsistent with an intimate associ-

mind and body:

Your dead

who

from the Old Testament, 70 seem to imply

life, .

.

.

their corpses will rise; awake, exult,

sleeping in the dust of the earth

some

I

to

all

of you

the land of ghosts will give birth. Isaiah 26:19

shame and

many

will

awake, some to

everlasting disgrace. Daniel 12:2

had been ordered. While

was prophesying, there was a I looked, and saw

I

sound of clattering; and the bones joined together.

that they

were covered with sinews;

flesh

was growing on them and skin was

covering them. Ezekiel 37:7-8

And you will know

that

from your graves,

my

And

from the

this

It is

the

I

am Yahveh, when I open your graves and raise you

people. Ezekiel 37:13

New

Testament: 71

same with the resurrection of the dead: the thing that is sown is Howbeit that was not first what is raised is imperishable.

perishable but

which

is

spiritual.

spiritual, 1

.

but that which

is

natural;

.

.

and afterward

that

which

is

Corinthians 15:42-46

These verses strongly suggest bodily resurrection indicates a belief that the

mind cannot

exist

in the afterlife,

which

independendy of the body;

279

SOULS OF SILICON

and Christianity preaches that a human being's soul and body are substantially united.

bodied

spirits,

72

Resurrected Christians

—

new body The soul as an

but in a

resurrected the body.

live after

as the

death not as disem-

Gospels claim that Christ

entity separate

from the body

influence from Eastern faiths, and Christian religions

is

do not make

an

it

a

tenet of their doctrines.

Other

The

religions do,

however, erect soul-body dualism

conflict with the materialistic

dogma.

view of the mind, since for the same soul

to live in several successive bodies implies a

and

as a

doctrine of reincarnation, for example, does indeed appear to

soul.

And

dichotomy between body

then there are the near-death experiences, as reported by

Raymond A. Moody 73 and a growing body of other researchers.

Patients

revived after approaching or reaching a state of clinical death recount strange experiences, claiming to have stepped out of their bodies and

observed the

Many recall

efforts

of the medical team to bring them back to

traveling through a dark tunnel

and emerging in

a bright

life.

and

peaceful light where they meet deceased friends and relatives. These

accounts cannot be dismissed

million adult Americans

Further,

For one

lightly.

of them: a Gallup poll performed

may have undergone

many

thing, there are too

1982 indicated that

in

as

many as

eight

a near-death experience.

74

NDE subjects are often in possession of knowledge they could

not have acquired by conventional means:

many who were unconscious

during the revival attempts can afterward describe those attempts in

Some

great detail.

subjects can even provide faithful accounts of events

occurring in other rooms, which they claim to have visited in their

disembodied It is still

the

mind

wound

life:

AI view of a

materialistic origin for

not incompatible with near-death experiences or reincarna-

is

tion. First,

during

state.

possible to argue that the

no

dualist

would deny the

otherwise,

essential unity

why would damage

of mind and body

to the brain (such as a head

or cerebrovascular accident) result in damage to the mind?

problem

is

what happens

at the

contains interesting speculations

moment of death, and on

the

AI

that very issue.

These concern the gradual and eventual replacement of brain electronic circuits with identical input-output functions.

philosopher

Zenon Pylyshyn used

The

folklore

this

cells

by

Although the

thought experiment as

a philo-

sophical argument in 1980, 75 the idea had been the subject of intense

debate in the AI grapevine for some years beforehand, under the

name

of "downloading." In 1988, Hans Moravec gave a dramatic description

280

Al

of the process ill

book Mind

in his

Children.

16

Assume you

patient in a twenty-first-century operating

A

substance.

your

and

local anesthesia

conscious.

fully

in

robot surgeon equipped with micromanipulators opens

under

skull

remain

are a critically

room, says Moravec

of neurons located

The surgeon

sets to

first

work on your

He

of your cortex.

in the periphery

brain.

You

concentrates on a small clump severs the

nerve-cell connections linking this assembly of cells to the rest of your

and replaces them with two-way connections. These

brain,

clump or

brain either to the

to an artificial

model of

it

link

He

scopic components, which the surgeon proceeds to build.

produces the structure of the it

cell

clump

in the artificial

your

made of micro-

so as to duplicate the exact behavior of the biological clump.

activating the switch that connects the rest original cluster or

When you no

accuracy of the modeling.

verify for yourself the

longer feel any difference

between the two positions of the switch, the biological clump removed, and the surgeon

number of

sets to

work on another clump. In

similar stages, he replicates

By

of your brain to either the

you can

electronic replica,

its

re-

model, and tunes

your entire brain

in

an

is

a large

artificial

no point do you experience an interruption of your end, your mind has been transferred to an artificial

construction; yet at

awareness. At the neural net.

This (so

far)

transferring a

imaginary process strongly suggests the possibility of

mind from one support

to another. Near-death experi-

ences and the survival of the "soul" after death could be explained by a similar transfer process. In this case, the receiving

be matter

do not

as

we know it, and

yet understand. It

would be required

pure

spirits;

though is,

certain,

for the information and organization that constitutes

our minds. Indeed, in death experiences

is

this regard,

some of those who underwent

insist that in the

rather,

disembodied

state,

near-

they were not

they inhabited another kind of "body" which,

invisible to living

humans, did possess

a definite structure.

of course, pure speculation to show that religious

larly the belief in survival after death, are

that the

support would not

would involve mechanisms we though, that some kind of support

the transfer

mind emerges from

physical

beliefs,

77

This

and particu-

not incompatible with the idea

phenomena.

11

HOW MANY BULLDOZERS FOR AN ANT COLONY?

now

should be clear by

It

define

it.

intelligent

Yet

it

system

that intelligence defies the heartiest effort to

equally clear that an essential ingredient of an

is

is its ability

the only function

common

to manipulate information. Indeed, this to brains

and computers. The

is

essential

ingredients of information are bits. Just as matter ultimately consists in

atoms,

all

the information that reaches us through our senses can be

broken down into

little

pieces of "yes " and "no." These particles

up any conversation, scenery, spanking, or

how

this is possible,

brain,

on clouds and water;

and waves twinkle

which

lets

experience.

a royal alley

make

To

see

no

us appreciate this beauty, has

inside our skull, our thinking organ

the sea as

would be

a

computer shuttered

Pentagon. In order to appreciate

this scene,

is

bits

Yet our

direct contact with just as

it.

removed from

basement of the

in the

our brain must reassemble

raw elements of data that our senses supply

corresponding to yes or no

of gold leads

like stars in the reflected light.

Locked up

the

we

consider the spectacle of the sun setting into the

ocean. Delicate hues play to the sun,

caress

it

as nerve impulses

of information.

In the case of sight, perception happens as follows:

The

cornea, a

transparent lens on the front of the eye, projects an image of the scene

onto the back of the ocular globe. In like a

this respect, the

eye works

camera, where a lens projects the image on photographic

much

film.

The

282 retina,

||

which plays the

of film

role

in

our eyes,

back of the ocular globe.

tissue covering the

sensitive nerve cells called "receptors."

a sheet

is

contains

It

Some of

of nervous

many

light-

these receptors

tell

how bright the receptor. Some other

other neurons, through a sequence of nerve pulses, projection of the image

receptors signal

at the location

is

how much

of the

red, green, or blue the

image contains

of pink, orange, and indigo

locations. (All the delicate hues

correspond to varying mixtures of these three primary

image

is

turned into discrete pulses in two ways:

becoming an

array

of a receptor

cell;

at their

in the sunset

The

colors.)

spatially,

first,

by

of dots, with each dot corresponding to the location

second, in the domain of brightness and color. Colors

become mixtures of discrete

more

hues, and brightnesses translate into

or less rapid firings of nerve

cells (the larger

the brightness, the faster

the firing rate). In similar ways, nerve cells in our ears turn the sounds

we

hear into pulse

Sensor

trains.

cells in

our skin do the same for

sensations of heat, cold, and pressure.

Thus, our brain it

bombarded with

constantly

is

what our senses

of pulses

trains

do with our

perceive. Intelligence has to

telling

ability to

manipulate these bits of information and use them to make sense of the world. Animals manipulate the information from their senses in a

ner that does not

them generate more than immediate

let

perceived threats or inducements.

We, however,

get

more mileage out

of the information we extract from our surroundings. into knowledge

and use

it

those of most

for planning

shows

mammals

their complexity.

We

can refine

it

for long-range planning and abstract reasoning.

do work on

Nevertheless, our brains animals. Dissection

man-

reactions to

the

same

lie

in the size

Thus, one can

of the structures present and

logically

and abstract thought are

principles as those of

between our brains and

that any differences

in

conclude that our capabilities

built

on

the

same basic

skills

that

allow animals to react to their environment. Further, this extra power

probably stems from the additional that the

more

abilities for

Thus, intelligence has to do with manipulate in a given time

measures information in elementary form,

is

compares the brain

(say,

bits,

how much

information one can

per hour or per second). Since one

one aspect of

intelligence, in

its

most

bits per second of raw processing power. If one

to a telephone switching station, this

correspond to the number of phone given time.

processing information

elaborate structure of our brains allow.

Of course,

there

is

more

lines the station

to intelligence than

power would

can switch in a

raw power, but

HOW MANY BULLDOZERS FOR let

283

AN ANT COLONY?

us not worry about this aspect of the problem right now. Let us just

recognize that no matter switching station

is, it

will

how

superbly structured and

simply not do

programmed

the

job if it can't process enough

its

connections in a given time. In the

first

questions:

unit of time,

benchmark? ment, and

part of this chapter

How many bits and

how

I shall

then look back

how

of our brains.

processing power

is

answer the following

to

do our present computers come

close

try to extrapolate

rise to the level

I shall try

of information can the brain manipulate per

at the history

long

it

of computer develop-

will take for

Finally, I shall

to this

our machines to

acknowledge that raw

not the only ingredient required for intelligence, and

discuss whether software powerful

enough

to emulate the

human mind

can be developed for the computers of the future.

THE HUMAN CORTEX AS CIRCUIT BOARD I he exposed

three

human

pounds of

brain

is

certainly

not an impressive

soft, jellylike, grayish tissue.

This

mushy

sight:

about

texture long

prevented anatomists from cutting the brain into clean thin

slices suit-

able for microscopic observation. Further, the uniform color of the

material kept

them from seeing

structural details. It

was only

in the late

nineteenth century that different hardening and coloring processes,

among them fine texture

the still-used Golgi stain, enabled anatomists to study the

of neural

tissue.

came as no surprise that, like other organs, the brain is made up of cells. They come in varying sizes and shapes, and neuroanatomists called them "neurons." One feature of the neurons, however, did astonish It

early researchers, including the Spaniard Santiago Italian

Ramon y Cajal and the

Camillo Golgi, developer of the staining process. They were

astonished by the intricacy and extensiveness with which these

connected to each other, each sending out that link

it

to as

many

other neurons.

literally

cells

thousands of tendrils

They make up

a

network of such

Byzantine complexity that Golgi, for one, firmly believed

continuous tissue extending throughout the brain.

He

it

formed one

defended

point of view, called "reticularism," in his 1906 Nobel address.

1

this

Later

284

U

—

observations

path

—

science progressed in

as

proved him wrong. Indeed,

neurons play

tedious, prodding

usual

its

we

as

gaps between

will see, the

workings of the brain.

a crucial role in the

Early in this century, researchers started to distinguish elements of

order in the apparent chaos of brain structure. that,

First, investigators realized

much

although neurons can differ from each other as

suckle bush does

and

different shapes

morphology

from

exist

Moreover,

brain).

mammals, from

a sequoia tree, they

sizes.

come

Only seven kinds of cells with

honey-

as a

in a small

number of

similar exterior

throughout the cortex (the largest structure cells

very similar to these

the higher primates

down

and bolts of our most abstract thoughts

make up

to the

in the

the brains of

puny mouse. The nuts

are thus the

same ones

that

support the mouse's instinctive reactions.

The

cortex, the brain structure responsible

motor responses, and about is

that

first

intellectual functions,

six millimeters (a

is

for our perceptions,

a thin sheet

quarter of an inch) in thickness.

of a square twenty inches on

of nerve

Its

a side: roughly the space that

personal computers used to take up on a desk.

To

fit it

IBM's

into our

Nature has had to fold the cortex; hence, the furrowed look of

skulls,

the naked brain.

The

cortex comprises six distinct layers, caused by an

uneven distribution of neurons of

makeup of the

layers vary

in the visual part, the

different types.

The

thicknesses and

over the area of the cortex. However, except

number of

cells

per unit area remains

fairly

stant at 146,000 per square millimeter. (Multiplying this figure

area of the cortex produces an estimated total

of about 30 cells

billion,

or 3 x

10 10 .)

The average

us from the

of the

is

likewise the

mouse

cells in

is

same

for

all

con-

by the

number of neurons

in

it

distribution of types of

throughout the cortex also remains constant. The density of

per unit area

To

cells,

surface area

mammals. What

cells

distinguishes

the area of our cortex, and not the kinds or density

it.

an engineer's eye, the cortex presents striking

similarities

with a

structure universally present in computers: the printed circuit board, a flat,

thin support holding integrated circuit chips,

cessing elements.

The board

which serve

as pro-

allows the chips to talk to each other

through conductive paths buried

in distinct layers

over

its

thickness.

Strangely enough, a typical board comprises six layers, just like the cortex.

Each chip on the board

(transistors, capacitors, diodes)

performs

is

made up of microscopic elements

of about the

a specific function within the

size

computer.

It

of

a neuron,

and

turns out that one

285

HOW MANY BULLDOZERS FOR AN ANT COLONY?

can also divide the cortex into chips, after a fashion. Experiments

conducted on animals by probing the sensory cortex with a microelectrode can detect firing impulses from single neurons. If you

move

the

electrode to and fro, in a direction perpendicular to the surface of the cortex,

you

only respond

However,

meet only nerve

will

For example,

if

you

if

and not on the

nonperpendicular direction by slanting the

in a

or the right eye, alternatively.

design our cortex as a circuit board?

both brain and board,

As

called "pyramidal cells,"

units.

of the brain

of the brain or cortex. Covered with an

of myelin, these

fibers

make up

a whitish tissue,

very different in appearance from the gray color of the cortex

about one

cell in a

of white matter

is

larger than that

kept them out of the cortex.

between the cortex and related to ease

of design.

a circuit It is

chips and connecting paths sions

would be

Only

of gray matter. Having

would have

with the direct communication between adjacent gray

why Nature

itself.

hundred extends beyond the cortex, yet the volume

in the brain

the "white cables" travel through the gray matter

bly

lies

Large neurons present in the cortex,

send nervous fibers downward, out of the

cortex, toward other regions

insulating greasy layer

depth.

We can conjec-

necessary to separate the closely

it is

in a circuit board, the cabling

underneath the computing

its

from the cabling connecting faraway

interacting processing elements

parts of the network.

left

the cortex were divided horizon-

It is as if

into different modules, each extending throughout

Why did Nature ture that, in

may

right.

meet neatly separated regions which respond to the

needle, you will

tally

process one kind of stimulus.

shine a light in the left eye,

you probe

if

cells that

buried into the visual part of the cortex, the probe

on

The

interfered

cells: this is

proba-

similarity in structure

board may have yet another reason,

already quite complicated to lay out the a flat surface.

a combinatorial

Doing

it

in three

dimen-

problem of monstrous proportions.

Perhaps Nature was no more willing to face engineers are! Whatever the reason for

convergent evolution of brain and

it,

circuit

this difficulty

than

human

one cannot contemplate the boards without wondering.

Let us go back to the basic building block of the brain, the neuron. Its

anatomy can

by the

brain.

appendages. a tree.

tell

us

more about

Extending from the

On

the cell

amount of computing performed body of the neuron are different

the input side, the dendrites look like the branches of

They connect with sensor cells, or other neurons, and receive The meeting points between dendrites and append-

their electric pulses.

ages of other cells are actually gaps, called "synapses." Although re-

286

Al

searchers had long suspected their existence, they could not prove

before the invention of electron microscopy

in the 1950s.

blow

vations of synapses then struck a final

it

Direct obser-

to the theory of nervous

system continuity, or reticularism, which Golgi had defended

until his

death in 1926.

When tween cell,

strong enough, nerve pulses can cross the synaptic gap be-

cells.

Pulses usually increase the electric potential of the receiving

which encourages

this cell to

generate a pulse of

Sometimes, however, arriving pulses decrease courage the receiving

cell

from

firing.

The

combines) the membrane potentials and function of this sum. a wire-like

The

cell

own, or

body sums

body

cell

called the "axon." It

"fire."

and

dis-

(or otherwise

fires at a rate that is

pulses generated by the

appendage of the neuron

its

this potential,

an /-shaped travel

along

may be

short

or very long: axons sometimes bundle together to form a nerve, which

can be as long as your arm. They also form the "white cabling" of the brain

I

treelike

have mentioned. The axon eventually branches out into another

network of

fibers.

These pass along

signals to other cells, or

activate muscles.

The

input and output ramifications of the neuron are

characteristic.

on

Extrapolations from counts

its

most

striking

electron micrographs

show there are from 10 14 to 10 15 synapses in the cortex. This means that, on the average, each neuron receives signals from about 10,000 other cells and sends its own messages to as many others. In this respect, the brain differs markedly from electronic circuits: on a circuit board, one component typically makes contact with fewer than five others. However, what computers lose in connectivity, they make up for in speed. In the brain, the pulses traveling from neuron to neuron are local

imbalances in

exacdy

how

salt

fast

concentrations

moving

in the order of 100 feet per second. This

from 10 to 100 milliseconds to reach the ever, pulses

They

moving from chip

travel at

at relatively

depends on the diameter of the nerve

two

thirds

is

why

low speed

fibers,

but

it is

sensor) stimuli take 7

cortex. In a computer,

how-

to chip are pure electromagnetic fields.

of the speed of

light

— about seven

million

times faster than nerve pulses!

The Brain's Processing Power Although our knowledge of the

brain's structure

mains sketchy. Recendy bold-hearted

scientists

is

progressing,

have

it

re-

tried to use this

HOW MANY BLLLDOZERS FOR

scanty evidence to estimate the the brain.

I shall

287

A\ ANT COLONY?

amount of raw computing going on

examine two such

tries

— by Jacob

T. Schwartz at

in

New

York University and by Hans Moravec at Carnegie Mellon University. It will come as no surprise that these professors achieved wildly different results. In fact, the very divergence of these estimates is a good illustra-

how

tion of

little

we

really

know about

the brain. Yet, because of the

accelerating pace of technological development, even guesses as poor

these provide useful estimates of

when we

as

be able to beat Nature

will

at brain building. I shall start

Courant

with the work of Jacob T. Schwartz, a professor at

Institute

neuron can

since a

about one hundred times per second,

fire

information to other neurons

much

whether to it

receives

lish is

at a rate

fire,

larger.

To

inside the

combine

to

first

from ten thousand other neurons. The

whether

this total is large

enough

for

it

to

cell

per is,

a second,

the signals

must then

The

fire.

all

sends

bits

neuron

one hundredth of

decide, every

the average neuron has

it

of about one hundred

The amount of information processed

second.

however,

NYU's

of Mathematical Sciences. 2 Schwartz estimates that

estab-

decision to

fire

complex, especially since some of the messages received from other

neurons may inhibit

firing rather

that to reach this decision, the

synapse

— perform

than promote

neuron must

—

Schwartz estimates

it.

for each firing, at each

the equivalent of calculations involving forty bits.

Since these operations involve intermediate steps, simulate them,

we have

per synapse, per

then a straightforward

bits

assume

that to

of information

affair to

amount of information processed by one neuron

overall

100

firing. It is

let's

one hundred

to manipulate

work out

in

the

one second:

per synapse per firing per neuron x 100 firings per second per

bits

synapse x 10,000 synapses per neuron per neuron.

From

there,

we

=100

million bits per second

the entire cortex: 100 million bits per second per neuron

neurons

in the cortex

=

power of x 3 x 10 10

get an estimate for the processing

3

x

10 18 bits per second of information

19 bits processing. Extrapolating to the entire brain, a total of about 10

per second

results.

Thus we have our

of brain power: 10 19

bits

first

estimate: Schwartz's estimate

per second.

Schwartz, however, puts a very strong qualifier on points out that computation rates,

might

safe to

suppose that what

really

this figure.

He

orders of magnitude lower,

suffice to represent the logical operations

is fairly

is

many

of the brain. Indeed,

it

matters to our thought processes

not the internal mechanics of a neuron, but

how

it

looks

like to

other

288

Al

may be

neurons. This

may be enough

more simple than

considerably

would show. Thus,

internal structure

stick-figure

the neuron's

models of neurons

to simulate the brain accurately. Moreover, our brain

is

accommodate a very large amount of redundancy, and much of complexity may be due to the constraints limiting its growth and

built to its

evolution (see the section entided "Avoiding Nature's Mistakes" later in this chapter).

Hans Moravec calculates the information-processing power of the manner different from Schwartz's, concentrating on the retina,

brain in a

the paper-thin layer of nerve cells and photoreceptors in the back of the eye.

3

amount of massaging on

After performing a certain

tion provided

by the receptors, the nerve

cells

send the

calculations to the brain through the optic nerve.

of the retina

that, in effect,

it

Such

the informa-

results is

of

their

the structure

makes up an extension of the

brain. Yet,

contrary to most brain structures, the functions the retina performs are well understood.

processing

They

are similar to those

of

artificial

vision systems

TV images. Of course, we know exacdy how much

comput-

power these operations require. Further, and by no coincidence, the resolution (number of receptors) of the fovea, the high-resolution part

ing

of the basis,

retina,

is

about equivalent to that of a television image.

Moravec estimates the processing power of the

He

then proceeds to extrapolate from entire brain,

and

is

this

large.

Which

figure the

that

retina,

computing

The

faced with a dilemma.

about 1,000 times as many neurons as the 100,000 times as

On

about one

per second.

billion operations

power of the

retina at

but

its

brain has

volume

is

figure correctly accounts for the larger

computing power of the brain?

We

can attribute the excess volume to

three factors. First, the connections between neurons in the brain are longer: the required cabling takes

up most of the space

in the brain.

Next, there are more connections per neuron in the brain. Finally, the brain contains nonneural tissues, such as the greasy myelin sheath of

many nerve

fibers.

Of

connections per neuron

Moravec, we

these three factors, only one

—

entails

shall thus take the

an increase

—

the excess of

in complexity.

Following

Solomonic decision of awarding the

power 10,000 times that of the retina. There follows an information-processing capability on the order of 10 13 calculations, which is Moravec's estimate of brain or about 10 14 bits, per second

brain a computing

—

power.

According to him, the brain

is

thus 100,000 times slower than

— HOW MANY BULLDOZERS FOR Schwartz's estimate of 10

one

19

per second. Moravec's procedure has

bits

crucial advantage: since

289

AN ANT COLONY?

sidesteps the need to rate the

it

factors required to adjust Schwartz's estimate,

we no

unknown

longer need to

guess the effective ("stick figure") processing power of an individual

we

neuron, and

are also spared the

need to assess the unnecessary

complexity with which evolutionary constraints burdened our brains.

Moravec's estimate probably

How do brain?

Not

computers

fare

well at

The

all.

closer to the truth.

lies

compared with the processing power of the computer

fastest

in existence in 1989, the

11 Cray-3, could process only 10 bits per second.

it is

therefore

1 ,000 times weaker than the

of the laboratory

level

the Cray-3

is

much

rat,

but

it

is

like the

As any computer calculates

is

How much

brain of 65 million neurons. Further,

AI work. Researchers must

Sun-4 workstation. At 2 x

500,000 times

would evenly match the

The Size of

its

too expensive to serve in

make do with machines second, the Sun-4

with

By Moravec's estimate, human brain, or at about the

1

less

powerful than a

00,000 neurons of a

1

8

human

bits

per

brain

snail!

Human Memory enthusiast knows, the rate at

but one measure of

its

which a given machine

power. Another crucial question

memory?

information can the computer hold in

is,

Similarly,

human mind, how well we think is very much a function how much we know. As it turns out, it is possible to estimate how much memory we need to function in the world. There are three ways to go about this. 4 One is to repeat what I have

in respect to the

of

just

done

for the brains calculating power: that

is,

examine the

anatomy and work out estimates from hardware considerations. direct

method

is

to survey the

could also deduce

how much

knowledge of an average adults

know from how

brain's

A more

adult. Third,

fast

we

they can learn,

how long they live. To start with the first

and

approach, what happens in our brain when we remember something? Scientists are still very much in the dark about memory. They know plenty about the periphery of brain operation, such as

how we

perceive the world through our senses, or

how we

activate

our muscles to act on the world. What happens between, though,

mains very

much

a conjecture. Researchers

do not

on one particular response to a perception, or memories on which we base this decision. One can make settle

re-

know how we how we store the

really

plausible

290

Al

assumptions, though. Consider a mouse that

of

snarl

a cat: this instinctive reaction

our own.

First,

sensor

the mouse's ears send nerve pulses to

cells in

other neurons in the brain, which start firing

of neurons assembles

activation pattern ally,

further

response to the

flees in

mechanism probably resembles response. Thus, an

in

mouse's brain. Eventu-

in the

waves of activation reach the neurons controlling the

legs,

which send the mouse running.

The

snarl

neurons

of the cat thus corresponds to an activation pattern of

in the

same way. What happens, in

to

A human

mouse's brain.

then,

brain

would represent

when we remember

some of the neurons become active again. By this

that at least

perceived a snarl

in the

the snarl of a cat

response to another cue, such as the sight of an angry cat?

assume

it

It is logical

when we last memory is also

that fired

token, a

a neural activation pattern.

Can we

identify in the brain the elements responsible for eliciting

What can

such an activation?

cause a certain group of neurons,

the billions present in the brain, to

become

active

of

all

a

among

sudden? All

evidence points to the synapses, these microscopic gaps between neural terminations.

The

average neuron makes contact with ten thousand

others through synapses of various conductivities.

through more or

Of the fire in

less

ten thousand

A

synapse can

let

of the nerve pulses emitted by the source neuron.

downstream neurons, those

that are

more

likely to

response are those with the more conductive synapses. Thus, one

can assume that highly conductive synapses connect the neurons representing an angry cat to those representing the snarl of a synapses

probably store the information causing the

If this

is

the case,

encoding memories of

a synapse

bits

we can

as follows:

can have sixteen

of information, since

a

this

mean

cat.

Hence,

of memories.

estimate the capacity of the brain for

Assume that the degree of conductivity values. Then the synapse can store four

sequence of four

numbers from to 15. The 10 15 synapses room for 4 x 10 15 bits.

Does

recall

bits

the brain can actively use that

tion? Probably not. Synapses are just the

can represent the

in the brain

would then hold

many

mechanism

bits

of informa-

that induces pat-

terns of neural activation in response to stimuli or other activation

however, many fewer neurons than synapses. If a

patterns.

There

memory

item corresponds to a group of neurons firing together, then

are,

there will be fewer such items than synapses also. In the past few years,

AI

researchers have devoted

much

attention to studying networks of

2 91

HOW MANY BILLDOZERS FOR AN ANT COLONY?

neurons. Experimental results, as well as mathematical theory,

artificial

show that the number of the number of neurons in fewer

bits

known

one can store

bits it.

Further, an

than there are neurons in

in

such a net depends on

artificial

net can store typically

For example,

it.

a type

of neural net

Hopfield network, containing n neurons, has a storage

as a

capacity of 0.1 5«. 5

Assuming a

a capacity of about

we end up with

similar ratio for the brain,

of usable memory.

5 billion bits

1

A more direct way of finding out how much each of us knows is the game of twenty questions. It involves two people. Player A thinks of a and player B must find out what

subject,

A

will

in

twenty questions or

only answer with yes or no.

and known to both primary information

not count. in just

It

The

less.

—

like

"300,286

turns out that a

player's

The

through.

sift

wins

if

B

can't guess the subject

must be

target item

that

clearly identifiable

one must deduce from other the product of 482 by 623" do

players. Facts that

good

about twenty questions

have to

A

by asking questions

it is

—

first

memory in two groups:

is

—

come to the answer how many items you

player can usually

this fact reveals

question

you

lets

partition the other

the items corresponding to a yes answer,

and those corresponding to no. The next question divides one of these groups in two again, and so on. Since twenty partitions are required for

you is

to

end up with

clearly

a single fact, the

2 20 or about ,

because players

,000,000.

1

We

number of items

It is

to

A

Eiffel

Tower. Items

only, or too sensitive for casual evocation, will be avoided.

not farfetched to multiply by another factor of 2 to compensate for

this effect:

A

this figure

choices to neutral items of

will typically limit their

mutual knowledge, such as Marilyn Monroe or the

known

choose from

to

must, however, correct

we

are

now up

to 2,000,000 items.

knottier issue concerns the hidden information corresponding to

unconscious or informal knowledge. For example,

does a recipe for

knowledge

how

that enables us to interpret

tions in people?

how much memory

to tie shoelaces, say, take up?

Such knowledge

will

body language or voice

scious?

Not

necessarily.

enough room

also

There are two reasons for a mental is

tions ("I desire mother, but

probably accounts for

activities

Are most of our memories, then,

remain unconscious: one

occur

cases,

awareness for

is

all

that there

his or her

is

at

uncon-

activity to

forbidden!"); the other reason,

many more

in a person's

inflec-

the repression of painful associated it is

the

never appear as an item of the

game. Psychologists consider that most of our mental the unconscious level.

What about

emowhich

simply not

mental

activities.

292

Al

For example, consider your behavior when you drive while earning on a conversation.

down not

You

will steer left

or right, watch for other cars, slow

or speed up as required without any conscious decision. That does

mean you

are ignorant

able to stop by stepping

of the

on

could

come

unconscious, merely

just a

that

it is

good

memory

An sents

7

Let us boldly "guesstimate" again

deal!

than one

Lincoln," to

A

lent

of

1

What

1

are

now up

to

it

would correspond

to an

example, "I was thinking of

of information does the Abraham sequence "Abraham Lincoln"

First, the

blank,

we

being.

have called an "item" repre-

I

says, for

how many bits

Lincoln structure correspond? contains 14 characters plus

human

Typically,

bit.

entire data structure: if player

Abraham

this effect:

in a typical

important question remains.

much more

game. So

not most of our memories that are

and multiply by 2 to compensate for 4,000,000 items of

being

You are simply too busy much unconscious behavior uses

to full awareness in the twenty-question

one might reasonably argue

as

the brake pedal.

to pay attention to these details. Thus, facts that

of driving, such

technicalities

which probably requires the equiva-

5 bytes of storage space in the brain. (A byte

is

a set

of eight

The sequence of phonemes corresponding to the pronunciation of the name probably requires about as much. A few years after history classes, most of us probably remember only a sketchy biography of the sixteenth president: "He taught himself law and campaigned against slaver)- as a congressbits. Digital

computers use bytes to represent

man. His election

North

as president caused the Civil

to victory, emancipated the slaves,

Address.

He was

assassinated in 1865

of

internal representation

from the a

string

tion requires

this

much

it

less

is

attending a play."

Our

certainly very different

takes to write

no reason, however,

is

War. Lincoln led the

and delivered the Gettysburg

when

information

of 265 characters

computer. There

characters.)

it

down

or store

it

in

to believe that this representa-

information in our brain. If

this

were

so,

we

a much more we needed much more information than a few store these characters, it would mean that our speech

would probably have developed

concise language and

writing. Similarly, if

hundred bytes to is

much more

efficient

than our thinking:

this is

hard to believe. Let us,

therefore, accept that this short biography of Lincoln requires the equivalent

of about 265 bytes of storage

What about

Lincoln's face?

in our mind's eye,

we

thousands of others.

Even

in if

our brain.

we cannot visualize him

precisely

could certainly recognize his photograph

How many

bits

among

of information does one need to

293

HOW MANY BULLDOZERS FOR AN ANT COLONY? store a recognizable likeness

TV images into

of a face? Not much,

it

turns out. Digitizing

of numbers, with varying resolutions, shows that

arrays

an image of 20 x 20 pixels (dots) of varying gray levels provides a very recognizable likeness. With 4 bits per dot, store such a picture,

which brings the

size

would take 200 bytes

it

to

of our Lincoln data structure

495 bytes.

to

Yet another item of our internal representation of Lincoln

the

is

emotional aura surrounding assassinated presidents. Something in the data structure

must be pointing

to the emotions awe or sadness.

require pointers to other items of information: the

War refer

Civil

words

was

information

we

a

man and

could tap

a politician

if

—

required.

can estimate

more

how many

bytes

these relations require by reference to conventional bases.

Some

by the LISP language, require pointers between data

These pointers serve much the same purpose

memory model

above. In a LISP data structure, as

reserved for the pointers as for the data. Thus, brain, the is

amount of memory required

equal to the

How much ture

computer data

kinds of them, such as relational data bases or the data

structures used

items.

we know still

categories that contain

We

also

congressman or

us to other complete data structures. Further,

that Lincoln

We

memory

let

as in our human much memory is

us assume that in the

for relationships

between items

required for the data items themselves.

information does one require,

"Abraham Lincoln"? The 495

for the data struc-

finally,

bytes above correspond to about

4,000 bits of direct information. Doubling this for pointers brings us to 8,000 bits for one item of the twenty-question game. For the 4 million

items of information the in

memory,

the

game and other

would amount

considerations

is

in surprising

agreement with the one

per neuron, or 15 billion

Looking

at

how we

we

got by assuming 0.15

bits.

As witnessed by

check on

this figure.

little

of our

table,

we must painfully learn

abilities are

inborn.

the helplessness of a

From walking to

virtually

all

of the

skills

newborn

the multiplication

and knowledge we

we learn we knew how fast we estimate how much of it the basic

in the world. Yet, in less than

the basic material that will support us in

life.

twenty years,

If

new information, we could "human knowledge base" contains. Certainly, we do not commit new information

can absorb

of

involved, this

gather information provides yet another cross-

baby,

need to function

show we hold

to about 32 billion bits. In view

number of approximations and informed guesses

figure bits

the total

to

memory

as fast as

294

Al

our senses feed

The

to us.

it

optic nerve, for one, sends over a hundred

million bits of information to the brain at each second. Yet

and remember only

happens when you

of

a tiny fraction try to learn a

this information.

we

interpret

Consider what

page of text by heart. At a resolution

equivalent to 300 dots per inch, your optic nerve can send over to the brain the entire contents of that page in about a second. Yet,

glance at an open

book

and then read the page

for a second,

mind's eye, you'll discover that you can retain

most

at

a

if

in

you

your

few words.

Further, rather than corresponding to the image of the words, which

memory

requires thousands of bits per letter to describe, your

highly abstracted description of the

be a

will

words you recognized while glanc-

ing at them. This encoding probably requires only a few bits to store a letter.

bles

In

fact,

show

that

experiments on memorizing random sequences of

we can absorb new

per second. 6 Learning entire rate,

even

at

100

information only

at rates

sylla-

of 100

bits

per second means memorizing an

bits

page of text (about 400 words) in

less

than three minutes. At that

an actor could memorize his lines by reading them once aloud! Yet, at

such breathtaking speed, twenty years of continuous learning

eight hours a day

would let you

at

digest only 21 billion bits of information.

We thus have four estimates of the size of human memory. Assessing it

from the number of synapses leads

million billion bits. But as

I

to the astronomical figure of 4

pointed out, there

the capacity of synapse storage

is

no

and the number of

direct link

between

explicit items repre-

The other three estimates give much lower values: number of neurons in the brain and neural-net theory yield 1 5 billion bits. The twenty-question game leads to 32 billion bits. Learning rate and duration give 21 billion bits. The relatively close sented in the brain.

considerations from the

agreement of these three estimates

somewhere

(2.5 gigabytes) as is still

a lot

lets

in the range they define.

an estimate of the

of information:

it

one hope

Thus,

I

that the true value lies

shall settle

memory

store that

billion bits

corresponds to slighdy more than

pages of printed text or twenty-five hundred books

Can computers

on 20

capacity of the brain. This

much

like this

a million

one.

information? Yes: by this yardstick,

The Cray-2 supercomputer, memory capacity. 7 Even AI research budgets allow scientists to come close: As I write this the typical AI workstation offers about 200 million bits of random access memory, only 100 times less than the brain. As we saw in chapter 10, the Cyc common-sense knowledge base will be slighdy smaller than our our machines have already overtaken

built in 1985, already

had 32

us.

billion bits

of

295

HOW MANY BULLDOZERS FOR AN ANT COLONY?

estimate of the brain's capacity (about 8 billion bits instead of 20).

workstations will have that

much random

access

memory

AI

at their dis-

posal in a very few years.

REACHING HUMAN EQUIVALENCE Despite

this essential parity

of the board

process information thousands of times

in

memory, machines

more

we

slowly than

still

do. Be-

it is tempting to compare Computer engineers, by contrast, like to think

cause of its myriads of cells working together, the brain to an ant colony.

of

their

mainframe machines

of data through deed, since in

our

it

skulls.

their

as bulldozers shoveling

about mountains

unique central processors. Puny bulldozers,

would take thousands of them

to

in-

match the ant colonies

many of the disappointments a jet engine, they have to make

This fact certainly explains

AI researchers have met. If the brain is do with the equivalent of bicycles! Rather than uncover the secrets of intelligence, they must spend most of their time programming around the weaknesses of their machines. Yet, as

closing the gap.

Soon computers

will

we

will see next, engineers are

approach the power of the human

brain.

The

Generation of Computers

Fifth

As I described in chapter 1, the first generation of computers was based on vacuum tubes: orange-hot filaments glowed in various computing machines from 1943 to 1959. Even during those years, progresses in vacuum tube technology cut down by a factor of 20 the time needed to perform an addition. The gain in cost per unit of computing power was even more impressive. In 1943, it cost about one hundred dollars to buy one

bit

per second of computing power. Sixteen years

later,

it

cost less

than ten cents.

Generation

new machines

2,

based on single transistors, accounted for most of the

until 1971.

From

then on, computers were built out of

integrated circuits: silicon chips containing

and

finally

first a

few, then hundreds,

thousands of microscopic elements. These formed the third

generation of computers, and lasted until 1980.

Around

that year,

it

296

Al

became possible single chip;

computer on

a

and by 1985, these microprocessor chips contained up to

a

to put the entire processing unit

quarter of a million elements.

Thus was born

of

a

the fourth generation of

computers.

As I write this in the early 1990s, the upward spiral of computer power continues to accelerate. If you've ever pondered the economics of replacing an aging computer, you may have

felt a

kinship with the

future space traveler faced with the star ship problem: a better time to

leave

always next year, because by then ships will be faster and will

is

get you to your destination sooner.

machine than

will,

on

same

this year's for the

Let

me

So

is it

with computers: next year's

the average, offer 50 percent

demonstrate

this

more computing power

price.

tendency by focusing on two particular

machines. 8

The Zuse-2, the first electromechanical computer built in 1939 by the German engineer Konrad Zuse, would then have cost about $90,000 in today's money and took 10 seconds to multiply two numbers. By contrast, the Sun-4 workstation, introduced in 1987, cost $10,000 and can multiply two numbers

400 nanoseconds.

in

In raw power, measured by the admittedly crude yardstick of the time required to multiply two numbers, the Sun-4

than

predecessor. If

its

power, the comparison less to

To what

do

is

we

25 million times faster

even more favorable:

a multiplication with the

it

costs 225 million times

Sun-4 than with the Zuse-2.

understand the staggering implication of these similar

improvements would bring about

A luxury car of 1938 — say, a Cadillac in today's

money.

It

what the Sun-4

twice the speed of

No

fairy

waved

compares the right after

it,

is

on

— would have

cost about $30,000

and do 3

it

billion miles

wand suddenly

per gallon!

to induce these changes. If

and progresses on to the Sun-4, there

improvements

in

were to the 1938

would cost only $3,300, run at

relay-activated Zuse-2 to electronic

in price or dramatic

figures, consider

applied to automobiles.

a gallon. If today's Cadillac

to the Zuse-2,

light,

a

if

reached a top speed of 60 miles an hour and

traveled about 15 miles car

is

consider the cost per unit of computing

one

machines introduced are

no abysmal drops

performance anywhere. Instead,

smooth evolutionary process is revealed. Hans Moravec courageously calculated and plotted the cost per unit of computing power of sixtya

seven consecutive machines, starting with a mechanical calculator built in 1891.

9

These data points

clearly

show

that, for the past sixty years, the

cost of computing has decreased by a constant factor of 2 every other

HOW MANY BULLDOZERS FOR As

year.

a result, the

mainframes of the 1970s are the desktoppers of

now

today.

Mainframes of the 1960s can

many

electronic wristwatches contain

machines

297

AN ANT COLONY?

be stored

in a single chip,

more elements than

and

these early

did!

Although speculators

who

blindly extrapolate stock prices

from past

tendencies usually end up broke, there are sound arguments for applying yesterday's trends to tomorrow's computers.

of the past

ress

in the evolution

than

In

one

we connect

even

could

essentially

is

cally possible to build a if

many of these arguments

fact,

equivalence problem

brain

staggering prog-

of the technology. Since we are also dealing with the

behavior of an industry, technical.

The

stems from profound structural processes

sixty years

are

argue

economic. Indeed,

economic rather that

the

brain-

it is

now

techni-

machine with the raw computing power of the thousand Cray-3 supercomputers. And, even

a

though managing such interconnection, and programming

form

like a

human

raw power would be could do

that,

would

brain,

still

it

to per-

formidable problems, the

raise

we

experiment with. Before

available for us to

however, we would have to deal with the small matter

of finding the twenty

billion dollars this

network would

cost.

the problem of building human-equivalent hardware boils

reducing the cost of processing power to affordable

examine, therefore,

why we

levels.

Thus,

down Let

to

me

can expect the sixty-year-old trend of de-

creasing prices to continue. First, the regularity

of the price curve

is

to a large extent the result

a self-fulfilling prophecy. Manufacturers, aware

of

of the tendency, plan the

introduction of new products accordingly; hence, the absence of drastic

jumps

in

cost/performance

ratios.

Manufacturers introducing a product

no

incentive to

much lower.

Instead, they

well ahead of the competition in performance have

reduce prices

pamper

drastically,

even

if their

their profits for a while, until

costs are

competition forces them to accept

lower prices.

Competition in the computer industry $150-billion world 10

is fierce.

computer market, companies

To

grab a share of a

are willing to scram-

number of people developing computers, and on the rise. Since computer companies spend a constant fraction of their revenues on research and development, resources for computer development grow about as fast as the computer market. They pushed ahead by about 1 5 percent a year since ble.

For

this reason, the

the resources at their disposal, are

1960. Since this growth

is

much

faster than that

of the economy,

it

will

298 slow

Al

down

eventually. (Otherwise, a continued

growth would lead to the

impossible situation of everybody developing or building computers.)

Even

if

the

development activity

number of people and

dollars

devoted to computer

levels off, the total intellectual resources available for this

would

increase exponentially.

still

The reason

because com-

is

puters are largely designed by other computers. Indeed, involving puters in their

own

conception can have dramatic

problem of planning the paths of

on printed

metallic traces

com-

Consider the

effects.

circuit

boards. In the assembled board, these traces connect together the pins

of different processing chips. They available

on

have to

all

the board, while maintaining

fit

in the restricted area

minimum

distances between

each other. Typically, out of a multitude of possible combinations of paths, only a few satisfy these constraints. In the 1960s and 1970s, lay-

ing out these paths with pencil and ruler used to take months. Worse,

changing a design

took almost as long as starting anew.

after testing

Nowadays, computers perform

this layout automatically in a

matter of

hours. Similar gains occurred in implementing those procedures at the

chip level: integrated circuits are also designed by computers. In coming years,

computers ever more powerful

of the design and construction of the design

(or,

will gradually

assume

a larger part

their successors, further speeding

up

reproduction) cycle.

Economies of

scale should also

speed up the rate of price decrease.

Present computers typically contain only one, expensive, processing unit.

Future machines, however,

will consist

of identical components which

ally millions,

of thousands, and eventuwill serve as

and processors. Manufacturing these components ties will

in

both memory

such large quanti-

give rise to economies of scale comparable to those affecting

memory chips. Since there are many memory chips in a computer, they come down in price faster than processing chips. Recognizing these new economics, the Defence Advanced Research Projects Agency's goal to double the pace

of cost reductions

instead of multiplying

government hopes But even

if

components,

coming up

coming

it

One light

year, the U.S.

will result if

Nature does not cooperate. Aren't we

against basic natural barriers that cannot be

or, as

computer

is

on,

ever larger resources to perfect electronic

obvious boundary which

—

From now

every year.

Won't we soon bump our noses on the outer of

years.

computer power by 2 every other

to double

we devote

little

in

is

fast

scientists

limits

approaching

sometimes

is

call

overcome?

of computation? that it,

of the speed

the "Einstein

HOW MANY BULLDOZERS FOR

299

AN ANT COLONY?

bottleneck." In a conventional computer equipped with a single process-

ing unit, information flows between the

to

memory and

the lone processor

between swings. Infinitesimal errors

as acrobats leap-fly

murderous crashes, and the

entire

in

timing lead

computer must operate

like a finely

tuned clock. Indeed, an electronic master clock beats time, keeping

components

drummer

in lockstep, as inexorably as the

all

in a slave ship.

For the drummer to be obeyed, there should be time enough for one beat to reach

all

parts

The

the next beat.

of the computer well before the clock generates

beats are electric signals: the fact that they travel at

of

close to the speed

but no

light,

imposes

faster,

a limit

on

the

frequency of the clock. For example, the time required for light to travel the entire width of a

maximum

implies a

1

computer

-foot- wide

(megahertz, in computerese).

Many

within a factor of 10 of that

limit.

One

solution

is

1.76 nanoseconds. This

clock frequency of 568 million beats per second

would be

desktop computers already operate

to keep walking

on the path we have so

profitably followed since the invention of the transistor: that of minia-

make

Let us

turization.

tighter space.

The

the

components smaller and cram them

signals will

have

less distance to travel,

in a

and we can

only will crowding

bumps into more compo-

amount of heat

generated, but

then speed up the clock. Alas, this approach immediately

another obstacle: heat removal.

Not

nents in a smaller space increase the

having them work faster ponent. Evacuating

down

requires

will also increase the

this extra

— when

These complications

it is

just

heat generated per

com-

heat to keep the machine from melting

possible at

all

—

technological prowesses.

about cancel any economies brought about by

the extra miniaturization.

Nature herself presents us with

a

way out of

operate at positively slumbering rates.

hundred impulses per second, second of

a digital

human

different

fastest

mode of

CPU

initially

facture the

will

brain

generate about a

to the millions

of beats per

a

find

it

easy to keep a cool head.

thousand times the information-processing

computers. This performance

is

due to the

operation of the brain.

Von Neumann's and

opposed

we normally

Yet the brain packs about of our

neuron

computer's clock. Being so sluggish, each neuron

generates litde heat, and

capability

as

A

Com-

this blind alley.

pared with those of computers, the components of the

suggestion to break up computers into a

offered obvious advantages.

memory

as

many

It

low-cost, identical

was possible cells.

memory to

manu-

Since there was

300

Al

onlv one processor, variety

could be as complex as required to perform a

it

of logic or arithmetic functions. Further,

programming

to the relatively simple task

one processing

instructions to the

unit.

this layout

of issuing

reduced

a single string

of

Unfortunately the setup also

when memories inmemory of a modern

introduced a major inefficiency that became clear creased in size to billions of

computer

that

it

compares

names and addresses of

A

metropolis of

elements.

its

Running

a

program

New

fashion. First load the

back.

New

is

also houses millions

one road connects these two

that only

and allows through only

few

a very

bits

TV

set in 7

driving: they

your

car, drive

iron, drive to

Los Angeles, and

to

it

LA, and come back. Take the

LA, and so on. To enhance matters

have improved the data-transfer

computer. Unfortunately,

this

amounts

computer

a bit, cities

instead of

between the parts of

rates

up the

to speeding

circuitry,

and soon bumps into the speed-of-light and heat dissipation

to

I

mentioned.

move

of

von Neumann computer is like moving your York to Los Angeles in the following senseless

Take the laundry

coffee pot, drive to

tions

of

cities.

in a

manufacturers have recently tried flying between the

a

could store the

it

York or Los Angeles.)

at a time.

household from

come

the

to a large city. (Indeed,

inhabitants of

all

takes a long time to travel

It

is

own, the processing unit

The problem

information

So huge

bits.

all

An

analogue to the obvious solution

items of your household at once

computer. Each

bit transferred

requires a separate wire,

—

—

limita-

using a van

not possible in a

is

between processing unit and memory

and there

is

a limit to

how many

of these can

be crammed into a machine. Over the years, manufacturers have wid-

ened the data path from 8 machines.

A

time to 32, and even to 128 for large

bits at a

small improvement, this

amounts

to

little

more than

letting

you move both the coffee pot and laundry iron together! In terms of is

my

two-city analog}

7

surprising. It consists in

mingling the two

New

York

memory.

cities

,

the solution adopted by our brains

moving Los Angeles

so you don't have to

plays the role of processing unit,

It

move

processing unit. In the brain, there

is

at

New all!

In

York and

my

and Los Angeles,

turns out that the brain does not

between these two functions: each neuron serves a

to

make any as

both

a

fable,

that

of

difference

memory and

indeed no clearly identified center

of intelligence comparable to the processing unit of computer. For example, despite long-standing

a

von Neumann

efforts, neurologists

have

never been able to pinpoint a center of consciousness. The neurosur-

HOW MANY BULLDOZERS FOR

geon Wilder Penfield suggested the upper brain stem

301

AN ANT COLONY? it

might

in the

lie

combined action of

and various areas of the cerebral cortex. 11 Others

down and

pointed out that consciousness has to do with laying

memories of the world. In

central role in this function, ness.

12

Another view holds

recalling

hippocampus, which plays a

this case, the

might qualify for the seat of conscious-

that our ability to

communicate makes up

our most obvious mark of intelligence: the language centers, located the left cerebral cortex,

good reasons

would then bear the palm. 13 There

many

areas.

14

in

however,

of the brain handle

to believe that, although various parts

special functions, consciousness arises

are,

from the combined operation of

down, appear

Likewise, long-term memories, once laid

distributed throughout large areas of the brain.

What

are the advantages

von Neumann a myriad

architecture?

of

this distributed configuration

For

starters,

of operations concurrently. This

the torrent of information your eyes send

is it

over the

allows the brain to perform

it

how your brain

(millions

and let you instandy recognize what you're looking

can analyze

of bits per second),

at.

Roughly speaking,

your brain separates the image into hundreds of thousands of dots, each separately analyzed

by several neurons.

A pure von Neumann machine

would, by contrast, slowly process each dot in succession.

Computer

scientists call "parallel

cation to a single task of ers.

many

processing" the simultaneous appli-

processors, be they neurons or comput-

In addition to the speedup inherent in getting

job, applying parallel processing to

advantage:

it

barrier. Since

are

amounts

computers

more workers on

the

offers another potential

to nothing less than breaking the light-speed

processors in a parallel machine

no longer enslaved

to the

semi-independent units could

drumbeat of

now

be made

work

separately, they

a central clock.

as small

and quick

These as

we

want them.

Through such parallelism, Nature will allow us power of our computers at a steady rate for

the

to keep

on increasing

a long time to come.

Eventually, individual processors will reach microscopic dimensions.

The emerging tures in

science of nanotechnology 15 will soon

which every atom plays

its

By common agreement among computer machines are those that implement way.

A

few of these machines are

Connection Machine,

built

let

us build struc-

assigned role.

parallel

now

scientists, fifth-generation

processing in an extensive

in existence: for

by Thinking Machines,

Massachusetts, with 250,000 processors.

As

I

Inc.,

example, the

of Cambridge,

said in chapter 8,

ma-

302

Al

chines based

on neural networks,

in

which microscopic components

will

emulate the neurons of our brains, are being contemplated.

Duplicating the Brain's Processing I

can

before

now attempt to we close the gap

human

brain?

answer the question raised in

How

earlier:

long

processing power between computers and the

have summarized

I

Power

and 11.2 the

in tables 11.1

estimates about the brain's computing

earlier

power and information-storage

capacity.

Since

we

still

know little about how the

brain works, different avenues

of investigation lead to extremely different

results.

cited for the information-processing capacity differ

by

a factor

the brain (table

of 100,000.

do not

1 1 .2)

The two

of the brain

estimates

I

(table 11.1)

My

estimates for the

memory

fare

any better, being

six

capacity of

orders of magni-

tude apart.

Our

mightiest computers offer only an insignificant fraction which-

ever value

we adopt

take for the

for the brain's processing power.

upward

spiral

Various answers appear in tables 11.3 and for a

speedup

in the rate

How long will

of hardware progress to close 1 1 .4.

this

it

gap?

Despite the arguments

of computer improvement,

I

have taken the

conservative view that the sixty-year-old tendency of doubling every

other year persists. I

have taken for benchmarks

in tables 11.3

and 11.4 the Cray-3

supercomputer, built in 1989, and the Sun-4 workstation, built

table

Two Estimates of the Computing Power

11.1

Argument

in 1987.

of the Brain

Estimate

Detailed modeling of neurons (Schwartz)

10 19 bits per second

Comparison of the

10 u bits per second

retina with similar

hardware

table 11.2 Various Estimates

of the Information Storage

Capacity of the Brain

Argument

Raw

synapse storage

Neural-net theory and number of neurons 20-question

Human

game

learning rate and duration

Estimate 4 x lO 13

x 10 9 32 x 10 9 9 21 x 10 15

bits

bits bits bits

HOW

MAW

303

BULLDOZERS FOR AN ANT COLONY? table 11.3 Estimates for

the

Year

Human

Supercomputers Will Reach

Which

in

Equivalence

Best Case

Worst Case

Processing power

2009

2042

Memory

1989

2023

table 11.4 Estimates for

the

Year

Which Desktop

in

Human

Computers Will Reach

Equivalence

Best Case

These

Worst Case

Processing power

2025

2058

Memory-

2002

2037

tables

list

the years in

which machines of a cost equivalent

to the

Cray-3 (about $10 million) and the Sun-4 (about $10,000) should reach

human

The

equivalence.

"best case" columns correspond to the weaker

power and memory in tables 11.1 and 11.2; the "worst case" columns correspond to the stronger estimates. The large discrepancies between estimates make remarkably little

estimates of brain-processing

difference

on

According to

dates.

supercomputers

will attain

table

human

1 1 .3,

if

the

weak

estimate

is

right,

equivalence in the year 2009. If the

strong estimate holds, this sets us back only thirty-three years, to 2042!

Indeed, is all it

if

computer power doubles even other 7

takes to

tables, the

improve by

roadblock

is

a factor

of 100,000. Also,

supercomputers

as

is

clear in

both

we will always reach the From the first line of table

processing power, since

required memory' about twenty years 11.3,

year, triirty-three years

will attain

earlier.

human

equivalence around 2025,* give

or take seventeen years. According to table 11.4, desktop machines will

have to wait

2041, with the same error margin.

until

After these dates,

than

we

abilities

are.

We

of the

we can

expect our machines to become more clever

have already done Nature one better for

human

body.

Our machines

all

physical

are stronger, faster,

more

may be advanced: the September 1992 of Electrical and Electronic Engineers) Price estimates exceed noted that "engineers expect teraflops machines by 1996. U.S. S100 million" (page 40). A teraflops is the approximate equivalent of our weaker estimate for brain power. This power, however, will come at ten times our target price of S10 million. Further, the degree of specialization of these early teraflops machines * Recent

developments indicate that

this date

issue of Spectrum (the journal of the Institute .

makes

it

.

.

unlikely that any

.

.

.

amount of programming could endow them with

intelligence.

304

Al

enduring, and

more

accurate than

we

are.

Some of them have

eyesight or hearing. Others survive in environments that

we

suffocate us. Shouldn't

sharper

would crush or

expect to improve upon our mental

abilities

just as well?

Avoiding Nature's Mistakes we build into our machines the strength of our minds, with much to spare, but we can also avoid duplicating the many weaknesses and inefficiencies of our brains. Indeed, when building artificial minds, we enjoy much more freedom that Nature had in Not

only can

eventually

building us. First,

on

we

of the limitations on material and structure imposed

are free

must grow, reproduce,

biological organisms. Living cells

selves,

and move over to

They must

their

proper positions

body

repair

them-

early in

life.

constantly absorb nutrient material from their environment

and evacuate waste. Most of their these ends.

in the

An

any of these

tasks.

internal structure

and functions serve

neuron, however, would not have to perform

artificial

function would reduce to generating electric

Its

of a biological neuron. Thus, we can expect the

signals similar to those

structure of an artificial

neuron to be much more simple than

natural one. Further,

could use materials that transmit impulses mil-

lions

of times

it

faster than

protoplasm and process

that

signals that

of a

much

faster.

Yet another

limitation our

machines

will

dispense with has to do with

blueprints. Nature's blueprint for our bodies, the

DNA molecule, does

not contain enough information to

connections of each

specify' the

make do with general instructions issued cells. What we know of neuroembryology

neuron. Instead, Nature must to entire classes of brain

shows

that in the early stages

through brain

tissue.

axon of the adult ends meet

cells

then bind to the

cell.

of

of life, brain

cells

emit filaments that travel

These filaments eventually form the dendrites and

They

travel

more or

less at

a kind that chemically attracts

cells in

random,

connections that become synapses.

stand the limitations this

mode of

performance, consider the following

until their

them. The filaments

To

under-

construction places on the brain's fable,

which

I

have called "Harry's

Plight."

new comOgomongo. Harry has

Harry, an electronics engineer, has just taken charge of a

puter assembly plant in the remote country of

MAW

HOW

accepted a mission no one else in the

computer

that the unskilled

from component

Ogomongans

the

305

BULLDOZERS FOR AN ANT COLONY?

ther can they

tell

chips.

Harry's dismay,

are incapable

wants: to design a

of reading

it

can assemble

soon becomes

easily

clear that

connection diagram. Nei-

a

apart chip models, except by their colors. Since dif-

ferentiating the pins

Harry has to

To

company

Ogomongo

workers of

on the chips is also a little hard for Ogomongans, mounting instructions that typically read: "Con-

settle for

nect any pin of a green chip to any pin of a yellow chip." Harry's considerable challenge to design chips of a kind that

connected in

this

haphazard way, produce

computer.

a

chance that compatible pins on different chips

number of

increases the

To

will,

when

increase the

will connect,

pins per chip. Second, he adds

now

It is

Harry

some

first

intelli-

gence inside the chips and decrees that each newly assembled computer

undergo

will

"running in" period of a month. During

a

this time,

each

chip sends out, through each of its pins, exploratory, low voltage pulses. It also listens

to pulses emitted

by other

Each pin of each kind

chips.

of chip emits a characteristic pulse pattern, enabling chips on each side

of

connection to check the validity of

a

mechanisms break

internal

tions of the right kind are maintained

Much to the this

There

and more expensive than

company wants improves

mongo

and somewhat to

his

own,

eventually does produce a working

com-

is

a

hundred times bulkier

number cruncher. Harry's The Ogomongans, however, feel it image, and insist on buying it. Since Ogo-

a conventional

to close the plant.

their international

sits

and strengthened.

only one snag: the machine

is

designed

of the wrong kind. Connec-

surprise of Harry's colleagues,

Rube Goldberg procedure

puter.

this link. Specially

off connections

on newly discovered

oil fields

amounting

to half the world's

reserves, they can well afford to.

This

is all

processes

fantasy,

of course

— but any resemblance

to existing biological

intentional!

is

In addition to the indiscriminate assembly of suffers

from

their lack

of

balanced mechanism, and

Yet we do not pensates for

Computers ways to

it

feel

also benefit

a price:

we

all

lose

A

a

neuron

is

a

parts, the brain

its

complex, delicately

hundreds of thousands every day.

any the worse for

by having

let their

must pay

reliability.

this loss

because the brain com-

large amount of redundancy

from redundancy, and engineers

machines

tolerate

minor component

making computers more

in

are

its

now finding

failures.

resilient requires

circuits.

Yet they

more com-

306

u

ponents. For this reason, building an intelligent machine out of parts

more simple and robust than neurons would increase its performance. The brain evolved through a process of small-scale, local changes spanning millions of years. intelligent designer

structure of the cortex

it

embodies many elegant features

shows. In

many

respects, the brain

schoolhouse turned into a major

one-room cabin with the children of the

a

first

school needed another tion.

Over

layered, circuit-board-like

prime example. Yet the brain's overall

a

is

wood few

city

Midwestern country

Then came

started with a

It

enough

stove, spacious

settlers.

room

like a

is

high school.

to

accommodate

the railway station: the

to handle the suddenly doubled popula-

To

the years, classrooms multiplied.

keep a studious atmo-

sphere, workers had to pare precious square feet from each

up linking

that an

expanded gradually, without benefit of advance planning,

architecture

and

It

would not disown. The

room

plumbing and

corridors. Later, installing indoors

to set

electricity

required major surgery, which gave the principal a severe headache.

When

it

meant

for one, the

became necessary to add mayor and the

The aldermen,

leery

a

second floor on

city*

a structure

never

engineer almost came to blows.

of raising taxes for

a

whole new building,

finally

overruled the engineer and hired a contractor themselves. Twenty years later,

congested plumbing and

wavering

lights

conditioning, corridor

air

traffic

jams, and

prevented any further expansion of the school. The

council voted the

site

and erected

into a park

a

new

city

school elsewhere.

Evolution does not have the option of starting over, and our brains still

contain the original cabin cum

wood

stove.

out of the upper end of the spinal cord. The the brain of our reptilian ancestors. tory

and attack prey or enemies,

around the

reptilian brain

this is the school's

second

is

it

Lemon-sized,

grows

Programmed

to stake out a terri-

Wrapped mammalian brain:

holds our darker instincts.

the limbic system, or old

floor.

it

reticularformation™ is in fact

Developed from centers

that

govern

mammals, the limbic system is the seat of emotions. It enabled our warm-blooded ancestors of a hundred million years ago to care for their young. Its programming often contradicts the reptilian brain, and many of our internal conflicts have no other origin. The smell in primitive

cerebral cortex

layer

holds our higher reasoning functions and forms the outer

of the brain.

It talks to

which somehow coordinate

the inner parts through its

action with theirs.

equivalent in our fictional country school. tect trying to design a better brain

At

many nerve fibers, The cortex has no

that level, a

human

archi-

probably would have started over.

I•

W

IA*

Our

1

Y

1 L L

B

Z E

IS

1

F

IITCtlflf!

II

old rriend the retina offers a striking example of

evolves impressively elaborate fixes to

Evolution

structures.

hit

upon

As you

into place.

make up

7

how Nature

no longer adequate

tor

the retina's peculiar layout early in the

development of vertebrates, and it

3

unthinking mechanisms

its

later

locked

the retina includes photoreceptors, which

recall,

turn light into nerve pulses, and layers of nerve cells that preprocess the

image. These

pack the number-crunching power of

cells

mainframe computer. Nature made the front, so light tors.

must pass through the

early mistake

a

modern

of placing them up

reach the photorecep-

cell layers to

This arrangement put a major design constraint on the data-

IBM

processing part of the retina: just imagine

make

trying to

their

computers perfectly transparent. Yet Nature rose to precisely that challenge in evolving our eyes: the nerve

There

is

cells in

the retina are transparent.

yet another difficulty: the nerve cells' position forces the optic

nerve to pass through the photoreceptive layer to reach the brain, creating

We

a blind spot in our field of vision.

more

sleight

the image and covers

""What

if

it

possible one?"

It

is

it

because, through

up the blind

spot.

realize

make

may

"Couldn't a

ask.

roundabout design the only

this

seems not, because the independendy evolved octopus

and squid do have Science,

see

wasn't an early blunder?" you

we do not

constraint

do not

of hand, our brain interpolates from neighboring parts of

their

photoreceptors up front. In a classic paper in

the eminent biologist Francois Jacob maintained that evolution

He

not a rational designer but a thinker.

many more examples of

illustrated his point

with

biology mixing slapdash foundations with "

prodigies of workmanship.

Many

1

find this iconoclastic view of evolution shocking, indeed, im-

perfection in Nature's creations contradicts

of cosmic order. Personally,

I

mistakes and keep forging ahead themselves.

And who knows:

Now

that

we

realize

ently than Nature. intelligent

artificial

Much

machines

will

minds

that

as airplanes

operate

on

a crucial

our imperfec-

it

will

pays to design a

same

probably

littie differ-

have wings but do not

the

its

next batch of intelligent beings.

our other duplications of natural funcuons, we

discover in building

view

for

the blunders

perhaps creating our brains was

we may help weed them out of the

m

ecologist's

make up

more impressive than

step in this self-correcting process? tions,

many an

rind Nature's ability to

flap

them,

principles as their natural

equivalents, but exploit these principles better. Streamlined, robust, and faster,

they

may

well surpass our

minds the way

airliners

do sparrows.

308

||

SOFTWARE: THE STRUGGLE TO KEEP UP »3o far

I

have compared the brain to a telephone switching station and

looked only

at

how

fast

can switch

it

lines.

the switching station has to be wired to

Since there

a lot

is

more

fact that

the right connections.

to intelligence than simple line switching,

time to ask this question: If we do develop, century, hardware powerful as the brain does, will

have neglected the

I

make

enough

we be

hardware? In other words,

will

in the early part

to process as

program

able to

manv

bits

it is

of the next per second

intelligence into this

software progress follow hardware devel-

opment? If the past

any indication, hardware and software development are

is

closely linked. In general, software needs can provide the motivation for

and point the way to appropriate directions

hardware development.

in

Conversely, weaknesses in hardware can not only act as a powerful

brake on software development but also divert is

no question

that the relative inadequacy

early progresses in artificial intelligence.

how

it

would

nets,

on word disambiguation simply because

never could

4).

We

also

chapter 6 that the advent of expert systems had to await the

availability

of computers with enough

knowledge and the programs needed

work was special

theory

test his

the computers of the mid-

1960s couldn't hold enough word definitions (see chapter in

me

recalled for

for years over a

toil

founder over lack of memory. 18 For example,

Ross Quillian, the inventor of semantic

saw

There

into blind alleys.

Marvin Minsky

early researchers (himself included)

program, only to see

it

of early computers hindered

deliberately

performed

memory

to hold large

to quickly sift

through

in toy task that did

amounts of it.

Early

not require

AI

much

knowledge.

This mind-set became so ingrained that researchers didn't always realize that they

were programming around

their

instead of addressing the real issues. Consider

block-manipulating program that

1970s (see chapter

guage used for

4).

Carl Hewitt,

SHRDLU,

PLANNER performed in those days,

made up

who

the

machines' weaknesses

SHRDLU,

the talkative

wonder hack of the

invented the

PLANNER

early lan-

pointed out the following to me:

so well because, and we weren't so conscious of it

by working on only one aspect of

a

problem

at a

time

it

R

1 M

N

ft

\

BILLDOZERS FOR AN

accommodated then.

itself to the

When it explored

C

\

L

3 09

K'

single

that

one branch.

If that solution didn't

work

out,

Despite such craftiness, Marvin Minsky told me,

amount of memory by

belittling the role

MITs

of

MIT AI

"The

it

would backtrack,

much

"took delivery of the

in

SHRDLU

still

required

Without

DARPA's

financial

I

might add,

as their genius in the success

of such

Laboratory," Patrick Winston remembered,

first

megabyte memory.

It

cost us a million dollars,

He added ruefully, "It is strange my portable PC these days." 20

a dollar a byte."

megabytes

went down

the standards of the time.

researchers,

largesse probably counted as projects.

it

19 the storage, and try another branch.

all

a formidable

we had

very small machine memories

possible solutions to a problem,

branch and only used the amount of storage needed for

one

recover

A \ T

Re-examining the history of AI

in the light

to think that

-

I

earn ten

of unrealized hardware

constraints can lead to interesting revisions of accepted explanations for

why

the field took certain orientations. For example, although the

demise of neural networks

and Papert's implacable

same

tide,

in the

criticism

1960s

is

widely attributed to Minsky

of Perceptions

in their

book of

that

Carnegie Mellon's James McClelland, a major contributor to

the revival of this field in the mid-1980s, suggested an alternative expla-

nation to me.

He

pointed out that most research on neural networks

them on

involves simulating

I

don't believe

it

was

that

research in the 1960s.

I

computers:

digital

book per

se

which discouraged Perceptron

think what actually happened

A

wasn't ready for neural networks. necessary before simulations

show

certain scale

Patrick

Winston

researchers to

"We

was

the

in their paths

towards progress:

of ideas we once rejected on the grounds

of computational impracticality have become the nght way explained

how

21

hardware limitations have often led

wrong rums

are discovering that a lot

is

do some

The computing power

totally insufficient for this.

also believed that

make

that the world

that neural networks can

things better than conventional computers. available in the early sixties

is

of computation

after all."

He

conventional robots control their movements by con-

stantly recalculating the control signals they

send to their arms. Recent

experimental robots, however, can use their increased parallel processors to learn gradually

by experience which

memones and efforts to exert

310

Al

under given circumstances. "This idea had been rejected twenty years ago," continued Winston, "and a lot of the efforts that went into motion

dynamics and the mathematical approach placed. In

my

now seem somewhat

view, one of the milestones of

five years is the realization that

we can do

mis-

AI research over

the last

on vasdy

parallel

things

computers that we couldn't do before."

However, Winston was quick not solve

Don't

all

to point out that hardware progress will

of AI's problems. Raising

infer

from what

I

said that

a cautioning finger,

we

should

just

he added:

stop software

research for twenty years and wait for the hardware to catch up. In fact,

on

I'm a

litde

schizophrenic on the subject of hardware. I'm saying,

the one hand, that the availability of better hardware allows the

discovery of

new ways of doing

things.

At the same

time,

I

believe to

it

would

fact, I

take us ten years worth of current software research

do hardware bad.

Minsky was,

for his part, convinced that if hardware

a bottleneck until the 1970s, the

shoe was

now on

in the 1980s, software turned into a millstone

machines right

now

this

had constituted

the other foot: that,

around AI's neck: "The

could be as smart as a person

if

we knew how

program them." Minsky's former student David Waltz

on

we

think

could do a whole lot more with the hardware we've got. In

to

later elaborated

point for me:

In the old days, machine memories were too small to hold the

knowledge researchers wanted

to

pour into them.

Now it's

the other

way around: you aren't ever going to fill the new machines with hand code. Nowadays almost all research on learning is really aimed at making use of hardware to

hand-code certain

would then feed computer

it

in a better way. Ideally,

initial

You

some form, which would allow new knowledge on its own. 22

experience in

to acquire

you should only have

circumstances into the machine.

the

311

HOW MANY BULLDOZERS FOR AN ANT COLONY?

CONCLUSION Ihus

would appear

it

that

AI software

scientists

have stepped into

seven-league boots too large for them. Their hardware colleagues have outfitted

them with machinery they

can't quite handle. Will

AI software

developers, then, remain hopelessly behind? Probably not: throughout the history of

computer

science,

hardware and software development

have kept leapfrogging each other. Software developers, periodically

overwhelmed by hardware suddenly grown ten times push

it

to

limits

its

and

start

as powerful,

soon

clamoring for more speed and memory.

Because of the subject matter's complexity,

it

hadn't happened before

AI software won't take the lead again, and it may already have happened in areas of AI other than symbolic reasoning. For example, in the Autonomous Land Vehicle project, which fell short of its objectives (see chapter 8), more powerful vision hardware might have made all the difference. Finally, although most of the AI programs described in this book ran on computers with about as much processing power as a snail's brain, these programs appeared much brighter than any snail. If AI software researchers could cajole that much performance out of such puny hardin AI.

Yet there

is

no reason

to believe that

ware, what will they not achieve with machines a million times as

powerful?

And, when

How will we

this

day

fare in a

not superior, to most chapter.

arrives,

what may

lie

in store for

world containing machines

human

beings

is

humankind?

intellectually equal, if

the subject of

my next,

and

final

12 THE SILICON CHALLENGERS

OUR FUTURE

IN

indeed, early in the next century, machines just as clever as human If, beings appear, the question arises of how we will interact with them,

and

how

new machines

they will affect our society. Perhaps the

will

simply relieve us of tedious chores, expand our intelligence, and bring

about universal peace and prosperity. But

of human experience embodied into ics strike a fatal

a massive

and then

blow

a

will

not the sight of a lifetime

few thousand

to our self-esteem? Will these

unemployment problem in business, science,

dollars

as they replace us first in factories,

and the professions? Even

ways to redistribute the wealth generated by automated businesses,

what

will

be

left

for

of electron-

machines not create

humankind

to

we do

find

factories

and

if

do? Having taken control

how do we know that machines will act in our best interests? If such comes about, how do we know that later of our

lives

through the economy,

generations of today's smart weaponry will not take forceful control of

our world? In order to bring out these scenarios which run the

we

shall see,

our future with our

what we make

it

to be.

issues,

gamut from paradise on silicon

I

have drawn up three

earth to apocalypse.

progeny

will

become

As

largely

THE SILICON CHALLENGERS

313

(MR FUTURE

IN

THE COLOSSUS SCENARIO Let's take the I

bad news

have borrowed

Forbin

first,

and consider the worst possible outcome.

this scenario's

name from

based on a novel by D.

Project,

United States entombs an

intelligent

pregnable vault, and gives

it

strategist.

As soon

1

969 movie 1

It tells

Colossus:

how

computer (Colossus) into an im-

as

it

enemy

attack faster than any

takes charge, Colossus discovers the

existence of its previously unsuspected Soviet equivalent, with silicon

commodore

has

two machines soon digital

superpower

more

affinity

than with

electronically merge,

dictates

The

a future

control of the nation's nuclear missiles in

the belief that the computer will react to

human

the

F. Jones.

its

its

human

which the

creators.

The

and the resulting composite

humanity.

will to

Farfetched, you think? Despite the end of the cold war and the

obvious foolishness of ever letting control of the nuclear button

from human hands, two of the experts

I

from

are tottering dangerously close to such a chasm. Their fear stems

the fact that

AI research

foremost a military the 1950s:

is,

affair. It all

at least in the

United

States, first

started with the launching

American backwardness

slip

have interviewed fear that we

and

of Sputnik in

in launching rockets generated a

crying need for miniaturization, and turned

NASA and the military into

first integrated circuits. The chips soon found bombs* and missile heads. As a result, the military, through its civilian funding arm of the Defense Advanced Research Projects Agency, became the most ardent supporter of innovation in electronics and computer science. The United States owes its position

avid consumers of the their

way

as a

world leader

into smart

in

computer technology,

progress in this field since the 1950s, to

observers have even

commented

as well as the breathtaking

DARPA's

support.

that the computerization

Some

of society

is

2

but a side effect of the computerization of warfare. AI departments and laboratories in

existence to

American

DARPA's

universities

owe

stressing open, unclassified basic research,

begun

known

to seek a return

their birth

and continued

funding. Although the agency has a history of

on

as the Strategic

its

it

has in the past several years

investment. Starting in 1983, a program

Computing

Initiative

has focused

*"Smart bombs" are equipped with controls that allow the bombs to toward their target under the guidance of a computer.

much AI

steer themselves

314

Al

research

on

three clearly identified military achievements.

Automated Land Vehicle,

undertakings, the

pilot

known

edge base called the

what better

projects

as the Pilot's Associate,

as

write

I

One of these

by the wayside

— an R2D2-like and ship-borne Batde Management System — seem

The other two

(see chapter 8).

fell

a

1989

in

electronic co-

strategic

knowl-

to fare

some-

this.

may on occasion set its research goals too high; but, as Gulf War showed, it can hardly be faulted on field results. The allied forces in the gulf owed their overwhelming superiority The

military

the 1991 Persian

largely to sophisticated this success.

computer technology. AI played no small part

"Some of

the things

in Saudi Arabia," Patrick

we

in

did did have a significant impact

Winston of MIT

told

me, "but these were not

we thought they would be." 3 In addition to cruise and smart bombs, much of the success of Desert Storm

necessarily the things missiles

stemmed from

prosaic

AI

applications, as

Hans Moravec of Carnegie

Mellon explained to me:

Computer mail* grew out of AI: there was a lot of that in use in the Gulf War. Everybody had workstations, even field troops. The American command was coordinated through E-mail: it was a very substantial contribution,

logistics also

How do

owed

but not a spectacular one. The planning and

a lot to

you pack

AI

techniques.

a transport plane?

I

mean

How do

simple things

like:

you physically arrange

programming problem, which at one AI problem. Also, scheduling is actually an expert-systems problem. You can do simple scheduling using numerithe supplies? That's a dynamic

point was considered an

cal algorithms, like the

system to solve

It is

when you

but

face a complicated scheduling

problem

timing and coordination of Desert Storm, you need an expert

extremely

it.

4

difficult in

AI

research,

Moravec

later

guess which of one's insights will turn into a weapon.

complained, to

Even

the

outwardly anodyne ideas sometimes find their way into the war

He remembered how

he spent part of

his

youth

most effort.

as a Stanford graduate

student looking for ways of making a robot cart cross a cluttered

room

without colliding with the furniture. Moravec was surprised to find that *Computer mail

computer users to send messages to each other over and display the messages on their terminals.

(or E-mail) allows

telephone or radio

links,

THE SILICON CHALLENGERS

some of his

IN

315

FITI RE

R

1

went to work for a Lockheed They adapted his methods to let a cruise target. Since Lockheed eventually lost the cruise

fellow graduate students later

research center in Palo Alto. missile find

way to

its

its

Moravec's technique wasn't used in the

missile contract to another firm,

Gulf War, but he expects the idea

to resurface in a later generation

of

cruise missiles.

This example

of many

in the

increasing

its

wherein

illustrates

AI

the danger: in spite of the desires

lies

modern weaponry

research community,

constantly

is

speed and savvy. This evolution, in turn, imposes new,

on

relentless constraints

field

combatants, which make them dependent

on information and advice provided by machines. The frenzy of modern

human

battlefield activity often leaves the

link in the military control

loop no choice but that of blind obedience to

its

electronic counselors.

home to me when when machines become truly

Daniel Dennett of Tufts University- drove the point I

asked him whether he thought

intelligent in the

that,

next century, they might seize control of our weapons

systems:

I

think you're looking too far

sooner.

Long before we

tions into

the

War Three by most

is

[1983],

of

won't turn his

as they

go through

to eliminate the

supposed

in

deep trouble. Consider

a child hacker almost causes

displays.

I

But

The

a

man

credits.

it's

a serious

chilling bit

postgame

drill.

You're

And

there

is

this

one

conditions are such that he's sup-

key and he won't do

The

World

think the

are putting their keys in the missile

this is the real thing.

his key.

pressure, and collapses.

come

Soviet attack, of which the personnel

who

launching locks believe that

posed to turn

where

mock

a

that the officers

who

will

of that movie happens during the opening

drill

informed by computer

guy

The dangers

breaking into the missile defense system.

chilling part

see a fire

shown

the line.

weapons systems, we're already

movie Wargames

You

down

build really serious and complicated inten-

is

analysis

it.

He's under tremendous

the reaction of the superiors

of the

fire drill.

They decide

because he couldn't perform the job he was

to.

Well look. If we only give those keys to people

whatever the machine says and do

it,

let's

who will

simply take

Throw human judgment

not kid ourselves.

away the keys and

just

playing

if it can't

stand up against computer judgment? Let's just

admit

and not delude ourselves about

it

put a wire

in. \XTiat

still

role

is

having

human

beings in

316

Al

the loop.

We

are already at a point in the standoff

judgment and human judgment where even pathological chutzpah to computer."

And

this

it

between machine

sometimes takes heroic or

say, "Well,

know

I

better than the

long before we've got intentions

is

really built

into computers. 5

MIT's Joseph Weizenbaum was of

a similar opinion,

when asked

about the possibility of computers taking control of our armaments:

To

a certain extent

we have

believe the eagerness with

the

crossed that threshold. For one thing,

which the American

Gulf War was intimately

jumped

related to their fascination with

erized weapons. In that sense, the their

military

weapons control

owners. As another case in point, consider the shooting of the

American

coming from Iran and shot

it

known

War

down.

It

turned out to be an Airbus

Had

the ship's captain

it

to

be

One of the

fired at.

reasons for this accident

the captain didn't have time to gather evidence and evaluate

forced to

make

a decision,

est link in the chain

is

always the

is

just that

is

that

He was

human

that in these cases the

I

don't think that holds

was the weakest

that the captain

weak-

being: consequently, the role

of humans ought to be even further reduced.

The reason

it.

and made the wrong one. What the

technocrats learned from such events

If

An

the plane to be an airliner, he would, of course, never have

ordered

water.

itself.

be under attack by an airplane

cruiser thought itself to

with two hundred and thirty people on board.

fully

comput-

the behaviors of

Airbus in the gulf about a year before the Gulf

is

I

into

link in the chain

he had to make the ultimate decision. Had the system been

automatic,

it

probably would have made the same decision. 6

humans can no longer

alter the

outcome of such

crisis situations,

we have already lost to computers much control over our armaments. The problem for now is that machines cannot be counted on to react any better than a human being under stress, and would certainly then

perform

far

options.

The

worse than

a

chilling angle

person with enough time to consider the is

that

it is

often the frantic rhythms forced

upon us by the machines that bring about if

we

are not careful, this situation

scenario.

As machines become

crises in the first place.

Yet

may develop into a much grimmer we may be tempted to give them

smarter,

THE SILICON CHALLENGERS

more

IN

B

III

control in situations where time

extend

their

317

FUTURE is

of the essence.

We

might even

dominion over circumstances more complex or without the

time limitation of the relatively simple knee-jerk situations considered so far.

Assuming

that such smarter

machines might indeed perform better

than time-pressed, or even comparatively inept, humans,

be faced with two dangers. The

syndrome,"

A

An

Space Odyssey.

bound

space ship

for Jupiter,

human crew when

one we could

computer

intelligent

principle,

chapter

I

drew

earlier

when

Feedback,

They

later

kills

of

a

the ship's

To

see

why, con-

between Norbert Wiener's feedback analysis

method

(see

introduced into gun-control mech-

first

anisms, produced wild mechanical oscillations that neers.

still

"Hal

faced with conflicting mission goals. Such behavior

and Newell and Simon's means-ends

3).

in control

Hal turns psychotic and

should be expected from early intelligent programs. sider the parallel

we would

label the

paranoid computer in Stanley Kubrick's 1968

after the

movie 2001:

first

initially

baffled engi-

explained this behavior as a side effect of the amplifi-

cation of the error signal between the actual and the desired aims of the

guns. Means-ends analysis, just like feedback, uses the difference be-

tween perceived and desired goal: this kind

of reasoning

of an AI program

as a

feedback loops. Just

is

states

of

AI

systems.

complex network of such interlocked symbolic

like

This

more

tion.

vulnerable.

difficulty is

artificial

not peculiar to gun control or AI programs. In large

systems, stability problems are the

Designing a structure that

mechanism problem

is

that performs that

more

its

will

norm

basic function

where they

rather than the excep-

withstand is

huge systems tend to amplify

vibrations to a point are

AI programs are more complex they

gun-control mechanisms,

subject to unforeseen and wild behavior; and the are, the

how to reach a One could think

decide

affairs to

frequent in

its

own

weight or a

comparatively easy.

The

small, naturally occurring

tear the structure apart. Large bridges

susceptible to the infantry lockstep

problem* than small ones;

supersonic aircraft have an unsettling tendency to nose dive and vibrate.

The generators in large electric power utilities have a positively unnerving way of going into spontaneous oscillations that can tear the interconnection apart, at times blacking out entire countries. The first machines to approach human intelligence will be incomparably more complex

*When many

soldiers cross a bridge in lockstep, they can set

tear the bridge apart. This

is

why

armies break step

when

up resonances

would on foot.

that

they cross a bridge

318

Al

than any

or electric

aircraft, bridge,

problems correspondingly more If

AI systems hold

instability, a typical

will

run more or

program steps

suddenly generate wildly inappropriate will reveal

faulty internal connections,

And

results.

Checking

nothing; the engineers will discover

and the programmers, no misplaced

All individual events within the

within specifications.

of

new program

in a

After a series of normal program runs, the

less like this:

individual

amount

bring about stability

will

true to the venerable engineering tradition

will

no

and

encounter with the phenomenon

computer

commas.

utility,

difficult to handle.

machine

will

have remained

yet the results of these faultless steps will

and unbalanced behavior: madness.

to irrational

Like bridge builders and electric power system engineers before them, the

AI

researchers involved will probably be taken by surprise.

have devoted

of thought to the

a great deal

hand, and probably weeded the obvious design.

And

instability

failure

yet this particular behavior will baffle

They will

problem before-

modes out of their them at first. After

days or weeks of head scratching, a bright young specialist will discover

how

finally

the combination of a specific set of operating parameters

mode, and a new chapter will be added to book on intelligent-system instability. With

resulted in this unique failure

the long (and unending) luck, such

problems

ever responsibilities

We

can expect

intelligent

will its

be spotted before the program assumes what-

designers planned for

that, in their first

systems will be

it.

It

may not

decades of functioning,

much more

human

beings

are.

evolution has had a million years to stabilize our design.

mas or

machines

will

all,

we

throw us out of kilter; emotional

trau-

tend to go mad, just as people do and perhaps

even for similar reasons. possibility

Yet

yet

deprivations lead us to depression or suicide. For a long time,

intelligent

ties to

After

And

balanced systems: barely measurable

critically

deficiencies in neurotransmitters

artificially

susceptible to the analogues of

paranoid or psychotic tendencies than

remain highly tuned and

always be so.

of madness and

We

will thus

irrationality

have to take into account the

before handing over responsibili-

future intelligent machines. if

one

is

willing to consider

unsettling possibility

machine

will

comes

probably develop

to deal with the

world

its

efficiently,

knowledge by drawing

its

all

to mind.

own

the implications, an even

As

I

more

said earlier, an intelligent

analogues of human feelings. In order it

will also

have the

ability to learn

new

conclusions from events. (To see why,

consider two household robots: one doesn't recognize your friends as

THE SILICON CHALLENGERS they

come

to visit,

and

is

319

OUR FUTURE

IN

unable to inform them of your whereabouts

The other learns to recognize your friends' faces, is able to distinguish them from unwanted solicitors, and learns their names if they come regularly; if the robot sees you step out to walk the if

you

dog,

are absent.

can infer that you

it

friend.

Which robot

is

be back

will

more

functions properly, a robot

its

in a

few minutes and

will, as I

have noted,

Such its

from acquitting

it

a

itself

or

its

perform

when

obstacles

mission.

machine would be constantly absorbing new knowledge from

environment and,

in effect, forever

modifying itself. Thus, a robot (or

missile-control program) certified sane

might well go crazy of

your

feel its equivalent

of satisfaction with well-done work, and frustration prevent

tell

useful?) Furthermore, in order to

real life.

Or,

like

it

into

at the factory

and mysteries

after confronting the contradictions

Colossus,

programmed

original goals

and well meaning

might come to the conclusion that the it

thy nation") are inappropriate to

("keep the house clean" or "protect its

own growth and well-being. may not turn out to be

Indeed, programming in good intentions simple as

it

as

seems. In this respect, the parallel between minds and

bureaucracies,

which both Herbert Simon and Marvin Minsky have

The

exploited in their theories, provides sobering food for thought.

spontaneous organization modes observed in bureaucracies, contend

both Simon 7 and Minsky, 8 provide insightful models of how minds, either natural or

artificial,

organize themselves. In democratic nations, govern-

—

ment bureaucracies are set up with the common good in mind tantamount, in computer parlance, to "programming in" good intentions. Yet

it is

a

common

under very

place that a bureaucracy will inevitably, unless held

tight leash,

grow out of

all

proportions to further

power. In former communist countries, bureaucracies, in life

fact,

its

own

sucked

out of the very nations they were supposed to pamper. The reason

all

is

probably that in order to stay alive and proficient in a competitive world,

any active entity

—

animal, bureaucracy, or machine

minimum of self-assertion and

danger of running amuck. Thus, a machine sight

— must

aggressiveness. This places that,

it

possess a

in perpetual

whether through over-

of its builders or sheer force of merit, acquires some control over

human affairs can be expected to strive for more influence. From these arguments, the Colossus scenario of gross military takeover by computers does not appear so implausible after all. To summathere is a real danger inherent in putting AI machinery in control of armaments, especially of nuclear weapons. Yet the unrelenting logic rize,

320 of the

AI

battlefield

is

pressing our military technicians ever further into

Today's

this direction.

artificial intelligences are

dumb

simply too

to

avoid the mistakes that fast-paced modern combat situations render probable. With the intelligence that will eventually

conditions better than

human

beings, will

let

come

them handle these

other uncertainties

about the machines' behavior and motivation. At present, not

know enough about

weapons

we

simply do

such matters to entrust the power of modern

to intelligent computers.

The time machines

is

them. For

to prevent the gradual

now, when we

this reason,

it

still

handing over of military power to

have a large measure of control over

may be more

urgent to work into disarmament

agreements anti-AI clauses than antinuclear ones. The present world-

wide reduction to

do

in

superpower

military tensions offers an ideal occasion

just that.

THE BIG BROTHER SCENARIO Less sensational, but equally grim takeovers might result from AI power

running amuck. Computers, either alone or in cooperation with a technical ruling class,

may come

to exert a

more

insidious

human

and dehuman-

izing control than gross military takeover.

Such

a situation

invasion which

would

is

ultimately result

from the

potential for privacy

modern information technology, and

inherent in

which AI could amplify without bounds. Daniel Dennett had the

fol-

lowing thoughts to offer about the matter:

It is trivially

by recording

easy

now with

their telephone or private conversations.

ever, a bottleneck: listen to

high tech to eavesdrop on people, simply

you need

There

is,

how-

trained, qualified, secure personnel to

those hundreds of hours of tape that you'll gather.

be horribly mind-numbing: thank

God

It

must

for that bottleneck! I'm sure

that in the CIA and other organizations the problem is finding people who will do the work. As Joe Weizenbaum pointed out years ago, AI

speech-recognition systems [similar to those investigated by

during the

SUR

program; see chapter

Long before you can make

5]

would provide

a

DARPA way

out.

a speech-recognition system that could

THE SILICON CHALLENGERS replace a stenographer,

321

OUR FUTURE

IN

you could make

system which could act as

a

good filter. It could be tuned to listen for a few hundred key words, which would increase the effective surveillance power of any single a

human monitor by out the tedious

orders of magnitude.

bits,

dred hours of tape

By

system

letting the

filter

an Al-assisted listener could process four hun-

in, say,

two hours. There

some evidence

is

that

it

has actually happened. In England, certainly, and probably in the National Security Agency, and the CIA.

There are other equally dangerous aspects of AI. Consider

its

possible application to electronic- funds transfer [which allows you to

pay for your purchases by credit or debit

What

cards].

if

EFT

pro-

ceeds to such a point that paying in cash becomes anomalous? Sup-

pose that it becomes a presumption that

must have something to will leave

form, as

hide. If the

our fingerprints

we do

if you are

paying in cash, you

anonymity of cash disappears, we

over the world in machine-readable

all

business electronically.

It's

hard to imagine a better

system of surveillance than the elimination of cash. to start a political

movement

for the use of cash

It

might be time

whenever

possible,

simply to preserve this political anonymity.

Other opportunities for electronic snooping include the two

billion

messages that Americans annually send to each other by electronic mail. Largely unprotected over the computer networks, they are a secret

dream.

police's

A

young German demonstrated

their vulnerability in

1988 by perusing the correspondence of United States military worldwide. 9 Yet another danger to our privacy

is

that

officers

of data-base

mining and matching by government agencies and private companies alike.

These organizations

are increasingly

drawing together from multi-

ple data sources information about people; credit ratings, voters'

tomers of various

how many

noticed

a purchase?)

lists,

magazine subscription

stores, together

bills,

lists,

and

lists

of cus-

with their purchases. (Have you

stores ask for your

Telephone

and these sources include

address after you

name and

make

bank, medical and criminal records, and

even Internal Revenue Service

files

have also been known to be so

pilfered.

The state

personal records reassembled from these various sources can

what your

whom

style

of

living

is,

what you

eat,

you associate with. By way of example,

where you

in

1988 the

travel, files

and

of the

322

Al

credit information

company TRW,

contained information on more

Inc.,

than 138 million people, including their income, marital status, sex, age,

telephone number, number of children, and type of residence. 10 Moreover, companies such as

TRW

These are based on

services.

often offer what they

statistical

"predictive"

call

techniques allowing the compa-

nies to predict a person's likely behavior (such as defaulting

on

a loan

or purchasing certain goods) from the characteristics in their data-base entries.

Such procedures force the persons so assessed into

defined categories which take is

done without

usually

door

little

a person's

to political repression

and

knowledge or consent,

it

this

opens the

social abuse.

Despite attempts by concerned legislators to erect against this sort of activity, the U.S. federal

without their consent. As early as the 1960s,

it

barriers

legal

government has

history of using data bases to gather information about

political dissidents.

arbitrarily

account of true individuality. As

was using IRS

The Justice Department then

its

a long citizens

files

against

created a special

com-

puter network to keep track of presumed agitators during the ghetto revolts that erupted across the continent

Angeles.

The department

later

added

protestors to this target group.

launched Operation fist

CHAOS

At

from Washington to Los

New Left activists

and

antimilitary

the peak of the cold war, the

CIA

between American

paci-

to unearth links

movements and communist powers. The project at one point han11 files on more that three hundred thousand individuals.

dled computer

For twenty

years, the National Security

Agency, with twice the budget

of the CIA, attempted to penetrate and control the world communication network. It eventually gained access, with the help

Union, and ITT, to

The Watergate

all

of RCA, Western

telegrams received or sent from the United States.

investigations eventually interrupted these operations in

no means brought an end to the government's electronic snooping. According to the American Civil Liberties Union, the number of data matches performed by the government tripled the early 1970s, but by

between 1981 and 1984. During

this period,

eleven cabinet-level depart-

ments and four independent agencies carried out 110 computer matching operations, comparing

The AI

more than two

researcher Roger Schank raised the issue of

in his The Cognitive Computer.

comes

He

facetiously

12

AI and privacy

remarked that

if

worse

means exist for the IRS to find out about a math grades in school and audit people who were poor at Schank then went on to explore a more unnerving possibility:

to worse, the

taxpayer's addition.

billion records.

THE SILICON CHALLENGERS It

used to be

INOIR out

difficult to find

323

FIT IRE

who

reads what. But today books are

who

ordered from general warehouses of booksellers records on computer.

Das

it

tend to keep their

be to determine which bookstores

most copies of Slaughterhouse Five last year? Change Slaughterhouse Five book associated with a clear radical movement

sold the for

How hard would

Kapital or any other

this kind doesn't seem so farsome bureaucracy decided to audit the property taxes paid bookstore in an effort to close it down? How hard would it be to get

and suddenly the idea of surveillance of

And

fetched.

by the a

if

copy of every

single

check or credit card transaction used by the customers

of that bookstore? Not hard Nazis began by simply

at

telling

all: all

information

this

Jews to

register.

is

on computers. The it became

After registration,

more and more difficult to escape the chain of events that the identification made possible. Access to information is a powerful thing. 13

process

Nowadays we like

or not.

it

are

all

It is

not clear that legislation

government bureaucracy from using AI

more control over our would have dreamed of. efficiency,

programs that

If the

human their

could use

collate data-base

nastier possibilities emerge.

Unimpeded by

resources. If exposed, a

to another

at will.

computer

in the

con-

They

program could simply send network, and

start

Using software viruses akin to those that can wreak havoc insert into

it

a diskette

of uncertain

disseminate enough copies of

How could

legal

may come

information (such as through blackmailing) to increase

power and

you

of

information for their

computer networks) could gather information

this

copy of itself

if

prevent

than any previous dictatorship

such a program (or programs: a multitude of them

to haunt the

their

now

lives

will suffice to

to acquire, in the interests

masters one day accede to intelligence and acquire desires of

own, even

straints,

whether we

registered with countless data banks,

such free-lance

origin,

a

over again. in

your

PC

such a program could

become virtually ineradicable. programs come into existence? One positself as to

mechanism would stem from the need for secrecy in the development of AI programs: Security considerations make secrecy necessary in sible

military applications,

and competitive pressures require

it

in

commercial

environments. At present, the single largest impediment to keeping the lid perfectly

closed

on how an AI program works

oughly document the code. This

opment of afterward.

a

Programs

Soar, could

is

mandatory

program by human beings and that learn

one day acquire

to.

is

the need to thor-

permit orderly devel-

for maintenance purposes

by themselves, such

target characteristics

as Allen

NewelTs

under loose human

324

Al

supervision, without a person's ever having to write a single line of code.

Such programs would most conveniently sidestep human secrecy, since they

they

come

would be self-developing and

into existence,

be against the interests of the military

will

it

or corporate authorities concerned to

let

human

innards too closely. This lack of scrutiny could in a It

let

beings monitor their deviant harmful

traits

program's "personality" go undetected.

upon their human AI programs and robots could deprive them of their livelihood.

has also been feared that, in addition to spying

victims, It

threats to

self-maintaining. If

might

start in the factories

fifty-y ear-old

where, as has already happened

Japan,

in

machinists would sweep around the robots that replaced

them. As robots become more

most of the twenty-five

versatile,

Americans working in production plants may

fall

million

victim to a similar

Their computerized replacements would work twenty-four-hour

never strike or

much

call in sick,

and

fate.

shifts,

amortization and running costs

entail

lower than a human's wages.

The management and

service sectors

of the economy

will

be

in

no

position to absorb displaced plant employees, because automation will

wreak havoc have already

in these activities also. drastically

Conventional computer applications

changed the nature of

clerical

and

secretarial

work. Nowadays most technical and office workers enter their reports or correspondence on

word

processors, gready reducing the need for

Future speech-understanding systems should eliminate typists

typists.

and accounting

altogether. Sales

recording a sale

is

now

clerks fare

no

better: in

most

stores,

reduced to scanning a bar code affixed to the

goods, thus generating an invoice or cash receipt which proceeds auto-

Gone

matically to an accounting computer. entries

goods or money soon be

reality.

With accounting programs a

company's

an ever smaller fraction of the work

What systems

all

transactions involving

are recorded electronically: the paperless office will

automated records, generating

it

to process the resulting

financial statements requires

used

to.

run-of-the-mill programs are doing to clerical work, expert will

soon do to middle management functions. Middle manag-

ers are typically those executives

who

gather and interpret information

make recommendations them after their bosses' More and more, computerized management information sys-

for their superiors.

on

are the laborious ledger

of yesteryears! In many organizations,

Middle managers

will also

the basis of such information and implement

approval.

tems can take care of the data-gathering part of a middle manager's

job.

I

THE SILICON CHALLENGERS

For example,

GM's

in

ment model of car is

IN

fully

OIR

325

TURE

F

automated Saturn

division,

upper manage-

can instandy discover through their computers what color or

to six

weeks

selling

responsibilities

take

them

— something

to find out.

over.

of

a

14

As

it

used to take a market analyst three

for the interpretation

and recommendation

middle manager, expert systems are beginning to

For example, an expert system could

correlate sales with

advertising campaigns and formulate adjustments to the ad schedules or

even revisions to

their contents. Likewise, a scheduling expert

could reorganize production runs to better follow

system could

also, to a large extent,

the Saturn type of manufacturing, automatically.

The

implement parts

all

its

sales.

system

In this case, the

recommendations. In

and supplies are ordered

expert system simply sends orders for the proper

parts to the plant's suppliers, together with a delivery schedule. It

the supplier's responsibility (or that of their

tem) to arrange for the parts to

show up

own

is

then

scheduling expert sys-

at the plant at the right

day and

hour. After unloading, the plant's automated machine tools and robots

process these parts as specified by the plant's scheduling expert system.

Other encroachments into middle management can be found banks

as

more and more simple

clerks, using expert

in

systems and data

bases holding borrowers' financial statements, perform loan analyses that formerly required the services

of experienced loan

In other

officers.

pinch of

industries, specialized technical personnel are also feeling the

AI. Just as

GE's

DELTA

system allows a techni-

(see chapter 8) expert

cian to carry out repairs to electric locomotives a highly skilled engineer, ter 6), the first expert

Edward Feigenbaum's

which used to require

DENDRAL (see chap-

system ever produced, allows any graduating

chemist to perform certain highly specialized analyses that were for-

merly the province of Ph.D.

Up

to

technical

scientists.

now, though, AI has

which account for two

There are several reasons for at

and service

particular, the professional

thirds

ern countries, have been largely

AI

of

and managerial work; and most people have never been ex-

posed to an expert system. In sectors,

affected only highly specialized areas

of the economic

immune

this state

to the

activity

of West-

encroachment of AI.

of affairs, but they may not keep

bay for long.

First,

the fact that

AI

programs

still

cannot converse

languages tends to restrict their applicability to areas where

in

human

they interact

with the environment through highly stylized symbols (such as bar

codes or inputs from measuring instruments), or

in

domains so narrow

326 as to

A

Al

be circumscribed by answers to

second and related problem

is

relatively

few standard questions.

common-sense

the

bottleneck, which

can lead a medical expert system to such ludicrous behavior as to

As we saw,

prescribe antismallpox drugs to a car showing rust spots.

continued research should gradually erode these restrictions during the next thirty years. In truth, though, these difficulties should not prevent the immediate

many

application of expert systems to ple, since a it

to

human and

should have no reason to confuse

reasons

professional activities: for exam-

medical expert system would operate in a

we

don't see

more expert systems

do with technology. One of these

that expert systems entail.

Who

is

in

not a garage,

clinic,

car bodies.

everyday

life

The

true

have nothing

the blurring of responsibilities

is

blame for

to

a professional fault

involving an expert system? In a medical context, the physician

who uses

the system, and the hospital that employs him, probably bear primary responsibility for a patient. parties

With an expert system,

would turn around and sue

original expert

company

who embodied

into

its it

that distributed the program.

it is

likely that these

programmers, along with the

his or her

What

knowledge, and the

with lawyers' tendency to

sue everyone in sight, such a lawsuit might also target the operators of the

computer system or network on which the expert system

nally

ran.

who

cording to some authorities, even the theoretical scientists

Ac-

origi-

developed the computer science principles implemented in the

expert system might be held liable under

some circumstances. 15

Medical authorities (or the ruling body of whichever concerned profession)

might eventually appoint committees to

of the rules in expert systems bearing not

clear,

however,

between the rules

rules,

were sound

human

how

upon

certify the

is

such evaluations could account for interactions

which might lead to

in themselves.

faulty

behavior even

The problem would worsen

experts do, the program were allowed to learn (that

itself) as a result

soundness

their respective trades. It

of its experiences. Not

surprisingly,

all

the

if all if,

is,

as

all

modify

professional and

technical people involved in the potential implementation of expert

systems in professional

activities

shy away from such a

legal

Damocles'

sword. In the early 1990s, medical expert systems remain confined to experimental or educational applications: virtually none are put into clinical use.

Doctors also object to expert systems on other grounds. entry: typing in

One

is

data

symptoms during an examination would be time con-

THE SILICON CHALLENGERS

327

OIR FIT IRE

IN

suming and unseemly. Further, since expert systems knowledge of human experts

in the first place, they

competence of individual physicians thus of

little

the

of expertise and are

use to them, except perhaps in a watchdog role to prevent

errors of fatigue or distraction. scrutiny. Further,

tence of

in their areas

embody

just

cannot exceed the

some

even

if

Many

physicians

would resent such

expert systems were to increase the compe-

would merely

physicians in certain areas, such programs

automate away the challenging and

intellectual parts

of a physician's

job.

Issues of lesser scientific import, like easing anxiety, or the tedious

routine of physical examinations ("smelling" the patient, as a surgeon friend put

to me),

it

Although these

would remain the

difficulties

physician's responsibility.

might seem to preclude the penetration of

expert systems into hospitals or doctor's offices for the foreseeable future, there

upon

is

a

way

in

which expert systems may force themselves

the medical (or other) professions in an undesirable manner. This

has to do with the exploding costs of medical malpractice to curb

them would be

in a flexible

and

easily accessible

medium, and

patient whether such standards

watchdog

to provide

One way

means

would conclusively show

physician to maintain records that

acting in a

suits.

to define the standards of good medical practice for a

for each

had been followed. Expert systems,

capacity, could probably accomplish this function.

This solution might lead to situations where physicians would make themselves more vulnerable to law

by not using an expert system

suits

than by using one. After being forced to rely on such an external standard of good practice,

some

physicians might abdicate their responsibility and blindly

follow the expert system's advice. already worried that trial

AI might

Some

observers of the AI scene are

our

intellectual elites as the indus-

affect

revolution affected craftsmen. 16

The

last century's

proud

class

of

blacksmiths, cabinetmakers, tinsmiths, and assorted glassblowers gradu-

know-how

became operators of ever mass-producing machinery. Some of them moved

ally lost their specialized

as they

more sophisticated on to become our modern designers and smaller number kept on working as before,

now who

being considered a luxury.

the product of their

large,

"de-skilling,"

The

skills

much skills

though, those craftsmen,

used to embody humanity's technical knowledge

have seen their

ties.

By and

engineers, while a

in earlier periods,

taken over by machinery. This process, called

may now

effect certainly

threaten to blunt is

many of our

not limited to those elites

intellectual abili-

whose

expertise can

328

AI

be embodied into expert systems.

we

place fifteen years ago, didn't

numbers

in

Now

our heads?

ing, isn't there

much

less

When

all

became common-

calculators

stop bothering about manipulating

that your typewriter can check your spell-

pressure to worry about the intricacies of

orthography? Such simple applications of computers have already taken

away the incentive surely carry

on

to learn

some

kinds of knowledge: expert systems will

the process at a

Embodying human

skills

into

much

higher intellectual

level.

machines may bring about other ad-

verse effects. Seeing knowledge acquired by one person over several years of hard

work

bottled

up into

a

few thousand

dollars'

worth of

hardware does not bolster one's respect for the value of human effort. Not only can the availability of machine standins lead to a decline in the number of human experts, but such a discouraging equivalence might reduce the morale of our entire species. The British philosopher and psychologist Margaret Boden compared the potential effect of this mechanistic analogy to the traumatic implications of cremation feared

by the Catholic Church

It

was

difficult

in past centuries:

enough for the

faithful to accept the

notion of bodily

body would eventually decay into the ground). But the image of the whole body being consumed by flames and changing within a few minutes to a heap of ashes was an even more powerful apparent contradiction of the theological claim of resurrection after having seen a burial (knowing that the

body resurrection

Day of Judgment. 17

at the

Likewise, an abstract belief in the physical origins of thought processes differs

altogether

machine"

from the gut

in one's skull. Seeing

replicated in a desktop

feeling

of the presence of

a

"meat

our very thought processes routinely

computer can be expected

to

undermine our

image of ourselves and devaluate our sense of responsibility and individuality.

As AI programs

take over

many human

functions, they

may

bring

about a gradual ossification of society into undesirable patterns. Most people occupying what see their

skill

we now

call

"white-collar jobs"

may

gradually

requirements reduced, together with the control they have

over their work. Their pay level

may go down

accordingly, and their jobs

turn into insecure, monotonous, and stressing chores. Meanwhile, a

group of professionals

mune

—

scientists

and managers lucky enough to com-

with the AI programs and participate in high-level decisions

THE SILICON CHALLENGERS

may

see their

standards.

work

Such

329

OUR FUTURE

IN

power and

enriched, together with their

social polarization

living

evokes the image of a police

where law enforcement mainly consists

in keeping

down

a

state

massive and

permanent underclass.

The

elite itself,

Once

however, might soon yield to another kind of petrifi-

brought about by the high cost of developing expert systems.

cation,

in place, there

is

a strong incentive to use such a system as long

as possible, despite progresses in its specialty or

context in which

is

it

applied.

By

changes to the social

thinning out the ranks of

experts and scientists, expert systems might also slow

of new knowledge, further reinforcing

The in

British philosopher Blay R.

the creation

this intellectual stagnation.

Whitby pointed out one puzzling way

which AI could damp human progress even

work

down

human

further: so far the

AI has been performed by people who, growing up without exposure to computers, introspected some of their thinking processes and implemented them into machines. 18 The home-micro generation, by contrast, may have grown up thinking like computers. pioneering

in

After the changing of the guard, can

artificial intelligence

continue to

progress?

THE BLISSFUL SCENARIO: LI FT-OFF? At one

point

when doing

the research for this book,

I

contemplated

these discouraging conclusions and despaired of the suicidal conse-

quences of trying to

instill

interviewing Gerald

Sussman

intelligence into machines.

unrelated question provided

at

my

MIT, and first

remembered of

this

as the

most

I

I

was

at the

time

answer to an apparently

inkling that there might be rosier

prospects for humankind's future after "Centuries from now,"

his

all.

asked Sussman, "what do you think

salient aspect

of AI

will

be

research in the latter half

century?"

After thinking a while, Sussman replied at

some

To

necessary to look back at

understand the answer

I'll

give you,

it's

length:

the intellectual history of humanity. Five thousand years ago, in

330 As

ancient Egypt, people started inventing geometry.

the

myth goes,

waters of the Nile overflowed every year and wiped out the land boundaries.

It

poses and for tians

was necessary telling

to reconstruct

them

for taxation pur-

people where to plant their seeds. So the Egyp-

invented geometry and surveying. Later the Greeks understood

this vision

and gave

it

a linguistic basis. Their

measurements and relationships among ways for people

words

for describing

spatial objects

to explain themselves to each other.

provided

these Greeks and Egyptians did thousands of years ago, you can tell

a ten-year-old child that, if

you build triangle.

you want to make

new

Because of what

a rigid

now

framework,

out of triangles: unlike a square, you cannot deform a

it

And

the child can understand those words, now.

The next breakthrough occurred around two thousand years ago when the Arabs and Hindus developed algebra, which is just a language to talk about generic numbers and relations among them. Because of what they did then, we can now say things like "The following

The

is

of

true

all

numbers."

advance happened about

third

the invention by Descartes, Galileo,

hundred years ago.

five

It

was

Newton, and Leibnitz of contin-

uous variables and functions involving them. Calculus,

in particular,

allowed them to account for motion in mathematics and

made mod-

ern science possible. And, again, because of what those people did five

now

hundred years ago, you can

tell

a child that a car crashed

against a tree at thirty miles an hour, and the child at least has an idea

of what that might

entail.

That idea was not

clear six or seven

hundred

years ago. I

believe that the

same kind of blossoming

We

twentieth century.

is

happening

are witnessing a breakthrough

can express complex ideas. For example, one can

complex algorithms, or procedures. The quite simple;

and

until recently,

in

now

earliest

in the late

how

people

specify very

algorithms were

no one had ever written down any

very complex algorithms because the means for expressing them did

not

exist.

The fun

part of such algorithms

is

that they allow

you

to

solve problems, rather than just specify the properties of the answer. Let's take the idea

square root

ofj

is

of square root, for example. the

number x such

But that statement doesn't

tell

you

that

x

You x

times

can is

say:

"The

equal toj.

how to find x. It is just a matheAn algorithm for finding x is

matical description of a square root.

more complicated

to explain;

and

a

few decades ago, mathematicians

THE SILICON CHALLENGERS

IN

had trouble getting such ideas

much more

algorithms

331

OIR FITIRE

Now we

across.

can

easily specify

complicated than the calculation of a square

root.

You might

think

it's

no big

deal,

but

found out otherwise while

I

teaching electrical engineering. Typical textbooks about electrical engineering contain plenty of formulas and explanations

on how

to

build sets of equations to solve network problems. These explana-

and ambiguous. You don't

tions are long-winded, poorly described,

them on first reading. Further, engineer finds out from a schematic what a

you look

how

usually understand

if

a real

circuit does, you'll

discover that hardly any equation writing

forms

lots

is

done.

of rather subde mental operations, and

nearly impossible to write

down what

it

was

that she

Well, as a professor of electrical engineering,

job to figure out what a professional does, so

I

[Sussman was referring to the

SCHEME

my

professional goes about across I

fifty

it.

years ago: the

believe that this

new

I

years

was doing.

tell

it was my my students.

intelligence

LISP

method

them

I

for finding

precisely

how

a

couldn't have carried these concepts

words

didn't exist.

capability will

have a profound influence on

humanity over a long period of time and

remembered many

was

complex procedures],

tells

it

it

thought

artificial

students a simple qualitative

the properties of an electric circuit:

engineer per-

recendy

language, a dialect of

that he developed specifically for explaining

could explain to

I

could

turned out that by using the language of

It

The

until

at

will

be the thing

that's

from now. 19

Sussman's view of AI as the Great Simplifier of complex ideas seems to carry promise. In fact, to

draw a

Indeed, effect

parallel

AI

is

a

mode of expression, and it is even possible

between the advent of AI and the invention of writing.

some of

the very misgivings expressed nowadays about the

of AI on moral values were voiced thousands of years ago about

writing. In Headrus, Plato

complaining that those

memory and become

quoted the Egyptian god-king Thamus

who

forgetful: they

dwells in writings, objected

as

practice writing will stop exercising their

might

Thamus, when

start believing that

it

wisdom

resides in the mind. Socra-

even made a remark that Hubert Dreyfus might not disallow: "You might suppose that written words understand what they are saying; if tes

you ask them what they mean by anything they simply return the same answer over and over again." 20

332

Al

The analogy between AI and

writing should help dispel our fears

about our eventual replacement by thinking machines, for the written

word

some of

already accomplishes

substitutes for

its

In a sense, a

this function.

do not depend on the presence of legislators. AI might

much

as writing did as

it

gradually took

intelligence amplifier in society.

permanence and allowing

on

By endowing

ideas to be

take

on

and philosophy

in

our culture

new

factual reports with

expanded

possible. It also allowed

new dimension

a

affect

the role of general-purpose

impossible in a single person's memory, writing matics,

book

author; and, to take effect, written laws or regulations

in a

degree of detail

made history, mathecommerce and law to

both space and time. Writing did not simply

gone on before; it increased their power and made new undertakings possible. 21 A similar power of innovation can be expected of AI, imbued as it is with a magic absent from mere inscriptions. In the world of computreplace the verbal activities that had

ers,

the

power to

Incredible as to

computer

evoking a

it

spell

out wishes

may sound

is

tantamount to making them happen.

to the profane, this notion

is

so ingrained in-

scientists that, latter-day spiritualists, they casually talk

program

to induce

its

execution.

The

aptness of the

revealed as one types out "Eliza" in luminous letters

on

word

presto!, Joseph

and

entertain. Spelling out

complex procedures

is

a dark screen,

Weizenbaum's mischievous creation appears

and

of

to chat

in the limpid language

of AI does more than allow us to understand them:

it

lets a

computer

execute these orders or control other machinery embodying these com-

pointed out:

it

Not

AI

ideas, as

Sussman

can also simplify complex machines or complex

activities.

plex processes.

only can

simplify

complex

Indeed, Donald Michie, the British dean of AI research, has called AI a

remedy

to "complexity pollution":

"AI

about making machines

is

more fathomable and more under the control of human beings, not

less.

Conventional technology has indeed been making our environment

more complex and more incomprehensible, and if it continues as it is doing now, the only conceivable outcome is disaster." 22 The increasing complexity of our machines and administration

is

probably the major

cause of the economic stagnation affecting developed countries. Solving the problem

would

tionally followed

require "a complete reversal of the approach tradi-

by technology, from one intended to get the most

economical use of machinery to one aimed

at

making the process of the

system clearly comprehensible to humans. For 23 to think like people."

this,

computers

will

need

THE SILICON CHALLENGERS

IN

For example, data bases, although as well.

333

FUTURE

01 R

a threat to privacy, have better uses

They can be invaluable in aiding a scientist carrying out a literature

search, a

businessman investigating market trends before launching a new

product, and even to anyone planning a vacation or in search of a

on

deal

certain

goods. Anyone with

modem

a

and

a

good personal computer

could tap hundreds of data bases if it weren't for one rub. In order to query a data base,

one must be familiar with its overall contents and the way these

are structured inside the data banks. Further, since

own query languages, one is

with their

different dialect

of computerese every time one switches to

a

new

data

These requirements turn away anyone (and that turns out to be most

base.

of us)

who doesn't have plenty of time and patience for searching through

thick manuals. will

many data bases come

forced to formulate questions in a

AI will soon replace

the thick manuals with programs that

understand questions in a close approximation of English, and find

out what

By

we want. 24

the turn of the

new

century data bases, coupled to expert systems,

much more individualized. Those our very own personal automated

should make possible services that are

of us so inclined may have access to librarian. It

on

may, for example, inquire about what you

a particular day,

combine these

acquired over

hints with

its

feel like

reading

knowledge of your

many such interactions with you, and come

personal

taste,

up with

selections to suit your

mood. Should you be

interested in re-

searching a particular subject (say, fishing), this librarian will review with

you the contents of the various books

available

one most appropriate to your needs. Or, chef, available at tion,

beck and

matching your

call

than the

whip up

tastes to the contents

book? Kristian Hammond's tives in 1986,

to

how

and help you

about a resident

recipes

of your refrigerator or pocket-

but required a hardware platform

power of these machines most households.

will

the day. 25

soon make

silicon

and menus combina-

CHEF program came close to

home computers of

select the

The

these objec-

much more powerful

exponential increase in

CHEF an affordable option for

and insurance agents are obvious extensions of this concept. Likewise, an expert system acting as a souped-up and interac-

Automated

tive

home

travel

medical encyclopedia might make for a healthier population

and even save

you of your

lives in

emergencies.

responsibilities

Home

and options

legal advisers

help you decide whether the expense of consulting a

warranted.

A home

might inform

in a particular situation,

human

and

lawyer

is

financial planner could help you invest a few hun-

334

Al

dred dollars with almost as

from

human

a

most

In

adviser

if

much wisdom and

invest.

cases, though, expert systems will not replace people in their

areas of competence. Instead, they will

users and simplify their tasks. for

savvy as you might get

you had tens of thousands of dollars to

many

years to

As

expand the

qualifications

of their

I've said earlier, expert systems will not,

come, exceed the savvy of the human experts they

model.

Expert systems will also need

and

their

someone

—

for the right information to be entered

recommendations interpreted

—

the help and

common sense of

domain of knowledge. Hence, an

already familiar with their

expert system's main function will be to enhance the knowledge and

performance of non-experts. For example, the dearth of medical ists in

remote areas might be

practitioners to

perform

alleviated

special-

by expert systems allowing general

as specialists.

Furthermore,

we

already have

expert systems allowing nurses to perform cardiorespiratory resuscitation in the

absence of a physician. Unfortunately, as I've mentioned, such

systems have never been implemented in emergency wards: such extensions of the

human

intellect lie

ahead of our present

legal

artificial

means of

defining and certifying competence.

In the long run, however, the pattern of our culture should rearrange itself to

accommodate them. At

the root of the problem

lies

our under-

standing and use of innovations in the context set forth by older practices.

The

earliest

carriages. It

automobiles, for instance, looked very

stations to turn the

like

automobile into a universal means of transportation;

to begin with, early drivers a dearth

much

took years for paved roads, mass production, and service

had to make do with few

of qualified mechanics. Likewise, AI

and shape both

itself

and

its

environment

will

outlets for fuel

have to find

in order to

its

and

place

bloom. Today's

expert systems are akin to the early Phoenician writings that just re-

corded a ship's cargo and the products of a

sale: early scripts

did not

engender history, philosophy, and mathematics overnight.

To what

equivalents of mathematics and philosophy, then, will

give birth to?

Much

as the scribe-accountants

of

AI

early Phoenicia could

not have predicted these disciplines, today's practitioners of AI are hard put to answer

this question.

Donald Michie,

for one,

made

the following

prediction:

We

can foresee a whole industry arising

industrial plant, the

"knowledge

.

.

refinery,"

.

based around

a

novel type of

which would take

in specialist

THE SILICON CHALLENGERS knowledge

in

creative gap-filling is

would be

significant if

precise, tested,

wisdom could be

we

will

pull

it

together, carry out

.

a fraction

.

.

of the world's accumulated

practical

brought together, and turned into accurate usable

sifted, 26

in The Cognitive Computer, explored

power of AI may

time,

even

in this way.

Roger Schank, sive

it,

whenever the need becomes evident, and turn out knowland certified correct. The boon to mankind

edge that

knowledge

335

OUR FUT IRE

form and debug

existing

its

IN

affect the sciences

how

the

new

be able to "develop understanding systems

particular fields." Stating exactly

how

expres-

and professions. For the

first

in various

doctors go about making their

diagnoses and lawyers assess a case, and what kinds of knowledge structure economists use to

make

decisions, will

endow

knowledge with unequaled investigative powers. "AI renaissance in practically every area

The main impact of thinking,

was

it

will

encourage

a

touches," claims Schank. 27

new ways of The same may well of AI in this respect

writing, in addition to creating

to revolutionize social organization.

hold true of AI. There are potential positive effects

as well as the negative ones. First, consider the effect

both the blue- and

these fields of

of automation

in

the white-collar worlds. It turns out that blue-collar

unions are usually supportive of robotics for two reasons: robots often replace workers in tedious, uncomfortable or hazardous jobs; and

most

unions acknowledge the need for productivity improvements which robots

effect.

AI and

robotics just might relieve us of the repetitive and

work brought about by the first industrial revolution. Consider loom operators: two hundred years ago, they were skilled manual workers enjoying definite professional respect. Then came the tedious parts of our

Jacquard loom, which automated the weaving of figured fabrics and

reduced the operator's job to one of feeding in materials and activating the machine: this skill.

Thanks

new

job description required no education and litde

to automation, the

taken a turn for the better.

I

loom

operator's profession has

now

recently visited a carpet manufacturing plant

where the looms mostly take care of themselves. Their overseers

are

engineers with the competence to suggest major design changes to the

equipment

if

needed, and submit written reports to

company's board of

would envy them

this effect to the

directors. Their eighteenth-century counterparts

their professional status.

But isn't one such loom engineer replacing scores of mechanical loom attendants and hundreds of earlier manual

loom operators? Haven't

336

||

these or their descendants

gone on

to swell the ranks

of the unem-

ployed? Since fewer than 10 percent of the population are unemployed in

most

tion

answer to

industrialized countries, the

ous no. Indeed, today's workers

of many of

is

an obvi-

their Victorian predecessors, but they also enjoy

incomparably superior standard of additional

question

this

match the combined produc-

typically

goods and

sen-ices

By and

living.

consumed by

large,

an

producing the

the average person occupies

the displaced workers. Also, in spite of the current tightness of the job

market, there

is

plenty of work to be done. 28 Given

enough resources,

education, health care, and psychological counseling provide an infinite

source of employment for people displaced by AI from other sectors.

Even though

it

may

take years to retrain them, and the

problem may

often not be resolved until the next generation, the key point the long run, the

employment problem

self-correcting.

is

is

that, in

The reason

plant engineers are replacing workers by robots, and managers are substituting

increase

it.

computers for

clerks,

growing amount of goods and there will

not to reduce production but to

is

Thus, automated economy

still

will

computers don't buy

their offspring) will

(or, if retraining is impractical,

be available to educate the young, take care of the

or help developing countries along the road to the

in the past

ing goods

cars,

be plenty of supplies around to meet the needs of humans.

Those people displaced by automation sick,

keep on generating an ever-

services. Since

hundred

years,

good

life.

Twice

we've developed mechanisms for redistribut-

whose production required

the population. During the

a drastically

first industrial

dwindled fraction of

revolution, manufacturing

jobs absorbed workers freed from farm work, which before had occu-

pied 80 percent of the American for only 9 percent

up the United

slack

work

force. Agriculture

accounts is

taking

from an ever-shrinking supply of manufacturing jobs. In the

States, the service sector occupies

we

now

of total employment, and the services sector

two

thirds

of the work

force.

means of spreading around the wealth generated by our emerging automated economy. At worst, we will end up with Surely

will

invent

more time on our hands than we can occupy with happens, then we'll

just

have to learn to place

less

full-time jobs. If this

moral value on work.

Indeed, AI will profoundly influence our values, including our perception

of ourselves. W"e must weigh any

induced by AI against the

intellectual skills available to

natural ability to acquire

fears

about the potential de-skilling

AI programs will make their embodied humans. Further, it is beyond anyone's

fact that

all

the

skills

one would wish

for.

For example an

~

CIULEKII*

TIE SHICfl

>„-i.: i.;:;:.i'T':::

:ti:r

:

:

77t :ti:

ur:

:

::

7_7U-7

7tr i~rr.~

:r tz

r