Spatial Information Theory: Foundations of Geographic Information Science: International Conference, COSIT 2001 Morro Bay, CA, USA, September 19-23, ... (Lecture Notes in Computer Science, 2205) 3540426132, 9783540426134

Citation preview

Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis and J. van Leeuwen

2205

3

Berlin Heidelberg New York Barcelona Hong Kong London Milan Paris Tokyo

Daniel R. Montello (Ed.)

Spatial Information Theory Foundations of Geographic Information Science International Conference, COSIT 2001 Morro Bay, CA, USA, September 19-23, 2001 Proceedings

13

Series Editors Gerhard Goos, Karlsruhe University, Germany Juris Hartmanis, Cornell University, NY, USA Jan van Leeuwen, Utrecht University, The Netherlands Volume Editor Daniel R. Montello University of California, Department of Geography Santa Barbara, CA 93106, USA E-mail: [email protected]

Cataloging-in-Publication Data applied for Die Deutsche Bibliothek - CIP-Einheitsaufnahme Spatial information theory : foundation of geographic information science ; international conference ; proceedings / COSIT 2001, Morro Bay, CA, USA, September 19 - 23, 2001. Daniel R. Montello (ed.). - Berlin ; Heidelberg ; New York ; Barcelona ; Hong Kong ; London ; Milan ; Paris ; Tokyo : Springer, 2001 (Lecture notes in computer science ; Vol. 2205) ISBN 3-540-42613-2

CR Subject Classification (1998): E.1, I.2, F.1, H.2.8, J.2, H.1 ISSN 0302-9743 ISBN 3-540-42613-2 Springer-Verlag Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law. Springer-Verlag Berlin Heidelberg New York a member of BertelsmannSpringer Science+Business Media GmbH http://www.springer.de © Springer-Verlag Berlin Heidelberg 2001 Printed in Germany Typesetting: Camera-ready by author Printed on acid-free paper SPIN 10840745

06/3142

543210

Preface The 5th International Conference on Spatial Information Theory, COSIT 2001, took place at the Inn at Morro Bay, California, USA, September 19-23, 2001. COSIT grew out of a series of workshops/NATO Advanced Study Institutes/NSF Specialist Meetings during the 1990s concerned with theoretical and applied aspects of representing large-scale space, particularly geographic or environmental space (this history is elaborated in the prefaces of previous COSIT proceedings). These are spaces in which (and on which) human action takes place, and which are represented and processed in digital geographic information systems. In these early meetings, the need for well-founded theories of spatial information representation and processing was identified, particularly theories based on cognition and on computation. This concern for theory provided an early foundation for the newly emerging field of geographic information science. COSIT is not backed by any particular scientific society but is organized as an independent enterprise. The conference series was established in 1993 as an interdisciplinary biennial European conference on the representation and processing of large-scale spatial information after a successful international conference on the topic had been organized by Andrew Frank et al. in Pisa in 1992 (frequently referred to as "COSIT 0"). After two successful European COSIT conferences with strong North American participation (COSIT ’93: Island of Elba, Italy; COSIT ’95: Semmering, Austria), COSIT ’97 moved across the pond to the United States, and was held in the Laurel Highlands, Pennsylvania. COSIT ’99 returned to Europe, being held in Stade, Germany. The 2001 site of Morro Bay, on the central coast of California, continued the COSIT tradition of holding the conference at somewhat remote but accessible sites. The participants stay together for the full period of the meeting to promote intensive interactions without distractions. The aim of COSIT is to bring together researchers from different disciplines for an intensive scientific exchange. This aim is facilitated by the presentation and discussion of a restricted number of papers in a single-track meeting format to ensure that all conference participants can get involved in the discussions of the papers. As has been typical, COSIT 2001 had about 100 participants, including

VI

Preface

university professors, university and industry researchers, and students (including doctoral candidates). COSIT is very multidisciplinary, and as it has evolved, has become increasingly interdisciplinary, with researchers increasingly sharing methods and concepts across disciplines. COSIT attracts a great variety of disciplines. The most strongly represented have been different specializations within geography, computer science, and psychology, but contributions have also come from anthropology, architecture, biology, cartography, design, earth science, economics, engineering, history, law, linguistics, mathematics, philosophy, planning, and surveying and geodesy. This pattern occurred again in 2001. The continued vitality of the COSIT program in the future will be signaled by increasing disciplinary diversity and increasing disciplinary integration. The conference program was determined by thorough peerreview of over 70 submitted full manuscripts by an international and interdisciplinary Scientific Committee. The reviews of the Scientific Committee were managed and evaluated by members of a Program Committee; in borderline cases, their judgments were in turn subjected to criteria of relevance, innovation, accessibility, and intellectual diversity by the Program Committee Chair. This interactive and timeconsuming process was intended to equitably identify the highest quality scientific contributions, effectively communicated, that would provide a balanced and spirited intellectual basis for the meeting that took place. Undoubtedly this process led to the rejection of worthy contributions and perhaps the expression of implicit biases of the COSIT community. As Chair of the Program Committee, I take final responsibility for these unfortunate shortcomings. To kick the conference off, a two-day workshop on Spatial Vagueness, Uncertainty, and Granularity took place at the Inn at Morro Bay on September 17-18. Organized by Matteo Cristani and Brandon Bennett, the workshop featured a series of papers on various aspects of this very important topic in geographic information science. COSIT proper started with a day of state-of-the-art tutorials on September 19. The tutorials were intended to help bridge boundaries between different disciplines involved in the conference. Tony Cohen presented "Qualitative Spatial Representations and Reasoning"; Mary Czerwinski and George Robertson presented "Navigating Information Spaces"; Jonathan Raper presented "Everything You Wanted to Know About GIS, But Were Afraid to Ask!"; and Jack Loomis and Andrew Beall

VII Preface

presented "Virtual Reality as a Research Tool for Studying Spatial Cognition". The second to fourth days of the conference were dedicated to the formal presentations and discussions of the research papers, including one invited Keynote Address by an eminent scientist each morning. On the fifth and final day, the "Doctoral Colloquium" was held. This was a forum for doctoral students to present and discuss their research with one another and with experienced research advisors. The goal of the colloquium was to give students experience presenting research in a public forum, and to give them feedback on their research and presentations. It also provided an opportunity for students and faculty to share insights on how to do a Ph.D. in an interdisciplinary field. Science is a social process too. The exchange of ideas and cementing of collaborations do not occur just in formal sessions. At COSIT 2001, these social activities included coffee breaks and lunches, a welcoming reception on the 19th, "birds-of-a-feather" dinner on the 20th, an afternoon expedition to Hearst Castle in San Simeon on the 21st, and a banquet that evening. The organizers hope that other occasions during the five days were found suitable for the social intercourse that plays such an important yet informal role in the advance of knowledge. I thank all members of the program, scientific, and organizing committees for making the meeting and this volume a success. Thanks also to the tutorial presenters for four stimulating and popular tutorials. The continued support of Springer-Verlag is gratefully acknowledged. The staff and setting of the Inn at Morro Bay made for an appealing week. The generosity of our sponsors is also appreciated. Finally, the core of any such enterprise is the participants and contributors. Their effort and enthusiasm made this worthwhile for me. July 2001

Daniel R. Montello

Program Committee Daniel R. Montello, University of California at Santa Barbara, USA (chair) Anthony Cohn, University of Leeds, UK Michel Denis, Université de Paris-Sud, France Max Egenhofer, University of Maine, USA Andrew Frank, Technical University of Vienna, Austria Christian Freksa, University of Hamburg, Germany Mary Hegarty, University of California at Santa Barbara, USA Steve Hirtle, University of Pittsburgh, USA Werner Kuhn, University of Münster, Germany David Mark, State University of New York at Buffalo, USA Barbara Tversky, Stanford University, USA

Scientific Committee Dave Abel, Australia Jochen Albrecht, USA Gary L. Allen, USA Michael Almeida, USA Kate Beard, USA Brandon Bennett, UK Tom Bittner, USA Mark Blades, UK Melissa Bowerman, The Netherlands Barbara Buttenfield, USA Roberto Casati, France Keith Clarke, USA Eliseo Clementini, Italy Helen Couclelis, USA Leila De Floriani, Italy Andreas Dieberger, USA Jack Du Bois, USA Geoffrey Edwards, Canada Gregory Elmes, USA Susan Epstein, USA Martin Erwig, USA Carola Eschenbach, Germany

Boi Faltings, Switzerland Jerome Feldman, USA Fred Fonseca, USA Scott M. Freundschuh, USA Alinda Friedman, Canada Mark Gahegan, USA Antony Galton, UK Janice Glasgow, Canada Christopher Gold, Canada Reginald Golledge, USA Suchi Gopal, USA Nicola Guarino, Italy Christopher Habel, Germany Daniel Hernández, Germany John R. Herring, USA Don Heth, Canada Kathleen Hornsby, USA Christian S. Jensen, Denmark Marinos Kavouras, Greece Rob Kitchin, Ireland Roberta Klatzky, USA Markus Knauff, Germany Benjamin Kuipers, USA

X

Organization

Scientific Committee (continued) Steve Levinson, The Netherlands Gerard Ligozat, France Jack Loomis, USA Paola Magillo, Italy Claudio Masolo, Italy Robert McMaster, USA Harvey Miller, USA Reinhard Moratz, Germany N. Hari Narayanan, USA Nora Newcombe, USA Dimitris Papadias, Greece Eric Pederson, USA Juval Portugali, Israel Ian Pratt-Hartmann, UK Jonathan Raper, UK Tony Richardson, USA John Rieser, USA Thomas Röfer, Germany Christoph Schlieder, Germany Markus Schneider, Germany

Michel Scholl, France Priti Shah, USA Jayant Sharma, USA Barry Smith, USA John Stell, UK Erik Stubkjær, Denmark Holly Taylor, USA Frank Tendick, USA Sabine Timpf, Switzerland Nectaria Tryfona, Greece Andrew Turk, Australia David Uttal, USA Achille Varzi , USA Laure Vieu, France Rob Weibel, Switzerland Karl F. Wender, Germany Steffen Werner, USA Michael Worboys, UK Wai-Kiang Yeap, New Zealand Benjamin Zhan, USA

Organizing Committee Daniel R. Montello (chair) Mary Hegarty Reginald Golledge Sarah Battersby (administrator) David Waller (tutorials) Anthony Richardson (doctoral colloquium) Mark Probert (computer systems)

Sponsors College of Letters & Science, UCSB College of Engineering, UCSB Division of Mathematical, Life & Physical Sciences, UCSB Graduate Division, UCSB Environmental Systems Research Institute (ESRI)

Table of Contents Keynote Lecture A Geographer Looks at Spatial Information Theory.......................... 1 M.F. Goodchild Geospatial Ontology and Ontologies I True Grid ........................................................................................... 14 B. Smith A Taxonomy of Granular Partitions .................................................. 28 T. Bittner, B. Smith A Geometric Theory of Vague Boundaries Based on Supervaluation .............................................................................. 44 L. Kulik Qualitative Spatio-Temporal Reasoning I When Tables Tell It All: Qualitative Spatial and Temporal Reasoning Based on Linear Orderings ....................... 60 G. Ligozat Computational Structure in Three-Valued Nearness Relations ........ 76 M. Duckham, M. Worboys Qualitative Spatio-Temporal Continuity ........................................... 92 S.M. Hazarika, A. G. Cohn Formalizations of Human Spatial Cognition Application of Supervaluation Semantics to Vaguely Defined Spatial Concepts .............................................. 108 B. Bennett Spatial and Cognitive Simulation with Multi-agent Systems .......... 124 A.U. Frank, S. Bittner, and M. Raubal A Virtual Test Bed in Support of Cognitively–Aware Geomatics Technologies ............................. 140 G. Edwards

XII

Table of Contents

Space, Cognition, and Information Systems I Evaluating the Usability of the Scale Metaphor for Querying Semantic Spaces ......................................................... 156 S.I. Fabrikant A Semantic Map as Basis for the Decision Process in the www Navigation .................................................................... 173 H. Hochmair, A.U. Frank Pragmatism and Spatial Layout Design ........................................... 189 S.L. Epstein, B. Moulin, W. Chaker, J. Glasgow, and J. Gancet Navigation: Human and Machine Approaches Spatial Frames of Reference Used in Identifying Direction of Movement: An Unexpected Turn ................................................ 206 C.R. Miller, G.L. Allen The Role of a Self-Reference System in Spatial Navigation ........... 217 M.J. Sholl The Utility of Global Representations in a Cognitive Map ............. 233 M.E. Jefferies, W.K. Yeap Keynote Lecture How Spoken Language and Signed Language Structure Space Differently .............................................................. 247 L. Talmy Language and Space Two Path Prepositions: Along and Past ........................................... 263 C. Kray, J. Baus, H. Zimmer, H. Speiser, and A. Krüger Ambiguity in Acquiring Spatial Representation from Descriptions Compared to Depictions: The Role of Spatial Orientation ....................................................... 278 H.A. Taylor, D.H. Uttal, J. Fisher, and M. Mazepa When and Why Are Visual Landmarks Used in Giving Directions? ....................................................................... 292 P.-E. Michon, M. Denis

Table of Contents

XIII

Space, Cognition, and Information Systems II Recognition of Abstract Regions in Cartographic Maps ................. 306 J.H. Steinhauer, T. Wiese, C. Freksa, and T. Barkowsky Geographical Information Retrieval with Ontologies of Place ........ 322 C.B. Jones, H. Alani, and D. Tudhope Qualitative Spatial Representation for Information Retrieval by Gazetteers .......................................... 336 C. Schlieder, T. Vögele, and U. Visser Keynote Lecture Spatial Representation and Updating: Evidence from Neuropsychological Investigations ......................... 352 M. Behrmann, J. Philbeck Cognitive Mapping Mental Processing of Geographic Knowledge ................................. 371 T. Barkowsky Spatial Cognition and the Processing of Verticality in Underground Evironments ........................................................... 387 S. Fontaine Grid Patterns and Cultural Expectations in Urban Wayfinding ....... 400 C. Davies, E. Pederson Qualitative Spatio-Temporal Reasoning II The House Is North of the River: Relative Localization of Extended Objects ......................................................................... 415 H.R. Schmidtke Double-Crossing: Decidability and Computational Complexity of a Qualitative Calculus for Navigation ......................................... 431 A. Scivos, B. Nebel Spatial Reasoning: No Need for Visual Information ....................... 447 M. Knauff, C. Jola, and G. Strube

XIV

Table of Contents

Geospatial Ontology and Ontologies II A Formal Theory of Objects and Fields ........................................... 458 A. Galton What’s in an Image? ......................................................................... 474 G. Câmara, M.J. Egenhofer, F. Fonseca, and A.M.V. Monteiro Features, Objects, and Other Things: Ontological Distinctions in the Geographic Domain ....................... 489 D.M. Mark, A. Skupin, and B. Smith Author Index ................................................................................... 503

A Geographer Looks at Spatial Information Theory Michael F. Goodchild 1 1

National Center for Geographic Information and Analysis, and Department of Geography, University of California, Santa Barbara, California 93106-4060, USA [email protected]

Abstract. Geographic information is defined as a subset of spatial information, specific to the spatiotemporal frame of the Earth’s surface. Thus geographic information theory inherits the results of spatial information theory, but adds results that reflect the specific properties of geographic information. I describe six general properties of geographic information, and show that in some cases specialization has assumed other properties that are less generally observed. A recognition of the distinction between geographic and spatial would allow geographic information theory to achieve greater depth and utility.

1 Introduction The term geographic might be said to refer to features and phenomena at or near the surface of the Earth, and if so, geographic information is information about such features and phenomena. More formally, geographic information might be defined as consisting of atomic pairs of the form where x is a location in space-time, and z is a set of properties of that location [10]; or of information that is reducible to such pairs. Thus geographic refers to a spatial domain consisting of the Earth’s surface and near-surface, and times extending forwards and backwards from the present. The term also implies a certain range of spatial resolution, from perhaps 1cm to 10km, that excludes any quantum or relativistic effects and is thus rigidly Newtonian. In this sense geographic is a subset or specialization of spatial, which by extension refers to any spatiotemporal frame, and any spatial resolution, and also includes nonCartesian spaces. The spaces defined by the human body, or an automobile, or the universe are instances of spatial. A spatial frame may contain the geographic frame, as in the case of the universe, but the geographic frame may also contain spatial frames that may move within it. Thus a human sees the geographic frame as a rigid and fixed structure, and other spaces as variously embedded within it. From this perspective the term geospatial is essentially identical to and redundant with geographic. While geographic inherits many of its properties from spatial, it also adds new ones, and thus specializes the definition. If “spatial is special”, as many have suggested [1], [13], then geographic should be even more special, and a theory of geographic information should be distinct from a theory of spatial information, inheriting all of the generality of the latter, but adding its own specifics. Thus when a geographer looks at spatial information theory, he or she logically asks not whether D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 1−13, 2001.  Springer-Verlag Berlin Heidelberg 2001

2

M.F. Goodchild

the conclusions of spatial information theory are useful, as they must necessarily be in any subclass, but whether the conclusions could be more useful if the specifics of the subclass were exposed. In this paper I examine the specific nature of geographic information, by discussing six principles that appear to be generally true of geographic information but not necessarily true of spatial information. By doing so I hope to demonstrate that while its generalities are undoubtedly useful, a theory of spatial information can be made even more useful and effective for geographic information if it recognizes and exploits those specifics. The specific nature of geographic information imposes constraints, narrowing the options that must be considered in the general case. It also suggests underlying structures and causal mechanisms that may further narrow the options, and allow theory development to proceed to deeper levels.

2 General Properties of Geographic Information

2.1 Positions in the Geographic Frame Are Uncertain First, consider the determination of position on the Earth’s surface. All measuring instruments are subject to error, and many of the instruments used to measure position are subject to substantial errors. For example, routine measurements using the Global Positioning System are subject to errors on the order of 10m. Monuments in supposedly fixed positions move as a result of tectonic activity and the movement of crustal plates. More fundamentally, the frame as defined by the Earth’s axis moves as the Earth wobbles, and along with it the Poles and Equator; and the ellipsoids and other mathematical functions used to approximate the shape of the Earth are defined only to limited precision. For all of these reasons, it is impossible to measure location on the Earth’s surface exactly, or to determine equality of position based purely on information about location. As a consequence the geometry underlying all geographic information technologies is approximate. Moreover, errors are normally much greater than the uncertainty inherent in using discrete numerical methods in computing systems, although these sometimes contribute significantly. For example, single-precision arithmetic normally offers 7 decimal digits; but 1 part in 107 of the Earth’s radius is less than a meter, and thus substantially more precise than the accuracy of most global databases. Double precision offers 14 decimal digits, which supports accurate positioning on the Earth’s surface at sub-micron levels, an absurd level of precision given the typical accuracy of geographic data. In practice, positional accuracy seems to fall within a fairly narrow range of 10-3 to 10-4 of the extent of a project for a variety of reasons [8]. Thus only when coordinates are represented by short integers is there a need to be concerned about machine precision. The limited precision of positional representations has motivated a number of projects, such as ROSE [11], that have developed algorithms that are consistent with a discrete rather than continuous space. In effect, these algorithms assume that numerical methods are implemented in discrete form in a space that is fundamentally

A Geographer Looks at Spatial Information Theory

3

continuous, and that position is knowable to an accuracy that is greater than the precision of the methods. But inaccuracy requires a somewhat different approach, because each point’s position must be conceptualized as located at the center of a circle of possibility in continuous space—one might term this an object-centered approach, to distinguish it from the space-centered approach of a discrete-space geometry (Figure 1). Because inaccuracies are likely to be many orders of magnitude greater than imprecisions, the object-centered approach seems to be much more strongly motivated for geographic information than the space-centered approach, but to have received much less attention.

(A)

(B)

Fig. 1. (A) In an object-centered approach, limited accuracy requires that the possible true locations of a point are located within a given distance of the point's apparent position. (B) In a space-centered approach, limited precision requires that points appear to be at the intersections of a fixed grid.

Thus while it is interesting to theorize about spatial information in ways that include the possibility of equality of position, in practice for geographic information it is almost never possible to determine equality. We cannot determine whether a point lies exactly on a line, or whether two lines are exactly equal, based on position alone. Thus point-in-polygon routines designed to determine enclosure normally offer only a binary response (in or out), and polygon overlay routines infer equality of position using user-defined positional tolerances, not by exact comparison. It is generally unwise to compute topology from geometry, and better to allow independently determined topology to over-ride geometry when the two conflict, as they often will. Because the distance between a house and its street centerline is often less than the accuracy of positioning of either, many databases code the house’s side of the street directly (e.g., TIGER and its derivatives). It is generally unsafe to rely on point-inpolygon operations to determine the parcel containing a point, such as a utility pole, and better to code the containment directly, and to allow this topological information to over-ride any information obtained from geometry. In summary, a theory of geographic information can often afford to drop the equality option, because it implies an unrealistic level of accuracy in positioning.

4

M.F. Goodchild

Moreover, the nature of inaccuracy suggests that an object-centered approach to imprecise geometry will be more productive than a space-centered approach.

2.2 Spatial Dependence Is Endemic in Geographic Information A variable is said to possess spatial dependence if correlations exist between its values at distinct points. Frequently the degree of similarity between the values at two points increases as the two points approach each other, implying a degree of continuity and smoothness. Geographic information is observed to possess this type of spatial dependence, and this observation is sufficiently general to warrant the status of a law, often identified with Waldo Tobler [18] and stated thus: “All things are similar, but nearby things are more similar than distant things.” The effect expressed in the law is easily measured by the Geary and Moran statistics of spatial autocorrelation, and by the variogram, and the field of geostatistics is founded on what Matheron [15] termed regionalized variables, or variables possessing strong spatial dependence in accordance with Tobler’s law. It is possible to distinguish between positive and negative spatial autocorrelation; in the positive case nearby pairs of points are more similar than distant pairs, while in the negative case nearby pairs are more different than distant pairs. But such measures are scale-specific, and it is generally impossible for a variable to be negatively autocorrelated at all scales. Thus the familiar chessboard shows strong negative autocorrelation between adjacent squares, but strong positive autocorrelation within squares. Zero spatial autocorrelation results when values at distinct points are uncorrelated, or statistically independent. This is a reasonable condition when the points are very far apart, or separated by what geostatistics terms the range of the variable. But consider a world in which spatial autocorrelation is zero at all scales. In such a world, an infinitesimal movement would be sufficient to encounter the entire range of the variable, and it would be impossible to construct descriptions or representations of the world that were less than infinitely large. In effect, spatial dependence is essential for description, mapping, and the very existence of geographic information as a useful commodity. A world without spatial dependence would be an impossible world to describe or inhabit. Many statistical methods assume independence of observations, and thus are problematic when applied to geographic information. Inferential tests associated with the Geary and Moran coefficients [5] invoke a null hypothesis of zero spatial dependence, which is virtually untenable with respect to geographic information. Thus any experiment which results in acceptance of this null hypothesis suggests a Type II statistical error—acceptance of the null hypothesis when in fact it is false. Tobler’s law is an observation about geographic space, and thus clearly not true of all spaces, although it seems that much theorizing about spatial information has assumed it implicitly. For example, Tobler’s law is clearly implicit in any discussion of uniform regions or polygons.

A Geographer Looks at Spatial Information Theory

5

2.3 Geographic Space Is Heterogeneous In the discipline of geography there is an ancient debate that is still annually rehearsed in seminars on geographic thought, concerning whether the purpose of research should be to discover general truths, or to document specific facts; the two positions are termed nomothetic and idiographic respectively. While the former is often presented as more scientific, it is also possible for idiographic description to follow scientific principles of replicability. In a geographic context the two are expressed as distinct strategies with respect to our understanding of the Earth’s surface; in the nomothetic strategy, research is successful if it uncovers principles that are true everywhere in the domain, while the idiographic strategy supports detailed study of the unique characteristics of places, that may or may not lead to generalizations about the entire domain. Clearly the nomothetic strategy requires some degree of homogeneity of the domain, not perhaps in its form, but probably in the processes that modify and shape it; and the search for such processes dominates the nomothetic approach. On the other hand the idiographic strategy requires no homogeneity at any level. One possible compromise between these two positions exploits the potential of geographic characteristics to repeat themselves. For example, all of the world is not like Bloomfield Hills, Michigan; but market researchers are well aware that the characteristics of Bloomfield Hills relevant to marketing are very much like those of Scottsdale, Arizona. Thus it may not be possible to generalize from one region to the entire planet, but it is often possible to generalize from one region to several similar regions; geography may not be uniform, but it may be repetitive. The strategy relies on our ability to define and measure suitable metrics of similarity. Recently another compromise strategy appears to be emerging, and to be gaining in popularity. This strategy argues that any general laws relevant to the geographic world are likely to be of limited predictive power, unlike, say, the general laws of physics. The unexplained variation in any law is likely to be geographically variable, because the Earth’s surface is essentially heterogeneous. Thus it is appropriate to define a law to the level of its inputs and outputs, but to regard one or more of the parameters of the law as geographic variables. For example, consider a law z = f(y), and assume that f is a linear function. We might expect the law to apply everywhere, but we might expect its constants a and b (as in z = a + bx) to vary geographically. Such variation can be readily exposed as shown in Figure 2. Geographically Weighted Regression (GWR; [7]) is one of a number of place-based analytic techniques that adopt this compromise between the nomothetic and idiographic strategies. The Earth’s surface exhibits enormous variation, and because of Tobler’s law it is often necessary to scan a large fraction of the surface to encounter all of its variability; a small area of the surface typically encompasses only a small fraction of any variable’s total variation. It follows that the results of an analysis almost always depend explicitly on the geographic bounds of the study region, and that a shift of boundaries will produce different results. A small region does not produce a representative sample of the Earth’s surface. As with spatial dependence, spatial heterogeneity appears to be a defining characteristic of geographic space.

6

M.F. Goodchild

2.4 The Geographic World Is Dynamic As noted in the previous section, the heterogeneous nature of the geographic world, coupled with the nomothetic need to generalize, leads inevitably to an emphasis on the study of process in preference to form. Geomorphologists, for example, have long argued that study of process is of greater significance and value than study of form; that understanding how the world works is more important than understanding how it looks. The processes of interest to geomorphologists are natural, but the argument applies as well to the human processes that modify the landscape, such as settlement and migration, as to the physical processes such as erosion and tectonic activity. The world of geographic information is also concerned with design, or the study of deliberate, normative modifications of the landscape by human action (e.g., [16]).

Fig. 2. Geographically Weighted Regression is conducted as follows: (1) Select one observation as reference point, and weight all other observations according to a decreasing function of distance from the reference (e.g., by a negative exponential function of distance); (2) Fit the constants a and b using points weighted in this way, and assign the derived values to the reference point; (3) Repeat for all observations, and interpolate complete surfaces for a and b (only one surface is shown).

By contrast, our perspective on the geographic world is relatively static, and most of our information comes in the form of snapshots at specific instants of time. The lack of attention to time in geographic information systems, which draw heavily from cartographic roots, is often recognized, as is the relative importance of information about change to the development of public policy and the making of decisions. Escaping a static view of the world remains one of the most important challenges of GIS. There are many kinds of geographic information. One normally cites maps and images as the most familiar examples, but geographic information can also take the form of text description, spoken narrative, and even music (the songlines of the Australian aborigine are a form of geographic information; [4]), since all of these meet the definition of geographic information given above. Information about dynamic processes is expressed in many different forms: as mathematical models, such as partial differential equations (PDEs; e.g., [17]); as conceptual models

A Geographer Looks at Spatial Information Theory

7

expressed in text or diagrams; and as computational models expressed in computer code. But none of these meet the definition, since none is reducible to the atomic form . Yet they are certainly expressible in binary form, given appropriate methods of coding. Dynamic process models are analogous to the transformations familiar to users of GIS, because they map the geographic world from one state to another. For example, a PDE expressed in numeric form as a finite difference computer code takes the initial state of the system, and computes future states based on appropriate functions and parameters. In that sense dynamic process models are similar to GIS operations such as buffering, which similarly accept input geographic information and produce new geographic information as output. From an object-oriented perspective, dynamic process models are akin to the methods that can be encapsulated with object classes. The field of geocomputation has emerged in recent years as an intellectual home for research on dynamic process models and their implementation. The relationship between geocomputation and geographic information theory has been discussed by several authors and in several presentations (e.g., [2], [6]), but remains controversial. If study of process trumps study of form, as it clearly does in many areas of science, and if it motivates much acquisition and analysis of geographic information, then an understanding of process is clearly important to effective theorizing about geographic information. I believe therefore that links to the study of process can enrich geographic information theory, and that dialog is essential between the geocomputation and geographic information theory communities.

2.5 Much Geographic Information Is Derivative The raw data of science often consist of original measurements made with instruments. The terms accuracy and precision refer to the fit between measurements and truth, and repeated measurements respectively. For many instruments these parameters are well known, and can be used to analyze the impacts of errors on subsequent analyses. The geographic information that is presented on maps and in databases is rarely composed of original measurements, however. A user of a soil database sees polygons with uniform classes, rather than the original measurements that were obtained by analyzing soils collected in pits, or the aerial photographs that were used to extrapolate the information obtained from pits to create a complete coverage. Much geographic information is similarly the result of compilation, interpretation, analysis, and calculation, almost all of which remains hidden from the user. The visible form of representation (classified polygons in this example) may have little relationship to the forms of representation used at earlier stages (e.g., point samples, rasterized aerial photographs, digital elevation models). Consider the soil database example in the context of uncertainty, and the impact of uncertainty on its polygons and homogeneous classes. Let uncertainty be interpreted as meaning that other databases might equally well have arisen through the process of derivation, and that in the absence of other information all such alternative databases should be taken to be equally likely. For example, the errors inherent in the measurement of properties in the field will eventually result in alternative databases.

8

M.F. Goodchild

Without any knowledge of the process of derivation, we have no guidance about the form that such alternative databases might take, and must therefore consider every possibility. Thus we have no reason to assume that alternative databases will have boundaries in the same positions, or even the same numbers of polygons, edges, and nodes. For example, different compilers will most likely have produced databases that are topologically as well as geometrically and thematically distinct, despite working from the same original data. To pursue this example a little further, as an instance of any representation of a nominal field (including databases of soils, land cover, land use, vegetation type, land ownership, etc.), it is clear that many of the models of uncertainty studied in spatial information theory represent somewhat arbitrary choices. The egg-yolk model, for example, focuses on individual polygons, and implies that alternative databases will have the same topology (the same boundary network). Moreover, it proposes that the region of uncertainty in each polygon will be adjacent to the boundary. Although this is in a sense consistent with Tobler’s law, there are several reasons why it and other similar models such as the epsilon band may be inappropriate for many types of geographic information. First, in the case of a database derived from remote sensing, the definition of a class is statistically based, and influenced by the relative abundances of pixels in different parts of the spectrum. If spectral responses vary systematically with distance inwards from the polygon boundary, then the responses typical of the periphery will be more common than the responses typical of the center. Thus the choice of a class for a polygon may depend more on the peripheral areas than the central area, just as the suburban class is more typical of a city than the central core. In this sense the periphery may be more certain than the core. Second, an important stage in the compilation of any soil map is cartographic—a cartographer ultimately determines the positions of polygon boundaries, and decides which small patches should be separate polygons, and which should be merged with their surroundings. Any mapped polygon will likely contain many small inclusions, or patches of some other class, that have been deleted by the cartographer. Now consider such an inclusion near the polygon edge, and assume it is similar to the class of the neighboring polygon across the edge (Figure 3). When the line is drawn, it may be able to accommodate the inclusion by modifying the polygon boundary. But inclusions near the core of the polygon must be ignored. In summary, the cartographic process of map compilation may lead to greater lack of homogeneity in the core of polygons than on the periphery. Finally, a distance-based epsilon band or egg-yolk model raises awkward issues of process, since it is difficult to think of real processes that might lead to a zone of uncertainty of uniform width inside a polygon. In a botanical example, it is possible that dispersion of seeds into an area from outside its boundary might produce a uniform gradient of uncertainty, but it is hard to imagine a similar process operating in the case of soils. Thus from a geographic perspective, there seem to be good reasons not to believe in epsilon bands or egg-yolk models, but to take a broader view of the alternatives that result from uncertainty. In such cases spatial information theory seems more restrictive than geographic information theory, which is counter to the arguments presented earlier.

A Geographer Looks at Spatial Information Theory

9

What is clearly lacking in all such discussions of the uncertainty associated with polygons is a clearly defined and reasonable model of how one specific set of polygons resulted from the process of compilation—such a model would also provide a basis for theorizing about uncertainty, by modeling the generation of alternative databases. What is missing, then, is a set of comprehensive models of uncertainty in nominal fields, that serve to frame the methods used to compile databases. Although many such models have been discussed in the literature, I will focus only on one [9] as an example to demonstrate the efficacy of this approach to conceptualizing uncertainty in one class of geographic information. I do not want to suggest that this model is in any way unique, or even the most appropriate in many instances. However it seems to provide one conceptual framework for the process by which the polygons of a soil database came into being.

Fig. 3. In drawing a polygon (heavy solid line) a cartographer will ignore inclusions of a different class that fall below the size of the minimum mapping unit. But an inclusion near the boundary may result in a modification of the boundary’s position. Thus there may be greater uncertainty in the center of the polygon than in the periphery.

Consider a set of fields {z1 (x),z2 (x),…} measured on continuous scales. Each field represents the spatial variation of one measurable quantity relevant to soil mapping, such as soil pH, depth to water table, or organic carbon content. Now consider a space defined by these variables (Figure 4 shows an example in the case of only two variables). Define c(z) as a function over this space, the discrete values of c defining a set of classes. The space and its function are analogous to the classifiers used in remote sensing (where the defining variables are spectral responses in the various bands of a sensor; for example, a parallelepiped classifier is named for the geometric form of the domains formed in z by values of c). For the purposes of this paper I term this a phase space by analogy to the physical states of a substance. Finally, map any

10

M.F. Goodchild

geographic location x to a class c(x) by determining its measurable quantities {z1 (x),z2 (x),…}, and identifying the class associated with those quantities. z 1(x)

z 2(x) z 3(x)

z2

c=4

c=2

c=1

c=3

z1

4

2

3

3 1 2

2

Fig. 4. A possible model for the derivation of polygons in a soil map. Variables are measured at sample points, and interpolated to form continuous fields. A phase space assigns every vector of field values to a class. Finally, the interpolated fields and phase space are combined to form a nominal field.

Now consider the implications of this model. First, successive determinations of the underlying variables z1 ,z2 ,… will be subject to the measurement errors inherent in the relevant instruments. In practice the variables will not have been measured everywhere, but will have been interpolated from point measurements, so interpolation errors will need to be included, perhaps using the techniques of geostatistics [12]. Thus it will be possible to simulate alternative measurements and

A Geographer Looks at Spatial Information Theory

11

interpolations (see specifically conditional simulation), and consequently alternative databases. Second, the implications of scale change can be examined by coarsening the underlying variables {z1 (x),z2 (x),…}, which is readily done using simple convolution filters. Third, the implications of coarsening or refining the classification scheme can be examined by making appropriate changes to the phase space (e.g., a class can be subdivided into two or more classes by subdividing its domain). The model provides an easy way of conceptualizing the implications of Tobler’s law. Since all of the underlying variables are geographic, we expect them to exhibit strong spatial dependence, and this of course is the basis for all techniques of spatial interpolation. It follows that two classes can be adjacent in geographic space if and only if they are adjacent in phase space. In summary, much geographic information is derivative, in the sense that it is the result of compilation, interpretation, analysis, and calculation from original measurements that are not normally exposed to the user; these processes can involve many stages and many individuals. A model such as that presented above provides a way of conceptualizing the process of creation of a nominal field (and the collection of polygons used to represent it). Moreover, uncertainty is represented explicitly in the model, in this case as measurement error in the original point observations, and errors in the process of interpolation used to create continuous fields. Thus the alternatives to be expected due to uncertainty can be modeled explicitly, as a comparatively narrow range of options. Models such as the epsilon band or egg-yolk, which assume no such background conceptual framework, can be examined to see if they are feasible within the framework, and to determine the degree of generality of their assumptions with respect to the framework.

2.6 Many Geographic Attributes Are Scale-Specific Consider the field defined by the elevation of the Earth’s surface. Overhanging cliffs, or locations x where the field is many-valued, are sufficiently rare to be ignored in most circumstances. Elevations are discontinuous at cliffs, where z(x+δx) does not tend to z(x) as δx tends to zero. More importantly gradients are discontinuous at ridges and sharp valleys, where the surface lacks well-defined tangents or derivatives. Such properties are characteristic of fractal surfaces, and Mandelbrot [14] has shown how fractal behavior is typical of many geographic phenomena. One of the commonest GIS functions applied to digital elevation models is the determination of slope. Since the elevation surface is already represented in such models as a finite-difference approximation, or a regular grid of point measurements, it is convenient to estimate slope by comparing elevations over a neighborhood of such points, typically a 3 by 3 neighborhood. Burrough and McDonnell [3] and others review the alternative estimating equations. Implicit in this approach is the dependence of the resulting estimates of slope on the grid spacing. But if the elevation surface lacks tangents, these slope values are not estimates of the derivatives of the surface, but explicitly scale-specific. In essence, there is no such thing as the slope of a geographic surface, only slope at a specific scale or grid spacing. This property of scale specificity is very general for geographic data, and extends well beyond the case of interval fields such as elevation. The derivation process for

12

M.F. Goodchild

nominal fields discussed in the previous section is also scale specific, as is the definition of many of the classes used in geographic databases. Consider the example of the land cover class urban. Scale is not often specific in its definition, but is clearly important. The pixels covering New York City may be roughly homogeneous in spectral response when seen from the AVHRR satellite, with a ground resolution of approximately 1.1km, but at the 4m resolution of the multispectral IKONOS sensor the homogeneity breaks down into grass, concrete, asphalt, roof materials, etc. Scale specificity has obvious implications for any theory of the effects of spatial and thematic resolution on geographic data. Rather than breaking down at finer scales, the domain urban in the phase space discussed in the previous section disappears completely, and its replacement classes of grass, concrete, asphalt, etc. may share none of its boundaries. Thus we cannot assume a hierarchical relationship between coarse and refined classes. Instead, it seems likely that new classes will be needed on the boundaries of coarse classes in phase space as well as within their domains, since it is probably here that the greatest heterogeneity exists.

3 Conclusion I have focused in this paper on the differences between spatial and geographic, defining those terms such that geographic is a specialization of spatial. The six general properties discussed above are clearly only a sample, and there may well be others that are equally or more important in specializing spatial. Each of these specializing properties provides a basis for extending spatial information theory, by narrowing the set of possibilities that it must consider, and thus allowing theory to be extended and deepened. In the case of the framework model, the geographic case provides a basis for additional theorizing through the formulation of a background framework, or model of the process by which geographic information was compiled. Finally, I have identified ways in which the specialization of spatial appears to have proceeded in a direction that is inconsistent with the general properties of geographic. The properties discussed in this paper are generalizations from empirical observations, and as such fall into a classic tradition of observations that serve to drive theory. Although there is value in theorizing in the absence of such general observations, there is clearly much greater practical value in theory that is grounded in empiricism. In this case, the domain of spatial is far greater than the domain of geographic, and many more subclasses exist, each of which can be expected to exhibit general properties that may or may not be similar to those exhibited by the geographic domain. Thus theorizing about spatial information results in conclusions that apply in all domains, whereas theories about geographic information may apply only to the geographic domain, and this potential disadvantage must be weighed against the advantages of domain-specific theory driven by empiricism.

A Geographer Looks at Spatial Information Theory

13

Acknowledgment The Center for Spatially Integrated Social Science is supported by the National Science Foundation.

References 1. Anselin, L.: What is Special About Spatial Data? Alternative Perspectives on Spatial Data Analysis. Technical Report 89-4. National Center for Geographic Information and Analysis, Santa Barbara, Calif. (1989) 2. Berry, B.J.L.: Towards Computational Geography. Paper presented at GIScience 2000, Savannah, Georgia, October 28 (2000) 3. Burrough, P.A., McDonnell, R.A.: Principles of Geographical Information Systems. Oxford University Press, New York (1998) 4. Chatwin, B.: The Songlines. Viking, New York (1987) 5. Cliff, A.D., Ord, J.K.: Spatial Processes: Models and Applications. Pion, London (1981) 6. Couclelis, H.: Geocomputation in Context. In Longley, P.A., Brooks, S.M., McDonnell, R., MacMillan, W., Geocomputation, A Primer. Wiley, New York (1998) 17–30 7. Fotheringham, A.S., Brunsdon, C., Charlton, M.E.: Quantitative Geography. Sage, London (2000) 8. Goodchild, M.F.: Scale in Remote Sensing and GIS. In McMaster, R., Sheppard, E.S., editors, Scale and Geographic Inquiry. Blackwell, Oxford (in press) 9. Goodchild, M.F., Dubuc, O: A Model of Error for Choropleth Maps with Applications to Geographic Information Systems. Proceedings, Auto Carto 8. ASPRS/ACSM, Falls Church, Virginia (1987) 165–174 10. Goodchild, M.F., Egenhofer, M.J., Kemp, K.K., Mark, D.M., Sheppard, E.: Introduction to the Varenius Project. Int. J. Geogr. Info. Sci. 13 (1999) 731–745 11. Güting, R.M., Schneider, M.: Realm-Based Spatial Data Types: The ROSE Algebra. VLDB J. 4 (1995) 100–143 12. Isaaks, E.H., Srivastava, R.M.: Applied Geostatistics. Oxford University Press, New York (1989) 13. Longley, P.A., Goodchild, M.F., Maguire, D.J., Rhind, D.W.: Geographic Information Systems and Science. Wiley, New York (2001) 14. Mandelbrot, B.B.: The Fractal Geometry of Nature. Freeman, San Francisco (1983) 15. Matheron, G.: The Theory of Regionalized Variables and Its Applications. Ecole National Superieure des Mines, Paris (1971) 16. McHarg, I.L.: Design with Nature. The Natural History Press, Garden City, N.Y. (1969) 17. Smith, T.R., Birnir, B., Merchant, G.E.: Towards an Elementary Theory of Drainage Basin Evolution I: The Theoretical Basis. Comp. Geosci. 23 (1997) 811–822 18. Tobler, W.R.: A Computer Movie: Simulation of Population Change in the Detroit Region. Economic Geography 46 (1970) 234–240

True Grid Barry Smith Department of Philosophy, Center for Cognitive Science and NCGIA University at Buffalo, NY 14260, USA [email protected]

Abstract. The Renaissance architect, moral philosopher, cryptographer, mathematician, Papal adviser, painter, city planner and land surveyor Leon Battista Alberti pmided the theoretical foundations of modem perspective geometry. Alberti's work on perspective exerted a powerful influence on painters of the stature of Albrecht Diirer, Leonardo da Vinci and Piero della Francesca. But his Dellapittura of 1435-36 contains also a hitherto unrecognized ontology of pictorial projection. We sketch this ontology, and show how it can be generalized to apply to representative devices in general, including maps and spatial and non-spatial databases.

Fig. 1 Albrecht Diirer's interpretation of 'The Draftsman's Net'

1 Through a Glass Clearly The Della pittura of the Renaissance artist and art theorist Leon Battista Alberti, dating from 1435-36, is the first modem treatise on painting. It defends a view according to which the proper goal of the artist is to produce a picture that will represent the visible world as ifthe observer of the picture were looking through a window. This open window conception reflects a time when painting is still an adjunct of architecture: paintings are designed to enhance one's home. The aesthetic experience of a building's interior and the aesthetic experience of the paintings on its walls are meant to be fused into one: the picture must be so painted that a spectator's imagination is drawn towards the wall-plane, not away from it. This is why Renaissance painters, acting as interior decorators, revived and elaborated the system of perspective already used by interior decorators at Pompeii and elsewhere in the ancient world. (Collingwood 1938, p. 153) D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 14−27, 2001.  Springer-Verlag Berlin Heidelberg 2001

True Grid

15

Alberti's conception of the painting was extremely influential. Indeed the art historian Erwin Panofsky argues that, while there are elements of perspectival foreshortening in earlier works of art, one can properly speak of a perspectival intuition of space only where a 'whole picture is as it were transformed into a "window" through which we should then believe ourselves to be loolang into the space'. (Panofsky 1927) To rub the same point home, Diirer, in his woodcuts, always depicts the process of perspective projection in such a way that this process is situated in a room in which a section of window clearly appears. ('Perspective' means, roughly, 'seeing through' or 'seeing clearly'.)

Fig. 2 Alberti's Reticolato

Alberti presented his ideas on perspective in terms of his so-called 'reticolato', also known as Alberti's 'grid' or 'grill' (graticola), a mechanical aid to painters in the execution of the fenestra aperta technique, which involved creating a grid across an actual window in order to enable the artist to transfer the scene visible through the window to a correspondingly gridded canvas. Because parallax is here so strong, it is unlikely that such,devices were ever in fact used by painters. Even the slightest movement on the painter's part will have a dramatic effect upon the scene perceived. We might think of the reticolato, rather, as a pedagogical device, designed to help the artist understand how perspective works. Figure 2 depicts rays extending from the abstractly represented (single) eye of the artist, passing out through the cells of the artist's grid and forming a visual pyramid along their way to their final destination: an array of planes in the background of the figure. To the right of the grid is a correspondingly gridded notepad to which the artist is supposed to transfer the contents of each successive cell, contents that have been 'measured' by the rays, which reach out like feelers to touch the corresponding portions of reality. In this way the artist can apprehend in systematic and accurate fashion the visual qualities in the scene before him. Diirer's treatise on measurement, his Underweysung der Messung of 1527, illustrates a range of similar machines by which an artist might 'scientifically' depict people and objects along these same lines. The machines employ a glass plate or frame divided into small squares by a net or veil of black thread. This allows the imagined artist to locate marks within the space of the painting in such a way that their shapes, sizes and relative positions conform to what we would see if we were observing corresponding objects in reality. At the same time the grids encourage a new way of seeing, through which a portion of the visible world is organized into a geometric composition.

16

B. Smith

2 Theatrum Orbis Terrarum The practical problem of projecting an array of objects existing in three-dimensional space onto a two-dimensional plane was solved at around the time of Brunelleschi, who is held to have created the first painting - of the Baptistery of St. John in Florence - in 'true perspective', sometime between 1415 and 1425. The problem was solved theoretically by Alberti in Book I of Della pittura, which presents the mathematical theory of the way in which a plane intersects a visual pyramid in exactly the way that is captured intuitively in images of the reticolato. Over the next century and a half Brunelleschi's and Alberti's work, and that of their contemporaries and successors, including not only Diirer but also Piero della Francesca and Leonardo da Vinci (all of whom were influenced by Alberti), transformed painting in a way which enabled European art for the first time to free itself from the inhibiting burden of those earlier traditions of visual representation which had remained unaware of perspective. The theoretical solution of the problem of perspective put forward in Della pittura was a scientific discovery of the first importance, and it ranks with the later contribution of Desargues in launching our contemporary understanding of projective geometry. But why did mankind have to wait until the fifteenth century, 1700 years after Euclid's Elements and Optics, to take what Panofsky calls 'the apparently small step of intersecting the visual pyramid with a plane'? How can this be lag explained, if perspective had been present in all seeing fiom the very start? Samuel Edgerton, in his The Renaissance Rebirth of Linear Perspective, presents a two-part solution to this problem, holding 1. that there arose among a certain group of citizens of Florence in the early years of the fifteenth century a new way of apprehending visual space as a structure ordered by an abstract uniform system of linear coordinates, and 2. that the decisive impetus towards this new was of seeing was inspired by developments in cartography, and specifically by the rediscovery of Ptolemy's Geographia, a work dating fiom around 140 A.D., which arrived in Florence in 1400 to great acclaim. In more traditional metaphysical systems, such as were employed, for example, by Aristotle, a distinction had been drawn between the realm of astronomy: which is subject to precise, intelligible mathematical laws, and the sublunar world of change and decay, which is only partially intelligible to mortals such as ourselves. The principal achievement of Ptolemy's Geogmphia turned on its demonstration of the possibility of using a regular mathematical grid system to map the entire known world. Ptolemy thereby showed how the earth below could be comprehended in a uniform way in terms of a single mathematical system. Essential to this achievement was the idea that the grid not only have the mathematical properties of an exhaustive tessellation, but also that it be transparent. Ptolemy's grid is not a part of any of reconstruction of some abstract mathematical realm. It is designed, rather, to help us to grasp this world, the world of sensate matter, as it really is. The impact of Ptolemy's transparent grid system was so great that atready by 1424 Florence has acquired the reputation of a center of cartographic and geographic study, and its influence may have extended, through commentaries on Florentine versions of the Geographia, to Christopher Columbus. Ptolemy's grid system also began to be taken up as a basis for territorial boundarydemarcation. Certainly grid systems had been used for surveying purposes since much earlier times, above all by the agrimensores who had introduced centuriation into many parts of the Roman Empire. But like the grids used in the seventeenth century in dividing up the Dutch polders Roman centuriation applied always to the demarcation of intratemtorial lines. During the wars of 1420, however, a longitudinal line was proposed as the boundary between the two states of Milan and Florence. Edgerton (1975, p. 115) conjectures that this may have been the first occasion when an imaginary mathematical line - a fiat boundary - was recognized as a political-territorial limit. As Veltman (1977) points out, there are a number of problems with the details of Edgerton's account. Yet the similarities between Ptolemy's method of projecting arcs of circles visible on a globe onto a planar map and the method of perspective painting encapsulated in Alberti's reticolato are strikingly close, and the hypothesis that Alberti recognized the significance of Ptolemy's cartographic projection method for painting is supported further by Alberti's own claims on behalf of his reticolato, for example that it 'affords the greatest assistance in executing

True Grid

17

your pictures, sinee you can see any object that is round and in relief, represented on the flat surface of the veil.' (Alberti 1435/36, pp. 68 f.) In his introduction to the English translation of Della pithrra, Spencer conjectures (1956) that Alberti may have arrived at his solution to the problem of perspective also through his own experiences in the domain of surveying.' Between 1431 and 1434, which is to say just before the completion of Della pittura, Alberti composed a small work entitled Descriptio urbk Romae in which he sets forth both a method for surveying and a table of sightings obtained in applying this ~ needs some means to determine the method to yield what he calls a 'picture' of ~ o m eA. surveyor proportionate distance between any two quantities. There can be no doubt that Alberti understood such a method,) and one which did not make use of trigonometry, which had not yet been invented. Further evidence that surveying is a source of Alberti's construction is provided by the privileged role awarded by Alberti to the measure of a staff held at arm's length. Spencer points out further that Piero della Francesca gives an account of a perspective construction - based on plan and elevation drawings connected by lines fiom a point of sight and cut by a perpendicular which is essentially Alberti's surveying method from the Descriptio urbis Romae moved indoors to the drawing board.

3 Fiat Lux Alberti's contribution to the history of cartography has been noted by others. Our purpose here is to show that Della pittura contains also a contribution to our understanding of the ontology of pictures which can be generalized to projective devices in general. There are, according to Alberti, two kinds of matter with which the painter must be concerned. On the one hand is the threedimensional matter of the observable world, which exists in space and light. On the other hand is the two-dimensional matter of the painting, a simulacrum of reality that is produced by the painter, who 'must find a means of controlling the matter of the macrocosm if he is to represent it in his microcosm.' (Spencer 1956, p. 19) The first kind of matter is composed of surfaces in threedimensional reality, the second of marks the artist makes on the flat plane of the canvas. (Compare Gibson 1980) This second kind of matter exists, if the artist is successful, in the form of a visual story (istoria) that is constructed out of points, lines and planes (marks) on a panel or canvas. The latter are grouped together to form (for example) limbs, bodies and groups of bodies related together in a way that is analogous, as Alberti sees it, to the way in which words, phrases, sentences and paragraphs are related together in natural language. Alberti develops rules for manipulating these various elements in an btoria, based on the four principles of dignita, varieta, modestia and verisimilitude. Together with geometry, these principles constitute the basis of a rational art or indeed of a science of painting.

The association between optics and s w e y i n g has a long tradition, as is shown not least by the inclusion of four theorems on sweying in Euclid's Optics. The philosopher Al-Farabi could write of optics in the tenth century that it 'makes it possible for one to know the measurement of that which is far distant, for example, the height of tall trees and walls, the width of valleys and rivers, the height of mountains and the depth of valleys, rivers' (from Velhnan 1999). An account of the instrument which Alberti invented for this purpose is given by Spencer as follows: a bronze disc [is] mounted parallel to the surface of the earth and divided on the circumference into 48 degrees. At the centre a metal or wooden ruler, divided into 50 degrees, is pivoted. ... When the ruler is placed at right angles to the line of sight, it becomes possible to compute the distance of the object given its width - or its actual width - given the distance - by means of the similarity of triangles. (Spencer 1956, pp. 113 f.) In his Ludi mathematici composed for Borso d'Este about 1450 he demonstrates the well known opera5on of determining the width of a stream by means of a staff and the similarity of triangles. (Spencer, 1956, pp. 114 f.)

18

B. Smith

No painter can paint well who does not know geometry.4 The observed scene, the scene that is visible and that is to be represented by the artist, is made of finite surfaces out there in the world. The painter's job is to find the appropriate shapes, sizes and positions for the counterparts of these surfaces within the microcosm of the painting in such a way as to constitute an istoria. The totality of surfaces in the macrocosm exists objectively, though it changes from moment to moment with changes in the ambient light. (It is as if the sun, by a sort of divine fiat, makes a selection of which surfaces shall belong to this totality from moment to moment.) In addition to this global selection, however, each observer effects his alun local selection from this totality in such a way as to yield a framed arrangement of observed surfaces of the sort which we see when we look through an open window. The a m y which results through this local selection is dependent upon the observer's position and on the scope and direction of his gaze. Moreover, some surfaces in the observed array are foreshortened because they have parts which are obstructed for this observer or are such as to fall outside the scope of what the artist will choose to represent. In this way there is created out of an in principle infinite totalitp a selection of a sort that can be comprehended by a finite mind.

4 Qualitative Geometry For all of this, however, the results of this act of selection are, because they fall within the first kind of matter, still something entirely objective: they belong to the world of space and light out there. Compare the on~~logical status of the events which take place on the stage in a theater. Certainly the latter constitute a play only because of the way they are perceived and understood (and separated off by fiat from the events around them); but they exist nonetheless objectively, as movements of bodies and props. These movements are however subject to a further series of effects because of the ways the spectators in the theater react towards them. They find one movement threatening, another welcoming, and so on. And so also in our present case: the objective array of surfaces is subjected, when viewed by an observer, to effects of an analogous sort. Some surfaces will appear to be larger or of a different color or shape, some figures will dominate, others will recede into the background. The artist's job, according to Alberti, is to project the objective array of surfaces into the microcosm of the painting in such a way as to achieve a maximally beneficial (moral) effect. The buildings, too, in which the painting is to be displayed, should likewise be designed on the basis of a combination of geometrical and moral principles, and the same applies also (Alberti was a pioneer of urban planning) to the city in which these buildings are to be arranged. (Westfall 1974) Alberti is sometimes described as the first universal genius, and his work, whether on painting, on architecture, on town planning, or on the morality of the family, always transcends the p u . l y theoretical sphere. This is no less true in the domain of mathematics, where Alberti was the first to present the geometrical principles of linear perspective. For e v y here his concerns point always in the direction of practical implications. As he himself expresses it: mathematicians examine the form of things as separated from their matter. Those, however, who wish the object to be seen 'will use a more sensate wisdom'. (1435-36, p. 42) Alberti's interest is accordingly not in form separated fiom matter, but rather in form as it is visible, which means: in the matter that is located in space and that is affected by ambient light. He thus develops a version of Euclid's geometry not in terms of abstractions but in terms of concrete visible 'signs' or 'marks' (recall that the term used by Euclid himself for what we call 'point' is 'semeion' or 'sign'): The first thing to know is that a point is a sign [signum] which one might say is not divisible into parts. I call a sign anything which exists on a surface so that it is visible to the eye. ... Points joined together continuously in a row constitute a line. So for us a line will be a sign whose length can be divided into parts, Alberti 1435-36, p. 89. Compare Leonardo's Non mi legga chi non e matematico. ('Let no one read me who is not a mathematician.')

True Grid

19

but it will be so slender in width that it cannot be split ... If many lines are joined closely together like threads in a cloth, they will create a surface. (Alberti 1435-36, p. 42)

It is in the same vein that Albexti proposes for the outer edgc by which a surface is bounded the terminology of 'brim' [oral or 'fringe' vmbria], terms connoting the edge of a piece of cloth or garment. In a separate tract entitled De punctis et lineis apudpictures Alberti writes: 'Points and lines among painters are not as among mathematicians, [who think that] in a line there fall infinite points.' (Edgerton 1975, p. 8 1) Alberti hereby anticipates contemporary work on so-called qualitative geometries (Bennett et al., 2000), which means: geometries based, not on abstract mathematical points, but rather on finite regions. Both his theory of perspective and his theory of the organization of marks or signs to form an istoria are formulated in qualitative-geometricalterms.

5 Rays of Marvelous Subtlety The surfaces in the objective array and their qualities of color, shape and size are, Alberti tells us, 'measured with sight'. What he means by this he explains by referring to 'the maxims of the philosophers' for whom there are rays that serve the sight 'which cany the form of the thing seen to the sense.' These visual rays, which are depicted in Figure 2 as extending between the single, fixed eye and the array of surfaces seen in the background, constitute what we have called a 'visual pyramid'. They are such that 'by a certain marvelous subtlety' they penetrate the air and 'all thin and clear objects' until they strike against something dense and opaque, where they strike with a point and adhere to the mark they make. Among the ancients there was no little dispute whether these rays come from the eye or the plane. This dispute is very difficult and is quite useless for us. It will not be considered. We can imagine those rays to be like the finest hairs of the head, or like a bundle, tightly bound within the eye where the sense of sight has its seat. The rays, gathered together within the eye, are like a stalk; the eye is like a bud which extends its shoots rapidly and in a straight line on the plane opposite. (Alberti 1435-36, pp. 44 f.)

Alberti's reference in this passage to 'the ancients' relates to the disputes among philosophers between so-called intromissionist and exfromissionist views of visual perception. For the intromissionists, vision is to be explained in terms of light passing fiom the object and into the eye. For the extromissionists, vision is an active process involving 'visual rays', which move fiom the eye and out into the world of surfaces beyond. When Euclid, in his Optics, demonstrates theorems about visual angles, then it is in terms of an extromissi~nisttheory of visual rays that these theorems are formulated. (The validity of the laws of geometrical optics work is not affected by the direction of the visual rays.) For Euclid visual rays are homogeneous. For Ptolemy, on the other hand, another extromissionist, the centermost visual ray, which flows directly fiom the eye and strikes at right angles the surface of what is seen, is privileged as contrasted with 'median' rays on the tiinges of the cone of rays emanating fiom the eye. When Galen isolated the eye's crystalline lens as the seat of visual power, he sees the lens, still in extromissionist terms, not as a receiver but rather as a transmitter of visual force. Why, we might reasonably ask, did Euclid and Ptolemy and Galen, and many other prominent thinkers of the ancient world, defend what must nowadays seem 90 counterintuitive a view of visual perception? One reason was the supposed power of cats and other nocturnal animals to see in the dark. The primary argument for extromissionism however turned on the atomism embraced by many ancient thinkers. The extromissionists pointed out that it would be impossible that every point on a large visual surface should be transmitted simultaneously to a single point via atoms of light. The 'effluxes of things so large as, say, a camel or a'mountain could not very well pass through the tiny pupil of the eye'. (Edgerton 1975, p. 67) It was the Arab thinker Alhazen who, by solving this 'large efflux' problem, established the viability of intromissionist optics by showing how refraction can filter out excess information in

20

B. Smith

the light. Alhazen showed how it was in fact possible even in atomistic terms for every point on the surface of an object seen in nature 'to convey its form to the seat of vision within the eye - in an exact one-for-one, place-for-place proportionate way.' (Edgerton 1975, p. 74) Alhazen's theory of refraction explained also the privileged status of rays close to the axis of sight: they travel unrefiacted through to the optic nerve. In his commentary on Aristotle's De sensu, Albertus Magnus distinguishes four positions on this dispute: a. extromission of visual rays (for example Empedocles); b. intromission of atoms (corporeal images: Democritus); c. intromission of forms (spiritual images: Aristotle); d. simultaneous extromission and intromission of rays (Plato); Extromissionism lives on in the thinking of Augustine and Anselm, but the success of the intromissionist theory is given a powerful boost when it is taken up by three thirteenth-century English scholars: Robert Grosseteste, Roger Bacon and John Pecharn, who saw in the new optical theories of the transmission of light the model of how God spreads the light of grace to his subjects in the world. As Veltman points out (1999), however, Leonardo could still defend a combined extromissionist-intromissionist view (and in presenting the arguments for an extromissionist component in the visual process Leonardo refers to the power which maidens have in their eyes 'to attract the love of men.') Even by the time of Kepler the debate was still not conclusively settled. Indeed, extromissionism still lives on today. And this is so even in spite of all subsequent developments in our understanding of the physics of light and of the physiology of the eye. It lives on in contemporary philosophy and cognitive science in the context of discussions of what is And it is in these terms, I suggest, that the visual rays of Alberti nowadays called 'intenti~nality'.~ - and perhaps even of Euclid6- are to be understood. 'Intentionality' is the term employed especially in the tradition of Brentano, Husserl and Ingarden to refer to the directedness of the mind towards its objects. Husserlians sometimes speak of the arrow of intentionality, and Husserl himself (1970) uses the terminology of 'mental rays' for example when he distinguishes between single-rayed and many-rayed acts of perception. All cognitive directedness towards objects, for Husserl, depends on perceptual intentionality and all perceptual intentionality depends on action. (Mulligan 1995) Language, including maps and diagrams, Husserl sees as a vehicle by which intentional directedness is leveraged beyond the realm of objects given in direct perception. Gibson's ecological psychology, too, can be understood in these terms. It represents a mixed intromissionist-extromissionistview, according to which each organism, in each given context, is tuned to certain specific types of invariants within its surrounding ocean of energy. The organism picks up the information available in the environment that is relevant to its actions in a spontane-

Compare also the extromissionist theory of vision propounded in Pylyshyn (1989)' which presents a view of the visual system as employing a limited number of visual indexes that go out into the world and adhere to whatever it is that the visual system wants to interrogate. These indexes are hypothesized to allow access to the objects that they individuate, and because they are sticky, they are able to track objects and to be updated automatically as the object moves about in the environment. On this reading, the visual rays which Euclid conceives as projecting from the eye are not to be conceived in physical terms at all. For the theory of perspective belongs not to physical but rather to geometric optics, which is what results when we adopt simplifLing assumptions to the effect that the wavelength of light is zero and that rays propagate through homogeneously refractive media along straight lines. Euclid's visual rays would then be analogous to the abstractly conceived (fiat) lines of his own geometry, rather than to rays of light in the proper, physical sense - or indeed to X-rays, or to any other physical manifestations of 'marvelous subtlety'.

True Grid

21

ous process. which involves, not inference or other cognitive processes, but rather (in our terminology) something like a transparent grid into the cells of which the affordances of the environment exactly fit. The views presented here can now be seen as a generalization of the Husserlian and Gibsonian accounts of organism-environment interactions to apply also to the projection onto reality that is involved in our uses of maps, pictures, databases, catalogues and taxonomies of various sorts. When we use a proper name to refer to an object, then a relation of projection comes into existence: the name projects out towards the object, be it present or absent, in virtue of an intentional ray. When we use a photograph to refer to an object, then a relation of projection likewise comes into existence: the photograph projects out towards the object in virtue of a whole pattern of intentional rays. And similarly, when using maps or spreadsheets we employ labelled grids to project in multi-rayed fashion onto corresponding objects in reality.

Fig. 3 The Periodic Table of the Chemical Elements

6 How to Tell the Truth with Maps A good map casts a transparent net over the surface of the earth in just the way Alberti's reticolato casts a transparent net over some portion of objective reality. As the painter's grid casts into relief a certain visual scene, so the grid of the map casts into relief a certain spatial region. There is a deep-seated analogy here; but it is an analogy that has nothing to do with perspective - for it obtains even in relation to maps and plans of strictly two-dimensional planar arrays. It has to do, rather, with the highly general concept of a transparent grid and with the associated highly general notion ofprojection, both of them notions which (as Figure 3 makes clear) can be applied even to types of organization which are entirely non-spatial. (Bittner and Smith, in this volume.) To see what all of these cases have in common we need to boil Alberti's reticolato down to its essential elements, which we can list as follows: 1. the eye (or point of projection), 2. projective rays, 3. the artist's grid, 4. the constituent cells of the artist's grid, 5. the totality of objective visible surfaces, 6. the target grid: the artist's grid as projected onto the objective visible surfaces, 7. the constituent cells of the target grid

22

B. Smith

Extending from the point of projection (1 .), projective rays (2.) bring about a one-one correspondence between the two arrays of cells (4. and 7.), within the artist's grid (3.) and the target grid (6.), respectively. We shall call a structure of this sort a true grid. A true grid is transparent to the corresponding objects in reality. Almost all our customary maps are true grids in the seme defined. In this case the term for the eye or station-point (1.) is replaced by that of the user of the map. The projective rays (2.) are replaced by relations of rigid designation (to be discussed below). The counterpart of the artist's grid (3.) is constituted not just by the rectilinear grid of the map but more generally by whatever is the pattern of contour and border lines and cartographic icons (4.) to be found on the map. The counterpart of (5.) is the corresponding portion of the earth's surface, and the counterparts of (6.) and (7.) are the results of projecting the grid of the map onto this more or less planar region.

Fig. 4 Cartographic Projection

As in the reticolato, so also here we need to distinguish in the ontological structure of the map between two distinct grids. On the one hand is the grid of the map itself, which is in the simplest case a system of regular or irregular cells, each cell enjoying a certain intrinsic position within the grid and thus also standing in certain determinate relations to its neighboring cells. On the other hand is an isomorphic grid on the side of the target portion of the surface of the earth (in Figure 4 a grid of English counties). Regular cells in such grids will standardly be identified by their coordinates within the grid itself; irregular cells will standardly be assigned proper or common noun labels such as 'Berkshire' or 'lateral geniculate nucleus'. In the case of the reticolato, the projective rays can be made to point in whichever direction the user might desire. In the case of a map, on the other hand, the projective rays tie the cells of the map rigidly to corresponding portions of reality. This means that when a user buys a map, he buys not any simple piece of paper but rather a complex cognitive device from out of which, as soon as he begins to use the map, there will project invisible arrows --mys of marvelous subtlety - which tie its constituent cells rigidly to corresponding features on the ground.

True Grid

23

7 Semantic Projection Such lines of projection are at the basis, too, of the so-called picture theory of meaning defended by Wittgenstein in the Tractatus: The pictorial relationship consists of the assignments of the picture's elements to the things. These correlations are, as it were, the feelers of the picture3 elements, with which the picture touches reality. (2.15 14 E, italics added)

A true proposition, for Wittgenstein, is a picture or map of a state of affairs in reality. It is a propositional sign in its projective relation to the world. (3.12) Each (atomic) proposition in the Tractarian framework consists of simple signs (names), which stand in a projection relation to corresponding simple objects. If the proposition is true, then these simple signs stand to each other in the propositional picture as the corresponding objects stand to each other in the world. Here the counterpart of the artist's grid, in Alberti's terminology, is the propositional sign, a complex of names arranged in a certain order, the names serving as the equivalent of the constituent cells. The counterpart of the target grid is a state of affairs in the world. It is a complex question how far Wittgenstein's picture theory of meaning can be extended to language in general. Where it does unproblematically apply is in relation to simple lists, which constitute true grids in the sense here intended -provided only that the items listed do indeed exist in reality. Even a single name, for example 'Mama', constitutes a true grid under the obvious ('Mama' - Mama) projective relation. Moreover, as Figure 3 once more reveals, a system of concepts, too, can form a true grid in the sense defined. The idea of a projective relation fiom concepts to corresponding categories on the side of target objects in reality is at work in the following passage fiom Millikan: The membership of the category 'cat,' like that of 'Mama,' is a natural unit in nature, to which the concept cat does something like pointing, and continues to point despite large changes in the properties the thinker represents the unit as having. ... The difficulty is to cash in the metaphor of 'pointing' in thii contexf. (Millikan 1998)

The generalized reticolato and the associated notion of projection can, 1 suggest, help to cash in Millikan's pointing metaphor in precisely the way required. The generalization from Alberti's reticolato to maps is in one sense simple. Both involve grids which are recognizable as such; both involve relations between spatial neighborhoods which can easily be defined in topological terms; both types of grid can also be subject to the same types of geometrical transformations. It is at first sight difficult to see how we can generalize beyond these sorts of cases to talk of semantic or conceptual grids. In light of recent advances in mereotopology and in the study of so-called 'conceptual neighborhoods' or 'continuity networks', and also in light of the work on granular partitions outlined in the paper by Bittner and Smith (in this volume), we can more readily understand what such generalized grids involve. They all share in common in the ideal case - the presence of a domain (the user's grid) and a co-domain (the target grid), with systems of mereotopological relations defined on each, and with a notion of correspondence or mapping connecting the two of a sort which - in the case of a true grid - preserves mereotopology. But not all grids are true. For while the examples dealt with so far have involved an isomorphism between cells in the grid and corresponding objects, grids may fall short of such perfection by involving some mismatch between user's grid and target grid. Such a mismatch can come about either because the projective relation is not well-defined (cells in the grid are putatively projected onto objects where there are no such objects) or because the cells of the grid do not stand to each other in relations isomorphic to the relations between the corresponding target objects. Even grids which satisfy both of these requirements may still fall short of the sort of perfection that is manifested in the examples of optical, cartographic and conceptual projection referred to above. The grids of the latter satisfy a requirement to the effect that the cells within the target grid

24

B. Smith

fit exactly to the corresponding cells of the user's grid. This condition can in various ways be weakened. Bittner and Stell (1998) offer an approach to spatial grids otherwise similar to the one advanced here but within which the restriction on cell-object fit is relaxed through the notion-of 'rough' location. Smith and Brogaard (2001,200 1a) show how the theory of true grids can be used to deveiop a new version of the supervaluationist theory of vagueness. Their work turns on the idea that grids are always such as to involve a certain coarse-grainedness, which implies that their cells trace over parts or features of reality which fa11 beneath a certain size. This in turn means that the latter can vary while the user's grid - which represents our cognitive access to the relevant objects - remains the same. The phenomenon of vagueness, from this perspective, is just the other side of the coin fiom the phenomenon of granularity. It arises because of the possibility of a variation that falls, in a given context, beneath the t h r e s h d of salience.

8 Directions of Fit Each true grid - be it optical, cartographic, or conceptual - effects a tiling of the portion of reality towards which it is directed. In some cases, as in the case of a gridded map or an Albertian grill, this imputed tiIing - a system of fiat boundaries in reality to which the grid directly corresponds is of no intrinsic significance. Even a purely arbitrary imp~tedtiling may, however, acquire significance if its fiat cells are put to specific practical purposes by colonial administrators or postal authorities. (Smith 2001) In some cases, however, the grid of a map reflects prior independently existing boundaries on the side of the objects themselves. This is so, for example, of the irregular grid depicted in Figure 4 above, which reflects not only the fiat division of England into counties but also the bona fide division between England and the sea, which (partially) surrounds it. In yet other cases - this is so above all in the case of cadastral maps - the grid of a map stands to its target grid in a symbiotic relationship. As the objects change (because the fiat boundaries of land parcels are redrawn or re-measured, or because the land itself has been subject to erosion), so corresponding changes are made in the grid of the cadastre; and as administratively motivated changes in the grid of the cadastre are effected, so this may bring about changes in the objects (land parcels) on the ground. We can thus distinguish, for true maps, three different sorts of cases: 1. the target grid depends exclusively on the grid of the map (a map-to-world direction of fit) 2. map grid and target grid are mutually dependent upon each other (as in the case of a cadastre) 3. the grid on the map reflects a pre-existing grid in reality (a world-to-map direction of fit). 3. can be divided into fbrther sub-cases, according to whether the grid of the map reflects 3a. bona fide boundaries on the side of the target objects themselves 3b. pre-existing fiat boundaries on the side of the target objects 3c. some combination of bona fide and pre-existing fiat boundaries. The same family of cases can be distinguished also in the domain of conceptual projection. Corresponding to 1. is the case where the distinctions in the world are mere reflections of our concepts (for example when the baseball coach divides up his team by assigning positions to his players at the beginning of the game). An example under 2., the symbiotic case, might be the set of prize categories used by dog shows, which both reflects the divisions on the side of the sample domain and a1so;over time, may itself bring about adjustments to these divisions. Corresponding to 3., finally, is the case where concepts reflect pre-existing distinctions among objects in reality, whether bona fide (3a.), for example the distinctions between the six different sorts of quark; or fiat (3b.), for example distinctions between different tax brackets; or mixed (3c.), for example the distinctions among bird species or among languages and dialects.

True Grid

25

9 Windowless Monads Epistemological skepticism is a view to the effect that conceptual classifications of type 3a. are forever beyond our reach. Such epistemological skepticism goes hand in hend with the views of many artists in recent times, who have been pleased to ignore perspective geometry, just as they have ignored Alberti's four principles of dignita, vurieta, modestiu and verisimilitude. From at least the time of Duchamp, the visual arts have, been freed from their connection to everyday life (and to beauty and harmony) and they have been rtcontextualized in the museum. The h c t i o n of painting, if it has one, is not at all that of providing a window on the world, but rather that of drawing attention to itself. A painting is no longer conceived as a transparent device enabling the perceiver to grasp the reality beyond. Rather; it is an object in its own right, and the viewer is called upon to relish its materiality and its quality of opaquenCss. Talk of a 'correct' perspectival representation, with its implication to the effect that there is some single detached master point of view, has at the same time come to be disparaged as a remnant of outmoded phallologocentric thinking. How can one or another method of painting be 'true' or 'correct', when there is no single notion of reality against which its results could be matched? As Henri Lefebvre puts it in his Production of @ace: The fact is that around 1910 a certain space was shattered. It was a space of common sense, of knowledge (savoir), of social practice, or political power ... a space, too, of classical perspective and geometry, developed from the Renaissance onwards on the basis of the Greek tradition (Euclid, logic) and bodied forth in Western art and philosophy, as in the form of the city and town. (1974, p. 25)

There is a simple argument fiom the realist side against all such nonsense. It is the same argument which can be used against all attempts to see reality as somehow dependent upon people's beliefs, and against all attempts to identi9 scientific truths with mere conventions of time or culture. In our present case the argument would run as follows: if perspective geometry is not inherent in the world - a structure waiting to be discovered - but rather a convention, which had to be invented, then this has the consequence that Renaissance men were living in a drfferent world from the world of their ,medieval predecessors. It is this consequence which makes the skeptical and antirealist positions seem so glamorous and exciting. But it is evidently a consequence no less absurd than the thesis that, with the dawning of the realization that the earth is round, the earth itself acquired a new geometry. To this, characteristically, the defender of the anti-realist view will respond that of course he is not wishing to be taken literally when he says that Renaissance men were living in a different world, or that they were 'producing' or 'shattering' a certain space. Such remarks are, he will say, mere metaphors. But then surely the interesting questions pertain, not to what metaphors have been fashionable at different points in human history, but rather to what the true structure of reality is - and this is a question which makes sense only from the realist perspective. As the physiologist M. H. Pirenne (1952) shows, even granting the simplifying assumptions of geometrical optics, penpective paintings correspond to the way we see the world around us with a very high degree of approximation. The best explanation of this correspondence lies in the thesis that the mathematical forms captured in the geometry of perspective are - modulo certain wellunderstood and in normal circumstances negligible simplifications - out there in the world, waiting for us to apprehend them through abstraction. They thus serve as truthmakers for the theory of perspective; and as Pirenne nicely puts it, the strange fascination which perspective had for the Renaissance mind was precisely 'the fascination of truth.' Certainly our understanding of perspective has developed over time. For whatever reason, it took a long time before people were ready to perform the abstraction of these mathematical forms. We can hazard that part of this reason turns on the need, before this step could be taken, for a certain detachment !+om the world of objects through the cultivation of the standpoint of the neutral, scientific observer, a standpoint which Renaissance thinkers, like some of their Greek predecessors, enjoyed, but which medieval thinkers lacked. Renaissance thinkers such as Alberti were able to grasp the world as an abstract, mathematical 'container' - as a stage upon which men move, and

26

B. Smith

have their exits and entrances. It is indeed in the Renaissance that the theatrical audience is for the first time separated fiom and forced to adopt a particular point of view (or as we might also say, a particular perspective) in relation to the qectacle on the stage.

10 Fit Happens There is nothing subjective in Alberti's reticolato. As Pirenne makes clear, the geometrical relztionship between an object and its projection on the picture plane obtains quite independently of whether there is an eye at the vanishing point. As the technology of laser-guided missiles reveals, the laws of perspective hold independently of the existence of subjects, observers, artists or cultures: they are laws governing the way light, space and the surfaces of objects are related together. The laws of perspective are laws of geometrical optics; they have nothing to do with neurology or psychology. Correspondingly, the picture drawn in perspective aims not at representing anything like the retinal image or any pattern of nervous stimulation on the side of an observer. Rather it aims to send to the eye the same distribution of light as that which the object itself would send. This corresponds to the theory of picture perception sketched by the great theorist of realism J. J. Gibson and encapsulated by his student Kennedy (1974) in the form of a definition of a picture as: 'a surface treated so that it yields light to a particular station point, usually on a normal to the picture surface, which could have come fiom a scene in the real world.' (Compare Gibsan 1978.) Gibson naturally recognizes that there are other sorts of pictures (including maps), some of which involve conventional elements (symbols, icons), which have nothing to do with the conveyance of light to the eye in a way which simulates the light that is projected fiom surfaces in threedimensional space. Even these pictures can, however, be interpreted in realist fashion on the basis of the general theory of projection sketched above. There are of course also many cases of pictorial images in which perspectival or other features of the represented scene are distorted in one or other way. As the Gibsonian realist can insist, however, the fact that pictures are sometimes made, for whatever reason, in such a way as to embody such distortions does not imply that all pictures are lacking in the sort of transparency for which the followers of Alberti strove. A11 maps must be of a certain scale or combination of scales, just as every grid must have a certain resolution or granularity of cells. And since reality itself (as Gibson 1979 emphasizes) contains entities accessible at many different scales, it follows that no single grid can be complete. Rather, as scientific practice shows, we need grids of many different resolutions if we are to do justice to reality in its many aspects. This implies, as the enemies of realism are fond of pointing out, that there is no 'God's eye perspective' or 'view from nowhere'. This does not, however, mean that we are justified in drawing the conclusion that every single one of the myriad perspectives which we have at o w disposal embodies a false view of reality. The inference fiom partiality to falsehood might indeed be valid, but only in a world without windows - a world in which no single one of our grids enjoys the condition of transparency. The fact that there are maps which deviate, for whatever reason, from the strictly veridical representation of reality does not take away fiom the fact that - leaving aside any small errors which may have been made in the application of the relevant projection systcm - aimost all maps are true of the corresponding portion of reality. This applies to Mercator's map, and it even applies to Saul Steinberg's View of the Worldfrom Ninth Avenue. Maps must of course embody some projection system in representing three dimensions on a planar surface. Yet those who see in this an argument to the effect that all maps must necessarily involve some form of systematic distortion are simply revealing their own misunderstanding of the nature of projection. They are like those who, on noticing that the Circle Line is represented on maps of the London Underground as a yellow band, complain of 'distortion' because yellow, rather than some other color, has been used.

True Grid

27

Acknowledgements

Support from the American Philosophicd Society, and from the NSF (Research Grant BCS-

9975557: "Geographic Categories: An Ontological Investigation") is gratefully acknowledged.

References Alberti, Leon Battista ca. 1435-1436 De pictura praestantissima, original (Latin) edition: Basel 1540 (reprinted Portland, Oregon 1972), Italian translation: Della pittura, Venice 1547. Cited according to the English translation: On Painting, 1956 (complete text available at http:Nwww.noteaccess.com). Bennett, B., Cohn, A. G., Tomni, P. and Hazarika, S. M. 2000 "A Foundation for Region-Based Qualitative Geometry", Proceedings of ECAI 2000, Berlin, 204-208. Bittner, Thomas and Stell, John G. 1998 "A Boundary-Sensitive Approach to Qualitative Location", Annals of Mathematics and Artificial Intelligence, 24,93-114. Bittner, Thomas and Smith, Barry (in this volume) "A Taxonomy of Granular Partitions". Collingwood, R G. 1938 The Principles of Art, Oxford: Oxford University Press. Edgerton, Samuel Y. 1975 The RenaissanceRebirth of Linear Perspective, New York: Basic Books. Gibson, James J. 1978 "The Ecological Approach to Visual Perception in Pictures", Leonardo, 11:3, 227235. Gibson, James J. 1979 The EcologicalApproach to Visual Perception, Boston: Houghton-Mifflin. Gibson, James J. 1980 "A Prefatory Essay on the Perception of Surfaces versus the Perception of Markings on a Surface", in M. Hagen (ed.), The Perception of Pictures, Volume I: Alberti's Window, New York: Academic Press, xi-xvii. Gibson, James J. 1982 Reasonsfor Realism. Selected Essays of James J. Gibson, Edward Reed and Rebecca Jones (eds.). Hillsdale, NJ and London: Lawrence Erlbaum. Husserl, Edrnund 1970 Logical Investigations, London: Routledge and Kegan Paul, 1970. Kennedy, John Miller 1974 A Psychology of Picture Perception: Images and Information, San Francisco: Jossey Bass. Lefevbre, Henri 1974 La Production de L 'Espace, Paris: Editions Anthropos. Millikan, Ruth Garrett 1998 "A common structure for concepts of individuals, stuffs, and real kinds: More Mama, more milk, and more mouse", Behavioral and Brain Sciences, 9: 1, 55-100. Mulligan, Kevin 1995 "Perception" in B. Smith and D. W. Smith, eds., The Cambridge Companion to Husserl, Cambridge and New York: Cambridge University Press, 168-238. Panofsky, Erwin 1927 "Die Perspektive als 'symbolische Form"', Vortrage der Bibliothek Warburg, 258330. English translation: Perspective as Symbolic Form, New York: Zone Books, 1991. Pirenne, M. H. 1952 "The Scientific Basis for Leonardo Da Vinci's Theory of Perspective", British Journal for the PhilosophyofScience, 3: 10, 169-18s. Pylyshyn, Zenon W. 1989 "The Role of Location Indexes in Spatial Perception: A Sketch of the FINST Spatial-Index Model", Cognition, 32, 65-97. Smith, Barry, 2001 "Fiat Objects", Topoi, 20: 2. Smith, Barry and Brogaard, Bent 2001 "Quantum Mereotopology", Annals of Mathematics and Artificial Intelligence (in press). Smith, Barry and Brogaard, Bent 2001a "A Unified Theory of Truth and Reference", Loqique et Analyse, 43 (in press). Spencer, John R 1956 "Introduction" to the English translation of Alberti 1435-36, 1-3 1. Veltman, Kim H. 1977 Review of Edgerton 1975, The Art Bulletin, 59: 2,281-282. Veltman, Kim H. 1986 Linear Perspective and the Visual Dimensions of Science and Art (Leonardo da Vinci Studies I), Munich: Deutscher Kunstverlag, 1986. Veltman, Kim H. 1999 Continuity and Discovery in Optics and Astronomy (Leonardo da Vinci Studies II), http://www.surnscorp.com. See also http://www.mmi.unimaas.nl. Westfall, Carroll William 1974 In This Most Perfct Paradbe: Alberti. Nicholas V, and the Invention of Conscious Urban Planning in Rome, 1447-55, University Park: Pennsylvania State University Press. Wittgenstein, Ludwig 1961 Tractatus Logico-Philosophicus, with English translation by D. F. Pears and B. F. McGuinness, London: Routledge and Kegan Paul.

A Taxonomy of Granular Partitions Thomas Bittner and Barry Smith Qualitative Reasoning Group, Department of Computer Science, Northwestern University, [email protected] Department of Philosophy, Center for Cognitive Science and NCGIA, State University of New York, Buffalo, [email protected]

Abstract: In this paper we propose a formal theory of granular partitions (ways of dividing up or sorting or mapping reality) and we show how the theory can be applied in the geospatial domain. We characterize granular partitions at two levels: as systems of cells, and in terms of their projective relation to reality. We lay down conditions of well-formedness for granular partitions, and we define what it means for partitions to project transparently onto reality in such a way as to be structure-preserving. We continue by classifying granular partitions along three axes, according to: (a) the degree to which a partition represents the mereological structure of the domain it is projected onto; (b) the degree of completeness and exhaustiveness with which a partition represents reality; and (c) the degree of redundancy in the partition structure. This classification is used to characterize three types of granular partitions that play an important role in spatial information science: cadastral partitions, categorical coverages, and the partitions involved in folk categorizations of the geospatial domain.

1 Introduction Imagine that you are (a) a geologist classifying soil samples or (b) a spatial analyst classifying the raster pixels of a digital image or (c) a hotel manager making a list of the guests in your hotel on a certain night. In each of these cases you are employing a certain grid of cells, and you are recognizing certain objects as being located in those cells. In case (a) the cells are labeled, for example, ‘clay’ or ‘sand’ and the objects you are recognizing as located in these cells are your soil samples. In case (b) the cells are labeled with the names of vegetation classes, each class being made to correspond to a particular spectrum of frequencies in the pixel array, and the objects that are located within those cells are raster cells within the partition which is the pixel image. In case (c) the cells correspond to the rooms in your hotel, the objects are the individuals or groups who are, according to the hotel register, assigned to these rooms on any given night. We shall call a grid of cells of the type used in these examples a granular partition, and we shall argue that granular partitions are involved in all listing, sorting, cataloguing and mapping activities. Granular partitions are ways of structuring reality in order to make it more easily graspable by cognitive subjects such as ourselves. Some partitions are flat: they amount to nothing more than a mere list (case c). Other partitions are hierarchical: they consist of cells and subcells, the latter being contained within the former. Some D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 28−43, 2001.  Springer-Verlag Berlin Heidelberg 2001

A Taxonomy of Granular Partitions

29

partitions are built in order to reflect independently existing divisions on the side of objects in the world (the subdivision of the animal kingdom into species and subspecies, the subdivision of heavenly bodies into galaxies, stars, planets, moons, etc.). Other partitions – for example the partitions created by electoral redistricting commissions – are themselves such as to create the necessary divisions on the side of their objects, and sometimes they create those very objects themselves. Some partitions involve the imposition of a layer of quasi-discreteness upon an underlying reality which in itself has the structure of a continuum. In Smith and Brogaard (2000) the notion of granular partition was introduced as a generalization of David Lewis’s (1991) conception of classes as the mereological sums of their singletons. Given its set-theoretical roots, our basic formal ontology of granular partitions will have two parts: (A) a theory of the relations between cells and the partitions in which they are housed, and (B) a theory of the relations between cells and objects in reality. The counterpart of (A) in a set-theoretic context would be the study of the relations among subsets of a single set; the counterpart of (B) would be the study of the relations between sets and their members. Division into units, counting and parceling out, listing, sorting, pigeonholing and cataloguing are activities performed by human beings in their traffic with the world. Granular partitions are the cognitive devices designed and built by human beings to fulfill these various purposes. As will be clear from what follows, the notion of granular partition that is hereby implied is only distantly related to the more familiar notion of a partition defined in terms of equivalence classes. The paper is structured as follows. We start with a discussion of properties of granular partitions as systems of cells in the sense of theory (A). We then consider granular partitions in their projective relation to reality in the sense of theory (B). This provides us with the tools to define what it means to say that a granular partition projects onto reality in a transparent and structure-preserving way. We then provide a classification of granular partition by characterizing various properties of the correspondence between partition and reality, and we go on from there to discuss relationships between set theory, mereology, and the theory of granular partitions as alternative tools for the purposes of formal ontology. We conclude by considering three classes of partitions that have an important role to play in the geographic domain.

2 Granular partitions as system of cells 2.1 Cells and subcells All granular partitions involve cells arranged together in some sort of structure. This structure is intrinsic to the partition itself, and obtains independently of whether there are objects located in its cells. Cells in granular partitions may be nested one inside another in the way in which species are nested within genera in standard biological taxonomies. Theory (A) studies properties granular partitions have in virtue of the relations between and the operations which can be performed upon the cells from out of which they are built. We say that one cell, z1, is a subcell of another cell, z2, if the first is contained in the latter (‘Cell’ is ‘Zelle’ in German). We write z1 ⊆ z2 in order to designate this relationship, and we postulate as an axiom or master condition:

30

T. Bittner and B. Smith

MA1: The subcell relation ⊆ is reflexive, transitive, and antisymmetric. Every granular partition A (‘partition’ is ‘Aufteilung’ in German) has a maximal cell defined as: DMax: Max(z1, A) ≡ Z(z1, A) and ∀z: Z(z, A) → z ⊆ z1 where ‘Z(z, A)’ means that z is a cell in the partition A. (In what follows the condition Z(z, A) will be omitted in cases where it is clear that we are talking about cells within some fixed partition A. In addition, initial universal quantifiers will be taken as understood.) We now demand that MA2: ∃z: Max(z, A) which ensures that every granular partition has a maximal cell in the sense of DMax. From the antisymmetry of the subcell relation it follows that this cell is unique. This root cell, denoted r(A), is such that all the cells in the partition are included in it as subcells. The nestedness of cells inside a partition yields chains of cells satisfying z1 ⊇ z2 ⊇… ⊇ zn. We shall call the cells at the ends of such chains minimal cells, and define: DMin: Min(z1, A) ≡ Z(z1, A) and ∀z: Z(z, A) → (z ⊆ z1 → z = z1) Another important aspect of granular partitions is then: MA3: Each cell in a partition is connected to the root by a finite chain. MA3 leaves open the issue as to whether granular partitions themselves are finite; thus it does not rule out the possibility that a given cell within a partition might have infinitely many immediate subcells. 2.2 Partition-theoretic sum and product of cells Every pair of distinct cells in a partition stand to each other within the partition either in the subcell relation or in the relation of disjointness. In other words: MA4: Two cells overlap only if one is a subcell of the other. Or in symbols: ∃z: (z = z1 ∩ z2) → z1 ⊆ z2 or z1 ⊃ z2. From MA3 and MA4 we can prove by a simple reductio that the chain connecting each cell of a partition to the root is unique. Following Smith (1991) we can define the partition-theoretic sum and product of cells within granular partitions as follows. The partition-theoretic sum z = z1 ∪ z2 of two cells in a partition is the ⊆-minimal cell satisfying z1 ⊆ z and z2 ⊆ z. The partition-theoretic product, z = z1 ∩ z2, of two cells is defined only if z1 and z2 are not mereologically disjoint. If it is defined, then it yields the largest subcell shared in common by z1 and z2. 2.3 Trees Philosophers since Aristotle have recognized that the results of our sorting and classifying activities can be represented as those sorts of branching structures which mathematicians nowadays called trees. Trees are rooted directed graphs without

A Taxonomy of Granular Partitions

31

cycles (Wilson and Watkins 1990). Every finite partition can be represented very simply as a rooted tree in such a way that the cells in the partition correspond to vertices in the tree and vertices are connected by an edge if and only if the corresponding cells stand to each other in an immediate subcell relation. We can represent a partition not only as a tree but also as a simple sort of Venn diagram. In a Venn diagram partition cells are represented as topologically simple and regular regions of the plane. Our partitions are Venn diagrams within which regions do not intersect. In the remainder we will often think of partitions as such planar maps – they are Venn diagrams without overlapping – and the minimal cells correspond to the smallest regions within such diagrams.

3 Granular partitions in their projective relation to reality 3.1 Projection Granular partitions are more than just systems of cells. They are built to serve as pictures or maps of reality. Granular partitions are systems of cells that project onto reality in something like the way in which a bank of flashlights projects onto reality when it carves out cones of light in the darkness. In some cases the cells of a partition project but there are no objects for them to project onto. (Consider the partition cataloguing Aztec gods.) Here, however, we are interested primarily in granular partitions which do not project out into thin air in this way. We write ‘P(z, o)’ as an abbreviation for: cell z is projected onto object o. In what follows we shall assume that a unique projection is defined for each granular partition. For a more general discussion see (Bittner and Smith 2001). The theory of granular partitions allows us to employ a very general reading of the term ‘object’. An object in the partition-theoretic sense is everything onto which some cell of a partition can project: an individual, a part of an individual, a group or class of individuals (for example a biological species), a spatial region, a political unit (county, polling district, nation), or even (for present purposes) the universe as a whole. Objects can be either of the bona fide or of the fiat sort (Smith 1995). Bona fide objects exist independently of human partitioning activity. They are, simply, recognized (highlighted) by partition cells. Fiat objects are objects created by our human partitioning activity. Hence it may be that the corresponding partition cells not only recognize their fiat objects but that the latter are in fact created through the very projection of partition cells onto the corresponding portion of reality. Examples are the States of Wyoming and Montana. For an extended discussion of the relationships between granular partitions and fiat objects see (Bittner and Smith 2001). 3.2 Location When projection succeeds then the corresponding granular partition represents the corresponding portion of reality transparently and in such a way that mereological structure is preserved. We write ‘L(o, z)’ as an abbreviation for: object o is located at cell z. When projection succeeds, then location is what results. Projection and location thus correspond to the two ‘directions of fit’ – from mind to world and from world to

32

T. Bittner and B. Smith

mind – between an assertion and the corresponding truthmaking portion of reality. (Searle 1983, Smith 1999) Location presupposes projection: an object is never located in a cell unless through the projection relation associated with the relevant partition. Thus MB1 L(o, z) → P(z, o). In the case where no errors have been made in the construction and the projection of a granular partition, L(o, z) holds if and only if P(z, o). This is because, in such a case, if a partition projects a given cell onto a given object, then that object is indeed located in the corresponding cell. MB2 P(z, o) → L(o, z). Very many granular partitions – from automobile component catalogues to our maps of states and nations – have this quality without further ado, and it is such granular partitions upon which we shall concentrate in what follows. Such granular partitions are transparent to the corresponding portion of reality. In this case projection and location are converse relations with respect to the partition in question. Formally we write: DTr: Tr(A) ≡ ∀z∀o: PA(z, o) ↔ LA(o, z). MB1 and MB2 jointly ensure that objects are actually located at the cells that project onto them. Notice however that a transparent partition, according to our definition, may still have empty cells. (Think of the Periodic Table, which leaves empty cells for chemical elements of types which have yet to be detected.) MB1 and MB2 tell us only that, if a cell in a partition projects upon some object, then that object is indeed located in the corresponding cell. They do not tell us what happens in case a cell fails to project onto anything at all. An object o is recognized by a cell z if and only if z is projected onto o and the object o is actually located at z. A partition recognizes a given object if and only if it has a cell that recognizes that object (Smith and Brogaard 2001). We shall sometimes use the term ‘recognition’ as a synonym for ‘transparent projection’ in what follows.

4 Functionality constraints 4.1 Projection is functional: The confused schoolboy The notion of transparency is still very weak. Thus it is consistent with ambiguity on the side of the cells in relation to the objects they target, that is with the case where one cell projects onto two distinct objects. Consider the partition created by a lazy schoolboy studying the history of the Civil War in England, which has just one cell labeled ‘Cromwell’. Thus it does not distinguish between Oliver, the Lord Protector, and his son Richard. Or consider the partition utilized by those who talk of ‘China’ as if the Republic of China and the People’s Republic of China were one object. To eliminate such ambiguity we lay down a requirement to the effect that each partition must be such that its associated projection is a functional relation: MB3: P(z, o1) and P(z, o2) → o1 = o2 For granular partitions satisfying MB3, cells are projected onto single objects (one rather than two).

A Taxonomy of Granular Partitions

33

4.2 Location is functional: The Morning Star and the Evening Star Consider a partition having root cell labeled ‘heavenly bodies’ and three subcells labeled: ‘The Morning Star’, ‘The Evening Star’, and ‘Venus’, respectively. As we know, all three subcells project onto the same object. This partition is clearly somewhat barren; but it is still perfectly consistent with the conditions we have laid out thus far. Its distinct subcells truly, though unknowingly, project onto the same object. It is not unusual that we give different names (or coordinates, or class-labels) to objects in cases where we do not know that they are actually the same. A good partition, though, should clearly be one in which such errors are avoided. Granular partitions manifesting the desired degree of correspondence to objects in this respect must accordingly be ones in which location, too, is a functional relation: MB4: L(o, z1) and L(o, z2) → z1 = z2 In granular partitions that satisfy MB4, location is a function, i.e., objects are located at single cells (one rather than two). The location function is however partial, since partitions are not omniscient. As MB3 rules out co-location (overcrowding), so MB4 rules out co-projection (redundancy).

5. Structural mapping MB1 and MB2 are, even when taken together with MB3 and MB4, still very weak. They thus represent only a first step along the way towards an account of correspondence to reality for granular partitions. Such correspondence will involve two further dimensions: of structural mapping, and of completeness. In the present section we address our attention to the topic of structural mapping. 5.1 Recognizing mereological structure Each granular partition reflects the basic part-whole structure of reality through the fact that its cells are themselves such as to stand in the relation of part to whole. This means that, given the master conditions expressed within the framework of theory (A) above, granular partitions have at least the potential to reflect the mereological structure of the relevant domain. And in felicitous cases this potential is realized. We say that the cells z1 and z2 reflect the mereological relationship between the objects onto which they are projected if and only if the following holds: DS1: RS(z1, z2) ≡ L(o1, z1) and L(o2, z2) and z1 ⊂ z2 → o1 < o2 If z1 is a proper subcell of z2 then any object recognized by z1 must be a proper part of any object recognized by z2. A partition reflects the mereological structure of the domain it is projected onto if and only if each pair of cells satisfies DS1: DS2: RS(A) ≡ ∀z1,z2: (Z(z1, A) and Z(z2, A)) → RS(z1, z2) We then impose a new master condition: MB5: All granular partitions are structure reflecting in the sense of DS2. Note that even MB5 is still very weak. Its effect is in a sense entirely negative: it merely ensures that granular partitions do not misrepresent the mereological relationships between their objects. But granular partitions might still be blind to (trace over) such relationships. Minimal cells might project onto objects which stand to each other in any one of the entire range of possible mereological relations

34

T. Bittner and B. Smith

(parthood, proper parthood, disjointness, and overlap). Pairs of cells z1 and z2 which do not stand to each other in the subcell relation are likewise neutral as to the mereological relations between their objects. This means that the corresponding partition does not know (or does not care) how o1 and o2 are related, which means that we are entitled to infer nothing at all about the mereological relations among the corresponding objects. Consider, for example, a partition that contains cells that recognize John and his arm, i.e., L(John, z1) and L(John’s arm, z2). Cell z1 need not be a proper subcell of the cell z2, because granular partitions may trace over mereological relationships between the objects they recognize. MB3 is however still strong enough to ensure that, if a partition tells us something about the mereological relationships on the side of the objects which it recognizes, then what it tells us is true. 5.2 The domain of a partition That upon which a partition is projected, its domain, is a certain mereological sum of objects in reality. It is, as it were, the total mass of stuff upon which the partition sets to work: thus it is stuff conceived as it is prior to any of the divisions or demarcations effected by the partition itself. The domains of granular partitions might comprehend not only individual objects and their constituents (atoms, molecules, limbs, organs), but also groups or populations of individuals (for example biological species and genera, battalions and divisions, archipelagos and diasporas) and their constituent parts or members. Granular partitions can be used to impose a division into discrete units upon continuous domains, for example through temperature or frequency bands. We shall see that maps of land use or soil type are another important family of granular partitions in the sense here advanced. Formally we define the domain of a partition simply as the object onto which its root cell is projected: DD D(A) = p(r(A)) MB1–5 already ensure (a) that everything that is located at some cell of the partition is part of what is located at the corresponding root cell; and (b) that for each partition there can be only one such object. We now demand that every partition has a non-empty domain: ∃x: x = D(A) MB6 We then say that a partition represents its domain correctly if and only if MA1–4 and MB1–6 hold. 5.3 Granularity A granular partition is granular in virtue of the fact that it can recognize an object without recognizing all its parts. The theory of granular partitions can thus provide the basis for understanding the selective focus of our maps and classifications and above all their ability to trace over parts below a certain level. To impose a partition on a given domain of reality is to foreground certain objects and features in that domain and to trace over others. Partition cells always project onto wholes. If a partition recognizes not only wholes but also one or more parts of such wholes, then this is because there are additional cells in the partition which do this recognizing job. Consider, for

A Taxonomy of Granular Partitions

35

example, a partition that recognizes human beings, i.e., it has cells that project onto John, Mary, and so forth. This partition does not recognize parts of human beings – such as John’s arm or the molecules in Mary’s shoulder – unless we add extra cells for this purpose. And even if a partition recognizes wholes and their parts, then as we saw above it is not necessary that it also reflects the mereological relationships between the two. The theory of granular partitions inherits from mereology the feature that it is consistent with both an axiom to the effect that atoms exist and with the negation of this axiom. The theory thus enables us to remain neutral as to the existence of any ultimate simples in reality from out of which other objects would be constructed via summation. This is due to the fact that granular partitions are by definition top-down structures. The duality with trees puts special emphasis on this aspect: we trace down from the root until we reach a leaf. A leaf need not necessarily be an atom in the sense that it projects upon something in reality which has no further parts. The fact that there are leaves simply indicates that our partition does not care about what, on the side of reality, lies beneath a certain level of granularity. An object located at a minimal cell is an atom only relative to the partition which we bring to bear.

6 Varieties of granular partitions In this section we discuss some of the more fundamental varieties of those granular partitions which satisfy the master conditions (MA1-4 and MB1-6) given above. We classify them according to: (1) degree of structural fit; (2) degree of completeness and exhaustiveness of projection; and (3) degree of redundancy. 6.1 Structural constraints We required of granular partitions that they reflect the mereological structure of the domain they recognize. Remember that such reflection is to be understood in such a way that it leaves room for the possibility that a partition is merely neutral about (traces over) some aspects of the mereological structure of its target domain. Taking this into account, we can order granular partitions according to the degree to which they do indeed succeed in representing the mereological structure on the side of the objects onto which they are projected. At the one extreme we have (1): granular partitions that completely reflect the mereological relations holding between the objects they recognize. At the other extreme are (2): granular partitions that completely trace over the mereological structure of the objects they recognize (except to the degree that they recognize them as part of the domain in question). Between these two extremes we have granular partitions that reflect some but not all of the mereological structure of the objects they recognize. Under heading (1) are those granular partitions which satisfy the weak converse of MB5, which means that if o1 is part of o2, and if both o1 and o2 are recognized by the partition, then the cell at which o1 is located is a subcell of the cell at which o2 is located. Formally we can express this as follows: CM: L(o1, z1) and L(o2, z2) and o1 < o2 → z1 ⊂ z2 We call granular partitions satisfying CM mereologically monotonic.

36

T. Bittner and B. Smith

6.2 Projective completeness So far we have allowed granular partitions to contain empty cells, i.e., cells that do not project onto any object. We now consider partitions which satisfy the constraint that every cell recognizes some object: CC: Z(z, A) → ∃o: L(o, z) We say that granular partitions that satisfy CC project completely. These partitions are of particular interest since in this case projection is a total function. 6.3 Exhaustiveness There may be objects in our target domain that are not located at any cell. The resulting granular partitions are not very satisfying: governments want all their subjects to be located in some cell of their partition of taxable individuals. They want their partition to satisfy an exhaustiveness constraint to the effect that every object in the pertinent domain is indeed recognized. But what does it mean to say that a partition exhausts its domain? Unfortunately we cannot capture this notion formally by using (*) o ≤ D(A) → ∃z: Z(z, A) and L(o, z), which asserts that if some object o is part of the domain of the partition A then there is a cell z in A that recognizes o. The tax authorities (as of this writing) do not want to tax the separate molecules of their subjects. To formulate an acceptable alternative to (*) will be a difficult matter. In fact we believe that it will be necessary to promote several restricted forms of exhaustiveness, each one of which will approximate in different ways to the (unrealizable) condition of unrestricted exhaustiveness expressed in (*). To see how one such exhaustiveness condition might work in first (schematic) approximation, let us introduce a sortal predicate ϕ that singles out the kinds of objects our taxation partition is supposed to recognize (for example, human beings rather than parts of human beings). We can now demand that the taxation partition recognize all of those objects in its domain which satisfy ϕ: CEϕ

o ≤ D(A) and ϕ(o) → ∃z: Z(z, A) and L(o, z).

Think of CEϕ as asserting the completeness of one partition relative to another, the ϕ-totalizer partition, which consists exclusively of minimal cells in which all and only the objects satisfying ϕ are located. We will discuss examples of other such conditions in section 8. 6.4 Redundancy Granular partitions are natural cognitive devices and the designers and users of such devices build them to serve practical purposes. This means that they will normally strive to avoid certain sorts of redundancy. One sort of redundancy – which we might call correspondence redundancy – is excluded already by condition CC. This consists in the presence of necessarily empty cells (cells whose labels tell us ex ante that no objects can be located within them). But partitions can manifest also what we might call structural redundancy, and this is not quite so trivial. Consider a partition with a cell labeled vertebrates, which

A Taxonomy of Granular Partitions

37

occurs as a subcell of the cell labeled chordates in our standard biological classification of the animal kingdom. Almost all chordates are in fact vertebrates. Suppose (for the sake of argument) that biologists were to discover that all chordates must be vertebrates. Then in order to avoid structural redundancy they would collapse into one cell the two cells of chordates and vertebrates which at present occupy distinct levels within their zoological partitions. A constraint designed to rule out such structural redundancy would be: CR: A cell in a partition never has exactly one immediate descendant.

7 Set theory, mereology, and granular partitions 7.1 Partition theory as an alternative to set theory and mereology The theory of granular partitions is intended to serve, first of all, as an alternative to set theory both as a tool of formal ontology and as a framework for the representation of human common sense. Currently it is the naïve portion of set theory that is used in almost all work on common-sense reasoning and in related investigations of natural language semantics. Kinds, sorts, species are standardly treated as sets of their instances; subkinds as subsets of these sets. Set theory nicely does justice to the granularity that is involved in our sorting and classification of reality by treating objects as elements of sets, i.e. as single whole units within which further parts are not recognized. But set theory also has its problems, not the least of which is that it supports no distinction between natural granular totalities (such as the species cat) and such ad hoc totalities as for example: {the moon, Napoleon, justice}. Set theory also has problems when it comes to dealing with the fact that biological species and similar entities may remain the same even when there is a turnover in their instances. For sets are identical if and only if they have the same members. If we model the species cat as the set of its instances, then this means that cats form a different species every time a new cat is born or dies. If, similarly, we model an organism as the set of its cells, then this means that it becomes a different organism whenever cells are gained or lost. Set theory also has problems when it comes to dealing with relations between objects at different granularities. An organism is a totality of cells, but it is also a totality of molecules, and it is also a totality of atoms. Yet the corresponding sets are distinct, since they have entirely distinct members. More recently, attempts have been made to solve some of these problems by using mereology as a framework for ontological theorizing. Mereology is better able to do justice in realistic fashion to the relations between wholes and their constituent parts at distinct levels of generality. All the above-mentioned totalities (of cells, molecules, atoms) can be recognized, when treated mereologically, as being one and the same. Mereology has one further advantage over set theory as a tool for the sort of middle-level ontological theorizing which the study of common-sense reasoning requires, namely that it does us not require that, in order to quantify over wholes of given sorts, one must first of all explicitly specify all the parts. On the other hand, however, mereology, too, has its problems. Above all it does not have the machinery for coping with the phenomenon of granularity; for if we quantify over wholes in a mereological framework, then we thereby quantify over

38

T. Bittner and B. Smith

all the parts of such wholes, both known and unknown, at all levels of granularity. Mereology can mimic the advantages of set-theory in this respect only if we depart from realism and make the idealizing commitment to atomism. (Galton 1999) Set theory and mereology are then in practice indistinguishable, since each whole becomes isomorphic to a certain set of atoms. The theory of granular partitions presented in this paper is the product of an effort to build a more realistic, and also a more general and flexible, framework embodying some of the strengths of both set theory and mereology while at the same time avoiding their respective weaknesses. At the formal level it assumes standard extensional mereology (Simons 1987) and adds the primitives and axioms of theories (A) and (B). It thereby avoids the disadvantages of the unrestricted partof relation via the intermediate formal machinery of cells, which adds to mereology the features of selectivity and granularity. 7.2 Partition theory and set theory Partition theory, as already noted, is a generalization of set theory understood in Lewis’s sense. At the formal level there are some obvious similarities between sets and granular partitions: (a) the subcell relation and the subset relation are both partial orders (MA1); (b) the minimal chain condition (MA2) is the analogue of the set-theoretic Begründungsaxiom; (c) the existence of a root cell of which all subcells are parts corresponds to the conception of sets as containers; (d) the transparency and functionality of projection and location (MB1-4) reflect analogous features of the element-of relation. At the same time there are a number of important differences between the two frameworks. Above all partition theory is designed to do justice to the fact that not all members of the powerset of a set are of interest in the sorts of natural contexts in which sorting and classifying occur. Partitions are cognitive artifacts. They comprehend only those subcell-cell relations which reflect some sort of natural inclusion relation – for example between a species and its genus – on the side of objects in the world. Some sets then have a structure which precludes them from being even considered as partitions in the sense defended here. Consider, for example, the set {{a, b}, {a, c}}. Since we have {a} ⊆ {a, b} and {a} ⊆ {a, c}, any corresponding partition would violate MA4, the condition designed to exclude double counting.

8 Granular partitions of geographic space Granular partitions are, we repeat, natural cognitive devices. We assume that the primary examples of partitions are transparent and structure reflecting (they satisfy all of the master conditions MA1–4 and MB1–6 above). If we imagine the system of cells of a partition as being ranged over against a system of objects, with all the cells of the partition being occupied by objects (under a certain relation of projection), then in the best case we have a partition that is mereologically monotone (CM) and such as to project completely (CC) and exhaustively (CEϕ) relative to some condition ϕ. Such ideal granular partitions are thereby also free of redundancy (CR). We find examples of such perfection above all in the abstract, fiat domains of databases and spatial subdivisions.

A Taxonomy of Granular Partitions

39

In what follows we discuss cadastral maps, which come close to representing granular partitions which are perfect in the sense defined. We then move on to discuss categorical coverages which fall short of this sort of exact fit between partition and the corresponding objects in reality. Finally we discuss the ‘folk’ categorizations of geographic reality. 8.1 The perfect cadastre The perfect cadastre is what exists in the databases of cadastral authorities. It is what you see when you examine cadastral maps. You see mathematically exact lines that separate land parcels. We are here assuming for the sake of simplicity that the cells on the map project onto corresponding parcels in reality (that the map contains no errors). We assume also that all and only parcels are recognized by the minimal cells of the cadastral partition. Partition cells are represented, for example, by entries in the German Grundbuch or in its computational equivalents. There are very strict rules for inserting, deleting, or changing cells in this partition, by means of which we seek to guarantee that the cadastral partition has the ideal properties set forth above. Land parcels are fiat objects. They are created (in no small part) through the very projection of the cells in the cadastre onto reality itself. This is a geodetic projection of a sort which is described by a small number of axioms. It is mathematically well defined and can even (within certain limits) be computed. This projection imposes fiat boundaries onto reality in the same way that the plotter draws the lines on a cadastral map. The projection (in our partition-theoretic sense) has the following properties. Cadastral partitions are transparent in the sense that cells correctly recognize objects, i.e., P(z, o) ↔ L(o, z). Projection and location are functional relations, i.e., one cell projects onto one land parcel and one parcel is located at one cell. Cadastral partitions are CEϕ-complete, where ϕ selects minimal cells that recognize pieces of land that are parcels. (Defining ϕ is a complicated matter of law, and currently there exist only informal definitions.) The intuition underlying this thesis is that there are no no-mans-lands, which means: no zones within the domain of the cadastral partition that are assigned to no cell within the partition itself. Cadastres satisfy also CC-completeness, in that they do not contain empty cells, i.e., cadastral entries that do not correspond to any piece of land. These properties are (in the cases of interest to us here) ensured by law and by extensive training on the part of those who are charged with the task of maintaining the cadastre. Cadastral partitions may recognize some mereological structure on the side of their objects. For example, a cadastral partition may recognize multi-parcel estates as well as separate single parcels. If a cadastral partition properly recognizes all the pertinent multi-parcel estates then it is mereologically monotone, i.e., CM holds. Cadastral partitions have the property that they recognize, too, some of the mereotopological structure on the side of their objects, in the sense that two cells are adjacent in the cadastre if and only if the corresponding land parcels are neighbors on the ground.

40

T. Bittner and B. Smith

8.2 Categorical coverages Area-class maps (W. Bunge 1966) or categorical coverages (Chrisman 1982) belong to a type of thematic maps that show the relationship of a property or attribute to a specific geographic area. A prototypical example of a categorical coverage is the land use map, in which a taxonomy of land use classes is determined (e.g., residential, commercial, industrial, transportation) and a specific area (zone) is then evaluated along the values of this taxonomy (Volta and Egenhofer 1993). Another prototypical example is soil maps, which are based on a classification of the soil covering the surface of the earth (into clay, silt, sand, etc.). The zones of a categorical coverage are a jointly exhaustive and pair-wise disjoint subdivision of the relevant region of space (Beard 1988). There are in fact two reciprocally dependent granular partitions involved in categorical coverages. On the one hand is the partition of the attribute domain (e.g., of land use or of soil types); we can think of the attribute domain with which we start as a continuum, which is then partitioned into discrete bands in light of our practical purposes, capabilities of measurement, and so forth. On the other hand is the partition of the surface of the earth into corresponding zones. Both of these partitions satisfy all of the master conditions set forth above. The close relationship between the two has been discussed for example by Beard (1988) and Frank et al. (1997). The same reciprocal relationship is illustrated in the way in which every categorical map (a partition of space) stands to its legend (a partition of the attribute domain represented on the map). Consider, first of all, the spatial component of a categorical coverage, which is a partition of some portion of the surface of the earth. Using the notions introduced in the foregoing we are now able to specify four properties of this partition more precisely as follows: First, the partition is complete in the sense that there are no empty cells (CC). Secondly, the minimal cells of the partition exhaust a certain domain (a part of the surface of the earth) in the sense of CEϕ , where ϕ selects topologically simple and maximal regions that are of one or other of the soil types recognized by the partition of the attribute domain. Consequently the root of the partition recognizes the mereological sum of all the regions (zones) recognized by its cells. Thirdly, the correspondence between the cells in the partition of the spatial component of a categorical coverage and the zones it recognizes is one-one and onto. The fact that projection and location are here total, functional and mutually inverse is exploited extensively in the formalization and representation of categorical coverges (e.g. Frank et al. 1997, Bittner and Stell 1998, Erwig and Schneider 1999). Fourthly, as in the case of cadastral maps, spatial partitions recognize the mereotopological structure of their domains in the sense that they are not only mereologically monotone in the sense of CM but also such that two cells in the spatial partition are adjacent if and only if the corresponding parcels are neighbors on the ground. This is the case because the geodetic transformations used to map features on the surface of the Earth onto planar maps preserve topological relations (assuming perfect transformations without error and modulo the feature of limited resolution). This implies that the part-of relation is also preserved by the given mappings. Spatial partitions can be considered as Venn diagrams and hence they can be transformed

A Taxonomy of Granular Partitions

41

into a partition structure where the part-of relation becomes the subcell relation along the lines described above. These properties of their spatial component and the close relationship between the spatial and attribute components of categorical coverages mean that the partition of the pertinent attribute domain also satisfies the following nice constraints: First, it is exhaustive relative to the spatial component. Every minimal cell in the spatial partition (a topologically simple zone of homogeneous coverage) has a corresponding minimal cell in the attribute partition. This immediately follows from the definition of the selection predicate ϕ for minimal cells of the spatial component. Consequently, the partition of the attribute domain exhausts the domain of all cases that actually occur in the region covered by the corresponding spatial partition. For example, if our spatial partition projects onto a desert, then the corresponding partition of soil types needs to be exhaustive for the different types of sand that occur in this area and which we find it important to distinguish, but it does not need to contain a cell labeled ‘clay’. Secondly, projection and location need both to be functional, otherwise the regions carved out on the spatial side would not be jointly exhaustive and pairwise disjoint. Both functions may however be partial as long as they are exhaustive relative to the pertinent spatial component. The location function is partial if there exist soil types that are not recognized by the attribute partition and the projection is partial if there are empty cells in the attribute partition. Partitions of attribute domains are not necessarily limited to partitions consisting only of minimal cells (and one root cell). Consider a partition of the attribute domain Land-Use/Land-Coverage. There might be, for example, a non-minimal cell labeled agricultural in this partition, with subcells labeled cultivated cropland, pasture, livestock, and poultry. Hierarchical partitions of attribute domains are often created by refinement, i.e., we start with a root cell recognizing the attribute domain as a whole and add layers of subcells in such a way that the mereological sum of everything that is recognized by the cells of one layer is recognized also by the root cell. Consider for example a partition of the attribute domain ‘Rainfall in inches’. There might be a layer of cells recognizing values falling within one or other of the three intervals [0, 5], [5, 10], [10, ∞), together with more refined layers recognizing values in: [0, 2.5], [2.5, 5], [5, 7.5], and so forth. Hierarchical partitions of the attribute domain create potentially hierarchical partitions of the spatial domain. Notice that the spatial component of hierarchical categorical coverages is not necessarily non-redundant in the sense of CR. In the spatial component of a hierarchical categorical coverage ‘Land Usage (Chicago)’ there might be one single region that is recognized by both the cells ‘Agricultural’ and ‘Cultivated Cropland’ where the second is a subcell of the first. In this case location is not a function since the region in question is located within both cells. Technically the problem is dealt with by letting only the most specific cell (the one farthest away from the root) project onto the region in question. It is important to see that the regularity of the given partition structures is due to the fact that the objects recognized are fiat objects carved out by the projecting partitions themselves. For example, in the categorical coverage for soil types there are certainly bona fide differences between sand and solid rock, but the distinction between the many soil-types in between are of the fiat sort. They are created by imposing a partition onto the attribute domain ‘soil on the surface of Earth’. (Smith and Mark 1999) This partition, on being projected, then creates as its target a spatial

42

T. Bittner and B. Smith

partition whose cells are separated by spatial fiat boundaries on the ground. The latter demarcate ‘categorical zones’, which are homogeneous at the level of granularity determined by the map. The given boundaries sometimes coincide with bona fide boundaries in reality, but in most cases they do not do so. 8.3 A folk categorization of water bodies We discussed spatial partitions or attribute partitions that induce spatial partitions. That given partitions are characterized by a high degree of structure and order is due not only to the fact that they are spatial subdivisions but also to the fact that there are well defined rules (of scientific methodology or of law) which govern their construction and projection. Granular partitions in general are much less well structured. loch

body of water

lake* narrow ocean pond

tarn reservoir lake* millpond

pool sea * = term appears twice

tank

Figure 1: Ontology of Water Bodies and Related Entities, based on Definitions in the American Heritage Dictionary (taken from Smith and Mark 1999)

Smith and Mark 1999 analyzed the partition of water bodies and related entities which can be extracted from the definitions contained in the American Heritage Dictionary. The graph-theoretic representation of this partition is given in Figure 1. If we analyze this graph, then we can see easily that it is not a tree, since it contains cycles (e.g., pond, tank, reservoir, pond). We also can see that there are two cells labeled ‘lake’. The latter clearly indicates that location is not a function relative to this partition. We hypothesize that there are special features of the definitions we find compiled in existing dictionaries in virtue of which their underlying taxonomies appear to deviate from the tree structure. Guarino and Welty (2000) have shown, however, that such taxonomies can very easily be reconstituted as trees in systematic fashion. This gives us some confidence that the ideas presented above may be of service also in providing a framework for the construction of more coherent taxonomies for use in dictionaries and data standards in the future. Acknowledgements This work was supported in part by DARPA under the Command Post of the Future program and the National Science Foundation under the Research on Learning and Education program. Support from the American Philosophical Society, and from the NSF (Research Grant BCS-9975557: “Geographic Categories: An Ontological Investigation”) is also gratefully acknowledged.

A Taxonomy of Granular Partitions

43

Bibliography Beard, K. 1988 “Multiple representations from a detailed database: A scheme for automated generalization.” Ph.D. thesis, University of Wisconsin, Madison Bittner, T. and Stell, J. G. 1998 “A Boundary-Sensitive Approach to Qualitative Location,” Annals of Mathematics and Artificial Intelligence, 24, 93–114. Bittner, T. and B. Smith 2001 “Vagueness and Granular Partitions.” to appear in C. Welty and B. Smith (eds.), Proceedings of FOIS-2001: The 2nd International Conference on Formal Ontology in Information Systems, Sheridan Press. Bunge, W. 1966 Theoretical Geography. Lund: Gleerup. Casati, R. and Varzi, A.C. 1995 “The Structure of Spatial Location”, Philosophical Studies, 82, 205-239 Casati, R. and Varzi, A. C. 1999 Parts and Places, Cambridge, MA: MIT Press. Chrisman, N. 1982 “Models of Spatial Analysis Based on Error in Categorical Maps.” Ph.D. thesis, University of Bristol, England Erwig, Martin and Schneider, Markus 1999 “The Honeycomb Model of Spatio-Temporal Partitions,” International Workshop on Spatio-Temporal Database Management (Lecture Notes in Computer Science 1678), Berlin: Springer, 39-59. Frank, A., Volta, G., and McGranaghan, M. 1997 “Formalization of families of categorical coverages,” International Journal of Geographic Information Science, 11: 3, 214-231 Galton, A. C. 1999 "The Mereotopology of Discrete Space", in C. Freksa and D. M. Mark (eds.) Spatial Information Theory: Cognitive and Computational Foundations of Geographic Science (Lecture Notes in Computer Science 1661), Berlin/New York: Springer, 251–266. Guarino N. and Welty, C. 2000 “Ontological Analysis of Taxonomic Relationships,” to appear in A. Laender and V. Storey, (eds.), Proceedings of ER-2000: The 19th International Conference on Conceptual Modeling (Lecture Notes in Computer Science), Berlin/New York: Springer-Verlag. Lewis, D. 1991 Parts of Classes, Oxford: Blackwell. Searle, J. R. 1983 Intentionality. An Essay in the Philosophy of Mind, Cambridge: Cambridge University Press. Simons, P. M. 1987 Parts: A Study in Ontology. Oxford: Clarendon Press. Smith, B. 1991 “Relevance, Relatedness and Restricted Set Theory”, in G Schurz and G. J. W. Dorn (eds.), Advances in Scientific Philosophy. Essays in Honour of Paul Weingartner, Amsterdam/Atlanta: Rodopi, 1991, 45–56. Smith, B. 1995 “On Drawing Lines on a Map”, in Andrew U. Frank and Werner Kuhn (eds.), Spatial Information Theory. A Theoretical Basis for GIS (Lecture Notes in Computer Science 988), Berlin/Heidelberg/New York, etc.: Springer, 475–484. Smith, B. 1999 “Truthmaker Realism”, Australasian Journal of Philosophy, 77 (3), 274–291. Smith, B. 2001 “True Grid”, in this volume. Smith, B. and Brogaard, B. 2000 “Quantum Mereotopology,” forthcoming in Annals of Mathematics and Artificial Intelligence. Smith, B. and Brogaard, B. 2001 “A Unified Theory of Truth and Reference,” Logique et Analyse, in press. Smith, B. and Mark, D. M. 1999 “Ontology with Human Subjects Testing: An Empirical Investigation of Geographic Categories,” American Journal of Economics and Sociology, 58: 2, 245–272. Volta, G. and Egenhofer, M. 1993 “Interaction with GIS Attribute Data Based on Categorical Coverages.” in: Frank, A. and Campari, I. (eds.) Conference on Spatial Information Theory, Proceedings. (Lecture Notes in Computer Science, 716). Wilson, R. J. and J. J. Watkins (1990). Graphs – An Introductory Approach. New York, John Willey and Sons, Inc.

A Geometric Theory of Vague Boundaries Based on Supervaluation∗ Lars Kulik Department for Informatics, University of Hamburg, Vogt-Kölln-Str. 30, D-22767 Hamburg, Germany [email protected] fax: +49-40-42883-2385 phone: +49-40-42883-2391

Abstract. The representation of geographical objects with vague or fuzzy boundaries still poses a challenge to current geographical information systems. The paper presents a geometric account to deal with spatial vagueness. This approach is based on ideas of the theory of supervaluation. To capture vague spatial information current geographical information systems mainly employ fuzzy set theory and fuzzy logic. The proposed geometric theory is contrasted with fuzzy theories regarding the representation of vague spatial objects and the inferences that can be drawn about the objects. Opposed to fuzzy theories, the proposed theory does not rely on a numerical representation to model spatial vagueness, but is still compatible with it. Therefore, the approach is able to support spatial databases in qualitative spatial inferences. Keywords. Axiomatics, Geometry, Geography, Spatial Reasoning, Vagueness.

1

Introduction

Nearly every geographic object has a vague boundary (cf. Couclelis, 1996): Mountains, seas, forests, deserts or the downtown of cities do not have precisely defined regions. An object with a vague boundary, called a ‘vague object’, does not allow a definite decision which spatial region belongs to the geographic object. Depending on the context and other factors there are different interpretations which region represents the geographic object. The localization of a boundary might be unknown as a result of incomplete information, different boundaries occur due to different evaluations of features of the geographic objects, the boundaries are time-dependent, the boundaries constitute continuous transitions, and so forth. Hadzilacos (1996) gives an overview concerning the different types of vague or indeterminate boundaries. There are only few exceptions of geographic objects, which have uniquely defined boundaries. The exceptions are, in general, defined regions like administrative zones, ∗

The research reported in this paper was supported by the Deutsche Forschungsgemeinschaft (DFG) in the project ‘Axiomatics of Spatial Concepts’ (Ha 1237-7). I am in particular indebted to Carola Eschenbach, Markus Guhe, Christopher Habel, and Inga Mau for their valuable comments.

D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 44−59, 2001.  Springer-Verlag Berlin Heidelberg 2001

A Geometric Theory of Vague Boundaries Based on Supervaluation

45

nature reserves, states, or man-made objects like streets or channels. Smith & Varzi (1997) call their boundaries fiat boundaries. They are sharp boundaries that separate an object from its surrounding and enable a clear distinction what belongs to the object and what does not. Due to the important role of vague objects, their classification (Couclelis, 1996), modeling and formal characterization plays an increasingly important role for geographical information systems and spatial databases (Erwig & Schneider, 1997; Worboys, 1998). Fuzzy methods are considered as the primary tools within geography (cf., for instance, Burrough, 1996; Fisher, 2000) as well as spatial databases (Schneider, 1999) for dealing with spatial vagueness. There is a controversial debate, however, to what extent fuzzy logic (Zadeh, 1975) should be employed to model reasoning about vagueness (cf. Elkan, 1994). There are only few alternative approaches available that are not based on fuzzy set theory to represent vague objects. These approaches, for instance the accounts of Cohn and Gotts (1996a) or Clementini and di Felice (1996), are considered to be qualitative approaches. Both accounts focus on topological or mereological relations and extend well-known theories that represent sharply bounded entities. Cohn and Gotts modify a mereological approach, the RCC-5 theory (Randell, Cui and Cohn, 1992), whereas Clementini and di Felice take up the 9-intersection model of Egenhofer and Herring (1991). However, neither the approach of Cohn and Gotts nor the one of Clementini and di Felice has been developed to cope with specific challenges of spatial vagueness like gradual transitions of two vague objects given for instance by a transition of a forest and a meadow. The presented work focuses on vague objects with gradual boundaries. In the case of a gradual boundary different spatial areas and points belong to the spatial extension of a vague object to different degrees. It is not possible to provide a unique criterion whether a point of a gradual boundary belongs to the object or not. Hence, there are no abrupt changes and the gradual transitions between two objects can be considered as blend-in constellations (Hadzilacos, 1996). Examples are the transition between a desert and a prairie, or the boundary of the northern part of Canada. To capture such smooth transitions we propose a theory of spatial vagueness that is based on ideas of the theory of supervaluation. Since the theory of supervaluation does not rely on numbers, it seamlessly fits into qualitative approaches. An overview of qualitative spatial reasoning theories is given by Cohn (1997). Many tasks in spatial reasoning about sharp and vague geographic objects involve qualitative knowledge like topology (Egenhofer & Herring, 1991) and ordering geometry for cardinal directions (Kulik & Klippel, 1999). Sharma, Flewelling & Egenhofer (1994) point out the advantages of qualitative methods in geographical reasoning and describe a qualitative spatial reasoner. Qualitative methods are used in particular if exact data is not known or principally not available like for the boundaries of ancient realms. They are very robust since they are not liable to numerical rounding errors. There are at least two perspectives in what way objects are considered as spatially vague: under one perspective the objects themselves are vague (Tye, 1994) whereas under another perspective the concepts or representations of the objects are vague (cf.,

46

L. Kulik

for example, Varzi, 2000). We consider vagueness as semantic vagueness since semantic vagueness is compatible with almost every theory about vagueness. Spatial vagueness of gradual boundaries under a semantic perspective corresponds to the fact that there are several regions of an object, which can be associated to the object in different degrees. The remaining part of Section 1 gives a short overview of vague boundaries in spatial databases, and outlines the theory of supervaluation as a semantic conception of vagueness. Section 2 provides a formalization of vague regions and shows how to represent the relative membership of points to gradual boundaries of vague objects without a numerical representation. Section 3 indicates in the first part how to reason about vague regions without numerical concepts. Still, a lot of data in geographical information systems relies on a numerical representation. The second part of Section 3 shows how to draw inferences involving different (numerical) degrees of truth in a supervaluational way, which has some essential advantages compared to fuzzy logic.

1.1

Vague Boundaries in Geographic Information Systems

Queries in spatial databases enable listings of object properties and support comparisons between different objects. A typical example is an information system for estate agents. On the one hand it basically lists the properties of a house like its price or its size. But, to support customers in their preferences it also has to be able to compare several houses regarding their locations, prizes, and so forth. An ordered list of houses according to preferences of a customer is in many instances sufficient as a result of such a query. The list certainly could contain numerical values, but the actual absolute values—apart from their order of magnitude—might not be very informative. Thus, a characterization of vague objects is desirable that does not require the use of a numerical representation but is still compatible with it. The treatment of queries about possible locations of a house near a city or a forest, or about the sphere of influence of a dumping site requires a representation of the gradual boundary of these objects. A lot of queries in geographic information systems involve boundaries of geographic objects. We list some queries that hold for sharply bounded objects as well as for vaguely bounded objects. Do two objects have a common (gradual) transition? For this purpose we have to check whether the (gradual) boundaries of both objects are co-localized. In the case of sharp regions, that means whether there are points or, more generally, lines that belong to both limits. The case of vague regions is considered in Section 2.1. Given a sharp boundary or a gradual boundary: which is the corresponding uniquely determined geographic object? Given a geographic object: what are all adjacent objects? In the last case it has to be tested, which sections of the entire boundary are frontiers of other regions.

1.2

The Classical Theory of Supervaluation

According to the theory of supervaluation (cf. Fine, 1975; Kamp, 1975) vagueness results from a semantic indecision: a vague predicate distinguishes entities to which it

A Geometric Theory of Vague Boundaries Based on Supervaluation

47

definitely applies and entities to which it definitely does not apply. Hence, a predicate like ‘forest’ singles out spatial regions or locations that are undeniably part of the forest from regions that are unquestionably not part of the forest. However, there might be still some remaining regions that cannot be clearly assigned to one of the two groups. Theories of supervaluation model this fact by assuming that there is not a single interpretation of a vague predicate but several equally good ones. Some interpretations consider the regions as part of the forest and others do not take them as part of the forest. All the regions that definitely belong to the forest constitute the positive extension of the forest, all regions that definitely do not belong to the forest are the negative extension, and the remaining ones represent the penumbra. Every single interpretation which assigns a meaning to a predicate like ‘forest’ is called admissible if it makes the predicate true in the positive extension, false in the negative extension, and either true or false in the penumbra. Hence, in a single interpretation every region of the penumbra either counts as belonging to the forest or as not being part of the forest (see Fig. 3b). It follows that every admissible interpretation subdivides the underlying space into two regions: one region that represents the spatial extension of the forest and another region that does not belong to the forest. Every admissible interpretation is precise and accordingly called a precisification.1 Each statement is either true or false on a given interpretation. The corresponding assignment of a truth-value to the statement is called a valuation. There is no reason to distinguish an interpretation as the right one. Accordingly, all interpretations are considered. The assignment of truth-values for all interpretations is called a supervaluation. A statement is supertrue (superfalse) if it is true (false) for all admissible interpretations. It is a remarkable feature that the technique of supervaluation maintains the law of excluded middle and the law of non-contradiction: given a statement p the formula p ∨ ¬p is supertrue whereas the formula p ∧ ¬p is superfalse even if p is based on a vague predicate. If there are interpretations for which the statement is true, and others for which it is false, then the classical theory of supervaluation assigns no truth-value at all.

1.3

Global and Local Reasoning about Spatial Vagueness

The presented approach takes the view that the reasoning techniques of supervaluation and fuzzy logic can be classified as local and global reasoning, respectively. Supervaluation uses the technique of local reasoning, which means that the conclusions are drawn in every precisification and then the vagueness is represented by quantifying over all precisifications. Therefore, the theory of supervaluation is able to maintain the law of non-contradiction and the law of excluded middle. Fuzzy logic just pursues the opposite strategy, the technique of global reasoning. First, it represents the spatial vagueness by assigning a numerical value to express the degree of membership to all interpretations and then reasons about the entirety of interpretations via the numbers. Tautologies for a proposition p like |p ∧ p|=0 and |p ∨ p|=1 do not 1

The term ‘precisification’ has become a standard term within the theory of supervaluation (cf., for instance, Fine, 1975; Keefe & Smith, 1996).

48

L. Kulik

hold in general anymore. The theory of supervaluation and fuzzy logic lead to different results regarding the inferences about vagueness. Chapter 3 elaborates the details.

2

A Geometric Theory of Vague Regions

The formal description of the spatial properties of vague regions with gradual boundaries is based on the axiomatic method. An axiomatic system does not define basic terms like ‘vague region’ or ‘gradual boundary’ but constitutes a system of constraints which determines the properties of the basic terms and specifies their relations. The characterization of vague regions with gradual boundaries is based on the idea to describe a vague region by sharp regions, which have uniquely determined spatial extensions. Therefore, the axiomatization of vague regions presupposes a corresponding characterization of sharp regions, which is omitted to focus on the specific features of spatial vagueness. The geometric description at first relates regions with sharp boundaries, called sharp regions, to vague regions, and subsequently introduces an ordering structure which compares two sharp regions of a vague region. The vague regions denote the spatial extensions of geographic objects and the points represent positions of point-like objects. We distinguish two types of vague boundaries: gradual boundaries and gradual transitions. If we focus on one spatially vague object, then its boundary is called a gradual boundary, whereas the boundary between two spatially vague objects with gradual boundaries is called a gradual transition. The basic geometric entities are points, sharp regions, vague regions and gradual boundaries. The capitals P, P', Q, and R denote points, R, R', R1, ... denote sharp regions, V, V', V1, ... denote vague regions, and B and B' denote gradual boundaries. The axiomatic characterization models the properties of the geometric entities by axioms. The entities are related by two relations, an incidence (ι) relation and an ordering relation (≺).

2.1

Axiomatic Characterization of Vague Regions

In agreement with the theory of supervaluation a vague region is specified by its sharp regions. The relation that relates sharp and vague regions is the incidence relation. A sharp region is incident with a vague region if it represents one admissible way to make the vague region precise and to stipulate its extension. A vague region is uniquely determined by the sharp regions that are incident with it (VI1)2 just as a sharp region is uniquely determined by the points that are incident with it (RI1). According to axiom (VI1) a vague region is not specified by its spatial extension and the points being incident with it. On the contrary, there can be different vague regions including the same points if they have different sharp regions. (RI1) ∀R R' 2

[∀P [P ι R ⇔ P ι R'] ⇒ R=R']

The implementation of the axiom (VI1) in a spatial database can be realized by sets. A vague region then is a set of sharp regions.

A Geometric Theory of Vague Boundaries Based on Supervaluation

(VI1) ∀V V'

49

[∀R [R ι V ⇔ R ι V'] ⇒ V=V']

To enable a comparison of the different sharp regions of a vague region, we introduce an ordering relation. A similar relation, although with different properties, has been introduced by Cohn & Gotts (1996b). The relation ≺ has to fulfill some requirements to take sharp regions as precisifications (see Section 1.2). If R and R' are two regions for a vague region V that fulfill the relation ≺(V, R, R' ), then the region R is called more restrictive than the region R' regarding the vague region V. In this case both sharp regions are incident with the vague region and the region R is a part of the region R'. A region R is a part of another region R' (symbolized as ) if all points of the region R are also points of the region R'. Hence, we obtain the following definitions: R  R'

⇔def ∀P [P ι R ⇒ P ι R']

≺(V, R, R' )

⇔def R ι V ∧ R' ι V ∧ R  R'

Corresponding to the positive and negative extension of a spatial predicate, the geometric notions of the core and the hull of a vague region V are introduced. They are symbolized as core(V) and hull(V), respectively. The core of a vague region is the most restrictive region of all regions that are incident with the vague region. Therefore, the core is contained in all sharp regions of a vague region. The core of a vague region of a geographic object corresponds to the positive extension denoted by the predicate referring to the object. The hull of a vague region on the other hand contains every sharp region of a vague region. Hence, the hull is the least restrictive region of a vague region. The negative extension of the spatial predicate is the result of all points that do not belong to the hull of the corresponding vague region. R=core(V)

⇔def R ι V ∧ ∀R' [R' ι V ⇒ ≺(V, R, R' )]

R=hull(V)

⇔def R ι V ∧ ∀R' [R' ι V ⇒ ≺(V, R', R)]

From the definition of the relation  immediately follows its transitivity. Accordingly, the relation ≺ is also transitive for a fixed vague region (TV1). Since the relation  is antisymmetric for a given vague region the antisymmetry also holds for the relation ≺ (TV2). (TV1) ∀V R R' R''

[≺(V, R, R' ) ∧ ≺(V, R', R'' ) ⇒ ≺(V, R, R'' )]

(TV2) ∀V R R'

[≺(V, R, R' ) ∧ ≺(V, R', R) ⇒ R=R']

A vague region requires more than a single sharp region for its description. The axiom (VB1) guarantees that there are at least two different sharp regions representing the vague region, the core and the hull of a vague region. This constraint ensures two aspects: every vague region has at least a core and a hull, and the minimal representation of vague regions consists of a pair of sharp regions. Cohn & Gotts (1996b) explicitly deny the condition that every vague region requires a core region and assume the opposite. However, this would lead to the fact that it is not possible to distinguish a single region for a geographic object that belongs undeniable to its spatial extension. The axiom (VB2) states that of two regions being incident with a

50

L. Kulik

vague region, one is more restrictive than the other. Hence, two regions can always be compared regarding the relation ≺. (VB1) ∀V ∃R R'

[R−R' ∧ R=core(V) ∧ R'=hull(V)]

(VB2) ∀V R R'

[R ι V ∧ R' ι V ⇒ ≺(V, R, R' ) ∨ ≺(V, R', R)]

The axioms (VB1) and (VB2) combined with the theorems (TV1) and (TV2) describe a geometric ordering structure. The axiom (VB1) guarantees that a vague region is at least described by two sharp regions. If there are no other regions as the core and the hull, we obtain the egg-yolk model of Cohn & Gotts (1996a). To describe gradual boundaries we need some prerequisites. A point is incident with a vague region V (or a gradual boundary B), if there is a sharp region of the vague region (gradual boundary) that contains the point. A region is the sum of two regions (denoted by ) if it consists of exactly the points of the two regions. Two regions are disjoint (denoted by dj) if they do not share a point. PιV

⇔def ∃R [P ι R ∧ R ι V]

R = R1  R2 ⇔def dj(R1, R2)

∀P [P ι R ⇔ (P ι R1 ∨ P ι R2)]

⇔def ∀P [P ι R1 ⇒ ¬(P ι R2)]

Vague regions and gradual boundaries condition each other. For every vague region V there is a gradual boundary such that for every sharp region RV of V there is a sharp region of the gradual boundary disjoint from the core of V that adds up with the core to RV (GB1). Conversely, for every gradual boundary B there is a vague region so that if a sharp region RB belongs to the gradual boundary then it is disjoint from

the core of the vague region and there is a sharp region of the vague region that can be decomposed into the core and the region RB (GB2). A gradual boundary is uniquely determined by its sharp regions (GB3), parallel to axiom (VI1). (GB1) ∀V ∃B ∀RV [RV ι V ⇔ ∃RB [RB ι B ∧ dj(RB, core(V)) ∧ (core(V)  RB=RV)]] (GB2) ∀B ∃V ∀RB

[RB ι B ⇔ dj(RB, core(V)) ∧ ∃RV [RV ι V ∧ (RV=core(V)  RB)]]

(GB3) ∀B B'

[∀R [R ι B ⇔ R ι B'] ⇒ B=B']

The axiom (GB3) enables us to define the uniquely determined gradual boundary (symbolized as gbd) of a vague region. B=gbd(V)

⇔def ∀RB [RB ι B ⇔ dj(RB, core(V)) ∧ ∃RV [RV ι V ∧ (RV=core(V)  RB)]]

The axioms (GB1) and (GB2) imply the decomposition principle (TV3): the spatial extension of a vague region can be decomposed into the spatial extension of the gradual boundary and the core of the vague region. (TV3) ∀V P

[P ι V ⇔ P ι core(V) ∨ P ι gbd(V)]

In addition to the gradual boundary of one vague region, we consider the gradual transition of two vague regions. The points of a gradual transition can be counted to

A Geometric Theory of Vague Boundaries Based on Supervaluation

51

both vague regions although in different degrees: the more one point belongs to one region, the less it belongs to the remaining region. The most characteristic case of a gradual transition is given by two regions where one region completely blends into another region. This type of gradual transition is called strong gradual transition. A weaker variation of a gradual transition of two regions is given if their gradual boundaries partially overlap. In general, gradual transitions of vague regions are not vague regions since they have uniquely defined extensions.3 They are also not sharp regions because of their internal structure. In the following we distinguish three types of gradual transitions of two vague regions, the weak gradual transition (see Fig. 1), the strong gradual transition, and the (path-)connected strong gradual transition.

Fig. 1. The figure shows three different ways for two vague regions to overlap resulting in three different gradual transitions of the vague regions. The left figure illustrates a weak gradual transition, the middle figure a strong one, and the right figure a strong one that is connected. The gray and the dashed lines depict the boundaries of the sharp regions of the vague regions

Two vague regions have a weak gradual transition if their gradual boundaries overlap.4 A weak gradual transition (denoted by wgt) is modeled as a binary relation of two vague regions V and V'. The overlapping region that consists of all the points that are incident with the gradual boundaries of V and V' is uniquely determined. This region is called the extension of the weak gradual transition (denoted by ewgt). wgt(V, V' )

⇔def ∃R ∀P [P ι R ⇒ P ι gbd(V) ∧ P ι gbd(V' )]

ewgt(V, V' )=R ⇔def wgt(V, V') ∧ ∀P [P ι R ⇔ P ι gbd(V) ∧ P ι gbd(V' )]

The notion of a strong gradual transition (denoted by sgt) corresponds to the notion of the penumbra of supervaluation. Two vague regions have a strong gradual transition if for every sharp region R of one of the vague regions there is a sharp region R' of the remaining vague region which is in external contact with R. If two sharp regions R and R' are in external contact we write ect(R, R' ). The regions are in external contact if parts of the boundaries of both regions coincide and the regions do not share a point5. The condition can be rephrased in a more suggestive way: in every interpretation of the strong gradual transition by two sharp regions of the vague regions, which are in external contact, every point of the transition belongs to one of the sharp regions. This condition ensures that a gradual transition is a “blend-in constellation” in the sense of Hadzilacos (1996). 3

4

5

This is only true if we consider vagueness as first-order vagueness. Details of higher-order vagueness can be found in Keefe and Smith (1997). It is possible to relax the characterization of a weak gradual transition even more by requiring that two vague regions have a weak gradual transition if their hulls overlap. Details on the contact of two regions and its implications can be found in Varzi (1997).

52

L. Kulik

⇔def ∀R [R ι V ⇒ ∃R' [R' ι V' ∧ ect(R, R' )]] ∧

sgt(V, V' )

∀R' [R' ι V' ⇒ ∃R [R ι V ∧ ect(R, R' )]] The (maximal) spatial extension R of a strong gradual transition of two vague regions V and V' (denoted by esgt) can be described in the following way: every point P lies in R if and only if for every sharp region R1 of one of the vague regions there is a sharp region R2 of the remaining vague region in external contact with R1 such that the point P lies in exactly one of the two sharp regions. esgt(V, V' )=R ⇔def sgt(V, V' ) ∧ ∀P [P ι R ⇔

∀R1 [R1 ι V ⇒ ∃R2 [R2 ι V' ∧ ect(R1, R2) ∧ (P ι R1 ∨ P ι R2)]] ∧ ∀R2 [R2 ι V' ⇒ ∃R1 [R1 ι V ∧ ect(R1, R2) ∧ (P ι R1 ∨ P ι R2)]]] The notion of a strong gradual transition can be tightened up. The description of a strong gradual transition still allows the case where its spatial extension R is disconnected. A formal characterization of a connected strong gradual transition could be carried out in at least two ways. The first alternative just uses the topological concept of connectedness whereas the second alternative relies on a geometric characterization of path-connectedness. The second alternative is still based on ordering geometry but requires a new class of entities, the curves, the introduction of a betweenness structure on curves (cf. Eschenbach, Habel & Kulik, 1999), and the characterization of the connectedness of curves. If a query in a spatial database requires the evaluation whether two vague objects have a gradual transition, the system first has to determine whether the hulls of the vague regions are not disjoint. If they are disjoint, there is no gradual transition of any type between the two vague regions. Otherwise, the system has to check if the additional conditions of a weak or a strong gradual transition are fulfilled.

2.2

Relative Membership of Points to Vague Regions

In Section 1.1 we motivated the advantages for geographical information systems of a relative description of the membership of locations to vague regions. The characterization of vague regions by sharp regions allows the comparison of points regarding their degree of membership to vague regions. A point P belongs more to a vague region V than another point P' (symbolized as ≺(V, P, P' )), if the point P lies in every sharp region in which also the point P' lies, and if there exists a sharp region of V including P but not P'. Two points belong to the same degree to a vague region V (symbolized as (V, P, P' )), if the points cannot be distinguished regarding the sharp regions: the point P is included in exactly all the regions, in which the point P' is included. A point P belongs less to a vague region V than a point P' (symbolized as (V, P, P' )) if the point P' belongs more to the vague region than the point P. ≺(V, P, P' )

⇔def ∀R [R ι V ⇒ (P' ι R ⇒ P ι R)] ∧ ∃R' [P ι R' ∧ R' ι V ∧ ¬(P' ι R' )]

(V, P, P' )

⇔def ∀R [R ι V ⇒ (P ι R ⇔ P' ι R)]

A Geometric Theory of Vague Boundaries Based on Supervaluation

(V, P, P' )

⇔def

53

≺(V, P', P)

R P

Q

Fig. 2. The figure illustrates a vague region described by three sharp regions denoted by three different gray values. The light gray region includes the medium gray region, which includes the dark gray region. The points P and Q belong to the same degree to the vague region, whereas the point R has higher a degree of membership than the points P and Q

2.3

Degrees of Membership for Supervaluation and Fuzzy Set Theory

There are two different strategies to represent spatial vagueness: an absolute and a relative characterization. The characterization proposed in Section 2.1 is a relative description. An absolute representation on the other hand assigns a fixed value to every point of a gradual boundary of a vague region. In the case of fuzzy theories the value is a number and depends on the degree of membership to the boundary or to the vague region. In the case of the theory of supervaluation there are different alternatives to obtain a numerical representation for the degree of membership (symbolized as µ) of a point to a vague region. We show two alternatives, one for a finite structure, and one for a dense or continuous structure. Assuming a finite structure, the degree of membership of a point incident with a vague region can be characterized by the number of sharp regions containing the point relative to the number of all sharp regions of the vague region. ⋅ denotes the power of a set. µ(P ι V)

=def {R | P ι R ∧ R ι V} / {R | R ι V}

It is clear that the degree of membership of points of a vague region can be used to define the degree of membership for a region R. It is equal to the degree of all those points that are included in R but not in any sharp region of the vague region, which is part of R. In a dense or continuous structure infinitely many sharp regions specify a vague region. In this case the Hausdorff metric of sets can be used to define a degree of membership of points of the vague region but we omit the details. The degree of membership of points and sharp regions of a vague region in a supervaluational approach and the representation of a vague region as a fuzzy set can be derived from each other. The link between a supervaluational and a fuzzy approach are the alpha-cuts. In the theory of fuzzy sets the alpha-cut (α-cut) Mα of a set M ⊂ M' is the set of elements of M' whose membership value (denoted by the characteristic function χM) is bigger or equal as the threshold value α ∈ [0, 1]: Mα

=def {x ∈ M' | χM(x)³α}, α ∈ [0, 1].

54

L. Kulik

The sets Mα fulfill the following consistency condition: the smaller the threshold value, the larger the set Mα: α' < α ⇒ Mα ⊂ Mα'. The resolution identity principle states that every fuzzy set can be represented as a weighted set of alpha-cuts: χM = sup

α ∈ [0, 1]

{αχMα}, or more suggestive: M =




α ∈ [0, 1]

{αMα}.

Conversely, according to this principle every fuzzy set can be reconstructed from a family of nested sets if the sets fulfill the consistency condition. The resolution identity principle reaches the gap between fuzzy sets and sharp sets of classical set theory. The principle connects the proposed characterization of spatial vagueness based on geometric supervaluation and a characterization based on fuzzy sets. The alpha-cuts constitute the sharp regions in the framework of geometric supervaluation. Hence, given a fuzzy set the degree of membership of a sharp region agrees then with the membership threshold value that all points of the region exceed. On the other hand, if the degrees of the sharp regions of a vague region are known, the resolution identity principle shows how to generate a fuzzy set from the degrees of the sharp regions. However, from the perspective of supervaluation the resolution identity principle is only one possibility to obtain an absolute representation like fuzzy sets.

3

Reasoning about Spatial Vagueness

This section outlines the advantages of geometric supervaluation in reasoning tasks. In a simple scenario we investigate whether a strong gradual transition of two vague regions V1 and V2 is a possible habitat of an animal A to show the differences between supervaluational and fuzzy reasoning. The spatial extension of the strong gradual transition is given by esgt(V1,V2). The animal A only settles in an area if it finds at least one of two different plants Pl1 and Pl2. The statement ‘the plant Pli is found at location P’ is abbreviated by p(Pli, P). The statement p(Pli, P) has to be understood as a matter of degrees since plants vary in their covering of regions. The average degree of covering—given by the number and the spread of plants within a predefined area—determines the degree of truth of p(Pli, P). To say p(Pli, P) is true to a certain degree means it is true that the point P is covered with the plant Pli by this degree. We assume two rules: the plant Pl1 is found everywhere in a forest region V1, and the plant Pl2 occurs all over a meadow region, denoted by V2 (see Fig. 3a, Fig. 3c). In a degree-based approach the two rules are formulated as (D i )

∀P [P ι Vi ⇒ p(Pli, P)],

i=1, 2.

For all points of the gradual boundary the rules imply that the plants can be found only partially depending on the membership degree of the points to the forest and the meadow. The rules (Di) can be reformulated in a supervaluational way to guarantee that they hold in every interpretation of the vague region as a sharp region. If a sharp region Ri is a precisification (see Section 1.2) of a vague region Vi and if P is a point that lies in Ri then it is true that under this interpretation the plant Pli can be found at P. It does

A Geometric Theory of Vague Boundaries Based on Supervaluation

55

not follow, however, that it is supertrue that the plant Pli can be found at the point P: for every point P of the gradual boundary of Vi there are other precisifications of the vague region which do not contain the point P. Nevertheless, it is supertrue that the plant Pl1 (Pl2) can be found everywhere at the core of the forest (meadow) because the points of the core of the forest (meadow) are contained in every sharp region that is a precisification of the vague region V1 (V2). We assume that both plants are in opposite competition so that they cannot be located at the same position. Therefore, every sharp region Ri that is a precisification of a vague region Vi is covered only by the plant Pli. The core of the forest is accordingly covered solely by the plant Pl1, the core of the meadow solely by the plant Pl2. a)

core of meadow

b)

c)

meadow meadow

forest forest

Forest forest

forest meadow

core of forest

meadow

forest forest

forest forest

meadow forest

meadow meadow

core of meadow

Fig. 3. The left figure shows a forest that is surrounded by a meadow. Since the forest has a vague spatial boundary the spatial extension of the transition of the forest and the meadow is a region. A white dashed line surrounds the core of the forest and the core of the meadow is the area between the black dashed line and the rectangle. The middle figure depicts three different interpretations of the vaguely bounded forest region as sharp regions. The darker the gray line, the more interpretations count the enclosed region as part of the forest. The right figure shows a distribution of two plants (symbolized as circles and triangles) in the forest and in the meadow

The question is whether the animal A considers the region esgt(V1,V2) of the gradual transition of the forest and the meadow as a possible habitat. Thus, we have to determine the truth value of the disjunction p(Pl1, P) ∨ p(Pl2, P) if P ι esgt(V1,V2). According to the construction of the scenario we expect that the animal clearly considers the gradual transition as a possible habitat (see Fig. 3c). At first we show how to reason without assuming degrees of truth. Since the gradual transition of the forest and the meadow is a strong one, in every interpretation of the transition by two sharp regions a point of the gradual transition belongs to one of these regions. If the point P belongs to a precisification of the forest region p(Pl1, P) holds, because a sharp region of the forest region is covered only by the plant Pl1. Otherwise, the point P has to belong to the meadow region such that p(Pl2, P) holds. Therefore, in every interpretation one of the plants Pl1 or Pl2 can be found at P. This means the statement p(Pl1, P) ∨ p(Pl2, P) is true for every interpretation and the disjunction is supertrue. As a result it turns out that the transition of the forest and the

56

L. Kulik

meadow is definitely a possible habitat for the animal A. This inference does not rely on numerical concepts and is based on ordering knowledge only. In the next step we show how to introduce degrees of truth into a geometrical account of supervaluation. The notion of degrees of truth seems to be associated with many-valued logics like fuzzy logic, but Kamp (1975) shows that degrees of truth can also be introduced in a supervaluational account. To say a predicate or statement is partially true means it is true that the statement is only applicable to an entity to a certain degree depending on the admissible interpretations. The main idea is to measure the set of admissible interpretations for a predicate. However, to define such a measure µ in the general case of an arbitrary predicate is very difficult. In the case of vague regions it is possible to introduce such a measure by using spatial knowledge like ordering information. According to Section 2.3, there are different ways to associate degrees of membership to a point of a vague region using its sharp regions. It is a remarkable feature of the theory of supervaluation that it is able to give up the truth-functionality: the truth-value of compound statements cannot be determined by the truth-value of its individual statements. This allows us to incorporate compatibility conditions: a compatibility condition indicates to what extent two spatial statements, abbreviated by p(P) and q(P), can hold simultaneously at a point P and thus specifies the truth value of the conjunction p(P) ∧ q(P). The lack of truth-functionality corresponds to the situation in probability theory: it is not possible to derive the probability (of the union or the intersection) of two events by using the probability of the single events. Therefore, we adapt a well-known theorem of probability theory, the formula of Sylvester-Poincaré, to determine the truth-value of compound statements. The corresponding formula relates the disjunction and conjunction of two spatial statements p(P) and q(P) and takes the following form: µ(p(P) ∨ q(P)) = µ(p(P))+µ(q(P))-µ(p(P) ∧ q(P)) This formula is not truth-functional because it is not possible to derive the truth-value of the disjunction by knowing the truth-values of p(P) and q(P). In the scenario every point P of the strong gradual transition belongs to the vague regions V1 and V2 to different degrees. According to Section 2.3 we can determine the degrees of membership if P ι V1 or P ι V2 holds. The membership function has to fulfill the condition µ(P ι V1)+µ(P ι V2)=1 for every point P of the gradual transition.6 Since the membership function of a vague region takes all sharp regions into account that includes the point P it follows from (Di): µ(P ι Vi)=µ(p(Pli, P)). Hence, we obtain µ(p(Pl1, P))+µ(p(Pl2, P))=1. Since the plants are in direct competition at every point P of the gradual transition it follows µ(p(Pl1, P) ∧ p(Pl2, P))=0. Therefore, the truth-value of the disjunction is equal to 1 at every point P. This result is identical to the result obtained by supervaluational reasoning without using numbers. We consider different compatibility conditions of the plants. If the plants do not compete, we can assume µ(p(Pl1, P) ∧ p(Pl2, P))=min(µ(p(Pl1, P)), µ(p(Pl2, P))) leading to µ(p(Pl1, P) ∨ p(Pl2, P))=max(µ(p(Pl1, P)), µ(p(Pl2, P))). If there is a partial influence of the plants a possible formula is µ(p(Pl1, P) ∧ p(Pl2, P))=µ(p(Pl1, P)) ⋅ µ(p(Pl2, P)). If the sum µ(p(Pl1, P))+µ(p(Pl2, P)) is bigger than 1 at some points this indicates that the 6

The membership function for the finite case of section 2.3 fulfills this condition.

A Geometric Theory of Vague Boundaries Based on Supervaluation

57

gradual transition is a weak one and the plants cannot be incompatible at the corresponding points. This short discussion shows that the truth-value of the disjunction of two statements does not solely depend on the truth-value of the statements but also on the truth-value of the compatibility condition in every point of the gradual transition of the vague regions. To show the differences of inferences between the presented account, which gives up the truth-functionality, and fuzzy logic its typical assumptions are compiled briefly. There are three main types of fuzzy logic for which completeness and soundness results are known (cf. Hájek, 1998). These types differ in the way they model the truth-value of the conjunction, the implication, and the disjunction of two statements p and q. We list the truth-values for the conjunction of each of the three logics. In the case of the Lukasiewicz logic µ(p ∧ q)=max(µ(p)+ µ(q)-1, 0) holds, in the case of the product logic µ(p ∧ q)= µ(p) ⋅ µ(q) holds, and in the case of the Gödel logic µ(p ∧ q)=min(µ(p), µ(q)) holds. We discuss only the Gödel logic since the main argument relies on the truth-functionality used in fuzzy logic and not on the specific type of fuzzy logic. In case of the Gödel logic the truth-value of the disjunction is µ(p ∨ q)=max(µ(p), µ(q)) and of the negation µ(¬p)=1-µ(p). To be able to reason with fuzzy logic, we use a system that ensures soundness and completeness. It is based on the proof-theoretic notion that a statement p is provable to (at least) a degree of α:
α p. The modus ponens has the following form: if
α p and
β (p ⇒ q) then
max{0,α+β-1} q. We use the soundness and completeness result stating that p is provable to a degree of α if and only if p is true in every model to a degree of α. For the scenario this leads to the following interpretation. A location P of the region R of the strong gradual transition belongs to the forest to a degree of αP ∈ ]0, 1[ (see Fig. 3a) and therefore to a degree of 1-αP to the meadow. Since the rules (Di) are definitely true, that is to say β=1, we obtain µ(p(Pl1, P))=αP and µ(p(Pl2, P))=1-αP. Hence, the truth value of the disjunction µ(p(Pl1, P) ∨ p(Pl2, P)) is equal to max{αP, 1-αP}. That means depending on the location P it is true to a degree between 0.5 and 1 to find at least one of the two plants Pl1 and Pl2. Therefore, it is possible to a degree of at least 0.5 that the animal A settles in R depending on the location P. Using the product logic, we obtain that it is true to a degree between 0.75 and 1 to find at least one of the two plants. The Lukasiewicz logic provides the value 1 for every point of R in agreement with the theory of supervaluation stating that it is definitely possible that the animal A settles in the region of the gradual transition. Comparing the results it turns out that every type of fuzzy logic has a different (inherent) assumption about the compatibility condition. Since the compatibility condition of the Lukasiewicz logic accidentally is the same as the one in this specific scenario it is able to derive the desired result whereas the Gödel logic and the product logic are not able to derive it. Nevertheless, in another scenario assuming a different compatibility condition the Lukasiewicz logic would fail to derive the desired inferences, as well. This holds for all types of fuzzy logic that are based on the assumption of truth-functionality. They all have to make an assumption how to determine the truth-value of the disjunction without knowing the truth-value of the conjunction (or vice versa) and thereby restricting themselves to the possible inferences. Since

58

L. Kulik

supervaluational reasoning does not rely on truth-functionality the scenario demonstrates that the incorporation of a compatibility condition allows a more appropriate way to draw inferences about vague spatial regions.

4

Conclusion and Outlook

The presented approach provides a formal specification of gradual boundaries of vague regions and of gradual transitions between vague regions using ordering information. It enables us to employ the methods of classical logic to reason about vagueness with and without numbers. The formalization characterizes vague regions of spatial entities as sets of sharp regions. Therefore, it is not necessary to design a spatial database from scratch to integrate reasoning about vaguely bounded entities, as it is the case for fuzzy set theory. The introduction of a compatibility condition of spatial concepts allows a more versatile way to reason about spatial vagueness. A theory that takes into account the fact that a gradual boundary itself does not have a uniquely determined beginning and ending, which means that its spatial extension is vague, too, has to cope with higher-order vagueness. Thus, a central objective of future work involves the characterization of higher-order vagueness. The representation of spatial vagueness and the reasoning in a supervaluational way opens a new perspective to deal with vagueness. The inferences demonstrated in Section 3 are drawn within a specific scenario. Therefore, a complete theory that does not presuppose truth-functionality to reason about vagueness is the next step towards an alternative approach describing vague geographical objects.

5

References

Burrough, P.A. (1996). Natural objects with indeterminate boundaries. In P.A. Burrough & A.U. Frank (Eds.), Geographic Objects with Indeterminate Boundaries (pp. 3–28). London: Taylor & Francis. Clementini, E., di Felice, P. (1996). An algebraic model for spatial objects with indeterminate boundaries. In P.A. Burrough & A.U. Frank (Eds.), Geographic Objects with Indeterminate Boundaries (pp. 155–169). London: Taylor & Francis. Cohn, A.G. (1997). Qualitative Spatial Representation and Reasoning Techniques. In G. Brewka, C. Habel & B. Nebel (Eds.), KI-97 – Advances in Artificial Intelligence (pp. 1–30), Berlin: Springer. Cohn, A.G., Gotts, N.M. (1996a). The ‘egg-yolk’ representation of regions with indeterminate boundaries. In P.A. Burrough & A.U. Frank (Eds.), Geographic Objects with Indeterminate Boundaries (pp. 171–187). London: Taylor & Francis. Cohn, A.G., Gotts, N.M. (1996b). Representing Spatial Vagueness: A Mereological Approach. In L.C. Aiello, J. Doyle & S. Shapiro (Eds.), Proceedings of the 5th conference on principles of knowledge representation and reasoning, KR ‘96 (pp. 230–241). San Francisco: Morgan Kaufmann.

A Geometric Theory of Vague Boundaries Based on Supervaluation

59

Couclelis, H. (1996). A typology of geographic entities with ill-defined boundaries. In P.A. Burrough & A.U. Frank (Eds.), Geographic Objects with Indeterminate Boundaries (pp. 45– 55). London: Taylor & Francis. Egenhofer, M.J., Herring, J. (1991). Categorizing Binary Topological Relationships between Regions, Lines and Points in Geographic Databases. Technical report, Department of Surveying Engineering, University of Maine. Elkan, C. (1994). The paradoxical success of fuzzy logic. IEEE Expert, 9 (4), 3–8. (Followed by responses and a reply.) Erwig, M., Schneider, M. (1997). Vague Regions. In M. Scholl & A. Voisard (Eds.), 5th Int. Symp. on Advances in Spatial Databases (pp. 298–320). Berlin: Springer. Eschenbach, Carola, Habel, C., Kulik, L. (1999). Representing simple trajectories as oriented curves. In A.N. Kumar & I. Russell (Eds.), FLAIRS-99, Proceedings of the 12th International Florida AI Research Society Conference (pp. 431–436). Orlando, Florida. Fine, K. (1975). Vagueness, truth and logic. Synthese, 30, 265–300. Fisher, P. (2000). Sorites paradox and vague geographies, Fuzzy Sets and Systems, 113 (1), 7–18. Hadzilacos, T. (1996). On layer-based systems for undetermined boundaries. In P.A. Burrough & A.U. Frank (Eds.), Geographic Objects with Indeterminate Boundaries (pp. 237–255). London: Taylor & Francis. Hájek, P. (1998). Metamathematics of Fuzzy Logic. Dordrecht: Kluwer. Kamp, J.A.W. (1975). Two theories about adjectives. In E.L. Keenan (Ed.), Formal Semantics of Natural Language (pp. 123–155). Cambridge: Cambridge University Press. Keefe, R., Smith, P. (Eds.) (1997). Vagueness: A Reader. Cambridge, MA: MIT Press. Kulik, L., Klippel, A. (1999). Reasoning about cardinal directions using grids as qualitative geographic coordinates. In C. Freksa & D.M. Mark (Eds.), Spatial Information Theory (pp. 205–220). Berlin: Springer. Randell, D.A., Cui, Z., Cohn, A.G. (1992). A spatial logic based on regions and connection. In Proceedings 3rd International Conference on Knowledge Representation and Reasoning (pp. 165–176). San Francisco: Morgan Kaufmann. Schneider, M. (1999). Uncertainty Management for Spatial Data in Databases: Fuzzy Spatial Data Types. In R.H. Güting, D. Papadias, F., Lochovsky (Eds.), 6th Int. Symp. on Advances in Spatial Databases (pp. 330–351). Berlin: Springer. Sharma, J., Flewelling, D., Egenhofer, M. (1994). A qualitative spatial reasoner. In Sixth International Symposium on Spatial Data Handling (pp. 665–681). Smith, B., Varzi, A. (1997). Fiat and Bona Fide Boundaries: Towards an Ontology of Spatially Extended Objects. In: S.C. Hirtle & A.U. Frank (Eds.), Spatial Information Theory: A Theoretical Basis for GIS (pp. 103–119). Berlin: Springer. Tye, M. (1994). Sorites paradoxes and the semantics of vagueness. In J. Tomberlin (Ed.), Philosophical Perspectives: Logic and Language (pp. 189–206). Atascadero, CA: Ridgeview. Varzi, A. (1997). Boundaries, Continuity, and Contact, Noûs, 31(1), 26–58. Varzi, A. (2000). Vague Names for Sharp Objects. In L. Obrst & I. Mani (Eds.), Proceedings of the KR Workshop on Semantic Approximation, Granularity, and Vagueness (pp. 73–78), Breckenridge, CO: AAAI Press. Worboys, M.F. (1998). Computation with imprecise geospatial data, Computers, Environment and Urban Systems, 22 (2), 85–106. Zadeh, L. (1975). Fuzzy logic and approximate reasoning. Synthese, 30, 407–428.

When Tables Tell It All: Qualitative Spatial and Temporal Reasoning Based on Linear Orderings G´erard Ligozat LIMSI, Paris-Sud University Bldg. 508, P.O. Box 133 F-91403 Orsay, France [email protected] Abstract. In [8] Bennett, Isli and Cohn put out the following challenge to researchers working with theories based on composition tables (CT): give a general characterization of theories and relational constraint languages for which a complete proof procedure can be specified by a CT. For theories based on CTs, they make the distinction between a weak, consistency-based interpretation of the CT, and a stronger extensional definition. In this paper, we take up a limited aspect of the challenge, namely, we characterize a subclass of formalisms for which the weak interpretation can be related in a canonical way to a structure based on a total ordering, while the strong interpretations have the property of aleph-zero categoricity (all countable models are isomorphic). Our approach is based on algebraic, rather than logical, methods. It can be summarized by two keywords: relation algebra and weak representation.

Keywords: temporal reasoning, spatial reasoning, relation algebra, weak representation, complete theory

1

Introduction

Many spatial and temporal reasoning formalisms represent and reason about useful information in terms of a finite vocabulary of binary relations holding among objects in space or time. The logical dependencies between relations may be stated in logical terms, e.g. as axioms of a first order logic. In many cases, a constraint-based approach is used for reasoning, based on the reformulation of the problem as a constraint-satisfaction problem (CSP) [23]. Typically, we consider a finite set B of basic dyadic relation symbols. The dependencies between the basic relations are specified using a composition table (CT): The CT defines a mapping B × B → 2B : For each pair (a, b) of elements in B, it specifies a subset of B called the composition of a and b, and denoted by (a ◦ b). In all cases we consider, it will be assumed that in the intended domains of interpretation the actual relations corresponding to the basic symbols in B form a jointly exhaustive and pairwise disjoint (JEPD) partition of the possible relations which can hold between pairs of objects. Subsets of basic relations, which are also called relations, are interpreted as disjunctions. D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 60−75, 2001.  Springer-Verlag Berlin Heidelberg 2001

When Tables Tell It All: Qualitative Spatial and Temporal Reasoning

61

Example 1 (Allen’s relations). A typical example is the set of basic Allen’s relations [1] between two intervals, which we denote by p, m, o, s, d, f, eq, p
, m
, o
, s
, d
, f
. 1.1

A challenge

In [8] Bennett, Isli and Cohn make the distinction between a weak, consistencybased interpretation of the CT, and a stronger extensional definition. In the consistency-based interpretation, we only impose that, whenever the interpretation of a holds between two objects x and y, and the interpretation of b holds between y and z, then for some c in (a ◦ b), the interpretation of c holds between x and z. In the stronger extensional definition, the converse must also be true, for any entry in the CT: If (the interpretation of) c holds between x and z, where c ∈ (a ◦ b), then there must exist some object y in the domain such that (the interpretation of) a holds between x and y, and (the interpretation of) b holds between y and z. Now the challenge consists in giving a general characterization of theories and relational constraint languages for which a complete proof procedure can be specified by a CT. 1.2

A full answer for a class of linear-ordering-based theories

In this paper, we take up a limited aspect of the challenge, namely, we characterize a subclass of formalisms for which the weak interpretation can be related in a canonical way to a structure based on a total ordering, while the strong interpretations have the property of ℵ0 -categoricity (all countable models are isomorphic). The class of formalisms we consider is based on linear orderings in the following sense: Each formalism is concerned with either sequences in linear orderings (generalized intervals), or with tuples of such objects in a Cartesian product of linear orderings (cardinal directions, n-points, n-blocks). Our approach will be based on algebraic, rather than logical, methods. The paper extends some of the results announced in [14] to a wide class of calculi. The completeness of Allen’s calculus in its “strong” version was first proved by Ladkin [12] using quantifier elimination. In the germane field of spatial databases, various notions of completeness have been investigated, especially in connection with Egenhofer and Franzosa [10], aka RCC-8 [19], relations. For recent results on topological queries, and further pointers to the literature, the reader is referred e.g. to [18, 21]. 1.3

Structure of the paper

The structure of the paper is as follows: In section 2, we introduce configurations, which are the kinds of situations the formalisms are meant to describe, first in an intuitive, then in a formal way: We give a precise sense to the notion, which

62

G. Ligozat

involves sequences or tuples in a linear ordering. In order to give a precise sense to what a description is, we introduce the necessary algebra in section 3. Then the relevant notions are the notion of a weak representation, which satisfies the axioms in a weak sense, and the notion of a representation, where they are valid in the strong (extensional) sense. Section 4 is devoted to weak representations and their relations to configurations. Based on the characterization of weak representations, we prove in section 5 that representations are in fact associated to universal configurations in linear orderings which are dense and without endpoints. Categoricity, hence completeness and decidability of the corresponding theories follow. We conclude in section 6.

2 2.1

Temporal and spatial configurations Configurations: An informal approach

Consider the examples in Fig 1-4. In each case, we have a temporal or spatial configuration. In the first example (Fig. 1), the configuration contains three intervals in a linear time-line.

A

A

❡ ✡ ❏
o
✡ ❏ m ❏❏ ✢✡ ✡  ❡ ✲❡

B C

✲

B

o


C

Fig. 1. A configuration for Allen’s calculus and its associated description

In the second example (Fig. 2), the configuration involves generalized intervals in the sense of [14, 15], i.e. finite increasing sequences of time-points. An interpretation could be that A represents a visit to a hospital, and B and C the periods when two patients have been hospitalized, the middle point representing some intervention. In the third example (Fig. 3), the configuration involves four points in 2D space (e.g. four cities). The configuration is described using symbols of the cardinal direction calculus [16]: n stands for north, s for south, etc. In the last example (Fig. 4), the configuration is about three rectangular objects with their axes parallel to the axes of reference.

When Tables Tell It All: Qualitative Spatial and Temporal Reasoning

A

63

A

❡ ✡ ❏ (2, 4)2,3 ✡ ❏ (3, 5)2,3 ✡ ❏❏ ✢ ✡  ❡ ✲❡

B C

✲

B

(2, 4, 6)3,3

C

Fig. 2. A configuration for the (2, 3)-interval calculus and its associated description

✻ A

✉

B

✉

A

nw ✲ ❡D ❡ ❅ ne ✒✻ ❅ ❅ n sw w ❅ ❅ ❘❡ ❅ ❄ ❡ ✲

D

✉

C

✉

✲

B

ne

C

Fig. 3. A configuration for the cardinal direction calculus and its associated description

✻

A A

❡ ✡ ❏
(o, d
) ✡ ❏ (o, o ) ❏❏ ✢✡ ✡  ❡ ✲❡

B

C

✲

B

(s
, p
)

C

Fig. 4. A configuration for the rectangle calculus and its associated description

64

G. Ligozat

2.2

Describing configurations

In order to describe such configurations, we use configuration languages based on a finite set of binary predicate symbols A. In the first case, B is the set of basic Allen relations. Using these symbols, the first configuration can be described by the graph on the right side of Fig. 1, which expresses that A is overlapped by B, that B is overlapped by C, and that A is met by C. In the second case, the language is about 2-intervals and 3-intervals in the sense of Ligozat. The set of basic relations is the union of Π2,2 , the set of basic (2, 2)- relations, i.e. Allen’s relations, of Π2,3 , Π3,2 , and of Π3,3 (more generally, Πp,q denotes the set of basic relations of one p-interval to a q-interval). Using the encoding of basic relations defined in [15], Πp,q can be identified with the set of non-decreasing sequences of length p of integers between 1 and 2q, where no odd number occurs more than once. Using these conventions, the second configuration can be described by the graph on the right side of the picture. In the third case, the language is about points in 2D-space, or by projection, pairs of points on each axis. The set of basic relations is called cardinal directions in [16]. Using the notations in that paper, the third configuration can be described by the graph on the right side of Fig. 3. For instance, it expresses that A is north of B, north-east of C and north-west of D. Finally, in the fourth case, the language is about rectangles in 2D-space, or by projection, pairs of intervals on each axis. The set of basic relations is determined by pairs of Allen’s relations: For instance, the relation of B to C can be denoted by (s
, p
) because the first projection of B is started by the first projection of C, while the second projection of B follows the second projection of C. Using this notation, the fourth configuration can be described by the graph on the right side of Fig. 4. Based on the preceding examples, we define the general notion of configuration: Definition 1. – Let S be a finite subset of positive integers. A S-interval configuration C = (U, W ) is defined by a linear ordering W and a subset U of p-intervals in W , where p ∈ S. – Let n be a positive integer. A n-point configuration C = (U, W1 , . . . , Wn ) is defined by linear orderings W1 , . . . , Wn and a subset U of W1 × . . . × Wn . – Let n be a positive integer. A n-block configuration C = (U, W1 , . . . , Wn ) is defined by linear orderings W1 , . . . , Wn and a subset U of intervals (i.e. 2intervals) in W1 × . . . × Wn . In this way, the examples described above are a 2-interval configuration, a {2, 3}-configuration, a 2-point configuration, and a 2-block configuration, respectively (we can assume that the underlying linear orderings are the real numbers).

When Tables Tell It All: Qualitative Spatial and Temporal Reasoning

2.3

65

Configuration languages

We now make precise which languages we use to describe configurations. We consider three classes of languages. Each language is the first order language whose non logical symbols are the elements of B, where B is defined as follows: – Let S be a finite subset of the set of positive integers. Then BS is the union of all Πp,q , for p, q ∈ S. In particular, the example above uses B{2,3} . When S has just one element, we denote the set of basic relations by Bn . Hence B1 corresponds to time-points, B2 is the set of Allen’s relations. – For each n ≥ 1, BPointsn is the set B1 n of n-tuples in B1 , representing the qualitative positions of two points in nD-space. The corresponding calculus is the n-point calculus studied in [6]. – For each n ≥ 1, BRectn is the set B2 n of n-tuples in B2 , representing the qualitative positions of two blocks in nD-space. The corresponding calculus is the n-block calculus studied in [4, 7, 5]. We refrain from considering more complex products of basic objects, but it is clear that we could also consider products of p-intervals, and get analogous languages. These might be useful for describing objects with complex shapes. What we intend to do is to study how the languages can be used to describe configurations, and to what extent a description determines a configuration. Remark 1. In the examples we have considered up to now, we only dealt with finite configurations (that is, configurations whose U is finite), and represented them as networks whose vertices are the elements of U and whose arcs are labelled by symbols in B. Actually, we will give a precise sense to the notion of representation after we have introduced the algebraic point of view. The possibility of reasoning about infinite configurations will prove crucial in the sequel.

3

An algebraic point of view

Structural operations. We are now in a position to introduce the algebraic setup. First, for each B, we define A as the set of subsets of B. As such, it is a Boolean algebra. Because it represents binary relations, A has the additional structure provided by conversion and composition: If b ∈ B holds between two objects x and y, then b
holds between y and x. Composition is defined by a composition table. Finally, each A has an identity element for composition. We denote it by 1 . In the case of S-intervals, it is the sum (the union) of all 1 p = (1, . . . , (2p−1))p,p (the equality between p-intervals, for p ∈ S). So it is only atomic if S contains just one integer. In the case of n-points and n-blocks, 1 is (eq, . . . , eq) (n times), where eq is the point, resp. the interval equality.

66

G. Ligozat

Relation algebras. The Boolean algebra A, together with the additional structure provided by converse, composition, and unit element for composition, is a relation algebra in Tarski’s sense [22, 11]: A relation algebra A = (A, +, 0, ·, 1, ◦, 1 , ) is a Boolean algebra (A, +, 0, ·, 1) together with a unary operation of converse (denoted by α → α
), a binary operation of composition (denoted by (α, β) → (α◦β)), and a distinguished element 1 , such that the following conditions hold: 1. For all α, β, γ, (α ◦ β) ◦ γ = α ◦ (β ◦ γ); 2. 1 is a unit element for composition: (α ◦ 1 ) = (1 ◦ α) = α, for all α; 3. For all α, β and γ, the following conditions are equivalent: (α ◦ β) · γ = 0; (α
◦ γ) · β = 0; (γ ◦ β
) · α = 0 Relation algebras were introduced by Tarski in order to axiomatize the structural properties of binary relation algebras (BRA), whose elements are actual binary relations, with transposition, binary relation composition and the identity relation as a neutral element. All relation algebras we consider here are actually BRAs, because of the following facts. First, consider AS , for a subset of positive integers S, and define ψ on AS as follows: Proposition 1. Let Ψ (a), for each atomic relation a ∈ Πp,q , where p, q ∈ S, be the set of pairs (u, v), where u is a p-interval and v a q-interval in Q, such that a describes the corresponding configuration. Then Ψ is an isomorphism of relation algebras. Explicit formulas for converse and composition are available for AS and can be found e.g. in [13]. Similarly, for the n-point and n-block algebra: consider the algebras APointsn and ABlocksn , whose atoms are BPointsn and BBlocksn , respectively. Conversion and composition are computed component-wise. Proposition 2. Let Ψ (a), for each atomic relation a ∈ BPointsn (resp. BBlocksn ), be the set of pairs (u, v) of n-points (resp. n-intervals), such that a describes the corresponding configuration. Then Ψ is an isomorphism of relation algebras. 3.1

Descriptions in algebraic terms: Weak representations

We are now in a position to describe in an intrinsic way what we mean by descriptions of configurations: The corresponding notion is called a weak representation (WR) of the corresponding relation algebra [14]. Intuitively, a weak representation is just a set U of elements, which stand for objects, together with the assignment to each atomic relation of a set of pairs (u, v) of elements in U (i.e. a binary relation in U ). To be interpreted as a model, these binary relations should satisfy the axioms corresponding to the algebraic properties. Definition 2. A weak representation of A is a map Φ of A into a product of algebras of subsets of U × U , where U is a non empty set, such that:

When Tables Tell It All: Qualitative Spatial and Temporal Reasoning

1. 2. 3. 4.

67

Φ is an homomorphism of Boolean algebras. Φ(α ◦ β) ⊇ Φ(α) ◦ Φ(β). Φ(1 ) = identity. Φ(α
) is the transpose of Φ(α).

Remark 2. Equivalently, weak representations are models of the associated firstorder weak theories: U is the domain of interpretation, and Φ is the interpretation function for the predicates associated to the symbols in B. Remark 3. When U is a finite set, a weak representation can be represented by a network, as already shown in the examples. This network is atomic, and none of the atoms is equality. Moreover, it is path-consistent: For any 3-tuple of vertices (i, j, k), the corresponding labels ai,j , aj,k and ai,k are such that (ai,j ◦ aj,k ) contains ai,k . Conversely, if a network is such that it is atomic, that none of its labels is equality, and if it is path-consistent, then it defines a weak representation. 3.2

From configurations to weak representations

Given a configuration, it is now clear how to associate a weak representation of the corresponding algebra to it: For each pair of elements in U , there is exactly one atomic relation a in the algebra such that a holds between u and v. Hence, the interpretation function Φ that, for every atom a, lists the corresponding set of pairs (u, v) in U ×U has the necessary properties to define a weak representation: Proposition 3. There is a well-defined construction G which, to any A-configuration, associates a weak representation of A. In the case of AS , this amounts to describing the configuration in terms of (p, q)-relations, for p, q ∈ S: For each pair (u, v) of elements in U , where u is a p-interval and v a q-interval, there is one well defined atomic (p, q)-relation holding between u and v. The weak representation is defined as (U, Φ), where: Φ(a) = {(u, v) ∈ U | a holds between u and v}. In the case of AP ointsn , the construction consists in describing the configuration in terms of the projections of the elements of U . Again, for each pair (u, v) of elements in U , where u, v ∈ W1 × . . . × Wn , there is one well-defined atomic n-point relation holding between u and v, and the associated WR is defined as above. We skip the detailed construction for n-blocks. It is quite analogous to the preceding ones.

4 4.1

Characterizing weak representations From configurations to weak representations

The central tool in this paper is a construction going the other way, from weak representations to configurations. Before describing it, we state the result we are after:

68

G. Ligozat

Proposition 4. There is a well-defined construction F which, to any weak representation U = (U, Φ) of A, associates a surjective A-configuration. By definition, a configuration is surjective if: – any element of W is a point of some generalized interval in U , in the case of S-intervals; – any element of W1 , . . . , Wn is a projection of some element in U , in the case of n-points or n-blocks. We describe the construction separately for the two cases. The construction in the AS case. Let (U, Φ) be a WR of AS . The intuition behind the construction is that, since an element in U is a p-interval, for some p in S, it defines a strictly increasing sequence in some linear ordering. Hence using p copies of U allows us to get a copy of each of the p points which appear in it. The challenge is to recover this information from the algebraic data. Firstly, consider the unit element for composition 1 in AS . It is a sum:
1 = p∈S 1 p Each 1 p is an atom in AS . Let Up = Φ(1 p ). Then Up is a subset of U , the subset of p-intervals, and U is the disjoint union of all Up , p ∈ S. Now, the idea is to build W using the points in U . Each element in U1 (if any) will contribute a point; each element in U2 , two points, and so on. This motivates the following definition (where s = max(S)): W  = (U1  U2 . . .  Us )  (U2 . . .  Us )  . . . (Us ) In this direct sum, the first summand corresponds to first points; the second to second points (hence points in U1 do not contribute); the third to third points, and so on; finally, the only subset contributing s-th points is Us . We denote by W  (i, p) the copy of Up in the i-th summand of W  . Hence i ≤ p ≤ s, and W  (i, p) corresponds to the i-th points in the p-intervals. Now W  makes a complete survey of all points. Of course, some points may appear more than once, e.g. as the i-th point of some element and simultaneously as the j-th of another. But the algebra tells us about it: Definition 3. Let ap,q i,j be the subset of (p, q)- relations in Πp,q whose i-th coordinate is (2j − 1). Hence a pair (u, v) is an atomic relation in ap,q i,j if the i-th point of u coincides with the j-th point of v. In that case, we have to identify the two points. Definition 4. Let W be the quotient set of W  by the following relation: u ∈ W  (i, p) is equivalent to v ∈ W  (j, q) if and only if (u, v) ∈ Φ(ap,q i,j ). In order for the definition to make sense, Φ(ap,q i,j ) has to be an equivalence relation. That this is so is a consequence of:

When Tables Tell It All: Qualitative Spatial and Temporal Reasoning

69

Lemma 1. The following conditions hold:  1. ap,p i,i ⊇ 1 p . p,q
2. (ai,j ) = aq,p j,i . q,r 3. ap,q ◦ a = ap,r i,j j,k i,k .

Proof. Note that the relations ap,q i,j are convex relations [15, 13]. Hence it is enough to check that the suprema and infima behave as described under composition. This is easily checked by explicit calculation.   Because of the lemma, and because Φ is the map of a weak representation (hence the image of a composition contains the composition of the images), Φ(ap,q i,j ) is indeed an equivalence relation: It is reflexive, symmetric, and transitive. Example 2. Consider the example in Fig. 5.

A

A

❡ ✡ ❏ (2, 4)2,3 ✡ ❏ (3, 5)2,3 ❏❏ ✢✡ ✡  ❡ ✲❡

B C c1

b1

a1 b2 a2

✲

b3

B

(2, 4, 6)3,3

C

Fig. 5. Construction of the configuration associated to a weak representation: an example for {2, 3}-intervals

Here S = {2, 3}, and U2 = {A}, U3 = {B, C}. We have to take two copies of U2 and three copies of U3 , hence 2 + 6 = 8 points for W  . Using lower-case letters and indexes to distinguish between the different copies of U (index i means that the i-th summand, corresponding to i-th points, is considered), the general construction yields in this case: W  (1, 2) = {a1 }, W  (1, 3) = {b1 , c1 }, W  (2, 2) = {a2 }, W  (2, 3) = {b2 , c2 }, W  (3, 3) = {b3 , c3 }. Since (A, C) ∈ Φ((3, 5)2,3 ), and (3, 5)2,3 ∈ a2,3 1,2 , a1 and c2 are the same point in W . For analogous reasons, a2 and c3 coincide in W , and these are the only identifications to be made. Consequently, W has 6 elements, which can be represented by {a1 , a2 , b1 , b2 , b3 , c1 }. Now that we have the set of underlying points, we define a binary relation on it in order to get a linear ordering. Again, the algebra provides us with the necessary information: Definition 5. Let bp,q i,j be the subset of (p, q)- relations in Πp,q whose i-th coordinate is strictly less than (2j − 1).

70

G. Ligozat

Definition 6. u ∈ W  (i, p) precedes v ∈ W  (j, q) if and only if (u, v) ∈ Φ(bp,q i,j ). Lemma 2. The following identities hold:











q,q p ,q 1. api
,i,p ◦ bp,q i,j ◦ aj,j
= bi
,i
. p,q q,p
2. bi,j ∩ (bj,i ) = ∅ . q,r p,r 3. bp,q i,j ◦ bj,k = bi,k .

Because of this lemma, the definition of precedence is in fact a relation on the set W of equivalence classes. Moreover, this induced relation is irreflexive and transitive (properties 2 and 3). Hence it is a strict ordering on W . Finally, this ordering is linear, because of: p,q q,p
Lemma 3. 1. Πp,q = bp,q i,j  ai,j  (bj,i ) p,p p,p
   2. In particular, 1 p ∈ bi,j if i < j, 1 p ∈ ap,p if i > j. i,i , and 1 p ∈ (bi,j )

Proof. For any atomic (p, q)-relation a, if the i-th coordinate of a is strictly greater than (2j − 1), then the j-th odd integer, which determines the j-th coordinate of a
, is left of the i-th point, hence the j-th coordinate of a
is strictly less than 2i − 1.   Since 1 p = (1, 3, . . . , 2p − 1), the last claim is obvious. Putting everything together, we get a linear ordering W . Moreover, since Up has p canonical maps into W  , we get for each p in S a sequence of maps of length p of Up into W . This sequence is strictly increasing, hence a p-interval, because, obviously, if u ∈ Up , then (u, u) ∈ Φ(1 p ), hence the corresponding element in W  (i, p) precedes that in W  (j, p) if i < j, by the last part of the lemma. This, finally, means that each element in Up maps to a p-interval in W . We denote by ϕ be the corresponding map ϕ : U → S-Intervals in W This concludes the construction of F . Proposition 5. There is a well-defined construction F which, to any weak representation U = (U, Φ) of A, associates a surjective S-configuration ϕ in W : ϕ : U → S-Intervals. Example 3. Consider the second example again. Since (A, B) ∈ Φ((2, 4)2,3 ), and since (2, 4)2,3 ∈ b2,3 1,2 (the first coordinate 2 is less than the second odd number 3), we have a1 ≺ b2 . The final ordering is c1 ≺ b1 ≺ a1 ≺ b2 ≺ a2 ≺ b3 . The canonical map ϕ sends A onto the 2-interval (a1 , a2 ), B onto the 3-interval (b1 , b2 , b3 ), and C to the 3-interval (c1 , a1 , a2 ). The construction in the AP ointsn case. Let now (U, Φ) be a WR of AP ointsn . Here each element in U stands for a point in a product set. Hence we take one copy of each element of U to stand for each of the n projections. For 1 ≤ i ≤ n, let ai be the set of atomic relations in BP ointsn whose ith coordinate is eq. Clearly, this is an equivalence relation. Hence, since Φ is associated to a weak representation, Φ(ai ) is an equivalence relation on U .

When Tables Tell It All: Qualitative Spatial and Temporal Reasoning

71

Let Wi be the quotient set of W  by Φ(ai ). For each i, there is a canonical map of U into Wi , hence a map ϕ : U → W1 × . . . × Wn . This map is injective, because the intersection of all ai is 1 , whose image by Φ is the diagonal relation in U . For the moment, each Wi is a set without any additional structure. We now provide it with an ordering relation: For 1 ≤ i ≤ n, let bi be the set of atomic relations in BP ointsn whose i-th coordinate is ≺. Since composition can be computed component-wise, the following lemma is easy to prove: Lemma 4. The following identities hold: 1. 2. 3. 4.

ai ◦ bi ◦ ai = bi bi ∩ b
i =∅ bi ◦ bi = bi
bi ◦ b
i = bi ◦ bi = BP ointsn

Hence we can define precedence on Wi : (the equivalence class of) u precedes (that of) v if and only if (u, v) ∈ Φ(bi ). Because of the lemma, this is welldefined, and induces a strict ordering relation on Wi which is in fact a linear ordering (obvious). Example 4. Consider the third example (Fig. 6).

✻ a2

b2

A

✉

B

✉

A

nw ✲ ❡D ❡ ❅ ne ✒✻ ❅ ❅ n sw w ❅ ❅ ❘❡ ❅ ❄ ❡ ✲

D

✉

C c2

✉

✲

B

ne

C

c1 a1 d1 Fig. 6. Construction of the configuration associated to a weak representation: an example for the cardinal direction calculus

Using the notations of [16], a1 is {n, eq, s}, while a2 is {e, eq, w}. Hence the classes of A and B coincide in W1 , while those of B and D coincide in W2 , and these are the only non-trivial equivalences. Hence both W1 nd W2 have three points. Moreover, b1 is {nw, w, sw}, and b2 is {ne, e, se}. Hence we get the orderings c1 ≺ a1 ≺ d1 on W1 and c2 ≺ b2 ≺ a2 on W2 .

72

G. Ligozat

4.2

The ARectn case

This case combines the two types of construction: 2-sequences (intervals in the sense of Allen) and products. We leave the detailed construction to the reader.

4.3

Provisional conclusions

The preceding results show that there is a full correspondance between the notion of a configuration in (a product of) linear ordering(s) and a weak representation of the corresponding algebra. In the next section, we use this fact to determine the representations of the algebra, which are models of the first order theory in the extensional sense. Remark 4. In the particular case of finite configurations, the associated weak representations are conveniently described using the language of constraint networks. Now a consequence of the constructions is the following: Corollary 1. Any atomic path-consistent network is consistent.

5 5.1

Classifying Representations Strong models: Representations

The stronger notion of a model specifies that the axioms embodied in the composition table should be interpreted as necessary and sufficient conditions: namely, if a pair (u, v) belongs to the relation interpreting γ, and if γ can be obtained by composing α and β, then there should exist w in U such that (u, w) is in the interpretation of α, and (w, v) in that of β. This stronger notion corresponds to the standard notion of a representation in algebra. A weak representation is a representation if it is one-to-one and condition (2) is replaced by: 5. Φ(α ◦ β) = Φ(α) ◦ Φ(β). Recall that a configuration is surjective if any point in the relevant linear ordering(s) can be obtained from U . Among the surjective configurations are the universal configurations: Definition 7. 1. A S-configuration (U, W ) is universal if U is the set of all p-intervals in W , where p ∈ S. 2. A n-point (resp. a n-block configuration (U, W1 , . . . , Wn )) is universal if U = W1 × . . . × Wn .

When Tables Tell It All: Qualitative Spatial and Temporal Reasoning

5.2

73

The main result

We have shown that, for any weak representation U of an algebra in the class we consider, there is a canonical map ϕ which interprets the underlying set U as the set U in a configuration C = (U, W ) (in the cases of S-intervals) or C = (U, W1 , . . . , Wn ) (in the case of n-points or n-blocks). Lemma 5. The configuration C associated to a weak representation by F is surjective. Proof. Indeed, this is true by construction, because W (resp. each Wi ) is a quotient set of U .   The central result in this paper is that, if U is a representation, then this configuration is in fact universal: Theorem 1. The configurations C canonically associated to representations are universal. Moreover, the underlying linear orderings of the configurations are dense and without end-points. Proof. The proof of surjectivity is interesting, but rather technical. We omit it because of space limitations. The interested reader is referred to the full version of this paper.   We prove the second part of the claim: Lemma 6. If (U, Φ) is a representation, then each linear ordering in the associated configuration is dense and without end-points. Proof (the case of S-intervals). Consider for instance the proof of density. Suppose that w and w are two points in W , with w ≺ w . Then, for some pair (u, v) in Up × Ur , and integers i and k, w is the i-th point in u and w the k-th point in v, and (u, v) ∈ Φ(bp,r i,k ). p,q q,r By Lemma 2 (3), bp,r i,k = bi,j ◦ bj,k . Now, since we are dealing with a representation, there exists an element u q,r   in U such that (u, u ) ∈ Φ(bp,q i,j ) and (u , v) ∈ Φ(bj,k ). Then ϕ(u ) is such that     w ≺ ϕ(u ) ≺ w . Proof (the n-points and n-block cases). The proof is quite similar to the preceding case. For instance, density follows from Lemma 4 (3), and the non-existence of end-points from Lemma 4 (4).   5.3

Categoricity and completeness

Let A denote any of the algebras we have been considering. Since any representation of A is based on dense linear orders, there is no finite representation of A. Moreover, by a theorem of Cantor, any countable, dense linear ordering without end-points is isomorphic to the ordering of the rational numbers. We already observed that the particular universal configuration based on Q is indeed a representation of A. Hence, up to isomorphism, it is the only one. This proves:

74

G. Ligozat

Theorem 2. Any countable representation of A is isomorphic to the corresponding representation based on Q. In other terms, the associated first order theory is ℵ0 -categorical. Since, on the other hand, this theory has no finite model, it is complete, by Vaught’s theorem [24]. As a consequence, it is decidable.

6

Conclusions

We have shown that, for a restricted class of calculi based on linear orderings, the algebraic notion of weak representation captures the concept of a qualitative configuration in a full sense: Each configuration has one well-defined WR associated to it, and, conversely, each weak representation can be realized in a canonical way as a configuration. In particular, using the language of constraint networks, any atomic pathconsistent network is consistent. Moreover, models of the corresponding calculi in the strong, extensional sense, correspond to representations of the corresponding relation algebra. Because of the correspondence just mentioned, these algebras have only one countable representation, up to to isomorphism, which is the standard one based on Q. Hence the strong theories are categorical. The calculi based on linear orderings are only particular cases of qualitative spatial and temporal calculi. Similar calculi such as temporal calculi based on partial orderings (partially ordered time, relativistic time) have also been considered [2, 9]. The calculus of cyclic intervals [3] has also to be mentioned. On the spatial side, the RCC family of calculi [19] are similarly related to relation algebras, and the same questions can be asked about them. For all the calculi mentioned, however, we cannot expect similar behaviours: For instance, a weak representation for RCC-8 is associated to many non isomorphic configurations [20], although the approach used here can be somewhat extended [17]; some weak representations for the cyclic interval algebra do not correspond to any configuration on a circle. Hence extending the techniques described here to more general cases is an open research problem.

References [1] Allen, J. F. Maintaining Knowledge about Temporal Intervals. Comm. of the ACM, 26:11, 832–843, 1983. [2] F.D. Anger, D. Mitra, and R.V. Rodriguez. Temporal Constraint Networks in Nonlinear Time. In Proc. of the ECAI-98 Workshop on Spatial and Temporal Reasoning (W22), pages 33–39, Brighton, UK. [3] P. Balbiani and A. Osmani. A model for reasoning about topologic relations between cyclic intervals. In Proc. of KR-2000, Breckenridge, Colorado, 2000. [4] Ph. Balbiani, J.-F. Condotta, and L. Fari˜ nas del Cerro. A model for reasoning about bidimensional temporal relations. In Proc. of KR-98, pages 124–130, 1998.

When Tables Tell It All: Qualitative Spatial and Temporal Reasoning

75

[5] Ph. Balbiani, J.-F. Condotta, and L. Fari˜ nas del Cerro. A new tractable subclass of the rectangle algebra. In Proc. of IJCAI-99, pages 442–447, 1999. [6] Ph. Balbiani, J.-F. Condotta, and L. Fari˜ nas del Cerro. Spatial reasoning about points in a multidimensional setting. In Proc. of the IJCAI-99 Workshop on Spatial and Temporal Reasoning, pages 105–113, 1999. [7] Ph. Balbiani, J.-F. Condotta, and L. Fari˜ nas del Cerro. A tractable subclass of the block algebra: constraint propagation and preconvex relations. In Proc. of the Ninth Portuguese Conference on Artificial Intelligence (EPIA’99), pages 75–89, 1999. [8] B. Bennett, A. Isli, and A. Cohn. When does a Composition Table Provide a Complete and Tractable Proof Procedure for a Relational Constraint Language? In Proc. of the IJCAI-97 Workshop on Spatial and Temporal Reasoning, pages 75–81, Nagoya, Japan, 1997. [9] M. Broxvall and P. Jonsson. Disjunctive Temporal Reasoning in Partially Ordered Models of Time. In Proc. of AAAI-2000, Austin, Texas, 2000. [10] M.J. Egenhofer and R. Franzosa. Point-set topological spatial relations. Int. J. Geo. Info. Sys. 5(2), 161–174, 1991. [11] B. J´ onsson and A. Tarski. Boolean algebras with operators, part II. American J. of Mathematics, 74:127–162, 1952. [12] P. Ladkin. Models of Axioms for Time Intervals. In Proc. of AAAI-87, 1987. [13] G. Ligozat. Generalized Intervals: A Guided Tour. In Proc. of the ECAI-98 Workshop on Spatial and Temporal Reasoning (W22), pages 11–18, Brighton, UK, 1998. [14] G. Ligozat. Weak Representations of Interval Algebras. In Proc. of AAAI-90, pages 715–720, 1990. [15] G. Ligozat. On generalized interval calculi. In Proc. of AAAI-91, pages 234–240, 1991. [16] G. Ligozat. Reasoning about Cardinal Directions. J. of Visual Languages and Computing, 9:23–44, 1998. [17] G. Ligozat. Simple Models for Simple Calculi. In C. Freksa and D.M. Mark, editors, Proc. of COSIT’99, number 1661 in LNCS, pages 173–188. Springer Verlag, 1999. [18] C.H. Papadimitriou, D. Suciu and V. Vianu. Topological Queries in Spatial Databases. In Proc. ACM SIGACT-SIGMOD-SIGART Symp. on Principles of Database Systems, pages 81–92, 1996. [19] D. Randell, Z. Cui, and T. Cohn. A spatial logic based on regions and connection. In B. Neumann, editor, Proc. of KR-92, pages 165–176, San Mateo, CA, 1992. Morgan Kaufmann. [20] J. Renz. A canonical model of the region connection calculus. In Proc. of KR’98, Trento, Italy, 1998. [21] L. Segoufin and V. Vianu. Querying Spatial Databases via Topological Invariants. In Proc. ACM Symp. on Principles of Database Systems, 1998. [22] A. Tarski. On the calculus of relations. Journal of Symbolic Logic, 6:73–89, 1941. [23] P.K. Tsang. Foundations of Constraint Satisfaction. Academic Press, 1993. [24] D. van Dalen. Logic and Structure. Springer, 1997.

Computational structure in three-valued nearness relations Matt Duckham and Michael Worboys Department of Computer Science Keele University Staffordshire, ST7 8AA, UK Fax: +44 17 82 71 30 82 Tel: +44 17 82 58 42 70 Email: [email protected], [email protected]

Abstract. The development of cognitively plausible models of human spatial reasoning may ultimately result in computational systems that are better equipped to meet human needs. This paper explores how human subjects perceive the qualitative spatial relation nearness within an environmental space. Based on experimental data, a three-valued nearness relation is analysed in two stages. First, the results are analysed with special reference to the existence of subsets of candidate landmark places, from which nearness relations between other places may be partially inferred. Second, the desirable properties of such landmark sets are considered and some of their formal properties are presented. These properties are then considered in the light of the data furnished by the experiment. The paper concludes with a discussion of the significance of the analyses and the scope for further work in this area.

Keywords: Nearness; qualitative spatial reasoning; landmarks; data mining; similarity relation.

1

Introduction

The qualitative spatial relation nearness is a basic, commonly used spatial relation that is both vague and context dependent. Nearness is a vague concept in the sense that it exhibits borderline cases: it can sometimes be difficult to decide whether certain places are near or not near each other. London may be considered near Oxford and not near Edinburgh, but there exists no clearly defined boundary between Oxford and Edinburgh where places stop being near to London. Nearness is context dependent in the sense that two places can be near in one context and not near in another. In the context of capital cities of the world, London might be considered near to Paris; at the same time, in the context of cycling to work in the morning, London is certainly not near to Paris. What makes nearness particularly interesting and challenging to spatial information theorists is that despite these apparent contradictions, humans are able to D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 76−91, 2001.  Springer-Verlag Berlin Heidelberg 2001

Computational Structure in Three-Valued Nearness Relations

77

effectively reason and communicate using vague, context dependent qualitative spatial concepts like nearness. Research in geographic information science has been concerned primarily with quantitative rather than qualitative spatial information. By contrast, humans are generally better at qualitative rather than quantitative spatial reasoning. Many spatial concepts commonly used by humans, such as nearness, present considerable difficulties within a computational setting. For example, Fisher [4] explores some of the problems arising from the inherently vague nature of many common spatial concepts. Gahegan [8] looks at the need for a qualitative userdefined context in understanding the significance of quantitative spatial information. The practical aim of such research is ultimately the development of improved GIS interfaces that are better equipped to support human concepts and decision making processes [15], for example by developing more user-friendly spatial query languages, decision support systems, and navigational aids. Building on experimental work reported in [25], this paper addresses the integration of experimental data and computational models concerning the qualitative spatial relation nearness. The aim of this research is to develop and explore cognitively plausible formal models of nearness as perceived by human subjects in an environmental space, the Keele University campus. Following a review of previous relevant work in §2, §3 presents a discussion of the experimental methods used to collect data about a group of human subjects’ perception of the qualitative spatial relation nearness. The analysis of this data is presented in two stages: §4 explores the use of data mining techniques to uncover computational structure within the experimental data; §5 develops elements of this structure into a more detailed formal model of qualitative vector space. Finally, §6 presents a review of the findings and an agenda for further work.

2

Background

Much research into qualitative spatial reasoning can be characterised as striking a compromise between computational and cognitive perspectives. Research from a computational perspective aims to develop formal models of qualitative spatial reasoning capable of being used within computational systems. Research from a cognitive perspective aims to provide insights into how humans use and reason with qualitative spatial information, with particular reference to experiments using human subjects. Although the two perspectives are closely related they are to some extent inherently incompatible: no formal logical system will ever be able to satisfactorily capture the diversity or flexibility of human spatial reasoning. However, the attempt to develop more cognitively plausible models of human spatial reasoning depends on the closer integration of these two perspectives. This section reviews the existing research from these two perspectives, focusing in particular on the nearness relation.

78

M. Duckham and M. Worboys

2.1

Computational and cognitive approaches to qualitative spatial information

In addition to inherently qualitative work on topology (eg [3]) and shape (eg [1]), work on computational aspects of qualitative distance (eg [9] and [7]) and qualitative direction (eg [14]) have appeared in the literature. Frank [6] develops a calculus usable for reasoning over both cardinal directions and qualitative distances, although the integration of distance and direction remains an unresolved research issue. A feature of many such computational approaches is that while qualitative, they often do not correspond well with the observed characteristics of human spatial cognition. Sadalla et al. [21] observed asymmetries in the human perception of nearness, with more significant reference points or landmarks generally being understood to be near to adjacent points more frequently than vice versa. Stevens and Coupe [23] and Hirtle and Jonides [12] provide evidence of distortions in human spatial cognition resulting from the apparent hierarchical arrangement of places according to spatial and semantic criteria. Further evidence of distortions congruent with the existence of hierarchies has been observed in the form of clustering for judgements of distances between and navigational paths through sets of places [13, 11]. Distortions such as asymmetry, vagueness, landmarks, hierarchies and clustering have proved difficult to integrate with the logical systems which form the backbone of computational approaches to qualitative spatial reasoning. Nevertheless, Tversky [24] argues that these distortions are important cognitive devices that help humans to organise spatial information. Ideally, cognitively plausible computational models of qualitative spatial information should be able to allow for such distortions. Some research has begun to provide a basis for closer integration between the computational and the cognitive. The ‘egg-yolk’ calculus [2] has proved particularly useful as a formal framework for reasoning about vague or indeterminate boundaries. Hirtle [10] provides a survey of three mathematical structures (trees, ordered trees and semi-lattices) capable of representing the hierarchical nature of spatial cognition. Robinson [18, 19] uses an adaptive algorithm to produce a fuzzy membership function for nearness based on human subjects’ responses to questions. In general, however, formal and computational work on qualitative spatial reasoning is still some way from being integrated with the observations from cognitive work. The remainder of this paper reports on work that takes a step toward providing better integration between computational and cognitive approaches to qualitative spatial reasoning.

3

Experimentation

The analyses explored in this paper are based upon experimental data concerning the qualitative spatial relation “near”, reported in more detail in [25]. Only a brief overview of the key features and results of the experiment is given here. Twenty-two subjects were asked to complete a series of questionnaires concerning the nearness of 22 places in Keele University campus, UK. The 22 places

Computational Structure in Three-Valued Nearness Relations

79

were selected as being well known places on Campus, identified using a preliminary study. Half the subjects (the truth group) were asked in a questionnaire to identify which places were near to a fixed reference place, drawn from the 22 places. The other half of the subjects (the falsity group) were asked to identify which places were not near to the same reference place. So for example, 8 of the 11 people in the truth group considered Keele Hall to be near to the Library, while 2 of the 11 people in the falsity group though that Keele Hall was not near the Library. With a break of at least one day between successive questionnaires, each subject was then asked to complete further questionnaires, one for each reference place, until information about all 22 reference places had been gathered for each subject. All the subjects were Keele University staff with some years’ experience of the Campus and were asked to complete questionnaires without reference to maps. Worboys [25] identifies three different approaches to the analysis of the results of the experiment which allow for the inherently vague nature of nearness: three-valued logic, higher-valued logics and fuzzy set theory. In this paper, we look more closely at the first of these approaches, the use of three-valued logic, although other work currently under way is looking at the other approaches. Table 1. Three-valued summary of aggregated nearness responses 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Place 24 Hour Reception Academic Affairs Barnes Hall Biological Sciences Chancellor’s Building Chapel Chemistry Clock House Computer Science Earth Sciences Health Centre Holly Cross/The Oaks Horwood Hall Keele Hall Lakes Leisure Centre Library Lindsay Hall Observatory Physics Student Union Visual Arts

1
? ? ? ? ? ⊥ ⊥ ⊥ ? ? ⊥ ? ? ⊥ ⊥ ? ⊥ ? ⊥
⊥

2 ?
⊥ ?

? ⊥ ? ? ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ? ⊥ ⊥ ? ? ⊥

3 ? ?
⊥ ? ? ⊥ ⊥ ? ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ? ⊥ ⊥ ? ⊥ ? ⊥

4 ⊥ ? ⊥
? ?
? ?
⊥ ? ⊥ ⊥ ⊥ ⊥ ? ? ⊥
? ?

5 ?
? ?
? ? ⊥ ? ? ⊥ ⊥ ⊥ ⊥ ⊥ ? ? ⊥ ⊥ ? ? ?

6 ? ? ⊥ ? ?
? ⊥ ? ? ⊥ ⊥ ? ? ⊥ ⊥
⊥ ⊥ ?
⊥

7 ⊥ ? ⊥
? ?
⊥ ? ? ⊥ ? ⊥ ⊥ ⊥ ⊥ ? ? ⊥
? ?

8 ⊥ ⊥ ⊥ ? ⊥ ? ?
⊥ ? ⊥ ⊥ ?
? ⊥ ?
⊥ ? ? ⊥

9 ⊥ ? ⊥ ? ? ? ? ⊥
? ⊥ ? ⊥ ⊥ ⊥ ? ? ? ⊥ ? ?


10 ⊥ ? ⊥
? ? ? ? ?
⊥ ⊥ ⊥ ? ⊥ ⊥
? ⊥
? ?

11 ? ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥
⊥
? ? ⊥ ? ⊥ ? ⊥ ? ⊥

12 ⊥ ⊥ ⊥ ? ⊥ ⊥ ? ? ? ? ⊥
⊥ ⊥ ⊥ ? ⊥
⊥ ? ⊥ ?

13 ? ⊥ ⊥ ? ⊥ ? ⊥ ? ⊥ ?
⊥
? ? ⊥ ? ⊥ ? ?
⊥

14 ⊥ ⊥ ⊥ ? ⊥ ? ⊥
⊥ ? ? ⊥


⊥ ? ? ⊥ ⊥ ? ⊥

15 ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ? ⊥ ⊥ ? ⊥


⊥ ? ⊥ ⊥ ⊥ ? ⊥

16 ? ? ? ⊥ ? ? ⊥ ⊥ ? ⊥ ⊥ ⊥ ⊥ ⊥ ⊥
⊥ ⊥ ⊥ ? ⊥ ?

17 ? ? ⊥ ? ?
? ? ⊥ ? ⊥ ⊥ ? ? ⊥ ⊥
? ⊥ ?
⊥

18 ⊥ ? ⊥
⊥ ? ? ? ? ? ⊥ ? ⊥ ? ? ⊥ ?
⊥ ? ⊥ ?

19 ? ⊥ ? ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ? ⊥ ? ⊥ ⊥ ⊥ ⊥ ⊥
⊥ ? ⊥

20 ⊥ ? ⊥
? ?
⊥ ?
⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ? ? ⊥
? ?

21
? ⊥ ? ?
⊥ ? ⊥ ? ? ⊥
? ? ⊥
⊥ ? ?
⊥

22 ⊥ ? ⊥ ? ? ⊥
⊥
? ⊥ ? ⊥ ⊥ ⊥ ? ⊥ ? ⊥ ? ⊥


A χ2 (chi-squared) test was used to determine from the subjects’ responses which places are regarded as near (denoted by ), not near (denoted by ⊥), or undecided (denoted by ?), using a significance level of P=0.001 (probability of 99.9%). Table 1 summarises the aggregated responses of the 22 subjects, so for each column header y we can read down the column all the places x for which it is true, false or undecided to say x is near to y. Taking P as the set of places in our environment, the Campus, table 1 describes a three-valued nearness relation,

80

M. Duckham and M. Worboys

ν on P , where for any two places p, p ∈ P the nearness relations pνp assumes one of the values , ⊥ or ?. The notion of ‘nearness’ is a similarity (or tolerance) relation, where the formal properties of equivalence (reflexivity, symmetry and transitivity) are weakened. Nearness is assumed to be reflexive by definition and the leading diagonal in table 1 contains only  values, although subjects were not actually asked questions of the form “is x near to x”. We can write a three valued reflexive property for the table, as in equation 1 below. ∀x ∈ P. xνx = 

(1)

The results exhibit the asymmetry that has already been noted as feature of human reasoning about nearness. From table 1 the Chapel is judged to be definitely near to Academic Affairs, but Academic Affairs is not definitely near to the Chapel. While the Clock House is definitely not near to Physics, the converse is not the case. Sadalla et al. [21] also observed this asymmetry, and attributed it to the relative prominence or importance of different features, although [25] provides evidence that scale effects may have a significant bearing upon the asymmetry. However, the results do preserve a degree of symmetry. There are no places in table 1 where x is judged to be near to y and y is judged to be not near to x. Consequently, we can write down a weak symmetry property that does hold for all places, given in equation 2 below. ∀x, y ∈ P. xνy =  implies yνx = ⊥

(2)

While as might be expected the relation is not transitive, it does also exhibit a degree of transitivity. The relatively high significance level used was chosen as the lowest tested value of P for which the results preserved weak transitivity, defined in equation 3 below. In short, the study found no places at the 0.001 significance level, for which x was near to y and y was near to z but x was not near to z, so the relation ν satisfies the property in equation 3 ∀x, y, z ∈ P. (xνy =  and yνz = ) implies xνz = ⊥

(3)

In summary, despite being derived from subjective data about which places are understood to be near to each other in Keele University campus, the data in its three valued form does exhibit some formal structure in terms of its reflexivity, weak symmetry and weak transitivity. The following section looks at how further cognitively plausible formal structure can be found in the data using data mining techniques.

4

Data mining analysis

In pursuit of greater integration between the computational and cognitive approaches to reasoning about nearness, this section looks at the use of data mining techniques in combination with the experimental data. The following two

Computational Structure in Three-Valued Nearness Relations

81

sections explore two related computational structures derived using the experimental data: faithful minimal frameworks and decision trees. These two computational structures can in turn be related to two analogous cognitive structures: landmarks and hierarchies. 4.1

Landmarks and faithful minimal frameworks

Landmarks can be defined as “... places whose locations are relatively better known and that serve to define the location of adjacent points” ([21] p. 517). Exactly which features in an environment constitute landmarks for a particular individual may depend upon a range of social, cultural, physical and spatial factors not covered in this experiment. Fisher and Orf [5] do attempt to relate subjective ideas about nearness to certain social characteristics of the experimental subjects, but in this study such effects are not considered. By looking at which places “serve to define the location of adjacent points” it should be possible to derive some candidate sets of landmarks. The existence of landmarks should be expressed as dependencies within the data, where non-landmark places can be discriminated simply with reference to their locations relative to the landmarks. Happily, the task of finding such dependencies can be achieved relatively simply using data mining techniques for finding and eliminating dependencies in data. Let P be the set of 22 reference places. We use the term framework to refer specifically to sets of reference places, A ⊆ P . In table 1, any two places that have exactly the same nearness values with respect to the framework P can be considered indiscernible from each other. For the full table of data, every place can be uniquely identified and no two places have exactly the same relationship to the framework P (no two rows are identical). One question is whether there are any smaller frameworks A ⊂ P for which it is still possible to uniquely identify every place in the data set. Formally we can define an equivalence relation A˜ on a subset of places U ⊆ P for any framework A ⊆ P , as in equation 4 below. A˜ = {(x, x )|x, x ∈ U and ∀y ∈ A. xνy = x νy}

(4)

A nonempty set A ⊆ P is termed a dependent set if for some proper subset A ⊂ A, both A and A have the same equivalence relation (A˜ = A˜ ), otherwise A is termed a minimal set ([16] p. 163). Here the terms dependent and minimal frameworks are used in place of dependent and minimal sets to highlight the fact that the discussion concerns sets of reference places. A desirable property of minimal frameworks is that they should still allow us to distinguish between every place in the data. This property is termed faithfulness and occurs when ∀(x, x ) ∈ A˜ .x = x . Faithful minimal frameworks provide us with the smallest sets of reference places with respect to which it is still possible to distinguish every place in the study. Finding faithful minimal frameworks using an exhaustive search is computationally intensive, but for this relatively small data set entirely possible. For a set of 22 reference places there are just over 4 million possible subset combinations to check. An exhaustive search of the data, achieved using Java code

82

M. Duckham and M. Worboys

written specially for the task, revealed 1951 faithful minimal frameworks. The three smallest faithful minimal frameworks contained only 5 reference places. To illustrate, knowledge of location (in terms of near, not near or undecided) with respect to just five reference places (for example, the framework {24 Hour Reception, Barnes Hall, Library, Lindsay Hall, Visual Arts}) is enough to uniquely identify each of the 22 places. While faithful minimal frameworks are clearly not the same as cognitive reference points or landmarks, they go some way towards closer integration of cognitive and computational approaches to nearness. Purely on the basis of the structure of the data, irrespective of social or physical influences, the faithful minimal frameworks represent good candidate sets of landmarks since they are derived from cognitive experimental data and can “serve to define the location of adjacent points”. 4.2

Information content

Unfortunately, the exhaustive search for faithful minimal frameworks is computational intractable, requiring O(2n ) time to complete, where n is the number of locations. Since the time complexity of the algorithm increases exponentially with the number of locations used, an exhaustive search for faithful minimal frameworks would not be practical for analysis large data sets. However, information theory is commonly applied as a heuristic that can significantly reduce the solution space in data mining, for example in the well known ID3 algorithm (see [20]). Knowledge about nearness to a single reference place will never allow all the places in the study to be distinguished. However, it does allow us to distinguish between some places. Shannon’s information content [22] offers a mechanism for quantifying how much an individual reference place allows us to distinguish between the different places in the study. Information content can provide a measure of how much information is gained by a knowing about nearness to a particular reference place. It is possible to quantify the information content I(p) for particular reference place p ∈ P a with respect to a set of places U ⊆ P using equation 5, after [22], where |X| denotes the cardinality of the set X. n∈{
,?,⊥}

I(r) =




    {x ∈ U |xνr = n} {x ∈ U |xνr = n} − log2 |U | |U |

(5)

Information content is additive, so for a framework A ⊆ P we can calculate the total information content I(A) by simply taking the sum of information content for each element of the set, as in equation 6. I(A) =

a∈A


I(a)

(6)

By selecting those reference places that maximise the information gained about the data set, it should be possible to obtain a reasonably small faithful framework at the same time as searching only a small portion of the data. Again, Java code was written to accomplish this task. There are a range of slightly

Computational Structure in Three-Valued Nearness Relations

83

different ways of using information content (see [16]) and depending on the specific details of the heuristic used, the algorithm applied to the data in table 1 resulted in a faithful dependent (not minimal) framework containing about 10 reference places. If these 10 locations are used to constrain a further exhaustive search for faithful minimal sets, 4 of the 10 locations can be eliminated. Although this is a still a sub-optimal result, resulting in a faithful minimal framework with cardinality 6, the process is much faster than a full exhaustive search. The time complexity of data mining algorithm is O(n) in the worst case, and may often be much better. In practice, when applied to the data for the 22 locations used in this study, the algorithm required less than a second or two to run on a standard PC, compared with more than 3 hours needed to complete an optimised exhaustive search. 4.3

Decision trees and hierarchies

A further use of information content is in the production of decision trees to describe the data set. Decision trees are an intuitive hierarchical structure for representing the data in table 1 in a much more compact, easily accessible form. While decision trees do not possess the same semantics as the hierarchies observed in [23, 12], they are analogous to such hierarchies and consequently represent a potentially useful, cognitively plausible computational structure. By recursively comparing the information content of each attribute in a faithful minimal framework, it is possible to build a decision tree that represents the framework with the most important information rich reference places nearer the root of the tree, and the least important information sparse reference places nearer the leaves. For some set of places U ⊆ P and framework A ⊆ P , we can generate a decision tree using the following algorithm, after [17]. 1. If the set of places U has only one element or if A = ∅, create new leaves in the decision tree with the values u ∈ U . 2. Otherwise for each reference place a ∈ A calculate the information gain associated with that place using the equation I(A) − I(A\{a}). Select the reference place r associated with the largest information gain. 3. Using the selected reference place a ∈ A partition U into disjoint sets Un = {x ∈ U |xνa = n}n∈{
,?,⊥} . A new decision node is then created to represent a and the algorithm is reiterated for each Un = ∅ using A\{a} as the framework. The decision trees produced by this algorithm provide a more compact, intuitive representation of the qualitative nearness information than data tables such as in table 1. An example decision tree for the faithful minimal framework mentioned in §4.1 (1, 24 Hour Reception; 3, Barnes Hall; 17, Library; 18, Lindsay Hall; 22, Visual Arts) is given in figure 1. The decision nodes (white boxes) correspond to decisions about whether a place is near one of the five reference places in the framework. The leaf nodes (black boxes) correspond to the 22 places which can be distinguished. Leaf node 21 (Student Union), for

84

M. Duckham and M. Worboys

Fig. 1. Example nearness decision tree (numbers correspond to table 1)

example, can be uniquely identified as the only place that is not near 22 (Visual Arts), near 17 (Library) and not near 18 (Lindsay Hall). Decision nodes nearer the root of the tree yield greater information gain than those nearer the leaves. It is worth noting that while 5 places are needed to distinguish every place on Campus, most places (18) can be uniquely identified with reference to between just 2 and 4 places.

5

Qualitative vector spaces

The previous section indicates that data mining techniques can go some way towards integrating computational and cognitive aspects of a three-valued nearness relation. In this section we will construct a qualitative vector space (QVS), using as a framework for the space a subset of places in the environmental space (the Campus, in our example). Places in the environmental space will then be capable of representation as qualitative vectors with respect to the framework. The QVS can itself be given a nearness structure. One of the key issues is how well the QVS with its nearness relation represents the environmental space and its nearness relation. We now set up the formal apparatus.

Computational Structure in Three-Valued Nearness Relations

5.1

85

Qualitative coordinates

Let E be a given environmental space, and P be the set of reference places in the space. The experimental data has provided a three-valued nearness relation ν on P . Let A ⊆ P be the framework for the coordinate system. For each place p ∈ P , we can define the qualitative coordinates of p with respect to A, denoted pA , as follows: (7) pA = {(a, pνa)|a ∈ A} We refer to pA as a qualitative vector with respect to A. If the framework A is understood, then the qualitative coordinates of p are just written p. It will also be useful to have a notation for a collection of qualitative vectors.  A as follows: Let Q ⊂ P . Then, define the qualitative vector set Q  A = {qA |q ∈ Q} Q

(8)

Again, if the framework A is understood, then we can drop the subscript. 5.2

Qualitative vector space

Given a set of places P , a three-valued nearness relation ν defined on P , and a framework A ⊆ P , we have defined a qualitative vectors set PA . Assume for the remainder of this section that A is fixed. We can give the qualitative vector set  P a structure by defining on it a nearness relation ν , as follows. Let x, y ∈ Q, then xν y if and only if  ∃a ∈ A. ((xνa =  ∧ yνa = ⊥) ∨ (xνa = ⊥ ∧ yνa = )) (9)  =< P , ν > is termed a qualitative vector space (QVS). The QVS The pair P  has some general properties, detailed below. P Reflexive property: ∀x ∈ P . xν x Symmetric property: ∀x, y ∈ P . xν y = yν x The symmetric property of ν shows up an immediate difference between ν and ν, as ν may well not be symmetric (see §3). The next subsections define properties that relate the closeness of fit between the environmental space and its representation as a QVS. The properties are all expressed in terms of the QVS, but we can equally talk of the properties relating to the frames underlying the QVS. Faithfulness A QVS provides a faithful representation of a nearness space if the representation provides an injective function from the vector set to the coordinate set. The intuition behind faithfulness is that the representation will represent different places with different sets of coordinates, as so different places can be distinguished in the vector space. Thus, the property of faithfulness for

86

M. Duckham and M. Worboys

QVS corresponds to the property of faithfulness in experimental data, introduced in §4. In detail, we have the following definition.  is faithful if and only if the following condition holds: Definition: The QVS P ∀x, y ∈ P. x = y ⇒ x = y

(10)

Adequacy The next properties of a QVS concern its ability to predict nearness and non-nearness relations in the environmental space. A QVS is adequate+ if whenever two vectors are near, then the places that they represent are near. In a similar way, a QVS is adequate− if whenever two vectors are not near, then the places that they represent are not near. We need to remember that the relation ν is Boolean while ν takes values from 3-valued logic. Formally, we have the following definitions.  is adequate + if and only if the following condition Definition: The QVS P holds: ∀x, y ∈ P. xν y ⇒ xνy =  (11)  is adequate − if and only if the following condition holds: Definition: The QVS P ∀x, y ∈ P. ¬xν y ⇒ xνy = ⊥

(12)

We can easily prove that if there are any occurrences of uncertainty in the relation ν, then no QVS can be both adequate+ and adequate− . To see this,  is adequate+ and adequate− . Suppose also that ∃x, y ∈ P suppose that QVS P such that xνy = ?. Then, Equation 11 implies that ¬xν y , while Equation 12 implies that xν y . As ν is a Boolean relation, we have a contradiction. We can weaken the adequacy definitions to make them more in accord with our nearness data.  is weakly adequate + if and only if the following conDefinition: The QVS P dition holds: ∀x, y ∈ P. xν y ⇒ xνy = ⊥ (13)  is weakly adequate − if and only if the following condiDefinition: The QVS P tion holds: ∀x, y ∈ P. ¬xν y ⇒ xνy =  (14)  is weakly adequate if and only if it is both weakly Definition: The QVS P + adequate and weakly adequate− . Note: It is useful to think of faithfulness and adequacy properties applying to the underlying framework of the faithful or adequate QVS. This leads to the following concepts of minimal and maximal frameworks.

Computational Structure in Three-Valued Nearness Relations

87

Minimal and maximal frameworks Clearly, if a framework A is faithful, and if A ⊆ A , then framework A is also faithful. Therefore, minimal faithful sets can be defined as follows. Definition: The framework A is minimal faithful if 1. A is faithful. 2. A ⊂ A implies that A is not faithful. It is also easy to see that if framework A is (weakly) adequate+ , and if A ⊆ A , then framework A is also (weakly) adequate+ . Similarly, if framework A is (weakly) adequate− , and if A ⊇ A , then framework A is also (weakly) adequate+ . It therefore makes sense to define minimal (weakly) adequate− QVS and maximal (weakly) adequate+ QVS in the obvious way. Definition: The framework A is minimal (weakly) adequate+ if 1. A is (weakly) adequate+ . 2. A ⊂ A implies that A is not (weakly) adequate+ . Definition: The framework A is maximal (weakly) adequate− if 1. A is (weakly) adequate− . 2. A ⊃ A implies that A is not (weakly) adequate− . 5.3

Experimental qualitative vector spaces

Having defined the nature and properties of qualitative vector spaces above it is worth reviewing what sort of qualitative vector space the experimental data describes. The definition of faithfulness in equation 10 corresponds to the definition given in §4.1, and the data mining techniques used in §4.1 uncovered 1951 such faithful (minimal) frameworks within the data. More than half the combinations of sets exhibits adequacy in at least one of the forms defined in equations 11–14, and more than 25000 of these frameworks are at the same time weakly adequate+ and weakly adequate− . However, there are no sets which are adequate+ . While it is possible for such sets to exist, they are heavily constrained. A framework A ⊆ P can only be adequate+ if for all places x, y ∈ P where xAν yA then xνy = yνx =  also hold. The tendency against strong symmetry in the environmental space (see §3) militates against such frameworks occurring. There are just three non-trivial frameworks that are adequate− , one of which is maximal ({Barnes hall, Observatory}). Significantly, Barnes Hall and the Observatory are peripheral locations not considered to be near to anywhere else, nor is anywhere considered to be near them (except reflexively themselves). Consequently, the framework A = {Barnes Hall, Observatory} possesses the rare symmetrical property that for all locations x, y ∈ P if xAν yA then xνy = yνx = ⊥ also hold. Finally, a search for the smallest frameworks which are weakly adequate+ , weakly adequate− and faithful yielded just 47 dependent frameworks with a

88

M. Duckham and M. Worboys

range of cardinalities from 10 to 16 places (eg {Academic Affairs, Health Centre, Holly Cross/The Oaks, Keele Hall, Leisure Centre, Library, Lindsay Hall, Physics, Student Union, Visual Arts}). The size of these frameworks suggests that the compromise needed to retain all three properties, weak adequacy+ , weak adequacy − and faithfulness leads to a degree of redundancy in the frameworks, making them relatively large when compared with the minimal faithful frameworks. However, it is still useful to note that there do exist frameworks within the experimental data the exhibit these three properties together. Computationally, finding such sets is relatively efficient. An exhaustive search is easily optimised using the properties of minimal (weak) adequacy+ and minimal faithfulness given above. From the minimally weakly adequate+ property it follows that for some A ⊂ A where A is known not to be weakly adequate+ then A is not weakly adequate+ . A similar optimisation is suggested by minimal faithfulness. While an unoptimised exhaustive search of the data, written using Java, required more than 24 hours to complete, constraining the search using the above optimisations reduced the search time to just over 30 minutes. A further possible optimisation might be based on the minimally weakly adequate− sets, where A ⊃ A and A is known not to be weakly adequate− then A is not weakly adequate. However, in practice the additional complexity introduced into the search by the additional constraint tended to increase rather than decrease the overall search time.

6

Discussion

The analyses presented above indicate how cognitive and computational approaches to qualitative spatial relations like nearness can be more closely integrated. The treatment in this paper provides an approach that is formally well founded and also cognitively plausible, in the sense of its derivation from human subject experimentation. The main contribution of the paper has been to develop the idea of frameworks, which provide as full as possible information about nearness relations within an environmental space. The frameworks form the basis of a QVS, that may exhibit two important formal properties: faithfulness and adequacy. These properties relate directly to important features of human perception of an environmental space. Faithfulness concerns the discrimination of different places, such that if two places are perceived as distinct, then their nearness relations to places in the framework are also distinct. Adequacy relates to the ability to extrapolate knowledge about nearness relations in a framework to the perceived nearness relations for the complete place set. The experimental data was found to exhibit several frameworks that are faithful, weakly adequate+ and weakly adequate− . The property of weak adequacy+ ensures that places that are near with respect to the framework are at least near or undecided in the perceived space. Conversely, the property of weak adequacy− ensures that places that are not near to each other with respect to the framework are not perceived as near to each other. In the light of §5, where it is shown that it is formally impossible for a framework to have all the strongest properties

Computational Structure in Three-Valued Nearness Relations

89

at the same time, this result indicates that the experimental data does retain interesting formal properties. While the results do indicate a cognitively plausible formal model of nearness, it is not yet clear to what extent the results are repeatable in different contexts. Consequently, future experimental work will study how the properties of QVS vary under different conditions. Planned further work includes studies of the effects of using: – different human subjects, such as random surveys of people using Keele University campus. – different environmental spaces, such as environmental spaces at different scales (eg regional, national or international scales) and dimensions (eg road or rail networks). – different qualitative spatial relations, for example qualitative direction, which in turn might serve as a basis for an integrated formal model of qualitative space. In addition to further experimentation with human subjects, future work will be in three directions. First, more formal work both on the underlying theory of qualitative vector spaces is planned and on different representations of the experimental data (for example using higher valued logics or fuzzy set theory). Second, further computational development will be pursued, including refinement of the data mining algorithms for finding suitable reference sets. Finally, finding suitable framework sets is highly desirable in many practical applications, from wayfinding to location specification. Work currently in progress, based on this theoretical work, aims to address the development of a prototype software able to assist in such a wayfinding and location specification scenario.

Acknowledgements The authors are grateful to Peter Jones for helpful discussions on appropriate statistical methods and to Tony Cohn for useful suggestions concerning the construction of decision trees. This research is supported by the UK EPSRC under grant GR/M 56685 “Managing vagueness, uncertainty and granularity in spatial information”.

References 1. A.G. Cohn, A hierarchical representation of qualitative shape based on connection and convexity, Spatial Information Theory: A Theoretical Basis for GIS (A.U. Frank and W. Kuhn, eds.), Lecture Notes in Computer Science, no. 988, SpringerVerlag, Berlin, 1995, pp. 311–326. 2. A.G. Cohn and N.M. Gotts, The ’egg-yolk’ representation of regions with indeterminate boundaries, Geographic Objects with Indeterminate Boundaries (Burrough, P.A. and Frank, A.U., eds.), GIS Data 2, Taylor and Francis, London, 1996.

90

M. Duckham and M. Worboys

3. M.J. Egenhofer and R.D. Franzosa, Point-set topological spatial relations, International Journal of Geographical Information Systems 5 (1991), no. 2, 161–174. 4. P.F. Fisher, Sorites paradox and vague geographies, Fuzzy Sets and Systems 113 (2000), no. 1, 7–18. 5. P.F. Fisher and T.M. Orf, An investigation of the meaning of near and close on a university campus, Computers, Environment and Urban Systems 15 (1991), 23–35. 6. A.U. Frank, Qualitative spatial reasoning about distances and directions in geographic space, Journal of Visual Languages and Computing 3 (1992), 343–371. 7. M. Gahegan, Proximity operators for qualitative spatial reasoning, Spatial Information Theory: A Theoretical Basis for GIS (A.U. Frank and W. Kuhn, eds.), Lecture Notes in Computer Science, no. 988, Springer-Verlag, Berlin, 1995, pp. 31–44. , Experiments using context and significance to enhance the reporting ca8. pabilities of GIS, Spatial Information Theory: A Theoretical Basis for GIS (S.C. Hirtle and A.U. Frank, eds.), Lecture Notes in Computer Science, no. 988, SpringerVerlag, Berlin, 1997, pp. 485–496. 9. D. Hern´ andez, E. Clementini, and P. Di Felice, Qualitative distances, Spatial Information Theory: A Theoretical Basis for GIS (A.U. Frank and W. Kuhn, eds.), Lecture Notes in Computer Science, no. 988, Springer-Verlag, Berlin, 1995, pp. 45– 57. 10. S.C. Hirtle, Representational structures for cognitive spaces: trees, ordered trees and semi-lattices, Spatial Information Theory: A Theoretical Basis for GIS (A.U. Frank and W. Kuhn, eds.), Lecture Notes in Computer Science, no. 988, Springer-Verlag, Berlin, 1995, pp. 327–340. 11. S.C. Hirtle and T. G¨ arling, Heuristic rules for sequential spatial decisions, Geoforum 23 (1992), no. 2, 227–238. 12. S.C. Hirtle and J. Jonides, Evidence of hierarchies in cognitive maps, Memory and Cognition 13 (1985), no. 3, 208–217. 13. S.C. Hirtle and M.F. Mascolo, Effect of semantic clustering on the memory of spatial locations, Journal of Experimental Psychology: Learning, Memory and Cognition 12 (1986), no. 2, 182–189. 14. G.F. Ligozat, Qualitative triangulation for spatial reasoning, Spatial Information Theory: A Theoretical Basis for GIS (A.U. Frank and I. Campari, eds.), Lecture Notes in Computer Science, no. 716, Springer-Verlag, Berlin, 1993, pp. 54–68. 15. D. Medyckyj-Scott and M. Blades, Human spatial cognition: its relevance to the design and use of spatial information systems, Geoforum 23 (1992), no. 2, 215–226. 16. T. Munakata, Fundamentals of the New Artificial Intelligence, Springer-Verlag, 1998. 17. J.R. Quinlan, Learning efficient classification procedures and their application to chess end games, Machine Learning: An Artificial Intelligence Approach (R.S. Michalski, J.G. Carbonell, and T.M. Mitchell, eds.), Morgan Kauffmann, California, 1983, pp. 463–482. 18. V.B. Robinson, Interactive machine acquisition of a fuzzy spatial relation, Computers and Geosciences 16 (1990), no. 6, 857–872. , Individual and multipersonal fuzzy spatial relations acquired using human19. machine interaction, Fuzzy Sets and Systems (2000), no. 113, 133–145. 20. S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall, New Jersey, 1995. 21. E.K. Sadalla, W.J. Burroughs, and L.J. Staplin, Reference points in spatial cognition, Journal of Experimental Psychology: Human Learning and Memory 6 (1980), no. 5, 516–528.

Computational Structure in Three-Valued Nearness Relations

91

22. C.E. Shannon, A mathematical theory of communication, The Bell System Technical Journal 27 (1948), 379–423, 623–656. 23. A. Stevens and P. Coupe, Distortions in judged spatial relations, Cognitive Psychology 10 (1978), no. 4, 422–437. 24. B. Tversky, Distortions in cognitive maps, Geoforum 23 (1992), no. 2, 131–138. 25. M.F. Worboys, Nearness relations in environmental space, International Journal of Geographical Information Science (2001), In press.

Qualitative Spatio-Temporal Continuity Shyamanta M Hazarika

Anthony G Cohn

School of Computing University of Leeds, Leeds LS2 9JT United Kingdom e-mail: smh,agc @comp.leeds.ac.uk


Abstract. We explore different intuitive notions of spatio-temporal continuity and give a formal characterization of continuity for space-time histories. We in-8 topological relations vestigate the types of transitions possible for the under each distinct notion of spatio-temporal continuity and provide a hierarchy of conceptual neighbourhood diagrams. 





Keywords : spatio-temporal reasoning, continuity, space-time, transitions, conceptual neighbourhoods.

1 Introduction In spite of a large amount of work in mereotopological theories as a basis for commonsense reasoning (see [6] for a review), very little work has been done on motion in a qualitative framework. Motion is nevertheless a key notion in our understanding of spatial relations and continuity remains an implicitly assumed notion. We want to formalize the intuitive notion of spatio-temporal (henceforth: s-t) continuity for a qualitative theory of motion 1. We consider regions in space to be temporally extended. This ontological shift is not entirely new (e.g. see [20, 2, 3]). More recently [13, 22] have considered all objects to be occurrents and regarded as s-t histories. Muller [17] has proposed a mereotopological theory of space-time. Muller presents an intuitive notion of s-t continuity and one that is perhaps nearest to a qualitative understanding of motion. Muller [16] purports to define transition relations which allow for a characterization of conceptual neighbourhoods of spatial relations2. Davis [9] has shown Muller’s interpretation of history-based theorems of transition not to be fully adequate. Davis analyses the conditions under which Muller’s theory can be said to be adequate and presents an alternative more comprehensive framework for characterisation of transitions in Muller’s first-order language over histories. However Davis does not re-characterise continuity. Apart from Muller, only Galton [11] addresses what continuity implies for a common-sense theory of motion. Galton characterises continuity as a set of logical constraints on transitions in a temporal framework. However he falls short of an explicit generic characterization of s-t continuity. 1 2

Motion can be seen as some form of spatial change and is used in such an interpretation here. However, as discussed in section 3.3, the notion of continuity proposed by Muller is unable to stop certain weird transitions and needs to be further refined.

D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 92−107, 2001.  Springer-Verlag Berlin Heidelberg 2001

Qualitative Spatio-Temporal Continuity

93

We investigate different intuitive notions of s-t continuity within a s-t framework. First, in section 2 we introduce the notion of a s-t history and present our s-t theory. Section 3 establishes intuitive s-t continuity as temporal continuity without spatial leaps (as previously proposed by [16]). However, such a notion of s-t continuity allows temporal pinching i.e., a history is allowed to disappear and re-appear instantaneously and weird transitions are possible (see fig. 9). To avoid ‘temporal pinching’ we introduce a notion of firm-connectedness and define firm continuity. Finally, in section 4 we investigate the types of transitions possible for the -8 topological relations [4] under each distinct notion of s-t continuity and provide a hierarchy of conceptual neighbourhood diagrams. 









2 Spatio-Temporal Theory Our underlying spatial entities are extended regions of space-time [13, 17]. The motivation for using such an ontology lies in two main reasons: The ontological distinction between continuants (objects of everyday world, which persists through time and cannot have temporal parts) and occurrents (states or events, bounded in time and which can have temporal parts) leads to some problems when considering change, specifically the ‘identity criteria’ – defining criteria to identify objects whose parts or properties are changing. This can even entail paradoxes [21]. The problem disappears when considering all material objects as occurrents, since properties become relative to temporal parts of these objects (see [21, page 117-127]). Such an ontology allows for simple reformulation of s-t facts in a rather intuitive way [17, page 63]. Further, it would allow dealing with spatial information directly in terms of objects in line with National Imagery and Mapping Agency (NIMA)’s recently announced vision for Integrated Information Libraries [15]. Space-Time Histories Following [13], s-t regions traced by objects over time are termed s-t histories. Fig 1 shows the s-t history for a 2-D object. In a n-D space, the s-t history is a n+1 dimensional volume. The object at any time is a temporal slice of its s-t history. time

History

Temporal Slice

space

Fig. 1. A spatio-temporal history is a n+1 dimensional volume in a n-D space.

94

S.M. Hazarika and A.G. Cohn

One important question about such s-t histories is whether it is possible to have a zero extent along the temporal dimension, i.e., is it possible to have instantaneous spatial objects? This is analogous to asking if the surface of a cube is a spatial object in our standard 3-D ontology [14, page 6]. The more pertinent question here is the spatial analog of the above: what does it mean for a n+1 dimensional s-t history to toggle instantaneously into n-1 dimensional spatial extent? i.e., for histories to disappear and reappear again instantaneously at the same spatial location. We term this as temporal pinching and discuss its implication for continuity under section 3.3. Note that following previous work within the group [19, 12, 4], we do not wish to admit lower dimensional entities – e.g. in our work on spatial mereotopology all regions were of the same dimension and we did not consider boundaries as spatial entities [19, 7]. Here too, we do not admit lower dimensional entities such as temporal points into our ontology for the same reasons as argued in [19, 12, 4]. Thus s-t histories may pinch to a spatial point at a temporal point, but we do not allow explicit reference to either of these points. However, in [5] we have developed descriptive apparatus to allow us to describe instantaneous transitions and histories which pinch to a spatial point instantaneously. 2.1 Connection Relations We will use three connection relations and for spatio-temporal, spatial and temporal connection respectively. Their intended interpretation is as shown3 in fig. 2. 

time















time

time

y



y

y

x

x

x

space

space

a.

space c.

b.

Fig. 2. a. Spatial b. Temporal and c. Spatio-Temporal connection.

Intuitively, spatial connection for s-t entities is the connection of their spatial projection. As shown in Fig. 2a, connection under spatial projection is interpreted along the temporal axis i.e., connection on projection to an infinitesimally thin ‘temporal slice’ : is spaat right angles to the temporal axis. Spatial connection is written as tially connected to . Here and are s-t regions whose closures have a spatial point in common, though not necessarily simultaneously. Temporal connection is written as : is temporally connected to . Here and are s-t regions whose closures have a temporal point in common, though not necessarily at the same place (Fig. 2b). The 







3



























Space is shown as 1D in fig. 2 and subsequent illustrations, but this is simply for ease of drawing. The defined concepts are applicable to 2D and other higher dimensional space.

Qualitative Spatio-Temporal Continuity

95

spatio-temporal connection primitive, : is spatio-temporally connected to is true just in case the closures of and at least share a s-t point (Fig. 2c). The axiomatisation of these connection relations are identical and follows [4]. Note that in this theory the closure and the interior cannot be distinguished. We have the following axioms: 





A1. A2. A3. 





!

where











-

"

0















.







2

4



2

 

 "







&

7





4

 "



(





+



(



 9

2.2 Mereo-Topological Relations From the topological connection relation parthood; : is a part of . :

D1.





 :







def ;



 

we define the mereological relation of 





!

"



 "



 

 "



(

The parthood relation is used to define proper-part , overlap and discreteness . and i.e., disconnected, external connected, partial overlap, equal, tangential proper-part and non-tangential proper-part respectively can also be defined. These relations, along with the inverses for the last two viz. and constitute the jointly exhaustive and pairwise disjoint (JEPD) relations of -8 (see [4] for definitions). We further introduce the following existential axioms4 . Axiom A4 ensures every region has a non-tangential proper-part. In A5 the individual is noted and represents the sum, where as in A6 it is noted and represents the difference. :

B

G

:

:

I



(





B

H







G





:

:

I

E







:

?





E

F





G

:

:



H

G

:

:

:

(



?









"



A4. A5. A6.

 !

Q

Q

T

"



:

!

H

G

R









(





:







:

R





Q "



K



(

(



& "

N



!

Y





 R

:





S

Y

 



[



 R

B



x

(

(



Y



(

& :



Y

"

(

(

x y

a.

y

b.

Fig. 3. a. Firm and b. Non-Firm connection between two entities ]

and . ^

We introduce the notion of firm connection. Figure 3 illustrates ‘firm’ connection and ‘non-firm’ connection. A firm-connection in n-D space is defined as a connection 4

We only need the ‘ ’ version of the axioms since the ‘ ’ and ‘ ’ versions are implied by A10. Also note that in [4] we claim that A4 is in fact a theorem (given A1-A3, A5 and A6), subsequently verified (in an algebraic setting) [10]. _

`

`

_

a

96

S.M. Hazarika and A.G. Cohn

wherein an n-D worm can pass through the connection without becoming visible to the exterior. In other words, for two regions to be firmly-connected, a direct conduit exists between the two [8]. To define firm-connection, we first define one-pieceness (i.e., s-t connectedness). A s-t region is spatio-temporally one-piece ( ) just in case all parts of are connected. Similarly we can define temporal connectedness: s-t region is temporally one-piece just in case all parts of are temporally connected. We can also define spatial connectedness: s-t region is spatially one-piece just in case connected. D4 states that a connection between two entities and all parts of are is a firm-connection just in case some one-piece part of ( ) and some onepiece part of ( ) is interior connected ( ). We have the following definitions: 









?



H





























D2. D3. D4. 

c

?





H

H

?



 H









def ;



?



?

 H

H

n





 !



def ;



c

 !

def ;

 "

Q

k

R

+

H





G

:

k p

 :

K

:









 R

( "



Q

 :



"

:

[





?



H

K



?

H







(

( "



 p



 



[

H





?

[ "

 H

H



G

[ R

:

:



?





 "

 H



[ 

[ p

c

H

?

H



?



H



( "





m

R

K p

(

m

2.3 Temporal Relations For clarity we will sometimes write the temporal relations as infix operators. Thereis also written as . Further, at times we will write fore temporal connection and as and respectively. In order to introduce a s-t interpretation we must capture a notion of temporal order. For temporal order we write the closure of strictly precedes the closure of in time. Axiom A7 establishes that temporal connection and temporal order are incompatible. Also temporal order is anti-symmetric (A8). Axiom A9 establishes the composition of temporal connection and temporal order. t



:









:

?







E

F

















u







w







s



s

;







y





A7. A8. A9.

t



s





s







T



s

 T



s



t



s



[



s

[ "

s "

( Y





s Y

(

Allen [1], and even before him Nicod [18], pointed out that if time is totally ordered then there are 13 JEPD relations (which can be defined in terms of meets) in which one one-piece interval can stand to another. We list here the ones that will be required for our subsequent discussion5 . Fig. 4 shows the different temporal relations. D5 is the definition for ‘meets’ which is a specialization of . D6 is the definition for a temporal interval ending with another interval . D7 is for a temporal interval starting with another interval . D8 defines interval to be between two distinct intervals and – this 3 place relation is not an ‘Allen’ relation. E















D5. D6. D7. D8. 5



|









‚

‰

†



E



;

def



;

def

‚



def ;

Š "

(



;





[

R

!

R



!

def



|





|

p

}



& R

| R

Q T



&



[

p

p

|

|

}



"

u



[ p



u



[ p



s p

}

(

( R

| R





"

(

(

Note that unlike Allen we want our definitions to work for multi-piece intervals. This is achieved through the second conjunct of D5.

Qualitative Spatio-Temporal Continuity

time

r

w

v;w r

r

u

v

v u

y

y

r

w

u

v

y

x

x

97

s

s

z

t u

y

v z

x

space Fig. 4. Temporal Relations over spatio-temporal regions

2.4 Spatio-Temporal Relations A s-t connection implies a spatial as well as a temporal connection (Axiom A10), though note that the converse is not necessarily true. Models must not be s-t alone, so s-t connection ( ) needs to be different from temporal as well as spatial connection. We have the following axioms: 



A10. A11. A12.









Q



Q



Q



Q





 







 









[





[

T



 

[ T









 

 





(

(





(

To define relations between s-t regions that may vary through time, we introduce the notion of a ‘temporal slice’, i.e., the maximal component part corresponding to a certain time extent [16]. D9. G

 Œ



def ;

:







[

Henceforth the notation when it exists (i.e., when

!

"

Ž



u

Y



:





"



[ "

u



(

denotes the part of )6 .

 :





 "

(

corresponding to the lifetime of 

Y

3 Component Connection Analysis In space-time the notion of continuity should implicitly capture the intuitive notion of motion. Nevertheless, weaker notions of s-t continuity can be defined and we will explore these below (note that these bear a strong relationship to various notions of connection in [8]). One way to characterize distinct notions of s-t continuity is based on whether all or some but not all (henceforth: sbna)7 parts of a history during an interval are connected to all or sbna parts of the history during another distinct interval. 6

The notation is purely syntactic sugar: for any atom . by We use to denote sbna which is purely syntactic sugar: . by 

’

“

”

”

7

]

“

™

¥

—

š

]

¤

]

^

›

£

]

›

¥

]

¦

¤

œ



ž

–

 

’

“

”

”

]

”

”

” 

‘

—

”

‘

–

could equivalently be replaced

–

—

£

]

¤

could equivalently be replaced

98

S.M. Hazarika and A.G. Cohn

Prior to such an analysis we differentiate between different types of relativised components8 of a s-t history. Fig 5 illustrates these components. E.g. the history has 3 spatio-temporal, 2 spatial and 1 temporal components during interval . 

"

time

time y

y

time

y

y

y

y

y

y

y

x x

x

x

z

x

z

space

z

x

space

space

a.

b.

c.

Fig. 5. a. Spatio-Temporal b. Spatial and c. Temporal components of a history ^

during . §

Even though there are three s-t components, two of them are connected which results in a single spatial component, making the total number of spatial components two. There being no temporal gaps in history during , has but a single temporal component during interval . We make the following definitions: 









"

"

DF 1 Relativised Spatio-Temporal, Spatial and Temporal Components A s-t component of a history during an interval is a maximal s-t region of . A spatial (resp. temporal) component of a history during an interval is a maximal spatial (resp. temporal) projection of the s-t region of . y







"

Ž¨







"

Ž¨

Below (D10) we give the object level definition for an -component based on DF 1. -

D10. 

©

ª

7

 



; "

def Q

R

k R

+

Ž¨ (

[

k

!

where -

.

0

2

4



2

7



4

9

Y



?

H



 

?

H

[



:

Y

 

[ :

[ R



Y

R

[ :

 

( Y

 Y

+ 



(

m

m

.

3.1 Multiple Component Analysis If a history contains multiple components, then we can consider how these relate to each other over time. In [5] we provided a set of definitions for various notions of continuity. However there was no formal analysis from which these definitions were recovered. Here we try to provide a more systematic analysis. Our starting point will be a qualitative analysis of the connection relationships between temporally adjacent parts of histories, which intuitively form the basis for any notion of continuity. We provide the following definitions based on the connection between components of a s-t history at a given interval to components at a later interval , where . "

8

}

"



"

|

}

"



These components are relativised to a temporal interval and are thus not necessarily topological components of the full history. When no confusion arises, we may drop the term ‘relativised’.

Qualitative Spatio-Temporal Continuity

99

DF 2 -Complete, -Hitting, -Covering and -Partial Connection Connection between parts of a history during to parts during is (resp. ) over -components if all -components of are connected to where all (resp. sbna) -components of Connection between parts of a history during to parts during is (resp. ) over -components if sbna -components of are connected to all (resp. sbna) -components of where 

­



­



­



y

­







°

­

I

4

4

I

±

²

"

-

"

}

¨

Ž

-

¨





Ž

-

µ

.

0

2

4



2

7



4





­

©

ª

7

®

¯

4

¯

­

³



9

´







:

­

¹

¸

4

I

¹

®

"

-

"

}

µ

Ž

¨

.

0

2

4



2

7



4



­



¨

-





Ž

-

©

·

¯

¸

I

±

²

­

³



9

´

From D11 to D14 we give the object level definitions for the different types of connection9 (defined over -components) based on DF2. 

­

-

D11.

-

 ­



©

ª

7

®

¯

4

 ¯



"

 }

( "

def ;





 !

 !

}

k 



©

ª

 7





 "

}

}

k

D12. D13.

-

 ­

 ­

°

I

4

4

I

±

 ²



"

 }

( "

def ;







©

·

¯

¸

I

±

 ²



"

 }

( "

 !

 !

}

def ;





!

k 

 º

 º

©

ª

 !

}

 7

k

k









ª

k





©



­

:

¹

¸

4

I

¹

®

 



"

}



"



(

def ;

!

 º

}

!

 º

k 



©

ª

7

 

 }

"

ª

©

[ }





 "



 7





[ "



 



m

}



­

m



 





­

m

}

m



}



D14.

 7

[ "

}

ª



}

 7

©

 "

}

©



ª



 7

©





ª





m "



7



"

[ 

 



 



­



­

m

}

 }



 

m

For notational convenience, let denote the above connections, where ranges over the types of components and ranges over the connection-types. For subsequent discussion in the paper is treated as a column vector (as below) and would refer to the element of . 

½

-

­

µ





½

½

­

­

¾



¿

À

Â

-

½

ÇÈ

­

Å

È



Å

Ã

ÄÅ

Æ



Å

É



Å



Ö

×

×

Ä



Ê

Ë

a

Ì

Í

`

Í

Ä

Î

Ï

`

`

Ï

Ð

Ñ

Ø

Ä



Ê

Ò

Í

Ó

Ï

Ð

Ñ

Ä

Ô

Õ

Ó

`

Ï

Õ

Ì

Any connection over -components can now be referred to by a pair (comprising of an index to refer to the type of connection from and the instantiation for ), for e.g . -





½

­

µ

­



+

½













©

·

¯

¸

I

±

²



Ù

Component Relation Label ( ) In order to allow further expressivity into our language we introduce component relation labels. A component relation label is of the form and specifies the connection relationship over -components of a history between and . D15 is a schematic definition for a component relation label. Ú



Ü

Þ

Ý

¾

ß

à



á



"

}



"



(





D15.

Û

"

-

"

}



 

Þ



Þ

 Ý

¾

ß

à



"

}



"



( 

;

á

def k

 ½



"

}



"



(

[

 ½

¾ à



"





"

}

(

m

á

where . Theoretically there are 144 cases possible. Of these only 58 are physically realizable for different components over and (see Table 1). â



ã

.

0

2

4



2

7



4

9

"

}

"



Temporal Transitivity Any of the connection relationships over a given class of components is said to be continuous iff it is temporally transitive. Our hypothesis is that we can base all our notions of continuity for s-t histories on this notion, i.e., if a history is continuous in some sense, then there must be a form of connectedness that is preserved over time. We use temporal transitivity to capture this notion of connectedness. A component relation label defined for a history is said to be temporally transitive 

9

These connection relations bears a strong relationship to the 

å

connection relation in [8].

100

S.M. Hazarika and A.G. Cohn

just in case for consecutive intervals and , the holding between and and between and implies that the same holds between and .We will to denote this notion of a transitive introduce the notation for a history . We will treat this as syntactic sugar; thus can be replaced by "

"



}



"



Ù

"





Ý

"

Ù

"

"

K

}

"

(



}

"



Ù

"



Þ

G

¸

¹

±

2

I

4

I

·

¯

Ý





¾

ß

à

(





Ý

á



Þ



G

¸

¹

±

2

I

4

I

·

¯

Ý



¾

Äë

“

—

§

æ

ç

§

è

ç

§ å

é

é

ì

í

î

ï Äë

“ ê

^

ç

§

æ

ç

§

è

–

›

ì

í

î

ï Äë

“ ê

^

ç

§

è

ç

§

– å

›

§

è

ð

“

§

æ

ñ

§

– å

ò

 

ì

í

î

ß

à



(

á

ï

“ ê

^

ç

§

æ

ó

§

è

ç

§ å

–

ò

–

Table 1 shows the component relation labels along with their transitive and nontransitive character. Note that any non-transitive connection label cannot be globally continuous and therefore cannot be any form of s-t continuity.

Labels

Æ

’

í Ä

ö ö

ê



æ 

æ

í Ä

ö

ö

ê 

è



Ä

å

í

ö ö

ø

ê

æ



æ

í Ä

ö

ö

ø

ê



è å

í Ä

ö

ê 

æ



Ä

æ

í

ö

ê



 å

æ

í Ä

ö

ø ö

ê  æ

æ

í Ä

ö

ø

ö

ê

è



Ä

å

í

ö

ê



æ 

æ

í Ä

ö

ê 

æ

 å

í Ä

ö

ø

ê

æ



Ä

æ

í

ö

ø

ê

æ

 å

í Ä

ö

ø

ê è



æ

í Ä

ö

ø

ê

è



Ä

å

í

ö

ø

ê



æ

å

í Ä

ö

ø

ê

 å

å

í Ä

ö

ø

ê

÷ 

Ä

æ

í

ö

ø

ê

÷ 

å

í Ä

ö

ø

ê



æ

æ

í Ä

ö

ø

ê



æ å

Ä í

ö

ø

ê

è



æ

í Ä

ö

ø

ê 

è å

í Ä

ö

ø

ê



æ å

Ä í

ö

ø

ê

 å

å

í Ä

ö

ø

ê 

æ

÷

í Ä

ö

ø

ê



÷ å

Ä

í

ö

ø

ö

ø

ê

æ

æ

í Ä

ö

ø

ö

ø

ê è å

í Ä

ê



æ 

æ

Æ

_

T T T NT T NT T NT T T T T NT NT T NT T NT T T NT

a

’

Æ

`

’

T T T T T T T NT NT T

_

`

T T T NT T T T NT T T T T T T T NT NT NT T T T T T NT NT NT T NT T

ô

õ

Labels

P1 P2 P1

Æ

’

í Ä

ö ö

ê





è

å

í Ä

ö

ö

ê 

÷



Ä

÷

í

ö ö

ø

ö

ø

ê



è å

í Ä

ö

ê



P1 P4 P1

÷

÷

í Ä

ö

ê 

è



÷

Ä



æ



æ

í

ö

ê

í Ä

ö

ø ö

ê 

è



÷

å

í Ä

ö

ø

ö

ê

÷

P1 P2 P1

Ä í

ö

ê



æ 

è



÷

í Ä

ö

ê 

æ

í Ä

ö

ø

ê

æ

Ä



è



÷



è



÷



è



÷



è



÷

í

ö

ø

ê

æ

P2

í Ä

ö

ø

ê è

í Ä

ö

ø

ê

è

P4

Ä í

ö

ø

ê

å

í Ä

ö

ø

ê

å

í Ä

ö

ø

ê

÷

Ä í

ö

ø

ê

÷

P1

í Ä

ö

ø

ö

ø

ö

ø

ê



è

æ

í Ä

ê



P4

÷

æ

Ä í

ê

è



è

í Ä

ö

ø

ö

ø

ö

ø

ö

ø

ö

ø

ê 

P2

÷

è

í Ä

ê



è å

Ä í

ê



÷ å

í Ä

ê 

è

÷

í Ä

ê



÷

÷

Ä

í

ö

ø

ö

ø

ö

ø

ê

è å

í Ä

ö

ø

ê ÷

÷

í Ä

ê



÷ 

÷

Æ

_

NT T NT NT T T NT NT NT T T NT T NT NT

a

’

Æ

`

’

_

NT -

T T NT NT T T NT NT T T NT T NT T T T NT T NT T NT T T T NT T NT NT NT

`

ô

õ

P4 P3

P2 P3

P4 P3

Table 1. Transitivity Table for component relation labels under Spatial ( ), Temporal ) and Spatio-Temporal ( ) Components. Here NT = Non Transitive, T = Transitive ( is an equivalence class column. Labels with identical and - = Undefined. The column marked numbers in this column fall into a single equivalence class (established through the Proposition of the same number). ö





ö

Ê

Ë

a





Ê

Ë

a



ô

õ

Ê

Ë

a

ø

Qualitative Spatio-Temporal Continuity

101

The transitive component relation labels in Table 1 clearly fall into three distinct groups. There are labels that are transitive under all three classes of components. Others are transitive under either two or a single class of components. It turns out that in most cases, having the same T / NT pattern means that the labels characterise the same possible class of histories. Propositions 1 and 4 below show this is the case. In Propositions 2 and 3 we show that for one particular pattern there are actually two separate equivalence classes. Prop 1 Component relation labels: and are equivalent (i.e., characterise identical classes of histories). 



Ý







}



ß





Ý





}



}



ß





Ý





}



}



ß



Ý





}



}



ß





Ý



}



}

 ß



Ý

 }

 }

ß

}



Ý



}



ß



}

Proof: For each of these labels, every component of a history in is connected with every component in . From DF 2 this is true for labels and label where . forces and simultaneously for all components and Index pair therefore all components connect spatio-temporally. This is true for both labels and . The labels are equivalent. 



Ž

¨





³





Ý

Ž

¨



Ý

´





}



ß





}

}



ß



}



Ý

µ





}

ß

­

.

0

2

4



2

7



4

9

2

}

7

ù

y

4

ù













Ý





}



ß

}



Ý

ú



}

ß





}

Prop 2 Component relation labels equivalent.

 

Ý







 ß





Ý 

Ù

}

 ß



and



Ý



Ù

}

 ß



Ý

Ù 



ß

}

are



Ý









ß

}

Proof: For each of these labels, a spatially disjoint s-t component appears and carries on forward as part of original history. For and , all components are spatio-temporally connected to sbna components in the forward direction. For sbna components are spatio-temporally connected to all components in the reverse with , there is a spatially disdirection. Therefore when moving from to joint s-t component in . Labels and are equivalent. In and , index forces and simultaneously leading to s-t connection. In either case it falls in one of or which are shown to be equivalent in the preceding paragraph. The labels are therefore equivalent. 



Ý

Ý







ß





Ù







ß



}



Ý



"

"

}



"













ß



"

}

ß





Ù





Ý

Ý

Ù









ß



}

}



ß



Ù



Ý

Ý



|

"





Ý



}





ß



4



}

}

ß





ù











Ù





Ý

Ý



}



ß



Ù







ß



}

ú

Prop 3 Component relation labels



and



Ý 





û

ß

Ý

}

}

 ß



û

are equivalent.



Ý





û ß





û

Proof: For each of these labels there exists equi-temporal components spatially disjoint throughout . For this is trivially true. For labels and this is true in one direction and index forces that there are no temporal gaps. The labels are equivalent. As noted above, the T / NT pattern in Table 1 for the labels in both Proposition 2 and Proposition 3 are identical (T - T). However these labels do not form a single equivalence class; rather they form two separate equivalence classes. Fig. 7 makes the difference between the two equivalence classes clear pictorially. 





Ý



Ý





û

ß





Ý



û

4



û



ß



}

}

ß





û

ù

ú

Prop 4 Component relation labels equivalent.





Ý







Ù ß

}



Ý



}

ß



 



Ý 





Ù ß





Ý

}

ß





and



Ý





Ù ß

}

are

Proof: Reasoning along same lines as that for Proposition 2 above. (Hint: For each of these labels a spatially disjoint s-t component disappears on going from to with ). "

"

}

|

"



ú

}

"



102

S.M. Hazarika and A.G. Cohn

3.2 Ú

Û

s and Space-Time Continuity Ü

over a given type of We define the notion of continuity based on transitivity of a component. This is the general case of continuity and we term it -continuity. 



Ý

-

DF 3 -continuity A s-t history is -continuous if component relation label is tran10 . sitive over -components where y

-



-

D16.

-

-

.

0

2

4



2

7



4

9

ý

á

þ

¾

ý

á

þ

û

¾



Þ

Þ



?

H

G

 

def ;

G

¸

¹

±

2

I

4

I

·

¯

 

Ý

¾ ü

ß

à

(

á

ü } ý

à

 ÿ



 ý



ý



Fig. 6 illustrates the concept of -continuity of a s-t history based on transitivity of labels. As shown in the figure, the history is temporally continuous as the label is transitive. This is because the same label between and is maintained between and . -



Ý



Ž

¨

³



Ž

¨

´

¨

Ž



}

ß





}

Ž

¨

³

¨

´



time

y

z3 z2

z1 z2

z1

space

Fig. 6. Transitivity of component relation label

í



ö ö

ê 

æ



æ

.

Having established the equivalence of several groups of s in Propositions 1 to 4, we now define a corresponding notion of continuity for each of these. In order to characterize the different types of s-t continuity, we define strong and weak s-t continuity. Further, we define spatial and temporal continuity. 



Ý

DF 4 Strong and Weak Spatio-Temporal continuity A s-t history is strongly (resp. weakly) spatio-temporally continuous if any component relation label with complete (resp. non-complete) s-t connection over s-t components is transitive. y



Definitions D17 and D18 are the object level definitions for strong and weak s-t continuity established through DF 4. D17. D18.

Œ

4

¸



?

H

 G





def ;

G

¸

¹

±



¾





?

H

G







;

def

2

ý

I

á

þ

4

I

·

¯









 Ý

}

ß







(

}

û

k G

¸

¹

±

2

I

4

I

·

¯









Ý

 

¾

 ß



(

m

á

ü ¾

ý

á

þ



We can also imagine a history to be spatially continuous, in the sense that its components spatially overlap, but there may be temporal gaps. Dually, we may allow for the possibility that no component endures for the entire history i.e., there may be spatial jumps providing there are no temporal gaps. We give the following definitions : ì

10

D16 is a definition schema for continuity and disjunction of the 144 cases referred to above.



ï

÷

ì



ï

ë

 

æ



î





ö



ö



ø





is syntactic sugar for a

Qualitative Spatio-Temporal Continuity

103

DF 5 Spatial and Temporal continuity A s-t history is spatially (resp. temporally) continuous if any component relation label with complete spatial (resp. temporal) connection over s-t components is transitive. y

D19. D20.





?



H



G



def ;

G

¸

¹

±

2

I

4

I

·

¯

?

H

G



G

¸

¹

±

2

I

4

I

·

¯

 



def ;



Ý







 }

 ß



(

}



Ý

 

}

ß



(

}

Labels in Proposition 1 are shown to be equivalent to which is transitive over s-t components. Therefore, component relation labels in Proposition 1 refer to strong s-t continuity. Labels shown equivalent by Proposition 3 are for multi-component histories where all components are equi-temporal. Groups of component labels shown to be equivalent by Proposition 2 and Proposition 4 represent weak spatio-temporally continuous histories. and are representative labels for spatial and temporal continuity. The validity of these statements follows from the definitions. 







Ý



}

ß





}



Ý

Ý





}



ß





}

}



ß

}

Forward Direction C-Hitting

C-Covering

C-Partial

C-Partial

C-Covering

C-Hitting

Reverse Direction

C-Complete

C-Complete

Fig. 7. Transitivity Chart for component relation label . The non empty entries in each row from top to bottom correspond to those labels identified in Proposition 1, 4, 2 and 3 respectively. ö

ì

í

ï



ö

ö

ê





Fig. 7 shows the different types of continuity for a s-t history based on the component relation label . Based on Proposition 2 and Proposition 4, definition D18 can be further refined to capture the two distinct notions, viz. components may come into existence (a kind of multiplication), but once they start, they carry on until the end as do the original component(s) and the natural dual to this, where all components start simultaneously but some may finish early (a kind of collapse). We give the following definitions: 







Ý

D21. D22.

¾

ß





á







©



®

®





?

H

?



H

G

 G









;

;

def def

G

¸

¹

±

2

I

4

I

·

¯



Ý

G

¸

¹

±

2

I

4

I

·

¯

 















ß







Ù ß





(

Ù

Ý



(



Finally, we state the following proposition: Prop 5 Strong s-t continuity is spatial and temporal continuity simultaneously. is transitive. Therefore from definition DF 4, is a label for Proof: strong s-t continuity. From Proposition 1 above, the label is equivalent to and 







Ý









Ý

}

ß





}

}

ß





}



Ý





}

ß



}

104

S.M. Hazarika and A.G. Cohn

, which is spatial and temporal connection simultaneously. Where ever defined, and is transitive. Therefore from definition DF 5, the labels are spatial and temporal continuity respectively. Thus strong s-t continuity is spatial and temporal continuity simultaneously. 

Ý



}



ß





}





Ý



Ý





}

ß





}

}

ß



}

ú

3.3 Strong Spatio-Temporal Continuity We have formally characterised strong s-t continuity as ‘temporal’ as well as ‘spatial’ continuity simultaneously. Intuitively, this notion is same as Muller’s definition of continuity [16]: any s-t region is defined as qualitatively continuous just in case it is temporally self-connected and it doesn’t make any spatial leaps11 . D23.

t



?

H

G













def ;

Y



?

H

[ Y

 !

!

R

G

 Œ

[ Y



s

[ R

:





R

Y

(

 





 R

(

À

However this definition of continuity is unable to stop histories from ‘temporal pinching’, i.e., exclude histories that disappear and reappear again instantaneously at the same spatial location. With temporal pinching, we have weird transitions possible: transitions that do not adhere to the conceptual neighbourhood diagrams for binary topological relations such as -8 (see fig. 9). In order to enforce a stronger notion of s-t continuity for histories we disallow temporal pinching and introduce the notion of firm-continuity. A non-pinched continuous s-t history is firmly continuous. 





time

time w w z space

space FCONT w

CONT w ;

FCONT w

a.

b.

Fig. 8. a. Firm-continuity and b. A non-firm history with ‘temporal pinching’ at the end of . §

Figure 8a shows a firm-connected history , while Figure 8b is for a history with “temporal pinching”. D24 is the definition of a non-pinched history and D25 defines firm-continuity. Y

Y

D24. D25. H

n

:



?

def ;

Y

H

G

Q T

Q



def ;

Y





k



?

:

H





[

G

Y











:









[ Y

[ Y

H

:



|



[ T

n



?

H









m

Y

If a history contains multiple components, we are interested in how these components relate to each other and thus produce different notions of continuity. D17 establishes the strongest notion of s-t continuity just in case all s-t components are 

11





Note that Muller uses a slightly different definition of one-piece/connectedness using closures. where is defined as the closure of def . His mereotopological theory follows [3] in having topological functions and interpreted as and share a point. Æ

'



"

$

ž

]

œ

¦

¥

]

æ

ç

]

è

“

ž

]

æ

ó

]

è

›

¦

“

%

]

æ

&

%

]

è

–

–

%

]



]

^

]

^

Qualitative Spatio-Temporal Continuity

105

connected to all other s-t components between and . From DF1 this would mean that all the components are equi-temporal. Therefore we have the following theorem, showing the equivalence of to the notion of strong continuity in [5]. Ž

¨

Œ

Th1. Œ

4

¸



?

H

 G





; !



k

4



¸



©

?

ª

H

 7



G









³

¨

Ž

´









;





m

The other notions of continuity (D18 through D22) established through the component connection analysis can be shown to be equivalent to the distinct intuitive notions of continuity defined in [5]. Each of these notions of continuity can be further refined to exclude temporal pinchings and also spatial leaps within a component. The strongest notion of s-t continuity will be both and . n

D26. Œ

4

¸

n



?

H

 G





def ;

Œ

4

¸



?

H



?

 G



H



G

Œ

[ n



?

4

H

¸



?

H



G



 G

Henceforth, when considering the spatial relationship over time between pairs of s-t regions, for convenience of reference the above doctrine of s-t continuity will be labelled -0. Allowing temporal pinching weakens -0 to -1 and -2 depending on whether temporal pinching of one or both histories is allowed respectively. Here, we will take the opportunity to introduce a corresponding notion for weak s-t continuity, which we label as -0. As with strong s-t continuity, temporal pinching would weaken this notion. Thus -1 and -2 denote weak s-t continuity with temporal pinching being allowed for one or both histories respectively. (

)

(

(

)

(

)

(

)

*

(

*

(

*

4 Continuous Transitions With -0, the intuitive transitions between histories hold. The -8 conceptual neighbourhood diagram [4] is one such transition network. Under space-time interpretations for relations and with ‘temporal pinching’, we can have a number of weird transitions. Figure 9a shows the transition from to for ‘temporal pinching’ of history . In figure 9b both the histories and undergo ‘temporal pinching’ and have a transition from to (which is not possible in the normal -8 transition network [4]). (

)













E







E



E

G

:

:



F







time

time

y

TPP

EQ x

EC

EC

a.

x

space

Fig. 9. Transition from a.

y space

ô

b.



to ™

Ô

Ô

and b. ô

to 

ô

,

.

The -8 spatial transition networks for -0, -1 and -2 are shown in figure 10. Allowing pinching of a single history means a direct transition between and or is possible. If pinching of both histories is allowed we have a direct transition between and . Note that the diagram for -2 differs slightly from 





(

)

(

)

(

)

E

G

:

:

G

:

:

I

E



E

F

(

)



106

S.M. Hazarika and A.G. Cohn

the conceptual neighbourhood given in fig. 10 of [9], e.g. his figure has a direct link from to . This depends on the interpretation of the spatial relationship holding when regions pinch to a spatial point. Davis considers the normalised (regularized) spatial cross section and isolated points will thus disappear, leading to the introduction of yet further links. We could also take this approach, in which case his fig. 10 and our diagram for -2 should be identical. B



(

G

:

:

)

TPP

DC

EC

NTPP

EQ

PO

NTPPi

TPPi

Fig. 10. Transition graph for -0, -1 and -2. Transitions for -0 are shown as single arcs; additional links for -1 are double arcs and for -2 are triple arcs. .

.

/

.

/

.

/

.

/

.

/

/

The transition network for is as in figure 11. For -0 it is similar to that of -0, with an additional direct transition from to . This is made possible with multiplication of one of the histories. With temporal pinching of one history a direct transition from to or is possible. For pinching of more than one history direct transition to from or is possible. (

(

*

(

)

E

B



G

E

:

:

G

F

B

:

:



:

I



E



TPP

DC

*

?

EC

NTPP

EQ

PO

TPPi

NTPPi

Fig. 11. Transition graph for -0, -1 and -2. Transitions for -0 are shown as single -1 are double arcs and for -2 are triple arcs. arcs; additional links for .

.

2

.

2

2

.

2

.

.

2

2

5 Conclusion We have formally characterised intuitive notions of s-t continuity and a strongest notion of s-t continuity is defined; we have also induced several weaker definitions of continuity. For two different notions of continuity we have given the conceptual neighbourhood diagrams for non-pinched as well as temporally pinched histories. A completely formal proof of the correctness of the -8 conceptual neighbourhood diagram is part of ongoing research. We also plan to consider the use of the various notions of continuity that we have developed here in a general theory of qualitative motion. 





Qualitative Spatio-Temporal Continuity

107

6 Acknowledgements The first author would like to acknowledge the financial assistance under reference INCS-1999-177 of the Commonwealth Scholarship Commission, United Kingdom. The second author gratefully acknowledges the financial assistance of the EPSRC under grant GR/M56807. We also thank the anonymous referees for their helpful comments.

References 1. J F Allen, ‘Towards a general theory of action and time’, Artificial Intelligence, 23(2), 123– 154, (1984). 2. R Carnap, Introduction to Symbolic Logic and its Applications, Dover Publications, Inc., New York, 1958. Translated by W H Meyer and J Wilkinson. 3. B L Clarke, ‘A calculus of individuals based on ‘connection’’, Notre Dame Journal of Formal Logic, 22(3), 204–218, (July 1981). 4. A G Cohn, B Bennett, J Gooday, and N Gotts, ‘RCC: A Calculus for Region based Qualitative Spatial Reasoning’, GeoInformatica, 1, 275–316, (1997). 5. A G Cohn and S M Hazarika, ‘Continuous transitions in mereotopology’, in Commonsense2001: 5th Symp. on Logical Formalizations of Commonsense Reasoning, New York, (2001). 6. A G Cohn and S M Hazarika, ‘Qualitative spatial representation and reasoning: An overview’, Fundamenta Informaticae, 46(1-2), 1–29, (2001). 7. A. G. Cohn and A. Varzi, ‘Connection relations in mereotopology’, in Proceedings of ECAI98, ed., H. Prade, pp. 150–154. John Wiley & Sons, (August 1998). 8. A G Cohn and A Varzi, ‘Modes of connection’, in Proceedings of COSIT-99, eds., C. Freksa and D.M. Mark, LNCS No. 1661, pp. 299–314. Springer-Verlag, (1999). 9. E Davis, ‘Continuous shape transformation and metrics of shape’, Fundamenta Informaticae, 46(1-2), 31–54, (2001). 10. I D¨untsch, H Wang, and S McCloskey, ‘A relation-algebraic approach to Region Connection Calculus’, Theoretical Computer Science, 255, 63–83, (2001). 11. A P Galton, Qualitative Spatial Change, Oxford University Press, 2000. 12. N M Gotts, J M Gooday, and A G Cohn, ‘A connection based approach to common-sense topological description and reasoning’, The Monist, 79(1), 51–75, (1996). 13. P. J. Hayes, ‘Naive physics I: Ontology for liquids’, in Formal Theories of the Commonsense World, eds., J. R. Hubbs and R. C. Moore, 71–89, Ablex Publ. Corp., Norwood, NJ, (1985). 14. M Heller, The Ontology of Physical Objects: Four Dimensional Hunks of Matter, Cambridge University Press, Cambridge, 1990. 15. NIMA (National Imagery and Mapping Agency), ‘The big idea framework’. Available from http: / / www. opengis. org/ thebigidea/ , (2000). 16. P. Muller, ‘A qualitative theory of motion based on spatio-temporal primitives’, in Proceedings of KR-98, ed., A. G. Cohn et al., pp. 131–141. Morgan Kaufman, (1998). 17. P. Muller, ‘Space-time as a primitive for space and motion’, in Formal ontology in Information Systems, ed., N. Guarino, pp. 63–76. IOS Press, (1998). 18. J. Nicod, Geometry in the Sensible World, Doctoral thesis, Sorbonne, 1924. English translation in Geometry and Induction, Routledge and Kegan Paul, 1969. 19. D. A. Randell, Z. Cui, and A. G. Cohn, ‘A spatial logic based on regions and connection’, in Proceedings of KR-92, ed., B. Nebel et al., pp. 165–176. Morgan Kaufmann, (1992). 20. B. A. W. Russell, Our Knowledge of the External World, Routledge, 1914. 21. P Simons, Parts: A Study In Ontology, Clarendon Press, Oxford, 1987. 22. L Vieu, S´emantique des relations spatiales et inf´erences spatio-temporelles: une contribution a´ l’´etude des structures formelles de l’espace en langage naturel, Ph.D. dissertation, Universit´e Paul Sabatier, Toulouse, 1991.

Application of Supervaluation Semantics to Vaguely Defined Spatial Concepts Brandon Bennett
School of Computing, University of Leeds, Leeds LS2 9JT, UK [email protected]

Abstract. The paper examines ways in which the interpretation of spatial concepts is affected by vagueness and suggests mechanisms for taking account of this within spatial information systems. The theory of supervaluation semantics is explained and applied to the spatial domain and to particular problems of defining geographical concepts such as ‘forest’.

Keywords: vagueness, supervaluation semantics, concept definitions, logic, spatial information systems

1

Introduction

Problems of indeterminacy, inaccuracy and imprecision of spatial information have been recognised as important aspects of data quality and much effort has been spent devising ways to handle such imperfections (Goodchild and Gopal 1989, Goodchild 1993, Heuvelink 1998, Burrough and Frank 1996). At the same time, high-resolution satellite images and sophisticated image interpretation software are yielding increasingly accurate geographical data. However, if this information is to be useful for high-level decision making about the environment, this detailed empirical information needs to be related to the vague natural concepts that we used to think and talk about the world. Geographers, and more especially surveyors and cartographers, have long been aware of the difficulties of giving precise definitions of spatial features (see for example (Maling 1989, chapters 5 and 12)); but, although the phenomenon of linguistic vagueness has been studied by a number of philosophers and logicians, applications of theories of vagueness to practical problems are largely undeveloped. In the fields of AI and GIS, vagueness has often been seen as more or less the same as uncertainty and accounts such as fuzzy logic (Zadeh 1975, Wang and Brent Hall 1996) and rough sets (Or6lowska 1997) are often supposed to encompass both phenomena. In the current paper I distinguish sharply between epistemic uncertainty of data and vagueness of the concepts in terms of which the data is expressed. Even when we have complete certainty about measurable information, we must still solve the problem of relating this data to the vague concepts of natural language.


This work was supported by the EPSRC under grant GR/M56807.

D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 108−123, 2001.  Springer-Verlag Berlin Heidelberg 2001

Application of Supervaluation Semantics to Vaguely Defined Spatial Concepts

109

The concerns of this paper overlap considerably with issues raised by those (e.g. (Frank 1997)) who have recognised the need for well defined ontologies for spatial information in order to avoid inconsistencies arising from the combination of incompatible representations. Whereas ontological formalisation replaces vague concepts with precise definitions, in treating vagueness itself, I am concerned with how to relate naturally vague concepts to possible precise interpretations. Problems of vagueness are also directly relevant to achieving interoperability (Vckovski 1997), where we want to combine data sources that may be based on different, perhaps incompatible, conceptualisations of spatial data. I propose that, in order to achieve their full potential, the architecture of spatial information systems needs to include some kind of Vagueness Reasoning Module (VRM). This will be a layer of the query processing system which will provide a bridge between the natural concepts occurring in queries and the precise but unnatural quantitative database. The structure of the paper is as follows. In the next section I examine the nature of vagueness and distinguish it from other phenomena that affect the interpretation of non-idealised information. Section 3 explains how vagueness can be modelled by supervaluation semantics and the sort of representations and implementations that can articulate the theory. In Section 4 I look at how vagueness relates to spatial extension and in Section 5 I apply the theory to definitions of geographical concepts. We then reach the conclusion.

2

Vagueness and Related Phenomena

I define vagueness as a lack of clearly defined critera for the applicability of a concept. Thus, it is a property of language not of the world itself. Typical examples of vague propositions are: ‘All mountains are very high’; and ‘That frog is green’. The words given in italics are the principal sources of vagueness. ‘Mountains’ is a vague classifier because there is no precise definition of the natural concept of a mountain. ‘very high’ is a vague adjectival phrase: it does tell us something about the actual measurable hight but does not fix any definite height range. The adjective ‘green’ is vague in (at least) two ways: firstly the exact range of colours that count as green is not precisely defined; and secondly, the concept of ‘being green’ is vague in that it is not clear exactly how much of the surface of an entity must be ‘green’ (the frog will almost certainly have some non-green parts). I distinguish sharply between vagueness and uncertainty, which I regard as a distinct (though interacting) phenomenon. I take ‘uncertainty’ to mean lack of exact knowledge about an object or situation. So uncertainty is an epistemic state not a feature of language.1 Although modelling of uncertainty is extremely important in the processing and interpretation of spatial information, it will not 1

There is a view that vagueness is always epistemic, in that it arises from a lack of knowledge regarding the applicability of language (see e.g. (Williamson 1994)). But this is still compatible with a sharp distinction being made between uncertainty about language (i.e. vagueness) and uncertainty about the state of the world.

110

B. Bennett

be considered in the current paper. I shall assume we are dealing with idealised data which is completely certain and accurate. Vagueness can often lead to uncertainty in that where a concept such as ‘forest’, ‘desert’ or ‘swamp’ is vague we will in many cases be uncertain how to demarcate the spatial extension of the concept. On the other hand, if we are not completely certain of the exact details of some information we want to report, we may employ vagueness as a means of increasing the certainty of what we say, while at the same time conveying a sense of imprecision. For example, a statement such as ‘The chair is in the corner of the room’ is vague but can often be said with certainty, whereas an exact specification of the location of a chair (or even a range of possible locations) will typically be uncertain. This example also illustrates the fact that vagueness is not merely a defect of language; it also often facilitates communication without the cumbersome language required to achieve precision. It is useful to distinguish between two kinds of vagueness, which I shall call conceptual vagueness and sorites vagueness. Conceptual vagueness occurs where there is no single completely adequate definition of conceptual term. Certain requirements may be clearly identifiable, whereas for other conditions it is arguable whether or not they are necessary. Certain combinations of these conditions may capture typical senses of the term but none is representative of all possible senses. Thus if we take the intersection of plausible definitions we get a concept that is much to strict (perhaps even unsatisfiable), whereas if we take their disjunction we get a concept that is overly general. Conceptual vagueness is closely related to ambiguity. If a word is ambiguous, it has two or more distinct senses that are clearly distinguishable. However, a conceptually vague term corresponds to a complex cluster of many overlapping senses, such that we cannot say exactly what senses make up the cluster. Moreover, one can meaningfully use a conceptually vague term without being committed to any one of its possible precise interpretations. What we can say about these clusters will be made clear below when I introduce the theory of supervaluation semantics. Sorites vagueness is the kind of indeterminacy that affects the thresholds at which we assert properties such as ‘tall’ or ‘heavy’. Such predicates classify entities with respect to some relevant measurable quantity, without being committed to any specific boundary value. In a pure case of sorites vagueness it is uncontroversial which factors are relevant to the ascription or how those factors should be measured, it is only the threshold that is at issue. However, many natural terms are affected by both types of vagueness. For example to precisely interpret the concept ‘tall man’ we have first to decide how we are to measure the height of a man: must he remove his shoes and hat? what about hair and prosthetic limbs? what about posture? Once we have resolved these conceptual issues we then still have to deal with sorites vagueness in setting the threshold for tallness. Both types of vagueness also interact strongly with contextual phenomena of various kinds. Many concepts exhibit some form of contextual variability. For

Application of Supervaluation Semantics to Vaguely Defined Spatial Concepts

111

example ‘tall’ in the context of ‘tall man’ has a different interpretation from in the context of ‘tall child’. In cases such as this we see that a sorites concept may be affected by its context so that location of its albeit vague threshold is shifted. The range of possible interpretations for a conceptually vague concept can also be affected, not so much by their immediate syntactic context but by their more general context within an information source or exchange. Because it is largely independent of vagueness, issues of contextual vagueness will not be addressed in the current work. I shall assume that context can be eliminated or ignored.2 Despite the fact that in many cases vagueness seems to be separable from context there may be cases where this distinction is blurred. A close connection between the phenomena is born out by the fact that formalisations of the logic of context (see e.g. (McCarthy 1993)) have much in common with the supervaluation approach to vagueness.

3

Supervaluation Semantics

On the supervaluation account of vagueness (Fine 1975) a vague language is one which can be made precise in many different and sometimes incompatible ways. A way of making a language precise is called a precisification. Each precisification p is identified with a precise interpretation, Ip , of the vocabulary of the language. In the simplest case this would be a classical propositional or 1st-order model. A supervaluation model then consists simply of a set of precisifications. Given a supervaluation model V, a proposition which is true under every interpretation Ip ∈ V is called super-true or — in my own terminology — unequivocally true. Supervaluation semantics by itself does not add anything interesting to logic at the object level. It is easy to see that those formulae that are unequivocally true in every model are just the classically valid formulae. However, the semantics does provide a framework within which we can define operators that articulate certain aspects of the logic of vagueness. 3.1

Modal Supervaluation Logic

One possibility is to take a modal approach and represent vagueness in terms of propositional operators (Bennett 1998). Uφ means that φ is unequivocally true — i.e. true for all precisifications; Sφ can be read ‘φ is in some sense true’ — i.e. true for some precisification. S is the dual of U and thus can be defined by Sφ ↔ ¬ U ¬φ. Logically U behaves as the
operator of the modal logic S5 and S as its dual, ♦. We can now qualify assertions according to whether they hold in some or all precisifications. For example S[Wood(‘Woodsley Clough’)] or U[Wooded(parcel1 )]. 2

For instance we might suppose that some transformation can be carried out that replaces contextually variable concepts with non-contextual concepts and explicit constraints; or, we could just consider composite concepts such as ‘tall man’ and ‘tall child’ as if they were syntactically atomic.

112

B. Bennett

We can also use these operators to specify dependencies between the meanings of vague concepts. Thus ∀x[Copse(x) → S Wood(x)] means anything which is a copse (i.e. a small group of trees) is in some sense a wood. Similarly, ∀x[Wood(x) → S Forest(x)] captures the intuition that any wood is arguably a (small) forest. If a copse is in some sense a wood and a wood is in some sense a forest, this does not mean that a copse is in some sense a forest; and indeed according to supervaluation semantics the formula U ¬∃x[Forest(x) ∧ Copse(x)] is consistent with the previous two formulae. This illustrates the ability of the theory to model the blurring of concepts, while still maintaining certain strong constraints. In the context of a computer database we will often be dealing with concepts that are precise sharpened version of natural terms. The supervaluation operators enable us to relate these artificial concepts to their vague natural language counterparts. For instance, the following formula asserts that Forest1 is a more precise version of the concept Forest: ∀x[Forest1(x) → (S Forest(x))] ∧ ∀x[(U Forest(x)) → Forest1(x)] By specifying such axioms, ‘soft’ constraints are placed on the meanings of natural concepts. Classifications in terms of artificial concepts can be combined with information containing natural concepts. Supervaluation semantics allows one to specify a number of different entailment relations of varying strength. Bennett (1998) defines the following (together with three weaker forms of ‘reliable’ entailment): |=S5U S((φ1 ∧ . . . ∧ φn ) → ψ) φ1 , . . . , φn |=arguable ψ iff iff |=S5U (Uφ1 ∧ . . . ∧ Uφn ) → Uψ φ1 , . . . , φn |=global ψ iff |=S5U U((φ1 ∧ . . . ∧ φn ) → ψ) φ1 , . . . , φn |=local ψ iff |=S5U (Sφ1 ∧ . . . ∧ Sφn ) → Uψ φ1 , . . . , φn |=reliable ψ The weakest entailment is ‘arguable’ which holds if there is any sense of the concepts in the formulae under which the implication corresponding to the entailment holds. This gives us entailments that hold under very flexible (perhaps even inconsistent) interpretations of the concepts involved. The strongest is ‘reliable’ entailment, which holds if: whatever senses the premisses are interpreted under, the conclusion holds in every sense. This can be used to derive entailments which must hold despite the presence of vagueness. For instance S Desert(x) → U ¬Marsh(x) might hold even where Desert and Marsh are very vague predicates. This ability to derive secure consequences involving vague concepts is perhaps the main advantage of supervaluation semantics over fuzzy logic, where fuzzy concepts cannot support completely reliable inferences. 3.2

Reified Precisifications

Although modal operators allow many logical properties of vague concepts to be expressed, they do not provide any way of referring directly to individual precisifications. However, in the environment of a GIS we will often want to record

Application of Supervaluation Semantics to Vaguely Defined Spatial Concepts

113

information about which interpretations can be given to information in particular datasets. To do this we need a language in which names of precisifications can be related explicitly to spatial formulae. In a general reified supervaluation semantics we could associate arbitrary propositions with precisification variables and constants. Thus, InPrec(p, φ) would assert that φ is true according to precisification p. At the expense of some elegance we can achieve the same expressive power by simply supplementing each predicate and function of an ordinary 1st-order language by an additional argument place. For instance a predicate Swamp(x) would be replaced by a relation Swamp(p, x) saying that, in precisification p, x is a swamp. If we use this approach we need not worry about axiomatising the logical predicate InPrec. Logical relationships between vague concepts that hold whatever reasonable way they are interpreted can now be represented by quantifying over the (possibly infinite) space of precisifications. Since a precisification fixes the meanings of all the vague vocabulary of a language, a classification which makes precise only part of the vocabulary may be common to a class of precisifications. In a formalism with reified precisifications, we can model this by introducing predicates of precisifications. For example, UNESCOF(p) ↔ Φ(p) might mean that the predicate UNESCOF applies to those precisifications satisfying some precise formal specification Φ of the UNESCO forestation classification given below in Table 1. The use of a language with reified precisifications is also motivated by the analysis of vague nominal expressions given in the next section, which seems to be difficult to express within a modal framework.

3.3

Vague Nominal Expressions

The established theory of supervaluation semantics models a situation where we wish to reason with vague predicates, which are applied to a perfectly definite domain of objects. The question of whether the objects referred to by a language can themselves be vague has also received attention from philosophers (Evans 1978, Hughes 1986, McGee 1997). There is no consensus on this issue but the more popular view seems to be that vagueness is a feature of language and not of the objects it describes. But even if this is so, one could still argue that the nominal expressions of a language may be vague in that they may not unequivocally refer to a unique entity, but rather may be understood in different senses as applying to different physical entities. The current paper concentrates primarily on vague concepts rather than nominals. However, I suggest that the vagueness nominals can be explained as derived from the concepts that they exemplify. Thus, an expression such as ‘Sherwood Forest’ refers to a particular instance of the vague sortal predicate ‘forest’ and the range of possible extensions that might be assigned to Sherwood Forest are determined by the range of possible precise senses that can be given to the concept ‘forest’.

114

B. Bennett

3.4

Implementing Supervaluation Semantics

The reified precisification approach to vagueness lends itself well to implementations in established logic programming languages such as Prolog. Alternatively one could use some description logic system (Calvanese, Lenzerini and Nardi 1998). In either case one could explicitly add precisification variables to the definitions of vague concepts. For most applications one would probably want to hide the apparatus of precisifications from the user and employ a more intuitive way of showing the results of vague queries. For instance, this might be achieved by expanding a query such as ‘?- show(forest).’ to a form something like ?- setof(R, (forest(P,F), Ext(P,F,R)), PossExts), illustrate(PossExts). P is a precisification variable, which parameterises both the sense in which ‘forest’ is interpreted and the sense in which its extension is determined by the Ext relation. ‘illustrate(PossExts)’ produces informative graphical output about the distribution of possible extensions under different interpretations. This example glosses over certain difficulties. In particular it assumes that a finite number of senses of forest are defined, whereas the space of possible senses might be better described in terms of certain continuous parameters (such as average tree height). To handle this a much more sophisticated procedure would be needed which might display the extensions of a range of possible interpretations taken at sample points within the space of precisifications.

4

Spatial Vagueness

Supervaluation semantics is a very general approach to vagueness but it can only be useful for reasoning about a specific domain if the peculiar logic of that domain is adequately modelled. With the exception of (Kulik 2000) I am not aware of any other work applying supervaluation semantics to spatial concepts. 4.1

Spatial Concepts and Extensions

The spatial properties that are easiest to understand semantically are those that can be defined in terms of properties of points — i.e. their extension consists of all points satisfying some given condition. Examples of such concepts are ‘the region of the Earth that is more than 1000m above sea-level. However, in general, a ‘region property’ will be associated with a property of the whole set of points, which cannot be explicitly reduced to properties of individual points. For example a ‘lake’ is not simply made up of the set of points which are covered by water, it is rather a particular maximal connected set of water covered points. Although maximal connectedness is one of the most important factors in the individuation of geographical features, only very basic types of feature can be regarded simply as maximal connected sets of points exhibiting a given property.

Application of Supervaluation Semantics to Vaguely Defined Spatial Concepts

a)

b)

115

c)

Fig. 1. Models of Vagueness and Extension

Typically, whether a set of spatial points can be taken as the extension of a feature of a given type, is dependent on much more complex constraints on the structure of this set (consider e.g. how we differentiate lakes from other hydrological features or how we might characterise a ‘building’). Some of these constraints may not even relate to physical properties (e.g. a ‘listed building’). I say that a region property is: integral iff when it is true of a region it is not true of any proper parts of that region; and, divisible iff whenever it holds of some region it holds of all parts of that region. According to this analysis, concepts describing geographical features, such as ‘lake’, ‘river’, ‘forest’ are integral; whereas land-type concepts such as ‘wooded region’, ‘marshy region’ are divisible. Many properties are neither integral nor divisible — e.g. ‘r encompasses a lake’. Whether properties are divisible is important in determining whether a conjunction of several spatial concepts applies to the intersection of their extensions of different spatial concepts (or of the same concept under different precisifications) A further issue that complicates the identification of regions with sets of points is the status of boundary points. For many spatial concepts it is not clear whether boundary points should be counted as included in the region to which they refer. This may be regarded as an example of conceptual vagueness. However, it is also a general ontological issue which applies to spatial concepts that are not in other ways vague. 4.2

Extensions of Vague Concepts

From the perspective of spatial information, the most important property of any object is its spatial extension. Hence, it is vital to model the way in which vagueness affects the attribution of extensions. The ‘Egg-Yolk’ theory (Lehmann and Cohn 1994, Cohn and Gotts 1996a, Cohn and Gotts 1996b) directly models the notion of a vague region in terms of its maximal and minimal possible extensions. The maximal extension is called the ‘egg’ and the minimal is the ‘yolk’, which is required to be a part of the egg (see Figure 1a). (The case where the yolk is equal to the egg is allowed, such cases corresponding to ‘crisp’ regions.) This analysis is simple and supports an account of some significant inferences involving relationship between vague

116

B. Bennett

regions. However, it cannot handle complex constraints on a region’s possible extensions between its maxima and minima. For instance, although a vague region such as an area of marshland might have maxima and minima as illustrated in Figure 1a, the area within the dotted line might not correspond to any reasonable precise interpretation of ‘marshland’. Supervaluation semantics is much more general in that it has the potential to model arbitrary constraints on the distribution of possible extensions, as illustrated in Figure 1b. However, the possible extensions of natural vague concepts will not be completely chaotic since, according to supervaluation theory, they correspond to a cluster of precise concepts with similar meanings. In the case of a purely sorites vague concept, where the vaguenss is in the choice of a suitable threshold for some observable, the possible extensions will typically (though not necessarily) be contoured as shown in Figure 1c. Each contour corresponds to a more or less strict sense of a spatial concept. For instance different definitions of ‘marshland’ may require more or less water to be present. Where we have mixed vagueness we will have several sets of contours each corresponding to varying the threshold for some conceptually unambiguous but still sorites vague concept.

5

Geographical Concepts and Features

Vagueness is pervasive in spatial and geographical concepts and tends to persist even where steps are taken to give them precise definitions. For example, in (Ordnance Survey 2000), the guide book to the Ordnance Survey’s Land-Line data set a road is defined as: “A metalled way for vehicles.” This does tell us something about what is meant by ‘road’ but the definition is still vague in many respects. We may be unsure about what surfaces count as ‘metalled’. Neither condition of the surface nor any restrictions on its spatial extent is specified. The term ‘way’ could be understood in many more or less general ways. ‘Vehicle’ is also too general a term for what is intended. The OS definition of road would seem to apply to bicycle paths, which may not be intended. It also seems to rule out cobbled or flagged streets which one might expect to be classified as roads. 5.1

When is Vagueness an Issue?

To illustrate how our theoretical approach might be applied to practical problems we focus in this paper on the example of the concept of a ‘forest’, looking at different ways in which the term can be interpreted and how these effect the determination of the spatial extension of a forest. Inventories of the properties and extent of woodland are vital to efficient forest management. However, problems of definition are not a major concern in the literature on forestry (the standard text book (Husch, Miller and Beers 1963) does not mention any definitional problems). This is not surprising when we consider the nature of forestry and the kinds of information it requires. For most purposes a forester can assume that a forest consists of a collection of stands whose boundaries are well-defined. The properties of each stand can then

Application of Supervaluation Semantics to Vaguely Defined Spatial Concepts

117

be determined by random sampling techniques; and from these measurements, economically important quantities such as ‘forest volume’ can then be derived by simple computations or by the use of empirically verified tables. The problem of determining boundaries is not of great importance because the statistical approach to measurements works with any reasonable bounding of the forest area, and, in all but exceptional cases, mitigates the effect of any uncertainty in this boundary. Although meta-questions about the nature of a forest may be largely irrelevant to the narrowly defined concerns of industrial forestry, they are certainly relevant to more general problems of determining and allocating land types. For instance, if we want to answer a question such as ‘How rapidly is the forested area of the earth shrinking?’ the problem of demarcating forest areas is central. Similar problems apply to the identification and classification of ‘deserts’ (and the problem of measuring and monitoring the progress of desertification). For instance, the Global Change Data Base produced by NOAA includes multiple data sets on topics such as soil, precipitation, vegetation, temperature, land cover, etc. Several of these data sets have ‘desert’ as a specific class, each with its own method of compilation and concept of what actually constitutes a desert — absence of vegetation, annual rainfall below a particular (and varying) threshold, number of months per year exceeding a precipitation threshold, type of soil, ecosystem characteristics, etc. The result is a set of maps that produces a very different distribution of deserts according to which classification you choose to use at any one time. Moreover, the concept of desert can itself be variously classified into sub-types (e.g. ‘desert, mostly bare’, ‘sand desert, partly blowing’, ‘other desert and semi-desert’, ‘polar desert’, ‘tropical desert’). A further example of the importance in environmental modelling of clarifying vague terms is provided by Alker, Joy, Roberts and Smith (2000) who consider issues in defining the concept of a ‘Brown-field’ which is often used in formulating development policies. 5.2

Defining ‘Forest’

I now examine the range of possible definitions which may be used to specify a precise concept corresponding to some reasonable sense of the natural language concept of ‘forest’. I start by considering a number of questions, each of which addresses one of the main aspects of vagueness associated with the term, and hence has no clear-cut answer: 1. Is a forest a natural feature or one determined by convention and legality? 2. Does ‘forest’ refer to an integral feature or can it be applied to an arbitrary region of land? 3. What type of vegetation can constitute a forest? (i.e. what species and how big must they be?) 4. How dense must the vegetation be? 5. How large an area must a forest occupy? 6. Are there any constraints on its shape?

118

7. 8. 9. 10. 11. 12.

B. Bennett

Must a forest be self connected, or can it consist of several disjoint parts? Must it be maximal or could it share a border with another region of forest? Is a clearing a part of or a hole in a forest? Are roads and paths going through a forest parts of the forest? How should seasonal and other temporal variations be taken into account? If part of a forest is felled and subsequently re-grown, does it remain part of the forest throughout?3

In the following subsections I shall suggest how the issues underlying these questions can be clarified by differentiating between many possible precise senses of ‘forest’. Preliminary analysis of some of these sense variations in terms of supervaluation theory will also be given.

5.3

Natural vs. ‘Fiat’ Forests

One of the most important aspects of the conceptual vagueness of the term ‘forest’ is the ambiguity between forests conceived of as a natural feature and forests as parcels of land upon which is legally or conventionally conferred the status of being a forest. Although it may be argued that forests are always originally identified with some natural feature, once they are named (and thus probably also owned) additional conventional and legal mechanisms may be employed to individuate forests. Smith (1995) has investigated the ontology of conventional regions of this kind, which he calls fiat regions. In axiomatising the vague term ‘forest’ it is clear that the natural and fiat interpretations will obey rather different axioms. Hence, any ontology of geographical concepts should split the concept into two specialisations. The following axioms ensure that in any precisification Fiat Forest and Natural Forest are sub-concepts of Forest and that all forests are of one of these two types (they do not rule out the possibility that something may be both): • ∀px[Fiat Forest(p, x) → Forest(p, x)] • ∀px[Natural Forest(p, x) → Forest(p, x)] • ∀px[Forest(p, x) → (Fiat Forest(p, x) ∨ Natural Forest(p, x))] Though free from a certain ambiguity, the predicates Fiat Forest and Natural Forest are still extremely vague, each will correspond to a wide range of possible senses and further subdivisions and axioms will be required to explicate these. In the rest of the analysis I shall dealing only with ‘natural’ forests, since these seem to be vague in a greater variety of ways; however, the semantics of fiat forests is no doubt also very complex. Henceforth the predicate Forest shall be used to mean Natural Forest. 3

Accompanying its ‘Land Usage of the World’ data the web site www.ecoworld.com gives the following definition of forest: “Forest: Land under natural forests or planted stands of trees. Also includes logged areas to be replanted in the near future, after logging.”

Application of Supervaluation Semantics to Vaguely Defined Spatial Concepts

5.4

119

Forest as Feature or Land Type

In clarifying the concept of ‘(natural) forest’ we immediately encounter a second fundamental ambiguity that affects this and many similar geographical concepts. When used with an article (‘a forest’ or ‘the forest’) the term typically refers to a particular integral feature whose boundary (albeit vague) is determined by the meaning of the concept. However, it can also be used in an adjectival sense to describe an arbitrary region as ‘forest’. These two uses are not really due to vagueness but rest on a logical distinction that ought to be explicit in the ontology of any GIS that supports high-level queries. Though ontologically distinct, features and corresponding land-type concepts have strong logical interdependence which must be formally specified (see (Eschenbach 2000)). Let us use the predicate Forest as a vague feature type and Forested as the corresponding vague land-type classifier and see what axioms one would expect to link the two concepts. We might be inclined to say that a region is ‘forested’ iff it is part of some forest. However, this definition suffers from a problem of granularity, since forests may contain pockets which are not at all forested. We can avoid this problem by taking forested as the more basic property. Using P for parthood and CON for connected we can define a forest as a maximal connected wooded region: Forest(p, x) ≡def (Forested(p, x) ∧ CON(x) ∧ ¬∃y[CON(y) ∧ P(x, y) ∧ Forested(p, y)]) . The scope of the precisification variable p ensures that under any given precisification the meaning of Forest is logically determined by the meaning of Forested under that same precisification. This would support various patterns of reliable inference that hold whatever reasonable sense we give the concepts. 5.5

Classifying Vegetation

Having defined ‘forest’ in terms of ‘forested’ we need to consider how observable measurements of the physical world determine which regions should be deemed ‘forested’; or rather, given our supervaluation methodology, we need to elucidate how these observables relate to different precise interpretations of ‘forested’. Table 1 shows a physiognomic classification, of levels of forestation that was proposed in (UNESCO 1973) and later adopted in (USGS 1994b). The range of different terms employed in the table illustrates how a precisification (or class of precisifications) is not merely associated with a collection of senses of individual terms but with a complex system of logical constraints concerning the meanings of multiple interrelated concepts. This classification carries with it a lot of implicit conceptual baggage which may not be compatible with other ways of defining forests. For instance, any precisification satisfying it must enforce the constraint that woodland and shrubland are necessarily disjoint. There is also some lack of specificity in the classification. It is not clear whether a population of fairly widely spaced tall trees growing among a dense cover of small shrubs should be counted as ‘sparse woodland’ or

120

B. Bennett

Percent Canopy Cover of Vascular Vegetation 100%–60% 60%–25% 25%–10% 10%–1% Plant-form/Height (interlocking) (touching) (spaced) Trees >5m Forest Woodland Sparse Woodland Shrubs/Trees 0.5–5m Shrubland Sparse Shrubland Sparsely Shrubs 0.05) indicate that these effects are relevant to the model and should be retained. Table 3: Hierarchical Log linear Models for the Scale Test Backward Elimination (p = .050) for DESIGN with generating class ZOOM*DIM*TYPE If Deleted Simple Effect is DF P ¤L2 ZOOM*DIMENSION*TYPE 2 0.7740 0.6792 DIM*TYPE 1 0.2280 0.6327 ZOOM*DIMENSION 2 15.2370 0.0005 ZOOM*TYPE 2 21.3340 0.0000 Shaded rows are selected for the Logit model. The best model has generating class: {ZOOM*DIM}, {ZOOM*TYPE} Likelihood ratio chi square = 1.00213 DF = 3 P = .801

Iteration 3 2 2 2

Evaluating the Usability of the Scale Metaphor

167

The Logit model of choice for zoom types (Z) and graphic variables dimension (D) and data type (T) contains the following terms (see Table 3 for ¤ L2 values):

Gijk = {ZD}{ZT }

(3)

Completing the model with the estimated parameters for two examples, the probability of a zoom-in with 2D points as a starting configuration is 83%, whereas not zooming in with 2D point clouds as a starting configuration is 63%. 4.8

Individual Differences

These results are also interesting in light of users’ background data. Spatial ability, as measured by the paper-folding test (Ekstrom et al., 1976), did not reveal a significant correlation with overall response time (Spearman’s r =-0.23, p = 0.12). There seems to be a significant negative correlation of spatial ability with time to first zoom, although it could be considered weak (Spearman’s r =-0.31, p = 0.03). Participants show high computer literacy. They all use computers on a daily basis (100%), and 50% of the participants use digital archives on a daily basis (e.g. access data on CD ROMs). Half the participants use online databases at least weekly (e.g. Web of Science, etc.). The majority (75%) has not had any formal training in cartography (mean = 0.5 yrs) and graphic design (mean = 1 yr.). About half of the subjects (50%) are not trained in computer graphics (mean = 1 yr.), or information retrieval (mean = 4.75 yrs). A considerable percentage of the participants use graphics (75%) and geographic data sets (40%) on a daily basis. There seems to be a weak positive correlation between spatial ability and GIS training (Spearman’s r = .289, p = 0.05), but none between mathematics training (mean = 5 yrs.) and spatial ability. Neither GIS background, nor mathematics training seem to have a relationship with overall response time or reaction time to first zoom. The post-test questionnaire also assessed participants’ subjective usability ratings and satisfaction with zoom tools and graphic displays utilized during the experiments. Overall people reacted very positively to the spatialized displays and query tool. Most participants were intrigued by the displays and would use them again to query a document archive (83%). Half of the subjects found them somewhat unique, and 42% found them very unique. Most participants were at least somewhat attracted by the graphics (83%), and found them somewhat interesting and worth exploring (67%). Seventeen percent found them very attractive, and very interesting (33%). Half the participants mentioned the use of color as a reason why they found the displays attractive. About half related their interest in the displays to being able to see an overall, spatial structure in 3D, including a combination of labels and graphics. The ability to rotate the graphics in real-time and at will seem to have been a powerful experience for many participants. Think-aloud protocols and direct observations confirm the overall very positive experience participants seem to have had interacting with the displays. Most people showed reactions of surprise or disappointment when reaching the end of the digital portion of the experiment. Most felt they had barely started and would have preferred to go on (this was least 30 minutes after the start of the experiment). Manifestations of astonishment, surprise and intriguedness were often apparent, particularly when participants could directly manipulate the 3D displays. Comments like “this is cool”, or “this if fun” were recorded with most participants. One participant, after having responded to the test question, got quiet while rotating the 3D representation in all directions for a very long time. This person

168

S.I. Fabrikant

tried to identify a pattern in the 3D point cloud. Other participants zoomed in and out of displays just to track how the change in data type affected their conceptualization of the information space. No signs of apparent frustration were detectable for any participant. One person mentioned in the post-test questionnaire having been frustrated at times, because of “not knowing the interrelation between displays”. 4.9

Discussion and Design Recommendations

Results from testing scale change suggests that people are able to associate graphical change in resolution (zoom-in) with different levels of detail in a document collection A statistically relevant association exists between zooming behavior and graphic representation. Zooms from point into point representations lead to the desired document faster than zooms performed from a surface display into a point representation. User group did not affect the use of the metaphor. Although results on reaction time until the zoom tool is applied are not significant, main effect type (F = 8.22, p= 0.007) and interaction of type and group come close (F = 4.94, p= 0.006). The main effects data type and dimension (controlled variables) are associated significantly with types of zooms. Graphic variables utilized to represent the database graphically will influence the usage pattern of the scale change metaphor embedded in the query tool. The factors dimension and data type both were significant, either for response times or for type of zooms. These results are difficult to interpret. One problem is the amount of factors that needs to be controlled for the scale change metaphor to strengthen the result of the experiment. Still, as a starting point for further research these outcomes are important to consider. In related experiments on the spatial metaphors distance and arrangement the graphic variables employed to render the semantic spaces were also shown to be important modifiers. Adherence to cartographic design principles (e.g. “more is darker”) enhances understanding of the metaphors. Color and shape are particularly strong visual variables in this study, as revealed in participants’ responses to open-ended questions in the post-test questionnaire. This has direct consequences for the design of the spatialized views. For query tasks that require zooming to change the level of detail of a document collection the primary design consideration relates to modifying the display from points to points or surface to points. Surface-to-point zooms in 2D, and point-to-point zooms in 3D to seem to work best when access time to find relevant documents is important. If the amount of zooms needs to be minimized, then surface-to-point representations seem to perform better overall.

5

Conclusions and Outlook

A usability evaluation was applied to a spatialized query metaphor, the zoom-in and zoom-out tool, to access a spatialized portion of GeoRef, a collection of geology and earth sciences documents. Response times and accuracy of responses of participants using the zoom tools were collected during experiments on querying spatialized views. A qualitative investigation was also pursued with the think-aloud method. Results indicate that people are able to associate hierarchical document relationships in a collection with the spatialized metaphor of scale change (zooms) for the information access scenarios described. For some displays it takes longer to make a decision. An important finding is that analysis of group membership did not yield a significant effect. Regardless of participants’ backgrounds, the tested metaphor seems to yield similar responses. The scale metaphor provides many research threads worth

Evaluating the Usability of the Scale Metaphor

169

exploring. The post-test questionnaire revealed that display attractiveness was directly connected with the ability to manipulate graphic representations in 3D, thus being able to explore an object from different view points. Current GIS typically represent geographic space (large-scale space) as pictorial space, or map space that are both non-manipulable space types (Freundschuh and Egenhofer, 1997). Assuming that direct manipulation and interaction with spatial representations enhances human’s cognition of spatial primitives and spatial object relations, could this “empowerment” factor improve the usability current GISystems? Direct manipulation interfaces with iconic representation of system commands for spatial analyses are not new, but these interfaces often lack validation through empirical usability evaluation. The interdisciplinary design framework adopted for this investigation, and derived design recommendations based on empirical results might be a starting point to reconsider the construction and design of more intuitive iconic GIS interfaces? Another research arena relates to “scalability” of the represented space. Human conception of space and spatial behavior are experience-based and scale-dependent (Freundschuh and Egenhofer, 1997). This study examines how people respond to a spatialized query tool to explore spatializations within a manipulable object space. This begs the question of how people’s association with scale change would differ, if the object space changed in scale? How would these spatializations be understood within a virtual semantic document space? Information seekers could navigate in full immersion through a traditional library, and could also manipulate and interact with more abstract spatialized representations to search for information. What is the optimal balance between increased realism for intuitiveness and wayfinding, and the level of abstraction to reduce cognitive overload? How would wayfinding and navigation in such large-scale virtual spaces affect spatial metaphor comprehension? Scale change in object space has implications on empirical testing procedures. It is doubtful that current usability evaluation methods are adequate for virtual environments (VE). These and other challenges have to be addressed, to maximize the potential that spatialization has to offer for knowledge discovery.

6

Acknowledgements

This study is a subset of the author’s dissertation completed at the University of Colorado at Boulder. Thanks are due to my Ph.D. advisor Barbara P. Buttenfield for her support and guidance throughout this study.

7

References

Card, S. K., Mackinlay, J. D., and Shneiderman, B. (1999). Readings in Information Visualization. Using Vision to Think, Morgan Kaufmann, San Francisco, CA. Catarci, T. (2000). What’s New in Visual Query Systems? Proceedings, First International Conference on Geographic Information Science, Savannah, GA, Oct. 28-31, 2000. (http://www.giscience.org/GIScience2000/invited/Catarci.pdf) Chen, C., Czerwinski, M., (Eds.) (2000). Special Issue on Empirical Evaluation of Information Visualisations. International Journal of Human Computer Studies, 53(5). Chen, C., Czerwinski, M., and Macredie R. (Eds.) (2000). Special Issue on Individual Differences in Virtual Environments. Journal of the American Society of Information Science, 51(6).

170

S.I. Fabrikant

Chen, C. and Yue, Y. (2000). Empirical Studies of Information Visualization: A Meta-Analysis. International Journal of Human-Computer Studies, 53: 851-866. Ekstrom, R. B., French, J. W., Harman, H. H., and Derman, D. (1976). Kit of FactorReferenced Cognitive Tests. Princeton, NJ, Educational Testing Service. Fabrikant, S. I. and Buttenfield, B. P. (2001). Formalizing Semantic Spaces For Information Access. Annals of the Association of American Geographers, 91: 263-280. Fabrikant, S. I. (2000). Spatial Metaphors for Browsing Large Data Archives. Unpublished Disseration, University of Colorado-Boulder, Department of Geography, Boulder, CO. Freundschuh, S. M. and Egenhofer, M. J. (1997). Human Conceptions of Spaces: Implications for GIS. Transactions in GIS, 2: 361-374. Golledge, R. G. (1995). Primitives of Spatial Knowledge. In Cognitive Aspects of Human-Computer Interaction for Geographic Information Systems, Nyerges, T. L., Mark, D. M., Laurini, R., and Egenhofer, M. J. (eds.), Dordrecht, Kluwer Academic: 29-44. Goodman, B. A. (1997). GeoRef Thesaurus. Alexandria, VA, American Geological Institute. Knoke, D. and Burke, P. J. (1980). Log-Linear Models. Sage University Paper Number 20. Newbury Park, CA, Sage Publications. Kraak, M. J. (1988). Computer-Assited Cartographical Three-Dimensional Imaging Techniques. Dissertation. Delft, The Netherlands, Delft University Press. Lakoff, G. (1987). Women, Fire, And Dangerous Things: What Categories Reveal About The Mind. Chicago, IL, University of Chicago Press. Lewis, C. (1991). Inner and Outer Theory in HCI. In Designing Interaction: Psychology at the Human-Computer Interaction Interface, Carroll, J. M. (ed.), Cambridge, MA, Cambridge University Press: 154-161. McNamara, T. P., Hardy, J. K., and Hirtle, S. C. (1989). Subjective Hierarchies in Spatial Memory. Journal of Experimental Psychology. Learning, Memory and Cognition, 15: 211-227. Montello, D. R., Golledge, R. G. (1999). Scale and Detail in the Cognition of Geographic Information. Varenius Project Report, Specialist Meeting, May 1416, 1998, Santa Barbara, CA. Morse, E., Lewis, M., and Olsen, K. A. (2000). Evaluating Visualizations: Using a Taxonomic Guide. International Journal of Human Computer Studies, 53: 637662. Nielsen, J. (1993). Usability Engineering. Boston, MA, Academic Press. Norman, D. A. (1993). Things That Make Us Smart. Defending Human Attributes in the Age of the Machine. Reading, MA, Addison-Wesley. Pirolli, P., Card, S. K., and Van Der Wege, M. M. (2000). Visual Information Foraging in a Focus + Context Visualization. Xerox PARC, Technical Report, IR-R-2000-14. URL: http://www.parc.xerox.com/istl/projects/uir/pubs/pdf/UIRR-2000-14-Pirolli-TechReport-VIFinFocusContext.pdf (Jan. 2001). Robertson, G. (1998). Keynote Address: Leveraging Human Capabilities in Information Perceptualizationd. IEEE Symposium on Information Visualization (InfoVis '98), Oct. 19-20, 1998, Research Triangle Park, North Carolina. URL: http://www.research.microsoft.com/~ggr/infovis98.ppt (Jan. 2001). Shneiderman, B. (1998). Designing the User Interface: Strategies for Effective Human-Computer Interaction. Reading, MA, Addison-Wesley.

Evaluating the Usability of the Scale Metaphor

171

Stasko, J., Catrambone, R., Guzdial, M., and McDonald, K. (2000). An Evaluation of Space-Filling Information Visualizations for Depicting Hierarchical Structures. International Journal of Human-Computer Studies, 53: 663-694.

zoom in

Appendix. 2D test screens for the scale change metaphor

Note to the reader: The test screens for the 3D tests can be accessed at the following Web address: htlp://www.geog.ucsb.ed~i~-wa/htm1ires~~~W~os0 1lappendid

L-"",*1

1

A Semantic Map as Basis for the Decision Process in the www Navigation Hartwig Hochmair, Andrew U. Frank Institute for Geoinformation, Technical University Vienna Gußhausstr. 27-29, A-1040 Vienna, Austria {hochmair,frank}@geoinfo.tuwien.ac.at Tel.: (+43) (0)1 58801-{12715,12710} Fax.: (+43) (0) 1 58801-12799

Abstract. In the physical world, decision making in common navigation strategies is based on a mental map which includes a mental representation of geometric features such as distances and directions between places. We propose that decision making in the web is also based on a mental map. Contrary to the physical world navigation, the mental map used for web navigation describes a part of the agent's epistemology of the world, hence consists of semantic relations between concepts and includes hardly any geometrical features. We focus on the navigation situation where the detailed web structure is unknown to the agent before the navigation, therefore the agent's decisions are completely based on his semantic mental map. We give a potential structure of the agent's mental map, simulated using WordNet, and simulate the agent's decision making process during the web navigation. The simulation of the strategies allows to assess existing web environments with regard to ease of navigability.

Keywords. semantic map, web navigation strategy, decision process, epistemology, web searching

1 Introduction

1.1 A Mental Map for Navigation Tasks Navigation towards an unknown goal is a common human activity. Many of the proposed navigation strategies in the literature are based on a mental representation or cognitive map of the environment. A cognitive map can be constructed through environmental observations combined with locomotional information during the exploration (Kuipers 1978), through reading an external map (Sholl 1987; Hochmair to app.) or through communication (Peuquet 1998). D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 173−188, 2001.  Springer-Verlag Berlin Heidelberg 2001

174

H. Hochmair and A.U. Frank

We propose that a web navigating agent uses a mental map for his navigation strategy, too. The agent navigates towards a novel web page and has a rough idea of the content of the target web page. The agent builds a semantic map around the content of the web site that he is searching for. The semantic map consists of several fields, of which certain elements together define the content of the web site. Because of navigating towards a novel goal, the detailed web structure of the web domain is unknown to the agent before the navigation. This means that the agent's mental map itself does not describe the linked positions of web pages. For semantic web navigation, such information is not needed. Directions and Euclidean distances, and path strategies are irrelevant for the web navigation as the web itself is not spatial (Svensson 1998) in the sense of direct bodily locomotion experience. 1.2 Searching in the Web Space Browsing and searching are common terms for web users. Literature shows a wide variation in the meaning of these terms. Marchionini (1995) distinguishes between direct browsing, semidirected browsing, and undirected browsing. The author also proposes a model of the information seeking process which is composed of eight subprocesses. Ellis et al. propose six categories of information seeking activities for general media: starting, chaining, browsing, differentiating, monitoring and extracting (Ellis and Haugan 1997). Our proposed web navigation strategy is related to the first of the categories proposed by Marchionini, and to the first four categories of seeking activities in (Ellis and Haugan 1997). At the beginning of the web navigation process, the agent determines a starting point in a web domain, possibly found using a search engine. During the navigation process, the agent is offered a number of links on each visited web site. On the user's computer screen, the links are realized either through colored hyperlink-texts, metaphorical graphics, or icons. If the associated information of a web link is part of the agent's semantic map, the agent is able to determine which of the links is semantically closest to the content of the web page that he is trying to find.

2 The Agent's Semantic Map

2.1 A Glance at Previously Discussed Ontologies in the Literature A wide range of models to categorize the world is proposed by philosophers. Smith (2001) claims that each scientific field has its own preferred ontology. For the philosopher, ontology is a theoretical construct which is invariant from the used language. Contrary to this, in the world of information systems, such as the web, ontology is a software artifact designed with a specific use and computational environment in mind. The classical theory uses categories with clear boundaries which are defined by common properties (Lakoff 1987). Wittgenstein (1953) contradicts the classical

A Semantic Map as Basis for the Decision Process in the www Navigation

175

theory and proposes that not all members of a category share one common property but are united by family resemblances. Conceptual embodiment (Lakoff 1987) claims that the properties of certain categories are a consequence of the nature of the human biological capacities and of the experience made in a social and physical environment. This contradicts the idea that concepts exist independent of a body and experience. In experiments concerning colors and physical categories, Rosch and Mervis (1975) found out asymmetries, so called prototype effects, between members of a category. Membership gradience (Barsalou 1987) describes the idea that some categories have degrees of membership and no clear boundaries. Graded structures are found in a wide range of category types (Rips, Shoben et al. 1973; Smith, Shoben et al. 1974; Lakoff 1987). 2.2 Proposed Structure of the Agent's Semantic Map For the proposed structure of the agent's semantic map, we use the basic idea of Aristotle's ontology which is based on substance and accident. Substances are things and bodies, accidents are qualities, events, and processes. Substances vary in chemical and physical composition, and are hierarchically structured. Some objects involve both substances and accidental parts, so that objects are partially bearers of accidents. A connection between objects and activity is also given through the idea of affordances (Gibson 1977). Raubal distinguishes between several types of affordances, e.g., social-institutional (Raubal to app.) or action affordances (Raubal and Worboys 1999). The latter describe what things or objects offer people to do with them. Affordances are highly based on an individual's life experience. Concerning computer interfaces, Norman (1999) distinguishes between physical affordances and perceived affordances, where the physical affordances are a synonym for action affordances. In the computer environment, the computer with its keyboard, display screen, pointing device, and mouse buttons affords pointing, touching, looking, and clicking on every pixel of the screen. In graphical, screen-based interfaces, the designer can control only perceived affordances. Norman explains perceived affordances as displayed elements, e.g., a cursor or an icon, that advertise an affordance. The design of a graphical object on the screen does not 'afford clicking', the object only provides a target and helps the user know where to click. The activity to click on an object on a screen with a pointing device is in Norman's view motivated by cultural conventions, which are conventions shared by a cultural group, and not through affordances of the designed object on the screen itself. We conclude that a displayed link on the screen provides two separate layers of information to the user: 1st layer: advertises to click on the hyperlink or icon (convention) 2nd layer: perceived affordance, which indicates that the object or content of the web site behind the link affords a specific action or contains information As a composition of Aristotle's ontology with substances and accidents, and Gibson's affordance theory, we propose the following fields for the structure of the agent's mental map regarding the web site the agent is searching for (italic font indicates Aristotle's terms):

176

-

H. Hochmair and A.U. Frank

action affordances (events) physical object hierarchy (substances) attributes (qualities)

A fourth field of the mental map is formed by the activities a web user wants to perform in the web. We name the field 'user intended actions'. All fields except the attributes are either structured in a partonomy or taxonomy. The highest hierarchical layer of each field expresses the most general term in a field, elements of a lower layer are either parts of or kinds of the term in an upper hierarchy. An example of a semantic map is shown in Fig. 1. The closer the content of a web page is to the lowest elements of the fields in the semantic map, the higher is the agent's subjective feeling of being close to the target page. Therefore, the agent's goal is to reach the lowest element of each field hierarchy. This mental goal can be represented as a set of elements in the semantic map. When analyzing various web pages, we found out that those elements of the mental goal that are not provided by the content of the actual web page can in most cases be found after clicking the link of the lowest element in the user intended action. This fact allows us to simplify the criterion that determines if the goal has been reached or not: The goal is reached if the link labeled with the lowest element in the field 'user intended actions' ('order' in Fig. 1) can be perceived. User intended actions in the web are not limited to seeking (Ellis and Haugan 1997; Wilson 1997) and browsing (Marchionini 1995) but include all potential activities in the internet, such as order a product online, send an e-mail, or view a city map. 2.3 Why Using a Predefined Ontology Due to human individual life experience and different semantic maps, it is not possible to model the navigation behavior for each individual human. For this reason we create a prototype agent with a semantic map that is based on WordNet (Miller 1995), a database for the English language. The online application of WordNet can be visited at http://www.cogsci.princeton.edu/~wn/w3wn.html. The prototype agent for our model uses a semantic map for the purpose to order sneakers (jogging shoes) in the internet. WordNet combines features of both a traditional dictionary and a thesaurus. All query results are given in form of synsets (Jones 1986) which describe sets of those words which can replace a particular word in a sentence without changing the way the sentence can be employed. The synsets are connected by a number of relations. Unlike in a thesaurus, the relation between concepts and words in WordNet are made explicit and labeled; users select the relation that guides them from one concept to the next and choose the direction of their navigation in conceptual space. WordNet allows semantic queries between nouns, verbs, and adjectives.

A Semantic Map as Basis for the Decision Process in the www Navigation

177

3 Filling the Semantic Map of the Prototype Agent

3.1 Physical Object Hierarchy A hierarchy of nouns is generated by hyponymy and hypernymy relations in WordNet. Usually a noun has only one hypernym but many hyponyms (Miller 1998). Available semantic relations for nouns among others are: -

coordinate terms (terms that have the same hypernym, 'sisters') hypernyms (generic term for a whole class) hyponyms (generic term used to designate a member of a class)

To fill the field 'physical object hierarchy' we request the hypernyms of 'shoe', as the physical part of sneakers. The bold terms of the result will be included in the mental map. shoe => footwear, footgear => covering => artifact, artefact => object, physical object => … Footwear has two meanings in WordNet (covering and clothing). We make an additional step and find hypernyms for footwear in the sense of clothing (see below). For the field 'physical object hierarchy', we unite the results of the two queries. footwear => clothing, clothes, apparel, vesture, wearing apparel, wear => covering => … 3.2 User Intended Actions Like nouns and adjectives in WordNet, verbs are grouped together as sets of synonyms (synsets). English has far fewer verbs than nouns, and verbs are approximately twice as polysemous as nouns (Fellbaum and Miller 1990). The elements within the field 'user intended actions' are described by verbs. In WordNet, the user intended action 'order' is not used in the context we use it, therefore, we request the hierarchy of 'order' used as a noun. Combined with the definition of the verb 'trade' in WordNet, we get the hierarchy 'do business' - 'buy' 'order' for the field 'user intended actions'.

178

H. Hochmair and A.U. Frank

3.3 Action Affordances The verb 'run' in the sense of doing outside sport is used as a noun (gerundive) in WordNet ('running'). We search for hypernyms of the word running in the noun category and start with the term 'track, running', which is listed as one of the meanings of running. We get the following result: track, running => track and field => sport, athletics => diversion, recreation => … 3.4 Attributes A physical object can have attributes, e.g., color or size. The attributes are expressed by nouns whereas attribute values are expressed by adjectives or values. Nouns can be said to serve as arguments for attributes. What is realized in WordNet so far, is the connection between attribute nouns and adjectives which express values of that attribute, e.g., between the noun size and the adjectives large and small or between the noun color and the adjectives red, yellow, green and so on. WordNet has not implemented adjective-noun pairs so far. This means that the database does not allow to determine which are important attributes of a noun.

Fig. 1. Mental map around 'order sneakers', based on WordNet

Fig. 1 shows the model of the agent's mental map. It can be seen that we add two attributes (brand and size) for 'sneakers' to the agent's semantic model. These fields

A Semantic Map as Basis for the Decision Process in the www Navigation

179

are also hierarchically structured. The agent imagines a subjective 'ideal' value of each attribute to be included in the content of the web page that is being sought. In our example, Nike and 9 1/2 are set as the goal values for the two attributes.

4. The Web Environment

4.1 Web Structure A web environment consists of web pages and connecting links. A web domain has a start page, from which one can move into deeper levels of the domain hierarchy. Links do not mediate any Euclidean and metric information but semantic relations between web pages. Abstracting a web domain, web pages can be represented as nodes, links as edges (Fig. 2).

Fig. 2. Part of the link structure of an existing web environment

Each link contains information of the web page to which it leads. Due to this information, a navigating agent gains the cumulative information of all the links along

180

H. Hochmair and A.U. Frank

the path from the start page to the actual position in the web. A part of the information offered on the web links is also contained in the agent's semantic map. This overlap is the basis for the agent's navigation decisions. Web domains are constructed from several categories, where categories themselves can be organized as taxonomies or partonomies. Cross links connect web pages of different categories, and allow web pages to be reached through several paths. For our model we take an existing web environment (http://www.yahoo.com/) and use parts of its structure that is visualized in Fig. 2. The visualized graph shows two main categories ('do business' and 'recreate') with their subcategories. Crosslinks between different categories are visualized as dashed arrows. Links which lead to a 'dead end' and require backtracking, are visualized as thin arrows. 4.2. Treatment of Space in the www Over the past few years, in the internet the number of applications which use at least some concept of spatiality in the way they describe themselves and how they structure interactions, is increasing (Dieberger 1998). Tools like chat rooms, news groups, email lists, and Multi User Dungeons (MUD) are in widespread use. Spatialization of the web can be provided through spatial metaphors (Kuhn 1996) which allow to mimic the real world or certain aspects of it. Our model proposes a semantic navigation strategy. As navigation in graphical virtual environments and interfaces using spatial metaphors is geometrically but not semantically based, we exclude the discussion of navigation strategies in such spaces. Many human activities are related to common places. Such common places provide a context for everyday action and a means for identification of the surrounding environment (Jordan, Raubal et al. 1998). The term 'place' in our meaning does not stand for a specific geographic area, such as a state or a town, but rather for a social or federal institution. A part of the activities which are related to common places, can be mapped to the web space. Examples are posting letters from a post office (send an email), buying goods in a store (do internet shopping), meeting people in a café (chat in a chatroom) or attending school (subscribe to an internet educational course). Such user intended actions in the web are implicitly connected to the attribute 'place', therefore, no extra field 'space' is included in the agent's cognitive map. Unlike the examples mentioned above, there exist activities that are not by default associated with a place. An example of such an activity is a person searching for a satellite picture of a certain geographical area in the internet: The activity of searching and browsing in the web is not spatial itself, merely the area of interest (the satellite picture) has a spatial component. Therefore, an additional attribute 'place' is added to the agent's mental map.

A Semantic Map as Basis for the Decision Process in the www Navigation

181

5. The Navigation Process in the www

5.1 Structure of the Navigating Agent Several types of agents are introduced in (Wooldridge 1999). For our approach we take a 'utility-based agent' that has an internal state which can store information about the agent and the environment. Further, the agent applies a utility function that maps a state onto a real number. This function allows the agent to select the activity that leads him as close as possible to his predefined goal. The agent's state consists of the semantic map, a goal, the agent's actual position, and a history of positions. The agent's actual position is split: One position describes the actually visited web page in the web graph, the other position refers to the mental map and describes which element of each field has been reached in the navigation process so far. 5.2 The Navigation Strategy The agent starts at the index page of the given web domain. During the navigation process, the agent perceives a number of links with the labeled information. The agent's strategy is to select the link, of which the corresponding element in the semantic map has the shortest path to its predefined goal in the mental map. The strategy serves for the local decisions at each node, but it is not capable of providing the shortest overall path to the target page in the web (in the sense of the number of mouse clicks to reach the target web page). Choosing the overall shortest path in the web would require knowledge of the web structure (e.g., through reading a site map or exploration of the web environment) or navigating along a familiar path. As we exclude knowledge of the web structure (and therefore the familiarity with any path) from our model, we assume that the agent relies on his semantic map and chooses the shortest path within that mental map. 5.3 Visualization of a Navigation Step We give an example to visualize the web navigation strategy (Fig. 3). The agent's web position is at the start node '1' of the web domain in Fig. 2. The agent perceives two links ('do business' and 'recreate') and finds the corresponding position of these terms in the mental map: 'do business' is part of the field 'user intended actions', 'recreate' is member of the field 'action affordances' (see Fig. 1). The agent determines the shortest path between the perceived links and their goals in the mental map. As the length from 'do business' to 'order' amounts to 2 steps, and the length from 'recreate' to 'running' amounts to 3 steps, the link 'do business' is chosen at this point of decision (see Fig. 3).

182

H. Hochmair and A.U. Frank

Fig. 3. Determining the best link through the corresponding element in the mental map

6. Simulation In this section we simulate a navigating agent in a web environment, following the web navigation strategy introduced before. We realize the navigation model through algebraic descriptions using classes with their functions. The model contains abstractions of the web environment, and a navigating agent whose decisions are based on a mental map. We implemented the algebraic specifications through the functional programming language Haskell (Thompson 1996). In this paper we show few lines of code and restrict ourselves mostly to describing the functionality of the model. The complete code can be downloaded from ftp://ftp.geoinfo.tuwien.ac.at/hochmair and will not be listed here. The implementation of the formalized model allows to assess a given web environment with regard to ease of navigability. Various software agents can be tested in existing web environments, 'dead ends' and loops can be detected while the agent is navigating from the start page to the target web page. The web environment and the agent's mental map stay invariant during the web navigation process whereas the agent's positions in the web and the mental map change with each navigation step. 6.1 The www Environment and the Mental Map For modeling the navigation process we abstract web pages in the web domain as nodes and the connecting links as edges. The web nodes, as shown in Fig. 2, consist of an integer identifier, but do not contain semantic information. The semantic information of the web environment is stored in the web links, they contain the semantic information of the web page they are leading to. A data type representing the web graph is built with the constructor function G EW NodeW, where G indicates a graph, EW stands for a list of elements (a list of NodeW here) with an additional string for the semantic information, and NodeW is the data type representing a web node. Similar to the web environment we abstract elements in the mental map as nodes and their connections as links. Contrary to web nodes it is the nodes in the mental map, which carry the semantic information. A field in the mental map is coded in the form Field EM NodeM, where Field constructs a graph with an additional string

A Semantic Map as Basis for the Decision Process in the www Navigation

183

(for the name of the field), EM stands for a list of elements (without any additional semantic information) and NodeM expresses a mental node. As the mental map consists of several fields (see Fig. 1), we unite all fields in a list to create the complete mental map. Square brackets symbolize a list in Haskell, therefore the data type of a mental map is represented as [Field EM NodeM]. The following lines demonstrate how to create the web environment of Fig. 2 in the simulation. The data are initially given in a list of triples consisting of start node, end node and the semantic content of the link (webStrings). The makeEW function takes one triple and converts it to a web link. The map function applies makeEW to each element of the webStrings data, and the insertG function inserts the resulting list of web links into an empty graph. webStrings = [(1,2,"do business"), (1,3,"recreate"),...] wmap = foldr insertG (G []) (map makeEW webStrings)

Creating the fields of the mental map is a similar process. We demonstrate the process for the field 'user intended actions' (mmapUia), which contains three nodes. To get the complete mental map, all fields are united in one list, which is labeled mmap. mmapUiaS = [("do business", "buy"),("buy","order")] mmapUia = foldr insertF (Field "Uia" (G [])) (map makeEM mmapUiaS) mmap = [mmapUia, mmapPhys, mmapAa, mmapBrand, mmapSize]

The agent's mental goal is formalized as a list of edges. Each edge describes the goal for one single field and consists of the fieldname (data type String) and the goal node (data type NodeM). For all elements of the mental goal see Fig. 1. 6.2 The Navigating Agent During the navigation, the agent stores the data in the agent's state (see section 5.1) which is needed for the further decision process. In the agent's state, the web position is stored as a web node, the mental position and the goal are stored as lists of edges (each of them consisting of a string and a mental node), the mental map as a list of fields, and the history as a list of tupels, where each tuple contains the agent's visited position in the mental map and in the web. The agent is created through the definition of a new data type Agent, which consists of the five elements of the agent's state. We construct a virtual agent who navigates in the given web environment visualized in Fig. 2. The following lines of code show how the agent's state is filled with data. The agent, called fred1, starts at the top page of the domain at web node 1 (posw). His mental position (posm) has an empty value for each field (defined in posstartM), the set of goal elements (goal1), and the mental map (mmap) are the ones defined above, the history list (hi) of previously visited nodes is empty. fred1 = Agent where posw posm goal

posm goal mentalmap posw hi = NodeW 1 -- position in web = posstartM -- position in mental map = goal1 -- mental goal

184

H. Hochmair and A.U. Frank mentalmap = mmap hi = []

-- mental map -- history list

To define the agent's operations - they cause changes in the agent's state - we define a class State. The class signature describes the parameters of the two included operations step and navigate. Both operations have a web graph (Web) and one parameter (agent) as input, and have the changed parameter as output. In the instance for the data type Agent, elements of the state, such as the agent's position or history, may change through one of the operations. The step function defines one single navigation step of the agent, whereas navigate performs a complete navigation process from the start page to the target using the step function recursively. class State agent where step :: Web -> agent -> agent navigate :: Web -> agent -> agent

The step function consists of several sub activities that are implemented in the instance of the class State for a data type Agent: -

a 'step' in the mental map which changes the actual position in the mental map a step in the web environment, which changes the position in the web graph update of the history list

We use three conditions to break the recursion in the autoNav function: The first condition is the perception of the link with the lowest element in the 'user intended actions' field. This indicates that the goal is reached. The second condition is given through a limitation of navigation steps. A specific function checks the number of elements in the history list to be smaller than a given maximum value. If the length of the history list is higher than the maximum value, this indicates that the agent is caught in a loop. The third condition for a break is that no 'useful' link can be perceived, which means that no information of the provided links can be matched with the elements of the semantic map. 6.3 The Decision Process The core of the decision process is the shortest path algorithm of Dijkstra (1959) as given by Kirschenhofer (1995). It is used for the assumption that the agent determines the shortest possible path in the mental map (see section 5.2). The shortest path function has two nodes and a graph (a field in the mental map) as input and returns a list of nodes. The result describes the shortest path between the two given nodes in the graph, in our case between an element of a mental field and its target element of the goal definition. To come to a decision, several substeps have to be performed mentally by the simulated agent: - determine all outgoing links from the actual web page - find the corresponding field and goal in the mental map for the information on each of the perceived links - calculate the length of the mental shortest path from each matched element to its goal in the mental map

A Semantic Map as Basis for the Decision Process in the www Navigation

-

185

select the link with the shortest mental distance from its corresponding goal in the mental map

7. Case Testing

7.1 Single Step This section demonstrates the simulated agent's navigation process in the web environment of Fig. 2. The agent starts at web node 1 and perceives two links, 'do business', and 'recreate'. We apply the step function on the agent to see the result for one single step. Test input> step wmap fred1 >> POSMentalMAP Uia 'do business', Phys '', Aa '', Brand '', Size '', POSWEB 2, HISTORY [ POSMentalMAP Uia 'do business', Phys '', Aa '', Brand '', Size '', PosWEB 2 ]

The result gives us information about the agent's position and the link he has decided to choose. We see that the agent's mental position (POSMentalMAP ) has changed to 'do business' in the field 'user intended actions' (Uia) of the mental map. The other fields have not been entered yet, which is indicated by the empty strings of the mental position. The agent's web position (POSWEB) changes to node 2 (see also Fig. 3). 7.2 Complete Navigation Process To perform a complete navigation process trough the web environment we apply the navigate function. Test input> navigate wmap fred1 >> POSMentalMAP Uia 'order', Brand 'Nike', Aa 'do sport', Phys 'shoe', Size '', POSWEB 22, HISTORY [ POSMentalMAP Uia 'do business', Phys '', Aa '', Brand '', Size '',PosWEB 2 ] , [ POSMentalMAP ...Uia 'buy'..., PosWEB 4 ] , [ POSMentalMAP ...Phys 'clothing'..., PosWEB 6 ] , [ POSMentalMAP ...Phys 'shoe'..., PosWEB 11], [ POSMentalMAP ...Brand 'brand'..., PosWEB 14 ], [ POSMentalMAP ...Aa 'do sport'..., PosWEB 16 ], [ POSMentalMAP ...Brand 'Nike'..., PosWEB 18 ], [ POSMentalMAP Uia 'order', Brand 'Nike', Aa 'do sport', Phys 'shoe', Size '', PosWEB 22 ] GOAL REACHED = True

A Semantic Map as Basis for the Decision Process in the www Navigation

187

map, navigation decisions can vary between individual users, although the same strategy is used. This fact means a challenge for web designers: The web environment and its links must be structured in such a way that the target web page can be reached via several paths, applicable for users with different mental maps. The simulation of the model helps to detect critical navigation situations in a web environment and to improve the link structure in a given web domain. As we restricted ourselves to proposing a web navigation strategy in this paper, we did not discuss in detail the types of potential errors during the navigation process, e.g., getting lost in a 'dead end' or being caught in a loop of links. Such errors result from differences between the web user's and the web designer's mental maps. As one cannot directly observe a person's mental map, human subjects testing is a method to obtain more realistic models (than the ones constructed from WordNet) of a web user's mental map. Through analyzing the decisions of the test persons who navigate through an unknown web domain, the content and structure of the semantic map may be constructed stepwise. The results may be used to improve the link structure of a specific web domain. We consider these ideas as part of the future work. Besides this, a goal for future work is to compare error taxonomies and navigation strategies in different environments (such as the web and the real world) using the algebraic structures of the formalized models. Another goal is to compare further features of navigation strategies such as shortcuts and deviations in different environments. A more general web navigation model with additional strategies and features may allow a more comprehensive comparison of navigation methods in the real world and the web.

References Barsalou, L. W. (1987). The instability of graded structure: implications for the nature of concepts. Neisser: 101-140. Dieberger, A. (1998). Social Connotations of Spatial Metaphors and Their Influence on (Direct) Social Navigation. Workshop on Personalized and Social Navigation in Information Space. A. M. K. Hook, D. Benyon. Kista, Sweden, Swedish Institute of Computer Science. Dijkstra, E. W. (1959). “A note on two problems in connection with graphs.” Numerische Mathematik 1: 269-271. Ellis, D. and M. Haugan (1997). “Modelling the Information Seeking Patterns of Engineers and Research Scientists in an Industrial Environment.” Journal of Documentation 53(4): 384-403. Fellbaum, C. and G. A. Miller (1990). “Folk psychology or semantic entailment? A reply to Rips and Conrad.” The Psychological Review 97: 565-570. Gibson, J. J. (1977). The Theory of Affordances. J. Bransford. R. E. Shaw. Hillsdale, NJ, Lawrence Erlbaum Associates. Hochmair, H. (to app.). Adapting One's Mental Model: An Essential Process for Successful Navigation in an Environment. Spatial Information in the Environment, Innovations in GIS 8, Taylor & Francis. Jones, K. S. (1986). Synonymy and semantic classification. Edinburgh, Edinburgh University Press.

188

H. Hochmair and A.U. Frank

Jordan, T., M. Raubal, et al. (1998). An Affordance-Based Model of Place in GIS. 8th Int. Symposium on Spatial Data Handling, SDH'98, Vancouver, Canada (July 11-15, 1998), International Geographic Union. Kirschenhofer, P. (1995). The Mathematical Foundation of Graphs and Topology for GIS. Geographic Information Systems - Materials for a Post Graduate Course. A. U. Frank. Vienna, Department of Geoinformation, TU Vienna. 1: 155-176. Kuhn, W. (1996). Handling Data Spatially: Spatializing User Interfaces. 7th Int. Symposium on Spatial Data Handling, SDH'96, Delft, The Netherlands, Faculty of Geodetic Engineering, Delft University of Technology. Kuipers, B. (1978). “Modeling Spatial Knowledge.” Cognitive Science 2. Lakoff, G. (1987). Women, Fire, and Dangerous Things. Chicago and London, The University of Chicago Press. Marchionini, G. M. (1995). Information Seeking in Electronic Environments. Cambridge, England, Cambridge University Press. Miller, G. A. (1995). “WordNet: A Lexical Database for English.” Communications of the ACM 38(11): 39-41. Miller, G. A. (1998). Nouns in WordNet. WordNet - an electronic lexical database. C. Fellbaum. Cambridge, Massachusetts, MIT Press. Norman, D. A. (1999). “Affordances, Conventions, and Design.” interactions. Peuquet, D. (1998). Cognitive Models of Dynamic Phenomena and their Representations, http://www2.sis.pitt.edu/~cogmap/ncgia/peuquet.html. Raubal, M. (to app.). “Ontology and Epistemology for Agent-based Wayfinding Simulation.” 15(7) IJGIS. Raubal, M. and M. Worboys (1999). A Formal Model of the Process of Wayfinding in Built Environments. Conference on Spatial Information Theory - Cognitive and Computational Foundations of Geographic Information Science. C. Freksa and D. Mark. Berlin-Heidelberg, Springer-Verlag. 1661: 381-399. Rips, L. J., E. J. Shoben, et al. (1973). “Semantic distance and the verification of semantic relations.” Journal of Verbal Learning and Verbal Behavior(12): 1-20. Rosch, E. and C. B. Mervis (1975). “Family resemblance: Studies in the internal structure of categories.” Cognitive Psychology(7): 573-605. Sholl, M. J. (1987). “Cognitive maps as orienting schemata.” Journal of Experimental Psychology: Learning Memory, and Cognition 13(4): 615-628. Smith, B. (2001). Objects and Their Environments: From Aristotle to Ecological Ontology. The Life and Motion of Socioeconomic Units. A. U. Frank, J. Raper and J.-P. Cheylan. London, Taylor and Francis. Smith, E. E., E. J. Shoben, et al. (1974). “Structure and process in semantic memory: A featural model for semantic decisions.” Psychol. Rev.(81): 214-241. Svensson, M. (1998). Social Navigation. Exploring Navigation; Towards a Framework for Design and Evaluation of Navigation in Electronic Spaces. N. Dahlbaeck. Kista, Sweden, Swedish Institute of Computer Science. Thompson, S. (1996). Haskell - The Craft of Functional Programming. Harlow, England, Addison-Wesley. Wilson, T. D. (1997). “Information Behavior: An Interdisciplinary P Processing & Management 33(4): 551-572. Wittgenstein, L. (1953). Philosophical Investigations. New York, Macmillan. Wooldridge, M. (1999). Intelligent Agents. Multiagent Systems - A modern Approach to Distributed Artificial Intelligence. G. Weiss. Cambridge, Massachusetts, The MIT Press.

Pragmatism and Spatial Layout Design Susan L. Epstein1, Bernard Moulin2, Walid Chaker2, Janice Glasgow3, and Jeremi Gancet2 1

Department of Computer Science, Hunter College and The Graduate School of The City University of New York, New York, NY 10021, USA [email protected] 2 Computer Science Department and Research Center of Geomatics, Laval University, Ste Foy, QC G1K 7P4, Canada [email protected], [email protected], [email protected] 3 Department of Computing and Information Science, Queen’s University, Kingston, Ontario, K7L3N6, Canada [email protected]

Abstract. Design problems address multiple, ill-defined goals in a computationally intractable search space. Our approach to spatial layout design capitalizes on devices that people use to control search. We advocate multiple passes at increasing, class-based levels of granularity, with heavy infusions of knowledge. Our approach makes brief, inexpensive searches from good starting points, seeking ideas that satisfy multiple criteria simultaneously. The result is an autonomous program for two-dimensional layout design of recreational parks. Keywords. spatial design, constraint-based reasoning, cognitive structure of spatial knowledge, social and cultural organization of space, structure of geographic information

1 Introduction Although computer science has made inroads in many areas of human expertise, design has thus far been mostly restricted to computer-aided design, with good reason. Design has many of the classic characteristics of an artificial intelligence (AI) problem — its practitioners make extensive use of domain-specific knowledge; it requires symbolic reasoning with both qualitative and numeric methods; it is laden with inexact, missing, or ill-defined information; and the problems themselves lack clear, clean algorithmic solutions. Nonetheless, the thesis of this work is that it is possible for a program to formulate original, task-specific solutions to a design problem. Our focus here is two-dimensional spatial layout design, and our investigative domain is recreational parks. In Anatomy of a Park, Molnar and Rutledge identify three phases of park construction: survey, analysis, and synthesis [1]. In the survey, the designer develops constraints and goals, seeking out purpose-directed reasoning as well as pre-existing resources and constraints. In the analysis, the designer extracts the relationships among the elements to be included, and in the synthesis a design is produced and then refined. Our work focuses only on synthesis; we assume that survey and D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 189−205, 2001.  Springer-Verlag Berlin Heidelberg 2001

190

S.L. Epstein et al.

analysis have already been performed by able humans. Thus the problem is to provide a design that incorporates all the elements while abiding by the constraints and goals. Our approach, pragmatic design synthesis, is a limitedly rational one to solution generation, an approach modeled on human designers [2-4]. For the two-dimensional layout design of a recreational park, pragmatic design synthesis capitalizes on the ways that people address the problem: object classes, levels of detail, and the infusion of knowledge. Our primary resources have been a successful empirical investigation for theoretical parks on an empty grid [5], and expert writings on architectural and landscape design for real-world data [1, 6-15]. The theory described here is the foundation for an ongoing, large-scale project that employs real-world data and provides urban and landscape architects with automated support for the design, or redesign, of parks. The next section of this paper defines spatial layout design, introduces our running example, explains the challenges inherent in spatial layout design, and targets some devices that people use to solve these problems. The third section constitutes our theoretical approach to the design of recreational parks. The fourth section discusses the roles in design of geographic information, alternatives, and navigability, and includes implementation results.

2 The Problem Informally, within some setting spatial layout design creates an arrangement of a set of partially-specified objects according to a set of objectives. The setting is called the design frame; it includes both the site, the geographical area in which the objects are to be arranged, and the periphery, the area immediately adjacent to the site. The design frame also includes any pre-existing objects deemed fixed, such as a body of water, a public utility location, or access to a transportation system. Fig. 1 begins our running example of a spatial layout design problem for a recreational park. The site includes a pond; the periphery includes two streets and two avenues. Each object in spatial layout design has a prespecified description, its property values. These may be detailed and rigid (e.g., a tennis court whose dimensions are fixed and whose orientation to sun and wind are relevant) or more flexible (e.g., a sandbox). Spatial layout design is said to anchor an object within a site when it sizes, fixes, and orients the object according to its property values. A building specified only as 600 square units, for example, might be anchored by making it a 20 × 30 rectangle centered on a particular point on the site, and oriented so that its shorter side faces due north. A problem in spatial layout design is specified by a design frame, a set of objects (e.g., the list in Fig. 1) with descriptions, and criteria, a set of specifications for the way the objects are to be anchored. The criteria are either required (constraints) or desired (principles). Constraints typically describe the objects themselves (e.g., size, shape, compass orientation) and their placement on the site, relative to the site’s boundaries and relative to each other (e.g., a parking lot near the periphery, a restroom near a playground). Principles serve as metrics for good design; they are more vague than constraints, and are generally conditions to be optimized, rather than tested. Examples of principles include ease of access from a particular location to the periphery, and aesthetic pleasure during traversal.

Pragmatism and Spatial Layout Design

191

Hospital

Avenue A

Bus stop

Subway

pond

Objects bandshelter restaurant 3 tennis courts sandbox swings administration building 7 benches 3 storage sheds

Key

Bus stop Subway

Avenue B

Bus stop

water line sewage line power source

Fig.1. A design frame for an urban park with a list of objects to be included

A solution to a problem in spatial layout design anchors all the objects within the site, and satisfies all the constraints, without any of the objects overlapping. Thus design synthesis accepts as input a design frame, objects to be anchored within the site, and criteria that describe the way those objects may be anchored. Design synthesis produces as output a set of designs that anchor all the objects in accordance with the constraints and are highly-rated by the principles. Problems in spatial layout design, along with its solution, can be treated as a cases (previously solved problems and their solutions) in a case-base of design experiences. Case-based reasoning in this domain can provide initial designs, adapt existing designs, or be used to evaluate possible solutions. Case-based reasoning has previously been applied to the problem of landscape design. Rather than focusing on the initial design layout, the CYCLOPS system [16] combines constraint-based solution generation with case-based reasoning to repair landscape architecture designs. 2.1 The Issues Design problems are hard because, in very large search spaces, they aspire to multiple ill-defined goals [2]. A solution to a design problem is expected to satisfy both specific, real-world constraints (e.g., size, cost) and less well-defined, but nonetheless relevant specifications (e.g., aesthetic properties). This section explores the difficulties design problems present within the traditional AI approach. Consider a design synthesis problem that specifies n objects to be anchored within an otherwise empty site. Each object is already sized and oriented; one need only locate the objects with respect to the single constraint that they do not overlap. In this event, anchoring need only specify a location, the coordinates for the center of each object. A design synthesis state could then be described as a vector of n elements, each of which

192

S.L. Epstein et al.

is a location or “unknown.” If any entry in the vector is “unknown,” the state is said to be partial; otherwise it is complete. If the design described by the vector abides by the no-overlap constraint, the state is said to be legal; otherwise it is illegal. A solution is simply a legal, complete state. The traditional AI approach to design synthesis would be to place one object at a time on the grid in a way that satisfies the constraints, until all objects are placed. If objects remain to be placed, but no unplaced object has a placement that does not overlap an already-placed object, the program would backtrack to some previous choice point, change that decision, and retract all the decisions that had followed it. This can be characterized as search for a solution through a space of legal, partial states. Unfortunately, the number of possible anchorings is enormous, even for a simple problem of this kind. One particular problem with only 13 objects had about 1.3 × 1026 possible complete states for those objects on a small grid, the vast majority of which were illegal [5]. Thus design synthesis appears to require search through an unmanageably large space. Equally problematic is that, although enlarging the site size may simplify a design problem for people, it makes it more difficult for this kind of search — each object then has more possible locations, which increases the branching factor. As described thus far, state-space search has also ignored a hallmark of design that distinguishes it from many other AI problems: its insistence upon multiple goals. The 13-object problem, for example, included 14 fairly simple constraints, such as “this object must be tangent to the northern boundary of the site” or “this object must have its center within 80% of the site’s center.” Additional constraints eliminate previouslylegal complete states, so that solutions in the state space are even more sparse. Recall that, in addition to its required constraints, the design problem has a set of desired principles. The correct, yet computationally intractable, approach for a solution that optimizes the principles is to examine every solution and select the one the principles rate highest. A program cannot afford to neglect these principles, for they include the aesthetics that have thus far made design the province of people. Principles, however, tend to be vague and computationally costly [3]. Yet, somehow, human designers produce satisfactory solutions. 2.2 Human Techniques The approach we advocate capitalizes on practical techniques human designers use to control search: object classes and granularity, knowledge infusion, and agent-based simulation. Human layout design experts typically assign specified objects to classes, sets of functionally and/or structurally similar objects [3]. Thus the sandbox and the swings of Fig. 1 would be in the playground class, the restaurant and the band shelter in the entertainment class. A particularly important object, such as the administration building, could form its own class. This perspective transforms the list of objects in Fig. 1 into the classification of Table 1, which groups objects by the activity they are intended to support. Here a use area is a section of the park intended for a particular activity, such as a playground or an entertainment area. All objects in a use area must belong to the same class, but there may be more than one use area with the same class of objects, for example, there may be two playgrounds in a large park.

Pragmatism and Spatial Layout Design

193

Table 1. Object classes for the problem of Fig. 1

Use area Maintenance Entertainment Playground Playground Playing fields Playing fields any any

Type major major major minor major minor minor minor

Objects Administration building Band shelter, restaurant Sandbox, swings Bench 1, bench 2, bench 3 Tennis court 1, tennis court 2, tennis court 3 Bench 4, bench 5 Bench 6, bench 7 Shed 1, shed 2, shed 3

Classification permits a designer to focus either on fewer objects at a time (those within a class) or, as we shall see, only on the use areas themselves [1]. Indeed, human experts produce designs of increasing granularity (level of detail). Their early designs focus on key objects or classes of objects, while subsequent efforts embellish the early work with details. 2.3 Computational Techniques In general, design problems in AI are defined by constraints, and the problem solver is expected to provide a solution that satisfies them. The constraints may either underspecify the problem, so that there are many possible solutions, or overspecify the problem, so that no set of anchorings satisfies all the constraints. To date, most automated design systems have supported human designers, rather than initiated designs of their own. For example, ARCHIE [17] helps architects carry out conceptual design, and AskJef [18] assists human-computer interface designers. A major result in AI is that properly-formulated knowledge expedites and refines search. Human experts rely heavily both on compendia of information (such as the 154-page soil treatise [19]) and on cases. The 13-object problem is difficult, in part because its site is an empty grid that offers no further restrictions. In the real world, sites provide extensive data, data that can be used to guide search. Section 4 further discusses the role of different knowledge sources in spatial layout design.

3 Pragmatic Design Synthesis Pragmatic design synthesis uses the reformulation of Table 1 to execute three passes, from sketch to plan and finally to design. Each pass works with a different level of granularity, and each involves multiple, resource-limited searches.

194

S.L. Epstein et al.

3.1 The Sketch The first step in the design process, at the coarsest level of granularity, positions use areas. A successful search in this pass ends in a sketch. Kinds of use areas include playgrounds, sports fields, water-related activities, entertainment, forests, meadows, food service, maintenance facilities, and container network skeletons. A container network is a connected graph intended to transport substances or objects. Examples of a container network include roads, power lines, and hiking trails. At its coarsest level, in the sketch, a container network is represented by a container network skeleton that gives the fundamental but unelaborated shape of the container network to be used in the sketch. For example, Frederick Law Olmstead, a famed designer of urban parks, favored an oval about three-fourths the size of the site for his road skeletons. The specification of the typical use area stipulates its left-right (longest embeddable horizontal line in the plane) and top-bottom dimensions (longest embeddable vertical line in the plane). It may also specify expected traffic (source and destination with respect to other use areas and the periphery, including estimated volume and frequency of access), and tolerances for deformation and rotation. Tolerance for deformation indicates the manner and the degree to which the boundary of the oval may be stretched or contracted. Each use area also has a refinement function, explained in Section 4.1. Container networks require special treatment — because their construction is computationally difficult, pragmatic design synthesis includes a case library of container network skeletons. Use area constraints frequently describe distances between pairs of use areas. One might, for example, specify that a picnic area be near a pond, that a playground be near a bus stop, or that the maintenance area be near the entertainment area but far from the picnic area.

Hospital

Avenue A Subway

Bus stop

playground

pond maintenance entertainment picnic area

playing fields

Bus stop Subway

Avenue B

Fig. 2. A sketch for the design frame of Fig. 1.

Bus stop

Pragmatism and Spatial Layout Design

195

A sketch anchors each use area in the site. A use area appears as a labeled oval whose area approximates its expected footprint (projection onto the plane of the site) in the final design. For the use areas of Table 1 and the design frame of Fig. 1, the sketch in Fig. 2 might be generated 3.2 The Plan Next, within each use area, the second pass positions major items. A successful search of this kind ends in a use area plan. Detailing the best sketches with the best use area plans for each of the use areas produces plans. Each use area contains any number of objects that are either preexisting within the site or constructed for it, such as a band shelter in the entertainment area or a sandbox in the playground. An object is major if it is costly (e.g., band shelter) or is of particular functional significance to its area (e.g., a sandbox), otherwise it is minor. Major object classes are determined by their use areas. For a playground they include swings, sliding boards, sandboxes, and climbing facilities; for a sports field, baseball diamonds, soccer/rugby/football fields, tennis courts, basketball courts, and skating rinks; for water-related activities, ponds, boat-houses, swimming pools, streams, and bridges; for entertainment, band shelters, performance stages, audience areas, merrygo-rounds, and zoos; for food service, restaurants and kiosks; for maintenance, large equipment storage, equipment repair facilities, and administration buildings; for container networks, branches; and for any use area, parking. A major object to be placed within a design space specifies a particular use area in which it may be anchored; a minor object may or may not do so. Each object has a leftright and a top-bottom dimension, and may also specify (or inherit as defaults) a footprint, scale (relative to the site and/or the other objects present), orientation to natural forces (e.g., sun, wind, precipitation), and cost (as a function of size and material). An object may either be specified with precise property values (e.g., a rectangular 100 by 60 foot tennis court) or by a range of values (e.g., a 6000 square-foot convex object with a height-to-width ratio in [2, 3]). Like a use area, an object may have a refinement function. (See Section 4.1 for details.) Once again, container networks require special treatment. In addition to its skeleton, each container network has objects called branches (major connection objects) from its skeleton to other objects. Branches must connect to the network, and serve to elaborate it. A branch is specified by its network and the object or objects it must connect. A point where a branch joins the skeleton or terminates at an object is called a node. If the object is a required branch in a container network, it will also have endpoints and a capacity (e.g., a two-lane road from a parking lot to the periphery). Thus each network is represented by an attribute graph on nodes that records distances and capacities. For each use area in a sketch, the second pass of pragmatic design synthesis creates a use area plan that anchors each major object in the site, retaining the borders of the use areas as fiat boundaries [20]. For the objects of Table 1, plus branches of the road network from the skeleton to the northern and western edges of the site, one possible plan from the sketch of Fig. 2 is Fig. 3.

196

S.L. Epstein et al. Hospital

Avenue A

Bus stop playground

Subway

pond

entertainment

maintenance

picnic area playing fields

Bus stop Subway

Key

water line sewage line power source road

Avenue B bandshelter restaurant

Bus stop

sandbox swings tennis court administration

Fig. 3. A plan for the sketch of Fig. 2

3.3 The Design The third and final pass, at the finest granularity level, has two stages. In its first stage, it refines a plan by positioning all the minor items. The result is a set of computeroriginated designs. Each object class has an owner, an agent realized by a set of design procedures specialized for that class. Minor objects may be anchored by their owners anywhere in the site that satisfies the constraints. (There is a general procedure to do this for any minor object not associated with a use area.) Because minor objects are expected to be small and have few restrictions, often any random location in the vicinity suffices. If the requirements for the running example included seven benches (three allocated to the playground, and two to the playing fields area) and three storage sheds, one possible design for the plan of Fig. 3 appears in Fig. 4. The function of the principles, the non-required design criteria, is to evaluate design alternatives. Every criterion has an evaluation function that measures the degree to which any state meets its particular specification. A constraint’s evaluation function is boolean — the state either meets it or does not. A principle’s evaluation function, however, is numeric; it may also be complex and make heavy computational demands. After all plans are completed, the second stage of the third pass tweaks them, that is, it re-anchors objects in an attempt to improve the scores received from the evaluation functions. Tweaking is directed by the principles; so, for example, if “buildings should

Pragmatism and Spatial Layout Design

197

Hospital

Avenue A

Bus stop playground

Subway

pond

entertainment

maintenance

picnic area playing fields

B

Bus stop Subway

Avenue B

Bus stop

Key water line sewage line power source road

bandshelter restaurant

B bench

sandbox swings tennis court administration storage shed

Fig. 4. A design for the plan of Fig. 3

be close together” is a principle, the system may shift one building toward another in a plan, re-rate the resultant plan, and choose the higher-scoring of the two. In empirical testing on simple problems [5], such tweaking was able to raise the best design’s score by as much as 30%. Principles that are particularly costly to evaluate may be reserved, so that they are used only in initial formulations (of a sketch or a plan) and in tweaking, but excluded from anchoring iterations.

4 Discussion 4.1 Geographic Information for Design Classification is, to our minds, the essential geographic knowledge for design: it links objects with their use areas and permits design to focus on fewer objects and the constraints relevant to them. Classification also supports the formulation of three kinds of criteria: those that pertain to a single object (object specifications), those for a set of objects in the same use area (intra-area specifications), and those for a set of objects in more than one use area (inter-area specifications). The object specifications derive from the survey data; the intra-area specifications are based on both survey data and

198

S.L. Epstein et al.

expert knowledge. Inter-area specifications, however, arise primarily from the needs people have within the park. For example, people may require efficient travel, aesthetic views, or informative views. The park management, meanwhile, may require efficient storage and retrieval of equipment, energy conservation, freedom of movement for vehicles, and control for safety. Once objects are classified, the initial states for both the sketches and the use area plans rely heavily upon knowledge infusion. Even the anchoring of half a dozen use areas to form a sketch can prove computationally expensive. As noted above, the survey phase of park design includes an inventory of pre-existing site features, used by landscape architects to restrict the number of possible design solutions. Landscape architects typically collect terrain data on features such as drainage, slope, soil types, and vegetation, and represent each set of measurements as a site map, a separate graphical model. People use site maps to help locate use areas [1]. Each site map can be thought of as a separate, fine-grained grid, where each grid cell represents the value of the corresponding measurement. For example, if slope measurements were categorized into five ranges, each grid cell would be assigned a value corresponding to one of the five ranges, based upon the survey data. In park design, the nature of the land within the site (e.g., slopes, drainage, soil, and existing vegetation) naturally provides terrain constraints. These constraints preclude many possible anchorings, and make some of the remaining ones far more attractive than others. One would not, for example, anchor a camping area on marshy ground, or a playground on a steep slope. Table 2 offers a simplified example of how categorized site data qualitatively determine a land type, and how land type then dictates construction appropriate to it. (For example, roads require terrain that is at least category III.) The efficient generation of use area plans is also directly attributable to classification. Given a sketch, each use area is detailed with its major objects, based upon the relevant inter-class constraints, to produce a use area plan. Recall that each object class Table 2. Categorized site properties characterize land types [1]. Each type is suited to a particular level of construction.

Property Slope 0-2% 2-4% 4-10% 10-20% 20%+ Drainage Good Fair Poor Soil Good Fair Poor Vegetation Good Fair Poor

I x x x x x x x x x x x x x x

IIA x x

Type IIB III IV x x x x x x x x x

x x

x x

x x

x

x x

x x

x

x

x

x

x x x

Type I IIA IIB III IV

Construction None Minimal Minimal Moderate Intensive

Usage trails playing fields picnicking roads, parking all structures

Pragmatism and Spatial Layout Design

199

has an owner (Section 3.3). The owner of an object class is responsible for the satisfaction of any intra-class constraints applicable to its objects. Each owner has a case-based library of ideal frameworks, use area plans for a set of objects that abide by all constraints, and are rated highly by all the owner’s principles, including any reserved ones. The playground owner, for example, has some ideal frameworks for playgrounds with much climbing equipment, and other ideal frameworks with objects geared to very young children. The owner selects and modifies a library plan to fit the set of major objects in the use area under consideration, much the way the sketch is constructed. Each iteration anchors a major object. It is the restricted focus to objects of the same class that makes these searches manageable. Refinement functions, both for use areas and for objects, also incorporate spatial knowledge into pragmatic design synthesis. The purpose of a refinement function is to define an object better, either when it is anchored or during tweaking. A refinement function might be expected to produce an aesthetically appealing footprint, or to layout the spaces in a parking lot. One refinement function might prevent an artificial pond from deformation into a stream; another might maintain the convexity of a building. If the object is an aggregate (e.g., forest or parking lot), its refinement function will also specify its components (e.g., trees, parking spaces). Thus geographic knowledge pertinent to layout design includes object categories, terrain constraints, ideal frameworks, and refinement functions. When infused with such knowledge, a program should be able to create a variety of legal sketches efficiently. 4.2 Alternatives as Design Devices Discussion of one important aspect of the process has been deferred until now: the encouragement of multiple alternatives. Although the process moves from sketch to plan to design, it does not focus on one “best” idea. Instead, like human experts, pragmatic design synthesis attempts to produce s different sketches, and p plans for each use area. As a result, with a use areas there may be as many as spa plans from which to produce a design. (All italicized letters here denote user-set search parameters.) To produce an individual sketch, search begins from an initial state (see Section 4.4) and, on each iteration, re-anchors one or more use areas. At each iteration the current sketch is scored by the (non-reserved) use-area principles specified in the problem. Each of the b most appropriate cases serves as the initial state for c searches, and each search is iterated d times. The final sketch in each search is retained only if it achieves a sufficiently high score. Once s sketches are produced this way, or there have been bc searches, the first pass ends. In the second pass, a set of use area plans for each use area is generated in a similar manner. Every owner has a recommendation function that generates proposals, suggestions on how to anchor one of its objects in a state. A recommendation function respects the properties of an object or use area (e.g., orientation) and can be tailored to an object’s class. For example, it may be appropriate to lop off the corner of a forest or bend a road around an obstacle, but neither of those would be appropriate for a tennis court. To produce a use area plan, search begins from any of e good matches for the current use area from among the owner’s ideal frameworks. Search iterates until p use

200

S.L. Epstein et al.

area plans are produced for each use area. An iteration entertains proposals from the owner of the use area to anchor (or re-anchor) a major object there. The (non-reserved) inter-area principles score the use area plan that would result from each proposal, and the f with the highest scores are used to create (full) plans. A preliminary plan is a combination of high-scoring use area plans. Each preliminary plan is then scored by the principles, and the best of them are forwarded as plans. In the third pass, each plan is detailed into a design, with anchorings proposed by the owners of the minor objects. Rather than iterate on these inexpensive decisions, the minor objects are distributed into each plan g times, and the h highest-scoring designs are selected. Every selected design is then tweaked, and the i best of those are output from the system. In the event that there are not enough appropriate matches to provide initial search states, cases may be generated at random, and the highest-scoring among them substituted for historical knowledge. Indeed, in the 13-object problem of Section 2 there were no cases, and randomly-generated initial states served quite well, as long as 2530% of all computation time was devoted to their production and evaluation [5]. In any event, some randomly-generated cases may be desirable for novelty. 4.3 Navigability and Other Principles Most principles are intended to capture aesthetic considerations. Once a use area or an object is anchored, one can determine the degree to which it is remote from a particular location (both metrically and, with some qualitative computation, visually). Similarly, once all the objects are anchored, one can also determine the degree to which a particular object is visually connected to other objects, visually contiguous with other objects, or visually isolated from other objects. At that point, for the entire design, one can evaluate the degree to which the objects together display massing, spacing, setback, unity, contrast, and continuity (clarity of form and closure). Metrics for all of these are currently under development. 4.4 Implementation Status Portions of the theory described here have been implemented, either in a pioneering system for urban parks on an empty grid [5] or in a system currently under development for recreational park design [21]. The two have tested many of the ideas discussed here; this section provides a status report on the latter. For the first pass, the use area sketch, our system is nearly complete. At this writing, however, use areas are represented by circles (whose diameters may be manually altered during search) rather than ovals, and container networks are not yet available. The user specifies a park problem (or project) as a set of use areas (with kinds and sizes) and proximity constraints. For example, Fig. 5 displays a sketch for a recreational park with five use areas: two picnic areas, two camping areas, and a playground. Two kinds of problem constraints on use areas are portrayed in Fig. 5: proximity constraints and terrain constraints. A proximity constraint specifies a bound on the distance between two use areas [1]. A proximity constraint is represented in our system

Pragmatism and Spatial Layout Design

Camp

201

Picnic

Camp

Playground

Picnic

Fig. 5. An implementation-produced sketch. Each circle represents a use area. Proximity constraints appear as lines — dotted lines for attraction constraints, and solid lines for repulsion constraints. Each circle’s shading represents the suitability of its current location; black for rejection, deep gray for acceptance, paler grays for moderate acceptance. The background shading is the acceptance map for the playground. Shaded circles and labels have been superimposed here for legibility.

as a link between the circumferences of two use areas. A proximity constraint is either an attraction constraint (represented here by a dotted line), which keeps two use areas at most a given distance from each other, or a repulsion constraint (represented here by a solid line), which keeps two use areas at least a given distance from each other. For example, Fig. 5 includes an attraction constraint between one camping area and a picnic area, and a repulsion constraint between the same camping area and the playground. Our system employs Intergraph’s Geomedia geographic information system [22] to program a module that supports user-specification of the site maps as bitmaps for a park design project. For each kind of use area in the project, our system uses this database to compute an acceptance map, which records how appropriate each grid cell is for such a use area. Instead of Table 2, our terrain constraints are based upon a database that includes the far more elaborate tables of the National Soil Survey [19]. Acceptance map calculations integrate all available measurements, including drainage, slope, soil

202

S.L. Epstein et al.

types, and vegetation. Fig. 5 is a gray-scale version of an acceptance map, displayed on our system as a set of colored grid cells. The color (in Fig. 5, the shading) of each grid cell reflects its level of suitability for that use area: red for rejection, yellow for moderate acceptance, green for full acceptance, or white when no data is available. (These appear in Fig. 5 as black, paler grays, deep gray, and white, respectively.) When a use area in a sketch is highlighted, our system retrieves and displays the acceptance map for that particular kind of use area; the color of the use area’s circle represents the mean value of all the circle’s grid cells. In Fig. 5, for example, the playground use area is highlighted, so the background displays the playground acceptance map, and the shading of the playground circle indicates that its current location is unsuitable. The construction of a sketch begins either with a case from the (terrain-independent, positive) cases in the sketch library or with a sketch provided by the user through a graphical user interface. The sketch case base currently contains 14 predefined sketches obtained from existing park designs, whose use areas and proximity constraints were identified by professional designers. The similarity of the current park project to any stored case is based upon the number of instances of each use area type. A solution to the sketch design problem anchors the use areas, so that they abide by both the proximity constraints and the terrain constraints. The system can search for solutions autonomously, or the user can move use areas about on the screen with the mouse. During autonomous search, heuristics seek solutions that agree with both proximity constraints and terrain constraints. Attention is directed first to the most constrained use areas within favorable portions of their corresponding acceptance maps. (Space limitations prevent further algorithmic detail here.) Each iteration re-anchors one or more use areas. Whether the search is automated or user-directed, the terrain constraints color the use areas as their locations improve or degrade, while the proximity constraints shift and pivot the cluster of circles about their links. Left to its own devices, our system searches for solutions and displays several candidate sketches that abide by the proximity constraints and best conform to terrain constraints. The user can choose any such sketch and modify it at will. Implementation of the second and third passes is current work. Meanwhile, we report here upon an agent-based simulation that computes an important principle: the navigability of a design, that is, the ease with which visitors can access particular locations the park. Potential destinations are marked on the sketch as interest points. The sketch is annotated with access points (ways in and out, e.g., doors or gates) for individual destinations, buildings, and use areas, and for the park itself. Through any of a set of access points, our system can currently introduce hundreds of simple, reactive agents into a design. Each agent is a member of a class (e.g., senior citizen, parent, sports enthusiast) whose profile determines the agent’s behavior, including speed, intended destination(s), duration of visit, interests, and willingness to stay on the paths. Nonetheless, each agent is an individual, whose behavior is randomized within the ranges specified by that profile. In real-time on a video display, our simulation can move several hundred agents from various classes through the design’s path network toward their intended destinations, gathering information about crowding and chosen paths. At every time step, each agent responds to the current state. On exit from the park, each agent scores its satisfaction with its visit, based upon its ability to reach its intended destinations, crowding, and any other criteria the user chooses to measure. The simulation’s cost is a function

Pragmatism and Spatial Layout Design

203

primarily of the complexity of the satisfaction function — more complex criteria may be accommodated by a reduction in the number of individual agents. For now, the navigation simulation serves as a reserved principle, but it also provides data that will soon be used to generate proposals to tweak the design. Fig. 6 is a snapshot from an agent-based simulation in process. Use areas appear as ovals linked by paths; points of interest appear as large black dots. (Access points are not shown.) The small shaded dots in use areas and on paths represent agents from various classes moving about in the park space. The rectangle at the lower left represents a building in which there is an interest point. One can run multiple simulations to obtain information about navigability of the space, including agents’ freedom to move along paths, identification of places subject to crowding, and so on. Our agent-based simulation is not limited to parks; it is applicable to any two-dimensional layout. 4.5 Limitations and Proposed Extensions Although the theory described here is for recreational parks, it could be extended in a variety of ways. Pragmatic design synthesis is not restricted to parks — it could be extended to planning a town, for example, where use areas would be schools and shopping centers and residential areas, probably with one or two additional levels of granularity dictated by the complexity of the objects. Moreover, although it considers slope

Fig. 6. An agent-based simulation of park usage. Use areas appear as ovals, linked by paths. Points of interest are large black dots; one in the lower left is within a building. Each smaller dot represents an agent; similarly shaded dots are agents of the same class. The park’s entry and exit point is at the center of the right margin

204

S.L. Epstein et al.

in the site maps, pragmatic design synthesis is thus far formulated for two-dimensional space. Although we are well aware of the complexity of aesthetic criteria in three dimensions, we see no other reason to prevent an extension that would include height. We remain convinced, however, that bootstrapping from the soil data narrows search in very practical ways. A set of flat, paved city blocks as a design frame would therefore be more difficult – a clean slate is not an easy tablet on which to design. One interesting extension would be to have the system extract cases for sketches and use area plans from the most successful (highest scoring) of its own solutions. In this manner pragmatic design synthesis could learn terrain-independent principles from its own experience. For two-dimensional layout design it could prove an effective technique. Our current approach incorporates case-based reasoning to retrieve initial designs which are then modified with a knowledge-driven search strategy. Future work may involve further applications of case-based reasoning. Case adaptation could be applied in tweaking. Moreover, if the case base contains instances of design decisions that were not ideal, case-based reasoning could be applied during evaluation to ensure that inappropriate design decisions were not repeated.

5 Conclusions Pragmatic design synthesis produces a variety of constraint-abiding, highly-rated designs for a variety of reasons. Object classification reduces the search space by focusing attention on fewer objects, or on use areas instead of objects. Multiple alternatives provide a broad variety of possibilities. Knowledge informs search: the case bases provide a wealth of historical experience that supports good initial search states, and the recommendation functions encode expert variations. Expensive computations (i.e., reserved principles) are deferred until a design is fully specified. In summary, pragmatic design synthesis is a form of restart hill-climbing from good initial states at different levels of granularity, one calculated to make search both manageable and productive.

References 1. Molnar, D.J., Rutledge, A.J.: Anatomy of a Park. McGraw Hill, New York (1986) 2. Goel, V., Pirolli, P., Motivating the Notion of Generic Design within Information Processing Theory: The Design Problem Space. AI Magazine. 10 (1989) 19-36 3. Goel, V., Pirolli, P., The Structure of Design Problem Spaces. Cognitive Science. 16 (1992) 395-429 4. Schraagen, J.M., How Experts Solve a Novel Problem in Experimental Design. Cognitive Science. 17 (1993) 285-309 5. Epstein, S.L. Toward Design as Collaboration. In Proceedings of the Fifteenth National Conference on Artificial Intelligence. Madison, WI: AAAI. (1998) 135-142 6. Alexander, C., Ishikawa, S., Silverstein, M.: A Pattern Language: Towns, Buildings, Construction. Oxford University Press, New York (1977) 7. Alexander, C.: A New Theory of Urban Design. Oxford University Press (1987)

Pragmatism and Spatial Layout Design

205

8. Cullen, G.: Townscape. Reinhold, New York (1968) 9. Stephen-Cutler, L., Stephen-Cutler, S.: Recycling Cities for People - The Urban Design Process. Van Nostrand Reinhold, New York (1983) 10. Gottdiener, M., Lagopoulos, A., (eds.): The City and the Sign: An Introduction to Urban Semiotics. Columbia University Press, New York (1986) 11. Hedman, R., Jaszewski, A.: Fundamentals of Urban Design. Planners Press, Washington, D.C. (1984) 12. Lynch, K.: What Time is This Place? MIT Press, Cambridge, MA (1971) 13. Lynch, K.: Good City Form. MIT Press, Cambridge, MA (1981) 14. Moughtin, C.: Urban Design: Green Dimensions. Butterworth-Heinemann, Oxford (1996) 15. Peps, J.W.: 1794-1918: An International Anthology of Articles, Conference Papers, and Reports selected, Edited, and Provided with Headnotes. (2000) Cornell University 16. Navinchandra, D.: Exploration and Innovation in Design: Towards a Computational Model. Springer Verlag, New York (1991) 17. Domeshek, E., Kolodner, J.: A Case-Based Design Aid for Architecture. In: J. Gero (ed.) Artificial Intelligence and Design. Kluwer Academic, Boston (1992) 18. Barber, J., Bhatta, S., Goel, A., Jacobsen, M., Pearce, M., Penberthy, L., Shankar, M., Stroulia, E.: AskJef: Integrated Case-Based Reasoning and Multimedia Technologies for Interface Design Support. In: J. Gero (ed.) Artificial Intelligence in Design. Kluwer Academic, Boston (1992) 19. SIRG: Soil Interpretation Rating Guides, National Soil Survey Handbook (Part 620). (2001) 20. Smith, B., Varzi, A., Fiat and Bona Fide Boundaries. Philosophy and Phenomenological Research. To appear (2001) 21. Moulin, B., Chacker, W., Epstein, S. Preliminary Design of a Software Tool for the Design of Geographic Space. In Proceedings of the GEOIDE General Conference 2000. Calgary, CA. (2000) CD-ROM 22. Intergraph: Geomedia Professional, . (2001) Intergraph

Spatial Frames of Reference Used in Identifying Direction of Movement: An Unexpected Turn Christy R. Miller and Gary L. Allen

Department of Psychology, University of South Carolina, Columbia, SC 29208 USA

[email protected] or [email protected]

Abstract. Despite extensive interest in the role of frames of reference in spatial representation, there is little consensus regarding the cognitive effort associated with various reference systems and the cognitive costs (if any) involved in switching from one frame of reference to another. Relevant to these issues an experiment was conducted in which accuracy and response latency data were collected in a task in which observers verified the direction of turns made by a model car in a mock city in terms of four different spatial frames of reference: fixed-observer (relative-egocentric), fixed-environmental object (intrinsic-fixed), mobile object (intrinsic-mobile), and cardinal directions (absolute-global). Results showed that frames of reference could be differentiated on the basis of response accuracy and latency. In addition, no cognitive costs were observed in terms of accuracy or latency when the frames of reference switched between fixed-observer vs. global frames of reference or between mobile object and fixed environmental object frames of reference. Instead, a distinct performance advantage was observed when frames of reference were changed. This unexpected result is attributed to a phenomenon analogous to release from proactive inhibition. Keywords: frames of reference; orientation; spatial perspective; spatial relations.

1 Spatial Frames of Reference: The Concept

The term “spatial frames of reference” has been used by researchers in several different but related areas of endeavor (e.g., perception, cognition, geography, information science, linguistics), but a shared nomenclature has been elusive. Across these various areas, however, it may be possible to find consensus. Fundamentally, a spatial frame of reference (FOR) is a conceptual basis for determining spatial relations. This description is applicable across situations in which person-to-object, object-to-object, and person-to-person spatial relations are perceived, imagined, judged, or described. The distinction between egocentric and allocentric reference systems is a standard point of departure in differentiating among FOR’s (for example, see Klatzky, 1998). Egocentrically determined relations involve locations that are represented with respect to the perspective of the person who is viewing those locations. Allocentrically determined relations involve locations that are represented with respect to concrete objects or abstract dimensions independent of the person who is viewing those locations. This basic distinction is valid and useful, but others are necessary for some expressions of spatial relations (see Frank, 1998). For example, both a deictic frame of reference, which specifies spatial relations from an observer’s viewpoint, and a relative reference frame, which is ‘body-centered’ and described in term of ‘front/back’ and ‘left/right’, could be considered egocentric in some ways. Similarly, both an intrinsic frame of reference, in which the position of D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 206−216, 2001.  Springer-Verlag Berlin Heidelberg 2001

Spatial Frames of Reference Used in Identifying Direction of Movement

207

an object is described with respect to another object, and an absolute frame of reference, which does not change regardless of the position of a speaker, can be considered allocentric. Although the matter of identifying different reference systems and determining the circumstances of their application has received considerable attention, little distinction has been made between FOR’s applied to static spatial situations and those applied to situations involving movement. Also, only recently have researchers begun to examine the ease with which a recipient of spatial information can switch from one frame of reference to another. These matters invite further inquiry. 1.1 Spatial Frames of Reference in Developmental Psychology In an early paper, Acredolo (1976) investigated the FOR’s used by young children to guide their movement toward a target location they had experienced previously. Guided by Piagetian theory at the time, she tested the assumption that children progress through three stages in developing references systems: egocentric (self-based), fixed (object-based), and then finally to an abstract system. She found that three year olds were most influenced by the egocentric frame of reference, while the four year olds were most influenced by the object frame of reference. Although this pattern was consistent with Piagetian theory, Acredolo and other researchers pursued this issue with infants using a looking response rather than a locomotor response. A number of investigations led the conclusion that multiple FOR’s can be used by infants, with their looking responses influence by variables such as environmental familiarity, attention-grabbing properties of the task, and perceptualmotor experience (Acredolo, 1990; Bremner, 1978; Cornell & Heth, 1979; Rieser, 1979). Based on logic and empirical results, the same conclusion was extended to young children using locomotor responses (Bremner, Knowles, & Andreasen, 1994; Heth & Cornell, 1980). If a variety of spatial FOR’s is available to children, are all reference systems equally easy to learn in a given task? Allen (1999) reported a study in which children from ages three to eight years learned to apply different FOR’s to locate a target object successfully over repeated trials. He found that younger children in “reaching” space learned to rely on an object-based system more easily than on either an egocentric (self-based) or place-based system. In “walking” space, the pattern was different, with both object-based and placed based FOR’s being easier to learn than a self-based system. By eight years of age, however, children were adept at applying any of the three FOR’s in either “reaching” or “walking” space. Allen concluded that in spatial tasks different FOR’s co-exist and compete in biased competition for dominance. Bias is based on previous experience with similar tasks and to some extent on situational factors. Reinforcement in the form of success in locating a target serves to build up strength for the successful FOR and perhaps inhibition for the situationally inappropriate FOR’s. This view is very similar to the one put forth by Newcombe and Huttenlocher (2000) in their theoretical analysis of the development of spatial cognition.

208

C.R. Miller and G.L. Allen

1.2 Spatial Frames of Reference in Attention and Perception Some researchers in cognitive psychology have described FOR’s as mechanisms of attention (e.g., Logan, 1996). In this perspective, a spatial FOR orients an observer’s attention to a specific aspect of an object or dimension in space. Evidence is mounting that the application of an FOR is largely a top-down or goaldirected process, which is influenced by both the intrinsic axes of objects and an observer’s bodily axes (Logan, 1996). What happens in the event of competing FOR’s? Carlson-Radvansky and Jiang (1998) examined this issue in the context of an object perception task that involved intrinsic object-based, absolute, and observer-based relative FOR’s. Their results indicated that multiple FOR’s are activated simultaneously despite instructions to participants to use a particular one. Furthermore, they found that application of a particular reference system was difficult and time-consuming when available FOR’s provided conflicting outcomes (for example, the up/down axis of an object versus the up/down axis of an observer). In order to apply only one FOR, observers apparently actively inhibit the other FOR’s. Such inhibition would lead to added cognitive costs when a switch between FOR’s is attempted. 1.3 Spatial Frames of Reference in Language Language researchers have dedicated considerable study to the issue of how spatial relations are communicated. Despite differences in theoretical orientation, a variety of researchers have reached similar conclusions regarding FOR's. Basically a reference system is considered a means of describing figure-ground or referentrelatum relations, and at least three of these means (intrinsic, absolute, and relative) can be convincingly differentiated from each other (Levelt, 1996; Levinson, 1996). An intrinsic FOR defines a location in terms of an object-based coordinate system, a relative FOR defines a location in terms of viewer-based axes, and an absolute FOR defines a location in terms of invariant directions due to gravity. Despite these generalizations, it should be noted that these FOR's do not appear to be linguistically universal and that there is little evidence to indicate a natural priority or universal evolutionary sequence among them (Levinson, 1996). Once an FOR has been selected, there may be cognitive costs associated with switching between frames when called on to do so. One way to overcome such costs might be to establish a mental model of spatial relations that is relatively free of specific FOR influences. This idea comes from a study by Franklin, Tversky, and Coon, 1992. They reasoned that readers of spatial descriptions may attempt to deal with relating a series of locations in two ways. One way is by forming mental models involving distinct relationships between objects in a scene. The readers would then switch between these smaller models in identifying objects’ locations. An alternative would be that readers create one larger integrated model containing several objects and relationships between them. The cognitive costs in using the smaller models can be subsumed by using this larger model. When one uses the larger model, verifying locations should ultimately be more efficient with respect to cognitive effort. Franklin et al. found that readers tended to construct integrated or perspective-free models that included all objects, observers, and relations. Only in the instance in which readers read about two protagonists in two different areas did

Spatial Frames of Reference Used in Identifying Direction of Movement

209

they form two small mental models rather than using one large all-encompassing model. Additional studies raise the prospect that either readers readily form perspective-independent mental models of spatial relations or switch from one FOR to another with relative ease. Taylor and Tversky (1992) reported that readers had little difficulty verifying spatial relations stated with respect to one FOR (either a body-based FOR incorporated into a route description or an abstract FOR incorporated into a survey description) after reading descriptions using a different FOR. Furthermore, in a study requiring individuals to describe an environment to another person, Tversky (1991) found that speakers used both route and survey perspectives and often mixed the two with apparent ease. 1.4 Summary The evidence of Developmental Psychology, Cognitive Psychology, and Psycholinguistics indicate that multiple FOR’s are simultaneously available to an observer when he or she perceives or describes an object or scene. FOR’s are basically means of focusing attention so as to make possible a particular way of responding behaviorally, cognitively, or verbally. Regardless of what an observer is instructed to attention to in terms of a reference system, alternatives are available and apparently compete with each other, although the competition may be biased by prior experience and contextual factors. It may be the case that competitor FOR’s must be suppressed to allow for a designated or target FOR to be applied appropriately. Thus, it is sensible to suggest that there is some cognitive cost involved in switching between FOR’s. However, while studies of FOR’s applied to object perception have supported this idea, studies of FOR’s applied to environmental descriptions suggest that frame-switching is common and easily accomplished. Previous investigations have examined FOR’s applied to the orientation of static objects, the orientation of objects within a static scene, and the orientation of a traveler within an environmental context. Yet, potential differences between these situations have not really been investigated systematically, and such differences have rarely been addressed. Object perception studies have tended to use pictures of static objects as stimuli, developmental studies have tended to use situations in which direction of movement is critical, and psycholinguistic analyses have often used static scenes to explain the difference between FOR’s. There is a need to expand the repertoire of experimental studies to include perception of mobile agents and mobile objects in spatial situations. Such studies would complement those conducted on the perception of static objects on the one hand and the comprehension of linguistic descriptions of mobile agents on the other hand.

2 Empirical Study

The present study was concerned with how observers can apply different frames of reference in a situation in which a stationary observer watches a mobile object (in this instance, a model car). Four frames of reference were incorporated into the study. The fixed-observer frame, synonymous with deictic or egocentric, referred to the subject’s own perspective in viewing the mobile object. This FOR was explained to the subject as “movement to the left or right with respect to you.”

210

C.R. Miller and G.L. Allen

The mobile object and the fixed-environmental object were similar to Levinson’s (1996) intrinsic frame of reference (object-centered). The mobile object FOR was explained as “movement to the car’s left or right,” and the fixed-environmental object FOR was explained as “movement toward a designated symbol.” The global reference system referred to an absolute FOR based on cardinal directions. This FOR was explained as “movement toward a compass direction.” The design of the study called for blocks of trials in which subjects would verify the direction of movement according to each FOR, with accuracy and response time data being collected. After a block of trials for each reference system, another series of trials was administered in which the FOR required varied from one trial to the next. This series allowed us to examine the effects of frame-switching on accuracy and response time. Two types of switches were examined: between the fixed-observer (you) and global (compass) FOR’s and between the mobile object (car) and the fixed-environmental object (symbol). 2.1 Hypotheses It was hypothesized that FOR’s would be differentiated by accuracy and response time data. Based on results of object perception studies involving changes of FOR, it was expected that cognitive costs would be observed when subjects were required to change from one FOR to another. 2.2 Method Participants. The participants were 35 female and 9 male students at a large southeastern university. The mean age was 20.82 years within a range of 18 to 38 years. Students participated to receive credit in undergraduate psychology courses. Materials. An area was configured on the laboratory floor to represent a city with several intersections. City blocks consisted of buildings represented by 36 squares, cut from black construction paper. The squares were 15 cm X 15 cm with 8 cm between them and were placed in a 6 X 6 configuration. The cardinal directions were designated by the first letter of each direction: N for north, E for east, S for south, and W for west. These letters were printed on 10 cm X 6 cm paper and were placed around the outside of the mock city. Each intersection contained a small piece of paper with a standard keyboard symbol printed upon it. There were four symbols used: #, $, *, and + (created using Microsoft Power Point in 96 font). These symbols were placed randomly among the intersections. A toy car was used as the mobile object in the experiment, with measurements of 11.5 cm X 5 cm. A computer program (written in Turbo Pascal) was developed to display different FOR’s and different directions based on the frame seen on the computer screen. There were four FOR’s: car referred to the mobile-object frame of reference (perspective of driving a car), you referred to the fixed-observer frame (perspective of watching a mobile object), symbol referred to the fixed-environmental object frame (four keyboard symbols), and compass referred to the global FOR (four cardinal directions). Subjects sat in a chair facing the area on the floor with a table with a computer monitor and keyboard placed on it immediately to their right. The keyboard was completely covered except for the “yes” and “no” keys and the

Spatial Frames of Reference Used in Identifying Direction of Movement

211

“enter” key. Reaction times in milliseconds and accuracy of responses were recorded for each subject. Procedure. All subjects were tested individually. Each participant read and signed an Informed Consent form before beginning the experiment. Participants were instructed to watch as the experimenter pushed the model car through a series of turns throughout the mock city. After each movement of the car, the experimenter told the subject to “press enter” on the computer keyboard and then look at the screen. Based on the statement that appeared on the computer screen and the movement of the car that just occurred, the subject chose either “yes” or “no” on the keyboard. Each trial statement that appeared on the computer screen was two words. An example of a trial statement was: “car left”; the first word indicated the frame of reference and the second word is the directional term (See Figure 1).

CAR……… LEFT

Fig. 1. Example of the format of statements viewed on the computer screen. In this example, a subject would watch as the car is moved, then decide if from the perspective of the car, did it go left? Then subjects would choose either “yes” or “no”. Once he/she made the choice of answer, they were instructed to look back at the area on the floor and watch as the experimenter moved the car again. Subjects pressed “enter” only when told to do so, which was after each movement of the car. The directional terms for the car frame included left, right, and straight. For the compass frame, the four cardinal directions (north, east, south, west) were used as the directional terms on the computer screen. The four keyboard symbols listed above were the directional terms for symbol frame. For the you frame, the directional terms varied from left, right, toward, and away. Each participant was presented with two blocks of six trials of each FOR (car, compass, you, and symbol) for a total of 48 blocked trials. These trials were used to estimate the subjects’ ability to utilize the different frames of reference. Another block of 48 trial items consisted of switching between FOR’s. Within these trials, the first 24 trials contained switches between the fixed-observer and the global FOR’s (you versus compass). The second group of 24 trials consisted of switches between the fixed-environmental object and the mobile object FOR’s (symbol versus car).

212

C.R. Miller and G.L. Allen

2.3 Results Differences in Using the Four Frames of Reference. A one-way ANOVA performed on the proportion of correct responses during the blocked trials revealed a significant difference between the frames of reference, F(3, 123) = 17.54, MSe = 0.010, p < .01. The frame responded to most accurately was compass (mean = 0.99) and the least accurate frame was you (mean = 0.84). Post-hoc analyses (Tukey) showed significant differences between compass (mean = 0.99) and car (M = 0.94), compass and you (mean = 0.84) and compass and symbol (mean = 0.96). Car differed significantly from the you frame of reference, but did not differ from symbol. The you FOR differed from all other FOR’s. For the response-time data during the blocked trials, a one-way ANOVA performed revealed a significant difference, F(3, 102) = 12.58, MSe = 99323.88, p < .01. The car frame of reference was responded to the fastest, while the slowest reaction time was found for the you frame of reference. Post-hoc analyses (Tukey) revealed that car (mean = 1756 ms) differed significantly from all other FOR’s: compass (mean = 1885 ms), you (mean = 2168 ms), and symbol (mean = 1881 ms). The you frame also differed significantly from all other FOR’s. The compass and symbol frames did not differ significantly from each another (see Figure 2).

2500 2000 1500 1000 500 0 car

compass

you

symbol

Frames of Reference

Fig. 2. Mean response times for different frames of reference. Cognitive Costs Associated with Frame-Switching. These analyses involved taking accuracy and reaction times from the trials in which FOR’s were switched and comparing that information to accuracy and reaction times from matched trials during the blocked trials in which no switches took place. A 2 (Frame type: you versus compass) X 2 (Consistency of frame type: consistent trials that were matched with switched trials versus switched trials) ANOVA conducted on accuracy data revealed a significant main effect for frame of reference, F(1, 42) = 30.21, MSe = 0.026, p < .01, but not for the main effect of consistency of frame type, F(1, 42) = 1.414, MSe = 0.006, p > .05. The interaction was not significant, F(1, 42) = 1.02, MSe = 0.08, p > .05. As indicated in the previous set of analyses, accuracy was

Spatial Frames of Reference Used in Identifying Direction of Movement

213

greater when directions were framed in terms of compass than when they were framed in terms of you. A 2 X 2 ANOVA was conducted on the corresponding response time data, which yielded a significant main effect for FOR, F(1, 41) = 5.68, MSe = 85859.32, p < .01, and for consistency, F(1, 41) = 70.74, MSe = 119952.38, p < .01, as well as a significant interaction, F(1, 41) = 12.73, MSe = 97528.18, p < .01. As expected, compass based responses were faster than were those based on you. Completely unexpected was the finding that trials involving a switch between FOR’s resulted in faster response times (mean = 1,496 ms) than did those involving no such switch (mean = 1,945 ms). The interaction showed that this advantage was greater when the comparison was between consistently using the you FOR and switching to the you FOR from the compass FOR (a 622 ms advantage after switching), but there was also a difference between consistently using the compass FOR and switching to it from the you FOR (a 278 ms advantage). Please refer to Figure 3.

2500 2000 1500 1000 500 0 You-No Switch

YouSwitch

CompassNo Switch

CompassSwitch

Condition

Fig. 3. Mean response times as a function of FOR type and consistency. A separate 2 X 2 (Frame type: symbol versus car) X 2 (Consistency of Frame type: consistent trials that were matched with switched trials versus switched trials) ANOVA was conducted on the proportion of correct scores for you versus car trials. The main effect for FOR was not significant, F(1, 41) = 1.76, MSe = 0.015, p > .05, and neither was the main effect for consistency of frame type, F(1, 41) = 3.27, MSe = 0.013, p > .05. However, the interaction was significant, F(1, 40) = 4.19, MSe = 0.014, p < .05. Performance was equally accurate when the symbol FOR (mean = .95) and car FOR (mean = .93) were consistently applied and when the FOR was switched from symbol to car (mean = .94). However, these levels of accuracy were each higher than that found when the FOR was switched from car to symbol (mean = .88). A 2 X 2 ANOVA was conducted on the corresponding response time data, which yielded a significant main effect for consistency of frame type effect, F(1, 40) = 4.68, MSe = 164595.05, p < .05. The main effect of frame of reference was not significant, F(1, 40) = 2.25, MSe = 119058.85, p > .05, nor was the interaction

214

C.R. Miller and G.L. Allen

resulted in F(1, 40) = 0.80, MSe = 103776.96, p > .05. As in the analysis involving the other two frames of reference, trials on which the FOR was switched resulted in more rapid responding (mean = 1,671 ms) than did trials on which the FOR was consistent (mean = 1,808 ms).

3 Discussion

Prior to the study, it had been hypothesized that the application of different spatial FOR’s in verifying the direction of an object’s movement would result in differential patterns of accuracy and response times. Consistent with this hypothesis, the results showed that the compass condition, which involved an absolute-global frame of reference, and the symbol condition, which involved an intrinsic-fixed object frame of reference, resulted in the highest accuracy and in equal and comparatively quick response times, while the you condition, which involved a fixed observer and a relative-deictic FOR, resulted in the lowest accuracy and the longest response times. The car condition, which involved an intrinsicmobile object FOR, resulted in the third highest accuracy level but the most rapid response times. Thus, it appears that the application of different frames of reference can be differentiated in a comparative sense using behavioral data from a relatively simple task involving the direction of an object’s movement. Additional studies in which the cardinal directions and fixed environmental objects are not visible are needed to substantiate this conclusion. It was also hypothesized that cognitive costs would be associated with switches between different FOR’s. Not only was this hypothesis disconfirmed, just the opposite was found to be the case. Switching references frames in this task provided a cognitive benefit in terms of faster response time. It mattered little which FOR’s reference were involved in the switch. Switches from relatively easy reference systems as identified by accuracy and response time data to relatively difficult ones were advantageous as well as the ones from difficult to easier reference systems. Past studies in object perception (Carlson-Radvansky &Jiang, 1998) and in language processing (Franklin, Tversky, & Coon, 1992; Bryant & Tversky, 1999) have suggested that cognitive costs should be associated with switching from different frames of reference or switching from different representations, such as from models to diagrams. Our methods differed greatly from theirs, and thus the fit between empirical outcomes is not obvious. Yet, it is also the case that Taylor and Tversky (1992) found that individuals could readily verify information across reference systems used in verbal descriptions, as in the case of verifying accurate route information after hearing a survey description. Individuals also switch perspectives with ease as they describe environments (Tversky, 1991). However, no previous findings prepared us for the results discovered here, that is, an actual advantage for frame-switching. At this point, it is reasonable to speculate that consistent application of any FOR over a number of trials results in the build-up of considerable response interference distributed across available responses within that frame. For example, when “north” is the correct answer, the tendency for that response must be excited up to a certain threshold, and the tendency to say “south” must be inhibited. On the next trial, “south” may be the correct answer, and so the pattern of excitation and

Spatial Frames of Reference Used in Identifying Direction of Movement

215

inhibition must be reversed. Over a block of trials, net excitation and inhibition waxes and wanes for each response, and slower response times result even when the correct answer is directly in view. When the FOR switches, the potential for interference is reduced as both excitation and inhibition associated with each response subsides. Tversky and her colleagues have taken the position that speakers normally and naturally engage in frequent changes of reference systems in the normal course of everyday spatial descriptions (Tversky, Lee, & Mainwaring, 1999). Our findings, which initially violated our expectations so radically, suggest a good reason for such switches.

4 References

Acredolo, L. P. (1976). Frames of reference used by children for orientation in unfamiliar spaces. In G. Moore & R. Golledge (Eds.), Environmental Knowing (pp. 165-172). Stroudsburg, PA: Dowden, Hutchinson, & Ross. Acredolo, L. P. (1990). Behavioral approaches to spatial orientation in infancy. In A. Diamond (Ed.), The development and neural bases of higher cognitive function (pp. 596-607). New York: New York Academy of Sciences. Allen, G. L. (1999). Children's control of reference systems in spatial tasks: Foundations of spatial cognitive skill? Spatial Cognition and Computation, 1, 413-429. Bremner, G. J. (1978). Egocentric versus allocentric spatial coding in nine-monthold infants: Factors influencing the choice of code. Developmental Psychology, 14, 346-355. Bremner, G. J., Knowles, L., & Andreasen, G. (1994). Processes underlying young children’s spatial orientation during movement. Journal of Experimental Child Psychology, 57, 355-376. Bryant, D. J. & Tversky, B. (1999). Mental representations of perspective and spatial relations from diagrams and models. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 137-156. Carlson-Radvansky, L. A. & Jiang, Y. (1998). Inhibition accompanies referenceframe selection. Psychological Science, 9, 386-391. Cornell, E. H., & Heth, C. D. (1979). Response versus place learning by human infants. Journal of Experimental Psychology: Human Learning and Memory, 5, 188-196. Frank, A. U. (1998). Formal models for cognition-Taxonomy of spatial location description and frames of reference. In C. Freksa, C. Habel, & K. F. Wender (Eds.), Spatial cognition: An interdisciplinary approach to representing and processing spatial knowledge (pp. 293-312). Berlin: Springer-Verlag. Franklin, N., Tversky, B., & Coon, V. (1992). Switching points of view in spatial mental models. Memory and Cognition, 20, 507-518. Heth, C. D., & Cornell, E. H. (1980). Three experiences affecting spatial discrimination learning by ambulatory children. Journal of Experimental Child Psychology, 30, 246-264. Klatsky, R. L. (1998). Allocentric and egocentric spatial representations: Definitions, distinctions, and interconnections. In C. Freksa, C. Habel, & K. F.

216

C.R. Miller and G.L. Allen

Wender (Eds.), Spatial cognition: An interdisciplinary approach to representing and processing spatial knowledge (pp. 1-17). Berlin: Springer-Verlag. Levelt, W. J. M. (1996). Perspective taking ellipsis in spatial descriptions. In P. Bloom, M. A. Peterson, L. Nadel, & M. F. Garrett (Eds.), Language and space (pp. 77-107). Cambridge, MA: MIT Press. Levinson, S. C. (1996). Frames of reference and Molyneux’s question: Crosslinguistic evidence. In P. Bloom, M. A. Peterson, L. Nadel, & M. F. Garrett (Eds.), Language and space (pp. 109-169). Cambridge, MA: MIT Press. Logan, G. D. (1995). Linguistic and conceptual control of visual spatial attention. Cognitive Psychology, 28, 103-174. Logan, G. D. (1996). Top-down control of reference frame alignment in direction attention from cue to target. In A. F. Kramer, M. G. H. Coles, & G. D. Logan (Eds.), Converging operations in the study of visual selective attention (pp 415437). Washington, D. C: APA. Newcombe, N. S., & Huttenlocher, J. (2000). Making space: The development of spatial representation and reasoning. Cambridge, MA: MIT Press. Rieser, J. J. (1979). Spatial orientation in six-month-old infants. Child Development, 50, 1078-1087. Taylor, H. A., & Tversky, B. (1992). Descriptions and depictions of environments. Memory & Cognition, 20, 483-496. Taylor, H. A. & Tversky, B. (1996). Perspective in spatial descriptions. Journal of Memory and Language, 35, 371-391. Tversky, B. (1991). Spatial mental models. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (pp. 109-145). San Diego, CA: Academic Press. Tversky, B., Lee, P., & Mainwaring, S. (1999). Why do speakers mix perspectives? Spatial Cognition and Computation, 1, 399-412.

The Role of a Self-Reference System in Spatial Navigation M. Jeanne Sholl Department of Psychology, Boston College, Chestnut Hill, MA 01772 [email protected]

Abstract. The self-reference system architecture developed to explain the retrieval of spatial knowledge from long-term memory is reviewed and expanded to include navigational tracking as one of its operations. A distinction is made between the operation of the self-reference system at a perceptual-motor and a representational level, and a case is made that if a representational self-reference system is to function as a navigational tracking device, it must be closely connected with the perceptual-motor level. Existing empirical evidence is reviewed and new empirical evidence is reported that explores the connections between the two levels of functioning. Additionally, the operations that would enable a self-reference system to function as a tracking device are derived from an animal model of sense of direction. The ideas explored in this manuscript rely on the premise that spatial memory systems evolved in the service of spatial navigation. Keywords. Self-reference system, navigational tracking device, cognitive map, geocentric heading, path integration, inertial dead reckoning.

1 Introduction How do we keep track of our orientation and location in a familiar large-scale space? At least one influential animal model of navigation answers this question by proposing that the animal has a geocentric cognitive map of the space and a tracking device that uses internal signals generated by the animal’s movement to compute the body’s location and heading relative to the cognitive map (e.g., Gallistel, 1990). By this account, internal idiothetic (self-motion) signals1 provide exclusive input to a tracking device that computes angular and linear displacements of the body and relates them to geocentrically referenced knowledge of inter-landmark relations, with only periodic visual fixes to correct for any error that may accumulate over time. The process of updating the body’s location and heading on the basis of the velocity and acceleration signals produced by self movement is called path integration or dead reckoning, when the emphasis is on velocity signals, or inertial dead reckoning, when the emphasis is on acceleration signals. By traditional definition, these simple navigational processes update the body’s current location and heading relative to an earlier location and heading, typically the starting point of the current 1

Idiothetic signals are produced by whole body movement and include internally generated vestibular, proprioceptive and motor-efferent signals, as well as external optical flow patterns.

D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 217−232, 2001.  Springer-Verlag Berlin Heidelberg 2001

218

M.J. Sholl

trajectory. In contrast, in the present context updating occurs relative to knowledge of stable inter-landmark relations, and hence it is geocentric. In an attempt to embed the retrieval of inter-landmark relations within a larger human navigational system, I have proposed a “self-reference system” model of spatial retrieval that follows animal models of navigation by differentiating between a self-reference system that tracks the person’s movement through large scale space and an object-to-object system that functions like a geocentric cognitive map (Easton & Sholl, 1995; Sholl, 2000, 1995; Sholl & Nolin, 1997). The self-reference system functions at the representational level as if it was superposed on the object-to-object system, and it tracks in representational space the person’s movement through physical space. It is in the interface between the self-reference and object-to-object systems that spatial retrieval occurs. Whereas previous research in my laboratory has studied the retrieval operations of the self-reference system, its navigational tracking function has only been theoretically inferred. The purpose of this chapter is to explore the psychological reality of a navigational tracking device in humans. Following a description of the functional architecture of the self-reference system, which distinguishes between perceptual-motor and representational operations, empirical evidence will be reviewed showing that the functional architecture has some of the key properties needed by a tracking device. Then, some processes that could be used by a self-reference system to compute the geocentric heading and the location of the body will be sketched out. A particular challenge for a human tracking device is the computation of geocentric heading. The difficulty lies in the fact that heading is defined in relation to an absolute environmental reference direction, and there is little evidence that humans are sensitive to the local visual (e.g., the sun’s azimuth, stellar constellations) or geomagnetic cues that specify global reference directions. An intriguing solution to this problem is provided by a computational model developed by McNaughton, Chen, & Markus (1991), which operates under some of the same constraints (i.e., no access to a geocentric reference axis) likely to characterize a human system. The model proposes that animals use inertial dead reckoning in combination with local views of the environment to compute changes in heading within a 360° geocentric frame of reference that is centered on the body. A brief description of the McNaughton et al. model is provided in the last section of the chapter, and it is tentatively proposed that the self-reference system updates heading and location using computations similar to those proposed by the model, but with a greater reliance on local views to compensate for the fact that humans are not very good at dead reckoning (Loomis, et al., 1993).

2. Self-Reference System Architecture: A Conceptual Model In the self-reference model of spatial retrieval, I have made a distinction between a self-reference system that operates at a perceptual-motor level and a self-reference system that operates at the representational level. This distinction will be reviewed in this section.

The Role of a Self-Reference System in Spatial Navigation

219

2.1 Perceptual-Motor Self-Reference System The architecture of the perceptual-motor self-reference system consists of the coordinate system formed by the physical axes of the body and a set of operations for computing the spatial coordinates of visible objects relative to the reference system. A primary function of the system is to compute the endpoint of object-directed activities such as reaching, pointing, and approaching. The up/down, front/back, right/left body axes form a global system of reference, within which are multiply nested, hiearchically organized local frames of reference for controlling the visually guiding motor output of specific body parts (Paillard, 1991). A global level of analysis is most relevant for present purposes, and the two-dimensional reference frame provided by the front/back and right/left axes of the body is of primary import. The right-left body axis divides the physical space surrounding the body into two equal regions. However these two regions of space are not psychologically equivalent. Conceptually, the region of space that is categorized as in front of the body is larger than the region of space categorized as behind the body (Franklin, Henkel, & Zangas, 1995). At a perceptual-motor level, the visual system is structurally designed to pick up electromagnetic energy in the anterior half of body space and the body is biomechanically designed to move in a forward direction. Others besides myself have pointed out that the asymmetries in the structural organization of the body render visual-spatial information in the anterior half of body space perceptually more accessible than in the posterior half (Clark, 1973; Franklin & Tversky, 1990; Shepard & Hurwitz, 1984).

2.2 Representational Self-Reference System At the representational level, the self-reference system operates in concert with an object-to-object system. Figure 1 provides a schematic illustration of an object-toobject system in which individual landmarks are represented by nodes and their metric inter-landmark distances and directions by the vectors connecting the nodes. The object-to-object system functions as if inter-landmark relations were coded independently of any momentary perspective of the viewer: that is, it functions like a geocentric cognitive map. The cognitive architecture of the representational selfreference system includes the following: a representation of the front/back, right/left axes of the body, a set of operations for shifting the location and heading of the selfreference system relative to the object-to-object system, and a set of operations for computing the distance and direction of objects represented in the object-to-object system. Figure 1 provides a schematic example of the part of the system architecture involved in spatial retrieval. The figure illustrates the retrieval of a target object from a reference location2 in representational space. Illustrated in the figure is the retrieval of Landmark A relative to Landmark E from a heading aligned with the front pole of the self-reference system. To retrieve relative location, the self-reference system is centered over the reference location, and then the spatial coordinates of a vector 2

The term reference location refers to both the location and facing direction (or heading) from which spatial relations are retrieved.

The Role of a Self-Reference System in Spatial Navigation

221

object-to-object vector space to be disentangled from its accessibility in bodycentered space. In this fashion, we were able to show that the locus of the “frontback” effect is in body-centered space. Thus, the asymmetries in visual-spatial information that are imposed by the physical characteristics of the body at the perceptual-motor level are mirrored at the representational level. Independent evidence for asymmetry in the accessibility of spatial information coded within a body-centered frame of reference comes from findings reported by Franklin et al. (1995). Franklin et al. developed a task that forced people to code and briefly hold in working memory the body-centered location of a target event. Participants were located at the center of a visually homogeneous, circular test space, and a brief target event occurred at randomly determined locations around the perimeter of the circle. The participant’s task was to point to the location of the target event after a short retention interval. In order to ensure that location was coded and stored in body-centered rather than geocentric coordinates, the participant rotated in place a randomly determined number of degrees during the retention interval. To illustrate the Franklin et al. (1995) task, a compass reference system is used to represent geocentric location (or absolute location in physical space) and a clock reference system to represent a body-centered system that moves with the body so that 12 on the clock-face always corresponds to the straight-ahead direction. Assume that on a sample trial, the participant faced north in geocentric space and that the target event occurred in the 3 o’clock position in body-centered space (or east in geocentric space). The participant then rotates in place so that they are facing west in geocentric space. The correct response is to point to 3 o’clock, which was the location of the target event in body-centered space, and not to 6 o’clock, which was the location of the event in geocentric space. Franklin et al. found the smallest pointing errors for target events that occurred in the anterior half of body space, suggesting that the locations of events anterior to the right-left body axis are coded and held in working memory with greater precision than posterior events. 3.2. The Interface between the Perceptual-Motor and Representational SelfReference Systems The asymmetry in the accessibility of visual-spatial information in representational space is consistent with the idea that the architecture of the representational system is organized similarly to the architecture of the perceptual-motor system. However, findings suggesting similar functional asymmetries do not address the extent to which the two systems interact with one another. If, as hypothesized, the representational self-reference system tracks body movement in relation to the object-to-object representational system, then its position in representational space should be tightly coupled to the body’s position of in physical space. Findings from perspective-taking tasks are informative in this regard. In perspective taking, the person is embedded within and oriented to a test space. Figure 2 schematically illustrates the body’s location within a configuration of objects that made up a test space used by Easton and Sholl (1995). The reference axes superposed on the body depict the self-reference system at the perceptual-motor level. To point toward an object’s visible location in perceptual-motor space, its location in retinocentric coordinates must be transformed into its location in body-centered coordinates, by adjusting for eye and head position.

222

M.J. Sholl

The typical perspective-taking task begins with a set of learning trials in which the participant first studies the locations of the surrounding objects and then closes his or her eyes and points in the remembered direction of each object. The act of closing one’s eyes transforms the task into working memory task, which relies on a representation of the test space as illustrated in the thought bubble in Figure 2. According to the self-reference system model of retrieval, pointing to the remembered location of the object involves computing its polar angle relative to the self-reference system in representational space. The computed polar angle is output to perceptualmotor self-reference system, providing the motor coordinates for the pointing response. In the learning trials, the imaged perspective and the actual perspective of the test space are the same. Once the object locations have been learned to an acceptable criterion, the perspective taking trials begin.

A

H G F

B C

E

A

H

B

D G F

C E

D

Fig. 2. On the right-hand side of the figure is a schematic illustration of a person located in a test space consisting of a configuration of objects, with each object depicted by a lettered circle. The reference system formed by the body axes at the percepetual-motor level is illustrated by the solid lines superposed on the body. A mental representation of the test space is schematically illustrated in the thought bubble. The reference system formed by the body axes at the representation level is illustrated by the dashed white lines in the representational space.

Recent research has differentiated perspective-taking performance in cases when the imagined perspective differs from the actual perspective by a rotation transformation from cases when the imagined perspective differs from the actual perspective by a translation transformation (Rieser, 1989). In the rotation transformation task, imagined facing directions vary across trials but imaged location stays constant and corresponds to the body’s actual location. In the translation transformation task, imagined location varies across trials, but imagined facing direction remains constant and corresponds to the body’s actual facing direction. In order to illustrate how the self-reference system model of retrieval explains perspective-taking performance, a sample trial from each type of task is described next, along with a schematic illustration of the operations used by the self-reference system to solve the perspective problem. Consider a typical rotation trial: Participants are instructed to point in the direction of Object H from their actual location in the test space but as if they facing Object D instead of Object A. At a conceptual level this entails aligning the self-reference

The Role of a Self-Reference System in Spatial Navigation

223

system so that its front pole is facing D and then computing the polar angle of H within its coordinate system, as illustrated in the left-hand panel of Figure 3. The computed polar angle is then output to the perceptual-motor self-reference system, which initiates the pointing response in the direction specified by the polar angle. Now consider a typical translation trial, in which the participant is instructed to point in the direction of Object H from their actual facing direction in the test space but as if standing at Object D instead of their actual location. At the conceptual level this entails positioning the representational self-reference system over Object D in representational space with its front pole parallel to the front pole of the perceptualmotor self-reference system. The polar angle of Object H is then computed and output to the perceptual-motor self-reference system which executes a response in the direction specified by the polar angle.

A

H G F

B

A

H

C E

D

G F

B C

E

Fig. 3. A schematic illustration of the retrieval of inter-object directional information at the representational level as described by the self-reference system model. The left-hand panel illustrates how the representational self-reference system computes the polar angle of Object H when the imagined perspective differs from the actual perspective by a rotation transformation and the right-hand panel illustrates the same operation when the imagined perspective differs from the actual perspective by a translation transformation. The actual perspective in this example is illustrated in Figure 2.

The distance of the imagined perspective from the actual perspective has been parametrically manipulated for both rotation transformations (Rieser, 1989) and translation transformations (Easton & Sholl, 1995). In both cases, the latency of the pointing response and the magnitude of the pointing error increased with the disparity between the actual and imagined perspectives, as the data from a translation perspective task illustrates in Figure 4. For the data illustrated, perspective taking was done in the test space illustrated in Figure 1, and similar results were found when the test space was the local college campus. A second finding reported by Easton and Sholl (1995) is of particular relevance to the current discussion. We averaged pointing latencies over reference locations separately for targets anterior and targets posterior to the reference location. We found a significant front-back effect both when the test space was a room-sized array of objects and when the test space was the college campus, thus providing additional evidence that retrieval occurred within a self-reference system. The linear relation between pointing performance and the disparity between the actual and imagined perspectives can be interpreted as follows (but see also, May, 2000; Rieser, 1995). Normally the representational self-reference system is tightly coupled to the perceptual-motor self-reference system. However, when imagining a perspective other than the actual perspective, the representational self-reference

224

M.J. Sholl

Pointing Response Latency (in secs)

system is uncoupled from the perceptual-motor system and either mentally rotated or translated depending on the perspective transformation required, to the imagined reference location in representational space. This mental transformation takes place in real time, so the greater the distance of the mental rotation or translation, the longer the response latency. 6.5 6 5.5 5 4.5 4 3.5 3 2.5 2

Y' = 0.18 X + 3.7 r2 = 0.42 0

2

4

6

8

10

Distance of Reference Object (in ft) Fig 4. Pointing latency as a function of the distance between the imagined perspective and actual perspective in a translation perspective task. Adopted from Easton and Sholl (1995).

3.3. The Perceptual-Motor and Representational Self-Reference Systems: Shared or Parallel Architectures?

The evidence reviewed so far is consistent with the idea that the perceptual-motor and representational self-reference systems share similar functional architectures. However, these findings leave unanswered the question of the extent to which the two systems are instantiated in the same physical architecture. At one extreme, the two systems may operate separately but in parallel, each instantiated in its own physical device and each with it own pool of resources. At the other extreme, the two systems may be instantiated in the same physical device and share a common pool of resources, in which case there is a single system operating simultaneously at the perceptual-motor and representational level. The question of architectural overlap was addressed with an interference paradigm, in which relative-direction judgments made when the perceptual-motor and representational self-reference systems were aligned were compared to those made when the two systems were misaligned. If the two systems are physically separate, then there should be no effect of alignment on performance. In contrast, if the two systems overlap physically, then alignment should have an effect. In this experiment, alignment was controlled within-subjects, with a 0º versus 90º facing-direction manipulation. In the 0º condition, the facing direction was perpendicular to the window (as illustrated in the arrow pointing to the left in the lefthand panel of Figure 5), and in the 90º condition, facing direction was a 90º clockwise

The Role of a Self-Reference System in Spatial Navigation

225

or counterclockwise rotation from 0º (as illustrated by the arrows pointing up and down, respectively). In a control condition, the perceptual-motor and representational self-reference systems were aligned in both the 0º and 90º conditions. That is, participants sat facing the window in the 0º condition, and physically rotated so that they were facing a quarter-turn to the right or the left in the 90º condition. In a visual interference and a non-visual interference condition, the reference systems were aligned in the 0º condition, but misaligned in the 90º condition. That is, participants sat facing the window in the 0º condition, and in the 90º condition they faced the window while imagining themselves rotated either a quarter turn to the right or the left. In the visual-interference condition, participants looked out the window at the view of the campus depicted in the right-hand panel of Figure 5 during both the 0º and 90º conditions. In the non-visual interference and the control conditions, participants first looked out the window to orient themselves and then were blindfolded during both the 0º and 90º conditions.

Fig. 5. The laboratory is located at the intersection of the arrows depicted on the map shown in the left-hand panel. The solid arrow indicates the 0º facing direction, and the dashed arrows indicate the 90º facing directions. The right-hand panel shows the view of the campus from the 0º facing direction. In the visual interference condition, participants pointed to target landmarks while looking at a trashcan positioned at the end of the arrowhead.

The interference task is similar to the perspective-taking task described earlier, except that the two tasks differ on one critical information-processing dimension, which is whether or not response latency includes a component attributable to the mental transformation of body position. On each perspective-taking trial, participants imagine themselves at a different reference location, so prior to computing the polar angle of the target object and executing the response, the representational selfreference system must be mentally shifted from its actual reference location in representational space to an imagined reference location. In contrast, the interference task tests a single imagined reference location over a block of trials, so processing on each trial includes computing the polar angle of the target and executing the response, but not mentally shifting the location of the self-reference system. To ensure that this was the case, the instructions preceding the 90º trials gave participants as much time as they needed to imagine the environment from the instructed facing direction. Then to ascertain that they had oriented themselves correctly, participants showed the experimenter the direction they would point to two practice targets from their imagined reference location. Moreover, the trials were self-paced, so if participants

226

M.J. Sholl

needed time to reorient themselves mentally between trials, they had an opportunity to do so. In the visual and nonvisual interference conditions, a within-subjects comparison between participants’ performance in the 90º condition, when the perceptual-motor and representational self-reference systems were misaligned, to their performance in the 0º condition, when they were aligned, provides the most sensitive measure of interference. However, the validity of this comparison is contingent upon the equal accessibility of spatial relations from the 0º and 90º reference locations, which operationally corresponds to no effect of facing direction in the control condition. Failure to meet this requirement calls for a less sensitive, between-subjects test of interference, wherein the performance of participants assigned to the 90º interference condition, in which the two reference systems were misaligned, is compared to the performance of participants assigned to the 90º control condition, in which they were aligned. An interference effect in the visual interference condition is attributable to either visual interference, caused by the incompatibility between the actual view from the window and the imagined view from the 90º reference location, or to reference-frame interference, caused by a misalignment between the perceptual-motor and representational reference axes. An interference effect in the non-visual interference condition can be attributed to reference-frame interference only. Thus, if the same amount of interference is observed in the visual and non-visual conditions, the locus of interference is likely to be in the reference system architecture; whereas, if interference is observed in the visual condition only, it is likely to have a specifically visual locus. Architectural interference, if observed, could be localized in the process of either computing the polar angle of the target landmark or executing the motor response, or both. Method. Participants. Participants were 36 Boston College undergraduates who had been on campus at least two semesters. The first 24 participants were randomly assigned to either the visual interference (n = 12) or control (n = 12) conditions. The non-visual interference condition (n = 12) was added after the other two conditions had been run. Procedure. The interference paradigm used a point-to-unseen-targets task. From either their actual or their imagined perspective of the environment, participants used a joystick to point in the direction of target landmarks. An Apple IIe computer controlled the experimental procedure. The software routines that controlled data collection procedures are described fully in Sholl (1987). The experimenter sat next to the participant and in front of a CRT screen, which was angled so that only the experimenter could see it. The experimenter read all the instructions to the participant. On each pointing trial, the experimenter read the name of the target as soon as it appeared on the screen. The computer recorded the latency and angle of the joystick response from the onset of the presentation of the target name on the screen. A set of target landmarks was selected from each of the quadrants formed by extending the axes shown in the left-hand panel of Figure 5. This distribution of target landmarks ensured that an equal number of targets were in the anterior and posterior half of body space from each imagined facing direction. None of the targets were visible from the window. The order of the 0° and 90° conditions was counterbalanced across participants, and half the participants within each betweensubjects condition faced in the 90° clockwise direction and the other half in the 90°

The Role of a Self-Reference System in Spatial Navigation

227

counterclockwise direction. These two between-subject variables were included in the design as control variables, and their outcomes are not reported. The two blocks of experimental trials were preceded by a block of practice trials in which participants learned how to use the joystick by pointing to the numbers on the face of a clock. Throughout the practice and experimental trials, participants in the control and nonvisual interference conditions were blindfolded, participants in the visual interference condition fixated a trashcan positioned at the center of their field of view. Results. A 3 (interference condition: control, non-visual interference, visual interference) x 2 (direction of rotation: clockwise, counterclockwise) x 2 (facing direction order: 0º – 90º or 90º –0º) x 2 (facing direction: 0º or 90º) analysis of variance was performed to generate the error term for planned comparisons. Mean pointing latencies and errors are shown in Figure 6. 0

o

90 o 4

45

3.5

40

3

35 30

2.5

25

2

20

1.5

15

1

10

0.5

5 0

0 Control

Nonvisual Interference

Visual Interference

Control

Nonvisual Interference

V isual Interference

Fig. 6. Mean pointing latencies on the left and pointing error on the right as a function of type of interference and facing direction. Error bars equal the standard error of the mean, computed from the omnibus, within-subjects mean square error term.

Pointing latency. A planned comparison showed no difference between the 0º and 90º control conditions, t (24) < 1.0, indicating that target angles are computed and pointing responses executed equally quickly from the two reference locations. Pointing latencies were significantly faster in the 0º than the 90º condition in both the non-visual, t (24) = 3.64, p = .0006, and visual, t (24) = 4.14, p = .0002, interference conditions. Pointing error. Mean pointing error was significantly larger in the 90º than the 0º control condition, t (24) = 1.82, p = .04. Therefore, pointing error in each of the 90º interference conditions was compared to error in the 90º control condition. There was no interference effect in the non-visual condition, t (24) < -1.0, but there was an effect in the visual condition, t (24) = 2.07, p = .02. It is notable that while there was a significant effect of facing direction in the control condition, there was no equivalent effect in the nonvisual interference condition. The failure to find an effect in the latter condition is likely to be a chance

228

M.J. Sholl

null result, especially given that the 0º and 90º nonvisual-interference means did not differ significantly from the 0º and 90º control means, respectively. Discussion. Visual interference was observed for both pointing accuracy and latency, and non-visual interference was observed for only pointing latency. In the case of pointing latency, the size of the interference effect was similar in the visual and nonvisual conditions, suggesting a nonvisual source of interference localized in the reference-system architecture. In contrast, for pointing error interference was observed only in the visual condition, which is consistent with a visual locus of interference. This pattern of results suggests that pointing latency and error may measure different underlying components of information processing. Interestingly, latency and error are statistically dissociable in multivariate analyses of psychometric and chronometric measures of spatial ability; findings which have led to the proposal that accuracy measures the quality of spatial representations (e.g., resolution, fidelity, etc.) and latency measures the efficiency of the operations performed on those representations (Pelegrino & Kail, 1982; Poltrock & Brown, 1984). If this interpretation is correct, then the present findings suggest that a perceived view of the local environment interferes with the quality of an imaged view, and a misalignment between the perceptual-motor and representational self-reference systems interferes with the efficiency with which the response angle is computed or executed, or both. To summarize, the latency results are consistent with the idea that the perceptualmotor and representational self-reference systems are instantiated in physically overlapping architectures.

4. The Role of the Representational Self-Reference System in Tracking Body Location in Physical Space The evidence reviewed thus far suggests that the self-reference system has the design characteristics suitable for navigational tracking. Of particular relevance in this regard are the findings from the perspective-taking and interference tasks that together suggest: (a) the self-reference system mentally updates body location relative to the object-to-object system in the absence of idiothetic input, (b) there is a the tight coupling between the body’s location in physical space and the self-reference system’s location in representational space, and (c) the perceptual-motor and representational self-reference systems are instantiated in overlapping physical architectures. These properties are consistent with a representational self-reference system that normally “moves” with the body, which is a key tracking function, but that can also be “detached” from the perceptual-motor system and moved to another reference location in representational space. In this section of the chapter, an animal model of sense of direction developed by McNaughton et al. (1991) will be adapted to the self-reference system architecture to describe the computations needed by a navigational tracking device. A device that tracks in representational space the body’s movement through physical space needs at minimum to compute the angular and linear displacements of the body and relate them to the geocentric reference frame provided by the object-toobject representational system. According to the McNaughton et al. (1991) model, the magnitude of the angular and linear displacements of the body are computed

The Role of a Self-Reference System in Spatial Navigation

229

relative to an inertial reference frame by doubly integrating with respect to time the linear and angular acceleration signals output by the vestibular system. Applying this account to the self-reference system model, the inertially based estimates of linear and angular displacement instruct the self-reference system’s “movement” relative to the object-to-object system. However, inertial dead reckoning cannot be the sole source of input to a human navigational tracking device for the following reasons: (a) humans simply are not very good at inertial dead reckoning, (b) inertial systems are prone to cumulative error, and (c) without some modification, an inertially based system does not allow a coherent sense of direction to emerge across locations in space that are visually disconnected from one another. The McNaughton et al. model deals with the latter two problems, which in turn allows a resolution of the first problem. 4.1. An Inertial System of Reference for Computing Geocentric Directional Signals In the McNaughton, et al. (1991) model, geocentric directional signals are computed relative to an inertial frame of reference and linked to local views of the physical environment. This linkage to local views allows for a mechanism, which will be described later, to correct for any error that accumulates in the inertial system. The directional signals that signify an animal’s heading with respect to geocentric space vary in value from 0° to 359°, as shown in Figure 7. An important property of the 360° directional system is that it is reset every time the body completes a full 360° rotation, so that a single value within the system always corresponds to the same direction in absolute space. For example, if the directional signal is initially set at 90° when an animal faces south, then, within the sensitivity limits of the system, whenever the animal faces south, the directional signal will be 90°, regardless of where the animal is located in the environment or what visual cues are visible from its location. Applying the construct of directional signals to the self-reference system model provides a process for computing geocentric heading, which corresponds to the orientation of the front pole of the front/back body axis in representational space. 0° 90°

270° 180°

Fig. 7. A 360° directional system in which the value of the directional signal increases in a clockwise direction and is reset to 0° with every full rotation of the body.

Head direction cells are the neural substrate for the directional signals. They have been discovered in the subicular complex and other regions of the mammalian brain connected to the hippocampus (e.g., Taube, et al., 1990; Mizumori & Williams, 1993; Robertson, et al., 1999). The cells have Gaussian response functions with peak firing rates when the animal’s head is oriented in the cell’s preferred geocentric direction,

230

M.J. Sholl

regardless of the animal’s location in the test space or the visual cues that are visible from that location. Preferred directions are evenly distributed throughout 360° and recordings taken simultaneously from two cells tuned to different geocentric headings show that they operate in synchrony. For example, as the rat changes its heading from Cell A’s preferred direction to Cell B’s preferred direction, Cell A’s firing rate decreases as Cell B’s firing rate increases. In order to illustrate how the 360° directional system works, it will be assumed that when a person enters a new environment, the directional signal is set at 0°. As long as the person does not turn his or her body, the directional signal will remain at 0°. But when the person turns to face in a new direction, the directional signal is updated to reflect the person’s new heading in the environment. The new directional signal is computed by adding the magnitude of the body’s angular displacement to the prior directional signal. The magnitude of the turning angle is computed from idiothetic cues that signal the body’s turning velocity. The model functions as if clockwise turns have positive value and counterclockwise turns, negative value. For example, if the animal’s first turn is 90° clockwise, the new directional signal is 90° (i.e., 0° + [+90°] = 90°). If instead, the animal turned 90° counter-clockwise, its new directional signal would be 270° (i.e., 0° + [-90°] = 270°). The updated directional signal becomes the new baseline to which the magnitude of the next angular displacement of the body will be added. 4.2. Visual Frame of Reference Because an inertial system is prone to error, McNaughton et al. (1991) propose that periodic visual fixes are needed to correct for error that will accumulate within the 360° directional system. To this end, as people familiarize themselves with a new environment, links are established between local visual views and internally computed directional signals. For example, local views visible when the person is facing in the 0° direction at various locations in the environment will be associated with the 0° directional signal. Likewise, local views visible from the 90° direction are associated with the 90° directional signal, and so on. With increasing environmental familiarity, the associations become stronger, so that eventually the directional signal linked to the currently visible local view will override the directional signal output by the directional system, if the two are in conflict. 4.3. Navigational Tracking Operations A basic premise of the McNaughton et al. (1991) model is that self movement produces idiothetic signals that are transformed into estimates of angular and linear body displacement. Over short time intervals, these estimates are reliable enough to allow the animal to keep track of its location and heading in a familiar large-scale environment, without any visual references to the environment. Over longer time intervals, associations formed between directional signals and local views serve a periodic corrective function. If the self-reference system does indeed function as a navigational tracking device, then its operations must be expanded to perform functions of the type proposed by McNaughton et al. (1991). At minimum, the system’s operations should include: a

The Role of a Self-Reference System in Spatial Navigation

231

process that converts idiothetic signals into estimates of linear and angular body displacement, a process that computes geocentric heading, and a process that associates geocentric heading with local views. Additionally, there must be a process that uses estimates of linear and angular displacement to shift the location and heading of the representational self-reference system relative to the object-to-object representational space and a process that corrects for mismatches between anticipated and perceived local views. The above is not intended as a complete list of the processes needed by a human tracking device, nor is it necessarily even a correct list; future research is needed to verify the psychological reality of the processes proposed. Moreover, additional work is needed to map out how these processes are interconnected with one another to form a coherent navigational tracking system.

5 Summary An empirical case has been made for the psychological reality of a representational self-reference system and for the idea that this system is designed architecturally to perform a navigational tracking function. Adopting an animal model of sense of direction to the self-reference system model offers a preliminary theoretical analysis of the operations needed by a tracking device. Future empirical work is needed to explore a self-reference system tracking function.

References Clark, H. H. (1973). Space, time, semantics, and the child. In T. E. Moore (Ed.), Cognitive development and the acquisition of language. New York: Academic Press. Easton, R. D., & Sholl, M. J. (1995). Object-array structure, frames of reference, and retrieval of spatial knowledge. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 483-500. Franklin, N., Henkel, L. A., & Zangas, T. (1995). Parsing surrounding space into regions. Memory and Cognition, 23, 397-407. Franklin, N., & Tversky, B. (1990). Searching imagined environments. Journal of Experimental Psychology: General, 119, 63-76. Gallistel, C. R. (1990). The organization of learning. Cambridge, MA: MIT Press. Loomis, J. M., Klatzky, R. L., Gollege, R. G., Cincinelli, J. G., Pellegrino, J. W., & Fry, P. A. (1993). Nonvisual navigation by blind and sighted: Assessment of path integration ability. Journal of Experimental Psychology: General, 122, 73-91. May, M. (2000, November). Imaginal repositioning in space: Transformation versus interference accounts. Paper presented at the Annual Meeting of the Psychonomic Society, New Orleans. McNaughton, B. L., Chen, L. L., & Markus, E. J. (1991). "Dead reckoning," landmark learning, and the sense of direction: A neurophysiological and computational hypothesis. Journal of Cognitive Neuroscience, 3, 190-202. Paillard, J. (1991). Motor and representational framing of space. In J. Paillard (Ed.), Brain and space (pp. 163-182). Oxford: Oxford University Press. Pellegrino, J. W., & Kail, R.J. (1982). Process analysis of spatial aptitude. In R. J. Sternberg (Ed.), Advances in the psychology of human intelligence (pp. 311-365). Hillsdale, NJ: Erlbaum.

232

M.J. Sholl

Poltrock, S. E., & Brown, P. (1984). Individual differences in visual imagery and spatial ability. Intelligence, 8, 93-138. Rieser, J. J. (1989). Access to knowledge of spatial structure at novel points of observation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 173188. Shepard, R. N., & Hurwitz, S. (1984). Upward direction, mental rotation, and discrimination of left and right turns in maps. Cognition, 18, 161-193. Sholl, M. J. (1987). Cognitive maps as orienting schemata. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 615-628. Sholl, M. J. (1995). The representation and retrieval of map and environment knowledge. Geographical systems, 2, 177-195. Sholl, M. J. (2000). The functional separability of self-reference and object-to-object systems in spatial memory. In S. O'Nuallain. (Ed.), Spatial Cognition: Foundations and Applications: Proceedings of the Mind III, Annual Conference of the Cognitive Science Society of Ireland, 1998. (pp. 45-67). Amsterdam: John Benjamins Publishing. Sholl, M. J., & Nolin, T. L. (1997). Orientation specificity in representations of place. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 1494-1507.

The Utility of Global Representations in a Cognitive Map M.E. Jefferies1 and W.K Yeap2 1 Department of Computer Science University of Waikato, Hamilton, New Zealand email: [email protected] 2 Artificial Intelligence Technology Centre Auckland University of Technology, Auckland, New Zealand email: [email protected]

Abstract. In this paper we propose the use of small global memory for a viewer’s immediate surroundings to assist in recognising places that have been visited previously. We call this global memory a Memory for the Immediate Surroundings (MFIS). Our previous work [1, 2] on building a cognitive map has focused on computing a representation for the different local spaces the viewer visits. The different local spaces which are computed can be connected together in the way they are experienced to form a topological network which is one aspect of a cognitive map of the spatial environment. The problem with topological representations is that using them one cannot easily detect that one is reentering a previously visited part of the environment if it is approached from a different side to the one used previously. Thus we have developed a cognitive map representation which comprises an MFIS working in cooperation with the topological network. The idea that a global map is present as part of the cognitive mapping process is increasingly appealing. Robotics researchers have used them from the early days of autonomous mobile robots. However, they have shown that it is difficult to compute an accurate global representation because of errors. There is now increasing evidence that a global map is used in animals and many simulation models have incorporated the use of such a map. In this paper we review these works, discuss this notion of a global map in cognitive mapping, and show how one could be computed with minimum effort.

key words: cognitive map, path integration, global spatial representation, local spatial representation

1 Introduction For researchers grappling with the basic issues involved in representing an individual’s experience of their spatial environment, the nature of the underlying representation is at the core of the problem. Psychologists and geographers examine the behaviour of the animal or human to determine the nature of the information that has been stored and how it is being used [3-5] whereas artificial intelligence and robotics researchers D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 233−246, 2001.  Springer-Verlag Berlin Heidelberg 2001

234

M.E. Jefferies and W.K. Yeap

are concerned with computational issues [6-8, 2]. Two themes which emerge from the studies of both groups are: (i) the notion of a representation for the local space i.e. the small area of the environment the individual is currently in, versus (ii) a global representation in which conceivably the individual’s total experience of their spatial environment could be represented using a single coordinate system. Related to these is the idea of a metric representation, where properties such as distance, size and location are explicitly or implicitly represented, versus a topological representation where relationships such as connectivity between individual elements are represented. Global representations in the sense the term is used in this paper are metric. The local space could be represented topologically as in for example the relationships between some key landmarks, or metrically where the structure of the space itself would be identified within some reference frame (see [9] for a discussion of topological versus spacebased representations of local space). One’s total memory for the environment could be stored in a topological representation, as a collection of local space representations, each with its own reference frame and the connections between pairs of local space representations would indicate that one could travel directly from one to the other. This idea of a topological network of metric local space representations is central to the computational theory of cognitive maps we have developed [10, 2]. We argue in [2] that as one traverses an environment one must initially compute a representation which identifies the space one currently occupies. The algorithm we use emphasises the importance of detecting exits in view from the surfaces perceived and from these exits a boundary for the local space is computed. Each local space is computed using its own cartesian coordinate reference frame. The resulting representation is called an Absolute Space Representation (ASR), a term which emphasises the independent, local nature of each local space visited. The ASRs for the different spaces visited form one’s cognitive map and if one remembers how one passed from one space into another they will be connected to form a topological representation of the traversed environment (see Fig. 1). However one of the limitations of this approach is that the location of the individual is defined only in terms of the local space they currently occupy. With this representation one cannot easily determine that one is re-entering a previously visited part of the environment from one’s location (see Fig. 2). The different spaces visited have not been integrated in such a way as to yield the information that the previous space visited, and the current one, occupy the same location in space. One could remember that one is reentering a familiar part of the environment if one remembered how the adjacent local spaces were connected in the cognitive map. But one would only realise neighbouring spaces were adjacent if they were experienced as such. Failing this a more computationally demanding process of recognition based on the identification of key features and matching must be employed. Kuipers and Byun [11] employ topological matching. They use local metric information to loosely match a robot’s representation for the place it is currently at with those of places already visited. If it appears that a match is possible its validity is checked by getting the robot to follow known routes to adjacent places and back to the current place. This route information would have been gathered on previous visits to the place. If the routes can be followed as expected then the current place is identified as the one encountered previously.

The Utility of Global Representations in a Cognitive Map

235

1 2 3 4 5

7

6

(a) 1

2

3

4

5

6

7

(b) Fig. 1. (a) The environment (b) The topological network of ASRs computed as a simulated viewer follows the path in (a). The ASRs are numbered in the order in which they are experienced. The numbered areas in the environment in (a) correspond to the same numbered ASRs in the topological network.

Our investigations into the recognition problem have led us to propose the use of a limited form of global representation, used in conjunction with the topological network of local space representations, to assist with recognition. Global representations are popular in robotics but the huge amount of information that needs to be stored makes them unwieldy to maintain. An added complexity is the need to correct for cumulative error so that the representation stays aligned with its environment. We favour instead a limited (in extent) global memory that moves with an individual as they traverse the environment. Deciding what the extent should be is a trade-off

236

M.E. Jefferies and W.K. Yeap

12

1

11

2 3 4

10 9

5

7

6

8 (a) 1 12 11

2

3

4

10 9

5

8

6

7

(b) Fig. 2. (a) the environment; (b) the topological network of ASRs computed as the path in the environment is traversed in our computer simulation of our theory of cognitive maps [2]. Following on from Fig. 1 ASRs 8-12 are computed. The darker shaded ASR, ASR12 is the ASR the viewer currently occupies. When ASR12 is computed the viewer has reentered ASR1. However the viewer cannot detect this from its cognitive map.

between how much information is remembered and (i) how much error will accumulate and (ii) how much effort is required to maintain the global memory. We propose that it should comprise the recently visited ASRs in the immediate vicinity of the ASR the viewer currently occupies so that it is possible to detect when these ASRs are being re-entered. The representation is thus termed a Memory for the Immediate Surroundings (MFIS) and a discussion of our early work on it can be found in [12, 2]. Our recent work on the ASR has led us to postulate that the initial ASR computed is a vague, uncertain representation that reflects the viewer’s initial attempt to make sense of the new space they have entered [1]. We briefly describe the fuzzy ASR in Section

The Utility of Global Representations in a Cognitive Map

237

4. In Section 5 we show how our initial ideas on the MFIS have been extended to include fuzzy ASRs. Our method of representing the spatial environment of the viewer combines metric, topological, global and local representations. Methods for representing the environment which combine metric and topological representations are common [6, 13, 8, 14]. Ones which combine local with global information are less common but two which are notable are those of Chatila [15] and Thrun [14]. Some of the key works by roboticists in this area are discussed in Section 3. In Section 2 we describe the work of psychologists on global memory systems. We present our conclusions in Section 6.

2 Global spatial representations in animals In many animals including humans the mechanism by which the animal keeps track of its location in a global framework is called path integration [16-20]. It involves the animal maintaining a fix on its position by updating each change in its position in a geocentric coordinate system [3]. Using it animals are able to compute a direct path “home” after following a circuitous route on the way out. However the usefulness of this global position in path integration coordinates is limited by its accuracy which gradually worsens as random and systematic errors accumulate [3]. It is not difficult to imagine the consequences of a systematic under or over estimation of the size of each step taken. The degree of turn would also be difficult to compute accurately. McNaughton [21] reports that research with rats suggests that when the environment is new the animal relies on path integration. As it becomes more familiar with the environment visual features are coded within the path integration framework. These features can be used to confirm or deny the results of the path integration system and can be used to realign the location obtained from path integration with the physical environment. Gallistel and Cramer [3] suggest that to overcome the problem of accumulated error, the animal takes a positional fix computing the discrepancy between its current position and orientation and what the animal thinks these values are. It uses the discrepancy to realign itself with the environment. Cheng [22] and Margules and Gallistel [23] showed that rats and Hermer [5] showed that young children, on becoming disoriented use the shape of the surrounding environment to reorient themselves. Several models of the path integration system have been proposed [24-27]. In some way they all attempt to account for the mathematics of the integration process. Some even attempt to account for random [24] and systematic [26] errors. Redish and Touretzky [27] represent the different locations the rat visits by individual place codes. However, the fact that each one of these place codes is tied to the location represented in path integration coordinates, means that Redish and Touretzky have constructed a global representation of the environment with all the inherent difficulties of error accumulation. Significantly though, they have realised the importance of explicitly tying the path integration system to what is actually observed in the environment. While the definition of the path integration system is that it operates independently of vision, at some point the animal will be aware that where it thinks it is, is not where it really is.

238

M.E. Jefferies and W.K. Yeap

To know this, there must be a strong link between the location recorded, the structure of the surrounding space and its identifying features. Redish and Touretzky [27] and McNaughton et. al. [21] address this issue, proposing that the representation for the environment is built around the different locations the animal visits. It is difficult to imagine that an animal could remember its total experienced environment in this way within a single framework. The environments used by Redish and Touretzky in their simulations and by their robot, and those in the experiments McNaughton et. al. refer to, are extremely small compared to the part of the world the animal could be expected to be familiar with. McNaughton et. al., however, suggest that the animal would typically use several reference frames for different parts of its environment, and Redish and Touretzky discuss the significance of multiple reference frames for their model. This notion is closer to the idea we have for a global memory which is limited in size, and which follows the viewer traversing the environment constantly changing the frame of reference.

3 Global spatial representations in robotics Global representations of the spatial environment are popular in autonomous mobile robots [28-30, 14] because such a representation is deemed essential to the problem of recognising unambiguously where one is in the environment at all times. The idea is to store the robot’s location in a global framework along with all the perceived sensory information. The robot’s location in this representation is adjusted every time the robot moves and the robot’s egocentric view is integrated into the representation at each turn or move of the robot. The problem with these global representations is that they are prone to error accumulation as the robot is unable to accurately determine how far it has moved or turned. The problem is similar to that of the path integration systems described in the previous section. The sensory data is often captured in a fine grained occupancy grid [31, 32, 30, 14] which if one imagines it laid over the physical environment then each cell records if that space in the environment is occupied or not. It follows that the representation determines the space in the environment where the robot is free to roam. Given the inaccuracies in sensing especially using sonar, recent implementations of grid-based methods record in each cell a value which somehow conveys the uncertainty associated with the cell being occupied or not. Pagac [30] uses the Dempster-Shafer inference rule to determine the probability of a cell being occupied, whereas Thrun [14] uses Bayes rule to represent in his cells the subjective belief that the cell is occupied. Of greater interest is how Thrun aligns the robot with its map so as to minimise the cumulative error due to wheel slippage and drift. The key is how he estimates the robot’s position. The robot’s wheel encoders, map matching and wall orientation are all used to acquire a good estimate of the robot’s location in its global environment. The wheel encoders can be used to work out the robot’s position but are inaccurate over large distances. Each time the robot takes a sensor reading it creates a local grid map. How well this local map correlates with the global map is determined by the robot’s position, and thus provides useful information as to the robot’s true position in its global map. The wall orientation method relies on the assumption that walls are par-

The Utility of Global Representations in a Cognitive Map

239

allel or orthogonal to each other or differ by at most 15o from these main orientations. With this method Thrun mostly manages to maintain the error at a sufficiently small amount so that the representation stays aligned with the physical environment and the robot is able to recognise previously visited locations. Thrun’s [14] results are impressive even though they are limited to regular buildinglike environments. However there is considerable computational overhead in maintaining such a large amount of information in a single representation. Reducing the complexity of the representation while still being able to recognise previously visited places are important issues in robotics.

4 Fuzzy ASRs The ASRs in Fig. 1 are exact and have been drawn this way so that the reader can see how each local space representation relates to the physical environment from which it was computed. Humans “see” the places they visit with much precision. Even though we can easily determine where objects are when we are looking directly at them, rarely are we able to reproduce from memory an exact description of the paces we have visited. Yet we are able to make good use of the vague and imprecise memories we have for our environment. The decisions we make on how to get from one place to another are often based on sketchy memories for the places we have visited along the way. We need to be able to accurately locate objects in front of us so that we can navigate around them and for the many other actions we perform in the local space. However once there is no longer immediate feedback from the physical environment it is difficult for animals and robots to maintain the same level of accuracy. Our solution is to only consider the current view a robot/human would have of their environment as “accurate”. We use the term accurate loosely here, as in the real world the degree of precision would depend on the accuracy of the information delivered by the robot or human sensory systems. Thus, the ASR representation has two parts, one which covers the current view and one which can cover the whole local space but which reflects the uncertainty which surrounds parts of the environment no longer in view. We call the latter a fuzzy ASR. Two important pieces of information that need to be computed early on in the fuzzy ASR (see [1, 2] for a more in depth discussion on this) are the rough extent of the local space and the approximate whereabouts of exits. Other information can be attached later as the viewer becomes more familiar with the environment. The rough extent of the local space is encapsulated in the ASR by a rectangle which roughly covers its area (see Fig. 3). This rectangle is then divided radially into octants. Exits are located in one or other of these octants and in some cases may span more than one octant. Importantly, the connections between neighbouring fuzzy ASRs are identified by the octants in each fuzzy ASR which contain the exit making the connection. Fig. 5 portrays fuzzy ASRs so computed and the corresponding cognitive map. These fuzzy ASRs are the ASRs which comprise both the topological network and the MFIS. Thus for the remainder of this paper whenever we use the term ASR we are in fact referring to the fuzzy ASR.

240

M.E. Jefferies and W.K. Yeap

s2 e2 e3 s3 e4

e1

e1

e1 e2 e3e4e3

s1 s4 s5 (a)

(b)

Fig. 3. (a) An exact ASR. (b) the fuzzy ASR description which devolves from the ASR in (a). Exits are allocated to their appropriate octants as shown.

5 A global memory for the immediate surroundings – the MFIS There are two significant problems in maintaining a global memory of the immediate surroundings. The first concerns the definition of the immediate surroundings itself. When one moves a step, has one’s immediate surroundings changed? If it has, the representation could be computationally very expensive to maintain. The second concerns the amount of information that needs to be tracked. How could one then maintain a reasonably accurate representation given that one’s perception of the world is inherently very noisy? Technically, the first problem concerns what frame of reference is appropriate for the MFIS. One could use either an egocentric or an allocentric frame of reference. It is clear from an implementation viewpoint it is inefficient to use an egocentric reference frame. To use an allocentric reference frame, one has to specify where the reference frame should be centred. The choice of this external point need not be chosen arbitrarily if we use the current ASR as the frame of reference. An ASR has an extent and any part of it would be suitable. In previous implementations of the MFIS we chose the entrance to the current ASR as the centre of the reference frame. However in fuzzy ASRs exits do not have an exact location. In fact the exact location of an exit can fall outside the fuzzy ASR boundary. Thus we choose the centre of the current ASR as the centre of the MFIS. When one moves out of the current local space, the origin of the MFIS is shifted to the centre of the next ASR. The extent of the MFIS can be defined arbitrarily but, more importantly, need not be defined exactly, say in metric terms. Varying its size is a trade-off between how much information is remembered and (i) how much error will accumulate and (ii) how much effort is required to compute it. Given an MFIS with a fixed size, part of an ASR will often be excluded as it lies outside the area covered by the MFIS. The advantage of having the MFIS is to help recognise that nearby local spaces were visited before. We, therefore, do not want to remove a part of an ASR (from the MFIS) because it falls

The Utility of Global Representations in a Cognitive Map

241

outside the area covered by the MFIS. Since the MFIS extent is defined arbitrarily, it is better to include the whole ASR if a part of it lies within the area covered by the MFIS. The second problem, which concerns the vast amount of information that needs to be tracked, can now be solved by observing that the spatial arrangement of individual surfaces in each ASR is already maintained in the ASRs themselves. Tracking them becomes effortless as long as we treat the ASR as a whole when maintaining the MFIS. We thus propose a limited global memory (the MFIS) containing the last few local spaces visited. It contains the same basic representation for the local space as the topological map, but in addition it contains global location information that allows the viewer to determine when a recently encountered local space is being revisited. Fig. 4 shows how the viewer can use the global MFIS to determine when a previously visited ASR is re-entered. Using an MFIS comprising fuzzy ASRs the viewer is able to maintain a rough notion of the whereabouts of recently visited ASRs. As the viewer moves from one local space to another the newly entered ASR is integrated into the MFIS. The centre of the new ASR becomes the centre of the MFIS. ASRs which fall outside the extent of the MFIS drop off. To determine if a recently visited local space has been encountered we check the viewer’s location against ASRs in the MFIS In the implementation of the MFIS reported in [12] its representation consisted of all the surfaces and exits (in exact coordinates) which could be found inside the predefined extent of the MFIS. Determining if a previously encountered ASR was being revisited was simply a matter of checking if one of its exits had been crossed. This method works well when ASRs are computed accurately and has allowed us to explore some of the limitations of the MFIS representation. However the brittleness of the above method becomes apparent when it is subjected to an MFIS comprising fuzzy ASRs. The problem is that exits are on the boundary of the ASR and in the fuzzy ASR the boundary is expressed very coarsely. Even allowing for some error tolerance, the exact exit coordinates would often not be inside the fuzzy ASR. To overcome this problem we instead use the centres of the ASRs contained in the MFIS. If the centre of one of these ASRs is contained inside this new ASR then the earlier representation is retrieved and is updated, if necessary, with any new information contained in the latest description. One problem occurs with this method. In some situations where the ASRs overlap due to their coarse boundaries it is possible for the centre of one ASR to catch the edge of the overlapping ASR even though they have been computed for quite distinct local spaces. Therefore when we find an ASR in the MFIS whose centre is within the boundary of the current ASR’s rough extent we perform a check to ensure that the centre of the current ASR is also inside the matched ASR. The advantage of this method is that only the centre of each ASR needs to be recorded in the global coordinate system of the MFIS. Figures 5 and 6 show a cognitive map (topological network and MFIS) being constructed as a viewer traverses the environment shown. The ASRs are numbered in the order in which they are experienced. In Fig. 5(a) the viewer is in the process of leaving ASR2. The MFIS is centred on ASR2. In Fig. 5 (b) the viewer is again leaving ASR2 having returned to it via ASRs 3-7. ASR2 has been recognised from its location in the

242

M.E. Jefferies and W.K. Yeap

12

1

11

3

2

4

10 9

5

8

(a)

7

6

(a)

1 12

2

11

3

4

10 9

5

8

6

7

(b)

1

1 2 3

11

4

10 9 8

5 6

7

(c) Fig. 4. (a) The viewer has traversed a circuitous route arriving back in the same room from which it started. (b) Using a topological representation in which each ASR has its own coordinate system the viewer cannot use its location to determine that ASR1 is being revisited and it thus constructs a new ASR, ASR12, for the same physical space. (c) An MFIS. The ASRs are laid out side by side to indicate a single coordinate system. Using this global representation one can detect from the viewer’s location that ASR1 is being re-entered. Note we have used exact ASRs here merely to demonstrate the basic MFIS process more clearly.

The Utility of Global Representations in a Cognitive Map

2

2

2

243

1

1

1

(a) 2 2 1

2 3

1 7 6

4

34

1

7

5

3

6

5

4

5

6

7

(b) 2 2

9

1 8

2 3

1 3 4 7 6

5

8

4

3

7

1

6 5

9

4 8

9

7

6

(c) Fig. 5. A cognitive map with the topological network and MFIS (left) the environment; the numbers correspond to the ASRs in the cognitive map. (centre) the topological network of ASRs (right) the MFIS. The marks the centre of the currently occupied ASR and also the centre of the MFIS’s coordinate system. The darker shaded octants in each circle mark the presence of an exit in that octant of the ASR.

MFIS and this description has been retrieved as the current ASR. ASR2 now has links to ASR1, ASR3 and ASR7. In Fig. 5(c) the viewer has reached ASR9 via ASR2, ASR1 and ASR8. At this point ASR5 drops out of the MFIS because it is no longer within the region the MFIS covers. In Fig. 6 the local space covered by ASR5 is reentered. ASR5 is no longer in the MFIS therefore a new fuzzy ASR, ASR11 is computed.

244

M.E. Jefferies and W.K. Yeap

2 3

1

8

34 9

1 8

7 6 10

9 5, 11

2

4

2 7

3 6 5

10

9

11

1 8

7 6 10

4

11

Fig. 6. Continuation of Fig. 5. A cognitive map with the topological network and MFIS (left) the environment; the numbers correspond to the ASRs in the topological network. (centre) the topological network (right) the MFIS. The marks the centre of the currently occupied ASR and also the centre of the MFIS’s coordinate system. The darker shaded octants in each circle mark the presence of an exit in that octant of the ASR.

As with the path integration described in the animal studies of Section 2, the usefulness of this global location information is limited. The quality of information will quickly degrade as the viewer moves further away from previously encountered ASRs and error accumulates. However, if the MFIS is restricted to include recently visited ASRs, not too far away from the viewer’s current location, then some useful recognition is possible.

6 Conclusion The MFIS plays an important role in structuring the cognitive map especially early on in the cognitive mapping process when the detail necessary for more comprehensive recognition is not yet available. It provides the framework for making connections in the cognitive map and importantly it also provides a sense of boundedness larger than that provided by a single ASR. In this sense its role is analogous to McNaughton’s [21] and Redish and Touretzky’s [27] notion of the path integration system underpinning the formation of a cognitive map in animals. The strength of the method lies in its simplicity, however the compromise is that it is not possible to recognise every local space that is revisited. This means that every connection possible between every local space is not made explicit in the topological network. The implication for navigation is that some planned paths will be longer than they need have been. The individual can still find its way around its environment, but less efficiently. Recognition is a very

The Utility of Global Representations in a Cognitive Map

245

complex process and draws on many sources of information. The method we have described here performs some useful recognition early in the cognitive mapping process. However in reality it would be but the first layer in a comprehensive recognition system.

References 1. Jefferies, M.E. and Yeap, W.-K.: Representing the local space qualitatively in a cognitive map. In: Proceedings of the Twelfth Annual meeting of the Cognitive Science Society. Madison, Wisconsin. (1998). 525-530 2. Yeap, W.K. and Jefferies, M.E.: Computing a representation of the local environment. Artificial Intelligence, 107 (1999) 265-301 3. Gallistel, C.R. and Cramer, A.E.: Computations on metric maps in mammals: getting oriented and choosing a multi-destination route. The Journal of Experimental Biology, 199 (1996) 211-217 4. Golledge, R.G., Dougherty, V., and Bell, S.: Acquiring spatial knowledge: survey versus route-based knowledge in unfamiliar environments. Annals of the Association of American Geographers, 85(1) (1995) 134-158 5. Hermer, L.: Internally coherent spatial memories in a mammal. Neuroreport, 8 (1997) 17431747 6. Kortenkamp, D. and Weymouth, T.: Topological mapping for mobile robots using a combination of sonar and vision sensing. In: Proceedings of the 12th AAAI. (1994). 979-984 7. Kuipers, B.: The spatial semantic hierarchy. Artificial Intelligence, 119 (2000) 191-233 8. Langley, P., Pfleger, K., and Sahami, M.: Lazy acquisition of place knowledge. Artificial Intelligence Review, 11 (1997) 315-342 9. Yeap, W.K. and Jefferies, M.E.: On early cognitive mapping. To appear, (2001) 10. Yeap, W.K.: Towards a Computational Theory of Cognitive Maps. Artificial Intelligence, 34 (1988) 297-360 11. Kuipers, B. and Byun, Y.-T.: A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations. Journal of Robotics and Autonomous Systems, 8 (1991) 47-63 12. Yeap, W.K., Jefferies, M., and Naylor, P.: An MFIS for computing a raw cognitive map. In: Proceedings of the 12th International Joint Conference on Artificial Intelligence. (1991) 13. Kuipers, B., Froom, R., Lee, W.-Y., and Pierce, D.: The semantic hierarchy in robot learning. In: Robot Learning, Connell, J. and Mahadevan, S., (eds.): Kluwer Academic Publishers. (1993) 14. Thrun, S.: Learning metric-topological maps for indoor mobile robot navigation. Artificial Intelligence, 99(1) (1998) 21-71 15. Chatila, R. and Laumond, J.-P.: Position referencing and consistent world modelling for mobile robots. In: Proceedings of the 1985 IEEE International Conference on Robotics and Automation. (1985) 16. Biegler, R.: Possible uses of path integration in animal navigation. Animal Learning and Behaviour, 28(3) (2000) 257-277 17. Etienne, A.S., Maurer, R., Georgakopoulos, J, and Griffin, A.: Dead reckoning (path integration), landmarks, and representation of space in a comparative perspective. In: Wayfinding Behavior - Cognitive Mapping and other Spatial Processes, Golledge, R.G. (ed.): Baltimore: The Johns Hopkins University Press. (1999) 197-228 18. Gallistel, C.R.: The Organisation of Learning: Cambridge, MA: Bradford Books/MIT Press. (1990)

246

M.E. Jefferies and W.K. Yeap

19. Loomis, J.M., Klatzky, R.L., Golledge, R.G., and Philbeck, J.W.: Human navigation by path integration. In: Wayfinding Behavior - Cognitive Mapping and other Spatial Processes, Golledge, R.G. (ed.): Baltimore: The Johns Hopkins University Press. (1999) 125-151 20. Maurer, R. and Seguinot, V.: What is modelling for? A critical review of the models of path integration. Journal of Theoretical Biology, 175 (1995) 457-475 21. McNaughton, B.L., Knierim, J.J., and Wilson, M.A.: Vector encoding and the vestibular foundations of spatial cognition: Neurophysiological and computational mechanisms. In: The Cognitive Neurosciences, Gazzaniga, M.S. (ed.) Cambridge, MA: Bradford/MIT press. (1995) 22. Cheng, K.: A Purely Geometric Module in the Rat’s Spatial Representation. Cognition, 23 (1986) 149-178 23. Margules, J. and Gallistel, C.R.: Heading in the rat: determination by environmental shape. Animal Learning and Behaviour, 16(4) (1988) 404-410 24. Benhamou, S.: On systems of reference involved in spatial memory. Behavioural Processes, 40 (1997) 149-163 25. Mittelstaedt, H. and Mittelstaedt, M.L.: Homing by path integration. In: Avian Navigation, Papi, F. and Wallraff, H.G. (eds.) Berlin: Springer Verlag. (1982) 26. Muller, M. and Wehner, R.: The hidden spiral: Systematic search and path integration n ants, cataglyphis fortis. Journal of Comparative Psychology A, 175 (1994) 525-530 27. Redish, A.D. and Touretzky, D.S.: Navigating with landmarks: Computing goal locations from place codes. In: Symbolic Learning, Ikeuchi, K. and Veloso, M. (eds.) Oxford University Press. (1996) 28. Chatila, R.: Path planning and environment learning in a mobile robot system. In: Proceedings of the European Conference on Artificial Intelligence. (1982) 29. Crowley, J.L.: Dynamic world modelling for an intelligent mobile robot using a rotating ultra-sonic ranging device. In: Proceedings of the 1985 IEEE International Conference on Robotics and Automation. (1985) 128-135 30. Pagac, D., Nebot, E.M., and Durrant-Whyte, H.: An evidental approach to map-building for autonomous vehicles. IEEE Transactions on Robotics and Automation, 14(4) (1998) 623629 31. Elfes, A.: Using occupancy grids for mobile robot perception and navigation. IEEE Computer, June (1989) 46-57 32. Moravec, H.P.: Sensor fusion in certainty grids for mobile robots. AI Magazine, Summer (1988) 61-64

How Spoken Language and Signed Language Structure Space Differently Leonard Talmy Department of Linguistics and Center for Cognitive Science University at Buffalo, State University of New York Linguistic research to date has determined many of the factors that structure the spatial schemas found across spoken languages. It is now feasible to integrate these factors and to determine the comprehensive system they constitute for spatial structuring in spoken language. This system is characterized by several features: It has a relatively closed universally available inventory of fundamental spatial elements that are combined to form whole schemas. It has a relatively closed set of categories that these elements appear in. And it has a relatively closed small number of particular elements in each category, hence, of spatial distinctions that each category can ever mark. An examination of signed language shows that its structural representation of space systematically differs from that in spoken language in the direction of what appear to be the structural characteristics of scene parsing in visual perception. Such differences include the following: Signed language can mark finer spatial distinctions with its inventory of more structural elements, more categories, and more elements per category. It represents many more of these distinctions in any particular expression. It also represents these distinctions independently in the expression, not bundled together into "pre-packaged" schemas. And its spatial representations are largely iconic with visible spatial characteristics. The findings suggest that instead of some discrete whole-language module, spoken language and signed language are both based on some more limited core linguistic system that then connects with different further subsystems for the full functioning of the two different language modalities.

Linguistic research to date has determined many of the factors that structure the spatial schemas found across spoken languages ( e g Gruber 1965, Fillmore 1968, Leech 1969, Clark 1973, Bennett 1975, Herskovits 1982, Jackendoff 1983, Zubin and Svorou 1984, as well as myself, Talmy 1983, 2000a, 2000b). It is now feasible to integrate these factors and to determine the comprehensive system they constitute for spatial structuring in spoken language. This system is characterized by several features. With respect to constituency, There is a relatively closed universally available inventory of fundamental spatial elements that in combination form whole schemas. There is a relatively closed set of categories that these elements appear in. And there is a relatively closed small number of particular elements in each category, hence, of spatial distinctions that each category can ever mark. With respect to synthesis, selected elements of the inventory are combined in specific arrangements to make up the whole schemas represented by closed-class spatial forms. Each such whole schema that a closed-class form represents is thus a "pre-packaged" bundling together of certain elements in a particular arrangement. Each language has in its lexicon a relatively closed set of such pre-packaged schemas (larger than that of spatial closed-class forms, due to polysemy) that a speaker must select among in depicting a spatial scene. Finally, with respect to the whole schemas themselves, these schemas can undergo a certain set of processes that extend or deform them. Such processes are perhaps part of the overall system so that a language's relatively closed set of spatial schemas can fit more spatial scenes. An examination of signed language2 shows that its structural representation of space An expanded version of the present paper is in Talmy (forthcoming) I here approach signed language from the perspective of spoken language because it is not at this point an area of my expertise. For their help with my questions on signed language, my thanks to Paul D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 247−262, 2001.  Springer-Verlag Berlin Heidelberg 2001

248

L. Talmy

systematically differs from that in spoken language in the direction of what appear to be the structural characteristics of scene parsing in visual perception. Such differences include the following: Signed language can mark finer spatial distinctions with its inventory of more structural elements, more categories, and more elements per category. It represents many more of these distinctions in any particular expression. It also represents these distinctions independently in the expression, not bundled together into pre-packaged schemas. And its spatial representations are largely iconic with visible spatial characteristics. When formal linguistic investigation of signed language began several decades ago, it was important to establish in the context of that time that signed language was in fact a full genuine language, and the way to do this, it seemed, was to show that it fit the prevailing model of language, the Chomskyan-Fodorian language: module. Since then, however, evidence has been steadily accruing that signed language does diverge in various respects from spoken language. The modem response to such observations -- far from once again calling into question whether signed language is a genuine language -- should be to rethink what the general nature of language is. Our findings suggest that instead of some discrete wholelanguage module, spoken language and signed language are both based on some more limited core linguistic system that then connects with different further subsystems for the full functioning of the two different language modalities.

2 Fundamental Space-Structuring Elements and Categories in Spoken Language An initial main finding emerges from analysis of the spatial schemas expressed by closedclass (grammatical) forms across spoken languages. There is a relatively closed and universally available inventory of fundamental conceptual elements that recombine in various patterns to constitute those spatial schemas. These elements fall within a relatively closed set of categories, with a relatively closed small number of elements per category. 2.1 The Target of Analysis As background to this finding, spoken languages universally exhibit two different subsystems of meaning-bearing forms. One is the "open-class" or "lexical" subsystem, comprised of elements that are great in number and readily augmented -- typically, the roots of nouns, verbs, and adjectives. The other is the "closed-class" or "grammatical" subsystem, consisting of forms that are relatively few in number and difficult to augment -- including such bound forms as inflections and such free forms as prepositions and conjunctions. As argued in Talmy (2000a, ch. l), these subsystems basically perform two different functions: open-class forms largely contribute conceptual content, while closed-class forms determine conceptual structure. Accordingly, our discussion focuses on the spatial schemas represented by closedclass forms so as to examine the concepts used by language for structuring purposes.

Across spoken languages, only a portion of the closed-class subsystem regularly represents spatial schemas. We can identifL the types of closed-class forms in this portion and group them according to their kind of schema. The type of closed-class forms with schemas for paths or sites include the following: (1) forms in construction with a nominal, such as prepositions like English across (as in across the field) or noun affixes like the Finnish illative suffix -:n4into', as well as prepositional complexes such as English in front of or Japanese constructions with a "locative noun" like ue 'top surface', (as in teeburu no ue ni 'table GEN top at' = "on the table"); (2) forms in construction with a verb, ,such as verb Dudis, Karen Ernrnorey, Samuel Hawk, Nini Hoiting, Marlon Kuntze, Scott Liddell, Stephen McCullough, Dan Slobin, Ted Suppala, Alyssa Wolf, and others, -- who are not responsible for my errors and oversights.

How Spoken Language and Signed Language Structure Space Differently

249

satellites like English out, back and apart (as in They ran out / back / apart); (3) deictic determiners and adverbs such as English this and here; ( 4 ) indefinites, interrogatives, relatives, etc., such as English everywhere / whither / wherever); ( 5 ) qualifiers such as English way and right (as in It's way / right up there); and (6) adverbials like English home (as in She isn 't home). 2.2 Determining the Elements and Categories A particular methodology is used to determine fundamental spatial elements in language. One starts with any closed-class spatial morpheme in any language, considering the full schema that it expresses and a spatial scene that it can apply to. One then determines any factor one can change in the scene so that the morpheme no longer applies to it. Each such factor must therefore correspond to an essential element in the morpheme's schema. To illustrate, consider the English preposition across and the scene it refers to in The board lay across the road. Let us here grant the first two elements in the across schema (demonstrated elsewhere): (1) a Figure object (here, the board) is spatially related to a Ground object (here, the road); and (2) the Ground is ribbonal -- a plane with two roughly parallel line edges that are as long as or longer than the distance between them. The remaining elements can then be readily demonstrated by the methodology. Thus, a third element is that the Figure is linear, generally bounded at both ends. if the board were instead replaced by a planar object, say, some wall siding, one could no longer use the original across preposition but would have to switch to the schematic domain of another preposition, that of over, as in The wall siding lay over the road. A fourth element is that the axes of the Figure and of the Ground are roughly perpendicular. If the board were instead aligned with the road, one could no longer use the original across preposition but would again have to switch to another preposition, along, as in The board lay along the road. Additionally, a fifth element of the across schema is that the Figure is parallel to the plane of the Ground. In the referent scene, if the board were tilted away from parallel, one would have to switch to some other locution such as The board stuck into /out of the road. A sixth element is that the Figure is adjacent to the plane of the Ground. If the board were lowered or raised away from adjacency, even while retaining the remaining spatial relations, one would need to switch to locutions like The board lay (buried) in the road. / The board was (suspended) above the road. A seventh element is that the Figure's length is at least as great as the Ground's width. If the board were replaced by something shorter, for example, a baguette, while leaving the remaining spatial relations intact, one would have to switch from across to on, as in The baguette lay on the road. Finally, an eighth element is that the axis of the Figure is horizontal (the plane of the Ground is typically, but not necessarily, horizontal). Thus, if one changes the original scene to that of a spear hanging on a wall, one can use across if the spear is horizontal, but not if it is vertical, as in The spear hung across the wall. / The spear hung up and down on the wall. Thus, from this single example, the methodology shows that at least the following elements figure in closed-class spatial schemas: a Figure and a Ground, a point, a line, a plane, a boundary (a point as boundary to a line, a line as boundary to a plane), parallelness, perpendicularity, horizontality, adjacency (contact), and relative magnitude. In the procedure of systematically testing candidate factors for their relevance, the elements just listed have proved to be essential to the selected schema and hence, to be in the inventory of fundamental spatial elements. But it is equally necessary to note candidates that do not prove out, so as to know which potential spatial elements do not serve a structuring function in language. In the case of across, for example, one can probe whether the Figure, like the board in the referent scene, must be planar -- rather than simply linear -- and coplanar with the plane of the Ground. It can be seen, though, that this is not an essential element to the across schema, since this factor can be altered in the scene by standing the board on edge without any need to alter the preposition, as in The board layflat / stood on

250

L. Talmy

edge across the railway bed. Thus, coplanarity is not shown by across to be a fundamental spatial element. However, it does prove to be so in other schemas, and so in the end must be included in the inventory. This is seen for one of the schemas represented by English over, as in The tapestry hung over the wall. Here, both the Figure and Ground must be planes and coplanar with each other. If the tapestry here were changed to something linear, say, a string of beads, it is no longer appropriate to use over but only something like against, as in The string of beads hung *over / against the wall. Now, another candidate element -- that the Figure must be rigid, like the board in the scene -- can be tested and again found to be inessential to the across schema, since a flexible linear object can be substituted for the board without any need to change the preposition, as seen in The board / The cable lay across the railway bed. Here, however, checking this candidate factor across numerous spatial schemas in many languages might well never yield a case in which it does figure as an essential element and so would be kept off the inventory. This methodology affords a kind of existence proof: it can demonstrate that some element does occur in the universally available inventory of structural spatial elements since it can be seen to occur in at least one closed-class spatial schema in at least one language. The procedure is repeated numerous times across many languages to build up a sizable inventory of elements essential to spatial schemas. The next step is to discern whether the uncovered elements comprise particular structural categories and, if so, to determine what these categories are. It can be observed that for certain sets of elements, the elements in a set are mutually incompatible -- only one of them can apply at a time at some point in a schema. Such sets are here taken to be basic spatial categories. Along with their members, such categories are also part of language's fundamental conceptual structuring system for space. A representative sample of these categories is presented next. It will be seen that These categories generally have a relatively small membership. This finding depends in part on the following methodological principles. An .element proposed for the inventory should be as coarse-grained as possible -- that is, no more specific than is warranted by cross-schema analysis. Correlatively, in establishing a category, care must be taken that it include only the most generic elements that have actually been determined -- that is, that its membership have no finer granularity than is warranted by the element-abstraction procedure. For example, the principle of mutual incompatibility yields a spatial category of "relative orientation" between two lines or planes, a category with perhaps only two member elements (both already seen in the across schema): approximately parallel and approximately perpendicular. Some evidence additionally suggests an intermediary "oblique" element as a third member of the category. Thus, some English speakers may distinguish a more perpendicular sense from a more oblique sense, respectively, for the two verb satellites out and o f , as in A secondary pipe branches out / o$from the main sewer line. In any case, though, the category would have no more than these two or three members. Although finer degrees of relative orientation can be distinguished by other cognitive systems, say, in visual perception and in motor control, the conceptual structuring subsystem of language does not include anything finer than the two- or three-way distinction. The procedures of schema analysis and cross-schema comparison, together with the methodological principles of maximum granularity for elements and for category membership, can lead to a determination of the number of structurally distinguished elements ever used in language for a spatial category. 2.3 Sample Categories and Their Member Elements

The fundamental categories of spatial structure in the closed-class subsystem of spoken language fall into three classes according to the aspect of a spatial scene they pertain to: the

How Spoken Language and Signed Language Structure Space Differently

251

segmentation of the scene into individual components, the properties of an individual component, and the relations of one such component to another. A sampling of categories and their member elements from each of these three classes is presented next. The examples provided here are primarily drawn from English but can be readily multiplied across a diverse range of languages (see Talmy 2000a, ch. 3). Categories Pertaining to Scene Segmentation. The class designated as scene segmentation may include only one category, that of "major components of a scene", and this category may contain only three member elements: the Figure, the Ground, and a secondary Reference Object. Figure and Ground were already seen for the across schema. Schema comparison shows the need to recognize a third scene component, the Secondary Reference Object. The English preposition near, as in The lamp is near the TV specifies the location of the Figure (the lamp) only with respect to the Ground (the TV). But localizing the Figure with the preposition above, as in The lamp is above the TV, requires knowledge not only of where the Ground object is, but also of the encompassive earth-based spatial grid, in particular, of its vertical orientation. Thus, above requires recognizing three components within a spatial scene, a Figure, a Ground, and a Secondary Reference Object. Categories Pertaining to an Individual Scene Component. A number of categories pertain to the characteristics of an individual spatial scene component. This is usually one of the three major components resulting from scene segmentation -- the Figure, Ground, or Secondary Reference Object -- but it could be others, such as the path line formed by a moving Figure. One such category is that of "dimension" with four member elements: zero dimensions for a point, one for a line, two for a plane, and three for a volume. Some English prepositions require a Ground object schematizable for only one of the four dimensional possibilities. Thus, the schema of the preposition near as in near the dot requires only that the Ground object be schematizable as a point. Along, as in along the trail, requires that the Ground object be linear. Over as in a tapestry over a wall requires a planar Ground. And throughout, as in cherries throughout the jello, requires a volumetric Ground. A second category is that of "number" with perhaps four members: one, two, several, and many. Some English prepositions require a Ground comprising objects in one or another of these numbers. Thus, near requires a Ground consisting of just one object, between of two objects, among of several objects, and amidst of numerous objects, as in The basketball lay near the boulder / between the boulders / among the boulders / amidst the cornstalks. The category of number appears to lack any further members -- that is, closed-class spatial schemas in languages around the world seem never to incorporate any other number specifications -- such as 'three' or 'even-numbered' or 'too many'. A third category is that of "motive state", with two members: motion and stationariness. Several English prepositions mark this distinction for the Figure. Thus, in one of its senses, at requires a stationary Figure, as in I stayed / *went at the library, while into requires a moving Figure, as in I went / *stayed into the library. Apparently no spatial schemas mark such additional distinctions as motion at a fast vs. slow rate, or being located at rest vs. remaining located fixedly. A fourth category is that of "state of boundedness" with two members: bounded and unbounded. The English preposition along requires that the path of a moving Figure be unbounded, as shown by its compatibility with a temporal phrase in for but not in, as in I walked along the pier for 10 minutes / *in 20 minutes. But the spatial locution the length of requires a bounded path, as in I walked the length of the pier in 20 Minutes / *for 10 minutes. While some spatial schemas have the bounded element at one end of a line and the unbounded element at the other end, apparently no spatial schema marks any distinctions other than the two cited states of boundedness, such as a cline of gradually increasing boundedness along a line. In addition to this sampling, some ten or so further categories pertaining to properties of an individual schema component, each category with a small number of fixed contrasts, can be readily identified.

252

L. Talmy

Categories Pertaining to the Relation of One Scene Component to Another. Another class of categories pertains to the relations that one scene component can bear to another. One such category was described earlier, that of "relative orientation", with two or three members: parallel, perpendicular, and perhaps oblique. A second such category is that of "degree of remove", of one scene component from another. This category appears to have four or five members, two with contact between the components -- coincidence and adjacency -- and two or three without contact -- proximal, perhaps medial, and distal remove. Some painvise contrasts in English reveal one or another of these member elements for a Figure relating to a Ground. Thus, the locution in the front of, as in The carousel is in the front of the fairground, expresses coincidence, since the carousel as Figure is represented as being located in a part of the fairground as Ground. But in front of (without a the) as in The carousel is in front of the fairground, indicates proximality, since the carousel is now located outside the fairground and near it but not touching it. The distinction between proximal and distal can be teased out by noting that in front of can only represent a proximal but not a distal degree of remove, as seen in that one can say The carousel is 20 feet in front of the fairground, but not, *The carousel is 20 miles in front of the fairground, whereas above allows both proximal and distal degrees of remove, as seen in The hawk is I foot / I mile above the table. The distinction between adjacency and proximality is shown by the prepositions on and over, as in Theji'y is on / over the table. Need for a fifth category member of 'medial degree of remove' might come from languages with a 'here / there / yonder' kind of distinction in their deictic adverbs or demonstratives. Some ten or so additional categories for relating one scene component to another, again each with its own small number of member contrasts, can be readily identified.

2.4 Properties of the Inventory By our methodology, the universally available inventory of structural spatial elements includes all elements that appear in at least one closed-class spatial schema in at least one language. These elements may indeed be equivalent in their sheer availability for use in schemas. But beyond that, they appear to differ in their frequency of occurrence across schemas and languages, ranging from very common to very rare. Accordingly, the inventory of elements -- and perhaps also that of categories -- may have the property of being hierarchical, with entries running from the most to the least frequent. Such a hierarchy suggests that the elements in the inventory, the categories in the inventory, and the elements in each category might not end at a sharp lower boundary but might trail off indefinitely.

2.5 Basic Elements Assembled into Whole Schemas The procedure so far has been analytic, starting with the whole spatial schemas expressed by closed-class forms and abstracting from them an inventory of fkdamental spatial elements. But the investigation must also include a synthetic procedure: examining the ways in which individual spatial elements are assembled to constitute whole schemas. Something of such an assembly was implicit in the initial discussion of the across schema. But an explicit example here can better illustrate this part of the investigation. Consider the schema represented by the English preposition past as in The ball sailed past my head at exactly 3 PM. This schema is built out of the following fundamental spatial elements (from the indicated categories) in the indicated arrangements and relationships. There are two main scene components (members of the "major scene components" category), a Figure and a Ground (here, the ball and my head, respectively). The Figure is schematiz-

How Spoken Language and Signed Language Structure Space Differently

253

point is moving (a member element of the "motive state" category). Hence it forms a onedimensional line (a member of the "dimension" category"). This line constitutes the Figure's "path". The Ground is also schematizable as a O-dimensional point (a member of the "dimension" category). There is a point P at a proximal remove (a member of the "degree of remove" category) from the Ground point, forming a l-dimensional line with it (a member of the "dimension" category). This line is parallel (a member of the "relative orientation" category) to the horizontal plane (a member of the "intrinsic parts" category) of the earthbased grid (a member of the major scene components" category). The Figure's path is perpendicular (a member of the "relative orientation" category) to this line. The Figure's path is also parallel to the horizontal plane of the earth-based grid. If the Ground object has a front, side, and back (members of the "intrinsic parts" category), then point P is proximal to the side part. A non-boundary point (a member of the "state of boundedness" category) of the Figure's path becomes coincident (a member of the "degree of remove" category) with point P at a certain point of time. The least understood aspect of the present investigation is what well-formedness conditions, if any, may govern the legality of such combinations. As yet, no obvious principles based, say, on geometric simplicity, symmetry, consistency, or the like are seen to control the patterns in which basic elements assemble into whole schemas. On the one hand, some seemingly byzantine combinations -- like the schemas seen above for across and past -occur with some regularity across languages. On the other hand, much simpler combinations seem never to occur as closed-class schemas. For example, one could imagine assembling elements into the following schema: down into a surround that is radially proximal to a center point. One could invent a preposition apit to represent this schema, as used in 1 poured water apit my house" to refer to my pouring water down into a nearby hole dug in the field around my house. But such schemas are not found. It could be argued that the recurrent schemas are simply the spatial structures most often encountered in everyday activity. But the different sets of spatial schemas found in different languages are diverse enough from each other that arguing on the basis of the determinative force of everyday experience is problematic. Something else is at work but it is not yet clear what that is.

2.6 Properties and Processes Applying to Whole Spatial Schemas It was just seen that selected elements of the inventory are combined in specific arrangements to make up the whole schemas represented by closed-class spatial forms. Each such whole schema is thus a "pre-packaged" bundling together of certain elements in a particular arrangement. Each language has in its lexicon a relatively closed set of such pre-packaged schemas. A speaker of the language must select among these schemas in depicting a spatial scene. We now observe that such schemas, though composite, have a certain unitary status in their own right, and that certain quite general properties and processes can apply to them. In particular, certain properties and processes allow a schema represented by a closed-class form to generalize to a whole family of schemas. In the case of a generalizing property, all the schemas of a family are of equal priority. On the other hand, a generalizingprocess acts on a schema that is somehow basic, and either extends or deforms it to yield nonbasic schemas. (see Talmy 2000a ch. 1 and 3, 2000b ch. 5). Such properties and processes are perhaps part of the overall spoken-language system so that any language's relatively closed set of spatial closed-class forms and the schemas that they basically represent can be used to match more spatial structures in a wider range of scenes. Looking first at generalizing properties of spatial schemas, one such property is that they exhibit a topological or topology-like neutrality to certain factors of Euclidean geometry. Thus, they are magnitude neutral, as seen in such facts as that the across schema can apply to a situation of any size, as in The ant crawled across my palm / The bus drove across the country. Further, they are largely shape-neutral, as seen by such facts as that, while the

254

L. Talmy

through schema requires that the Figure form a path with linear extent, it lets that line take any contour, as in I zig-zagged / circled through the woods. Thus, while holding to their specific constraints, schemas can vary freely in other respects and so cover a range of spatial configurations. Among the processes that extend schemas, one is that of "extendability in ungoverned dimensions". by this process, a scene component of dimensionality N in the basic form of a schema can generally be raised in dimensionality to form a line, plane, or volume aligned in a way not conflicting with the schema's other requirements. To illustrate, the English verb satellite out has a schema involving a point Figure moving along a radius away from a center point through a continuum of concentric circles, as in The boat sailed further and further out from the island. This schema with the Figure idealizable as a point is the basic form. But the same satellite can be used when this Figure point is extended to form a 1-dimensional line along a radius, as in The caravan of boats sailed further and further outfrom the island. And the out can again be used if the Figure point were instead extended as a 1-dimensional line forming a concentric circle, as in A circular ripple spread outfrom where the pebble fell into the water. In turn, such a concentric circle could be extended to fill in the interior plane, as in The oil spread out over the water from where it spilled. Alternatively, the concentric circle could have been extended in the vertical dimension to form a cylinder, as in A ring of fire spread out as an advancing wall of James. Or again, the circle could have been extended to form a spherical shell, as in The balloon I blew into slowly pufed out. And such a shell can be extended to fill in the interior volume, as in The leavened dough slowly pufed out. Among the processes that deform a schema, one is that of "feature cancellation", in which a particular complex of elements in the basic schema is omitted. Thus, the preposition across can be used in The shopping cart rolled across the boulevard and was hit by an oncoming car, even though one feature of the schema -- 'terminal point coincides with the distal edge of the Ground ribbon' -- is canceled from the Figure's path. Further, both this feature and the feature 'beginning point coincides with the proximal edge of the Ground ribbon' are canceled in The tumbleweed rolled across the prairie for an hour. Thus, the spoken language system includes a number of generalizing properties and processes that allow the otherwise relatively closed set of abstracted or basic schemas represented in the lexicon of any single language to be applicable to a much wider range of spatial configurations.

3 Spatial Structuring in Signed Language All the preceding findings on the linguistic structuring of space have been based on the patterns found in spoken languages. The inquiry into the fundamental concept structuring system of language leads naturally to investigating its character in another major body of linguistic realization, signed language. The value in extending the inquiry in this way would be to discover whether the spatial structuring system is the same or is different in certain respects across the two language modalities, with either discovery having major consequences for cognitive theory. In this research extension, a problematic issue is exactly what to compare between spoken and signed language. The two language systems appear to subdivide into somewhat different sets of subsystems. Thus, heuristically, the generalized spoken language system can be thought to consist of an open-class or lexical subsystem (generally representing conceptual content); a closed-class or grammatical subsystem (generally representing conceptual structure); a gradient subsystem of "vocal dynamics" (including loudness, pitch, timbre, rate, distinctness, unit separation); and an accompanying somatic subsystem (including facial expression, gesture, and "body language"). On the other hand, by one provisional proposal, the generalized sign language system might instead divide up into the following: a subsystem of

How Spoken Language and Signed Language Structure Space Differently

255

lexical forms (including noun, verb, and adjective signs); an "inflectional" subsystem (including modulations of lexical signs for person, aspect); a subsystem of size-and-shape specifiers (or SASS'S; a subsystem of so-called "classifier constructions"; a gestural subsystem (along a gradient of incorporation into the preceding subsystems); a subsystem of face, head, and torso representations; a gradient subsystem of "bodily dynamics" (including amplitude, rate, distinctness, unit separation); and an associated or overlaid somatic subsystem (including further facial expression and "body language"). In particular here, the subsystem of classifier constructions -- which is apparently present in all signed languages -- is a formally distinct subsystem dedicated solely to the schematic structural representation of objects moving or located with respect to each other in space (see Liddell forthcoming, Emmorey in press). The research program of comparing the representation of spatial structure across the two language modalities ultimately requires considering the two whole systems and all their subsystems. But the initial comparison -- the one adopted here -- should be between those portions of each system most directly involved with the representation of spatial structure. In spoken language, this is that part of the closed-class subsystem that represents spatial structure and, in signed language, it is the subsystem of classifier constructions. Spelled out, the shared properties that make this initial comparison apt include the following. First, of course, both subsystems represent objects relating to each other in space. Second, in terms of the functional distinction between "structure" and "content" described earlier, each of the subsystems is squarely on the structural side. Third, in each subsystem, a schematic structural form within an expression in general can be semantically elaborated by a content form that joins or replaces it within the same expression. To illustrate the classifier system, a spatial event that English could express as The car drove past the tree could be expressed in ASL as follows: The signer's dominant hand, used to represent the Figure object, here has a "3 handshape" (index and middle fingers extended forward, thumb up) to represent a land vehicle. The nondominant hand, used to represent the Ground object, here involves an upright "5 handshape" (forearm held upright with the five fingers extended upward and spread apart) to represent a tree. The dominant hand is moved horizontally across the signer's torso and past the nondominant forearm. Further though, this basic form could be modified or augmented to represent additional particulars of the referent spatial event. Thus, the dominant hand can show additional characteristics of the path. For example, the hand could move along a curved path to indicate that the road being followed was curved, it could slant upward to represent an uphill course, or both could be shown together. The dominant hand can additionally show the manner of the motion. For example, as it moves along, it could oscillate up and down to indicate a bumpy ride, or move quickly to indicate a swift pace, or both could be shown together, as well as with the preceding two path properties. And the dominant hand can show additional relationships of the Figure to the Ground. For example, it could pass nearer or farther from the nondominant hand to indicate the car's distance from the tree when passing it, it could make the approach toward the nondominant hand longer (or shorter) than the trailing portion of the path to represent the comparable relationship between the car's path and the tree, or it could show both of these together or, indeed, with all the preceding additional characteristics. The essential finding of how signed language differs from spoken language is that it more closely parallels what appear to be the structural characteristics of scene parsing in visual perception. This difference can be observed in two venues, the universally available spatial inventory and the spatial expression. These two venues are discussed next in turn. 3.1 In the Inventory The inventory of forms for representing spatial structure available to signed language has a greater total number of fundamental elements, a greater number of categories, and generally a

256

L. Talmy

greater number of elements per category than the spoken language inventory. More specifically, the classifier subsystem of signed language has many of the same spacestructuring categories as in the closed-class subsystem of spoken language, but it also has many categories not present there, whereas spoken language may have no categories that are absent from signed language. Comparing the membership of the corresponding categories in terms of discrete elements, the number of basic elements per category in signed language ranges from being the same as that for spoken language to being very much greater. Further, though, while the membership of some categories in signed language may well consist of discrete elements, that of others appears to be gradient. Here, any procedure of tallying some fixed number of discrete elements in a category must give way to determining the approximate fineness of distinctions that can be practicably made for that category. So while some corresponding categories across the two language modalities may otherwise be quite comparable, their memberships can be of two different types, discrete vs. analog. Altogether, then, given its greater number of categories, generally larger membership per category, and a frequently gradient type of membership, the inventory of forms for building a schematic spatial representation available to the classifier subsystem of signed language is more extensive and finer than for the closed-class subsystem of spoken language. This greater extensiveness and finer granularity of spatial distinctions seems more comparable to that of spatial parsing in visual perception. The following are some spatial categories in common across the two language modalities, but with increasing disparity in size of membership. First, some categories appear to be quite comparable across the two modalities. Thus, both the closed-class subsystem of spoken language and the classifier subsystem of signed language structurally segment a scene into the same three components, a Figure, a Ground, and a secondary Reference Object. Both subsystems represent the category of dimensionality with the same four members -- a point, a line, a plane, and a volume. And both mark the same two degrees of boundedness: bounded and unbounded. For certain categories, signed language has just a slightly greater membership than does spoken language. Thus, for motive state, signed language structurally represents not only moving and being located, but also remaining fixedly located -- a concept that spoken languages typically represent in verbs but not in their spatial preposition-like forms. For other spatial categories, signed language has a moderately greater membership than spoken language. In some of these categories, the membership is probably gradient, but without the capacity to represent many fine distinctions clearly. Thus, signed language can apparently mark moderately more degrees of remove than spoken language's four or five members in this category. It can also apparently distinguish moderately more path lengths than the two -- short and long -- that spoken language marks structurally (as in English The bugflew right / way up there). And while spoken language can mark at most three distinctions of relative orientation -- parallel, perpendicular, and oblique -- signed language can distinguish a moderately greater number, for example, in the elevation of a path's angle above the horizontal, or in the angle of the Figure's axes to that of the Ground (e.g. in the placement of a pole against a wall). Finally, there are some categories for which signed language has an indefinitely greater membership than spoken language. Thus, while spoken language structurally distinguishes some four path contours as seen in section 2.3.3, signed language can represent perhaps indefinitely many more, including zigzags, spirals, and ricochets. And for the category "locus within referent space", spoken language can structurally distinguish perhaps at most three loci relative to the speaker's location -- 'here', 'there', and 'yonder' -- whereas sign language can distinguish indefinitely many more within sign space.

How Spoken Language and Signed Language Structure Space Differently

257

Apart from membership differences across common categories, signed language represents some categories not found in spoken language. One such category is the relative lengths of a Figure's path before and after encounter with the Ground. Or again, signed language can represent not only the category of "degree of dispersion" (which spoken language can also represent), but also the category "pattern of distribution". Thus, in representing multiple Figure objects dispersed over a planar surface, it could in addition structurally indicate that these Figure objects are linear (as with dry spaghetti over a table) and are arrayed in parallel alignment, crisscrossing, or in a jumble. Overall, the additional structural spatial distinctions represented in signed language appear to be ones also regularly abstracted out in visual scene parsing and, if this can be demonstrated, would show a closer connection of signed than of spoken language to visual perception.

3.2 In the Expression The second venue, that of any single spatial expression, exhibits further respects in which signed language differs from spoken language in the apparent direction of visual scene parsing. Several of these are outlined next. Iconic Clustering of Elements / Categories in the Expression. The structural elements of a scene of motion are clustered together in the classifier subsystem's representation of them in signed language more as they seem to be clustered in perception. When one views a motion event, such as a car driving bumpily along a curve past a tree, it is perceptually the same single object, the car, that exhibits all of the following characteristics: it has certain object properties as a Figure, it moves, it has a manner of motion, it describes a path of a particular contour, and it relates to other surrounding objects (the Ground) in its path of motion. The Ground object or objects are perceived as separate. Correspondingly, the classifier subsystem maintains exactly this pattern of clustering. It is the same single hand, the dominant hand, that exhibits the Figure characteristics, motion, manner, path contour, and relations to a Ground object. The other hand, the nondominant, separately represents the Ground object. All spoken languages diverge to a greater or lesser extent from this visual fidelity. Thus, consider one English counterpart of the event, the sentence The car bumped along past the tree. Here, the subject nominal, the car, separately represents the Figure object. The verb bumped clusters together the representation of the fact of motion and the manner of motion, while its sister constituent, the satellite along represents the presence of a path of translational motion. The preposition past represents the path conformation, while its sister constituent, the nominal the tree, represents the Ground. Iconic Representation of ElementsJCategories in the Expression. The classifier subsystem of signed language appears to be iconic with visual parsing not only in its clustering of spatial elements and categories, as just seen, but largely also in its representation of them. For example, it marks one basic category opposition, that between an entity and its activity, by using an object like the hand to represent an object, and motion of the hand to represent motion of the object. More specifically, the hand or other body part represents a structural entity (such as the Figure) -- with the body part's configuration representing the identity or other properties of the entity -- while movements or positionings of the body part represent properties of the entity's motion, location, or orientation. For example, the hand could be held flat to represent a planar object (e.g. a sheet of paper), or curved to represent a cupshaped object. And, as seen, any such handshape as Figure could be moved along a variety of trajectories that represent particular path contours. But an alternative to this arrangement could be imagined. The handshape could represent the path of a Figure-- e.g., a fist to represent a stationary location, the outstretched fingers held flat together to represent a straight line path, the fingers in a curved plane for a curved path, and the fingers alternately

258

L. Talmy

forward and back for a zigzag path. Meanwhile, the hand movement could represent the Figure's shape -- e.g., the hand moving in a circle to represent a round Figure and in a straight line for a linear Figure. However, no such mapping of referents to their representations is found. Rather, the mapping in signed language is visually iconic: it assigns the representation of a material object in a scene to a material object in a classifier complex, for example, the hand, and the representation of the movements of that object in the scene to the movements of the hand. No such iconic correspondence is found in spoken language. Thus, while material objects are prototypically expressed by nouns in English, they are instead prototypically represented by verb roots in Atsugewi (see Talmy 2000b, ch. 1). And while path configurations are prototypically represented in Spanish by verbs, this is done by prepositions and satellites in English.

Many more elements / categories representable within a single expression. Perhaps the most striking difference between the signed and the spoken representation of space in the expression is that the classifier system in signed language permits the representation of a vastly greater number of distinct spatial categories simultaneously and independently. A spoken language like English can separately represent only up to four or five different spatial categories with closed-class forms in a single clause. As illustrated in the sentence The bat Jew way back up into its niche in the cavern, the verb is followed in turn by: a slot for indication of path length (with three members: "zero" for 'neutral', way for 'relatively long', right for 'relatively short'); a slot for state of return (with two members: "zero" for 'neutral', back for 'return'); a slot for displacement within the earth-frame (with four members: "zero" for 'neutral', up for 'positive vertical displacement', down for 'negative vertical displacement', over for 'horizontal displacement'); a slot for geometric conformation (with many members, including in, across, past); and perhaps a slot for motive state and vector (with two members: "zero" for 'neutral between location AT and motion TO' as seen in in /on, and -to for 'motion TO' as seen in into / onto). Even a polysynthetic language like Atsugewi has closed-class slots within a single clause for only up to six spatial categories: path conformation combined with Ground type, path length, vector, deixis, state of return, and cause or manner. In contrast, by one tentative count, ASL has provision for the separate indication of thirty different spatial categories. These categories do exhibit certain cooccurrence restrictions, they differ in obligatoriness or optionality, and it is unlikely -- perhaps impossible -for all thirty of them to be represented at once. Nevertheless, a sizable number of them can be represented in a single classifier expression and varied independently there. The table below lists the spatial categories that I have provisionally identified as available for concurrent independent representation. The guiding principle for positing a category has been that its elements are mutually exclusive: different elements in the same category cannot be represented together in the same classifier expression. If certain elements can be concurrently represented, they belong to different categories. Following this principle has, on the one hand, involved joining together what some sign language analyses have treated as separate factors. For example, the first category below covers equally the representation of Figure, instrument, or manipulator (handling classifier), since these three kinds of elements apparently cannot be separately represented in a single expression -- one or another of them must be selected. On the other hand, the principle requires making distinctions within some categories that spoken languages treat as uniform. Thus, the single "manner" category of English must be subdivided into a category of "divertive manner" (e.g. moving along with an up-down bump) and a category of "dynamic manner" (e.g. moving along rapidly) because these two factors can be represented concurrently and varied independently.

How Spoken Language and Signed Language Structure Space Differently

259

A. entity properties I. identity (form or semantic category) of Figure / instrument / manipulator 2. identity (form or semantic category) of Ground 3. magnitude of some major entity dimension 4. magnitude of a transverse dimension 5. number of entities B. orientation properties 6. an entity's rotatedness about its left-right axis ("pitch") 7. an entity's rotatedness about its front-back axis ("roll") 8. a. an entity's rotatedness about its top-bottom axis ("yaw") b. an entity's rotatedness relative to its path of forward motion C. locus properties 9. locus within sign space D. Motion properties 10. motive state (moving / resting / fixed) 11. internal motion (e.g. expansionlcontraction, form change, wriggle, swirling) 12. confined motion ( e.g. straight oscillation, rotary oscillation, rotation, local wander) 13. translational motion E. Path properties 14. state of continuity (unbroken / saltatory) 15. contour of path 16. state of boundedness (bounded / unbounded) 17. length of path 18. vertical height 19. horizontal distance from signer 20. left-right positioning 2 1. up-down angle ("elevation") 22. left-right angle ("direction") 23. transitions between motion and stationariness (e.g. normal, decelerated, abrupt as from impact) F. Manner properties 24. divertive manner 25. dynamic manner G. relations of Figure or Path to Ground 26. path's conformation relative to Ground 27. relative lengths of path before and after encounter with Ground 28. Figure's path relative to the Path of a moving Ground 29. Figure's proximity to Ground 30. Figure's orientation relative to Ground

It seems probable that something more on the order of this number of spatial categories is concurrently analyzed out by visual processing on viewing a scene than the much smaller number present in even the most extreme spoken language patterns. Elements / Categories Independently Variable in the Expression -- Not in Pre-Packaged Schemas. The signed-spoken language difference just presented was mainly considered for the sheer number of distinct spatial categories that can be represented together in a single classifier expression. Now, though, we stress the corollary: their independent variability. That is, apart from certain constraints involving cooccurrence and obligatoriness in a

260

L. Talmy

classifier expression, a signer can generally select a category for inclusion independently of other categories, and select a member element within each category independently of other selections. For example, a classifier expression can separately include and independently vary a path's contour, length, vertical angle, horizontal angle, speed, accompanying manner, and relation to Ground object. By contrast, it was seen earlier that spoken languages largely bundle together a choice of spatial member elements within a selection of spatial categories for representation within the single complex schema that is associated with a closed-class morpheme. The lexicon of each spoken language will have available a certain number of such "pre-packaged" spatial schemas, and the speaker must generally choose from among those to represent a spatial scene, even where the fit is not exact. The system of generalizing properties and processes seen in section 2.6 that apply to the set of basic schemas in the lexicon (including their plastic extension and deformation) may exist to compensate for the prepackaging and closed stock of the schemas in any spoken language. Thus, what are largely semantic components within a single morpheme in spoken language correspond to what can be considered separate individually controllable morphemes in the signed classifier expression. Classifier expressions', apparent general lack of pre-packaging, of a fixed set of discrete basic schemas, or of a system for generalizing, extending, or deforming such basic schemas may well accord with comparable characteristics of visual parsing. That is, the visual processing of a viewed scene may tend toward the independent assessment of spatial factors without much pre-packeting of associated factors or of their plastic alteration. If shown to be the case, then signed language will once again prove to be closer to perceptual spatial structuring than spoken language is.

4 Cognitive Implications of SpokedSigned Language Differences The preceding comparison of the space-structuring subsystems of spoken and of signed language has shown a number of respects in which these are similar and in which they are different. It can be theorized that their common characteristics are the product of a single neural system, what can be assumed to be the core language system, while each set of distinct characteristics results from the activity of some further distinct neural system. These ideas are outlined next. 4.1 Where Signed and Spoken Language are Alike We can first summarize and partly extend the properties above found to hold both in the closed-class subsystem of spoken language and in the classifier subsystem of signed language. Both subsystems can represent multifarious and subtly distinct spatial situations -that is, situations of objects moving or located with respect to each other in space. Both represent such spatial situations schematically and structurally. Both have basic elements that in combination make up the structural schematizations. Both group their basic elements within certain categories that themselves represent particular categories of spatial structure. Both have certain conditions on the combination of basic elements and categories into a full structural schematization. Both have conditions on the cooccurrence and sequencing of such schematizations within a larger spatial expression. Both permit semantic amplification of certain elements or parts of a schematization by open-class or lexical forms outside the schema. And in both subsystems, a spatial situation can often be conceptualized in more than one way, so that it is amenable to alternative schematizations.

4.2 Where Spoken and Signed Language Differ First, the two language modalities have been seen to divide up into somewhat different sets of subsystems without clear one-to-one matchups. Thus, the spatial portion of the spoken language closed-class subsystem and the classifier subsystem of signed language may not be

How Spoken Language and Signed Language Structure Space Differently

261

exactly corresponding counterparts, but only those parts of the two language modalities closest to each other in the representation of schematic spatial structure. Within this initial comparison, though, the classifier subsystem seems closer to the structural characteristics of visual parsing than the closed-class subsystem in the following ways: It has more basic elements, categories, and elements per category in its schematic representation of spatial structure. Its elements exhibit more iconicity with the visual in the pattern in which they are clustered in an expression, in their physical representation, in their progression through time, and in their gradient character. It can represent only a narrow temporal aperture in an expression (and only a narrow spatial aperture as well, though this difference from spoken language might not reflect visual fidelity). It can represent many more distinct elements and categories together in a single expression. It can more readily select categories and category elements independently for representation in an expression. And it avoids pre-packaged categoryelement combinations as well as generalizations of their range and processes for their extension or deformation. 4.3 A New Neural Model

In its strong reading, the Fodor-Chomsky model relevant here is of a complete inviolate language module in the brain, one that performs all and only the functions of language without influence from outside itself -- a specifically linguistic "organ". But the evidence assembled here challenges such a model. What has here been found is that two different linguistic systems, the spoken and the signed, both of them undeniably forms of human language, on the one hand share extensive similarities but -- crucially -- also exhibit substantial differences in structure and organization. A new neural model can be proposed that is sensitive to this finding. We can posit a "core" language system in the brain, more limited in scope than the Fodor-Chomsky module, that is responsible for the properties and performs the functions found to be in common across both the spoken and the signed modalities. In representing at least spatial structure, this core system would then further connect with two different outside brain systems responsible, respectively, for the properties and functions specific to each of the two language modalities. It would thus be the interaction of the core linguistic system with one of the outside systems that would underlie the full functioning of each of the two language modalities. The particular properties and functions that the core language system would provide would include all the spoken-signed language properties in section 4.1 specific to spatial representation, though presumably in a more generic form. Thus, the core language system might have provision for: associating individual concepts with overt physical representations, whether vocal or manual; recombining individual concepts in accordance with certain constraints into conceptual complexes and setting these in correspondence with particular sequential complexes of the physical representations (i.e. the basis for morphosyntax); and designating the schematic structure of a conceptual complex and representing it with a subpart of the physical complex. When in use for signed language, this core language system might then further connect with particular parts of the neural system for visual perception. I have previously called attention to the already great overlap of structural properties between spoken language and visual perception (see Talmy 2000a, ch. 2), which might speak to some neural connection already in place between the core language system and the visual system. Accordingly, the proposal here is that in the case of signed language, still further connections are brought into play, ones that might underlie the finer granularity, iconicity, gradience, and aperture limitations we have seen in signed spatial representations. When in use for spoken language, the core language system might further connect with a putative neural system responsible for some of the characteristics present in spoken spatial

262

L. Talmy

representations but absent from signed ones. These could include the packeting of spatial elements into a stable closed set of patterned combinations, and a system for generalizing, extending, and deforming the packets. it is not clear why such a further system might otherwise exist but, very speculatively, one might look to see if any comparable operations hold, say, for the maintenance and modification of motor patterns. The present proposal of a more limited core language system connecting with outlying subsystems for full language function seems more consonant with contemporary neuroscientific findings that relatively smaller neural assemblies link up in larger combinations in the subservance of any particular cognitive function. In turn, the proposed core language system might itself be found to consist of an association and interaction of still smaller units of neural organization, many of which might in turn participate in subserving more than just language functions. References

Bennett, David C. 1975. Spatial and temporal uses of English prepositions: An essay in stratijicational semantics. London: Longrnan. Clark, Herb. 1973. Space, time, semantics, and the child. In Timothy E. Moore (ed.) Cognitive development and the acquisition of language. New York: Academic Press. Emmorey, Karen. In Press. Language, cognition and the brain: Insights from sign language research. Lawrence Erlbaum. Fillmore, Charles. 1968. The case for case. In Emmon Bach & Robert T. Harms (eds.) Universals in linguistic theory. New York: Holt, Rinehart and Winston. Gruber, Jeffrey S. 1965. Studies in lexical relations. PhD dissertation, MIT. Reprinted as part of Lexical structures in syntax and semantics, 1976. Amsterdam: North-Holland. Herskovits, Annette. 1982. Space and the prepositions in English: Regularities and irregularities in a complex domain. PhD dissertation, Stanford University. Jackendoff, Ray. 1983. Semantics and cognition. Cambridge, MA: MIT Press. Leech, Geoffrey. 1969. Towards a semantic description of English. New York: Longrnan Press, 1969. Liddell, Scott. Forthcoming. Sources of meaning in ASL classifier predicates. In Karen Emmorey (ed.) Perspectives on classzjier constructions in signed languages (provisional title). Likely publisher: Cambridge University Press Talmy, Leonard. 1983. How language structures space. In Herbert L. Pick, Jr. & Linda P. Acredolo (eds.) Spatial orientation: Theory, research, and application. New York: Plenum Press. -----. 2000a. Toward a cognitive semantics, volume I: Concept structuring systems. Cambridge, MA: MIT Press. -----. 2000b. Toward a cognitive semantics, volume 11: Typology and process in concept structuring. Cambridge, MA: MIT Press. -----. Forthcoming. Spatial Structuring in Spoken and Signed Language. In Proceedings of the Berkeley Linguistics Society, 2001 Zubin, David & Soteria Svorou. 1984. Orientation and gestalt: conceptual organizing principles in the lexicalization of space. With S. Choi. In David Testen, Veena Mishra & Joseph Drogo. Lexical semantics. Chicago: Chicago Linguistic Society.

Two Path Prepositions: Along and Past Christian Krayl, Jorg Baus3, Hubert Zimmer2, Harry Speiser2, and Antonio Kriiger3 kray0dfki.de German Research Center for Artificial Intelligence (DFKI) {huzimmer ,h. speiser)0mx. uni-sb .de Saarland University, Dept. of Psychology {baus , krueger)@cs .uni-sb .de Saarland University, Dept. of Computer Science

Abstract. We present results from a series of experiments, where relevant factors for the use of path prepositions were examined. We were especially interested in the concepts behind the German prepositions "entlang" and "vorbei" (similar to "along" and "past"). After exploring the basic properties human beings attribute to these prepositions, we systematically varied those properties to investigate their impact on the selection process and the corresponding speech production latency. The results indicate that parallelism and distance between the outline of a reference object and a trajectory are key concepts in this context.

Key Words. communication of spatial information, spatial reasoning, empirical studies, path prepositions

1

Motivation

Although path prepositions such as along, across, past are often part of path descriptions or navigational instruct ions, only limited effort has been put into investigating their properties and into modeling them (see [4], [9], [I], [8]). Most research emphasizes the importance of turns and relations/prepositions such as distance-dependent (e. g., "close to"), directional (e. g., "left of' ), and topological relations (e. g., "in"). This imbalance is illustrated by the wide range of publications from different research communities on those relations (see, for example, [71, [31, (21, [Id], [GI, [lo]' [Ill, [161). This is unfortunate as path prepositions offer some unique means for route descriptions. On one hand, a lot of information can be conveyed using a single path relation [13]. Consider, for example, a path following the shape of a D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 263−277, 2001.  Springer-Verlag Berlin Heidelberg 2001

264

C. Kray et al.

river. Describing it can be achieved by the simple use of "along", while otherwise a sequence of instructions would have t o be generated. On the other hand, path prepositions relate to the shape of an object, whereas distance-dependent, and directional prepositions do not. Neither do topological relations, as they establish connections between sets. Therefore, path prepositions can enrich route instructions by introducing shape, and they can also contribute to reducing the complexity of route instructions. A first model for the computation of path relations and prepositions was introduced by Blocher et al. [I]. The approach was based on the determination of a conceptual trajectory, that was extracted by means of abstraction from either a static path, a motion trace, or a tight group of objects. The authors focus on the abstraction process and describe the procedure of computing a path relation only informally. A categorization of path relations into source oriented, goaloriented, sub-path locating and path locating relations is presented. Contextual factors (object sizes, speed, field of visual attent ion, communicative situation) are mentioned as being influential in the selection process of path relations. They then give a description of the computation for "along", which first calls for the identification of (a) suitable reference object(s) and/or clustering of several objects. This object (or cluster of objects) is then abstracted and a conceptual trajectory is extracted from its boundary. This trajectory is compared with the path to be described, and the "similarity" of both trajectories and their closeness determine the applicability of the relation. A second model was proposed by Kray and Blocher [8], where they introduced the notion of basic path relation along the lines of Gapp's work on spatial relations [ 5 ] . They defined six basic path relations, each modeling a change (or lack of change) of either distance or angle. As they were primarily concerned with identifying the basic meanings of path relations, they focussed on the analysis of simple straight lines. Subsequently, they extended the model to the more general case of arbitrary shaped poly-lines. They also tried to identify fundamental concepts underlying some common (German) path prepositions, and map them onto basic path relations. Nevertheless, they cautioned that there is no 1:lcorrespondence between path relations and prepositions, and that contextual factors need to be taken into account. However, a systematic empirical study on the relevant factors for path prepositions is still lacking. We present such a study on the German path prepositions "entlang" and "vorbei", which can roughly be translated to the English prepositions "along" and "past" We selected those two candidates for several reasons: Firstly, both approaches predict that "along" requires parallelism between the path and the outline of the reference object, which allows for a direct verification. Secondly, "along" is frequently used, especially in urban environments, e. g., when giving instructions t o follow a specific road [13].Thirdly, "past" is not rigidly specified in the formal models and we wanted to investigate its fundamental characteristics. Finally, these two path prepositions can also function

'.

We will use these rough translations instead of the German prepositions throughout the paper to facilitate reading.

Two Path Prepositions: Along and Past

265

as the opposite of each other, and me were interested in whether this applies always or just in some specific cases. It has to be mentioned that the linepistic properties of the terms used in this study were not the focus of our research. Therefore, we did not explicitly analyze, whether a term was used, e. g., as a preposition or an adverb. Throughout the paper, we denote all expressions, which describe a path relation in natural language, as 'path prepositions' (although they might not be used as prepositions in all cases). The term 'path relations' is used to identify semantical or geometrical relations between a number of path-like objects. However, Di Meola [12] has presented a linguistic analysis of the use of the German adposition "entlang" (along), where he he distin-&hed two components of its use: a PATHGOAL-scheme, which refers to a change of locations while moving from a point of departure to a destination, and a LINK-scheme, which establishes a (static) relation between the path and a reference object (RO). He further suggested that the distance between the trajectory and the RO should be small and more or less constant in order to allow for the use of "entlang". In emphasizing the importance of parallelism and closeness of a trajectory to the RO, Di kfeola's suggestions resemble ours in their geometrical aspects. However, unlike Di Meola, we grade the importance of parallelism and closeness: we assume that it is more important that a trajectory is parallel t o the RO than that it is close as long as the distance is small enough for perceiving the trajectory as being influenced by the RO. The remainder of this paper describes the empirical study and its e-xperiments in detail. In Section 2, we present a paper and pencil study, where subjects were asked to produce "ideal" trajectories for given path prepositions. The results from this experiment suggest that parallelism and closeness are important concepts, which we varied in the subsequent experiments. In Section 3 we first verified the importance of these concepts before systematically deviating from parallelism while manipulating closeness. The results from these experiments are discussed in Section 4. Section 5 sums up the paper and gives an outlook on future research.

2

Production of trajectories

The first experiment in the series was a paper and pencil study, where subjects were asked to produce "ideal" or "prototypical" tkajectories for given German path prepositions.

2.1

Method

Subjects Exactly 28 students of the Saarland University took part in the experiment. All subjects were native speakers, and were not paid for their participation.

266

C. Kray et al.

Material The experiment was designed as a paper and pencil test. Each item consisted of a reference object, a start point, an end point and a literal route description, e. g., "Gehe entlang des Gebaudes" (Go along the building). The subjects were also presented with a rating scale2, on which the subjects could mark how easy (or difficult) they found the task of drawing the trajectory between start and end point. Each item was printed on a DIN A4-sized sheet of paper. At the top there was the written description, followed by a drawn frame sized 16 cm x 20 cm and the rating scale at the bottom of the sheet. The RO, the start point, and the end point were displayed within this frame. We will report the results for two different ROs: The first one was a rectangle sized 2 cm x 8 cm, the second one consisted of two rectangles (sized 2 cm x 8 cm, and 3 cm x 2 cm). These rectangles were arranged in such a way that they formed an "L-shaped object standing on its head" (cf. Figure 1).In the case of the plain rectangle, the start point was located 6 cm t o the right and 4 cm in front (below) of the lower right corner of the RO. The end point was 6 cm to the right and 3 cm behind (above) the RO's upper right corner. In case of the L-shaped object the start point was 2.5 cm from the right and 4 cm in front of the lower right corner, the end point 3 cm the the right and 3 cm behind the upper right corner. Along with each of these two items we gave one of the following two descriptions: "Go along the building" or "Go past the building". The four resulting items of our interest have been tested in conjunction with other items. These other items differed in the shape of the RO and in the accompanying literal descriptions, e. g., "Go along the river" or "Go around the tower". Altogether, we designed 36 different items, each on separate sheet of paper. These 36 sheets were randomly shuffled and combined with a literal instruction for the experiment Procedure The subjects were tested in two groups at the beginning of two lectures on computer science. Every subject received the 36 different items and the instructions for the experiment. They were told to read the instructions carefully and to wait for the start signal. After the signal was given, they had to draw what they thought is the best matching trajectory between start and end point for each of the given combinations of RO and description, and then to judge the difficulty of the task.

2.2

Results

The subjects' drawings were then digitized and pr6pared with image processing met hods. We implemented a custom software system to compute critical parameters, which characterize the course of the trajectories in three regions: FA, the area in front of the RO; NA, the one next t o the RO, and BA, the area behind the RO (cf. Figure 2). We calculated the distance of the trajectory t to the RO in discrete steps, and interpolated the area between the trajectory and RO in the regions of interest. In the context of the questions investigated in this paper, region NA is most relevant: If parallelism and proximity are significant in the We will not report the ratings that were given by the subjects in this paper.

Two Path Prepositions: Along and Past

(a) "Go along the building"

(b) "Go past the building"

(c) "Go along the building"

(d) "Go past the building"

267

Fig. 1. The four cases of interest: Each picture shows a superimposition of the trajectories produced by the participants.

268

C. Kray et al.

e 'behind"

rg

"next-to"

b

Fig. 2. Schematic description of the regions used for the analysis

case of "along', the prototypical trajectory drawn t o characterize "along" should be closer to the RO than the one drawn in the case of "past", and its distance to the RO within NA should only vary minimally. This is what we did indeed observe. On the average, the entrance point of the "along7'-trajectory into NA (18 mm) was more proximal to the RO than the one for "past" (49 mm), t(27) = 2 . 4 , ~< .05. The same is true for the exit point (17 mm vs. 49 mm), and also holds for the average distance within NA (13 mm vs. 47 mm). Finally, within NA there was less variance in the case of "along" (5.02) than in the case of"pastn (7.14), t(27) = 2 . 3 9 , ~< .05. Obviously, subjects moved closer to the RO with "along" in mind than with "past''. They kept a constant distance relative to RO in both cases. A similar pattern was observed for the L-shaped RO (cf. Figure 1). However, in this case it became apparent that the parallel course of the trajectory for "past" in the previous example was accidental: While the subjects still moved closer t o the RO in the case of "along" (12 mm) than in the case of "past" (35 mm), t(26) = 9 . 2 7 , ~< .001, they only followed the shape of RO in the case of "along". (A more detailed description of the results, including the other conditions that were realized in this experiment can be found in (151.) The results from this experiment suggest that parallelism and proximity are important concepts for the discrimination of the two German path relations "ent lang" (along) and "vorbei" (past). To verify this hypothesis we designed a speech product ion experiment, where we systematically varied the

Two Path Prepositions: Along and Past

269

shape of the RO, and the shape/curvature of the trajectory between the start and the end point.

3

Production of prepositions

Following the discussion of the previous experiment, we prepared a speech production experiment, where we investigated how the concepts of parallelism and closeness influence the selection of one of the two path prepositions "along" and "past" .

3.1

Method

Subjects Sixteen students of the Saarland University took part in the experiment. All subjects were native German speakers and were paid for their participat ion. Material Each layout consisted of a n RO, a start point, an end point, and a trajectory connecting both points. We designed three different reference objects: a simple rectangle (2 x 8 cm), a rectangle of the same size, but tilted 20 degrees to the left, and another rectangle, which was bent in the middle to form a 160 degree angle. The start points for the trajectories were located 3 cm in front of (below) the R07s lower right corner and 2 cm respectively 6 cm to right of it (cf. Figure 3). The corresponding end points were always located 3 cm behind (above) and 9 cm to the right of the RO's upper right corner. Trajectories were drawn as lines of 1.5 mm width. For all items there was a mirrored counterpart with start and end points on the left side of the RO. Thus, 24 different layouts were developed. They differed in the kind of reference objects, in the location of the trajectory start point, and in the trajectory's shape/curvature. Systematically varying these variables should reveal the importance of the aforementioned concepts of parallelism and closeness t o the discrimination of the two path prepositions. The items were displayed on a 17 inch computer screen, with subjects seated one meter in front of the screen. The experiments were controlled by an IBM compatible PC running a Java 3D application, that was specifially built for the trials. Procedure Subjects were seated in front of the computer screen. Each trial had the following structure: A short warning signal (a beep) was given. One second later, the subjects saw one of the items. They had been instructed to describe aloud and as fast as possible the curvature of the trajectory in relation to the reference object. Subjects were only allowed to use one of German path prepositions "entlang7' (along) and "vorbei" (past). The subjects7 speech production triggered a voice key, which in turn caused the item to disappear from the screen, and the subjects's choice t o be recorded. After a break of one second, the next trial was started. Times were measured between the beginning of the presentation of an item and the beginning of the spoken response by the subject.

270

C. Kray et al.

Fig. 3. A first look a t the experimental layout

3.2

Results

The frequencies of the path prepositions used and speech production latencies were analyzed in order to obtain the results presented in the subsequent sections. Parallelism and Closeness In order t o more closely investigate the importance of these two concepts, we will report the results for the following two layouts of items: In case A, the RO was a rectangle, and we designed three different trajectories. Two trajectories were parallel t o the shape of the RO. Fkom the two parallel ones, trajectory tl was closer to the RO than trajectory t2 . The third trajectory t4 violated the concept of parallelism. In case B, the RO was a tilted rectangle. Again, we had two parallel trajectories t l and t2, where the first one was closer to the RO than the second one, and a third trajectory t5, which violated the concept of parallelism. The different layouts are shown in Figure 43. Table 1. Percentages of subjects producing "along"

I

I

Parallel Distance Case A Case B

Close 87.5 89.9

1

Far

1

77.3 87.5

Nonparallel

I

10.9 3.9

The average frequencies of selecting "along" are reported in Table 1. A 2 x 3 analysis of variance of this data with the factors 'type of item' (Case A or B) and Not all trajectories that were used in the experiment are shown in the picture.

Two Path Prepositions: Along and Past

(a) Case A: RO and the trajectories t l , t2, and t#

271

(b) Case B: RO and the trajectories t l , t2, and t5

Fig. 4. Item sets in the preposition production condition

Table 2. Latencies of subjects producing "along" for parallel trajectories (in ms)

D jstance Case A Case B

Close 744 709

Far 790 748

'course of trajectory' (parallel and close, parallel and distant, and nonparallel) yielded a significant effect for the trajectory's course, F(2,30) = 4 7 . 1 6 , ~< .001. Post hoc comparisons showed that the frequencies were the same for the two parallel cases, and that they were much higher than for the nonparallel case. The production latencies (cf. Table 2) were compared for the same two factors in a 2 x 2 analysis; the latencies for 'along' with a nonparallel trajectory were excluded because there was an insufficient amount of data for this case. This analysis also yielded a significant effect for the course of the trajectory, F(1,13) = 8 . 4 2 , ~< .05, which demonstrates that subjects produced 'Lalong" faster when describing the closer trajectory than in the case of the more distant one. From these results we can conclude that parallelism to the shape of RO is necessary precondition if a trajectory is t o be described using "along". We can also infer that closeness has only a weak effect on selection, but the production latencies are slightly shorter for trajectories closer t o the RO.

Deviation from Parallelism A further set of items consisted of those layouts, where the displayed trajectories differ in their deviation from a given RO's shape.

272

C. Kray et al.

Fig. 5. Case B: RO and trajectories t3, t4 and t5

The tilted rectangle serves as RO. The three trajectories t3, t4, t5 differ in their deviation from RO (cf. Figure 5). All these trajectories violate the concept of parallelism to the RO. Trajectories t3, t4 are partially parallel to each other, but not to the RO; t3 is closer to RO than t4; trajectory t5 leads away from the RO. Table 3 shows the frequencies and speech production latencies for the selection of "past" for the aforementioned combinat ions. In case of the nonparallel trajectories "past" was more frequently used than "along". We therefore analyzed the frequencies and the production latencies for "past" - those for "along" were 1 - f (past) - in a one-way analysis with three nonparallel ones were not (both 6 %), F(4,60) = 75.80, p < .001. We then investigated trajectories, which were only partially parallel to the RO. Again, we manipulated the distances of the trajectories, and also the course of the path in the nonparallel part of it. In the region formerly denoted as NA, half of the path was parallel to the RO, while the other one was not. The latter part was either passing or departing (cf. Figure 7). The production frequencies of "along" were counted. These frequencies are shown in Table 4 depending on the courses of the trajectories. Table 4. Percentages of subjects producing "along" for trajectories in Fig.7

I

Parallel then straight Parallel then departing

Start point far Distance Far Close 47.7 61.7 52.3

1 I

43.0

I

Start point close Trajectory Far 47.7 40.6

I

I

Two Path Prepositions: Along and Past

(a) Case C: RO and trajectories t2, t4, t5 and t6

273

(b) Case D: RO and trajectories t3, and t.4

Fig. 7. Partially parallel trajectories

We first compared these frequencies in a 2 x 3 analysis of variance with the factors 'degree of parallelism' (2) and 'distance' (3). In this analysis, only the degree of paraIleIism was significant. F(2,30) = 4.49, p < .05. "Along" was produced more often to describe the proximal trajectories (57%) than to describe one of the two more distant trajectories (46% and 45%). No other effect was significant. The latencies could not be analyzed, since too few data points remained (due to the case-wise deletion of subjects in the repeated measurement analysis). Next, we combined these partially parallel conditions and compared them with the averages of the other conditions, where trajectories were either completely parallel or nonparallel. We analyzed the data in a one-way analysis with the following levels: (1) completely parallel, (2) partially parallel and passing, (3) partially parallel and departing, and (4) nonparallel. The corresponding frequencies were 89%, 52%, 54%, and 6%' of which the difference was highly si,@icant, F(3,45) = 26.91, p < .001. The parallel trajectories were more often described as "along" than the partially parallel ones, which were not different, but these frequencies were still higher than those for the nonparallel trajectories. The latter ones were nearly exclusively described using "past".

4

Discussion

There are two main results to be drawn from the series of experiments we conducted. On one hand, it has become clear that parallelism between a trajectory and the outline of a reference object is a necessary precondition for the applicability of "along". In the path production experiment, the subjects took great

274

C. Kray et al.

detours in order to assure that their trajectory was at least partially parallel to the reference object. The subsequent trials, where subjects were asked to describe a trajectory by one of the path prepositions, supported this thesis. On the other hand, the effect of distance was not entirely clear. While closeness did yield faster response times in case of parallel trajectories, there were also trials where closeness induced a higher percentage of subjects choosing %long" in case of partially parallel trajectories. This implies that closeness is a secondary criterion that is called upon in cases where the degree of parallelism is not high enough to justify the selection of "along". However, the comparison of two specific items from the path production experiment indicates that parallelism is not sufficient in order to select "along". In Figure 1 these two items are shown: the subjects were given the description "Go along the building" in case (a) and "Go pa? the building" in case (b). Obviously, both groups of superimposed trajectories are parallel to the outline, yet trajectories in (b) were produced t o depict 'bpast". The reason for this result may well be that parallelism is the product of coincidence in case (b): the most direct route from source to target is a straight line that happens to be parallel to the reference object. This accidental parallelism may pose a problem to the computational modeling of "along" as its applicability seems to depend also on potential alternate routes. If this is the case, the direct mapping of a high degree of parallelism to this path preposition (which both models apply to some degree) may yield wrong results.

I Fig. 8. Distance threshold

Another observation that may require the adjustment of the computational models concerns the degree of closeness: while parallel trajectories closer to the reference object yielded faster response times (at a similar selection rate) than trajectories that were farther away, there seems t o be a threshold distance. Once a trajectory is farther away than that, "along' is almost never chosen. In Figure 8 both trajectories are equally parallel t o the reference object, yet "along" is selected by 77% for t2, but by only 43% for t3. The determination of the threshold value and relevant factors that influence it are subject of further research. Throughout the different trials, "past" seemed to be the less specific

Two Path Prepositions: Along and Past

275

case. "Past" was only consistently chosen, when the trajectory led straight from source to target (ignoring the shape of the reference object), or when it led away from the reference object. Otherwise, there was no clear trend as to when "past" was preferred over "along". These observations can be interpreted in several ways. On one hand, "past" may have a less specific meaning, whereas "along" is defined more crisply. Therefore, "past" is only chosen when the more crisp case does not apply. However, there is evidence against this interpretation as production latencies were similar in cases, where both prepositions were applicable (not reported in this paper). On the other hand, there may be inconsistent perceptions of what "past" indicates. In Figure l(b), most subjects drew a straight line from source to target to depict "past". However, quite a few drew lines closer to the building, which was the most frequent behavior in the "along" condition. Finally, "past" may be the default relation subjects use, when they just want the establish a relation between the trajectory and the RO, and are not willing (or unable) to specify it in more detail (e. g. when they intend t o rule out a competing RO.) F'rom a different perspective, one may argue that the applicability of "along7' depends on the intention of the producer. By using it instead of the less specific case of "past", a pragmatic goal is achieved such as making sure that the listener gets to see a certain sight, or does not get lost? This argument can also explain the effects of distance that we observed: once a threshold distance is passed, the intention behind the use of "along" can no longer be fulfilled. This may also be the reason why subjects went to great detours in order to approach the RO when drawing trajectories for "along" (cf. Fig. l(a)). Since neither of the two models presented in Section 1 currently incorporates a concept of intention, it may be worthwhile to extend them in order t o accommodate such a concept. However, modeling intentions is quite complicated, as it would require the inclusion of, e. g., an explicit user model, a dialog history, etc.

5

Conclusion

We presented a series of experiments that were aimed at investigating the relevant factors for the selection of the German path prepositions "entlangl and "vorbei" (corresponding to "along" and -'past7'). In the paper and pencil trials, where subjects were asked to draw trajectories according to given prepositions, we identified the concept of parallelism and closeness as being of importance in this context. In the speech production experiments subjects were asked to describe a given trajectory using one of the two path prepositions. The results indicate that parallelism is a necessary precondition for the use of "along', while closeness mainly influences the production latency. However, there were some problematic cases such as accidental parallelism and trajectories that are very E. g., when the area is crowded and the object to walk along allows the listener to constantly reassure that he is on the right way.

276

C. Kray et al.

far away from the reference object but still parallel. These cases indicate that further research is needed in order t o completely model the path prepositions in question, and that current models need t o be refined. In the future, we plan to follow several research tracks to deepen our understanding of path prepositions. On one hand, there are currently several new experiments underway, where we investigate how dynamic trajectories will influence the selection of prepositions. Addit ionally, we are planning further trials on the impact of the presentation medium (verbal descriptions, maps. virtual walk-t hroughs) . Further more, we want t o examine the distance effect more t horoughly. On the other hand, we are in the process of finishing a prototypical implementation of a mobile system for navigational assistance, which incorporates a computational model of path relations. Once we dispose of a working system, we would like to conduct some field tests with 'real people in the real world' in order to determine the appropriateness of the underlying model.

Acknowledgements The research reported in this paper was funded by the Deutsche Forschungsgemeinschaft (DFG) in the context of the 'Collaborative Research Center 378: Resource-adaptive cognitive processes', and by the Klaus Tschira Foundation (KTS) through the projects 'MapTalk' and 'SISTO '.

References [I] A. Blocher, F. Essig, A. Kriiger, and W. hlaai3. Towards a computational semantics of path relations. In P. Olivier, editor, Spatial language: cognitive and computational perspectives. Kluwer Academic Publishers, to appear. (21 A. G. Cohn. Calculi for qualitative spatial reasoning. In J. Calmet, J. A. Campbell, and J. Pfalzgraf, editors, -4rtifical Intelligence and Symbolic Mathematical Computation (LNCS 1138), pages 124-143. Springer, Berlin, 1996. [3] M. J. Egenhofer and J. R. Herring. A mathematical framework for the definition of topological relations. In K. Brassel and H. Kishimoto, editors, Fourth International Symposium on Spatial Data Hadling, Zurich, Switzerland, 1990. [4] A. Rank and M. Raubal. Formal specifications of image schemata - a step to interoperability in geographic information systems. Spatial Cognition and Cornputation 1, pages 67-101, 1999. [S] K.-P. Gapp. Basic meanings of spatial relations: Computation and evaluation in 3d space. In Proceedings of AAAI-94, Seattle, WA, 1994.

[6] W. G. Hayward and M. J. Tam. Spatial language and spatial representation. Cognition, 55, pages 39-84, 1995. [7]A. Herskovits. Language and Spatial Cognition - A n Interdisciplinary Study of the Prepositions i n English. Cambridge University Press, Cambridge, UK, 1986. [8] C. Kray and A. Blocher. Modeling the basic meanings of path relations. In Proceedings of the 16th IJCAI. Morgan Kaufmann, Sun Francisco, CA, pages 384-389, 1999. [9] A. Kriiger and W. Maai3. Towards a computational semantics of path relations. In Workshop on Language and space at the 14th National Conference on Artijical Intelligence ( AA A I 97),1997.

Two Path Prepositions: Along and Past

277

[lo] G. D. Logan. Linguistic and conceptual control of visual spatial attention. Cognitive Psychology, 28, pages 103-174, 1995. [ll] G. D. Logan and D. D. Sadler. A computational analysis of the apprehension of spatial relations. In P. Bloom, M. A. Peterson, L. Nadel, and M. F. Garrett, editors, Language and space. Language, speech, and communication. MIT Press, Cambridge, MS, 1996. [12] C. D. Meola. Semantisch relevante und semantisch irrelevante Kasusalternation am Beispiel von entlang. Zeitschrift fir Spmchutissenschaft 17.2, pages 204-235, 1999. [13] B. Tversky and P. U. Lee. How space structures language. In C. F'reksa, C. Habel, and K. F. Wender, editors, Spatial Cognition. Springer, Berlin, 1998. [14] B. Tversky and P. U. Lee. Pictorial and verbal tools for conveying routes. In Spatial Information Theory (Proceedings of C O S I T 99). Springer, Berlin, pages 51-64, 1999. [IS] H. Zimmer, H. Speiser, 3. Baus, and A. Kriiger. dritical features for the selection of verbal descriptions for path relations (in press). Cognitive Processing, 2001. [16] H. D. Zimmer, H. R. Speiser, J. Baus, A. Blocher, and E. Stopp. The tfse of locative expressions in dependence of the spatial reIation between target and reference object in two-dimensional layouts. In C. Freksa, C. Habel, and K. F. Wender, editors, Spatial cognition. A n interdisciplinary approach to representing and processing spatial knowledge. Springer, Berlin, 1998.

Ambiguity in Acquiring Spatial Representation from Descriptions Compared to Depictions: The Role of Spatial Orientation Holly A. Taylor1, David H. Uttal2, Joan Fisher2, and Marshall Mazepa2 1

2

Tufts University, Department of Psychology, Paige Hall, Medford, Massachusetts, USA 02155 [email protected] Northwestern University, Department of Psychology, 2029 Sheridan Road, Evanston, IL, USA 60208-2170 {duttal, joanfisher, m-mazepa}@northwestern.edu

Abstract. Adults can make judgments about multiple spatial relations based on information gained from different kinds of input, including maps, descriptions, and through navigation [1]. However, factors such as spatial orientation influence performance. We investigated spatial orientation effects on learning from different media. In Experiment 1, participants learned a house from a map or a description. They then judged surrounding locations while imagining being in each room and they reconstructed the house. Participants who learned from a description performed worse on both tasks. Errors suggested they interpreted the term “in front” differently than intended [2]. Experiment 2 tested this hypothesis by examining two factors influencing interpretation of "in front”, specific interpretation instructions and orientation information. The orientation information influenced performance more than the explicit interpretation of “in front.” Taken together, the results indicate multiple influences on the spatial reference frame participants use to interpret spatial terms.

Key words: spatial descriptions, reference frame, spatial orientation, map use, spatial judgments, perspective taking

1

Introduction

People can acquire spatial information about an environment through different sources, including navigation, maps, and verbal descriptions. Navigation provides direct experience with the environment. Acquisition from maps or through descriptions is secondary, since others must provide this information [3]. How might mental representations differ when derived from these two different secondary sources? Research has addressed differences between direct experience D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 278−291, 2001.  Springer-Verlag Berlin Heidelberg 2001

Ambiguity in Acquiring Spatial Representation from Descriptions

279

through navigation and learning through maps, but little research has explicitly examined differences between the secondary sources. All of these media can present the same basic spatial information, but thy differ in ways potentially significant for how the information is ultimately represented. 1.1

Spatial Perspective

The spatial perspective that one acquires during learning could influence how one mentally represents the space. Perspective most commonly refers to the viewpoint one takes on the environment, either within the environment (route perspective) or above it (survey perspective). Navigation provides a route perspective and maps give a survey perspective. Spatial descriptions can present one or both perspectives. Research that has examined the relation between acquired representations and perspectives has yielded mixed results. Some studies support perspective-based representation differences [4-8]. Other studies have either shown no perspectivebased differences or have found changes in representation over time[8-10]. The lack of consensus in this research may stem from how perspective is defined. Spatial perspective is not defined solely by viewpoint. A complete definition must also consider the reference system for locating new landmarks, whether the orientation is stable or dynamic, and the amount of information available at a given time. Survey perspectives locate new landmarks with respect to known ones, keep a stable orientation to the environment, and have a substantial amount of information available. Route perspectives use the observer's current location to locate new landmarks, continually change orientation with respect to the environment, and have limited information available at a given time. Because research on spatial perspective generally relies on only a partial list of these features, it is not surprising that conclusions might differ. Verbal descriptions of space provide a special case of presenting spatial perspective. Verbal descriptions can present either a survey or a route perspective [1, 11]. A route perspective description has all the characteristics of a route perspective, including hypothetical movement through the environment. A survey perspective description has all the characteristics of a survey perspective, save one: because spatial information is related through language, only limited information is available at a given time. This characteristic of survey descriptions is similar to the route perspective. Unlike navigation and maps, spatial descriptions are not limited to a single perspective. Taylor and Tversky [12] found that participants produced route, survey, and mixed perspective descriptions both after studying a map and after learning an environment through navigation. Mixed perspective descriptions took one of two formats, either redundant or alternating. Interestingly, the switches between perspectives were rarely signaled. A third perspective can also be presented through descriptions, referred to as the gaze perspective. This perspective has elements of both survey and route perspectives. At first glance, the gaze perspective appears similar to the route perspective, because both imply a tour of the layout. However, the route perspective is an implied physical tour; the gaze perspective tours using only the eyes. Gaze tours

280

H.A. Taylor et al.

maintain a fixed position with respect to the environment and relate object locations to other object locations, like survey descriptions. At the same time, they use a reference frame defined by the speaker, like a route perspective. Ehrich and Koster [13] found that participants used gaze tours when describing a dollhouse room, viewed from the outside. Shanon [14] found similar results for dorm room descriptions; Levelt [15] found the same in descriptions of node-link networks. 1.2 Reference Frames Presenting spatial information through descriptions creates the possibility of language-based ambiguity. One source of ambiguity is the availability of more than one reference frame. Even a simple conversation between two individuals can lead to reference frame ambiguity. For example, Phil and Kathy stand face-to-face in conversation. During the conversation, Kathy points out, “You have something stuck to the side of your face.” Phil responds, “which side?” Kathy responds, “the right side.” Fifty percent of the time, Phil probably reaches for the wrong side of his face. Why? The phrase, “the right side,” could be using Kathy’s reference frame or could be using Phil’s. Because they face one another, right using Kathy’s reference frame equals left using Phil’s. Since neither conversant stated the reference frame they were using, there is a fifty-fifty chance of using the correct one. Ambiguity is most likely to arise when reference frames use the same spatial terms and the frames are misaligned, such as in the example above. When frames are misaligned, does one frame receive priority processing? A pure dominant reference frame seems unlikely. Evidence indicates that reference frame use may be situation dependent. For example, object features, such as movement, make the object’s intrinsic properties more salient [16]. A functional relationship between two objects, also influences frame selection [17], as does physical proximity [18]. Reference frame selection also differs depending on whether one is talking to a live conversational partner or is alone [19]. Despite the importance of reference frames, they do not account for all spatial term ambiguity. Interpretation of terms using a single reference frame can also be fraught with ambiguity. Hill [2] discusses reference fields used to impose a relative reference frame on objects without intrinsic sides or asymmetries. The phrase “the keys are to the right of the ball” can be interpreted using the speaker’s reference frame. Hill [2] argues that when interpreting phrases such as this, the speaker sets up an orienting field, generally parallel with his/her own body. However, he points out that different spatial terms may use different orienting fields. The terms right/left generally use an aligned field, such that the keys are to the right side of the ball as defined by the speaker’s right side. The majority of English speakers, however, use a facing field for the terms front/back. For the phrase “the keys are in front of the ball”, most speakers of English would interpret the keys as being between themselves and the ball. If they had been using an aligned field, the keys would be interpreted as being on the far side of the ball. Hill [2] also argues that not all languages use the same alignment of the orienting field for the same terms. Native Hausa speakers generally use an aligned field for their equivalent of front/back. This changes, however, if one object occludes another. Levinson [20] found that speakers of Tzeltal

Ambiguity in Acquiring Spatial Representation from Descriptions

281

also differ in their assumptions about reference frame interpretation compared to speakers of Western languages, such as Dutch. Extended spatial descriptions, compared to single sentence descriptions, bring additional challenges for reference frame interpretation. First, the reference frame may change during the description. Levelt [15] asserted that extended spatial descriptions would adopt a single reference frame. This assertion, however, has not been supported in empirical work [12, 19]. Further, Taylor and Tversky [12] found that reference frame shifts are rarely signaled. Second, recipients of extended spatial descriptions must both determine the reference frame used and integrate newly described spatial locations with those previously described. In other words, they must interpret new information while maintaining information about multiple other spatial relations in memory.

2. Present Research The present work began as a pilot study to investigate the children's ability to acquire and form spatial representations from different sources. Hence the stimuli take a simple form. The results of pilot testing of university students’ learning an environment from either a description or a map prompted separate research on ambiguity in spatial descriptions, focusing on reference frames and signals for orientation within a reference frame. We report here the work with adults. The present research had two goals. The first was to directly compare spatial representations derived from different secondary sources. The second was to identify variables that affect the interpretation of and representations acquired from spatial descriptions. Participants learned a description of a six-room house either from a gaze tour description or from a map. We selected a gaze tour description rather than a survey description based on the initial developmental goal of the study. We felt that young children would have difficulty both learning the description and interpreting the canonical locative terms (north, south, east, west). Also, the gaze tour description, because it incorporates the terms right, left, in front, and behind, creates the possibility of linguistic ambiguity, which was a focus of our work. We also created tasks that required different perspectives. After learning the environment, participants completed two tasks in counterbalanced order. In one, participants imagined being in one of the rooms, facing a particular direction, and pointed to other, out-of-sight rooms. This task took a route, within-environment viewpoint. In the second task, participants created a model of the house using cutouts. This task took a survey, above-environment viewpoint. Both tasks required consideration of multiple relations among multiple locations. 2.1 Experiment 1 In Experiment 1, we compared the performance of participants who learned a spatial layout either from a map or from a verbal description.

282

H.A. Taylor et al.

Participants. The participants were 48 adults, with equal numbers of males and females. Most were university students who received course credit for their Introductory Psychology course; a few students who were not enrolled in the class also participated and were paid for their time. Apparatus and Materials. Within the lab space, we constructed a 6 x 6 x 7 foot room using PVC pipes with blue curtains as walls. We used six stuffed animals – pig, cat, rabbit, dog, bear, and frog -- to indicate the room participants should imagine themselves to be in on a given trial. During the learning phase, participants sat at a small table. Some of the participants learned the layout of animal’s rooms from a simple map, which consisted of six squares on cardboard with photographs of the animals arranged in the appropriate positions. Square cards with animal photographs attached were also used to reconstruct the layout. The verbal description provided a gaze tour with a viewpoint from above (as in a survey perspective), but using terms right, left, and in front. Procedures. Participants were assigned to one of two conditions. The Map group learned the room locations from the map. The Description group learned the room locations from a verbal description. The experimenter began by telling the participants that they would learn where six animals lived in six separate but connected rooms. The experimenter also told the participants that they would not be able to see all six rooms; instead, they would see the one room in the middle of the testing space and would have to imagine themselves in a specific animal's room, with the other rooms around them. The experimenter then showed the participant the stuffed animals, and asked him or her to identify each by name. For the map condition, the experimenter then produced the cardboard map. One by one, the experimenter placed the animal’s photograph in the middle of each square, saying, “This is the ---‘s room. Can you point to where the --- lives?” For each new animal added, the entire sequence was repeated until all six animal pictures were correctly positioned. The experimenter allowed the participant to study the map for as long as she/he wished before removing all study materials from sight. For the verbal description condition, participants first identified the animals. Next, the experimenter read a description and asked the participant to repeat it from memory. Follow-up questions confirmed whether participants learned the description. For example, if the experimenter said, “The cat lives on the right side of the pig,” the follow-up question was, “Where does the cat live?” This procedure was repeated until the participant could say the entire description twice in succession without error. Across all conditions, the rooms of the house were described/depicted in one of two counterbalanced orders, as indicated below (also see Figure 1). Description, Order 1 • The cat's room is on the left side of the rabbit's room • The pig's room is on the left side of the cat's room • The dog's room is in front of the pig's room • The bear's room is on the right side of the dog's room • The frog's room is on the right side of the bear's room.

Ambiguity in Acquiring Spatial Representation from Descriptions

283

Description, Order 2 • The cat's room is on the right side of the pig's room. • The rabbit's room is on the right side of the cat's room. • The frog's room is in front of the rabbit's room. • The bear's room is on the left side of the frog's room. • The dog's room is on the left side of the bear's room

Pig

Cat

Rabbit

Dog

Bear

Frog

Fig.1. Layout of the six rooms. Once the participants expressed confidence in knowing the room locations, we assessed their knowledge using two tasks, presented in counterbalanced order. For the pointing task, participants were instructed to imagine that each time they walked into the constructed room, a different animal lived there and the other animal’s rooms were all around. The experimenter explained that the participant would be asked to point to where other animals lived. Participants then entered the constructed room with the experimenter. For example, they were asked to imagine that they were in the pig’s room. To designate each room, an animal (e.g., the pig) was placed in the middle of the testing room. The participant was asked to stand on one side of the animal, and the experimenter stood on the other side. Thus the participant, the animal, and the experimenter stood side-by-side, facing the same direction. To ensure consistency, all participants faced south, although they were not asked about cardinal directions. Participants completed 18 points, three from each room. Table 1 shows the set of requested points from each imagined room. For example, when they imagined that they were in the rabbit's room, the participants were asked to point to the cat, dog, and bear (See also Figure 1). A participant’s point was recorded as falling within one of eight categorical directions-- toward one of the four walls or one of the four diagonals (i.e., corners).

284

H.A. Taylor et al.

Table 1. Direction of points from each imagined room Imagined Room rabbit cat pig dog

cat rabbit cat rabbit

frog bear

cat rabbit

Requested Points dog pig dog pig dog pig

bear frog bear frog bear frog

The construction task took place at the same table where learning took place. Participants arranged the cut outs (with the animal photographs affixed) to construct the house layout. Participants received the cutouts in a randomized stack. The experimenter recorded the completed configuration. Results and Discussion. We first assessed whether the participants’ performance was affected by how they learned the layout-- from the map or from the description. We examined performance on both the pointing and construction tasks. The primary dependent variable of the pointing task was the total number of correct points, out of 18 possible. The design involved a 2 (Condition: Map or Description) by 2 (Order: Construction Task First Vs. Pointing Task First) by 2 (Sex) ANOVA. The main effect of Condition was significant, F (1, 40) = 6.44, p < .05. Participants who learned the layout from the map (M = 11.75, SD = 7.42) pointed more accurately than those who learned from the description (M = 7.17 , SD = 5.58). The interaction between Condition and Sex was also significant. Women (M = 15.33, SD = 5.21) who in the map condition performed much better than women in the description condition (M = 6. 83, SD = 5.76). The performance of men did not differ based on condition (M = 8.17, SD = 7.74 for the map condition; M = 7.5, SD = 5.62 for the description condition). No other main effects or interactions reached significance. Analyses of participants’ constructions measured how well participants preserved spatial relations. We compared where participants placed each photograph in the construction to the correct location. Eighteen participants in the map condition reconstructed the entire configuration correctly, but only 6 participants in the description condition performed this well, χ2 (1, N = 48) = 12.00, p < .001. A more detailed examination of construction performance revealed distinct error patterns between conditions. Eight of the description participants, but none of the map participants, reversed the rows, χ2 (1, N= 48) = 13.71, p < .001 (see Figure 2); these participants placed the animals from the top row of the correct layout on the bottom row of their reconstructions (and vice-versa). In contrast, the map group errors involved swapping positions of only two animals. Few description group participants made this type of error. The tendency of the description group to reverse the rows is particularly interesting in light of our focus on orientation in the task. In particular,

Ambiguity in Acquiring Spatial Representation from Descriptions

285

errors that preserved the correct placements of animals within a row but reversed the rows likely stems from misinterpreting “in front.”

Pig

Cat

Rabbit

Dog

Bear

Frog Cat

Pig

Rabbit

Dog

Bear

Frog

(a)

Dog

Bear

Frog (c)

Pig

Cat

Rabbit

(b)

Fig. 2. Examples of constructions. 2(a) represents a correct construction. 2(b) illustrates a row reversal error. 2(c) illustrates switching of two locations. The results presented thus far indicate that participants in the description conditions made more errors than those in the map condition. We reasoned that at least some of the description participants’ errors might have stemmed from the tendency to interpret "in front" differently than expected [2]. To test this hypothesis, we rescored the pointing task, using the participant’s reconstructed map as the correct configuration. In other words, we asked whether participants pointed in a manner consistent with their reconstruction. This adjusted pointing score measured consistency between pointing based on memory and reconstruction of the map based on memory. The analysis revealed a Condition by Sex interaction, F (1, 40) = 10.41, p < .01. Map-group women (M = 15.42, SD = 5.23) performed better than description-group women (M = 9.67, SD = 7.48). In contrast, map group men (M = 7.92, SD = 7.86) performed worse than description-group men (M = 14.75, SD = 5.17). The most important result for the adjusted-total analysis was the lack of a significant main effect for Condition. Although both the original and adjusted pointing data showed a Condition by Sex interaction, a close look at the pointing

286

H.A. Taylor et al.

scores in each case indicates that men in the description group seemed most prone to misinterpreting the meaning of “in front” in the description. The analysis of the adjusted pointing scores revealed, however, that men in the description group used a consistent mental representation for both tasks. 2.2 Experiment 2

The results of Experiment 1 suggest that participants who learned locations from descriptions were more likely to reverse the rows of the house than were participants who learned from a map. This result supports the conclusion that misinterpretation of spatial terms, in particular the term “in front,” contributed to the relatively poor performance of the description group. The concordance between participants reconstructions and pointing seem to indicate that the description group formed a mental representation of the layout and used this representation in the pointing task. In other words, they performed consistently across tasks. In Experiment 2 we examined two factors that might influence participants’ construal of the term “in front”. The first was the position of the animal within the room. In Experiment 1, we placed the animals in a direction that accorded with the ground-level view experienced in the room. In other words, the participant, experimenter, and animal all faced this direction in the room. However, if participants’ adopted a survey-like perspective, then seeing the animals in this ground-level orientation may have caused confusion. We therefore examined whether participants used the animal’s facing direction to interpret orientation information. Accordingly, in Experiment 2, some of the participants saw the animals facing up, as they would be depicted on a map. The remaining participants saw the animals facing in orientation used in Experiment 1. Second, we examined whether providing information about the interpretation of “in front,” by using an arrow, would facilitate the formation and use of an accurate mental representation. We used the arrow during the learning phase to show participants our intended interpretation of "in front." Because our focus was on the factors influencing performance in the description condition, all participants in this experiment learned the layout from a description. Participants. The participants were 72 university students, with equal numbers of men and women. The participants were recruited from the same sources as in Experiment 1. Participants received course credit. In addition, we included as a comparison group the 24 participants from the description group in Experiment 1. Materials. We used a ball, a block, and an arrow constructed out of cardboard to indicate directions. We also used a small bucket to prop the animals face-up in the middle of the blue room. All other materials were identical to those of Experiment 1. Procedures. All participants learned the configuration from the description used in Experiment 1. The procedures differed from those of Experiment 1 in two ways. First, for half of the participants (the Arrow group), we provided specific interpretations of spatial terms [both right and front) using the arrow, ball, and block. Before reading the description to participants, we placed the block on the right side of

Ambiguity in Acquiring Spatial Representation from Descriptions

287

the ball and used the arrow to illustrate that “in front” meant toward the participant. Second, we varied the direction that the animals faced when participants were in the room. For half of the participants, we placed the animals facing up, toward the ceiling. The remaining participants saw the animals facing in the direction used in Experiment 1. Results and Discussion. As in Experiment 1, we first analyzed the pointing task, using the total number of correct points (out of 18) as the dependent variable. In this case, the design of the analyses was a 2 (Arrow: Present or Absent) by 2 (Animal Direction: Same as Participant or Facing Up) by 2 (Order: Pointing Task First or Construction Task First) by 2 (Sex) ANOVA. The main effect of the Animal Direction was significant, F (1, 88) = 15.54, p < .001. Participants who saw the animals facing up (M = 12.58, SD = 5.46) pointed more accurately than those who saw the animals facing in the same direction as themselves (M = 8.15, SD = 5.74). The main effect of Order was also significant, F (1, 88) = 4.07, p < .05. Participants who performed the pointing task first (M = 11.50, SD = 5.56) performed significantly better than those who performed the construction task first (M = 9.23, SD = 6.26). No other main effects or interactions reached significance, including any involving Sex. Performance on the construction task generally reflected performance on the pointing task. Thirty of the forty-eight participants who saw the animals facing up had correct constructions, but only twenty of those who saw the animals facing in the same direction as themsevles had correct constructions, χ2 (1, N= 96) = 4.18, p < .05. Additional examination of the constructions revealed important error pattern differences between the two groups. For example, participants who saw the animals facing in the same direction as themselves reversed the rows, but only 3 of the participants who saw the animals facing up made this kind of error, χ2 (1, N=96) = 4.76, p < .05. Taken together, these results suggest that many participants who learned from the description had difficulty establishing an orientation within the room and, consequently, interpreting "in front" during learning. This problem is reflected in the relatively large number of row reversals seen during reconstruction. Because seeing the animals facing up helped to reduce these kinds of errors substantially, we conclude that participants may have used the animal’s facing direction to orient themselves while pointing.

3. General Discussion Our results help to shed light on how people interpret and use spatial information acquired from maps and from descriptions. The description group experienced some specific problems that the map group did not; these problems appear to involve, at least in part, reconciling differences in orientation as defined by different perspectives. Participants in this condition learned the environment from one perspective, and then performed tasks that required them to take a different perspective.

288

H.A. Taylor et al.

3.1 Spatial Term Ambiguity

We defined the term “in front” from a survey perspective, assuming that the gaze descriptions would motivate participants to adopt a survey perspective. Indeed, instructions prior to reading the description asked participants to imagine a scale model on the table in front of them when hearing the description. Hill [2] noted that most English speakers use a facing field for interpreting terms “in front” and “behind,” particularly when the located objects do not have intrinsic sides. We based our descriptions on this observation. Our participants, however, appeared to use an aligned field for interpreting “in front.” In other words, they used their own reference frame. Consequently, they reversed the rows of the house in reconstruction. Performance on the pointing task mirrored the reconstruction results, showing consistency in using their own reference frame to interpret “in front.” This result indicates that part (although not all) of the burden of learning the layout from the description involved interpreting the reference frame corresponding to the term “in front.” Experiment 2 further supports this interpretation and also provides evidence that participants used additional information (the animal’s facing direction) to orient themselves . Although we believed that specific instructions on how to interpret the spatial terms would best facilitate development of an accurate mental representation from the description, the subtler cue to reference frame interpretation—the direction that the animals faced-- influenced performance to a greater extent. This cue involved the facing direction of the animal, either up or south (same orientation as the participant). When the animal faced up, participant performed better. In Experiment 1, participants tended to use their own reference frame to interpret “in front,” perhaps influenced by the southward facing animal in that study. The misalignment of the animal and the participant’s reference frame perhaps signaled that different frames could be used to interpret “in front,” thus leading to more accurate representations. It is, however, interesting that this subtle clue influenced performance where the explicit instructions did not. This may be related to the difficulties of extended spatial descriptions. Switches of reference frame use in extended descriptions are rarely signaled [12]. Further, tracking information about multiple spatial relations imposes a memory load. Under conditions of memory load, people may use the easiest or most available reference frame, i.e. their own. 3.2 Individual Differences

The present studies provide an interesting opportunity to examine sex differences in both spatial and verbal ability, two areas where sex differences have frequently been implicated [21]. There were unexpected main effects and interactions involving gender. In Experiment 1, the pointing performance of men was substantially worse than that of women, particularly men in the map group. An explanation of this finding is necessarily post-hoc, but also appears relevant to use of reference frames and interpretation of spatial terms. Four men and one woman in the map condition, and three men and one woman in the gaze condition earned particularly low total pointing scores and, because their map construction was

Ambiguity in Acquiring Spatial Representation from Descriptions

289

correct, also received low scores on the adjusted pointing total. However, closer inspection revealed that these participants pointed according to a 180-degree rotation of the configuration. In other words, they reconstructed the configuration accurately but then rotated the relations when pointing. Consequently, scoring their points according to their constructions did not improve their score. All but one of these participants saw the animals facing south, and all of them did the construction task first. This implies that constructing the map first allowed these participants to integrate the relative locations in mind. Then, seeing the animal facing south prompted them to rotate their perspective so that they were, in essence, "facing" the animal. Although the main effect of sex was not significant in Experiment 2, there was an interaction between sex and condition (also seen in Experiment 1). This interaction in Experiment 2, compared to that in Experiment 1, indicates that the men improved when the animal faced up. Presumably, this orientation of the animal did not compel them to reorient their mental representation. 3.3 Reference Frames

Different reference frames can be used to interpret spatial locative terms [20]. Further, different interpretations of terms can exist within a single reference frame [2]. The studies presented here indicate that reference frames, and the resultant linguistic ambiguity, influenced how people learned a house layout from a description. The difficulties, however, were also unexpected given previous research showing that adults can develop accurate representations from spatial descriptions [1]. Some of the difficulties may have arisen because we used the least common spatial description perspective, the gaze tour. While studies have shown that individuals produce gaze tour descriptions in some situations [13-15], Taylor and Tversky [12] found only one gaze tour description in their corpus of spatial descriptions. The main difficulty with the gaze tour lies in the interpretation of the terms “in front” (or “behind”, which we did not use). Interpretation of these terms in the gaze tour is not relative to the body, but relative to another object or location. It appears that when this reference object does not have intrinsic sides, interpretation of “in front” (or “behind”) is ambiguous. 3.4 Conclusions

Our results provide further evidence that situational variables strongly influence reference frame use. Situational influences range from the presence of a conversational partner [19] to language culture [20] to the spatial [18] and/or functional [17] relationship between objects. Further, participants use reference frames differently depending on whether they recall them from memory or have them visually available [22]. Here, the presence of an object (stuffed animal) with a welldefined reference frame influenced participants’ interpretation of the term “in front.” Namely, they interpreted it in line with the animal’s orientation. Most importantly, the present experiments show that reference frame and spatial orientation interpretations cannot be assumed. Too many external factors

290

H.A. Taylor et al.

influence their interpretation. Further, the presence of different, non-aligned reference frames may influence individuals to consider alternative reference frame interpretations. In other words, there is not standard case for determining reference frame and spatial orientation selection.

Author Note This research was funded partly through a Project Varenius Seed Grant from the National Center for Geographic Information Analysis (NCGIA), University of California, Santa Barbara to Holly A. Taylor and David H. Uttal.

References 1. 2.

3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.

Taylor, H.A. and B. Tversky: Spatial mental models derived from survey and route descriptions. J. Mem. Lang. 31 (1992) 261-292 Hill, C.: Up/down, front/back, left/right: A contrastive study of Hausa and English. In: J. Weissen and W. Klein, Editors: Here and there: Crosslinguistic studies on deixis and demonstration. Benjamins, Amsterdam (1982) 13-42 Presson, C.C. and M.D. Hazelrigg: Building spatial representations through primary and secondary learning. J. Exp. Psychol. Learn. 10 (1984) 716-722 Evans, G.W. and K. Pezdek: Cognitive mapping: Knowledge of real-world distance and location information. J. of Exp. Psychol.--Hum L. 6 (1980) 1324 Leiser, D., J. Tzelgov, and A. Henik: A comparison of map study methods: Simulated travel vs. conventional study. Cah. Psychol. Cogn. 7 (1987) 317334 Perrig, W. and W. Kintsch: Propositional and situational representations of text. J. Mem. Lang. 24 (1985) 503-518 Sholl, M.J.: Cognitive maps as orienting schemata. J. Exp. Psychol. Learn. 13 (1987) 615-628 Thorndyke, P.W. and B. Hayes-Roth: Differences in spatial knowledge acquired from maps and navigation. Cognitive Psychol. 14 (1982) 560-589 Golledge, R.G. and N.A. Spector: Comprehending the urban environment: Theory and practice. Geogr. Anal. 14 (1978) 305-325 McNamara, T.P., J.K. Hardy, and S.C. Hirtle: Subjective hierarchies in spatial memory. J. Exp. Psychol. Learn. 15 (1989) 211-227 Taylor, H.A. and B. Tversky: Descriptions and depictions of environments. Mem. Cognition. 20 (1992) 483-496 Taylor, H.A. and B. Tversky: Perspective in spatial descriptions. J. Mem. Lang. 35 (1996) 371-391 Ehrich, V. and C. Koster: Discourse organization and sentence form: The structure of room descriptions in Dutch. Discourse Process. 6 (1983) 169195 Shanon, B.: Room descriptions. Discourse Process. 7 (1984) 225-255

Ambiguity in Acquiring Spatial Representation from Descriptions

15. 16. 17. 18. 19. 20. 21. 22.

291

Levelt, W.J.M.: Cognitive styles in the use of spatial direction terms. In: R.J. Jarvella and W. Klein, Editors: Speech, place, and action. Wiley, Chichester, United Kingdom (1982) 251-268 Levelt, W.J.M.: Some perceptual limitations on talking about space. In: A.J. van Doorn, W.A. van der Grind, and J.J. Koenderink, Editors: Limits in perception. VNU Science Press, Utrecht (1984) 323-358 Carlson-Radvansky, L.A., E.S. Covey, and K.M. Lattanzi: "What" effects on "where"? Functional influences on spatial relations. Psychol. Sci. 10 (1999) 516-521 Carlson-Radvansky, L.A. and D.A. Irwin: Frames of reference in vision and language: Where is above? Cognition. 46 (1993) 223-244 Schober, M.F.: Spatial perspective-taking in conversation. Cognition. 47 (1993) 1-24 Levinson, S.C.: Frames of reference and Molyneux's question: Crosslinguistic evidence. In: P. Bloom, et al., Editors: Language and space. The MIT Press, Cambridge, MA (1996) 109-169 Halpern, D.F., Sex differences in cognitive abilities. 3rd ed. 2000, Mahwah, NJ: L. Erlbaum Associates. 420. Taylor, H.A., et al.: Is the donut in front of the car? An electrophysiological study examining spatial reference frame processing. Can. J. Exp. Psychol. 55 (2001) 177-186

When and Why Are Visual Landmarks Used in Giving Directions?∗ Pierre-Emmanuel Michon & Michel Denis Groupe Cognition Humaine, LIMSI-CNRS, Orsay, France {[email protected], [email protected]}

Abstract. Route directions describe the sequence of actions a moving person needs to take to reach a goal in an environment. When generating directions, speakers not only specify what to do. They also refer to landmarks located along the route. We report two studies intended to identify the cognitive functions of landmarks. In the first study, participants learned a route in an urban environment. They then generated route directions to help pedestrians unfamiliar with this environment to find their way. We found that landmarks were reported more frequently at specific points on the route, especially at reorientation points. The second study showed that pedestrians perceived landmarks as a useful part of route directions. We conclude that reference to landmarks is intended to help movers to construct a mental representation of an unfamiliar environment in advance and to prepare them cognitively to get through difficult or uncertain parts of that environment. Keywords. Landmarks, spatial cognition, route directions, urban environment, navigation.

1. Introduction In spatial cognition studies, considerable attention has been devoted to the processes involved in generating and comprehending route directions. This trend reflects the value attached by researchers to the investigation of the dynamic aspects of spatial cognition, rather than to approaches limited to processing static scenes or environments. It also illustrates the need for accounts of the cognitive processes on which most navigational aids are based. This is especially critical in the context of designing human-computer devices intended to provide pedestrians and drivers with navigational instructions (e.g., Chalmé et al., 2000; Chown, Kaplan, & Kortenkamp, 1995; Jackson, 1998; Werner et al., 1997). Route directions belong to the broad category of procedural discourse, which is intended to assist an agent to carry out an action so that it has a measurable, adaptive effect (cf. Dixon, 1987; Glenberg & Robertson, 1999). In the situation of route directions, the desired effect is for a human agent (or a robot) eventually to reach a ∗

The experiments reported in this paper were conducted as part of a collaborative research program of LIMSI-CNRS with Bouygues Telecom. D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 292−305, 2001.  Springer-Verlag Berlin Heidelberg 2001

When and Why Are Visual Landmarks Used in Giving Directions?

293

new position in a three-dimensional world. Once the agent has successfully reached this new position, an observer can plot the course that has been followed. Route directions can be summarized as the set of instructions that prescribe the actions required in order to execute that course, step by step, in an appropriate manner (cf. Allen, 2000; Denis, 1997; Denis et al., 1999; Fontaine & Denis, 1999; Golding, Graesser, & Hauselt, 1996; Lovelace, Hegarty, & Montello, 1999; Schneider & Taylor, 1999). The basic function of route directions is to prescribe actions. These actions succeed one another in a specific order. For the moves to lead an agent to a succession of specific locations along a route, reorientation procedures will be required. The default move, in the absence of any explicit instruction, is to move straight on, along the back-front axis of the moving agent. However, it is not only necessary to prescribe specific reorientations, but also to specify exactly where they take place. While progressing along a route, movers collect direct perceptions of their environment, and this is important if they are to be able to relate their moves to this environment. Route directions actually rely on the fact that moving agents are also perceptive agents, and this is reflected by their ability to describe the environment when they are invited to do so, especially at points where reorientation is necessary. The objective of a person generating route directions is thus to deliver a combined set of procedures and descriptions that allow someone using them to build an advance model of the environment to be traversed. The discourse will therefore include information that makes it possible for the user to create such an internal representation. The representation is created in such a way that it reflects frontal views of the environment, as it will be viewed along the route (rather than survey views) (cf. Schweizer et al., 1998; Taylor & Tversky, 1992, 1996). Most route directions can therefore be expected to include a rich set of descriptive components (descriptions of scenes, objects, topological relationships between objects, relationships between objects and the moving agent). In order to generate route directions, a speaker will have to refer to three types of entities. The first entities to be referred to are those on which the moves are executed, such as streets or roads. These entities have a two-dimensional extension. They can be described in terms of strips having both a length and a width. They are usually assimilated to a linear entity, or vector. This vector can be specified by its type ("street", "path", "avenue", etc.) and, optionally, by its proper name ("rue de Rivoli"). Width is neglected as long as it remains within certain limits, but it may be taken into account in some situations. For instance, it may be useful to specify which sidewalk (left or right) of the Champs-Elysées to walk on, whereas this will not be crucial in a narrow street. The second set of entities to be referred to are points on these vectors, used to signal the position of a landmark, or the place where reorientation should occur. A variety of linguistic expressions can be used to signal these points, or small regions assimilated to points. Examples include "at the end of the street", "at the top of the stairs", "by the

294

P.-E. Michon and M. Denis

middle of the street", "halfway", "at 200 meters", "at number 28". Such points have a metric value in a system of coordinates. They are conceptually distinct from objects that may be located at those points. The third set of entities to be described are precisely the objects that are found along the vectors. They correspond to points or regions of limited size. Even if they are not strictly speaking point-like, they can be assimilated to points. They are two- or threedimensional entities. When used in route directions, they may serve a variety of functions. The first, and probably most crucial function is to signal sites where actions, and particularly reorientations, are to be accomplished. The second function of landmarks is to help locate other landmarks, which are supposed to trigger a specific action. The third function is that of confirmation; the speaker mentions landmarks situated along the route in a context of lengthy actions, to confirm that the moving person is indeed on the right route. There are good reasons to assume that landmarks play an important role in route directions. This is a general feature, although some individual differences have been consistently observed. For instance, it has been shown that women refer to and make use of landmarks more readily than men do (cf. Denis, 1997; Galea & Kimura, 1993). However, despite such differences, landmarks are generally considered to be key components for constructing the representations used during navigation. The two studies reported below investigated the role of landmarks as components of navigational aids in urban environments.

2. Study 1: Collecting route directions This study involved collecting a corpus of route directions in the city of Paris. We paid special attention to the spatial distribution of the landmarks mentioned in route directions and to how often they were mentioned. 2.1. Method After having learned a route by navigating it, the participants were invited to generate route directions. They were told that these directions should successfully guide someone who was totally unfamiliar with the environment. The same procedure was repeated for two different routes. Routes. The two routes were located in two districts of Paris. Route 1 started from the fountain at the place Saint-Michel and ended at the Medical School, which is located in the rue des Saints-Pères. It was 1200 meters long, and included three segments (a short one, a long one, and then another short one) and it took in three streets. Two reorientations were required, at the end of the first and second segments, respectively. In the middle of the longest segment (850 meters), the route crossed a wide-open space consisting of two contiguous squares. Route 2 started from the Opera House, located in the place de la Bastille, and it ended at Victor Hugo’s House, located in the place des Vosges. It was 700 meters long, and consisted of four segments. The first segment involved walking round the place de la Bastille, and the next three segments

When and Why Are Visual Landmarks Used in Giving Directions?

295

involved three streets. Thus, three reorientations were necessary, one at the end of each of the first three segments. In a pilot study, these two routes had been generated by people living in these districts as the best routes between the starting points and destinations. Participants. The participants were 10 women and 10 men, 18-50 years in age. They were recruited from amongst the general population, and were paid for taking part. We confirmed that they were unfamiliar with the environments studied. Procedure. The participants were tested individually. For each route, the learning phase consisted of two stages. First, the participant was guided by the investigator along the route to be learned, and was instructed to pay attention to all aspects that would allow him/her to give adequate route directions later to someone else. After reaching the destination point, the participant was brought back to the starting point by another route, without walking back along any section of the original route. Back at the starting point, the participant was asked to follow the same route to the destination point again, while being monitored by the investigator. At the destination point, the participant was required to give directions for the route he/she had just learned. These directions were tape-recorded. The procedure was the same for the second route. The order in which the two routes were learned was balanced amongst the participants. Elaboration of data. The route directions collected were transcribed, and revealed considerable variability in terms of length and content. Each individual set of instructions, or protocol, was formatted as a set of minimal units of information, according to the method developed by Denis (1997). For example, the sentence "You come to a boulevard lined by plane trees that you have to cross" was considered to be composed of three units: "You come to a boulevard", "The boulevard is lined by plane trees", and "You have to cross the boulevard". The list of landmarks mentioned by each participant for each route was established. We classified the landmarks into two broad categories: on the one hand, public thoroughfares, such as streets, boulevards, and squares, which we called 2D landmarks since they are essentially twodimensional, and, on the other hand, the buildings, shops, statues, public gardens, etc., which we characterized as 3D landmarks. 2.2. Results Number of landmarks. Although the two routes differed in terms of layout, length, and landmarks, a correlation of r (18) = 0.61, p < .005, was found between the number of landmarks (both types considered) mentioned for the two routes. For the 2D landmarks, this correlation was r (18) = 0.42, p < .07, and for the 3D landmarks it was r (18) = 0.56, p < .02. The participants who mentioned more landmarks for one route also seemed likely to mention more landmarks for the other route. No significant correlation was found between the numbers of 2D and 3D landmarks mentioned for Route 1, r (18) = 0.29, or Route 2, r (18) = 0.11. This supported the view that the two categories of landmarks serve distinct functions in route directions. The mean number and standard deviation of both types of landmarks mentioned in route directions are shown in Table 1.

296

P.-E. Michon and M. Denis

Directions for the two routes included fairly similar numbers of each type of landmarlts. This was to be expected for the 2D landmarlts. The two routes did differ fi-om each other, but both included roughly similar numbers of streets and squares. This finding was more surprising for the 3D landmarlts, as the series of buildings, shops, public gardens, and other items encountered along the two routes was unique to each route. This finding supports the idea that the similarities of route structures elicited similar needs for detailed explanations. Similar numbers of difficulties along the two routes (mainly changes in direction) may have induced this recourse to similar numbers of clues (mainly landmarlts) when giving directions. An alternative hypothesis, although not one directly tested here, is that there is an optimum amount of information to be included in any route directions, irrespective of their length or complexity, which is essentially constrained by the limits of the processing capacities of people listening to directions (cf. Denis et al., 2001).

Table 1. Mean numbers of 2D and 3D landmarlts for each route. 2D landmarks 3D landmarks Route 1 4.5 (1.70) 6.8 (3.04) Route 2 5.0 (2.19) 6.7 (2.89) Both routes 4.7 (1.95) 6.8 (2.92) We calculated how many landmarlts were reported by women and men respectively. Overall, women tended to mention more 2D landmarlts than men, whereas there was no difference between the genders in referring to 3D landmarlts.

Distribution of the landmarks. The spatial distributions of the landmarlts mentioned by the participants for Routes 1 and 2 are shown in Figures 1 and 2, respectively. The figures show all the landmarlts reported by the participants as a whole.

Figure 1. Spatial distribution of landmarlts on Route 1.

When and Why Are Visual Landmarks Used in Giving Directions?

297

A total of 34 different landmarks were mentioned on Route 1, and 28 on Route 2. These numbers are quite low if one considers the virtually infinite number of potential landmarks that can be seen along these routes, as in any downtown urban environment.

Figure 2. Spatial distribution of landmarks on Route 2. On each route, the landmarks seemed to be spread along the whole route. This apparently random spatial distribution is compatible with the assumption that landmarks are simply used as beacons along the routes. According to this view, pedestrians simply progress along a route by directing themselves towards a landmark. When they reach that landmark, they then direct themselves towards the next one they can see, and so on until they reach their destination. In short, landmarks are essentially used in directions as sub-goals along the route (cf. Allen, 2000). However, the frequencies with which landmarks are mentioned reveals differences that forces us to consider another interpretation of their role. Frequency with which landmarks were mentioned. Figures 3 and 4 show the same data as the previous figures, but the frequency with which the landmarks were mentioned is reflected by the size of the corresponding circles. This presentation reveals major differences in the frequency with which landmarks were mentioned at different points along the routes. Higher density references to landmarks correspond to locations of several types. Firstly, and unsurprisingly, large numbers of landmarks were mentioned around the starting point. Similarly, at the other end of the route, the frequency with which landmarks were mentioned increased in the vicinity of the arrival point. In between, points where a change in direction was called for elicited numerous mentions of landmarks. This was also the case for some points, especially along long segments,

298

P.-E. Michon and M. Denis

where wide-open spaces resulting from major street intersections or squares may have been identified by describers as places where errors were likely to occur. This elicited increased reference to landmarks, even though no change in direction was called for at this intersection. Points where a change in direction was required or could be made by accident were also treated by describers as critical nodes calling for a more elaborate description.

Figure 3. Frequency of landmarlts mentioned along Route 1.

Figure 4. Frequency of landmarlts mentioned along Route 2.

When and Why Are Visual Landmarks Used in Giving Directions?

299

In order to test the relationship between the mentioning of a landmark and its location relative to crucial nodes, we computed the correlations between the frequency with which landmarks were mentioned and the distance separating them from the nearest node. For the 34 landmarks on Route 1, a correlation coefficient of r (32) = -.40, p < .01, was found. When the analysis was limited to the 21 landmarks along the second segment (boulevard Saint-Germain), the coefficient was r (19) = -.47, p < .05. A similar pattern was found for the 28 landmarks on Route 2, although the resulting correlation coefficient remained below the level of significance, r (26) = -.35. Landmarks were therefore more likely to be mentioned when they were closer to a node. This implies that the participants describing the routes were sensitive to the need for information people experience when they approach critical nodes along a route (cf. Golding, Graesser, & Hauselt, 1996). 2.3. Discussion This study reveals a major function of landmarks in route directions. The spatial distribution of landmarks and the frequency with which they are mentioned are closely related to specific regions along a route. In particular, landmarks are more likely to be mentioned when they are close to critical nodes. This finding supports the idea that people giving directions spontaneously stress those parts of their discourse that are related to segments of routes where special difficulty will be encountered. They anticipate the potential difficulty pedestrians will experience in trying to find their way by introducing local descriptions of the route, which are expected to help them construct a detailed representation of the environment. Our study also supports the assumption that information is needed by a pedestrian not only at points where actual changes in direction must occur, but also at points where several possible directions could be followed (cf. Allen, 2000).

3. Study 2: Collecting suggestions of how to improve route directions Study 1 provided clear evidence that landmarks play a special role in route directions by signaling the portions of a route to which people following them should pay special attention. However, this evidence is rather indirect, since it is inferred from an analysis of the verbal directions produced. More direct evidence was expected from the next study, involving the collection of explicit statements from people attempting to follow the route about what information they felt to be of primary importance in route directions. 3.1. Method People were invited to walk along routes with the help of quite minimal navigational instructions. The instructions only contained procedural statements referring to the names of the ways to walk along and the directions to take (such as "Take the rue Saint-Antoine on your right"). Participants were invited to express their perceived difficulties as they followed the route, and in particular to identify any gaps in the

300

P.-E. Michon and M. Denis

directions they had received. They were also invited to reformulate the directions by introducing their own corrections or revisions. Routes. The two routes were the same as those used in Study 1. Participants. The participants were 10 women and 10 men, between 18-50 years of age. None of them had participated in Study 1. They were recruited from amongst the general population, and were paid for taking part. They were unfamiliar with the two routes. Procedure. The participants were tested individually on each route. They were equipped with small microphone and tape recorder. Before they set out to follow the route, they were given a minimal set of written instructions, consisting of a simple series of instructions of the type "Take Street X on the left/right", and ending with "Arrival point at Number Y of Street Z". The participants were invited to provide their comments and suggestions in situ, as they followed the route. What they said was tape-recorded as they progressed, and the investigator invited them to reformulate any instruction they found inadequate. Elaboration of data. After transcribing the individual protocols, we listed the problems reported by the participants, as well as the solutions they proposed (involving correcting the original instructions or adding new ones). Some participants recommended solutions without necessarily making explicit the problem they had identified. There were also some participants who proposed several solutions for the same problem. 3.2. Results On both routes, participants reported experiencing problems due to the extremely concise directions they were given. Table 2 lists the problems mentioned by the participants while they were proceeding along Route 1, and the solutions they proposed to improve the directions. For each problem mentioned, the number of participants who suggested a solution referring to a 3D landmark is shown in bold type. The beginning of the route appeared to be the most challenging part of the route, as it was mentioned as a problem by all but one of the participants. This was to be expected as to set out requires that the first direction to be followed must be specified, and this cannot be done in terms of any previous orientation. Thus, the first segment cannot be located in terms of "to your right/left", since the pedestrian is not yet facing in any specific direction. The solution preferred by the majority of participants was to use a landmark located near to the street they were to take. Participants varied quite a lot in the specific landmark that they referred to, but all of them used a 3D landmark located at one corner of the street, thus clearly marking the beginning of the segment. Some 2D entities (a square or the Seine river) were also used to help locate the first segment, but fewer participants used them.

When and Why Are Visual Landmarks Used in Giving Directions?

Table 2. Problems and solutions verbalized by participants along Route 1. Starting Point Problem Locating the first segment 19 Recommended solutions Locate the first segment relative to a fountain/bookshop/café Locate the first segment relative to a nearby square Locate the first segment relative to the Seine river Describe the configuration of the square First Segment (No problem mentioned) Second Segment Problem Locating the beginning of the third segment 18 Recommended solutions Indicate the length of the second segment (in distance or time) Indicate that one has to pass a church/café/subway station/bank Indicate how many streets to go past Indicate that the third segment is located just after a public garden Indicate that one has to pass a square Indicate the names of the streets to be crossed Indicate street numbers at reorientation points Third Segment Problem Locating the end point 1 Recommended solution Indicate that the street numbers on the two sides do not match Whole Route Problem Locating the reorientation points 3 Recommended solutions Indicate the total length of the route Indicate the length of each segment Use landmarks instead of street names Locate streets relative to conspicuous permanent points

301

16 5 2 1

13 10 5 2 1 1 1

1

2 1 1 1

The next major problem was related to the second segment, a long segment containing a critical point. Many of the participants said that the directions should have described the spatial extension of this segment, either by making its length explicit (in terms of distance or time), or by mentioning a 3D landmark to be passed or to be located just before the reorientation point. Some instructions using 2D landmarks were also introduced, referring to the names or numbers of streets to go past.

302

P.-E. Michon and M. Denis

Table 3 shows the problems and solutions verbalized by the participants while proceeding along Route 2. Inspection of the data confirms the special informational value the participants attributed to visual landmarks. Table 3. Problems and solutions verbalized by participants along Route 2. Starting Point Problem Proceeding towards the second segment 11 Recommended solutions Locate the second segment relative to a monument at the center of the square Locate the second segment relative to the starting point (Opera House) First Segment Problem Locating the beginning of the second segment 13 Recommended solutions Locate the second segment relative to a monument/restaurant/ bank/café/kiosk Locate the second segment relative to the starting point (Opera House on the opposite side of the square) Indicate the number or names of the streets to be passed Indicate that one has to walk past a restaurant Locate the second segment relative to a landmark at a street corner Second Segment Problem Locating the beginning of the third segment 6 Recommended solutions Indicate the length of the second segment Indicate the number of streets to be passed Indicate that one has to pass a monument Third Segment (No problem mentioned) Fourth Segment (No problem mentioned) Recommended solution Indicate the length of the segment Whole Route (No problem mentioned) Recommended solution Indicate the total length of the route

7 5

11 10 7 2 1

4 4 1

1

1

Initiating progression and proceeding along the first segment were reported as difficult by participants, and most of them mentioned that the directions were not detailed enough at the starting point and along the first segment. Participants mentioned the difficulty of locating the beginning of the second segment, a street

When and Why Are Visual Landmarks Used in Giving Directions?

303

leading off from the square. Although this street would have been encountered eventually by simply walking around the square, participants mentioned it would have been easier if they could have identified this street with the help of a visible landmark before starting to move. For the wide-open space that offered many possible directions to take, most solutions referred to 3D landmarks. Despite the large number of streets surrounding the square, only a few of the recommended solutions referred to streets. The second major problem concerned the change in direction to be made between the second and the third segments. Here, the participants generally favored a reference to the length of the segment or to the number of streets to be passed. The virtual absence of reference to landmarks is interpreted as resulting from the fact that there were no clearly distinctive landmarks and this may have induced the describers to prefer other strategies to any reference to landmarks in giving directions. 3.3. Discussion The results of Study 2 confirmed the importance of descriptive components, namely landmarks, in route directions to allow a pedestrian to anticipate local difficulties when finding his/her way. These descriptive components are introduced to allow the users of directions to construct an anticipatory visual representation — albeit a very sketchy one — of the places where difficulties are likely to occur. The visual content of this representation is substantiated by the landmarks mentioned, buildings and monuments, that are immediately perceptible to a pedestrian looking around the environment. Streets are also sometimes referred to in these descriptive parts of directions. But they are less distinctive, and their names can only be seen by moving and looking for signs. Consequently, streets are often cited in terms of roads to cross or go past, with rank order being given relative to the street eventually to be taken. This type of description requires maintaining a global representation of the route throughout and sustained attention to what has been passed and what remains to be passed. Such a strategy may not be ideally suited to a prolonged and complex type of behavior, such as following a route.

4. Conclusions Our findings confirm that route directions spontaneously tend to include numerous references to landmarks. Furthermore, when confronted by directions restricted to a list of street names and left/right turns, people react to the absence of landmarks. Their suggestions for improving the directions call for the inclusion of landmarks. Landmarks may serve several distinct functions, such as signaling where a crucial action should take place, helping to locate another less visible landmark, or confirming to a pedestrian that he/she is still on the right way. In any case, the general function of landmarks is to provide information about important maneuvers to perform (or not to perform) at points in a route where changes in direction are likely to occur. Landmarks also contribute to creating a visual model of critical parts of an environment, as seen from a route perspective, which prepares the moving agent to react appropriately to situations involving a decision.

304

P.-E. Michon and M. Denis

References 1. Allen, G. A. (2000). Principles and practices for communicating route knowledge. Applied Cognitive Psychology, 14, 333-359. 2. Chalmé, S., Denis, M., Briffault, X., Gaunet, F., & Nathan, F. (2000). Aides verbales à la navigation automobile: L'impact des instructions directionnelles sur le comportement d'un pilote à l'approche de carrefours. Le Travail Humain, 63, 353-376. 3. Chown, E., Kaplan, S., & Kortenkamp, D. (1995). Prototypes, location and associative networks (PLAN): Towards a unified theory of cognitive mapping. Cognitive Science, 19, 1-51. 4. Denis, M. (1997). The description of routes: A cognitive approach to the production of spatial discourse. Current Psychology of Cognition, 16, 409-458. 5. Denis, M., Daniel, M.-P., Fontaine, S., & Pazzaglia, F. (2001). Language, spatial cognition, and navigation. In M. Denis et al. (Eds.), Imagery, language, and visuospatial thinking (pp. 137-160). Hove, England: Psychology Press. 6. Denis, M., Pazzaglia, F., Cornoldi, C., & Bertolo, L. (1999). Spatial discourse and navigation: An analysis of route directions in the city of Venice. Applied Cognitive Psychology, 13, 145-174. 7. Dixon, P. (1987). The structure of mental plans for following directions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 18-26. 8. Fontaine, S., & Denis, M. (1999). The production of route instructions in underground and urban environments. In C. Freksa & D. M. Mark (Eds.), Spatial information theory: Cognitive and computational foundations of geographic information science (pp. 83-94). Berlin: Springer. 9. Galea, L. A. M., & Kimura, D. (1993). Sex differences in route-learning. Personality and Individual Differences, 14, 53-65. 10. Glenberg, A. M., & Robertson, D. A. (1999). Indexical understanding of instructions. Discourse Processes, 28, 1-26. 11. Golding, J. M., Graesser, A. C., & Hauselt, J. (1996). The processing of answering direction-giving questions when someone is lost on a university campus: The role of pragmatics. Applied Cognitive Psychology, 10, 23-39. 12. Jackson, P. G. (1998). In search of better route instructions. Ergonomics, 41, 1000-1013. 13. Lovelace, K. L., Hegarty, M., & Montello, D. R. (1999). Elements of good route directions in familiar and unfamiliar environments. In C. Freksa & D. M. Mark (Eds.), Spatial information theory: Cognitive and computational foundations of geographic information science (pp. 65-82). Berlin: Springer. 14. Schneider, L. F., & Taylor, H. A. (1999). How do you get there from here? Mental representations of route descriptions. Applied Cognitive Psychology, 13, 415-441. 15. Schweizer, K., Hermann, T., Janzen, G., & Katz, S. (1998). The route direction effect and its constraints. In C. Freksa, C. Habel, & K. F. Wender (Eds.), Spatial cognition I: An interdisciplinary approach to representing and processing spatial knowledge (pp. 19-38). Berlin: Springer. 16. Taylor, H. A., & Tversky, B. (1992). Spatial mental models derived from survey and route descriptions. Journal of Memory and Language, 31, 261-292.

When and Why Are Visual Landmarks Used in Giving Directions?

305

17. Taylor, H. A., & Tversky, B. (1996). Perspective in spatial descriptions. Journal of Memory and Language, 35, 371-391. 18. Werner, S., Krieg-Brückner, B., Mallot, H. A., Schweizer, K., & Freksa, C. (1997). Spatial cognition: The role of landmark, route, and survey knowledge in human and robot navigation. In M. Jarke, K. Pasedach, & K. Pohl (Eds.), Informatik '97 (pp. 41-50). Berlin: Springer.

1

Recognition of Abstract Regions in Cartographic Maps Joe Heike Steinhauer, Tom Wiese, Christian Freksa, Thomas Barkowsky Department for Informatics University of Hamburg Vogt-Kölln-Str. 30 22527 Hamburg, Germany {steinhauer,wiese,freksa,barkowsky}@informatik.uni-hamburg.de Phone: +49 40 42883 2416 Fax: +49 40 42883 2385

Abstract. In the human interpretation of cartographic maps the areas we shall call abstract regions consist of several symbols (map objects), which are grouped to a single object. This abstraction process is an important part of human map interpretation. Abstract regions exist as mental objects in a mental map in the interpreter’s mind. In this article we describe an approach to automate the human process of recognizing abstract regions in cartographic maps by technical processes. We designed and implemented a system for defining abstract regions by hierarchical descriptions. The hierarchies are represented by attributed grammars that can be translated by a compilercompiler to yield a parser for abstract regions. With this parser, abstract region candidates that were identified by simple rules can be evaluated to check if they conform to the definition provided by the user. Our approach combines cognitive considerations on human abstraction with techniques from theoretical computer science and artificial intelligence. Keywords. cognitive modeling, formal methods, map generalization, map interpretation, region abstraction, spatial concepts

1

Introduction

An important component of map interpretation is the cognitive process that classifies and organizes the symbols depicted on the map. A model of the map interpretation process must include a model of this cognitive process. This is quite a challenge as people carry out most parts of this process subconsciously. It would be advantageous to have a mechanism that visualizes this cognitive process by making the implicit interpreter’s knowledge (concepts) explicit. The automatic region recognizer presented in this paper is an interactive system that allows a user to formally specify concepts of abstract regions. On the one hand our formalism is intuitive and easy to understand as it corresponds to the structure of 1

Support by the Deutsche Forschungsgemeinschaft (DFG) is gratefully acknowledged (grant Fr 806-8, Spatial Cognition Priority Program).

D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 306−321, 2001.  Springer-Verlag Berlin Heidelberg 2001

Recognition of Abstract Regions in Cartographic Maps

307

human concepts. On the other hand it allows for automatic calculation of results on the basis of well-founded theoretical methods. By comparing the computed result with the desired result the user can interactively improve the formal description. At the end a correct formal description of the region is obtained. This approach allows us to identify correspondences between mental knowledge structures and their formal descriptions. In such experimental studies we also can determine where the formalism fails to capture human concepts and where further research is required. We shall handle a model in which people studying a map first see the printed symbols. Shortly afterwards they recognize objects that are not directly depicted but are perceived through the formation of directly represented objects. We call these combined objects composite entities or abstract regions. In Fig. 1 houses are symbolized by solid squares; their location and shape in the map represents their location and shape in the nature. Villages neither have their own symbols nor precise locations or shapes. Nevertheless, villages can be recognized as abstract regions triggered by the symbols for houses. In the work presented here, we try to understand the process of abstract region recognition and to formalize it into a set of algorithms that acts as an automatic region recognizer modeling the human region recognition capabilities. Our aim is not only to achieve results comparable to human map interpretation but also to develop technical means Fig. 1. Section of a cartographic map and data structures that resemble the steps and intermediate results of the human region recognition process as closely as known and as technically possible. As we imitate a part of the human map interpretation our work is related to cartography. Cartography has been working on human map interpretation processes for centuries and the results on cartographic generalization are very important for our project. The study of human mental processes also is the topic of cognitive psychology. For an adequate representation of the intermediate results and for our implementation of recognition processes we are mostly interested in cognitive knowledge representation. A difference between region identification in geographic information systems (GIS) and our automatic recognizer is that the intended similarity to the human process is very relevant to our approach. We formalize recognition as a two step process. The first step uses a rule of thumb to extract region candidates that only needs a parameterized neighbor-to relation. Other relations (directional etc.) can be expressed in the second step that performs a precise test of these candidates. The language for this step needs to be flexible and extendable at runtime. In Section 2 we will explain what we call the ‘intuitive process’ and what will be the model for our own system design. In Section 3 the region candidate generator and the region synthesizer that are the main parts of the recognizer are described. In Section 4 we will introduce the fundamental hierarchical structure of the human mental concepts and the attributed context-free grammars that provide an adequate formal model of

308

J.H. Steinhauer et al.

the hierarchical structures. In Section 5 the parsing process is demonstrated by a complex example. Section 6 presents our conclusions and points out directions for further research.

2

The Intuitive Process

To model the human region recognition process, we need to precisely specify our understanding of this process. Therefore we analyze research results from psychology and cartography, especially on generalization and on the hierarchical character of knowledge representation and we explore our own experiences in map interpretation by introspection. Of course we can not guarantee that the resulting process is identical to the corresponding recognition process of a human map interpreter. But by providing a computer implementation of our model we offer a tool for further studying human recognition processes. We will demonstrate our intuitive process with a simple example for which we use the sketch map shown in Fig. 2 that only contains a few objects. All of these can be identified as houses, streets, forests, or gardens by matching with the symbols in the map legend. After recognizing individual objects (elementary analysis) the human map interpreter analyzes the relationships between these objects (complex analysis; Hüttemann, 1981). In our example, the map interpreter’s task is to identify the groups of map objects that correspond to his concept of a village. Such a concept might contain information about the objects involved, how many of them are required, which number should not be exceeded, how much space is around the objects, how far apart these objects may be, and in which constellation they need to be found. This is a subjective concept of a map interpreter. It depends on the task the interpreter is working on, on his experience with map interpretation, and on his background on the task. It is important to point out that these concepts are like ideas about the complex objects that must be recognized on maps. With regard to map interpretation we are dealing with people who do not have a concrete picture of a specific region. Their concepts can be very specific in certain parts and rather flexible in others. Concepts are like schemata in cognitive psychology. A schema can be seen as a collection of pieces of knowledge organized in a way that is technically similar to the 'record' construct used in programming languages (Anderson, 1985) or the frame concept used in artificial intelligence (Minsky, 1975). Abstractly, a concept can be described as a structured set whose members are individuals or other complex concepts (Anderson & Bower, 1974). Due to different needs and knowledge of the interpreters, the recognition of abstract regions can have different results (Head, 1991). The first thing a map interpreter has to do is to find out which symbols on the map correspond to which objects in reality. Then he has to translate his concept of the abstract region into the map language, so that it can be identified in the map. The human interpreter is only concerned with the objects that are relevant to his concept. In a concept there is no information about irrelevant entities (Palmer, 1977). On a map there may be symbols that are meaningless for the interpreter in a given context. Objects not relevant in the concept will be overseen while the map is scanned for relevant objects. We will classify the different types of objects to be able to speak about them.

Recognition of Abstract Regions in Cartographic Maps

309

While the interpreter is scanning the map, some objects will make him stop and look closer to find out if there is an abstract region in that area. We will call these objects triggers, while object types belonging to the abstract regions due to their proximity to triggers will be called annexed objects. Object types definitely not belonging into an abstract region are classified into excluded objects and preventing objects. Excluded objects are to be excluded from the abstract region whereas preventing objects make it impossible for the abstract region to be around. An example for a preventing object could be a nuclear power plant for an abstract region ‘beautiful landscape’. All these object types are relevant, as the existence of an abstract region depends on their presence or absence. Object types that can be found inside an abstract region as well as outside are not relevant for the definition of the concept for that abstract region and therefore are called neutrals. One example for concept of an abstract region ‘village’ is shown in Fig. 2. This example gives the concept for a village a fictive user in our example might have, not a general definition for a village. Houses are the only triggers for the abstract region ‘village’, in this example. There are no annexed objects, no preventing objects, forests are excluded objects, and streets and gardens are neutrals. The interpreter has an intuition of how many objects (cardinality) of every trigger and annexed object are needed to accept a cluster as the wanted abstract region, and how close together neighboring objects must be to qualify as potential members of the same abstract region. He probably does not know a precise value of this distance. The abstract region does not only contain the objects themselves but also the space between them and some space around them. This again is intuitive knowledge the interpreter uses. Legend: Symbol:

Concept:

Explanation:

Object class:

FOREST

relevant

STREET

neutral

HOUSE

relevant

GARDEN

neutral

excluded trigger

• Distance of the houses • Area around the houses • Cardinality of the houses

Fig. 2. Simplified map with legend and concept

With this concept of ‘village’ a human map interpreter would scan a map for triggers; in this specific case he would scan for houses. As soon as he has found one, he would look for neighboring objects, using his own criteria for what is called a neighbor. Neutrals would be ignored. Then he would accumulate a region by grouping the first triggering object, its neighbors, their neighbors and so on, as long as these neighbor objects belong to the village concept. The next step is a generalization process, where the special region is generalized to an object of the concepts type. For this task the borders of the abstract region must be found. (Again, human interpreters do this with a great amount of intuition). We can imagine that the result may be as if the map interpreter would draw a line

310

J.H. Steinhauer et al.

around the group of objects belonging to the region. A possible process of collecting the candidate objects and the generalization for our example is shown in Fig. 3.

Fig. 3. Steps of abstract region recognition and abstraction

The new region not only includes the relevant triggers and annexed objects but also the neutrals, as far as they are inside the calculated region borders. (After recognizing abstract regions, human map interpreters do not memorize all included objects but just the newly found region). The generalization of map objects follows the psychological aspects of landscape recognition: Details are forgotten after a while and only a generalized image of a scene is remembered (Wilhelmy, 1990). The individual objects (houses, gardens, and streets) that form an abstract region remain on the map, but in the mental image of the map in the interpreter’s mind the identified region is stored as one object whose constituent parts are ignored. During further interpretation steps the recognized regions can be used as basic objects to recognize more complex regions on a higher level of abstraction. In our example this could be a rural area or some kind of recreation area in the countryside. This iterative interpretation scheme corresponds to the psychological observation that our spatial knowledge is organized in categorical and hierarchical structures (McNamara, 1991; Timpf, 1998; Car, 1997). Anderson (1985) speaks about categorical or conceptual knowledge. The hierarchy is partly a part-of hierarchy and partly an is-a hierarchy. In artificial intelligence this structure is found in frames or frame systems (Minsky, 1975). It is similar to the class hierarchies in object-oriented software development. The hierarchical structures will be shown in detail in Section 4.

3

The Region Candidate Generator and the Region Synthesizer

We will demonstrate the use of the provided information and the candidate generator and region synthesizer by a small example. The user wants to find villages on the map sketched in Fig. 4a. His concept can be given, following the map legend shown in Fig. 4b and adding parameters like those in the concept in Fig. 4b. The first step is to eliminate the neutrals as shown in Fig. 4c. The second step is to find the neighbors of the relevant objects. To this end, the Voronoi diagram of the relevant objects is

Recognition of Abstract Regions in Cartographic Maps

311

calculated. Voronoi diagrams partition a territory of several objects into those regions that contain only one object such that each location belongs to the region that is closest to the object contained (Aurenhammer & Klein, 1996; Klein, 1989). This partition of an area closely corresponds to an intuitive partition carried out by a human (Gold, 1992). The Voronoi diagram is illustrated in Fig. 4d. Legend: Symbol:

(a)

(c)

(b)

• • •

Concept:

Explanation: Object class: FOREST

relevant

STREET

irrelevant

HOUSE

relevant

GARDEN

irrelevant

excluded index

Max distance of the houses: 50 m Area around the houses: 20 m 5 ≤ n ≤ 100 Cardinality of the houses:

(d)

Fig. 4. Calculating the neighborhood of the relevant objects (a) Initial map, (b) Legend and concept, (c) Relevant objects, (d) Voronoi diagram

The Voronoi diagram allows to compute the neighbors of each object. Two objects are neighbors if they share a region boundary in the Voronoi diagram. The candidate generator checks if the distance between the neighboring objects is within the specified bounds. If not, the neighboring object does not qualify as an object for the new abstract region. For objects that are close enough the candidate generator must check which belongs into the abstract region. This is done like in the intuitive process by taking a triggering object and traversing all qualifying neighbors and all their qualifying neighbors in turn, and so on, in a depth-first search. Depth-first search is used to preserve the structure of the region while expanding the syntax tree. In Fig. 5a the algorithm will find two groups which could be villages. The first group consists of six houses in the center of the map, the second is a oneobject group in the lower right corner. This house does not belong to the first group, as the distance to its Voronoi neighbors is above the specified threshold. The second parameter that must be tested, is the upper and lower cardinality bound of the relevant objects in the abstract region. In our example, a village was specified to have at least five but no more than 100 houses, so that the one-object group is rejected. After checking the critical parameters, the structure parser checks the remaining region candidates for their structural validity. In the village example, no structural constraints have been imposed; therefore the only remaining region candidate is a valid abstract region ‘village’.

312

J.H. Steinhauer et al.

This takes us to the region synthesizer that handles the integration of the new regions into the map. The task to be carried out here is to find the boundaries of the region. For this step the user specified the surrounding areas of the included relevant objects in the concept in Fig. 4b. The recognizer will give the space around every included object like in Fig. 5b. But we do (a) not want to lose the intuitiveness of the Voronoi edges and, of course, we will not allow any parts of excluded objects and their Voronoi regions to slip into the new abstract region through the added surrounding areas. Therefore the surrounding area of each object is restricted by the edges of its Voronoi region. The resulting areas, the object zones, are shown in Fig. 5c. These zones include the object and the (b) surrounding space that belongs to that object. Now the boundary of the whole abstract region can be calculated. We defined this as the optimal hull of the vertices of the object zones. The optimal hull is derived from the convex hull by taking into account special requirements. Fig. 6 illustrates an example how the optimal hull of a group of points is calculated. (c) In Fig. 6a a group of points is depicted that is arranged in ‘u’- shape. If all these points are included in the new abstract region, the convex Fig. 5. Construction of object zones hull in Fig. 6a will lose the ‘u’- shape and (a) Region candidates, (b) Maximal produce a region shaped as a half circle. To surrounding areas, (c) Object zones avoid this, we will modify the convex hull of the included points. The convex hull as shown in Fig. 6b is computed by Delaunay triangulation (Aurenhammer & Klein, 1996; Klein, 1989). There the convex hull is given by the outer Delaunay edges. The optimal hull is computed by restricting the length of the edges of the Delaunay edges according to a specified granularity threshold. Edges that exceed this threshold in length are deleted and the next inner Delaunay edges take their place in the boundary of the abstract region. In Fig. 6c we see that the ‘u’- shape is preserved much better by this optimal hull. The resulting area is reduced by areas belonging to excluded objects. Fig. 7a shows the convex hull, Fig. 7b shows the derived optimal hull reduced by excluded areas. After this restriction, the recognizer will abstract the region. As for a human viewer, the details of the region may not be relevant, once the region has been recognized correctly. The details, i.e. the relevant houses and the neutrals added in Fig. 7c, can be ignored. Therefore the recognizer cuts the region with all overlapping objects, deletes the objects that are completely inside the region, and provides a new symbol for the abstract region, shown on the map in Fig. 7d. This symbol is added to the map legend

Recognition of Abstract Regions in Cartographic Maps

313

shown in Fig. 7e. In further interpretation steps this region can be used as basis for the definition of other abstract regions, just as for other map objects.

(a)

(b)

(c)

Fig. 6. Optimal hull construction (a) Convex hull, (b) Delaunay triangulation, (c) Optimal hull

(b)

(a)

Legend: Symbol: Explanation: FOREST STREET HOUSE GARDEN

(c)

(d)

(e)

Fig. 7. Abstract region synthesis (a) Convex hull, (b) Optimal hull without excluded regions, (c) Objects inside the region, (d) Abstracted region, (e) New legend

4 The Hierarchical Structure of Human Definitions and Formal Grammars In this section we present an example to demonstrate the use of hierarchical definitions of concepts of an abstract region. For this we need to introduce a more complex example that includes additional constraints concerning further relations between the map objects, the concept of geological hot spots. Hot spots are more or less immobile active up-welling regions in the deeper mantle of the earth that every now and then lead to volcanic activity when the up-welling is strong enough to break through the tectonic plate above it. As the tectonic plate moves more or less steadily, episodic volcanic activity produces a chain of volcanoes, forming a chain of seamounts. The tips of the chain of seamounts manifest themselves as volcanic islands (Oceanography

314

J.H. Steinhauer et al.

Course Team, 1991). The mechanism is illustrated in Fig. 8. For the purpose of the example we will concentrate on those hot spots forming chains of islands in the ocean. Well-known instances are the Hawaiian and the Galapagos Islands. To recognize this mechanism as the origin of a chain of islands on a map, the interpreter must know what he is looking for. He could give a simple description of a hot spot as follows: A hot spot is a chain of islands. It consists of one active volcanic island followed by a chain of extinct volcanic islands that form a line that follows approximately the direction of the movement of the tectonic plate. The active volcanic island is an island containing an active volcano. The extinct volcanic islands are islands with extinct volcanoes. This description has a hierarchical structure. The hierarchy is shown in Fig. 9. We need to find a way of describing abstract regions that closely corresponds to the user’s intuition. In computer science formal grammars can be used to describe hierarchic structures in a rather natural way. People are used to describe structures by means of grammars. The structure of English sentences for instance is described in the English grammar. With this one grammar the structure of a lot of different sentences can be described. This indicates that a structure description specified by a grammar can provide considerable flexibility. We need to find a way of describing abstract regions that closely corresponds to the user’s intuition. In computer science formal grammars can be used to describe hierarchic structures in a rather natural way. People are used to describe structures by means of grammars. The structure of English sentences for instance is described in the English grammar. With this one grammar the structure of a lot of different sentences can be described. This indicates that a structure description specified by a grammar can provide considerable flexibility. hot spot

active volcanic island

1..n island active volcano

Fig. 8. The hot spot mechanism (Oceanography Course Team, 1991)

chain of extinct volcanic islands

1..n extinct volcanic island

1..n empty island

1..n island extinct volcano

island

Fig. 9. The hierarchical hot spot concept

We need to find a way of describing abstract regions that closely corresponds to the user’s intuition. In computer science formal grammars can be used to describe hierarchic structures in a rather natural way. People are used to describe structures by means of grammars. The structure of English sentences for instance is described in the English grammar. With this one grammar the structure of a lot of different sentences can be described. This indicates that a structure description specified by a grammar can provide considerable flexibility.

Recognition of Abstract Regions in Cartographic Maps

315

In this paper we will concentrate on a specific type of formal grammars called attributed context-free grammars, as for these grammars the syntactic correctness of sentences can be checked automatically, meaning that it is possible to provide an automated process of building a grammar checker. Attributed context-free grammars are used in compilers to check the syntax of computer programs for correctness while the programs are translated into machine code. There are numerous formal grammars that provide even more useful possibilities to express regional structures, as for instance graph grammars and picture layout grammars. However, to our knowledge, no parser generator exists for these grammars. The leaves us with the problem of the linearity of context-free grammars can be addressed by linearizing a virtual neighboring graph as a tree in depth-first manner. Formal grammars allow the description of hierarchical structures in a top-down manner. This is one of the features we require as the user can explain the structure of abstract regions in a top-down manner as we have seen in the hot spot example. Therefore we will now elaborate on the formal definition of these grammars. The following definitions are basic concepts in computer science (see for example Hopcroft & Ullman, 1979; Aho & Ullman, 1972; Sippu & Soisalon-Soininen, 1988, 1990). Our notation is mostly derived from Hopcroft and Ullman (1979). We start off with context-free grammars: A context-free grammar is denoted as G = (V, T, S, P) where V and T are finite sets of variables and terminals, respectively. It is mostly assumed that V and T are disjoint. P is a set of productions, also called the rules of the grammar; each production is of the form A → α, where A is a variable and α is a string of symbols from (V∪T)*. Finally, S is a special variable called the start symbol. Variables are usually included in “” and can contain capital letters, while terminal symbols neither have “” nor capital letters. An attributed context-free grammar is a context-free grammar expanded by attributes, semantic actions, routines, and context conditions: It can formally be denoted as G = {V, T, S, P, A, ψ, AGL, ξ} where V, T, S, P correspond to the definition of context-free grammars, A is a set of attributes, ψ is a A function: (V∪T) → 2 , mapping sets of attributes to the symbols of the grammar (V∪T), AGL is a set of functions of type b = f (c1, c2, ..., cn) and predicates of type true/false = Q (c1, c2, ..., cn), with ci and b as attributes or further function calls and ξ is AGL a function P → 2 , mapping subsets of AGL to the productions of the grammar. When we write down a specific grammar, we usually specify the productions and note the semantic actions, routine calls, and context conditions directly at each rule. The attributes can be extracted from this, and the mapping is clear by the notation. Context conditions are marked by “CC:” and semantic actions and routine calls by “SA:”. The signs and symbols used in a map can be seen as a language, a cartographic language (Head, 1991). Every sign used in the map can be considered as a terminal symbol of a cartographic grammar. A map interpreter that recognizes abstract regions, groups some of the terminal symbols, following specific criteria and abstracting from details. In cartography this process is known as aggregation and is just what we do when we generalize abstract regions. The abstract regions correspond to nonterminal symbols in the grammar, and the specific grouping rules are similar to the grammar’s

316

J.H. Steinhauer et al.

rules. We will now illustrate the representation by a grammar for the hot spot example. The description of the concept of hot spots using the attributed grammar shown in Fig. 10 is similar to the natural language description we gave before. This works especially well, reading “→” as “is”. For a grammar of this kind a compiler-compiler can automatically generate a parser. This parser then checks cartographic regions just like the syntax checker of a programming language compiler checks the syntactical correctness of a computer program (Bochmann & Ward, 1978). The generated parser can check whether or not the structure of the cartographic signs conforms to the specified map syntax. For a parser the data must be presented in a linear sequence. It can only check whether or not the whole sequence is correct. It is not possible to detect parts in the sequence that are correct and other parts that do not correspond to the grammar. Therefore it is not a good idea to parse the whole map data at once, as it is likely that only parts of the map are abstract regions of the kind we are looking for. We need a mechanism that extracts parts of a map that are worth being checked by the parser. We call this mechanism the candidate generator. It generates abstract region candidates by looking for triggers and testing their surroundings by simple rule of thumb criteria. The valid candidates are then given to the parser for a profound structure check. If the structure of the candidate conforms to the grammar, the candidate is classified as an abstract region. The generalized object is then included in the map while all constituent objects are erased. The result is a map that corresponds to the mental map a human interpreter would have in mind after abstracting the region.

→ CC : .Direction = Tectonic Direction (.Start) → CC: AboutInLine (.Centre, ) NumberOf (, ) > NumberOf (, ) SA: .Direction = Direction (.Centre, Furthest (, .Centre) ) .Start = .Centre

→ active_volcano island {active_volcano} SA: .Centre = island.GetCentrePoint() → {} → | → island extinct_volcano {extinct_volcano} → island

Fig. 10. The hot spot concept grammar (in EBNF notation)

The generated parser can check whether or not the structure of the cartographic signs conforms to the specified map syntax. For a parser the data must be presented in

Recognition of Abstract Regions in Cartographic Maps

317

a linear sequence. It can only check whether or not the whole sequence is correct. It is not possible to detect parts in the sequence that are correct and other parts that do not correspond to the grammar. Therefore it is not a good idea to parse the whole map data at once, as it is likely that only parts of the map are abstract regions of the kind we are looking for. We need a mechanism that extracts parts of a map that are worth being checked by the parser. We call this mechanism the candidate generator. It generates abstract region candidates by looking for triggers and testing their surroundings by simple rule of thumb criteria. The valid candidates are then given to the parser for a profound structure check. If the structure of the candidate conforms to the grammar, the candidate is classified as an abstract region. The generalized object is then included in the map while all constituent objects are erased. The result is a map that corresponds to the mental map a human interpreter would have in mind after abstracting the region. This leads to an architecture with a static component, the candidate generator, and a dynamic component, the candidate verifier, shown in Fig. 11a. The candidate verifier is dynamic, as it is automatically generated according to the region grammar. This can be seen in the detailed architecture shown in Fig. 11b. The first step of the region recognition process is the candidate generation, done by the candidate generator. For this step the user must provide information as to which objects are relevant and which are not. We divide the given objects of the map into those five classes we already used in the intuitive process: triggers, annexed objects, excluded objects, preventing objects, and neutral objects.

(a)

map

concept

Candidategenerator and map manipulator

Candidates

new map

concept

Input

map

relevance, distance, etc

grammar-converter

candidate-generator

Candidateverifier

map-data

Verified Candidates

region synthesizer candidate- generator and map- manipulator

Output (b)

Input

grammar

grammar file candidates verified candidates

new map

compiler-compiler and compiler parse

parser

candidate verifier

Output

Fig. 11. The recognizers architecture (a) Overview of the architecture, (b) Detailed architecture

Furthermore, the user must provide some parameters. As the objects that are collected in an abstract region should be located in the same vicinity, the user must specify a maximum distance to constrain the neighbor-to relation. And, of course, a region does not consist only of the objects included, but also of some space between and around them, see Fig. 3. For every combination of relevant objects the user can specify a distance value. We also provide a default value so that the user does not have to bother with requirements that would only be accessible with a great amount of thinking and calculating.

318

J.H. Steinhauer et al.

5

The Parsing Process

We will now return to the more complex hot spot example. This example, illustrated in the map sketch in Fig. 12a and the legend in Fig. 12b, shows how region candidates will be accepted and rejected by the parser. For illustration purposes, we moved the Aleutian Islands a little to the south and show them on the same map as Hawaii. As we will see later on, one of these chains of islands corresponds to the hot spot concept while the other does not. The information the user must provide for the candidate generator is shown in Fig. 12c in form of a table. All objects on the map are relevant for the abstract region ‘Hot Spot’, so the candidate generator does not delete any. The Voronoi diagram in Fig. 12d is calculated to find the neighbors of all objects. Afterwards the distances of the neighbors are checked, and some Voronoi neighbors are found to be too far away to be included in the new abstract region. The next step is to build the region candidates. One trigger is taken and the depth-first search through all its neighbors leads to a region candidate. If there are triggers that are not in the present candidate region, a new grouping is started, beginning with these triggers. In the present example two groups are found, one in the north (the Aleutian Islands) and one in the south (Hawaii). Both island chains are region candidates as shown in Fig. 12e and are passed to the parser for a structure check. Legend:

Objects: Island Extinct Volcano Active Volcano

Symbol: Explanation: ISLAND

Distances: Extinct Volcano – Island Active Volcano – Island Island – Island

ACTIVE_VOLCANO EXTINCT_VOLCANO

Cardinality: Island

(a)

(b)

(d)

Relevance: Surrounding: annexed 0m annexed 0m trigger 0m

Min: 3

200 m 200 m 120 000 m Max:

(c)

(e)

Fig. 12. Region candidate generation (a) Map, (b) Legend, (c) Concept information, (d) Voronoi diagram, (e) Region candidates

The parser is a single look-ahead top down parser that takes the objects in linear sequence, built in the depth-first search manner through all the annexed objects. The

Recognition of Abstract Regions in Cartographic Maps

319

first group (Hawaii) leads to the syntax tree shown in Fig. 13a were the terms are abbreviated as listed in Fig 13b. The map objects (terminals like active_volcano, extinct_volcano, island) can be aggregated to abstract regions (variables, like < Island >, < Active Volcanic Island >, < Extinct Volcanic Island >, < Island Chain >, etc). These again are aggregated to the start symbol < Hot Spot >. The result is, that Hawaii is a hot spot that conforms with the geologic understanding of a hot spot. The test of the second group (the Aleutian Islands) leads to the syntax tree in Fig. 13 that shows that, when reading the third island, an active volcano is found. The parser expects an island that can be interpreted as an or can be aggregated to an . As this is not possible, the parser stops with the result ‘syntax error’. This means that the Aleutian Islands do not have the right structure to represent a hot spot corresponding to our concept. This result is geologically correct. Even though they form a chain of islands, there is a tectonic movement, and there are many active and extinct volcanoes, the Aleutian Islands have a different mechanism of origin.

(a)





(b)





av ev i

= = = = = = = = = =





active volcano extinct_volcano island



(c)















or







Object stream: av bject stream:av i av av

i

ev ev i ev

i ev ev i ev i

i

i

ev

i

av

i ev

Fig. 13. Syntax trees of the region candidates (a) Syntax tree for Hawaii (left recursive parse), (b) Explanation of Abbreviations, (c) Syntax tree for the Aleutian islands

This means that the Aleutian Islands do not have the right structure to represent a hot spot corresponding to our concept. This result is geologically correct. Even though they form a chain of islands, there is a tectonic movement, and there are many active and extinct volcanoes, the Aleutian Islands have a different mechanism of origin. For the positively identified region Hawaii the optimal hull shown in Fig. 14a is calculated and inserted to the map as shown in Fig. 14b. The new legend with the new symbol for is illustrated in Fig 14c.

320

J.H. Steinhauer et al.

Legend: Symbol: Explanation: ISLAND ACTIVE_VOLCANO EXTINCT_VOLCANO

(a)

(b)

(c)

Fig. 14. Region abstraction (a) Optimal hull, (b) New map, (c) New legend

6

Conclusions

In this paper we have shown that the hierarchically structured information about regions in the human mind can be expressed adequately by attributed context-free grammars. The formulation of the concept of an abstract region using this formal method corresponds to the intuitively given natural language explanation of the respective region. At the same time the attributed context-free grammars allow the efficient implementation of a fast verifier. Together with a region candidate generator this results in a flexible region recognizer that provides a number of benefits: • The implementation provides means of experimenting with different concepts to determine what is needed for a full description of the relevant factors for the definition of abstract regions. This allows further exploration of the system formed by the map together with the map interpreter. By experimenting with the recognizer we can evaluate the difficulties of expressing mental concepts and improve the criteria for the concept definitions. • The modular architecture makes it possible to adapt the recognizer to new criteria. In most cases only the required predicates must be implemented and integrated in the context conditions of the attributed grammar. • It can be used as the working background engine for studies on user interface design for the negotiation of concepts between people and machines. The interface only must convert the input into a formal grammar. • As cartographic maps can be seen as a special class of pictorial diagrams, the recognizer could also be modified to allow exploration studies in diagrammatic reasoning.

Acknowledgment We thank the reviewers of this paper for their insightful and helpful remarks.

Recognition of Abstract Regions in Cartographic Maps

321

References Aho, A. V., & Ullman, J. D. (1972). The Theory of Parsing, Translation and Compiling Vol. I: Parsing. New Jersey: Prentice-Hall. Anderson, J. R., & Bower, G. H. (1974). Human associative Memory. New York: Wiley. Anderson, J. R. (1985). Cognitive Psychology and Its Implications. 2. Series New York: Freeman. Aurenhammer, Franz, Rolf, Klein (1996). Voronoi Diagrams. Informatik Berichte 198 – 5 / 1996. Fernuniversität Gesamtschule Hagen. Fachbereich Informatik. Bochmann, G. V., & Ward, P. (1978). Compiler writing systems for attribute grammars. In: Computer Journal 21, 2 (pp. 144-148). London: British Computer Society Car, A. (1997). Hierarchical Spatial Reasoning: theoretical Consideration and its Application to Modeling Wayfinding. Dissertation. Technical University Vienna. Gold, C. M. (1992). The Meaning of “Neighbour“. In: Frank, A., U., Campari, I., Formentini, U. (Eds.). Theories an Methods of Spatio-Temporal Reasoning (pp. 120-135). Berlin: Springer. Hake, G. & Grünreich, D. (1994). Kartographie 7. Auflage. Berlin: de Gruyter. Head, C. G. (1991). Mapping as a Language or Semiotic System: Review and Comment. In D. M. Mark & A. U. Frank (Eds.). Cognitive and Linguistic Aspects of Geographic Space. (pp 237-262). Dordrecht: Kluwer Academic Publishers. Hopcroft, J. E. & Ullman, J. D. (1979). Introduction to Automata Theory, Languages and Computation. Reading, Massachusetts: Addison-Wesley. Hüttemann, A. (1981). Karteninterpretation in Stichworten. Teil I Geographische Interpretation topographischer Karten. 2. Auflage. Kiel: Ferdinand Hirt. Klein, Rolf (1989). Concrete and Abstract Voronoi Diagrams. Goos, G. & Hartmanis, J. (Eds.). Lecture Notes in Computer Science. Berlin: Springer Verlag. Minsky, M. L. (1975) A framework for representing knowledge. In Winston, P. H. (Ed.). The Psychology of Computer Vision, (pp 211-277). New York: Mc Graw Hill. McNamara, T. (1991). Memory’s view of space. The psychology of learning and motivation: advances in research and theory, 27. (pp. 147 – 186). Oceanography Course Team, Open University (1991). Case studies in oceanography and marine affairs. Oxford: Pergamon Press, in association with the Open University, Walton Hall, Milton Keynes, England. Palmer, Stephen E. (1977). Hierarchical structure in Perceptual Representation. Cognitive Psychology, 9, (pp. 441 – 474). New York, London: Academic Press. Sippu, S., & Soisalon-Soininen, E. (1988). Parsing Theory, Vol. I: Languages and Parsing. W. Brauer, G. Rozenberg, & A. Salomaa (Eds.). Berlin: Springer. Sippu, S., & Soisalon-Soininen, E. (1990). Parsing Theory, Vol. II: LR(k) and LL(k) Parsing. W. Brauer, G. Rozenberg, & A. Salomaa (Eds.). Berlin: Springer. Timpf, S. (1998). Hierarchical Structures in Map Series. Dissertation. Technical University Vienna. Wilhelmy, H. (1990). Kartographie in Stichworten. 5. Auflage. Unterägeri: Ferdinand Hirt.

Geographical Information Retrieval with Ontologies of Place Christopher B. ones', Harith ~ l a n i 'and Douglas ~ u d h o ~ e ' '~epartmentof Computer Science, Cardiff University Queens Buildings, PO Box 916, Newport Road, Cardiff CF24 3XF,United Kingdom email: c.b. iones@,cs.cf.ac.uk Department of Electronics and Computer Science, University of Southampton '~choolof Computing. University of Glamorgan

Abstract. Geographical context is required of many information retrieval tasks in which the target of the search may be documents, images or records which are referenced to geographical space only by means of place names. Often there may be an imprecise match between the query name and the names associated with candidate sources of information. There is a need therefore for geographical information retrieval facilities that can rank the relevance of candidate information with respect to geographical closeness as well as semantic closeness with respect to the topic of interest. Here we present an ontology of place that combines limited coordinate data with qualitative spatial relationships between places. This parsimonious model of place is intended to suppon information retrieval tasks that may be global in scope. The ontology has been implemented with a semantic modelling system linking non-spatial conceptual hierarchies with the place ontology. An hierarchical distance measure is combined with Euclidean distance between place centroids to create a hybrid spatial distance measure. This can be combined with thematic distance, based on classification semantics, to create an integrated semantic closeness measure that can be used for a relevance ranking of retrieved objects.

Keywords: relevance ranking, similarity measures, thesauri, gazetteers

1 Introduction For users of the world-wide web, information retrieval has become an everyday activity. The search engines associated with web browsers are used to find information relating to most domains of human activity. A large proportion of that information may be regarded as embedded in geographical space and, as a consequence, many users will wish to specify geographical place names as part of their query. A characteristic of much of the research into data access methods for,geographical information systems is that it has been targeted towards handling coordinate-based geometric representations of space. Yet most people use place names to refer to geographical locations, and will usually be entirely ignorant of the corresponding coordinates. The previous emphasis upon coordinates is fully justified by the need to retrieve, analyse and display graphically the wealth of map-based and primary-surveyed data that are referenced to geographical and map grid coordinates. However, as more information becomes available both to environmental and social scientists and to non-specialists, the need arises to provide more intelligent information retrieval facilities that can recognise natural language concepts of space and time (Agosti et al. 1993). It is also the case that much geographically-referenced infonnation is identified by place names and other non-spatial classification terms that are not directly accessible by coordinate-based indexing methods. Examples of such data arise in modem and historical textual documents, including records of cultural and naturalenvironmental events and descriptions of material kept in museums, research institutes and other archival repositories. D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 322−335, 2001.  Springer-Verlag Berlin Heidelberg 2001

Geographical Information Retrieval with Ontologies of Place

323

Simple word matching of the sort that is used in search engines is not adequate for many purposes of geographical retrieval. There is a need for systems that perform imprecise matching of place-name terminology. Thus if the user specifies a place name in a query, then the retrieval system should find references to the same or similar places that may be referred to by different names, or may be at different levels of the administrative or topographical hierarchy, or may 'be nearby due to connectivity or to some other measure of proximity (Walker et al. 1992; Jones et al. 1996; Larson 1995; Moss et a1 1998). Having found candidate matches for the query it should also be possible to rank them according to relevance to the user. This requires the use of similarity or closeness measures expressed in terms of place name concepts. Assuming that the user has expressed an interest in some non-spatial concept then available information should also be matched for relevance to that concept. A final ranking of search results might then be expected to combine spatial and "thematic" concepts. The earliest attempts to enable users of geographical information to refer to place names when making queries were based on the use of simple gazetteers in which each place name is associated with a map-grid or geographical coordinate. The coordinate is then used typically as the basis of a conventional coordinatebased query to a GIs. The information processing is therefore brought immediately back into the world of coordinate space. The potential of the simple gazetteer-based retrieval facilities may be increased by encoding relationships between names. The Thesaurus of Geographic Names (TGN) was a notable development in this respect (Harpring 1997). In the TGN hierarchical relationships between names relating to administrative areas and to some physical features are recorded. In addition the TGN stores alternative versions of place names as well as a single geographical coordinate. A metadata approach to gazetteer standardisation has been proposed in Hill et al. (1 999) in the context of the Alexandria Digital Library..This supports the possibility of encoding semantic or spatial relationships between places. It also leaves flexibility regarding the storage of a geometric "footprint". In practice a footprint may be no more than a single representative point, or: centroid, that is located within the areal extent of the place, but it might also be a polygon or a set of points. The need for information retrieval facilities that recognise domain-specific terminology has led to various efforts to construct and exploit ontologies that model the associated concepts (e.g Guarino et al. 1999). In the field of information science, research into thesauri has led to the development of a range of semantic net and thesaurus-mediated information retrieval techniques, for which a variety of semantic closeness metrics have been designed (e.g. Rada et al. 1989; Lee et al. 1993; Richardson et al. 1994). Although the importance of ontologies for representing aspects of geographical information has been clearly highlighted in several studies, notably with regard to the representations of boundaries of geographical phenomena (e.g. Smith 1995 and Smith and Mark 1998) relatively little progress has been made on the practical representation of concepts of place specifically for purposes of information retrieval. This paper is concemed with creating a model of place that may be exploited for purposes of information retrieval. In this respect place names play an essential role. The motivation here is to provide a method for matching a specified place name with place names that refer to equivalent or nearby locations. We are not concemed in the present study with the problem of fixiding places that are conceptually similar but possibly entirely separate in location. Within the substantial body of literature on the subject of place (e.g. Relph 1977; Yuan 1977; Gould and White 1986; Johnson 1991; Curry 1996; Jordan et a1 1998) a common theme is that it reflects human experience of space.and the meanings that we impose upon space. Thus Couclelis (1992) locates the term place in the context of "experiential space". There is little doubt that individual places may be imbued with personal associations for users of the place name, but these experiential aspects cannot be assumed to be relevant in the use of the place name as a locator. Many place names refer to regions representing official administrative categorisations of space and these regions are typically embedded within regional hierarchies that have considerable potential to assist in expanding queries that employ place names to include geographically related places. The use of the term place in this context may be regarded therefore as perhaps nearer to that of Johnson (1991) who adopts for place some of the concepts of Paasi concerned with institutionalisation of regions. With regard to the various types of ontology (Guarino 1997). our use is that of a quite narrowly focused domain ontology. It is intended however that the approach should be extensible to support richer models of place that may be used in a wider variety of applications. An objective here is to rank the results of imprecise geographical queries using place names, that might be global in scope across a wide range of scales. This raises questions of the appropriate types of relationships and semantic attributes to maintain for such applications and leads to the idea of parsimonious

324

C.B. Jones, H. Alani, and D. Tudhope

spatial models that record the least amount of information necessary to process the queries. Equally important for us is the development of similarity measures that are based on the model of place and that can be used for ranking search results. In the next section of the paper we discuss some of the options for place ontologies, for use in information retrieval. In Section 3 we introduce a place schema that has been implemented in the context of a cultural heritage information system, using a semantic data modelling system. Sections 4 and 5 propose some measures of semantic distance for purposes of ranking search results based on queries that include place names and concepts. Section 6 summarises some experimental results of ranking search results on the basis of the proposed closeness measures. The paper concludes in Section 7 with a discussion of the results and hture work.

2. Ontologies of Place 2.1 Requirements

As indicated above, we are concerned with a conceptualisation of place that supports the measurement of locational similarity between named places. The objective is to implement procedures that match a given named place to named places that are equivalent or similar in geographical location. It is assumed that a place may refer to any geographical phenomenon, provided that it has been given a name or literal description. Examples of referents of place include therefore physical features of the Earth's surface such as forests, lakes, rivers and mountains, in the natural realm, and cities, counties, roads, and buildings in the human-made environment. This scope of types of place encompassses those with fiat and bona fide boundaries in the terminology of Smith (1995), that may have either crisp or fuuy boundaries, as determined by physical, political, social or other cognitive parameters. As regards the interpretation of similarity, this will be context dependent. Thus it should be possible to expand a search for similar places to any distance from the given named place. This expansion should be possible with respect to contained and containing places and with respect to overlapping, connected and separate places. In the search for an appropriate ontology of place for information retrieval, we must also take account of the possible requirement for global geographical extent at multiple levels of detail. An important question in this context is whether there is a requirement for inclusion of references to mapped coordinate-based representations of named places. If coordinate data for a global fepresentation of geographic space were ever to be available in a single database, that database would of course be extremely large. Given the need to deal with places for which there may be no exact coordinate-based representation then, even if such a database existed, the procedures that operated upon it would still only apply to a part of the domain of interest.

2.2 Parsimonious Spatial Models It is proposed here that for many practical purposes, detailed geometric data are not necessary, as well as not being desirable for the reasons just stated A more pragmatic approach may be to maintain a parsimonious spatial model of geographical place, in which minimal coordinate data, such as centroids, are maintained in combination with qualitative spatial relationships of topology and proximity. This model of place is related therefore very much more closely to that of geographical thesauri and "rich" gazetteer models than it is to a conventional GIs. The approach adopted is based on the assumption that a great deal of important spatial information for purposes of information retrieval can be pre-computed from coordinate-based data, or computed on-the-fly from the sparse coordinate data that are stored. Firm topological relationships of connectivity and overlap between regions with digitised boundaries can easily be extracted from digital map datasets. Similarly attributes that might be of significance when evaluating locational similarity measures, such as area and the length of common boundaries, can again be derived from the digital map data.

Geographical Information Retrieval with Ontologies of Place

325

2.3 Approximating Spatial Footprints Examples of Euclidean distance measures that might not so easily be pre-computed and stored are those of the distance of a centroid to a boundary, and a boundary to another boundary, since for any given centroid or boundary there may be many other boundaries to which distance might need to be computed. A solution to this problem, that avoids storing boundary data explicitly, is to compute approximate boundaries on-thefly fiom stored centroids, prior to calculating the distances. One method of doing this is to construct Voronoi diagrams of the set of centroid-referenced locations known to be inside a region of interest, in combination with nearby centroids known to be outside. The region is then approximated by the set of Voronoi polygons associated with the internal sites. Knowledge of containment and exclusion can be derived fiom stored topological relations of region containment and of region connectivity. Experiments reported in Alani et al. (2001) demonstrate that these methods can result in reasonably good quality estimates of regional area and boundary location, with area and boundary length approximation typically within 5% of the corresponding values measured on the original digital map data. An attraction of Voronoi-based methods for region approximation is that they can be applied to the estimation of imprecise regions for which there may be no existing digitised boundary but for which there may be knowledge of included and external places.

3 Modelling Place in OASIS Here we explain how place has been modelled in an experimental terminology system called OASIS (Ontologically-Augmented Spatial Information System). It has been built using the Semantic Index System (SIS) which is an object-oriented hypermedia management system developed by ICS-FORTH (Doerr and Fundulaki 1998). SIS provides multiple classification levels starting with the Token level, above which are a Simple Class and successive Meta Class levels. Both classes and their associated attributes are treated in SIS as objects that can have names, attributes and relationships to other levels. Access to the SIS database can be programmed with the Programmatic Query Interface (PQI) functions, in combination with Ci-t programs. OASIS has been used to maintain cultural information about archaeological finds and historic buildings that have been classified using terms from the Art and ~rchitdctureThesaurus (AAT) and referenced geographically using place names associated with the data and linked into the Thesaurus of Geographic Names. A Place class has been implemented in OASIS as a type of Geographical Concept. The thematic categories of place are specified by current and historical place types. A place may have multiple place types allowing instances of place to be characterised by physical, cultural or administrative classes. In our implementation, instances of place are taken mostly from the Thesaurus of Geographic Names, augmented with Bartholomews digital data for the UK,and the associated place types are from the AAT. Figure 1 illustrates the schema for Place and shows how it inherits various attributes and relationship types from Geographical Concept. A Geographical Concept has a Standard Name (or Preferred Term) and Alternative Names won-Preferred Terms). These names are associated with alternative spellings, a date of origin and a language. A scope note provides a verbal explanation of the concept. Geographical Concepts may be associated with a location defined by a geometry object, an area measurement value and spatial relationships. In the Place class, location is specialised to a centroid, defined by latitude and longitude coordinates, and spatial relationships are specialised into the meet, overlap and partOf relationships. The Artefact class has attributes of date-found, type and description and relationships of made-of to the Material class (not shown here). As illustrated in the figure, the Artefact class is associated with the Place class via found-at and made-at relationships. Figure 2 illustrates the classification of City of Edinburgh as an instance of the Place class. Note that it has three meets topological relationships with the administrative adjoining regions of Midlothian, East Lothian and West Lothian and that it is the parent for a set of partof relationships with multiple places that are referenced in the figure by the MANYpartOf object. It should be noted here that the partof relationship itself has attributes, notably of dates, enabling temporally-specific query expansion. When there is more than one place with the same name, only one instance of the repeated name is maintained, but a unique name for each of the associated places is created. These unique name objects are then linked via Standard Name or Alternative Yame

326

C.B. Jones, H. Alani, and D. Tudhope

Fig. 1. Place as a type of geographical concept

-7 Place

1~idbthianh East Lothian]* West l o t h i a h ,MANYpartO

instance

meets meets

meets

.

'-+ '-

+

City of

partof

centroidr

latitude + Scotland

5555N

longitude

--+ 000 15 W

Edinburgh

unitary authorid

Fig. 2. An example of an instance of place

provided in Figure 3 in which the name Hull is the standard name for the Hull in Canada (Canada-Hull) while it is the alternative name for the Hull in the UK (UK-Hull), the official (standard) name of which is Kingston upon Hull. This approach facilitates place name disambiguation, since access to the non-unique place name leads directly to all associated places. The user can then select the place of interest.

(Alternative Name)

Fig. 3. Example of the use of a single place name to that refers to multiple places

Geographical Information Retrieval with Ontologies of Place

327

Fig. 4. Example of a particular artefact as illustrated in the OASIS user interface.

OASIS has been populated with cultural heritage data from the Royal Commission on the Ancient and Historical Monuments of Scotland (RCHMS). An example of the schema for a particular artefact, axe number DE121, is illustrated in Figure 4 which illustrates several relationships including the found-at relationship to a place.

4 Locational Similarity Metrics 4.1 Relevant Criteria In developing or choosing closeness metrics for ranking places in information retrieval we focus upon those aspects of geographical space that are concerned primarily with location and proximity. We are not concerned with matching places that may be similar in shape or structure unless they refer to the same or similar locations on the Earth's surface. We treat place names as descriptors for regions of space. Thus if location is regarded as a substrate for concrete objects (Guarino 1997), then place names refer to specific locations. The referents of place serve to locate specified phenomena that are of interest to users of the place names. For example an archaeological query may request "axes in Edinburgh", in which the user may be interested in occurrences of axes in places that are inside Edinburgh, or refer to the same location as Edinburgh or its constituent parts, as well perhaps as places that are somehow near to Edinburgh. Potential criteria for assessing locational similarity between a specified place and a candidate place, when searching for information, include the following: = = = =

distance in map or geographical coordinate space between query and candidate; travel time between query and candidate; number of intervening places; spatial inclusion of the candidate within the query place; containment of the query place by the candidate; containment of candidate within, or overlap of candidate with, regions that contain or overlap the query place; boundary connectivity between query and candidate.

Because the motivation here is information retrieval we do not make any assumptions about the user's familiarity with the places which they specify. In this context therefore cognitive measures of closeness as perceived by people living in or familiar with the places may not be relevant to determining locational

328

C.B. Jones, H. Alani, and D. Tudhope

similarity. The criteria are therefore based on more concrete attributes of places. This is not to rule out the potential of cognitive measures but we regard them as a special case. In the present study we have focused initially on the use of geographical hierarchies in combination with Euclidean distances. Search expansion methods based on these aspects of space will automatically find connected places and will tend to give them higher priority than disconnected places. Euclidean disjance whether in map-grid space or as measured on the Earth's surface leads to a ranking based on physical proximity, and introduces the possibility of constraining the expansion of a search for similar places according to specified distance thresholds. While Euclidean distance is undoubtedly valuable for measuring locational similarity, it fails to take account of characteristics of place as determined by physical, social and political factors. Regional hierarchies have great potential in distinguishing between the relevance of nearby places since they record facets of interpretation of the place of interest with regard to the topographic and human environment and allow for the possibility of making distinctions between the importance of different aspects of geographical space according to the interests of the user. In this regard the possibility of inclusion within multiple hierarchies is essential. Travel time is not considered here at this stage since it is so dependent upon the means of transport, which would need to be assumed, or specified by the user. Euclidean distance however acts as a surrogate for travel time that may or may not be appropriate depending on the local topography and the interests of the user. The number of intervening places between query and candidate place provides the possibility of a qualitative measure of distance. It is a familiar concept in analysing geographical accessibility and has been proposed as a measure of spatial similarity for purposes of information retrieval (Jones et a1 1996). It has not been included in the present study, partly due to negative results of some preliminary user tests of its significance. However it is still regarded as potentially useful. One method of determining similarity between two objects is based on their common and distinctive (non-common) features (Tversky 1977). An example of the application of a feature-based spatial entity class similarity measure is found in Rodriguez et al. (1999), who integrated measures representing parts, functions and attributes respectively. If a conceptual hierarchy is available then terms within the hierarchy can be compared by measuring the distance between them along the branches of the corresponding graph. Following Rada et al. (1989) this distance is then equal to the number of connecting links in the shortest path in the graph. Calculation of distance within hierarchies can be refined by applying weights to links in the graph (Kim and Kim 1990). The weight or importance of a node in the graph may also be related to the inverse of its depth in the hierarchy. In the context of hierarchies and poly-hierarchies, an alternative approach to similarity or distance measurement is to consider all the non-common parents (at whatever level) of the respective nodes, each of which may have a weight inversely proportional to the depth in the hierarchy. This introduces a measure of the degree to which the nodes differ in their inherited categories, that is sensitive to the hierarchical levels. Semantic distance between a pair of terms increases in proportion to the number of distinctive (noncommon) parents. The use of non-common super-classes has been advocated by Spanoudakis and Constantopoulos (1994) and Sintichakis and Constantopoulos (1997) who have demonstrated the approach in the context of similarity metrics based on a combination of generalisation relations, classification and attributes. 4.2 Hierarchical Distance Measure

Here we adapt the methods based on non-common super-classes to geographical poly-hierarchies using generic part-of relations that may be interpreted spatially as inside or overlap. The parent regions provide units of space, or substrates, that assist in determining commonality of location, if places lie in the same parent region, and differences of places that belong to separate regions. It is assumed that a place is characterised by the sum of the geographical regions, or other parent places, to which it belongs either directly or by inheritance within a hierarchy. A town for example may be inside or overlap a county that itself is part of the formal hierarchical administrative subdivision of a nation, which is itself part of a global geopolitical hierarchy. The same town might belong to a physiographical hierarchy based on predominant features of the landscape, giving rise to descriptors such as "Great Plains", "Mackenzie Delta", "Southern Uplands" within which might be subdivisions consisting of particular named river valleys, mountains or

Geographical Information Retrieval with Ontologies of Place

329

marshes. There might be single level subdivisions, based for example on national parks, military installations or ethnic groups that further characterise the place either completely or partially. We define the Hierarchical Distance Measure (HD) between query place q and candidate place c as foIlows:

The L,, L, and Lzvalues represent the hierarchical levels of the individual places within their respective hierarchies. The set of places x are those distinctive super-parts of the query term that belong to it but not to the candidate, while places y are the distinctive super-parts of the candidate that are not shared with the query. The places z are the query and candidate terms themselves. The sets of terms q.PartOf and c.PartOf refer to the transitive closure of the super-parts of q and c respectively in the part-of hierarchy. The weights a, B and y provide control over the application of the measure. In particular the weights a and flprovide the option of asymmetry, making candidates that are sub-parts of the query more (or less if required) similar to the query than are candidates that are super-parts. Tversky (1977) reported experiments indicating that asymmetry is observed in people's perception of the similarity of terms where one is more important in some sense than the other, for example if one t e r n is the category or super-class of the other (e.g. bird vs sparrow). The purpose of the weight y is to provide control over the use of the query and candidate term levels in the distance measure. It is envisaged that if both the query and candidate are members of the same hierarchy then y should be a non-zero value. Thus if q and c are both sub-regions of the same parent region within a particular hierarchy, this would result in a non-zero distance between them. However, if q and c are not both members of a regional hierarchy then in general y would be zero. The consequence of this is that if both of the latter places belonged to the same parent region(s) in a hierarchy there would be no difference between them with respect to the regzonal hierarchy. In general when applying the hierarchical distance measure, distance between query and candidate increases according to the number of non-common parents, i.e. the distinguishing regions. The level values increase with increasing depth in the hierarchies with the result that there are smaller differences between pairs of places deeper down the hierarchy than there would be higher up. This is intended to reflect the idea that branches higher up a hierarchy introduce more significant differences than lower down. It should be noted that the formula measures distance explicitly with regard to distinguishing super-parts, while closeness is regarded as implicit within the branching structure of the hierarchies. Examples of the application of the measure are provided in section 6 . 4.3 Euclidean Distance Measure (ED)

Because we are concerned here with applications that may be global in extent, we base measurement of Euclidean distance on latitude and longitude values for centroids. The Euclidean Distance measure calculates the great circle distance. As pointed out above, use of only the centroids of places that have areal extent produces ED values that reflect the separation of the approximated centres of the respective places. As indicated in Section 2.3 it is possible to create region approximations that may be used to measure distances between approximated boundaries, or between point sites and approximated boundaries. 4.4 A Combined Spatial Closeness Measure The two locational distance measures can be combined in a weighted combination referred to as the Total Spatial Distance (TSD) as follows:

where we and whare weights of the ED and HD respectively. These weights lie in the range 0 to I. In order to calculate a weighted combination of the individual distance measures as above, it is necessary to normalise both of the measures to a range between 0 and 1 prior to use.

330

C.B. Jones, H. Alani, and D. Tudhope

5 Thematic Distance In order to measure the semantic similarity between non-spatial concepts we introduce a thematic distance measure that is applied here within the mono-hierarchical classifications of the AAT. The purpose of the measure is to determine similarity between the query-specified phenomenon of interest and a candidate information object, with a view to combining this measure with those of spatial similarity. It may be noted however that a non-spatial similarity measure could also be used to compare places with regard to their place type categories, which as previously noted are also taken from the AAT in our current system. The primary semantic relationships in the AAT are those of broader term (BT) which relates a term to its parent term within a hierarchy, narrower term (NT)which does the converse, and related term (RT). The BT and NT relations are the fundamental links between terms in individual hierarchies. In the context of the AAT, the BT relationship is one of semantic generalisation and hence is equivalent to the is-a relationship. The RT relationship records associations between terms that may be in different hierarchies. A detailed discussion of the application of the RT terms in OASIS can be found in Tudhope et a1 (2001). Here we apply a weighted shortest path procedure based on Tudhope and Taylor (1997) that is a modification of the method of Rada et a1 (1989). It is based on the principle of measuring the weighted distance between a pair of classification terms by the shortest number of links that separate them in the semantic net of classification terms. The weighting is affected by an inverse hierarchical depth factor that is analogous to the hierarchical level in the hierarchical spatial distance measure. Thus the thematic distance between two terms a and b is given by:

where each ratio on the right hand side refers to a link in the shortest path between a and b. The values L, represent the levels of the terms i, while the values C,& represent weights attached to those links between the respective terms j and k. Each different type of relationship may be given a different weight. In our experiments BT and NT relationships are given equivalent weights, while the RT relationship is given a larger weight, resulting in greater computed distance.

6 Ranking results 6.1 Examples

We now provide examples of the application of the spatial similarity measures. An example hierarchy is illustrated in Figure 5 in which several hills are associated via partofand overlap relationships with four administrative region places (Scottish Borders, West Lothian, Midlothian and City of Edinburgh). Note that the four administrative regions share a common parent in the region Scotland. Scotland is at hierarchical level 4 as it is part of the United Kingdom, that is part of Europe, that is part of the World Thus the other two levels are numbered 5 and 6 respectively.

City of Ediribwgh

Fig. 5. An example of part of a place name poly-hierarchy

Geographical Information Retrieval with Ontologies of Place

331

Examples of the application of the hierarchical distance measure are as follows: I. HD (Henshaw Hill, West Cairn Hill) = 0 reflecting the fact that the two places both overlap the same two regions of Scottish Borders and West Lothian and no other regions. 1 2. HD (Henshaw Hill, East Cairn Hill) = level of City of Edinburgh = 115 = 0.2 reflecting the fact that East Cairn Hill overlaps the City of Edinburgh, but Henshaw Hill does not. I

3. HD (Henshaw Hill, Carnethy Hill) = 1 1 1 level of Scottish Borders level of West Lothian = (115 + 115) + 115 = 0.6 In this case the Scottish Borders and West Lothian are both distinctive super-parts of Henshaw Hill, while Midlothian is a distinctive super-part of Carnethy Hill.

4. HD (Henshaw Hill, Harbour Hill) = (115 + 1/5) + (115 + 115) = 0.8 This example shows that Harbour Hill is at a larger distance from Henshaw Hill due to the former's two distinctive super-parts of Midlothian and City of Edinburgh in addition to the two super-parts of Henshaw Hill. To illustrate the application of asymmetry, the values of a and P may be set to 1 and 0.5 respectively. When the query term of Scotland is compared with the candidate term Henshaw Hill we obtain the following result: 5. HD (Scotland, Henshaw Hill) = 1 1 1 0 + 0.5 level of Scottish Borders level of West Lothian level of Scotland = 0.5 (115 + 115 + 116) = 0.28 Thus Scotland has no distinctive super-part, while Scottish Borders, West Lothain and Scotland are distinctive super-parts of Henshaw Hill. Note that Scotland is not regarded as a super-part of itself. 6. HD (Henshaw Hill, Scotland) = 1 1 level of Scottish Borders level of West Lothian = (115 + 115 + 1/6) = 0.57

1 level of Scotland

which indicates that Scotland is found to be more distant from Henshaw Hill than vice versa. For the purposes of finding things that are in Scotland or in Henshaw Hill this appears to be appropriate. It is important to note however for other types of query referring to these places this might not be an appropriate setting of the weights. Finally we introduce a score that combines nomalised values of the thematic distance measure and the spatial distance measures: Score = 100 - 100( w,TD, + (w, (w,ED, + whHD,) )

(4)

where each of the measurements refer to the distance between the query and the relevant candidate terms whether places or non-spatial objects. The subscript n refers to normalisation and w,, w, are weights for thematic and spatial measures respectively. These weights can all be set via the OASIS user interface. Table 1 illustrates the results of ranking retrieved data for a query that requested "axes in Edinburgh". Here weights for w, and w, have been set to 0.4 and 0.6 respectively and the weights for we and wh have been set to 0.6 and 0.4 respectively. Note that in the figure, place names are paired with their immediate parent. Thus "Edinburgh'Currie" refers to the town of Currie which is inside the City of Edinburgh, while "EdinburghlEdinburgh" refers to the candidate place of Edinburgh itself. Due to the sparse distribution of

332

C.B. Jones, H. Alani, and D. Tudhope

certain types of artefact in our database, several simulated data objects have been inserted in to the database for demonstration purposes. These include "tomahawks". 6.2 Query Expansion and Weight Specification

The search for matches with the query terms involves traversal of the place name poly-hierarchies and of the thematic, concephlal and poly-hierarchies belonging to the phenomenon of interest. In order to constrain expansion, threshold values for the TD, ED and HD can be set in the user interface. A further control on search is the number of links traversed from the query term. The user interface also includes provision to set weights for the thematic BT, NT and RT relationships as well as the weights for the combination of the ED and HD and for the combination of the spatial and thematic similarities. As each node in the respective hierarchies is encountered, the distance measures are calculated and compared with the relevant threshold values. Search will continue to expand in the respective hierarchies until all thresholds have been met or there are no more paths to follow.

ARTEFACT

Axes (weapons) Axes (weapons) Axes (weapons) Tomahawks (weapons) Tomahawks (weapons) Tomahawks (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Axes (weapons) Throwing axes Throwing axes Axes (weapons) Axes (weapons) Axes (weapons) Tomahawks (weapons) Axes (weapons) Axes (weapons) Axes (weapons)

PLACE FOUND

Edinburgh'Edinburgh Edinburgh'Edinburgh Edinburgh'Edinburgh Edinburgh'Edinburgh Edinburgh'Edinburgh Edinburgh'Edinburgh Edinburgh'Leith Edinburgh'Leith Edinburgh'Corstophine EdinburghlLuddingston Edinburgh'Currie Edinburgh'Cume Edinburgh'Cunie EdinburghlDalmeny Edinburgh'Ratho Edinburgh'Ratho Edinburgh'Kirkliston Edinburgh'Kirkliston Edinburgh'Kirkliston Edinburgh'Edinburgh Edinburgh'Edinburgh East Lothian'Musselburgh East Lothian'lnveresk East Lothian'lnveresk Edinburgh'Cume MidlothianlDalkeith Midlothian'Borthwick West Lothian'Kirknewton

SCORE 100 100 100 83 83 83 81 81 79 78 74 74 74 70 69 69 68 68 68 60 60 60 59 59 57 56 56 54

Table 1. Ranked results for a query on "axes in Edinburgh" using both spatial and thematic distance measures.

7 Conclusions and Discussion In this paper we described an ontology of place that may be used to derive semantic distance measures for use in geographically-referenced information retrieval. The proposed ontology is characterised by a mix of qualitative and quantitative spatial data including topological relations and sparse coordinate data representing the spatial footprints of places. Places are classified according to their geographical categories and are linked to instances of non-geographical phenomena classified by conceptual hierarchies. Places are associated with other places via topological relationships. Terminology is classified into standard (preferred) and non-standard (or non-preferred) terms, and terms and relationships are qualified with dates.

Geographical Information Retrieval with Ontologies of Place

333

In using a parsimonious model of space that records very limited coordinate data, the approach is closely related to information retrieval based on gazetteers and geographical thesauri. The primary contribution lies in diversifying the types of information maintained and hence developing integrated semantic closeness measures that can be used for automatic ranking of query results. In an implementation of the ontology, using data from existing thematic and geographical thesauri, fiom a cultural history source, and some simulated data, we have demonstrated how a combination of a hierarchical distance measure and an Euclidean distance measure can be used to rank retrieved archaeological site information in terms of geographical relevance. An overall ranking is obtained via a weighted combination of the spatial distance measures and a thematic measure. The hierarchical distance measure is notable for distinguishing between places according to the extent to which they belong to or overlap with different geographical regional hierarchies. While the techniques presented here appear to have considerable potential, there is no doubt that the distance measures could be refined and extended in various ways. The hierarchical distance measure presented here combines "proper part of' (inside) relations with overlap relations. It would be possible to weight overlap relations according to the degree of.over1ap as proposed by Beard and Sharma (1997). To do so would require measurement of that overlap with coordinate-based data. With the spatial model proposed here, that would not be possible directly from the centroid data. However, an approximation of overlap could be obtained using Voronoi methods that approximated regions by the union of the Voronoi cells of their contained places. An alternative would be to pre-compute overlap of ail represented regions using more detailed map data. The presented scheme gives equal weight to super-parts that are at the same hierarchical level. It may be that some geographical hierarchies are of more significance than others for a particular application, leading to the possibility of weighting members of individual hierarchies differently, or omitting them, according to context. The hierarchical distance measure distinguishes between places that are in different parts of a polyhierarchy, but it does not take account of whether adjacent places are connected or not. If two candidates are at equal distances from the query place, but one was regionally connected to the query place while the other was not, then the former might be regarded as closer. Clearly the meet relation could be employed to introduce a weighted connectivity term into the total spatial distance measure. The weighting of this term could be a function of the length of the common boundary. The Euclidean distance measure employed here is based solely on measurement between centroids, but it may be more appropriate to take account of distances between boundaries of regional places, or between regional boundaries and the centroid of a point-referenced place. Voronoi techniques again may help in these measurements. This paper has focused on the use of place as a locator, and for purposes of similarity measurement the only thematic (non-spatial) information taken into account when comparing places is the categories of the regions to which places are referenced via part-of and overlap relationships. It is possible that the place class may be relevant when comparing places for purposes of location and it would certainly be expected to be relevant when searching for places that were similar to the query place. It would be quite simple to extend the existing measures to include a thematic similarity measure based on the place types. This could employ the existing TD measure presented here. Alternatively, assuming that places may be associated with multiple place types, it may appropriate to consider a method based on non-common superclasses, analogous to the hierarchical distance measure, or on the feature-based methods exemplified by Tversky's ratio model. In the course of this study some limited user tests were camed out that did provide support for the techniques presented. It would be useful to carry out more systematic tests to evaluate alternative versions of the similarity measures. Experiments with users might also lead to methods to adapt the weights in the distance measures to the context of the query.

,

334

C.B. Jones, H. Alani, and D. Tudhope

Acknowledgements We would like to thank the J. Paul Getty Trust and Patricia Harpring in particular for provision of their TGN and AAT vocabularies; Diana Murray and the Royal Commission on the Ancient and Historical Monuments of Scotland for provision of their dataset; and Martin Doerr and Christos Georgis from the FORTH Institute of Computer Science for assistance with the SIS.

References AAT (2000) Art & Architecture Thesaurus hm://~ww.getty.edu~research~tooIs'vocabuIarv/aat~ Agosti, M., F. Crivellari, G. Deambrosis and G. Gradenigo (1993). "An architecture and design approach for a geographic information retrieval system to support retrieval by content and browsing." Computers, Environment and Urban Svstems 17: 32 1-335. Alani, H.. C. B. Jones and D. S. Tudhope (2001). "Voronoi-based region approximation for geographical information retrieval with gazetteers." International Journal of Geographical Information Science: accepted for publication. Beard, K. and V. Sharma (1997). "Multidimensional ranking for data in digital spatial libraries." International Journal of Digital Libraries 1: 153- 160. Couclelis, H. (1992) Location, place, region and space. Geography's Inner Worlds, RF. Abler, M.G. Marcus and J.M. Olson (eds), 2 15-233. Rutgers University Press, New Jersey. Curry M.R. (1996) The Work in the World - Geographical Practice and the Written Word. University of Minnesota Press, Minneapolis. Doerr, M. and I. Fundulaki (1998) "SIS - TMS: A Thesaurus Management System for Distributed Digital Collections". In Second European Conference on Research and Advanced Technologyfor Digital Libraries, ECDL198.Nikolaou, C. and Stephanidis, C. (eds), Heraklion, Crete, Greece, 215-234. Gould P. and R. White (1986) Mental Maps. Allen and Unwin, London. Guarino, N. (1997). Some organizing principles for a top-level ontology. Padova, National Research Council, LADSEB-CNR Int. Rep. 02/97. Guarino, N. (1997). Semantic Matching: Formal Ontological Distinctions for Information Organization, Extraction, and Integration. In M. T. Pazienza (ed.) Information Extraction: A Multidisciplinary Approach to an Emerging Information Technology. Lecture Notes in Computer Science 1299, Springer Verlag: 139- 170. Guarino, N., C. Masolo and G. Vetere (1999). " Ontoseek: Content-Based Access to the Web." IEEE Intelligent Systems 14(3): 70-80. Harpring, P. (1997). "Proper words in proper places: The Thesaurus of Geographic Names." MDA Information 2(3): 5- 12. Hill, L. L., J. Frew and Q. Zheng (1999). "Geographic Names. The implementation of a gazetteer in a georeferenced digital library." Digital Library 5(1): www.dlib.org~dlib/january99/hiIUO 1hill.htm1. Johnson, R.J. (1991) A Question of Place: Exploring the Practice of Human Geography. Blackwell. Jones, C. B., C. Taylor, D. Tudhope and P.Beynon-Davies (1996). "Conceptual, spatial and temporal referencing of multimedia objects". Advances in GIS Research II. M. J . Kraak and M. Molenaar (eds). London, Taylor and Francis: 33-46. Jordan T, M. Raubal, B. Gartrell and M.J. Egenhofer (1998) "An affordance-based model of place in GIs". Proceedings 8th International Symposium on Spatial Data Handling, T.K. Poiker and N. Chrisman (eds), International Geographical Union, 98-1 09. Kim, Y. W. and J.H. Kim (1990) "A Model of Knowledge Based Information Retrieval with Hierarchical Concept Graph". Journal of Documentation, 46(2), 113- 136. Larson, R., R. (1995) Geographic Information Retrieval and Spatial Browsing. In CIS and Libraries Patrons. Maps and Spatial Information, Linda Smith and Myke Gluck (eds), Urbana-Champaign : University of Illinois, 1996. @. 8 1- 124). Lee, J. H., M. H. Kim and Y. J. Lee (1993). "Information retrieval based on conceptual distance in IS-A hierarchies." Journal of Documentation 49(2): 113- 136. Moss A., E. Jung and J. Petch (1998) "The construction of WWW-based gazetteers using thesaurus techniques". Proceedings 8th International Symposium on Spatial Data. International Geographical Union, 65-75.

Geographical Information Retrieval with Ontologies of Place

335

Rada, R., H. Mili, E. Bicknell and M. Blettner (1989). "Development and application of a metric on semantic nets." IEEE Transactions on Systems, Man and Cybernetics 19(1): 17-30. Relph E. (1 977) Place and Placelessness. Pion Limited. Richardson, R., A. F. Smeaton and J. Murphy (1994). "Using WordNet for conceptual distance measurement". Information Retrieval: New Systems and Current Research: 100- 1 23. Rodriguez, M. A., M. J. Egenhofer and R. D. Rugg (1999). "Assessing semantic similarities among geospatial feature class definitions". Interop199.A. Vckovski, K. Brassel and H.-J. Schek (eds). Berlin, Springer. Lecture Notes in Computer Science 1.580: 189-202. Sintichakis, M. and P. Constantopoulos (1997). A method for monolingual thesauri merging. 20th Annual ACM SIGIR Conference on Research and Development in Information Retrieval, ACM Press, 129-138. Smith, B. (1 995). "On drawing lines on a map". Spatial Information Theory A Theoretical Basisfor GIs. A. U. Frank and W. Kuhn. Berlin, Springer. Lecture Notes in Computer Science 988: 475-484. Smith, B. and D. M. Mark (1998). "Ontology and geographic kinds". 8th International Symposium on Spatial Data Handling SDH'98, Vancouver, International Geographical Union, 308-320. Spanoudakis, G. and P. Constantopoulos (1994). Measuring Similarity Between Software Artifacts. 6th International Conference on Software Engineering & Knowledge Engineering (SEKE '94), Junnala, Latvia, 387-394. TGN (2000) Getty Thesaurus of Geographic Names. http://www.getty.edu/research~tools~vocabulary/tgn/ Tuan Ti-Fu (1977) Space and Place: the Perspective of Experience. Edward Arnold. Tudhope, D. and C. Taylor (1997). "Navigation via Similarity: Automatic Linking Based on Semantic Closeness." Information Processing and Management 33(2): 233-242. Tudhope, D., H. Alani, C. Jones (2001) "Augmenting thesaurus relationships: possibilities for retrieval". Journal of Digital Information Vol. l(8): http://jodi.ecs.soton.ac.uklArticles/v01/i08/rudhope/ Tversky, A. (1977). "Features of similarity." Psychological Review 84(4): 327-352. Walker, D., I. Newman, D. Medyckyj-Scott and C. Ruggles (1992). "A system for identifying datasets for GIs users." International Journal of Geographical Information Systems 6(6): 51 1-527.

Qualitative Spatial Representation for Information Retrieval by Gazetteers C. Schlieder, T. V¨ogele, U. Visser TZI - Center for Computing Technologies, University of Bremen, Universit¨atsallee 21-23, D-28359 Bremen, Germany Email: {cs|vogele|visser}@informatik.uni-bremen.de Abstract Intelligent and efficient information retrieval becomes increasingly important. Analogous to thesauri in the realm of spatial concepts, gazetteers offer a controlled vocabulary that can be used for spatial queries. Gazetteers use geographic footprints to link place names to geographic locations. Which geographic footprint representation is chosen has a strong impact on the quality of spatial queries. However, the footprint representations currently used in standard gazetteers such as points, lines, grid cell representations, and bounding boxes do not offer enough topological information to support refined spatial queries. We propose a new type of spatial footprint that can be described as a qualitative representation of the spatial decomposition of geographic entities. It holds enough topological and ordinal information enable refined spatial queries without being subject to the constraints of exact polygon representations. The proposed spatial representation was developed to be combined with terminological reasoning techniques used in systems for intelligent information integration.

Keywords: Spatial Reasoning, Gazetteers, Information Retrieval and Inference, Semantic Interoperability

1

Introduction

In recent years, a lot of effort has been put into the development of methods that support information retrieval from distributed and heterogeneous data sources based on thematic relevance criteria. A basic component of many approaches is the development of thematic thesauri and ontological representations [Stuckenschmidt et al., 2000]. They define a controlled and structured vocabD.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 336−351, 2001.  Springer-Verlag Berlin Heidelberg 2001

Qualitative Spatial Representation for Information Retrieval by Gazetteers

337

ulary that can be used both to annotate data sources with metadata and to provide search terms for information retrieval. However, thematic relevance is not the only criterion that can be used for information retrieval: most data objects have some sort of reference to objects in geographic space. These include ”real” objects, like cities, roads, rivers, mountains, as well as abstract objects like census areas, districts, or natural regions. In analogy to the thesauri used to define thematic concepts, a controlled and structured vocabulary of place names is needed as a basis for spatial information retrieval. This is done with the help of gazetteers. Gazetteers link the names of spatial objects to thematic concepts and to their spatial representation, the ”geographic footprint” [Hill, 1999]. In this paper, we focus on the representation of geographic footprints with a new method. In chapter 2, we have a closer look at gazetteers, their basic components, and the similarities and differences between gazetteers and standard geographic information systems (GIS). We examine the different methods used to represent geographical footprints, and point out their strengths and weaknesses with respect to the specification of spatial queries. In chapter 3, graph-based footprints are proposed as an extension to the repertoire of standard footprint representations. We briefly discuss basic aspects of the decomposition of geographic entities and the relevance criteria for spatial queries that can be extracted from such decompositions. Then we present a concept for the qualitative representation of such decompositions in form of a connection graph. We show how such footprints can be integrated in the existing gazetteer content standards to enhance gazetteer interoperability in the context of online information and data-brokering systems. In chapter 4, we discuss the interaction between spatial and terminological reasoning for information retrieval. In this context we introduce systems for intelligent information integration. Using a practical example, we show how a special data-brokering service and its capabilities for semantic data integration can be enhanced by an extended gazetteer.

2

Gazetteers

Generally speaking, a gazetteer can be described as a ”geospatial dictionary of geographic names” [Hill, 2000]. Hill identifies three core components of a gazetteer: a name, a location, and a type. Gazetteers can therefore be seen as tools that define geographic objects by linking their names to spatial and thematic concepts.

2.1

Query by Place Names

Names of geographic objects are typically expressed in plain language and may include variant names. Location information links the objects to geographic coordinates representing point, line, or areal objects. Type information is used to

338

C. Schlieder, T. Vögele, and U. Visser

classify the geographic object in terms of membership in a hierarchy of thematic concepts. The typical question that can be answered with the help of a gazetteer is: ”What information sources (e.g. electronic documents, databases) related to the place name X are available?” From this point of view gazetteers offer a subset of the functionality usually provided by geographic information systems (GIS). In fact, gazetteers can be seen as specialized GISs that are tailored to handle a number of very specific tasks: • Indirect georeferencing: Most of the the information available digitally has some reference to locations or objects in geographic space. However, only a fraction of it is properly georeferenced, i.e. indexed using a geographic coordinate system. In most cases, simple place names and other non-formalized descriptions are used. By linking place names and coordinates, gazetteers are an important tool for the retrieval of indirectly georeferenced information [Brandt et al., 1999]. • Vertical data integration: Standard GISs are optimized for horizontal data integration, i.e. the projection and display of data onto a horizontal plane. As a result of this paradigm, 2-D thematic maps are still the main output format used in standard GISs. However, vertical data integration, i.e. fast and simultaneous access to a large number of thematic properties of a given location, has become increasingly important as a key issue for information retrieval. Gazetteers are well equipped to perform such vertical data integration because they focus more on the retrieval of attribute data than on the exact representation of space. In addition, the database-type architecture of gazetteers facilitates the integration in online information systems. • Handling large data sets: Gazetteers are typically designed to handle very large numbers of geographic place names. The gazetteer used by the Alexandria Digital Library, for example, covers more than six million names in the U.S. and world wide [Hill et al., 1999]. The efficient handling of such large geographic data sets requires specific database solutions not available in standard GISs. As a result, gazetteers are effective tools to support information retrieval based on place names. In combination with thematic thesauri, they play an increasingly important role in the construction of online information and databrokering systems. In the following, we briefly describe some of the basic features of a gazetteer. We specifically look at the advantages and shortcomings of standard methods used to implement spatial representations for geographic footprints. Although much of the research in the area of gazetteer development and gazetteer integration has focused on the integration of place names and thematic type information, we claim that geographic footprint representations play an important role with respect to the useability and interoperability of gazetteers.

Qualitative Spatial Representation for Information Retrieval by Gazetteers

2.2

339

Geographic Footprint Representations

The second basic component of a gazetteer is a geographic footprint linked to the place name. Geographic footprints are formalized spatial representations of geographic gazetteer objects. Depending on the scale and type of the representation chosen, a geographic footprint is a more or less crude abstraction from reality. However, the type of footprint representation used to describe a geographic objects has a strong impact on the usability of a gazetteer and the quality of search results. Figure 1, for example, shows a polygon representation of the administrative districts of Bremen, a city in northern Germany. In a gazetteer, the districts would typically be represented with their names and a single point, thus using the lat/long coordinates of the polygon’s centroid as geographic footprint for the whole district. Such a simplifying representation could yield an answer to a simple query such as ”Which data objects are related to the place name ”Mahndorf”?”. It could also be used to find other districts within a certain buffer zone around the center point of the Mahndorf district. However, the degree of spatial relevance obtained by this method is limited because only those districts will be retrieved whose center points are located within the buffer zone. The result of the query will depend heavily on the specification of the buffer zone: If Figure 1: Districts in the city of Bremen the buffer was too small, many of the larger neighboring districts would be overlooked. If the buffer was too large, districts that are not direct neighbors of Mahndorf would erroneously be included as well. A more realistic representation of 2-D areal geographic objects would be to use bounding boxes. Bounding boxes are relatively easy to obtain and, because they allow to use efficient algorithms for inclusion and overlapping, they offer better tools for the selection of spatially relevant geographic objects. However, bounding boxes have some serious drawbacks: For one, depending on the geometry of the geographic object, bounding boxes tend to be much larger than the actual object. The bounding box of a specific district, for example, would cover a considerable part of the bounding box of another district as well. Secondly, the topological information represented by bounding boxes is rather imprecise: it cannot be concluded without doubt that the overlapping of two bounding boxes means the overlapping of the respective geographic objects. A third abstraction used by some gazetteers such as the GTU, the gazetteer

340

C. Schlieder, T. Vögele, and U. Visser

of the German Environmental Information Network GEIN (www.gein.de), is a grid-based representation. Geographical objects are described in terms of gridcell coverage, and objects sharing the same grid cell are defined to be spatially related. However, the approach depends heavily on the resolution of the grid. In addition, the fact that each geographic object can only have two states with respect to the grid cell in question, i.e. inside the grid cell or outside the grid cell, doesn’t allow us to specify refined spatial queries. Probably the optimal solution with respect to the accuracy of a query would be polygon-based footprint representations. Using exact polygon footprints, full-scale GIS functionality could be applied to select all geographic objects within a given region, to determine neighboring polygons, or to perform other complex spatial queries. However, exact polygon representations have a number of disadvantages that make their integration in a gazetteer problematic: • How exact is exact? Intuitively, one may think that the more detailed a polygon is, the better. However, there is some evidence that this is true only to a certain extent, and that the gain of more detail, for example by adding more and more vertices to a footprint-polygon, is outweighed by the loss of efficiency due to the increasing amounts of data that have to be handled. Hill [2000] proposes to use the attribute ”satisficing” as a criterion for the selection of footprint representations for specific purposes. ”Satisficing” describes the fact that in many real-world applications, the best practical solution is to find a compromise between the (technically) optimal solution for a given problem, and a minimization of the costs involved. Applied to the area of gazetteers, this means that for certain tasks, like information retrieval and metadata annotation, more or less abstracted footprint representations may be the best way to go. • Polygon maps are often proprietary data: A major problem with exact polygon footprint representations is that polygon maps are often proprietary and not meant to be distributed freely over the Internet. This can be due to secrecy reasons, like in the case of military sites or sensitive cooperate information, but most likely commercial interests are involved. A mapping agency like the cadaster office, for example, may not be willing to give away a detailed vector map of land use classifications without charging a hefty fee. • Unavailability of polygon data: For many geographic objects exact polygon representations are simply not available. For some objects, like for example historic regions, natural regions or colloquial names for geographic areas, the information necessary to create exact polygons does not exist. Such ”fuzzy footprints” cannot be properly expressed using a polygon representation. • Efficiency constraints: The processing and management of detailed polygon data is computationally intensive. Despite more and more efficient

342

C. Schlieder, T. Vögele, and U. Visser

can contain other polygons. In analogy to partonomies we introduce decompositions which are defined recursively as hierarchical data structures for encoding the spatial part-of relation together with the type of arrangement of the parts. Definition 3 A triple (P, r, D) is a decomposition of the polygon P if r is a relation symbol from {undecomposed, tesselation, patchwork, covering} and D = {D1 , . . . , Dk } is set of decompositions, where all Di = (Pi , ri , Di ) satisfy one of the following conditions: r = undecomposed and D = ∅; . . .; r = covering and covering(P, {P1 . . . Pk }). A decomposition is called homogeneous if it consists of a single type, that is, only one kind of relation symbol is used. By abstraction from the type of spatial arrangement one obtains the partonomy that underlies a decomposition. This partonomy is encoded by the decomposition tree which has the same nodes as the decomposition and whose edges denote the binary part-of relation between polygons (fig. 2).

3.2

Criteria for Spatial Relevance

A central idea behind gazetteers is that they give the user access to information items based not just on thematic but also on spatial relevance. This raises the computational problem of deciding which geographic footprints are relevant to a given footprint. Generally, the problem is solved by defining an appropriate metric on the space of geographic footprints. For points, a chessboard metric is easily obtained by superimposing a grid onto the map space. Points lying in the same grid cell as the given point (distance 0) are considered most relevant; next come points from the four immediately neighboring cells (distance 1). We analyze the different possibilities of defining a relevance metric on a polygonal decomposition. The analysis concentrates on the most important case, homogeneous decomposition by tessellations. In a homogeneous decomposition by tessellation two kinds of structure with spatial character interact. Firstly, there is the recursive structure of the decomposition reflected by the decomposition tree. Secondly, there exists a neighborhood structure which is due to fact that a polygon shares each of its edges or each of its vertices with at most one other polygon. In this paper, we focus on

Figure 2: Homogeneous decomposition by tessellation

Qualitative Spatial Representation for Information Retrieval by Gazetteers

343

direct neighborhoods, i.e. neighbors defined by shared edges. In the example, P6 , but not P5 , is a neighbor of P8 because these polygons possess a common edge. The neighborhood structure is expressed by a graph (fig. 3). Definition 4 The neighborhood graph of a homogeneous decomposition by tessellation is a graph N = (V, E) with the set of undecomposed polygons as nodes V and all pairs of neighboring polygons as edges E.

Figure 3: Neighborhood graph of a homogeneous decomposition into tessellations If there are no interesting information items linked to a polygonal footprint, a good place to search for further information are the neighboring polygons. Alternatively, one could search in those polygons that are part of the same decomposition. Obviously, this leads to two different criteria of spatial relevance. In other words, a spatial relevance metric can be based on either the decomposition tree or the neighborhood graph (fig. 3). We denote the graph-theoretical distance between two nodes Pi and Pj in the decomposition tree by δ(Pi , Pj ) and in the neighborhood graph by ν(Pi , Pj ). By definition, δ and ν are metrics. A combined metric for spatial relevance is obtained by using the linear combination aδ(Pi , Pj ) + (1 − a)ν(Pi , Pj ) in which the parameter a determines the proportion to which the criteria are effective.

3.3

Inferring Relevance from Spatial Neighborhood

Spatial inference consists in making implicit spatial information explicit. This does not necessarily require a logical framework. In fact, computational approaches for spatial reasoning often take a different approach to inference, for example constraint satisfaction (see [Cohn, 1997] for an overview of this line of research). Constraint-based spatial reasoning has been applied successfully to various GIS-related problems. We will therefore investigate whether this approach provides an adequate solution to inferences about spatial relevance. The basic inference problem derives from the standard way that gazetteers are used. A query is formulated which contains a place name. The user expects the system to return a ranked list of footprints, in our case polygons, which contain references to information items relevant to the place name in the query. Obviously, one footprint to return is the polygon that is designated

344

C. Schlieder, T. Vögele, and U. Visser

by the name. Other footprints are those that are close to this polygon with respect to the metric ν in the neighborhood graph. The elementary inference step consists in determining from information about the relative position of the polygons which are the neighbors, and which the neighbors of neighbors etc. of a polygon. This type of inference deals with topological information. Frequently, topological reasoning is formalized by representing the problem in the relational algebra of the region connection calculus (RCC, [Randell et al., 1992], see also [Egenhofer, 1991]) and using a finite domain constraint solver to compute the inferences. There are eight relations in the relational algebra of RCC describing the relative positions of two not necessarily polygonal regions: disconnected (DC), externally connected (EC), partially overlapping (PO), equal (EQ), tangential proper part (TPP), non-tangential proper part (NTPP), and the converse relations TPP−1 , and NTPP−1 . In homogeneous decompositions by tessellation, polygons are either disconnected or externally connected, that is, the only relations needed are DC and EC. The same is true if arbitrary regions are allowed as elements of the tessellation. The arrangement of five connected regions depicted in figure 4 could be represented by the following set of RCC-formula: EC(A, B), EC(A, C), EC(A, D), EC(B, C), EC(B, D), EC(B, E), EC(C, D), EC(C, E), EC(D, E). These formula specify the relation between nine of the ten pairs of regions. Only the relation between A and E is left unspecified. Can it be inferred in the calculus whether or not A is a neighbor of E? Intuitively, one might think so.

Figure 4: 2D and 3D models It is easy to see that if connected regions are arranged in the plane such that the formulas are satisfied then A and E must be disconnected. Formally, this follows from the fact that adding the edge AE to the neighborhood graph turns it into the complete graph K5 which is known to be non-planar. Since neighborhood graphs are planar, the graph with the edge AE cannot be a neighborhood graph. Unfortunately, it is not possible to obtain the result of this graph-theoretical argument with RCC. Assuming that A and E are connected, i.e. adding EC(A, E) to the formulas, does not lead to a contradiction in RCC. There are models of this set of formulas, e.g. the 3-dimensional arrangement of regions shown above which connects A and E by a bridge (fig. 4). The example

Qualitative Spatial Representation for Information Retrieval by Gazetteers

345

shows that RCC allows us to deal with topological inferences to the extent that they are valid for regions of arbitrary dimension. If the intended models are regions or, as in our case, polygons in the plane, then RCC turns out to be ”too incomplete” to provide satisfactory results.

3.4

A Qualitative Representation of Decompositions

From the example given in the previous section we can not only learn about the limitations of a particular topological calculus but also about the strength of graph-theoretical arguments in the context of reasoning about spatial neighborhood. It became clear, that an adequate qualitative abstraction from the spatial information of a homogeneous decomposition by tessellation must preserve at least the information encoded in the neighborhood graph. We will now show that the qualitative representation of the decomposition must preserve even more information, namely some type of ordinal information. If only the neighborhood graph is used, then the two arrangements of polygons shown below cannot be distinguished (fig. 5). Both have the same neighboring graph but they differ fundamentally with respect to neighborhood: neighbors of P1 and P3 can never be neighbors of P2 if the polygons are arranged as in (a) while they can in the arrangement (b). The problem is linked to multiple neighborhoods, that is, the fact that in (a) P1 and P3 have two disconnected edges in common. Therefore, the qualitative representation of the decomposition should be able to encode multiple neighborhood relations between two polygons.

Figure 5: Multiple neighborhood relations As solution to the problem of finding an adequate abstraction for a decomposition, we propose to represent it by a connection graph. Definition 5 The connection graph of a homogeneous decomposition by tessellation with neighborhood graph N = (VN , EN ) is a graph C = (VC , EC ) together with the combinatorial embedding of C in the plane. VC = VN ∪ {E} where E is the exterior, unbounded polygonal region. EC contains an edge (Pi , Pj ) for each connected sequence of polygon edges that Pi and Pj share. The combinatorial embedding of C consists in the circular ordering of the edges from EC at each vertex from VC .

346

C. Schlieder, T. Vögele, and U. Visser

Figure 6 shows the connection graph C of a homogeneous decomposition by tessellation D. Each polygon from D is represented by a vertex from C. In addition there is the node 1 representing the external polygonal region. The edges from C which are incident with a vertex are easily obtained together with their circular ordering by scanning the contour of the corresponding polygon. For polygon 10 the following circular sequence of neighbors is obtained: 1, 2, 3, 4, 6, 8, 9, 8. Note that polygon 8 appears twice in this list because it shares with 10 two disconnected polygon edges. On the other hand, polygon 9 which shares three edges with 10 appears only once because the three edges are connected. As the example shows, the connection graph is a multi-graph in which several edges can join the same pair of vertices. Technically speaking, the connection graph consists of the dual of the tessellation together with the combinatorial embedding of the dual.

Figure 6: Connection graph representation of a decomposition by tessellation The connection graph representation supports a number of graph-theoretical operations which can be used to draw inferences about spatial relevance (spatial neighborhood): • Finding polygonal footprints spatially relevant to a given footprint: By breadth-first search in the connection graph the distance of all polygons to the given polygon can be determined. More complex operations consist in graph-theoretical equivalents to morphological operators (dilatation, erosion, opening, closure). • Finding polygonal footprints spatially relevant to a given set of footprints: If the set contains just two footprints then polygons lying on the shortest path(s) in the connection path between the two footprints are good candidates for spatially relevant polygons. More complex operations like filling ”holes” in a set of polygonal footprints make use of the ordinal information given by the combinatorial embedding.

Qualitative Spatial Representation for Information Retrieval by Gazetteers

3.5

347

Application to Gazetteers

A metadata standard for gazetteers, the Gazetteer Content Standard (GCS) [Brandt et al., 1999], has been developed by the Alexandria-Digital-Libraryteam. Apart from the non-spatial attributes of a geographic feature, such as name, feature type information, and feature source information, the GCS also specifies how the spatial location of a feature, i.e. the feature’s footprint, can be described. In the GCS, spatial footprint representations can be specified either as bounding boxes, or as a ”detailed spatial geometry representation”. The latter includes points, linear features, bounding boxes, and polygons. Attributes to be listed include the number of points, the point order, and their lat/long coordinates. The qualitative representation we proposed, the connection graph, easily fits into the GCS. Each place name is associated with a footprint consisting of a vertex of the connection graph and a circular list of neighboring vertices. In this representation, polygons are used as footprints without specifying their metrical properties.

4

Extended Gazetteers and Semantic Interoperability

We have seen that extended gazetteers are able to reason spatially about place names. This makes them a valuable extension to information and data integration systems that so far focused mainly on terminological reasoning about thematic concepts. In this section, we discuss the relevance and applicability of our ideas with respect to intelligent information integration. There are several approaches for intelligent information integration, e.g. SHOE [Heflin and Hendler, 2000], KRAFT [Preece et al., 1999], OBSERVER [Mena et al., 1996], Buster [Stuckenschmidt and Wache, 2000], etc.). Here, we give insight in the Buster system (Bremen University Semantic Translator for Enhanced Retrieval).

4.1

The BUSTER Approach

Whenever a user sends a query to the information broker, the system has to decide which subset of a large number of pre-registered data sources contains data that may be able to satisfy the query. The Buster approach (www.semantictranslation.com) tackles this challenge by providing a common interface to heterogeneous information sources in terms of an intelligent information broker. The query is processed by an interaction of several components (see fig. 7). On the syntactic level, wrappers are used to establish a communication channel to the data source(s), this is independent of specific file formats and system implementations.

348

C. Schlieder, T. Vögele, and U. Visser

Figure 7: The Buster approach On the structural level, a mediator uses information obtained from wrappers and ”combines, integrates and abstracts” [Wiederhold, 1992] them. In the Buster approach, we use generic mediators which are configured by transformation rules. These rules describe the way in which data from several sources can be integrated and transformed to the data structure of original source in a declarative style. On the semantic level, we use two different tools specialized in solving semantic heterogeneity problems. Both of these tools are responsible for the context transformation, i. e. transforming data from a source-context to a goal-context. So far, the prototypical implementation of the system uses ontology-driven search. This means, that every source has to be formally described and annotated with formal semantics. Once this is done, the registered sources are comprehensive enough to fulfill the requirements of the above mentioned steps. At the moment, the Buster system is based on FaCT, a logical reasoner which can be used to check consistencies within ontologies and to compute subclass relations not explicitly contained in the ontology [Horrocks, 1999].

4.2

Extended Gazetteers and Information Retrieval

How can abstracted footprints as part of an extended gazetteer help us to solve spatial queries? The process of site analysis might serve as an example. For this, we connect the Buster system with our extended gazetteer approach. Our example is based on a few assumptions: The user consults the Buster system as a de facto information broker. Buster already has a few ontologies registered, amongst them an ontology about the ATKIS catalogue system (the official topographic cartographic information system in Germany). Also, the ATKIS data (or parts of them) are available as an extended gazetteer according to section 3.

Qualitative Spatial Representation for Information Retrieval by Gazetteers

349

One concept in one location Assume a user is looking for eligible industrial areas within a suburb in Bremen. He uses the Buster interface to define what he is looking for, e.g. the concept ’industrial area’ at the location ’Mahndorf’ (see 1 on page 4). The system is looking for eligible documents within the given frame but fails. The next step is that Buster is using its ontology about the ATKIS catalogue system and therefore can reason terminologically that the concept ’industrial area’ is subsumed by the Figure 8: Districts in the city of Bremen concept ’areas of commercial use’. The system can then search again and is looking for ’areas of commercial use’ in Mahndorf and might find eligible documents for the user. In addition, Buster can reason that the concept industrial area subsumes another concept, namely ’industrial site’. The system can also search for documents connected to this new concept in order to find some useful information. With the ATKIS data also available as extended gazetteers we can also check whether regions next to Mahndorf are of any relevance. One concept in more than one location If the reasoning and search processes described above retrieve no documents, the query may be extended by specifying ’industrial areas’ in ’Mahndorf’ and ’Hemelingen’ (see fig. 8). The system then would be able to reason that if these two areas are of relevance to areas in between these areas, e.g. ’Arbergen’ might be of interest as well. Here, the user benefits from spatial reasoning. In summary, we get documents not only about industrial areas in Mahndorf and/or Hemelingen, but also about industry and commercial sites in other relevant areas. This clearly demonstrates the benefits of a combination of terminological and spatial reasoning.

5

Discussion and Outlook

Gazetteers provide a controlled vocabulary of place names that can be used in particular for the otherwise difficult search through large repositories of indirectly georeferenced data. However, the very simplifying abstractions of geographic footprints used in state-of-the-art gazetteers impose serious constraints on the expressiveness of spatial queries. On the other hand, exact polygon representations, which could be used to define much more powerful queries, are difficult to integrate into the gazetteer architecture for various reasons.

350

C. Schlieder, T. Vögele, and U. Visser

We proposed a new type of footprint representation, in the form of connection graphs derived from a qualitative abstraction of tessellation polygons. We showed that these connection graphs hold enough topological and ordinal information to allow for expressive spatial queries. This representation is different to other approaches formalizing regions in a spatial hierarchy (e.g. [Timpf and Frank, 1997], [Remolina et al., 1999]) in several ways. An important difference is the encoding of ordinal information by specifying the combinatorial embedding. As shown in [Schlieder, 1998] fast graph-matching algorithms can be devised if the combinatorial embedding is known. A sequel paper will use the proposed representation to address the matching of connection graphs in the context of integrated gazetteers. This is particularly important with respect to gazetteer interoperability, and the simultaneous use of multiple distributed and heterogeneous gazetteers in online information brokering systems.

References [Bl´ azquez et al., 1998] Bl´ azquez, M., Fern´ andez, M., Garc´ıa-Pinar, J., and Gomez-Perez, A. (1998). Building ontologies at the knowledge level using the ontology design environment. In KAW’98, Banff, Canada. [Brandt et al., 1999] Brandt, L., Hill, L. L., and Goodchild, M. F. (1999). Digital gazetteer information exchange (dgie) - final report. In Digital Gazetteer Information Exchange Workshop. [Cohn, 1997] Cohn, A. (1997). Qualitative spatial representation and reasoning techniques. In Brewka, G., Habel, C., and Nebel, B., editors, KI-97, Lecture Notes in Articial Intelligence, pages 1–30. Springer. [Egenhofer, 1991] Egenhofer, M. (1991). Reasoning about binary topological relations. In G¨ unther, O. and Schek, H.-J., editors, Symposium on Large Spatial Databases SSD, volume 525 of Lecture Notes in Computer Science, pages 143–160. Springer. [Heflin and Hendler, 2000] Heflin, J. and Hendler, J. (2000). Semantic interoperability on the web. In Extreme Markup Languages 2000. [Hill, 1999] Hill, L. L. (1999). Foundations of shareable gazetteer data. In Digital Gazetteer Information Exchange Workshop. Transcribed and edited from audiotape. [Hill, 2000] Hill, L. L. (2000). Core elements of digital gazetteers: placenames, categories, and footprints. In Borbinha, J. and Baker, T., editors, ECDL 2000, Research and Advanced Technology for Digital Libraries, pages 280– 290, Lisbon, Portugal. [Hill et al., 1999] Hill, L. L., Frew, J., and Zheng, Q. (1999). Geographic names: The implementation of a gazetteer in a georeferenced digital library. D-Lib Magazine, 5(1).

Qualitative Spatial Representation for Information Retrieval by Gazetteers

351

[Horrocks, 1999] Horrocks, I. (1999). Fact and ifact. In Lambrix, P., Borgida, A., Lenzerini, M., M¨ oller, R., and Patel-Schneider, P., editors, Proceedings of the International Workshop on Description Logics (DL’99), pages 133–135. [Mena et al., 1996] Mena, E., Kashyap, V., Sheth, A., and Illarramendi, A. (1996). Observer: An approach for query processing in global information systems based on interoperability between pre-existing ontologies. In Proceedings 1st IFCIS International Conference on Cooperative Information Systems (CoopIS ’96). Brussels. [Preece et al., 1999] Preece, A., Hui, K.-J., Gray, W., Marti, P., Bench-Capon, T., Jones, D., and Cui, Z. (1999). The kraft architecture for knowledge fusion and transformation. In Proceedings of the 19th SGES International Conference on Knowledge-Based Systems and Applied Artificial Intelligence (ES’99). Springer. [Randell et al., 1992] Randell, D. A., Cui, Z., and Cohn, A. G. (1992). A spatial logic based on regions and connection. In Nebel, B., Swartout, W., and Rich, C., editors, Knowledge Representation and Reasoning KRR, pages 165–176, Cambridge. Morgan Kaufman. [Remolina et al., 1999] Remolina, E., Fernandez, J., Kuipers, B., and Gonzalez, J. (1999). Formalizing regions in the spatial semantic hierarchy: An ah-graphs implementation approach. In Freksa, C. and Mark, D., editors, Conference on Spatial Information Theory (COSIT), volume 1661 of Lecture Notes in Computer Science, pages 109–124, Stade, Germany. Springer. [Schlieder, 1998] Schlieder, C. (1998). Diagrammatic transformation processes on two-dimensional relational maps. Journal of Visual Languages and Computing, 9:45–59. [Stuckenschmidt and Wache, 2000] Stuckenschmidt, H. and Wache, H. (2000). Context modelling and transformation for semantic interoperability. In Knowledge Representation Meets Databases (KRDB 2000). [Stuckenschmidt et al., 2000] Stuckenschmidt, H., Wache, H., V¨ ogele, T., and Visser, U. (2000). Enabling technologies for interoperability. In Visser, U. and Pundt, H., editors, Workshop on the 14th International Symposium of Computer Science for Environmental Protection, pages 35–46, Bonn, Germany. TZI, University of Bremen. [Timpf and Frank, 1997] Timpf, S. and Frank, A. (1997). Using hierarchical spatial data structures for hierarchical spatial reasoning. In Hirtle, S. and Frank, A., editors, Conference on Spatial Information Theory (COSIT), volume 1329 of Lecture Notes in Computer Science, pages 69–83. Springer. [Wiederhold, 1992] Wiederhold, G. (1992). Mediators in the architecture of future information systems. IEEE Computer, 25(3):38–49.

Spatial representation and updating: Evidence from neuropsychological investigations Marlene ~ehrmann',and John philbeck2

' Department of Psychology, Carnegie Mellon University,

Pittsburgh, PA 15213-3890 USA [email protected] Department of Psychology, The George Washington University, 2125 G. Street, N.W., Washington, DC 20052 USA [email protected]

Abstract. How spatial information is represented and updated over time and over changes in the position of the stimulus and/or observer is considered in the context of a population of patients who have impairment in spatial perception. We present data both from our own research with patients suffering from hernispatial neglect as well as from studies in the literature. Taken together, these studies suggest that spatial information is coded in more than one spatial frame of reference simultaneously and that the choice of reference frame depends on the demands of the tasks. Once the stimulus is located, however, the patients are able to update the position of the stimulus dynamically when walking or when undergoing passive rotation. The insights obtained from this neuropsychological population provides converging evidence for the psychological and neural mechanisms which mediate spatial representation and dovetail well with existing single unit recording and functional imaging data. Keywords. Spatial representation, Spatial updating, Spatial reference frames, Neuropsychology, Hemispatial neglect

1 Introduction The simple act of reaching for a cup of coffee on one's desk is achieved effortlessly and rapidly. This apparently simple act, however, emerges from a host of complicated underlying computations, involving not only the perception of the object and derivation of motor commands to drive the muscles to the desired object but also the representation of the spatial position of the object and the updating of that position as the hand reaches out. The focus of this paper is on the psychological and neural mechanisms involved in the processes that are concerned with locating an object in space. The parietal cortex, which forms part of the dorsal cortical visual stream, is thought to be pre-eminent among cortical areas responsible for spatial perception. Single unit recording studies with nonhuman primates and functional imaging studies with humans provide strong support for this contention [4, 51. Adopting a converging approach, our research involves the study of individuals who, following a lesion to parietal (and often, adjacent) cortex suffer from hernispatial neglect. The critical aspect of this deficit is that the patients fail to orient towards or report information that appears on the conualateral side of space for recent reviews, [ l , 3,6] D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 352−370, 2001.  Springer-Verlag Berlin Heidelberg 2001

Spatial Representation and Updating

Patient A

Patient B

353

fl?

Fig. 1. Copies of a target clock and daisy by two patients with left-sided neglect, revealing the omission of features on the left.

Neglect occurs most frequently following lesions to the inferior parietal lobule and adjacent temporal cortex especially in the right hemisphere [8, 9, 121; hence, we refer to neglect as 'left-sided' throughout this chapter. Patients with neglect may fail to notice objects on the left of a scene, may ignore words on the left of a page or food on the left of a plate, and typically, omit to copy features on the left of a figure while preserving the corresponding features on the right (see Figure 1). They may also show neglect of contralesional information in other sensory modalities, such as audition, somatosensation and olfaction, and the deficit may even impair their ability to plan contralesional saccades or manual movements [5, 61. Importantly, the failure to process contralateral information is not attributable to a primary sensory or motor problem. Rather, neglect is thought to occur because neurons in one hemisphere have predominant, although not exclusive, representation of the contralateral side; removing neurons therefore impairs, to a greater extent, spatial representations for contralateral than ipsilateral positions [57,61]. In this paper, in the context of this neuropsychological population, we examine two central questions. The first is, When the patients ignore or neglect information on the contralateral left, what is it 'left' of? Because spatial position can not be defined absolutely, but only with respect to a set of coordinates or reference frame with an origin and axes, this question investigates the reference frame in which spatial position is defined such that, following damage, information to the left of its midline is neglected. We then ask whether the patients are able to update the position of the stimulus during various types of action such as locomotion or passive rotation. Answers to these questions may provide an understanding of the spatial maps coded in parietal cortex and the dynamic processes by which the maps are updated during action.

2 Spatial reference frames There are a number of potential reference frames that can be used to define positions in space. These can be divided into two broad classes: objects and locations can be

354

M. Behrmann and J. Philbeck

distance from fixation

Fig. 2. Schematic depiction of experiment on eye movements and reference frames. The subject is seated in arc of LEDS with a speaker used to emit auditory signals to help elicit and maintain subjects' fixation: (a) Baseline condition with midline of eyes, head and trunk aligned with the environmental midline; (b) Eyes right (ER) and (c) Eyes left (EL) with the midline of the eyes rotated 15' right or left but the midline of the head and trunk aligned with the environmental midline. The dashed line indicates the position of the eyes and the solid line the position of the head and trunk. (Adapted from [6]).

defined egocentrically, relative to the vantage point of the viewer, or allocentrically, from an extrinsic vantage point that is independent of the viewer's position.

2.1 Egocentric reference frames Many studies with neglect individuals have examined whether the information neglected is defined by an origin and axes aligned with the midline of (a) the eyes or vertical meridian of the visual field, (b) the head, (c) the trunk, or (d) the longitudinal axis of the limb that is involved in executing an action, such as the arm. To

Spatial Representation and Updating

355

determine the role of different reference frames in coding spatial position, the experiments typically probe the patients' ability to respond to a target that lies on the left or right of one midline which is rotated out of alignment from another. For example, in a recent experiment designed to examine the individual contribution of a reference frame centered on the eyes, we recorded the latency and accuracy of saccades in neglect patients to targets presented individually at 5, 10 or 15 degrees to the left or right of the midline of the eye [6]. In the baseline condition, the midline of the eyes was aligned with the midline of the head and trunk as well as the environment (i.e. subjects looked straight ahead; see Figure 2a). In two other conditions, the eyes were deviated to the right or left while the head, trunk and midline remained straight ahead (see Figure 2b and 2c). In the baseline, the detection of the targets fell along a gradient with best performance on the right and poorest on the left (of all reference frames). When the midline of the eyes was rotated out of alignment out of from the other midlines, latency (and also accuracy to some extent) was affected by the position of the stimulus, defined relative to the eyes: detection was good for targets to the right of the retinal axis and poor for targets to its left. Interestingly, there was further modulation of detection with the position of the eye in the orbit; when the eyes were deviated 15 degrees to the right, and the targets to the left of the fixation sampled, neglect was significantly ameliorated compared to when the eyes were straight ahead. When the eyes were deviated to the left, there was no change in performance, probably because these targets (right of fixation) are already detected well and there is no room for additional improvement. Evidence in support for retinocentric coding in neglect patients is also provided by many other studies [26,45,49, 671 and consistent evidence for an influence of line-of-sight or orbital position has been obtained both in humans [12] and animals [2, 31. A spatial code defined with respect to the midline of the head is still somewhat controversial. Although Karnath and colleagues [41,44] found no modulation of neglect with changes in head orientation in neglect patients, the combined influence of target defined retinally and modulated by orbital position reported above [6] provides some support for a coding with respect to the head. There is also support for a head-based reference frame in nonhuman primates [13]. Evidence for coding with respect to the trunk midline is more robust. Karnath and colleagues, for example, have argued that the midline of the trunk (body-centered reference frame) plays a fundamental (perhaps exclusive) role, serving as the anchor or midline for dividing space into left and right [41,42] [44]. In these studies, there was significant amelioration of neglect when the patient's trunk was rotated to the left compared to the baseline condition although, the neglect was not exacerbated by trunk rotations to the right, a result which they acknowledge is puzzling [see also [40] for further discussion and consideration of vestibular and optokinetic variables, and [29] for a more general evaluation of these findings]. Support for the role of the midline of the trunk is also obtained from studies by [lo, 171. Rather less research has been done to evaluate the role of the position of the limb on neglect performance. In one tactile exploration study, Bisiach and colleagues [12] manipulated the placement of the right limb such that the workspace of the limb either fell along the midline of the trunk or extended into the right side of space (as the board to be explored tactually was placed to the right). Performance did not differ in these conditions, suggesting that the limb coordinates are not crucial in affecting neglect (but

356

M. Behrmann and J. Philbeck

for affirmative evidence in monkeys, see [32]). A recent study, however, suggests that there may be some involvement of limb coordinates in neglect although this may primarily involve the spatial position of the limbs in relation to each other. Aglioti, Smania and Peru [I] applied bilateral stimulation to the dorsum of the hands when the hands were either placed straight ahead (anatomical position) or were placed one over the other. When the hands were crossed, the crossing could occur across the midline of the body or just in the right or in the left hemispace. Whereas the stimulus on the left-hand was not detected when the hands were in the anatomical position, there was both improved detection of the stimulus delivered to the left hand as well as poorer detection of the right hand stimulus in the crossed position, and this occurred irrespective of whether the hands were positioned on the left, right or across the midline of the trunk. These findings suggest that spatial position of a tactile stimulus delivered to one hand is coded with some sensitivity to the location of the other limb, and is independent of the midsaggital plane of the trunk. The studies reviewed thus far clearly point out the modulation of the severity of the neglect as a function of the gaze angle or line-of-sight, the midline of the trunk, and the position of the limb and perhaps, albeit to a lesser extent, by the midline of the head. Whether or not these various egocentric frames are truly separable from each other and, hence, independent, or whether they are contingent on each other to varying degrees still remains to be determined.

2.2 Allocentric reference frames Just as a number of different reference frames can be defined egocentrically, and can influence performance differentially, so too can different allocentric reference frames. Most research has focussed either on a reference frame defined with respect to the midline of a visual scene or environment, or on one defined with respect to the midline of individual objects or perceptual units in the scene. The derivation of an environment-centered frame requires computations involving gravitational forces on the otolith organ of the vestibular system, visual input to define environmental landmarks with respect to gravity, and proprioceptive and tactile information to provide a sense of the body's posture in relation to gravity. Mennemeier, Chatterjee and Heilman [47] have argued that the environmental frame is perhaps the most important reference frame, even more salient than a viewer-based, egocentric frame. Their conclusion is based on a line bisection study in which the environmental and bodycentered frames were brought into opposition through rotating the subjects' body in left, right, prone and supine positions. The critical finding was that the patients' bisections errors were predicted better by the environmental than body-centered frames, leading the authors to conclude that environment coordinates dominate in coding spatial position. In the last few years, considerable evidence has accumulated suggesting that spatial position may also be coded with respect to the midline of an individual object. The evidence comes from several studies showing that patients fail to report information appearing to the left of the object midline even when this information is located to the right of the midline of the viewer andlor the environment [ 7 , 9 , 22, 38, 53,721 but see [28] for contradictory evidence]. One of the earliest documented examples of object-based neglect is from patient NG, who had right-sided neglect and who failed to read the rightmost letters of a

Spatial Representation and Updating

357

word. This was true when the word was presented vertically, in mirror-reversed format and even when she was required to spell words backwards [15, 16, 361. Arguin and Bub [4] also showed that their patient's inability to report a target letter in a horizontal array of four elements depended on the object-relative position of the letter not the viewer-relative position. In a series of studies, Humphreys, Riddoch and their colleagues have also documented object-based neglect, showing that patients neglect letters positioned to the left of individual words [37, 38, 591. Interestingly, these same patients show neglect for information on the right in multiple-stimulus displays simultaneous with the object-based effects, providing support for accounts that posit the involvement of multiple spatial frames and coding between- as well as within-objects (see [34,63] for discussion of neglect dyslexia and other object-based neglect findings). Although all of the studies cited above use letters or words as stimuli, objectbased neglect has also been reported in studies that use other types of stimuli. For example, [72] reported that their patient performed poorly at identifying the left half of chimeric faces even when the faces were presented upside down and the relative left chimera occupied a position on the right side of space, again suggesting that the left of the object is disadvantaged even when it appears on the right of the viewer. The studies of Pavlovskaya et al. [53]and Grabowecky et al. [311 used geometric shapes and showed that information falling to the left of the center of mass of an object was less well detected than information appearing to the right. These data presuppose a computation of a center-of-mass that is specific to the object, the subsequent determination of the object midline, and the neglect of information to the left of this midline (see also, [21, 22,431. The failure to orient towards and process the left half of the chimera is also evident in eye movements; [69] reported that their patient, RR, restricted his fixations to the right side of an individual object. This object-based pattern could not be attributed to the failure to fixate the left of a display as RR could scan both the left and right of scenes and could also make left saccades when the left half of an object was presented in his left visual field [68, 701. The existence of an object-centered representation has not, however, gone without challenge. Driver and colleagues [23, 241 for example, have suggested that there is no need to invoke a reference frame that is tied to an individual object. Rather, they have argued that the left and right of an object may be coded solely from one's initial egocentric (and viewpoint dependent) encounter with the object. The claim is that when an object is viewed, left and right are assigned in a purely egocentric manner in accordance with the strength of an underlying attentional gradient [20]; for additional evidence of an attentional gradient, see (Kinsbourne 1993)l. A similar claim is made by Pouget and Sejnowski [56, 581 and by Mozer 1481 in their modeling work; because the left of the object always appears at the poorer end of the gradient relative to the right of the object, both in absolute and relative egocentric space, the ipsilesional information will always dominate over the contralesional information, which will then be neglected. This perspective suggests that object-centered coding is not necessary and that the same pattern of data may be obtained from simply assuming an egocentric gradient. Indeed, Mozer [48] has conducted simulations of so-called object-centered neglect in the context of a computational model, MORSEL, which assigns spatial position purely egocentrically (by virtue of a retinotopic attentional gradient) and does not have any object-centered representation. He shows that this implementation can account for a host of object-centered neglect effects

358

M. Behrmann and J. Philbeck

(4 Static condition

1250 -1

I Ed

Rotating

I

I

Moving condition

Initial presentation

-

Time

Left

Right

Side of target

Fig.3. (a) Depiction of the static and rotating conditions in the barbell paradigm with identical final displays. In the experiment, one circle of the barbell was colored red and one colored blue. (b) Mean detection time for 4 neglect patients for targets on the left and right in the static and moving conditions. Note that, because a fifth subject made so many errors, his data are not included here in the RT analysis, but his data reveals the same pattern with accuracy as the dependent measure (Adapted from 191.

(e.g., [4, 21,22, 531. In all of these cases, the left of the object always appears further left than the object right, both absolutely and relatively, and so is less activated. An experimental paradigm in which the left of the object does not always appear further left than the right of the object can, however, also reveal neglect [8,9, 651. In one such paradigm (see Figure 3), a barbell appears on a screen, with the left and right circles colored in blue or red (and the color remains constant for a single subject but is counterbalanced across subjects). In the first, static condition, a position on the right or left is probed and this position is both right and left in both viewer- and object-coordinates and serves as a baseline against which to compare performance in the second condition. In the critical, rotating condition, the barbell is previewed and then undergoes a rotation of 180 degrees so that the left, defined by the barbell, appears on the right of the viewer, and the right of the barbell appears on the left of the viewer. When a spatial position on the viewer-defined right or left is probed, both accuracy and speed of detection is influenced by whether this position occupies a right or left position, defined by the object. Thus, when the probe appears on the viewer's right but is on the left of the barbell (which rotated into that side), detection is poorer than when the position is both viewer-right and object-right. Similarly, when the probe appears on the viewer's left, detection is better when the position occupies the right of the barbell (which rotated into that side) compared to when it is both viewer-left and object-left. Support for a representation of spatial position, defined with respect to the midline of an individual object, has also been obtained from studies with animals. Olson and

Spatial Representation and Updating

359

colleagues obtained recordings of single neurons in monkeys who were required to move their eyes to the left or right of objects [50-521. Interestingly, the results showed that neurons in the supplementary eye field, a premotor area in frontal cortex, as well as in parietal cortex participate selectively when the monkey is planning to make an eye movement to the left of an object, while other neurons are activated when the monkey plans an eye movement to the right of an object. This object-based directional selectivity occurs regardless of the direction of the eye movement required and the retinal position of the object, regardless of the exact visual features of the object and regardless of whether the monkey was specifically following an object-centered instruction. These results point directly to a neural mechanism that might be responsible for locating positions in an object-based reference frame. Damage to neurons with object-left spatial selectivity would then give rise to the object-based neglect that is revealed by the patients; see [71] for review of spatial representation studies in monkeys.

3 Spatial updating Thus far, we have considered a rather contrived and artificial situation in which a stimulus is typically presented and remains stationary over the course of the trial, as does the observer (except for the barbell experiment where there is movement of the stimulus). Under more natural conditions, however, the observer typically interacts with the stimulus in some way by moving the eyes towards it or reaching out for it manually or the stimulus itself might move or shift position. Once the stimulus and/or observer are no longer static, the original spatial position of the stimulus must be updated with every movement in order to track the current position of the stimulus. A number of studies have suggested that the reference frame in which the stimulus is represented is linked to an output system which drives the relevant effector, and the updating of the target position is done in this same reference frame [18]. For example, for reaching, information is coded (and updated) in limbcoordinates 114, 331 whereas saccade-related information is coded (and updated) in oculocentric coordinates [25]. In an elegant single unit recording study, Duhamel and colleagues have shown that neurons in the lateral intraparietal area (LIP) which code spatial position retinotopically even respond to a target which is no longer present but which, with an appropriate eye movement, would have fallen within the receptive field of the neuron. Every time a monkey makes a saccade, the representation in LIP shifts into a new coordinate system whose origin is the postsaccadic center of gaze. The question then in the context of patients with spatial deficits is whether, once a stimulus is represented in a reference frame, its position can be correctly updated. If so, this would suggest that the problem arises in the initial representation of the stimulus but not in the ability to update it. There have, however, been some studies, which have sugges-tedthat there may be a specific problem in updating the stimulus position in such patients. For example, using a double-step saccade paradigm [26, 351 have shown that individuals with unilateral posterior parietal lesions were impaired at updating the eye position signal, but only when the eyes were directed into the contralesional hernispace (defined relative to the median sagittal plane of the head and body, which were aligned in this study). In the double-step saccade paradigm, the observer foveates a central fixation

360

M. Behrmann and J. Philbeck

point in a darkened room. Two visual targets are flashed in rapid succession above and to either side of the fixation point. After the flashes, the observer's task is to fixate the locations of the flashed targets in the order of presentation. Conditions in which both targets are on the same side of the fixation point allow generalized deficits in moving the eyes to the left or right to be measured and taken into account. In this paradigm, individuals with parietal lesions were unable to make accurate saccades to the second target when fixation of the first target made their eyes point into the contralesional hemifield. Significantly, these impairments were observed even though the required eye movement was toward the ipsilesional side (i.e., toward the hemifield that is typically not neglected by the patients), arguing against motoric neglect as the basis of the deficit. Thus, the parietal patients exhibited hemifieldselective impairments in updating the remembered location of the second target on the basis of the eye position signal (efference copy). One of the most complicated situations in which spatial position must be updated in the service of action is during walking and we have evaluated the ability of patients with parietal involvement to update stimulus position during locomotion. A great challenge in attempting to assess updating during locomotion is to control the enormous variety of stimulus information that can be used to determine one's position. Individuals can, for example, use vision to identify landmarks at known locations and then determine their position with respect to those landmarks. Humans, however, do not need to directly sense their external environment in order to keep track of their location during locomotion. Walking generates a rich source of information about self-motion in the form of limb movement, limb position, and vestibular signals. When walking along linear paths, in fact, humans are remarkably good at updating their location on the basis of these internally-generated signals, even when vision is obscured. After seeing a visual target up to 20 m away or more, the average person can walk to that target without vision and stop at its location with virtually no systematic error [19,27,46,55,60,62,64]. This basic result has important implications for studying the neural underpinnings of updating during locomotion. Because vision is obscured during the walk, landmark-based updating is not possible; the only sensory information available for updating comes from internally-generated sources, such as limb movement. Accuracy in the nonvisual walking task indicates that fundamental perceptual and cognitive processes are being performed well: in particular, the target's initial location was visually perceived and remembered, and nonvisual self-motion information during the walk was accurately registered. To explore the link between updating during locomotion and parietal cortex, we tested a group of 6 male, right-handed patients with lesions to the posterior parietal cortex [lo]. Except for one patient whose lesion involved a small portion of the superior PPC, the parietal involvement was confined to the inferior region. Four of the patients demonstrated mild hemispatial neglect (as determined by the Behavioral Inattention Test, BIT; [13]) while the rest performed within normal limits. Three walked well without assistance, and the rest usually walked with a cane due to mild hemiplegia. We also tested two groups of control participants, one with no history of neurological disease, and the other with brain injuries of various etiologies but no lesion in the parietal lobe. The primary purpose of this study was to test the ability of the parietal patients to walk to previously-viewed targets without vision. To perform this task well, individuals must not only visually perceive the target's initial location, but also sense their self-motion on

Spatial Representation and Updating

361

the basis of non-visual information. Either of these two subprocesses might be impacted by parietal damage and contribute to behavioral deficits, so, in addition to the visuallydirected walking task, we used a variety of other means to isolate the functioning of each subprocess individually. One important methodological concern was to insure that the participants did in fact use self-motion information when making their responses. It is possible, for instance, to waik accurately to a target by estimating (during the visual preview) the number of steps that it would take to reach the target; one would merely need to count off paces while walking blindfolded in order to reach the target-very little perception of self-motion would be required. We therefore discouraged pace-counting strategies in two ways. First, we asked observers to say a nonsense phrase aloud while walking, to prevent subvocal pace counting. Second, we varied the walking speed on each trial so that the observers could not predict with certainty how many paces might be required. The participants held onto the arm of an experimenter when walking; this provided physical support and allowed the experimenter to control the walking speed and direction. To confirm that the patients could visually localize the target, we obtained verbal estimates of the target distance. After the patients viewed the stimuli on these trials, we asked them to turn around and pick out a marker on a graduated scale on the floor in front of them that had the same distance as the target they just viewed. This task not only verified that the patients could remember the target locations, but also used a response that did not require verbalizing numbers. To test self-motion sensing in a task that did not require visual perception, we guided the participants along a straight path without vision and asked them to indicate the distance they had just walked. They did this by giving a verbal estimate and then attempting to reproduce that distance by walking (still without vision). As in the visually-directed walking task, participants repeated a nonsense phrase while walking and did not know what walking speed would be used on each trial. The study was conducted in a large, well-lit workspace and the target distances ranged from 2 to 6 m. A secondary interest in this study was to look for performance differences that depended upon the target's initial location to the left or right side of the body midline. Neglect patients show a decreased likelihood of detecting targets or events on the contralesional side of the body midline, but given sufficient time or a salient cue, they will eventually acknowledge the presence of targets in that region. For our purposes, it was essential that the patients detected the targets, so we verified this by asking them to call out the color of the target, which changed from trial to trial. However, we thought that even if a neglect patient successfully detected a target on the contralesional side, deficits in representing its location or updating while walking towards it might remain. To check this, we presented the targets 45 degrees to the left and right of straight ahead. To our surprise, the parietal patients performed quite normally on all tasks in this study (see Figure 4). Not only were they able to visually perceive the distance to targets, they were also able to walk that distance accurately without vision. Performance was equally good for targets on either side of the body midline; once the patients had detected the target, they apparently perceived its distance normally and had no trouble sensing their self-motion when walking toward it. The patients also performed normally when estimating walked distances in tasks that did not require visual perception. These results strongly indicate that the right hemisphere parietal cortex does not play a crucial role in sensing and integrating linear self-motion signals generated by locomotion.

362

M. Behrmann and J. Philbeck

b Vision / Verbal

Vision l Walking

Panetal Group Bra~n-Injured Control Group

d

c Proprioception / Verbal

Proprioception/ Walking

Stimulus Distance (m) Fig 4. Average indicated distances for the three subject groups: data from the four combinations of stimulus input modality (vision or proprioception) and response modality (verbal distance estimation or walked reproduction) are shown. Data have been collapsed over side of presentation as this factor had no effect on performance. Error bars denote +I- 1SD of the mean; solid horizontal lines indicate the physical stimulus distance. This study does not rule out a role for the parietal cortex in updating during locomotion entirely, however. Natural locomotion involves rotational body movements as well as linear translations, and the parietal cortex may be more specialized for updating during rotations. There have been several studies investigating this issue, but the results have been mixed 130,39, 661. The inferior parietal cortex borders a region thought to be a primary processing area for vestibular information; interpretation of studies investigating the role of the posterior parietal cortex for rotational processing can be complicated by the possibility that parietal lesions may also encroach upon the nearby vestibular cortical area. In a large study involving many patients and a variety of lesion sites, including the posterior parietal cortex and the vestibular cortical region, Israel and her colleagues (1995) concluded that the posterior parietal cortex is not critically involved in sensing rotations. Even this result, however, does not rule out a role for the posterior parietal cortex in updating during body rotations. When moving about in the world, we must not only keep track of our own body and its limbs, but also the positions of objects around us. We need to have this capability so that we can predict the future location of moving objects and keep track of objects as they pass out of view by being occluded or moving behind the head. The parietal cortex, therefore, may be important for making use of self-motion information-specifically for the purpose of updating the locations of objects external to the body.

Spatial Representation and Updating

363

But how can one distinguish self-motion updating from target updating, experimentally? They are indeed closely related processes, but if they are dissociable, one would expect that, for a given amount of perceived self-motion, some targets should be updated better than others. We studied a group of neglect patients to investigate this possibility [54]. Instead of asking the patients to update a target based on locomotor self-motion signals, we used passive, whole-body rotations as the stimulus to updating. We predicted that for a given magnitude of perceived rotation, targets would be updated more poorly if their updated trajectories carried them into the contralesional side of the body midline. We tested a group of 6 patients (average age 64 years), each of whom showed mild to moderate symptoms Five of the patients had right of neglect on a standard neglect battery, the BIT [13]. hemisphere injuries due to stroke, while the sixth had a left hemisphere tumor that had been surgically removed. We compared the performance of the neglect group with a group of neurologically intact control participants and a group of brain-injured control participants whose injuries spared the parietal lobe. The study was conducted in a well-lit indoor environment. The participants sat in a swivel chair in the middle of a square table (2.5 m on each side; see Figure 5). There were four possible target locations on the perimeter of the table, at 25 and 75 degrees left and right of straight ahead. To specify a target, a small light was flashed at that location. After acknowledging the presence of the target, the participant donned a blindfold and was passively rotated by an experimenter through an angle of 25 to 125 degrees. At the end of this rotation, the task was to use a manual pointing device, mounted on the chair just in front of the participant, to indicate the position of the target. We used a variety of control tasks to verify that each person could perceive the location of the target, remember it for several seconds, and use the pointer to indicate its location after a delay. Although the parietal patients sometimes took longer to localize the target on their contralesional side, once they had done so, they were able to use the pointing device to point to it accurately. We did notice that when the parietal patients attempted to point straight ahead after a passive rotation (relative to the patient's orientation at the end of the rotation), there appeared to be a slight shift in the perceived midline (about 5 deg) toward the ipsilesional side, but subsequent analyses showed that this did not have a large influence on the target updating data. The most striking finding was that although all the participants tended to underestimate the magnitude of the passive rotations to some degree, the neglect patients tended to underestimate much more so than the two control groups. These deficits, however, were not differentiated by the direction of the rotation. The pattern of underestimation is consistent with some previous work (e.g., [7]) and may be due to encroachment of the brain injury on the vestibular cortical region. We were more interested, however, in how well the patients could point to targets whose trajectories during the whole-body rotation carried them into the contralesional side of the patient's midline. The large underestimations in the magnitude of the rotations complicated this analysis, however. Even if a target's initial location was on the patient's "good" (ipsilesional) side and then physically crossed over to the contralesional side during the rotation, the patients had no knowledge of the physical crossover because their eyes were covered. Because of the underestimations in the magnitude of the rotation, they sometimes perceived the target to remain on the same side of the body midline even though it had physically crossed over. We therefore used the participants' pointing

364

M. Behrmann and J. Philbeck

Fig 5. Schematic diagram of the experimental apparatus, seen fi-om overhead. The observer sits in a swivel chair and uses a pointing device to indicate the direction to specified targets. The pointing device is mounted on the chair and rotates with it. The chair itself sits in the middle of a circular aperture cut into a square table. Target lamps are placed on the table surface at 25 and 75 deg on either side of the observer's starting orientation.

responses on each trial as a guide for determining when the target was perceived to cross the body midline. Using this scheme, we divided the data into broad categories according to how much of the target's trajectory was perceived to be on either side of the body midline (all, some or none). Contrary to our predictions, we found that the neglect patients did update targets more poorly when most or all of the updated trajectory lay on the contralesional side of the body midline. Once the patients had localized the target, they updated its location equally well, regardless of whether or not it passed into their "bad" (contralesional) side during the rotation. This indicates that the posterior parietal cortex does not play a crucial role in making use of information about body rotation to update the location of remembered targets. The absence of a spatial asymmetry when the body moves in space has also been obtained in two other, related studies. In one of these studies [2], patients were blindfolded, seated in a wheelchair and then wheeled along two sides of a triangle or rectangle. When the wheelchair was stopped, the patients were required to point back to the origin with a laser pointer. The results of this study did not show any

Spatial Representation and Updating

365

significant differences between neglect or control subjects in indicating the starting point when comparing the rightward and leftward path. In the other experiment [ l l ] , subjects were required to replicate the distance of rectilinear whole-body translations imposed by a specially designed robot. The participants could replicate a passively imposed whole-body displacement by moving the robot in the same direction (leftward-leftward, rightward-rightward and forward-forward) or in a different direction (leftward-forward; forward-rightward etc.). Again, there was no difference in the spatial reproduction by the patients with parietal lesions (with and without hemispatial neglect) and the control subjects. How can we reconcile the discrepancy between the neurophysiological findings indicating an involvement of parietal neurons in updating spatial infomation and the neuropsychological data from the double-step paradigm and the absence of an apparent updating deficit in the studies of parietal patients, as reviewed above. One possible explanation concerns the exact methods that are used: whereas the targets are not initially identified in the double-step paradigm, in the other updating studies, subjects know, at the outset, where the target is. For example, in the locoinotion study [lo], the subjects were required to identify the color of the target to ensure that they knew the spatial position that required updating. The same was true in the passive rotation study by Philbeck and colleagues [54] and in the origin pointing study by Bisiach and colleagues [ l l ] . Indeed, in order to conduct these studies, one has to ensure that the subject has coded the stimulus to be updated in the first instance otherwise the evaluation of updating is for nought. A possible reconciliation between the different findings, then, is that if the subjects have represented the targets adequately, they will be able to update the location. The deficit in the patients then appears to arise predominantly (although perhaps not exclusively) in the initial representation of the target in a particular reference frame. If the patient is assisted in doing so by cueing or verbal instruction, the spatial position of the target can be monitored and updated during action.

4 General discussion In this paper, we have addressed two related issues concerning how spatial information is represented and how it is dynamically monitored and updated as it changes either by virtue of its own movement or the movement of the observer. Our approach has been to investigate these questions in a population of individuals who, following damage to parietal cortex, are impaired at spatial processing. We have presented data from a series of our own studies of patients with hemispatial neglect and have reviewed others, all of which suggest that spatial position can be coded in parietal cortex in multiple reference frames, both egocentric and allocentric, and that the reference frame of choice may depend on the task demands or effector required for response. The data also seem to indicate that, provided that the stimulus is initially registered adequately, there does not appear to be an obvious problem in monitoring its changing position over time. These findings shed light on the nature of the spatial maps derived in human parietal cortex and dovetail well with concurrent findings from neurophysiological and functional imaging studies.

366

M. Behrmann and J. Philbeck

Authors' notes: This research was supported by NIH R 0 1 MH54246 to M.B. and by NIH grant F32 MH11791 and James S. McDonnell Foundation grant 97-17 to J.W.P. Part of this chapter was written while the first author received a Weston Visiting Fellowship at the Weizmann Institute of Science, Rehovot, Israel. Additional support for the sabbatical leave came from the James McKeen Cattell Foundation and the National Institutes of Mental Health.

References [I] Aglioti S, Smania N, Peru A. Frames of reference for mapping tactile stimuli in brain-damaged patients. Journal of Cognitive Neuroscience 1998; 11: 67-79. [2] Andersen R A, Essick G K, Siege1 R M. Encoding of spatial location by posterior parietal neurons. Science 1985; 230: 456-458. [3] Andersen R A, Snyder L H, Bradley D C, Xing J. Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Annual Review of Neuroscience 1997; 20: 303-330. [4] Arguin M, Bub D. Evidence for an independent stimulus-centered spatial reference frame from a case of visual hemineglect. Cortex 1993; 29: 349-357 . [5] Behrmann M, Ghiselli-Crippa T, Di Matteo I. Impaired initiation but not execution of eye movements in patients with hemispatial neglect. Behavioral Neurology 2001; in press:. [6] Behrmann M, Ghiselli-Crippa T, Sweeney J, Dimatteo I, Kass R. Mechanisms underlying spatial representation revealed through studies of hemispatial neglect. Journal of Cognitive Neuroscience in press. [7] Behrmann M, Moscovitch M. Object-centered neglect in patients with unilateral neglect: Effects of left-right coordinates of objects. Journal of Cognitive Neuroscience 1994; 6: 1-16. [8] Behrmann M, Tipper S P. Attention accesses multiple reference frames: Evidence from neglect. Journal of Experimental Psychology: Human Perception and Performance 1999; 25: 83-101. [9] Behrmann M, Tipper S P. Object-based attentional mechanisms: Evidence from patients with unilateral neglect. In: Attention and Performance XV: Conscious and nonconscious information processing, Umilta C Moscovitch M, editors. MIT Press: Cambridge, MA, 1994. p. 35 1-375. [lo] Beschin N, Cubelli R, Della Sala S, Spinazzola L. Left of what? The role of egocentric coordinates in neglect. Journal of Neurology, Neurosurgery and Psychiatry 1997; 63: 483-489. [ l l ] Bisiach E. The spatial features of unilateral neglect. In: Parietal lobe contributions to orientation in 3-D space, Karnath HO Thier P, editors. Springer-Verlag: Heidelberg, 1997. p. 465-495. 1121 Bisiach E, Capitani E, Porta E. Two basic properties of space representation in the brain: Evidence from unilateral neglect. Journal of Neurology, Neurosurgery and Psychiatry 1985; 48: 141-144. [13] Brotchie P R, Andersen R A, Snyder L H, Goodman S J. Head position signals used by parietal neurons to encode locations of visual stimuli. Nature 1995; 375: 232-235.

Spatial Representation and Updating

367

[14] Caminiti R, Johnson P B, Galli C, Ferraina S, Burnod Y. Making arm movements in different parts of space: the premotor and motor cortical representations of a co-ordinate system for reaching to visual targets. Journal of Neuroscience 1991; 11: 1182-1197. [15] Caramazza A, Hillis A E. Levels of representation, co-ordinate frames and unilateral neglect. Cognitive Neuropsychology 1990; 13: 39 1-446. [16] Caramazza A, Hillis A E. Spatial representation of words in the brain implied by studies of a unilateral neglect patient. Nature 1990; 346: 267-269. [17] Chokron S, Imbert M. Egocentric reference and asymmetric perception of space. Neuropsychologia 1993; 3 1: 267-275. [18] Colby C L, Goldberg M E. Spatial representations. In: Handbook of Neuropsychology, 4., Behrmann M, editor. Elsevier Science: North Holland, 2001. p. 45-65. [19] Corlett J. The role of vision in the planning and guidance of locomotion through the environment. In: Vision and motor control, Proteau L Elliott D, editors. North Holland: Elsevier Science, 1992. p. 375-397. [20] Driver J. Egocentric and object-based visual neglect. In: The hippocampal and parietal foundations of spatial behavior, Burgess N, Jeffery KJ O'Keefe J, editors. Oxford University Press: Oxford, 1999. p. 67-89. [21] Driver J, Baylis G C, Goodrich S, Rafal R D. Axis-based neglect of visual shape. Neuropsychologia 1994; 32: 1353-1365. [22] Driver J, Halligan P W. Can visual neglect operate in object-centered coordinates: An affirmative study. Cognitive Neuropsychology 1991; 8: 475-496. [23] Driver J, Pouget A. Object-centered visual neglect, or relative egocentric neglect. Journal of Cognitive Neuroscience 2000; 12: 542-545. [24] Driver J, Spence C. Attention and the crossmodal construction of space. Trends in Cognitive Sciences 1998; 2: 254-262. [25] Duhamel J R, Colby C L, Goldberg M E. The updating of representations of visual space in parietal cortex by intended eye movements. Science 1992; 225: 90-92. [26] Duhamel J R, Goldberg M E, Fitzgibbons E J, Sirigu A, Grafman J. Saccadic dysmetria in a patient with a right frontoparietal lesion: The importance of corollary discharge for accurate spatial behavior. Brain 1992; 115: 1387-1402. [27] Elliott D. The influence of walking speed and prior practice on locomotor distance estimation. Journal of Motor Behavior 1987; 19: 476-485. [28] Farah M J, Brunn J L, Wong A B, Wallace M, Carpenter P. Frames of reference for the allocation of spatial attention: Evidence from the neglect syndrome. Neuropsychologia 1990; 28: 335-347. [29] Farnk A, Ponti F, Lhdavas E. In search of biased egocentric reference frames in neglect. Neuropsychologia 1998; 36: 61 1-623. [30] Farrell M J, Robertson I H. The automatic updating of egocentric spatial relationships and its impairment due to right posterior cortical lesions. Neuropsychologia 2000; 38: 585-595. [311 Grabowecky M, Robertson L C, Treisman A. Preattentive processes guide visual search : Evidence from patients with unilateral visual neglect. Journal of Cognitive Neuroscience 1993; 5: 288-302. [32] Graziano M, Gross C G. Multiple pathways for processing visual space. In: Attention and Performance XVI, Inui T McClelland JL, editors. Bradford Book, MIT Press: Cambridge, MA, 1996. p. 181-207. [33] Graziano M S A, Yap G S, Gross C G. Coding of visual space by premotor neurons. Science 1994; 266: 1054-1057.

368

M. Behrmann and J. Philbeck

[34] Haywood M, Coltheart M. Neglect dyslexia and the early stages of visual word recognition. Neurocase 2000; 6: 33-43. [35] Heide W, Blankenburg M, Zimmerman E, Kompf D. Cortical control of double-step saccades - Implications for spatial orientation. Annals of Neurology 1995; 38: 737-748. [36] Hillis A E, Rapp B. Unilateral spatial neglect in dissociable frames of reference: a comment on Farah, Brunn, Wong, Wallace and Carpenter. Neuropsychologia 1998; 36: 1257-1262. [37] Humphreys G W, Riddoch M J. Attention to within-object and between-object spatial representations: Multiple sites for visual selection. Cognitive Neuropsychology 1994; 11: 207-24 1. [38] Humphreys G W, Riddoch M J. Separate coding of space within and between perceptual objects: Evidence from unilateral visual neglect. Cognitive Neuropsychology 1994; in press:. [39] Israel I, Rivaud S, Gaymard B, Berthoz A, Pierrot-Deseillgny P. Cortical control of vestibular-guided saccades in man. Brain 1995; 118: 1169-1183. [40] Karnath H 0 . Neural encoding of space in egocentric coordinates. In: Parietal lobe contributions to orientation in 3D space, Thier P Karnath HO, editors. Springer-Verlag: Heidelberg, 1997. p. 497-520. [41] Karnath H 0 , Christ K, Hartje W. Decrease of contralateral neglect by neck muscle vibration and spatial orientation of the trunk midline. Brain 1993; 116: 383-396. [42] Karnath H 0 , Fetter M, Dichgans J. Ocular exploration of space as a function of neck proprioceptive and vestibular input - observations in normal subjects and patients with spatial neglect after parietal lesions. Experimental Brain Research 1996; 109: 333342. [43] Karnath H 0 , Niemeier M. Task-dependent differences in the exploratory behaviour of patients with spatial neglect. submitted manuscript 2001; . [44] Karnath H 0 , Schenkel P, Fisher B. Trunk orientation as the determining factor of the contralateral deficit in the neglect syndrome and as the physical anchor of the internal representation of body orientation in space. Brain 1991; 114: 1997-2014. [45] Kooistra C A, Heilman K M. Hemispatial visual inattention masquerading as hemianopia. Neurology 1989; 39: 1125-1127. [46] Loomis J M, Da Silva J A, Fujita N, Fukusima S S. Visual space perception and visually directed action. Journal of Experimental Psychology: Human Perception & Performance 1992; 18: 906-92 1. [47] Mennemeier M, Chatterjee A, Heilman K M. A comparison of the influences of body and environment- centered reference frames on neglect. Brain 1994; 117: 10131021. [48] Mozer M C. Frames of reference in unilateral neglect and visual perception: A computational perspective. submitted manuscript 2001; . [49] Nadeau S E, Heilman K M. Gaze dependent hemianopia without hemispatial neglect. Neurology 1991; 41 : 1244-1250. [SO] Olson C. Object-based vision and attention in primates. Current Opinion in Neurobiology 200 1; 11: 171- 179. [51] Olson C, Gettner S N. Brain representation of object-centered space. Current Opinion in Neurobiology 1996; 6: 165-170. [52] Olson C R, Gettner S N. Object-centered directional selectivity in the macaque supplementary eye field. Nature 1995; 269: 985-988.

Spatial Representation and Updating

369

[53] Pavlovskaya M, Glass I, Soroker N, Blum B, Groswasser Z. Coordinate frame for pattern recognition in unilateral spatial neglect. Journal of Cognitive Neuroscience 1997; 9: 824-834. [54] Philbeck J W, Behrmann M, Loomis J. Spatial updating during self-rotations after right posterior parietal lesions. in Annual meeting of the Psychonomic Society. 1999. Los Angeles, CA. [55] Philbeck J W, Loomis J M. Comparison of two indicators of perceived egocentric distance under full-cue and reduced-cue conditions. Journal of Experimental Psychology: Human Perception & Performance 1997; 23: 72-85. 1561 Pouget A, Deneve S, Sejnowski T J. Frames of reference in hemineglect: a computational approach. In: Neural modeling of brain disorders: Progress in Brain Research, Reggia J, Ruppin E Glanzman D, editors. North-Holland: Elsevier, 1999. [57] Pouget A, Driver J. Relating unilateral neglect to the neural coding of space. Current Opinion in Neurobiology 2000; 10: 242-249. [58] Pouget A, Sejnowski T J. Spatial transformations in the parietal cortex using basis functions. Journal of Cognitive Neuroscience 1997; 9: 222-237. [59] Riddoch M J, Humphreys G W, Luckhurst L, Burroughs E, Bateman A. "Paradoxical neglect": Spatial representations, hemisphere-specific activation and spatial cueing. Cognitive Neuropsychology 1995; 12: 569-604. [60] Rieser J J, Ashmead D H, Talor C R, Youngquist G A. Visual perception and the guidance of locomotion without vision to previously seen targets. Perception 1990; 19: 675-689. [61] Rizzolatti G, Berti A, Gallese V. Spatial neglect: Neurophysiological bases, cortical circuits and theories. In: Handbook of Neuropsychology, 1., Boller F Grafman J, editors. Elsevier Science: North-Holland, Amsterdam, 2000. p. 503-538. [62] Steenhuis R E, Goodale M A. The effects of time and distance on accuracy of target-directed locomotion: Does an accurate short-term memory for spatial location exist? Journal of Motor Behavior 1988; 20: 399-415. [63] Subbiah I, Caramazza A. Stimulus-centered neglect in reading and object recognition. Neurocase 2000; 6: 13-30. [64] Thomson J A. Is continuous visual monitoring necessary in visually guided locomotion? Journal of Experimental Psychology: Human Perception & Performance 1983; 9: 427-443. [65] Tipper S P, Behrmann M. Object-centred not scene-based visual neglect. Journal of Experimental Psychology: Human Perception and Performance 1996; 22: 1261- 1278. [66] Tropper J, Mellvill Jones G, Bloomberg J, Fadallah H. Vestibular perceptual deficits in patients with parietal lobe lesions: A preliminary study. Acta Otolaryngologica (suppl.), 1991; 48 1: 528-533. [67] Vuilleumier P, Valenza N, Mayer E, Perrig S, Landis T. To see better when looking more to the right: Effects of gaze direction and frames of spatial coordinates in unilateraI neglect. Journal of the International Neuropsychological Society 1999; 5: 75-82. [68] Walker R, Findlay J M. Eye movement control in spatial- and obect-based neglect. In: Parietal lobe contributions to orientation in 3D space, Thier P Karnath HO, editors. Springer-Verlag: Heidelberg, 1997. p. 201-2 18. [69] Walker R, Findlay J M, Young A W, Lincoln N B. Saccadic eye movements in object-based neglect. Cognitive Neuropsychology 1996; 13: 569-6 15. [70] Walker R, Young A W. Object-based neglect: An investigation of the contributions of eye movements and perceptual completion. Cortex 1996; 32: 279-295.

370

M. Behrmann and J. Philbeck

[71] Wardak C, Olivier E, Duhamel J-R. Neglect in monkeys: effect of permanent and reversible lesions. In: The Cognitive and Neural Bases of Neglect, Karnath HO, Milner AD Vallar G, editors. Oxford University Press: Oxford, 2001. p. [72] Young A W, Hellawell D J, Welch J. Neglect and visual recognition. Brain 1991; 115: 51-71.

Mental Processing of Geographic Knowledge

1

Thomas Barkowsky Department for Informatics University of Hamburg Vogt-Kölln-Str. 30 22527 Hamburg, Germany [email protected] fax: +49-40-42883-2385 phone: +49-40-42883-2368

Abstract. The contribution presents a computational modeling approach to geographic knowledge processing in mind. Geographic knowledge is assumed to be stored in a piecemeal manner. Spatial knowledge fragments form a hierarchical structure of lean knowledge. An actual mental image representation is constructed when needed to perform a specific task. In this construction process missing information is complemented to create a determinate mental image. – First, the artificial intelligence perspective taken is elaborated. After a short review of conceptions on mental processing of spatial knowledge from psychology and artificial intelligence we outline the model MIRAGE. The internal structure and the operating of the model is elaborated using an exemplary scenario. Problems in constructing mental images from given pieces of knowledge are demonstrated and discussed. The paper concludes with a discussion of the approach with respect to its modeling objective. We point to further research questions and to potential applications.

Keywords. Cognitive maps, spatial knowledge construction, mental imagery, diagrammatic reasoning, experimental computational modeling.

1

The Construction of Geographic Knowledge

Mental representations of geographic or large-scale spaces are commonly referred to as cognitive maps (Tolman, 1948; Downs & Stea, 1977). However, numerous research findings in cognitive psychology have revealed that mental representations of large-scale spaces differ from maps in important respects. For example, mental representations of spatial knowledge are distorted, fragmented, and incomplete (for an overview, see Montello, 1992; Tversky, 1993; Hirtle & Heidorn, 1993). Processing geographic information can be described as a mental construction process in memory (Tversky, 1992; Portugali, 1996a) rather than a mere recall of static spatial information. This conception is supported by findings that indicate that mental 1

Support by the Deutsche Forschungsgemeinschaft (DFG) is gratefully acknowledged (grant Fr 806-8, Spatial Cognition Priority Program).

D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 371−386, 2001.  Springer-Verlag Berlin Heidelberg 2001

372

T. Barkowsky

representations often are not based on facts that are known, but rather on assumptions that are likely to be true and that are filled in during retrieval processes (Bransford et al., 1972; Sulin & Dooling, 1974; Intraub & Hoffman, 1992; Friedman & Brown, 2000). 1.1

Motivation

Although mental representations of spatial configurations are not map-like in a literal sense, it is generally assumed that spatial mental representations are organized in spatio-analogical form. There is neuropsychological evidence that the same neural subsystems are involved in mental reasoning about spatial configurations as for visual comprehension of external scenes. For example, thinking about geographic configurations activates the same neural systems as studying (external) geographic maps (Kosslyn, 1987; Kosslyn et al., 1994). Mental images (Finke, 1989; Kosslyn, 1980, 1994) are constructed in working memory when needed using pieces of information retrieved from long-term memory. The perspective on spatial knowledge processing as a construction process that involves mental images points to a very efficient way of dealing with geographic information. We identify the following features of mental image construction: (1) Mental representations of spatial configurations can be customized with respect to the specific task to be solved (which entities are involved, which scale and resolution is needed, what characteristics of the representation are needed etc.). (2) More or less scarce (or lean) knowledge about space can be efficiently stored in memory and can be used in a flexible manner. (3) Although adequate pieces of information may not be available in memory for all conceivable tasks, the construction on demand allows for compensating for missing information by using default knowledge. Default knowledge may fill gaps with details that are likely to be true. (4) Knowledge from different information sources and of different modalities can be combined in a unique representation. Especially, the two general types of knowledge distinguished in artificial intelligence (AI) and cognitive psychology, propositional and pictorial knowledge (Paivio, 1971; Larkin & Simon, 1987), are combined in a common representation in the mental image. Thus, the mutual advantages of both forms of representation can be exploited (cf. Freksa et al., 1999). (5) Both forms of knowledge may be used to exhibit information that is only implicitly contained in the knowledge stored in memory by constructing a quasipictorial representation for exploring the task to be solved. This characteristic is related to the ideas of diagrammatic reasoning in AI (Koedinger, 1992; Glasgow et al., 1995). The following example illustrates the construction of geographic knowledge in memory. 1.2

An Example

In a famous experiment aimed at exhibiting the hierarchical structure of human memory Stevens and Coupe (1978) asked the participants (students at University of California, San Diego) to decide about the relative orientation of some well known

Mental Processing of Geographic Knowledge

373

locations with respect to a global geographic reference system. For example, they asked which of the two cities Reno (Nevada) and San Diego (California) is located farther west. Presumably, the participants never had been explicitly asked this question before. Nevertheless they were able to answer the question. However, most participants answered that San Diego is farther west than Reno, which is not the case. Stevens and Coupe explained this effect by the fact that California is west of Nevada. They argued that the participants derived the relative location of the two cities from the relative position of the two states in which they are located. We will assume that the participating subjects had no explicit representation of the relative location of the two cities in their minds (otherwise they should have known the answer correctly). Therefore, to answer the question they had to construct a mental image of the relative position of the two cities on the basis of some other information available – here the relative orientation of California and Nevada. 1.3

An Artificial Intelligence Perspective

The above example points to an interesting class of phenomena in spatial cognition. We must frequently conceive of spatial configurations that we have never seen and that are not explicitly represented in memory. Nevertheless, we need a fast and pragmatic decision procedure. On the basis of some available information a tentative reconstruction of the real situation is built up to answer a question or to solve a task. From an AI point of view we would like to answer the question of how the cognitive processes and representations are structured and how they can be described in a computational model. In the present paper we report about an experimental computational modeling approach to answer this question. The three notions experimental, computational, and modeling will be further elaborated in the following. We are concerned with modeling, in so far as we want to provide a construction that maps certain types of phenomena to structural descriptions. The question is approached from an architect's point of view (cf. Braitenberg, 1984; Sloman, 1994): Which structures explain the behavior of the cognitive system in a given situation? For this purpose we must provide a 'metadescription' (Kosslyn, 1980) of the model to be designed. This metadescription must bridge the gap between the theoretical assumptions (e.g. derived from psychological findings) and the computational model. The resulting model is intended to serve as an embodiment of the underlying theories. This goal can be achieved by documenting the theoretical principles and their correspondence to components in the model. In this way, we separate between aspects in the model that are intended as literal modeling components and those which are needed to hold those components together in an implemented computer program. The modeling task is performed using a computational approach. This means that a system will be described by specifying representational structures and processes that can be implemented on a digital computer. As a consequence, the model's dynamic operation can be observed in the computer simulation. This is why the method is called experimental computational modeling: The observation of the running model allows for experimentation under various conditions. This enables critical reflection of the model's preconditions as well as of the computer implementation in a similar way as experimentation with human participants. Alternative design decisions can be tested to extend, to elaborate, and to refine the model. In comparison to an exclusively theoretical explanation of cognitive

374

T. Barkowsky

phenomena computational modeling provides a concrete realization of a dynamic system. This method forces us to completely specify every component of the model up to the degree necessary for computer implementation. A model built up in this way is open to criticism regarding the modeling decisions taken by the designer. Usually there is no definite reason for particular modeling decisions, as the observed phenomena do not determine the internal structure of a system (cf. Anderson, 1978). Nevertheless, a computational model provides a concrete embodiment of scientific conceptions that formerly existed as a bunch of – frequently disconnected – theoretical descriptions that each accounted for a different phenomenon. So a computational model is a specifically instantiated form of a scientific conception, and it provides a new basis for further discussions and explorations of a cognitive phenomenon. In the work described here we will provide the computational model MIRAGE that describes geographic knowledge processing in mental images. It starts from pieces of spatial knowledge stored in memory, describes the construction of quasi-pictorial representations in working memory, and deals with the exploration and refinement of the representations when required. The remainder of the paper is structured as follows. In Section 2 we provide a short review of existing conceptions on mental processing of geographic knowledge in cognitive psychology and AI. Section 3 presents an outline of the MIRAGE model and explains its substructures and the processes that operate on them. We discuss examples for mental image constructions. The paper concludes with a discussion of MIRAGE with respect to its modeling objective. We indicate essential issues for further research and point to promising perspectives for the application of the approach in intelligent spatial assistance systems.

2

Conceptions on Mental Processing of Geographic Knowledge

In this Section we review metaphorical conceptions on mental processing of geographic knowledge from cognitive psychology. The relation between mental models, human memory, and mental imagery is sketched out. AI approaches to spatial knowledge processing in qualitative spatial reasoning and diagrammatic reasoning are outlined. 2.1

Cognitive Maps and Other Metaphors

Metaphorical conceptions play an important role in scientific development. They allow for transferring well-tried ideas between research areas, they ease communicating about phenomena that are only roughly understood, and they are of importance in theory and model building processes (Kuhn, 1993; Hirtle, 1998). Numerous metaphorical conceptions have been proposed to capture the shortcomings of the cognitive map metaphor. Among them are spatial images and rubber sheet maps (Lynch, 1960), spatial schemata (Lee, 1968), environmental images (Appleyard, 1970), cognitive atlases (Kuipers, 1982; Hirtle, 1998), spatial mental models and cognitive collages (Tversky, 1991; 1993), (human) geographic

Mental Processing of Geographic Knowledge

375

information systems (GIS) (Peterson, 1995; Hirtle, 1998), and inter-representation networks (Portugali, 1996b). The most interesting metaphors in the present context are the cognitive collage metaphor, the cognitive atlas metaphor, and the spatial mental model conception. The former two emphasize that spatial mental representations are generally incoherent (involving different reference systems), spatially distorted, multimodal, hierarchically organized, and partially contradictory. The latter emphasizes the characteristic of mental representations of large-scale spaces as mental constructions. The GIS metaphor can be considered an extension of the cognitive atlas metaphor. Both involve structural aspects of internal representation (i.e., whether it resembles a raster or a vector representation format, cf. Couclelis, 1992), issues of varying accuracy, scale, and resolution, and the combination of partial representations held in different 'layers' of spatial information. The spatial mental model metaphor also relates to the construction of working memory representations for reasoning and problem solving; it is closely related to employing mental images for thinking about spatial configurations. 2.2

Mental Images, Human Memory, and Mental Models

Johnson-Laird (1983) proposed mental models to grasp mental reasoning processes that require the integration of a set of premises into a common representation to solve a given task. The representation is assumed to exhibit a representation structure analogical to the structure of the domain represented. For visual and spatial information (as well as for abstract information that can be mapped to a spatial structure) mental models are realized by mental images (Kosslyn, 1994). Mental images are evoked and operated with in working memory. Working memory for visual information according to Baddeley (1986) comprises a spatioanalogical representation structure (the visuo-spatial scratchpad) that is controlled by a central executive module (that also drives other short-term storage subsystems). So working memory for visual and spatial information comprises both a quasipictorial representation structure for short-term storage and a structure that holds the underlying facts and controls their treatment in the image proper. The facts that are used in evoking the mental image stem from long-term memory. They are retrieved for forming a mental image to reason about some question at hand. Retrieval from long-term memory is assumed to be done by activation of stored pieces of knowledge to make them vivid for subsequent usage, for example for mental imagery. Mental images comprise both, facts that are retrieved from long-term memory and inventions of states of affairs not explicitly contained in memory (Finke, 1989). As resulting from construction in working memory they are not retrieved as a whole from long-term memory. Regarding the type of knowledge processed they make use of pictorial as well as of propositional pieces of knowledge. 2.3

Spatial and Diagrammatic Reasoning

In AI, the mental capability to operate on spatial and pictorial structures has been adopted in qualitative spatial reasoning (QSR) (Freksa & Röhrig, 1993; Cohn, 1997; Vieu, 1997) and diagrammatic reasoning (DR) (Glasgow et al., 1995).

376

T. Barkowsky

Qualitative spatial reasoning investigates processing of spatial knowledge without relying on exact metric measurements. Humans usually deal with their spatial environment using qualitative rather than quantitative information (even though precise metric information may be available and is also utilized in reasoning tasks, cf. Montello, 1998). In qualitative spatial reasoning all types of spatial knowledge like topological information, orientation knowledge (directions), comparative distance information, and combinations of them are investigated. Besides the type of spatial relationships QSR also focuses on the ontological type of entities involved in spatial reasoning (e.g., whether entities are conceptualized as point-like or as spatially extended objects). Diagrammatic reasoning uses spatio-analogical representation structures to make use of the medium's properties for reasoning. These structures may be positional or relational (which refers to the conception of representing space per se versus representing objects in space, respectively). The core idea is that the properties of the spatial representation medium can be employed in reasoning: The spatial properties of the medium restrict the possible relationships between entities which may reduce reasoning to just representing (i.e., mapping to the spatial medium). A diagrammatic reasoning approach that explicitly refers to mental imagery is the computational imagery system by Glasgow and Papadias (1992). It distinguishes between a deep representation structure (related to long-term memory), a spatial and a visual representation structure (related to working memory and short-term representations, respectively).

3

The MIRAGE2 Model

This section presents the MIRAGE model that utilizes pieces of geographic knowledge for the construction of image-like representations. In doing so, it compensates for missing pieces of information by employing default knowledge for spatial properties and relations. The overall structure of the model is depicted in Fig. 1. MIRAGE is structured according to three memory substructures: non-activated longterm memory, activated long-term memory, and short-term memory3. Working memory is constituted by activated long-term memory and short-term memory. In Fig. 1 three subsystems of the model are identified: the long-term memory activation subsystem, the image construction subsystem, and the image inspection subsystem. All three systems operate in parallel. Thus further pieces of knowledge can be retrieved while an image is under construction on the basis of previously retrieved knowledge; or an image can be explored by the image inspection subsystem while the image is still being constructed or refined. The components of the three subsystems are described in the following.

2 3

MIRAGE stands for Mental Images in Reasoning About Geographic Entities. As retrieval from long-term memory is conceived of as being done by activation (cf. Section 2.2) two long-term memory systems are distinguished. This is done for functional reasons in the model and does not imply a physical information transfer. In the brain, information in long-term memory may be activated or not, thus belonging to one of the two systems distinguished in the model.

Mental Processing of Geographic Knowledge

activated LTM

non-activated LTM

short-term memory

spatial knowledge fragment

access

refinement

hierarchical LTM representation

377

construction

image construction

activated LTM representation

LTM activation

conversion

working memory

enriched representation

maintenance

visualization

visual buffer

inspection

inspection result

image inspection Fig. 1. Overview of the MIRAGE model with its three subsystems long-term memory (LTM) activation, image construction, and image inspection. The model is structured according to the three memory structures of non-activated LTM, activated LTM, and short-term memory (see text). The latter two constitute the working memory.

3.1

Long-term Memory Activation

The long-term memory activation subsystem comprises the underlying hierarchical long-term memory representation in non-activated long-term memory, which is utilized by an access process to obtain a spatial knowledge fragment. Spatial knowledge fragments are further processed by a construction process that builds up the activated long-term memory representation in working memory. The activated

378

T. Barkowsky

long-term memory representation forms the basic representation for the image construction process (see Section 3.2). Spatial knowledge fragments are represented by n-ary spatial relations between geographic entities that are uniquely identified. The relation is annotated by the type of spatial information and its degree of resolution. Currently, topological and orientation information (cardinal directions, cf. Frank, 1992) are represented as propositional knowledge, whereas shape information of extended objects is represented in pictorial form. Both knowledge types can be represented at different stages of resolution. The hierarchical long-term memory representation is a directed graph structure whose nodes are formed by identifiers of geographic entities and whose edges are the relations that hold between the respective entities. The edges encode the information represented in the spatial knowledge fragments. A visualization of an exemplary hierarchical long-term memory representation is given in Fig. 2a. a)

b) Nevada

U.S. intop,1 shape_NV

sha,1

Nevada

touchestop,1 ori,1

E

intop,1 shape_CA

sha,1

California

oto,1

in

top,1

Reno

NE

in

top,1

S_in

San Diego

intop,1 San Diego

in Reno

shape_NVsha,1

ori,2

W_in

California

Eori,1

top,1

oto,1

Nevada W_inoto,1 Reno

sha,1

shape_CA

ori,2

NE

touches

California top,1

S_inoto,1 San Diego

Fig. 2. Example of a hierarchical long-term memory representation (a) and two stages of corresponding activated long-term memory representations (b). Compared with the hierarchical LTM representation the activated memory representation only contains geographic relations relevant for the task to be solved (i.e., to determine the position of Reno with respect to San Diego). In the example, geographic entities are connected by topological and orientation relations.

The access procedure is modeled as a graph search procedure driven by (i) the type of relation wanted, (ii) the entities involved, and (iii) the hierarchical structure that is encoded in the long-term memory representation. So the access procedure tests the structure for a path between the entities in question that delivers that wanted type of relation. The spatial knowledge fragments encoded in this path are returned and passed to the construction process that builds the activated long-term memory representation. The activated long-term memory representation is defined like the hierarchical long-term memory representation with the following restrictions: The activated longterm memory representation only contains information relevant to the question at hand due to the access procedure; between two geographic entities there are no spatial relations of the same type at different levels of granularity. Figure 2b shows two stages of activated long-term memory representations corresponding to the hierarchical long-term memory representation shown in Fig. 2a. The activated long-

Mental Processing of Geographic Knowledge

379

term memory representation forms the basis for the image construction subsystem described in Section 3.2. The construction process builds an activated long-term memory representation from spatial knowledge fragments provided by the access process. To meet the specification of the activated long-term memory representation it checks for granularity conflicts between relations of the same type between a given set of geographic entities. 3.2

Image Construction and Image Inspection

The image construction subsystem starts with the activated long-term memory representation. It comprises a conversion procedure that yields the enriched representation. The enriched representation is the preliminary stage to the image generation proper. The image is generated in the visual buffer by the visualization process. The visual buffer is inspected by the inspection process to yield the inspection result. The enriched representation complements missing information in the activated long-term memory representation prior to visualization. It is defined like the activated long-term memory representation with the following additions: (i) Every spatially extended entity is assigned a specific shape; (ii) all spatial relations are complemented to enable an immediate visualization in the subsequent visualization process (see below). In the enriched representation no annotations of relational type or granularity are required. The conversion procedure builds the enriched representation from the activated long-term memory representation. First, for every spatial entity that does not come with a specific shape from long-term memory, an ontological type is assigned. To ease further processing, point-like entities are used as far as possible. Finally, further spatial relations are assigned for visualization. These relations are qualitative, i.e., no specific values are determined so far. Figure 3 shows two resulting enriched representations that correspond to the examples in Fig. 2b. a)

b)

default_shape Nevada

E_disjoint

default_shape California

shape_NV

centered_in centered_in Reno point-like

shape_CA

Nevada W_in

San Diego point-like

Reno point-like

NE_touches

California S_in San Diego point-like

Fig. 3. Two resulting enriched representations corresponding to the activated LTM representations shown in Fig. 2b. In both cases it has been determined whether geographic entities are point-like or extended. In (a) default shapes have been assigned to extended objects whereas in (b) shapes retrieved from LTM are employed. The relations that hold between the entities have been made determinate with respect to orientation and topology.

380

T. Barkowsky

The visual buffer models the quasi-pictorial medium that contains the image proper. In the model it is realized as an image representation in vector format. This format appears suitable to describe visuo-spatial working memory that exhibits behavior observed in humans; however, it is not intended to model structural properties of the visual buffer in the brain. The visual buffer forms the basis for the subsequent inspection process. The visualization procedure transforms the representation in the enriched representation and maps it into the visual buffer. This is done in a two step manner. First, specific (metric) values are assigned to the entities and the relations between them (image specification). This is done to prepare the subsequent second step of the visualization. In the second step the image is mapped into the visual buffer (image mapping). For this purpose, clipping and scaling must be performed on the representation generated in the first step. This is done because the visual buffer is restricted both in spatial extension and in resolution (Kosslyn, 1980; 1994). So to map the information of interest as suitable as possible for inspecting the information wanted an appropriate positioning and zooming has to be performed. When it cannot be computed in advance which parts of the image must be focused on in the visual buffer, the visualization process may be necessary to be performed in an iterated manner. Figure 4 shows two examples of the two steps of the visualization procedure (cf. Fig. 3a and b). In Fig. 4a and b default shapes (squares) are assigned to the extended entities; Fig. 4c and d show the two visualization steps with the proper shapes retrieved from long-term memory. a)

c)

10

b) California

Nevada

San Diego

Reno

0 20

10 Nevada Reno

10

California San Diego

0

20

d)

California

Nevada

San Diego

Reno

Reno Nevada California San Diego

10

Fig. 4. Two examples of the two subsequent steps (image specification and image mapping) of the visualization procedure. In (a) squares are assigned as default shapes, whereas in (c) proper shapes are employed. In both cases the objects' positions have been specified metrically. Figures (b) and (d) show the results of the image mapping step corresponding to (a) and (c), respectively, that generate the image in the visual buffer focusing on the entities Reno and San Diego.

Mental Processing of Geographic Knowledge

381

The image inspection process interprets the image representation in the visual buffer to yield the spatial relation wanted. For this purpose the graphical relations of objects in the visual buffer are translated to qualitative spatial relations. This is done in an iterated manner such that modifications in the visual buffer immediately result in a updated inspection result. 3.3 Dealing with Conflicting Situations in Mental Image Construction Visualization is not always as straightforward as demonstrated in the example. Figure 5 shows a more complicated scenario. In Fig. 5a the given activated LTM representation is depicted. In this example the orientation relation between the two cities Nice (France) and Geneva (Switzerland) is wanted. As can be observed, the representation structure is a cyclic graph structure, i.e., each spatial entity is restricted by spatial relations to two other entities. a)

b)

Lake_Geneva ori,1

ori,1

W

S

Geneva

Nice

top,1

c)

FR

top,1

in

in

Switzerland E

ori,1

Geneva

Lake G. CH Nice

FR

Geneva

e)

CH

CH

Nice

France

d) Geneva

Lake G.

f) Geneva

Lake G. FR Nice

Lake G.

FR Nice

FR

CH Geneva Lake G.

CH Nice

Fig. 5. A more complicated activated LTM representation (a) and five possible images (b-f) constructed according to the strategies described. None of them visualizes all five relations represented in the underlying representation.

When trying to visualize this scenario in a straightforward way as demonstrated in the previous section, we can see that no consistent visualization is possible. Depending on the order in which the entities and the relations between them is dealt with, five possible images can be constructed (Fig. 5b-f). None of them allows for integrating all relations represented in the underlying working memory representation. What are the consequences for answering the question using mental imagery? Three types of consequences are conceivable: (i) The image may be unstable; (ii) conflicting facts may have to be ignored and omitted from the image; or (iii) the preceding image generation steps have to be revised.

382

T. Barkowsky

An unstable image occurs when in a subsequent visualization step an object already contained in the image must be placed elsewhere due to additional spatial constraints. As image maintenance is performed by repeatedly updating entities depicted in the image (cf. Section 4.1) this would lead to an object being moved to another location. This behavior can be compared to the interpretation of 'impossible figures' used in empirical investigations in cognitive psychology (e.g., Schacter et al., 1991). When a spatial relation causes a conflict with already visualized entities the relation may be omitted. In this case the image remains incomplete and it cannot be detected which parts of the information cause conflicts and whether another solution might allow for the integration of all facts. However, under resource restrictions the strategy of omitting facts may be helpful. The revision of previous image generation steps is the most difficult but most promising option when a complicated situation needs to be solved. Modifications in the image generation strategies can be made both in the conversion procedure and in the image specification step of the visualization process. In any case this option is resource consuming and it is not clear in advance whether a solution (i.e. an adequate image) can be found at all. For each of the cases the following problems must be addressed: First, it must be detected that a complication has occurred in the image. Second, the cause of this complication must be assessed and options for solving the complication must be evaluated. Third, when having decided for a revision of the visualization strategies, the main difficulty is how the available resources in working memory can be efficiently utilized. The efficient use of chunking and intermediate storage facilities is known to be one of the crucial factors in efficiently employing visual mental imagery in complex situations (Kosslyn, 1994; Hegarty, 2000).

4

Conclusion and Outlook

The MIRAGE model is designed to mimic the characteristics of the construction of geographic knowledge representations in mind. This section reviews the model's features and relates it to principles in human mind that have been addressed in cognitive science research. We will point to open research questions and to potential further applications of the system. 4.1 Discussion The hierarchical LTM representation models a lean knowledge structure. Many relations between geographic entities are not represented. Since spatial knowledge usually is not acquired systematically, most pieces of geographic information needed in a specific situation are not stored explicitly in mind. Often spatial facts must be inferred from related information that is available. Fortunately, spatial facts and relations have a strong interdependence on various levels that makes the derivation of useful information from less suitable information possible. Nevertheless, also redundant knowledge is represented in the hierarchical LTM representation. For example, orientation information may be available at different

Mental Processing of Geographic Knowledge

383

levels of granularity. As the cognitive collage and atlas metaphors for mental spatial knowledge suggest, spatial knowledge may be available at different stages of accuracy and on different levels of resolution. The hierarchical LTM representation is built from spatial knowledge fragments, which form the elementary units of spatial knowledge in the model. They encode both geographic properties and spatial relations that hold between one, two, or more geographic entities. Knowledge for geographic entities is known to be fragmented in memory rather than globally coherent. The LTM representation in the model is organized in a hierarchical manner imposed by the type of relation encoded and the degree of resolution. From results in cognitive psychology it is known that facts stored in memory always are structured by some sort of hierarchy, be it imposed by a given superordinate structure (e.g., Hirtle & Jonides, 1985), be it constructed by the person according to some idiosyncratic principle (McNamara et al., 1989). The hierarchical organization enables retrieval processes that provide semantically related pieces of knowledge. In the model, retrieval is done by the access process that utilizes the hierarchical structure with respect to the problem to be solved. In MIRAGE's image construction subsystem spatial relations held in the activated LTM representation are complemented using default knowledge to obtain the required information from the underlying lean knowledge. This together with the image specification step in the visualization process relates to the categorial-to-coordinate conversion subsystems claimed in mental imagery (Kosslyn, 1994), which convert types of entities to exemplars for visualization. Compared to the underlying lean LTM representation the image representation in the visual buffer is fully instantiated. This relates to mental models (Johnson-Laird, 1983) in which humans for decision taking purposes instantiate just one potential solution rather than considering all possible solutions (cf., preferred mental models, Schlieder, 1999). Mental images require periodical rehearsal to prevent them from fading out. In the model, image maintenance is realized by iterated image constructions based on the activated LTM representation. The activated LTM representation is more persistent than the image (Kosslyn, 1994); so modifications in the underlying working memory representation cause the image to be updated in the next visualization step. In MIRAGE, the three subsystems LTM activation, image construction and image inspection - though they rely on each other - operate independently of each other and in parallel. Inside each of these systems operations are performed sequentially. This construction principle leads to an anytime characteristic of the model: An image represented in the visual buffer can already be used at an early stage of image construction, while the underlying image representation in working memory is still being updated and refined. 4.2

Further Research Questions

The architecture of MIRAGE has been devised as a modeling framework for describing imagery processes for mental reasoning about geographic configurations. Its conception for the use of mental images for constructing spatial representations exhibits a number of degrees of freedom that leave room for experimentation and that raise further research questions. The model presented does not provide answers to

384

T. Barkowsky

these questions, but it can be used to point out and further elaborate issues that may lead to a more accurate modeling conception. Most of the answers needed require further empirical investigations. Focusing on the image construction the following questions are of interest: • How is missing specificity compensated for in working memory prior to image construction? Especially, which shapes are employed when reasoning about extended entities, and how are qualitatively represented relationships specified to be mapped into the mental image? • When straightforward strategies fail, how are parameters modified and constraints relaxed, which alternative strategies can be employed, and how is the respective situation assessed to come up with alternative control strategies in the image construction processes? • What are the feedback structures that enable tuning between the interacting subcomponents for efficient strategies in complex image generation tasks? • What are the strategies that compensate for the resource restrictions in working memory when alternative images have to be compared, or when complex situations require more and more pieces of knowledge being included in the image? 4.3

Further Application Perspectives

MIRAGE's primary objective is in basic research in spatial cognition. However, there are also further application perspectives. Visual thinking in mental images can be improved using suitable external pictorial media. For example, people use paper and pencil to draw sketches for extending visual thinking processes in mind in the external medium. In the same way intelligent technical assistance systems can support and complement the processes of spatial reasoning with mental images. Capacity limits and other resource restrictions in working memory can be compensated for by external systems that assist reasoning processes involving visual mental images. Modeling imagery processes for reasoning about problems in large-scale space may serve as a first step in externally extending the internal pictorial space together with the intermediate reasoning processes involved. Application perspectives can be seen in the interactive presentation of geographic information provided by geographic information systems (e.g. for planning tasks), in interactive design systems, or in the cognitively adequate presentation of environmental information in spatial assistance systems (e.g. in tutorial systems or in wayfinding assistance).

Acknowledgments I would like to thank Mary Hegarty and Dan Montello who commented on earlier stages of this work. Special thanks are due to Christian Freksa whose critical comments and constructive suggestions helped to significantly improve the paper.

Mental Processing of Geographic Knowledge

385

References Anderson, J. R. (1978). Arguments concerning representations for mental imagery. Psychological Review, 85 (4), 249-277. Appleyard, D. (1970). Styles and methods of structuring a city. Environment and Behavior, 2, 100-118. Baddeley, A. D. (1986). Working memory. New York: Oxford University Press. Braitenberg, V. (1984). Vehicles - Experiments in synthetic psychology. Cambridge, MA: MIT Press. Bransford, J. D., Barclay, J. R., & Franks, J. J. (1972). Sentence memory: A constructive versus interpretative approach. Cognitive Psychology, 3, 193-209. Cohn, A. G. (1997). Qualitative spatial representation and reasoning techniques. In G. Brewka, C. Habel, & B. Nebel (Eds.), KI-97: Advances in Artificial Intelligence (pp. 1-30). Berlin: Springer. Couclelis, H. (1992). People manipulate objects (but cultivate fields): Beyond the raster-vector debate in GIS. In A. U. Frank, I. Campari, & U. Formentini (Eds.), Theories and methods of spatio-temporal reasoning in geographic space (pp. 65-77). Berlin: Springer. Downs, R. M., & Stea, D. (1977). Maps in minds: reflections on cognitive mapping. New York: Harper & Row. Finke, R. (1989). Principles of mental imagery. Cambridge, MA: MIT-Press. Frank, A. (1992). Qualitative spatial reasoning with cardinal directions. Proc. of the Seventh Austrian Conference on Artificial Intelligence, Vienna (pp. 157-167). Berlin: Springer. Freksa, C., Barkowsky, T., & Klippel, A. (1999). Spatial symbol systems and spatial cognition: A computer science perspective on perception-based symbol processing. Behavioral and Brain Sciences, 22 (4), 616-617. Freksa, C., & Röhrig, R. (1993). Dimensions of qualitative spatial reasoning. In N. P. Carreté & M. G. Singh (Eds.), Qualitative reasoning and decision technologies, Proc. QUARDET'93 (pp. 483-492). Barcelona. Friedman, A., & Brown, N. R. (2000). Reasoning about geography. Journal of Experimental Psychology: General, 129 (2), 193-219. Glasgow, J., Narayanan, H., & Chandrasekaran, B. (Eds.) (1995). Diagrammatic reasoning: Computational and cognitive perspectives. Cambridge, MA: MIT-Press. Glasgow, J., & Papadias, D. (1992). Computational imagery. Cognitive Science, 16, 355-394. Hegarty, M. (2000). Capacity limits in diagrammatic reasoning. In M. Anderson, P., Cheng., & V. Haarslev (Eds.), Theory and application of diagrams (pp. 194-206). Berlin: Springer. Hirtle, S. C. (1998). The cognitive atlas: using GIS as a metaphor for memory. In M. Egenhofer & R. Golledge (Eds.), Spatial and temporal reasoning in geographic information systems (pp. 267-276). Oxford University Press. Hirtle, S. C., & Heidorn, P. B. (1993). The structure of cognitive maps: Representations and processes. In T. Gärling & R. G. Golledge (Eds.), Behavior and environment: Psychological and geographical approaches (pp. 170-192). Amsterdam: North-Holland. Hirtle, S. C., & Jonides J. (1985). Evidence of hierarchies in cognitive maps. Memory & Cognition, 13 (3), 208-217. Intraub, H., & Hoffman, J. E. (1992). Reading and visual memory: Remembering scenes that were never seen. American Journal of Psychology, 105 (1), 101-114. Johnson-Laird, P. N. (1983). Mental models. Cambridge, MA: Harvard University Press. Koedinger, K. R. (1992). Emergent properties and structural constraints: Advantages of diagrammatic representations for reasoning and learning. AAAI Spring Symposion on Reasoning with Diagrammatic Representations, Stanford University, March 27-29. Kosslyn, S. M. (1980). Image and mind. Cambridge, MA: Harvard University Press. Kosslyn, S. M. (1987). Seeing and imagining in the cerebral hemispheres: a computational approach. Psychological Review, 94, 148-175.

386

T. Barkowsky

Kosslyn, S. M. (1994). Image and brain - The resolution of the imagery debate. Cambridge, MA: MIT Press. Kosslyn, S. M., & Shin, L. M. (1994). Visual mental images in the brain: Current issues. In M. J. Farah & G. Ratcliff (Eds.), The neuropsychology of high-level vision (pp. 269-296). Hillsdale, NJ: Lawrence Erlbaum. Kuhn, W. (1993). Metaphors create theories for users. In A. U. Frank & I. Campari (Eds.), Spatial information theory - A theoretical basis for GIS (pp. 366-376). Berlin: Springer. Kuipers, B. (1982). The 'map in the head' metaphor. Environment and Behavior, 14 (2), 202220. Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65-99. Lee, T. R. (1968). Urban neighborhood as a socio-spatial schema. Human Relations, 21, 241268. Lynch, K. (1960). The image of the city. Cambridge, MA: MIT Press. McNamara, T. P., Hardy, J. K., & Hirtle, S. C. (1989). Subjective hierarchies in spatial memory. Journal of Experimental Psychology: Learning, Memory and Cognition, 15 (2), 211-227. Montello, D. R. (1992). The geometry of environmental knowledge. In A. U. Frank, I. Campari, & U. Formentini (Eds.), Theories and methods of spatio-temporal reasoning in geographic space (pp. 136-152). Berlin: Springer. Montello, D. R. (1998). A new framework for understanding the acquisition of spatial knowledge in large-scale environments. In M. J. Egenhofer & R. G. Golledge (Eds.), Spatial and temporal reasoning in geographic information systems (pp. 143-154). New York: Oxford University Press. Paivio, A. (1971). Imagery and language. In S. J. Segal (Ed.), Imagery: Current cognitive approaches (pp. 7-32). New York: Holt, Rinehart & Winston. Peterson, M. (1995). Interactive and animated cartography. Englewood Cliffs, NJ: Prentice Hall. Portugali, J. (Ed.) (1996a). The construction of cognitive maps. Dordrecht: Kluwer Academic Publishers. Portugali, J. (1996b). Inter-representation networks and cognitive maps. In J. Portugali (Ed.), The construction of cognitive maps (pp. 11-43). Dordrecht: Kluwer Academic Publishers. Schacter, D. L., Cooper, L. A., Delaney, S. M., Peterson, M. A., & Tharan, M. (1991). Implicit memory for possible and impossible objects: Constraints on the construction of structural descriptions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17, 319. Schlieder, C. (1999). The construction of preferred mental models in reasoning with interval relations. In G. Rickheit & C. Habel (Eds.), Mental models in discourse processing and reasoning (pp. 333-357). Amsterdam: North-Holland. Sloman, A. (1994). Explorations in design space. In A. G. Cohn (Ed.), Proceedings of the 11th Conference on Artificial Intelligence (ECAI'94) (pp. 578-582). Chichester et al.: Wiley. Stevens, A., & Coupe. P. (1978). Distortions in judged spatial relations. Cognitive Psychology, 10, 422-437. Sulin, R. A., & Dooling, D. J. (1974). Intrusion of a thematic idea in retention of prose. Journal of Experimental Psychology, 103, 255-262. Tolman, E. C. (1948). Cognitive maps in rats and men. The Psychological Review, 55 (4), 189208. Tversky, B. (1991). Spatial mental models. The Psychology of Learning and Motivation, 27, 109-145. Tversky, B. (1992). Distortions in cognitive maps. Geoforum, 23 (2), 131-138. Tversky, B. (1993). Cognitive maps, cognitive collages, and spatial mental models. In A. Frank & I. Campari (Eds.), Spatial information theory (pp. 14-24). Berlin: Springer. Vieu, L. (1997). Spatial representation and reasoning in artificial intelligence. In O. Stock (Ed.), Spatial and temporal reasoning (pp. 5-41). Dordrecht: Kluwer Academic Publishers.

Spatial Cognition and the Processing of Verticality in Underground Environments Sylvie Fontaine Groupe Cognition Humaine, LIMSI-CNRS, BP 133, 91403 Orsay, France [email protected]

Abstract. Verticality is a relevant feature in many environments. Despite its relevance, little is known about how it is processed. In work conducted with the RATP, we approached this problem by comparing three forms of graphic aids intended to provide information about the vertical dimension of subway stations. These graphic representations were : floorplans of each level of the station, the same floorplans associated with a frontal view of the station and a three-dimensional axonometric representation of the station. Use of these representations was compared to learning of the station by navigation. Sixty four persons took part in the experiment. Participants had to perform routes, to locate landmarks and to compare distances. The axonometric representation was found to be the easiest to learn. The results also showed that this representation enabled individuals to perform well on the different tasks by allowing them to elaborate a correct mental representation of the vertical relations between the levels of the station, and between the underground and the outside. Thus, this kind of representation seems to be an efficient navigational aid, allowing users to achieve good planning for their displacements. Keywords : spatial cognition, verticality, navigational aids, underground and urban environments.

1 Introduction In most current research in navigation, space is approached in a two-dimensional way. This is appropriate in many contexts. In a city, buildings or districts are distributed along a two-dimensional surface. Changes in elevation are important and can be taken into account to control effort but they are not critical for localization. Nevertheless, localization in large three-dimensional spaces can be necessary. For example, in many situations, people have to deal with environments in which the third dimension, their verticality, is a relevant feature and has to be processed for planning displacements.

This research was conducted with the support of the RATP (Régie Autonome des Transports Parisiens). D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 387−399, 2001.  Springer-Verlag Berlin Heidelberg 2001

388

S. Fontaine

1.1 Previous studies In a previous experiment in underground environments, Fontaine and Denis (1999) showed that the vertical organization of a subway station is not easily understood by subway users. Participants learned routes beginning on a station platform and ending in the city by navigation. After learning, they were asked to describe the routes and to locate landmarks. They had to locate the arrival point (outside) in reference to the starting point (underground) and to locate the arrival point in reference to the exit stairs of the station (outside). Few participants succeeded in giving the correct orientation when the estimate was made from the platform. We suggested that participants did not have a single and global representation of the station but had two unconnected representations : one of the underground section and one of the outside section. So we decided to study the processing of verticality by comparing three kinds of graphic aids providing information on the vertical relations within the subway station and between the station and the city. As reported by some studies (Bronzaft, Dobrow, & O'Hanlon, 1976 ; Passini, 1984), the vertical dimension can be a source of difficulty in the processing of a space, and consequently a source of disorientation. Passini (1984) collected protocols from people wayfinding in multilevel shopping complexes. Participants clearly needed knowledge about vertical relationships to make decisions about the use of stairways and elevators. Two other studies examined specifically the processing of verticality, one took place in an open environment and the other in a building. Gärling, Böök, Lindberg and Arce (1990) wanted to know if elevation could be integrated in mental spatial representations. The studied city was built on a series of hills. Participants had to judge if one place in the city was above or below another one. Responses showed that elevation was actually encoded in mental representation and its integration was independent of the processing of distances. Montello and Pick (1993) examined the integration of verticality in a closed environment. The experiment took place in a multilevel building. Participants learned two routes by navigation, each one on a different level. The vertical link between the two routes was verbally described by the experimenter. Results showed that when verbal information on a vertical link was given, people could integrate the two routes within their configurational knowledge. These data suggest that people are able to build and to use integrated representations of spaces located on different levels. So, what about the processing of verticality in underground spaces ? 1.2 The studied station The studied station is the new station (Madeleine) which belongs to the 14th line of the Paris subway. Stations belonging to this new line are constructed according to new architectural concepts. The aims were to make the spaces more legible, allowing an easier comprehension of them. The architectural particularity of Madeleine is that it is constructed around a central cylinder from which several subway lines and several exits are accessible. This cylinder is open to all the accesses, so from the base of it, all the floors are visible.

Spatial Cognition and the Processing of Verticality

389

1.3 Three graphic representations Three types of graphic representations were tested : floorplans of each level of the station, floorplans of each level associated with a frontal view, an axonometric representation of the station. The use of these representations was contrasted with a learning experiment involving only navigation. Numerous studies have been interested in the use of graphic navigational aids but the studied aids are generally two-dimensional maps or floorplans. As far as we know, neither the use of a frontal view nor the use of an axonometric representation has ever been tested. So it is interesting to examine the comprehension and the use of these kinds of representations. On floorplans, no rendering of object volumes is given (an example of a floorplan is shown in Figure 1). Floorplans have to be integrated together in order that the vertical link emerges. The superposition of the boards can simulate verticality but the cognitive construction of this link requires effort and is difficult. When floorplans are associated with a frontal view, the vertical links are visualized (the frontal view is shown in Figure 2). The connection points between floorplans are more easily and more rapidly identified. Nevertheless, in this situation, the view remains split and hence requires effort to integrate. An axonometric representation is a two-dimensional representation of a three-dimensional object (see Figure 3). It is a mathematical construction which does not correspond to the perceived object (in opposition to a perspective representation). The main advantage of this representation is that it provides a global view of the whole environment at a single time. All the organization is visible : the horizontal organization, the organization in depth but also the vertical organization. Because the objects have a rendering of volume, their identification is easier. This representation does not correspond to what the individual perceives, however it does support the simulation of displacements.

Figure 1. Floorplan of the underground level –3 of the station

390

S. Fontaine

Figure 2. The frontal view of the station

Figure 3. The axonometric representation of the station

We hypothesised that tasks requiring a global representation of the station, such as navigation or object localization, would be better performed by individuals who learned the axonometric representation. The association with a frontal view was expected to favor the processing of floorplans. On the other hand, for estimations of distances, we expected better performances with floorplans because of the distance deformations in the axonometric representation.

Spatial Cognition and the Processing of Verticality

391

2 Method 2.1 Participants The participants were 32 men and 32 women aged 20-45. Each group of participants was divided according to their familiarity with the subway, i.e. 32 people familiar and 32 not. A participant was rated as familiar if he or she used the subway every day, and not familiar if less than once a month. Participants were distributed to four learning conditions : Floorplans, Floorplans with Frontal View, Axonometric Representation and Navigation. No one knew the station being studied. The participants were paid for taking part in the experiment. 2.2 Material The three kinds of graphic representations were presented on separated boards in A3 size. No color was used. Only shades of grey were added on the frontal view and on the axonometric representation to differenciate the underground from the outside. The same information was present on the three types of representations. On each representation, 7 landmarks were localized and identified by a name : - 4 underground landmarks : the ticket office, the middle of the platform, the mezzanine and the escalator. - 3 outside landmarks : the Eglise de la Madeleine, Le Café Madeleine and the store Fauchon. The floorplans Each level of the station was represented by a floorplan. On each floorplan were drawn the landmarks present at that level, including the stairs and the escalators. Five floorplans were drawn : from the outside level, that is the map of the city above the station, to the deepest level of the station. The frontal view All the levels were represented on the frontal view on a single board. The seven landmarks were identified at their respective levels. The axonometric representation The whole station and the city above the station were represented on a single board. The seven landmarks were identified at their respective levels. 2.3 Procedure The participants were tested individually. They were distributed according to the four learning conditions defining the type of representation given. During learning, participants had to learn the location of the seven landmarks within the station and outside. No limit of time was given. The learning was considered as finished if participants relocated correctly twice the seven landmarks on blank maps. For the navigation condition, the experimenter guided the participants in the station by showing and naming the different landmarks. All the participants were told to

392

S. Fontaine

learn the position of the seven landmarks but were informed that they would have to execute routes in the station. After the training, several tasks were given to the participants. Only three will be presented here. First, they had to perform two routes : one connecting two underground landmarks and the other connecting an underground landmark and an outside one. Then participants had to locate four landmarks in relation to reference landmarks. For each landmark to locate, they received a sheet on which was represented in the middle of a circle the reference landmark (taken among the seven). They indicated by a cross the position of the landmark to locate. Two landmarks had to be located in relation to a reference landmark placed in the same environment (underground/ underground or outside/ outside) and two landmarks had to be located in relation to the reference landmark placed in a different environment (underground/outside). Six comparisons of distances in relation to a standard distance were asked. The standard distance was the distance between the escalator and the end of the platform (on the platform of line 14). Participants made their comparisons with respect to the escalator. Hence during these comparisons, the standard distance was visible. They were asked to indicate if each distance was smaller or longer than the standard distance. Three of the distances were smaller than the standard distance and three longer. For each task, time and responses were recorded.

3 Results Three between-subject factors were considered for analysis : Learning conditions, Gender and Familiarity. Analyses were conducted both on times and scores. Two within-subject factors were added to provide further information concerning these analyses. 3.1 Learning Time The average learning time was about four minutes. The analysis of variance showed a significant effect of the learning condition (F(3,46)=16,55 ; p=1,98.10-7). Learning time for the axonometric representation was found to be the shortest (cf. Table 1). Only the difference between the Axonometric group and Floorplans+Frontal View group is significant (specific comparison : F(1,46)=5,62 ; p=0,02). No effect of gender or familiarity was found. Table 1. Learning time for each graphic representation (seconds) Floorplans Mean time

276

Floorplans + Frontal View 303

Axonometric representation 195

Spatial Cognition and the Processing of Verticality

393

3.2 Route Performance Time analysis Each of the two routes lasted about three minutes. The navigation group was the most rapid. No significant differences appeared between the other groups. For route 1, analysis of variance showed a significant effect of learning condition (F(3,46)=5 ; p=0,004). For route 2, a similar effect of the learning condition was found (F(3,46)=5,85 ; p=0,001). Analysis of responses Navigation performance was scored by measuring directional errors, hesitations and requests for help. The sum of these measures gave us a global error score. Participants exhibited few errors for the two routes. When the two routes were considered together, analysis of variance showed a marginally significant effect of learning condition (F(3,48)=4,42 ; p=0,07) (cf. Table 2). No global effect of gender or familiarity was found. Men tended to have more efficient displacements than women. Familiar participants tended to make less errors than unfamiliar participants. Table 2. Mean global error score Floorplans 2 routes combined Route 2

1.16 1.12

Floorplans + Frontal View 0.88 1.19

Axonometric representation 0.75 0.75

Navigation 0.23 0.26

As previously described, the first route connected two underground landmarks and the second route connected an underground landmark to an outside one. So we examined whether the involvement of the two environments could affect performance. For the first route, no effect of the different factors was found. The performance of the different groups were equivalent. For the second route different effects appeared. The learning conditions had a significant effect on performance (F(3,48)=4,42 ; p=0,007). The navigation group made less errors. The navigation group and axonometric group performed better than the two other groups (cf. Table 2). We observed a significant interaction between familiarity and learning condition (F(3,48)=2,93 ; p=0,04). Familiar participants have their best performances when they have learned the station by navigation and with the axonometric representation. Performances of unfamiliar participants changed little as a function of learned representation (cf. Figure 4).

394

S. Fontaine

2

Mean score

1.5

familiar

1

unfamiliar

0.5

0

Floorplans

Floorplans+Frontal View

Axonometric representation

Navigation

Learning groups

Figure 4. Mean global error of familiar and unfamiliar participants

Mean score

2 1.5

men

1

women

0.5 0 Floorplans

Floorplans + Frontal View

Axonometric representation

Navigation

Learning groups

Figure 5. Mean global error score of men and women Analysis showed also a significant interaction between gender and learning condition (F(3,48)=3,12 ; p=0,03). While women were less efficient than men when they learned floorplans, floorplans and frontal view, they became more efficient than men when they learned the axonometric representation (cf. Figure 5). 3.3 Localization of landmarks Time analysis We observed no significant differences between the four groups but the axonometric one tended to be the more rapid. No global effect of the different factors was observed. As we were interested in time differences between the four landmarks, we considered a new factor Landmarks defined by four modalities corresponding to the four landmarks. Hence we processed times as repeated measures. We observed that

Spatial Cognition and the Processing of Verticality

395

localization times were different as a function of environment (F(3,138)=22,32 ; p=7,5.10-2). Landmarks to be located were either in the same environment as the referent landmark or in a different environment (underground/outside). Specific comparisons showed that locating an outside landmark in relation to an underground one took more time (18 s) than locating a landmark in relation to a reference landmark placed in the same environment (9 s) (underground or outside) (F(1,46)=42,17 ; p=5,24.10-8). Analysis of responses The participants located the landmark in a circle representing all the possible locations, the reference landmark being the middle of the circle. To measure participants' performance, we considered two levels of success. Two points were given when a landmark was located in an error margin of plus or minus 20° and one point was given when a landmark was located in an error margin of plus or minus 40°. Outside these margins, no score was given. The mean score was 3 (in relation to the maximum score 8). The learning conditions had no significant effect but we observed the following tendency : the axonometric group and the navigation group had the same mean score and were more accurate than the other groups (cf. Table 3). Table 3. Mean score of the different learning groups for landmark localization Floorplans Mean score

2.50

Floorplans + Frontal View 3.13

Axonometric representation 3.81

Navigation 3.81

When mean scores were processed as repeated measures, the scores for the different landmarks were significantly different (F(3,144)=15,09 ; p=1,35.10-8). The same effect observed on times was found on scores. Landmarks placed in the same environment as the referent landmark were located significantly more efficiently than those placed in different environment (mean scores were respectively : 0.99 and 0.55) (F(1,48)=21,77 ; p=2,48.10-3). 3.4 Distance comparisons Time analysis Beyond the six distance comparisons, two concerned distances between two underground landmarks, two concerned distances between two outside landmarks and two concerned distances between underground and outside landmarks. For each of these pairs, one of the two distances was longer than the standard distance and the other shorter. We observed a significant effect of gender (F(1,46)=4,85 ; p=0,03). Women were more rapid than men (respectively 4.1s and 5.5s). Learning condition and familiarity had no effect. We also introduce here a new factor Distances defined by six modalities corresponding to the six studied distances. Hence time comparisons were processed as repeated measures. Times were found to be significantly different for the

396

S. Fontaine

six comparisons (F(5,230)=8,74 ; p=1,31.10-7). We compared the two kinds of judgements : longer than the standard distance / shorter. There was a significant difference between them (specific comparison : F(1,46)=16,29 ; p=0,0002). Judgements of smaller distances took less time (4 s) than longer distances (5.6 s). Analysis of responses The correct responses to the comparisons were added together in order to obtain a global score. The mean score for distance estimates was 3,6. Mean scores were processed as repeated measures. Learning conditions had a significant effect on the performances (F(3,48)=5,9 ; p=0,001). The Floorplans+Frontal view performed better than the other groups but significant differences appeared only with the navigation group (specific comparison F(1,48)=17,07 ; p=0,0001) (cf. Table 4). No global effect of gender or familiarity was found. Table 4. Mean score of the different learning groups for distance comparisons Floorplans Mean score

3.75

Floorplans + Frontal View 4.19

Axonometric representation 3.63

Navigation 2.88

When scores were processed as repeated measures, we observed that mean scores were significantly different for the six comparisons (F(5,240)=7,3 ; p=2,17.10-6) (cf. Figure 2). We also observed that the comparisons of distances which are shorter than the standard distance were made more efficiently than comparisons of longer distances (mean score were respectively 0.75 and 0.46) (F(1,48)=19,26 ; p=6,21.105).

4 Discussion In this section, we examine the results in relation to our hypothesis. We observed that, regardless of the learning conditions, navigation performance for the two routes was quite good. As expected, learning the station by navigation was the most efficient method for this task. Indeed participants wandered through the station and consequently were aware of the main routes in the station. Hence their task consisted essentially of recognising places and making appropriate decisions. On the other hand, the remaining groups had to match the representation of places in memory to the real places and they had to mentally build routes before executing them. Out of these three groups, individuals who learned the axonometric representation showed better performance than the others. The second route tested required a good representation of spatial relations between the underground and the outside. This second route seemed to be more discriminating than the first. The different observed interactions suggest that among the learned representations, some may be more appropriate for certain classes of participants. The representation which allows the most rapid and the most efficient displacements for familiar participants is the axonometric representation. Unfamiliar participants did not show such preference. Familiar participants seem to benefit more from the learned

Spatial Cognition and the Processing of Verticality

397

representations than unfamiliar participants. We can suggest that their familiarity with movements in subway stations helped them to elaborate from the graphic aids a more complete mental representation. Some differences appeared between men and women. While floorplans or floorplans combined with the frontal view allowed men to have the most rapid and the most efficient displacements, the axonometric representation helped womens’ displacements. For this task requiring an integration of the vertical relations between the underground and the outside, the axonometric representation was the most efficient aid for women. For the localization of landmarks, the results for the different learning groups are not clearly distinct. However, the axonometric group tended to be the most rapid and the most efficient to locate landmarks. The results for performing routes and for locating landmarks showed that the vertical relations were integrated and understood. This suggests that participants have elaborated a global representation of the station. Analysis also showed that landmarks placed in the same environment as the reference landmarks were located more rapidly and more efficiently than landmarks placed in different environments. This suggests that even if individuals have elaborated a global representation of the station, the transition between the underground and the outside requires effort and remains difficult. The vertical links between the underground and the outside are known and represented but their use requires more time and more processing than links between different levels in the station. We can suppose that locating an underground landmark in relation to an outside one requires one to make intermediate levels "transparent" in order to make visible only the two landmarks concerned in a global frame of reference. As we expected, the floorplans group and the floorplans+frontal view group performed better than the others for the distance comparisons. This suggests that the deformations of distances on the axonometric representation affected the mental representation of distances. This effect remains even though participants had experimented with distances during the navigation tests. We observe that the navigation group had the lower score for this task. This group, through learning, had experienced the different distances directly. Perception of distances underground may be affected by the feeling of enclosure and by the limits of the visual field. Moreover, we observed that distances smaller than the standard distance were processed more rapidly and judged more efficiently than distances longer than the standard distance. These data suggest that individuals tend to underestimate distances in these environments.

5 Conclusions In most studies on wayfinding, authors emphasize the impact of the environment on navigation performance. The performance of our navigation group reinforces suggestions by Chown, Kaplan, and Kortenkamp (1995) and Heft (1996) who consider a relational and reciprocal perspective between the individual and the environment. Indeed, it appears that the structure of the environment has a central and specific role in the elaboration of a mental representation. The efficiency of

398

S. Fontaine

displacements in the Madeleine station, at least for the navigation group, is due to the fact that environmental features of this station correspond to those features that favor navigation according to Weisman (1981). Here are some illustrations. In this station, there is a large visual access which provides a good view of landmarks in the station. The mezzanine allows a view of the platforms. The floors in the central cylinder provide a global view of the different levels and their localization. Moreover, architectural differenciation is a major element according to Weisman (1981), which helps recognition of landmarks and areas. The central cylinder in Madeleine station is an example of architectural differenciation which constitutes a relevant element. This feature allows one to link the different areas in the station and breaks the uniformity of space. As Arthur and Passini (1992) suggested, built environments and their different parts must operate like a communication device. It seems that the structure of the Madeleine station operates like this. Resolution of the localization and distance comparison tasks confirms that metric and topological information are present within the mental spatial representation. Our data suggest that information concerning the vertical dimension can also be integrated in spatial mental representations in at least two contexts. On the one hand, verticality can be integrated when this information is learned first from graphic representations which show the vertical organisation of the environment. On the other hand, the vertical dimension can be integrated when the space is sufficiently legible, such as when the architectural structure allows one to identify the vertical relations. We suggest that in order to elaborate a global mental representation, two steps must be carried out to integrate the vertical dimension. First, in the navigation space concerned here, it is necessary to integrate elements of the environment (landmarks, routes) in the plane. Second, the different planes must be integrated together according to the vertical relations between them. These two steps make the processing very demanding. In subway stations, the presence of signs could free individuals from engaging in such processing. As verticality is little studied, it would be interesting to develop our investigations for other environments and with other procedures such as spatial priming.

6 References 1. Arthur, P., & Passini, R.E. (1992). Wayfinding: People, signs, and architecture. McGraw-Hill Ryerson, Toronto. 2. Bronzaft, A.L., Dobrow, S.B., & O'Hanlon, T.J. (1976). Spatial orientation in a subway system. Environment and Behavior, 28, 185-203. 3. Chown, E., Kaplan, S., & Kortenkamp, D. (1995). Prototypes, location and associative networks (PLAN) : Towards a unified theory of cognitive mapping. Cognitive Science, 19, 1-51. 4. Fontaine, S., & Denis, M. (1999). The production of route instructions in underground and urban environments. In C. Freska & D.M. Mark (Eds), Spatial information theory: Cognitive and computational foundations of geographic information science (pp. 83-94). Berlin: Springer. 5. Gärling, T., Böök, A., Lindberg, E., & Arce, C. (1990). Is elevation encoded in cognitive maps. Journal of Environmental Psychology, 10, 341-351.

Spatial Cognition and the Processing of Verticality

399

6. Heft, H. (1996). The ecological approach to navigation: A Gibsonian perspective. In J. Portugali (Ed.), The construction of cognitive maps (pp. 105-132), Kluwer Academic Publishers. 7. Montello, D. R., & Pick, H. L., Jr. (1993). Integrating knowledge of vertically aligned large-scale spaces. Environment and Behavior, 25, 457-484. 8. Passini, R.E. (1984). Spatial representation, a wayfinding perspective. Journal of Environmental Psychology, 4, 153-164. 9. Weisman, J. (1981). Evaluating architectural legibility. Wayfinding in the built environment. Environment and Behavior, 13, 189-204.

Grid Patterns and Cultural Expectations in Urban Wayfinding Clare Davies1 and Eric Pederson2

1

University of Surrey, Dept of Psychology, Guildford GU2 7XH, UK University of Oregon, Dept of Linguistics, Eugene OR 97403, USA [email protected] and [email protected] 2

Abstract. Much of the literature on human spatial cognition and language in large-scale environments has been based on 'simplified' grid-pattern layouts with orthogonal intersections and parallel paths/streets. However, these are not the prevailing urban structure in many countries. This field study considered the possibility that different cultural expectations for typical urban environments would affect even long-term residents' mental models and behavior regarding urban wayfinding and locational knowledge. Residents of two grid-pattern cities, one in the UK, where such layouts are rare, and another one in the US, performed a battery of tasks including confidence ratings, sketch map drawing, verbal route directions, and pointing to non-visible landmarks. The results show that the UK group placed less emphasis on the central grid in their sketch maps, and showed a systematic error in their pointing direction. The results are discussed in the light of previous research on orientation biases. Further crosscultural analysis and studies are planned. Keywords: route directions, mental maps, cognitive mapping, urban navigation, cross-cultural psychology, pointing tasks, landmarks, cardinal directions

1

Introduction

Within psychology and cognitive geography, urban wayfinding holds particular importance, not only to theoreticians modeling spatial cognition, but also as an application of our theoretical ideas. However, many urban wayfinding studies have taken place in locations where a regular grid pattern of streets is not only locally present, but commonly expected within the national culture. So it is scarcely surprising that people depend on it when interpreting building numbers, judging orientations and giving route directions (e.g. Lynch 1960, Downs & Stea 1977, Montello 1991, Freundschuh 1992). In many cultures, such as much of Europe, grid-pattern cities are relatively rare, and people's culturally-influenced reference systems for wayfinding are forced to depend on other spatial cues (Canter 1977, Rapoport 1977). Junctions have irregular shapes, vistas are more varied and systematic naming or numbering is absent or unreliable. The prioritised knowledge types, in people's thinking about a city and planning movement around it, and in their language when describing it, might also be expected to differ according to this different reference system, even where the native language is the same across cultures. Therefore we should not assume a cross-cultural D.R. Montello (Ed.): COSIT 2001, LNCS 2205, pp. 400−414, 2001.  Springer-Verlag Berlin Heidelberg 2001

Grid Patterns and Cultural Expectations in Urban Wayfinding

401

generalizability of the wayfinding studies conducted in grid-pattern cities. Determining cultural as well as geographic effects on wayfinding may have important theoretical consequences -- as well as implications for future urban planning. Cornell and Heth (2000) suggest that although subjects do not need to know about either cardinal directions or city grid layouts in order to learn and recall routes, they are likely to become aware of them and to integrate them into an areal representation. However, this could arguably depend on extent of experience, and on the cultural and personal approach to wayfinding adopted by the individual. The present exploratory study was prompted by the hypothesis that cultural expectations (deriving in part from stereotypical urban landscapes) affect the reference frame utilized for wayfinding and the organization of local navigational knowledge. The practical effects of these expections may be strong enough to be measurably different across different cultural groups even with long-term inhabitants of cities with similar layouts. We compared the spatial reference frames and linguistic terms used by long-term inhabitants of the only grid-pattern city in England (Milton Keynes), and those of a similarly-sized grid-pattern US city (Eugene, Oregon). English and US cultural expectations of city structures are very different. Preliminary anecdotal evidence, including national media commentary, suggested that most English people have great difficulty adapting to the grid pattern in the relatively new city of Milton Keynes, since their reliance on broad visual and semantic cues is restricted in a more uniform environment. Long-term personal observations by one of the authors suggested that even long-term MK residents continue to use wayfinding strategies more appropriate to other UK cities. Such residents appear not to use the geodirectional features of their grid which are apparently so readily adopted by US residents. This preliminary study is the first we know of to investigate this issue. Data were gathered using methods which were both practical in the field, and designed to complement each other in the information provided about participants' mental models ('cognitive maps') of the two cities. In the present study, the most obvious ways in which the two groups of participants might be expected to differ were 1. Differences in directional knowledge, due to the use of different strategies for determining direction; 2. Differences in wayfinding information preferences, and hence in strategies used for helping an imaginary stranger to the city, based on cultural norms and in the subjective confidence felt about this; 3. Differences in content of residents' mental models, e.g. more landmark knowledge encoded by British participants rather than dependence on a regular grid1; 4. Differences in the language used about the city, as reflected in route directions, and particularly in the spatial relational terms used (e.g. “north”, “left”, “down”), possibly reflecting different spatial reference frames as seen in other cultural linguistic studies (e.g. Pederson 1993, Levinson 1996, Pederson et al. 1998). As Kitchin (1996) showed, task demands differ in subtly important ways in their capture of human environmental cognition. Thus, different tasks need to be used to 1

Even those residents actually born in Milton Keynes would not have witnessed the completion of MK's grid until the late 1980's.

402

C. Davies and E. Pederson

capture complementary aspects of people's knowledge and strategies, even where the subject of that knowledge (the same set of local landmarks and routes) is identical. This study differs from some others in its attempt to focus on more mature and longterm residents of an urban non-campus environment, and to capture all data 'in the field' in ecologically realistic tasks: pointing, route directions, and sketchmaps. In the everyday environment, one may be approached by a stranger for verbal directions, and several studies have already examined the structure and components of these (e.g. Allen 1997, Lovelace et al 1999). If the directions are complex, the authors have witnessed many people spontaneously grabbing a pen to draw a simple sketch map for the newcomer; Wright et al (1995) found that most literate adults preferred to have this for wayfinding and found it an effective form of communication, although only some would spontaneously produce it. These tasks are not without their drawbacks, of course. Sketch maps, in particular, are often criticised for being affected by individual 'drawing ability'. In the authors' view this criticism is overstated: a key part of the cognitive aptitudes that may be involved, in either producing or interpreting any drawing, is surely visualisation ability, and that visualisation (and its limitations) is precisely what we hope to capture in the drawn map. Sketch map-based studies have fallen foul of this 'problem' only when the emphasis was on judging the 'quality' and 'accuracy' (compared with Euclidean space). When used instead to explore the knowledge elicited and the strategy followed for drawing the map, within the constraints of the task, great richness can be revealed in the data, as seen in classic studies such as Milgram and Jodelet's study of residents' maps of Paris (Milgram and Jodelet, 1976). Pointing, similarly, can seem slightly artificial, and socially awkward to some people, but is nevertheless a common (and sometimes unstoppable!) part of people's attempts to convey spatial information, such as routes, to each other, and thus again has high ecological validity. As suggested by Hirtle and Heidorn (1993), measures of angular bearings or distance judgements, unlike paper-based tasks, are not constrained by 2D Euclidean space; additionally, with pointing, no transformation of scale is involved. The use of direct pointing can thus add information about people's spatial mental model, and any distortions within it, which could be confounded by other variables in the more complex but information-rich verbal directions and sketch maps. Rossano and Warren (1989) showed that a pointing task could be affected by having learned from a misaligned map, and so we might expect any distortions in subjects' local grid knowledge to be reflected in the bearing accuracies. Montello (1991) showed that pointing to locations in a grid city (Phoenix) while at an off-grid location similarly reduced the accuracy of people's pointing direction, suggesting that awareness of 'griddedness' is important at least to US grid-city residents. A more detailed comparison of our results with Montello’s work follows in the discussion section below. Due to length constraints and continuing detailed analyses, the main results reported here come from the sketch map and pointing tasks, which shed some light on numbers 1-3 above, and on the previous pointing studies. Linguistic differences between two populations of English language speakers are subtler and more complex. For current purposes, we assume that population differences on the minimally linguistic pointing and sketch map tasks are the result of non-linguistic cultural differences in mental maps and wayfinding.

Grid Patterns and Cultural Expectations in Urban Wayfinding

2

403

Method

We sought to compare subjects from two cities with as similar as possible physical geography. Both cities in this study have a well-defined central grid plan, with dissimilar off-grid peripheral areas. From the city center, off grid destinations thus involve travel through an irregular as well as regular layout. Of course, especially at the level of smaller details, there inevitably will be a number of distinctively American features to the landscape of the US city, and English features to the English city. Nonetheless, we believe that the primary difference between these two cities is not physical but is that the English city residents live in a culture which does not expect grid features in a city, and the American city residents live in a culture which quite typically expects and relies on grid features for navigation. 2.1

Milton Keynes

Milton Keynes (MK) is a modern metropolis of almost 250,000 residents, planned and begun in the 1960's, and located about 50 miles (80km) NW of London. MK’s main road grid, although the roads are not uniformly straight or regular, is aligned roughly 20-30 degrees W of true N. The 11 main roads running parallel with this axis are labelled 'V' (vertical), and numbered sequentially from W to E; they are crossed by 10 'H' (horizontal) roads numbered sequentially from N to S. All these roads are also given proper names (e.g. “V7 Saxon Street”), and many residents refer only to those names. Despite its greenery, extensive leisure facilities, full employment and uncluttered highways, 'MK' is ridiculed nationally; British visitors are invariably lost in its unusually regular and landmark-scarce grid. 2.2

Eugene

Eugene was plotted out in 1853 along the south and west banks of the Willamette river. The city of Springfield is located to the north and east of the Willamette. A few relatively small townships are also contiguous with Eugene-Springfield giving an effective population of the combined urban area of approximately 225,000, or essentially the same size as MK. The basic layout of Eugene’s downtown and neighboring area is a grid aligned perfectly with actual (not-magnetic) NSEW directions. The main EW roads are numbered (e.g. 20th Ave.) and the NS roads are all named. One cluster of these roads is named after early US Presidents2 and residents often speak of the “numbered” vs. the “president” streets. The historically major NS throroughfare through Eugene (Willamette Ave.) divides the numbered roads into east and west sections. Two “buttes” mark the approximate endpoints of Willamette Ave; it is unusual to be able 2

Several Eugene subjects reported confusion with the sequence of these roads because they “could not remember the historical order of the presidents” even though, in fact, these roads are not named in historical sequence.

404

C. Davies and E. Pederson

to see both landmarks, but they define a clear NS axis through the heart of the city. A third of our subjects reported relying partly on these landmarks, though they are often confused by which butte has which name. 2.3

Tasks

Both groups of subjects were run in an outdoor and uncrowded public area of a downtown shopping area. None of the locations queried in this study were visible from the testing location, but both sites had a clear view of a navigationally useful road on the downtown grid. Subjects were presented with several different tasks. These were identical for both the MK and Eugene subjects except in place names. The locations used for each of the tasks were reasonably well balanced for distance from subject, on- or off-grid address, public vs. private building, and compass direction from the subject. 1. Personal background questionaire. Age, length of residence, etc. 2. Self-report confidence ratings that subjects could direct a stranger to 40 local locations (to assess confidence in their familiarity with the city). 3a. Pointing to ten unseen locations (from the previous list) 3b. Giving driving directions to these same ten locations 3c. Drawing a sketch map for six of the unseen locations 4. General opinion questionaire on the city and its layout; followed by debriefing. The tasks 3a-c were presented in balanced orders across subjects (abc/cab/bac/cba) in such a way that pointing and direction giving were always adjacent and were repeated as a pair of tasks for each of the ten locations. Further, locations were presented in different orders across subjects to minimize any order effect within each task. Subject responses were tape recorded and, with the exception of verbal directions, recorded on sheets at the time of the experiment. Pointing and directions task. Subjects were asked to stand and point as accurately as possible and to give directions to each of ten well-known buildings, parks, and shopping areas around the city (All locations were selected from pilot work.) These locations were matched as closely as possible across the two cities: they were in a scattered array of directions and ranged from 100 to 3000 meters distant. Locations were both on and off the main road grid. Since subjects were at the required location of origin, and were free to face the direction of their pointing, no contradiction between egocentric and other frames of reference could develop (unlike, e.g., Werner and Schmidt 1999). Half of the subjects would first point to a location and then give directions to it; the other half would first give directions and then point. The subject was asked to assume that the researcher was a stranger asking driving directions from the field site, and that each set of directions was with a new stranger, so they should not make reference to directions to previous locations. All subjects were able to follow these instructions. When the initial starting direction was indicated non-verbally, this was noted. Pointing direction was measured by line of sight with a compass to an estimated accuracy of ±3°. After each point and after each directions, subjects gave a 0-5

Grid Patterns and Cultural Expectations in Urban Wayfinding

405

confidence rating of the accuracy of the point and of the likelihood that a stranger would reach the destination using the directions. Sketch map task. Subjects were given a sheet of A4 or 8½x11 inch paper with printed instructions to draw a single, rough, overview sketch map of the city, as an aid for a fictional stranger navigating to six locations from the field site. Subjects were asked to finish in six to seven minutes and were stopped after about eight minutes if necessary. The maps were drawn with a pen and no erasing was possible. There were no instructions as to how to lay out or label the map. After completing the map, the researcher checked that each location was clearly marked and if not, asked for a correction. 2.4

Subjects

Using primarily a mailing list (UK) and notice board adverts (US), we recruited 56 (32 UK and 24 US) adults. 32 were female (57%); ages ranged between 19 and 74 (mean = 41, s.d. = 12). All but one had lived in the city for at least 3 years, with a maximum of 45 years (mean = 15, s.d. = 11). The MK and Eugene samples differed significantly on only two subject variables: the Eugene sample were more likely to be non-drivers (p=0.035 in Fisher's Exact Test), and tended to walk around the downtown more frequently than the MK sample (Mann-Whitney U = 106, p