Facets of Virtual Environments: First International Conference, FaVE 2009, Berlin, Germany, July 27-29, 2009, Revised Selected Papers (Lecture Notes ... and Telecommunications Engineering, 33) 3642117422, 9783642117428

Citation preview

Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering Editorial Board Ozgur Akan Middle East Technical University, Ankara, Turkey Paolo Bellavista University of Bologna, Italy Jiannong Cao Hong Kong Polytechnic University, Hong Kong Falko Dressler University of Erlangen, Germany Domenico Ferrari Università Cattolica Piacenza, Italy Mario Gerla UCLA, USA Hisashi Kobayashi Princeton University, USA Sergio Palazzo University of Catania, Italy Sartaj Sahni University of Florida, USA Xuemin (Sherman) Shen University of Waterloo, Canada Mircea Stan University of Virginia, USA Jia Xiaohua City University of Hong Kong, Hong Kong Albert Zomaya University of Sydney, Australia Geoffrey Coulson Lancaster University, UK

33

Fritz Lehmann-Grube Jan Sablatnig (Eds.)

Facets of Virtual Environments First International Conference, FaVE 2009 Berlin, Germany, July 27-29, 2009 Revised Selected Papers

13

Volume Editors Fritz Lehmann-Grube Technische Universität Berlin Center for Multimedia in Education and Research (MuLF) Straße des 17. Juni 136 10623 Berlin, Germany E-mail: [email protected] Jan Sablatnig Technische Universität Berlin Institute of Mathematics Straße des 17. Juni 136 10623 Berlin, Germany E-mail: [email protected]

Library of Congress Control Number: 2009943510 CR Subject Classification (1998): K.8, I.2.1, K.4.2, K.3, J.4, I.3.7 ISSN ISBN-10 ISBN-13

1867-8211 3-642-11742-2 Springer Berlin Heidelberg New York 978-3-642-11742-8 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © ICST Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering 2010 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12985821 06/3180 543210

Preface

In recent years, the popularity of virtual worlds has increased significantly and they have consequently come under closer academic scrutiny. Papers about virtual worlds are typically published at conferences or in journals that specialize in something entirely different, related to some secondary aspect of the research. Thus a paper discussing legal aspects of virtual worlds may be published in a law journal, while a psychologist's analysis of situation awareness may appear at a psychology conference. The downside of this is that if you publish a virtual worlds paper at an unrelated conference in this manner you are likely to be one of only a handful of attendees working in the area. You will not, therefore, achieve the most important goal of attending conferences: meeting and conversing with like-minded colleagues from the academic community of your field of study. Virtual worlds touch on many well-established themes in other areas of science. Researchers from all these fields will therefore be looking at this new, interesting, and growing field. However, to do effective research related to these complex constructs, researchers need to take into account many of the other facets from other fields that impact virtual worlds. Only by being familiar with and paying attention to all these different aspects can virtual worlds be properly understood. We therefore believe that the study of virtual worlds has become a research field in its own right. To date, this research field can claim only a relatively small community, because interested researchers from more established fields largely keep to themselves. FaVE was born to change that. We wanted to start creating a multidisciplinary community of academic researchers all interested in virtual worlds and their applications; and we wanted everyone to talk to each other, regardless of their original field, because we do believe that every one of these researchers has something to say that will be of interest to the rest. After much organizational work and with lots of help from collaborators all over the world (and of course some sleepless nights), the conference was finally held during July 27–29, 2009. The tracks and sessions were organized with our multidisciplinary goal in mind: that is, we attempted to create sessions with a combination of presenters who are working on similar subjects, albeit perhaps coming from different angles. Over the course of the conference, our attendees did indeed see the advantages of the format. By the end of the conference, there were vivid and vibrant discussions going on, bringing all the diverse viewpoints to the table––surprisingly similar in some cases and surprisingly different in others. The first set of papers presented at the conference talked about the application of virtual worlds to science, both for research and for education. Virtual worlds are seen as a means to solve problems that have been known to science for a while, but which are expected to become more pronounced in the near future––such as data visualization and extending the reach of scientific teaching. The following papers were presented:

VI

Preface

• “Exploring the Use of Virtual Worlds as a Scientific Research Platform: The Meta-Institute for Computational Astrophysics (MICA)” by S. G. Djorgovski, P. Hut, S. McMillan, E. Vesperini, R. Knop, W. Farr, M. J. Graham • “Dual Reality: Merging the Real and Virtual” by Joshua Lifton and Joseph A. Paradiso • “Development of Virtual Geographic Environments and Geography Research” by Fengru Huang, Hui Lin, Bin Chen The next few papers addressed how people behave and react in existing virtual worlds. This not only characterized how people move and navigate, but also included very tangible advice on how one might improve the usability and acceptance of virtual worlds, such as by adding landmarks and improving the virtual weather. These papers comprised: • • •

“Landmarks and Time-Pressure in Virtual Navigation: Towards Designing Gender-Neutral Virtual Environments” by Elena Gavrielidou and Maarten H. Lamers “Characterizing Mobility and Contact Networks in Virtual Worlds” by Felipe Machado, Matheus Santos, Virgilio Almeida, and Dorgival Guedes “The Effects of Virtual Weather on Presence” by Bartholomäus Wissmath, David Weibel, Fred W. Mast

Next, we took a look at what can be done to make virtual worlds easier to use for the end user. This ranged from a shop assistant who attempts to understand typed speech, through a visualization plug-in architecture, to an analysis of current virtual worlds' Terms of Service and how those may be improved. The papers here were: • • •

“The Role of Semantics in Next-Generation Online Virtual World-Based Retail Store” by Geetika Sharma, C. Anantaram, and Hiranmay Ghosh “Complexity of Virtual Worlds' Terms of Service” by Holger M. Kienle, Andreas Lober, Crina A. Vasiliu, Hausi A. Müller “StellarSim: A Plug-in Architecture for Scientific Visualizations in Virtual Worlds” by Amy Henckel and Cristina V. Lopes

We subsequently discussed the theory and practice of collaboration in virtual worlds. A formal description of virtual world collaboration was developed that may be used to describe workflow in a virtual world setting. Also, an actual workflow was studied experimentally and some requirements for characters controlled by artificial intelligences in interacting efficiently with human users were set out. The papers were: • “Formalizing and Promoting Collaboration in 3D Virtual Environments - A Blueprint for the Creation of Group Interaction Patterns” by Andreas Schmeil and Martin J. Eppler • “Usability Issues of an Augmented Virtuality Environment for Design” by Xiangyu Wang and Irene Rui Chen • “Conceptual Design Scheme for Virtual Characters” by Gino Brunetti and Rocco Servidio

Preface

VII

Finally, we focused on the social aspects of using virtual worlds. While in traditional media the media produces content and consumers consume it, these lines are blurred in virtual worlds. This touches on many important questions such as ownership and rights. Does a user of a virtual world even have rights? The mixing of play and work that is becoming noticeable in many virtual worlds was also explored. The Papers were: • “The Managed Hearthstone: Labor and Emotional Work in the Online Community of World of Warcraft” by Andras Lukacs, David Embrick, and Talmadge Wright • “Human Rights and Private Ordering in Virtual Worlds” by Olivier Oosterbaan • “Investigating the Concept of Consumers as Producers in Virtual Worlds: Looking Through Social, Technical, Economic, and Legal Lenses” by Holger M. Kienle, Andreas Lober, Crina A. Vasiliu, Hausi A. Müller The papers are an interesting read and we hope that you take the time to peruse a few that may not be quite in your area of research.

Organization

Steering Committee Imrich Chlamtac Sabine Cikic Viktor Mayer-Schönberger

Create-Net, Italy Technische Universität Berlin, Germany Harvard University, USA

General Conference Chair Richard A. Bartle

University of Essex, UK

General Conference Vice Chair Sven Grottke

University of Stuttgart, Germany

Technical Program Chair Jan Sablatnig

Technische Universität Berlin, Germany

Workshops Chair Fritz Lehmann-Grube

Panels Chair Julian R. Kücklich

University of Arts London, UK

Local Arrangements Chair Sabine Cikic

Technische Universität Berlin, Germany

Publicity Chair Sebastian Deterding

Publications Chair Fritz Lehmann-Grube

Utrecht University, The Netherlands

X

Organization

Web Chair Sharon Boensch

Technische Universität Berlin, Germany

Sponsorship Chair Sabina Jeschke

University of Stuttgart, Germany

Conference Coordinator Gabriella Magyar

ICST

Program Committee Katharina-Maria Behr Anja Beyer Sabine Cikic Julian Dibbell Sebastian Deterding Martin Dodge Sean Duncan David England James Grimmelmann Sven Grottke Shun-Yun Hu Jesper Juul Fritz Lehmann-Grube Andreas Lober Claudia Loroff Viktor Meyer-Schönberger Claudia Müller Heike Pethe Thomas Richter Albert 'Skip' Rizzo Jan Sablatnig Uwe Sinha Matthew Sorell Marc Swerts Anton van den Hengel Xiangyu Wang Marc Wilke Leticia Wilke Theodor G. Wyeld Tal Zarsky

Hamburg Media School, Germany Ilmenau University of Technology, Germany Technische Universität Berlin, Germany Utrecht University, The Netherlands University of Manchester, UK University of Wisoconsin-Madison, USA Liverpool John Moores University, UK New York Law School, USA University of Stuttgart, Germany National Central University Taiwan Singapore-MIT GAMBIT Game Lab, Singapore Technische Universität Berlin, Germany Schulte Riesenkampff, Lawyers Institut für Innovation und Technik, Germany Harvard University, USA University of Stuttgart, Germany University of Amsterdam, The Netherlands University of Stuttgart, Germany University of Southern California, USA Technische Universität Berlin, Germany Technische Universität Berlin, Germany University of Adelaide, Australia Tilburg University, The Netherlands Australian Centre for Visual Technologies, Australia The University of Sydney, Australia University of Stuttgart, Germany University of Stuttgart, Germany Flinders University Adelaide, Australia University of Haifa, Israel

Table of Contents

FaVE 2009 – Track 1 Development of Virtual Geographic Environments and Geography Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fengru Huang, Hui Lin, and Bin Chen Dual Reality: Merging the Real and Virtual . . . . . . . . . . . . . . . . . . . . . . . . . Joshua Lifton and Joseph A. Paradiso Exploring the Use of Virtual Worlds as a Scientific Research Platform: The Meta-Institute for Computational Astrophysics (MICA) . . . . . . . . . . S. George Djorgovski, Piet Hut, Steve McMillan, Enrico Vesperini, Rob Knop, Will Farr, and Matthew J. Graham

1 12

29

FaVE 2009 – Track 2 Characterizing Mobility and Contact Networks in Virtual Worlds . . . . . . Felipe Machado, Matheus Santos, Virg´ılio Almeida, and Dorgival Guedes Landmarks and Time-Pressure in Virtual Navigation: Towards Designing Gender-Neutral Virtual Environments . . . . . . . . . . . . . . . . . . . . . Elena Gavrielidou and Maarten H. Lamers The Effects of Virtual Weather on Presence . . . . . . . . . . . . . . . . . . . . . . . . . Bartholom¨ aus Wissmath, David Weibel, and Fred W. Mast

44

60 68

FaVE 2009 – Track 3 Complexity of Virtual Worlds’ Terms of Service . . . . . . . . . . . . . . . . . . . . . . Holger M. Kienle, Andreas Lober, Crina A. Vasiliu, and Hausi A. M¨ uller

79

The Role of Semantics in Next-Generation Online Virtual World-Based Retail Store . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Geetika Sharma, C. Anantaram, and Hiranmay Ghosh

91

StellarSim: A Plug-In Architecture for Scientific Visualizations in Virtual Worlds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Amy Henckel and Cristina V. Lopes

106

XII

Table of Contents

FaVE 2009 – Track 4 Formalizing and Promoting Collaboration in 3D Virtual Environments – A Blueprint for the Creation of Group Interaction Patterns . . . . . . . . . . Andreas Schmeil and Martin J. Eppler

121

Conceptual Design Scheme for Virtual Characters . . . . . . . . . . . . . . . . . . . . Gino Brunetti and Rocco Servidio

135

Usability Issues of an Augmented Virtuality Environment for Design . . . Xiangyu Wang and Irene Rui Chen

151

FaVE 2009 – Track 5 The Managed Hearthstone: Labor and Emotional Work in the Online Community of World of Warcraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andras Lukacs, David G. Embrick, and Talmadge Wright Human Rights and Private Ordering in Virtual Worlds . . . . . . . . . . . . . . . Olivier Oosterbaan

165 178

Investigating the Concept of Consumers as Producers in Virtual Worlds: Looking through Social, Technical, Economic, and Legal Lenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Holger M. Kienle, Andreas Lober, Crina A. Vasiliu, and Hausi A. M¨ uller

187

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

203

Development of Virtual Geographic Environments and Geography Research Fengru Huang1, Hui Lin1, and Bin Chen2 1

Institute of Space and Earth Information Science, Chinese University of Hong Kong, Shatin, N.T., Hong Kong 2 Institute of Remote Sensing and Geographic Information System, Peking University, Beijing, China {huangfengru,huilin}@cuhk.edu.hk, [email protected]

Abstract. Geographic environment is a combination of natural and cultural environments under which humans survive. Virtual Geographic Environment (VGE) is a new multi-disciplinary initiative that links geosciences, geographic information sciences and information technologies. A VGE is a virtual representation of the natural world that enables a person to explore and interact with vast amounts of natural and cultural information on the physical and cultural environment in cyberspace. Virtual Geography and Experimental Geography are the two closest fields that associate with the development of VGE from the perspective of geography. This paper discusses the background of VGE, introduces its research progress, and addresses key issues of VGE research and the significance for geography research from Experimental Geography and Virtual Geography. VGE can be an extended research object for the research of Virtual Geography and enrich the contents of future geography, while VGE can also be an extended research method for Experimental Geography that geographers can operate virtual geographic experiments based on VGE platforms. Keywords: Virtual Environment, Virtual Geography, Experimental Geography, Virtual Geographic Experiment.

1 Introduction Geographic environment is a combination of natural and cultural environments under which humans survive, and traditional geography takes geographic environments in the real world as its study object. Geography aims to study the physical, chemical, biological and human processes of the geographic environment (the earth surface system), analyze the relationships between the interfaces of each geo-spheres, and interaction mechanisms between various natural and human processes, thus to explore the precepts of coordinative and sustainable development of resources, environments and human activities. As the development of information technologies such as Internet, Web and Virtual Reality goes further, both new opportunities and challenges are generated for the development of geographic information sciences and technologies, as well as for geography sciences. Virtual Geographic Environment (VGE) was first proposed in F. Lehmann-Grube and J. Sablatnig (Eds.): FaVE 2009, LNICST 33, pp. 1–11, 2010. © Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering 2010

2

F. Huang, H. Lin, and B. Chen

early 2000 by geography and geographic information science researchers [1, 2, 3, 4]. VGE is a new multi-disciplinary initiative that links geosciences, geographic information sciences and information technologies. A VGE is a virtual representation of the natural world that enables a person to explore and interact with vast amounts of natural and cultural information on the physical and cultural environment, in cyberspace. From the perspective of geography, VGE is an environment concerned with the relationship between avatar-based humans and 3-dimension (3D) virtual worlds. From the perspective of information systems, VGE is an advanced information system that combines GIS (Geographic Information System) with VR technology [1, 2, 3]. At present, there has launched much research into VGE theory, technology and applications [5, 6, 7, 8]. Those works focus on different aspects of VGE research and thus raise broader and more complicated research such as topics on geo-data, geo-models, geosciences knowledge acquisition, GeoComputation, geo-visualization, geocollaboration, interaction mode, virtual geographic experiments and Virtual Geography. To address this, this paper aims to discuss the background of VGE, introduce its research progress, and address key issues on VGE research and the significance for geography research from the perspectives of Experimental Geography and Virtual Geography. This paper is organized as follows. In section 2, we discuss background and research progress of VGE, as well as its research contents and key issues. In section 3, we present revolution of geography research method and geographic language. Section 4 and Section 5 discuss development of Virtual Geography and development of Experimental Geography, respectively. Section 6 contains some final discussion and remarks on VGE and geography research.

2 Background and Research Progress of VGE 2.1 What Is VGE? VGE was first proposed as a concept of a virtual world that was referenced to the real world, which had five types of space, namely Internet space, data space, 3D graphical space, personal perceptual and cognitive space, and social space [2]. To this concept regard, there are three stages in the evolutionary process of a VGE: virtual crowds, virtual villages and virtual cities. In this sense, VGE research focuses on the differences and extension of life content and life style from the real world to virtual worlds, or between the real world and a virtual world, and thus relate to research of Virtual Geography or other terms alike. To make emphasis on representation of geographic process and phenomena in the real world, such as visualization and simulation of geomodals in diverse geosciences, the concept of VGE has been supplemented as a new generation of information platform that can be used for geo-phenomena representation and simulation, and geo-knowledge publishing and sharing [9]. Such a VGE represents an ideal interface of geo-information scientists for geographic representation and research, that is ‘immersive experience and beyond the understanding of reality’. VGE systems have five characteristics:

Development of Virtual Geographic Environments and Geography Research

3

1, Integrated management and interoperation on geo-models and GIS data; 2, Multi-dimension geo-visualization, including visualization of geometric models (represent static objects) and geo-models (represent dynamic geographic processes); 3, Immersive virtual interaction: users can ‘step’ into the virtual geographic world and be a part of the environment, thus have an immersive interaction with the virtual environment. 4, Distributed geo-collaboration: geographic experts from different places/locations of the real world can carry out professional discussion and decision-making with the support of VGE platform; 5, Public participation: VGE emphasizes on the role of social public participation, so the users are not just experts and professional users, but also the general public. 2.2 Why VGE Rising? The rising of VGE has a profound background that includes not only development of geographic sciences, but also currently rapid development of computer technology, information technology and social sciences. The development of VGE is closely related to the development of Earth System Science and will ultimately serve the research of global environment change and human sustainable development. 1, Earth System Science research needs a new research tool and information platform in which scientific computation and virtual representation are the two important characteristics, to facilitate simulation and prediction on natural complex phenomena that can not be experimented in the real world conditions, such as prediction on the whole cycle of the Earth's atmosphere-ocean, global warming, Earth's crust change, earthquake occurrence, and human behavior simulation in emergency public accident or natural disasters, so as to help manage on environmental resources and human activities to achieve sustainable development. 2, Current rapid development of Earth information technologies provides technical support for the emergence of VGE. As the development of mathematical scientific methods (for example, scientific computation, cellular automation, fractal geometry, fuzzy mathematics, etc), and computer science and technologies (such as computer communication, networks, databases, distributed computing, artificial intelligence, human-computer interaction and virtual reality) goes further and is being applied to geographic science and Earth System Science, there has been continuous development from different angles in the field of Earth information technologies. This provides support for the rising and development of VGE, which integrates with Remote Sensing (RS), Global Position System (GPS), Geographic Information System (GIS), computer network, virtual reality technology, and other computer technologies. 3, The field of social and cultural sciences require a research platform or a window like VGE to learn about human development trends in the age of post-modernism. The style of post-modern society has the basic characteristics as "information age", "knowledge economy" and "learning society", and has actually penetrated into various aspects of contemporary human society, quickly and fully. In recent years, geography research activities and literature have been increasing with regard to the impact of modern information technology on geography. For example, Batty [10, 8] proposed "invisible cities", "Cyberspace Geography" and "Virtual Geography" in terms of geographic space–place, espace, cyberspace, and cyberplace. Increasing public is being

4

F. Huang, H. Lin, and B. Chen

familiar with and a part of virtual environments, virtual earth, or virtual worlds. The new styles of learning, working and living, such as e-tourism, e-education, eshopping, virtual communities, virtual office, virtual banking, virtual stock market, virtual games, and virtual art appear in succession and show a strong vitality, and may represent human development trends and directions in the post-modern age. Therefore, from the perspective of social scientists who study socio-economic, political, legal, cultural, and human psychology, behavior and life styles of the post-modern age, something like VGE as a research window is needed to help to explore characteristics and development trends of human society of the post-modern age. 2.3 Related Work VGE is developed with the support of the advancement in computer science and technologies, geosciences, Geographic Information science and techniques. Only by combination of those theories and technologies to construct an integrated platform can we meet the need of the development of Earth System Science for global environmental change and sustainable development research. In recent years, much progress has been made on such a next-generation geographic information platform from different aspects. Chinese scholars have been engaging actively in relevant research since VGE was put forward a decade ago. Lin and Gong explored basic theory, technology and application of VGE through a series of academic work and papers [1, 2, 3, 4, 9, 11]. Tang et al. studied on visual geographic modeling and construction of VGE [12]. Researchers in the Electronic Visualization Laboratory (EVL) of The University of Illinois have focused on the development of tools, techniques and hardware to support real-time and highly interactive visualization [13], and the platform GeoWall [14] was developed with the characteristics of users’ immersive interaction with the virtual environment which was displayed to the big screen. MacEachren developed a system named Dialogue Assisted Visual Environment for Geoinformation (DAVE_G), in which the earlier multi-modal interface framework and two test-bed implementations: iMap and XISM [15] were built on and extended. Batty, M. established virtual city and explored Virtual Geography [8, 10, 16]. Yano built Virtual Kyoto through 4DGIS and Virtual Reality to show social customs and traditional culture in Japan [17]. Google, Microsoft, Linden Lab and other companies started to build community, city, region, or even global 3D virtual environments. Google developed Google Earth for public searching the high resolution digital map freely [18], and Google SketchUp [19] for 3D models building. Microsoft launched Virtual Earth project, which was built up by using photos and offered a higher sense of reality [20]. Linden Lab created and opened Second Life® to the public since 2003, and now it owns the largest amount of virtual residents and many kinds of applications such as virtual meeting, virtual class, virtual industry, etc., in its virtual world [21]. As one of the approaches of VGE application construction, some GIS-based multi-user virtual environment applications are being carrying out based on virtual world platforms such as Second Life®, OpenSimulator [22] or other similar projects. We can therefore see that, as a new generation of geographic information platform, VGE development has a broad prospect for geography research.

Development of Virtual Geographic Environments and Geography Research

5

2.4 Research Contents and Key Issues of VGE In contrast to current data-centered GIS, a VGE is a human-centered environment. A VGE system can present immersive multi-dimension visualization, support multi-user collaborative work, and provide a natural way of perception and interaction between avatars or users, or between users and virtual environments. Thus, VGE can be an integrative innovation and its research contents may involve multi-discipline issues, such as geo-modeling, geographic simulation, GeoComputation, geo-visualization, computer network, geo-collaboration and interaction, geo-knowledge discovery and sharing, and virtual geographic experiments. Those are as well as the key issues of VGE research. On the other hand, VGE extends the research range of traditional geography with virtual extended geographic environments. Thus, the research contents of geography extend from place and space of real geographic environment to placespace and relationship in virtual environments or interaction between those two. This paper will discuss further on two extended research fields: Virtual Geography and Experimental Geography in the subsequent sections.

3 Revolution of Geography Research Methods and Geographic Languages There has always been a thread of research thoughts of "Pattern - Structure - Process -Mechanism" throughout geography studies. However, the research methods in traditional physical geography are mostly field-site inspection and the use of maps and data analysis. Geographer Baranshiy once said, "Map is the second language of geography". Using maps for thinking and analyzing is the most important research method that makes geography different from other subjects. Development of GIS is based on a combination of map, mathematical methods, and modern information technologies. To date, GIS has become the most common carrier and platform of geographic information. Chen argued "GIS is the third-generation language of geography" [23]. Along with constant improvement of ability and means to access digital spatial data and expansion of GIS applications, limitation of traditional GIS (map-centered and data-driven mechanism) has hindered the development of new methods in the field of geographic information representation and services. Virtual reality technology can be used as an immersive human-computer interface for 3D visualization, collaborative work and group decision making through integration with traditional GIS and 3D GIS. Thus, development of VGE can be seen as a higher level of GIS that integrates traditional GIS, virtual reality, network technology, geo-models, humancomputer interaction technology, and systematic methods. Lin argued VGE can be a new generation of geographic language in that VGE had the ability of abstract expression of multi-dimensional, multi-viewpoint, multiple details of multi-model visualization, supporting for a variety of natural interaction and multi-spatial cognition [4, 11]. Fig. 1 shows the developing process from map and GIS to VGE.

6

F. Huang, H. Lin, and B. Chen

Field Survey and Mapping Geographic Science Earth System Science

Mathematical Methods Map Digitization

Computer Technologies

Map Systematic Methods

GIS 2D GIS

Network Technology

GIS visualization (2D->2.5D)

Virtual Reality Virtual Map

3D Virtual GIS

Network Based GIS

Geo-informatic Tupu Geo-spatial Cognition Geo-graphical Thinking Geo-Knowledge Reasoning ……

Distributed Computing Grid Computing

Collaborative Information Sysem

Distributed Collaborative 3D Virtual GIS Geographic Models and Geometry Spatial Database Integration

VGE

Fig. 1. Process from map and GIS to VGE

4 Development of Virtual Geography 4.1 VGE Extends Geographic Environment in the Real World Geography is the science of place and space [24]. Traditional geography focuses on place and space of geographic environment in the real world. However, information science and technology provide open and distributed environments like VGE in the Internet or in other cyberspace. In those information worlds, the importance of geographic distance and place has gradually decreased [2]. Online communities or virtual companies exist in cyberspace with virtual places in virtual environments, but with their locations at “elsewhere” or even nowhere in the real world. Thus, space-place becomes virtual space-place and this leads to a deep thinking and wide discussion for geographers in the context of future geography [25, 26, 27, 28]. Geography Research has extended from traditional geographic environment to virtual geographic environments that Virtual Geography focuses on.

Development of Virtual Geographic Environments and Geography Research

7

4.2 Virtual Geography Virtual geography, cyber geography, and imagine geography are all the similar terms in the present literatures that show the impacts of modern technology on geography [2]. Batty proposed virtual geography and focused on the relationship and interaction between cyberspace and the real world, and argued that the boundary between space and place in cyberspace turned blurred, while Crang et al. examined virtual geography mainly from the aspect of complicated social relationships in virtual environments. Lin and Gong [1, 2] argued that virtual geography was a new dimension of geography studying the characteristics and laws involving VGE, and the relationship and interaction between VGE and real geographic environments. In comparison to traditional geography, research contents of this new initiative of geography may include: 1, cybercartography: this is to study the principles and methodology of cybermapping. 2, Development, planning and building of 3D virtual worlds. 3, Spatial perception, cognition and behavior of post-human in 3D virtual environments. 4, Issues in the evolution process of VGE, such as boundary and relationship among various 3D virtual worlds, mechanism of driving forces of evolution of VGE, etc. 5, Relationship and interaction between VGE and real geographic environments in population, landscape, social, political, and economic structures.

5 Development of Experimental Geography 5.1 Experimental Geography Experiment is an important feature as well as a symbol of development of modern science. That means a scientific experiment can be repeated and be verified. Experience, observation, practice, and experiment are of great importance in geography research. From 1950-1960, Chinese geographers have come to realize the importance of experiments for scientific theories and methods of geography development. Huang Bingwei, a modern Geography pioneer of China, pointed out that, the old methods such as empirical and descriptive study in geography research were inanimate, and Experimental Geography was a major development direction of the forward-looking geography [29]. Experimental Geography applies specific experimental ideas, experimental methods, observation equipments and instruments to learn about the spatial structure, time series, human-earth relationship of geographic environment, discover the basic law of geography information accumulation and provide evidence to form a measurable, comparable, controllable geographic system. Therefore, experimental design and experimental execution theories and methods together constitute the research contents of Experimental Geography. The purpose of all experimental work is to identify geographical relations through accessing geographic information by the means of an extension of human senses.

8

F. Huang, H. Lin, and B. Chen

Traditional methods used in Experimental Geography include field experiments and indoor physical modeling and experiments. However, those traditional experimental methods show much limitation when its processed object, geographic system, is a complex giant system with geographic issues of multi-dimension, multi-scale, ambiguity and uncertainty. At present, geographic mathematical modeling, remote sensing information modeling and computer simulation calculation and experimental methods are various and complicated, thus, an organic integration of those modern methods from the perspective of Experimental Geography is needed to be achieved. 5.2 Virtual Geographic Experiments for Experimental Geography Virtual experiments are defined as digital and virtual environments to carry out scientific experiments with the support of computer and network technologies. As development of information technology and simulation technology goes further, currently, virtual experiments are applied to a large number of research areas, including biology, chemistry, physics, human motion, and manufacturing, and has become a hot issue in those research fields. However, virtual experiment applications in geosciences are relatively few due to the giant system and highly complex nature of geographic environment. In recent years, as development of VGE and related research that has been carried out, as well as learning from virtual experiment applications in experimental economics, experimental medicine and other areas, virtual geographic experiment has gradually formed a new direction of research methods for Experimental Geography. 5.3 VGE as a Virtual Geographic Experiment Platform We argue that VGE, a virtual geographic world, can be a virtual laboratory in which Virtual Geographic Experiment can be carried out. Virtual Geographic experiment aims to establish and visualize geographic models to verify and represent geographic phenomena and processes by calculation, simulation, visualization, real-time human participation, interaction and manipulation based on geosciences data. It may correspond to geographic positioning field experiments, or indoor physical modeling experiments. It may also be some virtually constructed experiments based on specific geographic features, phenomena and laws that are difficult to be carried out as physical experiments in the real world. Virtual Geographic Experiment can be widely used not only in traditional experimental geography focused research areas of physical geography, but also in economic geography and human geography as a major research method. With the support of such an integration platform of interactive and collaborative work and geographic experimental environment provided by VGE, geographers can analyze the represented geographic phenomena and processes and carry out joint research, knowledge discovery, communication and decision-making in its immersive way. Thus, VGE extends the research methods of Experimental Geography (Fig. 2).

Development of Virtual Geographic Environments and Geography Research

Methods of Experimental Geography

9

Virtual Geographic Environment Virtual geographic experiment

Field investigate

Geo-knowledge discovery and sharing

Field observation and survey

Geo-collaboration interaction Computer simulation experiment Interior experiment and analysis Interior physical simulation experiment

Remote sensing information modeling

and

Multi-D geo-visualization Geo-system Simulation Scientific geo-computation

Mathematical Geographic Modeling

Geo-modeling

Fig. 2. VGE extends the research methods of Experimental Geography

6 Discussion and Conclusion In recent years, multi-user virtual environments have come into widespread use on the Internet. Virtual environment technologies and virtual world platforms (e.g. the classical virtual world "Second Life"®) are used not only for games but also for various non-game purpose applications [30]. Moreover, Roush argued that the World Wide Web will soon be absorbed into the World Wide Sim: an immersive, 3D visual environment combining elements of social virtual worlds ( e.g. Second Life®) and mapping applications (e.g. Google Earth), and what’s coming is a larger digital environment-a 3D Internet [31]. Many relevant issues are being developed or need to be developed to explore both on theory, technology and various applications on those subjects. VGE combines elements of all these technologies and research on relevant frontier issues from the perspective of geography. However, current VGE research focuses more on geometry modeling and visualization or realistic representation that inherits and extends from 2D GIS functionalities, there are limitations with VGE but are important aspects of VGE are dynamic geographic processes modeling and visualization, geo-collaboration, interaction under a 3D virtual environments that support for the capability of people to better understand the real geographic environment. Virtual Geography and Experimental Geography are two closest fields that associate with the development of VGE. Virtual Geography has VGEs as its research object and extend geographic issues from traditional geographic environment to virtual environments and the spaces, places, avatars, and all the other elements and relations in it. Experimental Geography might have VGE as a new medium to establish virtual experiments on geographic processes with a way of immersive visualization, geocollaboration and natural interaction. Development of VGE represents a new field in

10

F. Huang, H. Lin, and B. Chen

geographic information and geographic research in the coming 3D Internet age. Much work should be developed from different aspects of this new field.

Acknowledgements This research is partially supported by The National “863”High Technology Research and Development Program of China (No. 2006AA12Z207, 2007AA120502), and Direct Grant from CUHK (No. 2020967). We would also like to thank the three anonymous reviewers for their valuable suggestions on previous version of this paper.

References [1] Gong, J., Lin, H.: Virtual Geographic Environments—A Geographic Perspective on Online Virtual Reality. High Education Press, Beijing (2001) [2] Lin, H., Gong, J.: Exploring Virtual Geographic Environments. Geographic Information Sciences 7(1), 1–7 (2001) [3] Lin, H., Gong, J.: On Virtual Geographic Environments. Acta Geodaetica et Cartographica Sinica 31(1), 1–6 (2002) [4] Lin, H., Gong, J., Shi, J.: From Maps to GIS and VGE-A Discussion on the Evolution of the Geographic Language. Geography and Geo-Information Science 19(4), 18–23 (2003) [5] Jiulin, S.: An Exploration of Virtual Recreation Environment on Resources and Environment Sciences. Resources Science 21(1), 1–8 (1999) [6] Jun, G., Yunjun, X., Xiong, Y.: Application of Virtual Reality in Terrain Environment Simulation. People’s Liberation Army Press, Beijing [7] Dykes, J., Moore, K., Wood, J.: Virtual environments for student field work using network components. International Journal of Geographical Information Science 13(4), 397– 416 (1999) [8] Batty, M., Smith, A.: Virtuality and Cities: Definitions, Geographies, Designs. In: Fisher, P.F., Unwin, D.B. (eds.) Virtual Reality in Geography, pp. 270–291. Taylor and Francis, Abington (2002) [9] Lin, H., Xu, B.: Some Thoughts on Virtual Geographic Environments. Geography and Geo-Information Science 23(2), 1–7 (2007) [10] Batty, M.: Virtual Geography. Future 29(4/5), 337–352 (1997) [11] Lin, H., Zhu, Q.: The Linguistic Characteristica of Virtual Geographic Environments. Journal of Remote Sensing 9(2), 158–165 (2005) [12] Tang, W., Lv, G., Wen, Y., et al.: Study of Visual Geographic Modeling Framework for Virtual Geographic Environment. Geo-information Science 9(2), 78–84 (2007) [13] Jeong, B., Renambot, L., Jagodic, R., Singh, R., Aguilera, J., Johnson, A., Leigh, J.: High-Performance Dynamic Graphics Streaming for Scalable Adaptive Graphics Environment. In: Proceedings of SC 2006, Tampa, FL, November 11-17 (2006) [14] Johnson, A., Leigh, J., Morin, P., Van Keken, P.: GeoWall: Stereoscopic Visualization for Geoscience Research and Education. IEEE Computer Graphics and Applications (11/01/2006 - 12/31/2006) [15] MacEachren, A., Cai, G., Sharma, R., Rauschert, I., Brewer, I., Bolelli, L., Shaparenko, B., Fuhrmann, S., Wang, H.: Enabling collaborative Geoinformation access and decisionmaking through a natural, multimodal interface. International Journal of Geographical Information Science 19(3), 293–317 (2005)

Development of Virtual Geographic Environments and Geography Research

11

[16] Smith, H., Evanss, Batty, M.: Building the virtual city: Public participation through edemocracy. Knowledge, Technology & Policy 18(1), 62–85 (2005) [17] Keiji, Y.: Virtual Kyoto through 4D-GIS and Virtual Reality, http://www.ritsumei.ac.jp/eng/newsletter/winter2006/gis.shtml [18] http://www.Earth.google.com [19] http://www.sketchup.google.com [20] http://www.preview.local.live.com [21] http://www.secondlife.com [22] http://www.opensimulator.org [23] Chen, S.: Geographic Information System Exploration and Experiments. Scientia Geographica Sinica 3(4), 287–302 (1983) [24] AAG, What is a geography (2001), http://www.aag.org/ [25] Couclelis, H.: he Death of Distance. Environment and Planning B: Planning and Design 23, 387–398 (1996) [26] NCGIA, Project Varenius (1998), http://www.ncgis.ucsb.edu/varenius/ [27] Crang, M., Crang, P., May, J.: Introduction in Virtual Geography: Bodies, Space, and Relations. In: Crang, M., Crang, P., May, J. (eds.), pp. 1–20. Routledge, London (1999) [28] Dodge, M.: Cybergeography. Environment and Planning B: Planning and Design 28, 1–2 (2001) [29] Tang, D.: Experimental Geography and Geographical Engineering. Geographical Research 16(1), 1–10 (1997) [30] Quinn, B.: Immersive 3D Simulator-based GIS. Bay Area Automated Mapping Association, 3–16 (2009) [31] Roush, W.: Second Earth. Technology Review 7/8, 39–48 (2007)

Dual Reality: Merging the Real and Virtual Joshua Lifton and Joseph A. Paradiso MIT Media Lab

Abstract. This paper proposes the convergence of sensor networks and virtual worlds not only as a possible solution to their respective limitations, but also as the beginning of a new creative medium. In such a “dual reality,” both real and virtual worlds are complete unto themselves, but also enhanced by the ability to mutually reflect, influence, and merge by means of sensor/actuator networks deeply embedded in everyday environments. This paper describes a full implementation of a dual reality system using a popular online virtual world and a humancentric sensor network designed around a common electrical power strip. Example applications (e.g., browsing sensor networks in online virtual worlds), interaction techniques, and design strategies for the dual reality domain are demonstrated and discussed. Keywords: dual reality, virtual worlds, sensor network.

1

Introduction

At the heart of this paper is the concept of “dual reality,” which is defined as an environment resulting from the interplay between the real world and the virtual world, as mediated by networks of sensors and actuators. While both worlds are complete unto themselves, they are also enriched by their ability to mutually reflect, influence, and merge into one another. The dual reality concept, in turn, incorporates two key ideas – that data streams from real-world sensor networks are the raw materials that will fuel creative representations via interactive media that will be commonly experienced, and that online 3D virtual worlds are an ideal venue for the manifestation and interactive browsing of the content generated from such sensor data streams. In essence, sensor networks will turn the physical world into a palette, virtual worlds will provide the canvas on which the palette is used, and the mappings between the two are what will make their combination, dual reality, an art rather than an exact science. Of course, dual reality media will complement rather than replace other forms of media. Indeed, the end product, that which can be consumed and shared, is unlikely to outwardly resemble current forms of media, even if it is just as varied. Browsing the real world in a metaphorical virtual universe driven by a ubiquitous sensor network and unconstrained by physical boundaries approaches the concept of a digital “omniscience,” where users can fluidly explore phenomena at different locations and scales, perhaps also interacting with reality through distributed displays and actuators. Indeed, a complete consideration of dual reality must also include the possibility of “sensor” data from the F. Lehmann-Grube and J. Sablatnig (Eds.): FaVE 2009, LNICST 33, pp. 12–28, 2010. c Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering 2010 

Dual Reality: Merging the Real and Virtual

13

Fig. 1. An environmental taxonomy as viewed on the real-virtual axis (left). Sensor networks seamlessly merge real and virtual to form dual reality (right).

virtual world embodied in the real world. Insofar as technically feasible, dual reality is bi-directional – just as sensed data from the real world can be used to enrich the virtual world, so too can sensed data from the virtual world be used to enrich the real world. Of the many axes along which various virtual worlds can be compared, the most relevant for this work is the real-virtual axis, which indicates how much of the constructed world is real and how much virtual. See Figure 1. A rough taxonomy can further compartmentalize the real-virtual axis into reality, which is simply life in the absence of virtual representations of the world; augmented reality, which has all aspects of reality, as well as an “information prosthetic” which overlays normally invisible information onto real objects [1,2]; mixed reality, which would be incomplete without both its real and virtual components, such as the partially built houses made complete with blue screen effects for use in military training exercises [3]; and virtual reality, which contains only elements generated by a computer in an attempt to mimic aspects of the real world, as exemplified in some popular computer games [4]. Contrast this with the taxonomy given by Milgram and Kishino in [5]. Each of these environments represents what is supposed to be a single, complete, and consistent world, regardless of which components are real or virtual. Although this taxonomy can be successfully applied to most enhanced reality efforts, it does not address well the concept of dual reality, which comprises a complete reality and a complete virtual reality, both of which are enhanced by their ability to mutually reflect, influence, and merge into each other by means of deeply embedded sensor/actuator networks. See Figure 1.

2

Background

By their nature, sensor networks augment our ability to understand the physical world in ways beyond our innate capabilities. With sensor networks and a record of the data they generate, our senses are expanded in space, time, and modality. As with previous expansions of our ability to perceive the world, some of the first and perhaps in the long run most important upshots will be the stimulation of new creative media as artists working in dual reality strive to express sensed phenomena into strong virtual experiences. The work described

14

J. Lifton and J.A. Paradiso

here begins to explore directions for such self-expression as it takes shape in the interplay between sensor networks and virtual worlds. There is no definition of online virtual worlds that is both agreed upon and useful. The term itself is vague enough to encompass a full spectrum of technologies, from text-based multiple user domains (MUDs) originating in the late 1970s [6] to visually immersive online 3D games commercially available today [7,8]. This work primarily focuses on the concept of virtual world as introduced in science fiction works by authors such as William Gibson [9] and Neil Stephenson [10]. This type of online virtual world is characterized by an immersive 3D environment, fluid interactions among inhabitants, and some level of ability for inhabitants to shape their environment. The goal may not be, and probably should not be, to replicate all aspects of the real world, but rather only those that facilitate the interaction in a virtual environment. In light of this, imbuing virtual worlds with the ability to sense aspects of the real world is a technique with significant potential. The real world portions of this work use the 35-node Plug sensor network described in [11,12,13] and reviewed in a later section. The virtual world portions of this work focus exclusively on Second Life, an online virtual world launched in 2003 and today still maintained by Linden Lab [14]. A comprehensive review of all online virtual worlds is beyond the scope of this work and better left to the several websites that specialize in such comparisons [7,8,15]. Second Life was chosen because of its technical and other advantages in implementing many of the dual reality ideas explored here. For a more detailed introduction to Second Life, see Linden Lab’s official guide book and the Second Life website [16,14]. 2.1

Self-expression in Virtual Worlds

Virtual worlds today are largely social in nature – people enter these worlds in order to meet other people and build connections with them through shared experiences. As in the real world, social interactions in virtual worlds revolve around self-expression. Taking Second Life as a representative example of the state-of-the-art in this respect, a resident of Second Life can express herself via the appearance and name of her avatar, the information revealed in her avatar’s profile (favorite places, preferences, etc.), her avatar’s scripted or explicitly triggered actions (dancing, laughing, running, etc.), text chat on public channels (received only by those nearby in the virtual world), text chat on private channels (received by a user-determined list of people regardless of their location in the virtual world), and live voice chat using a headset. A typical encounter when meeting another person for the first time, especially someone new to Second Life, revolves around explanations of how names and appearances were chosen, elaborations of details in avatar profiles, and exhibitions of clothing or animations. A less explicit although arguably more compelling form of self-expression in Second Life is the ability to build objects, from necklaces to cars to castles, and imbue them with a wide range of behaviors. The skill level needed to do so, however, is on par with that needed to build compelling web sites. As such, this form of self-expression is limited to a small proportion of the total virtual

Dual Reality: Merging the Real and Virtual

15

world demographic. However, those who can build and script in Second Life can express themselves to a far wider audience than those who cannot. Compared to the real world, self-expression in Second Life and other virtual worlds is limited; missing are rich sources of information taken for granted in the real world, such as scent, body language, and the telltale signs of daily wear and tear. It’s not that these sources of information were forgotten, just that they are difficult to emulate in meaningful ways in the virtual world. For example, virtual wind causes virtual trees to sway, a virtual sun and moon rise and set periodically, and virtual clouds form and disperse in Second Life, but there is no meaning or cause behind any of these phenomena and their effect on the virtual world is superficial at best. Overall, the demand for richer forms of selfexpression in virtual worlds is apparent. Data collected from real-world sensor networks can help meet this demand by importing into the virtual world the inherent expressiveness of the real world. 2.2

The Vacancy Problem

The vacancy problem is the noticeable and profound absence of a person from one world, either real or virtual, while they are participating in the other. Simply put, the vacancy problem arises because people do not currently have the means to be in more than one place (reality) at a time. In the real world, the vacancy problem takes the form of people appearing completely absorbed in themselves, ignoring everything else. In the virtual world, the vacancy problem takes the form of virtual metropolises appearing nearly empty because there are not enough avatars to fill them. In part, this virtual vacancy is due to technical barriers preventing large numbers (hundreds) of people from interacting within the same virtual space. However, the vacancy problem will remain, even as processor speeds, network bandwidth, and graphics fidelity increase to overcome these technical difficulties. In a world nearly unconstrained by geography or physics, the currency of choice is people rather than real estate or possessions. As of this writing, there are over 10 million registered Second Life accounts, but only about 50,000 users logged into Second Life at any given time [17], providing a population density of 10 people per square kilometer (vs. over 18,000 for real-world Manhattan). The vacancy problem is a fundamental characteristic of today’s virtual worlds. More closely linking the real world with the virtual world, as the dual reality concept suggests, can work to mitigate the vacancy problem – just as real cities require special infrastructure to allow for a high population density, so too will virtual cities. We can envision people continuously straddling the boundary between real and virtual through “scalable virtuality”, where they are never truly offline, as sensor networks and mobile devices serve to maintain a continuous background inter-world connection (an early exploration of this idea was given in [18]). This can be tenuous, with virtual avatars passively representing some idea of the user’s location and activity and the virtual world manifesting into reality through ambient display, or immersive, with the user fully engaged in manipulating their virtual presence.

16

2.3

J. Lifton and J.A. Paradiso

Mapping between Realities

There are numerous challenges in designing exactly how the real and virtual will interact and map onto each other. A direct mapping of the real to virtual and virtual to real may not be the most appropriate. For example, the sensor data streams collected from a real person may be better mapped to the virtual land the person’s avatar owns rather than to the avatar itself. One possible mapping strategy is to shape the virtual world according to our subjective perceptions of the real world. In essence, the virtual world would be a reflection of reality distorted to match our mind’s eye impressions as discerned by a network of sensors. For example, the buildings on a virtual campus could change in size according to the number of inhabitants and virtual corridors could widen or lengthen according to their actual throughput. 2.4

Related Work

Work that couples the real world with virtual worlds falls into several broad categories. There are several efforts to bring a virtual world into the real world by using positioning and proximity systems to cast real people as the actors of an otherwise virtual world, such as Human Pacman [19], Pac Manhattan [20], ARQuake [21], and DynaDOOM [22]. Such work remains almost exclusively within the realm of converting video games into live action games and, aside from location awareness, does not incorporate other sensing modalities. Magerkurth et al. provide a good overview of this genre of pervasive games, as well as other more sensor-rich but physically confined games [23]. In an attempt to make Second Life more pervasive in the real world, Comverse has created a limited Second Life interface for cell phones [24]. Virtual worlds are being used to involve citizens in the collaborative planning of real urban areas [25], although this type of system relies more on GIS data than sensor networks embedded in the environment. More advanced and correspondingly more expensive systems are used for military training [26]. Most of the systems mentioned above support only a handful of simultaneous users. Among efforts to bring the real world into the virtual world, it is standard practice to stream audio and video from live real events, such as conferences and concerts, into Second Life spaces built specifically for those events [27]. More ambitious and not as readily supported by existing technologies is the IBM UK Laboratories initiative in which the state of light switches, motorized blinds, the building’s electricity meter, and the like in a real lab space are directly reflected and can be controlled in a Second Life replication [28]. Similar efforts on a smaller scale include a general-purpose control panel that can be manipulated from both the real world and Second Life [29], and a homebrewed virtual reality wearable computer made specifically to interface to Second Life [30]. The convergence of Second Life, or something like it, with popular real-world mapping software to form a “Second Earth” has been broadly predicted [31]. Uses of such a “hyper reality” include analyzing real-world data (“reality mining”), as was done in the Economic Weather Map project [32]. Such ideas have appeared

Dual Reality: Merging the Real and Virtual

17

before as interactive art pieces. For example, the Mixed Realities juried art competition organized by Turbulence (a net art commissioning organization [33]) in collaboration with Ars Virtua (a media center and gallery within Second Life [34]) recognizes projects that mix various aspects of the real and virtual [35]. Sensor network-enabled dual realities may naturally merge with or evolve from the life logging work pioneered by Gordon Bell [36,37] and popularized by web applications such as MySpace [38], Facebook [39], and Twitter [40]. Central to the dual reality concept is the expressive and social intent of the participants, which separates dual reality from the broader field of information visualization [41,42]. For example, consider services like Google Maps [43] and Traffic.com [44], that visualizes traffic congestion in a large metropolitan area. Traffic information might be gathered from numerous sources, such as cell towers, arial imagery, or user input, and displayed in a variety of ways, such as on the web, in a 3D virtual environment, or text messaging. The primary use of this service is to allow participants to intelligently plan their daily commute. Although hardly social by most standards, this service does form a social feedback loop; a user of the service will change her route according to the data presented and in doing so change the nature of the data presented to the next user. However, the motivation or intent of the service is entirely devoid of self-expression, and therefore does not readily fall under the rubric of dual reality. Closer to dual reality is VRcontext’s ProcessLife technology [45], which uses high-fidelity 3D virtual replicas of real environments to visualize and remotely influence industrial processes in real-time, though the potential for social interaction and rich metaphor appears low, as does the granularity of the sensor data visualizations.

3 3.1

Design and Implementation Real World Implementation

This work utilizes the previously developed “Plug” sensor network comprising 35 nodes modeled on a common electrical power outlet strip and designed specifically for ubiquitous computing environments [11,12,13]. A Plug offers four standard US electrical outlets, each augmented with a precision transformer for sensing the electrical current and a digitally controlled switch for quickly turning the power on or off. The voltage coming into the Plug is also sensed. In addition to its electrical power sensing and control features, each Plug is equipped with two LEDs, a push button, small speaker, analog volume knob, piezo vibration sensor, microphone, light sensor, 2.4GHz low-power wireless transceiver, and USB 2.0 port. An external expansion port features a passive infrared (PIR) sensor motion sensor, SD removable memory card, and temperature sensor. All the Plug’s peripherals are monitored and controlled by an Atmel AT91SAM7S64 microcontroller, which is based on the 32-bit ARM7 core, runs at 48MHz, and comes with 16KB of SRAM and 64KB of internal flash memory. Figure 2 shows Plug node with and without the external expansion. An extensive library of modular firmware can be pieced together into applications at compile time.

18

J. Lifton and J.A. Paradiso

Fig. 2. A Plug sensor node with (right) and without (left) an external expansion

3.2

Virtual World Implementation

The following sections describe objects or effects in the Second Life virtual world that were designed as an example of interfacing with the real world through sensor networks. Everything in Second Life exists as some combination of land, avatars, objects, and scripts. Land in Second Life is mapped directly to Linden Lab server resources, such as computing cycles, memory, and bandwidth. Avatars are the virtual manifestation of real people using Second Life. Objects are built from one or more primitive three-dimensional solids (“prims”), such as spheres, cubes, tori, and cones. A script is a program written in the Linden Scripting Language (LSL) and placed in an object to affect the object’s behavior. Data Ponds. A single “data pond” is meant to be an easily distinguishable, locally confined representation of the sensor data from a single Plug node. See Figure 3. The data pond design consists of a cluster of waving stalks growing out of a puddle of water and an ethereal foxfire rising from among the stalks, as might be found in a fantastic swamp. The mapping between a Plug’s sensor data and its corresponding data pond is easily understood once explained, but still interesting even without the benefit of the explanation. The particular mapping used is detailed in Table 1. The data ponds allowed sensed phenomena in the physical world to be efficiently browsed virtually, and proved effective, for example, in seeing at a glance which areas of our lab were more active than others. A real version of the data pond complements the virtual version. The real version follows the virtual’s tentacle aesthetic by using a standard desk fan shrouded in a lightweight, polka dotted sheet of plastic. The air flow through the shroud and therefore the height, sound, and other idiosyncrasies of the shroud can be finely controlled by plugging the fan into the outlet of a Plug device and pulse width modulating the supply voltage accordingly. See Figure 3. Virtual Sensing. Whereas real sensor networks capture the low-level nuance of the real world, virtual sensor networks capture the high-level context of the

Dual Reality: Merging the Real and Virtual

19

Fig. 3. A virtual data pond reflects real data near a virtual wall (left) and a real data pond reflects virtual data near a real wall (right)

Table 1. The mapping from a real-world Plug’s sensor data to its corresponding virtual data pond Plug Sensor Data Pond Mapping Modality Attribute light stalk length the stalk height is proportional to the maximum light level over the most recent one-second window temperature stalk color the color of the stalks varies linearly from blue to yellow to red from 18◦ C to 29◦ C motion stalk motion the stalks sway gently when no motion is detected and excitedly when motion is detected over the most recent one-second window sound puddle size the diameter of the water puddle is proportional to the maximum sound level over the most recent one-second window electrical current fire intensity the height and intensity of the fire is proportional to the total average absolute value of the electrical current over the most recent one-second window

20

J. Lifton and J.A. Paradiso

Fig. 4. Side view of the final implementation of Shadow Lab, which includes data ponds. A human-sized avatar is standing in the foreground - our particular labspace is rendered in detail, while the rest of the building was represented by a map. In the background are buildings belonging to unrelated neighbors.

virtual world. For example, in reality, there are literally an infinite number of ways a person can touch a table, but in Second Life, there is exactly one. This work uses embedded and wearable virtual sensing schemes. The embedded sensing scheme entails seeding every object of interest in the virtual environment to be sensed with a script that detects when an avatar touches or otherwise interacts with the object and then reports back to a server external to Second Life with a full description of the interaction, including avatar position, speed, rotation, and identity. The wearable sensing scheme requires each avatar in the region of interest to wear a sensing bracelet. The sensing bracelet reports back to the same external server every five seconds with a full description of its avatar’s location, motion, and public channel chat. As incentive for avatars to wear the sensing bracelet, the bracelet also serves as an access token without which the avatar will be ejected from the region being sensed. Shadow Lab. Shadow Lab is a space in Second Life modeled after our real lab in which the Plug sensor network is deployed and exemplifies our real space to virtual space mapping. The primary feature of Shadow Lab is the to-scale two-dimensional floor plan of the third floor of our building. Only a small portion of the entire space is modeled in three dimensions. In part, this is due to the difficulty and resource drain of modeling everything in three dimensions. However, it is also a design decision reflecting the difficulty in maneuvering an avatar in a to-scale three dimensional space, which invariably feels too confining

Dual Reality: Merging the Real and Virtual

21

Fig. 5. Avatar metamorphosis (left to right) as real-world activity increases

due to wide camera angles, quick movements, and the coarseness of the avatar movement controls in Second Life. Moreover, the two-dimensional design lends itself more readily to viewing the entire space at once and drawing attention to what few three-dimensional objects inhabit it. Figure 4 shows the latest version of Shadow Lab, which consists of the map of the lab, approximately 30 data ponds positioned on the map according to the positions of their corresponding Plugs in the real lab, and a video screen displaying a live video stream, when available, from a next-generation Tricorder [13] device equipped with a camera. Metamorphosis. The only unintentional body language exhibited in Second Life is the typing gesture avatars make when the user is typing a chat message, the slumped over sleeping stance assumed when the user’s mouse and keyboard have been inactive for a preset amount of time, automatically turning to look at nearby avatars who have just spoken, and a series of stances randomly triggered when not otherwise moving, such as hands on hips and a bored slouch. All other body language and avatar actions must be intentionally chosen by the user. Clearly, there is room for improvement. Metamorphosis explores mapping real space to a virtual person. See Figure 5. In this prototype, the avatar begins as a typical human and transforms into a Lovecraftian alien according to several parameters drawn from the sensor streams of the Plug sensor network spread throughout the real building. While this particular example is outlandish and grotesque, in practice the mapping used in a metamorphosis is arbitrary, which is exactly its appeal as a method of self-expression – metamorphosis can be mapped to other arbitrary stimuli and unfold in any fashion. Virtual Atrium. The translation of our lab’s atrium into Second Life attempts to retain that which is iconic about the original and at the same time take advantage of the freedom of the virtual world. See Figure 6. The virtual atrium is defined by the intersection of two perpendicular walls of tile, one representing the total activity level of the real world as sensed by the Plug network and the one representing the total activity of the virtual world as sensed by the virtual sensing systems mentioned above. The physical extent and color scheme of the virtual atrium walls change accordingly. Each tile has a blank white front face, four colored sides, and a black back face. Touching a tile will cause it to flip over, at which point the black back face comes to the front and changes to reveal a

22

J. Lifton and J.A. Paradiso

Fig. 6. The real lab atrium (left) and the virtual version (right). A real person and an avatar show their respective scales.

Fig. 7. Side view of the Ruthenium region

hidden movie or image. All tiles in a given wall share the same image or movie when flipped, although the exact image or movie displayed is variable. Dual Reality Open House. At the time of this writing, the state-of-the-art in large events that bridge the real and virtual worlds amounts to what is essentially video conferencing between a real auditorium and a virtual auditorium [46]. As a prototype demonstration of moving beyond this by employing sensor networks, a dual reality open house was constructed to introduce residents of Second Life to the lab and visitors of the lab to Second Life. The dual reality open house premiered at a one-day technical symposium and held in the atrium of our lab [47]. The real portion of the event consisted of talks and panel discussions in the building’s main auditorium, interspersed with coffee breaks and standup meals in the atrium among tables manned by lab students demonstrating various lab projects related to virtual worlds. The virtual portion of the open house was located in a typical 256-meter by 256-meter region of Second Life [48] called “Ruthenium.” The server running the Ruthenium region is limited to 40 simultaneous avatars and 15,000 simultaneous prims. In preparation for the open

Dual Reality: Merging the Real and Virtual

23

house, Ruthenium was terraformed and filled with static information kiosks and live demonstrations of various projects from around the lab. More details about the projects displayed can be found in [11]. The virtual atrium described in 3.2 framed the space where the virtual portion of our event took place. Data ponds and an avatar metamorphosis were featured as well. See Figure 7. The entire Ruthenium region employs the virtual sensing schemes described earlier.

4

Dual Reality Event and Discussion

The dual reality open house described earlier has the potential to explore the real data and virtual data collection systems. (See [12,11] for more detailed evaluations of the Plug sensor network.) Sensor data from both the real world and virtual world were collected during the day-long event. The real-world data originated from the Plug sensor nodes used throughout the real lab atrium at the various open house demo stations. Motion, sound, and electrical current data from a typical Plug are shown in Figure 8. Also collected but not shown here are data for each Plug’s light, voltage, vibration, and temperature sensors. The virtual-world data originated from the virtual sensing system previously detailed as deployed throughout the virtual portion of the dual reality open house described earlier. Such an extensive data set from a single event spread across both real and virtual worlds had not previously been collected. By the nature of the event and its presentation in each world, very little correlation between the real and virtual data was expected. However, each data set does speak to how people interact within each world separately and what the possibilities are for using data from one world in the other. The real-world sound and motion data shown in Figure 8 clearly follows the structure of the event as attendees alternate between the atrium during break times and the auditorium during the conference talks - the auditorium is noisier during breaks, during which demo equipment was also generally switched on and people are moving around the demos. On the other hand, the light data (not shown) indicate physical location more than attendee activity – direct sunlight versus fluorescent lights versus LCD projector light. See [11] for more detail. Of the various data collected from the virtual world during the day-long event, Figure 9 shows the distribution over time of touch events (avatars touching a virtual object equipped with the virtual embedded sensing system) and avatar movement events (the virtual wearable sensing system checks if its avatar is moving approximately once per second) collected from 22 avatars, of which 16 chose to wear the access bracelet virtual sensing system. Due to a network glitch, data collected from the virtual sensing system started being logged at approximately 11 AM rather than at 8 AM, when the event actually started. The spike of avatar movement at around noon is likely due to the pause in the live video stream from the auditorium when the talks broke for lunch, thus giving avatars watching the video stream incentive to move to another location to interact with other aspects of the virtual space. The relatively constant motion thereafter might indicate the exploratory nature of the participants and/or the space. Of all avatar-object

24

J. Lifton and J.A. Paradiso

Fig. 8. Electrical current, sound level, and motion versus time from a typical Plug node during the dual reality open house

interactions, 83% were between an avatar and a virtual atrium wall tile, that displayed the live video feed from the real auditorium. This trial could have been improved in several respects. For example, the number of virtual attendees could have been increased with better advertising. Also (and most crucially), a stronger connection between real and virtual premises could have been made and “connectedness” metrics formulated and tested. These are being addressed in another dual reality event that we are hosting soon. 4.1

Discussion

In a completely fabricated virtual world, the entropy of a real-world data stream can dramatically alter the virtual ambiance. Certainly, a cleverly utilized pseudorandom number generator could do the same, but meaning derives more from perception than from the underlying mechanism, and it is much easier to weave a story from real data than from pseudo-random numbers. The act of weaving a story from sensor data is essentially the act of designing and implementing a mapping from data to a real or virtual manifestation of the data. A successful story must be meaningful to tell as well as to hear, and using sensor data grounded in either the real or virtual world helps achieve this. In essence, the act of creation must be as gratifying as the act of consumption. The creative aspects of dual reality, the mapping of real or virtual sensor data to some manifestation, will likely follow the trend of another recent medium – blogs. While blogs have allowed some creative geniuses an outlet and given them a wide, appreciative, and well-deserved audience, the quality of most blogs, at least as a consumptive medium, is far below previous mass media standards. Of course, their quality as a creative medium and the value they bring to their creators in that regard far exceed previous standards by virtue of their relatively low barrier to entry alone. These trends will be exaggerated in the context of dual reality for two reasons. First, the medium is much richer, involving virtual 3D worlds and complex social interactions and is therefore accessible to a wider

Dual Reality: Merging the Real and Virtual

25

Fig. 9. Avatar movement and interaction during the dual reality open house

audience. Second, once the mapping of data to manifestation is set, the act of creation is nearly automatic (sitting somewhere between an interactive installation and a performance) and therefore a wider range of talent will participate. In short, the worst will be worse and the best will be better, a hallmark of successful mass media. As with other creative media, virtuosity will still play a critical role in dual reality, namely in the conception, implementation, and honing of the specific mappings between sensor data and their manifestations. These ideas are further discussed in [49]. While mapping sensor data to manifestation may be at the highest level of the dual reality creative process, once the mappings are in place, people can still intentionally express themselves in many ways, depending on the exact nature of the mapping. The evolution of emoticons in text messages is one example of such expression using a current technology. Another is the habit of maintaining an active online presence, such as used in Internet messaging clients, by jogging the computer’s mouse occasionally. In the same way, users of dual reality environments will modify their behavior so as to express themselves through the medium.

5

Conclusion

Various technologies have fundamentally altered our capacity to consume, share, and create media. Most notably, television and radio made consumption

26

J. Lifton and J.A. Paradiso

widespread and the Internet made sharing widespread. In comparison, creation of media is still difficult and limited to a small subset of the population. The promise of dual reality is to use sensor/actuator networks as a generative tool in the process of transforming our everyday experiences in the real world into content shared and experienced in the virtual world. Just as the data created by a movie camera are shared and consumed in a theater, the data collected from sensor networks will be shared and consumed in virtual worlds. This holds the potential to revolutionize sensor network browsing, as participants fluidly explore metaphoric representations of sensor data - similarly, virtual denizens can manifest into real spaces through display and actuator networks. If sensor networks are the palette, then virtual worlds are the canvas that usher in a new form of mass media.

References 1. Feiner, S., et al.: Knowledge-based Augmented Reality. Comm. of the ACM 36(7), 53–62 (1993) 2. Sportvision. Virtual Yellow 1st and Ten (1998), http://www.sportvision.com/ 3. Dean Jr., F.S., et al.: Mixed Reality: A Tool for Integrating Live, Virtual & Constructive Domains to Support Training Transformation. In: Proc. of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) (2004) 4. Electronic Arts. SimCity (2007), http://simcity.ea.com/ 5. Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Trans. of Information Systems E77-D(12) (December 1994) 6. Rheingold, H.: The Virtual Community: Homesteading on the Electronic Frontier. Addison-Wesley, Reading (1993) 7. Good, R.: Online Virtual Worlds: A Mini-Guide (April 2007), http://www.masternewmedia.org/virtual reality/virtual-worlds/ virtual-immersive-3D-worlds-guide-20071004.htm 8. B. Book. Virtual Worlds Review (February 2006), http://www.virtualworldsreview.com/ 9. Gibson, W.: Neuromancer. Ace Books (1984) 10. Stephenson, N.: Snow Crash. Bantam Books (1992) 11. Lifton, J.: Dual Reality: An Emerging Medium. Ph.D. Dissertation, M.I.T., Dept. of Media Arts and Sciences (September 2007) 12. Lifton, J., et al.: A Platform for Ubiquitous Sensor Deployment in Occupational and Domestic Environments. In: Proc. of the Sixth Int’l Symposium on Information Processing in Sensor Networks (IPSN), April 2007, pp. 119–127 (2007) 13. Lifton, J., et al.: Tricorder: A mobile sensor network browser. In: Proc. of the ACM CHI 2007 Conference - Mobile Spatial Interaction Workshop (April 2007) 14. Linden Lab. Second Life (2003), http://www.secondlife.com 15. Lifton, J.: Technology Evaluation for Marketing & Entertainment Virtual Worlds. Electric Sheep Co. Report (2008), http://www.electricsheepcompany.com/publications/ 16. Rymaszewski, M., et al.: Second Life: The Official Guide. Wiley, Chichester (2007) 17. Linden Lab. Economic Statistics (2007), http://secondlife.com/whatis/economy_stats.php

Dual Reality: Merging the Real and Virtual

27

18. Musolesi, M., et al.: The Second Life of a Sensor: Integrating Real-world Experience in Virtual Worlds using Mobile Phones. In: Fifth ACM Workshop on Embedded Networked Sensors (HotEmNets) (June 2008) 19. Cheok, A.D., et al.: Human Pacman: A Mobile Entertainment System with Ubiquitous Computing and Tangible Interaction over a Wide Outdoor Area. In: Fifth Int’l Symposium on Human Computer Interaction with Mobile Devices and Services (Mobile HCI), September 2003, pp. 209–223 (2003) 20. PacManhattan (2004), http://pacmanhattan.com 21. Thomas, B., et al.: ARQuake: An Outdoor/Indoor Augmented Reality First Person Application. In: Fourth Int’l Symposium on Wearable Computers (ISWC 2000) (2000) 22. Sukthankar, G.: The DynaDOOM Visualization Agent: A Handheld Interface for Live Action Gaming. In: Workshop on Ubiquitous Agents on Embedded, Wearable, and Mobile Devices (Conference on Intelligent Agents & Multiagent Systems) (July 2002) 23. Magerkurth, C., et al.: Pervasive Games: Bringing Computer Entertainment Back to the Real World. ACM Computers in Entertainment 3(3) (July 2005) 24. Roush, W.: New Portal to Second Life: Your Phone. Technology Review (2007), http://www.technologyreview.com/Infotech/18195/ 25. MacIntyre, J.: Sim Civics. Boston Globe (August 2005), http://www.boston.com/news/globe/ideas/articles/2005/08/07/ sim civics/ 26. Miller, W.: Dismounted Infantry Takes the Virtual High Ground. Military Training Technology 7(8) (December 2002) 27. Jansen, D.: Beyond Broadcast 2007 – The Conference Goes Virtual: Second Life (2006), http://www.beyondbroadcast.net/blog/?p=37 28. IBM. Hursley Island (2007), http://slurl.com/secondlife/Hursley/0/0/0/ 29. ciemaar. Real Life Control Panel for Second Life (2007), http://channel3b.wordpress.com/2007/01/24/ real-life-control-panel-for-second-life/ 30. Torrone, P.: My wearable computer – snowcrash (January 2006), http://www.flickr.com/photos/pmtorrone/sets/1710794/ 31. Roush, W.: Second Earth. Technology Review 110(4), 38–48 (2007) 32. Boone, G.: Reality Mining: Browsing Reality with Sensor Neworks. Sensors Magazine 21(9) (September 2004) 33. Turbulence (2007), http://www.turbulence.org/ 34. Ars Virtua (2007), http://arsvirtua.org/ 35. Turbulence. Mixed Realities Commissions (2007), http://transition.turbulence.org/comp_07/awards.html 36. Bell, G.: A Personal Digital Store. Comm. of the ACM 44(1), 86–91 (2001) 37. Gemmell, J., et al.: MyLifeBits: A Personal Database for Everything. Comm. of the ACM 49(1), 88–95 (2006) 38. MySpace (2007), http://www.myspace.com/ 39. Facebook (2007), http://www.facebook.com/ 40. Twitter (2007), http://twitter.com/ 41. Tufte, E.R.: The Visual Display of Quantitative Information. Graphics Press (1983) 42. Chen, C.: Information Visualisation and Virtual Environments. Springer, Heidelberg (1999)

28 43. 44. 45. 46.

J. Lifton and J.A. Paradiso

Google. Google Maps (2007), http://maps.google.com Navteq. Traffic.com (2007), http://www.traffic.com VRcontext. ProcessLife (February 2009), http://www.vrcontext.com/ Verbeck, S.: Founder and CEO of The Electric Sheep Company. Personal comm. via e-mail, July 9 (2007) 47. IBM. Virtual Worlds: Where Business, Society, Technology, & Policy Converge (June 15, 2007), http://www.research.ibm.com/research/press/virtualworlds_agenda.shtml 48. Lifton, J.: Media Lab Dual Reality Open House (2007), http://slurl.com/secondlife/Ruthenium/0/0/0/ 49. Lifton, J., Laibowitz, M., Harry, D., Gong, N., Mittal, M., Paradiso, J.A.: Metaphor and Manifestation: Cross Reality with Ubiquitous Sensor/Actuator Networks. IEEE Pervasive Computing Magazine (Summer 2009)

Exploring the Use of Virtual Worlds as a Scientific Research Platform: The Meta-Institute for Computational Astrophysics (MICA) S.G. Djorgovski1,∗, P. Hut2,*, S. McMillan3,*, E. Vesperini3,*, R. Knop3,*, W. Farr4,*, and M. J. Graham1,* 1

California Institute of Technology, Pasadena, CA 91125, USA The Institute for Advanced Study, Princeton, NJ 08540, USA 3 Drexel University, Philadelphia, PA 19104, USA 4 Massachusetts Institute of Technology, Cambridge, MA 02139, USA [email protected] 2

Abstract. We describe the Meta-Institute for Computational Astrophysics (MICA), the first professional scientific organization based exclusively in virtual worlds (VWs). The goals of MICA are to explore the utility of the emerging VR and VWs technologies for scientific and scholarly work in general, and to facilitate and accelerate their adoption by the scientific research community. MICA itself is an experiment in academic and scientific practices enabled by the immersive VR technologies. We describe the current and planned activities and research directions of MICA, and offer some thoughts as to what the future developments in this arena may be. Keywords: Virtual Worlds; Astrophysics; Education; Scientific Collaboration and Communication; Data Visualization; Numerical Modeling.

1 Introduction Immersive virtual reality (VR), currently deployed in the form of on-line virtual worlds (VWs) is a rapidly developing set of technologies which may become the standard interface to the informational universe of the Web, and profoundly change the way humans interact with information constructs and with each other. Just as the Web and the browser technology has changed the world, and almost every aspect of modern society, including scientific research, education, and scholarship in general, a synthesis of the VR and the Web promises to continue this evolutionary process which intertwines humans and the world of information and knowledge they create. Yet, the scientific community at large seems to be at best poorly informed (if aware at all) of this technological emergence, let alone engaged in spearheading the developments of the new scientific, educational, and scholarly modalities enabled by these technologies, or even new ideas which may translate back into the better ways ∗

All authors are also associated with the Meta-Institute for Computational Astrophysics (MICA), http://mica-vw.org

F. Lehmann-Grube and J. Sablatnig (Eds.): FaVE 2009, LNICST 33, pp. 29–43, 2010. © Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering 2010

30

S.G. Djorgovski et al.

in which these technologies can be used for practical and commercial applications outside the world of academia. There has been a slowly growing interest and engagement of the academic community in the broad area of humanities and social sciences in this arena (see, e.g., [1, 2, 3, 4, 5], and references therein), but the “hard sciences” community has barely touched these important and potentially very powerful developments. While a few relatively isolated individuals are exploring the potential uses of VWs as a scholarly platform, the scientific/academic community as a whole has yet to react to these opportunities in a meaningful way. One reason for this negligence may be a lack of the real-life examples of the scientific utility of VWs. It is important to engage the scientific community in serious uses and developments of immersive VR technologies. With this growing set of needs and opportunities in mind, following some initial explorations of the VWs as a scholarly interaction and communication platform [6, 7], we formed the Meta-Institute for Computational Astrophysics (MICA) [8] in the spring of 2008. Here we describe the current status and activities of MICA, and its long-term goals.

2 The Meta-Institute for Computational Astrophysics (MICA) To the best of our knowledge, MICA is the first professional scientific organization based entirely in VWs. It is intended to serve as an experimental platform for science and scholarship in VWs, and it will be the organizing framework for the work proposed here. MICA is currently based in Second Life (SL) [9] (it initially used the VW of Qwaq [10]), but it will expand and migrate to other VWs and venues as appropriate. The charter goals of MICA are: 1. Exploration, development and promotion of VWs and VR technologies for professional research in astronomy and related fields. 2. To provide and develop novel social networking venues and mechanisms for scientific collaboration and communications, including professional meetings, effective telepresence, etc. 3. Use of VWs and VR technologies for education and public outreach. 4. To act as a forum for exchange of ideas and joint efforts with other scientific disciplines in promoting these goals for science and scholarship in general. To this effect, MICA conducts weekly professional seminars, bi-weekly popular lectures, and many other regularly scheduled and occasional professional discussions and public outreach events, all of them in SL. Professional members of MICA include scientists (faculty, staff scientists, postdocs, and graduate students), technologists, and professional educators; about 40 people as of this writing (March 2009). A broader group of MICA affiliates includes members of the general public interested in learning about astronomy and science in general; it currently consists of about 100 people (also as of March 2009). The membership of both groups is growing steadily. We have been very proactive in engaging both academic community (in real life and in SL) and general public, in the interests of our stated goals. Both our membership and activities are global in scope, with participants from all over the world, although a majority resides in the U.S.

Exploring the Use of Virtual Worlds as a Scientific Research Platform

31

MICA is thus a testbed and a foothold for science and scholarship in VWs, and we hope to make it both a leadership institution and a center of excellence in this arena, as well as an effective portal to VWs for the scientific community at large. While our focus is in astrophysics and related fields, where our professional expertise is, we see MICA in broader terms, and plan to interact with scientists and educators in other disciplines as well. We also plan to develop partnerships with the relevant industry laboratories, and conduct joint efforts in providing innovation in this emerging and transformative technology. The practical goals of MICA are two-fold. First, we wish to lead by example, and demonstrate the utility of VWs and immersive VR environments generally for scientific research in fields other than humanities and social sciences (where we believe the case is already strong). In that process, we hope to define the “best practices” and optimal use of VR tools in research and education, including scholarly communications. This is the kind of activity that we expect will engage a much broader segment of the academic community in exploration and use of VR technologies. Second, we hope to develop new research tools and techniques, and help lay the foundations of the informational environments for the next generation of VR-enabled Web. Specifically, we are working in the following directions: 2.1 Improving Scientific Collaboration and Communication Our experience is that an immediate benefit of VWs is as an effective scientific communication and collaboration platform. This includes individual, group, or collaboration meetings, seminars, and even full-scale conferences. You can interact with your colleagues as if they were in the same room, and yet they may be half way around the world. This is a technology which will finally make telecommuting viable, as it provides a key element that was missing from the flat-Web paradigm: the human interaction. We finally have a “virtual water cooler”, the collegial gathering work spaces to enhance and expand our cyber-workspaces. VWs are thus a very green technology: you can save your time, your money, and your planet by not traveling if you don't have to. This works well enough already, at almost no cost, and it will get better as the interfaces improve, driven by the games and entertainment industry, if nothing else. This shift to virtual meetings can potentially save millions of dollars of research funding, which could be used for more productive purposes than travel to collaboration or committee meetings, or to conferences of any kind. We have an active program of seminars, lectures, collaboration meetings, and freeform scholarly discussions within the auspices of MICA, and we are proactive in informing our real-life academic community about these possibilities. We offer coaching and mentoring for the novices, and share our experiences on how to best use immersive VR for scientific communication and collaboration with other researchers. In addition, starting in a near future, we plan to organize a series of topical workshops on various aspects of computational science (both general, and specific to astrophysics), as well as broader-base annual conferences on science and scholarship in VWs, including researchers, technologists, and educators from other disciplines. These meetings will be either entirely based in VWs (SL to start), or be in “mixed reality”, with both real-life and virtual environment gatherings simultaneously, connected by streaming media.

32

S.G. Djorgovski et al.

Fig. 1. MICA members attending a regular weekly astrophysics seminar, in this case by Dr. M. Trenti, given in the StellaNova sim in SL. Participants in these meetings are distributed worldwide, but share a common virtual space in which they interact.

Genuine interdisciplinary cross-fertilization is a much-neglected path to scientific progress. Given that many of the most important challenges facing us (e.g., the global climate change, energy, sustainability, etc.) are fundamentally interdisciplinary in nature, and not reducible to any given scientific discipline (physics, biology, etc.), the lack of effective and pervasive mechanisms for establishment of inter-, multi-, or cross-disciplinary interactions is a serious problem which affects us all. One reason for the pervasive academic inertia in really engaging in true and effective interdisciplinary activities is the lack of easy communication venues, intellectual melting pots where such encounters can occur and flourish. VWs as scientific interaction environments offer a great new opportunity to foster interdisciplinary meetings of the minds. They are easy, free, do not require travel, and the social barriers are very low and easily overcome (the ease and the speed of striking conversations and friendships is one of the more striking features of VWs). To this end, we will establish a series of broad-based scientific gatherings, from informal small group discussions, to full-size conferences. We note that once a VR environment is established, e.g., in a “sim” in SL, the cost (in both time and money) of organizing conferences is almost negligible, and the easy and instant worldwide access with no physical travel makes them easy to attend. Thus, we have developed a dedicated “MICA island” (sim), named StellaNova [11] within SL. This is intended to be the Institute’s home location in VWs; it is currently in SL as the most effective and convenient venue, but we will likely expand and migrate to other VW venues when that becomes viable and desirable. StellaNova is used as a staging area for most of our activities, including meetings, workshops, discussions, etc. It is intended to be a friendly and welcoming virtual environment for scholarly collaborations and discussions, very much in the tradition of academe of the golden age Athens.

Exploring the Use of Virtual Worlds as a Scientific Research Platform

33

A part of our exploration of VWs as scientific communication and collaboration platforms is an investigation in the mixed use of traditional Web (1.0, 2.0, … 3.0?) and VR tools; we are interested in optimizing the uses of information technology for scientific communications generally, and not just exclusively in a VR context, although a VR component would always be present. We plan to evaluate the relative merits of these technologies for different aspects of professional scientific and scholarly interaction and networking – while the Web mechanisms may be better for some things, VWs may be better for others. Finally, we intend to investigate the ways in which immersive VR can be used as a part of scientific publishing, either as an equivalent of the current practice of supplementing traditional papers with on-line material on the Web, or even as a primary publishing medium. Just as the Web offers new possibilities and modalities for scholarly publishing which do not simply mimic the age-old printed-paper media publishing, so we may find qualitatively novel uses of VWs as a publishing venue in their own right. After all, what is important is the content, and not the technical way in which the information is encoded; and some media are far more effective than others in conveying particular types of scholarly content. 2.2 A New Approach to Numerical Simulations Immersive VR environments open some intriguing novel possibilities in the ways in which scientists can set up, perform, modify, and examine the output of numerical simulations. In MICA, we use as our primary science environment the gravitational N-body problem, since that is where our professional expertise is concentrated [12, 13, 14, 15, 16, 17], but we expect that most of the features we develop will find much broader applicability in the visualization of more general scientific or abstract data sets. Our goal is to create virtual, collaborative visualization tools for use by computational scientists working in an arbitrary VW environment, including SL [9], OpenSim [18], etc. Here we address interactive and immersive visualization in the numerical modeling and simulations context; we address the more general issues of data visualization below. For an initial report, see [40]. We started our development of in-world visualization tools by creating scripts to display a set of related gravitational N-body experiments. The gravitational N-body problem is easy to state and hard to solve: given the masses, positions, and velocities of a collection of N bodies moving under the influence of their mutual Newtonian gravitational interactions, according to the laws of Newtonian mechanics, determine the bodies' positions and velocities at any subsequent time. In most cases, the motion has no analytic solution, and must be computed numerically. Both the character of the motion and the applicable numerical techniques depend on the scale of the system. Most of the essential features of the few-body problem can be grasped from studies of the motion of 3-5 body systems, in bound or scattering configurations. The physics and basic mathematics are elementary, and the required programming is straightforward. Yet, despite these modest foundations, such systems yield an extraordinarily rich spectrum of possible outcomes. The idea that simple deterministic systems can lead to complex, chaotic results is an important paradigm shift in many students' perception of physics. Few-body dynamics is also critically important in the determining

34

S.G. Djorgovski et al.

the evolution and appearance of many star clusters, as well as the stability of observed multiple stellar systems. These systems are small enough that the entire calculation could be done entirely within VWs, although we would wish to preserve the option of also importing data from external sources. This tests the basic capabilities of the visualization system – updating particles, possibly interpolating their motion, stopping, restarting, running backwards, resetting to arbitrary times, zooming in and out, etc. The next level of simulation involves broadening the context of our calculations to study systems containing several tens of particles, which will allow us to see both the few-body dynamics and how they affect the parent system. Specifically, the study of binary interactions and heating, and the response of the larger cluster, will illustrate the fundamental dynamical processes driving the evolution of most star clusters. We will study the dynamics of systems containing binary systems, a possible spectrum of stellar masses, and real (if simplified) stellar properties. These simulations are likely to lie at the high end of calculations that can be done entirely within the native VW environments, and much of the data may have to be imported. The capacity to identify, zoom in on, and follow interesting events, and to change the displayed attributes of stars on the fly will be key to the visualization experience at this level. The evolution of very large systems, such as galaxies, is governed mainly by largescale gravitational forces rather than by small-scale individual interactions, so studies of galaxy interactions highlight different physics and entail quite different numerical algorithms from the previous examples. It will not be feasible to do these calculations within the current generation of VWs, or to stream in data fast enough to allow for animation, so the goal in this case will be to import, render, and display a series of static 3-D frames, which will nevertheless be “live” in the sense that particles of different sorts (stars, gas, dark matter, etc.) or with other user-defined properties can be identified and highlighted appropriately. The choice of N ~ 50,000 is small compared to the number of stars in an actual galaxy, and it is more typical of a large star cluster. However, with suitable algorithms, galaxies can be adequately modeled by simulations on this scale, and this choice of N is typical of low-resolution calculations of galaxy dynamics, such as galaxy collisions and mergers, that are often used for pedagogical purposes. It also represents a compromise in the total amount of data that can be transferred into the virtual environment in a reasonable time. The intent here will be to allow users to visualize the often complex 3D geometries of these systems, and to explore some of their dynamical properties. This visualization effort in this case will depend on efficient two-way exchange of data between the in-world presentation and the external engine responsible for both the raw data and the computations underlying many aspects of the display. Our first goal is thus to explore the interactive visualization of simulations running within the VWs computational environments, thus offering better ways to understand the physics of the simulated processes – essentially the qualitative changes in the ways scientists would interact with their simulations. Our second goal is to explore the transition regime where the computation is actually done externally, on a powerful or specialized machine, but the results are imported into a VW environment, while the user feedback and control are exported back, and determine the practical guidelines as to how and when such a transition should be deployed in a real-life numerical study of astrophysical systems. The insights gained here would presumably be portable to

Exploring the Use of Virtual Worlds as a Scientific Research Platform

35

Fig. 2. A MICA astrophysicist immersed in, and interacting with, a gravitational N-body simulation using the OpenSim environment

36

S.G. Djorgovski et al.

other disciplines (e.g., biology, chemistry, other fields of physics, etc.) where numerical simulations are the only option of modeling of complex systems. 2.3 Immersive Multi-Dimensional Data Visualization In a more general context, VWs offer intriguing new possibilities for scientific visualization or “visual analytics” [19, 20]. As the size, and especially the complexity of scientific data sets increase, effective visualization becomes a key need for data analysis: it is a bridge between the quantitative information contained in complex scientific measurements, and the human intuition which is necessary for a true understanding of the phenomena in question. Most sciences are now drowning under the exponential growth of data sets, which are becoming increasingly more complex. For example, in astronomy we now get most of our data from large digital sky surveys, which may detect billions of sources and measure hundreds of attributes for each; and then we perform data fusion across different wavelengths, times, etc., increasing the data complexity even further. Likewise, numerical simulations also generate huge, multi-dimensional output, which must be interpreted and matched to equally large and complex sets of measurements. Examples include structure formation in the universe, modeling of supernova explosions, dense stellar systems, etc. This is an even larger problem in biological or environmental sciences, among others. We note that the same challenges apply to visualization of data from measurements, numerical simulations, or their combination. How do we visualize structures (clusters, multivariate correlations, patterns, anomalies...) present in our data, if they are intrinsically hyper-dimensional? This is one of the key problems in data-driven science and discovery today. And it is not just the data, but also complex mathematical or organizational structures or networks, which can be inherently and essentially multi-dimensional, with complex topologies, etc. Effective visualization of such complex and highly-dimensional data and theory structures is a fundamental challenge for the data-driven science of the 21st century, and these problems will grow ever sharper, as we move from Terascale to Petascale data sets of ever increasing complexity. VWs provide an easy, portable venue for pseudo-3D visualization, with various techniques and tricks to encode more parameter space dimensions, with an added benefit of being able to interact with the data and with your collaborators. While there are special facilities like “caves” for 3D data immersion, they usually require a room, expensive equipment, special goggles, and only one person at a time can benefit from the 3D view. With an immersive VW on your laptop or a desktop, you can do it for free, and share the experience with as many of your collaborators as you can squeeze in the data space you are displaying, in a shared, interactive environment. These are significant practical and conceptual advantages over the traditional graphics packages, and if VWs become the standard scientific interaction venue as we expect, then bringing the data to the scientists only makes sense. Immersing ourselves in our data may help us think differently about them, and about the patterns we see. With scientists immersed in their data sets, navigating around them, and interacting with both the data and each other, new approaches to data presentation and understanding may emerge.

Exploring the Use of Virtual Worlds as a Scientific Research Platform

37

Fig. 3. MICA scientists in an immersive data visualization experiment, developed by D. Enfield and S.G. Djorgovski. Data from a digital sky survey are represented in a 6-dimensional parameter space (XYZ coordinates, symbol sizes, shapes, and colors).

We have conducted some preliminary investigation of simple multi-dimensional data visualization scripting tools within SL. We find that we can encode data parameter spaces with up to a dozen dimensions in an interactive, immersive pseudo-3D display. At this point we run into the ability of the human mind to easily grasp the informational content thus encoded. A critical task is to experiment further in finding the specific encoding modalities that maximize our ability to perceive multiple data dimensions simultaneously, or selectively (e.g., by focusing on what may stand out as an anomalous pattern). One technical challenge is the number of data objects that can be displayed in a particular VW environment; SL is especially limiting in this regard. Our next step is to experiment with visualizations in custom VW environments, e.g., using OpenSim [18], which can offer scalable solutions needed for the modern large data sets. However, even an environment like SL can be used for experimentation with modest-scale data sets (e.g., up to ~ 104 data objects), and used to develop the methods for an optimal encoding of highly-dimensional information from the viewpoint of human perception and understanding. Additional questions requiring further research include studies of combined displays of data density fields, vector fields, and individual data point clouds, and the ways in which they can be used in the most effective way. This is a matter of optimizing human perception of visually displayed information, a problem we will tackle in a purely experimental fashion, using VWs as a platform. The next level of complexity and sophistication comes with introduction of the time element, i.e., sequential visualization of changing data spaces (an obvious example is the output of numerical simulations of gravitational N-body systems, discussed in the previous section). We are all familiar with digital movies displaying such information in a 2-D format. What we are talking about here is immersive 3-D data cinematography, a novel concept, and probably a key to a true virtualization of

38

S.G. Djorgovski et al.

scientific research. Learning how to explore dynamical data sets in this way may lead to some powerful new ways in which we extract knowledge and understanding from our data sets and simulations. Implementing such data visualization environment poses a number of technical challenges. We plan experiment with interfacing of the existing visualization tools and packages with VW platforms: effectively, importing the pseudo-3D visualization signal into VWs, but with a goal of embedding the user avatar in the displayed space. We may be able to adopt some emergent solutions of this problem from the games or entertainment industry, should any come up. Alternatively, we may attempt to encode a modest-scale prototype system within the VW computational environments themselves. A hybrid approach may be also possible. 2.4 Exploring the OpenGrid and OpenSim Technologies Most of the currently open VWs are based on proprietary software architectures, formats, or languages, and do not interoperate with each other; they are closed worlds, and thus probably dead ends. OpenSimulator (or OpenSim) [18] is a VW equivalent of the open source software movement. It is an open-source C# program which implements the SL VW server protocol, and which can be used to create a 3-D VW, and includes facilities for creating custom avatars, chatting with others in the VR environment, building 3-D content and creating complex 3-D applications in VW. It can also be extended via loadable modules or Web service interfaces to build more custom 3-D applications. OpenSim is released under a BSD license, making it both open source, and commercially friendly to embed in products. To demonstrate the feasibility of this approach, we have conducted some preliminary experiments in the uses of OpenSim for astrophysical N-body simulations, using a plugin, MICAsim [21, 22]. We have modified the standard OpenSim physics engine as a plugin, to run gravitational N-body experiments in this VW environment. We found that it's practical to run about 30 bodies in a gravitational cold-collapse model with force softening to avoid hard binary interactions in the simulator, where a few simulator seconds corresponds to a crossing time. We believe that we could get another factor of two in N from code optimizations in this setting. We will continue to explore actively the use of OpenSim for our work, and in particular in the arena or numerical simulations and visualization, and pay a close attention to the issues of avatar and inventory interoperability and portability. A start along these lines is ScienceSim [23]. Having an immersive VR environment on one’s own machine can bypass many of the limitations of the commercial VW grids, such as SL, especially in the numbers of data points that can be rendered. It is likely that the convergence of the Web and immersive VR would be in the form whereby one runs and manages their own VR environment in a way which is analogous to hosting and managing one’s own website today. OpenSim and its successors, along with a suitable standardization for interoperability, may provide a practical way forward; see also [24]. 2.5 Information Architectures for the Next Generation Web One plausible vision of the future is that there will be a synthesis of the Web, with its all-encompassing informational content, and the immersive VR as an interface to it,

Exploring the Use of Virtual Worlds as a Scientific Research Platform

39

since it is so well suited to the human sensory input mechanisms. One can think of immersive VR as the next generation browser technology, which will be as qualitatively different from the current, flat desktop and web page paradigm, as the current browsers were different from the older, terminal screen and file directory paradigm for information display and access. A question then naturally arises: what will be the newly enabled ways of interacting with the informational content of the Web, and how should we structure and architect the information so that it is optimally displayed and searched under the new paradigm? To this effect, what we plan to do is to investigate the ways in which large scientific databases and connections between them (e.g., in federated data grid frameworks, such as the Virtual Observatory [25, 26, 27]) can be optimally rendered in an immersive VR environment. This is of course a universal challenge, common to all sciences and indeed any informational holdings on the Web, beyond academia. Looking further ahead, many of the new scientific challenges and opportunities will be driven by the continuing exponential growth of data volumes, with the typical doubling times of ~ 1.5 years, driven by the Moore’s law which characterizes the technology which produces the data [35, 36]. An even greater set of challenges is presented by the growth of data complexity, especially as we are heading into the Petascale regime [37, 38, 39]. However, these issues are not limited to science: the growth of the Web constantly overwhelms the power of our search technologies, and brute-force approaches seldom work. Processing, storing, searching, and synthesizing data will require a scalable environment and approach, growing from the current “Cloud+Client” paradigm. Only by merging data and compute systems into a truly global or Web-scale environment – virtualizing the virtual – will sufficient computational and data storage capacity be available. A strong feature of such an environment will be high volume, frequent, low latency services built on message-oriented architectures as opposed to today’s serviceoriented architectures. There will be a heterogeneity of structured, semi-structured and unstructured data that will need to be persisted in an easily searchable manner. Atop of that, we will likely see a strong growth in semantic web technologies. This changing landscape of data growth and intelligent data discovery poses a slew of new challenges: we will need some qualitatively new and different ways of visualizing data spaces, data structures, and search results (here by “data” we mean any kind of informational objects – numerical, textual, images, video, etc.). Immersive VR may become a critical technology to confront these issues. Scientists will have to be increasingly immersed into their data and simulations, as well as the broader informational environment, i.e., the next generation Web, whatever its technological implementations are, simply for the sake of efficiency. However, the exponential growth of data volumes, diversity, and complexity already overwhelms the processing capacity of a single human mind, and it is inevitable that we will need some capable AI tools to aid us in exploring and understanding the data and the output of numerical models and simulations. Much of the data discovery and data analysis may be managed by intelligent agents residing in the computing/data environment, that have been programmed with our beliefs, desires and intents. They will serve both as proxies for us reacting to results and new data according to programmed criteria expressed in declarative logic languages and also as our interface point into the computing/data environment for

40

S.G. Djorgovski et al.

activities such as data visualization. Interacting with an agent will be a fully immersive experience combining elements of social networking with advances in virtual world software. Thus, we see a possible diversification of the concept of avatars – as they blend with intelligent software agents, possibly leading to new modalities of human and AI representation in virtual environments. Humans create technology, and technology changes us and our culture in unexpected ways; immersive VR represents an excellent example of an enabling cognitive technology [28, 29]. 2.6 Education and Public Outreach VWs are becoming another empowering, world-flattening educational technology, very much like as the Web has already done. Anyone from anywhere could attend a lecture in SL, whether they are a student or simply a science enthusiast. What VWs provide, extending the Web, is the human presence and interaction, which is an essential component of an effective learning process. That is what makes VWs such a powerful platform for any and all educational activities which involve direct human interactions (e.g., lectures, discussions, tutoring, etc.). In that, they complement and surpass the traditional Web, which is essentially a medium to convey pre-recorded lectures, as text, video, slides, etc. Beyond the direct mappings of traditional lecture formats, VWs can really enable novel collaborative learning and educational interactions. Since buildings, scenery, and props are cheap and easy to create, VWs are a great environment for situational training, exploration of scenarios, and such. Medical students can dissect virtual cadavers, and architects can play with innovative building designs, just moving the bits, without disturbing any atoms. Likewise, physicists can construct virtual replicas of an experimental apparatus, which students can examine, assemble, or take apart. There is already a vibrant, active community of educators in SL [30, 31], and many excellent outreach efforts are concentrated in the SL SciLands virtual continent [32]. MICA’s own efforts include a well-attended series of popular talks, “Dr.Knop talks astronomy” [33], which includes guest lecturers, as well as informal weekly “Ask an Astronomer” gatherings. We will continue with these efforts, and expand the range of our popular lectures. Under the auspices of MICA, we are starting to experiment with regularly scheduled classes and/or class discussions in SL, and we will explore such activities in other VW environments as well. These may include an introductory astronomy class, or an advanced topic seminar aimed at graduate students. We will also try a hybrid format, where the students would read the lecture materials on their own, and use the class time for an open discussion and explanations of difficult concepts in a VW setting. We also plan to conduct a series of international “summer schools” on the topics of numerical stellar dynamics, computational science, and possibly others, in an immersive and interactive VW venue.

3 Concluding Comments In MICA, we have started to build a new type of a scientific institution, dedicated to an exploration of immersive VR and VWs technologies for science, scholarship, and

Exploring the Use of Virtual Worlds as a Scientific Research Platform

41

education, aimed primarily at academics in physical and other natural sciences. MICA itself is an experiment in the new ways of conducting scholarly work, as well as a testbed for new ideas and research modalities. It is also intended to be a gateway for other scholars, new to VWs, to start to explore the potential and the practical uses of these technologies in an easy, welcoming, and collegial environment. MICA represents a multi-faceted effort aimed to develop new modalities of scientific research and communication using new technologies of immersive VR and VWs. We believe that they will enable and open qualitatively new ways in which scientists interact among themselves, with their data, and with their numerical simulations, and thus foster some genuine new “computational thinking” [34] approaches to science and scholarship. We use the VWs as a platform to conduct rigorous research activities in the fields of computational astrophysics and data-intensive astronomy, seeking to determine the potential of these new technologies, as well as to develop a new set of best practices for scholarly and research activities enabled by them, and by a combination of the existing Web-based and the new VR technologies. In that process, we may facilitate new astrophysical discoveries. We also hope to generate new ideas and methods which will in turn stimulate development of new technological capabilities in immersive VR and VWs, both as research and communication tools, and in the true sense of human-centered computational engineering. The central idea here is that immersive VR and VWs are potentially transformative technologies on par with the Web itself, which can and should be used for serious purposes, including science and scholarship; they are not just a form of games. By conveying this idea to professional scientists and scholars, and by leading by example, we hope to engage a much broader segment of the academic community in utilizing, and developing further these technologies. This evolutionary process may have an impact well beyond the academia, as these technologies blend with the cyber-world of the Web, and change the ways we interact with each other and with the informational content of the next generation Web. While at a minimum we expect to develop a set of “best practices” for the use of VR and VWs technologies in science and scholarship, it is also possible that practical and commercial applications may result or may be inspired by this work. If indeed immersive VR becomes a major new component of the modern society, as a platform for commerce, entertainment, etc., the potential impact may be very significant. In our work, we are assisted by a large number of volunteers, including scientists, technologists, and educators, most of them professional members of MICA. Some of them are actively engaged in the VWs development activities under the auspices of various governmental agencies, e.g., NASA. We have also established a strong network of international partnerships, including colleagues and institutions in the Netherlands, Italy, Japan, China, and Canada (a list which is bound to grow). We are also establishing collaborative partnerships with several groups in the IT industry, most notably Microsoft Research, and IBM, and we expect that this set of collaborations will also grow in time. This broad spectrum of professionally engaged parties showcases the growing interest in the area of scientific and scholarly uses of VWs, and their further developments for such purposes.

42

S.G. Djorgovski et al.

Acknowledgments. The work of MICA has been supported in part by the U.S. National Science Foundation grants AST-0407448 and HCC-0917817, and by the Ajax Foundation. We also acknowledge numerous volunteers who have contributed their time and talents to this organization, especially S. McPhee, S. Smith, K. Prowl, C. Woodland, D. Enfield, S. Cianciulli, T. McConaghy, W. Scotti, J. Ames, and C. White, among many others. We also thank the conference organizers for their interest and support. SGD also acknowledges the creative atmosphere of the Aspen Center for Physics, where this paper was completed.

References 1. Bainbridge, W.S.: The Scientific Research Potential of Virtual Worlds. Science 317, 472– 476 (2007) 2. Journal of Virtual Worlds Research, http://jvwresearch.org/ 3. TerraNova blog, various authors, http://terranova.blogs.com/ 4. Boellstorff, T.: Coming of Age in Second Life: An Anthropologist Explores the Virtually Human. Princeton University Press, Princeton (2008) 5. Convergence of the Real and the Virtual, the first scientific conference held inside World of Warcraft, May 9-11 (2008), http://mysite.verizon.net/wsbainbridge/convergence.htm 6. Hut, P.: Virtual Laboratories. Prog. Theor. Phys. Suppl. 164, 38–53 (2006) 7. Hut, P.: Virtual Laboratories and Virtual Worlds. In: Vesperini, E., et al. (eds.) Proc. IAU Symp. 246, Dynamical Evolution of Dense Stellar Systems, pp. 447–456. Cambridge University Press, Cambridge (2008) 8. The Meta-Institute for Computational Astrophysics (MICA), http://www.micavw.org/ 9. Second Life, http://secondlife.com/ 10. Qwaq Forums, http://www.qwaq.com/ 11. MICA SL island, StellaNova, http://slurl.com/secondlife/StellaNova/126/125/28 12. Hut, P., McMillan, S. (eds.): The Use of Supercomputers in Stellar Dynamics. Springer, New York (1986) 13. Hut, P., Makino, J., McMillan, S.: Modelling the Evolution of Globular Star Clusters. Nature 363, 31–35 (1988) 14. The Art of Computational Science, http://www.ArtCompSci.org 15. The Starlab Project, http://www.ids.ias.edu/~starlab 16. MUSE: a Multiscale Multiphysics Scientific Environment, http://muse.li 17. Hut, P., Mineshige, S., Heggie, D., Makino, J.: Modeling Dense Stellar Systems. Prog. Theor. Phys. 118, 187–209 (2007) 18. OpenSim project, http://opensimulator.org/ 19. SL Data Visualization wiki, http://sldataviz.pbwiki.com/ 20. Bourke, P.: Evaluating Second Life as a Tool for Collaborative Scientific Visualization. In: Computer Games and Allied Technology 2008 conf. (2008), http://local.wasp.uwa.edu.au/~pbourke/papers/cgat08/ 21. Johnson, A., Ames, J., Farr, W.: The MICASim plugin (2008), http://code.google.com/p/micasim/ 22. Farr, W., Hut, P., Johnson, A., Ames, J.: An Experiment in Using Virtual Worlds for Scientific Visualization (2009) (paper in prep.)

Exploring the Use of Virtual Worlds as a Scientific Research Platform 23. 24. 25. 26. 27. 28.

29. 30. 31. 32. 33. 34. 35. 36. 37.

38.

39. 40.

43

ScienceSim project wiki, http://sciencesim.com/ Virtual World Interoperability wiki, http://vwinterop.wikidot.com/ The U.S. National Virtual Observatory, http://www.us-vo.org/ The International Virtual Observatory Alliance, http://www.ivoa.net/ Djorgovski, S.G., Williams, R.: Virtual Observatory: From Concept to Implementation. ASP Conf. Ser. 345, 517–530 (2005) Roco, M., Bainbridge, W. (eds.): Converging Technologies for Improving Human Performance: Nanotechnology, Biotechnology, Information Technology, and Cognitive Science, NSF report. National Science Foundation, Arlington (2002) Bainbridge, W., Roco, M. (eds.): Managing Nano-Bio-Info-Cogno Innovations: Converging Technologies in Society, NSF report. National Science Foundation, Arlington (2005) Second Life Education Wiki, http://simteach.com/ The Immersive Education Initiative, http://immersiveeducation.org/ SciLands Virtual Continent, http://www.scilands.org/ MICA popular lectures series, http://mica-vw.org/wiki/index.php/Popular_Talks Wing, J.: Computational Thinking. Comm. ACM 49, 33–35 (2006) Szalay, A., Gray, J.: The World-Wide Telescope. Science 293, 2037–2040 (2001) Szalay, A., Gray, J.: Science in an Exponential World. Nature 440, 15–16 (2006) Emmott, S. (ed.): Towards 2020 Science. Microsoft Research Publ. (2006), http://research.microsoft.com/ en-us/um/cambridge/projects/towards2020science/ Djorgovski, S.G.: Virtual Astronomy, Information Technology, and the New Scientific Methodology. In: Di Gesu, V., Tegolo, D. (eds.) Proc. CAMP 2005: Computer Architectures for Machine Perception, IEEE Conf. Proc., pp. 125–132 (2005) Bell, G., Hey, T., Szalay, A.: Beyond the Data Deluge. Science 232, 1297–1298 (2009) Farr, W., Hut, P., Ames, J.: An Experiment in Using Virtual Worlds for Scientific Visualization of Self-Gravitating Systems. JVWR (in press, 2009)

Characterizing Mobility and Contact Networks in Virtual Worlds Felipe Machado, Matheus Santos, Virg´ılio Almeida, and Dorgival Guedes Department of Computer Science Federal University of Minas Gerais Belo Horizonte, MG, Brasil {felipemm,matheus,virgilio,dorgival}@dcc.ufmg.br

Abstract. Virtual worlds have recently gained wide recognition as an important field of study in Computer Science. In this work we present an analysis of the mobility and interactions among characters in World of Warcraft (WoW) and Second Life based on the contact opportunities extracted from actual user data in each of those domains. We analyze character contacts in terms of their spatial and temporal characteristics, as well as the social network derived from such contacts. Our results show that the contacts observed may be more influenced by the nature of the interactions and goals of the users in each situation than by the intrinsic structure of such worlds. In particular, observations from a city in WoW are closer to those of Second Life than to other areas in WoW itself. Keywords: Multi-player On-line Games, Virtual Worlds, social networks, complex networks, characterization.

1

Introduction

Virtual worlds are an important emerging form of social media that have recently caught the attention of the research community for their growth, their potential of applications and the new challenges they pose [1,2]. According to the companies responsible for those worlds, as of December 2008, World of Warcraft (WoW) is being played by more than 11.5 million subscribers worldwide and Second Life total residents are more than 16.5 million. Other data suggests that there are more than 16 million players of massively multi-player on-line games (MMOGs), where players control one or more characters in virtual worlds. Not only that, but users spend a significant amount of time on-line: in Q3/2008, residents spent 102.8 million hours in Second Life. Each virtual world fosters the creation of an active market both inside them and in other sites in the Internet, moving billions of dollars in the entertainment industry [3]. The environments provided by such virtual worlds are usually complex, providing a variety of opportunities for players to interact, fight and develop their characters. The virtual worlds are often divided in zones that may represent continents, islands, cities and buildings, where characters must move. Players may be forced to cooperate with others in order to achieve certain goals, and have to F. Lehmann-Grube and J. Sablatnig (Eds.): FaVE 2009, LNICST 33, pp. 44–59, 2010. c Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering 2010 

Characterizing Mobility and Contact Networks in Virtual Worlds

45

fight elements of other groups according to the rules of each environment. Even Second Life can be analysed in such a manner, although in that world there are no explicit competitive situations other than those arising in usual social interactions. All the possibilities offered by those environments create a highly complex virtual reality where a variety of characters seek different goals. Although some aspects of the virtual worlds may be quite detached from reality (like the multitude of different forms of intelligent life and the presence of magic forces), other aspects can be quite similar to the real world. After all, characters are controlled by real people, and interactions are often based on rules also existing outside the virtual environments. Information extracted from such virtual worlds may be directly useful to understand the way users behave in them, but can also be applied to other problems. For example, information about user mobility may be used in studies of how viruses spread among people, how information disseminates through their contacts, or how malware may spread among wireless devices carried by them [4]. Our goal in this work is to provide a first analysis of those worlds in terms of the way players move through the game and how they interact. That is achieved through a spatio-temporal analysis of mobility patterns in both worlds. From those patterns, we derive the social networks based on the users’ contact patterns and study them considering the similarities and differences of the two environments. While in Second Life interactions are mostly cooperative, in WoW they also have a competitive nature, leading to mixed behaviors. That difference is visible in some of the results. As previously mentioned, the information we provide here can be useful for those interested in the development and analysis of virtual worlds, as well as an input for experiments that depend on movement and contact data for real people, such as in epidemiological studies or research on mobile networks, for example. In the Sections that follow, we start by discussing related work in Section 2. Section 3 provide a general description of the virtual worlds considered, while Section 4 discusses our approach to monitoring them and deriving the metrics we used. The subsequent Sections that follow present the results of our analysis in terms of mobility patterns and contact social networks. Finally, Section 7 provides some conclusions and discusses future work.

2

Related Work

Virtual worlds have recently become the focus of researchers looking for data that could be used to model real world mobility patterns. The Second Life virtual environment has been monitored to collect information about avatar movements to mirror movement in enclosed spaces [5]. Metrics used included time to first contact, contact time, inter-contact time, and covered distance, among others. They also analyzed the users’ contact network using complex networks metrics such as node degree, network diameter and clustering coefficients. We use similar metrics in this work.

46

F. Machado et al.

Characterization of on-line games has been an interest for some time now, but a lot of effort has been focused on studying the network traffic produced by them, not in understanding the mechanics of their virtual worlds [6,7,8]. In relation to the particular worlds considered in this study, there has been previous work characterizing Second Life and World of Warcraft from the point of view of the users, by collecting traffic in the client applications [9,10], but again with little insight into the virtual worlds themselves. With that in mind, this work is, to the best of our knowledge, the first one to consider two different virtual worlds with different interaction patterns and objectives. It is also the first one to consider the behavior of avatars in World of Warcraft from a social network perspective derived from their contacts.

3

Virtual Worlds: Background

Both environments considered can be seen as examples of massive multi-player on-line games (MMOGs) based on the Role-Playing Game model (RPG). In such games, players perform their roles through their characters in the game, which interact based on behavioral rules defined by the game environment. For the sake of completeness, this Section provides a brief description of both worlds. 3.1

Second Life

In Second Life, each user controls a virtual character (avatar) that can own objects, real estate, stores, etc. Usually there is no concept of game levels, since the game is entirely focused on social interactions. Hierarchies and class divisions are left for the players. Basically, an avatar sets itself apart from others based on its looks and its possessions. Differently from a traditional RPG, there are no clearly stated goals to Second Life, no missions or tasks defined by the game for the users to complete. The idea is just to allow users to interact socially, talking, performing collective activities, or trading, for example. Users can create virtual groups, which are just used to bring together users with common interests, like the appreciation for a certain location, the desire to meet other people or just as a means to make it simpler to keep contact over time. Avatars can become friends with others, leading to an underlying social network, although the environment does not offer tools to build such networks explicitly. The game territory is quite large, being composed by different continents and many islands. All of it is divided in smaller regions called lands, usually in the form of 256 meter-sided squares. Each land has a defined maximum occupancy and is kept associated with a specific server in order to make load distribution simpler. Management of user actions are therefore distributed among the servers. 3.2

World of Warcraft

World of Warcraft (WoW) adheres strongly to the concept of RPG. It takes place in a virtual world divided in large continents, each one with its special

Characterizing Mobility and Contact Networks in Virtual Worlds

47

characteristics and sub-divisions. In the game, each user can have multiple characters, but can control only one at a time. The goal of the game is, just like in most RPGs, to evolve the characters based on a hierarchy defined by the game and to defeat the enemy, which can be another player or a programmed entity running on the game servers. For that end there are different resources and possibilities, like items that characters can obtain during the game, their professions and special abilities they can develop. To help characters in their quests and facilitate interaction and trade among users, various cities exist in the territories offering supplies, shelter and training for characters. In WoW, each character belongs to one faction, race and class. They must belong to one of the two existing enemy factions, the Horde and the Alliance, bound to fight each other. For that reason, a meeting of characters of different factions cannot be collaborative, but instead must be surrounded by a clear form of dispute. Cities can belong to one of the factions or declare themselves neutral grounds, the only place where members of different factions can meet without open confrontation. The auction houses in such cities can mediate trade between the factions. Continents are divided in zones with different shapes larger but similar to Second Life’s lands in the way they restrict movements between them to a few points of transit. In that way, each zone can be controlled independently of the others. Eastern Kingdoms and Kalimdor are the older continents in the game, while Outlands is a newer continent added during an expansion named the Burning Crusade. There is also the concept of instances, regions of the map that are duplicated to restrict the occupancy to certain groups each time. If various groups go to a certain region to complete a mission, game servers instantiate one copy of that region for each group, in case the goal is to allow each group to work on the mission without affecting the others’ progress. That leads, in practice, to areas with externally controlled populations.

4

Methodology

In order to understand the behavior of characters in WoW and Second Life, we collected data from WoW at different levels, so we could analyse behavior in terms of the large continents, controlled regions (instances) and a city, which we expected to be a region with characteristics closer to those of an island in Second Life. Table 1 shows some general information about the data collected for each of the virtual regions we considered. The headers used for each of the first five columns refer to elements from WoW: main continents (Eastern Kingdoms, Kalimdor, and Outlands), an instance of a region (Instance 18 ), and a city (Stormwind ). The last column refers to Second Life. Rows show, for the duration of the logs, the total number of distinct characters seen in each region, the average and maximum number of concurrent users actually on-line, and the average session length in hours. The two worlds considered differ significantly in their operations, what led to the use of different data harvesting techniques. The details of each process are discussed next.

48

F. Machado et al. Table 1. General information about the collected data

E. K. Characters 1276 Avg. concurrent users 109 Max. concurrent users 340 Avg. session length (h) 1.4

WoW S.L. Kal. Inst18 Outl. SW. 1039 750 611 511 511 105 88 56 109 31 299 225 123 340 49 1.6 1.6 1.4 1.2 0.05

To collect data from Second Life we implemented a client for the game using the libsecondlife library1 . This automated client connects to the server as a player, interacting with the world following a pattern defined by the programmer. For this work, the client moved in large circles around the center or the territory, since it was found that a moving avatar draws less attention. Once the resulting avatar reaches one of the lands it begins receiving information about the general conditions of the land and all other characters in that region (their IDs, their position relative to the land and whether they are online or offline. The client stores that information once every five seconds in a record containing the number of online users in that land at the time, followed by a list with character ID and position for each avatar. The logs used in this work were selected to hold a continuous 24 hour period. The region used was the Dance Island2 , a popular location in Second Life which contains a dance floor and a bar, among other things. Besides the official World of Warcraft (WoW) game servers, there are currently other versions of those servers, developed through reverse engineering, maintained by users around the globe. For this work we used a message log obtained from one of those user-maintained servers for version 3.5 of the game. The log was created by instrumenting the private Mangos server to log every network message received or sent by it over a 24 hour period. That resulted in a 33 GB data log with more than one hundred million messages, being 15 million sent from clients to the server, and approximately 96 million sent by the server. If the server showed any interruption in its execution the period of the fault was removed from the logs and users returned to their activities where they had left them at the moment of the problem, avoiding any impact to the players movements. Coordinates in the WoW messages are relative to the main continents and instances the characters are in, so there is no global coordinate system that can be equally applied to all characters. To take that into account, all the following analysis considered each continent separately. As previously mentioned, we considered the continents Eastern Kingdoms, Kalimdor, and Outland. We also analysed separately one of the major cities in the game, Stormwind, to compare with the results from Second Life, since a city in WoW offered an area more similar to a land than a complete continent. Finally, we also added an instance of a replicated region of the game, identified as Instance 18, where the number of players was controlled by the game server. 1 2

http://www.libsecondlife.org/ http://slurl.com/secondlife/Dance%20Island/

Characterizing Mobility and Contact Networks in Virtual Worlds

49

An anomaly identified in the game, when compared to the real world, was the presence of different forms of teletransportation 3 . In some of the analysis, we experimented with removing that functionality from character behavior to try to get patterns closer to the real world, since teletransportation would allow them to travel unlimited distances in practically no time, something clearly impossible in the real world. To achieve that, each time a character used teletransportation, disappearing from one location and materializing at another one, we considered that the first character left the game at the earlier position and a new one entered the game at the materialization spot. We also analyzed the movements as they happened originally, with teletransportation. Once data was collected from WoW, we extracted from the log all messages carrying character positions with the ID of the character, its position and the message timestamp. That information was then processed to create a final log with the same format of that created for Second Life, with all active characters’ positions recorded every five seconds. After a single record format was available for both worlds they were processed using the same algorithms to derive information such as covered distances, demographic density and contact events. Contacts were considered to occur whenever two characters were closer than a certain distance r, considered 10 meters in this case. That definition allows us to consider not only direct character interaction but also close encounters, which have been identified in the literature as relevant for multiple purposes, such as epidemiological studies and wireless network interactions [11]. From the contact information we built the network of contacts, one of the main focus of this paper, and derived also a temporal analysis of contacts. For the temporal analysis, we computed time to first contact, the time it took characters to establish their first contact in the environment, contact time, the times characters spent in contact with others, and inter-contact time, the times between two successive contacts by each pair of characters. The results of the analysis of the metrics derived are discussed in the following Sections.

5

Spatio-temporal Analysis

5.1

Spatial Analysis

In this section we analyze and compare character movements in the two worlds, both in terms of distances traveled and demographic densities. Distances traveled. Figure 1 shows distances traveled (both as a probability density function, PDF, and a cumulative probability density function, CDF) for both worlds in log scale, with and without teletransportation in WoW. As expected, based on the dimensions of each area, probability of short travels is higher in Second Life, while distances in WoW with teletransportation may be significantly larger. 3

In Second Life avatars can also use teletransportation, but only between lands. Since we consider only one land, such events were seen as a user leaving the region.

50

F. Machado et al. 1

1

P[X x] (CCDF)

SL - Dance Stormwind instance 18 E Kingdoms Kalimdor Outland

1

0.01 0.001 0.0001

SL - Dance Stormwind instance 18 Kalimdor E Kingdoms Outland

1e-05 1e-06

1e-06 1e-07

51

1e-07 1

10

100

1000 Density

10000 100000

1

10

100 1000 10000 100000 Aggregate density

Fig. 2. Probability distribution, and complementary cumulative distribution of the aggregate demographic density

Figure 2 shows PDF and CCDF (complementary cumulative probability density function) of the aggregate density computed as the number of characters seen at each square. We can see that the PDF for Second Life stays constant for most of the densities, with some oscillation for lower concentrations. WoW, on the other hand, has a much more skewed distribution for all large areas, with a behavior close to a power law for most of the range considered. Stormwind, the city in Wow, being a restricted area, has a behavior closer to that of the Second Life land, although still closer to the general WoW pattern. From the CCDF, we can see that the three continents and the instance in WoW, being larger areas, spent most of the day with no visitors (about 1% of the area had at least one visitor during the period, except for Outland, in which case less than 0.5% of the area was visited. Even in Second Life, more than 50% of the area was not visited according to the log. Again, the curve for the city, Stormwind, is closer to that of Second Life. It might be the case that they would be even closer if their areas were more close to each other. 5.2

Temporal Analysis

To better understand the nature of the interactions in each world, we considered the temporal dynamic of the contacts. The metrics used, time to first contact, contact time and inter-contact time, were discussed in Section 4. Considering the strictly social nature of Second Live, time to first contact and inter-contact times should be shorter and contact time should be longer than for WoW. Second Life users enter the world mostly to socialize, so they seek other people as soon as they get on-line, reducing time to first contact. For the same reason, after they meet a character or a group, they tend to start a conversation in stead of just pass by and go somewhere else. That should be particularly true for Dance Island. As Table 2 and Figure 3 show, that is exactly the case, except in a few cases. Stormwind, being a city, again shares some of the characteristics of Second Life. Cities serve as temporary bases and support facilities, so people tend to

52

F. Machado et al.

Avg

Table 2. Contacts temporal metrics (averages in seconds)

E. K. First contact 2170 Contact time 89 Inter-contact time 384

WoW Kal. Inst. 18 Outland 1943 2695 520 170 316 128 405 435 112

S.L. S.W. 195 163 474 284 1222 387

seek populated places, like markets, banks and training sites once they reach them, leading to early contacts, so they have similar times to first contact. In the city, however, long sessions where players seek to improve their user experience (trading, grouping, training skills, seeking quests, chatting) seems to dominate contact times, making them even longer than for Second Life. Also, after characters part in Stormwind, they take much longer to meet again (if they ever do), as the average inter-contact time indicates. That was mostly due to the nature of the game: once characters part after training or conducting business they tend to leave the city for new quests, returning much later. Both features are also visible in Fig. 3, where we can see that approximately 50% of the intercontact times in Stormwind are longer than 100 seconds, against only 30% in Second Life, and also the longer contact times for Stormwind (roughly 5% are longer than 2.5 hours). Other elements of interest in Table 2 are the lower inter-contact time for Outland and high first-contact times and longer contact times in the Instance18. Those are also explained by the nature of the game. Outland is a continent visited by advanced characters in their quest to improve their rankings even further. In that condition, collaboration with other characters is important and they tend to meet often to exchange information, if for nothing else. That reduces inter-contact time. Instances are mostly places were collaborative game play is essential. Characters usually join outside an instance and enter them together. Once inside, they proceed together (getting closer or farther apart as the situation requires) but with no contacts with characters other than those in their group. We only registered the (eventual) moments when characters get more separated and then get closer again. On the other hand, contact times and inter-contact times capture the together-again-apart-again nature of the action. From Fig. 3 we see that Second Life has fewer short-lived contacts: characters tend to at least try to start a conversation each time they meet, so contacts tend to last at least a little longer (only 20% last less then 30 seconds). On the other hand, in WoW is more common for characters to just pass by others while en route to a farther destination, without ever stopping — although that is, again, a little less common for Outland and Stormwind, for the reasons discussed. In both, there are some short-lived contacts but also some long-lived ones. We can see basically five categories in terms of time to first contact in Fig. 3. Clearly Second Life is the one with lower values (almost 80% of the first contacts happen in less than 8 seconds, while the opposite is true for instance 18 (50% take longer than 4 minutes). Outland and Stormwind, since they have conditions

Characterizing Mobility and Contact Networks in Virtual Worlds 1

SL - Dance Outland Stormwind Kalimdor E Kingdoms instance 18 10

100 1000 10000 Time to first contact (sec)

E Kingdoms Kalimdor instance 18 Outland Stormwind SL - Dance

P[X