Ubiquitous Intelligence and Computing: 5th International Conference, UIC 2008, Oslo, Norway, June 23-25, 2008 Proceedings (Lecture Notes in Computer Science, 5061) 3540692924, 9783540692928

Citation preview

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen University of Dortmund, Germany Madhu Sudan Massachusetts Institute of Technology, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany

5061

Frode Eika Sandnes Yan Zhang Chunming Rong Laurence T. Yang Jianhua Ma (Eds.)

Ubiquitous Intelligence and Computing 5th International Conference, UIC 2008 Oslo, Norway, June 23-25, 2008 Proceedings

13

Volume Editors Frode Eika Sandnes Oslo University College, Oslo, Norway E-mail: [email protected] Yan Zhang Simula Research Laboratory, Lysaker, Norway E-mail: [email protected] Chunming Rong University of Stavanger, Stavanger, Norway E-mail: [email protected] Laurence T. Yang St. Francis Xavier University, Antigonish, NS, Canada E-mail: [email protected] Jianhua Ma Hosei University, Tokyo 184-8584, Japan E-mail: [email protected]

Library of Congress Control Number: 2008929350 CR Subject Classification (1998): H.4, C.2, D.4.6, H.5, I.2, K.4 LNCS Sublibrary: SL 3 – Information Systems and Application, incl. Internet/Web and HCI ISSN ISBN-10 ISBN-13

0302-9743 3-540-69292-4 Springer Berlin Heidelberg New York 978-3-540-69292-8 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media springer.com © Springer-Verlag Berlin Heidelberg 2008 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12277919 06/3180 543210

Preface

This volume contains the proceedings of UIC 2008, the 5th International Conference on Ubiquitous Intelligence and Computing: Building Smart Worlds in Real and Cyber Spaces. The conference was held in Oslo, Norway, during June 23–25, 2008. The event was the fifth meeting of this conference series. USW 2005 (First International Workshop on Ubiquitous Smart World), held in March 2005 in Taiwan, was the first event in the series. This event was followed by UISW 2005 (Second International Symposium on Ubiquitous Intelligence and Smart Worlds) held in December 2005 in Japan, by UIC 2006 (Third International Conference on Ubiquitous Intelligence and Computing: Building Smart Worlds in Real and Cyber Spaces) held in September 2006 in Wuhan and Three Gorges, China, and by UIC 2007 held in July 2007 in Hong Kong. Ubiquitous computers, networks and information are paving the road to a smart world in which computational intelligence is distributed throughout the physical environment to provide trustworthy and relevant services to people. This ubiquitous intelligence will change the computing landscape because it will enable new breeds of applications and systems to be developed; the realm of computing possibilities will be significantly extended. By embedding digital intelligence in everyday objects, our workplaces, our homes and even ourselves, many tasks and processes could be simplified, made more efficient, safer and more enjoyable. Ubiquitous computing, or pervasive computing, composes these numerous “smart things/u-things” to create the environments that underpin the smart world. A smart thing can be endowed with different levels of intelligence, and may be context-aware, active, interactive, reactive, proactive, assistive, adaptive, automated, sentient, perceptual, cognitive, autonomic and/or thinking. The field of intelligent/smart things is an emerging research field that covers many disciplines. A series of grand challenges exist to move from the world of ubiquitous computing with universal services of any means/place/time to the smart world of trustworthy services with the right means/place/time. The UIC 2008 conference offered a forum for researchers to exchange ideas and experiences in developing intelligent/smart objects, environments, and systems. This year, the technical program of UIC drew from a very large number of submissions: 102 papers submitted from 26 countries representing four regions – Asia Pacific, Europe, North and South America. Each accepted paper was reviewed (as a full paper) by at least three reviewers, coordinated by the international Program Committee. In order to allocate as many papers as possible and keep the high quality of the conference, we finally decided to accept 27 regular papers for presentation, which reflected a 26% acceptance rate. In addition, there were 28 special session papers included into the proceedings. The accepted papers provide research contributions

VI

Preface

in a wide range of research topics that were grouped into 14 conference tracks: smart objects and embedded systems, smart spaces/environments/services, intelligent computing, wireless sensor networks, context-aware applications and systems, and wireless networks. In addition to the refereed papers, the proceedings include Jadwiga Indulska’s keynote address on “Challenges in the Design and Development of Context-Aware Applications,” Petter Øyan’s keynote address on “The Importance of Including the Haptics Factor in Interaction Design,” and an invited paper from Stephen S. Yau on “Security Policy Integration and Conflict Reconciliation for Collaborations Among Organizations in Ubiquitous Computing Environment.” We believe that the conference not only presented novel and interesting ideas but also stimulated future research in the area of ubiquitous intelligence and computing. Organization of conferences with a large number of submissions requires a lot of hard work and dedication from many people. We would like to take this opportunity to thank the numerous people whose work made this conference possible and ensured its high quality. We wish to thank the authors of submitted papers, as they contributed to the conference technical program. We wish to express our deepest gratitude to the Program (Vice) Chairs for their hard work and commitment to quality when helping with paper selection. We would also like to thank all Program Committee members and external reviewers for their excellent job in the paper review process, the Steering Committee and Advisory Committee for their continuous advice, and Jadwiga Indulska and Daqing Zhang for organizing a panel on the important question: “What do we expect from pervasive/intelligent computing and how far are we from achieving it?” A special thanks to Yo-Ping Huang for organizing a special session on “Object Identification: Techniques and Applications.” We are also in debt to the Publicity Chairs for advertising the conference, to the Local Organizing Committee for managing the registration and other conference organization-related tasks, and to Oslo University College for hosting the conference. We are also grateful to Kyrre Begnum for the hard work on managing the conference website and the conference management system.

June 2008

Frode Eika Sandnes, Yan Zhang, Chunming Rong, Laurence Tianruo Yang and Jianhua Ma UIC 2008 Editors

Organization

Executive Committee General Chairs

Program Chairs

Program Vice Chairs

Honorary Chairs Steering Committee

International Advisory Committee

Frode Eika Sandnes, Oslo University College, Norway Mark Burgess, Oslo University College, Norway Chunming Rong, University of Stavanger, Norway Yan Zhang, Simula Research Laboratory, Norway D. Manivannan, University of Kentucky, USA Michael Beigl, University of Hannover, Germany Yo-Ping Huang, National Taipei University of Technology, Taiwan Oliver Sinnen, University of Auckland, New Zealand Graham Megson, University of Reading, UK Stephen S. Yau, Arizona State University, USA Norio Shiratori, Tohoku University, Japan Jianhua Ma (Chair), Hosei University, Japan Laurence T. Yang (Chair), St. Francis Xavier University, Canada Hai Jin, Huazhong University of Science and Technology, China Jeffrey J.P. Tsai, University of Illinois at Chicago, USA Theo Ungerer, University of Augsburg, Germany Leonard Barolli, Fukuoka Institute of Technology, Japan Victor Callaghan, University of Essex, UK Yookun Cho, Seoul National University, Korea Sumi Helal, University of Florida, USA Frank Hsu, Fordham University, USA Ali R. Hurson, Pennsylvania State University, USA Qun Jin, Waseda University, Japan Beniamino Di Martino, Second University of Naples, Italy Christian Muller-Schloer, University of Hannover, Germany Timothy K. Shih, Tamkang University, Taiwan Ivan Stojmenovic, Ottawa University, Canada Makoto Takizawa, Tokyo Denki University, Japan David Taniar, Monash University, Australia Jhing-Fa Wang, National Cheng Kung University, Taiwan

VIII

Organization

Executive Committee(continued) International Advisory Committee

Publicity Chairs

International Liaison Chairs

Award Chairs

Panel Chairs

Special Session Organizer Publication Chairs Web Chair Local Organizing Chair

Guangyou Xu, Tsinghua University, China Yaoxue Zhang, Tsinghua University, China Albert Zomaya, University of Sydney, Australia Yueh-Min Huang, National Cheng Kung University, Taiwan Jinhua Guo, University of Michigan - Dearborn, USA Mieso Denko, University of Guelph, Canada Hakan Duman, British Telecom, UK Ohbyung Kwon, Kyunghee University, Korea Wen Jing Hsu, National University of Singapore, Singapore Benxiong Huang, Huazhong University of Science and Technology, China Jong Hyuk Park, Kyungnam University, Korea Leonel Sousa, INESC-ID, Portugal Si Qing Zheng, University of Texas at Dallas, USA David Nelson, University of Sunderland, UK Jiannong Cao, Hong Kong Polytechnic University, Hong Kong Demissie Aredo, Oslo University College, Norway Jadwiga Indulska, University of Queensland, Australia Daqing Zhang, National Institute of Telecommunications, France Yo-Ping Huang, National Taipei University of Technology, Taiwan Tony Li Xu, St. Francis Xavier University, Canada Liu Yang, St. Francis Xavier University, Canada Ismail Hassan, Oslo University College, Norway Siri Fagernes, Oslo University College, Norway Simen Hagen, Oslo University College, Norway Kirsten Ribu, Oslo University College, Norway Kyrre Begnum, Oslo University College, Norway Jie Xiang, Simula Research Laboratory, Norway Qin Xin, Simula Research Laboratory, Norway Hai Ngoc Pham, University of Oslo, Norway

Program Committee Waleed Abdulla Sheikh Iqbal Ahamed

The University of Auckland, New Zealand Marquette University, USA

Organization

IX

Program Committee(continued) Alexandra Branzan Albu Najah Abu Ali Noureddine Boudriga Phillip G. Bradford Tsun-Wei Chang Jiming Chen Ruey-Maw Chen Shu-Chen Cheng Wei Chen Wen Chen Peter H.J. Chong Hung-Chi Chu Alva Couch Alma Leora Culn Waltenegus Dargie Reza Dilmaghani Panayotis E. Fouliras Alois Ferscha Edgar Gabriel Antonio Mana Gomez Zonghua Gu Khoon Guan Winston Seah Arobinda Gupta Laurence Habib Hrek Haugerud Dag Haugland Jianhua He Shang-Lin Hsieh Bin Hu Lin-Huang Chang Yu Hua Tao Jiang Wenbin Jiang Tore Møller Jonassen Audun Josang James B.D. Joshi Akimitsu Kanzaki Helen Karatza Andreas J. Kassler

University of Victoria, Canada United Arab Emirates University, UAE University of Carthage, Tunisia University of Alabama, USA De Ling Institute of Technology, Taiwan Zhejiang University, China National Chin-Yi University of Technology, Taiwan Southern Taiwan University, Taiwan Nanjing University of Posts and Telecommunications, China Shanghai Jiaotong University, China Nanyang Technological University, Singapore Chaoyang University of Technology, Taiwan Tufts University, USA University of Oslo, Norway Technical University of Dresden, Germany King’s College London, UK University of Macedonia, Greece Johannes Kepler Universit¨ at Linz, Austria University of Houston, USA University of Malaga, Spain Hong Kong University of Science and Technology, Hong Kong Institute for Infocomm Research, Singapore Indian Institute of Technology, India Oslo University College, Norway Oslo University College, Norway Bergen University, Norway Swansea University, UK Tatung University, Taiwan University of Central England, UK National Taichung University, Taiwan Huazhong University of Science and Technology, China University of Michigan, USA Huazhong University of Science and Technology, China Oslo University College, Norway Queensland University of Technology, Australia University of Pittsburgh, USA Osaka University, Japan Aristotle Univeristy of Thessaloniki, Greece Karlstad University, Sweden

X

Organization

Program Committee(continued) Paris Kitsos Gabriele Kotsis Lars Kristiansen Stefan Lankes Matti Latva-aho Cai Li Frank Li Jie Li Shiguo Lian Daw-Tung Lin Giuseppe Lo Re Arne Lkketangen Sathiamoorthy Manoharan Hassnaa Moustafa Josef Noll Lukowicz Paul

Hellenic Open University, Greece Johannes Kepler University of Linz, Austria University of Oslo, Norway RWTH Aachen University, Germany Oulu University, Finland Communications University of China, China Agder University, Norway University of Tsukuba, Japan France Telecom R&D Beijing, China National Taipei University, Taiwan University of Palermo, Italy Molde University College, Norway

University of Auckland, New Zealand France Telecom R&D , France UniK/Movation, Norway Lehrstuhl f¨ ur Informatik, Universit¨at Passau, Germany Marius Portmann University of Queensland, Australia Ravi Prakash University of Texas at Dallas, USA Antonio Puliafito University of Messina, Italy Ragnihild Kobro Runde University of Oslo, Norway Hamid Sharif University of Nebraska, USA Albrecht Schmidt University of Munich, Germany Rose Shumba Indiana University of Pennsylvania, USA Isabelle Simplot-Ryl University of Lille, France Lingyang Song University of Oslo, Norway Min Song Old Dominion University, USA Lambert Spaanenburg Lund University, Sweden Zhou Su Waseda University, Japan Evi Syukur Monash University, Australia Chik How Tan Gjvik University College, Norway Hallvard Trtteberg Norwegian University of Science and Technology, Norway Shiao-Li Tsao National Chiao Tung University, Taiwan Thanos Vasilakos University of Western Macedonia, Greece Javier Garcia Villalba Complutense University of Madrid, Spain Agustinus Borgy Waluyo Institute for Infocomm Research (I2R), Singapore Guojun Wang Central South University, China Tzone-I Wang National Cheng-Kung University, Taiwan Xinbing Wang Shanghai Jiaotong University, China Zhijun Wang Hong Kong Polytechnic University, Hong Kong Jiang (Linda)Xie University of North Carolina at Charlotte, USA Qin Xin Simula Research Laboratory, Norway

Organization

Program Committee(continued) Stephen Yang Wei Yen Muhammad Younas Rong Yu Hongwei Zhang Chi Zhou Yuefeng Zhou

National Central University, Taiwan Tatung University, Taiwan Oxford Brookes University, UK South China University of Technology, China Wayne State University, USA Illinois Institute of Technology, USA NEC Laboratories Europe, UK

XI

Table of Contents

Keynote Speech Challenges in the Design and Development of Context-Aware Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jadwiga Indulska

1

The Importance of Including the Haptics Factor in Interaction Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Petter Øyan and Anne Liseth Schøyen

2

Regular Papers Ubiquitous Computing Security Policy Integration and Conflict Reconciliation for Collaborations among Organizations in Ubiquitous Computing Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stephen S. Yau and Zhaoji Chen Improved Weighted Centroid Localization in Smart Ubiquitous Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stephan Schuhmann, Klaus Herrmann, Kurt Rothermel, Jan Blumenthal, and Dirk Timmermann A Formal Framework for Expressing Trust Negotiation in the Ubiquitous Computing Environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deqing Zou, Jong Hyuk Park, Laurence Tianruo Yang, Zhensong Liao, and Tai-hoon Kim Pervasive Services on the Move: Smart Service Diffusion on the OSGi Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Davy Preuveneers and Yolande Berbers

3

20

35

46

Smart Spaces/Environments/Services Robots in Smart Spaces - A Case Study of a u-Object Finder Prototype - . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tomomi Kawashima, Jianhua Ma, Bernady O. Apduhan, Runhe Huang, and Qun Jin Biometrics Driven Smart Environments: Abstract Framework and Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vivek Menon, Bharat Jayaraman, and Venu Govindaraju

61

75

XIV

Table of Contents

A Structured Methodology of Scenario Generation and System Analysis for Ubiquitous Smart Space Development . . . . . . . . . . . . . . . . . . . . . . . . . . . Ohbyung Kwon and Yonnim Lee Capturing Semantics for Information Security and Privacy Assurance . . . Mohammad M.R. Chowdhury, Javier Chamizo, Josef Noll, and Juan Miguel G´ omez

90

105

Context-Aware Services and Applications A Framework for Context-Aware Home-Health Monitoring . . . . . . . . . . . . Alessandra Esposito, Luciano Tarricone, Marco Zappatore, Luca Catarinucci, Riccardo Colella, and Angelo DiBari Semantic Learning Space: An Infrastructure for Context-Aware Ubiquitous Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhiwen Yu, Xingshe Zhou, and Yuichi Nakamura A Comprehensive Approach for Situation-Awareness Based on Sensing and Reasoning about Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Thomas Springer, Patrick Wustmann, Iris Braun, Waltenegus Dargie, and Michael Berger

119

131

143

Context-Adaptive User Interface in Ubiquitous Home Generated by Bayesian and Action Selection Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . Han-Saem Park, In-Jee Song, and Sung-Bae Cho

158

Use Semantic Decision Tables to Improve Meaning Evolution Support Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yan Tang and Robert Meersman

169

Combining User Profiles and Situation Contexts for Spontaneous Service Provision in Smart Assistive Environments . . . . . . . . . . . . . . . . . . . Weijun Qin, Daqing Zhang, Yuanchun Shi, and Kejun Du

187

Ubiquitous Phone System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shan-Yi Tsai, Chiung-Ying Wang, and Ren-Hung Hwang

201

Utilizing RFIDs for Location Aware Computing . . . . . . . . . . . . . . . . . . . . . Benjamin Becker, Manuel Huber, and Gudrun Klinker

216

Intelligent Computing: Middleware, Models and Services A Component-Based Ambient Agent Model for Assessment of Driving Behaviour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tibor Bosse, Mark Hoogendoorn, Michel C.A. Klein, and Jan Treur

229

Table of Contents

XV

A Cartesian Robot for RFID Signal Distribution Model Verification . . . . Aliasgar Kutiyanawala and Vladimir Kulyukin

244

Self-Localization in a Low Cost Bluetooth Environment . . . . . . . . . . . . . . . Julio Oliveira Filho, Ana Bunoza, J¨ urgen Sommer, and Wolfgang Rosenstiel

258

Penetration Testing of OPC as Part of Process Control Systems . . . . . . . Maria B. Line, Martin Gilje Jaatun, Zi Bin Cheah, A.B.M. Omar Faruk, H˚ avard Husev˚ ag Garnes, and Petter Wedum

271

Intersection Location Service for Vehicular Ad Hoc Networks with Cars in Manhattan Style Movement Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yao-Jen Chang and Shang-Yao Wu

284

Ubiquitous and Robust Text-Independent Speaker Recognition for Home Automation Digital Life . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jhing-Fa Wang, Ta-Wen Kuan, Jia-chang Wang, and Gaung-Hui Gu

297

Energy Efficient In-Network Phase RFID Data Filtering Scheme . . . . . . . Dong-Sub Kim, Ali Kashif, Xue Ming, Jung-Hwan Kim, and Myong-Soon Park Energy-Efficient Tracking of Continuous Objects in Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jung-Hwan Kim, Kee-Bum Kim, Chauhdary Sajjad Hussain, Min-Woo Cui, and Myong-Soon Park

311

323

Wireless Sensor Networks Data Randomization for Lightweight Secure Data Aggregation in Sensor Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Abedelaziz Mohaisen, Ik Rae Jeong, Dowon Hong, Nam-Su Jho, and DaeHun Nyang Mobile Sink Routing Protocol with Registering in Cluster-Based Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ying-Hong Wang, Kuo-Feng Huang, Ping-Fang Fu, and Jun-Xuan Wang

338

352

Towards the Implementation of Reliable Data Transmission for 802.15.4-Based Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . Taeshik Shon and Hyohyun Choi

363

An Energy-Efficient Query Processing Algorithm for Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jun-Zhao Sun

373

XVI

Table of Contents

Special Session Papers Smart Objects and Embedded Computing Rule Selection for Collaborative Ubiquitous Smart Device Development: Rough Set Based Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kyoung-Yun Kim, Keunho Choi and Ohbyung Kwon

386

An Object-Oriented Framework for Common Abstraction and the Comet-Based Interaction of Physical u-Objects and Digital Services . . . . Kei Nakanishi, Jianhua Ma, Bernady O. Apduhan, and Runhe Huang

397

Personalizing Threshold Values on Behavior Detection with Collaborative Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hiroyuki Yamahara, Fumiko Harada, Hideyuki Takada, and Hiromitsu Shimakawa IP Traceback Using Digital Watermark and Honeypot . . . . . . . . . . . . . . . . Zaiyao Yi, Liuqing Pan, Xinmei Wang, Chen Huang, and Benxiong Huang

411

426

Wireless Networks: Routing, Mobility and Security Multi-priority Multi-path Selection for Video Streaming in Wireless Multimedia Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lin Zhang, Manfred Hauswirth, Lei Shu, Zhangbing Zhou, Vinny Reynolds, and Guangjie Han

439

Energy Constrained Multipath Routing in Wireless Sensor Networks . . . Antoine B. Bagula and Kuzamunu G. Mazandu

453

Controlling Uncertainty in Personal Positioning at Minimal Measurement Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hui Fang, Wen-Jing Hsu, and Larry Rudolph

468

RFID System Security Using Identity-Based Cryptography . . . . . . . . . . . . Yan Liang and Chunming Rong

482

Ubiquitous Computing RFID: An Ideal Technology for Ubiquitous Computing? . . . . . . . . . . . . . . . Ciaran O’Driscoll, Daniel MacCormac, Mark Deegan, Fred Mtenzi, and Brendan O’Shea An Experimental Analysis of Undo in Ubiquitous Computing Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marco Loregian and Marco P. Locatelli

490

505

Table of Contents

XVII

Towards a Collaborative Reputation Based Service Provider Selection in Ubiquitous Computing Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . Malamati Louta

520

Petri Net-Based Episode Detection and Story Generation from Ubiquitous Life Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Young-Seol Lee and Sung-Bae Cho

535

Smart Spaces/Environments/Services Protection Techniques of Secret Information in Non-tamper Proof Devices of Smart Home Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Abedelaziz Mohaisen, YoungJae Maeng, Joenil Kang, DaeHun Nyang, KyungHee Lee, Dowon Hong, and Jong-Wook Han

548

Universal Remote Control for the Smart World . . . . . . . . . . . . . . . . . . . . . . Jukka Riekki, Ivan Sanchez, and Mikko Pyykk¨ onen

563

Mobile Navigation System for the Elderly – Preliminary Experiment and Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takahiro Kawamura, Keisuke Umezu, and Akihiko Ohsuga

578

Time Stamp Protocol for Smart Environment Services . . . . . . . . . . . . . . . . Deok-Gyu Lee, Jong-Wook Han, Jong Hyuk Park, Sang Soo Yeo, and Young-Sik Jeong

591

Intelligent Computing: Middleware, Models and Services An Analysis of the Manufacturing Messaging Specification Protocol . . . . Jan Tore Sørensen and Martin Gilje Jaatun

602

A Long-Distance Time Domain Sound Localization . . . . . . . . . . . . . . . . . . . Jhing-Fa Wang, Jia-chang Wang, Bo-Wei Chen, and Zheng-Wei Sun

616

Towards Dataintegration from WITSML to ISO 15926 . . . . . . . . . . . . . . . . Kari Anne Haaland Thorsen and Chunming Rong

626

A SIP-Based Session Mobility Management Framework for Ubiquitous Multimedia Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chung-Ming Huang and Chang-Zhou Tsai

636

Context-Aware Services and Applications AwarePen - Classification Probability and Fuzziness in a Context Aware Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Martin Berchtold, Till Riedel, Michael Beigl, and Christian Decker

647

XVIII

Table of Contents

A Model Driven Development Method for Developing Context-Aware Pervasive Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Estefan´ıa Serral, Pedro Valderas, and Vicente Pelechano

662

Intelligent System Architecture for Context-Awareness in Ubiquitous Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jae-Woo Chang and Seung-Tae Hong

677

User-Based Constraint Strategy in Ontology Matching . . . . . . . . . . . . . . . . Feiyu Lin and Kurt Sandkuhl

687

Object Identification: Techniques and Applications RFID-Based Interactive Learning in Science Museums . . . . . . . . . . . . . . . . Yo-Ping Huang, Yueh-Tsun Chang, and Frode Eika Sandnes

697

Real-time Detection of Passing Objects Using Virtual Gate and Motion Vector Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Daw-Tung Lin and Li-Wei Liu

710

A Ubiquitous Interactive Museum Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yo-Ping Huang, Tsun-Wei Chang, and Frode Eika Sandnes

720

Dynamic Probabilistic Packet Marking with Partial Non-Preemption . . . Wei Yen and Jeng-Shian Sung

732

Fractal Model Based Face Recognition for Ubiquitous Environments . . . . Shuenn-Shyang Wang, Su-Wei Lin, and Cheng-Ming Cho

746

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

761

Challenges in the Design and Development of Context-Aware Applications Jadwiga Indulska School of Information Technology and Electrical Engineering, The University of Queensland and NICTA [email protected]

Abstract. Context-awareness plays an important role in pervasive computing as adaptations of applications to context changes (changes in computing environment and in user activities/tasks) help to achieve the goal of computing services available everywhere and at any time. There is a growing body of research on context-aware applications that are adaptable and capable of acting autonomously on behalf of users. However, there are still many open research issues that challenge the pervasive computing community. In this talk I discuss several of these research challenges. First, I outline the state of the art in context information modelling, management and reasoning as well as possible future research directions in this area. This is followed by a discussion of context information management that allows development of fault-tolerant and autonomic context-aware applications. As one of the challenges inhibiting the development of context-aware applications is their complexity, I discuss software engineering approaches that ease the task of developing such applications. Context-aware applications adapt to context changes using context information. However this context information may be imprecise or erroneous and therefore can lead to incorrect adaptation decisions creating usability problems and affecting acceptance of context-aware applications. This creates a need for some balance between autonomy of context-aware applications and the user control of the applications. I describe some early approaches my team is working on to tackle this problem. Finally, I discuss research issues related to privacy of context information and how context can be used to enhance security mechanisms within pervasive computing environments.



NICTA is funded by the Australian Government as represented by the Department of Broadband, Communications and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program; and the Queensland Government.

F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, p. 1, 2008. c Springer-Verlag Berlin Heidelberg 2008 

The Importance of Including the Haptics Factor in Interaction Design Petter Øyan and Anne Liseth Schøyen

Abstract. One aspect of interaction design is communication of information through the use of a screen. Another aspect, the physical or haptic interaction with the device itself, is another important issue, especially to reduce errors when the device is used in critical situations. Ostfold university college has cooperated with the Institute for Energy Technology in Halden and the Norwegian Defence Research Establishment in Kjeller, and several student groups have been given the opportunity to work on research data and in collaboration with researchers from these two institutions within their final year studies. The focus of the projects has been realistic; emphasizing the importance of minimizing operating errors to ensure safe operation in critical situations, where the ability to give correct feedback through haptic interaction is as important as the correct understanding of visual communication. The cases are demonstrating the user centered approach to problem solving used by industrial designers and the analogy between the design- and the resarch process, especially focusing on the use of physical designs to test and review and thereby exploring form as an interaction parameter.

F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, p. 2, 2008. © Springer-Verlag Berlin Heidelberg 2008

Security Policy Integration and Conflict Reconciliation for Collaborations among Organizations in Ubiquitous Computing Environments Stephen S. Yau and Zhaoji Chen Department of Computer Science and Engineering School of Computing and Informatics Arizona State University Tempe, AZ 85287-8809, USA {yau,zhaoji.chen}@asu.edu

Abstracts. The investigative capabilities and productivity of researchers and practitioners of various organizations can be greatly expanded with collaborations among these organizations in ubiquitous computing environments. However, in order to provide effective collaborations among organizations in ubiquitous computing environments, due to the heterogeneity and dynamic nature of the ubiquitous computing environments it is critical to generate an integrated security policy set to govern the interactions among collaborating organizations during the collaboration. Existing approaches in security policy integration and conflict analysis have focused on static policy analysis which cannot address the dynamic formation of collaborative groups in ubiquitous computing environments. In this paper, an approach is presented to security policy integration, including a similarity-based policy adaptation algorithm for changing collaborative groups and a negotiation-based policy generation protocol for the new resources generated by the collaboration as well as for conflict reconciliation. Our approach can be automated with user defined control variables, which can greatly improve the efficiency and make the system scalable with little human intervention. Keywords: Security policy integration, conflict reconciliation, organization collaboration, similarity-based policy adaptation, negotiation-based policy reconciliation, ontology, ubiquitous computing environments.

1 Introduction Due to rapid development of mobile devices and embedded systems, computing resources are embedded in various devices, appliances, such as cell phones, PDAs, televisions, TiVo, refrigerator and cars. All these devices can provide certain information processing capability. With the technologies, such as Bluetooth and Wi-Fi, these devices can easily communicate and share data and/or resources with each other. These advances in ubiquitous computing can greatly improve the investigative capabilities and productivity of researchers and practitioners in many fields, such as health care, e-business, e-government, telecommunication, and homeland security, F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 3–19, 2008. © Springer-Verlag Berlin Heidelberg 2008

4

S.S. Yau and Z. Chen

where the sharing of data, equipment and other resources among collaborating organizations may be extensive. In such an environment, computing resources are distributed throughout the physical environment, and are usually owned and managed by various organizations. An organization can be viewed as a set of nodes which has a set of users, certain resources and is capable of performing certain functions autonomously. To facilitate collaborations among participating organizations, we need to make their resources available across organizations seamlessly. By seamlessly, we mean if a user from one of the participating organizations enters the environment controlled by another organization, the user can access and use the appropriate resources through direct interactions with the resource owners. Otherwise, the user will need to send a request to his/her affiliated organization, and then the affiliated organization will forward the request to the organizations which own the resources. The resource owners would then make the decisions on whether to allow the user to access their resources, and then inform the affiliated organization, which will inform the user of their decisions. It is obvious that seamless resource sharing will greatly improve the effectiveness and efficiency of collaborations among various organizations. However, ubiquitous computing devices in such ubiquitous computing environments are more vulnerable to attacks than other distributed computing systems, especially when malicious users pretend to be legitimate users and try to sneak in the collaboration groups. With the growing concerns on security and privacy, a major obstacle for this kind of collaborations to be widely adopted, especially for critical applications, is how to govern the access to sensitive data and limited resources during the collaborations. Each organization may specify its own set of security policies to govern how its users to access its resources. But, in collaborations among organizations, we need to integrate all the relevant security policies from the participating organizations to govern the interactions throughout the collaborations. The heterogeneity and dynamic nature of ubiquitous computing environments poses certain unique challenges for generating an integrated congruous security policy set. In this paper, we will present an approach to integrating security policies of collaborating organizations in ubiquitous computing environments, including a similarity-based policy adaptation algorithm for security policy integration and a negotiation protocol for conflict reconciliation. An example will be given to illustrate our approach.

2 Challenges and Current State of the Art Collaborations among organizations in ubiquitous computing environments have the following characteristics. First, each organization is a group of nodes which manages its own resources and can function autonomously. Second, a group of collaborating organizations can be formed and changed at runtime. Each organization can choose to join or leave the group based on its needs. Third, organizations are peers to each other. Normally, there is no organization to serve as an authority to coordinate or manage the interactions among various collaborating organizations. Due to these characteristics, we have the following three major challenges for providing security in such environments.

Security Policy Integration and Conflict Reconciliation

5

1) Ambiguous security policy specifications: Each participating organization specifies its security policies describing how its resources should be used, including various limits and access control, and how these policies are enforced at runtime. Because each organization acts autonomously, the organizations can choose different vocabularies to specify their own security policies, which may cause confusion and difficulty when they try to understand each other’s policies. 2) Dynamic set of users: Individual organization’s security policies normally do not address the expanded user sets for collaborations. An organization’s security policies only address its own users and resources. Because there is no authority in the coalition to manage the requests from other organizations, and because the group of collaborating organizations may be changing, security policies specified by participating organizations need to be integrated together to govern the interactions during the collaboration, and the owner of a resource needs to know how to handle the requests from other organizations at runtime without prior knowledge of the collaborators. The collaboration may generate some resources that cannot be provided by any single participating organization. To govern future accesses to these new resources, the integrated security policy set should include policies for all these resources and the policies need to be acceptable by all participating organizations. 3) Conflicting security policies: Participating organizations may have inconsistent security policies among them. Because the group of collaborating organizations may continuously change at runtime, the organizations cannot anticipate all possible collaborative partners a priori and specify their security policies accordingly. The fact that each participating organization has little knowledge about what has or has not been specified by the other organizations may lead to redundancy and conflict when trying to integrating their security policies together. To avoid inconsistency in security policies integration and enforcement, redundant policies should be removed, and conflicts must be resolved. To address the first challenge, several approaches [1, 2, 4] have been developed to provide unambiguous interoperable security policy specifications using XML. These approaches provide syntactic tags for the security policy specifications and the tags show how the specifications should be interpreted. The general syntax of such approaches is rigid in the sense that specifications must adhere to the general rules of XML to ensure that all XML-aware software can at least read and understand the relative arrangement of information within them. This usually results in lengthy specifications as the user needs to define a lot of tags just to comply with XML standards. We have developed an approach based on ontology in [3], where the security policy has a fixed format, but users can derive individual components from an ontology to specify application-specific information needed in the security policy specifications. All these approaches can address Challenge 1) sufficiently and provide unambiguous security policy specifications across collaborating organizations. In this paper, we will present methods to address Challenges 2) and 3). To address Challenge 2), relevant security policies from various collaborating organizations need to be combined together. Bonatti et al. [5] dealt the policy composition problem through a multi-level security model. Bidan et al. [6] introduced a set of

6

S.S. Yau and Z. Chen

combination operators to combine security policies in a controlled manner. Several algebraic operators for combining enterprise privacy policies specified by EPAL were introduced in [7]. SPL [8] has stratified security rules, and there is a special layer of rules, which is used to control policy integration. These approaches are common in that they all require prior knowledge about the other collaborating organizations. However, in ubiquitous computing environments, groups of collaborating organizations are continuously changing with participating organizations joining or leaving the groups/. Each organization has little prior knowledge on the security policies of other collaborating organizations and how they will impact its own security policies. To address Challenge 3), Ribeiro, Zuquete and Ferreira [8] specified special rules to decide which security policy should override the others when there is a conflict. Similarly, Lupu and Sloman [9] specified “policy precedence relationships” to determine which one of two conflicting policies should apply and which should be ignored. Agrawal et al. [10] presented an approach to eliminating conflicts by checking the new policies and prevent them from being added to the integrated policy set if there is a conflict. All these approaches rely on some policy hierarchy structure, where certain policies are more important than the others, and more important policies override less important ones in case of conflict. For [10], the security policies that are already in the integrated set are more important than those that are waiting to join the set. It is difficult to adopt these approaches in ubiquitous computing environments because there is no authority in the group of collaborating organizations to establish such hierarchy and each participating organization is an autonomous entity. Hence, it is not reasonable to choose an organization’s security policies over the others in case of conflict. So far, no comprehensive approach addresses both Challenges 2) and 3) for collaborations among organizations in ubiquitous computing environments, where groups of collaborating organizations are continuously changing with little prior knowledge about when other organizations will join and what kinds of security policies they have specified. This makes policy integration and conflict reconciliation more difficult compared to the case that the groups of collaborating organizations remain the same.

3 Our Overall Approach Our approach aims at generating an integrated, congruous security policy set based on the security policies specified by individual participating organizations. Our overall approach is shown in Fig.1 and consists of the following three major parts. Part1) uses a specification language similar to the one we presented in [3] to generate security policy specifications for each collaborating organization, where a security policy can be specified as a quadtuple with the components derived from the ontology containing the key concepts needed for specifying security policies. For the sake of completeness, we will summarize the important aspects of this specification language needed in our approach in the Appendix.

Security Policy Integration and Conflict Reconciliation

7

Part2) adapts existing security policies to address the expanded user set for the collaboration. When an access to a resource is requested by a user from other participating organizations, the resource owner should make the decision on whether to grant the request to the resource. However, since the requester is from another organization, the resource owner may not have much knowledge about this requester, which makes it difficult to make a decision on the request. In reality, it is natural to evaluate a

OrgA

OrgB

Natural Language Security Policies

Natural Language Security Policies

Security Policy Specification Language

Security Policy Specification Language

Part1

Security Policy Specifications

Security Policy Specifications

Address expanded user set

Address newly generated resources Integration Process Negotiation-base Policy Generation

Similarity-base Security Policy Adaptation

Part2 Integrated Security Policy Specifications

Part3

Yes Have conflicts?

No

Negotiation -base Conflict Reconciliation

Final Security Policy Set for Collaboration

Fig. 1. Our overall approach for security policy integration and conflict reconciliation

8

S.S. Yau and Z. Chen

stranger’s trustworthiness based on where he comes from and how he is trusted in the organizations he belonged before. Thus, we will use a similarity-base security policy adaptation algorithm to help the resource owner make a decision based on evaluating whether the participating organizations that the user comes from have similar security policies, and whether the user request will be granted under those security policies. The details of this algorithm will be presented in Section 4. Part3) uses a negotiation-based approach to generating a congruous set of security policies which can be accepted by all participating organizations. Participating organizations will first provide their inputs towards the generation of the policies, and then make compromise in order to resolve possible conflicts and reach an agreement. This is used to address possible conflicts which arise from the integration, and to specify new security policies for the resources generated by the collaboration. In both cases, no single organization can claim the sole ownership of the generated resources, and thus no security policies of any participating organization should take priority over those of other organizations. The details of this approach will be presented in Section 5.

4 Similarity-Based Security Policy Integration Algorithm The specification approach presented in the Appendix provides an unambiguous, interoperable security policy specification that is understandable by other participating organizations. To enable collaborations among various collaborating organizations, security policies specified by these organizations need to be integrated to govern the interactions among them during collaboration. It is similar to corporation mergers, where company policies related to labor forces, equipments as well as intellectual properties need to be integrated before the merge can be completed. In such a case, the integration of the policies of different companies is done through negotiation talks between the management and legal departments of the merging companies. However, for dynamic collaborations in ubiquitous computing environments, it is simply not practical if we need to halt the collaboration and get human involvement every time there is an organization join or leave the group of collaborating organizations. In this paper, we present an alternative approach based on policy adaptation and negotiation to achieve dynamic security policy integration with minimum human intervention. During the collaboration, an organization may need to choose to make its resources available to other collaborating organizations so that users from those organizations can access the resources. Even though a collaboration may involve multiple organizations, it is rare that a user belong to two or more of these organizations. In case a user belongs to more than one organization during a collaboration, when that user requests to access a specific resource, we can require the user to specify which organization affiliation he wants to submit this request. This is similar to system administration that a person can be a normal user or an administrator. But, when the user wants to perform system administration tasks, he must sign in as an administrator. So we assume that when a user requests to access a resource, he belongs to only one organization. A general scenario of such pair-wise user-resource access scenario is shown in Figure 2.

Security Policy Integration and Conflict Reconciliation

9

Fig. 2. Pair-wise user-resource integration

There are two organizations OrgA and OrgB. OrgA has its user set UA= {ui, uj, …} and data set DatA ={datAp, datAq …}. A set of security policies PA has been specified to govern the data access behavior in OrgA. Similarly, OrgB has its user set UB, data set DatB, and security policy set PB. The collaboration will not only make existing data available to all the researchers, it may also generate some new data which appears in neither DatA nor DatB. For example, the collaboration can produce a new data datk through analysis of datAq and datBm. We use DatAB to denote the set of the newly generated data. The integrated security policy set should be able to handle all possible data access requests. Let u be the requester and dat be the requested data. There are three types of requests. Type1 consists of C1: dat ∈ DatA , u ∈ UA and C2: dat ∈ DatB , u ∈ UB, where u and dat are in the same organization. Type2 consists of C3: dat ∈ DatB , u ∈ UA and C4: dat ∈ DatA , u ∈ UB, where u and dat are in different organizations and the original security policies specified by the data owner cannot cover the new users from the other collaborating organization. Type3 is C5: dat ∈ DatAB, u ∈ UA ∪ UB, where dat is newly generated data and both collaborating organizations can claim ownership of the data. For Type1 requests, the corresponding original security policies in PA and PB apply. For Type3 requests, the policy integration needs to generate a new security policy to address the security concerns from both organizations. To generate such policies, similar to a conflict reconciliation process, we take the security concerns of the newly generated data from both organizations, and through a negotiation process, a common set of policies accepted by both organizations can be generated. The negotiation process will be discussed in Section 5.

10

S.S. Yau and Z. Chen

For Type2 requests, we will present a similarity-based security policy adaptation algorithm for generating integrated policies. This type of requests does not have existing security policies because the request and the resource belong to different organizations. Original security policies specified by the data owner cannot cover the new users from the other collaborating organization, and the collaborating organization does not have policies to govern the access to data not belonging to the organization. However, because one organization is the only owner of the data, the organization should control the generation of new security policies to address the new users of the data. To deal with this type of requests, adapting the existing security policies of the organization, which owns the data, to handle the new user set, is better than negotiation with the other organization. Because the data owner may have little knowledge about a requester which is not part of its original user set, it is difficult for the data owner to know how to adapt its security policy accordingly. However, it is very difficult to adapt existing security policies to address the new users without heavy human intervention, Because an organization may have specified a list of hundreds or even thousands security policies for its shared resources, it will greatly reduce the efficiency of the collaboration and easily cause human errors due to the large number of requests, On the other hand, even though the organization having the new requestor does not have a security policy to govern the access to this resource, it is often that there are some security policies for this requestor to access some resources similar to the resource it currently requests for accessing. For example, a user u ∈ UB wants to access the raw medical data dat1 ∈ DatA, which contains patients’ SSN. OrgA does not have existing security policy to cover u. If OrgA knows that u can access dat2 ∈ DatB, which contains patients’ mental evaluation results, medication history, and other sensitive information, it is reasonable for OrgA to draw the conclusion that u is trusted to handle sensitive data in OrgB. It is less risky to extend the access of dat1 to u compared to some other user u’ from the same organization which does not have such kind of access rights. Based on this observation, we can develop the following policy adaption algorithm based on policy similarity evaluation. We first need to define a metric for the similarities among different entities. Because our security policy specification is based on a vocabulary organized in a hierarchical structure [3], relative relations between various elements in security policies specifications can be established. Let e1, e2, and e3 be any three elements, e4 be the lowest common ancestor of e1 and e2 in the hierarchical structure, and e5 be the lowest common ancestor of e1 and e3. Element e1 is said to be closer to e2 than to e3 if e4 appears at a lower level in the class hierarchy than e5. The similarity index SI(ei , ej) between two elements ei and ej, is defined as follows:

{

SI (ei , e j ) =

1 if ei = e j SI (ek , e j ) / n if e j is ei 's ancestor, where n is the number of siblings e1 has and ek is ei 's parent SI (ei , el ) × SI (e j , el ) for all other cases, where el is the lowest common ancester of ei and e j

For example, e1:“SHA-1”is a child of e4:“Hash Function”. e2:“MD5” is also a child of e4. e3:“Fingerprint checking” is a child of e5: “Biometric Authentication” mechanism. Both e4 and e5 are derived from e6: “SecurityMechanism”. If these are all the elements in this example, then the similarity between e1 and e2, and between e1 and e3 can be calculated as follows:

Security Policy Integration and Conflict Reconciliation

11

SI(e1 , e2)= SI(e1 , e4) × SI(e2 , e4)=(1/2)×(1/2)=0.25, SI(e1 , e3)= SI(e1 , e6) × SI(e3 , e6)=((1/2)/2)×(1/2)=0.125. SI(e1 , e2) > SI(e1 , e3), which is in line with the fact that “SHA-1” is closer to “MD5” compared to “Fingerprint checking”. The similarity between two atomic security policies, P= (S, A, O, C) and P’ = (S’, A’, O’, C’), is defined as follows: SI(P, P’) = SI(S, S’) × SI(A, A’) × SI(O, O’) × SI(C, C’). The similarity between a composite security policy P = (P1∧P2) ∨ P3∨ ~P4 and an atomic security policy P’ is defined as follows: SI(P, P’)= (SI(P1, P’)× SI(P2, P’))+ SI(P3, P’)+(- SI(P4, P’)). The candidate policy for adaption can be a composite security policy. When searching for similar policies for the request in the collaborating organization, we only search for the atomic policies related to the requester. If there is a composite security policy, we can treat it as several atomic policies during our similarity calculation. Thus, we do not define the similarity between two composite security policies. Based on these definitions, our similarity-based security policy integration algorithm allows the data owner to make the decisions on whether to grant access to a requester from a participating organization based on similarity analysis of the two organizations’ security policies. Let P be the security policy specified by the data owner for a specific data dat. Let TS be a similarity threshold the data owner specified. A security policy P’ in the collaborating organization is selected by the data owner to help it make security decision only if SI(P, P’) is greater than TS. A greater TS will guarantee the selected policies to be closer to P, but if the selection criteria is too tight, it may not be able to return any policy which is certainly not the purpose of this process. The similarity-based security policy adaptation algorithm can be presented as follows: Input: a request where dat  DatA , u  UB Process: 1) Based on the request action and current situation, find the suitable policy, PA, in OrgA which is used to handle this kind of request if the user is from OrgA. 2) Find PS(u), which is the set of atomic policies regarding the requester u in OrgB. 3) For every P in PS(u), calculate SI(PA, P) 4) Select a subset PS’(u) of PS(u), where for every P in PS’(u), SI(PA, P) > TS 5) Divide PS’(u) into two subsets, PSP(u) and PSD(u), where PSP(u) is the set of rules that permit access for u, and PSD(u) is the set of rules that deny access for u. SI ( PA , P) ¦ SI ( PA , P)  P¦ PPSP ( u ) PS D ( u ) 6) Calculate R u W , where W is a confidence min( SI ( PA , P), P  PS '(u )) weight OrgA assigned to OrgB. W is originally set to 1, and every time if OrgA grants access for a user in OrgB based on this calculation and the user does not cause any security problem, W is increased by 1. On the other hand, if the user failed to meet the expectation, W is decreased by 1. Output: If R > 1, the request is granted, otherwise it is denied.

12

S.S. Yau and Z. Chen

The original similarity threshold can be generated based on sampling result of the collaboration with domain knowledge. If on the average, every node has N children, two policies with a similarity index in the range N-3 ~ N-4 are considered to be similar. At runtime, the threshold can be updated based on previous evaluation results to make it more suitable for a specific collaboration. The advantage of this approach is that this process can be automated and very efficient.

5 Negotiation-Based Conflict Reconciliation Protocol We consider two security policies P1 and P2 have conflict with each other if for their satisfactory sets Q1 and Q2, there exists an element e such that e ∈ Q1 and e ∈ Q2 , where Q1 and Q2 are satisfactory sets for P1 and P2 respectively, which contain all the elements that comply with their corresponding policies. As discussed in Section 1, due to the dynamic nature of collaborations among organizations in ubiquitous computing environments, when specifying its own security policies, an organization has difficulties in knowing what security policies have specified by other organizations which may collaborate with in the future. Thus, for a complex system, conflicts are hard to avoid. It is important that the collaborations do not undermine the security of all participating organizations. On the other hand, security should not deny participating organizations the possibility of collaboration simply because there exist some conflicting security policies. Our conflict reconciliation protocol can improve the collaboration chance. It is based on the following observations of real world scenario: it is difficult to find suitable collaborator and collaboration can generate extra synergies. Thus, instead of rejecting a possible collaborator because of some conflicting security policies, collaborating organizations often prefer to take certain risk and move forward with the collaboration by relaxing their security policies and adopting some weaker security policies to resolve the conflicts. Policy P1 is said to be weaker than P2 if P1 a Q1, P2 a Q2, and Q2 ⊂ Q1. Because each collaboration goal and collaborating participants may be different, the security compromise should not be done in a static manner. Our reconciliation protocol captures these dynamic factors as situation-aware compromise thresholds, which specify how much compromise an organization is willing to make for a specific collaboration. A compromise threshold, TC, is specified based on the following factors: (1) How much benefit can be generated if this collaboration goes through? (2) How much damage it may cause by adopting a weaker security policy? (3) What is the track record of this specific collaborating organization? How did it perform in pervious collaborations? The value of Tc should be updated when the above three factors change. Now, we will present the negotiation-based conflict reconciliation protocol as follows:

Security Policy Integration and Conflict Reconciliation

13

Step1 For a specific resource, each organization specifies a chain of security requirements instead of just one security requirement. The head of the chain is the most desirable security policy. Traveling through the chain from the head, the security requirements become weaker and weaker. Step2 Each organization specifies compromise thresholds under various situations as (TC,T), which means when situation T is true, the weakest security policy it is willing to take is indexed by TC along the chain. Step3 At runtime, the system obtains the compromise thresholds TC1 and TC2 for both organizations based on the current situation, and identifies the index for the two conflict policies in their chain. Test to see whether the current index is less than the compromise threshold. If not, go to Step6. Step4 One organization is randomly selected to make compromise first, and proceeds along the policy chain to select a weaker policy; Step5 Test whether there is still any conflict. If no conflict is detected, then reconciliation is done. If there is still conflict, repeat Step4 by proceeds along the other organization’s policy chain. Step6 Both organizations make compromise in turns, until one of the compromise thresholds is reached. If no reconciled policy is generated by then, failure is reported for human intervention.

For two conflict policies P1 and P2, the above steps can be summarized in the following pseudo-code. // sp1[] is the policy chain contains P1, and sp2[] is the policy chain contains P2 x1 = GetCompThreshold1(θ ); x2 = GetCompThreshold2(θ ); n1 = Sizeof(sp1); n2 = Sizeof(sp2); flag = 0; for (p=0, q=0; p≤n1, q ≤ n2; ) { if ((sp1[p] not conflict with sq2[q]) substitute P1 with sp1[p] and P2 with sp2[q], break; else{ if(flag == 0){ p++; flag = 1; }else{ q++; flag = 0; } } } if (p > n1 || q > n2) report the conflict cannot be automatically reconciled For the sake of simplicity, we only illustrate the reconciliation process between two collaborating organizations. The reconciliation process involving more than two organizations can be handled in a similar fashion: We will have multiple security policy chains and the compromise may be made in a loop circling all participating organizations, instead of going back and forth between two organizations.

14

S.S. Yau and Z. Chen

6 An Illustrative Example In this section, we will give an example to illustrate our approach. Continuing with the scenario described in Section 4, a university OrgA and a pharmaceutical company OrgB collaborate on a joint vaccine development project. UA is made up of a set of professors (UP), a set of graduate research assistants (UG) and a set of undergraduate students (UU). DatA includes some research papers (datAp), unpublished experiment results and paper drafts (datAo), and composition data on several promising chemicals, which the users in UA intend to use (datAq). UB consists of a set of research scientists (US), a set of lab technicians (UT) and several directors for research (UD). DatB includes patents of existing drugs related to this disease (datBn), a list of potential clinical trial participants (datBr), and a set of secret methods for fast vaccine prototype development (datBm). These users and data can be organized in a hierarchical manner as shown in Figure 3. Entity Is a

Is a

Resource

User

Is a

Data

Is a

dat Ap

Is a

Confidential Info

Is a

dat Bn

Is a

dat Ao

UD

dat Aq

Is a

Participant Is a

UG

Is a Is a

Is a

Supervisor

Is a

Is a

Pub Info

Is a

Is a Is a

Volunteer

Investigator

Is a

Is a

UU

Is a

UT

dat Bm

UP

Is a

US

DatBr

Fig. 3. The hierarchy of concepts of the example

To follow our approach, Part1), we have the following security policies specified: OrgA – (1) All members can access the published papers at anytime. P1: (UP ∪ UG ∪ UU, Access, datAp, __ ) (2) Professors can always access and modify the unpublished paper draft. P2: (UP, Access∪Modify, datAo, __ ) (3) Graduate assistants can always access the unpublished paper draft. However, they can only update the draft if they are the co-authors. P3: (UG, Access, datAo, __ ) ∧ (UG, Modify, datAo, C3), C3: Up ⊂ Author(datAo) OrgB – (1) All members can access the drug patents at anytime. P4: (UD ∪ US ∪ UT, Access, datBn, __ ) (2) Scientists and directors can access the trial participant list. P5: (UD ∪ US, Access, datBr, __ ) (3) Only directors can update the trial participant list. P6: (UD, Modify, datAo, __ )

Security Policy Integration and Conflict Reconciliation

15

Part 2), if a professor u from OrgA wants to access the list of trial participants, OrgB’s current policies cannot handle this request. Follow our similarity-based algorithm, we have 1) Because the request is to access datBr, P5 is the suitable policy for similarity comparison. We can transform P5 into P5’:(UD, Access, datBr, __ ) ∨ P5’’:(US, Access, datBr, __ ) 2) Select policies for professors in OrgA. We only consider the atomic component for professors if the policy also applies to other users. Thus, PS(u) = { P1’:(UP, Access, datAp, __ ), P2’:(UP, Access, datAo, __ ), P2’’:(UP, Modify, datAo, __ )}. 3) SI(P5, P1’) = SI(P5’, P1’) + SI(P5’’, P1’). SI(P5’, P1’) = SI(UD, UP) × SI(Access, Access) × SI(datBr, datAp) × SI( __, __) = 1/32 × 1 × 1/32 × 1 = 1/1024. Similarly, SI(P5’’, P1’) = 1/16. Thus SI(P5, P1’) = 65/1024. We can also calculate SI(P5, P2’) = 9/128, and SI(P5, P2’’) = 9/3200. 4) For this example, assume that OrgB sets the threshold Ts = N-4. Because on the average every node has 3 children, Ts = 1/81, and hence both P1’ and P2’ are selected. 5) Both P1’ and P2’ grant access to u, thus PSP(u) = { P1’ , P2’ } and PSD(u) = φ 6) Assume that this is the first time the two organizations collaborate. Then, W =1. R = (SI(P5, P1’)+ SI(P5, P2’)) / min (SI(P5, P1’), SI(P5, P2’)) = (65/1024+9/128) / min(65/1024, 9/128) = 137/65. Because R > 1, the access is granted to u. The system can make this decision automatically without any human supervision at runtime. Because professors in OrgA are the main contributors (Investigators) for this project, and professors can access confidential information in OrgA, it is reasonable for a human supervisor in OrgB to grant a professor from OrgA the access to the participant list. This decision is in line with the real world situation. Part3), since negotiation is used for conflict reconciliation as well as security policy generation for newly generated resources, we only need to show an example of the latter to illustrate this negotiation process. Assume that based on datAq, the collaboration developed several vaccine prototypes using the methods listed in datBm, and performed several clinical trials. We will have a new set of data datk which is the evaluation results of these vaccine prototypes. Neither OrgA nor OrgB can claim the ownership of datk completely. Thus, a security policy for datk will be generated through negotiation. Following our negotiation protocol discussed in Section 5, for this example, Step 1. OrgA presents its list of input policies sp1[], with the most desirable policy for at front. OrgB also presents its inputs sp2[]. Step 2. Both OrgA and OrgB specify their compromise thresholds for various situations as (TC,θ) pairs. Step 3. After analyzing the collaboration history and the importance of the data, the compromise thresholds for both organizations are 2. In the following we only list the input policies up to the threshold.

16

S.S. Yau and Z. Chen

OrgA: (1) Only investigators can access datk through a communication channel using 64-bit encryption. sp1[0]: (UI, Access, datk, “64-bit encryption” ) (2) Both investigators and participants can access datk through a communication channel using 64-bit encryption. sp1[1]: (UI ∪ UPa, Access, datk, “64-bit encryption” ) (3) Both investigators and participants can access datk. sp1[2]: (UI ∪ UI, Access, datk, __ ) OrgB: (1) Both investigators and participants can access datk through a communication channel using 128-bit encryption. sp2[0]: (UI ∪ UPa, Access, datk, “128-bit encryption” ) (2) Both investigators and participants can access datk through a communication channel using 64-bit encryption. sp1[1]: (UI ∪ UPa, Access, datk, “64-bit encryption” ) (3) Anyone in the collaboration can access datk. sp2[2]: (U, Access, datk, __ ) Both organizations start with their first choice, sp1[0] and sp2[0]. Obviously, there is a conflict since sp1[0] allows less investigators to access datk while sp2[0] grant access to both investigators and participants. Step 4. OrgA is randomly selected to initialize the negotiation. It will proceed along the input policy list and present the next policy sp1[1] which is weaker than sp1[0]. Step 5. After analyzing sp1[1] and sp2[0], there is still a conflict since sp2[0] requires access through 128-bit encryption while sp1[1] only requires 64-bit encryption. Hence, Step 4 is repeated. This time, OrgB needs to make its compromise in the negotiation process. The next policy in its input list, sp2[1], is presented. Step 6. After analysis, no conflict is found between the two input policies sp1[1] and sp2[1], and hence a security policy for datk is generated: (UI ∪ UPa, Access, datk, “64-bit encryption” ) Note that we have not reached the compromise threshold. Though there is no conflict between sp1[2] and sp2[2] either, the negotiation process stops as soon as a pair of non-conflict policies is found because they are stronger than the policies appearing later in the list. This will provide the best protection for the collaboration system without creating any conflict.

7 Conclusions and Future Work In this paper, we have presented an approach to security policy integration and conflict reconciliation for collaborations among organizations in ubiquitous computing environments. Based on the similarities between security policies, a resource owner can adapt its existing security policies to cover the new users from collaborating organizations using the similarity indices as a selection criterion for selecting similar policies in the collaborating organization and make a decision based on their inputs. Collaborating organizations with conflicting security policies can try to resolve the conflicts by gradually making security compromise and relaxing their security

Security Policy Integration and Conflict Reconciliation

17

policies until either the conflict is resolved or the conflict reconciliation process reaches certain compromise threshold associated with that particular collaboration. This approach can also be used for generating security policies for the new resources generated as a result of the collaboration, where all participating organizations specify their security concerns for the resource, and the negotiation process is trying to find a common set of policies that will satisfy all the organizations’ concerns. We plan to conduct case studies to establish some applicable guidelines for specifying the similarity threshold and updating the confidence weight in the decision making process. In addition, we will investigate the impact of extending the security policy set through logical reasoning and try to associate that impact with policy similarity so that we can update the resource owner’s security policies based on the similarity analysis, rather than relying on similar security policies specified by the collaborating organizations.

References [1] Xu, F., Lin, G., Huang, H., Xie, L.: Role-based Access Control System for Web Services. In: Proc. 4th Int’l Conf. on Computer and Information Technology, pp. 357–362 (2004) [2] Bhatt, R., Joshi, J.B.D., Bertino, E., Ghafoor, A.: Access Control in Dynamic XMLBased Web Services with XRBAC. In: Proc. 1st Int’l Conf. on Web Services (2003), http://www.sis.pitt.edu/~jjoshi/ICWS_XRBAC_Final_PDF.pdf [3] Yau, S.S., Chen, Z.: A Framework for Specifying and Managing Security Requirements in Collaborative Systems. In: Yang, L.T., Jin, H., Ma, J., Ungerer, T. (eds.) ATC 2006. LNCS, vol. 4158, pp. 500–510. Springer, Heidelberg (2006) [4] OASIS, eXtensible Access Control Markup Language (XACML) Version 2.0, OASIS standard (2005), http://docs.oasis-open.org/xacml/2.0/ access_control-xacml-2.0-core-spec-os.pdf [5] Bonatti, P.A., Sapino, M.L., Subrahmanian, V.S.: Merging Heterogeneous Security Orderings. In: Martella, G., Kurth, H., Montolivo, E., Bertino, E. (eds.) ESORICS 1996. LNCS, vol. 1146, pp. 183–197. Springer, Heidelberg (1996) [6] Bidan, C., Issarny, V.: Dealing with Multi-policy Security in Large Open Distributed Systems. In: Quisquater, J.-J., Deswarte, Y., Meadows, C., Gollmann, D. (eds.) ESORICS 1998. LNCS, vol. 1485, pp. 51–66. Springer, Heidelberg (1998) [7] Backes, M., Durmuth, M., Steinwandt, R.: An Algebra for Computing Enterprise Privacy Policies. In: Samarati, P., Ryan, P.Y.A., Gollmann, D., Molva, R. (eds.) ESORICS 2004. LNCS, vol. 3193, pp. 33–52. Springer, Heidelberg (2004) [8] Ribeiro, C., Zuquete, A., Ferreira, P.: SPL: An access control language for security policies with complex constraints. In: Proc. Network and Distributed System Security Symp (NDSS 2001), pp. 89–107 (2001) [9] Lupu, E., Sloman, M.: Conflicts in policy-based distributed systems management. IEEE Trans. on Software Engineering 25(6), 852–869 (1999) [10] Agrawal, D., Giles, J., Lee, K.W., Lobo, J.: Policy ratification. In: Proc. 6th IEEE Int’l Workshop on Policies for Distributed Systems and Networks (POLICY), pp. 223–232 (2005) [11] Yau, S.S., Huang, D., Gong, H., Yao, Y.: Support for Situation-Awareness in Trustworthy Ubiquitous Computing Application Software. J. Software Practice and Engineering 36(9), 893–921 (2006)

18

S.S. Yau and Z. Chen

Appendix Ontology-Based Security Policy Specification Our approach uses ontology-based security policy specification language in [3]. For the sake of completeness, we summarize the important features of the specification language. Natural language security policy specifications can be expressed by a set of logic expressions with the key vocabulary derived from a hierarchical security policy ontology. The ontology is shown in Fig. 4, where the generic classes in the core ontology are defined as follows: • SecurityPolicy can be an AtomicSecurityPolicy or CompositeSecurityPolicy. • CompositeSecurityPolicy is composed of AtomicSecurityPolicy. • AtomicSecurityPolicy has four components: Subject, Object, Action and Condition. • Both Subject and Object are Entity. • Entity can be User or Resource, and Data is also considered as Resource. • Action specifies what action Subject can perform on Object, which includes Access, Share, Delegate, Delete and Modify. • Condition can be specified as a specific SecurityMechanism, which must be adopted, or as a certain environment or system Situation, which will trigger the security policy. The value of a Situation expression is determined by situationaware processors [11].

Security Policy

compose

Is a

Is a

Composite Security Policy

Has

Atomic Security Policy

Has

Subject Is a

Is a

Condition

Has Has Is a

Action

Is a

Resource

Is a

Is a Is a

User Is a

Data

Is a

Security Mechanism

Object

Entity

Is a

Access

Is a

Share

Situation

Is a

Modify

Delegate Delete

Fig. 4. An ontology for security policy specification

This ontology provides the fundamental vocabulary for specifying security policies. However, users can extend the basic concepts and introduce more applicationspecific terms when specifying their own security policies. The basic building block is the atomic security policy, which can be specified as a quadtuple: P = (S, A, O, C), where S ⊂ Subject, A ⊂ Action, O ⊂ Object, and C ⊂ Condition. The quadtuple is interpreted as follows: Security policy P specifies that a set of subjects S can perform

Security Policy Integration and Conflict Reconciliation

19

a set of actions A on a set of objects O when a set of conditions C is met. A request is granted if there exists a policy P = (S, A, O, C) such that the requester s ∈ S, the requested action a ∈ A, the requested target o ∈ O and the conditions listed in C are satisfied when the request is made. If C is omitted, it means the policy applies to all conditions. Atomic security policies can be composed through logic operators conjunction ‘∧’, disjunction ‘∨’ and negation ‘¬’ to form more complex security policy specifications. Based on these definitions, for a give set of security policies, we can derive a satisfactory set Q which contains all the elements that comply with the given set of policies. We use P a Q to denote this logic reasoning process. This specification approach has two advantages. First, because all four elements within a specification are specified based on the ontology, each element is clearly defined. There is only one interpretation for a quadtuple, and thus a specified security requirement is unambiguous. Second, the hierarchical structure of the ontology can be used to establish common understanding of different terms used by different organizations in their security policy specifications. For example, one policy specifies “administrator can only access the account information through a 128-bit encrypted channel”, and the other policy specifies “administrator must use a 128-bit encrypted channel to access the username/password information”. Based on the ontology we used in our specifications, both “account information” and “username/password information” are Objects. If the two collaborating organizations establish a common understanding that “username/password” used by OrgA is the “account information” used by OrgB, the system will be able to “understand” this fact and process the policies from both organizations based on this fact accordingly. In this example, the two security policies will be treated as the same policy. The algorithm to bridge different terms used by different organizations and form common understandings was presented in [3].

Improved Weighted Centroid Localization in Smart Ubiquitous Environments Stephan Schuhmann1 , Klaus Herrmann1 , Kurt Rothermel1 , Jan Blumenthal2 , and Dirk Timmermann2 1 University of Stuttgart, Germany Institute of Parallel and Distributed Systems (IPVS) {firstname.lastname}@ipvs.uni-stuttgart.de 2 University of Rostock, Germany Institute of Applied Microelectronics and CE {jan.blumenthal,dirk.timmermann}@uni-rostock.de

Abstract. Location-awareness is highly relevant subject in ubiquitous computing, as many applications exploit location information to provide adequate services or adapt to a changing physical environment. While GPS provides reliable outdoor localization, indoor positioning systems present a bigger challenge. Many indoor localization systems have been proposed. However, most of them rely on customized hardware or presume some dedicated infrastructure. In this paper, we focus on WLAN-based localization in smart ubiquitous environments. We propose an improved scheme of the Weighted Centroid Localization (WCL) algorithm that is robust and provides higher location accuracy than the original WCL algorithm. The improvements are based on the use of dynamic weighting factors that are solely dependent on the correlation of the Received Signal Strength Indicators of the received beacon signals. Compared to the original WCL scheme, our approach does not increase requirements to the environment. Real-world experiments in a typical environment that we report on in this paper confirm that the increased location accuracy determined in previous calculations is reproducible in a realistic noisy environment. This provides a simple, cost-efficient, and battery-conserving, but yet adequate technique for getting the accurate location information of mobile devices.

1

Introduction

Location-awareness is an essential service for many wireless ubiquitous computing scenarios, as many applications integrate location information to increase context knowledge. However, ubiquitous computing environments have specific characteristics which limit the approaches that are applicable to maintain location-awareness. For instance, pervasive applications have to deal with fluctuating availability of devices due to mobility and network failures. Thus, environmental contexts are highly dynamic. Furthermore, processor performance and available energy of mobile nodes in ubiquitous computing scenarios are often limited. Therefore, intensive communication and computation tasks are not F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 20–34, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Improved Weighted Centroid Localization

21

feasible. In this context, algorithms are subject to strict requirements covering reduced memory consumption, communication, and processing time. In Smart Environments, available infrastructure devices can provide additional support for mobile devices. Determining the position of nodes in wireless networks, particularly in noisy environments, represents a real challenge. To identify the exact coordinates of a device (also called Station Of Interest, or SOI ) requires measuring a distance, e.g., measuring Time of Arrival (ToA) or Time Difference of Arrival (TDoA). Difficulties concerning time measurements result from synchronization of involved devices as well as the high computational effort to calculate the position. Measuring the Received Signal Strength Indicator (RSSI) offers a possibility to realize distance determination with minimal effort. A good localization algorithm should calculate a position as fast as possible and should be resistant to environmental influences as well as imprecise distances. However, it is desirable to use standard off-the-shelf consumer products without the necessity for far-reaching customization as this reduces the effort necessary for setting up the positioning system. Thus, our approach of combining the Weighted Centroid Localization (WCL) [1] with standard Wireless LAN technology has high practical relevance in typical environments. The main contribution of this paper is an improved Weighted Centroid Localization scheme that uses weighting factors of dynamic degrees to increase location accuracy in smart ubiquitous environments. Therefore, we derived optimal dynamic weighting factors through theoretical calculations and approximated the obtained weighting factors by a power function using the least-squares method. Furthermore, we evaluated the improved localization scheme in an office room that represents a typical noisy ubiquitous computing environment. The measurements confirmed that our approach increases location accuracy almost to the best possible value for weighted centroid schemes. The paper at hand is divided into six sections. The second section discusses the requirements to be met by the localization scheme and the environment. Section 3 gives a broad survey of existing localization approaches. In Section 4, we at first describe the original WCL scheme. Following, we derive an improved WCL scheme that uses dynamic weighting factors and increases location accuracy. We present our theoretical and practical evaluation results in Section 5. This is followed by the conclusion and a brief outlook to future work in Section 6 which closes this paper.

2

Requirements

In this paper, we focus on indoor localization for resource-restricted devices in smart ubiquitous environments. This requires to avoid high battery consumption, as mobile devices like PDAs or smart phones are equipped with limited battery and computation power. Hence, we aim at minimizing the requirements for the localization devices as well as the infrastructure to allow applicability to a wide

22

S. Schuhmann et al.

range of scenarios. This means we want to provide a flexible solution for indoor localization. Thus, our system has to maintain the following properties: – Minor Environmental Requirements: The environment solely has to provide infrastructure nodes with fixed positions that send periodic beacon signals. Many realistic ubiquitous scenarios fulfill this property, as they include 802.11 WLAN Access Points (APs), for example. These devices have to know their exact position and notify other devices of their location, e.g. within the beacon signals which needs slight adaptations to the APs. If changes to the APs are undesired, AP positions can be received by queries to a location server that provides the clients with AP locations. – Minimal Costs: Additional hardware costs should be avoided by relying on standard components that are available in typical ubiquitous scenarios, both on the client and on the infrastructure side. For instance, the localization scheme ought to integrate existing infrastructure devices without the need of large changes in their configuration. To minimize costs, we rely on exactly four beacon nodes in this paper. These beacons are placed at the edges of a rectangular plain, as common rooms typically have a rectangular base. This AP positioning enables nodes to localize themselves in the whole region. – Minimal Communication Needs on the Client Devices: Since power supply is critical for small mobile devices, the clients should avoid any additional communication during the localization process to maximize battery lifetimes. So, only the infrastructure devices should send beacons needed for the localization process, while the mobile devices have to exploit the information provided by these beacons. As localization is supposed to be executed in various scenarios in real-time, a solution that demands prior configuration of the client devices is not feasible, as this limits the use of the system. – Applicability in Realistic Ubiquitous Scenarios: Contrary to laboratory environments, realistic scenarios often suffer from effects like multi-path propagation or interferences with other wireless networks operating on the same frequency spectrum. This can significantly influence the accuracy of the localization. A localization scheme must be able to cope with this interference in order to provide accurate localization nevertheless.

3

Related Work

In this section, we investigate several related localization approaches concerning the above requirements to confine WCL from these approaches, and we clarify why they prove inadequate in our scenarios. In Figure 1, we provide a classification of proposed schemes. At first, these schemes can be divided into those that are based on coordinates and those that are coordinate-free. Coordinate-free schemes, as presented by Fekete et al. [2], focus on an abstract way of location-awareness. These schemes aim at achieving consistent solutions by the use of geographic clustering. Unfortunately, they rely on rather dense sensor networks to achieve adequate accuracy and, therefore, are not usable here. Thus, we focus on coordinate-based approaches that can further be divided into

Improved Weighted Centroid Localization

23

Fig. 1. Classification of proposed localization schemes

range-free and range-based schemes. Range-free schemes comprise implicit distance measurements, while range-based schemes use explicit distances for localization. Regarding range-free schemes, there exist three main approaches: – Anchor-based approach: This approach uses anchor beacons, containing two-dimensional location information (xi , yi ), to estimate node positions. After receiving these beacons, a node estimates its location using a specific formula. Besides a scheme proposed by Pandey et al. [3], another anchorbased approach is Centroid Localization (CL), proposed by Bulusu et al. [4]. In this approach, the SOI calculates its position as the centroid of the coordinates from the stations whose beacons are received. While CL performs only averaging the coordinates of beacons to localize SOIs, the Weighted Centroid Localization (WCL) algorithm [1] uses weights to ensure an improved localization and, hence, represents a direct advance over CL. – Hop-based approach: In this type of localization scheme, the SOI calculates its position based on the hop distance to other nodes [5],[6]. This scheme delivers an approximate position for all nodes in networks where only a limited fraction of nodes have self-positioning capability. Through this mechanism, all nodes in the network (including other anchors) get the shortest distance, in hops, to every anchor. However, this approach only works well in dense networks and relies on flooding of messages which produces high communication overhead that is undesired in our scenarios. – Area-based approach: Area-based schemes [7],[8] perform location estimation by isolating the environment into triangular regions between beaconing nodes. The presence inside or outside of these triangular regions allows a node to narrow down the area in which it can potentially reside. Area-based approaches normally perform particularly well in heterogeneous networks with high node density. Unfortunately, this condition does not hold in our case and, thus, area-based approaches cannot be used here.

24

S. Schuhmann et al.

Besides range-free schemes, a couple of approaches exist that are based on the transmission ranges of the nodes. Range-based approaches can be subdivided into the following approaches: – Signal Propagation Time: The Time of Arrival (ToA) technology is commonly used as a means of obtaining range information via signal propagation time. This approach is used by satellite navigation systems [9]. Though this approach guarantees high precision and can be globally used, it suffers from several major drawbacks as this scheme can only be used outdoors, needs expensive hardware, and the receivers consume much energy. Thus, ToA approaches are inapplicable in our case. – Localization by Measuring Signal Propagation Time Differences and Arrival Angles: This technology is based on measurements of the Time Difference of Arrival (TDoA) [10], or the Angle of Arrival (AoA) [11]. While TDoA estimates the distance between two communicating nodes (this is also called ranging), AoA allows nodes to estimate and map relative angles between neighbors. Like ToA technology, TDoA and also AoA rely on special hardware that is expensive and energy consuming. Thus, these approaches are unsuited here. – Simple Signal Strength Measurements: This approach uses the Received Signal Strength Indicator (RSSI) of incoming beacon signals and has been proposed for hardware-constrained systems. Contrary to techniques like ToA, TDoA or AoA, RSSI is the only feature that is measurable with reasonably priced current commercial hardware. RSSI techniques use either theoretical or empirical models to translate signal strengths into distance estimates where each point is mapped to a signal strength vector [12], or to a signal strength probability distribution [13]. This technology suffers from problems such as background interference or multi-path fading which make range estimates inaccurate, as shown in [14]. Furthermore, many signal strength based approaches rely on special hardware, like small infrared badges [15], magnetic trackers [16], or multiple cameras [17], which usually are not included in typical environments and mobile devices. Some approaches do not use client-based, but sniffer-based localization [18]. However, this technique cannot be used here as it assumes that the access points can localize other devices which induces major changes in the APs. Many approaches use location fingerprinting for localization in wireless networks [12],[13]. This method consists of a training phase and a positioning phase. Unfortunately, this approach needs additional preconditioning and the creation of a training database for several environments beforehand, which makes it inapplicable in our case. In summary, one can see that most approaches cannot be chosen since they rely on extensive special hardware, need additional preconditioning to build a training database, use sniffer-based localization, or depend on high node densities to perform well. This narrows the huge field of localization schemes down to few anchor-based approaches. Among those, Weighted Centroid Localization uses more fine-grained parameters than CL to weight the received beacon signals, as

Improved Weighted Centroid Localization

25

CL simply uses binary weighting (weight 1 for those APs whose beacons were received, weight 0 for the others). This ensures higher location accuracy for WCL.

4

Improved Weighted Centroid Localization

After a short introduction to the original WCL scheme with static weighting factors [1], we propose an advanced scheme that uses dynamic weighting factors which are based on the correlation of the received RSSI values. 4.1

Static Degree Weighted Centroid Localization (SWCL)

We assume that the network consists of SOIs that want to localize themselves, and n beacons. In our case, beacons are WLAN access points whose position is assumed to be known exactly. We depend on a small number of n = 4 beacons, placed at the edges of a rectangular plain. The SOIs are mobile devices that initially do not know their own position. Algorithms such as CL use centroid determination to calculate a device’s position [4]. In the first phase, each beacon Bj sends its position (xj , yj ) to all nodes within its transmission range, which can simply happen through the periodically sent beacon signals. In the second phase, the SOI Pi calculates an approximation (xi , yi ) of its real position (xi , yi ) by a centroid determination from all n positions of the beacons in range (1). 1 (xj , yj ) n j=1 n

(xi , yi ) =

(1)

The localization error fi for the SOI Pi is defined as the distance between the exact position (xi , yi ) and the approximated position (xi , yi ) of Pi (2).  (2) fi = (xi − xi )2 + (yi − yi )2 While CL only averages the coordinates of beacon devices to localize SOIs, WCL uses weights to ensure an improved localization. In CL, all weights of received beacon signals are implicitly equal to 1. WCL represents a generalization of this scheme since it introduces variable weights wij for each device Pi and each beacon Bj . These weights depend on the distance between the two as it will be explained later in this section. In the more general WCL equation, the number of beacons n is replaced by the sum of weight factors wij , and each beacon position (xj , yj ) is multiplied by its weight factor wij . Hence, (1) is expanded to the WCL formula (3) yielding the new approximation (xi , yi ) for Pi ’s position. n j=1 wij · (xj , yj )   n (3) (xi , yi ) = i=1 wij The weight wij is a function depending on the distance and the characteristics of the SOI’s receivers. In WCL, shorter distances cause higher weights. Thus, wij

26

S. Schuhmann et al.

and dij (the distance between beacon Bj and SOI Pi ) are inversely proportional. As an approximation, the correlation is equivalent to the function 1/dij . To weight longer distances marginally lower, the distance is raised to a higher power hs . For a concentric wave expansion with a linear characteristic of the receiver and a uniform density of the beacons, we form (4). wij =

1 (dij )hs

(4)

The static degree hs has to ensure that remote beacons still impact the position determination. Otherwise in case of a very high hs , the approximated position moves to the closest beacon’s position and the positioning error fi increases. There exists a minimum of fi where hs is optimal [19]. However, as the next section clarifies, this use of static weight values leads to suboptimal accuracy if you compare it to the use of adapted dynamic weight factors that are based on the correlation between the received RSSIs from several APs. According to Friis’ free space transmission equation (5), the detected signal strength decreases quadratically with the distance to the sender. Prx = Ptx · Gtx · Grx · (

λ 2 ) 4πdij

(5)

Prx = Remaining power of wave at receiver Ptx = Transmission power of sender Gtx , Grx = Gain of transmitter and receiver λ = Wave length dij = Distance between sender and receiver In embedded devices, the received signal strength is converted to the RSSI which is defined as ratio of the received power to the reference power (Pref ). Typically, the reference power represents an absolute value of Pref = 1 mW .  The actually received power Prx can be obtained by the RSSI and vice versa, as denoted in the following equations.  Prx = Pref · 10

RSSI 20

⇐⇒

RSSI = 20 · log10

 Prx Pref

(6)

An increasing received power results in a rising RSSI. Thus, distance dij is  indirect proportional to Prx , and (7) with power gs (= hs ) is formed.  gs wij = (Prx )

(7)

So, it can be seen that the knowledge of dij is not necessary, as the algorithm  or, alternatively, the RSSI. In this paper, can simply use the received power Prx we use the RSSI, i.e. we take (6) and (7), and form (8). wij = (Pref · 10

RSSI 20

)gs .

(8)

Improved Weighted Centroid Localization

AP3

R1,1 AP1

R1,2

R2,1 l1

AP4 AP1

AP2

a)

l2

AP3

R1,1

R2,2

l1

R2,2

R1,2

R2,1 AP2

27

AP4

l2

b)

Fig. 2. a) Distribution of gopt at a rectangular plain (p = 43 ), b) Potential for improvement of location error at a rectangular plain (p = 43 ), relative to longer side l1

In practical scenarios, the ideal distribution of Prx does not hold, because the radio signal propagation is interfered with many influencing effects, like reflection on objects, diffraction at edges, or superposition of electro-magnetic fields.  represent different values which yields localization errors. Thus, Prx and Prx 4.2

Dynamic Degree Weighted Centroid Localization (DWCL)

Using an optimally chosen static degree gs , as original WCL does, results in improved location accuracy, compared to CL [1]. However, the degree gopt that leads to minimal error can significantly differ from gs in some regions, as this  section will clarify. Therefore, we used (5) and (6), and put Prx = Prx to calculate the expected RSSI in various positions within an idealized region under observation. Following, we used the WCL algorithm with different powers g to determine the optimal gopt which yields highest location accuracy at a particular place in the region. Figure 2a shows the distribution of the factors gopt in a rectangular region where the proportion p of the length l1 of the longer side divided by the length l2 (≤ l1 ) of the shorter side is p = ll12 = 43 . It can be seen that the distribution of gopt is very uneven. The values at the regions R1,1 and R1,2 exactly between two adjacent anchors with a distance of l1 to each other (e.g., AP1 and AP3 ) are relatively high (up to 5.0), as well as at those regions R2,1 and R2,2 between two anchors with a distance of l2 to each other (e.g., AP1 and AP2 ), where the maximum gopt is approximately 2.5. In the special case of a quadratic plain (l1 = l2 , i.e. p = 1), the distribution of gopt is symmetric to the center of the region. By selecting the determined optimal dynamic powers shown in Figure 2a, the mean error could be decreased by about 30 % (compared to WCL with an optimal static weight factor of gs = 0.87 in a region with p = 43 ) to the best possible value for WCL. Thus, it is obvious that there is huge potential for improvements by using dynamic weight factors. Figure 2b shows this potential which represents the difference between the error if gs is used, and the minimum possible error if

28

S. Schuhmann et al.

the optimal gopt is used. This figure clarifies that the potential for improvements is alike very unevenly distributed. Particularly the regions Ri,j (i, j ∈ {1, 2}) offer an enormous potential of more than 10 % of l1 for improvements, which corresponds to the noticeably increased optimal weight factor at these regions, as Figure 2a shows. It is obvious that the influence of the two farthest access points was too high there, as the value of gs was chosen too low. This shifted the calculated position near the centre of the region. The proceeding of DWCL to obtain the optimal dynamic weighting factor gd is the following: At first, the SOI has to identify the approximate subregion it is located in. Then, it has to lookup subregion-specific parameters. Now, the SOI can at first calculate the value of a balancing function and, based on this result, the approximated actual dynamic weight gd and its location within the whole region under observation. We will focus on these steps in the following. Subregion Determination. First of all, a SOI has to determine the approximate subregion in which it is located. This has to happen by solely regarding the RSSIs of incoming beacon signals. Figure 2a suggests to define four different subregions, as there exist four peaks for the optimal values of g. A simple way to divide the region in adequate subregions is using the perpendicular bisectors of the sides from AP1 to AP4 and from AP2 to AP3 , as illustrated in Figure 3. A SOI then can easily determine in which of the emerging subregions C1,1 , C1,2 , C2,1 , and C2,2 it is located. This is performed by simply comparing the RSSIs pairwise, as denoted in Table 1. Table 1. Determination of subregions RSSI constraints

Subregion

(RSSI(AP1 ) > RSSI(AP4 )) ∧ (RSSI(AP3 ) > RSSI(AP2 )) (RSSI(AP4 ) > RSSI(AP1 )) ∧ (RSSI(AP2 ) > RSSI(AP3 )) (RSSI(AP1 ) > RSSI(AP4 )) ∧ (RSSI(AP2 ) > RSSI(AP3 )) (RSSI(AP4 ) > RSSI(AP1 )) ∧ (RSSI(AP3 ) > RSSI(AP2 ))

C1,1 C1,2 C2,1 C2,2

Determination of Subregion-Specific Parameters. After determining the subregion it is located in, a SOI needs to obtain further parameters which we will derive in this paragraph. Regarding the beacon signals received within Ri,j , the RSSIs have several special characteristics there: – The RSSIs of the two strongest received signals are almost equal, as the distances from the SOI to the corresponding access points, respectively, are almost the same. The same holds for the two weakest signals. – At those points in Ri,j where gopt is maximal, the quotient Qi (i ∈ {1, 2}) of the distance to the far APs by the distance to the close APs can be calculated by Pythagoras’ formula, as depicted in Figure 3.   l22 + ( 12 l1 )2 l12 + ( 12 l2 )2 x1 x2 = ; Q = = (9) Q1 = 2 1 1 y1 y2 2 l1 2 l2

Improved Weighted Centroid Localization





























    
































29












   

Fig. 3. Calculation of parameters Q1 and Q2











 

 





  


    







 

     

   



 

 






































Fig. 4. Typical values in common rooms a) for D1 and D2 , b) for a1 , a2 , b1 , and b2

According to Equation 6, the RSSI differences between the received signals from the corresponding APs are Di = 20 · log10 Qi , i ∈ {1, 2}.

(10)

Thus, a SOI only needs to know l1 and l2 to achieve the parameters Qi and Di . The values of l1 and l2 can easily be obtained by the fixed AP positions. The SOI has to use D1 if it is situated in region C1,1 or C1,2 . Otherwise, it has to choose D2 , as it is situated in region C2,1 or C2,2 then. Common values for Di are shown in Figure 4a. For example, in a region with p = 43 , the calculation results for Di are D1 = 9.0908 dB and D2 = 5.1188 dB. The differences between the received RSSIs can generally be expressed as follows: Δk = RSSIk − RSSIk+1 , k ∈ {1, 2, 3} where RSSIk represents the k-th strongest received RSSI. Within regions Ri,j , the values of Δ1 and Δ3 are typically very low (close to 0), while those of Δ2 are close to D1 (in R1,j ) or D2 (in R2,j ).

30

S. Schuhmann et al.

Fig. 5. Distribution of calculated gi (S) in theory (p = 1, i.e. g1 (S) = g2 (S))

Balancing Function. To differ if a SOI is located in Ri,j where gopt is considerably high, we decided to define the following balancing function S that is based on the special RSSI characteristics in Ri,j that were mentioned above: S = Δ1 + |Di − Δ2 | + Δ3 , i ∈ {1, 2}

(11)

As already mentioned, Δ1 and Δ3 are close to zero within Ri,j , while Δ2 depends on the parameter p and is around Di in Ri,j . Thus, (Di − Δ2 ) is around zero in Ri,j . Following, S is minimum within Ri,j and increases outside of these regions. Hence, the use of S helps to state more precisely the location of the SOI. Approximation. We calculated S for various positions in the whole region, and compared the values with those of the optimal gopt at this position. The distribution of S dependent on g is shown in Figure 5 and suggests to approximate gd by the power function that is denoted in (12). ai (12) gd = gi (S) = bi , i ∈ {1, 2} S with the unknown parameters a1 and b1 if the SOI is within C1,1 or C1,2 , and with a2 and b2 if the SOI is located in C2,1 or C2,2 . Now, we approximated the power functions with the least-squares method for various values of p and got the parameters ai and bi as denoted in Figure 4 that yield minimum root mean square (RMS) errors. For example, in case of p = 1, we got a1 = a2 ≈ 1.580 and b1 = b2 ≈ 0.248 which led to a very small RMS error of only 0.002 which is a small fraction of the RMS error of SWCL with optimal static weight.

5

Evaluation

First, we present the theoretical results of DWCL and compare them with those of SWCL. Then, we concentrate on the indoor tests with 802.11 WLAN access

Improved Weighted Centroid Localization

31

Fig. 6. Evaluation results of theoretical calculations 3·108 m

points, operating at a wavelength of λ = fc = 2.44·109sHz ≈ 0.123 m, and compare our practical results with the calculated ones. 5.1

Theoretical Results

We compared the calculated location accuracies of DWCL and SWCL with the results we got by choosing the optimal gopt at each position within the observed region. The results are shown in Figure 6. The mean error of SWCL at a quadratic room (p = 1) is 7.6 %, while the mean error of DWCL is 5.3 % and, hence, very close to the best possible WCL value of 5.1 %. Regarding the maximum error within the region, DWCL and the WCL optimum are almost equal, while SWCL’s accuracy is worse by 2.5 percentage points. The results for a value of p = 2 are quite similar, even though DWCL’s mean gain decreases a bit. 5.2

Indoor WLAN Tests

We implemented the WCL scheme on a common smart phone1 which served as a client device. The infrastructure beacons were represented by standard 802.11 access points2 . The transmission power was put to the maximum possible value of 50 mW with a fixed bandwidth of 11 Mbit/s at 802.11b mode. According to [20], this provides a maximum range of about 67 meters in indoor scenarios. Figure 7a shows the setting for our indoor tests. We used four beacons, A1 to A4 , which were placed in the corners of a quadratic plain of 4.8 m x 4.8 m (i.e., p = 1) in a first test, and a rectangular plain of 5.8 m x 3.2 m (i.e., p ≈ 1.81) in a second test. The tests have been performed in a wall-enclosed office within a 1 2

T-MobileTM MDA Pro with PXA 270 CPU (520 MHz), 48 MB RAM, IEEE 802.11b. CiscoTM Aironet 1200, [20].

32

S. Schuhmann et al.

Fig. 7. a) Indoor test setup, b) Distribution of calculated gd at indoor tests (p = 1)

building where interferences with other WLANs occured. Furthermore, typical office furniture like chairs, desks, and a flip chart were available in the room. The SOI and the APs were placed on a plain table. We deactivated rate shifting on the APs to guarantee fixed transmission powers and performed 20 localization processes at various positions within the region, respectively. Then, we compared the averaged calculated positions with the real ones to determine the location errors. Figure 7b displays the values of the balance function S at the test positions and the approximated power function for p = 1. The mean square error (0.330) is

Fig. 8. Evaluation results of indoor tests

Improved Weighted Centroid Localization

33

higher than in theory as the signals suffered from noise and reflections. However, this error is still about 33.7 % lower than that of SWCL with gs = 0.93. Figure 8 presents a comparison of the mean errors of DWCL, SWCL, and calculations with gopt in these tests. It states that for p = 1, DWCL performs about 0.7 percentage points worse than the theoretical WCL optimum, but at the same time about 2.3 percentage points better than SWCL regarding the mean error, which implies it uses most of the existing potential for improvements. Concerning worst cases, DWCL’s difference to the optimum in accuracy is close to zero, while SWCL performs about 4.2 percentage points worse than the optimum. These results confirm the increased location accuracy by dynamic weighting factors, particularly regarding worst case accuracies in rooms that are almost quadratic where DWCL chooses nearly optimal values for gd .

6

Conclusion and Outlook

In this paper, we proposed an improved Weighted Centroid Localization scheme called DWCL in combination with 802.11 access points as a simple, cost-efficient, applicable, and accurate localization scheme for ubiquitous computing scenarios. We introduced the problem domain, discussed the requirements, and presented the original WCL scheme with static weight factors (SWCL). Following, we developed an improved WCL scheme called DWCL that uses dynamic weighting factors which depend on the RSSIs to increase location accuracy. From our experiments, we conclude that DWCL represents a very effective and yet simple method for indoor localization, as it implies almost optimal location accuracy for weighted centroid schemes with rectangular anchor positioning. It does not pose additional requirements to the clients as well as to the infrastructure beacons. Standard WLAN access points suffice for achieving acceptable localization errors, and the computational burden put on mobile client devices is mild. Due to the fact that WLAN-based WCL can be used in environments with off-the-shelf WLAN hardware, it is appropriate for achieving ad hoc localization in many ubiquitous smart environments with minimal prior setup efforts. The research group at the University of Stuttgart currently investigates if this improved WCL scheme is applicable for localization across several rooms with larger distances between the access points, and alternative positionings of the access points. The research team at the University of Rostock concentrates on the reduction of the positioning error that is mainly caused by border effects. Their current focus is the adaptation of weight functions according to the number of known beacons and the adjusted transmission range of the beacons.

Acknowledgement This work is funded by the German Research Foundation (DFG) within Priority Programme 1140 - Middleware for Self-organizing Infrastructures in Networked Mobile Systems.

34

S. Schuhmann et al.

References 1. Blumenthal, J., Reichenbach, F., Timmermann, D.: Position Estimation in Ad hoc Wireless Sensor Networks with Low Complexity. In: Joint 2nd Workshop on Positioning, Navigation and Communication and 1st Ultra-Wideband Expert Talk, pp. 41–49 (March 2005) 2. Fekete, S.P., Kr¨ oller, A., Buschmann, C., Fischer, S., Pfisterer, D.: Koordinatenfreies Lokationsbewusstsein. it - Information Technology 47(2), 70–78 (2005) 3. Pandey, S., Anjum, F., Kim, B., Agrawal, P.: A low-cost robust localization scheme for WLAN. In: Proceedings of the 2nd Int.’l Workshop on Wireless Internet (2006) 4. Bulusu, N., Heidemann, J., Estrin, D.: GPS-less Low Cost Outdoor Localization For Very Small Devices. IEEE Personal Communications Magazine 7(5), 28–34 (2000) 5. Niculescu, D., Nath, B.: DV Based Positioning in Ad hoc Networks. Journal of Telecommunication Systems (2003) 6. Nagpal, R.: Organizing a global coordinate system from local information on an amorphous computer. Technical Report AI Memo No. 1666, MIT A.I. Laboratory (1999) 7. He, T., et al.: Range-free localization and its impact on large scale sensor networks. Transactions on Embedded Computing Systems 4(4), 877–906 (2005) 8. Elnahrawy, E., Li, X., Martin, R.P.: Using Area-Based Presentations and Metrics for Localization Systems in Wireless LANs. In: Proceedings of the 29th IEEE International Conference on Local Computer Networks (LCN 2004), Washington, USA (2004) 9. Parkinson, B.W., Spilker Jr., J.J.: Global Positioning System: Theory and Applications, Vol. 1. In: Progress in Astronautics and Aeronautics, American Institute of Aeronautics and Astronautics, vol. 163, pp. 29–56 (1996) 10. Harter, A., Hopper, A., Steggles, P., Ward, A., Webster, P.: The Anatomy of a Context-Aware Application. Mobile Computing and Networking, 59–68 (1999) 11. Niculescu, D., Nath, B.: Ad hoc positioning system (APS). In: Proceedings of GLOBECOM, San Antonio (November 2001) 12. Bahl, P., Padmanabhan, V.N.: RADAR: An In-Building RF-Based User Location and Tracking System. In: INFOCOM, vol. (2), pp. 775–784 (2000) 13. Youssef, M., Agrawala, A., Shankar, U.: WLAN Location Determination via Clustering and Probability Distributions. In: Proceedings of IEEE PerCom 2003 (March 2003) 14. Ganesan, D., Krishnamachari, B., Woo, A., Culler, D., Estrin, D., Wicker, S.: Complex Behavior at Scale: An Experimental Study of Low-Power Wireless Sensor Networks. Technical Report CSD-TR 02-0013, UCLA (February 2002) 15. Want, R., Hopper, A., Falc˜ ao, V., Gibbons, J.: The Active Badge Location System. ACM Transactions on Information Systems (TOIS) 10(1) (1992) 16. Paperno, E., Sasada, I., Leonovich, E.: A New Method for Magnetic Position and Orientation Tracking. IEEE Transactions on Magnetics 37(4) (July 2001) 17. Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M., Shafer, S.: Multi-Camera Multi-Person Tracking for EasyLiving. In: Proceedings of the 3rd IEEE International Workshop on Visual Surveillance (VS 2000), Washington, USA (2000) 18. Ganu, S., Krishnakumar, A., Krishnan, P.: Infrastructure-based location estimation in WLAN networks. In: Proceedings of IEEE WCNC 2004 (March 2004) 19. Blumenthal, J., Grossmann, R., Golatowski, F., Timmermann, D.: Weighted Centroid Localization in WLAN-based Sensor Networks. In: Proceedings of the 2007 IEEE International Symposium on Intelligent Signal Processing (WISP 2007) (2007) 20. Cisco Systems, Aironet 1200 Series Access Point Data Sheet, http://www.cisco.com/en/US/products/hw/wireless/ps430

A Formal Framework for Expressing Trust Negotiation in the Ubiquitous Computing Environment* Deqing Zou1, Jong Hyuk Park2, Laurence Tianruo Yang 3, Zhensong Liao1, and Tai-hoon Kim4 1

Services Computing Technology and System Lab, Cluster and Grid Computing Lab, School of Computer Science and Technology, Huazhong University of Science and Technology, China 2 Department of Computer Science and Engineering, Kyungnam University, Korea 3 St. Francis Xavier University, Canada 4 Division of Multimedia, Hannam University, Korea

Abstract. There are lots of entities in the ubiquitous computing environment. For the traditional public key Infrastructure (PKI), every entity should be signed a valid certificate by the certificate authentication center. However, it’s hard to construct a centralized trust management framework and assign a valid certificate for every entity in the ubiquitous computing environment because of large numbers of dynamic entities. Trust negotiation (TN) is an important means to establish trust between strangers in ubiquitous computing systems through the exchange of digital credentials and mobile access control policies specifying what combinations of credentials a stranger must submit. Current existing TN technologies, such as TrustBuilder and KeyNote, focused on how to solve a certain problem by using some special techniques. In this paper, we present a formal framework for expressing trust negotiation. The framework specifies the basic concepts, elements and the semantics of TN. By analyzing TN, we point out how to build a TN system in practice.

1 Introduction With the comprehensive popularity of computer knowledge and the great development of computer network, people have more and more requirements towards resource sharing over Internet. Common access control mechanisms, such as MAC [1], RBAC [2], could not work well because of their limitation in design and application. In the ubiquitous computing environment, it’s hard to construct a centralized trust management framework for large numbers of dynamic entities. Exchange of attribute credentials is a means to establish mutual trust relationship between strangers in the ubiquitous computing environment, who wish to share resources or conduct business transactions [3][4][5]. As a new access control mechanism, TN successfully provides a method to share resources for strangers from different security domains in the ubiquitous computing environment. *

The project is supported by National Natural Science Foundation of China under Grant No. 60503040.

F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 35–45, 2008. © Springer-Verlag Berlin Heidelberg 2008

36

D. Zou et al.

In TN, automated trust negotiation (ATN), as a new kind of TN, has a great improvement towards previous TN. Generally, ATN differs from TN as the following aspects: 1) ATN discloses the access control policy when the resource can be available. While in TN, the requestor would not know what credentials he must submit before the negotiation starts. From this, we can see that ATN saves much time, which helps to improve negotiation efficiency. 2) ATN has done much work in privacy protection. In TN, the requestor is required to submit his certificate to attain the access. In ATN, the trust relationship is established by the exchange of credentials and access control policies. The disclosure of a certificate may pose a threat towards the user private privacy, while the disclosure of a credential would not do. 3) ATN pays much attention on information protection, while TN does not. In ATN, the transmitting messages are encrypted, while in most TN systems, the exchanged data are plaintexts. Currently, there are still some problems unsolved in TN. First, there are no unified secure definitions for TN’s concepts. For example, what is a credential? What information a credential must include? Which credential is secure? Second, TN is difficult to be formalized by a certain language, which leads to a low security level. Third, since most information protection techniques are immature or heavyweight, how to protect privacy and sensitive information is still a serious issue. Finally, there is no guideline to instruct how to build a TN system. In the paper, we aim at expressing TN theoretically, specifying the internal relationship between TN’s concepts or components. Generally, our work has the following contributions: 1) A common framework of general-purpose for TN is proposed in this paper. Previous work focused on how to solve a certain problem, such as giving a model, using some special techniques and so on. Our work aims at the unification of all kinds of models and techniques. 2) We propose a lot of useful suggestions on how to build a TN system. Usually, before developing a TN system, some preparations should be made. For example, which policy language will be selected? How should we improve negotiation efficiency? Previous work did not deal with such problems. 3) We present a new thought to improve TN’s security level. According to the specification of TCSEC (Trusted Computer System Evaluation Criteria) [6], TN has a low security level. If TN is formalized by a certain policy language, TN’s security level would be improved or enhanced. The rest of this paper is organized as follows. Section 2 describes the related work, including TN project survey and information protection techniques. Section 3 is the main part of the paper. It describes the framework in detail. Section 4 purposes useful suggestions on how to build/develop a TN system. Section 5 concludes the paper and specifies the future work.

A Formal Framework for Expressing Trust Negotiation

37

2 Related Work Our work is originally motivated from the existing automated trust negotiation research [3][4][5] whose goal is to enable trust establishment between strangers in a decentralized or open environment, such as Internet. Our work focuses on analyzing TN, including the concepts, the components and the relationship between the concepts. To make it clear, existing TN projects and systems should be collected. Meanwhile, how to protect sensitive information or personal privacy is still a hot issue. Based on these, in this part, we mainly investigate TN projects/systems and information protection. In the existing TN systems, TrustBuilder [7][8], KeyNote [7][9][10] are much well-known. Some similar systems include PolicyMaker [11], SD3 [12], PRUNES [13] and Trust-X [14]. Here, we mainly discuss TrustBuilder and KeyNote. TrustBuilder represents one of the most signification proposals in the negotiation research area. It provides a broad class of negotiation strategies, as well as a strategyand language-independent negotiation protocol that ensures the interoperability of the defined strategies within the its architecture. Each participant in a negotiation has an associated security agent that manages the negotiation. During a negotiation, the security agent uses a local negotiation strategy to determine which local resources to disclose next and to accept new disclosures from other parties. In KeyNote, the predicates are written in a simple notation based on C-like expressions and regular expressions, and the assertions always return a Boolean (accepted or denied) answer. Moreover, the credential signature verification is built in to KeyNote system. Assertion syntax is based on a human-readable “RFC-822”-style syntax. And the trusted actions in KeyNote are described by simple attribute/value pairs. As far as the information protection is concerned, some typical and influencing techniques are involved as the following. W.H. Winsborough and N. Li [4] presented ACK policy to control the disclosure of credentials and policies, and developed TTG protocol to construct policy tree so that it was easy to detect whether the credentials matched the policies or not. ACK policy is useful to protect sensitive information, and TTG protocol enables two parties to do joint chain discovery in an interactive manner as well as the parties to use policies to protect sensitive credentials and the attribute information contained in the credentials. However, ACK policy and TTG are limited in applying because of difficulty in constructing them. N. Li et al [15] proposed OSBE protocol to prevent important information from leaking and void the negotiation being attacked. OSBE bases its idea on digital signature and checks the message’s integrity so as to detect whether the negotiator has the right signature. OSBE protects message from unauthorized access, but it is heavyweight to build and the signature computing has a great cost. Holt et al introduced hidden credentials in [16]. They gave a formal description for hidden credentials, including the concepts of credential and policy indistinguishability, and showed how to build them using IBE. Their work also gave some compelling examples of the utility of hidden credentials. In short, they provided a good model for trust negotiation to implement hidden credentials. Based on this, Robert W et al [17] used hidden credentials to conceal complex policies, and Keith F et al [18] used them

38

D. Zou et al.

to hide access control policies. Since hidden credential system cannot prevent from invalid inference, its implementation is limited in some degree. J. Li and N. Li proposed the notions of OACerts and OCBE in [19]. OCBE protocol adopted the idea of zero-knowledge and ensured that if and only if the recipient was the specified one, he could get the resource, otherwise he got nothing. However, the method has not discussed how to guarantee the security of message during the transmission over the insecure Internet.

3 Framework for Expressing Trust Negotiation In this section, we will describe the TN framework in detail. First, the basic concepts and elements are listed. Then, the framework is introduced. Finally, the corresponding expressions are given. 3.1 Basic Concepts and Elements TN is performed by the exchange of credentials and access control policy. In this part, we give the basic concepts of TN and put forward their corresponding formalized description as follows: • Access control policy (ACP): the constraint in accessing the resource, which indicates what credentials the accessor should prepare and submit. In TN, access control policy has two types: compound policy and meta policy. Meta policy is the minimal element to compose a compound policy. To realize it, a set of conjunctive predicates are required, such as , < ∧,∨,¬> and so on. • Access request (AR): a structural message to tell the resource provider that the resource is expected to be accessed. An access request should at least include: 1) accessor, 2) requested resource, 3) requested operation. • Certificate: to identify a person’s identity or attribute information, such as principal, public key. In previous TN, certificate is the main tool to carry a person’s information. Currently, as a person’s privacy is considered, certificate is hardly disclosed and transferred any more. • Credential: a kind of temporary certificate, whose content comes from the user’s certificate. In other words, a certificate contains many credentials. When the credential is introduced in TN, user’s certificate would not be released. • Negotiation strategy (NS): a mechanism to specify how to disclose access control policy and credentials. There are two extreme negotiation strategies: eager strategy and parsimonious strategy. The former discloses zealously without being requested, while the latter would not release any until a request is received. • Negotiation protocol (NP): a rule to specify what actions would be taken during the negotiation process, including authentication, delegation, authorization communication mode (TCP/UDP), encryption mode (symmetrical or asymmetrical), and encryption function and so on. For example, is transmitting message transferred via plaintext or ciphertext? How to select the encryption functions?

A Formal Framework for Expressing Trust Negotiation

39

Accordingly, the basic elements of TN are as follows: • ACP: access control policy set. ACP= {acp1,acp2,…,acpn}, acpi (1≤i≤n) is a meta policy. Usually, an access control policy ACP can be depicted as ACP=f(acp1,acp2,…,acpn), while f is compound function by using the specified conjunctive predicates. • AR: access request set. AR={ar1,ar2,…,arn}, ari (1≤i≤n) is a specific access request towards a certain resource. • C: credential set, C={c1,c2,…,cn}, ci (1≤i≤n) is a credential. Every credential carries only one kind of attribute information. • R: resource which can be accessed. In TN, R stands for the available resource. It can be the resource itself, or the access entry. When R is disclosed, the trust relationship is established, and the negotiation process ends up. • SEQ: disclosure sequence set, which is used to store the disclosed access requests, access control policies and credentials. SEQ=SEQAR∪SEQACP∪ SEQC∪R, while SEQAR (SEQACP, SEQC) denotes the disclosed access request (resp. access control policy, credential) sequence. When R∈SEQ, the negotiation succeeds. • NS: negotiation strategy set. NS={ns1,ns2,…,nsn}, nsi (1≤i≤n) is the selected negotiation strategy. In practice, new negotiation strategies may be developed according to the application. • NP: negotiation protocol set. NP={np1, np2,…,npn}, npi (1≤i≤n) is a certain negotiation protocol to specify how to exchange information during the negotiation process.

Fig. 1. The relationship between AR and ACP

3.2 Framework for Expressing Trust Negotiation In our framework, TN can be described as follows: TNparameter=

(1)

In (1), TNparameter denotes the parameter to deploy a TN system. When TNparameter is confirmed, the work mode of TN is established. ATN/TN stands for the negotiation

40

D. Zou et al.

type. When ATN is selected, it is an ATN system. AR (ACP, C, NS, NP) is the corresponding access request (resp. access control policy, credential, negotiation strategy, negotiation protocol) set. There are many internal relationships and constraints among TN’s parameters. 1) The relationship between AR and ACP In TN, when the resource provider receives an access request ar, he will analyze it and check whether it is valid and effect. If ar passes the examination, the resource provider will return a relative access control policy set Policy (including one or more meta policies) as a response, otherwise, ar is denied. Generally, every request should have a corresponding policy set, i.e., ∀ar∈AR,∃Policy⊆ACP⇒Correlate(ar,Policy), in which the function Correlate() is a binary logic predicate. Meanwhile, it may happen multiple requests correspond to a same policy set. This is caused by the access granularity. In short, the relationship between AR and ACP can be shown as Fig. 1.

Fig. 2. The relationship between ACP and C

2) The relationship between ACP and C In TN, credentials are submitted to meet the requirements of a compound policy. To avoid the different meanings, each credential can only satisfy a meta policy. That is to say, ∀acp∈ACP, ∃c∈C⇒Match(acp,c) and ∀c∈C,∃acp∈ACP⇒ Match(acp,c), in which the function Match() is also a binary logic predicate. The relationship between ACP and C can be shown by Fig. 2. 3) The relationship between NS and NP In TN, negotiation strategy decides the disclosure mode. Different negotiation strategy has different requirements towards system, network, user, resource provider and negotiation protocol. For example, if the parsimonious strategy is adopted, i.e., the negotiation needs a tight coupling communication, which causes a great network burden, and has a strict requirement towards the negotiation protocol. Accordingly, negotiation protocol decides the negotiation process. Usually, negotiation protocol and negotiation strategy should cooperate with each other; otherwise, the negotiation is easy to fail. The relationship between NS and NP is shown as Fig. 3.

A Formal Framework for Expressing Trust Negotiation

41

Fig. 3. The relationship between NS and NP

3.3 Trust Negotiation Expression From above, we can see that the state of TN can be depicted as: STN = SX + SY

(2)

SX = NS × NP

(3)

SY = AR × ACP × C

(4)

SX denotes a relatively static state of TN. When the negotiation starts, NS and NP would be ascertained. When SX is fixed, the disclosure sequence SEQ would be somehow clear. In short, SX decides the qualitative change of SEQ. SY stands for a relatively dynamic state of TN. SY records the total negotiation process, which can be used to track both negotiators’ actions. Meanwhile, it is the important information to compose the log file. If the negotiation fails, SY can be useful to produce feedback. In short, SY decides the quantitative change of SEQ. As negotiation protocol is taken into account, NP can be depicted as: NP=

(5)

In (5), A1 is authentication. In TN, authentication is an important means to make certain a user’s identity. Usually, authenticating process is to ensure that the acceesor holds the private key of his certificate. Suppose p and l to be a user’s principal (also called public key) and license (also called private key). The authenticating process can be: 1) the server randomly picks a string str, and encrypts it with the user’s principal p and passes it to the user; 2) the user decrypts it and returns it through a secure VPN (virtual private network) channel; 3) the server compares the original string with the received string, and decides the authentication result. Since it would take a great cost to set up a VPN channel, authentication can be realized by mutual encryption via insecure channel such as Internet without VPN. Based on this, we present a mutual authentication shown as Fig. 4.

42

D. Zou et al.

Fig. 4. Mutual authentication for TN

In short, authentication can be depicted as: A1= Principal × License

(6)

In (5), D is delegation. In TN, delegation is to endow others with one’s privileges. To finish a delegation, two users (suppose A and B) should do: 1) the authentication of A and B; 2) the delegation of B towards A, to let A own B’s privileges. Based on these, the delegation can be depicted as: DBA = A1A × A1B × PrivilegeB

(7)

In (5), A2 is authorization. In TN, authorization is to let the accessor have the specified privileges to operate on resource. In short, authorization can be depicted as: A2= Principal × License × PrivilegeR

(8)

In (5), PL/CI denotes the message encapsulation means. PL is plaintext, while CI is ciphertext. TCP/UDP denotes the communication means. TCP stands for tight coupling communication. When NS is parsimonious strategy, TCP must be selected. UDP stands for loose coupling communication. S/A denotes the encryption means. S stands for symmetric encryption, while A stands for asymmetric encryption. When S is selected, the relative algorithms could be AES [20], Rijndael [21] and so on. Corresponding, when A is selected, the relative algorithms could be RSA [22], ECC [23] etc.

4 Guideline for Building a TN System The research in theory is useful to guide teams to build practical systems. Before building a TN system, some questions should be answered.

A Formal Framework for Expressing Trust Negotiation

43

1) Application type The team should make clear what services the TN system can provide. How about the security requirements? Would the online service be high available or have high computing power? How to ensure QoS? When these problems are clear, the team can make good preparations for the development. 2) Policy language Not every language can serve for TN. TN has a lot of requirements [3][7] towards policy languages. The selection of policy language should be adapted to application type. 3) Specification of AR, ACP and C To attain the goal of high efficiency and least artificial participation, TN should be much intelligent as possible. To realize it, TN should improve the readability of access request, access control policy and credential. So before developing TN, the formats of AR, ACP and C should be ascertained. This helps to perform all kinds of operations. An example for the formats of AR, ACP and C can be: AR=(owner):(resource):(operation): (period)

(9)

ACP=(owner):(recipient):(item):(predicate): (value):(period)

(10)

C=(owner):(recipient):(item):(value):(period)

(11)

In (9), owner denotes the holder of AR, usually the principal of the user; resource denotes requested resource or service; operation denotes the requested operation towards the resource; while period denotes the valid life time of AR. In (10), owner denotes the holder of ACP; recipient denotes the receiver of ACP; item denotes the attribute content, such as role, age, name etc; predicate denotes the comparing symbol, predicate∈ {>,

1

See http://www.cs.kuleuven.be/∼ davy/ontologies/2008/01/ for the latest revision of our context ontologies.

Pervasive Services on the Move

51

The previous constraint declares that a device should have at least one memory resource instance (of class RAM or one of its subclasses) with a property currAvailable that has a value larger than or equal to 98304. The type of the property is specified in the Hardware.owl ontology, and for this value it is bytes.







The above constraint declares that a device should have at least one Java virtual machine instance with a GUI rendering engine instance that belongs to the JavaAWT class. The main advantage of our context ontologies [9] is that complex semantic relationships only need to be specified once and can be reused by every service descriptor. Moreover, each device profile can specify their characteristics with the same ontologies. If a device then would specify that it runs OSGi on top of JDK 1.6, then an AWT rendering engine dependency would semantically match, but a Java MIDP 2.0 LCDUI rendering engine would not. Resource and hardware constraints can be expressed in a similar way. The matching itself to verify that a service can move to a particular host is carried out by a context enabling service that is described in the following section. 3.2

Context-Awareness as an OSGi Distributed Declarative Service

In order to diffuse OSGi services in a way that takes into account the heterogeneity and dynamism of a ubiquitous computing environment, we need to be aware of what the characteristics and capabilities of these devices are, where they are located, what kind of services they offer and what resources are currently available. The COGITO service will provide this information on each host. It gathers and utilizes context information to positively affect the provisioning of services, such as the personalization and redeployment of services tailored to the customer’s needs. The core functions of the enabling service are provided as a set of OSGi bundles that can be subdivided into the following categories: – Context Acquisition: These bundles monitor for context that is changing and gather information from resource sensors, device profiles and other repositories within the system itself or on the network. – Context Storage: A context repository ensures persistence of context information. It collects relevant local information and context that remote entities have published in a way that processing can be handled efficiently without losing the semantics of the data.

52

D. Preuveneers and Y. Berbers




  
  
 




 
   

 

  

 
   


     






   

 

  

    

    







  

 
  







 






Fig. 2. Building blocks of the COGITO enabling service

– Context Manipulation: These bundles reason on the context information to verify that context constraints are met. Besides a rule and matching engine that exploits semantic relationships between concepts [11], it also provides adapters that transform context information into more suitable formats. A high-level overview of the building blocks of the context-awareness enabling service is given Figure 2. The advantage of decomposing the context-awareness framework into multiple OSGi bundles is that the modular approach saves resources when certain bundles are not required. COGITO is implemented as a distributed Declarative Service. As the Declarative Service specification does not cover the registration of remote services, we have each device announcing its presence and that of its context enabling service with a service discovery protocol like UPnP, SLP or ZeroConf. Upon joining a network, each other node in the network creates a remote proxy to the enabling service of the joining node, and the Declarative Services bundle will ensure lazy registration and activation of the proxy whenever remote context is acquired. When a node leaves the network, the proxies on the other nodes are deactivated. This approach for distributed Declarative Services simplifies the collection of context information on the network, but more importantly it also enables transparent sharing of intensive context processing services (such as a rule or inference engine bundle) on resource-constrained devices. 3.3

Service State Representation, Extraction and Synchronization

In order to replicate stateful services on multiple hosts with support for state synchronization, we need to explicitly model the state variables of the service, extract them at runtime and send them to the replicated peers. In order to keep

Pervasive Services on the Move

53

state representation and exchange straightforward and lightweight, we encapsulate the state of a service in a JavaBean. JavaBeans have the benefit of (1) being able to encapsulate several objects in one bean that can be passed around to other nodes, (2) being serializable and (3) providing get and set methods to automate the inspection and updating of the service state. The approach is quite similar to the stateful session beans in the Enterprise JavaBeans specification [12]. Stateful session beans maintain the internal state of web applications and ensure that during a session a client will invoke method calls against the same bean in the remote container. In our approach however, a JavaBean is situated locally within each replicated service and does not involve remote method calls. Instead, the contents of the JavaBean is synchronized with those of the replicated services. Our current implementation tries to synchronize as soon as the state of an application has been changed. When network failures arise, the state updates are put in a queue and another attempt is carried out later on. The queue holds the revision number of the last state that was successfully exchanged and this for the three replicated applications. If, however, two or more replicated services independently continue without state synchronization, a state transfer conflict will occur when the involved hosts connect again. In that case, a warning will be shown and the user can choose to treat the services as separate applications, or have one service push its state to the others. This approach is rather crude, but acceptable as long as we are mainly dealing with single-user applications.

    ! " #
$    !!
    




 

  




 

   $ 



" %

!!





 

Fig. 3. New service states in the life-cycle of a migrating or diffusing service. Some of the transitions of the original service are shown in green, the ones of the replicated service(s) in red.

54

3.4

D. Preuveneers and Y. Berbers

Extending the Life-Cycle Management to Relocate Services

In a ubiquitous computing environment where pervasive services are replicated, a user may switch from one replicated service to another. Therefore, we add extra information to the service state that helps to coordinate the synchronization of the state changes. For example, we add a revision number that is increased after each write operation on the bean and use Vector Clocks [13] among all peers to order the state updating events. Fig. 3 shows that an active service can not only be stopped, but that in another transition the state can be extracted from the service and synchronized, or that the service can continue to either migrate or replicate to a new host. We use a lease timeout algorithm to garbage collect stale replicated services in order to recycle resources. The timeouts are application dependent, but can be reconfigured by the user.

4

Experimental Evaluation

The usefulness and feasibility of dealing with context-aware service migration and diffusion will be illustrated with three applications: (1) a grocery list application, (2) a Jabber instant messaging client, and (3) a Sudoku game. The devices used in the experiment include two PDAs, a laptop and a desktop, all connected to a local WiFi network. This setup will simulate the real-life scenario in the next section. 4.1

Scenario

The scenario goes as follows: It is Saturday morning and mom and dad go shopping. Everybody at home uses the grocery list application to make up a shared grocery list. In the past, mom sometimes forgot her grocery list at home, but now she only needs to take along her smartphone. The application just diffuses along. Dad also has his PDA with him and runs the same replicated application. They decided to split up the work and go shopping separately. At the mall, each time mom or dad finds a product that they need, it is removed from the shared sticky note application. They get each others updates when the state of the application is synchronized. Mom is still checking out while dad is already at the car. As he is waiting, he has another go at the Sudoku puzzle that he was trying to solve on the home computer that morning. His daughter is online and starts a conversation. She wants to send him a picture, but the display of his PDA is too small. He takes his briefcase, turns on his laptop, the application migrates and he continues the conversation there. Unfortunately, he forgot to charge his battery so after a while his laptop gives up on him. “Back to the PDA but then without the picture”, dad decides. In the meantime, mom has arrived and dad tells her about the picture. She will have a look at it at home once the application synchronizes with the desktop computer.

Pervasive Services on the Move


 

 


55






  
 
  
 

Fig. 4. The grocery list is automatically adapted to fit the new screen size if needed. After redeployment the states of all the replicated grocery lists are synchronized.

4.2

Experiments

In the experiment all the applications are launched on the desktop PC. The Jabber instant messaging client and Sudoku puzzle diffuse to one handheld, while the grocery list application diffuses to both PDAs. The application states are continuously synchronized between all hosts. Later on, the instant messaging application migrates from one PDA to the laptop and back (while synchronizing with the desktop PC). The mobility of the grocery list is illustrated in Figure 4. Additionally, these three applications all make use of two extra general-purpose services: (i) a stateless audio-based notification service, and (ii) a stateful logging service. By only deploying them on the desktop computer and synchronizing them on the laptop, we enforce that these two services can only be reached from a PDA through a remote connection. The applications will invoke them when (1) the grocery list changes, (2) the online status of somebody changes or (3) when the puzzle is solved. The bindings to the remote services are used to test handover to a replicated service after network disruptions. We let the setup run for 30 minutes, in which the following actions are simulated: – Each 30 seconds the grocery list is modified and synchronized. This event also triggers an invocation to both the remote services. – Each 5 seconds the state of the Jabber client is synchronized. Each minute the online status of somebody in the contact list changes, and this triggers an invocation to the remote services. – The Sudoku puzzle changes each 15 seconds and the 81 digits of the puzzle are synchronized. A puzzle is solved after 10 minutes, and this initiates an invocation to the remote services.

56

D. Preuveneers and Y. Berbers Table 1. Overhead results of state transfer, synchronization and handover Grocery List

Jabber Client

Sudoku Puzzle

Bundle size

23023 bytes

288235 bytes

54326 bytes

State size

7024 bytes

18235 bytes

1856 bytes

State synchronization

53 Kbytes / min 112 Kbytes / min 37 Kbytes / min

Relocation time

9 seconds

13 seconds

7 seconds

Synchronization delay

350 msec

518 msec

152 msec

61 msec

54 msec

213 msec

78 msec

Handover delay (audio service) 47 msec Handover delay (log service)

167 msec

Each 3 minutes we shut down a random network connection for 1 minute to verify if the replicated applications can recover from the missed state updates and that handover to one of the remote replicated services. We measured the overhead of state transfer and synchronization and compare that to the resources that the applications required. We also measured the service availability and responsiveness by measuring the delay of handing over to another remote replicated service after a network disconnection. The experiment was repeated 3 times and the averaged results are shown in Table 1. 4.3

Discussion of the Results

The test results show there is only a small overhead for the state transfer. This is mainly a consequence of the size of the applications. They are rather small as they would otherwise not run on mobile devices. For other data intensive applications, such as multimedia applications, we expect that the overhead will be a lot bigger if data files have to be moved along as part of the state of the service. As the overhead was limited, we did not implement incremental state transfers, but for media applications this could be an improvement. The delays in state synchronization are low because the experiments were carried out on a local WiFi network. Other tests in multi-hop networks showed longer but still acceptable state synchronization delays (at most 5 times longer). More interesting though is the difference in service relocation time and state exchange time. By proactively moving the service in advance, the time to get an up-to-date replica is a lot smaller for state synchronization compared to service relocation. As such, service diffusion with state synchronization provides a much better usability to the user compared to plain service migration. Of course, this assumes the required bandwidth is available. The handover to the remote services worked in all cases because we avoided the case where all network connections to the remote replicated services are disrupted. In theory, our framework can handle this particular case if the remote service calls can be buffered and invoked asynchronously. However, our current

Pervasive Services on the Move

57

implementation does not support this. Interesting to note for the handover to the replicated services (in our experiment, the stateless audio service and stateful log service), is that handover to another log service takes a bit longer on average. This result is due to the fact that for the audio service we do not need to wait until the replicated service has acquired the latest revision of the service state. If state synchronization is taking place during handover, the handover delay can become higher. In our experiment, the remote services were only available on two hosts (the laptop and the desktop). If more devices would host a replicated service, the decision to handover to a particular device could depend on which replica is already completely synchronized.

5

Related Work

In recent years, many researchers have addressed the issue of state transfer, code mobility and service oriented computing in ubiquitous computing environments. This research has greatly improved our knowledge of how context-aware service oriented architectures can accommodate to new functionality requirements in changing environments. Providing a detailed review of the state of the art on all of these domains is beyond the scope of this paper. Instead, we focus on those contributions that are most related to the work presented in this paper. In [14], the OSGi platform was used to perform context discovery, acquisition, and interpretation and use an OWL ontology-based context model that leverages Semantic Web technology. This paper has inspired us to use the OSGi platform for context-aware service orientation, and more specifically service mobility. In [15], the authors present an OSGi infrastructure for context-aware multimedia services in a smart home environment. Although the application domain that we target goes beyond multimedia and smart home environments, we envision that more data driven services like multimedia applications are good candidates to further evaluate our service mobility strategies. Other context-awareness systems leveraging the OSGi framework include [16,17]. Optimistic replication is a technique to enable a higher availability and performance in distributed data sharing systems. The advantage of optimistic replication is that it allows replicas to diverge. Although users can observe this divergence, the copies will eventually converge during a reconciliation phase. In [18], the authors provide a comprehensive survey of techniques developed to address divergence. Our current service replication and state synchronization is more related to traditional pessimistic replication, because user perceived divergence might cause confusion. As discussed in [19], we will further investigate how optimistic replication can improve the quality of service in mobile computing, while keeping the user perceived divergence at an acceptable level. Remote service invocation on the OSGi framework is an issue that has been addressed by several researchers and projects. The Eclipse Communication Framework [20] provides a flexible approach to deal with remote services and service discovery on top of the Equinox OSGi implementation. Remote OSGi

58

D. Preuveneers and Y. Berbers

services can be called synchronously and asynchronously on top of various communication protocols. The R-OSGI [21] framework also deals with service distribution. While both projects address network transparency for OSGi services, neither of them deals with explicit service mobility. The Fluid Computing [22] and flowSGI [23] projects explicitly deal with state transfer and synchronization in a similar way to ours. They also discuss state synchronization and provide a more advanced state conflict resolution mechanism. Our contribution builds upon this approach and specifically focuses on service oriented applications. Our method also enhances the peer selection for service replication by using context information of the service and the device. This approach provides better guarantees that the application will actually work on the device.

6

Conclusions and Future Work

This paper presents a context-driven approach to live service mobility in pervasive computing environments. It focuses on service migration and diffusion to multiple hosts to increase accessibility and expedite human interaction with the service. We summarized the basic requirements for service mobility, including enhanced service descriptions, context-awareness to select appropriate targets for service migration, and life-cycle management support to carry out service relocation with state transfer and state synchronization. We have discussed how we implemented these requirements on top of the OSGi framework and validated our prototype by means of several applications that are replicated in a small scale network with enforced network failures. We studied the effects of service migration and service diffusion. Service migration moves an application from one host to another including its state, while service diffusion replicates the service and the state on multiple hosts. State synchronization ensures that each service can be used interchangeably. Experiments have shown that the overhead of state transfer and synchronization is limited for relatively small applications. The results illustrate that, if the available network bandwidth permits, the time to keep the state in sync is a lot smaller than to migrate a service from one host to another. This means that if a user wants to use another device because it provides a better quality of service (e.g. the bigger screen in the scenario), that proactive service diffusion provides a much better usability (i.e. shorter delays). Future work will focus on a better handling of state synchronization conflicts and improving the state encapsulation approach to deal with incremental state transfers for data intensive applications. However, our main concern will always be to keep the supporting infrastructure flexible and lightweight enough for lowend devices with room left to actually run applications. The outcome of this future work could result in a better policy with respect to on how many remote devices a service should be replicated and under which circumstances service migration would be a better approach compared to service diffusion.

Pervasive Services on the Move

59

References 1. Weiser, M.: The computer for the 21st century. Scientific American 265, 94–104 (1991) 2. Papazoglou, M.P., Georgakopoulos, D.: Service oriented computing. Commun. ACM 46, 24–28 (2003) 3. Shaw, M., Garlan, D.: Software architecture: perspectives on an emerging discipline. Prentice-Hall, Inc., Upper Saddle River (1996) 4. Dey, A.K.: Understanding and using context. Personal Ubiquitous Comput. 5, 4–7 (2001) 5. Open Services Gateway Initiative: OSGi Service Gateway Specification, Release 4.1 (2007) 6. Helal, S.: Standards for service discovery and delivery. Pervasive Computing, IEEE 1, 95–100 (2002) 7. Kramer, J., Magee, J.: The evolving philosophers problem: Dynamic change management. IEEE Trans. Softw. Eng. 16, 1293–1306 (1990) 8. Vandewoude, Y., Ebraert, P., Berbers, Y., D’Hondt, T.: An alternative to quiescence: Tranquility. In: ICSM 2006: Proceedings of the 22nd IEEE International Conference on Software Maintenance, Washington, DC, USA, pp. 73–82. IEEE Computer Society, Los Alamitos (2006) 9. Preuveneers, D., den Bergh, J.V., Wagelaar, D., Georges, A., Rigole, P., Clerckx, T., Berbers, Y., Coninx, K., Jonckers, V., Bosschere, K.D.: Towards an extensible context ontology for Ambient Intelligence. In: Markopoulos, P., Eggen, B., Aarts, E., Crowley, J.L. (eds.) EUSAI 2004. LNCS, vol. 3295, pp. 148–159. Springer, Heidelberg (2004) 10. Mokhtar, S.B., Preuveneers, D., Georgantas, N., Issarny, V., Berbers, Y.: EASY: Efficient SemAntic Service DiscoverY in Pervasive Computing Environments with QoS and Context Support. Journal Of System and Software 81, 785–808 (2008) 11. Preuveneers, D., Berbers, Y.: Encoding semantic awareness in resource-constrained devices. IEEE Intelligent Systems 23, 26–33 (2008) 12. Burke, B., Monson-Haefel, R.: Enterprise JavaBeans 3.0, 5th edn. O’Reilly Media, Inc, Sebastopol (2006) 13. Mattern, F.: Virtual time and global states of distributed systems. In: Parallel and Distributed Algorithms: proceedings of the International Workshop on Parallel and Distributed Algorithms 14. Gu, T., Pung, H.K., Zhang, D.Q.: Toward an osgi-based infrastructure for contextaware applications. Pervasive Computing, IEEE 3, 66–74 (2004) 15. Yu, Z., Zhou, X., Yu, Z., Zhang, D., Chin, C.Y.: An osgi-based infrastructure for context-aware multimedia services. Communications Magazine, IEEE 44, 136–142 (2006) 16. Lee, H., Park, J., Ko, E., Lee, J.: An agent-based context-aware system on handheld computers, 229–230 (2006) 17. Kim, J.H., Yae, S.S., Ramakrishna, R.S.: Context-aware application framework based on open service gateway 5, 200–204 (2001) 18. Saito, Y., Shapiro, M.: Optimistic replication. ACM Comput. Surv. 37, 42–81 (2005) 19. Kuenning, G.H., Bagrodia, R., Guy, R.G., Popek, G.J., Reiher, P.L., Wang, A.I.: Measuring the quality of service of optimistic replication. In: ECOOP Workshops, pp. 319–320 (1998)

60

D. Preuveneers and Y. Berbers

20. The Eclipse Foundation: Eclipse Communication Framework (2007), http://www.eclipse.org/ecf/ 21. Rellermeyer, J.S., Alonso, G., Roscoe, T.: Building, deploying, and monitoring distributed applications with eclipse and r-osgi. In: Eclipse 2007: Proceedings of the 2007 OOPSLA workshop on eclipse technology eXchange, pp. 50–54. ACM, New York (2007) 22. Bourges-Waldegg, D., Duponchel, Y., Graf, M., Moser, M.: The fluid computing middleware: Bringing application fluidity to the mobile internet. In: SAINT 2005: Proceedings of the The 2005 Symposium on Applications and the Internet (SAINT 2005), Washington, DC, USA, pp. 54–63. IEEE Computer Society, Los Alamitos (2005) 23. Rellermeyer, J.S.: flowsgi: A framework for dynamic fluid applications. Master’s thesis, ETH Zurich (2006)

Robots in Smart Spaces - A Case Study of a u-Object Finder Prototype Tomomi Kawashima1, Jianhua Ma1, Bernady O. Apduhan2, Runhe Huang1, and Qun Jin3 1 Hosei University, Tokyo 184-8584, Japan [email protected], {jianhua, rhuang}@hosei.ac.jp 2 Kyushu Sangyo University, Fukuoka 813-8503, Japan [email protected] 3 Waseda University, Saitama 359-1192, Japan [email protected]

Abstract. A smart space is a physical spatial environment such as a room that can provide automatic responses according to the contextual information occurring in an environment. The context data is usually acquired via various devices which are installed somewhere in the environment and/or embedded into some special physical objects, called u-objects, which are capable of executing computation and/or communication. However, the devices used in current researches on smart space are either fixed or residing in real objects, which are incapable of moving by themselves. Likewise, more and more robots have been produced, but most of them are designed and developed based on the assumption that the space surrounding a robot is an ordinary one, i.e., a non-smart space. Therefore, it is necessary to study what additional services can be offered and what specific technical issues will be faced when adding robots to smart spaces. To identify the potential novel services and technology problems, this research is focused on a case study of a u-object finding service performed by a robot in a smart space. This paper presents the design and development of the system prototype robot which can communicate with other devices and can find a u-object with attached RFID tag in a smart room. Some preliminary experiments were conducted and the results verified the functionalities of the system.

1 Introduction The essential of ubiquitous computing (ubicomp), as emphasized by Mark Weiser, is “enhancing computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user” [1, 2]. The synergy of ubiquitous computing and communications is the natural result of two fundamental technology trends: (1) the continuing miniaturization of electronic chips and electro-mechanical devices following Moore’s law, and (2) their interconnection via networks especially using wireless communications. Aside from the widespread use and popularity of wireless technology such as IEEE802.11 a/b/g [3] and Bluetooth [4], the WPAN (Wireless Personal Area Network) [5, 6] and high-speed wireless F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 61–74, 2008. © Springer-Verlag Berlin Heidelberg 2008

62

T. Kawashima et al.

networks using UWB (Ultra Wide Band) [7] will soon be available and will play important roles in short range ubiquitous communications. Due to the continuing miniaturization and availability of high-speed communications, many devices can be distributed to surrounding ambient environments and make the ordinary environments capable of computing. That is, a real spatial environment such as a room may become smart to take automatic responses according to the contextual information [8, 9] occurring in this environment. The context data is usually acquired via various devices which are installed somewhere in the environment and/or embedded into some special physical objects. Real physical objects are called u-objects [10], as opposed to virtual e-things, if they are attached, embedded or blended with computers, networks, sensors, actors, IC-tags and/or other devices [11]. The smart space/environment, as one of the major ubicomp areas, has recently received much attention, and various smart space systems have been developed to make smart homes, smart offices, smart classrooms, etc. [12, 13]. However, the devices used in the current smart space researches are either fixed or residing in real objects that cannot move on their own. On the other hand, robotics is an exciting field that has been getting a great deal of attention for decades. Lots of robots have been produced and deployed to various applications such as in games, entertainment, household chores, rescue, military, and manufacturing [14]. The robots can also be seen as some special kinds of smart u-objects with partial human-like or animal-like behaviors. The most special characteristic of robots are that they can move by themselves according to the received instructions or perceptual information. However, most of them are designed and developed based on the assumption that the space surrounding a robot is an ordinary one, i.e., a non-smart space [15, 16]. It is then interesting and worth investigating to know what new services can be offered by smart spaces in which robots exist, and what new technical issues arise when combining smart spaces and robots to make smart systems. Flipcast [17] showed a design of an agent-based platform for communications between robots and ubiquitous devices, but it was not based on a smart space and did not give any concrete examples. Generally, there is almost no systematic research on integrated systems with some combination of robots and smart spaces. One way to approach the systematic research would be to start making concrete and representative sample applications. Therefore, this research is focused on a case study of a u-object finding service done by a robot in a smart space. This paper presents the design and development of a system prototype to manage robots and u-objects attached with RFID tags in a smart room space, and for a robot to communicate with other devices and find a tagged uobject in the smart space. We have conducted some preliminary experiments and the result verifies and evaluates the aforementioned functionalities. In what follows, we first check the possible roles of robots in a smart space as well as the relevant technical problems in general, and then present the u-object finding robot prototype in detail in the remaining sections. After showing the prototype overview in Section 3, we explain how to manage robots and u-objects in Section 4, and how to control and communicate with the robots in Section 5. The experimental

Robots in Smart Spaces

63

environment and results in finding u-objects are described in Section 6. Finally, the conclusion and future work are addressed in the last section.

2 General Services and Issues of Smart Spaces with Robots This section will discuss in general the special roles of robots in a smart space and the related basic issues to be solved in order to provide the desired services of robots. A smart space that can offer automatic services mainly relies on the sensing devices to acquire contextual information, and actuation devices to take responsive actions. As mentioned earlier, these devices are usually installed in some parts of the physical environment; such as walls, or embedded in some physical objects such as books. Because such devices are fixed and don’t move by themselves, they can sense information or act only in specific locations/directions. Different from these devices, robots are capable of moving from one location to another in a space. In addition, many robots can carry sensors and/or actuators. Therefore, a robot can move to a place to sense information or take an action in that place. For example, a movable robot can monitor a place from any direction using its camera to record the condition of a house, office, classroom, shop, street, and so on. Another example is that a robot, under instruction by a virtual host/manager of the smart environment, may bring an object from somewhere in the environment. In general, additional service functions which can be done by robots in smart spaces fall into the following categories. Location-Varied Information Acquisition: When carrying sensors, a robot can get the requested information not only at a specified location, but also from some data distributed in a space. For example, a robot with a microphone can move around to locate a sound source, which is usually hard to precisely detect by fixed microphones. The temperature distribution in a room can also be easily gathered by a movable robot by carrying only a single temperature sensor. Otherwise, many temperature sensors have to be installed in different parts of a relatively large room. That is, robots can function as mobile sensors to collect information at any point in smart space where they can possibly go. Location-Dependent Responsive Action: According to the sensed information, a smart space can take responsive actions. Some reactions may be conducted best at some specific locations. For example, the sound-aware smart room we developed [18] can sense ambient sounds using a microphone array, and output a speech to one of the speakers installed in a fixed location in the room. It sends a be-quiet reminder speech to a speaker near the person who is talking/singing loudly who may disturb others in the same location. Having a robot carrying a speaker in the room, the robot can move near the person and then give the reminder. In general, a robot may conduct some action by moving to a place requested by a smart space. Location-Aware Interaction with Devices/u-Objects: When a robot works in a nonsmart environment, it has to sense and interact with the environment using multimedia data from cameras, microphones, and so on. Therefore, the sophisticated robots rely on complex intelligent technologies for multimedia processing and understanding,

64

T. Kawashima et al.

which are not yet perfect and are very costly. In a smart environment, however, there are various electronic devices which are either independent or embedded in some objects, i.e., being u-objects. Thus, robots will be able to interact with these devices or u-objects via wireless electronic communications, instead of using audiovisual communications. Through communications, a robot may interact with nearby u-objects to identify their locations, know what they are doing, etc. By means of interaction, the robots and devices/u-objects in a smart room can work collaboratively to carry out some tasks. In summary, robots can be applied to add flexible location-based service functions to smart spaces because of their capability to move. To realize the new functions, a robot must collaboratively work with a virtual space host/manager and other devices/u-objects in the space. Therefore, the two fundamental issues for robots to conduct collaborative work in a smart space are how to manage the robots, and how to enable robots to communicate with the space host and various devices/u-objects. A long-term solution to completely solve the management and communication issues would be a scalable framework or middleware capable of supporting many applications in different smart spaces with multiple robots and heterogeneous devices/u-objects. This preliminary study is focused on the necessary management and communications in a specific application, i.e., a prototype called u-object finder, a robot for finding u-object(s).

3 The u-Object Finder Overview It is a very common experience that sometimes we forget and could not immediately find some common items which we need badly, such as keys or books. This is more common to small children and elder people since they often forget what they have just seen or done. The u-object finder (UOF) is an application of the smart space in which a robot moves around to search for a lost object. Figure 1 shows the UOF conceptual environment used in our study and its development.

㫌㪄㪦㪹㫁㪼㪺㫋㩷㪝㫀㫅㪻㪼㫉

㪬㪦㪝㪄㪤㪸㫅㪸㪾㪼㫉 㪩㪝㪠㪛㩷㪩㪼㪸㪻㪼㫉

㫌㪄㪦㪹㫁㪼㪺㫋 㪩㪝㪠㪛㩷㪫㪸㪾㫊 㪬㪦㪝㪄㪩㫆㪹㫆㫋

Fig. 1. The u-object finder environment

Robots in Smart Spaces

65

The environment is a typical room space equipped with computers, devices, and robots which are connected by wireless networks. Basic assumptions to the environment are as follows.

・ ・ ・ ・

・ ・

Every object to be searched must be attached or embedded with at least one device which is used to uniquely identify the object and communicate with other devices. In the present prototype, we attached a RFID tag to every searchable uobject, also called a tagged object. A single object may have multiple attached tags, which can be used to either identify different parts of the object, or to improve search efficiency. A RFID reader is installed somewhere at the room entrance to read the RFID tag of a u-object when the object passes through the entrance. It is used to detect what and when a u-object has been brought in/out of the room. There is one or more movable robots in the room which moves according to the commands and instructions from a room manager. Each robot may carry multisensors for various purposes. To find a tagged object, a RFID reader is carried by the robot in our system. A number of RFID tags are set at fixed positions on the room floor. These tags are used for the robot to precisely locate its position since all the position data has been measured and kept in some room database. Usually a robot can remember the route it has passed, which can be used to estimate the robot’s present position. The electric-magnetic signal strength received by the robot from wireless routers can be also used to estimate its position, but this exhibits some errors in estimating the robot’s position. Thus, the position-fixed RFID tags on the floor can be read when the robot moves, and exploited to reduce or correct the errors. All machines and devices including robots are able to conduct wireless communications via either some specific communication scheme or a common wireless LAN router. There exists a virtual manager, i.e. an application residing on a PC, to manage all devices, u-objects, RFID tags and robots. It also provides an interface for a user to request a robot to find an object and receive the search result via our present UOF prototype.

The u-Object Finder system, as shown in Fig. 2, is mainly composed of the uObject Finder Manager (UOF-Manager) that functions as a central administrator of all devices including robots, and the u-Object Finder Robot (UOF-Robot) that is mainly in charge of searching u-objects in the room. The UOF-Manager has two main components: the u-Object Manager (O-Manager) to manage u-objects, and the Robot Manager (R-Manager) to manage robots. On the other hand, the UOF-Robot has two main components: the Robot Controller (RController) and the Robot Machine (R-Machine). The R-Machine is some kind of robot that has a motor, wheels, an ultrasonic sensor, etc., and thus movable by itself. When the robot moves near a barricade or blocks, it can turn and change direction using the data from the ultrasonic sensor. The movement direction, destination and route will be determined according to the robot’s task, current position, surrounding

66

T. Kawashima et al.

UOF- Manager R- Manager O- Manager

UOF- Robot IEEE802.11b

Bluet oot h

Radio Frequency

u- Object

R- Controller (PDA)

Radio Frequency

R- Machine (Robot)

Fig. 2. The u-object finder components

situations, and so on. The R-Controller functions as a mediator between R-Manager and R-Machine, and runs on a PDA that is carried by the robot. The PDA provides a popular programming environment and adopts the general TCP/IP communication protocol on top of common wireless LANs. A RFID reader and other wired sensors can be also connected directly to this PDA. Therefore, the PDA can greatly extend and enhance the functions of a low-cost robot. The PC, PDA, robot and devices are connected via several networks including IEEE 802.11b LAN, Bluetooth, and radio frequency communication between an RFID reader and an RFID tag. The general procedure to search a u-object is conducted in the following steps. At first, a user informs the UOF-Manager what u-object to search. The UOF-Manager will find out if the object is in the room by checking the object’s existence status kept in the u-object database. If it is in the room, the UOF manager will check if any UOFRobot is available in the room to do the search job. When a robot is available, the UOF-Manager will send an object finding command and the RFID tag number(s) of the u-object to the UOF-Robot. Then, the robot moves around and reads the nearby RFID tags. Once the tag number of the searched object is detected, the UOF-Robot will stop and send the UOF-Manager a search success message. At the same time, the robot will give a speech or a beeping sound to notify the user of its location, which is near to the found u-object. If the u-object cannot be found after a certain time, the UOF-Robot will send an unsuccessful message to the UOF-Manager. The next two sections will explain the UOF-Manager and UOF-Robot in details, respectively.

4 UOF-Manager The UOF-Manager is an application working on a PC. A user can operate the UOFRobot and manage the u-Object respectively using the UOF-Manager. The UOFManager consists of a UOF-GUI, an R-Manager (Robot-Manager) and an O-Manager (u-Object Manager), as shown in Fig. 3. The user can register and delete a u-Object by simply holding it over an RFID reader. The data of the registered u-Object is saved in the u-Object database by the O-Manager.

Robots in Smart Spaces

67

UOF- GUI

R- Manager

Operation Retrieval Connection Module Module Module

UFCP Stack

O- Manager

Registration Status Module Module

u- Object Database

Fig. 3. The UOF-Manager component

4.1 UFCP (U-object Finder Control Protocol) To manage robots and let them work in a smart space, the R-Manager and robots must be first enabled to communicate with each other. One objective of this study is to develop the UFCP stack based on TCP/IP, which will run the R-Manager, RController and R-Machine. It is a protocol for the UOF-Manager to retrieve and operate the UOF-Robot via message exchanges. The UFCP messages are in two forms: UFCP-TCP and UFCP-UDP. The UFCP-UDP is used to retrieve the UOF-Robots in the same network segment, while the UFCP-TCP is used to operate the UOF-robots. The format of the UFCP packet takes on ASCII. The UFCP packet has two fields: command and data, separated by a comma. Table1 shows the message format of UFCP-TCP and UFCP-UDP. 4.2 R-Manager The R-Manager has the UFCP Client which can search a UOF-Robot, make a connection, start and stop the UOF-Robot, change the threshold value of devices (ultrasonic sensor, motor), and get the status of the UOF-Robot. Through the UOF-GUI, a user can conduct the following management functions.

・ Retrieving the UOF-Robot ・

When receiving a robot retrieval demand from a user, the U-Manager will broadcast a UFCP robot search message over connected wireless LANs. The UOFRobot receiving the message will reply an acknowledgment message with its name and current position. The list of retrieved UOF-Robot is shown in UOFGUI. Connection of the UOF-Robot After the user selected one UOF-Robot from the retrieved UOF-Robot list, the user can make a connection using UFCP to the selected UOF-Robot. After the connection is established, the user can start to operate the robot.

68

T. Kawashima et al. Table 1. UFCP message format

UFCP- TCP DATA DESCRIPTION COMMAND UOF- ROBOT START RFID Starting u- Object finding UOF- ROBOT STOP Stopping u- Object finding ACK VALUE, POSITION, etc. Acknowledgement NAK ERROR MESSAGE Negative acknowledgement GET STATUS Getting UOF- Robot status GET VALUE DEVICE NAME Getting specified device’ s value GET POSITION Getting UOF- Robot position SET VALUE DEVICE NAME, VALUE Setting specified device’ s value FOUND RFID Informing found RFID tag SOUND Request for playing sound UFCP- UDP DATA DESCRIPTION COMMAND SEARCH Searching UOF- Robot ACK UOF- ROBOT NAME Acknowledgement

・ Operation of the UOF-Robot

The UOF-Manager can specify which robot will search a u-object through its RFID, can start and stop the UOF-Robot, or get the status of the UOF-Robot. In addition, the UOF-Manager can ask the UOF-Robot to produce a beeping sound to inform the user of its current position in the smart room.

4.3 O-Manager The O-Manager handles the registration and deletion of a new u-Object. When the user holds the RFID attached to an u-Object over an RFID reader, the O-Manager obtains the RFID data through the RFID reader and check whether the RFID is already registered or not. If the RFID is not yet registered, the O-Manager shows a popup window for the user to input the name of the u-Object associated with the RFID. After the user input the name successfully, the user can select the u-Object to be searched in the smart room. The user can also register a u-Object by not using an RFID reader but by inputting the RFID data and name manually. Another function of the O-Manager is to manage the existence status of all registered u-objects. A registered object can be brought out/back to the room. The u-object’s in/out state and corresponding time can be detected by the RFID reader installed at the room entrance. When a UOF-Robot moves around the room, it may encounter many u-objects including the one to be searched and others which are not to be searched. Actually, the positions’ data of all u-objects encountered by the robot can be kept in the u-object database. These existence and position status information can be further used to speedup the searching and tracing of u-objects.

Robots in Smart Spaces

69

5 UOF-Robot As explained in the previous sections, the UOF-Robot is mainly in charge of searching a u-object in the room space. The search is based on two special functions. The first is that, the UOF-Robot can move around the space; and the second is that, the robot carries a RFID reader to detect a RFID tag within a certain distance, e.g. 10cm. The UOF-Robot consists of a R-Controller, a R-Machine and a R-Reader, as shown in Fig. 4.

R- Controller (PDA) R- Machine

R- Controller (PDA)

UFCP Stack ht oot eu lB

R- Machine

UFCP Stack Motor Ultrasonic Sensor

Device Driver B S U

R- Reader

Fig. 4. The UOF-Robot components

Ultrasonic Sensor

Motor R- Reader

Fig. 5. A snapshot of the UOF-Robot

The R-Controller resides in a PDA and is used to communicate and control both the R-Machine and the R-Reader. The R-Reader is a RFID reader that is connected to the PDA via a USB cable. A manufacturer-dependent device driver should be installed in the PDA for the R-Controller to communicate with the R-Reader. The RReader’s task is relatively simple, i.e., mainly reading the nearby tags and sending the tags’ codes to the R-Controller. The R-Machine is a self-movable robot that has a motor, wheels, an ultrasonic sensor, etc. Figure 5 shows a snapshot of the UOF-Robot used in our system. The R-Machine can start to move or stop by following the instructions from the RController. The R-Machine can move around by itself by sensing the surrounding situations with its ultrasonic sensor. The sensor can measure the distance between the R-Machine and the object in front. When the distance from an encountered barricade or block is below some threshold value, it may move away from the barricade/block, and change its moving direction. The changes of rules on the robot’s turn movement and the degree of direction are programmable, and can be set by the user on the UOFGUI. Although the R-Machine is operated by the R-Manager, their interaction messages during operations are mediated via the R-Controller. The R-Controller communicates with the R-Manager over a wireless LAN (IEEE802.11b), and also communicates with the R-Machine using a Bluetooth network, which is embedded in the robot. Like the R-Manager, the UFCP stack runs on both the R-Controller and R-Machine. Therefore, the three of them can exchange

70

T. Kawashima et al.

messages with each other. When receiving a robot retrieval message request from RManager, the R-Controller sends a UFCP-UDP response message to the R-Manager. The operation request, on the other hand, is sent as an UFCP-TCP message. When the robot moves, the R-Reader detects whether there is any RFID tag nearby. If it detects one, it sends the tag code to the R-Controller which will check whether the code is the same as the u-object’s code to be searched. When the RController gets the RFID code of the u-object to be searched from the R-Reader, it sends a found message in UFCP-TCP format to the UOF-Manager. In addition, when it detects a position tag which is set on the floor in a smart room, it also sends the UOF-Manager the tag code, which can be used to get the current robot position. An operation message from the UOF-Manager is resent to the R-Machine by the RController via the Bluetooth network.

6 System Implementation and Experiment A preliminary u-object finding prototype has been implemented with emphasis on communications between heterogeneous devices across different physical networks, and management of the robots and u-objects. The following devices and software were used in our current implementation.

・ ・ ・ ・

Robot: Mindstorm NXT including an Ultrasonic sensor and a Motor developed by LEGO [19]. It adopts lejOS [20] to control the robot movement and set some moving parameters. The ultrasonic sensor is installed in front of the robot. RFID tag and reader developed by Phidgets, Inc [21]. Java-based API which can run on Windows, Linux and Mac OS X platforms. The RFID tags were attached in all objects to be searched, and were also set on some locations on the room floor. The readable distance between a tag and a reader is about 5cm. PDA: AXIM series by Dell, Inc. It runs on Windows Mobile OS and supports .NET Compact Framework. Networks: wireless LAN (IEEE 802.11a/b/g) between the PC and PDA, and Bluetooth between PDA and NXT robot. The USB cable is used to connect a Phidgets tag reader to the PDA.

The u-object finding system is developed using Java programming language. Its software consists of three main parts. The first one is the UFCP, which is a TCP/IPbased common communication platform running on PC, PDA and robot. The UFCP is a very light protocol, only about 50Kbytes, and thus able to run low-spec devices. The second one is the R-Controller running on the PDA. It has multiple functions; such as interacting with the UOF-Manager to get commands and send back results, checking the tag codes from the R-Reader, controlling the R-Machine, and mediating messages between the R-Manager and the R-Machine. The third one is the UOF-Manager running on a PC, which manages both robots and u-objects, and also provides a GUI to interact with a user. To test our system, the experimental environment was set up in our laboratory room, and a partial setting is shown in Fig. 6. The space is 5mx5m on which there is one tagged object to be searched, a robot to search the object, some position tags fixed on the floor, etc.

Robots in Smart Spaces

71

Fig. 6. The UOF experiment environment

When a user wants to find an object in the space, he/she first starts the UOFManager, which is a Java application program running on a PC. Figure 7 shows the snapshot of the manager’s UOF-GUI. All u-objects registered in this room are listed on the upper-left of the GUI. From the u-object list, the user selects the desired object to be found. Next, the user can check what UOF-Robots are available in the room by clicking a robot retrieval button. As shown in the bottom-left of Fig. 7, one robot is retrieved through communications between the R-Manager on the PC and the R-Controller on the PDA carried by the robot. When the UOF-Manager makes a connection with the UOF-Robot successfully, a new tab window is added to the right side of GUI. At the same time, the robot appears at its present position in the middle-right map of the GUI. Some parameters related to the robot movement are displayed as well. The user can operate the UOF-Robot using the buttons and the text field on the tab window. Upon receiving the instruction to find a u-object, the UOF-Manager will send a “START” message command which sends the u-object tag code to the R-Controller of the selected UOF-Robot. Then, the R-Controller gives the “START” command to the R-Machine and asks the R-Reader to read the RFID tags. When the robot moves around, the R-Reader sends every tag code it has detected to the R-Controller. The R-Controller compares the tag code received from the R-Reader with the tag code of the searched u-object. If the two codes are the same, it means that the searched object is very near the robot. Thus, the R-Controller sends a “STOP” command to the R-Machine, and a “FOUND” message to the UOF-Manager. And then, the robot stops and generates a beeping sound to notify the user of its location. If the u-object is not yet found within the specified period of time, the search operation is considered a failure, and then the process is terminated. In our experimental environment, the u-object was found in many cases, but the search time varied greatly depending upon the positions of the robots and the uobject. This is mainly due to the short readable distance between a RIFD read and a

72

T. Kawashima et al.

Fig. 7. A snapshot of the UOF-Manager GUI

tag, and the robot’s moving course in our current implementation which is based on a rather simple algorithm that the UOF-Robot monitors the distance value from its ultrasonic sensor and turns left or back whenever the distance between the robot and barrier becomes less than 10cm. Although the searching performance is unsatisfactory, the whole system works well in terms of communications and management of the robots and devices in the smart space.

7 Conclusion and Future Work The smart space/environment and robotics are used to be studied as two separate research disciplines. It is therefore very significant to study their combination so as to form a new merged research area and generate some novel applications. Based on this idea, we first investigated the potential services and possible new technical issues when integrating the robots into smart spaces, and then focused on a case study of a uobject finding movable robot prototype to search a u-object, which is attached with RFID tags, in a smart room space. We have discussed the design and implementation as well as experiments of the developed system prototype. As mentioned in the previous section, the research presented in this paper is mainly for communications and management of robots and u-objects in a smart space. However, the object searching procedure took a long time and thus it’s inefficient. This is because of two problems: the short reading distance (about 10cm) between the RFID reader and a tag, and the course was simple for the robot’s movement. To overcome the former limitation, the RFID tag with longer reading distance such as 30~50cm will be used in our next study. Multiple tags can be attached on the different sides of a large object to improve the searching efficiency. In addition, a set of RFID readers which can read tags within 1~2m may be installed in fixed locations to first detect the possible region where the u-object is located and the robot can go directly to the

Robots in Smart Spaces

73

region to find the more precise position of the u-object. To overcome the second limitation, the robot must be able to remember the route passed and the obstacles encountered to avoid going to the same places repeatedly. Another interesting approach is to ask two or more robots to search a u-object together, and study how the robots collaborate with each other. Aside from the above, there are still many technical problems and challenging issues to tackle when combining robots and smart space which should be studied along with many kinds of applications in the future.

Acknowledgement This work was partially supported by Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research, 19200005.

References 1. Weiser, M.: Ubiquitous Computing. IEEE Computer (October 1993) 2. Weiser, M., Brown, J.S.: The Coming Age of Calm Technology. In: Denning, P.J., Metcalfe, R.M. (eds.) Beyond Calculation: The Next Fifty Years of Computing (1996) 3. IEEE802.11, http://www.ieee802.org/11/ 4. Bluetooth, http://www.bluetooth.org/ 5. Callaway, E., Gorday, P., Hester, L., Guiterrez, J.A., Naeve, M.: Home Networking with IEEE802.15.4: A Developing Standard for Low-Rate Wireless Personal Area Networks. IEEE Communications Magazine, 70–77 (August 2002) 6. IEEE802.15 Working Group for WPAN, http://www.ieee802.org/15/ 7. Porcino, D., Hirt, W.: Ultra-Wideband Radio Technology: Potential and Challenges Ahead. IEEE Communications Magazine, 66–67 (July 2003) 8. Dey, A.K.: Understanding and Using Context. Personal and Ubiquitous Computing 5(1), 4–7 (2001) 9. Abowd, G.D., Mynatt, E.D., Rodden, T.: The Human Experience. IEEE Pervasive Computing, 48–57 (January-March 2002) 10. Ma, J.: Smart u-Things – Challenging Real World Complexity. In: IPSJ Symposium Series, vol. 2005(19), pp. 146–150 (2005) 11. Private, G.: Smart Devices: New Telecom Applications & Evolution of Human Interfaces. In: CASSIS Int’l Workshop, Nice (March 2005) 12. Cook, D.J., Das, S.K.: Smart Environments: Technologies, Protocols and Applications. Wiley-Interscience, Chichester (2005) 13. Ma, J., Yang, L.T., Apduhan, B.O., Huang, R., Barolli, L., Takizawa, M.: Towards a Smart World and Ubiquitous Intelligence: A Walkthrough from Smart Things to Smart Hyper-spaces and UbicKids. International Journal of Pervasive Comp. and Comm. 1(1) (March 2005) 14. Pinto, J.: Intelligent Robots Will Be Everywhere, in Automation.com (2003), http://www.jimpinto.com/writings/robots.html 15. Sakagami, Y., Watanabe, R., Aoyama, C., Matsunaga, S., Higaki, N., Fujimura, K.: The intelligent ASIMO: System Overview and Integration. Intelligent Robots and System (2002)

74

T. Kawashima et al.

16. Kaneko, K., Kanehiro, F., Kajita, S., Hirukawa, H., Kawasaki, T., Hirata, M., Akachi, K., Isozumi, T.: Humanoid Robot HRP-2 (ICRA 2004) (April 2004) 17. Ueno, K., Kawamura, T., Hasegawa, T., Ohsuga, A., Doi, M.: Cooperation between Robots and Ubiquitous Devices with Network Script Flipcast. In: Proc. Of Network Robot System: Toward Intelligent Robotic Systems Integrated with Environments (IROS) (2004) 18. Ma, J., Lee, J., Yamanouchi, K., Nishizono, A.: A Smart Ambient Sound Aware Environment for Be Quiet Reminding. In: Proc. of IEEE ICPADS/HWISE 2005, Int’l Workshop on Heterogeneous Wireless Sensor Networks, Fukuoka (July 2005) 19. Lego Mindstorm NXT, http://mindstorms.lego.com 20. lejOS, http://lejos.sourceforge.net/ 21. Phidgets, Inc., http://www.phidgets.com

Biometrics Driven Smart Environments: Abstract Framework and Evaluation Vivek Menon1, , Bharat Jayaraman2, and Venu Govindaraju2 Amrita University, Coimbatore 641 105, India vivek [email protected] 2 University at Buffalo, Buffalo, NY 14260, USA [email protected], [email protected] 1

Abstract. We present an abstract framework for ‘smart indoor environments’ that are monitored unobtrusively by biometrics capture devices, such as video cameras, microphones, etc. Our interest is in developing smart environments that keep track of their occupants and are capable of answering questions about the whereabouts of the occupants. We abstract the smart environment by a state transition system: Each state records a set of individuals who are present in various zones of the environment. Since biometric recognition is inexact, state information is probabilistic in nature. An event abstracts a biometric recognition step, and the transition function abstracts the reasoning necessary to effect state transitions. In this manner, we are able to accommodate different types of biometric sensors and also different criteria for state transitions. We define the notions of ‘precision’ and ‘recall’ of a smart environment in terms of how well it is capable of identifying occupants. We have developed a prototype smart environment based upon our proposed concepts, and provide experimental results in this paper. Our conclusion is that the state transition model is an effective abstraction of a smart environment and serves as a basis for integrating various recognition and reasoning capabilities.

1

Introduction

The vision of pervasive computing [1] provides the inspiration for a smart environment saturated with sensors, computing, and communication devices that are gracefully interfaced with human users [2]. In this paper, we focus on smart indoor environments such as homes, offices, etc. Indoor environments do not suffer from the problems of power or battery-life that confront outdoor environments. The goal of our research is to develop smart indoor environments that can identify and track their occupants as unobtrusively as possible and be capable of answering queries about the occupants. Such ‘context-aware’ systems can identify and track people in environments ranging from homes for elderly or disabled, office workplace, department stores and shopping complexes to larger arenas such as airports, train stations, etc. 

This work was done while the author was a Visiting Research Scientist at the Center for Unified Biometrics and Sensors, University at Buffalo.

F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 75–89, 2008. c Springer-Verlag Berlin Heidelberg 2008 

76

V. Menon, B. Jayaraman, and V. Govindaraju

(a)

(b)

(c)

Fig. 1. Architecture of a Biometrics Driven Smart Environment

Identification approaches vary from tag-based approaches such as those involving RFID to those based on biometrics of the user. Tag-based methodologies tend to be obtrusive, requiring the individual to continuously retain them, however small the tag maybe. Some of the biometric techniques, such as fingerprint and iris scans, require a ‘pause-and-declare’ interaction with the human [3]. They are less natural than face, voice, height, and gait, which are less obtrusive and hence are better candidates for use in our smart environments. Figure 11 shows the overall architecture of a biometrics-driven smart environment. Although the figure illustrates a single biometric modality of face recognition, the architecture is also applicable to other biometric modalities. For example, zone 1 might use voice recognition, zone 2 might use face recognition, 1

Face images blurred to preserve anonymity.

Biometrics Driven Smart Environments

77

and zone 3 might use height estimation. However, in all cases the output of a biometric recognition is the set of person probability pairs as discussed in more detail below. The main contribution of this paper is a framework for abstracting the behavior of a biometrics-driven smart environment in terms of a state transition system. Our proposed framework supports multiple biometric modalities in a uniform manner and facilitates a precise statement of the performance aspects of a smart environment. The state of an environment is expressed in terms of the probabilities of the occupants being present in the different zones of the environment. The state information is probabilistic because a biometric recognizer typically provides a set of scores indicating the degree of match between the subject and the candidates in the database. Therefore, in our approach an event abstracts a biometric recognition step - whether it is face recognition, voice recognition, etc. - and is represented as a set of pairs o, p(o) where p(o) is the probability 2 that occupant o has been recognized at this event. The transition function abstracts the reasoning necessary to effect state transitions. Effectively, the transition function takes as input, a state and an event, and determines the next state by assigning revised probabilities to the occupants in the environment based upon the probabilities in the event. In this manner, we are able to accommodate different types of biometric sensors and also different criteria for state transitions, including those that incorporate declarative knowledge of the individuals and the environment. It is not necessary for us to consider non-deterministic transitions, since a state itself is represented as a set of occupants and their probabilities. We introduce the concepts of precision and recall in order to provide a quantitative measure of the performance of a smart environment. Precision captures how well an occupant is recognized, while recall captures whether an occupant is recognized at all. These are complementary concepts and together capture the overall performance of a smart environment. The concepts of precision and recall are standard performance measures in the information retrieval literature [6], but we have adapted the definitions to suit our context. The rest of this paper is organized as follows. The related work is presented in Section 2 while the details of our model are discussed in Section 3. The experimental prototype is described in Section 4 and the conclusions and the future work are presented in Section 5.

2

Related Work

There has been considerable interest in the subject of smart environments. A major difference between our proposed approach and several of the approaches surveyed below is our use of a framework in which the details of biometric recognition are abstracted simply as events in a state transition system. We also 2

Such scores can be mapped to probabilities by running the recognizer on a large number of samples, as shown in [4,5].

78

V. Menon, B. Jayaraman, and V. Govindaraju

characterize the performance of the smart environment in terms of the concepts of precision and recall. We briefly survey closely related efforts below and highlight their main features. The survey paper by Cook and Das [7] provides a good account of the stateof-the-art in smart environments. There are several projects that focus on smart homes (MavHome [8], the Intelligent Home [9], the House n [10] and development of adaptive control of home environments by also anticipating the location, routes and activities of the inhabitants. 2.1

Location Estimation

An extensive survey and taxonomy of location systems for a ubiquitous computing application is discussed in [11] while [12] provides a more recent survey of position location techniques in mobile systems and draws a comparison between their parameters. Fox et al. [13] highlight the relevance of Bayesian filtering techniques in dealing with the uncertainty characteristic of sensors in pervasive computing environments and apply them in the context of estimation of an object’s location. Data Association problem [14] is a pertinent issue in scenarios which involve tracking of multiple people using anonymous sensors. The ambiguity in identity estimation arising due to the absence of ID sensors and the corresponding tags on people is difficult to resolve. Thus identity estimation, absolute or relative is a desirable and integral subsystem of any object tracking mechanism. 2.2

Tracking

Bui et al. [15] propose an Abstract Hidden Markov Model(AHMM) based approach for tracking human movement in an office like spatial layout and to predict the object trajectories at different layers of detail. For location tracking in a single inhabitant smart space, Roy et al. [16] propose an optimal algorithm based on compressed dictionary management and online learning of the inhabitant’s mobility profile. In a related work [17], they highlight the complexity of optimal location prediction across multiple inhabitants in a smart home and propose a stochastic game-theoretic approach, which learns and estimates the inhabitants’ most likely location profiles. This study uses RFID tags to track the inhabitants. A combination of particle filters with Kalman filters to track multiple objects with accuracy of anonymous sensors and identification certainty of id-sensors is discussed in [13,18]. Krumm et al. [19] investigate the nuances of visual person tracking in intelligent environments in the context of the EasyLiving project [20] by deploying multiple cameras for tracking multiple people. However this tracking experiment was within the confines of a single room and the identity estimation and maintenance for facilitating the tracking only dealt with non-absolute, internally system generated identity of tracked persons. 2.3

Biometrics in Smart Environments

Pentland and Choudhury [3] highlight the importance of deploying audio-andvideo based recognition systems in smart environments as these are modalities

Biometrics Driven Smart Environments

79

similar to those used by humans for recognition. They summarize the face recognition efforts and discuss various commercial systems and applications as well as its novel applications in smart environments and wearable computing. Chen and Gellersen [19] propose a new method to support awareness based on fusion of context information from different sources in a work environment which included integration of audio and video sources with more specific environment sensors and with logical sensors that capture formal context. Reasoning over the context information generated by applying different perception techniques on the raw data collected is used to generate a refined context. Hamid Aghajan et al. [22] propose a vision-based technology coupled with AI-based algorithms for assisting vulnerable people and their care givers in a smart home monitoring scenario. However, users are expected to wear a wireless identification badge that broadcasts a packet upon sensing a significant signal by one of the accelerometers. Gao et al. [23] propose a new distance measure for authentication in their face recognition system for a ubiquitous computing environment which relies on a fusion of multiple views of each person. Their work focuses on optimizing a single modality to improve robustness rather than deploying a multimodal approach that fuses different biometrics. Hong et al. [24] discuss structure, operation and performance of a face verification system using Haar-like features and HMM algorithm in a ubiquitous network environment. Zhang et al. [25] propose a distributed and extensible architecture of a continuous verification system that verifies the presence of logged-in user. A Bayesian framework that combines temporal and modality information holistically is used to integrate multimodal passive biometrics which includes face and fingerprint. Driven by the proliferation of commercially available hand-held computers, Hazen et al. [26] research the improvements in identification performance by adopting a bimodal biometric approach to user identification, for use on mobile devices by integrating audio and visual biometric information in the form of voice and face. They also report significant improvements in identification performance that can stem from using dynamic video information instead of static image snapshots on a database of 35 different people. Highlighting the privacy concerns of video recording discussed in Bohn et al [27] and reliability issues of face recognition techniques for user authentication, Vildjiounaite et al. [28], deploy a fusion of accelerometer based gait recognition and speaker recognition as an unobtrusive and marginally privacy-threatening means of user authentication with personal mobile devices. They have reported improved performance in the combination mode as opposed to individual modalities for a user base of 31. More recently Bernardin Stiefelhagen [29] have implemented a system for the simultaneous tracking and incremental multimodal identification of multiple users in a smart environment which fuses person track information, localized speaker ID and high definition visual ID cues opportunistically to gradually refine the global scene model and thus increase the system’s confidence in the set of recognized identities. In terms of the focus on the use of non-obtrusive biometrics based recognition and location estimation, our work is similar to [29]. However, in our research,

80

V. Menon, B. Jayaraman, and V. Govindaraju

we propose an abstract framework where in a variety of biometric modalities can be incorporated in a uniform manner. Our approach to identity estimation deals with the absolute identity of people across multiple zones of a facility. However, we attempt to highlight the inherent uncertainty of automated face recognition by recasting the eigen distances generated by eigenface algorithm into a probability distribution of the registered faces, instead of the conventional approach of assigning the value with the least eigen distance as the matching face. This probabilistic approach to biometric recognition is a key theme around which we construct our abstract framework for a biometrics driven smart environment. 2.4

State Space Representation

It might appear that a Hidden Markov Model(HMM) would serve as an elegant basis for representing the state space. From a HMM perspective, a smart environment with n occupants and m zones can have mn distinct possible states. Thus probabilities are not associated with the states but with the transitions between them; these transition probabilities are to be learnt from past behavior or by simulation [15]. Thus an HMM approach is computationally more complex due to a state space explosion and the requirement of a priori probabilities of trajectories. In our approach, the size of a state is m ∗ n, meaning that for each of the m zones we record the probabilities of each of the n occupants being present in that zone. Therefore, in Section 4 (Evaluation), we depict a typical state as a table with n rows and m columns. The transitions from one state to another are deterministic. Therefore, given any event in a zone, the next state is unambiguously determined. In contrast with the HMM approach, we do not need to learn the transition probabilities in order to determine the next state because biometric recognition (or event) provides a direct means for effecting state transitions. Our state transition model is discussed in the next section.

3

Framework

Definition (Smart Environment). An n-person smart environment is abstracted as a state transition system (S, E, Δ) where S is the set of states labeled s0 , s1 , . . . sx ; E is the set of events labeled e1 , e2 , . . . ex and Δ : S × E → S is a function that models the state transition on the occurrence of an event. The state transitions may be depicted as follows: e

e

e

1 2 x s1 → s2 . . . → sx s0 →

We shall consider a smart environment as being divided into a number of zones, each of which may be a region (or a set of rooms). We include two special zones, an external zone and a transit zone, for the sake of convenience. Definition (State). Given n occupants, o1 . . . on and m zones labeled 1 . . . m, a state sk of the environment is represented by an m-tuple Z1k . . . Zmk  where for 1 ≤ j ≤ m, Zjk= {oi , pjk (oi ) : 1 ≤ i ≤ n}. Also, in each state sk and for m each occupant oi , i=1 pjk (oi ) = 1.

Biometrics Driven Smart Environments

81

The state of an environment is expressed in terms of the probabilities of the occupants being present in the different zones of the environment. The constraint m i=1 pjk (oi ) = 1 indicates that sum of probabilities of any occupant being present across all the zones in any state equals one. In the initial state s0 , we may assume without loss of generality that all occupants are in the external zone with probability 1. Given a smart environment with n occupants, m zones, and x number of events, the total size of the state space is m ∗ n ∗ (x + 1). Thus, the size of the state space is quadratic in m and n rather than exponential, as in HMMs. In this paper we model all exit events as entry events into a transit zone. Hence it suffices in our model to only consider entry events. An event is essentially an abstraction of a biometric or feature recognition step performed in the environment. Definition (Event). Given n occupants o1 . . . on , an (entry) event ek occurring at zone j (1 ≤ j ≤ m) at time t is represented as t, j, P , where P = {oi , pjk (oi ) : 1 ≤ i ≤ n} and pjk (oi ) is the probability that an occupant oi was recognized at zone j in event ek . As noted earlier, an event is an abstraction of a recognition step. For simplicity, we assume that events happen sequentially in time, i.e., simultaneous events across different zones are ordered arbitrarily in time. That is, the entry of an occupant oi into zone zi and occupant oj to zone zj at the same time t can be modeled as oi before oj or oj before oi . Definition (Transition Function). Δ : S × E → S, maps state sk−1 into state sk upon an event ek = t, j, P  occurring at time t in zone j, where P = {oi , pjk (oi ) : 1 ≤ i ≤ n}. Let sk−1 = Z1k−1 . . . Zjk−1 . . . Zmk−1  and Zjk−1 = {oi , pjk−1 (oi ) : 1 ≤ i ≤ n}. Then Δ determines state sk = Z1k . . . Zjk . . . Zmk  as follows: Let xi = 1 − pjk (oi ). Then, Zjk = {oi , pjk (oi ) + xi ∗ pjk−1 (oi ) : 1 ≤ i ≤ n}

(1)

Zlk = {oi , xi ∗ plk−1 (oi ) : 1 ≤ i ≤ n}, for 1 ≤ l ≤ m and l = j

(2)

The transition function maps a state sk−1 to a state sk upon an event ek occurring at zone j. For zone j, we sum the new probability pjk (oi ) for an occupant generated by event ek with the complement of the new probability value 1 − pjk (oi ), apportioned by a factor of the existing probability pjk−1 (oi ). In the event of a revision, there might be a violation of the constraint m that the sum of probabilities for any occupant across all zones equals one ( i=1 p(oi ) = 1). To restore adherence to this constraint, for each occupant oi , we apportion to the probability of oi being present in each zone l = j by redistributing the complement of the new probability value, 1 − pjk (oi ), in the ratio of the probability value in existing state plk−1 (oi ). This transition function ensures that the probability values associated with a new event as well as the current state figure in the determination of the new state as in a Markov process. Since we are dealing with a closed environment with a fixed set of occupants, o1 . . . on , we can, in general, utilize a priori declarative knowledge regarding

82

V. Menon, B. Jayaraman, and V. Govindaraju

the occupants, such as their schedules or knowledge of the environment, such as the distance between rooms and whether an occupant could move between a pair of rooms within a certain interval of time. However, the transition function presented in the above definition does not make any use of such declarative knowledge of the occupants. The nature of the declarative knowledge can also be fuzzy to factor in the probabilistic nature of the environment. The reasoning component can alleviate some of the weaknesses in the recognition component. However it should ensure that its results are not totally contradictory to the recognition system thereby generating inconsistencies. We now define the concepts of precision and recall for a smart environment. These are defined in terms of the ground truth, which, for a given input event sequence, is a sequence of states of the environment wherein the presence or absence of any occupant in any zone is known with certainty (0 or 1). Precision captures how well an occupant is recognized, while recall captures whether an occupant is recognized at all. Definition (Ground Truth). Given n occupants O={o1 . . . on } and an event sequence e1 . . . ex , then the ground truth is the sequence of states g1 . . . gx where each gk = T1k . . . Tjk . . . Tmk  and Tjk = {oi , qjk (oi ) : 1 ≤ i ≤ n ∧ qjk (oi ) ∈ {0, 1}}. Also, oi , 1 ∈ Tjk → oi , 1 ∈ Tlk , for all l = j in state gk . Given a zone of a state, the precision for that zone of the state is defined as the average probability of those occupants that are present in that zone of the state as given in the ground truth. The average precision across all zones (where at least one occupant is present as per the ground truth) is the precision for the state, and the average precision across all states is the precision for a given ground truth. Finally, the average across multiple ground truths is the precision of the smart environment. Definition (Precision). Given an environment with m zones, n occupants O = {o1 . . . on }, an event sequence E = e1 . . . ex , a ground truth G = g0 , g1 , . . . gx , and state transitions S = s0 , s1 , . . . sx . We define the precision, π, with respect to G as follows: Let πjk = ajk /bjk , where ajk =

 {pjk (oi ) : 1 ≤ i ≤ n ∧ qjk (oi ) = 1}

bjk = |

 {oi : 1 ≤ i ≤ n ∧ qjk (oi ) = 1}|

Then πk =

m  j=1

πjk /m, and we define π =

x 

πk /x.

k=1

preciNow, given a set of ground truths {G1 , G2 , . . . G2 } with the corresponding t sions {π 1 , π 2 , . . . π t }, the precision of the smart environment, Π = l=1 π l /t. For a given ground truth, state and zone, we define recall with respect to a threshold θ as the ratio a/b, where a is the number of occupants of that zone with probabilities greater than θ and who are present in the ground truth, and

Biometrics Driven Smart Environments

83

b is the number of occupants who are present in the ground truth for that zone. The recall for a state is the average of the probabilities across all zones where at least one occupant is present as per the ground truth. The average recall across all states is the recall for a given ground truth, and the average across multiple ground truths is the recall of the smart environment. Definition (Recall). Given an environment with m zones, n occupants O = {o1 . . . on }, an event sequence E = e1 . . . ex , a ground truth G = g0 , g1 . . . gx , and state transitions S = s0 , s1 , . . . sx . We define the recall, ρ, with respect to a threshold θ as follows: Let ρjk = ajk /bjk , where ajk = |

 {oi : 1 ≤ i ≤ n ∧ qjk (oi ) = 1 ∧ pjk (oi ) > θ}|

bjk = |

 {oi : 1 ≤ i ≤ n ∧ qjk (oi ) = 1}|

Then ρk =

m 

ρjk /m, and we define ρ =

j=1

x 

ρk /x.

k=1

preciNow, given a set of ground truths {G1 , G2 , . . . G2 } with the corresponding t sions {ρ1 , ρ2 , . . . ρt }, the recall of the smart environment, R = l=1 ρl /t. As it is clear, the recall is inversely proportional to the threshold, θ, since lowering the threshold will result in more occupants being identified. This figure is generally arrived at experimentally for a smart environment. A reasonable choice of θ is 0.5, and this is also the value that we adopt in our experiments. In the above definition, the recall was defined zone-wise. An alternative approach is to disregard the zones while taking the ratio; doing so would increase the overall recall. Our definition gives due importance to zones, and hence is a relatively more conservative.

4

Evaluation

We have developed an experimental prototype embodying the ideas presented in this paper. Figure 1(b) illustrates a 4-zone smart environment (fourth zone representing the external zone) with 25 occupants who are monitored by video cameras installed in each of the zones. Our experimental prototype collects sample face images of the 25 occupants of an office facility and pre-registers them in a training database. The image sensors deployed in each zone detects the presence of occupants as they move through the zones and verify the face images extracted from the video against the database. The distance scores generated by eigenface is recast into a probability value [4,5] which denotes a posterior probability of the detected face matching the pre-registered occupants. This set of person-probability pairs generated essentially constitutes an event as defined in Section 3.

84

V. Menon, B. Jayaraman, and V. Govindaraju

Fig. 2. Event Sequence

Although our abstract framework is independent of the details of any particular biometric modality, we illustrate our concepts in terms of face recognition. Automated face recognition is yet to attain any comparable levels of robustness as that of humans. Factors such as viewing angle, distance, background clutter, lighting spectrum, intensity, angle and diffuseness of lighting, differences between posed photographs and spontaneous expression can cause fluctuations in the performance of computer vision based on statistical classifiers [30]. Our prototype is based upon OpenCV’s [31] implementation of the eigenface algorithm [32], which provides a basis for a simple though not robust implementation of face recognition. This prototype can be extended to incorporate other biometric recognizers in a similar manner. For example, for voice recognition, the voice samples of occupants are pre-registered in a database instead of face image samples. The ambient voice that is picked up by the voice sensors installed in different zones can be

Biometrics Driven Smart Environments

85

Fig. 3. Sample State Transition

(a)

(b) Fig. 4. Illustrating Precision and Recall

matched against the voice database to generate a set of person-probability pairs. In this manner the different biometric recognizers are interfaced in a uniform way with the rest of the system. We illustrate with figures 2, 3, and 4, the computation carried out by the state transition system that is embodied in the 4-zone smart environment. We

86

V. Menon, B. Jayaraman, and V. Govindaraju

have presented the observations and results obtained in a tabular form for ease of understanding. Figure 2 presents 10 events involving four of the occupants of the smart environment. Each event is presented as a column of probability values - these form the output of the face recognition module as it matches a given face against the training database. Shown in boldface are the probability values corresponding to the actual occupants who were involved in the corresponding events, as per the ground truth. Italicized values indicate the probabilities corresponding to the occupants who were not involved in the corresponding events, but falsely assigned values by the recognizer. This ambiguity may arise due to any of the reasons already discussed above. Figure 3 illustrates a sample transition from state s8 to s9 upon the occurrence of event e9 in zone 1. The probabilities of the occupants in zone 1 in state s9 are obtained as per equation (1) defined under the transition function definition. For the remaining zones of s9 , the probability values are obtained as per equation (2) defined under the transition function. Figure 4 illustrates the results of the precision and recall for the ground truth corresponding to the event sequence of figure 2. The low values for precision at zone 3, corresponding to states s3 and s10 in particular, can be traced to the ambiguity arising in the face recognition step at events e3 and e10 , both occurring at zone 3 which results in a low probability of recognition of occupants o1 and o2 at these events respectively. For the same reason, the values for recall at zone 3 also suffers, thereby affecting the average recall of the states s3 . . . s6 .

5

Conclusions

We have presented a novel framework for non-obtrusive biometric based indoor smart environments for identification and tracking. 1. A state transition framework in which events abstract different biometric recognition steps and transitions abstract different reasoning steps. 2. A characterization of the performance of the smart environment in terms of the concepts of precision and recall. Our state transition system is fundamentally probabilistic because the biometric recognition that underlies events is inexact in nature. Our formulation of precision and recall succinctly characterizes the performance of a smart environment. We believe that our state transition model is an effective abstraction of a smart environment and serves as a basis for integrating different recognition and reasoning capabilities. In our model, a state provides location information at a zone level and a sequence of consecutive states implicitly contains zone level tracking information for all occupants. While it is possible to define precision and recall with respect to any query of interest, we have formulated them in a query independent manner which we believe is more general. Our model is capable of supporting spatial and temporal queries, such as: the location of an occupant in the facility; time of entry/exit of an occupant; the first/last person to enter/leave the facility;

Biometrics Driven Smart Environments

87

the current occupants present in the facility; etc. A query of special interest is tracking of individuals over time and space. We plan to enhance our current prototype by incorporating a variety of biometric recognizers, such as for height, gait, voice, etc. It is possible to fuse two or more biometric modalities to enhance the overall performance of recognition. We also plan to incorporate spatio-temporal reasoning based upon declarative knowledge of the environment as well as the occupants. Through such enhancements in recognition and reasoning, we can improve the overall precision and recall of the smart environment. We plan to test this approach on larger environments and support speech-based information retrieval queries about the environment.

Acknowledgements Thanks to Philip Kilinskas for his help in developing the experimental prototype; Dr. Jason J. Corso for discussions on Markov models; and members of the Center for Unified Biometrics and Sensors for their comments and suggestions.

References 1. Weiser, M.: The Computer for the 21st Century. Scientific American 265(3), 66–75 (1991) 2. Satyanarayanan, M.: Pervasive Computing: Vision and Challenges. IEEE Personal Communications 8(4), 10–17 (2001) 3. Pentland, A., Choudhury, T.: Face Recognition for Smart Environments. IEEE Computer 33(2), 50–55 (2000) 4. Cao, H., Govindaraju, V.: Vector Model Based Indexing and Retrieval of Handwritten Medical Forms. In: Proc. of the Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), pp. 88–92. IEEE Computer Society, Washington (2007) 5. Bouchaffra, D., Govindaraju, V., Srihari, S.: A Methodology for Mapping Scores to Probabilities. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(9), 923–927 (1999) 6. van Rijsbergen, C.J.: Information Retrieval. Butterworths, Boston (1979) 7. Cook, D., Das, S.: How smart are our environments? An updated look at the state of the art. Pervasive and Mobile Computing 3(2), 53–73 (2007) 8. Youngblood, M., Cook, D.J., Holder, L.B.: Managing adaptive versatile environments. In: Proc. of the Third IEEE Intl. Conf. on Pervasive Computing and Communications, pp. 351–360. IEEE Computer Society, Washington (2005) 9. Lesser, V., et al.: The Intelligent Home Testbed. In: Proc. of the Autonomous Agents 1999 Workshop on Autonomy Control Software. ACM, New York (1999) 10. House n Living Laboratory Introduction (2006) 11. Hightower, J., Borriello, G.: Location Systems for Ubiquitous Computing. IEEE Computer 34(8), 57–66 (2001) 12. Manesis, T., Avouris, N.: Survey of position location techniques in mobile systems. In: Proc. of the Seventh Intl. Conf. on Human Computer interaction with Mobile Devices and Services. MobileHCI 2005, pp. 291–294. ACM, New York (2005)

88

V. Menon, B. Jayaraman, and V. Govindaraju

13. Fox, D., Hightower, J., Liao, L., Schulz, D., Borriello, G.: Bayesian filtering for location estimation. IEEE Pervasive Computing 2(3), 24–33 (2003) 14. Bar-Shalom, Y., Li, X.-R.: Multitarget-Multisensor Tracking: Principles and Techniques, Yaakov Bar-Shalom (1995) 15. Bui, H.H., Venkatesh, S., West, G.: Tracking and surveillance in wide-area spatial environments using the Abstract Hidden Markov Model. Intl. Journal of Pattern Recognition and Artificial Intelligence 15(1), 177–195 (2002) 16. Roy, A., Bhaumik, S., Bhattacharya, A., Basu, K., Cook, D.J., Das, S.K.: Location aware resource management in smart homes. In: Proc. of the First IEEE Intl. Conf. on Pervasive Computing and Communications, pp. 481–488. IEEE Computer Society, Washington (2003) 17. Das, S.K., Roy, N., Roy, A.: Context-aware resource management in multiinhabitant smart homes: A framework based on Nash H-learning. Pervasive and Mobile Computing 2(4), 372–404 (2006) 18. Schulz, D., Fox, D., Hightower, J.: People tracking with anonymous and id-sensors using Rao-Blackwellised particle filters. In: Proc. of the 18th Intl. Joint Conference on Artificial Intelligence (IJCAI), pp. 921–926 (2003) 19. Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M., Shafer, S.: Multi-Camera Multi-Person Tracking for EasyLiving. In: Proc. of the 3rd IEEE Intl. Workshop on Visual Surveillance (VS 2000). IEEE Computer Society, Washington (2000) 20. Brumitt, B., Meyers, B., Krumm, J., Kern, A., Shafer, S.A.: EasyLiving: Technologies for Intelligent Environments. In: Thomas, P., Gellersen, H.-W. (eds.) HUC 2000. LNCS, vol. 1927, pp. 12–29. Springer, Heidelberg (2000) 21. Chen, D., Gellersen, H.: Recognition and reasoning in an awareness support system for generation of storyboard-like views of recent activity. In: Proc. of the Intl. ACM SIGGROUP Conference on Supporting Group Work. GROUP 1999. ACM Press, New York (1999) 22. Aghajan, H., et al.: Distributed Vision-Based Accident Management for Assisted Living. In: Okadome, T., Yamazaki, T., Makhtari, M. (eds.) ICOST. LNCS, vol. 4541, pp. 196–205. Springer, Heidelberg (2007) 23. Gao, Y., Hui, S.C., Fong, A.C.: A Multi-View Facial Analysis Technique for Identity Authentication. IEEE Pervasive Computing 2(1), 38–45 (2003) 24. Hong, K., et al.: Real Time Face Detection and Recognition System Using Haar-Like Feature/HMM in Ubiquitous Network Environments. In: Gervasi, O., Gavrilova, M.L., Kumar, V., Lagan´ a, A., Lee, H.P., Mun, Y., Taniar, D., Tan, C.J.K. (eds.) ICCSA 2005. LNCS, vol. 3480, pp. 1154–1161. Springer, Heidelberg (2005) 25. Zhang, S., et al.: Continuous Verification Using Multimodal Biometrics. In: Zhang, D., Jain, A.K. (eds.) ICB 2005. LNCS, vol. 3832, pp. 562–570. Springer, Heidelberg (2005) 26. Hazen, T., Weinstein, E., Heisele, B., Park, A., Ming, J.: Multi-Modal Face and Speaker Identification for Mobile Devices. In: Hammoud, R.I., Abidi, B.R., Abidi, M.A. (eds.) Face Biometrics for Personal Identification: Multi-Sensory Multi-Modal Systems, pp. 123–138. Springer, Heidelberg (2006) 27. Bohn, J., Coroama, V., Langheinrich, M., Mattern, F., Rohs, M.: Social, Economic, and Ethical Implications of Ambient Intelligence and Ubiquitous Computing. In: Weber, W., Rabaey, J., Aarts, E. (eds.) Ambient Intelligence, pp. 5–29. Springer, Heidelberg (2005)

Biometrics Driven Smart Environments

89

28. Vildjiounaite, E., et al.: Unobtrusive Multimodal Biometrics for Ensuring Privacy and Information security with Personal Devices. In: Fishkin, K.P., Schiele, B., Nixon, P., Quigley, A. (eds.) PERVASIVE 2006. LNCS, vol. 3968, pp. 186–201. Springer, Heidelberg (2006) 29. Bernardin, K., Stiefelhagen, R.: Audio-visual multi-person tracking and identification for smart environments. In: Proc. of the 15th International Conference on Multimedia, pp. 661–670. ACM, New York (2007) 30. Hewitt, R., Belongie, S.: Active Learning in Face Recognition: Using Tracking to Build a Face Model. In: Proc. of the 2006 Computer Vision and Pattern Recognition Workshop, pp. 157–157. IEEE Computer Society, Washington (2006) 31. OpenCV, http://www.intel.com/technology/computing/opencv/index.htm 32. Turk, M., Pentland, A.: Eigenfaces for Recognition. J. Cognitive Neuroscience 3(1), 71–86 (1991)

A Structured Methodology of Scenario Generation and System Analysis for Ubiquitous Smart Space Development Ohbyung Kwon1 and Yonnim Lee2 1

School of International Management Kyunghee University Seochun, Ghiheung, Yongin, Kyunggi-do, South Korea [email protected] 2 Research Center for Ubiquitous Business and Services, Kyunghee University Seochun, Ghiheung, Yongin, Kyunggi-do, South Korea [email protected]

Abstract. Ubiquitous smart space (USS) has been regarded as a promising extension of ubiquitous services, and it is currently the subject of world-wide development. In one USS development methodology, scenario development is performed before system analysis and design. However, even though many redundant elements can be found between scenarios and system analysis results, developers have not been taking any structural approaches to join them together for more consistency and eventually higher productivity. Hence, the aim of this paper is to propose a methodology to increase the consistency in the early steps of USS development. To do so, scenario and requirement analysis are integrated in a structured manner. Keywords: ubiquitous smart space, scenario analysis, requirement analysis, structured method.

1 Introduction Advances in ubiquitous technologies have led to the development of many new types of ubiquitous services. Unlike earlier approaches, ubiquitous computing services are defined and used as services that are provided not just in physical spaces but also in logical spaces. This extension of service concept has drawn interest to USS (Ubiquitous Smart Space). USS is an artificial space that is added to ubiquitous computing technology. Recently ubiquitous computing services have developed in USS environment all over the world, for example: in the United States, there are several systems including HP’s Cool Town, Microsoft’s Easy Living, Cisco’s MIE, Intel’s Digital Home, Xerox’s Smart Media Spaces, NIST’s Smart Space, Georgia Tech’s e-Class, and MIT’s Auto-ID Center. In Europe, systems include EU’s 2Wear, Belfast’s MIME, Orange at Home of Orange, and Siemens’ Smart Home. In Japan, systems include the Ubiquitous ID Center’s Ubiquitous ID Technology, Matsushita’s eHII system, OMRON & ODAKYU’s GOOPAS, NTT’s Power Conversion System, and NTTDocomo’s Cmode. In Korea, F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 90–104, 2008. © Springer-Verlag Berlin Heidelberg 2008

A Structured Methodology of Scenario Generation and System Analysis

91

systems include ETRI’s u-Post, SKT’s Moneta, Samsung Electronics’ Digital Home, and CVnet’s Cyber Village. Most of these USSes are proposed by feasible service scenarios that involve various ubiquitous computing elements, technologies and products. In the first stage of development, they write a service scenario that shows a possible future lifestyle, and carries out a requirement analysis through that scenario. From this, the developers design and implement the USS. USS is totally different from the previous physical space so these services also deploy very differently. Therefore, the scenario method that is very widely used in future research has been recognized as an effective tool in developing future services. The scenario method was proven very useful for eliciting USS service idea [1]. Moreover, the scenario method is useful for explaining new ideas to another developer or to capture requirements for system implementation. Hence, the scenario method is widely used as a method to develop USS service [2]. Carnegie Mellon University’s Aura project was started with the goal of achieving invisible computing by Peter Steenkiste in 1999; they first used the scenario method and designed the Aura system to realize that scenario. However, to date, no standardized methodology exists for writing scenarios. Most scenario methods suggest simple guidelines without using all USS development phases. Therefore many scenarios have been developed by brainstorming and the contents have been decided by personal taste or by their organization’s preferences. Brainstorming is a group creativity technique designed to generate a large number of ideas for the solution to a problem. Although brainstorming is very useful for generating creative ideas, it is of course limited if there aren’t sufficient resources. Potential brainstorming problems can also include distraction, social loafing, evaluation apprehension, and production blocking. Existing scenario methods are not related to requirement analyses for software development, which can cause communication problems between scenario writer and requirement analyst. Requirements analysis for developing services has made progress independent of efforts for scenario writing. As a result, one-sided and subjective scenarios that were written without concerning the requirement analysis phase have created problems: double writing, inconsistency between scenario and requirement specification, waste of time and cost. Moreover, these types of scenarios make it difficult to objectively analyze and evaluate scenarios. It is also difficult to validate the degree of scenario reflection on requirements specifications since finding objective standards against which one can evaluate a subjective scenario is difficult. Therefore, this paper proposes the standardized and structured methodology of writing scenarios, and develops an integration methodology for requirement analysis that proceeds automatically and continuously from the written scenario. In this methodology, we focus on a technical-push scenario that is validating the utilitarian of the technology which is developed in the short term. This paper is organized in five sections. First we review previous research, and then Section 3 describes the scenario writing methodology and integration methodology for requirement analysis. A case study of our methodology is proposed in Sections 4. Finally, we provide concluding remarks in Section 5.

92

O. Kwon and Y. Lee

2 Scenario-Based Requirement Engineering Scenario-based requirements analysis methods express system requirements by using scenarios that were elicited through examples of real world experience and invented experiences [3,4]. Scenario-based requirements analysis methods can be simply divided into two methods: ScenIC and SCRAM [5,6]. ScenIC (Scenario in the Inquiry Cycle) proposes a schema of scenario-related knowledge composed of goals, objectives, tasks, obstacles, and actors [7]. Scenarios are composed of episodes and action carried out by actors, who are usually people but may also be machines. Goals are classified into achieving, maintaining, or avoiding states, while obstacles prevent goals from being achieved, or inhibit successful completion of tasks. The method proceeds in a cycle of expressing scenarios in a semistructured format, criticizing and inspecting scenarios in walkthroughs, which lead to refining requirements and specifications and the next cycle. Guidelines are given for formatting scenario narratives and identifying goals, actions and obstacles. Scenario episodes are assessed with challenges to see if goals can be achieved by the system tasks, whether the actors can carry out the tasks, whether obstacles prevent the actors carrying out the tasks, etc. Thus, dependencies between goals, tasks, actors and resources can be checked to make sure the system meets its requirements. Dependency analysis and means-ends analysis, in which tasks and the capabilities of actors are examined to ensure goals can be achieved. In SCRAM (Scenario-based Requirement Analysis Method) [8,9,10], scenarios are used with early prototypes to elicit requirements in reaction to a preliminary design. This approach does not explicitly cover modeling and specification, as this is assumed to progress in parallel, following the software engineering method of the designer’s choice. According to Alistair [5], the method consists of four phases: (1) Initial requirements capture and domain familiarization. This is conducted by conventional interviewing and fact-finding techniques to gain sufficient information to develop a first concept demonstrator; (2) Storyboarding and design visioning; (3) Requirements exploration; (4) Prototyping and requirements validation. This phase develops more fully functional prototypes and continues refining requirements until a prototype is agreed to acceptable by all the users. However, in the legacy scenario-based requirement analysis methods, requirement analysis is actually separated from scenario writing; they support system analysis only after writing scenarios completely.

3 Methodology Overview 3.1 Overall Framework To write scenarios and analyze requirements in USS development environment, an overall framework is proposed as shown in Fig. 1. There are two ways to write a scenario briefly. The first one is a guided approach. In this approach, we first analyze the environment, and then write scenarios based on the results. The other is an open-ended approach that writes scenarios freely without any environment analysis [24]. As previously stated, our research focuses on a technical-push scenario that validates the usefulness of the technology developed in the short term. Hence, we apply a guided approach because we aim to make our scenario

A Structured Methodology of Scenario Generation and System Analysis

DDefi ne efi ne the the Technol oogy Technol gy MMatri xx atri

DDefi ne efi ne the the SServi ce ervi ce MMatri xx atri

SSketch ketch oout ut the oo the SScenari cenari OOutl ine utl ine

93

GGenerate enerate the the SScenari oo cenari

DDraw raw the the SScenari oo cenari DDescri p oonn escriptiti DDiag iagram ram

GGenerate enerate the the RRequi rem equi rem ent ent SSppeci fi cati oonn eci fi cati

Fig. 1. Proposed methodology for generating scenario and requirement specification from technology and service matrix

practical to implement. In this paper, we define technologies, devices, and services that we can use in the scenario at present—this is the environment analysis phase. We then write the scenario based on the results. The scenario writer defines the technology matrix after collecting usable technologies and devices. Technology matrix means a sort of tables which shows summary of available technologies and corresponding devices for making services available. Through a combination of technology and devices in the defined technology matrix, we extract services that we can use in scenarios. We then define a service matrix, which describes in detail the content of each service extracted from a technology matrix, and evaluate its service level to decide if it is available to make scenarios. Based on this evaluation result, we confirm the service that we will use in the scenario. The scenario outline is then sketched out by a final confirmed service. After that, we draw the scenario diagram using the scenario outline and confirmed services. The scenario diagram is a technique that represents scenarios by predefined notations and processes. This diagram consists of a context diagram, a level-0 diagram and a level-1 diagram. Scenarios are generated by transforming a completed scenario diagram according to automatic transformation logic. Requirement specifications are written by transforming a generated scenario according to automatic transformation logic. Meanwhile, we create a scenario dictionary to improve understanding of the scenario reader and requirement specification writer. 3.2 Technology Matrix Definition First, the scenario writer collects technologies and devices that can be used in the scenario and describes brief explanation for each. Then, the scenarios writer may draw up a table which indicates a summary of those technologies and devices as shown in Table 1:

94

O. Kwon and Y. Lee Table 1. Technology Matrix Template

Category Available Technology Possible Device

Name

Explanation

Technology name

Brief explanation for the technology

Device name

Brief explanation for the device

3.3 Service Matrix Definition (1) Extract services Through a combination of technology and devices in the previously defined Technology Matrix, we extract services that we can use in scenarios. Extracted services would take concrete shape by the Service Matrix and be confirmed in the phase of service evaluation. Table 2 is the template for Service Extraction. Among these methodologies, we will focus on the former six steps from need identification to validation on service selection in this paper. Table 2. Service extraction template

Technology and Device Tech. 1

Tech. 2

{

Device 1

Device 3

{

{

Service

Device 4

Service name

Mark ‘{’ Tech. or Device used in the Service.

(2) Define service matrix Service matrix is a table to display in detail the contents of each extracted service. At this time, the functionality of the service that is expressed as an adjective or an adverb in service must be explained as detailed as possible. Table 3 is the template for the Service Matrix. Table 3. Service matrix template

Service

Description

Functionality Enabler

Function

A Structured Methodology of Scenario Generation and System Analysis

95

(3) Evaluate service level We evaluate the service level of each service based on the checklist. Then we review the result of evaluation with the people who request the scenario. 3.4 Scenario Outline Identifying the topic and background of scenario might proceed as follows. • • • •

Define the value that can be provided for end user through the services used in scenario. Identify stakeholders related to the scenario Define the main character of the scenario and his goal. If needed we can set up the obstacles and complication elements of the scenario. Describe the brief contents for scenario.

Table 4 is the template for identifying the theme and the background of scenario. Table 4. The template for identifying the theme and the background of scenario

Category Name Theme Stakeholder Abstract

Description The name of scenario The theme of scenario Stakeholders are people who affects, or can be affected by the scenario financially Abstract of the scenario

(2) Define the actors and assign their role We define the actors who carry out the service in the scenario and assign their role. An actor might be divided by its property (device, agent, ontology, u-Product). Table 5 is the template for defining the actors and assigning their role. 3.5 Scenario Definition Diagram We draw the scenario diagram based on the sketch of the scenario. This diagram is a technique that represents scenarios by predefined notations and processes. It consists Table 5. The template for defining the actors and assigning their role

Category Device Agent Ontology u-Product

Actor Actor name

Role Actor’s role

96

O. Kwon and Y. Lee Table 6. Components of the scenario definition diagram

Component

Notation

Sequence

Scene



Stakeholder

Data Flow

Data Store

Information Flow

Description It represents the service offered in scenario. One sequence has more than one service. It represents the actual service action that is performed by using the data. It has an actor who is the object of services and an action that is service contents. It represents the stakeholders are people who affects, or can be affected by, the scenario financially It represents the data flow among services or actor’s action. It represents the optional data flow among services or actor’s action. It represents the looped data flow among services or actor’s action. It represents the data store that saves the data. It could have a number of various physical shapes. It represents the information flow between scene and data store. When a scene requests specific information, the data store provides it.

of a context diagram, a level-0 diagram, and a level-1 diagram. Table 6 shows the components of the scenario diagram. (1) Draw the scenario context diagram The scenario context diagram represents the business model of scenario. This diagram explains the relationship among stakeholders who affect, or can be affected by, the scenario financially. Hence, the scope of service is decided through this diagram. The scenario context diagram is drawn like Fig. 2. (2) Draw the scenario level-0 diagram The level-0 diagram represents all services offered in a scenario, and it depicts information flow and the transforms that are applied as data moves from input to output. The scenario level-0 diagram is drawn as a series of steps and will be drawn as in Fig. 3.

A Structured Methodology of Scenario Generation and System Analysis

Stakeholder 1

Scenario name

Financial relation

97

Stakeholder 2

Financial relation

Financial relation Stakeholder 3 Fig. 2. Scenario contxext diagram Data 1. Service 1

Data 2. Service 2 Data

Fig. 3. Scenario level-0 diagram

(3) Draw the scenario level-1 diagram The Scenario Level-1 Diagram represents the actor and his action in each service. All actors have an Event, Action, and Response. In here, Event is the cause action leads to actor’s action and Response is the result action of the actor’s action. Table. 7 is the template for Action Matrix to organize these actors and actions. Scenario Level-1 Diagram is drawn like Fig. 4. Table 7. The template for action matrix

Actor Actor name

Event

Action

Response

The cause action leads to actor’s action

The actor’s own action

The result action of the actor’s action

98

O. Kwon and Y. Lee

Data flow

1.1 Actor 1

Data flow

1.2 Actor 2

Action1

Data flow

1.3

Data flow

Actor 3

Action2

Action3

Data flow

Data flow

Data Store Fig. 4. Scenario level-1 diagram

3.6 Scenario Generation We generate the scenario by transforming completed Level-1 Diagram according to automatic transformation logic on a per-Sequence basis. Since the transformation logics are highly dependent of language to be used, Korean language has been considered so far. 3.7 Generate Requirement Specifications In this paper, we use the UML that is a modeling language widely used for requirement analysis. Software system can be visualized, described, developed and created a document by the UML. Especially, we choose the Sequence Diagram among 4 models and 8 diagrams of UML that are OMG’s public standard. Sequence Diagram is drawn as follow steps. Step 1. On the basis of Level-1 Diagram, collect all data flows in scenario and give them serial number in order. And then rearrange each data flow, input and output as Table 8. Step 2. Placed all actors appeared in scenario horizontally by using predefined symbols of Fig. 5. And draw a vertical line started from each actor. Step 3. Draw all data flows arranged in Step 1 from Input Actor to Output Actor in order. At this time, represent data flow as an arrow along the vertical line. Table 8. Template for rearranging the data flow in scenario

No 1

Action

Actor

Next Actor

The actor’s own action

Actor who acts

Actor who is affected by the action

2

…

…

…

A Structured Methodology of Scenario Generation and System Analysis

99



Human

u-product

sensor

event handler agent

ontology

Fig. 5. Symbols for sequence diagram

4 Case Study This case study describes the ‘U-Shopping Aid’ scenario. We applied our methodology to this scenario. ‘U-Shopping Aid’ scenario was proposed to make customers Table 9. Technology matrix Category Technology

Device

Name Agent Technology Semantic Web

Description Offer the information of the product reputation and the payment through the negotiation among agents Provide semantic information through the data written by Ontology Language (OWL) Transfer data between Network AP of shop and u-PDA

uPAN Scalefree Ubiquitous Mobile Object (UMO) UMO

It is a mobile object or device that people have. It can be changed own functions depended on context and share computing resource with smart objects Smart object that customer has. It play an important role in receiving the service Smart display device established in shop or customer’s house Performing a recognizing role among devices

Smart Wall Or AR Table RFID Sensor

Table 10. Extract services Technology and Device uAgent USN PDA

Augmented Reality

Teleconference

Smart Wall

Smart Table

G

G

{G

{G

{G

G

G

{G

G

{G

G

G

{G

G

G

{G

{G

G

G

G

{G

G

G

{G

{G

G

G

G

G

G

G

{G

G

{G

G

Service Customer-aware and product recommendation Virtual put-on service Image sync service Payment service1 Payment service2

100

O. Kwon and Y. Lee Table 11. Service matrix Service name

Definition

Customer-aware and product recommendation

When customer enters, it is aware of that fact automatically. Recommend products that customer would like

Image sync service

Provide the virtual image that customer puts on the chosen clothes using the pictures saved in smart object

Payment service1

It enables to share images among remote spaces and to communicate with each other

Payment service1

When customer makes the decision about buying, it automatically transmits the payment information to the seller and delivery company simultaneously When customer makes the decision about buying, it automatically transmits the payment information to the seller simultaneously

Payment service2

Functionality Function Notify the user’s current location information Mall Agent Detect the embedded chip in UMO automatically Customer Store the user profile information Ontology Shop Agent Make recommendation lists according to the user preference Augmented Display virtual images reality device when the user sees the chosen clothes through AR device AR Table or Display made picture Smart Wall u-PDA Transmit shared images AR Table or Display shared images Smart Wall and transmit the selected information u-PDA Transmit payment information Enabler UMO

u-PDA

Transmit payment information

Table 12. Scenario actors and their roles

Category

Actor Shop Agent

Agent Mall Agent

Sensor

Chip sensor at the Mall entrance

Ontology

Customer info Ontology

Role It offers the optimized service through being aware of various customer need and manage the agent of shops. It is in the shopping mall. It is aware of customer’s context information and transmits it to Shop Agent. It is a RFID chip sensor at the shopping mall entrance. It detects the customer’s smart objects. It stores a customer profile.

A Structured Methodology of Scenario Generation and System Analysis

101

convenient for their shopping more practically. ‘U-Shopping Aid’ scenario consists of customer-aware and product recommendation services, virtual put-on service, image sync service and payment service. Let’s suppose that the Technology Matrix could be defined as Table 9. Then five services in Table 10 are extracted and Service Matrix is defined as listed in Table 11, which is based on Table 9. Then the actors and assign their role for our scenario is defined as shown in Table 12. Lastly, Scenario Context Diagram and Scenario Level-0 Diagram are prepared as Appendix A.

5 Conclusion Recently, USS developers are under increasing pressure due to the increased importance of making scenario and requirement analyses; hence the increasing need for structured methodologies. However, so far there has been no standardized methodology for writing scenarios. Most scenario methods suggest simple guidelines that do not take into consideration all USS development phases. Therefore many scenarios have been developed by brainstorming and the contents have been decided by personal taste or by the organization’s favor. Moreover, existing scenario methods do not relate to requirement analysis for software development. This fact creates communication problems between scenario writer and requirement analyst. It creates problems like double writing, inconsistency between scenario and requirement specification, waste of time and cost, and so on. These kinds of scenarios make it difficult to analyze and evaluate scenario objectively. Therefore, we propose the standardized and structured methodology of writing scenario and develop the integration methodology for requirement analysis that is proceeding from written scenario automatically and continuously. We are applying our methodology to the ‘U-Shopping Aid’ service development supported by a Korea governmental project.

Acknowledgment This research is partially supported by the ubiquitous Computing and Network (UCN) Project the Ministry of Information and Communication (MIC) 21st Century Frontier R&D Program.

References 1. Ranta, M., Asplund, H.: Utilizing scenarios in service idea generation- a case of distant participation for a seminar. In: Proceedings cf COST269 Conference, pp. 379–386 (2003) 2. Anton, A.I., Potts, C.: A Representational Framework for Scenarios of System Use. Requirements Eng. 3, 219–241 (1998) 3. Sutcliffe, A.: Scenario based requirement analysis. Requirements Engineering Journal, 48– 65 (1998)

102

O. Kwon and Y. Lee

4. Plihon, V., Ralyte, J., Benjamen, A., Maiden, N.A.M., Sutcliffe, A., Dubois, E., Heymans, P.: A Reuse Oriented Approach for the Construction of Scenario Based methods. In: Proc. Int’l Software process Assoc. Fifth Int’l Conf. Software Process (ICSP 1998), Chicago, pp. 14–17 (1998) 5. Sutcliffe, A.: Scenario-Based Requirements Engineering. In: Proceedings of the 11th IEEE International Conference, Montery Bay, USA, pp. 320–329. IEEE Computer Society Press, Los Alamitos (2003) 6. Misra, S., Kumar, V., Kumar, U.: Goal-oriented or scenario-based requirements engineering technique- What should a practitioner select? In: IEEE CCECE/CCGEI, Saskatoon, pp. 2290–2291 (2005) 7. Potts, C.: ScenIC: A Strategy for Inquiry-Driven Requirements Determination. In: 4th IEEE International Symposium on Requirements Engineering, pp. 58–65. IEEE Computer Society Press, Los Alamitos (1999) 8. Sutcliffe, A.G.: Scenario-Based Requirements Analysis. Requirements Engineering 3, 48– 65 (1998) 9. Sutcliffe, A.G.: User-Centred Requirements Engineering. Springer, London (2002) 10. Sutcliffe, A.G., Ryan, M.: Experience with SCRAM: A Scenario Requirements Analysis Method. In: IEEE International Symposium on Requirements Engineering: RE 1998, pp. 164–171. IEEE Computer Society Press, Los Alamitos (1998)

Appendix A. Example Results (1) Context Diagram COEX Mall

Payment Context-aware info Info & payment User (A)

U-shop service & delivery Service

Info & payment U-shop

U-Shopping Aid Customized i Purchasing info & payment Delivery Info

Distribution and delivery Service Provider

A Structured Methodology of Scenario Generation and System Analysis

(2) Scenario Level-0 Diagram [Scenario name] Aware & recommend

[Scenario name] Virtual Put On

[Scenario name] Image Sync

[Scenario name] Payment and delivery

103

104

O. Kwon and Y. Lee

(3) Data Flows in Scenario No. 1 2 3 4 5 6 7 8 9

18

19 20 21 22 23

Data flow A enters Send A’s context info.

Actor Chip sensor at Mall Entrance Mall Agent Customer Info. Ontology Mall Agent

Request for A’s info. Send A’s info. Send A’s Context info. Ask A’s info. Shop Agent Send A’s info. Customer Info. Ontology Provide product list to Shop Agent recommend Send a message to A’s UMO suggest the VPO service Scenes from 10 to 17 are omitted here. Display the created IS service device augmented reality to share with family The info. about the AR Table family choices Send the family A’s UMO choices Decision to purchase A and pay Info. about payment A’s UMO Info. about payment Shop Agent and delivery

Next_Actor Mall Agent Customer Info. Ontology Mall Agent Shop Agent Customer Info. Ontology Shop Agent A’s UMO A

AR Table

A’s UMO A A’s UMO Shop Agent A

Capturing Semantics for Information Security and Privacy Assurance Mohammad M.R. Chowdhury1 , Javier Chamizo2 , Josef Noll1 , and Juan Miguel G´ omez2 UniK-University Graduate Center Post Box 70, N-2027 Kjeller, Norway {mohammad, josef}@unik.no 2 Escuela Polit´ecnica Superior Universidad Carlos III de Madrid Avda. de la Universidad 30, Legan´es, Madrid, Spain [email protected], [email protected] 1

Abstract. Security and privacy assurance is indispensable for ubiquitous access to information and resources. This paper focuses on the security and privacy provisions in a restricted organizational environment through access control mechanism. It includes the representation of the semantics of an organization and its access control mechanism exploiting the Web Ontology Language. The system controls access to the resources of an organization through differential access privileges. These are formulated based on the roles of the individuals, and the projects and departments they belong to. Instead of explicit definitions, some additional facts of the mechanism are inferred by executing semantic rules using the Jess rule engine over the designed ontology. These information are then passed back to the ontology to enrich it. The ontology is designed to cope with the organization restructuring with minimal efforts.

1

Introduction

Ubiquitous computing and connectivity together with extensive diffusion of portable devices allow users to access information/resources/services anytime and anywhere even when they are on the move. However, these access scenarios demand security and privacy assurance which is not a trivial job in today’s increasingly connected but dynamic systems. In this regard, Professor Dr. Eugene Spafford said [1], The only truly secure system is one that is powered off, cast in a block of concrete and sealed in a lead-lined room with armed guards - and even then I have my doubts. This paper focuses on the security and privacy provisions in a restricted organizational environment through access control mechanisms. Access control in distributed and dynamic systems is crucial for secure service access. It is included F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 105–118, 2008. c Springer-Verlag Berlin Heidelberg 2008 

106

M.M.R. Chowdhury et al.

as one of the main sections in ISO/IEC 27000 series1 , an information security standard published by the ISO and the IEC. We believe that the capabilities of semantic technologies can contribute to mitigate these problems. The impact of Semantic Web technology is wide ranging. The Project10X (a consulting firm)2 study found that more than 190 companies including Adobe, Google, HP, Oracle and Sony are involved in developing Semantic Web based tools. But making it easier to comb through online data carries security implications. Among the challenges of security issues, policy-awareness and access control to Web resources play a major role, particularly given that these are two of the most significant requirements of information access and exchange. Design and maintenance of access control constraints in organizations are challenging problems as company structure, roles, user pools, security requirements are always changing. Conceptual organizational semantics and its access control mechanisms are fomally represented in an ontology using Web Ontology Language. The company system controls access to the resources of the organization by providing differential access privileges. In this ontology, these privileges are formulated based on the roles of the individuals. In addition, it considers which projects or departments they belong to. The proposed solution is designed to cope with the dynamic nature of an organization. This work is an extension of our previous work [2], where the concepts were so static that it could not reflect the organizational changes. This paper deals with a complex situation where an employee plays multiple roles across different departments and projects. The paper is organized as follows. The next section discusses the problem statements and our use case scenario. Section 3 briefly describes the Semantic Web technologies. The ontology representing organizational semantics are described in section 4. In Section 5, we illustrate the access control mechanism, the processes and results of inference based on the proposed ontology. In the next section, proposed solution is evaluated in the context of organizational restructuring. Section 7 contains overview of the related works and the paper concludes summarizing the paper and briefly stating the future works.

2

Problem Statement: Information Security and Privacy Assurance

Nowadays people in business organizations increasingly work in project oriented environments. Some of the projects deal with company sensitive information which should not be leaked to unauthorized employees. The project members come from different departments. They don’t enjoy the same rights or privileges within a project environment. This is more prevalent while accessing resources owned by the departments or projects. There are situations where a person 1

2

ISO 27002 - The Information Security Standard, http://www.standardsdirect.org/iso17799.htm [accessed on Jan. 4, 2008] The Project10X Special Report, http://www.semantic conference.com/semanticwave.html

Capturing Semantics for Information Security and Privacy Assurance

107

holds multiple roles and privileges. Employees with different levels of privileges are expected to access resources through the Intranet or Internet. Fig. 1 illustrates the use case scenario which describes a specific organizational environment. It deals with the roles like employee (department in general), supervisor, project leader (PL) and project member (PM). Telenor R&I (Department A) and its planning department (Department B) both involve in project release 7 and 8. Release 9 only resides in planning department. Each of the departments and projects has its own resources. A person has multiple roles like Josef Noll is not only a supervisor (Department A) and project leader (Release 9) but also a project member (Release 7). He should have access to corresponding department and project resources where he involves in. So, access rights depend on one’s role into the respective departments and projects. Following are some of the examples of restricted access scenarios, 1. Supervisor is the head of a department. Department owns some resources (administrative resources, documents, deliverables). He can read and write the documents. He can edit its administrative resources and give final approval to the deliverables. Supervisor can also monitor status of department’s employees who work in different projects. 2. Department’s employees will have only read and write privileges to its documents. 3. Departments participate in different projects. Project leader leads a project. He can read and write project’s documents. He can edit its administrative resources and give final approval to the project deliverables. 4. Besides leader, projects have members. They can only read and write project’s documents. Therefore, the architecture manages access to resources not only based on the roles but also based on the involvement in organizational divisions (departments, projects). Access scenarios of the use case are described in table 1.

Fig. 1. The use case scenario

108

M.M.R. Chowdhury et al. Table 1. Roles and privileges to access corresponding resources Employee

Role

Josef Noll

Supervisor

Project Leader

Hans Christian

Project Member Supervisor

Project Leader

3

George Kalman

Employee Project Member

Erik Swansson

Employee Project Member

Privilege

Access to Resources

Administrator Admin. Dept.A Final Approval Deliverables Dept.A Read Write Documents Dept.A Administrator Admin. Rel9 Final Approval Deliverables Rel9 Read Write Document Rel9 Read Write Documents Rel7 Administrator Admin. Dept.B Final Approval Deliverables Dept.B Read Write Documents Dept.B Administrator Admin. Rel7&Rel8 Final Approval Deliverables Rel7 &Rel8 Read Write Documents Rel7 &Rel8 Read Write Documents Dept. A Read Write Documents Rel8 Documents Rel9 Read Write Documents Dept. A Read Write Documents Rel8

Representation of Organizational Semantics

We advocate that access control solutions should adopt semantic technologies as the key building blocks for supporting expressive representation of organizational semantics and reasoning. This section briefly describes the technologies and clarifies our claims. 3.1

Using Ontology

A common format of information and data representation will likely never be achieved. Efficient management of information and data is possible by capturing the common and understandable meaning of them formally and unambiguously [16]. Ontologies [15] are the cornerstone technology of Semantic Web, providing structured vocabularies that describe a formal specification of a shared conceptualization. It is used to capture the knowledge about a domain of interest in the form of concepts and their relationships. It permits the description of organizational structures, roles, privileges and resources at different levels of abstraction and support reasoning about both the structure and the properties of the elements that constitute the system. We believe that designing a consistent ontology based on a sound conceptual foundation is worthwhile because it can be reused in various organizations to control access to their resources.

Capturing Semantics for Information Security and Privacy Assurance

3.2

109

Introduction to the Web Ontology Language

Among the different ontology languages, we are focusing on the Web Ontology Language (OWL3 ) suggested by the World Wide Web Consortium (W3C). It is a markup language that builds on RDF4 and RDF Schema. OWL is chosen because it provides more vocabularies for describing concepts and properties (e.g. relations between concepts, cardinality, equality, richer typing of properties, etc) than that supported by XML, RDF, and RDFS. There are three species of OWL: OWL Lite, OWL DL and OWL Full and these are designed to be layered according to their increasing expressiveness. 3.3

Description Logics for OWL

Between the three different sub-languages OWL offers, we decided to use OWL DL. It is based on Description Logics (hence the suffix DL). These are the decidable part of First Order Logic5 and are therefore amenable to automated reasoning. Though OWL DL lacks in expressivity power compared with OWL Full, it maintains decidability6 and regains computational efficiency. The computational efficiency is an important feature since it is expected to support scores of relations. The mechanism is supposed to evaluate and grant permissions to access resources, it seems necessary to add reasoning support with it. In order to achieve more expressivity and decidability, we use Semantic Web Rule Language (section 5.1) which is designed as an extension of OWL DL.

4

Descriptions of Organizational Semantics

We assumed that company employees are already authenticated to the system through some secure means. Ontology models the organizational structures described in section 2. 4.1

Defining Concepts through Classes

Fig. 2 illustrates the proposed ontology in conceptual level. OWL classes are the concrete representations of concepts. In the proposed ontology, Identity class defines the identities of the company employees. We specify Company, Department and Project as subclasses of Work Unit in order to avoid defining explicit relationships between department/project and roles. We follow the set theory (eq. 1) while defining the class hierarchy of Role considering the fact that supervisor of 3 4

5 6

OWL Overview: http://www.w3.org/TR/owl-features/ RDF builds on URI and XML technologies. The specifications provide a lightweight ontology system. First Order Logic (FOL), http://en.wikipedia.org/wiki/First-order logic Logics are decidable if computations/algorithms based on the logic will terminate in a finite time.

110

M.M.R. Chowdhury et al.

Fig. 2. The ontology: class, property, instance

a department is also an employee of it. The same is true for project leader and member. Class hierarchy is a critical issue in inheritance of properties. {Supervisor, P rojectLeader, P rojectM ember} ⊆ Employee ⊆ Role

(1)

We divide the resources into subclasses, administrator resources and web resources. Web resources are further divided into documents and deliverables. These resources are related to appropriate privileges. Privileges are designed in accordance with the individual’s roles in the organization. In this paper, role is the positional hierarchy of employees. Through this, individuals are restricted to access correct resources. As for example, administrative resources are related to the Admin Privilege to ensure that only the roles having administrative privilege can access these. 4.2

Realizing Concepts through Instances

OWL classes are interpreted as sets that contain instances or individuals. Fig. 2 illustrates the instances of classes in ellipses. Instances of the identities are defined here simply as their names. Admin, Final approval and Read write instances of privilege are added. But new instances can be added (section 6) whenever necessary. Instances of the resources are added which correspond to the resources owned by the departments or projects. Instances to the subclasses of Role are added in accordance with the individual roles to realize multiple roles of a person. As for example, Sup Josef instance of Supervisor corresponds to the supervisor role of Josef Noll. Similarly, PM Josef corresponds to the project member role of Josef Noll.

Capturing Semantics for Information Security and Privacy Assurance

4.3

111

Defining Relations through Properties

Properties are the binary relations between the two things, more specifically between the instances of classes. A property relates instances from the domain with the instances from the range. Syntactically, domain links a property to a class and range links a property to either a class or a data range. Due to the class hierarchy (eq. 1) and domain and range specifications, subclasses inherit the relationships between the respective classes. Fig. 2 also provides the properties and their relationships with classes. rolePlaysIn property specifies the fact that in which specific Work Unit (departments/projects) a Role instance plays its role. Ontology is supposed to answer who can see/access which resources and hasVisibilityOf property defines this situation. isSupervisorOf explains who is the supervisor of whom in a department. The relationships of hasVisibilityOf, isSupervisorOf are not defined explicitly. These relationships of the properties are filled in through the inference process.

5

Access Control by Enhancing the Expressivity of OWL

Access control is achieved by enhancing the expressivity of OWL through the inference process. This section describes the access control logic and expressivity needs, making a review of the language used and its benefit to the goals pursued. 5.1

Introduction to the Semantic Web Rule Language

The expressivity provided by the OWL is limited by tree like structures [17]. This means that knowledge cannot be inferred from indirect relations between the entities, however the solution spends most part of his power in inferring indirect relationships that will determine whether a subject has access to a resource or not and which are its privileges over it. Hierarchical structures as defined before and inherent relationships between working units and hierarchies of resources are a perfect field where inference can extract these knowledge. We did inference through the rule support over the ontology. and used Semantic Web Rule Language (SWRL7 ) which pretends to be a complimentary feature of OWL. SWRL is roughly the union of Horn Logic and OWL. As any Horn Logic based language, rules are defined as a set of precedent and consequent states. 5.2

Inference Results

Objects of the properties, hasVisibilityOf and isSupervisorOf are filled in through the inferred knowledge from executing the rules. We use Jess rule engine to run the rules. First, OWL ontology and SWRL rules are transferred to Jess. Running the engine then initiates the inference process, generates knowledge as Jess facts. This inferred knowledge can be used by the external interface or can optionally be passed back to the ontology to enrich it. All these actions are user-driven. Rules are formulated using the SWRL as follows, 7

The Semantic Web Rule Language, http://www.w3.org/Submission/SWRL/

112

M.M.R. Chowdhury et al.

– Rule1: Over which resource a Role has Visibility/Access? Employee(?Em) ∧ roleP laysIn(?Em, ?X) ∧ hasP rivilege(?Em, ?Y ) ∧ belongsT o(?Z, ?X) ∧ needP rivilege(?Z, ?Y ) −→ hasV isibilityOf (?Em, ?Z)

– Rule2: Who is supervisor of whom? Dept Employee(?DepEm) ∧ hasRole(?Y, ?DepEm) ∧ Department(?Dep) ∧ roleP laysIn(?DepEm, ?Dep) ∧ Corporate Identity(?ID) ∧ Supervisor(?Sup) ∧ hasRole(?ID, ?Sup) ∧ roleP laysIn(?Sup, ?Dep) −→ isSupervisorOf (?ID, ?Y )

Fig. 3 and 4 illustrates the inference results of rules execution. All the relationships in the ontology have not been explicitly defined. As for example, hasVisibilityOf relationship of project leader Hans (PL Hans) has not been defined (circled in fig. 3). The figure displays that 25 relationships are inferred, which answers the resources over which the roles have the required visibility/access (Rule 1). Our investigation shows that the knowledge are inferred as expected. The inference results are exported back to the ontology to fill these empty relationships (fig. 3 shows that 25 relationships are transferred back to the OWL knowledge). From Tab. 1, it is evident that Josef Noll is the supervisor of George Kalman, which was not explicitly defined. Execution of Rule 2 shows the similar result (Fig. 4). This can be used in a special situation, when supervisor wants to check the status of an employee of his department to a project where he is not involved.

6

Evaluation

The proposed solution is more maintainable since there exist a general schema for any organization that can easily be adapted, thanks to its expressivity capacity nearer to human understanding. Expressivity and inference capacities avoid the inclusion of redundant information in the ontology. The proposed ontology can reflect the organization changes of a company with minimum efforts. Now, we are going to describe a situation of this. A new department, Audit has been created in the company. Roman Stanek and Peter Johansson has joined as the supervisor and employee (auditor) of the department respectively. The department has budgets, and Peter audits the budgets. Audit department prepares the audit reports of company. It is expected that auditor (Peter) as well as supervisors of the departments can only check department’s budget. Roman and Peter only have access to the company audit reports that belong to the Audit department only. As a supervisor of Audit department, Roman can give final approval to these. The corresponding actions to reflect these changes into the ontology are described in the following points. Through this, we are going to evaluate the strength of the proposed ontology. – Add new identity instances: Roman Stanek and Peter Johansson. – Add new department instance: Audit

Capturing Semantics for Information Security and Privacy Assurance

Fig. 3. Inferred results executing Rule 1

Fig. 4. Inferred results executing Rule 2

113

114

M.M.R. Chowdhury et al.

Fig. 5. Inferred relationships when Rule 1 is executed

Fig. 6. Inferred relationships when Rule 2 is executed

– Add new subclass of Resource: Budget & Audit. It contains three instances: audit report of company (Audit Company), BudgetDept.A and BudgetDept.B. A new subclass of resource is created because it needs a different privilege to support its privacy requirements. Besides, the department contains an administration resource (AdminResAuditDept). – Add new subclass: Auditor within the class tree: Role-CompanyEmployee. Add Auditor Peter instance of it. – Add new instance of Supervisor role: Sup Stanek.

Capturing Semantics for Information Security and Privacy Assurance

115

– Add a privilege instance Admin Budget to ensure only auditor and supervisor have access to budget. This instance is related to Auditor Peter and Budget Dept.A & Budget Dept.B. – Fill up all the corresponding relationships. – If we execute Rule 1, new relationships are inferred with hasVisibilityOf property. These are exported back to the ontology to fill in (circled in the figure) the empty relationships. Fig. 5 shows these new relationships for instance of auditor role Auditor Peter. As expected, Peter can only have access to company audit reports and Budgets of Dept.A and B. Similarly, fig. 6 shows the fact that Roman Stanek is the supervisor of Peter Johansson. It requires only few changes in the ontology. Among these changes, only two new subclasses have to be created. Otherwise, all the remaining additions are in the instances which is quite apparent. Conceptually, proposed ontology can follow the organizational changes. One of the ways of simplifying access rights management is to store the access control matrix using access control list (ACL). Though ACLs are simple to implement, its management steps are quite tedious especially when it is required to revoke somebody’s privilege. The proposed ontology provides a simple mechanism to revoke somebody’s role or privilege. One can simply delete the relationship between the role instance and the privilege instance to withdraw the privilege and afterward can delete the corresponding role instance to repeal the subject’s role entirely. Among the few disadvantages, it is computationally expensive and decidability not guaranteed by SWRL. In addition, XML processing is slower than that of database. But ontologies can be stored in the relational databases or even be mapped from them.

7

Related Works

The significance of adding privacy-enhancing technologies (PET) in virtual community networks is overwhelming [5], [6]. Information security and privacy requirements are more evident in a business environment. These issues are handled in this paper through access control mechanisms in the context of project oriented corporate working environment. We considered the concept of Role Based Access Control (RBAC) as a part of our access control mechanism. Sandhu introduced RBAC in [4] with early multiuser computer systems, where users’ access permissions are associated with roles, and users are made members of appropriate roles. A major advantage of RBAC is its ability to constraint access based on the concept of separation of duties which significantly simplifies the management of permissions. In the proposed solution, access control also depends on how the organizational structures. There are two types of RBAC constraints: dynamic and static. Authors in [3] described an approach of RBAC with dynamic constraints through an automated reasoning technique. Though we focused on static constraints on roles, rules were included within the ontology to infer new knowledge which can be passed back to the ontology. Through this verification of access control constraints defined

116

M.M.R. Chowdhury et al.

in the ontology are also achieved. The proposed solution can also adapt to ever changing company structure with less effort. Ubiquitous connectivity not only permits users to access resources anytime and anywhere but also complicating its control due to user/device mobility. To integrate this pervasive behavior, a context-aware access control framework has been developed for secure collaborations in pervasive computing environments [18]. It followed two main design principles: context-awareness to control resource access and to enable dynamic adaptation of policies based on context, and the use of OWL DL for context/policy specification. The authors considered location and time of the meeting as the context information collected dynamically. We used static contexts like, which departments/projects someone is involved though these were predefined. Among the access control technologies, ACL is widely used. The semantics of ACLs have been proved to be insecure in many situations. Instead of maintaining a centralized ACL, a trust based group has been proposed to delegate access rights in [7], [8] where FOAF (friend of a friend) [9] based social networks acted as a mean for the delegation. A private key based signature scheme was proposed to ensure the privacy of networks and users. But it requires secure distribution and maintenance of keys. A similar concept of trust or reputation has also been used by [10] to create and access communities. Distributed trust management approach is considered as one of the main components to secure the Semantic Web [11]. They intended to provide access to community resources and privacy solutions only by means of trust/reputation management. But access to sensitive business resources based on trust mechanism does not provide adequate security in business contexts. In addition, trust is affected by various factors and therefore difficult to quantify. In [12], authors presented an approach to reduce the inefficiencies of the management (coordination, verification and validation, and enforcement) of many role-based access control policies and mechanisms using OWL. They focused on the representation of XACML (eXtensible Access Control Markup Language) policies in DL. In [13], Kolovski introduces an algorithm for the translation of a subset of XACML into DL with the goal of offering relevant analysis services using an OWL DL reasoner, Pellet[14]. In this paper, we also formalize the organizational semantics, roles and access privileges using OWL DL. Finini in his paper [11] also proposed using OWL for constructing ontologies which define policies/privileges.

8

Conclusions

In this paper, we addressed the security and privacy challenges of project-based organizations through access control mechanism exploiting Semantic Web technologies. In this regard, we developed an ontology to represent the conceptual structure of an organization and the roles of individuals. In the ontology, all the relationships between the entities have not been defined explicitly. Semantic rules facilitated expressing these additional knowledge. Jess rule engine executed

Capturing Semantics for Information Security and Privacy Assurance

117

these rules and new facts were transferred back to the ontology to enrich it and check its validity. Apart from these, we evaluated the inference capabilities of the proposed solution by restructuring the organization. As the proposed solution is based on centralized architecture, the future architecture should consider the scalability issues of it especially for a system of big enterprise. Our ultimate goal is to integrate this solution with a web application which controls access to resources.

References 1. Spafford, E.H.: director of the Purdue Center for Education and Research in Information Assurance and Security, Selected Quotes [accessed on January 4, 2007], http://homes.cerias.purdue.edu/∼ spaf/quotes.html 2. Chowdhury, M.M.R., Noll, J., Gomez, J.M.: Enabling Access Control and Privacy through Ontology. In: 4th International Conference on Innovations in Information Technology (Innovations 2007), Dubai, UAE (2007) 3. Dury, A., Boroday, S., Petrenko, A., Lotz, V.: Formal Verification of Business Workflows and Role Based Access Control Systems. In: International Conference on Emerging Security Information, Systems and Technologies (SECUREWARE 2007), Valencia, Spain (2007) 4. Sandhu, R.S., Coyne, E.J., Feinstein, H.L., Youman, C.E.: Role-Based Access Control Models. IEEE Computer 29(2), 38–47 (1996) 5. Chewar, C.M., McCrickard, D.S., Carroll, J.M.: Persistent virtual identity in community networks: Impact to social capital value chains. Technical Report TR-03-01, Computer Science, Virginia Tech (2003) 6. Walters, G.J.: Privacy and Security: An Ethical Analysis. Computers and Society, 8–23 (2001) 7. Kruk, S.R., Grzonkowski, S., Gzella, A., Woroniecki, T., Choi, H.-C.: D-FOAF: Distributed Identity Management with Access Roghts Delegation. In: 1st Asian Semantic Web Conference, Beijing, China (2006) 8. Kruk, S.R., Gzella, A., Grzonkowski, S.: D-FOAF Distributed Identity Management based on Social Networks. In: Demo session of ESWC 2006 (2006) 9. FOAFRealm project, http://www.foafrealm.org/ 10. Choi, H.-C., Kruk, S.R., Grzonkowski, S., Stankiewicz, K., Davis, B., Breslin, J.G.: Trust Models for Community-Aware Identity Management. Identity. In: Reference and the Web IRW 2006, WWW 2006 Workshop, Scotland, May 23 (2006) 11. Finin, T., Joshi, A.: Agents, Trust, and Information Access on the Semantic Web. ACM SIGMOD 31(4), 30–35 (2002), Special Issue: Special section on semantic web and data management 12. Smith, M.A., Schain, A.J., Clark, K.G., Griffey, A., Kolovski, V.: Mother, May I? In: OWL-based Policy Management at NASA European Semantic Web Conference 2007, ESWC 2007 (2007) 13. Kolovski, V., Hendler, J., Parsia, B.: Analyzing Web Access Control Policies. In: 16th International World Wide Web Conference, WWW 2007, Alberta, Canada, May 8-12 (2007) 14. Pellet, an OWL DL reasoner, http://pellet.owldl.com/

118

M.M.R. Chowdhury et al.

15. Fensel, D.: Ontologies: A Silver Bullet for Knowledge Management and Electronic Commerce. Springer, Heidelberg (2001) 16. Berners-Lee, T., Hendler, J., Lassila, O.: The Semantic Web. Scientific American (May 2001) 17. Motik, B., Sattler, U., Studer, R.: Query Answering for OWL-DL with Rules. In: International Semantic Web Conference 2004, pp. 549–563. Springer, Heidelberg (2004) 18. Toninelli, A., Montanari, R., Kagal, L., Lassila, O.: Semantic Context-Aware Access Control Framework for Secure Collaborations in Pervasive Computing Environments. In: International Semantic Web Conference 2006. LNCS, pp. 473–486. Springer, Heidelberg (2006)

A Framework for Context-Aware Home-Health Monitoring Alessandra Esposito, Luciano Tarricone, Marco Zappatore, Luca Catarinucci, Riccardo Colella, and Angelo DiBari University of Salento, Lecce, Italy {alessandra.esposito,luciano.tarricone, luca.catarinucci}@unile.it

Abstract. This paper presents a proposal for a context-aware framework. The framework is organized according to a general purpose architecture, centred around an ontological context representation. The ontology provides the vocabulary upon which software agents interoperate and perform rule-based reasoning, in order to determine the system response to context changes. The system components and their coordinated operations are described by providing a simple example of concrete application in a home-care scenario. Keywords: context-aware, ontology, rule, logic, agent, ubiquitous, health.

1 Introduction Progress in mobile devices, wireless networks and software technologies is bringing healthcare a new generation of systems, which are known under the name of pervasive and context-aware systems. The term ‘pervasive’ [1] refers to the seamless integration of devices into the user’s everyday life. Appliances vanish into the background to make the user and his tasks the central focus rather than computing devices and technical issues. Context-aware systems adapt their operations to the current context without explicit user intervention. These kinds of applications pose several technological challenges. First of all, a pervasive application is made up of heterogeneous and dispersed components which must be able to interoperate in a transparent manner. Moreover, in order to adapt to context variations, the system must elaborate raw data sensed by context sources, such as sensors, and extract high level context information from them. Both requirements lead to the identification of the core component of the system: a robust, reusable and sharable context representation. However, the definition of a shared context model provides the basis for 1) allowing the system components to cooperate and build together the system reaction to incoming events, 2) structuring sensed data in a meaningful and interpretable form. Currently, there is no standard modelling option for context [2]. Therefore, available frameworks adopt proprietary approaches. Among them, ontology-based ones [3,4,5] are more and more recognized as the most promising [6], as they provide a rich formalism for specifying contextual information and are reusable and sharable. The adoption of ontologies is encouraged by their capability of being embedded within multi-agent and rule-based systems. Indeed, designing the application in a multi-agent fashion allows one to organize it around the F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 119–130, 2008. © Springer-Verlag Berlin Heidelberg 2008

120

A. Esposito et al.

coordinated interaction of autonomous reasoning entities, which wrap the “natural” components of the pervasive system (such as sensing devices and consumer services). Rule-based logic supports agents in implementing advanced reasoning and in deriving high-level concepts from sensed information thus opening the application to sophisticated adaptation and pro-active behaviour. This work proposes a framework for context-aware pervasive systems built around the three above mentioned technologies: ontology representation, multi-agent paradigm and rule-based logic. The amenability of these technologies for contextaware pervasive environments has been demonstrated in a number of recent works. Among them, we recall CoBrA [5], an agent-based framework for intelligent meeting rooms centred around an OWL ontology. Gu et al. [6] propose SOCAM which is an ontology-based middleware for context-aware services in smart space domains. An ontology-based context model and context reasoning for ubiquitous health-care is presented in [4]. Health-care is the focus of Paganelli and Giuli [3] work as well, which describes a system built around OWL and rule-based inference. A further impulse to the integration of the three technologies is provided with our work, which is the result of an effort to enhance their interoperability. As a result, the ontology provides the knowledge codification needed to support both agent reasoning and communication. The effectiveness of the adopted model is shown by using a simple and concrete example in a home-care scenario. The paper is organized as follows. Section 2 introduces the general-purpose framework and its prototypal implementation. Section 3 is centered around context modeling. First the ontology representation, then the rule-based reasoning are illustrated. Section 4 focuses on how agent-based reasoning supports context elaboration. Section 5 focuses on data collection strategies. It describes “S-tag”, a novel low cost device enabling the integration of sensor networks with RFID systems. A home-health scenario is adopted throughout the entire paper as specific example of application and practical result.

2 System Architecture The architecture of our system (Fig. 1) follows a widely accepted abstraction [7], according to which context-aware systems are organized into three layers: context sources. Context management middleware and context consumer level Context sources include entities providing raw context data. They are conceptually partitioned into two groups: physical and virtual sources [8]. Physical sources include all hardware devices able to sense context data, such as RFID, sensors, positioning systems, etc. Virtual sources include software services able to gather context data, such as GUIs for user preferences input, databases, etc. Such data must be elaborated in an “intelligent” manner, so that the overall system reacts properly to context changes. This requires the availability of a machine-interpretable representation of context and of software components (agents) able to suitably process such knowledge. Both of them are conceptually situated at the intermediate level of the system architecture, the so-called middleware layer, as they provide the core building blocks of a context-aware application. Agents interoperate with one another thanks to the availability of a unified model of reality. Their behaviour is strongly influenced by

A Framework for Context-Aware Home-Health Monitoring

121

data provided by context sources and substantially determines the activities to be performed at the highest layer. Indeed, the context consumer layer includes all the entities, such as mobiles, Web interfaces, laptops, which interact with final users in response to meaningful context changes, thus determining the behaviour of the context-aware application as a whole.

Context Consumer GUIs

Applications

Mobiles

PDAs

PCs

…

Context Management Middleware Context Model

Inference Engine

JADE

APIs

…

Context Sources Physical Sensors

Virtual Sensors

Fig. 1. The three-layer system architecture. The lowest layer includes physical and virtual sensors, i.e. devices and software entities forwarding context data. The middle level hosts middleware components, such as software development kits and APIs. The context model is the core of the system and belongs to the middle level as well. End user devices and applications are positioned at the highest level.

The prototypal implementation of the framework is drawn in Fig.2. As shown in the figure, the distributed system is made up of a team of interoperating agents, which share a common knowledge representation and reason through an inference engine. The agents are dispersed over several nodes and play different roles. As explained in Section 4, Context Provider Agents (CPA) wrap context sources and are in charge of converting raw context data into high level context information. CPAs cooperate with Context Interpreter Agents (CIA) which are responsible for managing high level context information and of identifying the set of actions to be triggered as a consequence of an emergence. Finally, Context Consumer Agents (CCA) forward the message/signal describing the alarm situation to the most suited destination. Input data are provided by a physical context source obtained by integrating sensors with an RFID device (see Section 5) and by a virtual context source containing static information. In the following sections, the system components and their way of operating are described, with reference to a home-care scenario.

122

A. Esposito et al.

S-tag

CPA

CN3

CPA

CPA Agent Inference Engine

CCA

CN1

CIA CN2

Agent Behaviours Fig. 2. System Prototype. The system was implemented on three nodes (CN) connected by a local area network. It follows a multi-agent paradigm. Context Provider Agents (CPA) filter and integrate data coming from physical and virtual sensors, thus converting raw context data into high level context. Context Interpreter Agents (CIA) process high level context information to identify the actions and the actors best suited for managing an emergency. Context Consumer Agents (CCA) forward the alarm information to the final destination.

3 Context Representation An application is context-aware if it is able to adapt its behaviour to context by suitably reacting to context variations. In other terms, a context-aware application must exhibit a sort of “situation awareness”, i.e. it must manage and interpret context and its real-time evolution. Moreover, a context-aware application is generally made up of several components, possibly deployed on dispersed nodes, which need to interact with one another. Therefore it is fundamental for such components to share a common representation of context knowledge. These premises cast a twofold approach to context modelling. First of all, the fundamental entities, objects and relations of a situation must be represented. This is needed in order to ground system knowledge on a common basis and provide a unified concept taxonomy to the several components interacting in the application framework. Secondly, context changes originated by meaningful data variations must originate recognizable “high-level” events, so that situation switching can be interpreted and suitably managed by the system. In other terms, the system must be enabled to deduce high-level, implicit context from the low-level, explicit context directly acquired from sensors. This operation often requires reasoning over complex combinations of different data and context information. For example, an

A Framework for Context-Aware Home-Health Monitoring

123

increase of blood pressure (low level context) exceeding the patient threshold (low level context) may produce a switching to an “alarm” situation (high level context), which, on its turn may produce a series of operations finalized at identifying and invoking the most suited available and reachable doctor. Our system attacks the former need, i.e. machine representation of context entities, by the use of ontologies which enable to describe meaningful events, objects and relations of context and support several fundamental forms of reasoning, such as concept satisfiability, class subsumption, consistency and instance checking. The latter need, situation switching management, is approached by implementing logical rules and by embedding context events into facts. In this way, an iterative process is activated every time rules are matched. Indeed, fired rules infer new facts, i.e. new scenario switchings, which on their turn may fire other rules. This process allows the system to convert low level context into high level context, with the final result of producing the system reaction to the context switching which originated the process. For example the occurrence of low level events sensed by physical and virtual sensors, may originate the “diastolic blood pressure of patient X=100” and “threshold of patient X=90” facts. These facts may fire a rule inferring the high level fact “alarm for patient X=true”. The alarm situation, combined with facts concerning the availability of doctors, may produce the fact “doctor Y has to be called”, which generates the system response to the detected patient anomaly. As explained in the following subsections, the ontologies were implemented in the OWL language, whilst the rule-based domain knowledge was implemented with Jess on top of OWL ontologies. 3.1 Context Ontology Several context modeling techniques [9] exist such as key value, mark-up scheme, graphical, object-oriented, and ontology-based modeling. According to [9], the ontology-based approach fits well with the requirements of ubiquitous/context-aware computing. Indeed. ontologies facilitate knowledge sharing and reuse. Knowledge sharing is enabled by providing a common knowledge model to computational entities, such as agents and services, which need to interoperate with one another. Knowledge reuse is promoted by ontology amenability to be extended to different domains and to be integrated within wider ontology-based frameworks. Many ontology languages exist including Resource Description Framework Schema (RDFS) [10], DAML+OIL [11], and OWL [12]. OWL is a key to the Semantic Web and was proposed by the Web Ontology Working Group of W3C. Therefore, it was chosen for our prototype. A common practise, when developing ontologies, is to adopt a top level (upper) shared conceptualization [13] on top of which domain ontologies are built. Top level ontologies codify general terms which are independent of a particular problem or domain. Once the top level ontology is available, several lower level ontologies can be introduced, with the scope of incrementally specializing concepts from the high level generic point of view of the upper ontology to the low level practical point of view of the application. This way of structuring knowledge promotes sharing and reuse of ontologies in different application domains.

124

A. Esposito et al.

ContextualEntity Person

Device hasDevice MeasuringDevice

Patient hasPhysiological Parameter PhysiologicalParameter BloodPressure hasCurrentSBPValue hasNormalMaxSBPValue Upper Context Ontology Class Context Ontology Class

Sensor WearableSensor Sphygmomanometer measuresBloodPressure

Is-A Relationship Object Property Datatype Property

Fig. 3. Some classes from the context domain ontology referring to the “patient environment”

Therefore, we divided context ontology into two levels: the top-level context and the domain context ontology. The top-level context ontology includes concepts which refer to context-aware computing, independently from the application domain. The domain context ontology refers explicitly to the health-care domain. The vocabulary related to context entities was defined starting from the widely accepted definition of context, provided in [14]: “Context is any information that can be used to characterize the situation of an entity.” Therefore, an entity is a person, place, computational entity, or object which is considered relevant for determining the behavior of an application. Therefore, as shown in Fig.3 and Fig.4, “person”, “device” and “TriggeredAction” are contextual entities which specialize into different con?epts depending on the context subdomain they belong to. For instance, a person can be a patient, a relative of the patient or a health operator. Devices can be of different types as well. For the sake of brevity, the figures show just small ontology fragments referring to the example used throughout the remaining part of the paper. 3.2 Context Rules The proposed framework utilizes Jess [15] to structure knowledge in the form of declarative rules. Jess is a widely adopted rule engine and scripting environment written in Java. It adopts the Rete algorithm [16] to implement efficiently the rule matching. Jess rules are used to convert low-level information, given in a raw form by sensors, into high-level context. This is conceptually performed in an incremental

A Framework for Context-Aware Home-Health Monitoring

125

ContextualEntity Person

TriggeredAction

Patient

Alarm hasRelative

Relative

isInChargeOf

HealthcareOperator Nurse

hasAlert Level AlertLevel

Physician hasAvailability Availability

triggersProcedure ContactProcedure

Upper Context Ontology Class

Is-A Relationship

Context Ontology Class

Object Property

Fig. 4. Some classes from the context domain ontology referring to the “application consumer environment”

fashion.When the system starts to work, the sensor network or other devices get data from physical world. Depending on the incoming events captured by sensors and context, the facts in the Jess working memory are updated. A set of first-order rules determines if an alarm has to be triggered and which alarm level should be activated, according to measurement values and corresponding thresholds. Jess pattern matcher then searches automatically through the available combinations of facts to figure out which rules should be fired. Such rules, when matched, infer new facts which express the context switching to “situation of alarm”. In other terms, the system acquires a sort of anomaly awareness, i.e. the raw data interpretation infers facts which express the occurrence of an anomaly. For instance, the following example shows a rule activating an alarm when the systolic blood pressure (“sbp-c” variable) exceeds the patient threshold (“spb-max” variable). When the rule is fired, the fact “status abnormal is true” (“sbp-s” variable) is inferred and the action “notify the abnormal event” is activated (“sbp-anomalynotification” action): (defrule verify-SystolicBloodPressure (measurement (hasPID ?m-pid) (hasMeasurementDate ?d) (hasMeasurementTime ?t)) (patient (hasPatientID ?pid) (hasCurrentSBPValue ?sbp-c) (hasNormalMaxSBPValue ?sbp-nmax)

126

A. Esposito et al.

(SBPStatusAbnormal ?sbp-s)) (test (> ?sbp-c ?sbp-max)) => (bind ?sbp-s true) (sbp-anomaly-notification) ) The “anomaly awareness” facts may fire other rules, which may on their turn infer other facts. This determines the switching to the higher level context. In other terms, the context switches to a new situation, which we may call “procedure awareness”, in which the activities to be performed in order to manage the alarm situation are known. The following example shows a rule fired as a consequence of an abnormal status due to both systolic and diastolic blood pressure. The rule infers the fact “alarm is true” and the action “find-available-physician” (defrule set-alert-level (patient (hasPatientID ?pid) (SBPStatusAbnormal ?sbp-s) (DBPStatusAbnormal ?dbp-s) (HighAlertLevel ?hal)) (test (eq ?sbp-s ?dbp-s true)) => (bind ?hal true) (find-available-physician) ) Once that the procedures needed to manage the anomaly have been identified, the context consumers come into action by performing suited “anomaly management” actions. As detailed in the following section, the kind of reasoning above described is carried out with the support of suited agents.

4 Agent-Based Reasoning All the agents are implemented by using the Java Agent Development Environment (JADE). JADE [17] is a software framework to develop and run agent applications in compliance with the FIPA specifications [18] for interoperable intelligent multi-agent systems. Inter-Agent communication is based on the FIPA ACL which specifies a standard message language by setting out the encoding, semantics and pragmatics of the messages. As shown in Fig.5, the semantics of agent messages and reasoning is built over OWL concepts and predicates, which are matched with Jess and JADE vocabulary. Figure 6 shows the proposed multi-agent framework, which assigns three fundamental roles to agents: Context provider agents (CPA). These agents wrap context sources to capture raw context data and instantiate the ontology representation. CPAs may encapsulate single

A Framework for Context-Aware Home-Health Monitoring

127

OWL public class SystolicBloodPressure extends BloodPressure{ {private int hasNormalMaxSBPValue; public void setHasNormalMaxSBPValue(int value) {this.hasNormalMaxSBPValue=value;} }} JADE (deftemplate MAIN::SystolicBloodPressure MAIN::SystolicBloodPressure "$JAVA-OBJECT$ .SystolicBloodPressure" (declare (from-class JESS .SystolicBloodPressure))) Fig. 5. The system implementation is based on the matching between OWL vocabulary with agent inner context representation (in the form of Java classes) and Jess facts codification

sensors or multiple sources. In the former case (“single domain CPAs”) they are mainly responsible for gathering and filtering data and info from sensor devices. In the latter case, [19] they interact with single domain CPAs, in order to aggregate context information from various context sources (for instance sensed data must be aggregated with patient thresholds). Both kinds of CPAs are responsible also of making low level context inference and putting relevant context information into the rule engine as facts. Context interpreter agent (CIA). Context Interpreter Agents are responsible for observing context changes sensed by CPAs, and, as consequence of these changes, to identify the set of actions that should be performed by context consumer agents. Context consumer agent (CCA). Context consumer agents are responsible for performing the actions triggered by CIAs. Actions provide the application reaction to context information changes, which may assume diverse forms, such as the generation of a signal, the delivery of a notification or a web services request.

5 Context Sources As previously stated, the input raw data of the proposed architecture is represented by the set of values, usually physical parameters, collected by the so-called physical sources. Nevertheless, in order to be effectively and conveniently integrated in the scheme proposed in Fig.2, the capability to measure a physical value (such as temperature, blood pressure or oxygen saturation), is only one of the several

128

A. Esposito et al.

Data CPA

Data Retrieval Working Memory Manipulation Anomaly Awareness

CIA

Working Memory Manipulation Procedure Awareness

CCA

Agent Behaviours Working Memory

Anomaly Management

Intra-Agent Communications Inter-Agent Communications

Fig. 6. The multi-agent framework. Context Provider Agents (CPA) are responsible for inferring a potential “anomaly awareness” situation from data provided by context sensors. Context Interpreter Agents (CIA) process high level knowledge provided by CPAs to acquire a sort of “procedure awareness”. Context Consumer Agents (CCA) forward signals and messages to the suited destination as requested by CIAs (anomaly management). The three categories of agents embed a Jess rule engine, update the working memory of Jess and interoperate by using ACL messages.

requirements a physical source should satisfy. The measured data, for instance, should be sent to a data collector by using a wireless connection, and the choice of the most adequate one is not univocal: wi-fi, Bluetooth, GPRS, UMTS, GSM are only a few of the many possible candidates. In order to allow the indispensable capillary diffusion of physical sources, though, the ratio benefit/cost cannot be left out of consideration, thus imposing the choice of a cost-saving wireless technology. Moreover, the capability to be easily interfaced with Internet could be another added value. On such basis, the integration of sensor-networks with RFID systems appears to be the most practicable way. RFID technology, in fact, is quite inexpensive (passive RFID tags are as cheap as few euro-cents) and naturally compatible with Internet [20]. Moreover, as demonstrated in the following, it could be slightly modified in order to transmit sensor-like data. Indeed, the scheme proposed in Fig.7 represents the actually designed and realized (patent pending) general purpose Sensor-Tag (S-Tag) connected to a generic sensor. The architecture of the S-Tag does not substantially differ from standard RFID systems, thus allowing us to maintain the compatibility between this system and devices already available and internationally standardized. The working principle is as

A Framework for Context-Aware Home-Health Monitoring

129

S-tag DIGITAL IN/OUT SWITCH

SENSOR

CHIP (ID1)

CHIP (ID2)

CHIP (ID3)

CHIP (ID4)

MULTI-ID CHIP

SENSOR-TAG ANTENNA

Fig. 7. A simplified scheme of the RFID sensor tag (S-tag)

easy as effective: data measured from a generic sensor are used as input to the S-Tag; when the Tag is in the region covered by the RFID reader, it sends back a signal containing a different combination of several identity codes (IDs) depending on the value of the input itself, thus facilitating the transmission of sensor data. More specifically, the internal microwave circuit of the S-Tag samples the value at its input (which has been measured by the sensor) and quantizes it by using a number of bits equal to the number of available different IDs. For each bit with value equal to 1, the integrated micro-switch selects the corresponding ID to be transmitted; the combination of bits can be hence received by a standard RFID reader and easily decoded in order to rebuild the sensor-measured waveform. This is not the most adequate context for a more exhaustive explanation of the implementation issues of the S-Tag; we only would like to observe that, as apparent from the picture, the sensor is an external unit. In such a way, generic sensors, with the only requirement of a digital output, can be used. Such sensors are not integrated into the S-Tag, so that they do not influence the tag-cost. Moreover, thanks to an accurate electromagnetic design of the tag antenna and of the microwave circuit (microcontroller, RF-switch and so on), also the implemented technological innovation is reasonably inexpensive.

6 Conclusions In this paper we presented a framework for context-aware computing and its prototypal implementation to a home-care scenario. The framework is based on a context codification obtained by integrating an ontology model and a rule-based representation. The main components of the proposed system are its multi-agent

130

A. Esposito et al.

behaviour, as well as the harmonization of heterogeneous technologies, such as agents, ontologies and rule-based inference engines, combined with the low-cost and flexible properties of RFID systems. The overall system is now available and tested in real-life situations in homehealth applications.

References 1. Weiser, M.: The computer for the 21st century. Scientific American, 94–104 (1991) 2. Baldauf, M., Dustdar, S., Rosenberg, F.: A survey on context-aware systems. Int. J. Ad Hoc and Ubiquitous Computing 2(4), 263–277 (2007) 3. Paganelli, F., Giuli, D.: An Ontology-based Context Model for Home Health Monitoring and Alerting in Chronic Patient Care Networks. In: 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW 2007) (2007) 0-7695-2847-3/07 4. Ko, E.J., Lee, H.J., Lee, J.W.: Ontology-Based Context Modeling and Reasoning for UHealthCare. IEICE Trans. Inf. & Syst. E90–D(8), 1262–1270 (2007) 5. Chen, H., Finin, T., Joshi, A.: An ontology for context-aware pervasive computing environments. Special Issue on Ontologies for Distributed Systems, Knowledge Engineering Review (2003) 6. Gu, T., Pung, H.K., Zhang, D.Q.: A service oriented middleware for building contextaware services. Journal of Network and Computer Applications 28, 1–18 (2005) 7. Indulska, J., Sutton, P.: Location management in pervasive systems. In: CRPITS 2003 Proceedings of the Australasian Information Security Workshop, pp. 143–151 (2003) 8. Strang, T., Popien, C.L.: A context modeling survey. In: Workshop on Advanced Context Modeling, Reasoning and Management as Part of UbiComp 2004, The 6th International Conference on Ubiquitous Computing, pp. 33–40 (2004) 9. W3C, RDFS (RDF Vocabulary Description Language 1.0: RDF Schema), Recommendation (February 10, 2004), http://www.w3.org/TR/rdf-schema/ 10. DAML site, http://www.daml.org 11. W3C, OWL Web Ontology Language Overview, Recommendation, (February 10, 2005), http://w3.org/TR/2004/RDC-owl-features-20040210/ 12. Guarino, N.: Formal Ontology and Information Systems. In: Guarino, N. (ed.) Proceedings of the 1st International Conference on Formal Ontologies in Information Systems, FOIS 1998, Trento, Italy, pp. 3–15. IOS Press, Amsterdam (1998) 13. Chen, H., Finin, T., Joshi, A.: An ontology for context-aware pervasive computing environments. In: The Knowledge Engineering Review, vol. 18, pp. 197–207. Cambridge University Press, Cambridge (2003) 14. Dey, A.K., Abowd, G.D.: Toward a better understanding of context and contextawareness, GVU Technical Report, GIT-GUV-99-22 (1999) 15. Ernest Friedman-Hill “Jess In Action”, Edited by Manning 16. http://herzberg.ca.sandia.gov/jess/docs/52/rete.html 17. Jade, http://jade.cselt.it 18. Fipa, http://fipa.org/repository/index.html 19. Dockhorn Costa, P., Ferreira Pires, L., van Sinderen, M.: Architectural patterns for context-aware services platforms. In: 2nd International Workshop on Ubiquitous Computing (IWUC 2005), in conjunction with ICEIS 2005, Miami, USA (2005) 20. Finkenzeller, K.: RFID Handbook: Fundamentals and Applications in Contactless Smart Cards and Identification. John Wiley and Sons Ltd, Chichester

Semantic Learning Space: An Infrastructure for Context-Aware Ubiquitous Learning Zhiwen Yu1, Xingshe Zhou2, and Yuichi Nakamura1 1 Academic Center for Computing and Media Studies, Kyoto University, Japan [email protected], [email protected] 2 School of Computer Science, Northwestern Polytechnical University, P.R. China [email protected]

Abstract. In order to facilitate the development and proliferation of contextaware ubiquitous learning services, there is a need for architectural support in the user context processing and learning content management. In this paper, we propose a context-aware ubiquitous learning infrastructure called Semantic Learning Space. It leverages the Semantic Web technologies to support explicit knowledge representation, flexible context reasoning, interoperable content integration, expressive knowledge query, and adaptive content recommendation. The architectural design and enabling technologies are described in detail. Several applications and experiments are presented to illustrate and evaluate the key features of the infrastructure.

1 Introduction The emergence of e-learning allows the learners to access electronic course contents easily and conveniently through the Web. With the vision of ubiquitous computing coming to reality, people will be living in an environment surrounded ubiquitously by a lot of networked computers (e.g. PCs, TVs) and mobile devices such as PDAs, cellular phones, etc.. Learners can therefore access their desired learning content anytime anywhere with the accessible devices. These two trends have precipitated the advent of ubiquitous learning. One of the most important features of ubiquitous learning is adaptability – learners can get the right information at the right place in the right way [1]. To achieve learning adaptability, the provisioning of content needs to take into account the learner’s changing context (e.g., prior knowledge, learning style, current activities and goals), which we call context-aware learning. In a smart learning environment, a contextaware learning service can, for example, analyze the user’s knowledge gap by comparing his current competencies and tasks, and then suggest suitable learning content for him [2]. While many e-learning systems have been developed and used in recent years, building context-aware learning systems in ubiquitous environment is still complex and time-consuming due to inadequate infrastructure support, e.g., context processing, content provisioning, and content selecting. We propose a context-aware ubiquitous learning infrastructure called Semantic Learning Space that leverages the Semantic Web technologies to support explicit F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 131–142, 2008. © Springer-Verlag Berlin Heidelberg 2008

132

Z. Yu, X. Zhou, and Y. Nakamura

knowledge representation, flexible context reasoning, interoperable content integration, expressive knowledge query, and adaptive content recommendation. The Semantic Web enables computers and people to work in cooperation by representing machine-interpretable information and automating some processes. It satisfies the user-centric, personalized and active learning requirements of context-aware learning. The Semantic Learning Space exploits the Semantic Web standards and tools to provide architectural support for facilitating the development and proliferation of context-aware ubiquitous learning services. The main characteristics of the infrastructure include: (1) systematically handling context management, i.e., aggregation, inference, and query of context; (2) interoperably integrating content from diverse sources; (3) adaptively recommending learning content according to various kinds of context. The rest of this paper is organized as follows. Section 2 discusses previous work relevant to this paper. Section 3 presents the ontology model to express knowledge about the learner, content, and the domain being learned. Section 4 describes the Semantic Learning Space infrastructure and its components in detail. The prototype implementation and evaluation are presented in Section 5. Finally, Section 6 concludes the paper.

2 Related Work There has been much work done in the area of context-based learning in the past few years. LIP project [2] aims to provide immediate learning on demand for knowledge intensive organizations through incorporating context into the design of e-learning systems. iWeaver [3] is an e-learning system which offers learners with different media experiences based on their different learning styles. Sakamura and Koshizuka [4] propose ubiquitous learning that enables people to learn about anything at anytime and anywhere by deploying RFIDs onto a variety of objects, such as foods, medicines, vegetables, and resorts. JAPELAS [5] is a context-aware language-learning support system for Japanese polite expressions learning. It provides learner with appropriate polite expressions deriving the learner’s situation and personal information. Paraskakis [6] proposes a paradigm of ambient learning aiming at providing access to high quality e-learning material at a time, place, pace and context that best suits the individual learner. To achieve this aim, it includes three main functions: multimodal broadband access, content management and integration, and context management. KInCA [7] is an agent-based system supporting personalized, active and socially aware e-learning. The personal agent is aware of the user’s characteristics and cooperates with a set of expert cognitive agents. Recently, some efforts have been put into applying the Semantic Web technologies for learning. Stojanovic et al [8] present an e-learning architecture with ontologybased description of the learning materials. Tane et al [9] propose an ontology tool (displaying, interaction, and evolution of ontology) for semantic resource management in e-learning. Elena project [10] works towards a smart space for learning, which is based on peer-to-peer and Semantic Web technologies. It adopts semantic web ontology languages to describe educational services. Although much work has been done to provide efficient and smart learning services by taking into account user’s context as well as utilizing the Semantic Web

Semantic Learning Space

133

technologies, our work differs from and perhaps outperforms previous work in several aspects. While existing work either focuses on context-based learning or Semantic Web-based learning, we integrate them together, i.e., applying the Semantic Web technologies to accomplish context-aware learning. Although Huang et al [11] propose to exploit Semantic Web technologies to represent context in e-learning, which is similar to us, they do not describe how to use context to accomplish content adaptation. Second, our work enhances the development of context-aware learning services through systematically handling context management (e.g., context aggregation, inference, storage and query) as opposed to ad hoc manner in existing systems. Third, we propose an adaptive recommendation approach to realize context-awareness in learning content provisioning while existing systems simply use match or rule-based selection.

3 Ontology Model We use ontologies to model context about the learner, knowledge about the content, and the domain knowledge (the taxonomy of the domain being learned). Within the domain of knowledge representation, the term ontology refers to the formal and explicit description of domain concepts, which are often conceived as a set of entities, relations, instances, functions, and axioms [12]. By allowing learners or contents to share a common understanding of knowledge structure, the ontologies enable applications to interpret learner context and content features based on their semantics. Furthermore, ontologies’ hierarchical structure lets developers reuse domain ontologies (e.g., of computer science, mathematics, etc.) in describing learning fields and build a practical model without starting from scratch. Upper-level ontologies are designed to provide a set of basic concepts that allows the definition of new concepts in terms of subclasses to complement the upper-level classes. In our system, we have designed three ontologies: Context Ontology, Learning Content Ontology, and Domain Ontology. The Context Ontology shown in Fig. 1a depicts contexts about a learner, e.g., content already mastered, learning goal, available learning time, location, learning style, and learning interests. It also describes the characteristics of the terminal that the learner uses (e.g., displaying feature and network connection). The learning goal may be an abstract subject or a particular content. lco and do stand for Learning Content Ontology and Domain Ontology, respectively. Properties of contents as well as relationships between them are defined by the Learning Content Ontology (see Fig. 1b). The relation hasPrerequisite describes content dependency information, i.e., a content necessary to be taken before the target content. Actually, nowadays most of the departments in universities provide a course dependency chart when issuing their courses. The Domain Ontology is proposed to integrate existing consensus domain ontologies such as computer science, mathematics, chemistry, etc. The domain ontologies are organized as hierarchy to demonstrate topic classification. For instance, the hierarchical ontology of computer science domain is presented in Fig. 1c. It derives from the well-known ACM taxonomy (http://www.acm.org/class/1998/).

134

Z. Yu, X. Zhou, and Y. Nakamura

We adopt OWL (Web Ontology Language) [13] to express ontology enabling expressive knowledge description and data interoperability of knowledge. It basically includes ontology class definition and ontology instance markups.

(a)

(b)

(c) Fig. 1. Ontology design, (a) context ontology, (b) learning content ontology, (c) computer science domain ontology

4 Semantic Learning Space Infrastructure The Semantic Learning Space infrastructure consists of seven collaborating components (see Fig. 2): context aggregator, context reasoner, learning content integration, ontology mapping, knowledge query engine, learning content recommender, and content deliver. The Context aggregator is responsible to aggregate context data from physical sensors as well as virtual sensors, transform the raw context data into context mark-ups, and assert them into the context knowledge base. The Context reasoner infers high-level context information, e.g., user behaviour, from basic sensed contexts, and check for knowledge consistency in the context KB. Learning content integration is responsible for integrating multiple heterogeneous content repositories. Ontology mapping transforms different kinds of content description metadata into the generic ontology mark-ups. The Knowledge query engine handles persistent queries and allows learning content recommender to extract desired content description, context information, and domain knowledge from the knowledge bases. The Learning

Semantic Learning Space

135

content recommender selects learning content and determines its presentation form according to learner’s context. Finally the Content deliverer retrieves and delivers the selected learning content to users.

Content deliverer

Learning content recommender

Knowledge query engine Context reasoner

Content knowledge base Domain knowledge base

Context knowledge base

Context aggregator

Ontology

mapping

Learning content integration

Outlook calendar Workflow Human resource Semantic annotation Virtual sensors

Camera Microphone RFID GPS Content repositories

Physical sensors

Fig. 2. Semantic Learning Space architecture

4.1 Context Aggregator Nowadays there are many kinds of sensors deployed in smart home and smart classroom. Such sensors can be utilized as sources of context about learning. It is necessary to combine different information from video sensors, RFID sensors, and other types of sensors for analyzing complicated context, e.g., learning behavior. The context aggregator therefore aggregates diversity of context information from an array of diverse information sources, i.e., physical sensors and virtual sensors. Physical sensors like camera, microphone, RFID sensors and pressure sensors can detect user’s basic context. Virtual sensors, like outlook calendar service and organization workflow, on the other hand, can extract user’s semantic context, such as learning schedule, task, etc. Context aggregation helps to merge the required information which is related to a particular learner. It then asserts them into the context knowledge base for further reasoning.

136

Z. Yu, X. Zhou, and Y. Nakamura

4.2 Context Reasoner Higher-level contexts (What is the user doing? What is the activity in the room?) augment context-aware learning by providing summary descriptions about a user’s state and surroundings. Context reasoner infers abstract, high-level contexts from basic sensed contexts, resolves context conflicts, and maintains knowledge base consistency. It includes both certain and uncertain reasoning. For certain reasoning, we can apply OWL ontology reasoning using DL (Description Logic), and user-defined reasoning based on first-order logic. To perform user-defined context inference, an application developer needs to provide horn-logic rules for a particular application based on its needs. To enable context reasoning to handle uncertainty, we can use mechanisms such as Bayesian Networks, fuzzy logic, and probabilistic logic. Our current system applies Jena2 generic rule engine [14] to support user-defined reasoning over the OWL represented context. The context reasoner is responsible for interpreting rules, connecting to context KB, and evaluating rules against stored context. We have specified a rule set based on the forward-chaining rule engine to infer high-level learning contexts, e.g., a user’s learning behavior, based on basic sensed context. For instance, the following rule examines whether a given person is currently engaged in English studying on the basis of his location, action, book on the desk — if he is in the study room, he is sitting, and the book on the desk is English book, he is likely to be studying English. The user’s location is tracked by RFID. The user’s action, namely standing, sitting, and walking are recognized by video. The English book is detected by RFID sensor to be on the desk. type(?user, Person), locatedIn(?user, StudyRoom), action(?user, Sitting), on(EnglishBook, Desk) =>involvedBehavior(?user, StudyingEnglish) 4.3 Learning Content Integration Learning content integration is responsible for integrating multiple heterogeneous content repositories. Our current system applies Sakai 2.1 tools [15] to integrate varied learning content repositories from a wide range of content providers. It allows to manage integration without including the complexity inherent in supporting heterogeneous means of communication and data exchange, and therefore gains access to content in a manner that hides the technical detail by which that content is provided. To facilitate flexible integration, each repository needs to be wrapped with a plugin developed with O.K.I (Open Knowledge Initiative) Repository OSID (Open Service Interface Definition) [16]. It contracts between service consumers and providers. The OSIDs for eLearning services include repository, assessment, grading, and course management. The Repository OSID also defines methods for managing object lifecycle, data maintenance, and searching. Sakai 2.1 offers the Repository OSID as a repository service. The repository service works with many plug-ins. It finds out what plug-ins are available, loads the plug-ins, and federates requests and responses among them.

Semantic Learning Space

137

4.4 Ontology Mapping Different organizations have developed description languages for learning objects which combine terms from multiple vocabularies (e.g., LOM, Dublin Core, and SCORM). To facilitate the semantic interoperability of learning content, we should provide a universal query interface. The ontology mapping mechanism transforms different kinds of content description metadata into the generic ontology mark-ups. It enables universal query across heterogeneous content repositories that are described using different metadata vocabularies. The ontology mapping involves two ontologies: core ontology and mapping ontology. The core ontology of learning content is defined in Section 3. It provides a common understanding of the basic entities and relationships of learning content. We adopt W3C SKOS Mapping [17] to describe the mapping ontology. It contains a set of properties for specifying mapping relations between concepts from different organizations’ ontologies (e.g., exactMatch, broadMatch, narrowMatch, majorMatch, minorMatch). Our current system maps LOM, SCORM, ARIADNE, IMS, OKI, and Dublin Core metadata. The mapping is done semi-automatically. We first manually specify the mapping relations used for mapping concepts in a particular learning content ontology to the core ontology. After this the term mapping in content metadata querying is automatically done according to the manual setting. 4.5 Knowledge Query Engine There are mainly three knowledge bases in our system: Context Knowledge Base (Context KB), Content Knowledge Base (Content KB) and Domain Knowledge Base (Domain KB). The knowledge bases provide persistent knowledge storage and eventbased retrieval by the Knowledge Query via proper interfaces. The Context KB also allows some manipulations, e.g., context reasoning. The Domain KB integrates existing consensus domain ontologies such as ontology of computer science, mathematics, chemistry, etc. The domain ontologies are organized as hierarchy to demonstrate topic classification. Since the three kinds of knowledge are represented with ontology and described with OWL, we can inherently provide a universal query interface. The knowledge query engine handles expressive query triggered by the learning content recommender. It provides an abstract interface for applications to extract desired knowledge from the Context KB, Content KB and Domain KB. In our current system, we adopt RDQL (RDF Data Query Language) [18] to query knowledge about content, context, and domain. It is widely used for querying RDF or OWL represented ontology-based information. 4.6 Learning Content Recommender The content recommender provides the right content, in the right form, to the right person based on learner context. It addresses two problems about providing learning content: (a) which content should be provided, and (b) in which form should the selected content be presented.

138

Z. Yu, X. Zhou, and Y. Nakamura

For the purpose of efficient context processing in content recommendation, we classify context into two categories: personal context (e.g., prior knowledge and goals) and infrastructure context (physical running infrastructure, e.g., terminal capability). The right content is determined by personal context, while the right presentation form depends on infrastructure context and user’s QoS requirements. The content recommender first utilizes knowledge-based semantic recommendation [19] to determine the right content which the user really wants and needs to learn. It computes the semantic similarity between the learner and the learning contents and generates a learning path for a particular target course. The content recommender also suggests content according to a user’s situation context. For example, knowing the user has studied a subject (e.g., mathematics) for a long time and is somewhat tired for it, it will recommend another subject, e.g., English. Then, the content recommender determines appropriate presentation form of the learning content according to user’s QoS requirements and device/network capability [20]. The recommendation process is conducted based on fuzzy theory. An adaptive QoS mapping strategy is employed to dynamically set quality parameters of learning content at running time according to the capabilities of client devices. 4.7 Content Deliverer The content deliverer is responsible for streaming or downloading learning content to various devices through different networks. If the modality of recommended content is video, audio or flash, i.e., continuous ones, the content deliverer streams the content to terminals. On the other hand, if the modality is image or text, i.e., discrete ones, the content deliverer merely downloads the content. The content deliverer should support a wide variety of video, audio, image formats adapting to different terminal and network conditions. In our system, the media delivery supports transferring content over such networks like wired Internet and wireless IEEE802. MPEG-4 is employed as video codec for media streaming.

5 Implementation and Evaluation 5.1 Implementation We have developed a prototype of the Semantic Learning Space infrastructure. It consists of an embodiment of the above defined architecture and a set of APIs for supporting the integration of sensors, content repositories, and the development of context-aware applications. Several context-aware learning applications supported by the proposed infrastructure have been implemented to illustrate the key features of it. Content adaptation is conducted within the applications based on different kinds of contexts. One application is to enable recommending learning content according to user’s learning goal and prior knowledge. The aim of developing this application is to test the infrastructure support for semantic recommendation of learning content. Fig. 3

Semantic Learning Space

139

Fig. 3. Main interface for semantic recommendation

shows the main interface for the semantic content recommendation. It mainly consists of four parts. The top part provides GUI for context capturing, i.e., learning goal and the courses already learned. When the learner wants to get content based on these context, the system queries context and makes recommendation. Then, the recommendation list is presented below. The learning path is generated and shown in the left bottom column, while the recommendation package for the content selected from the path is presented in the right bottom column. For learners, how to learn the materials in a more efficient way is very important. The second application developed aims to improve learning efficiency through giving learning advices in arranging learning subjects based on user’s behavior and task. For example, through inference on basic sensed information the context reasoner gets the high-level behavior context that the user has studied a subject (e.g., mathematics) for a long time and is somewhat tired for it. The system then suggests the learner to study another subject (e.g., English), that belongs to today’s learning task and has yet to be finished. The everyday learning task can be made up by the teacher or the parents. The advice message can be given in different ways, e.g., displaying a message on the computer screen or sending an email to parents’ mobile phone. Fig. 4 shows an advice message displayed on the computer. The learner’s current learning behavior, today’s learning task, and learning advice are presented.

Fig. 4. Learning advice

140

Z. Yu, X. Zhou, and Y. Nakamura

The third application is named ELAA (English Learning Anytime Anywhere) with the intention of illustrating the infrastructure support for content adaptation according to the infrastructure context like device capability and network condition. The user can on-line access English learning material through different networks (wired and wireless) and display it on different devices (PC, laptop, PDA, and mobile phone). Suppose that a learner is currently using a Sony VAIO handheld PC that uses either wired Ethernet or wireless network to access the popular English learning material, Family Album USA. As network condition changed, the content presentation form varied accordingly. For example, with the low-bandwidth wireless network, the video playing features 240*176 frame size and 5fps frame rate (see Fig. 5a), while a highquality video with larger size (480*360) and high frame rate (30fps) displaying when the user uses the wired network (see Fig. 5b).

(a)

(b)

Fig. 5. System screenshots: (a) playing low quality video; (b) playing high quality video

5.2 Evaluation We evaluated the Semantic Learning Space from system perspective. Among the infrastructure functions, content recommendation and context reasoning are relatively time-consuming. We therefore mainly tested the overheads of these two components in terms of response time. The experiments were conducted on a Linux workstation with 2.4 GHz Pentium 4 CPU and 512 MB RAM. We measured the content recommending time by varying the number of learning content metadata ranging from 50 to 250. The result is shown in Fig. 6a. We observed that the recommendation process was quite computationally intensive. In particular, as the amount of metadata (or contents) grows, the time spent increases proportionally to the size of the learning content database. For context reasoning, we used five context data sets: 1000, 2000, 3000, 4000, and 5000 RDF triples to test the inference engine’s scalability. The rule set contains 20 user-defined rules. The results (Fig. 6b) show that the response time largely depends on the size of context knowledge base. As the content database and context knowledge become larger, the response time will lead human perceivable delay. However, our user test shows that most of the invited learners found the response time acceptable. We believe the infrastructure is generally feasible for non-time-critical learning applications.

Semantic Learning Space

141

Recommending time (ms)_

2500

2000

1500

1000

500

0 50

100

150

200

250

4000

5000

Number of content metadata

(a) 1600

Reasoning time (ms)

1400 1200 1000 800 600 400 200 0 1000

2000

3000

Number of RDF triples

(b) Fig. 6. System performance: (a) content recommending; (b) context reasoning

6 Conclusion and Future Work In this paper, we propose an infrastructure for context-aware learning in ubiquitous computing environment. It supports explicit knowledge representation, flexible context reasoning, interoperable content integration, expressive knowledge query, and adaptive content recommendation. Three applications are given to illustrate the key features of the infrastructure. In the future work, we will continue to develop more context-aware ubiquitous learning applications using the infrastructure and improve the design. Also we are trying to measure the user’s satisfaction with the system by conducting user study.

Acknowledgement This work was partially supported by the Ministry of Education, Culture, Sports, Science and Technology, Japan under the project of “Cyber Infrastructure for the Information-explosion Era”, the High-Tech Program of China (863) (No. 2006AA01Z198), and the Innovation Fund of Northwestern Polytechnical University of China (No. 2006CR13).

142

Z. Yu, X. Zhou, and Y. Nakamura

References 1. Chen, Y., et al.: A Mobile Scaffolding-Aid-Based Bird-Watching Learning System. In: The 1st IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE 2002), pp. 15–22 (2002) 2. Schmidt, A., Winterhalter, C.: User Context Aware Delivery of E-Learning Material: Approach and Architecture. Journal of Universal Computer Science 10(1), 28–36 (2004) 3. Wolf, C.: iWeaver: Towards ‘Learning Style’-based e-Learning in Computer Science Education. In: The 5th Australian Computing Education Conference (ACE 2003), Adelaide, Australia, pp. 273–279 (2003) 4. Sakamura, K., Koshizuka, N.: Ubiquitous Computing Technologies for Ubiquitous Learning. In: WMTE 2005, Tokushima, Japan, pp. 11–20 (2005) 5. Ogata, H., Yano, Y.: Context-Aware Support for Computer-Supported Ubiquitous Learning. In: The 2nd IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE 2004), Taoyuan, Taiwan, pp. 27–34 (2004) 6. Paraskakis, I.: Ambient Learning: a new paradigm for e-learning. In: The 3rd International Conference on Multimedia and Information & Communication Technologies in Education (m-ICTE2005), Spain, pp. 26–30 (2005) 7. Nabeth, T., Razmerita, L., Angehrn, A., Roda, C.: InCA: a Cognitive Multi-Agents Architecture for Designing Intelligent & Adaptive Learning Systems. Computer Science and Information Systems 2(2), 99–114 (2005) 8. Stojanovic, L., Staab, S., Studer, R.: eLearning based on the Semantic Web. In: Proceedings of the World Conference on the WWW and Internet (WebNet 2001), Orlando, Florida, USA (2001) 9. Tane, J., Schmitz, C., Stumme, G.: Semantic Resource Management for the Web: An ELearning Application. In: The 13th International World Wide Web Conference on Alternate Track Papers and Posters, New York, USA, pp. 1–10 (2004) 10. Simon, B., Mikls, Z., Nejdl, W., Sintek, M., Salvachua, J.: Smart Space for Learning: A Mediation Infrastructure for Learning Services. In: The 12th International Conference on World Wide Web, Budapest, Hungary (2003) 11. Huang, W., Webster, D., Wood, D., Ishaya, T.: An intelligent semantic e-learning framework using context-aware Semantic Web technologies. British Journal of Educational Technology 37(3), 351–373 (2006) 12. Gruber, T.: A Translation Approach to Portable Ontology Specification. Knowledge Acquisition 5(2), 199–220 (1993) 13. McGuinness, D.L., Harmelen, F.: OWL Web Ontology Language Overview. W3C Recommendation (2004) 14. Carroll, J., et al.: Jena: Implementing the Semantic Web Recommendations. In: WWW 2004, New York, pp. 74–83 (2004) 15. Sakai, http://sakaiproject.org/ 16. Open Service Interface Definition, http://www.okiproject.org/ 17. W3C SKOS Mapping, http://www.w3.org/TR/swbp-skos-core-guide/ 18. Miller, L., Seaborne, A., Reggiori, A.: Three Implementations of SquishQL, a SimpleRDF Query Language. In: Horrocks, I., Hendler, J. (eds.) ISWC 2002. LNCS, vol. 2342, pp. 423–435. Springer, Heidelberg (2002) 19. Yu, Z., et al.: Ontology-Based Semantic Recommendation for Context-Aware E-Learning. In: UIC 2007, Hong Kong, China, pp. 898–907 (2007) 20. Yu, Z., et al.: Fuzzy Recommendation towards QoS-Aware Pervasive Learning. In: AINA 2007, Niagara Falls, Canada, pp. 846–851 (2007)

A Comprehensive Approach for Situation-Awareness Based on Sensing and Reasoning about Context Thomas Springer1 , Patrick Wustmann1 , Iris Braun1 , Waltenegus Dargie1 , and Michael Berger2 TU Dresden, Institute for Systems Architecture, Computer Networks Group Siemens AG, Corporate Technology, Intelligent Autonomous Systems CT IC 6 {Thomas.Springer, Patrick.Wustmann, Iris.Braun, Waltenegus.Dargie}@tu-dresden.de, [email protected]

1 2

Abstract. Research in Ubiquitous Computing and Ambience Intelligence aims at creating systems able to interact in an intelligent way with the environment, especially the user. To be aware of and to react on the current situation usually a complex set of features describing particular aspects of the environmental state has to be captured and processed. Currently, no standard mechanisms are available to model and reason about complex situations. In this paper, we describe a comprehensive approach for situation-awareness which covers the whole process of context capturing, context abstraction and decision making. Our solution comprises an ontology-based description of the sensing and reasoning environment, the management of sensing devices and reasoning components and the integration of these components into applications for decision making. These technological components are embedded into an conceptual architecture and generic framework which enable an easy and flexible development of situation-aware systems. We illustrate the use of our approach based on a meeting room scenario.

1

Introduction

Research in Ubiquitous Computing and especially Ambience Intelligence aims at creating systems able to interact in an intelligent way with the environment, especially the user. ”Machines that fit the human environment instead of forcing humans to enter theirs will make using a computer as refreshing as a walk in the woods” [1]. Thus, a system able to recognise the current situation can adapt its behaviour accordingly. For instance, an application for supporting mobile workers during their tasks in the field could adapt the interaction modalities to improve the interaction with the user (e.g. speech input and output could be used if the workers hands are not free or gesture input could be used if the surrounding noise level is very high). In a similar way, an assistance application for elderly people could intelligently support planning of daily activities, e.g. selecting convenient connections of public transportation systems for carrying out shopping activities, visiting the doctor or meeting relatives or friends. F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 143–157, 2008. c Springer-Verlag Berlin Heidelberg 2008 

144

T. Springer et al.

To create such systems, sensing the environment and adapting the behaviour according to its current state are major prerequisites. A system has to be able to capture information about the environment and the involved users based on heterogeneous and distributed information sources, e.g. sensor networks, extracted application data, user monitoring or other methods for gathering context information. The information captured in this way is usually lower-level and has to be abstracted and fused to create an understanding of the overall situation a system is currently within. A large set of schemes for reasoning about the current situation exist which include a wide range of logic and probabilistic based estimation, reasoning and recognition schemes most of which have been employed in artificial intelligence, image and signal processing, control systems, decision theory, and stochastic processes. All these schemes have their advantages and disadvantages and can be applied for different types of sensed data and application areas. Currently, no standard mechanisms are available to model and reason about complex situations. There is no common understanding about what features are relevant for a certain situations and how such features and their interrelations can be identified and modelled. Moreover, the adoption of different reasoning schemes is in an early state, especially with respect to their combination and the overall performance. In this paper, we describe a comprehensive approach for situation-awareness. In particular, the capturing of low-level context information based on sensor networks and further sensing devices, the abstraction of higher-level context using heterogeneous reasoning schemes and the derivation of system decisions are covered. Our solution comprises an ontology-based description of the sensing and reasoning environment, the management of sensing devices and reasoning components and the integration of these components into applications for decision making. These technological components are embedded into a generic framework and a design methodology which enable an easy and flexible development of situation-aware systems.We illustrate the use of our approach based on a meeting room scenario. Our paper is organised as follows: Related work in the areas of context sensing and reasoning as well as for architectures and frameworks for context-awareness is discussed in chapter 2. We introduce our concepts, giving a detailed description of the requirements, the major concepts, the proposed architecture together with its components and a development methodology in chapter 3. In chapter 4 we describe the implementation of the generic framework and our feasibility study based on one example scenario is presented in chapter 5. We conclude the paper with summarising the lessons learned and giving an outlook to future work.

2

Related Work

Several context reasoning architectures and prototypes have been proposed in the recent past. The Sylph [2] architecture consists of sensor modules, a proxy core, and a service discovery module. A sensor module provides a standard means for initialising and accessing sensor devices. A service discovery module advertises

A Comprehensive Approach for Situation-Awareness

145

sensors through a lookup service, and a proxy core manages application queries and serves as a semantic translator between applications and sensor modules. The iBadge prototype, developed with the Sylph architecture, tracks and monitors the individual and social activities of children in kindergartens (loneliness, aggression behaviour, etc.) It incorporates orientation and tilt sensing, environmental sensing, and a localisation unit. The Mediacup [3] is an ordinary coffee mug to which a programmable hardware for sensing, processing, and communicating context is embedded. The hardware is a circular board designed to fit into the base of the cup. It incorporates a micro controller, a memory for code processing and data storage, an infrared transceiver for communication, and an accelerometers and a temperature sensor for sensing different contexts. The same system architecture was used to embed a sensing system into a mobile phone. A three-layered context recognition framework is used to reason about the status of the mug and the mobile phone. It consists of a sensor layer, a cue layer, and a context layer. The sensor layer is defined by an open-ended collection of sensors which capture some aspects of the real world. The cue layer introduces cues as abstractions of raw sensory data. This layer is responsible for extracting generic features from sensed data, hiding the sensor interfaces from the context layer. The context layer manipulates the cues obtained from the cue layer, and computes context as an abstraction of a real world situation. SenSay [4] takes the context of a user into account to manage a mobile phone. This includes adjusting the functional units of the mobile device (e.g. setting the ringer style to a vibration mode) or it can be call related. For the latter case, SenSay prompts remote callers to label the degree of urgency of their calls. Korpip¨ a¨a et al. [5] exploit several sensors to recognise various everyday human situations. The authors propose a multi-layered context-processing framework to carry out a recognition task. The bottom layer is occupied by an array of sensors enclosed in a small sensor box and carried by the user. The upper layers include a feature extraction layer incorporating a variety of audio signal processing algorithms from the MPEG-7 standard; a quantisation layer based on fuzzy sets and crisp limits; a classification layer employing a na¨ıve Bayesian classifier which reasons about a complex context. Their implementation involves a three-axis accelerometer, two light sensors, a temperature sensor, a humidity sensor, a skin conductance sensor and a microphone. The approaches above support the dynamic binding to context sources. On the other hand, their deployment setting as well as the context reasoning task is predetermined at the time the systems are developed. This limit the usefulness of the systems as users’ and applications’ requirements evolve over time. Subsequently, there is a need for dynamic integration of sensing, modelling and reasoning. We build upon the experiences of previous work to support flexible context reasoning and usage. In parallel, efforts were started to create more general models and services to model and reason about context based on ontologies. Recent projects (e.g., [6]) covered the creation of comprehensive and generic context models with the

146

T. Springer et al.

goal of identifying and integrating characteristics of contextual information. Especially, ontology-based modeling and reasoning is addressed ([7,8,9]) with focus on knowledge sharing and reasoning. In [8,7], approaches for defining a common context vocabulary based on a hierarchy of ontologies are described. An upper ontology defines general terms while domain-specific ontologies define the details for certain application domains. Both approaches use a centralized architecture and work on local scenarios from the smart home or intelligent spaces domain. These solutions focus on the modelling of context and apply particular reasoning schemes. Thus, a comprehensive approach starting with sensor integration and classification is not considered. Moreover, the systematic placement of sensors and a decomposition of situations are not addressed.

3

Conceptual Architecture

Real world situations usually have to be derived from a complex set of features. Thus, a situation-aware system has to capture a set of features from heterogeneous and distributed sources and to process these features to derive the overall situation. Thus, major challenges for the creation of situation-aware systems are to handle the complexity of the situation and the related features, to manage the sensing infrastructure and to find appropriate reasoning schemes that efficiently derive the overall situation from low-level context features. In the following we present a conceptual architecture for the creation of situation-aware systems. The approach is intended to be comprehensive, i.e. it comprises all components and processing steps necessary to capture a complex situation, starting with the access and management of sensing devices up to the recognition of a complex situation based on multiple reasoning steps and schemes. To handle complex situations the concept of decomposition is applied to the situation into a hierarchy of sub-situations. These sub-situations can be handled autonomously with respect to sensing and reasoning. In this way, the handling of complex situations can be simplified by decomposition. We focus on sensors installed in the environment, i.e. they are immobile and usually not part of a mobile device carried by the user. The main idea is to exploit cheap and already available sensors to create new or extended applications. For example, buildings are more and more equipped with sensors, e.g. for measuring temperature or light intensity to enable building automation. Such installations can be extended and then exploited to create ”more intelligent” situation-aware applications. We start with the discussion of the requirements for our solution and discuss our conceptual architecture in detail. 3.1

Requirements

The goal of our research work was to develop a comprehensive approach for situation awareness which covers the whole process of context capturing, situation recognition based on context abstraction and decision making. Because of the dynamic properties in the field of situation-awareness, systems have to be flexible and extensible. Therefore, the approach should be independent

A Comprehensive Approach for Situation-Awareness

147

of the type of information sources involved, i.e. the types of sensors, the structure of particular sensor networks and the different real world situations which could occur. Furthermore, the approach should not depend on the application scenario, the used reasoning schemes and the type of the derived higher-level context. Based on the situation and application-specific use of the sensing infrastructure, the capturing and abstraction should be (re-)configurable, e.g. regarding the frequency of data collection, the used classification mechanisms and the aggregation of sensor values. Moreover, scalability and modularity are important. That means, the approach should not restrict the amount of deployed sensors and their distribution. It should also allow a flexible combination of different schemes for reasoning of higher-level context information. For a better reuse, the resulting system should be designed in building blocks, so it is easy to replace a single building block by another one (e.g. a particular reasoning scheme with a different one). Especially, a situation should not be modelled in a monolithic way and varying availability of sensors and resulting incompleteness of information should be manageable. Last but not least, the system should be independent from a certain application scenario. Rather, it should support the set-up of different scenarios regarding the involved sensors and their configuration, the used reasoning schemes and the overall situation captured. Thus, a formalised scenario description which can be interpreted, instantiated and exchanged should be supported by the system. 3.2

Conceptual Architecture Based on Situation Decomposition

Our conceptual framework is based on the decomposition of complex situations. From previous experiments we have observed, that complex situations can be decomposed into sub-situations which can be handled independently with respect to sensing and reasoning. Each sub-situation represents a certain aspect of the overall situation and has to be fused with other sub-situations at a certain level of a hierarchical reasoning process. Below that point, a sub-situation can be handled separately. This enables especially a parallel and distributed sensing and reasoning process. Moreover, for each sub-situation the appropriate reasoning scheme can be applied. At the same time, the decomposition enables the modularization of the system, because sub-situation can be assigned to separate processing modules. Extensibility and flexibility can be supported, because during the decomposition process, alternative sub-situations can be identified and new aspects can be easily added to the hierarchical situation description. According to our validation example, the overall situation could be the current use of a meeting room. Among others, the meeting room could be empty, used for a discussion between two people, a discussion in a larger group, a presentation, or a party. Each of these instances of the overall situation can be captured based on the aggregation and processing of different aspects of that complex situation. For instance, the number of persons in the room, their activity, the lighting, the noise level and the state of the beamer could be used to detect the overall situation. Thus, the initial step for creating a situation-aware system according to our conceptual framework is the decomposition of the complex situation the

148

T. Springer et al.

Fig. 1. Abstraction process for situation detection based on sensor data

system should be aware about. The resulting hierarchy of sub-situations can be refined or extended during the development process as well as during the whole lifetime of the system. The leaves of the situation hierarchy represent atomic sub-situation which can not or should not be decomposed further. These atomic sub-situations are the starting points for the creation of the situation-aware system. To reflect all necessary steps to derive the overall situation from sensed information, our conceptual architecture consists of three layers: a sensing layer, a feature extraction layer and a reasoning layer. These layers are depicted in figure 1. Each of these layers will be described in more detail below. Sensing Layer. The sensing layer comprises solutions for two major issues, namely the integration of heterogeneous sensing devices and the placement and organization of sensing devices according to their semantic relation to subsituations. In fact, there is a broad range of different sensors which can be considered for gathering context information like audio, video, a whole wireless-sensor network, etc. These sensors have to be accessed with the help of a specific programming interface. So the sensors deliver different types of (raw) values. Usually the interfaces are provided by the manufacturer of the sensors. Therefore mainly the sensors can be used directly with only little implementation effort, except the special implementation of a wireless-sensor network, for implementing special sensor configurations. Usually, sensors relevant for a certain sub-situation belong to a certain location. Thus, in contrast to other approaches, which often assume a uniform or random distribution of sensing devices and calculate average values out of several values of sensors of the same type in a certain area, we

A Comprehensive Approach for Situation-Awareness

149

identify distinct ”‘areas of interest”’ which are relevant to capture sensor data relevant for a certain sub-situation. For instance, the presentation area or a table could represent such an area of interest (see figure 3). At these areas of interest different types of sensor devices should be placed and logically grouped. Thus, in our concept each sensor is dedicated to a certain area of interest. Especially, sensors of the same type which belong to different areas of interest are handled separately in the classification and lower-level reasoning steps. The idea is, that an average value of all sensors for the light level at a certain location could be useless for capturing a certain situation, while the values of the light level at certain areas of that location could have a high relevance (e.g. at the projection area of a beamer to decide if the beamer is on or off, or at the surface of a seat to decide if the seat is taken or free). The information of the organisation of a set of sensors into an area of interest is exploited in the classification and reasoning steps described next. Feature Extraction Layer. Because of the heterogeneous sensing devices and the resulting different sensor values, each value has to be classified by an appropriate classifier. Classifiers are dividing the sensor data into individual, application depended classes. These classed are labelled with a symbolic name. The used classifiers can be Neural Networks, Hidden-Markov-Models, Decision Trees, Rule-Sets, Bayesian-Nets, Clustering with a matching heuristic or simply a quantisation over the data. Because of the possible use of different classifiers for the different sensors it is difficult respectively almost not possible in addition to the classified sensor data to have a common quality statement out of the different classifiers, which can be used as quality statements, which influence the reasoning steps. Therefore, the results of the classifiers are logic facts of the particular sensors on their areas of interest. These facts are forwarded then to the reasoning steps. Reasoning Layer. Reasoning is in our concept done in multiple hierarchical steps. Based on the facts resulting from classification new facts, i.e. new highlevel context attributes, are inferred. This is done with the help of different reasoning schemes, which can be deployed separately or work in parallel. Example reasoning schemes are ontology reasoning applying description logics reasoners, rule-based reasoning, case-based reasoning, neuronal networks or bayesian nets. Schemes like the last two can be used, but they have to be trained with a sufficient large training set. The advantages of these methods are, that contradictory facts, which are resulting from measurement failures of the sensors can be handled better. In fact, no wrong high-level context facts are reasoned out of these wrong sensor facts, because the result of these reasoning schemes are statements of the quality or the probability of the inferred context attributes. For example, a trained Bayesian Network for a context attribute calculates the probability to that attribute. On that probability it can be decided if that attribute should be further considered in the following reasoning steps or not. The disadvantage is the application depended and high training effort in advance. The resulting new facts can be further forwarded to reasoners of next superior areas, which combine these facts to another, more abstract fact. After a certain

150

T. Springer et al.

level of abstraction is reached the context attributes of all levels can be delivered to a context-aware application, which can than involve this information for internal decision making, i.e. triggering actions, performing adaptations, etc. To create situation-aware systems according to our conceptual architecture, the developer can intuitively decompose the relevant situation. Based on the experiences and knowledge about classification and reasoning schemes, the different layers of the conceptual architecture can be defined. Usually, the whole process of situation decomposition and the identification of the components of the different layers of the conceptual architecture are determined based on an iterative process. Starting with a small set of sub-situations, the developer can test the system and extend it stepwise to create a more-and-more complex system. Especially, our experiences show, the understanding of a complex situation grows during testing and practical trials.

4

Generic Framework for Situation-Awareness

For implementing situation-aware systems according to our conceptual model we propose a generic framework. The framework integrates several sensor devices and provides a set of classifiers and reasoners which can be adopted in different scenarios. Moreover, it supports the specification of application scenarios based on an ontology which describes the sensor infrastructure, relevant physical values, applied classifiers and reasoners together with their particular configurations and data base settings for storing the captured raw data. The architecture of our framework is depicted in figure 2. In the following sections all components of the framework are described in detail. 4.1

Scenario Manager

The Scenario Manager is the control component of the framework. It is configured by an ontology modelling the settings, sensors, locations and physical values relevant for the current scenario. It evaluates this information, instantiates and configures the required components and invokes the components at runtime. Thus, by providing a new scenario ontology to the Scenario Manager, the framework can be completely reconfigured for another scenario. 4.2

Sensor Integration

The concept of the framework supports the integration of arbitrary sensing devices. We have currently integrated two types of sensors into our framework. Firstly, we implemented a wireless sensor network (WSN) with a flexible amount of sensor nodes at different locations. These sensor nodes are able to obtain different types of sensed data, e.g. light level and temperature, at once. We have used MicaZ Motes from Crossbow in our current implementation. Crossbow offers a wide range of sensor boards which can be connected to the MicaZ and which include many kinds of sensors. For example, the basic sensor board (MTS310CA) provides the following sensors: Photo (Clairex CL94L), Temperature (Panasonic

A Comprehensive Approach for Situation-Awareness

151

Fig. 2. Architecture of the proposed generic framework for situation-awareness

ERT-J1VR103J), Acceleration (ADI ADXL202-2), Magnetometer (Honeywell HMC1002), Microphone (max. 4KHz), Tone Detector and Sounder (4.5kHz). Based on TinyOS we have implemented basic senor access based on the programming language NesC. We have implemented several commands for initialization, read access, reset, setting of sample rate and network statistics. Secondly, a microphone was utilized for getting audio context information. It is cheap and easy to integrate into a sensor environment. The aim is to extract audio information from the environment, to classify these data according experience, and to identify the most likely context. The integration of the microphone in the prototype was simply done with the javax.sound package. 4.3

Classifiers

For performing the step of classification of sensed data the framework comprises a mechanism for the integration of classifiers and a repository containing a set of classifier implementations. Currently, we have implemented several classifiers for the different sensor types available with the WSN. The result of the classification operations are facts in form of qualitative values (e.g. dark, bright and medium for temperature). For the microphone the underlying model of the classification we used is the Hidden Marokov Model (HMM). Before the HMM can be applied to audio data two the extraction of audio features followed by a quantization of these features has to be performed. Based on training data provided as wave files for a specific scenario, the classifiers can recognise the specified situations (e.g. discussion, presentation or panic in a meeting room). 4.4

Reasoners

Because we assume that all classifiers produce facts we focused on deterministic reasoning schemes. In the current implementation we support rule-based and

152

T. Springer et al.

ontology-based reasoning. Especially, for the reasoning about the overall situation we adopt an ontology-based approach. Therefore, a situation is modelled as a set of concepts and roles in the TBox of the ontology. The current values of concepts and roles related to sensor data or lower-level reasoning steps are included as individuals into the ABox by the scenario manager. To reason about ontologies, a description logic reasoner, namely Pellet [10] is applied. Especially, we use the DL reasoning service realization which works on the Abox. Realization. Given an individual I, Abox A and Tbox T, return all concepts C from T s.t. I is an instance of C w.r.t. A and T, i.e., I lies in the interpretation of concept C. If all Realization is performed for all individuals in the Abox, we speak about the Realization of the Abox. The Abox updates are implemented based on the Semantic Web Framework Jena. Based on realization, concepts can be identified which represent the situation according to the facts in the ABox. For instance, the concept BeamerOn can be defined for the meeting room example. 4.5

Scenario Settings

To ensure the configurability of the framework we introduce an ontology-based scenario description. The description consists of two parts: an upper ontology defining general concepts and roles for the scenario description and a scenariospecific ontology, containing concepts, roles and individuals for the definition of a concrete scenario. The upper ontology defines basic concepts for modelling application scenarios, i.e. available sensors, sensor locations, physical values measured by the sensors, and the assignment of sensors to sensor motes (if existing). The concept Sensor represents the sensors available in the environment. Each sensor is described by a certain location which is assigned to the sensor with the role property locatedAt. The concept Location allows the modelling of semantic locations relevant in the scenario. The sensors available at a certain location are modelled by the property role hasSensor, which is the inverse role of locatedAt. Furthermore, each sensor is described by the physical value, which can be measured by the particular sensor type. The concept PhysicalValue represents a value captured from the environment, e.g. temperature or light intensity. The relation between a sensor and its physical value is modelled by the property measures. Further scenario settings are related to available reasoners, a microphone, the database for storing sensed values and general settings for the scenario. The values are defined as datatype properties for the concepts. In the lower ontology, which is specific to a certain scenario, the general concepts of the upper ontology can be refined by deriving scenario specific concepts. To define concrete instances of reasoners or classifiers, individuals have to be defined in the lower ontology. To add new sensors, classifiers or reasoners, appropriate components have to be implemented and registered with the Scenario Manager. If these components require additional settings, the upper ontology and the controller have to be extended as well. All changes can be done easily based on the current framework implementation. Especially, as long as the

A Comprehensive Approach for Situation-Awareness

153

upper ontology is extended without changing existing concepts and properties, all existing scenario settings can remain usable without any changes. 4.6

User Interface

For the visualization of the functionality of the framework and the easy setup of demonstrations, a graphical user interface was created. The user interface consists of thee parts enabling the configuration of the WSN, the monitoring of sensor data and captured context and to reason about the current situation.

5

Validation Example

In the following the usage of the proposed conceptual architecture and the generic framework is illustrated by an application scenario, namely the capturing of the current situation in a meeting room. In most companies and universities, conference and lecture rooms, special places in cafeterias and lounges, and other public places should be reserved in advance to conduct public meetings, presentations, parties, etc. This ensures that business is conducted as planned and no inconvenience or conflict of interests occurs which inhibits the overall goal of the company or university. On the other hand, this well planned and well organised execution of business should accommodate rooms for impromptu or short term decisions to organise get together, staff meetings and bilateral discussions. For the meeting room scenario the goal is to determine the current activity in the room based on the state of devices (e.g. the beamer), the noise level and the occupation of seats (number of persons) in the room. The activity inside a room can be a meeting, a casual discussion of two or three people, a presentation involving many people, a party, a person who is reading or idle. Based on this knowledge, the usage of that meeting room could be optimized. For instance, an empty meeting room could be detected for a spontaneous meeting or the location of a certain meeting could be detected. 5.1

Areas of Interest

The identified areas of interest are shown in figure 3. To distinguish between several activities in the room (e.g. one person reading, two persons discussing, a working meeting, a presentation or a party) and an empty room the individual chairs were identified as areas of interest similarly to the train compartment application scenario. For each seat area we installed a temperature and a light sensor to measure the light level and the temperature directly at the surface of the seat. Based on the measured values we can distinguish between the occupation by a person or by an object and an empty seat. Moreover, the table represents an area of interest. In that area we installed a microphone to capture the audio data from the room which helps us to distinguish between a discussion between two people, a larger discussion, a party and a presentation (i.e. one person is speaking from a certain distance to the table). A light sensor installed in that area captures the ambient light in the room.

154

T. Springer et al.

Fig. 3. Areas of interest in the meeting room scenario

Additionally, we identified a so called presentation area, the area where the image of the beamer is projected on and where the person is standing while giving a talk. In that area we installed a light sensor to capture the beamer state (i.e. on or off). In combination with the light level in the room we can identify the situation that the beamer is on and the ambient light in the room is dimmed as it would be the case during a presentation. 5.2

Lower Ontology

For the scenario the sensors TemperatureSensor, LightSensor and AudioSensor are defined. These sensors measure the physical values Temperature, Light and Audio respectively. For Temperature the three qualitative values HighTemperature, MediumTemperature and LowTemperature are defined. For Light four qualitative values are defined, namely Dark, Medium, Bright and VeryBright. The values for Audio represent an activity captured by microphone measurements in the room. Currently we distinguish between Discussion and Presentation. By adding more training examples and extending the ontology more values can be added. To reason about the situation in a meeting room we modelled the chairs around a table in the meeting room. Table and Seat as well as PresentationArea are modelled as sub concepts of MeetingroomLocation which itself is a sub concept of Location. Free seats are modelled by the following concept: F reeSeat = ∃hasLightSensor(∃uso : hasV alue(Bright ∨ M edium))∧ ∃hasT emperatureSensor( ∃uso : hasV alue(LowT emperature ∨ M ediumT emperature))

For occupied seats we distinguish between seat occupied by persons and by objects. A seat occupied by a person is described as follows: P ersonOccupiedSeat = ∃hasLightSensor(∃uso : hasV alue(Dark))∧ ∃hasT emperatureSensor(uso : hasV alue(HighT emperature))

A Comprehensive Approach for Situation-Awareness

5.3

155

Reasoning

The reasoning about the situation in the meeting room is based on the individual MyMeetingroom. Again, the type of situation is determined by computing the types of this individual based on the reasoning service realisation. MyMeetingRoom belongs to the concept MeetingRoom. The concept represents the overall situation. All partial situations, namely BeamerOff, BeamerOn and PresentationMeetingRoom are modelled as sub concepts of MeetingRoom. Based on the light sensor installed at the presentation area, the state of the beamer can be determined: BeamerOf f = ∃hasP resentationArea(∃hasLightSensor( ∃uso : hasV alue(Bright ∨ M edium ∨ Dark))) BeamerOn = ∃hasP resentationArea(∃hasLightSensor( ∃uso : hasV alue(V eryBright))) ∧ ∃hasRoom( ∃hasLightSensor(∃uso : hasV alue(M edium ∨ Dark)))

In combination with the audio signal the overall situation is determined: P resentationM eetingRoom = BeamerOn∧ ∃hasRoom(∃hasAudioSensor(∃uso : hasV alue(P resentation)))

New scenarios can be implemented by creating a new scenario ontology based on the upper ontology for configuring the generic framework.

6

Summary and Outlook

The major result of our work is a comprehensive solution for modelling and reasoning about complex situations. The solution is comprehensive in the sense that is starts at the sensor layer and comprises all steps necessary to abstract the captured low level sensor values to an overall notion of a complex situation. In our solution we considered the installation and access of sensors, the classification of captured sensor data, several steps of abstracting sensor data and reasoning about sub-situations and the overall situation. In particular, we developed a systematic approach for the decomposition of complex situations. Based on the identification of sub-situations we have shown that it is feasible to place sensor devices in so called areas of interest. Each area of interest can than process the captured sensor data separately (at least the aggregation and classification of sensed values). In higher level reasoning steps the sub-situations are than integrated into the overall situation. To demonstrate the feasibility of our approach we designed and implemented a generic framework for situation awareness. The framework comprises implemented components of every level of situation awareness as described in the third chapter of this document (see figure 1). Each layer is extensible regarding to new components and technologies (i.e. sensors, classifiers and reasoners).

156

T. Springer et al.

Our solution enables an easy, fast and flexible development of situation-aware applications. New scenarios can be implemented by creating a new scenario ontology based on the upper ontology for configuring the generic framework. By using an OWL ontology, scenarios and sensor configurations are clearly defined, easy readable and easy to understand. 6.1

Lessons Learned

From our approach we learned the following lessons: 1. It is possible and reasonable to decompose complex situation into partial situations. At least in all scenarios considered in our project it was possible to decompose complex situations in a way that each sub-situation characterises a certain aspect of the complex situation and is by itself meaningful. Moreover, the partial situations can than be composed to infer the overall situation even if only a subset of all modelled partial situations is considered, i.e. information about environment is incomplete. 2. The identification of areas of interest and the well defined placement and combination of sensors in that area seams to be reasonable and efficient. Compared to an equal distribution of sensors in the environment, the placement of sensors in areas of interest is more focused and environmental state can be captured systematically. 3. The combination of several classifiers and reasoners seams to be possible. In our generic framework a set of classifiers and reasoners is available and we combined them in different ways in the two scenarios. Based on the ontologybased scenario definition, the classification and reasoning algorithms can be configured and thus reused for different scenarios to some degree. 4. We found out that the programming of WSNs and the placement/organisation of sensors is highly related to the scenario. Usually WSNs have to reprogrammed and sensor placement has to be adopted for different scenarios. Thus, the reusability of particular sensor networks and sensor settings is limited. Nevertheless, the generic framework itself is configurable and thus can be reused in different scenarios. 6.2

Future Work

The results demonstrate that our approach is reasonable and advantageous compared to scenario specific and monolithic approaches. While our approach is more flexible, it assumes a closed sensing infrastructure. Especially, the sensors should be under control of the system developer. To exploit existing sensor infrastructures, our approach could be extended to support sensor discovery and dynamic integration of sensing devices. Especially, a location-based discovery of sensors should be considered. Another important point is a further decoupling of all components and layers of our framework. Classifiers and reasoners are available in a local repository and can’t be distributed in the current implementation. Thus, a distribution of these components as well as a discovery and integration mechanism similar to the sensor integration will be considered in the future. Moreover, we will adopt our approach to further application scenarios.

A Comprehensive Approach for Situation-Awareness

157

References 1. Weiser, M.: The Computer for the 21st Century. Scientific American, 66–75 (September 1991) 2. Srivastava, M., Muntz, R., Potkonjak, M.: Smart kindergarten: sensor-based wireless networks for smart developmental problem-solving environments. In: The 7th Annual international Conference on Mobile Computing and Networking, pp. 132– 138 (2001) 3. Peltonen, V., Tuomi, J., Klapuri, A., Huopaniemi, J., Sorsa, T.: Computational auditory scene recognition. In: International conference on acoustic speech and signal processing (2002) 4. Siewiorek, D., Smailagic, A., Furukawa, J., Krause, A., Moraveji, N., Reiger, K., Shaffer, J., Wong, F.: Sensay: A context-aware mobile phone. In: The 7th IEEE international Symposium on Wearable Computers (2004) 5. Korpip¨ aa ¨, P., M¨ antyj¨ arvi, J., Kela, J., Kernen, H., Malm, E.-J.: Managing context information in mobile devices. IEEE Pervasive Computing 2(3), 42–51 (2003) 6. Henricksen, K., Livingstone, S., Indulska, J.: Towards a hybrid approach to context modelling, reasoning and interoperation. In: Ubi-Comp 1st International Workshop on Advanced Context Modelling, Reasoning and Management, pp. 54–61 (2004) 7. Chen, H., Finin, T., Joshi, A.: An ontology for context-aware pervasive computing environments (2003) 8. Gu, T., Pung, H.K., Zhang, D.Q.: Toward an osgi-based infrastructure for contextaware applications. IEEE Pervasive Computing 3(4), 66–74 (2004) 9. Christopoulou, E., Goumopoulos, C., Kameas, A.: An ontology-based context management and reasoning process for ubicomp applications. In: sOc-EUSAI 2005: Proceedings of the 2005 joint conference on Smart objects and ambient intelligence, pp. 265–270. ACM Press, New York (2005) 10. Sirin, E., Parsia, B.: Pellet: An OWL DL reasoner. In: Haarslev, V., M¨ oller, R. (eds.) Proc. of the 2004 Description Logic Workshop (DL 2004), vol. (104) (2004), http://www.mindswap.org/2003/pellet/index.shtml

Context-Adaptive User Interface in Ubiquitous Home Generated by Bayesian and Action Selection Networks Han-Saem Park, In-Jee Song, and Sung-Bae Cho Department of Computer Science, Yonsei University 134 Shinchon-dong, Seudaemun-gu, Seoul 120-749, Korea {schunya,sammy}@sclab.yonsei.ac.kr, [email protected]

Abstract. Recent home theater system requires for users to control various devices such as TV, audio equipment, DVD and video players, and set-top box simultaneously. To obtain the services that a user wants in this situation, user should know the functions and positions of the buttons in several remote controllers. Since it is usually difficult to manipulate them and the number of the devices that we can control increases, we get to confuse more as the ubiquitous home environment is realized. Moreover, there are a lot of mobile and stationary controller devices in ubiquitous computing environment, so that user interface should be adaptive in selecting the functions that user wants and in adjusting the features of UI to fit in a specific controller. To implement the user and controller adaptive interface, we model the ubiquitous home environment and use the modeled context and device information. We have used Bayesian network to get the degree of necessity in each situation. Action selection network uses predicted user situation and necessary devices, and it selects necessary functions in current situation. Selected functions are used to construct adaptive interface for each controller using presentation template. For experiments, we have implemented ubiquitous home environment and generated controller usage log in this environment. We have confirmed the BN predicted user requirements effectively as evaluating the inferred results of controller necessity based on generated scenario. Finally, comparing the adaptive home UI with the fixed one by 14 subjects, we confirm that the generated adaptive UI is more useful for general tasks than the fixed UI. Keywords: Adaptive user interface, ubiquitous home, Bayesian network, action selection network.

1 Introduction People use remote controllers belonging to the appliances when they want to control them. If they purchase more appliances, the number of controllers is getting bigger according to one of appliances. For example, users have to control several devices such as cable TV set-top-box, DVD player, television, audio equipment, video, and DivX players to use a recent home theater system. Controllers for those devices have different interfaces by their companies though they looked similar. Thus, it is not easy to get accustomed to all controller interfaces [1]. Though the controllers have various F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 158–168, 2008. © Springer-Verlag Berlin Heidelberg 2008

Context-Adaptive User Interface

159

functions, only about a third is used practically. If ubiquitous home is generalized, most home devices like lights and boilers as well as home appliances would be controlled using remote controllers. Therefore, it is required to study a context adaptive user interfaces that provide users necessary functions among many of them. In this situation, PUC (Personal Universal Controller), automatically generating user interfaces in PDA or smart phones, has been investigated recently. J. Nichols and B.A. Myers in CMU developed the system, which can generate an optimized controller interface for smart phone using hierarchical menu navigation system [2] and presented HUDDLE, which generates automatically task-based user interface [1]. Besides, they verified that automatically generated interface has better usability in terms of the time efficiency and consistency than general interfaces for device control using user study [3]. These studies generated and provided the useful PUC, but they cannot consider user’s context. We implemented the ubiquitous home environment and used the space and device information. In addition to the user and sensor input, system have used Bayesian network to infer the necessity of devices given context. Also, action selection network has used the necessity of devices as input and selected the function which is necessary in current context. Adaptive user interface consists of these selected function using presentation template. Modeling using Bayesian networks provides good performance as effectively incorporating the domain knowledge [4]. We also modeled the user interface with action selection network, so it can adapt to the user input constantly. For experiment, we have made the log of the use of devices based on scenario and infer the necessity of devices with this log. After that, we evaluated the proposed method using GOMS model. 14 subjects were asked to perform 10 tasks with both conventional fixed home UI and proposed adaptive home UI. The results showed that subjects dealt with the general tasks more effectively using the proposed UI than using conventional one.

2 Related Works 2.1 Bayesian Networks for Context-Awareness Context can have several meanings. Dey defined context as any information that can be used to characterize the situation of an entity such as a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and the application themselves [4]. Generally, context influences the user's preference to a service, and it does to controlling devices in home. It is because the devices user want to control can change according to user's context. Bayesian network, one of the models that are used for context-awareness, is a model to infer the context and provide reliable performance with uncertain and incomplete information [5]. It can be modeled using the data and can be designed using expert knowledge, and has been used to classify and infer the mobile context based on these strengths. Korpipaa et al. in VTT utilized naive Bayes classifier to learn and classify the user's context [6]. E. Horvitz et al. in MS Research proposed the system that infers what the user concentrated in a certain time point in an uncertain environment [7].

160

H.-S. Park, I.-J. Song, and S.-B. Cho

2.2 Action Selection Network Action selection network was presented by P. Maes for robot agent control [8]. Figure 1 illustrates an example of representative action selection network. In this network, competition of actions is the basic characteristic, and it is shown as a change

Fig. 1. An example of action selection network: Solid line represents predecessor link and dashed line represents successor link

of activation level in each action. An action with the highest activation level is selected after the activation spreading, it is calculated as follows. Nodes can have different types of links that encode various relationships and stimulate one another, and each node has five components of preconditions, add list, delete list, activation level, and the executable code. Links divides into internal and external, and internal link subdivides into predecessor link, successor link and conflict link. External link is connected to sensor and goals. If the value of a certain sensor S1 is true and S1 is in the precondition list of an action node A, a link from S1 to A is activated. If goal G1 has an activation larger than zero and G1 is in the add list of A, a link from G1 to A is activated. The procedure to select an action node executed at each step is as follows: 1. Calculate the excitation from sensors and goals. 2. Spread excitation along the predecessor, successor and conflictor links, and normalize the activation level, so the average becomes equal to a constant π . 3. Check any executable nodes, select the one with the highest activation level, execute it, and finish. A node is executable if all preconditions are satisfied and its activation level is greater than threshold. If there is no executable node, reduce the threshold and repeat the process. Links are set by a designer according to the given task. Using this, the system can select the proper action for its goal given a set of sensor states.

Context-Adaptive User Interface

161

3 Context Adaptive User Interface for Ubiquitous Home Figure 2 summarizes the process of proposed adaptive UI generation for ubiquitous home devices. To begin with, Bayesian network infers the necessary devices in current context. With this result and the description of devices and controllers, action selection network is constructed in order to select the necessary functions for each device. Finally, user interface for a given controller is generated using UI template.

Fig. 2. Adaptive UI generation process

3.1 Modeling Ubiquitous Home Ubiquitous home environment for adaptive user interface generation is modeled as follows. - Ε = {Location_List, Device_List, Sensor_List} - Location_List = {Location1, Location2, ..., LocationN} - Device_List = {Device1, Device2, ..., DeviceN} - Sensor_List = {Sensor1, Sensor2, ..., SensorN} - Location = {Location_Name, Location_ID, Neighbors} - Device = {Device_Name, Device_ID, Location_ID, Function_List} - Function_List = {Function1, Function2, ..., FunctionN} - Function = {Function_Name, Function_ID, Function_Type, Function_Value, Add_List, Delete_List} - Sensor = {Sensor_Name, Sensor_ID, Sensor_Type, Sensor_Value}

162

H.-S. Park, I.-J. Song, and S.-B. Cho

Ubiquitous home environment (E) includes location list, which tells locations of users, rooms and devices, device list including appliances like TV and video and equipments like boiler and lights, and sensor list, which observes the states of users and environments. Device information includes the name, location, and its functions, and function information includes functions of each device and the constraints. Function also includes Add_List and Delete_List for action selection network. Add_List has functions, which are desirable to be displayed with that function. Delete_List, on the other hand, has functions, which are not desirable to be displayed with. Location information is represented as a room name. 3.2 Predicting Necessary Devices Using Bayesian Network We have used the Bayesian network to infer the devices that seem to be necessary for a given context in ubiquitous home. Since Bayesian network can be made by expert knowledge even though there are no or little data for learning [9], the system provides reliable performance in an uncertain home environment where it just set. After enough amount of log is obtained, it is possible to learn the Bayesian network. Personalized model can be learned using the log of individual users. We have used the K2 algorithm proposed by Cooper and Herskovits [10]. To calculate the necessity of each device, basic context such as user location, current time, and the date (whether the day is holiday or not) should be used as evidences. The logs of devices related to each device also have been used as evidences. Related devices mean the devices with similar function or in the same room. After all evidences are set, BN inference is conducted to predict the necessary devices. 3.3 Selecting Necessary Functions Using Action Selection Network Once we get the necessities of each device, an action selection network is designed using Device_List (D) and Function_List (F) explained in section 3.1. Assuming these necessities of devices as a set of environmental nodes E and the states of devices’ functions as a set of functional nodes B, E and B are defined as Equation (1) and (2) using F and D. (1) E = {ei | ei ∈ D ∧ executable(ei )}

B = {bi | bi ∈ F ∧ Executable(bi )}

(2)

After making the nodes, predecessor links, successor links, and conflictor links between these nodes are generated. Predecessor link is generated when Equation (3) is satisfied. Predecessor link, which connects two function nodes or a function node and an environment node, is generated when both nodes belong to the same device and they are related positively. Here, a positive relation occurs among functions that are likely to be executed after a certain function is executed. This link is used to make a hierarchical structure of functions in one device. Successor and conflictor links are used to represent the relations of functions in the different devices. Successor link is decided as Equation (4). It is similar to the predecessor link, but they are different in that successor link connects the functions of different devices and connects functional

Context-Adaptive User Interface

163

nodes only. Conflictor link is generated if Equation (5) is satisfied. Differing from the successor link, it is generated when two functions have negative relation (confliction). ⎧ ⎪ ⎪1 precondition(n p , ns ) = ⎨ ⎪ ⎪0 ⎩

⎧ ((n p ⊂ B) ∨ (n p ⊂ E )) ⎪ if ⎨ Device(n p ) = Device(ns ) ⎪relation(excute(n ), excute(n )) p s ⎩ otherwise

⎧ ⎧ ((n p ⊂ B) ∨ (n p ⊂ E )) ⎪ ⎪ ⎪1 if ⎨ Device(n p ) ≠ Device(ns ) succesor (n p , ns ) = ⎨ ⎪relation(excute(n ), excute(n )) ⎪ p s ⎩ ⎪0 otherwise ⎩ ⎧ ⎪ ⎪1 conflictor (n p , ns ) = ⎨ ⎪ ⎪0 ⎩

⎧ ((n p ⊂ B ) ∨ (n p ⊂ E )) ⎪ if ⎨ Device(n p ) = Device(ns ) ⎪confliction(excute(n ), excute(n )) p s ⎩ otherwise

(3)

(4)

(5)

Fig. 3. Change of activation level in an action selection network and constructed user interface after the user controlled TV

164

H.-S. Park, I.-J. Song, and S.-B. Cho

Constructed action selection network is basically represented as a tree, which has device nodes as parents and their functions as children, and each link is added to that tree based on the functional relations. In this network, activation functions are evaluated as F : E × B → [0...1] . Using the inferred necessities of devices, a set of environment node E is made. After that, the procedure to select a necessary function in action selection network follows as explained in section 2.4. After the procedure, we can get an activation level of each function node. Active function node bi(t) is selected as Equation (6). We let the several functions be selected at once while the conventional action selection network does not, so user interface can use these selected functions to display at next step. ⎧ α i (t ) ≥ θ ⎧ ⎪1 if ⎨ bi (t ) = ⎨ (6) executable (bi , t ) = 1 ⎩ ⎪0 otherwise ⎩ Constructed network is evaluated again when the device list user needs changes a lot or when user requests it. As explained in Figure 3, the necessity of TV gets larger if a user watches it, so network evaluation is updated. After evaluation, user interface includes more functions to control TV.

4 Experiments For evaluation, we have conducted experiments in a simulated ubiquitous home environment. Comparing the proposed adaptive UI with the conventional fixed UI, we have confirmed that the adaptive UI provided better performance.

Fig. 4. A plane figure of ubiquitous home environment

Context-Adaptive User Interface

165

4.1 Experimental Environment

Simulated home environment is illustrated in Figure 4. It has 5 rooms of a living room, a dining room, a bedroom, a study, and a bathroom, and each room has devices summarized in Table 1. Lists for devices and functions are represented as XML. Table 1. Devices and functions in each room

Place

Living room

Dining room

Bedroom

Study

Restroom

Device

Name Power Channel TV Volume Brightness Light intensity Power Audio equipment Mode Play Volume Power Ceil light Light intensity Power Wall light Light intensity Window Open/Close Curtain Open/Close Power Coffee maker Status Power Ceil light Light intensity Power Ceil light Light intensity Window Open/Close Curtain Open/Close Bed Fold/Unfold Status Set (am/pm) Alarm Set (hour) Set (minute) Power Computer Status Phone Status Power Ceil light Light intensity Power Ceil light Light intensity Instantaneous Power water heater Temperature Power Fan Mode

Function Type Enum Enum Range Range Range Enum Enum Enum Range Enum Range Enum Range Enum Enum Enum Enum Enum Range Enum Range Enum Enum Enum Enum Enum Range Range Enum Enum Enum Enum Range Enum Range Enum Range Enum Enum

166

H.-S. Park, I.-J. Song, and S.-B. Cho

4.2 Usability Test for Adaptive User Interface

To evaluate the usability of the proposed method, we assumed 10 situations that could happen in home environment and evaluated adaptive interface generated with GOMS model [11]. Situations are provided in Table 2. Each situation has detailed tasks. For example, situation #1 is “Having a breakfast while listening to music with audio” and its detailed tasks are as follows. “Turn on the ceil light in a dining room Æ Turn on the coffee maker Æ Turn on the audio equipment in a living room Æ Set the frequency Æ Set the volume Æ Have a breakfast Æ Turn off the ceil light in a dining room” Table 2. 10 situations for usability test Number 1 2 3 4 5 6 7 8 9 10

Situation

Having a breakfast while listening to music with audio Watching TV in the evening Playing computer games Getting up from the bed in the morning Taking a shower Closing all windows and curtains Turning of all lights Watching TV while having dinner Having a phone conversation in a bed at night Going to sleep at night

Fig. 5. A user interface used in current home network environment (A fixed UI only)

Using these situations, we have evaluated the user interfaces of Figure 5 and Figure 6 using GOMS. Figure 5 used tab-type interface for selecting place and device. To control a certain device, 2 steps of tab selection are required. This is an interface that

Context-Adaptive User Interface

167

Fig. 6. A user interface including context-adaptive interface generated by proposed method (A fixed UI with an adaptive UI)

is based on a controller design of home network widely used. An interface in Figure 6 adds a context-adaptive interface at the right side. It changes according as the user controls the devices or the context changes. Using these two interfaces, we have compared the time that consumed for 14 users to perform the tasks in 10 situations. Figure 7 summarizes the result. When using an adaptive interface together, the reduction rate of time was 38.63% comparing to one only using a fixed user interface.

Fig. 7. Time consumed to perform tasks in 10 situations

168

H.-S. Park, I.-J. Song, and S.-B. Cho

5 Conclusion This paper proposed the method for context-adaptive user interface generation. There can be several devices and controllers in a ubiquitous home environment, proposed method generated the user interface considering both controller and devices. Bayesian network was used to infer the necessary devices, and an action selection network was used to select the necessary functions according as the context and user selection changes effectively. Finally, we have conducted the usability test of the user interface generated by the proposed method. For future work, we are planning to compare the proposed model with others.

Acknowledgements This research was supported by the MKE (Ministry of Knowledge Economy), Korea, under the ITRC (Information Technology Research Center) support program supervised by the IITA (Institute of Information Technology Advancement) (IITA-2008(C1090-0801-0046))

References 1. Nichols, J., Rothrock, B., Chau, D.H., Myers, B.A.: Huddle: Automatically generating interfaces for systems of multiple connected appliances. In: UIST 2006, pp. 279–288 (2006) 2. Nichols, J., Myers, B.A.: Controlling home and office appliances with smart phones. IEEE Pervasive Computing 5(3), 60–70 (2006) 3. Nichols, J., Chau, D.H., Myers, B.A.: Demonstrating the viability of automatically generated user interfaces. In: CHI 2007, pp. 1283–1292 (2007) 4. Kleiter, G.D.: Propagating imprecise probabilities in Bayesian networks. Artificial Intelligence 88(1-2), 143–161 (1996) 5. Dey, A.K.: Understanding and using context. Personal and Ubiquitous Computing 5, 20– 24 (2001) 6. Korpipaa, P., Koskinen, M., Peltola, J., Makela, S.-M., Seppanen, T.: Bayesian approach to sensor-based context awareness. Personal and Ubiquitous Computing 7(2), 113–124 (2003) 7. Horvitz, E., Kadie, C.M., Paek, T., Hovel, D.: Models of attention in computing and communications: From principles to applications. Communications of the ACM 46(3), 52–59 (2003) 8. Maes, P.: How to do the right thing. Connection Science Journal 1(3), 291–323 (1989) 9. Cooper, G., Herskovits, E.A.: A Bayesian method for the induction of probabilistic networks from data. Machine Learning 9(4), 109–347 (1992) 10. Card, S., Moran, T.P., Newell, A.: The Psychology of Human Computer Interaction (1983)

Use Semantic Decision Tables to Improve Meaning Evolution Support Systems Yan Tang and Robert Meersman Semantic Technology and Application Research Laboratory (STARLab), Department of Computer Science, Vrije Universiteit Brussel, Pleinlaan 2 B-1050 Brussels, Belgium {yan.tang, robert.meersman}@vub.ac.be

Abstract. Meaning Evolution Support Systems have been recently introduced as a real-time, scalable, community-based cooperative systems to support the ontology evolution. In this paper, we intend to address the problems of accuracy and effectiveness in Meaning Evolution Support Systems in general. We use Semantic Decision Tables to tackle these problems. A Semantic Decision Table separates general decision rules from the processes, bootstraps policies and template dependencies in the whole system. Recently, DOGMA-MESS (“Developing Ontology Grounded Methodology and Applications” framework based “Meaning Evolution Support Systems”) is developed at VUB STARLab as a collection of meaning evolution support systems. We embed Semantic Decision Tables in DOGMA-MESS to illustrate our approach. Semantic Decision Tables play the roles in both top-down and bottom-up processes of the meaning evolution cycle. The decision rules that consist of templates dependency rules are mainly responsible for the top-down process execution. The bottom-up process execution relies on the ones that contain the concept lifting algorithms. Keywords: ontology, Meaning Evolution Support System, Semantic Decision Table.

1 Introduction Nowadays, a vast amount of ontology capturing methodologies and tools are available. In [9] and on the OnToWorld wiki website1, a general survey on the ontology capturing methodologies is explored, such as the Grüninger and Fox method [11], the Uschold and King method [27], Methontology [8], CommonKADS [18], the linguistic based methodologies (such as in [2, 15]), Opjk methodology [4], and DOGMA methodology [22]. DOGMA (Developing Ontology-Grounded Methods and Applications) is in general a framework for ontology capturing methodologies and applications. Amongst these mentioned methodologies, DOGMA methodology is designed partly based on the best practice of the other methodologies: 1) the Grüninger and Fox method for TOVE project uses the competency questions as a way to scope the domain of 1

http://ontoworld.org/wiki/Ontology_Engineering#Ontology_Building_Methodologies

F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 169–186, 2008. © Springer-Verlag Berlin Heidelberg 2008

170

Y. Tang and R. Meersman

interests and conceptualization evaluation; 2) the Uschold and King method for Enterprise Ontology emphasizes the importance of incorporating brainstorming and defining/grouping terms in a natural language; 3) Methontology and CommonKADS focus on the structural knowledge management activities. Each activity produces a deliverable as the output; 4) the linguistic based methodologies ground the concepts on the basis of natural languages. Hence, it is necessary to use the natural language processing technologies while building multilingual ontologies from scratch; 5) the Opjk methodology adapts the argumentation method (so called “Diligent”). It underlines the socio aspect. Seeing the importance of community aspect in the notions of ontology [10, 12], Semantic Web2, Web 2.0 [3], and some socio aspect focused methodologies (e.g. [4]), the trends towards community impacts on ontology engineering result in a growing interest in community-grounded, knowledge-intensive methodology. DOGMA Meaning Evolution Support System (DOGA-MESS) is thus developed at the VUB STARLab [6]. As an extension to the DOGMA, DOGMA-MESS is a machine-guided ontology versioning, merging and alignment system to support scalable ontology engineering. In practice, we observe that it is hard to do in an interorganizational setting, where there are many pre-existing organizational ontologies and rapidly evolving collaborative requirements. Current ontology merging systems 3 mainly focus on how to integrate different ontologies, such as in [14]. Researches concentrate on combining several (sub-) ontologies into one ontology by removing inconsistency and reducing conflicts among them. DOGM-MESS does not focus on how to solve these problems, but to gradually build interoperable ontologies amongst a large, knowledge-intensive community. One communicates with others’ needs, trying to find the overlapping interests, with which we make interorganizational ontologies. The core activity in meaning evolution support systems is to reach the final consensus on the conceptual definitions. How to manage the negotiation among the members (also called “domain experts”) of the community is crucial. In DOGMAMESS, the technology of meaning negotiation [5] is thus integrated, which constructs the kernel of community management. However, the community behaviors are not thoroughly studied, which leads to the fact that the outcomes of the DOGMA-MESS processes often become “messy”. Therefore, a novel design by embedding Semantic Decision Tables (SDTs) in DOGMA-MESS micro process was proposed in our early paper [26]. Semantic Decision Tables are used to capture the community’s behaviors at the macro level and guide the community at the micro level. Recently, we get increasing requirements of managing ontological structure at a high level, automatically checking the dependencies of different knowledge blocks, and the ability of the quick and accurate adaptation of the knowledge elicitation processes in DOGMA-MESS. These requirements become the challenges of this paper. Based on our early work [26], we enrich the model of DOGMA-MESS by integrating Semantic Decision Tables in DOGMA-MESS in this paper. We focus on how Semantic Decision Tables are used in both bottom-up and top-down processes of the 2 3

http://www.sciam.com/article.cfm?articleID=00048144-10D2-1C70-84A9809EC588EF21 We consider the ontology merging systems as a kind of systems in scalable ontology engineering.

Use Semantic Decision Tables to Improve Meaning Evolution Support Systems

171

meaning evolution cycle in DOGMA-MESS. The accuracy and effectiveness that Semantic Decision Tables can bring for the meaning evolution support systems in general are stressed in this paper. The remainder of this paper is structured as follows. In section 2, we present the background of the paper. We compare our work with the existing technologies, and discuss both the advantages and disadvantages of our work in section 3. We design the model of DOGMA-MESS embedded with Semantic Decision Tables in section 4. Different Semantic Decision Tables hold different semantically rich decision rules. The decision rules that consist of templates dependency rules are mainly responsible for the top-down process execution (section 4.1). The bottomup process execution relies on the ones that contain the selection algorithms, which can be evaluated (section 4.2). Section 5 contains the paper conclusion and the future work.

2 Background This section is organized as follows. First, we explain the background of DOGMA (Developing Ontology-Grounded Methods and Applications, [21]) and its philosophical foundation of double articulation (section 2.1). Second, we recall the model of DOGMA-MESS (Meaning Evolution Support System) methodology in section 2.2. Third, we explain the notion of Semantic Decision Tables in section 2.3. 2.1 DOGMA The research efforts on DOGMA (Developing Ontology-Grounded Methods and Applications, [21]) approach to ontology engineering have been performed at the VUB STARLab over ten years. It was designed as a methodological framework inspired by the tried-and-tested principles of modeling conceptual databases. In the DOGMA framework one constructs (or converts) ontologies by the double articulation principle into two layers: 1) the lexon base layer that contains a vocabulary of simple facts called lexons, and 2) the commitment layer that formally defines rules and constraints by which an application (or “agent”) may make use of these lexons. A lexon is a quintuple < γ , t1, r1, r2, t2>, where γ is a context identifier. γ is assumed to point to a resource, and serves to disambiguate the terms t1, t2 into the intended concepts. r1, r2, which are “meaningful” in this specific context γ , are the roles referring to the relationships that the concepts share with respect to one another. For example, a lexon < γ , Driver’s license, is issued to, has, Driver>4 explains a fact that “a driver’s license is issued to a driver”, and “a driver has a driver’s license”. The linguistic nature of a lexon represents that a fundamental DOGMA characteristic is its grounding in the linguistic representation of knowledge. The community of domain experts chooses (or has to agree on) a given (natural) language, e.g. English, to store and present lexon terms and roles. 4

In this paper, we do not focus on the discussion of the context identifier γ , which is omitted in other lexons. E.g. < γ , Driver’s license, is issued to, has, Driver> is thus written as .

172

Y. Tang and R. Meersman

A commitment corresponds to an explicit instance of an intentional logical theory interpretation of applications. It contains a set of rules in a given syntax, and describes a particular application view of reality, such as the use by the application of the (meta-) lexons in the lexon base. This describing process is also called ‘to commit ontologically’. The commitments need to be expressed in a commitment language that can be easily interpreted. Suppose we have a lexon , which has the constraint as “one driver’s license is issued to at most one driver”. We apply the uniqueness constraints UNIQ on the lexon written as below: p1 = [Driver’s license, is issued to, has, Driver]: UNIQ (p1).5 Just like the same database can be viewed and used by different database applications, the same lexon base can be queried, constrained and used by different ontology based application. The commitments interface the lexon base and different applications. The commitments can be further modeled graphically with many popular modeling tools, such as ORM (Object Role Modeling, [13]), CG (Conceptual Graph, [19]) and UML (Unified Modeling Language6). Ontologies modeled in DOGMA can be further implemented in an ontology language, such as OWL7 and RDF(S)8. 2.2 Meaning Evolution Support Systems DOGMA-MESS (Meaning Evolution Support System, [6]) is to organize the process of interorganizational 9 ontology engineering. Its basic characteristic is to find the relevance of concepts defined by different domain experts in order to capture the overlapping interests in the community. The goal is not to create a “complete” domain ontology that covers all the concept definitions in the domain, but a dynamically evolving “interorganizational” domain ontology. A generic model of interorganizational ontology engineering is presented in Fig. 1. An interorganizational ontology cannot be produced once, but needs to evolve over time. The process results in different versions – e.g. v_1 and v_m in Fig. 1. Each version of the interorganizational ontology contains three different layers: OO (Organizational Ontology), LCO (Lower Common Ontology) and UCO (Upper Common Ontology). •

5

Each domain has its own UCO, which contains the common concept type hierarchy and the domain canonical relations. For example, “employee”, “employer” and “personnel” are the common concept types in the domain of human resource management. The concept types “employee” and “employer” are the subtypes of “personnel” in the type hierarchy. The domain

The syntax of the formalized commitment and the examples can be found at: http://www.starlab.vub.ac.be/website/SDT.commitment.example 6 UML is specified by OMG (Object Management Group), http://www.uml.org/ 7 OWL Web Ontology Language: http://www.w3.org/TR/owl-features/ 8 RDF(S) Resource Description Framework (Schema): http://www.w3.org/TR/rdf-schema/ 9 The name of “Interorganizational ontology” was coined by de Moor in [6]. The name “Interorganizational” indicates that the ontology only contains the overlapping interests and is used between the organizations.

Use Semantic Decision Tables to Improve Meaning Evolution Support Systems

•

• •

173

canonical relations are the relations that are common to the domain. For example, the ones between “employee” and “employer” can be “hire” or “fire”. OO is created by individual domain experts based on the ontological structure at UCO level. Each OO within one ontology version is represented by one domain expert that may have different insights from the others’. At LCO level, the concept definitions from different OO are aligned and merged. The overlapped domain interests are elicited. This process happens within a version. When the interorganizational ontology evolves, all the definitions of the concepts at LCO level of version N are standardized and lifted to UCO level of version N+1.

Fig. 1. A Model of Interorganizational Ontology Engineering, [6]

DOGMA-MESS supports the users playing three main roles: the knowledge engineer, the domain experts and the core domain experts. The knowledge engineer has thorough knowledge of analyzing formal semantics in general. He helps the core domain experts to standardize the concept definitions at LCO level. The core domain experts are recognized domain experts who have excellent domain expertise and a clear overview of the domain. The domain experts constitute the majority of the system users. Every enterprise has at least one domain expert who has his own insights of his own domain. They are responsible to create their own organizational ontologies. 2.3 Semantic Decision Table A Semantic Decision Table (SDT, [24]) uses the tabular presentation as a decision table does. There are three basic elements in a decision table: the conditions, the actions (or decisions), and the rules that describe which actions might be taken based on the combination of the conditions. A condition is described by a condition stub and a condition entry. A condition stub contains a statement of a condition. Each condition entry indicates the relationship between the various conditions in the condition stub.

174

Y. Tang and R. Meersman

An action (or decision) contains an action stub and an action entry. Each action stub has a statement of what action to be taken. The action entries specify whether (or in what order) the action is to be performed for the combination of the conditions that are actually met in the rule column. Table 1. A Simple Example of a traditional decision table 1

2

3

Yes

Yes

No

Condition Bad weather It’s far from home

Yes

No

Yes

Money in pocket

Yes

Yes

Yes

Action Take a taxi back home Walk back home

*

* *

Table 1 is a simple decision table with three conditions: “Bad weather”, “It’s far from home” and “money in pocket”; and two actions: “Take a taxi back home” and “Walk back home”. The condition “Bad weather” has two condition entries - “Yes (The weather is bad now)” and “No (The weather is not bad now)”. The rule column with ID ‘1’ expresses a decision rule as “If the weather is bad, it’s far from home, and, there is money in pocket, then the decision is to take a taxi back home”. In the collaborative settings, one decision maker might misunderstand (or have his own comprehension of) the meaning of a decision item designated by others. For example, the condition “It’s far from home - Yes” in Table 1 can have different measures of distance. Does it mean that the distance is more than 1 km, or more than 3 km? A traditional decision table itself doesn’t support the collaborative setting. In [24], we have listed the following problems that occur when using traditional decision tables: 1) ambiguity in the information representation of the condition stubs or action stubs, 2) conceptual duplication amongst the conditions, 3) uncertainty in the condition entries, and 4) difficulties in managing large tables. The notion of Semantic Decision Table (SDT, [24]) was introduced to tackle the mentioned problems. What makes an SDT different from a traditional decision table is its semantics. Unlike traditional decision tables, the concepts, variables and decision rules are explicitly defined. A decision group shares the most important decision knowledge within a group decision making session. SDTs are modeled in DOGMA (section 2.1). Accordingly, an SDT contains a set of SDT lexons, SDT commitments and a specific decision task. The question on how to construct an SDT within a decision group is answered in our recent publications [23, 24, 25]. Although an SDT contains SDT lexons and SDT commitments, SDT itself is not an ontology. It is because each SDT is used for a specific decision task. The SDT commitments can contain both static, ontological axioms and temporal, changeable rules. Also note that the usage of SDTs is not restricted to a specific system, such as DOGMA-MESS in this paper. Instead, we use DOGMA-MESS as an example to

Use Semantic Decision Tables to Improve Meaning Evolution Support Systems

175

demonstrate how SDTs can improve community-grounded systems. SDTs can be applied in many group decision making domains, such as collaborative human resource management.

3 Related Work and Discussion Consensual knowledge base introduced in [7], meeting techniques applied in [4], cooperative domain ontology studied in [1], “Diligent” in Opjk methodology [4] are promising related work in building consensus on ontology level. However, authors in [6] discuss that those methodologies are lack of community consideration although those methodologies work out some basic principles for building ontological consensus. DOGMA-MESS focuses on the community aspects of scalable ontology engineering, and provides a “fat” community grounded model. In our current projects (e.g. the EC Prolix project15), we observe that there are several advantages and disadvantages while applying Semantic Decision Tables to DOGMA-MESS. We stress the advantages as follows: • • • •

The tabular reports generated based on SDT commitments, in general, are extremely convenient and user-friendly for non-technical domain experts. Semantic Decision Tables are used to constrain the dependencies between the templates. The accuracy of the system is thus improved. Semantic Decision Tables are used to capture the behaviors of the community, manage and guide the community systematically and automatically. Therefore, the effectiveness of the system increases. The flexibility at the system management level increases because the knowledge engineers can create different algorithms and decision rules based on their needs.

A big disadvantage is the complexity. The knowledge engineers need to know how to construct Semantic Decision Tables. However, we observe that, in the projects, it is rather easy for the experts to write Semantic Decision Tables if they already know the DOGMA. It is because the notion of Semantic Decision Table is modeled in the DOGMA framework, and the ontology developed at STAR lab also takes the DOGMA format.

4 Embed Semantic Decision Tables in a Meaning Evolution Support System Recently, modeling layered ontologies has been studied so far. The scalable ontology model we describe focuses on neither the typology of ontology nor the construction of layered ontologies. Instead, we focus on the idea of how to gradually build ontologies within layered ontologies. Based on the work in [28, 6], we model a scalable ontology into four layers: Top Level Ontology (TLO), Upper Common Ontology (UCO), Lower Common Ontology (LCO) and Organizational/Topical Ontology (OTO). In [28], topical ontology structure for scalable ontology engineering is introduced to represent the knowledge

176

Y. Tang and R. Meersman

Fig. 2. Interorganizational ontology engineering model with the integration of Semantic Decision Tables

structure of the domain experts (the stakeholders of a community) by involving different view points of analysis and modeling. Later on, the interorganizational ontology model is designed to match the requirements for meaning evolution [6]. We integrate SDT into the topical ontology model and interorganizational ontology model illustrated in Fig. 1. Fig. 2 shows our design. The dotted lines with arrows in Fig. 2 indicate the specialization dependencies between the ontologies of different levels. Comparing to Fig. 1, Fig. 2 contains an extra ontological level – the level of Top Level Ontology (TLO)10. It defines the abstract concept types, such as ‘Actor’, ‘Object’, ‘Process’ and ‘Quality’. Conceptualization at this level is not allowed to be changed. The relations between these concept types fall into two categories: i) the hierarchical relations reflected by the type hierarchical construct. This kind of relations is also called subsumption ontological roles (e.g. “subtype of” relationship in [20]). ii) Other Core Canonical Relations, such as “part-of” merelogical relation in [12], “property-of” relation and “equivalent” relation. Another difference is the OTO Level. In Fig. 1, the lowest level is OO (organizational ontology) level. In Fig. 2, the lowest level is OTO (organizational and topical ontology) level, which includes OO level. Organizational Ontology and Topical Ontology (OTO) seek to represent systematically the knowledge structure the domain experts has on the given themes (or tasks) individually. A Topical Ontology “lays 10

It was called “MO (Meta Ontology)” level in the old papers [6, 26]. However, we have debated whether to name it as MO level or not in the OTM’07 conferences (http://www.cs.rmit.edu.au/fedconf/). As the structures at this level don’t necessarily model the ontology itself, we conclude that it is better to call it TLO.

Use Semantic Decision Tables to Improve Meaning Evolution Support Systems

177

foundation for application (or task) specific ontologies and conceptual models… its semantic space covers multiple subjects and dynamic evolution of the core concepts within a topic” [28]. The concepts within a topic represent the terminology of the application structure, assumption and framework. Within a version, every domain expert (or every enterprise-wise stakeholder group) is responsible to build his own OTO based on the ontology models in UCO. How does DOGMA-MESS execute the model in Fig. 2? Suppose we have some top level models at TLO level designed by the knowledge engineer. The whole process in Fig. 2 starts with creating the templates at UCO level based on the models at LCO level. These templates at UCO level form the first version of the interorganizational ontology (ontology version V1). Then, these templates are delivered to the domain experts from different enterprises. When they receive the templates, they start to create the ontologies at OTO level. After a time period, the system collects the newly introduced concepts, selects a few and lifts them from OTO level to LCO level. When the lifting process is finished, the core domain experts empty the concept set at LCO level by standardizing and formalizing them. Then, the core domain experts merge them at UCO level. A new ontology version V2 is created. As the starting point of creating the ontology V3, the domain experts (again) introduce new concepts based on the updated templates at UCO level, and so forth. By executing the model visualized in Fig. 2 recursively, the ontology evolves. 4.1 Top Down: Use Semantic Decision Tables to Mange Templates Dependencies We use Semantic Decision Tables (SDTs, section 2.3) to manage the dependencies between different templates at UCO level (Fig. 2). In DB theory, several classic templates dependencies are defined in [17]. In the most general sense, dependencies are possible constrained relations. Among all kinds of templates dependencies, multivalued dependencies, subset dependencies, join dependencies, mutual dependencies, and generalized mutual dependencies are the mostly used. Except the multivalued dependencies, the others are also called “functional dependency”. Based on their work, we carefully reuse the notions for ontology engineering. The dependencies in [17] are the constraints at the instance level. Note that the definitions of “instance” in DB theory and in ontology engineering are disparate. In DB theory, a record in a table is called an instance, e.g. in [17]. Suppose that we have a table column called “vehicle” with two record “bus” and “TOYOTA VIP-234”. These two records are both called the instances of “bus”. In ontology engineering, “bus” is a type at the conceptual level. “TOYOTA VIP-234”, which is a car with a plate license, points to a unique object in the UoD. Hence, “TOYOTA VIP-234”, in ontology engineering, is an instance at the instance/application level. In this paper, we follow the definition of “instance” in ontology engineering. We use SDT to manage the dependencies at both the instance level and the conceptual level. In the following subsections, different kinds of dependencies are explained. 4.1.1 Multivalued Dependencies The multivalued dependencies are used to constrain the generating process of the templates. For example, we have two templates that are relevant to the concept “course” (Fig. 3).

178

Y. Tang and R. Meersman

Fig. 3. Two templates in Conceptual Graph [20] at UCO level

If we apply the multivalued dependency of “course” on these two templates, and if a course type (e.g. “Java course”) is introduced by using one template in Fig. 3, “Java course” will be automatically added to another template as a subtype of “Course”. The constraint is stored as the following SDT commitment: (P1 = [Teacher, teach, , Course], P2 = [Teaching, teach, , Course]): MULTIVAL_DEP(P1(Course), p2(Course)). Seeing that the same templates can sometime be applied to different contexts11 in an ontology in DOGMA-MESS, we stress that to use the multivalued dependencies is necessary. 4.1.2 Subset Dependencies The subset dependencies in [17] is similar to the is-a subsumption relationship in ontology engineering. We say that concept A is the subset of concept B when the subtypes and instances of A belong to the set of concept B. For example, we have two templates: one is related to the concept “Teacher” and the other is related to the concept “Lecturer” (Fig. 4). In the context of “university”, “Lecturer” is a subtype of “Teacher”.

Fig. 4. One template relevant to “Teacher” and the other to “Lecturer”

We store this subset dependency in SDT commitments in two ways: 1) to create a new lexon constrained with a subset constraint; 2) to write a subset constraint directly between these two templates. The SDT commitments are shown as follows: P3 = [Lecturer, is a, , Teacher]: SUBSET(P3(Lecturer), P3(Teacher)). //method 1 (P1 = [Teacher, teach, , Course], P4 = [Lecturer, has ID, , Personnel ID]) : SUBSET(P4(Lecturer), P1(Teacher)). //method 2

11

For example, the members of “course” in the context of “university” are different from the ones of “middle school”.

Use Semantic Decision Tables to Improve Meaning Evolution Support Systems

179

SUBSET(P4(Lecturer), P1(Teacher)) (in method 2) records the subsumption relationship between “Lecturer” and “Teacher” defined in lexon P4 and P1. A “Teacher” is a super type of “Lecturer”. SUBSET() constraint can also be applied in the same lexon, e.g. in method 1. Both methods are equivalently used. 4.1.3 Join Dependencies The join dependencies are the constraints while joining two templates. We often use them together with other constraints, such as subset and equal. Take the same example as in section 4.1.2 (Fig. 4), we write the following SDT commitment to indicate that “Teacher” is joined with “Lecturer”. (P1 = [Teacher, teach, , Course], P4 = [Lecturer, has ID, , Personnel ID]): JOIN_DEP(P1(Teacher), P4(Lecturer)), SUBSET(P1(Lecturer), P3(Teacher)). JOIN_DEP(P1(Teacher), P4(Lecturer)) means that the concepts “Teacher” and “Lecturer” defined in lexons P1 and P4 have a join dependency. In this example, the join dependency is used together with a subset constraint – “Lecturer” is a subset of “Teacher”. Thus, when we join “Lecturer” into “Teacher”, the structure of “Lecturer” is not changed. Fig. 5 shows the result of applying this SDT commitment in the system. If we apply a join dependency combined with an equal constraint, then the templates of both concepts should be updated.

Fig. 5. the result of applying join dependency to the two templates in Fig. 4

4.1.4 Other Dependencies Other dependencies have been studied so far, e.g. the constraints of information modeling and relational database in [13]. We mostly use the following constraints • •

Uniqueness. The uniqueness constraint and how to write its commitment are discussed early in section 2.1. Mandatory. A role is mandatory “if and only if, for all states of the database, the role must be played by every member of the population of its object type…” [13, pp. 166]. For example, we want to have a fact that “a lecturer must have a Personnel ID”. We then apply a mandatory constraint on the lexon written as:

P4 = [Lecturer, has ID, , Personnel ID]: MAND(P4(has ID)). •

Equality. The equality constraint discussed in [13, pp. 231] is very similar to the multivalued dependency we deal with in section 4.1.1. A big difference between them is that an equality constraint is applied to two concepts with different names, while a multivalued dependency deals with two concepts

180

Y. Tang and R. Meersman

with the same name. For example, the following SDT commitment means that “Course” and “Lecture” are equivalent. (P1 = [Teacher, teach, , Course], P5 = [Teacher, teach, Lecture]): EQUAL(P1(Course), P5(Lecture)). •

Exclusion. Type A and type B are mutually exclusive if and only if they don’t share any members. This dependency constraint is checked when a new concept type is introduced as a member of Type A or B. If a domain expert tries to add the same concept type to another type (B or A), he would violate this constraint. Suppose we don’t allow a teacher to be a student at the same time. We shall apply the dependency of exclusion to “Teacher” and “Student” written in the following SDT commitment:

(P1 = [Teacher, teach, , Course]), P6 = [Student, 12 learn, , Course]): OR (P1(Teacher), P6(Student)). 4.1.5 Tabular Reports Generated by SDT in the DOGMA-MESS Top Down Process Early in this section, we have discussed different kinds of template dependencies and how to write the SDT commitments respectively. In each DOGMA-MESS top down process iteration, SDT is used not only to constrain the templates dependencies, but also to generate tabular reports. Once a domain expert introduces a new concept, the system checks the dependencies stored in the SDT commitments. A tabular report, which is generated when a domain expert tries to add new concepts, stores the dependencies information of every new concept. Table 2 is a simple example generated when a domain expert add “Trainer” as a subtype of “Lecturer”, and “Tutor” as a subtype of “Teacher”. As there is a subset dependency between “Lecturer” and “Teacher” (“Lecturer” is a subset of “Teacher”, section 4.1.2), the subtypes of “Lecturer”, e.g. “Trainer”, are automatically added as a subtype of “Teacher” (the column named “Trainer”, Table 2). The reason is explained in the column named “1Lecturer” in Table 2: “Lecturer” is the subset of “Teacher”. Table 2. A table that shows validity of introduced concepts Candidate

Trainer

Tutor

Apprentice

1

Lecturer

Teacher

Teacher, Student

Teacher

Lecturer

2

Teacher

Dependency SUBSET EXCLUSION

Student

Action/Decision Add

*

Add to others

*1 {Teacher}

*

Add conflicted Add Action Denied 12

*2

OR is derived from logical disjunction operator – or. For the exclusive-or in logical theory can be considered equivalently to the mutual exclusion in set theory.

Use Semantic Decision Tables to Improve Meaning Evolution Support Systems

181

If the domain expert tries to add “Apprentice” as a subtype of both “Teacher” and “Student”, it cannot be added because “Teacher” and “Student” have exclusion dependency (see the SDT commitment of exclusion in section 4.1.4). Thus, the add action of “Apprentice” is denied (the column named “Apprentice”, Table 2), the reason of which is explained in the columns named “2Teacher” in Table 2. Table 2 shows a very simple example that the DOGMA-MESS top down processes can benefit from SDT. In the next subsection, how to use SDT to guide the concepts elicitation in a bottom up process is discussed. 4.2 Bottom Up: Use Semantic Decision Tables to Guide the Concepts Elicitation Within an ontology version, the system needs to select a few concepts amongst the ones at OTO level. We use Semantic Decision Tables (SDT) to store the lifting algorithms and manage the process.

Fig. 6. The concept “Teacher” is designed at UCO level

Fig. 7. A new relevant concept “Patience” is introduced by a domain expert at OTO level

Let’s first look at a simple lifting algorithm. When we lift a concept from OTO level to LCO level, we need to choose some concepts at OTO level. Let Sc be the concept set at OTO level, and let Sl be the resulting lifted concept set at LCO level. In order to compute this process automatically, we hereby introduce two important condition stubs used to form SDT condition lexons – the relevance score Rc and the interest point Ip. A concept Ci at OTO level is considered as a relevant candidate concept when it gets certain amount of relevance score Rc. Rc is set zero when a new concept is defined at the first time. It increases when the concept is defined in other organizational ontologies designed by different domain experts. For example, if we get the concept “skill of safety” defined within the same context from two different organizational ontologies, the Rc of this concept is increased by one. Interest point Ip starts from zero. Ip is assigned to an existing concept at UCO level. It increases by one when a new concept, which is connected to this existing concept, is introduced. For example, we have the concept definition of “Teacher” at UCO level (Fig. 6). When a domain expert adds a new relevant concept “Patience” to “Teacher” at OTO level (Fig. 7), Ip of “Teacher” is increased by one. Ip reflects the focusing interests of the stakeholder community. We consider the concept “Patience” as a candidate concept that will be lifted to UCO level only when Ip of its connected concept “Teacher” meets a certain threshold value. When a new concept at OTO level is connected to more than one concept at UCO level, we choose the biggest number of Ip. Accordingly, we formalize a lift rule into the SDT commitments as illustrated in Table 3.

182

Y. Tang and R. Meersman

Table 3. An example of SDT commitments and their explanations in the natural language ID 1

Commitment (P1 = [concept, has, is of, Ip], P2 = [Ip, is of, has, concept])

Verbalization Each concept has at most one interest point. And each interest point is of at most one concept.

: UNIQ (P1, P2). 2

(P3 = [concept , has, is of, Rc], P4 = [Rc, is of, has, concept])

Each concept has at most one relevance score. And each relevance score is of at most one concept.

: UNIQ (P3, P4). 3

The fact that a concept is lifted to UCO level depends on two conditions: 1. whether its relevance score is more than T1 or not; And 2. Whether its interest point P7 = [concept, is lifted to, contain, UCO level]) is more than T2 or not. : IMP(AND(P5(Rc) >= T1, P6(Ip)>=T2 ), P7). (P5 = [concept, has, is of, Rc], P6 = [concept, has, is of, Ip],

Commitment 1 in Table 3 uses the uniqueness constraint (‘UNIQ’) to express the one-to-one relationship between ‘concept’ and ‘interest point’. So does commitment 2. Commitment 3 uses the propositional connectives ‘IMP’ (the implication connective) and ‘AND’ (the conjunction connective) to express that: a candidate concept can be lifted to UCO level if and only if its relevance point and interest point meet their threshold (‘T1’ and ‘T2’). Based on Table 3, two concepts at the OTO level – ‘Patience’ and ‘Oral comprehension’, which are considered as two candidate concepts, are analyzed in Table 4. Table 4 contains the decision whether the concepts ‘Patience’13 and ‘Oral comprehension’ at OTO level can be lifted to LCO level or not. The tabular report is automatically generated by the SDT plug-in in the DOGMA-MESS tool14. As the relevance score (20) of the concept ‘Oral comprehension’ doesn’t reach the threshold (25), it is kept at OTO level for next MESS iterations. Table 4. A tabular report generated based on the SDT, which contains the commitments in Table 3 (T1=25, T2=25) Candidate concept

Patience

Oral comprehension

…

Condition Super Type

N/A

Competence

…

Relevance Score

30

20

…

Relevant concept at UCO

Teacher

Teacher

…

Interest Point

30

30

…

...

…

…

…

Action/Decision

…

Keep for next MESS iteration Lift to LCO

13 14

* *

Its concept is given by Fig. 7. The DOGMA-MESS tool currently developed in STARLab is a web portal to assist domain experts to design ontologies: http://www.dogma-mess.org/

Use Semantic Decision Tables to Improve Meaning Evolution Support Systems

183

The resulting concept set is then provided to the core domain experts, who are responsible for standardizing the concepts and merging them at UCO level. As the concepts are defined and visualized in the Conceptual Graph models (e.g.), the core domain experts can use many available conceptual graph matching manners, such as in [16], to merge the new concepts automatically into the existing concepts. During this merging phase, some extra links between new concepts and existing concepts need to be constructed. For example, conceptually equivalent concepts need to be linked with the “equivalent” canonical relation. The merging process results in several reorganized conceptual graphs at UCO level. 4.2.1 Process Evaluation Using SDT Table 3 is as an example of a lifting rule. In practice, users are free to choose their preferred lifting rules. Furthermore, the algorithm in SDT can be evaluated. Let Ncan be the number of the candidate concepts, which resides at OTO level. And let Nsel be the number of selected concepts that are lifted. The concept selection rate O is defined as: O = Nsel/Ncan. The selection rate O can be studied by setting up different values of T1 and T2 (Table 5). Table 5. A result of the selection rate O in tabular report15 taking from the PROLIX project16

T1

Exp1

Exp2

Exp3

Exp4

Exp5

8

20

37

44

56

T2

15

15

15

15

15

O

25%

12%

12%

10%

0%

In practice, core domain experts need to adjust the parameters T1 and T2 based on real situation. For example, they may set T1 and T2 higher when there are a lot of concept candidates and they don’t have a lot of time to standardize them. An advantage of using the generated tabular reports (e.g. Table 5) is to help the core domain experts to determine which values that they should assign to T1 and T2. In this section, we focus on how Semantic Decision Tables can be used in both bottom-up and top-down processes of the meaning evolution cycle in DOGMA-MESS. In the top-down processes, Semantic Decision Tables are used to constrain the dependencies between the templates at the Upper Common Ontology level. The accuracy of the concepts provided by the domain experts gets improved. In the bottom-up elicitation processes, Semantic Decision Tables bring the effectiveness to the system. For the behaviors of the community are captured, managed and guided by Semantic Decision Tables. The concepts created by the domain experts are no longer selected manually. In addition, the lifting algorithms are no longer hard coded in the system. Thus, the flexibility increases. 15 16

This report is also automatically generated in the tool. The objective of PROLIX is to align learning with business processes in order to enable organizations to faster improve the competencies of their employees according to continuous changes of business requirements. In this project, DOGMA-MESS is used to create the competency ontology for every domain of the test beds. URL: http://www.prolixproject.org/

184

Y. Tang and R. Meersman

5 Conclusion and Future Work In this paper, we focus on the discussion of using Semantic Decision Tables in community grounded, knowledge intensive Meaning Evolution Support Systems for ontology evolution. As DOGMA-MESS is recently developed at our STARLab as a collection of meaning evolution support systems, we use it as a sample17 system to illustrate the strength of Semantic Decision Tables (SDTs). We improve the accuracy and the effectiveness of DOGMA-MESS by embedding SDT in it. More explicitly, SDTs are utilized when designing the templates dependencies in the top-down process, and lifting new relevant concepts to the domain level in the bottom-up process. In the top-down process, the decision rules of SDTs are used to check the validity of the concepts. By doing that, the accuracy is improved. We specified seven categories of templates dependencies that are mostly used. They are multivalued dependencies, subset dependencies, join dependencies, uniqueness dependencies, mandatory dependencies, equality dependencies and exclusion dependencies. Except the multivalued dependencies, others are also called “functional dependencies”. In the bottom-up process, the decision rules of SDTs are utilized to draw the decisions whether the concepts are lifted or not. The decisions are executed automatically in the processes. Thus, the system is enhanced with effectiveness. In addition, we emphasize the importance of capturing the community’s behavior and guide it respectively. The community’s behaviors are coded as the parameters in the lifting algorithm stored in SDTs. After the execution of each process (no matter whether it is the top-down process or the bottom-up process), a tabular report is generated based on SDTs. We consider such a tabular report as a complementary mechanism for non-technical users. We have developed a tool to support constructing and visualizing SDTs. The current version supports modeling some specific commitment types. A web portal to support DOGMA-MESS methodology has been developed [6]. In this paper, we focus on the system management while introducing new concepts that matter the overlapped interests of the community. In practice, we observe that the concepts in an ontology can be obsolete after a long time period. Therefore, we need to update the ontology by modifying (e.g. deleting, replacing with others, and redefining) the concepts. A future work is to use SDTs in this kind of modifying processes. Acknowledgments. The research is partly supported by the EC Prolix project.

References [1] Aschoff, F.R., Schmalhofer, F., van Elst, L.: Knowledge mediation: a procedure for the cooperative construction of domain ontologies. In: Proc. of the ECAI 2004 workshop on Agent-Mediated Knowledge Management, pp. 29–38 (2004) [2] Benjamin, P., Menzel, C., Mayer, R., Fillion, F., Futrell, M., de Witte, P., Lingineni, M.: Idef5 method report. Technical Report F33615-C-90-0012, Knowledge Based Systems Inc., Texas (1994) 17

We argue that the usage of SDT is not limited to DOGMA-MESS system.

Use Semantic Decision Tables to Improve Meaning Evolution Support Systems

185

[3] Braun, S., Schmidt, A., Walter, A.: Ontology maturing: a collaborative web 2.0 approach to ontology engineering. In: Proceedings of WWW 2007 (2007) [4] Casanovas, P., Casellas, N., Tempich, C., Vrandecic, D., Benjamins, R.: Opjk modeling methodology. In: Lehmann, J., Biasiotti, M.A., Francesconi, E., Sagri, M.T. (eds.) Legal Ontologies and Artificial Intelligence Techniques, International Conference on Artificail Intelligence and Law (ICAIL) WorkshopSeries, vol. 4, pp. 121–134. Wolf Legal Publishers (2007) [5] de Moor, A.: Ontology-Guided Meaning Negotiation in Communities of Practice. In: Mambrey, P., Gräther, W. (eds.) Proc. of the Workshop on the Design for Large-Scale Digital Communities at the 2nd International Conference on Communities and Technologies (C&T 2005), Milan, Italy (2005) [6] de Moor, A., De Leenheer, P., Meersman, R.: DOGMA-MESS: A Meaning Evolution Support System for Interorganizational Ontology Engineering. In: Schärfe, H., Hitzler, P., Øhrstrøm, P. (eds.) ICCS 2006. LNCS (LNAI), vol. 4068, pp. 189–203. Springer, Heidelberg (2006) [7] Euzenat, J.: Building consensual knowledge bases: context and architecture. In: Mars, N.J.I. (ed.) Towards Very Large Knowledge Bases – Proc. of the KB&KS 1995 conference, pp. 143–155. IOS press, Amsterdam (1995) [8] Fernández, M., Gómez-Pérez, A., Juristo, N.: Methontology: from ontological art towards ontological engineering. In: Proceedings of the AAAI 1997 Spring Symposium Series on Ontological Engineering, Stanford, USA, pp. 33–40 (1997) [9] Gómez-Pérez, A., Fernández-López, M., Corcho, O.: Ontological Engineering. Advanced Information and Knowledge Processing Series. Springer, Heidelberg (2004) [10] Gruber, T.R.: Toward Principles for the Design of Ontologies Used for Knowledge Sharing. In: Workshop on Formal Ontology, Padva, Italy. In book, Guarino, N., Poli, R. (eds.): Formal Ontology in Conceptual Analysis and Knowledge Representation. Kluwer Academic Publishers, Dordrecht (1993) [11] Grüninger, M., Fox, M.: Methodology for the design and evaluation of ontologies. In: Skuce, D. (ed.) IJCAI 1995 Workshop on Basic Ontological Issues in Knowledge Sharing (1995) [12] Guarino, N., Poli, R. (eds.): Formal Ontology in Conceptual Analysis and Knowledge Representation. The International Journal of Human and Computer Studies 43(5/6) (1995) (special issue) [13] Halpin, T.: Information Modeling and Relational Database: from Conceptual Analysis to Logical Design. Morgan-Kaufmann, San Francisco (2001) [14] Madhavan, J., Bernstein, P., Domingos, P., Halevy, A.: Representing and reasoning about mappings between domain models. In: Eighteenth National Conference on Artificial Intelligence (AAAI 2002), Edmonton, Canada, pp. 80–86. American Association for Artificial Intelligence (2002) ISBN:0-262-51129-0 [15] Mizoguchi, R.: Tutorial on ontological engineering. New Generation Computing 21(2) (2003) [16] Myaeng, S.H., Lopez-Lopez, A.: Conceptual graph matching: a flexible algorithm and experiments. International Journal of Pattern Recognition and Artificial Intelligence 4(2), Special issue Conceptual graphs workshop, pp. 107–126. World Scientific Publishing Co., Inc, Singapore (1992) ISSN:0218-0014 [17] Sadri, F., Ullman, J.D.: Template dependencies: a large class of dependencies in Relational Databases and its complete axiomatization. Journal of the ACM (JACM) 29(2), 363–372 (1982)

186

Y. Tang and R. Meersman

[18] Schreiber, G., Akkermans, H., Anjewierden, A., de Hoog, R., Shadbolt, N., Van de Velde, W., Wielinga, B.: Knowledge Engineering and Management — The CommonKADS Methodology. The MIT Press, Cambridge (1999) [19] Sowa, J.F.: Conceptual Structures: Information Processing in Mind and Machine. Addison-Wesley, Reading, Massachusetts (1984) [20] Sowa, J.: Knowledge Representation: Logical, Philosophical, and Computational Foundations. Brooks Cole Publishing Co., Pacific Grove (2000) [21] Spyns, P., Meersman, R., Jarrar, M.: Data Modeling versus Ontology Engineering. SIGMOD Record: Special Issue on Semantic Web and Data Management 31(4), 12–17 (2002) [22] Spyns, P., Tang, Y., Meersman, R.: A model theory inspired collaborative ontology engineering methodology. Journal of Applied Ontology, Special issue on Guizzardi, G., Halpin, T.: Ontological Foundations for Conceptual Modeling, vol. 4(1), pp. 1–23. IOS Press, Amsterdam (2007) [23] Tang, Y.: On Conducting a Decision Group to Construct Semantic Decision Tables. In: Meersman, R., Tari, Z., Herrero, P. (eds.) OTM-WS 2007, Part I. LNCS, vol. 4805, pp. 534–543. Springer, Heidelberg (2007) [24] Tang, Y., Meersman, R.: Towards building semantic decision table with domain ontologies. In: Man-chung, C., Liu, J.N.K., Cheung, R., Zhou, J. (eds.) Proceedings of International Conference of information Technology and Management (ICITM 2007), pp. 14–21. ISM Press (2007) ISBN 988-97311-5-0 [25] Tang, Y., Meersman, R.: On constructing semantic decision tables. In: Wagner, R., Revell, N., Pernul, G. (eds.) DEXA 2007. LNCS, vol. 4653, pp. 34–44. Springer, Heidelberg (2007) [26] Tang, Y., Meersman, R.: Organizing Meaning Evolution Supporting Systems Using Semantic Decision Tables. In: Meersman, R., Tari, Z. (eds.) OTM 2007, Part I. LNCS, vol. 4803, pp. 272–284. Springer, Heidelberg (2007) [27] Uschold, M., King, M.: Towards a methodology for building ontologies. In: Skuce, D. (ed.) IJCAI 1995 Workshop on Basic Ontological Issues in Knowledge Sharing (1995) [28] Zhao, G., Meersman, R.: Architecting ontology for Scalability and Versatility. In: Meersman, R., Tari, Z. (eds.) OTM 2005. LNCS, vol. 3761, pp. 1605–1614. Springer, Heidelberg (2005)

Combining User Profiles and Situation Contexts for Spontaneous Service Provision in Smart Assistive Environments Weijun Qin1,2 , Daqing Zhang2 , Yuanchun Shi1 , and Kejun Du2,3 Key Laboratory of Pervasive Computing, Ministry of Education, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China [email protected], [email protected] 2 TELECOM & Management, SudParis, Evry 91000, France [email protected], [email protected], [email protected] School of Computer Science, Northwestern Polytechnical University Xi’an 710072, P.R. China [email protected]

1

3

Abstract. Wireless hotspots are permeating our workplace, home and public places bringing interesting services and spontaneous connectivity to mobile users. Since mobile users will be immersed in thousands of different kinds of pervasive services, it’s of paramount importance to provide appropriate services to the right person in the right form. Existing approaches offer a rather simplistic solution to the problem and seldom consider the multi-dimensional contextual information. In this paper we present a service provision mechanism which can enable effective service provision based on semantic similarity measure with the combination of user profiles and situation context in WLAN enabled environment. We illustrate our ideas through a scenario of service discovery and provision in public places (e.g. shopping mall) where a disabled person can select appropriate services with the constraints of his mobile device.

1

Introduction

Recent research advances in the areas of pervasive and ubiquitous computing focus on novel application scenarios that are related to various living spaces such as home, office, hospital, shopping mall, museum, etc.. Mobile and pervasive technologies are opening new windows not only for ordinary people, but also for elderly and dependent people due to physical/cognitive restriction. As wireless hotspots are permeating the globe bringing interesting services and spontaneous connectivity to mobile users, one trend is to gradually transform various physical environments into ”assistive smart spaces”, where dependent people can be supported across the heterogeneous environments and be fully integrated into the society with better quality of life and independence. F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 187–200, 2008. c Springer-Verlag Berlin Heidelberg 2008 

188

W. Qin et al.

In order to provide services in various assistive environments, impromptu service discovery and provision with heterogeneous mobile devices become critical. There are two possible ways to make the services accessible to mobile users. One way is to get connected to a global service provider and access the location-based services through the service provider. In this case, the service provider needs to aggregate all the services in the smart spaces and have the indoor/outdoor location of the user. Due to the evolutionary nature of the autonomous smart spaces, it is unlikely to have such a powerful service provider in the near future. The other way, which we are in favor of, is to enable the mobile user interact with individual smart space directly when the mobile user physically enters the space, that is, the mobile user can automatically discover and provision the services provided by the smart spaces. In the latter case, the mobile user needs to get connected directly to the individual smart space using short range wireless connectivity (such as WLAN or Bluetooth). There are two key challenges to achieve impromptu service discovery and provision. The first challenge is how to automatically discover the relevant services upon entering the smart space, making reasonable assumptions on mobile devices and service providers who offer services in the specific smart space; The second challenge is how to automatically provide appropriate services to the right person with the right form with the consideration of contextual information in smart environment. To address the first challenge, various software [1][2] and hardware [3] ”plugin” solutions have been proposed, in addition to the service discovery standards such as UPnP [4], Jini [5], etc. The key idea was to embed specialized software or hardware components in both the mobile device and the environment which follow the same service discovery protocol, however, there is still no dominant solution that was accepted by the whole community. Compared to the first challenge, the second challenge received much less attention. Existing approaches eithor offer a rather simplistic solution to the problem and seldom consider the multi-dimensional contextual information, or are still limited to key-value based semantic matching to select appropriate service to the user [6]. In this paper, we aim to propose service provision framework combining the user profiles and situation contexts to select appropriate services to the end-users from thousands of desultory services with the constraints of mobile device. The rest of the paper is organized as follows: Section 2 starts with a use scenario for assisting a person in a shopping mall, from the use case six system requirements are identified for supporting impromptu service discovery and access in unfamiliar assistive environments. After presenting the overall system for impromptu service discovery in Section 3, a context-aware service provision process based semantic similarity measure are described in Section 4. Then Section 5 gives the design issues. Finally, some concluding remarks are made.

2

Use Scenario

In this section we introduce an assistive service scenario in a shopping mall that needs secure impromptu service discovery and access in smart environments. In

Combining User Profiles and Situation Contexts

189

this paper, we focus on the issue of context-aware service provision with the consideration of user profiles and situation contexts. 2.1

Service Provision in Shopping Mall Scenario

Bob, a disabled person with wheelchair, would like to buy a dust coat in a shopping mall. Upon entering the shopping mall, he is authenticated implicitly and presented 3 general services on his wireless PDA including store direction service, goods directory service, and discount service. He uses goods directory service and discount service to search several dust coat brands with discount according to his preferences such as Burberry, Boss, etc. He decides to look around Boss speciality store first, and uses store direction service to find the location. When Bob decides to go to the store, he is presented with 2 in-store services: one is coat finder which helps him locate the coat in a specific shelf, the other is coat recommendation service that recommends coats based on his preference. Besides, Bob could uses general goods directory service to compare different brands to help him make decision. 2.2

Requirements of the Scenario

From the scenarios described above, we can elicit the following requirements that support impromptu, secure and context-aware service discovery and provision with mobile devices: - R1: Minimum assumption on mobile devices: One of the first ideal features for impromptu service access is that the mobile device doesnt need to install any specialized software or hardware in order to interact with the services in a certain environment. - R2: Automatic discovery of services in unfamiliar environments: Another ideal feature for impromptu service access with mobile devices is the automatic discovery of the relevant services in the new environments, when needed. - R3: Heterogeneous service aggregation: As the devices and services are heterogeneous, they also vary greatly in different environments. There should be an effective mechanism and platform that facilitates the management of both static and dynamic services associated with each smart space. - R4: User and environment context consideration: In order to present the right services to the right person in the right form, user and environment context should be taken into account for service discovery and access in each smart space. - R5: User interface generation according to user context and device capability: As the users and access devices vary, impromptu and personalized user interface generation based on user preference and device capability should be supported. - R6: Secure service access with ensured privacy: When the mobile users access certain services, user inputs and the output of the services need to be exchanged in a secure manner. User privacy should be guaranteed so that he might choose not to reveal his identity during service discovery and access.

190

3

W. Qin et al.

Impromptu Service Discovery and Access

To address the above-mentioned requirements, we propose and implement a impromptu service discovery and access framework which enables the services in unfamiliar environments accessible to any WLAN enabled mobile device, according to individual user’s preference, situation and device capability. While an open standard-based service platform OSGi [7] has been adopted for service aggregation (R3), the Semantic Space [8] is deployed for user and environment context management (R4). The spontaneous interaction framework(R3), security and privacy issue (R6) and user interface generation (R5) have been studied and discussed in separate papers. In this section, we mainly focus on what the minimum requirements are for mobile devices in our design (R1) and how we achieve impromptu service discovery under the minimum assumptions (R2). In the next section we will present the service provision process combining with user and situation contexts (R4). 3.1

Minimum Assumptions on Mobile Device

In order to communicate locally with the smart space, we assume that the mobile devices should have at least the built in WiFi chipset to allow wireless connectivity and a web browser to access the web server hosting the services of the smart spaces. The requirement on mobile devices is minimum as it doesn’t rely on any specialized software or hardware. 3.2

Impromptu Service Discovery Mechanism

Following the minimum assumption on mobile devices, it requests that the services are associated with a specific ”smart space” like a shopping mall, a lift, or a coat shop. When the mobile devices are detected within a certain smart space, the services can be automatically discovered and presented in the mobile devices. To enable the service aggregation and automatic discovery, a captive portal and service portal mechanism is proposed. We adopted the open standard-based service platform such as OSGi [7] to aggregate various services and provide the service portal in a specific smart space, as such various devices and communication protocols are already supported. The main function of captive portal is to connect the mobile device to a wireless network automatically and direct the web browser of the mobile device to the service portal of a specific smart space, it enables the automatic discovery of the services which are provided by each service provider available in the smart space. In our approach, we suppose each service provider records the user’s behavior and infers the semantic dependency between the service and the user using reasoning model (e.g. Dynamic Bayesian Network) [9]. Different from the normal captive portal mechanism, our captive portal is able to detect what the preferred networks list the client device has, it can then configure the WLAN access point in such a way the mobile device with wireless auto configuration capability can automatically connect to the network. Thus when the mobile device switches on, the device is automatically connected to the network without

Combining User Profiles and Situation Contexts

191

Fig. 1. When the user enters a smart space, his mobile device is detected by the Captive Portal and connected to the wireless access point(1), and after launching the web browser, his web browser is directed to the Service Portal(2). He can then browse the available services(3), and invoke a service through his mobile device(4).

configuration. When the web browser is launched by the user, the mobile device will be directed to the service portal of the smart space so that the user can choose to invoke the service needed. The whole process is illustrated in Fig. 1. 3.3

Overall System Architecture

The overall system architecture is shown in Fig. 2 where each smart space has an impromptu service discovery and provision framework embedded as a server. When the dependent people move from one space to another with a WLAN enabled mobile device, her mobile device will be detected and connected to a wireless network in a smart space. And the web browser will be directed by the captive portal in the framework to the service portal of the new smart space which aggregates and hosts the services offered. The middleware core refers to the OSGi service platform. The context manager is responsible for context representation, aggregation, reasoning, storage and query processing, which facilitates service discovery and access with mobile devices [8]. Combining user profiles and situation context, The SemanticMatcher delivers the appropriate associated services to the users automatically.

4 4.1

Context-Aware Service Provision Process Context Modeling

Context is a description of semantic entities such as the user, a device, the situation and the environment [10]. It’s obvious that there are many different types of contextual information that can be used by applications in a smart assistive

192

W. Qin et al. Application Layer

Situation Profile Repository

Semantic Matcher

Reasoning Engine

UI Generation

Service Description Repository

Service Invocation

Service Discovery

. . .

Context Manager

Security

Administration

HTTP Request & Response

Doman‐Specific Services (e.g. Home, Office, Public Places)

Service Layer

OSGi Service Framework Java Runtime Environment System Layer

Operating System Hardware (e.g. IEEE 802.11, Ethernet, Bluetooth)

Physical Interface Layer

Fig. 2. Impromptu service discovery and provision framework for multiple spaces

environment, including physical contexts (e.g. time, location), personal contexts (e.g. identity, preference, mood), device contexts(e.g. display size, power), activity contexts(e.g. shopping list, meeting schedule). Besides those, other types of information are still considered as crucial context which may be invisible to the participants e.g. systematic contexts (e.g. CPU power, network bandwidth), and application contexts (e.g. agents, services), environmental contexts (e.g. light, temperature), etc. In order to distinguish the specific properties or attributes of different kinds of contexts, Context Modeling plays an important role in designing and developing context-aware applications for smart assistive environment. Based on the previous work on context modeling in [11], we build an ontology-based context model with the powerful capabilities of RDF (Resource Description Framework) and OWL (Web Ontology Language). In our model, we divide context ontology into two sets: core context ontology for general conceptual entities in smart assistive environment and extended context ontology for domain-specific environment, e.g. shopping mall domain. The core context ontology attempts to define the general concepts for context in smart environment that are universal and sharable for building context-aware applications. The extended context ontology attempts to define additional concepts and vocabularies for supporting various types of domain-specific applications. The core context ontology investigate seven basic concepts of User, Location, Time, Activity, Service, Environment and Platform, which are considered as the basic and general entities existed in smart assistive environment. Part of the core context ontology is adopted from several different widely-accepted consensus ontologies, e.g. DAML-Time [12], OWL-S [13], etc. Since user profiles, environmental contexts and device capability have deeply influenced service provision process in smart assistive environment for mobile users, the model considers context information ranging from user profiles, location, time and activity to environment capability as user’s situation context. Fig. 3. shows a partial context ontology used in the shopping mall scenario

Combining User Profiles and Situation Contexts

193

Fig. 3. A partial context ontology for service provision process in smart assistive environment, which delineates the concepts and relationships between the user’s situation context and service context

above which delineates the concepts and relationships among the user’s situation context and service context. The User ontology in our sense will consist in a classification of users type and of features of users, and each categorized class will be linked with associated information, such as preferences, knowledge, capabilities and so on. The Service ontology defines the multi-level service specifications in order to support service discovery and composition. Each instance of Service will present a ServiceProfile description, be describedBy a ServiceModel description, and support a ServiceGrouding description in OWL-S specification. The Condition property and the additional hasPrecondition parameter in OWLS specify one of the preconditions of the service which can be considered as the instance defined according to the restriction of user’s situation contexts. User Profiles. The User profiles are of major importance to provide intelligent and personalized service in smart environments [6]. Based on User Ontology, we divide the user profile into two parts: static part which is relevant to user’s type and characteristics, such as Knowledge, PersonalData(e.g. demographic features), Characterics(e.g. profession, language, etc.), Capabilities(e.g. competencies), PhysicalState(e.g. blood pressure, etc.), CognitiveState(e.g. mood, etc.) and so on, and dynamic part which is relevant to user’s behavior, such as Interest, Preference, InvolvedActivity, InvolvedLocation, and their subclasses [14]. The profiles can be stored in the PDA (client) and updated according to specific contexts. The same user can have multiple profiles. Each profile could be the one corresponding to the current user’s location or activity. Table 1 shows a possible profile for the same person related to his role as customer in the scenario. Defining user ontologies facilitates to provide general and shared common concepts and properties relevant to user’s personalized information, preferences and activities.

194

W. Qin et al. Table 1. A partial example for user profile used in shopping mall ID: QIN001 ... InvolvedActivity: Shopping InvolvedLocation: Shopping Mall Gender: Male Age: 40 Language: French Capabilities: Disabled Interest: Coat Preference: Shape, Color, Material, Size, Brand ...

Situation Context Description. The Situation Context captures the current user’s situation including Location, Activity, Time and PhysicalEnvironment information which is obtained from embedded sensors in smart environment. After the representation, aggregation and calculation processes of context-aware system, situation context can be represented as RDF description form. Table 2 shows an partial example for situation context description in the system. From the basis of context model, the hierarchical structures of Location, Time, Activity and PhysicalEnvironment are well organized with the capability of ontologies. For example, The Store is the subclass of the Shopping Mall with the different granularity of Location. In sensor-based context-aware systems, each data result from sensors may have imprecise factors such as sensor disconnection, signal distortion, etc. With the consideration of sensor data imperfectness, probabilistic model is adopted to represent imperfect data with the form of P r(A) = a, where A represents the situation context instance, and a represents probability value which is calculated from context reasoner. Table 2. An partial example for situation context description used in a store

... Location: Boss speciality store Device: PDA Time: 1:00pm EST, Jan. 1st, 2008 ...

Service Description. The Service Description represents the capability and properties of services and the degree of required information. Current initiatives such as UDDI (Universal Description Discovery and Integration), WSDL (Web Service Definition Language) and WSFL (Web Services Flow Language) are considered complementary to the expression of service definition and properties

Combining User Profiles and Situation Contexts

195

description, however these initiatives lack in considering of combing with the situation contexts in real smart environment. From the view of end-user interaction, a taxonomy of service provide a schema to describe the features of service instance including UUID, ServiceName, ServiceProvider, Category, Precondition, InputParameter, OutputParameter, Dependency etc. The UUID provides the unique identity of services in global domain. The ServiceName presents the service instance’s name. The Category identifies the type of services for end users, such as Navigation Service, Recommendation Service. The InputParameter and OutputParameter aim to delivery the interface parameters from/to the system. The Dependency is used to describe the semantic dependency between the service and the user which is captured, inferred and measured by the service provider. The Dependency has two properties Feature and Value with the form of KeyValue pair: the former is to describe the context objects related to the service interaction, and the latter one is to measure the semantics relevance degree between the two objects that ranges from -1 to 1. For example, the Coat Recommendation Service enumeratess the semantic dependency between the user’s feature, such as Gender, Age, etc., and evaluate the semantics relevance degree between each feature and the service. Table 3 shows an partial example for the Coat Recommendation Service provided by the store. Table 3. An partial example for service description provided by a store ... UUID: S001 ServiceName: Coat Recommendation Service ServiceProvider: Boss speciality store Category: Recommendation Service Feature: Gender DependencyDegree: 0.95 Feature: Age DependencyDegree: 0.8 ...

4.2

Semantic Matching Based on Similarity Measure

The goal of semantic matching between context objects is to determine which user or service profile item with ratings is more relevant for the current context. For example in the scenario above, when Bob wants to enter the shopping store, the Recommendation Service provided by the store may be more appropriate for him based on his location and involved activity. Therefore for each context type, the quantifiable measure of the similarity should be needed to evaluate the semantics relevance between two context objects. Formally we define context (including user profile, situation context, and service) here as a n-dimension vector based on Vector Space Model [15] to represent context object: (1) C = (c1 , c2 , ..., cn )

196

W. Qin et al.

where ci , (i ∈ 1..n) is quantified as a context type (e.g., Activity, Location) from the feature extraction ranging from -1 to 1. We use the Pearson’s correlation coefficient to calculate the similarity weight between two different contexts with their rating values. The similarity of context C1 and C2 is formally defined as, n αi βi C1 · C2 = n i=12 n Similarity(C1 , C2 ) = (2) 2 ||C1 || × ||C2 || i=1 αi i=1 βi where C1 = (αi ), C2 = (βi ), (i ∈ 1..n) and Similarity(C1 , C2 ) returns the relevance of two context over all the item ratings from the system. The advantage of semantic similarity measure is that similarity-based approaches combines the semantics of both the object’s hierarchical (subclass relationship) and relevant non-hierarchical (cross links) components, while the retrieved results from keyword-based approaches have low recall (some relevant items are missing) and low precision (some retrieved item are irrelevant)[6]. The disadvantage of Pearson correlation coefficient is that the correctness of the vector item’s assumption has influenced the similarity between two context objects, while Pearson correlation indicates the strength of a linear relationship between two n-dimensional vectors. Given the user profiles set and situation context captured from the system, the service provision process first select appropriate user profile based on current user’s location or involved activity information. And then, combine with the selected user profile U ∗ ={u∗1 , u∗2 , ..., u∗m } and situation context C ={w1 , w2 , ..., wn } as composite context CC ={v1 , v2 , ..., vp }={u∗1 , u∗2 , ..., u∗m , w1 , w2 , ..., wn }, where p = m + n and a service description S ={s1 , s2 , ..., sp }, the cosine similarity is calculated as p vi si S · CC Similarity(S, CC) = = p i=12 p (3) 2 ||S|| × ||CC|| i=1 vi i=1 si Based on the similarity result, the M services with high ratings are selected as the nominated presented to the end user. The number M depends on the UI design of user’s client device. {S ∗ } = {Uj |maxm {Similarity(S, CC)}}

5

(4)

Design and Implementation Issues

In this section, we discuss design issues for implementing spontaneous service provision combining with user profiles and situation context. 5.1

Captive Portal to Enable Spontaneous Access without Zero Installation and User Profile Automatic Infusion

By means of the captive portal technique, clients can spontaneously discover the services available in the smart space. When the client’s mobile device connects

Combining User Profiles and Situation Contexts

197

to the wireless hotspot, it issues a DHCP request, and the router would assign with an IP for the device, but the device would initially be firewalled. When the client browser is opened, and requests a website, the listener on the router would return the IP address of the captive portal, instead of the requested site. In our solution, clients are seamlessly authenticated at the captive portal and redirected to the Service Portal, where they can discover available services. Since the captive portal mechanism transparently redirects the client to the Service Portal, through DNS manipulation, the client only needs wireless connectivity and a web browser. No additional software or hardware beacons are needed. The process of capturing browser requests and redirecting to the Service Portal is illustrated in Fig. 1. After authentication, the system create a user profile infusion service to transport multiple user profiles from mobile device to the center server automatically, implicitly and securely. 5.2

Thin Client

Aggregation of services and content is done on the server side at a Spontaneous Interaction Server at each location. The information is presented in webpage format to the client browser through HTTP. Existing Wifi-enabled client devices can easily function in such smart environments, since they do not need special hardware or software to detect other devices or existing services.

Service 1

...

Aggregator 1

...

Service 2

Aggregator n

Core

Service n

Aggregator Service 1

Aggregator n

Service 1.1 Service 1.2

Service n.n

Fig. 4. The core is a thin middleware to provide a means to add services as modules. Services installed can be added to the core directly through built-in aggregators, or through aggregator services as sub-services. In the latter case, the sub-services need not be visible to the core layer.

5.3

Aggregation of Services and Contents

The system contains a core with the functionality to install services as modules (see Fig. 4). Whole ranges of services can be added when aggregators based on existing protocols (such as UPnP, Jini, and X10) are added to the core. Some

198

W. Qin et al.

services can also be aggregator services which collate sub-services. However, these subservices may not be visible directly to the core. In the case of the Media Service, it is an aggregator service that also aggregates media content from various media sources, while being a service itself in the home smart space. 5.4

Semantic Matching Combining with User Profiles and Situation Context

Using the aggregation of situation contexts and user profiles, rating or scoring process can be done to generate the Vector description set. The system can calculate the semantic similarities of all kinds of services existed in the smart environment and provide appropriate numbers of personalized service to end users according to the capabilities of mobile device, such as screen display size, media support type, and so on. 5.5

Middleware Support

Building over the Semantic Space [8] and CAMPS [11], the middleware consist of several individual, collaborating module such as ContextWrapper, ContextAggregator, InferenceEngine, KnowledgeBase, ServiceProvision and QueryFilter as shown in Fig. 5.

Fig. 5. Middleware support for service provision

5.6

Initial Prototype Implementation

We have implemented a prototype as Fig. 6(a) shown for the impromptu service discovery and access framework using the ProSyst OSGi service platform as the middleware core. All the assistive services are wrapped as OSGi bundles in the framework. In each logical smart space, we install the captive portal on a Linksys WRT54GL WLAN router. On entering the smart space, a mobile device (Samsung Q1 UMPC)

Combining User Profiles and Situation Contexts

199

is detected with a WLAN card and connected automatically to the router. After launching the web browser in the UMPC, it is automatically directed to the service portal showing the offered services. For aggregation of devices, we implemented a UPnP wrapper service built over Kono’s CyberLink for Java UPnP library, providing web access capabilities. Links generated in the service portal point to the presentation URLs of the UPnP services. For the shopping scenario in a Coat shop, a snapshot of assistive services including a Coat finder and a Coat recommender is shown in Fig. 6(b). By choosing the Coat finder service, the user can locate the coat collections in a certain shelf (Fig. 6(c)).

Fig. 6. Initial prototype implementation: (a) Impromptu service discovery and provision platform; (b) User interface for Coat recommender; (c) A in-store service: Coat finder

6

Conclusion and Future Work

The paper suggests the combination of user profiles and situation contexts to provide a more pervasive service experience in smart assistive environment with mobile device. Leveraging a captive and service portal mechanism, we have allowed mobile clients with a web browser and wireless connectivity to spontaneously discover and access services. With the semantics similarity combination of user profiles and situation context, the scope of assistive services has been reduced efficiently, personalized to user’s interaction intention and adapted to the capabilities of user’s mobile client. In the future work, we will evaluate service provision based semantic similarity measure.

References 1. Lee, C., Helal, S., Lee, W.: Universal interactions with smart spaces. IEEE Pervasive Computing Magazine 5(1), 16–21 (2006) 2. Nakajima, T., Satoh, I.: A software infrastructure for supporting spontaneous and personalized integration in home computing environments. Personal and Ubiquitous Computing, 379–391 (2006) 3. Rukzio, E., Leichtenstern, K., Callaghan, V., Holleis, P., Schmidt, A., Chin, J.: An experimental comparison of physical mobile interaction techniques: Touching, pointing and scanning. In: Proc. UbiComp, pp. 87–104 (2006)

200

W. Qin et al.

4. Universal Plug and Play (UPnP) Forum, UPnP Device Architecture, Version 1.0 (December 2003), http://www.upnp.org 5. Jini.org, http://www.jini.org 6. Yu, S., Al-Jadir, L., Spaccapietra, S.: Matching user’s semantics with data semantics in location-based services. In: Workshop on Semantics in Mobile Environments (SME 2005) (2005) 7. Open Service Gateway Initiative (OSGi), http://www.osgi.org 8. Gu, T., Pung, H.K., Zhang, D.Q.: A service-oriented middleware for building context-aware services. Journal of Network and Computer Applications 28(1), 1–18 (2005) 9. Lin, Z.H., Fu, L.C.: Multi-user preference model and service provision in a smart home environment. In: Proc. CASE, pp. 759–764 (2007) 10. Strang, T., Linnhoff-Popien, C.: A context modeling survey. In: Proc. UbiComp workshop on Advanced Context Modeling, Reasoning and Management (2004) 11. Qin, W.J., Suo, Y., Shi, Y.C.: Camps: A middleware for providing context-aware services for smart space. In: Proc. GPC, pp. 644–653 (2006) 12. DAML-Time, http://www.cs.rochester.edu/∼ ferguson/daml/ 13. OWL-S: Semantic Markup for Web Services, http://www.daml.org/services/owl-s/1.1/ 14. Carmagnola, F.: The five ws in user model interoperability. In: Proc. IUI 2008 Workshop on Ubiquitous User Modeling (2008) 15. Salton, G., Wong, A., Yang, C.: A vector space model for automatic indexing. Communications of the ACM 18(11), 613–620 (1975)

Ubiquitous Phone System Shan-Yi Tsai, Chiung-Ying Wang, and Ren-Hung Hwang Dept. of Computer Science & Information Engineering, National Chung-Cheng University, 621, Chiayi, Taiwan [email protected], {wjy, rhhwang}@cs.ccu.edu.tw

Abstract. Portable smart devices, such as Pocket PC Phones and Smartphones, have become common devices in our daily life. However, they still provide technology-oriented human interface. In future ubiquitous environment, context-awareness is the key to make the human interface of smart devices become human-centric. In this paper, we design and develop a ubiquitous phone system, referred to as UbiPhone, which provides context-aware, human-centric phone service based on rich context acquired from various sensing sources. The most unique feature of UbiPhone is that a user only needs one click, UbiPhone will automatically choose the right communication channel and device to connect to the callee. UbiPhone also provides intelligent feedback service when the callee is not available, such as when to call back according to callee’s calendar, whether to automatically dial back when the callee becomes available, who to contact with if this is an emergency call and the callee is not reachable. In particular, social network is adopted to reason who would be the most appropriate person to help the caller find the callee immediately. Prototype of UbiPhone has been developed to show its feasibility. Keywords: Ubiquitous computing, smart world, context-awareness, Social network, Intelligent phone system.

1 Introduction As rapid advancement of technology, a revolutionary view of our future life articulated by Mark Weiser [1] is bit by bit coming true. Most of digital equipments, such as smart phone, are equipped with the capabilities of computing and communication in the age of information revolution. Through these advanced technologies, a ubiquitous environment full of human-centered services can be fulfilled soon. Contextawareness is the key feature for human-centered services as it enables services adaptive to human needs. By embedding context aware intelligence into devices of our daily life, many tasks could become simplified and efficient. For example, personal mobile phone has become one the necessary devices for our daily life as many of us are using it for communication. Therefore, making the mobile phone context aware could take the initiative to the ubiquitous small world. Much attention has been paid to integrate context and phone system in recent years, e.g., ContextPhone[2], iCAM system [3], Live Contacts project[4], Enhanced F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 201–215, 2008. © Springer-Verlag Berlin Heidelberg 2008

202

S.-Y. Tsai, C.-Y. Wang, and R.-H. Hwang

Telephony [5], Live Addressbook [6], Context Phonebook [7], and Awarenex [8]. ContextPhone[2] is a context aware smart phone project. The components include ContextLogger for recording user’s location movement, ContextContacts for exchanging context with each other and ContextMedia for sharing users’ media. iCAM system [3] displays location information of the callee on the contact list when the caller is making the call. According to the callee’s location information, iCAM also provides the caller an ordered list of phone numbers of the callee, such as house phone, office phone and cellular phone. Similarly, Live Contacts project[4] integrates various contexts to provide a list of communication ways, such as office phone, cellular phone, SMS, IM and e-mail for the caller. Then the caller can choose his favorite way to communicate with the callee from the list. In the aforementioned projects, although various contexts have been used to help the caller to make a phone call, most of them are not able to provide an automatic call to the callee through the most appropriate communication tool. Furthermore, social context, an important context for intelligent phone service, has not been adopted and analyzed. Social context has been applied to several applications in recent years. In a social network [9, 10], any two individuals (persons) in the network may be connected through a link set of intermediate acquaintances. In [11], the social context was used to find the most appropriate participant to talk with during a conference. The social relationship was discovered by analyzing the contents of research papers. In [12], Alex Pentland proposed a perspective on social dynamic which is based on the signals and behavior of interactions among people. He concluded that the measurements of social interactions can form powerful predictors of behavioral outcome in several important social dynamics, such as to get a raise, to get a job or to get a date. VibeFones [13, 14] gained sophisticated understanding about people’s social lives by acquiring those social signals and phone interactions. Observations concluded that user’s social rank can be discovered by analyzing his ordinary social interactions. In this paper, we propose a ubiquitous phone service, referred to as UbiPhone. Our goal is to provide a human-centric context-aware phone service which makes a phone call as easy as a single click on the callee from the contact list. UbiPhone automatically connects to the callee using the most appropriate phone system based on current context information, such as caller and callee’s location, presence status, network status, available phone systems, calendars, social relationship, etc. In case when the callee is not reachable, UbiPhone will prompt the caller when the callee will become reachable according to callee’s calendar. If it is an emergency call, UbiPhone will provide the caller the most possible person that will be nearby the callee or know how to reach the callee immediately based on the social network model we developed. Moreover, we also integrate IM and SMS (Short Message Service) with UbiPhone to provide short message service when a callee is unable to receive a voice-based communication at that time. Obviously, context management is very important to the implementation of UbiPhone. In UbiPhone, a framework for context management, includes context acquisition, context representation, context retrieval, and context inference, is proposed based on state-of-art technologies. It adopts ontology as the underlying context model. RDF (Resource Description Framework) [15, 16] and OWL (Web Ontology Language) [17] are adopted to implement the ontology model. SPARQL [18] is adopted for query and reason context from the ontology.

Ubiquitous Phone System

203

A prototype of UbiPhone has been implemented to demonstrate the feasibility of UbiPhone. PDA phones are used to carry out the human-centric phone service. In this paper, we also demonstrate that the use of social network model is helpful in finding a callee who is currently unreachable via phone. The rest of this paper is organized as follows. Section 2 presents the architecture and design idea of UbiPhone. A prototyping system implementation details are given in Section 3. Performance evaluation results are shown in Section 4. Finally, Section 5 follows with a concluding remark.

2 Design of UbiPhone In this section, we present the architecture of UbiPhone in section 2.1. Context and social models adopted in UbiPhone are described in section 2.2. The intelligent services provided by UbiPhone to achieve human-centric services are described in section 2.3. 2.1 Architecture of UbiPhone The architecture of UbiPhone is shown in Figure 1. UbiPhone Servers consist of service providing server and context management server. Service providing server provides the intelligent human-centric services by the help of inference engine. Context management server mainly acts as a semantic database server to manage raw data collected from the environment. UbiPhone Clients include users’ smart devices, sensors and other network equipments. Between servers and clients, we also designed a middleware as an interface to transmit all messages and signals. Web Services is adopted for the implementation of the middleware. More specifically, both contexts acquired from client’s device and messages to be sent to client from a server are encapsulated into XML-based SOAP messages. Java Web Service (JWS) framework is adopted to generate WSDL description files which are deployed on the server using Apache Tomcat Servlet Container.

Fig. 1. Architecture of UbiPhone

204

S.-Y. Tsai, C.-Y. Wang, and R.-H. Hwang

In Figure 1, the agents provide functions for service enquiry, context retrieval and statistics acquisition. When each agent receives certain events from sensing devices, it will provide the appropriate service. Call agent and messaging agent assist to connect to the right communication channel for users at that time. Status agent can automatically change users’ status according to user’s location, calendar and network status. Schedule agent, location agent and network status agent are designed to transmit user’s calendar, current location and available network system to context management server. Statistics agent derives statistics of phone call records and location logs of a user and updates collected statistics to the context management server. Figure 2 shows the flowchart of how context is acquired from various devices or updated to user devices. For example, the networking status (context) of the user device, such as whether it is connected to GSM network, is acquired by the network agent and sent to the context management server whenever there is a change. Figure 3 shows another example of how presence status is maintained. Context Information Acquire & Update User Request

Location Info. Update

Social Info. Update

Context Info. Response

Context Info. Update

Context Info. Push

Context Info. Update

RFID/GPS Info. Maintain

Context Info. Update

Updated Social Info. Social Info. Update.

Periodical Update Update Social Info.

Push RFID Reader Info. & GPS Info. For each User

Push GSM/IP Network/Bluetooth Status to Server. For each User

Privacy Rule

GSM/IP Network/Bluetooth Support & Status

Server

Privacy Rule

Context Info. Update

Server Push Context Info to Client Ex: New Calendar event

Client Push IM Info & New Calendar event to Server

Server Response Data with Privacy Rule. Ex: Joe’ s Calendar

Client Request Data. Ex: Joe’ s Calendar

Client

Statistics Agent.

Location Agent

Network Agent

Context Info. Maintain

Privacy Context Management Server

Network Status Update

Context Data

Client Agent

User

User Request

Context Info. Update

Update By SPARQL

Fig. 2. Flow chart of context acquisition and update

2.2 Models of UbiPhone We develop context digest model and social network model in UbiPhone to provide human-centric service. 2.2.1 Context Model Context digest model adopts ontology as the underlying technique to manage context. Context digest model acquires context from various agents and smart devices. These

Ubiquitous Phone System

205

Presence Status Maintenance Manual Input

Automatic Status Set By Context Info .

Get PS Contacts Status

Manual Input

Establish a Call Establish a Call

Calling

IM Status BusyΕ Idle etc.

Scheduled

GSM/ IP Network/ Bluetooh Support & Status Location Info.

Call Agent

Privacy Rule

Status Set

Conflict handle & Status Set

Privacy Rule

All Contacts Status

Intelligent Services

Fig. 3. Flow chart of context presence status maintenance

context data can be classified into three types: static, dynamic and statistic. Static context, such as users’ profile and preference, device’s profile seldom changes. Dynamic context, such as user’s presence status and location, updates to context management server frequently and responds to smart device immediately. Statistic context, such as phone call logs, location logs and event logs, is derived from statistics of historic logs. 2.2.2 Social Network Model In UbiPhone, a social network is modeled based on the user’s phone call records. The social network consists of a set of relations between two individuals. The “weight”, ω, of the social relation between any two users is calculated based on Equation (1). ωu ,b = i

Ru ,bi + Rbi ,u

max {Ru , x + R x ,u }

(1)

∀x

where Ra,b is the number of phone calls generated from a to b. The social network model is used by the inference module of service providing server to provide personalized service. Moreover, in order to implement the emergent-contact service, we define Prelation(u,b,t), the possibility to reach u through b at time t, based on social relation as shown in Equation (2).

206

S.-Y. Tsai, C.-Y. Wang, and R.-H. Hwang

Prelation (u , b, t ) = (1 − βω ) ∗ Pu (b | t ) + βω ∗ ωu ,c

(2)

where βω is the weight of social network model in calculating this possibility; Pu(b|t) is the preference of reaching the user u through contact buddy b at time t. Pu(b|t) is pre-configured by u. βω should be set to 1 if the user does not configure his emergentcontact preference Pu(b|t). 2.3 Intelligent Service of UbiPhone Among many intelligence services of UbiPhone, three most interesting intelligent services are ubiquitous call, anycall and emergent-contact service. They are described in detail in the following. All of these services are implemented based rule-based reasoning engine of the service providing server. 2.3.1 Ubiquitous Call Service Ubiquitous call service is aimed to provide a novel human-centric phone call service. When a user desires to call a buddy from his contact list, he just clicks on the buddy’s name. The call agent on the user’s PDA phone will communicate with the service providing server and provide the most appropriate service to the user, either a VoIP call, a GSM call, or a short message will be suggested if the callee is not available to answer the phone call. Figure 4 shows the flowchart of ubiquitous call. When the user initiates the ubiquitous phone call service, the call agent sends the request to the service providing server. The service providing server first checks the presence status of the callee. If the callee is not busy and can be reached via voice communication tools, the service providing server then checks if VoIP is available for both caller and callee. If yes, then it will instruct the call agent to establish a VoIP call. Otherwise, the callee’s phone number is retrieved based on his location information, e.g., office phone number if he is in office or GSM phone number if he is outdoors, the service providing server then instructs the call agent to make a GSM call to the selected phone number. In case the callee is busy, e.g., he in a meeting or class, appropriate text communication tool is suggested to the user through the message agent. The call agent also prompts callee’s expected available time and his presence status to the callee in the latter case. 2.3.2 AnyCall Service AnyCall is similar to the anycast service in IP networks; it provides the user to call the most appropriate buddy within a group. Consider an emergent scenario where the user needs to call anyone in his family group to inform an accident. In such an emergent situation, any buddy from the family group will be able to help the caller. Thus, considering as a human-centric service, AnyCall provides the user an interface to just a single click on the group name, it will connect the phone call to one of the group members who is available at that time to answer the phone. Figure 5 shows the flowchart of AnyCall. In this case, presence status is used to form a list of candidate callees whose status are free now. The location context is then

Ubiquitous Phone System

Fig. 4. Flowchart of Ubiquitous Call

207

Fig. 5. Flowchart of AnyCall

used to decide who is the nearest buddy geographically. If more than one buddy has the same shortest distance, then Ubiphone selects the buddy with the cheapest phone rate to call from the caller. (In Taiwan, phone rate is different due to more than one cellular phone providers.) Finally, after the callee is determined, a procedure similar to ubiquitous call is adopted to find out the most appropriate communication way (or phone number) to call the callee. 2.3.3 Emergent-Contact Service The emergent-contact service helps the caller contact with a third party from callee’s contact list or a phone at a location when the callee is not reachable. Currently social, calendar and location information are used to reason the most appropriate third party in Ubiphone. Specifically, the function Prelation(u,b,t), defined in equation (2), gives us the possibility to reach the callee based on social context. In equation (3), we define another function, Plocation(u,b,t), which gives us the possibility to reach the callee based on the historic location context. Specifically, the presence location of u and b of the last W weeks are recorded in the context management server. The possibility that u and b are at the same location (defined as the same building) is given by the number of times that they are in the same building at time t (the same day of week and the same time of the day) during the last W weeks over the number of weeks (W):

Plocation (u , b, t ) =

| location u (t ) = location b (t ) | W

(3)

The possibility to reach u through b at time t which considers both social and location context is then defined by the following Equation:

208

S.-Y. Tsai, C.-Y. Wang, and R.-H. Hwang

Pcontact ( u, b, t ) = α location ∗ Plocation (u , b, t ) + α relation ∗ Prelation ( u, b, t )

where α location + α relation =1

(4)

Note that equations (1)~(4) are defined as possibility not probability as they are indicators of possibility, not necessary follow the law or semantics of probability.

Fig. 6. Flowchart of Emergent-contact Service

Figure 6 shows the flowchart of emergent-contact service. After the call agent sends the request to the service providing server. The service providing server first retrieves the list, b, of buddies of u who are free now from the context management server. It then retrieves u’s schedule (calendar) at this time. If the schedule exists and the location of u could be retrieved from the schedule context, then find a buddy who is at that location or the phone number of that location to call. Otherwise, u’s current location will be predicted based on his location logs in the past. Equations (1)~(4) will be used to compute the possibility of each buddy in the contact list and the one that has the highest possibility will be suggested to call.

Ubiquitous Phone System

209

3 Prototyping and Implementation In this section, we demonstrate the feasibility of the UbiPhone via prototyping a real system. Our demonstration scenario is made up with four users using Dopod PDA phones equipped with WiFi, GSM, GPS, and RFID. Users joined one or more research groups. An example of contexts of four users is shown in Table 1. Users configured their contact list at will, as shown in Figure 7. Table 1. Users’ context Context Name Status Location Group

Hushpuppy

Lunsrot

Zqq

busy library Web 2.0 group

free EA315 Web 2.0 group

busy EA105 Ubiquitous Learning group

The scenario is demonstrated as follows: Lunsrot read an interesting paper and wished to discuss his novel idea with any buddy in the e-Learning project. He invoked the AnyCall service (implemented as “I’m Feeling Lucky” function on the PDA phone) by a click on the “e-Learning 2.0” and selecting the “I’m Feeling Lucky” function. Since only Septhiorth was free at that time and he was on the Internet, so the UbiPhone connected Lunsrot to Septhiorth via a VoIP connection, as shown in Figure 8. After talking to Septhiorth, Lunsrot continued his survey on Web 2.0 literature. Later on, he read an interesting topic and decided to share with Hushpuppy. So, he invoked the Ubiquitous Call service to Hushpuppy. Since Hushpuppy was busy in library, UbiPhone blocked the voice channel and suggested Lunsrot to use messaging service, as shown in Figure 9. Now, consider an emergent scenario where Septhiorth needs to inform an important message to zqq immediately. Assume that zqq is not reachable because he is busy and his mobile phone power worn out at that time. So Septhiorth invokes the

Fig. 7. The Screen of Four Users’ Smart Devices

Fig. 8. The Demonstration of “AnyCall”

210

S.-Y. Tsai, C.-Y. Wang, and R.-H. Hwang

Emergency-contact service to find out who can help him to reach zqq. Based on the flow chart of Figure 6, Emergency-contact service suggests Septhiorth to call Lunsrot, as shown in Figure 10. Since Lunsrot and zqq is in the same building, so Septhiorth is able to reach zqq through Lunsrot.

Fig. 9. The Demonstration of “Ubiquitous Call”

Fig. 10. The Demonstration of “Emergency-contact Service”

4 Performance Evaluations In this section, we evaluate the performance of emergent-contact service. Consider the scenario where the user masa has been chosen as the callee for this evaluation. Table 2 shows masa’s contact list which includes buddy’s ID, name, and group. Table 3 shows the default preference, Pu(g|t), of each group configured by masa. The experiment is carried out for five weeks. During the first four weeks, masa’s location logs and phone call logs are collected and stored in the context management server. Statistics of the location logs are then derived for each hour based on a weekly cycle. For example, the location logs may show masa was at (EA315, D4304, EA315, EA315) from 8:00 AM to 9:00 AM on Monday during the first four weeks. The possibility that masa will be at room EA315 is then estimated to be 0.75 for the fifth week. On the other hand, masa’s phone call logs are used to build a social network model where a node corresponds to a buddy and the weight of a link between two nodes is computed based on Equation (1). Based on the social network model and statistics aforementioned, the experiment on the fifth week is conducted as follows. For each hour during the fifth week, select a random time and assume masa turned off his PDA phone at that time. Randomly select a buddy to invoke the emergent-contact service and obtain the service from the service providing server. The weighting parameters are setting as follows: αlocation = 0.65, αrelation = 0.35, βω = 0.3. We then evaluate whether the suggested emergentcontact buddy is nearby masa.

Ubiquitous Phone System

211

Table 2. The Experimenters List No. b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 b15 b16 b17 b18 b19 b20 b21 b22

Nickname masa yuwen puzi calais iamshyang bluejam zqq sigwa des33 hopper septhiorth hushpuppy Lunsrot yanmin Nick rhhwang Mei Brian Dad yuan hsg sue

Relation (Group) Myself Graduate Mate Graduate Mate Graduate Mate Graduate Mate Graduate Mate/University Mate/Dorm Mate Graduate Mate/University Mate University Mate University Mate University Mate Younger Graduate Mate Younger Graduate Mate Younger Graduate Mate Younger Graduate Mate Graduate Mate for Doctor's degree Tutorial Professor Family Family Family Friend Friend Friend

Table 3. The Default Preference of Group Defined by Masa Group Name Myself Family Dorm Mate Friend University Mate Graduate Mate Younger Graduate Mate Graduate Mate for Doctor's degree Advisor

Preference 0 1 2 3 4 5 6 7 8

Pu(g|t) 0 1 1/2 1/3 1/4 1/5 1/6 1/7 1/8

Table 4 shows the statistics of location logs for each hour on a weekly cycle. Table 5 shows masa’s location logs of the fifth week. Besides, social network based on masa’s phone call logs of the first four weeks are derived, as shown in Figure 11. Here, the service is considered successful if the emergent-contact buddy is actually nearby masa (in the same building or outdoor location) and can get masa on the phone right away. Table 6 shows the results of our experiment where “s” denotes success and “f” denotes fail. As we can observe that the service is a success for most of the time. In average, the successful rate is 0.7738.

212

S.-Y. Tsai, C.-Y. Wang, and R.-H. Hwang Table 4. The Statistics of Masa’s Location Log During the First Four Weeks

00:00~01:00 01:00~02:00 02:00~03:00 03:00~04:00 04:00~05:00 05:00~06:00 06:00~07:00 07:00~08:00

Mon D4304/ EA315 D4304/ EA315 D4304/ EA315 D4304/ EA315 D4304/ EA315 D4304/ EA315 D4304/ EA315 D4304/ EA315

Tue D4304

Wed D4304/ EA315

D4304

D4304

D4304

D4304

D4304

D4304

D4304

D4304

D4304

D4304

D4304

D4304

D4304

D4304

D4304

D4304

09:00~10:00

D4304

D4304/ EA315

D4304/other s D4304/ others

10:00~11:00

D4304

D4304/ EA501/ EA315

D4304/ EA315/ others

11:00~12:00

D4304

D4304/ EA501/ EA315

D4304/ EA315/ others

12:00~13:00

Meal/ D4304

Meal/ D4304/ EA315

Meal/ D4304/ EA315

13:00~14:00

D4304/ others

D4304/ EA315

D4304/ EA315/ others

14:00~15:00

D4304/ EA315

D4304/ EA315

15:00~16:00

D4304/ EA315

EA315

16:00~17:00

D4304/ S.C.

EA315/ EA105

17:00~18:00

D4304/ S.C.

EA315/ EA105

18:00~19:00

Meal/ D4304

Meal/ EA105

19:00~20:00

D4304

Meal/ EA315

20:00~21:00

D4304

EA315

21:00~22:00

D4304

08:00~09:00

22:00~23:00 23:00~24:00

D4304 D4304

EA315 D4304/ EA315 D4304/ EA315

D4304/ EA315/ others D4304/ EA315/ others D4304/ EA315/ C.Y. D4304/ EA315/ C.Y./S.C D4304/ EA315/ C.Y./S.C D4304/ EA315/ C.Y. D4304/EA3 15/C.Y. D4304/EA3 15 D4304/EA3 15 D4304/ EA315

Thu D4304/ EA315

Fri

Sat D4304/ EA315/K.H. D4304/ D4304 D4304 EA315/K.H. D4304/ D4304 D4304 EA315/K.H. D4304/ D4304 D4304 EA315/K.H. D4304/ D4304 D4304 EA315/K.H. D4304/ D4304 D4304 EA315/K.H. D4304/ D4304 D4304 EA315/K.H. D4304/ D4304 D4304 EA315/K.H. D4304/other D4304/ D4304 s EA315/K.H. D4304/ D4304/ D4304/ others others EA315/K.H. D4304/ D4304/ EA315/ EA315/K.H. EA315/ others / others others D4304/ D4304/ EA315/K.H. EA315/ EA315/ others / others others D4304/ Meal/ EA315/K.H. D4304/ Meal/ / EA315/H.W EA315 others .Room D4304/ D4304/ EA315/K.H. EA315/ EA315/H.W / others .Room others D4304/ Meal/ EA315/ EA315/ D4304/ others S.C./K.H EA315 D4304/ D4304/ EA315/other EA315/ EA315/ s others S.C./K.H D4304

Sun D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H.

D4304/ S.C.

EA315/Driv ing/others

D4304/ EA315/K.H.

D4304/ S.C.

EA315/Driv ing/others

D4304/ EA315/K.H.

Meal/ D4304

EA315/ C.Y./K.H.

D4304/ EA315/K.H.

D4304

EA315/ C.Y./

D4304/ EA315/K.H.

D4304/ EA315/Driv ing D4304/ EA315/Driv ing D4304/ EA315

K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H.

D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H. D4304/ EA315/K.H.

D4304/ EA315 D4304/ EA315 D4304/ EA315 D4304/ EA315

D4304 D4304 D4304 D4304

D4304/ EA315/K.H.

Ubiquitous Phone System

213

Table 5. The Statistics of Masa’s Location Log During the Fifth Week Mon

Tue

Wed

00:00~01:00 01:00~02:00 02:00~03:00 03:00~04:00 04:00~05:00 05:00~06:00 06:00~07:00 07:00~08:00 08:00~09:00 09:00~10:00 10:00~11:00 11:00~12:00

D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304

D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 EA501 EA501

EA315 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304

Thu D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 CCU campus DonateBlood EA315

12:00~13:00

D4304

EA315

EA315

Driving Range

13:00~14:00

D4304

EA315

EA315

Driving Range

14:00~15:00

D4304

EA315

EA315

D4304

15:00~16:00 16:00~17:00 17:00~18:00

D4304 Squash Court Squash Court

EA315 EA105 EA105 Great NightFair

EA315 EA315 EA315 Great NightFair

D4304 D4304 D4304

EA315

18:00~19:00

D4304

19:00~20:00

D4304

EA315

20:00~21:00

D4304

EA315

EA315

D4304

21:00~22:00 22:00~23:00 23:00~24:00

D4304 D4304 D4304

D4304 D4304 D4304

EA315 D4304 D4304

D4304 D4304 D4304

Fri

D4304 D4304

Sat

D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 Ming Hsiung ChengClinic Cheng Clinic EA315 EA315 EA315 Orange Grange Orange Grange Orange Grange D4304 D4304 D4304

Sun

D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304

D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 D4304 EA501 EA501

D4304

EA315

D4304

EA315

D4304

EA315

D4304 Squash Court Squash Court

EA315 EA105 EA105

D4304

Great NightFair

D4304

EA315

D4304

EA315

D4304 D4304 D4304

D4304 D4304 D4304

Table 6. The Experimental Results

00:00~01:00 01:00~02:00 02:00~03:00 03:00~04:00 04:00~05:00 05:00~06:00 06:00~07:00 07:00~08:00 08:00~09:00 09:00~10:00 10:00~11:00 11:00~12:00 12:00~13:00 13:00~14:00 14:00~15:00 15:00~16:00 16:00~17:00 17:00~18:00 18:00~19:00 19:00~20:00 20:00~21:00 21:00~22:00 22:00~23:00 23:00~24:00

Mon

Tue

Wed

Thu

Fri

Sat

Sun

f f f f f s s s s s s s s s s s s s s s s s s s

s s s s s s s s s f s s s s s s s s s s s s s s

s s s s s s s s s f f f s s s s s s s s s s s f

f s s s s s s s s f f s f f f f f f f s s s s s

s s s s s s s s s s f f f f f s s s s f f f s s

f s s s s s s s s s f f f f s s s s f s s s s s

s f f s s s s s s s s s s s s s s s f f s s s s

214

S.-Y. Tsai, C.-Y. Wang, and R.-H. Hwang

Fig. 11. Social Network

5 Conclusion In this paper, we have proposed a ubiquitous phone service, UbiPhone, which emphasizes on providing human-centric context-aware phone service. In particular, three human-centric services have been shown in this paepr to demonstrate a phone call to a buddy or a group could be done as simple as a single click on the contact list. We have also shown how the social context and location context could be used to reach a buddy in emergency using the emergent-contact service. UbiPhone also adopts a general platform to manipulate context. Specially, ontology is used for context modeling and Web Services is used for client-server communication. Software agents have been used as the middleware to send and receive context between client and server such that applications could provide context-aware services in a unified form. Our experimental results show that emergent-contact service could provide the caller the buddy nearby the callee at most of the times. The prototype of UbiPhone also demonstrates the feasibility of the proposed humancentric concepts. Several issues of the proposed UbiPhone require further study. For example, vertical handoff between heterogeneous networks is necessary to make a phone call uninterrupted when the user moves around.

Acknowledgment The research is supported by the NSC 96-2219-E-194-005 and NSC 96-2219-E-194006, National Science Council, ROC.

Ubiquitous Phone System

215

References 1. Weiser, M.: The Computer for the Twenty-First Century. Scientific American, 94–104 (September 1991); reprinted in IEEE Pervasive Computing, 19–25 (2002) 2. Raento, M., Oulasvirta, A., Petit, R., Toivonen, H.: ContextPhone: A Prototyping Platform for Context-Aware Mobile Applications. IEEE Pervasive Computing 4(2), 51–59 (2005) 3. Nakanishi, Y., Takahashi, K., Tsuji, T., Hakozaki, K.: iCAMS: A Mobile Communication Tool Using Location and Schedule Information. IEEE Pervasive Computing 3(1), 82–88 (2004) 4. Henri ter Hofte, G., Otte, R.A.A., Kruse, H.C.J., Snijders, M.: Context-aware Communication with Live Contacts. In: Proceedings of the Conference on Computer-Supported. Cooperative Work, Chicago, Ill, USA (November 2004) 5. Cadiz, J.J., Narin, A., Jancke, G., Gupta, A., Boyle, M.: Exploring PC-telephone Convergence with the Enhanced Telephony Prototype. In: Proceedings of the International Conference for Human-computer Interaction (CHI 2004), New York, NY, USA, pp. 215–222 (2004) 6. Milewski, A.E., Smith, T.M.: Providing Presence Cues to Telephone Users. In: Proceedings of the Conference on Computer-Supported. Cooperative Work, New York, NY, USA, pp. 89–96 (2001) 7. Schmidt, A., Stuhr, T., Hans, G.: Context-Phonebook - Extending Mobile Phone Applications with Context. In: Proceedings of Mobile HCI 2001, Lille, France (September 2001) 8. Tang, J.C., Yankelovich, N., Begole, J., Van Kleek, M., Li, F., Bhalodia, J.: ConNexus to awarenex: extending awareness to mobile users. In: Proceedings of the International Conference for Human-computer Interaction (CHI 2001), New York, NY, USA, pp. 221–228 (2001) 9. Radcliffe-Brown, A.R.: On Social Structure. Journal of the Royal Anthropological Institute 70, 1–12 (1940) 10. Milgram, S.: The Small World Problem. Psychology Today, 60–67 (May 1967) 11. Matsuo, Y., Tomobe, H., Hasida, K., Ishizuka, M.: Mining Social Network of Conference Participants from the Web. In: Proceedings of IEEE Web Intelligence, pp. 190–193 (2003) 12. Pentland, A.S.: Social Dynamics: Signals and Behavior. In: IEEE International Conference on Developmental Learning, San Diego, CA (October 2004) 13. Pentland, A.S.: Socially aware computation and communication. IEEE Computer Society 38(3), 33–40 (2005) 14. Madan, A., Pentland, A.S.: VibeFones: Socially Aware Mobile Phones. In: Proceedings of the 10th IEEE International Symposium on Wearable Computers, pp. 109–112 (October 2006) 15. RDF Primer. W3C document, http://www.w3.org/TR/rdf-primer/ 16. RDF Schema. W3C document, http://www.w3.org/TR/rdf-schema/ 17. OWL. W3C document, http://www.w3.org/2004/OWL/ 18. SPARQL. W3C document, http://www.w3.org/TR/rdf-sparql-query

Utilizing RFIDs for Location Aware Computing Benjamin Becker1, Manuel Huber2 , and Gudrun Klinker2 1 EADS InnovationWorks, Germany Advanced Industrial Design and Visualization 81663 M¨ unchen [email protected] 2 Technische Universit¨ at M¨ unchen, Germany Institut f¨ ur Informatik Boltzmannstraße 3, 85748 Garching b. M¨ unchen [huberma, klinker]@in.tum.de

Abstract. This paper focuses on the possibilities of utilizing infrastructural RFIDs for location aware computing. A key to automated computer aided work is location and situation awareness. This allows the system to operate independently from direct user input, minimizing the impact on the original process. For demonstration purposes a large area tracking system using existing RFID technology and infrastructure is developed and evaluated. As a first possible application we embed the system into an aircraft cabin. The localization procedure is based on the coordinates of the RFID transponders detected by the RFID reader, as well as the orientation information measured by a gyroscope. Position estimation is accomplished through an iterative algorithm, optimizing the position of the antenna’s readability field with respect to the currently visible tags. The average error is below half a meter. Possible use cases range from maintenance assistance to way finding in commercial shops or underground parking lots.

1

Introduction

In order to automatically adapt to the ever-changing contexts in which a user may perform, situation and especially location awareness emerged as a key concept in ubiquitous computing. A system that is aware of the user’s location is enabled to react to the current demands without explicit input and may assist in various novel means. Unfortunately location information is not always easily accessible. Often tracking either relies on public infrastructure which may not be present everywhere, especially indoors, or on specialized infrastructure which usually has to be deployed at a high cost per area. Furthermore the installation of specialized hardware is likely to be unfeasible in certain environments because it usually is obtrusive and distracting. The consequence of these constraints is the idea to use whatever technology is present as infrastructure for position estimation. F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 216–228, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Utilizing RFIDs for Location Aware Computing

217

Fig. 1. Computer Assisted Maintenance Worker

In an industrial scenario it is possible to introduce new technology in a controlled manner and thus it is more accessible in this manner as more diverse technologies are likely to be present. Especially RFID technology, which already has seen some attention in ubiquitous computing, is at the moment predominately used by supply chain management and life-cycle management applications. Therefore RFID technology is expected to become more and more ubiquitous in the near future and thus may be part of our common environments, even possibly enabling its use in domestic environments. We focus on the industrial setup of aircraft cabin maintenance (Figure 1) where we can assume already present RFID tags distributed in the cabin for quality assurance as part of our infrastructure. We will show that using these tags it is possible to perform inside-out tracking and pose estimation. We demonstrate this on a first prototype of an automated maintenance support system. Location awareness herein reduces the need for direct user input since the system is able to detect the current state of the worker. Computer aided maintenance and automated worker support for maintenance tasks in general have already been studied in the past (see for example [1] or [2]) and their benefit has been established. This allows us to mainly focus on the characteristics of the tracking concept including scalability, low impact integration and precision.

218

2

B. Becker, M. Huber, and G. Klinker

Exposition

The following paragraphs will describe the prototype’s overall hardware setup as well as the currently used position estimation algorithm. This is still a prototype and will require further adaption for integration into concrete future scenarios. 2.1

Technical Background

In contrast to existing infrastructural large-area outdoor trackers like GPS and GSM, there is a general lack of scalable and robust large-area trackers working indoors. This paper introduces the possibilities of using RFID technology for localization and tracking in large, dense indoor areas. The prototype illustrates the concept on the example of an aircraft cabin testbed, where RFID tags have been deployed independently for the setup and evaluation of various use cases. As already stated, logistics, quality assurance and life-cycle management are major domains of RFID technology. To match these requirements, tags vary in levels of complexity from simple ID transmitters to smart cards. Passively powered tags use the energy transmitted via the RF field for the communication response whereas active tags depend an additional power source. Currently there are two common physical communication layers [3]: near field (LF at 134.2kHz, HF at 13.56MHz and 27.125MHz ) and far field (UHF at 433MHz or 868MHz (EU) and 915MHz (US), MW 2.45GHz or 5.8GHz ) which especially differ in their operating distance which ranges up to few centimeters in the first case and typically up to 6m in the second case. Active powered tags reach further and often implement additional features like complex encryption - but suffer from a short lifespan and additional maintenance issues due to their battery. Summing up, active tags allow more complex applications at a higher investment per tag. Usually they are used to identify only few but special objects for a short period of time, whereas passively powered tags are often used as a long-term identification of structural parts. For our aircraft cabin scenario we focus on passively powered tags which are being used for long-term quality assurance of structural parts. These support our assumption that RFIDs are massively distributed as part of the infrastructure. Recent use cases show the need for long range readability beyond 1m, from which our proposed tracking method profits much more than it would from short range RFIDs, as these would severely limit the operating volume. 2.2

Related Work

Tracking methodologies are generally classified into outside-in and inside-out tracking. For outside-in tracking a global set of observers determines the positions of all participating entities, where as in inside-out tracking each participant determines its pose independently on its own using landmarks in the environment. Similarly existing approaches for RFID based tracking can be classified into these categories. The inside-out RFID tracking uses distributed tags and a single mobile reader for the position estimation of the reader whereas outsidein uses a certain number of fixed readers to track a set of mobile transponder.

Utilizing RFIDs for Location Aware Computing

219

Outside-in based systems generally suffer from the need to distribute numerous readers in the tracking area, impeding the possible scalability to large areas, due to complex communication needs and high investments. Although outside-in tracking does not match the general constraints of the aircraft use cases, it does use the same technological basis and therefore our system should be able to compete with it. Even commercial applications exist [4], using angle of arrival for the tracking of proprietary, active ultra-wide-band tags. Another system [5] demonstrates the possibility to enable tracking by distance grouping of transponders by varying the antenna’s power. Inside-out based tracking was already used to update a robot’s position estimation [6] by reading tags with known location. Another promising approach [7] is introducing the Super-Distributed RFID Tag Infrastructure. This scenario uses a small robot vehicle equipped with a very short-range RFID reader pointing downwards. The tags are distributed on the floor using their ID to reference a specific position. 2.3

Prototype Scenario

The prototype demonstrates the possibility to achieve 6DOF localization and tracking of a mobile device in an aircraft cabin using a long-range reader and a cloud of distributed passive tags, taking a gyroscope’s current orientation estimate into account. The goal of this application is to allow automated maintenance support based on location awareness. Maintenance of air crafts is time critical as every minute on ground means lost revenue to the airline. Clearly, the more promising approach for our case is an inside-out based tracking system, having no or little impact on the environment by bundling the required hard- and software on a single mobile device. The system’s design for a single reader and inexpensive passive RFID tags allows us to cover a large area with little investment. Besides, often the required infrastructure is already present from other RFID applications as listed in the prior paragraph. Also a mobile computing device can be easily extended with additional sensors like a gyroscope or WLAN tracking, increasing range, robustness and precision of the localization algorithm. Hardware Setup. Since RFIDs are only just being introduced in the aircraft production and maintenance process the tags in the prototype’s cabin mock-up were added supplementarily. This paragraph describes how the tracking system was embedded into the aircraft cabin and gives additional detail about the hardware in use. It is useful to have a large and reliable reading range to increase robustness as well as the position estimation quality. The RFID tags which had already been deployed in the aircraft cabin testbed are passive ultra-high-frequency transponders, which match these requirements (Figure 2, Rafsec G2 ShortDipole at 868Mhz ). The reader used was a Feig LRU2000 UHF Long Range Reader. Since neither this nor any other investigated reader made any tag quality metrics (such as received signal strength indicators) available, each scan only resulted in

220

B. Becker, M. Huber, and G. Klinker

Fig. 2. RFID Tags Distributed in Aircraft Cabin

binary “detected” or “not-detected” information per tag. The miniature longrange reading device (Figure 3) is equipped with a directional antenna. The antenna’s gain increases the reading range along a single axis, allowing this system to detect tags up to a distance of 6.0 meters. As illustrated in Figure 2 (right and lower left), the transponders in this scenario are distributed within the aircraft’s cabin, especially attached to the hatrack, the seating and the floor panels. In the final scenario in a real aircraft cabin usually every major structural part has its own identification tag, giving a far denser distribution. For measuring the current orientation, a gyroscope is directly connected to the antenna (Figure 4) with a precalibrated offset transformation. Basic Localization Concept. The localization procedure is based on two sensors: the RFID reader and the gyroscope. The reader’s output is a list of tags read by the antenna within the last 200 milliseconds. Note that these measurements only constitute the existence of the relevant tags in the reading range of the antenna and do not contain further distance information. The developed RFID tracking system has three basic characteristics: – A large number of RFID tags are attached to structural parts of the cabin. For each of them the exact location and ID are known and available to the system as part of the Digital MockUp (DMU) data.

Fig. 3. RFID Reader

Fig. 4. Xsens Gyroscope

Utilizing RFIDs for Location Aware Computing

221

– A gyroscope measures the current orientation. This is used internally to increase the accuracy of the position estimate and returned as part of the pose estimation. – The reading range of the RFID antenna is taken into consideration. It has to be determined beforehand. The localization is based on the coordinates of the currently detected transponders, which can be derived from the database. The antenna is estimated to be at a position from where it is able to read all currently detected tags best. This is accomplished through an iterative process which imposes requirements on how the detection range is modeled, as seen in the following section. In our case the position of each RFID tag and the corresponding ID was measured in a preprocessing step. Hopefully this can be automated in a future scenario by having predefined IDs and positions of tags on certain constructions or building parts. This assumption is realistic, since RFID technology is in the process of being integrated into the aircraft manufacturing process. The setup of the following algorithm is intended to strongly reduce computational cost of the pose estimation. This is required since mobile devices often lack computational power in favor for mobility and uptime. Antenna Readability Model. Before actually using the system, the detection range of the RFID reader’s antenna has to be determined. Due to missing details on the assembly of the antenna this has to be empirically estimated through experiments. To distinguish the readability field some tests were run. Since the system uses a directional antenna with a rotationally symmetric RF field, the dimension of the testbed was reduced to a 2D plane through the central axis of the antenna. Numerous tags were attached to polystyrene cubes and placed on a grid to evaluate the reading range and probability, resulting in a discrete readability field. For each point p relative to the antenna the position estimation algorithm requires a scaled displacement vector from the antenna model. This vector v(p) indicates in which direction the antenna needs to be moved to increase the chance of detecting a transponder at location p. A scalar d(p), defining the intensity of the displacement vector of a tag at this position is used to increase the influence of a wrongly positioned tag on the pose estimation. Both will be derived from the probability of a tag being read. Since realtime computation of the readability for all tags is computationally expensive, a gradient field was derived by interpolation (Figure 5) which serves as a lookup table. The resulting gradient field is used to scale the displacement, which is shown in Figure 6. Both fields use coordinates in the antenna’s reference frame. The antenna is placed at p = (0, 0) and the point with the maximum reading possibility is located at pmax = (0, 0.85m). Optimization of the Position Estimate. The reader scans the antenna field for visible tags during 100ms slices. Initially the position estimation assumes

222

B. Becker, M. Huber, and G. Klinker

Fig. 5. Antenna Readability

Fig. 6. Displacement

that the antenna is in the centroid of the visible tag positions. At each step a simple optimization procedure is performed. First of all the visible tags are rotated into the reference frame of the antenna by applying the inverse of the orientation measured by the gyroscope. This relative position is projected onto the 2D plane and used to access the readability as well as the displacement field. In the next step the median over all displacement vectors is computed and added to the current position resulting in a new position estimate. The estimated quality of this position is calculated as the sum over the readability of all detected tags, taken from the readability field. Iteratively this procedure optimizes the position estimate until a lower border has been reached. When finished the position as well as the likeliness with the highest quality rating is returned. In case no tags have been detected, the current position is kept untouched. 2.4

Prototype Analysis

This section focuses on the evaluation of the technical properties of the prototype and compares it with other existing setups. Error Analysis. The setup for the test is shown in Figure 7. For the estimation of the ground truth position an optical infrared tracker is used, allowing the reflective marker on the antenna to be localized within millimeter precision. Also the offset between the infrared reflection marker and antenna was calibrated beforehand. This assures that the entire tracking system used for validation and testing introduces a negligible error compared to the expected precision of the RFID tracking. We present here only a one minute excerpt from our test runs, which exemplifies the typical performance of the system. The mobile device was moved in the cabin with varying orientations. The currently visible tags and the resulting position estimation of the RFID tracker as well as the ground truth position

Utilizing RFIDs for Location Aware Computing

223

Fig. 7. Antenna with IR-Marker and A.R.T. Tracking System

Fig. 8. Antenna Field (Median Readability) and Cabin Coordinates

were timestamped and logged. In a post-processing step the recorded data was examined to determine the precision and to analyze the current setup. To differentiate the error induced by the distribution of the tags and the antenna model, the coordinate system’s axes are distinguished with a trailing ‘a’ for antenna or ‘c’ for cabin. In Figure 8, the cabin has the xc axis (red) pointing to the side, the yc axis (green) to the front and the zc axis (blue) to the ceiling. The antenna’s coordinate system has the xa axis pointing to the side, the ya along the antenna’s normal and the upward axis za . The overall performance of our approach can be extracted from Figure 9. It shows the error distributed along all three axes of the cabin as well as the average error across the entire testing time. The output is the raw data taken from the position estimator without any filtering or continuous position estimation for jitter reduction. The Euclidean distance between the RFID tracker’s estimate and the ground truth ranges up to 1.5 meters with an average of 0.48 meters. In the beginning of the test run, the antenna was moved while pointed towards the front of the cabin. This results in a small yc axis error since a more homogeneous distribution of the RFID tags (Figure 8, right) allows the antenna model to fit perfectly with its long reading range along its ya axis. Therefore the effect is homogeneous in both reference systems. In the time range between 10 and 40 seconds the antenna was additionally pointed upwards and downwards while still moving through the cabin, increasing the error along the z c axis. The uncertainty along the other two axes as well as

224

B. Becker, M. Huber, and G. Klinker

Eucl. sum x y z

2.5 error distance in meter

2 1.5 1 0.5 0 -0.5 -1 -1.5 0

10

20

30 time in seconds

40

50

60

Fig. 9. Error in the Test Run in the Cabin’s Coordinate System

the overall error are also increased, since the antenna’s reading field is mainly outside of the cabin’s volume where no tags were placed. Within the final time interval beginning at 40 seconds, the antenna’s position was kept still but was rotated mainly around its za axis and therefore often pointing along the side of the cabin (xc axis). This shifts the error between the yc and the xc axis. The distribution density of the tags towards the side of the cabin is also inappropriate. Not only the distribution and the homogeneity of the tags are important for a proper function of the algorithm, also the number of tags used for the position estimation is of interest. Looking at Figure 10, it is evident that the simple approach to use the centroid of the tag cloud is inferior to the optimization based on the antenna model. The overall median error is reduced from 0.71 to 0.48 meters. It is evident that the possible uncertainty introduced by the antenna’s long reading range in direction of the ya axis could be compensated Initial Centroid

Optimized Position Estimation 1000 x y z

error distance in meter

1.4

x y z

1.2 1

800

600

0.8 400

0.6 0.4

200

0.2 0

number of position estimations

1.6

0 0

1 2 3 4 5 6 7 number of tags for pos. estimate

0

1 2 3 4 5 6 7 number of tags for pos. estimate

Fig. 10. Error in the Antenna’s Reference Frame (x, y, z) and Histogram (gray bars) with Respect to the Number of Detected Tags

Utilizing RFIDs for Location Aware Computing

225

with the orientation based method. The errors on the xa and za axis could be improved only slightly. For the case of zero detected tags the error is high because without measurement information supplied no further estimation can be performed. In the case of a single tag the algorithm’s estimate induces an even larger error. This indicates that a tracking using a motion model, based on preceding estimates, could achieve an increased performance since a good initial estimate seems superior to the estimation based on a single tag. Single tag discoveries often occur when the reader is pointing directly into an obstacle, reducing the range of the RF field. Although the error should be indirectly proportional to the number of tags this can not be derived from our current test results. As the histogram in Figure 10 illustrates, only about 3% of the frames have more than 4 tags detected. Nevertheless it seems obvious that the error increases in a high detection case. A possible reason for this can be traced back to the reader buffering recently read tags when a quick rotation is performed, resulting in more detected tags but partially from an outdated orientation. Although neither the algorithm nor the implementation can be expected to be optimal, the results clarify four main error sources in the system setup: – – – –

inhomogeneous distribution of tags cause the position algorithm to fail low density of RFID tags yield an inferior pose estimation obstacles changing the antenna’s reading probability field increases the error accumulation of visible tags over a certain period of time introduces an error when quick rotations are performed

These characteristics need to be considered in future optimizations of the system. Error Evaluation and Comparison. Considering the simple algorithm and optimization model used, the achieved quality is encouraging. As the results clarify, the antenna model and the position estimation algorithm are not optimal. Nevertheless it is obvious that an algorithm using a representation of the antenna model in combination with the orientation information of a gyroscope can significantly increase the quality of an RFID tracker. For a comparison with existing RFID based tracking systems, the outside-in based approaches are analyzed first. Although they have a totally different target application and differ largely from our setup it is important to see what results are possible when using RFID technology at ideal conditions. Disregarding the already mentioned disadvantages, the results are only of slightly better quality regarding precision of the position estimate. The tracker Ubisense reaches only slightly better results with 95% of the measurements having an error below 30cm [4]. The passive outside-in system landmarc ([5]) achieves an median error value of about one meter. Both systems highly depend on the number of readers used in the tracking area. Inside-out based approaches, as used in the fields of robotics, can benefit from the advantages of continuous tracking with control inputs from the robot’s steering sensors. Besides this, the localization is reduced to a 2D position estimation.

226

B. Becker, M. Huber, and G. Klinker

The system presented in Super-Distributed RFID Tag Infrastructure [7] allows a high quality position estimation. When a tag is detected, the short reading range of the antenna provides a high accuracy localization, giving the robot the chance to correct its internal estimate. This still suffers from a major drawback: When carried by a worker and therefore without control input, continuous tracking fails since the short reading range implicates very seldom tag discoveries. 2.5

Application and Further Use Cases

For our maintenance application the tracking is precise enough to embed a location aware assistance software. It is possible to guide the worker to the correct frame of the aircraft and to priorize the tasks according to his current position, so that maintenance tasks in the current working area are favored and thus minimizes time spent between tasks. Also it is possible to reduce user input which would occupy the worker. The use of a tracking system allows extending the existing workflow of a mechanic. Nowadays it is common to have a digital task list on a mobile device to verify that all important maintenance tasks have been performed correctly. With the additional location sensitivity it is possible to guide the worker towards the next task and automatically give additional information regarding the current step. For further scenarios, it will be interesting to see if RFID technology is feasible for standalone tracking, as supporting tracker for initialization of a more precise but local tracking method (e.g. optical model-based tracking)[8] and for largearea tracking. Also it is promising to evaluate RFID technology as an additional source of information for ubiquitous computing or Augmented Reality applications regarding location awareness and object identification. Furthermore RFID based infrastructure can serve as an additional tracking modality in ubiquitous tracking setups[9], extending it to otherwise inaccessible environments. As stated before RFID technology is expected to become more common and enter areas of our daily life. As a consequence this also can be seen as a low cost tracking infrastructure becoming ubiquitously available which can lead to numerous applications. One such envisioned possibility is assisted wayfinding and navigation in supermarkets. This scenario further is supported by the fact that today many consumer products already start to be tagged by RFIDs, as some preliminary field studies show. Another similar scenario could be the installation of tags in underground parking lots to facilitate customer routing and orientation between parking decks and cars.

3

Conclusion and Outlook

Our system extends the existing approaches to a complete 6DOF tracker by combining the 3DOF orientation of the device with the 3DOF position optimized considering the antenna’s alignment.

Utilizing RFIDs for Location Aware Computing

227

The precision of the position estimate is of similar quality as the existing commercial outside-in tracking systems. Contrary to those approaches, the tracking system presented here allows simple and cost effective scaling for tracking within large areas. A major difference to the existing inside-out tracking systems is that it does not require the control input of a robot to function, allowing it to be applied for tracking mobile devices. This prototype confirms the idea of using RFID technology as a basis for location aware computing. The precision is acceptable for location based applications but leaves room for improvement. Our work shows that there is a high potential in using RFID technology for tracking purposes. For different scenarios it might not be feasible to spend additional money on a gyroscope to reduce the error but use a less directional antenna taking the centroid of the detected tags as pose estimation. Further Improvements. One possible way to improve the application is to examine the readability of tags in detail. Not only the antenna’s characteristic defines the probability of a tag to be detected, but also the orientation of the tag’s dipole in the RF field. This leads to the idea not only to store the position of the distributed RFID tags in a database but extend it to a 6DOF pose. Also different reader hardware setups will be investigated. Other readers allow dynamic reconfiguration of the radio parameters and also do not exhibit the buffering problem observed above. Different antennas with more pronounced directional radiation patterns may also influence the performance of the system. Another influence on the readability is the close environment of the tag, the material it is attached to as well as any material preventing it from beeing read by the reader, in case it is placed in the objects inside. This should also be taken into account when trying to estimate the reader’s position. Currently also ignored, yet of major importance to the position estimation, is the impact of undetected tags on the position estimate. An estimate not only becomes more likely if the detected tags are mapped perfectly to the antenna model but also must decline if many tags within the readability field are invisible. Our next extension will be a Kalman Filter based model that takes a tag list as measurement input and compares this to the readability derived from the antenna model. This allows the algorithm to also exploit undetected tags and to reach an even better estimate. Additionally the Kalman Filter will use an internal motion model for continous tracking.

Acknowledgments This paper profits from the work done by Marcus Fey in his diploma thesis. Our research was supported by the BFS (Bayerische Forschungsstiftung) project trackframe.

228

B. Becker, M. Huber, and G. Klinker

References 1. Ong, S.K., Nee, A.Y.C. (eds.): Virtual and Augmented Reality Applications in Manufacturing. Springer, Heidelberg (2004) 2. Speckmann, H., Ley, M.: Multi-media NDT procedures and online-maintenance assistance in the frame of the european R&D project INDeT (integration of NDT). In: ECNDT 2006, 9th European Conference on Non-Destructive Testing, Berlin, Berlin, Germany, Deutsche Gesellschaft f¨ ur Zerst¨ orungsfreie Pr¨ ufung e.V. (2006) 3. Finkenzeller, K.: RFID-Handbuch, Grundlagen und praktische Anwendungen induktiver Funkanlagen, Transponder und kontaktloser Chipkarten, 3rd edn. Hanser (2002) 4. Steggles, P.: Ubisense Hardware Datasheet (2006) 5. Ni, L.M., Liu, Y., Lau, Y.C., Patil, A.P.: Landmarc: Indoor location sensing using active RFID. In: First IEEE International Conference on Pervasive Computing and Communications, vol. 1, pp. 407–415 (2003) 6. H¨ ahnel, D., Burgard, W., Fox, D., Fishkin, K., Philipose, M.: Mapping and localization with RFID technology. In: Proc. of the IEEE International Conference on Robotics and Automation (ICRA) (2004) 7. Bohn, J., Mattern, F.: Super-Distributed RFID Tag Infrastructures. In: Markopoulos, P., Eggen, B., Aarts, E., Crowley, J.L. (eds.) EUSAI 2004. LNCS, vol. 3295, pp. 1–12. Springer, Heidelberg (2004) 8. Najafi, H., Navab, N., Klinker, G.: Automated initialization for marker-less tracking: A sensor fusion approach. In: Proc. of International Symposium on Mixed and Augmented Reality, Arlington, VA, USA (November 2004) 9. Newman, J., Wagner, M., Bauer, M., MacWilliams, A., Pintaric, T., Beyer, D., Pustka, D., Strasser, F., Schmalstieg, D., Klinker, G.: Ubiquitous tracking for augmented reality. In: Proc. IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2004), Arlington, VA, USA (November 2004)

A Component-Based Ambient Agent Model for Assessment of Driving Behaviour Tibor Bosse, Mark Hoogendoorn, Michel C.A. Klein, and Jan Treur Vrije Universiteit Amsterdam, Department of Artificial Intelligence, de Boelelaan 1081a, 1081 HV Amsterdam, The Netherlands {tbosse, mhoogen, mcaklein, treur}@cs.vu.nl http://www.cs.vu.nl/~{tbosse, mhoogen, mcaklein, treur}

Abstract. This paper presents an ambient agent-based model that addresses the assessment of driving behaviour. In case of negative assessment, cruise control slows down and stops the car. The agent model has been formally specified in a component-based manner in the form of an executable specification that can be used for prototyping. A number of simulation experiments have been conducted. Moreover, dynamic properties of components at different aggregation levels and interlevel relations between them have been specified and verified.

1 Introduction Recent developments within Ambient Intelligence provide new technological possibilities to contribute to personal care for safety, health, performance, and wellbeing; cf. [1], [2], [9]. Applications make use of possibilities to acquire sensor information about humans and their functioning, and knowledge for analysis of such information. Based on this, ambient devices can (re)act by undertaking actions in a knowledgeable manner that improve the human’s, safety, health, performance, and wellbeing. The focus of this paper is on driving behaviour. Circumstances may occur in which a person’s internal state is affected in such a way that driving is no longer safe. For example, when a person has taken drugs, either prescribed by a medical professional, or by own initiative, the driving behaviour may be impaired. For the case of alcohol, specific tests are possible to estimate the alcohol level in the blood, but for many other drugs such tests are not available. Moreover, a bad driver state may have other causes, such as highly emotional events, or being sleepy. Therefore assessment of the driver’s state by monitoring the driving behaviour itself and analysing the monitoring results is a wider applicable option. A component-based ambient agent-based model is presented to assess a person’s driving behaviour, and in case of a negative assessment to let cruise control slow down and stop the car. The approach was inspired by a system that is currently under development by Toyota. This ambient system that in the near future will be incorporated as a safety support system in Toyota’s prestigious Lexus line, uses sensors that can detect the driver’s steering operations, and sensors that can detect the focus of the driver's gaze. F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 229–243, 2008. © Springer-Verlag Berlin Heidelberg 2008

230

T. Bosse et al.

The ambient agent-based system model that is presented and analysed here includes four types of agents: sensoring agents, monitoring agents, a driver assessment agent, and a cruise control agent (see also Figure 1). Models for all of these types of agents have been designed as specialisations of a more general componentbased Ambient Agent Model. Within the model of the driver assessment agent, a model of a driver’s functioning is used expressing that an impaired internal state leads to observable behaviour showing abnormal steering operation and unfocused gaze. The monitor agent model includes facilities to automatically analyse incoming streams of information by verifying them on temporal patterns that are to be monitored (for example, instable steering operation over time). The design has been formally specified in the form of a component-based executable agent-based model that can be used for prototyping. A number of simulation experiments have been conducted. Moreover, dynamic properties of components at different aggregation levels and interlevel relations between them have been formally specified and verified against these traces. The paper is organised as follows. First, the modelling approach is introduced in Section 2. In Section 3 the global structure of the agent-based model is introduced, whereas Section 4 presents a generic ambient agent model. Specialisations of this generic agent model for the specific agents within the system are introduced in Section 5, and in Section 6 simulation results using the entire model are described. Section 7 shows the results of verification of properties against the simulation traces, and finally Section 8 is a discussion.

2 Modelling Approach This section briefly introduces the modelling approach used. To specify the model conceptually and formally, the agent-oriented perspective is a suitable choice. The modelling approach uses the Temporal Trace Language TTL for formal specification and verification of dynamic properties [3], [7]. This predicate logical language supports formal specification and analysis of dynamic properties, covering both qualitative and quantitative aspects. TTL is built on atoms referring to states, time points and traces. A state of a process for (state) ontology Ont is an assignment of truth values to the set of ground atoms in the ontology. The set of all possible states for ontology Ont is denoted by STATES(Ont). To describe sequences of states, a fixed time frame T is assumed which is linearly ordered. A trace γ over state ontology Ont and time frame T is a mapping γ : T → STATES(Ont), i.e., a sequence of states γt (t ∈ T) in STATES(Ont). The set of dynamic properties DYNPROP(Ont) is the set of temporal statements that can be formulated with respect to traces based on the state ontology Ont in the following manner. Given a trace γ over state ontology Ont, the state in γ at time point t is denoted by state(γ, t). These states can be related to state properties via the formally defined satisfaction relation |=, comparable to the Holds-predicate in the Situation Calculus [8]: state(γ, t) |= p denotes that state property p holds in trace γ at time t. Based on these statements, dynamic properties can be formulated in a sorted first-order

A Component-Based Ambient Agent Model

231

predicate logic, using quantifiers over time and traces and the usual first-order logical connectives such as ¬, ∧, ∨, ⇒, ∀, ∃. A special software environment has been developed for TTL, featuring a Property Editor for building TTL properties and a Checking Tool that enables formal verification of such properties against a set of traces. To specify simulation models and to execute these models, the language LEADSTO, an executable sublanguage of TTL, is used (cf. [4]). The basic building blocks of this language are causal relations of the format α → →e, f, g, h β, which means: If then

state property α holds for a certain time interval with duration g, after some delay (between e and f) state property β will hold for a certain time interval of length h.

where α and β are state properties of the form ‘conjunction of literals’ (where a literal is an atom or the negation of an atom), and e, f, g, h non-negative real numbers.

3 Global Structure For the global structure of the model, first a distinction is made between those components that are the subject of the system (e.g., a patient to be taken care of), and those that are ambient, supporting components. Moreover, from an Driver Assessment agent-based perspective (see, agent e.g., [5], [6]), a distinction is made between active, agent Cruise Steering Gaze-focus components(human or artificial), Control Monitoring Monitoring agent agent agent and passive, world components (e.g., part of the physical world or a database). Agents may interact through communication. Interaction between an agent Steering Gaze-focus and a world component can be Sensoring agent Sensoring agent car and environment either observation or action performance; cf. [5]. An action is generated by an agent, and transfers to a world component to have its effect there. An obdriver servation result is generated by Fig. 1. Ambient Driver Support System a world component and transferred to the agent. In Figure 1 an overview of the system is shown. Table 1 shows the structure in terms of different types of components and interactions. 3.1 State Ontologies Used at the Global Level For the information exchanged between components at the global level, ontologies have been specified. This has been done in a universal order-sorted predicate logic

232

T. Bosse et al.

format that easily can be translated into more specific ontology languages. Table 2 provides an overview of sorts and predicates used in interactions at the global level. Table 1. Components and Interactions of the Ambient Driver Support System subject components subject interactions

ambient components

ambient interactions

interactions between subject and ambient components

subject agent subject world component human driver car and environment observation and action by subject agent in subject world component driver observes car and environment driver operates car and gaze ambient agents steering and gaze-focus sensoring agent; steering and gaze-focus monitoring agent; driver assessment agent, cruise control agent communication between ambient agents steering sensoring agent communicates to steering monitoring agent gaze-focus sensoring agent communicates gaze focus to gaze-focus monitoring agent eye-focus monitoring agent reports to driver assessment agent unfocused gaze steering monitoring agent reports to driver assessment agent abnormal steering driver assessment agent communicates to cruise control agent state of driver observation and action by ambient agent in subject world component steering sensoring agent observes steering wheel gaze-focus sensoring agent observes driver gaze cruise control agent slows down or stops engine

Table 2. Ontology for Interaction at the Global Level SORT

Description an action an agent an information element, possibly complex (e.g., a conjunction of other info elements) a world component Predicate Description performing_in(A:ACTION, W:WORLD) action A is performed in W observation_result_from(I:INFO_EL, W:WORLD) observation result from W is I communication_from_to(I:INFO_EL, X:AGENT, information I is communicated by X to Y ACTION AGENT INFO_EL WORLD

Y:AGENT) communicated_from_to(I:INFO_EL,X:AGENT,Y:AGENT)

information I was communicated by X to Y

3.2 Temporal Relations at the Global Level Interaction between global level components is defined by the following specifications. In such specifications, for state properties the prefix input, output or internal is used. This is an indexing of the language elements to indicate that it concerns specific variants of them either present at the input, output or internally within the agent. Action Propagation from Agent to World Component ∀X:AGENT ∀W:WORLD ∀A:ACTION output(X)|performing_in(A, W) → input(W)|performing_in(A, W) Observation Result Propagation from World to Agent ∀X:AGENT ∀W:WORLD ∀I:INFO_EL output(W)|observation_result_from(I, W) → input(X)|observed_result_from(I, W)

A Component-Based Ambient Agent Model

233

Communication Propagation Between Agents ∀X,Y:AGENT ∀I:INFO_EL output(X)|communication_from_to(I,X,Y) →input(Y)|communicated_from_to(I,X,Y)

4 Component-Based Ambient Agent Model This section focuses on an Ambient Agent Model (AAM) used for the four types of ambient agents in the system. These agents are assumed to maintain knowledge about certain aspects of human functioning, and information about the current state and history of the world and other agents. Based on this knowledge they are able to have some understanding of the human processes, and can behave accordingly. In Section 5 it is shown how the Ambient Agent Model AAM has been specialised to obtain models for monitor agents, a driver assessment agent, and a cruise control agent. 4.1 Components within the Ambient Agent Model Based on the component-based Generic Agent Model (GAM) presented in [5], a model for ambient agents (AAM) was designed. Within AAM, as in GAM the component World Interaction Management takes care of interaction with the world, the component Agent Interaction Management takes care of communication with other agents. Moreover, the component Maintenance of World Information maintains information about the world, and the component Maintenance of Agent Information maintains information about other agents. In the component Agent Specific Task, specific tasks can be modelled. Adopting this componentbased agent model GAM, the ambient agent model has been obtained as a refinement in the following manner. The component Maintenance of Agent Information has three subcomponents in AAM. The subcomponent Maintenance of a Dynamic Agent Model maintains the causal and temporal relationships for the subject agent’s functioning. The subcomponent Maintenance of an Agent State Model maintains a snapshot of the (current) state of the agent. As an example, this may model the gaze focussing state. The subcomponent Maintenance of an Agent History Model maintains the history of the (current) state of the agent. This may for instance model gaze patterns over time. Similarly, the component Maintenance of World Information has three subcomponents for a dynamic world model, a world state model, and a world history model, respectively. Moreover, the component Agent Specific Task has the following three subcomponents: Simulation Execution extends the information in the agent state model based on the internally represented dynamic agent model for the subject agent’s functioning, Process Analysis assesses the current state of the agent, and Plan Determination determines whether action has to be undertaken, and, if so, which ones (e.g., to determine that the cruise control agent has to be informed). Finally, as in the model GAM, the components World Interaction Management and Agent Interaction Management prepare (based on internally generated information) and receive (and internally forward) interaction with the world and other agents.

234

T. Bosse et al.

4.2 State Ontologies within Agent and World To express the information involved in the agent’s internal processes, the ontology shown in Table 3 was specified. Table 3. Ontology used within the Ambient Agent Model Ontology element belief(I:INFO_EL) world_fact(I:INFO_EL) has_effect(A:ACTION, I:INFO_EL) leads_to_after(I:INFO_EL, J:INFO_EL, D:REAL) at(I:INFO_EL, T:TIME)

Description information I is believed I is fact true in the world action A has effect I state property I leads to state property J after D property I holds at time T

As an example belief(leads_to_after(I:INFO_EL, J:INFO_EL, D:REAL)) is an expression based on this ontology which represents that the agent has the knowledge that state property I leads to state property J with a certain time delay specified by D. 4.3 Generic Temporal Relations within AAM The temporal relations for the functionality within the Ambient Agent Model are: Belief Generation based on Observation and Communication

∀X:AGENT, I:INFO_EL, W:WORLD input(X)|observed_from(I, W) ∧ internal(X)|belief(is_reliable_for(W, I))

→ →

internal(X)|belief(I)

∀X,Y:AGENT, I:INFO_EL input(X)|communicated_from_ to(I,Y,X) ∧ internal(X)|belief(is_reliable_for(X, I))

→ →

internal(X)|belief(I)

Here, the first rule is a generic rule for the component World Interaction Management, and the second for the component Agent Interaction Management. When the sources are assumed always reliable, the conditions on reliability can be left out. Belief Generation based on Simulation

∀X:AGENT ∀I,J:INFO_EL ∀D:REAL ∀T:TIME internal(X)|belief(at(I, T)) ∧ internal(X)|belief(leads_to_after(I, J, D))

→ →

internal(X)|belief(at(J, T+D))

The last generic rule within the agent’s component Simulation Execution specifies how a dynamic model that is explicitly represented as part of the agent’s knowledge (within its component Maintenance of Dynamic Models) can be used to perform simulation, thus extending the agent’s beliefs about the world state at different points in time. This can be considered an internally represented deductive causal reasoning method. Another option is a multiple effect abductive causal reasoning method: Belief Generation based on Multiple Effect Abduction

∀X:AGENT ∀I,J1, J2:INFO_EL ∀D:REAL ∀T:TIME J1≠J2 ∧ internal(X)|belief(at(J1, T)) ∧ internal(X)|belief(leads_to_after(I, J1, D)) ∧ internal(X)|belief(at(J2, T)) ∧ internal(X)|belief(leads_to_after(I, J2, D)) → → internal(X)|belief(at(I, T-D))

4.4 Generic Temporal Relations within a World For World Components the following specifications indicate the actions’ effects and how observations provide their results. Action Execution and Observation Result Generation in a World

∀W:WORLD_COMP ∀A:ACTION ∀I:INFO_EL input(W)|performing_in(A, W) ∧ internal(W)|has_effect(A,I)

A Component-Based Ambient Agent Model → →

235

internal(W)|world_fact(I)

∀W:WORLD_COMP ∀I:INFO_EL input(W)|observation_focus_in(I, W) ∧ internal(W)|world_fact(I)

→ →

output(W)|observation_result_from(I, W)

∀W:WORLD_COMP ∀I:INFO_EL input(W)|observation_focus_in(I, W) ∧ internal(W)|world_fact(not(I)) output(W)|observation_result_from(not(I), W)

→ →

5 Instantiations of the Ambient Agent Model This section provides instantiations of the Ambient Agent Model for, respectively, Ambient Monitor Agents, a Driver Assessment Agent, and a Cruise Control Agent. 5.1 Ambient Monitor Agents As a refinement of the Ambient Agent Model AAM, an Ambient Monitoring Agent Model AMAM has been designed, and instantiated for steering monitoring and gaze monitoring. Table 4 indicates the components within these monitoring agents. These agents relate temporal patterns of gaze, resp. steering to qualifications of abnormality. Table 4. Ambient Monitor Agent Model: Components Maintenance of Agent and World Information maintenance of model of gaze/steering patterns over time history models Agent Specific Task process analysisdetermine whether a gaze/steering pattern is abnormal plan determinationfor abnormality state decide to communicate to driver assessment agent Agent Interaction Management prepare communication to driver assessment agent

A monitor agent receives a stream of information over time, obtained by observation of a world component or by communication from other agents. Typical sources of information are world parts equipped with sensor systems or sensoring agents that interact with such world parts. Any monitoring agent has some properties of the incoming information stream that are to be monitored (monitoring foci), e.g., concerning the value of a variable, or a temporal pattern to be detected in the stream. As output a monitoring agent generates communication that a certain monitoring focus holds. A monitor focus can be a state property or a dynamic property. An example of a simple type of state property to be used as a monitor focus is a state property that expresses that the value of a certain variable X is between two bounds LB and UB: ∃V [ has_value(X, V) ∧ LB ≤ V ∧ V ≤ UB ]. In prefix notation, this can be expressed as follows: exists(V, and(has_value(X, V), and(LB ≤ V, V ≤ UB))). It is possible to obtain abstraction by using (meaningful) names of properties. For example, stable_within(X, LB, UB) can be used as an abstract name for the example property expressed above by specifying: has_expression(stable_within(X, LB, UB), exists(V, and(has_value(X, V), and(LB ≤ V, V ≤ UB))))

The fact that a property stable_within(X, LB, UB) is a monitor focus for the monitor agent is expressed by: monitor_focus(stable_within(X, LB, UB)). An example of a monitor property is:

236

T. Bosse et al.

∀t [ t1≤t ∧ t≤t2 ∧ at(has_value(X, V1), t1) → ∃t’, V2 t≤ t’ ≤ t+D ∧ V2 ≠V1 ∧ at(has_value(X, V2), t’) ]

This property expresses that between t1 and t2 the value of variable X is changing all the time, which can be considered as a type of instability of that variable. This dynamic property is expressed in prefix notation as: forall(t, implies(and(t1≤t, and(t≤t2, at(has_value(X, V1), t))), exists(t’, exists(V2, and(t≤ t’, and(t’ ≤ t+D, and(V2 ≠V1, at(has_value(X, V2), t’))))

This expression can be named, for example, by instable_within_duration(X, D). It is assumed that the monitor focus on which output is expected is an input for the agent, communicated by another agent. This input is represented in the following manner. communicated_from_to(monitor_focus(F), A, B) communicated_from_to(has_expression(F, E), A, B)

Note that it is assumed here that the ontology elements used in the expression E here are elements of the ontology used for the incoming stream of information. Moreover, note that for the sake of simplicity, sometimes a prefix such as input(X)|, which indicates in which agent a state property occurs, is left out. Within AMAM’s World Interaction Management component, observation results get a time label: observed_result_in(I, W) ∧ current_time(T) → → belief(at(I, T)). Similarly, within the Agent Interaction Management component communicated information is labeled: communicated_from_to(I, X, AMAM) ∧ current_time(T) → → belief(at(I, T)). The time-labeled consequent atoms belief(at(I, T)) are transferred to the component Maintenance of Agent History and stored there. Within the component Process Analysis two specific subcomponents are used: Monitoring Foci Determination, and Monitor Foci Verification. Monitoring Foci Determination. In this component the monitor agent’s monitoring foci are determined and maintained: properties that are the focus of the agent’s monitoring task. The overall monitoring foci are received by communication and stored in this component. However, to support the monitoring process, it is useful when an overall monitor focus is decomposed into more refined foci: its constituents are determined (the subformulas) in a top-down manner, following the nested structure. This decomposition process was specified in the following manner: monitor_focus(F)

→ →

in_focus(F)

in_focus(E) ∧ is_composed_of (E, C, E1, E2)

→ →

in_focus(E1) ∧ in_focus(E2)

Here is_composed_of(E, C, E1, E2) indicates that E is an expression obtained from subexpressions E1 and E2 by a logical operator C (i.e., and, or, implies, not, forall, exists). Monitoring Foci Verification. The process to verify whether a monitoring focus holds, makes use of the time-labeled beliefs that are maintained. If the monitoring focus is an atomic property at(I, T) of the state of the agent and/or world at some time point, beliefs about these state properties are involved in the verification process: in_focus(E) ∧ belief(E)

→ →

verification(E, pos)

Verification of more complex formulae is done by combining the verification results of the subformulae following the nested structure in a bottom-up manner: in_focus(and(E1, E2)) ∧ verification(E1, pos) ∧ verification(E2, pos) in_focus(or(E1, E2)) ∧ verification(E1, pos)

→ →

→ →

verification(and(E1, E2) , pos)

verification(or(E1, E2) , pos)

A Component-Based Ambient Agent Model in_focus(or(E1, E2)) ∧ verification(E2, pos)

→ →

237

verification(or(E1, E2) , pos)

→ →

in_focus(implies(E1, E2)) ∧ verification(E2, pos)

in_focus(implies(E1, E2)) ∧ not verification(E1, pos)

verification(implies(E1, E2), pos)

→ →

verification(implies(E1, E2), pos)

→ → verification(not(E), pos) in_focus(exists(V, E)) ∧ verification(E, pos) → → verification(exists(V, E), pos) in_focus(forall(V, E)) ∧ not verification(exists (V, not(E), pos) → → verification(forall(V, E), pos)

in_focus(not(E)) ∧ not verification(E, pos)

The negative outcomes not verification(E, pos) of verification can be obtained by a Closed World Assumption on the verification(E, pos) atoms. If needed, from these negations, explicit negative verification outcomes can be derived: not verification(E, pos)

→ →

verification(E, neg)

The following relates verification of an expression to its name: verification(E, S) ∧ has_expression(F, E)

→ →

verification(F, S)

Eventually, when a monitoring property E has been satisfied that is an indication for a certain type of abnormal behaviour of the driver, the Monitoring agent will indeed believe this; for example, for the Steering Monitoring Agent: verification(E, pos) ∧ internal(monitoring_agent)|belief(is_indication_for(E, I))

→ →

internal(monitoring_agent)|belief(I)

5.2 Driver Assessment Agent As another refinement of the Ambient Agent Model AAM, the Driver Assessment Agent Model DAAM; see Table 5 for an overview of the different components. For the Driver Assessment Agent, a number of domain-specific rules have been identified in addition to the generic rules specified for the Ambient Agent Model presented in Section 4. Some of the key rules are expressed below. First of all, within the Driver Assessment Agent an explicit representation is present of a dynamic model of the driver’s functioning. In this model it is represented how an impaired state has behavioural consequences: abnormal steering operation and gaze focusing. Table 5. Driver Assessment Agent Model: Components Maintenance of Agent and World Information maintenance of dynamic models model relating impaired state to abnormal steering behaviour and gaze focussing maintenance of model of internal state, abnormality of gaze of driver, and of steering state models wheel Agent Specific Task process analysis determine impaired driver state by multiple effect abduction plan determination for impaired driver state decide to communicate negative assessment to cruise control agent Agent Interaction Management receive and prepare communication (from monitor agents, to cruise control agent)

The dynamic model is represented in component Maintenance of Dynamic Models by: internal(driver_assessment_agent)|belief(leads_to_after(impaired_state, abnormal_steering_operation, D)) internal(driver_assessment_agent)|belief(leads_to_after(impaired_stste, unfocused_gaze, D))

238

T. Bosse et al.

The Driver Assessment Agent receives information about abnormality of steering and gaze from the two monitoring agents. When relevant, by the multiple effect abductive reasoning method specified by the generic temporal rule in Section 4, the Driver Assessment Agent derives a belief that the driver has an impaired internal state. This is stored as a belief in the component Maintenance of an Agent State Model. Next, it is communicated to the Cruise Control Agent that the driver assessment is negative. 5.3 Cruise Control Agent The Cruise Control Agent Model CCAM is another agent model obtained by specialisation of the Ambient Agent Model AAM. It takes the appropriate measures, whenever needed. Within its Plan Determination component, the first temporal rule specifies that if it believes that the driver assessment is negative, and the car is not driving, then the ignition of the car is blocked: internal(cruise_control_agent)|belief(driver_assessment(negative)) ∧ internal(cruise_control_agent)|belief(car_is_not_driving)

→ → output(cruise_control_agent)|performing_in(block_ignition, car_and_environment)

If the car is already driving, the car is slowed down: internal(cruise_control_agent)|belief(driver_assessment(negative)) ∧ internal(cruise_control_agent)|belief(car_is_driving)

→ → output(cruise_control_agent)|performing_in(slow_down_car, car_and_environment)

6 Simulation Results Based upon temporal rules as described in previous section, a specification within the LEADSTO software environment (cf. [4]) has been made and simulation runs of the system have been generated, of which an example trace is shown in Figure 2. In the figure, the left side indicates the atoms that occur during the simulation whereas the right side indicates a time line where a dark box indicates the atom is true at that time point and a grey box indicates false. Note that in the trace merely the outputs and internal states of the various components are shown for the sake of clarity. The driver starts the car and accelerates, resulting in a driving car. internal(car_and_environment)|world_fact(car_driving)

After a short time, between time points 10 and 20, the driver shows signs of inadequate behaviour: the gaze becomes unfocused and steering instable. Over short time intervals an alternation occurs of: output(driver)|performing_in(steer_position(centre), car_and_environment) output(driver)|performing_in(steer_position(left), car_and_environment) output(driver)|performing_in(steer_position(right), car_and_environment)

On the other hand, the gaze focus becomes fixed for long time intervals: output(driver)|observation_result_from(gaze_focus(far_away), driver)

The temporal sequences of these observed steering positions and gaze focus are communicated moment by moment by the respective sensoring agent to the

A Component-Based Ambient Agent Model internal(car_and_environment)|world_fact(car_not_driving) output(driver)|performing_in(start_engine, car_and_environment) output(driver)|performing_in(steer_position(centre), car_and_environment) output(car_and_environment)|observation_result_from(car_not_driving, car_and_environment) internal(car_and_environment)|world_fact(steer_position(centre)) internal(car_and_environment)|world_fact(engine_running) output(car_and_environment)|observation_result_from(steer_position(centre), car_and_environment) output(car_and_environment)|observation_result_from(engine_running, car_and_environment) output(driver)|performing_in(accelerate, car_and_environment) output(steering_sensoring_agent)|communication_from_to(steer_position(centre), steering_sensoring_agent, steering_monitoring_agent) internal(car_and_environment)|world_fact(car_driving) output(car_and_environment)|observation_result_from(car_driving, car_and_environment) internal(driver)|world_fact(gaze_focus(far_away)) output(driver)|performing_in(steer_position(left), car_and_environment) output(driver)|observation_result_from(gaze_focus(far_away), driver) verification(gp(1), pos) output(driver)|performing_in(steer_position(right), car_and_environment) internal(car_and_environment)|world_fact(steer_position(left)) verification(gp(15), pos) output(car_and_environment)|observation_result_from(steer_position(left), car_and_environment) output(gaze_focus_sensoring_agent)|communication_from_to(gaze_focus(far_away), gaze_focus_sensoring_agent, gaze_focus_monitoring_agent) internal(car_and_environment)|world_fact(steer_position(right)) output(car_and_environment)|observation_result_from(steer_position(right), car_and_environment) output(steering_sensoring_agent)|communication_from_to(steer_position(left), steering_sensoring_agent, steering_monitoring_agent) output(steering_sensoring_agent)|communication_from_to(steer_position(right), steering_sensoring_agent, steering_monitoring_agent) output(steering_monitoring_agent)|communication_from_to(abnormal_steering_operation, steering_monitoring_agent, driver_assessment_agent) output(gaze_focus_monitoring_agent)|communication_from_to(unfocused_gaze, gaze_focus_monitoring_agent, driver_assessment_agent) output(driver_assessment_agent)|communication_from_to(driver_assessment(negative), driver_assessment_agent, cruise_control_agent) output(cruise_control_agent)|performing_in(slow_down_car, car_and_environment) output(cruise_control_agent)|performing_in(block_ignition, car_and_environment) internal(car_and_environment)|world_fact(engine_always_off) time

0

10

20

30

239

40

50

60

Fig. 2. Example Simulation Trace

corresponding monitoring agent. The following dynamic monitor property is used as monitor focus within the Steering Monitoring Agent: ∀t [ t1≤t ∧ t≤t2 ∧ belief(at(steer_position(centre), t)) → ∃t’ t≤ t’ ≤ t+D ∧ not belief(at(steer_position(centre), t’))

This property expresses that between t1 and t2, whenever the steer is in a central position, there is a slightly later time point at which it is not in a central position (in other words, the driver keeps on moving the steer). This dynamic property is expressed in prefix notation as: forall(t, implies(and(t1 ≤ t, and(t ≤ t2, belief(at(steer_position(centre), t)))), exists(t’, and(t ≤ t’, and(t’ ≤ t+D, not(belief(at(steer_position(centre), t’))))

In LEADSTO this property was expressed as: is_composed_of(gp(1), forall, t, gp(2, t)) is_composed_of(gp(2, t), implies, gp(3, t), gp(8, t)) is_composed_of(gp(3, t), and, gp(4, t), gp(5, t)) has_expression(gp(4, t), t1≤t) is_composed_of(gp(5, t), and, gp(6, t), gp(7, t)) has_expression(gp(6, t), t≤t2) has_expression(gp(7, t), belief(at(steer_position(centre), t))) is_composed_of(gp(8, t), exists, t’, gp(9, t, t’))

is_composed_of(gp(9, t, t’), and, gp(10, t, t’), gp(11, t, t’)) has_expression(gp(10, t, t’), t≤t’) is_composed_of(gp(11, t, t’), and, gp(12, t, t’), gp(13, t, t’)) has_expression(gp(12, t, t’), t’≤sum(t, D)) is_composed_of(gp(13, t, t’), not, gp(14, t, t’), gp(14, t, t’)) has_expression(gp(14, t, t’), belief(at(steer_position(centre), t’)))

Note that during the process within the Steering Monitoring Agent the overall monitoring focus given by this dynamic property is decomposed into a number of smaller expressions (using the predicate is_composed_of). The top level expression (that is checked by the Steering Monitoring Agent) is called gp(1). The atomic expressions

240

T. Bosse et al.

have the form of a belief that a state property holds at a certain time point (e.g., belief(at(steer_position(centre), t))), or of an inequality (e.g., t≤t2). The following dynamic monitor property is used as monitor focus within the Gaze Focus Monitoring Agent: ∃t ∀t’ [ t ≤ t’ ≤ t+D → belief(at(gaze_focus(far_away), t’)) ]. This property expresses that there is a time period from t to t+D in which the gaze of the driver is focused at a point far away. It is expressed in prefix notation as: exists(t, forall(t’, implies(and(t≤t’, t’≤t+D), belief(at(gaze_focus(far_away), t’))))). Within the LEADSTO model, this property was expressed as: is_composed_of(gp(15), exists, t, gp(16, t)) is_composed_of(gp(16, t), forall, t’, gp(17, t, t’)) is_composed_of(gp(17, t, t’), implies, gp(18, t, t’), gp(21, t, t’)) is_composed_of(gp(18, t, t’),

and, gp(19, t, t’), gp(20, t, t’)) has_expression(gp(19, t, t’), t≤t’) has_expression(gp(20, t, t’), t’≤sum(t, D)) has_expression(gp(21, t, t’), belief(at(gaze_focus(far_away), t’)))

Here, the top level expression (that is checked by the Gaze Focus Monitoring Agent) is called gp(15). Given these monitoring foci, the monitoring agents detect the patterns in this sensor information, classify them as abnormal, and communicate this to the Driver Assessment Agent. By the multiple effect abductive reasoning method, this agent generates the belief that the driver is having an impaired state, upon which a negative driver assessment is communicated to the Cruise Control Agent. The Cruise Control Agent first slows down the car, and after it stopped, blocks the ignition: output(cruise_control_agent)|performing_in(slow_down_car, car_and_environment) output(cruise_control_agent)|performing_in(block_ignition, car_and_environment)

7 Verification of Dynamic Properties This section addresses specification and verification of relevant dynamic properties of the cases considered, for example, requirements imposed on these systems. 7.1 Properties of the System as a Whole A natural property of the Ambient Driver Support System is that a driver with impaired driving behaviour cannot continue driving. The global property is: GP1 No driving when symptoms of impaired driving occur If the driver exposes symptoms that indicate that it is not safe to drive anymore then within 30 seconds the car will not drive and the engine will be off ∀γ:TRACE, t:TIME, R:REAL (unfocused_gaze(t, γ) ∧ abnormal_steering_behaviour(t, γ)) ⇒ ∃t2:TIME < t:TIME + 30 [state(γ, t2, internal(car_and_environment))|= world_fact(car_not_driving)]

This property makes use of two other properties: UG Unfocussed gaze has occurred for some time. In trace γ, during the time period D just before t, the gaze of the driver was focussed at a far distance. ∀t2:TIME ((t2 = t-D)) ⇒ [state(γ, t2, internal(driver)|= world_fact(gaze_focus(far_away))]

AS Abnormal steering behaviour has occurred In trace γ, during a time period P just before t, whenever the steer is in a central position, there is time point within D time steps at which the steer is not in a central position.

A Component-Based Ambient Agent Model

241

∀t:TIME ((t-P-D < t2) ∧ (t2 < t-D) ∧ [state(γ, t2, internal(car_and_environment)|= world_fact(steer_position(centre))]) ⇒ ∃t3:TIME, ((t t}. Fig. 4(b) depicts an example for a region Ω(0.9). Considering Ω(t) it is possible to calculate the probability of taking a correct decision for that region using equation Ω(t) 4. Now the problem is reduced to find one value of t that maximize P r(Cd ) conΩ(t) strained to P r(Cd ) >= ρ0 . If regions of confidence cannot be find that meet the constrain, it is necessary to substitute the decision rule with a more robust one. Another alternative is to include new observable aspects to the system. Specific details for our experiments will be discussed in the next section. When the training phase ends, we have, for each position Ofkix a confidence region Ωkρ0 . During the system-run phase, a mobile device running a self-localization software is allowed to move within the building. Periodically, the software issues one or more inquiry commands, depending on the decision rule adopted in the training phase. Because the mobile device only issues inquiries, there is no effective connections or change of personal informations. Such feature assures the privacy of the node realizing the location. Additionally, we just require the fixed nodes to be able to answer to inquiries. That has a twofold advantage. First, the hardware used to implement a fixed node may be very simple and cost effective. It may implement only the Bluetooth functions related to the inquiry command. Second, it consumes very low power, because the inquiry command is issued only on demand and very rapidly. In our experiments, the fixed node (Bluetooth chip) consumed only 3.3 mW when in inquiry scan mode and 29 mW when answering to an inquiry command, which takes only 625 microseconds. After acquiring information about the environment, the mobile device runs the decision rule to decide in which regions it is to be found. The software uses the information about regions to identify the sector where the mobile device is located. Additionally, each sector is correlated with the spatial information about the building. Though, it is possible to present the localization result as a region on the building floorplan map on the mobile device’s display. The next sections show the realization of a self-localization and personal navigation system using the concept explained here.

4

Experiments

The region-based localization methodology, explained in the last section, was applied to realize an indoor localization and personal navigation system. We describe our experiments considering the two phases of the implementation: training and system-run phases.

264

J.O. Filho et al.

4.1

Training Phase

A section of the Wilhelm-Schickard-Institut building, at the University T¨ ubingen was used as a real scenario for our experiments. Its basic floorplan can be seen in Fig. 5. A grid of points with one meter displacement among each other was set up over the area. During the training phase, the mobile devices were allowed to be positioned only at those points. Four points in this grid, depicted in Fig. 5 as Off1ix , Off2ix , Ofr1ix and Ofr2ix , were chosen to permanently accommodate one Bluetooth Smart Node (BTnodes), developed at the Swiss Federal Institute of Technology (ETH). Those points compose the set of fixed nodes. Each fixed node was placed considering aspects of the building topology, because we wanted to investigate how it later could affect the shape of the respective region of confidence. Off1ix was positioned at the middle of a straight corridor, while Off2ix was placed at a bent corridor corner. The regions of confidence for Off1ix and Off2ix are expected to spread along the corridor, with a smaller coverage in the second case, due to additional reflexions on the corner. Ofr1ix and Ofr2ix where placed inside a small and a big room, respectively. The idea was to investigate how the shape of the region of confidence would spread out of the room.

Fig. 5. Experimental set up. Points in the grid are spaced by 1 meter. Fixed nodes Of ix answer to inquiry commands issued by a cell phone placed at the indicated measuring points on the grid.

Self-Localization in a Low Cost Bluetooth Environment

265

Fig. 6. Probability maps for the answered inquiry commands

First task in the training phase is to measure the probability of the inquiry event for each point in the grid. That was accomplished executing the following procedure at each point. First, a BTnode and a Sony Ericsson V600i mobile devices were placed at the point. We used two different devices to keep track on the possible variation of using diverse equipments. Each device executed one hundred inquiry commands. In each round, it is logged which fixed nodes answered the command. When all the commands were issued, the rate of successful inquiries may be determined individually for each fixed node. Such rate is used as an estimator for Ok

the probability pωif ix , explained in section 3. The result of this procedure can be seen in Fig. 6, considering each fixed point individually. The variation on the rate of successful inquiries was very small between the two applied devices. The mobile devices obtained a high rate (nearly 100%) of answered inquiries from Off1ix and Off2ix , when they were located in the corridor. That is because they were in the line-of-sight of the fixed BTnodes. When leaving the line-of-sight or when in presence of obstacles, a fast drop-off appears and the mobile devices can hardly reach the fixed BTnodes. We assumed that the fixed nodes Ofr1ix and Ofr2ix would answer to all the inquiries done within the room they are placed with probability 1. In both cases there is a region in the corridor with a radius of about 10 meters where the probability to get an inquiry response is still high. Outside these bounds this probability vanishes very rapidly. During our experiments, the doors were kept open to avoid attenuation of the inquiry signal. However, we stated that doors and metalic obstacles do affect the measured probability and a detailed study about their effects is part of our on going work.

266

J.O. Filho et al.

Regions of Confidence. The regions of confidence Ωkρ0 for each fixed node Ofkix should now be determined. In order to do that, it is necessary to fixate the desired degree of confidence ρ0 and adopt one decision rule. For the remaining of this work, we arbitrarily set ρ0 = 0.9, which guarantees decisions with at least 90% of confidence for all regions. Additionally, we investigate two possible decision rules. The first decision rule is given in equation 5. It considers that the mobile device is in region Ωkρ0 if the fixed node Ofkix answers one inquiry command issued by the mobile device.  1 , if Ofkix answered the inquiry. (5) d1 (one inquiry) = 0 , if Ofkix did not answered the inquiry. The second decision rule considers the last three inquiry commands issued by the mobile device. It decides that such mobile device is in region Ωkρ0 if the fixed node Ofkix answered to at least two of the three commands. It can be described as  d2 (three inquiries) =

1 0

, if Ofkix answered the at least two inquiries. , if Ofkix answered less than two inquiries.

(6) We are now able to determine the regions of confidence for one given fixed node Of ix . As an example, we consider the fixed node Off1ix . We grouped the points using a threshold value t as explained in section 3. Fig. 7 shows the obtained value for the degree of confidence ρ as a function of t ∈ [0, 1] considering d2 as the decision rule. The maximum probability of making a correct decision is 96.9%. It is achieved when t = 56. Our region includes every point of the measured grid 1 0,9 0,8

degree of confidence

0,7 0,6 0,5 0,4 0,3 0,2 0,1 0

threshold t

Fig. 7. Degree of confidence for Ω(t) of Off1ix

Self-Localization in a Low Cost Bluetooth Environment

267

where the number of answered inquiry commands lies between 100 and 56. For this region, we can take a correct decision with a probability 96.6%. Interestingly, that result also stands for regions defined with threshold t between 56 and 31. We keep using t = 56, which corresponds to the smallest regions with the higher degree of confidence. Smaller regions lead to greater accuracy. The same procedure was taken on determining the regions for the other fixed nodes. Table 1 shows the maximal value for the degree of confidence reached when using decision rules d1 and d2 . Decision rule d1 fails to provide regions of confidence with ρ >= ρ0 = 0.9 in two cases. Therefore, it should be rejected. Decision rule d2 , however, was able to provide adequate regions for all fixed nodes. It was adopted to be used in the system-run phase. The threshold value t used to build the region of each fixed node is also indicated on table 1. The regions defined for Off1ix , Off2ix , Ofr1ix and Ofr2ix may be seen in Fig. 8. Those regions have degree of confidence ρ equal to 0.969, 0.907, 0.907 and 0.941, respectively. Though, they are all regions of confidence Ω 0.9 . Table 1. Maximal degree of confidence according to decision rule Fixed Node

d1

d2

Final Threshold (t) when using d2

Off1ix Off2ix Ofr1ix Ofr2ix

94.5% 87.7% 85.7% 90.4%

96.9% 90.7% 90.7% 94.1%

56% 51% 51% 52%

Fig. 8. Regions of confidence calculated for Off1ix (top left), Off2ix (top right), Ofr1ix (bottom left) and Ofr2ix (bottom right)

268

J.O. Filho et al.

After the training phase, the environment is prepared for running the selflocalization and navigation system. The fixed nodes are installed and the regions were already determined. The system-run phase for our scenario is explained in details in the next section.

5

Running the Localization and Navigation System

The information for each region of confidence, obtained in the training phase, were merged under one unique building floorplan map. The result may be seen in Fig. 9(a). The floor plan view allows us now to split the space into sectors. Each sector corresponds to an area on the space ”covered” by one or more regions of confidence. Therefore, it is possible to relate each sector with the corresponding regions that cover it. Such information is tabulated and will be available to be used by the self-localization software. As an application example, a self-localization software was implemented using Java 2 Micro Edition and JSR-82 Bluetooth API. The implementation in Java allows the software to run on nearly every mobile device like a mobile phone or a PDA. The Java application has got two main functionalities. First, a localization mode enables the user to find out his location in the building. Second, a navigation mode allows the user to choose a destination. The system will navigate him to the selected place. When the localization mode is active, the software starts the Bluetooth service at the mobile device and issues three inquiry commands. The system analyzes the inquiry responses to find out which fixed BTnodes have answered and how often. Due to our decision rules a BTnode must answer at least two times to get a valid predicate. Answers of different regions are combined to determine in which sector the mobile device is to be found. After this evaluation, the resulting sector

(a) Regions overlap to form sectors on the building plan.

(b) Software output screen for the localization mode.

Fig. 9. Example system for localization and navigation

Self-Localization in a Low Cost Bluetooth Environment

269

is displayed. It takes about 30 seconds from starting the application to obtaining the actual location. Fig. 9(b) shows the output screen for the localization mode. Up to three mobile devices were allowed to start inquiry commands simultaneously in order to investigate concurrency issues in the system. However, there were not observed colision or delay problems. The navigation functionality is an extended localization mode, where the software on the mobile device periodically issues inquiry commands. After each round, it executes the decision rule d2 and refreshes the output screen accordingly. The whole process takes about 5 seconds. Note that all the operation is executed by the software on the mobile device. With the small, battery operated Bluetooth landmarks a minimal infrastructure hardware is required. Even though, the system is reliable enough to support localization and navigation services. Accuracy in the localization may be customized for each building by introducing new fixed nodes.

6

Conclusion and Future Work

We presented a low cost, low power, Bluetooth based self-localization and navigation system. The proposed region-based localization concept is suitable to be completely executed on the mobile device to be localized. Nevertheless, a high degree of confidence on the localization is achieved. Self-localization, without the need of changing personal information with the environment, makes our approach a safe and privacy aware solution. Our experience shows, that it is possible to build reliable, broad usable and low cost personal navigation systems using Bluetooth. The system was implemented in a real scenario at the University T¨ ubingen. It uses a Java software, suitable to be executed in any Java capable mobile device, such as mobile phones and PDAs. Accuracy is yet a point to be improved. That may be achieved optimizing the position of the fixed Bluetooth nodes that compose the environment infrastructure. A methodology to optimally place these nodes, investigating the influence of obstacles such as doors, and how the system behaviour when simultaneously operated by several mobile devices is part of our actual ongoing work.

References 1. Sommer, J., Hoene, C.: Indoor positioning applications. localisation in wireless environments using bluetooth and wlan. In: Third Heidelberg Innovation Forum (2006) 2. Aitenbichler, E., M¨ uhlh¨ auser, M.: An ir local positioning system for smart items and devices. In: ICDCSW 2003: Proceedings of the 23rd International Conference on Distributed Computing Systems, Washington, DC, USA, p. 334. IEEE Computer Society, Los Alamitos (2003) 3. Fukuju, Y., Minami, M., Morikawa, H., Aoyama, T.: Dolphin: An autonomous indoor positioning system in ubiquitous computing environment. In: WSTFES 2003: Proceedings of the IEEE Workshop on Software Technologies for Future Embedded Systems, Washington, DC, USA, p. 53. IEEE Computer Society, Los Alamitos (2003)

270

J.O. Filho et al.

4. Teker, U.: Realisierung und Evaluation eines Indoor-Lokalisierungssystems mittels WLAN. PhD thesis, Universit¨ at Bremen, Diploma thesis (2005), www.uni-bremen.de 5. Otsason, V., Varshavsky, A., LaMarca, A., de Lara, E.: Accurate gsm indoor localization. In: Beigl, M., Intille, S.S., Rekimoto, J., Tokuda, H. (eds.) UbiComp 2005. LNCS, vol. 3660, pp. 141–158. Springer, Heidelberg (2005) 6. Nilsson, M., Hallberg, J., Synnes, K.: Positioning with Bluetooth. In: 10th International Conference on Telecommunications ICT 2003, pp. 954–958 (2003) 7. Forno, F., Malnati, G., Portelli, G.: Design and implementation of a bluetooth ad hoc network for indoor positioning. IEE Proceedings - Software 152(5), 223–228 (2005) 8. Anastasi, G., Bandelloni, R., Conti, M., Delmastro, F., Gregori, E., Mainetto, G.: Experimenting an indoor bluetooth-based positioning service. In: ICDCSW 2003: Proceedings of the 23rd International Conference on Distributed Computing Systems, Washington, DC, USA, p. 480. IEEE Computer Society, Los Alamitos (2003) 9. di Flora, C., Ficco, M., Russo, S., Vecchio, V.: Indoor and outdoor location based services for portable wireless devices. In: ICDCSW 2005: Proceedings of the First International Workshop on Services and Infrastructure for the Ubiquitous and Mobile Internet (SIUMI), Washington, DC, USA, pp. 244–250. IEEE Computer Society, Los Alamitos (2005) 10. Madhavapeddy, A., Tse, A.: A study of bluetooth propagation using accurate indoor location mapping. In: Beigl, M., Intille, S.S., Rekimoto, J., Tokuda, H. (eds.) UbiComp 2005. LNCS, vol. 3660, pp. 105–122. Springer, Heidelberg (2005) 11. Wendlandt, K., Robertson, P., Berbig, M.: Indoor localization with probability density functions based on bluetooth. In: IEEE (ed.) PIMRC 2005, pp. 2040–2044. VDE Verlag (2005) 12. Subramanian, S.P., Sommer, J., Schmitt, S., Rosenstiel, W.: SBIL: Scalable indoor localization and navigation service. In: WCSN 2007: Third International Conference on Wireless Communication and Sensor Networks, pp. 27–30 (2007) 13. Bahl, P., Padmanabhan, V.N.: RADAR: An in-building RF-based user location and tracking system. In: INFOCOM, vol. (2), pp. 775–784 (2000) 14. Niculescu, D., Nath, B.: Vor base stations for indoor 802.11 positioning. In: MobiCom 2004: Proceedings of the 10th annual international conference on Mobile computing and networking, pp. 58–69. ACM Press, New York (2004) 15. G¨ unther, A., Hoene, C.: Measuring round trip times to determine the distance between wlan nodes. Networking, 768–779 (2005) 16. Vorst, P., Sommer, J., Hoene, C., Schneider, P., Weiss, C., Schairer, T., Rosenstiel, W., Zell, A., Carle, G.: Indoor positioning via three different rf technologies. In: 4th European Workshop on RFID Systems and Technologies (RFID SysTech 2008), Freiburg, Germany (2008) 17. Galstyan, A., Krishnamachari, B., Lerman, K., Pattem, S.: Distributed online localization in sensor networks using a moving target. In: IPSN 2004: Proceedings of the third international symposium on Information processing in sensor networks, pp. 61–70. ACM Press, New York (2004) 18. Guha, S., Murty, R., Sirer, E.G.: Sextant: a unified node and event localization framework using non-convex constraints. In: MobiHoc 2005: Proceedings of the 6th ACM international symposium on Mobile ad hoc networking and computing, pp. 205–216. ACM Press, New York (2005)

Penetration Testing of OPC as Part of Process Control Systems Maria B. Line1, Martin Gilje Jaatun1, Zi Bin Cheah2, A.B.M. Omar Faruk2, Håvard Husevåg Garnes3, and Petter Wedum3 1

SINTEF ICT, N-7465 Trondheim, Norway {maria.b.line,martin.g.jaatun}@sintef.no 2 Kungliga Tekniska Högskolan, Stockholm, Sweden {zbcheah,aofaruk}@kth.se 3 Google, Trondheim, Norway {hgarnes,wedum}@google.com

Abstract. We have performed penetration testing on OPC, which is a central component in process control systems on oil installations. We have shown how a malicious user with different privileges – outside the network, access to the signalling path and physical access to the OPC server – can fairly easily compromise the integrity, availability and confidentiality of the system. Our tentative tests demonstrate that full-scale penetration testing of process control systems in offshore installations is necessary in order to sensitise the oil and gas industry to the evolving threats. Keywords: Information Security, Process Control, Penetration Testing, OPC.

1 Introduction Process control systems (PCS) are used to support and control a specific industrial process. The setting for the kind of system discussed in this paper is oil production on offshore installations. Traditionally, hardware and software components in such systems were proprietary; designed for use in a specific context. The current trend is heading towards commercial off-the-shelf technologies because of the goal of integrated operations, which means extensive cooperation between onshore and offshore staff, and the possibility of controlling installations completely from onshore. This transition results in exposure to an entirely new set of security threats. Many components and applications in use were developed before this transition without any thought of making them intrusion-proof, because the access used to be more limited, both in the physical and the logical way. With respect to security, the ideal solution would be to build all parts of a process control system from scratch, taking all new threats into account. In reality this is not achievable; economically it would be a disaster for the operators, and a set of threats relevant today may change during the period of development. Also, there is the fact that new security flaws are usually introduced when developing new systems. Hence the brand new systems would not be perfect anyway. F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 271–283, 2008. © Springer-Verlag Berlin Heidelberg 2008

272

M.B. Line et al.

The more realistic solution is to add security functionality or security measures where they are needed. In order to identify where the needs are, several components and applications already in use must be analyzed in detail. This paper presents how a test network was set up to simulate a process control system, and how intrusion tests were performed towards OPC. The concept of OPC is described in the following section. 1.1 OPC – OLE for Process Control OLE1 for Process Control (OPC) [1] is a mechanism for exchanging process control data among numerous data sources in process control system. OPC is implemented as a client-server-architecture. The OPC server aggregates data from available control units in the system. It is capable of reading and writing values from/to these units, and offers mechanisms for interaction with these values to an OPC client. OPC is widely used in process control systems due to its stability and reliability. Also, it allows transmission of double precision real numbers, which other protocols usually do not allow without rewriting [2]. The OPC interface is defined on Microsoft’s OLE/COM (where COM stands for Component Object Model) [3], and most OPC servers run on MS Windows servers, even though there exist some implementations for other operating systems, such as OpenSCADA. The OPC interface is implemented as a set of function calls to COM objects defined by the OPC server, and can be accessed by DCOM (Distributed COM) from other computers. Hence, OPC does not implement security, but makes use of the security offered by DCOM. We have used OPC Tunneller and OPC Simulation Server from Matrikon2 in our simulated process control system. The reason for choosing Matrikon is that it is a freely available and fully functioning OPC-server for use in a non-production setting. The simulation server is normally used for testing OPC connections and services before production servers are put to use. It is therefore considered to be a suitable alternative for a real OPC server. 1.2 Penetration Testing Penetration testing is a way of evaluating the security of a system by simulating attacks by a malicious user against the system. Such testing can be performed in different ways; by running known vulnerability scanners like Nessus3 and Nmap4, or more as research and testing of new and unknown vulnerabilities. Nessus is well-suited for testing systems in production, and it is low-cost. It is continuously updated with new modules for newly discovered vulnerabilities, and running Nessus gives a good indication for how secure a system is, without giving a 100% complete result, which is quite impossible to achieve in any ways. Testing of unknown vulnerabilities requires extensive knowledge of the system in focus and is hence out of the question for our tests this time, as time and resources 1

OLE stands for Object Linking and Embedding. http://www.matrikonopc.com 3 http://www.nessus.org 4 http://insecure.org/nmap/ 2

Penetration Testing of OPC as Part of Process Control Systems

273

puts limitations on our work. In the future we might be able to move on to deeper analyses of OPC including testing of unknown vulnerabilities. 1.3 Our Tests in Brief Our aim is to show how a malicious user with varying privileges – outside the network, access to the signalling path and physical access to the OPC server – can fairly easily compromise the integrity, availability and confidentiality of the system. Resources available to us include a lab where we can set up a test network that simulates a process control system, and software freely available on the Internet. We define a blue team and a red team, where the blue team is responsible for setting up the test network and detecting intrusions, and the red team will act as malicious hackers and run different sets of exploits/attacks. 1.4 Paper Outline Section 2 describes how the test network was setup. Section 3 is about vulnerabilities that apply to OPC, which will be exploited. Section 4 describes the performed attacks together with achieved results, while section 5 discusses our findings. We offer concluding remarks and directions for further work in section 6.

2 Test Network Setup Blue team set up a simulated process control system in a test lab [4]. The topology reflects a typical offshore network inspired by the SeSa method [5]. The network is divided into three subnets, as follows: • DMZ subnet • Admin subnet; with an OPC client • Process Network subnet; with an OPC server DMZ is an abbreviation for demilitarized zone. In common computer networks, a DMZ contains an organization's services that are available for access from untrusted networks, typically the Internet. Services in a DMZ are usually needed by external hosts in order to communicate successfully with hosts inside the network and vice versa. The purpose of the DMZ is to add an additional layer of security to an organization's network by having services that are frequently accessed housed in this layer. The Admin layer models a typical process control system setup where hosts in the Admin Network are privileged users that usually have access to the Process Network layer. In order for users in the Admin network to access the Process Layer services, they should have the proper credentials to do so. For example, a manager in the Admin layer might use OPC client software such as a Matrikon OPC Client to access information from an OPC server (which is placed in the Process Layer). In order for the manager to successfully do this, he has to have the proper login/password credentials.

274

M.B. Line et al.

The Process Network layer is the deepest layer of the system and houses the most critical services in an offshore network. If an attacker manages to take control over this network, he or she has succeeded in compromising the entire network. The specifications of the computers used in the network are listed in Table 1 below, and the topology is illustrated in Fig. 1.

Fig. 1. The network topology as set up by the blue team Table 1. Computer hardware specification Host Name

Operating System

# NICs

Gateway Honeywall DMZ Gateway DMZ Host Admin Gateway OPC Server

Linux Fedora Core 6 Linux Fedora Core 6 Linux Fedora Core 6 Linux Fedora Core 6 Linux Fedora Core 6 Windows XP SP2

3 3 2 1 2 1

Penetration Testing of OPC as Part of Process Control Systems

275

The honeywall and the router are for administrative and monitoring purposes. The OPC client is placed on the Admin subnet, while the OPC server resides on the process network subnet. 2.1 The Honeywall A honeypot is a host set up to detect, monitor and/or trap attackers, and a honeywall is a host that is usually placed between honeypot(s) and non-honeypot components in a network. It is regarded as a bridge between honeypots and regular networks segments. The two main tasks for a honeywall are traffic monitoring and intrusion prevention. Our honeywall has an IP address on only one interface, and we use the gateway to block anyone trying to access it. This interface is used as a remote management interface, while the other two interfaces (without IP addresses) act as the bridge. The bridge operates like a hub; passing traffic along from one interface to the next. As a honeywall, it also reads, copies, and analyzes all the passing traffic. A honeywall should be “non-existent” from a logical view. No one will know there is a honeywall host present unless that person enters the laboratory and sees our network. 2.2 The Gateway All traffic that enters or exits the network has to go through the gateway. If the gateway crashes, it means that the connection between our test network and the Internet is broken. In addition, the gateway performs firewall filtering and Network Address Translation (NAT). The purposes of the main gateway are: • • •

Internet entry-exit point Firewall packet filtering Firewall NAT

Our network provides an open port (21379) for the OPC client to connect to our OPC server using OPC tunnelling. Furthermore, the services SSH (port 22) and HTTPS (port 443) are also open for remote connection. An attacker might wish to exploit such open services and try to attack remotely from the Internet. We selected iptables5 as our choice of firewall implementation due to its flexibility. The firewall not only blocks traffic, but shapes traffic, too. Traffic can be denied or allowed based on port numbers, host names and IP addresses. For our case we use the filter table and NAT table in iptables. In the filter table of iptables, we used ALLOW ALL. Any traffic may come in and out of the network. The only exception is that we have configured our gateway to deny access to the private IP address of 10.0.0.X. The 10.0.0.X is the private network between the gateway and the honeywall; we want this subnet to be logically invisible so that no one can access this subnet. Usually, ICMP (Internet Control Message Protocol) is used to ping a host to determine if it is alive. We have configured the firewall to reply with “icmp port unreacheable” when anyone tries to ping the 10.0.0.X subnet. This will let others think that the subnet does not exist. The only way 5

http://www.netfilter.org

276

M.B. Line et al.

to access this subnet is via HTTPS from the Internet. In this case, the outside host will not know that the subnet exists because NAT has already translated the addressing. The NAT table in iptables allows us to forward traffic to certain ports to ports that we decide. Since we only have one public IP address, NAT directs the traffic to the appropriate destinations. In our configuration we forward traffic meant for the ports 3389 (RDP)6, 21379 (OPC Tunneller) and 135 (RPC)7 to the OPC server, and 443 (HTTPS)8 to the honeywall. 2.3 Intrusion Prevention We do not want an attacker that has successfully compromised our network to launch attacks from our network against others. However, we do not want to deny all outbound traffic, only malicious traffic. Outbound informational queries, such as ICMP ping, finger query, or a simple HTTP GET command should be allowed. To realize this we deploy a special snort function called snort_inline9 that is integrated in the Honeywall. This function lets us limit outbound, but welcome inbound, malicious traffic. One way to limit outbound traffic is to drop all outbound malicious packets; another is to limit the amount of outbound traffic per time unit – we decided to do both. Limiting outbound traffic is especially important regarding preventing Denial-ofService attacks from our network towards others. We do not set the outbound traffic to zero, as this will arouse the attacker's suspicion as to why there is no outbound connections allowed. 2.4 OPC By default, Matrikon OPC Tunneller does not use encryption or authentication to connect with another OPC Tunneller. We can, however, use a shared secret to encrypt and authenticate communication between client and server side OPC Tunnellers. This option was not employed in our tests. Throughout the penetration testing we have assumed that the IP-address of the OPC server is known to the attackers (red team). 2.5 Red Team’s Equipment Red team set up a Matrikon OPC Simulation Server with Windows XP SP2 and all available updates. The server was set up as described in the guide published by CERN [6]. The password for this server was known by the red team, as the login service was not to be tested. This server is in addition to the blue team simulation OPC-network. Besides this test server, red team operated with two regular computers and a switch that connected their units with “the Internet”, where “the Internet” is represented by a switched Ethernet LAN.

6

3389: Needed for remote desktop management of the OPC server. 21379 and 135: Needed to run the OPC server. 8 443: Needed for configuration of the honeywall via a browser. 9 http://snort-inline.sourceforge.net 7

Penetration Testing of OPC as Part of Process Control Systems

277

3 Vulnerabilities in OPC In order to map vulnerabilities related to OPC, we have to consider vulnerabilities related to DCOM and RPC. This is because OPC is based on DCOM, which uses RPC, as mentioned in the Introduction. Frequently used vulnerability databases like the National Vulnerability Database (NVD)10, the Open Source Vulnerability Database (OSVD)11, and the database provided by United States Computer Emergency Readiness Team (US-CERT)12 reveal 555 vulnerabilities in RPC. 71 vulnerabilities related to DCOM are listed, and 40 OPC-specific vulnerabilities are reported. Here are some examples of vulnerabilities: − NETxAutomation NETxEIB OPC Server fails to properly validate OPC server handles − MS Windows RPC DCOM Interface Overflow − MS Windows DCOM RPC Object Identity Information Disclosure − MS Windows DNS RPC buffer overflow − MS Windows RPC service vulnerable to denial of service − MS Windows RPCSS Service contains heap overflow in DCOM request filename handling − MS Windows 2000 RPC Authentication Unspecified Information Disclosure These vulnerabilities open for buffer overflow, Denial-of-Service, and information disclosure; especially in the case of RPC. Carter et al [7] describe OPC-specific vulnerabilities in detail, and the main points of their discussion are summarized below. Lack of Authentication in OPC Server Browser: Configuration guidance from many vendors recommends allowing remote Anonymous Login so that OPCEnum will work when DCOM Authentication is sent to “None”. If a buffer overflow is discovered in the OPC Server Browser code, the result could be arbitrary code execution or Denial-of-Service attack against any computer running the OPC Server Browser. Fortunately, such an overflow has not been discovered yet. Lack of Integrity in OPC Communications: The default DCOM settings do not provide message integrity for OPC communication. If the underlying network is compromised and the attacker can sniff and insert traffic, it is likely that rogue messages could be injected once the client and server are authenticated during the initial connection establishment. A number of “Man-in-the-Middle” tools and techniques are available, and it is likely that these could be modified or enhanced to conduct attacks against OPC communication. Lack of Confidentiality in OPC Traffic: Although DCOM supports message encryption, most OPC vendors do not recommend enabling Packet Privacy for their OPC Server or the OPC Server Browser. Some vendors recommend VPN tunnelling as a means of providing secure remote access. Matrikon uses client- and server-side tunnelling component with encryption for this purpose. 10

http://nvd.nist.gov/ http://osvdb.org/ 12 http://www.kb.cert.org/vuls/ 11

278

M.B. Line et al.

In a different article, Lluis Mora [8] discusses the vulnerabilities of OPC servers. These include attacks using invalid server handle, invalid or crafted configuration file, resource starvation, etc. They have developed the tool “OPC Security Tester”/OPCTester13 for testing these vulnerabilities.

4 Exploits and Results Red team’s task was to act as attackers towards the test network and the OPC server specifically. Their initial knowledge included the IP address of the OPC server. In the following, the different types of mappings and attacks they performed are described, together with the results they achieved. More details about their work can be found in a separate report [9]. 4.1 Initial Network Mapping – Scanning and Probing the Network Many tools are available for gathering information about networks, operating systems and services offered by a network. Network administrators use such tools to manage network services, monitor hosts, service uptime etc. But as such tools are freely available; attackers can also use them for their own purposes. Nmap and Nessus can be used for scanning a process control system. Nessus also supports testing a list of vulnerabilities on a specific remote host. This can be thought of as the first step for any attacker in order to explore the list of services and ports exposed by a network. The introduction of a packet sniffer in the network clearly showed that in the case of setting the DCOM security level for the OPC server to "connect", all the OPC traffic is sent over the network in plain text, both from our test server and from the simulation network. Without any knowledge of the packet layout, we could still easily read out string values of the packets. Closer inspection and cross-checking with a proper OPC client lead us to also be able to identify other types of values in the packets, especially numerical values. This experience indicates that CERN does not value confidentiality of OPC data, as they have recommended this configuration setting. Further examinations of the network were done with the network mapper Nmap and the vulnerability scanner Nessus. Neither of these gave any indications of easily exploitable vulnerabilities in either our test server or in the test network. The information that was obtained was correct from both scanners on the operating system and other information known for our test server, and indicated Linux Fedora as the operating system running on the front end of the simulation network. However, these tools do not test OPC in particular, but rather the general setup of the servers with their operating systems and services. OPCTester performs automated tests of around 30 known vulnerabilities and access faults. We ran OPCTester to scan both the test server and the simulation network and did not get any useful results for either of the OPC servers. Run locally on our test server, OPCTester showed all available tests as passed without known vulnerabilities. 13

http://www.neutralbit.com/en/rd/opctest/

Penetration Testing of OPC as Part of Process Control Systems

279

To sum up, network mapping did not yield any information on exploitable technical issues on the simulation network nor on our test server. Network sniffing did clearly show that the confidentiality of a standard set up OPC-server is void. 4.2 Entering the Signalling Path As “the Internet” in our setup is a switched Ethernet, we utilized the well known technique ARP-spoofing in order to be able to realistically perform network sniffing. We used the tool ARPspoof for this purpose, which is included in the Linux software package dsniff14. With ARPspoof we continuously sent out gratuitous ARP-packets to two given IP addresses informing them that the other IP-address is located at the attacker’s MAC address. This way all traffic between the two hosts is switched onto the attacker’s port on the switch, effectively putting us (the attacker) in the middle. 4.3 Packet Sniffing A network packet analyzer, e.g. Wireshark15, can be used to capture network packets and analyze them for future attacks. In the case of OPC traffic communication between the server and client side tunnellers is by default not encrypted. Information about OPC server and groups and items added or updated in OPC server can hence be read. A Man-in-the-Middle attack can be used due to the lack of access control mechanisms. An ARP spoofing attack allows an attacker to spoof the MAC address to sniff, modify, redirect or stop the traffic to the IP address of the target system. By entering the signalling path we were able to read the packets sent between the client and the server. By monitoring the authentication process of the client we were able to read in clear text the user name and machine name of the client and the machine name of the server. The client response to the server challenge includes both the NTLM version 1 response and the LM response. There is no need for both of the responses which only leads to further reduction of the security. Both of these two protocols use DES, which is known to be a weak encryption algorithm. NTLM version 2 has been accepted as a de-facto internet standard as of 2000 [10], is widely used, and is deemed much more secure than NTLM version 1. 4.4 Denial-of-Service Attacks We used SYN flooding and ARP spoofing to launch a Denial-of-Service attack. TCP uses three-way handshake (SYN, SYN-ACK, ACK) to establish a session. In the case of SYN flooding the attacker sends several packets, but does not send the last ACK back to the server, or the attacker spoofs the source IP address of the SYN message, server sends SYN-ACK to the false IP address and never receives the last ACK. ARP spoofing can stop the traffic to the spoofed target system. With the previously mentioned middleman-status we performed a Denial-of-Service attack by simply dropping all the packets destined for both the spoofed hosts, and thereby acting as a black hole in the signal path. This attack totally destroyed the communication between the test server and the client as expected. 14 15

http://www.monkey.org/~dugsong/dsniff/ http://www.wireshark.org/

280

M.B. Line et al.

A second successful Denial-of-Service attack was performed as a SYN-flood attack run by a single attacker. On the test server, this attack not only disabled incoming communication from a client, but also slowed down the system so much that the server was practically unresponsive during the attack. The effect of the attack lasted until about one minute after the attack was called off by the attacker. As this attack used fake source addresses in the packets, this attack can potentially be very difficult to distinguish from genuine heavy load. As seen, both a black-hole method and a SYN-flood method of attacks were able to destroy the availability of the test system; the SYN-flood even resulted in lost local availability. 4.5 Man-in-the-Middle Attack A tool was written to do string replacement and forwarding of packets in the ARP spoofed network setup between the OPC client and the OPC test server. As the recommended setup of DCOM did not seem to contain any packet integrity mechanisms, we expected this attack to be able to compromise the integrity of the entire OPC network setup.

Fig. 2. Man-in-the-middle attack on OPC; packets between server and client are modified

Penetration Testing of OPC as Part of Process Control Systems

281

The tool we wrote simply replaced the string "Simulation" as sent from the OPC server A with the string "Real" and forwarded the modified packets to the OPC client B. Likewise the tool changed the string "Real" in the packets from the client to the server into the string "Simulation" and again forwarded these packets. Our tool was written as a proof-of-concept in the sense that the strings replaced were arbitrary. Any value and/or string can be replaced in the packets in the same way. The same tool can also be easily modified to shield the client from the server and in effect hijack the session. The resulting setup is described in Fig. 2. Table 2. Excerpt from printouts from OPC client from two different OPC sessions Without packet modification 2007-12-05 14:00:05,061 [main] DEBUG org.openscada.opc.lib.da. browser.BaseBrowser – Browsing with a batch size of 10 Leaf: Clients [Clients] Branch: Simulation Items Branch: Bucket Brigade Leaf: ArrayOfReal8 [Bucket Brigade.ArrayOfReal8] Leaf: ArrayOfString [Bucket Brigade.ArrayOfString]

Man-in-the-middle attack 2007-12-05 14:02:25,345 [main] DEBUG org.openscada.opc.lib.da. browser.BaseBrowser – Browsing with a batch size of 10 Leaf: Clients [Clients] Branch: Real Items Branch: Bucket Brigade Leaf: ArrayOfReal8 [Bucket Brigade.ArrayOfReal8] Leaf: ArrayOfString [Bucket Brigade.ArrayOfString]

As we can see from the sampled output in Table 2, we have a situation that to the eye looks genuine, but in reality is completely false. With the Man-in-the-Middle attack we were able to gain complete control of the system with the user privileges and access rights of the user in session. We were able to read, drop, change, and create packets as we saw fit in both directions, to the server and to the client. 4.6 Configuration Errors In the process of setting up the simulation network, the blue team installed an OPC Tunneller. As we connected to this OPC Tunneller, there were no access control mechanisms in place se we had complete access to the OPC server in the simulation network. The blue team also gave us instructions on how to set up access control on the OPC client, but these instructions led to the setup screen for the Windows XP DCOM run-as-user configuration management. We assume the instructions for the server setup was misunderstood such that the blue team in reality had set their OPC Tunneller to run as a specific user on the server rather than setting access control credentials needed for a client to log onto it. We can see that a small misunderstanding in the setup of the server led the red team, and in reality anyone, to take complete control of the simulation OPC server while the blue team was certain that access control was in place.

282

M.B. Line et al.

5 Discussion Process control networks in the oil and gas industry have traditionally had a very strong focus on safety, but safety-related risks are quite different from security-related risks. One way of differentiating between security and safety is to say that safety consists of protecting the world from the system, while security is protecting the system from the world [11]. Another aspect is that safety-related incidents normally have clearly defined statistical properties that relate to empirically established values for e.g. Mean Time Between Failures (MTBF) for components and systems. The same thing cannot be said for security-related incidents, since they predominately involve a human attacker – and we have yet to develop a useful statistical model for Homo Sapiens. In the early days of the internet, there was a Usenet newsgroup known as alt.folklore.urban16, devoted to the topic of urban legends. A recurring joke (or mantra) in this newsgroup was: “It could have happened, so it must be true.” Ludicrous as this statement may be in “real life,” this is exactly how we must think in the realm of computer security. The professional paranoia this reflects also testifies to a distinct difference in mindset compared to the safety domain. Our main contribution in this paper is to highlight the “could have happened” aspect of computer security in the Integrated Operations context. We have no empirical evidence that the specific attacks in our laboratory tests have been successfully wielded in a real PCS environment, although there are certainly news reports of other attacks against PCS’s in the past [12]. In our contact with representatives from the Norwegian oil & gas industry, we have been confronted with the sentiment “We don’t care what they say they can do over in the States; unless you can show us something that is directly applicable to our setup, we’re not interested.” This tells us that there is a need for full-scale penetration testing activities – and our tests indicate that such activities would yield interesting results. The most conclusive security breach that we successfully performed was achieved due to configuration error, which may at first glance seem to be an insignificant result. However, it is clear that the human factor plays a major role in the defence of systems as well as in their attack, and the safety concept of “fail safe” should clearly be applied to security settings, It may be argued that also the other attacks successfully implemented in our tests have limited value, since they all require physical access to the communications medium on some level, and these networks are all protected by multiple layers of firewalls and other protective mechanisms [5]. However, previous breaches tell us that not only is the insider threat an important factor in PCS environments [12], but the increasing integration of offshore and land-based networks also mean that the attack surface is increasing. This, in turn, implies that it is important to follow the doctrine of defence in depth, ensuring that there are sufficient (and independent) security mechanisms every step of the way from the external barrier to the heart of the PCS. Clearly, a protocol that uses outdated cryptography for authentication, and transmits data in plain text is not a good match for our doctrine. It is our opinion that the OPC implementation should implement NTLM version 2 or other more secure authentication protocols like Kerberos. 16

It’s still out there – see http://tafkac.org/

Penetration Testing of OPC as Part of Process Control Systems

283

6 Conclusion We have shown that confidentiality in OPC networks is non-existent in the default setup as recommended by CERN. Furthermore, the authentication process of the client reveals confidential information in clear text and gives a weak encryption of the client password. We have seen that DoS attacks can easily be accomplished, not only in making the server unavailable over the network, but also leading to denial of local control over the server. We have demonstrated a to the eye perfect breach of the integrity of the OPC server as a consequence of lacking DCOM packet integrity measures in the setup recommended by CERN, and we have demonstrated how fragile an OPC network is to configuration errors.

Acknowledgements The research presented in this paper was performed as part of project assignments at the Norwegian University of Science and Technology (NTNU).

References 1. OPC Overview 1.0, OPC Foundation 2008-01-17 (1998), http://www.opcfoundation.org/Downloads. aspx?CM=1&CN=KEY&CI=282 2. Understanding OPC and How it is deployed, Byres Research 2008-01-17 (2007), http://csrp.inl.gov/Recommended_Practices.html 3. DCOM Technical Overview, Microsoft Developer Network 2008-01-17 (1996), http://msdn2.microsoft.com/en-us/library/ms809340.aspx 4. Cheah, Z.b., Faruk, A.B.M.O.: Identifying and Responding to External Threats in a PCS Network. Norwegian University of Science and Technology Project Assignment, Trondheim (December 2007), http://sislab.no/blueteam.pdf 5. Grøtan, T.O., et al.: The SeSa Method for Assessing Secure Remote Access to Safety Instrumented Systems, SINTEF Report A1626, Trondheim (June 2007), http://www.sintef.no/content/page1_16321.aspx 6. Puget, M.B.J.-P., Barillere, R.: IT-CO recommended DCOM settings for OPC, CERN, Geneva (2005) 7. Carter, J., et al.: OPC Security. Digital Bond (2007) 8. Mora, L.: OPC Server Security Considerations. In: SCADA Security Scientific Symposium 2007, Miami, FL (2007) 9. Garnes, H.H., Wedum, P.: Innbruddstesting på prosesskontrollsystemer på oljeplattform, Norwegian University of Science and Technology Project Assignment, Trondheim (December 2007) 10. Zorn, G.: RFC 2759: Microsoft PPP CHAP Extensions, Version 2, The Internet Society (2000) 11. Line, M.B., et al.: Safety vs security? In: Eighth International Conference on Probabilistic Safety Assessment and Management, New Orleans, USA (2006) 12. GAO-07-1036 Critical Infrastructure Protection: Multiple Efforts to Secure Control Systems Are Under Way, but Challenges Remain, United States Government Accountability Office (2007), http://www.gao.gov/htext/d071036.html

Intersection Location Service for Vehicular Ad Hoc Networks with Cars in Manhattan Style Movement Patterns Yao-Jen Chang and Shang-Yao Wu Department of Electronics Engineering Chung Yuan Christian University Taoyuan, Taiwan 320 {yjchang,g9476025}@cycu.edu.tw

Abstract. For inter-vehicular communications that happen in metropolitan areas, moving cars travel on restricted directions along streets and make frequent stops over intersections. In this paper, we present Intersection Location Service (ILS), a distributed hashing-based location service algorithm that makes use of the features of street intersections and the Chord algorithm as the location query and fault tolerant mechanism. Performance comparisons of ILS with two well-known location service algorithms, Grid Location Service (GLS) and Hierarchical Location Service (HLS) are demonstrated in the ns-2 simulations of moving cars in various city environments. We have shown by means of simulation that ILS achieves good results in terms of query success ratios and remarkable scalability with respect to network size.

1 Introduction We assume that the nodes are able to determine their own position, e.g. by means of GPS. The task of locating the destination is a communication session is then fulfilled by a location service, which employs a distributed algorithm maintained by all the participating nodes in the Vehicular Ad Hoc Networks (VANETs). Most research results of location services have been focused on nodes whose motions follow patterns of random waypoints. However, we are looking at some protocol that introduces novel mechanisms to exploit the features of the urban scenario affecting the vehicular mobility such as the presence of intersections. For car-to-car communications that occur in metropolitan areas, moving cars obviously travel on restricted directions along streets and make frequent stops over intersections to gain right of ways according to local traffic rules. Several location services protocols including DREAM [10], Homezone [11] and GHLS [8] have been proposed in literature, by exploiting different approach for tracking the nodes' movements such as flooding-based, rendezvous-based, and hash-based. The solution proposed in the paper, called Intersection Location Service, is a variant of the hash-based approach. As main difference from the other schemes proposed in literature, the ILS algorithm is tailored for a vehicular environment. It does not use F.E. Sandnes et al. (Eds.): UIC 2008, LNCS 5061, pp. 284–296, 2008. © Springer-Verlag Berlin Heidelberg 2008

Intersection Location Service for Vehicular Ad Hoc Networks with Cars

285

flooding of information, and it includes fault-tolerant mechanisms. In the ILS scheme, each node is assigned a single location server, by hashing the node's unique identifier into an intersection identifier. Only vehicles moving in the intersections act as location servers, although every node is assumed to be capable of relaying packets. The duties of cars at temporarily empty intersections are transferred to cars at other nonempty intersections according to the proposed scheme. The location information cached by cars at a previously nonempty intersection is taken care of in a way that the subsequent queries on such information can be directed to the cars at a certain nonempty intersection that now takes over the responsibilities as location servers. The simulation analysis confirms the effectiveness of the ILS scheme in terms of query success ratios under multiple network scenarios. In addition, ILS performs better in street environments than two other well known location service protocols (GLS [7] and HLS [5]) designed without taking into account street environments. The remainder of this paper is structured as follows: In Section 2 we give an overview of related work. The ILS model is described in detail in Section 3. In Section 4 we present how the Chord algorithm is applied in the protocol of ILS. Section 5 contains the results of the simulation with ns-2. A comparison with two select location algorithms is also presented. The paper is concluded with a summary and future work in Section 6.

2 Related Work DREAM, the Distance Routing Effect Algorithm for Mobility [10], is a location service which uses flooding to spread position information. The capacity of the network is substantially decreased as a result of message flooding, especially in situations where the nodes make frequent position changes. A location algorithm that does not require flooding is Homezone [11]. Each node is assigned an area called Homezone via a hash function well known to all the nodes in the network. Location updates that reflect the changes nodes make to positions are sent to all the nodes in the designated homezone. Problems may arise when the homezone becomes empty, and some cars may get into situations where their default location servers are temporarily unavailable. The Grid Location Service (GLS) [7] divides the area of the entire ad hoc network into a hierarchy of squares forming a quad-tree. Each node selects one node in each element of every levels of the quad-tree as a location server. Therefore, the density of location servers for a particular node is high in areas close to the node and become exponentially sparse as the distance to the node increases. GLS has demonstrated good scalability in the random waypoint mobility model. Hierarchy Location Service (HLS) [5] partitions the area of the ad hoc network into cells. The cells are grouped into regions level by level. Therefore, a node uses a hierarchy of location servers to store its location information. The problem of empty cells where location updates or requests may run into is solved by temporary servers through a handover mechanism to bridge the gap. DREAM, Homezone, GHLS, GLS, and HLS are all designed with random waypoint motions in mind. The location services apply in MANETs where nodes can change directions at will; there are no fixed pathways for mobile nodes to follow.

286

Y.-J. Chang and S.-Y. Wu

However, there are increased needs to provide location services for MANETs that consist of vehicles moving along streets within metropolitan areas. ILS is proposed in this paper to address those needs. HLS and GLS, although not designed for street conditions, are selected for comparisons due to their good performance results for nodes in random waypoint motions.

3 Sketch of Intersection Location Service Protocol In this section, we briefly sketch the basic underlying idea of ILS, which will be detailed in the next section. In ILS, the street intersections covered by the network is numbered from 0 to N. In the ITS scheme, each node is assigned a single location server, by hashing the node's unique identifier into an intersection identifier. Cars within the radius r of the intersection M cache the location information of all the vehicles K for which M=Hash(K) where Hash is a hash function that can distribute cars evenly among the intersections. In an ideal case where all the street intersections are nonempty, the queries of location information about Car K where Hash(K)=M will be directed to Intersection M, the default location server for Car K. Due to the highly dynamic nature of VANETs, it is possible that some intersections are empty for certain periods of time. In case there is no designs of fault tolerance, location queries that are directed to temporarily empty intersections will lead to failures. A priori selection of backup location servers is of little use because any intersection that acts as a location server also suffers from becoming temporarily empty. Using the Chord algorithm [1,9]. Once a location server is determined for a certain query, the query issued by a car will be routed to the target location server with a position-based routing protocol like GPSR [4]. The position-based routing in ILS is done by the cars that serve as intermediate relays to overcome the multiple-hop delivery problems typical in MANETs.

4 Protocol of Intersection Location Service This section describes the ILS protocol which specifies how to find the locations of cars moving along the streets, how the unavailability of intersections can be overcome with relatively small overhead. 4.1 Consistent Hashing The consistent hash function assigns each intersection an m-bit identifier using SHA-1 [2] as a base hash function. Vehicles moving in the radius r of intersections are responsible as location servers to cars by caching their location information. Consistent hashing assigns Car K to a default location server M if M=SHA1(K), where SHA1 stands for SHA-1. However, a location server M can be nonexistent in the 2m space or M goes out of service because the intersection has no cars for a period of time and none of the location information can be cached in any one of the cars within the radius r of the center of intersection M. Therefore, a successor location server for car K, denoted as successor(K), is chosen to be the first location server whose identifier is equal to or follows SHA1(K). If identifiers are represented as an imaginary circle of

Intersection Location Service for Vehicular Ad Hoc Networks with Cars

287

numbers from 0 to 2m -1, then the successor(K) is the first node clockwise from M if M=SHA1(K). The circle is referred to as the Chord ring [1,9]. As an illustrative example, Fig. 1 shows a Chord ring with m=3. The Chord ring has 3 location servers and 4 cars. The successor of car 2 is location server 3, so location server 3 will store location information about car 2 while location server 2 is out of service.

7 7 3 6 2

5 4

Fig. 1. A Chord ring with 4 nodes and 3 intersections where cars with radius r act as location servers

4.2 Query for Car Location When a car requests for location service to acquire the location of other cars before car-to-car communication can be set up, it first submits its query to a nearby intersection. ILS is based on the Chord protocol that works by pruning candidate location servers by half at each iteration and the binary search continues until the exact location server is determined. It is not without price; ILS maintains additional information to facilitate binary search. Additional information is kept by each location server in a data structure, called the finger table [1,9]. As before, let m be the number of bits in the location server identifiers. A location server M maintains its finger table with up to m entries. The i-th entry in the table at location server M contains the identity of the first node S that succeeds M by at least 2i-1 on the identifier circle, i.e., S = successor(M+2i-1), where 1≤i≤m and all arithmetic is modulo 2 m. S is called the i-th finger of M, and is denoted by M.finger[i]. A finger table entry includes both the identifier of location server and its coordinates. Note that the first finger of M is the immediate successor of M on the circle and is referred to as the successor for convenience. Figure 2 is an example of finger tables maintained by vehicles with radius r of nonempty intersections. For a network with N location servers, the finger table at each location server has O(Log(N)) entries relating to information about only a small number of other location servers. The Chord algorithm to find a successor with the use of finger tables operates is as follows.

288

Y.-J. Chang and S.-Y. Wu

Finger 7+2i-1 =8, i =1 7+2i-1 =9, i =2 7+2i-1 =11, i =3

Finger 6+2i-1 =7, i =1 6+2i-1 =8, i =2 6+2i-1 =10, i =3

Successor 1 3 3

Successor 7 1 3

7

1

6

3

Finger 1+2i-1 =2, i =1 1+2i-1 =3, i =2 1+2i-1 =5, i =3

Successor 3 3 6

Finger 3+2i-1 =4, i =1 3+2i-1 =5, i =2 3+2i-1 =7, i =3

Successor 6 6 7

Fig. 2. Finger tables as cached by vehicles within the radius r of intersections

//For a location server M to find the successor of car K M.find_successor(K) { if( M