Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011 [1 ed.] 9781614990925, 9781614990918

Citation preview

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

SIMULATION AND MODELING RELATED TO COMPUTATIONAL SCIENCE AND ROBOTICS TECHNOLOGY

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Studies in Applied Electromagnetics and Mechanics Series Editors: K. Miya, A.J. Moses, Y. Uchikawa, A. Bossavit, R. Collins, T. Honma, G.A. Maugin, F.C. Moon, G. Rubinacci, H. Troger and S.-A. Zhou

Volume 37 Previously published in this series: Vol. 36. Vol. 35. Vol. 34. Vol. 33. Vol. 32. Vol. 31. Vol. 30. Vol. 29. Vol. 28. Vol. 27.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Vol. 26. Vol. 25. Vol. 24. Vol. 23. Vol. 22. Vol. 21.

B.P.C. Rao, T. Jayakumar, K. Balasubramanian and B. Raj (Eds.), Electromagnetic Nondestructive Evaluation (XV) T. Chady, S. Gratkowski, T. Takagi and S.S. Udpa (Eds.), Electromagnetic Nondestructive Evaluation (XIV) S. Wiak and E. Napieralska-Juszczak (Eds.), Computer Field Models of Electromagnetic Devices J. Knopp, M. Blodgett, B. Wincheski and N. Bowler (Eds.), Electromagnetic Nondestructive Evaluation (XIII) Y.-K. Shin, H.-B. Lee and S.-J. Song (Eds.), Electromagnetic Nondestructive Evaluation (XII) A. Tamburrino, Y. Melikhov, Z. Chen and L. Udpa (Eds.), Electromagnetic Nondestructive Evaluation (XI) S. Wiak, A. Krawczyk and I. Dolezel (Eds.), Advanced Computer Techniques in Applied Electromagnetics A. Krawczyk, R. Kubacki, S. Wiak and C. Lemos Antunes (Eds.), Electromagnetic Field, Health and Environment – Proceedings of EHE’07 S. Takahashi and H. Kikuchi (Eds.), Electromagnetic Nondestructive Evaluation (X) A. Krawczyk, S. Wiak and X.M. Lopez-Fernandez (Eds.), Electromagnetic Fields in Mechatronics, Electrical and Electronic Engineering G. Dobmann (Ed.), Electromagnetic Nondestructive Evaluation (VII) L. Udpa and N. Bowler (Eds.), Electromagnetic Nondestructive Evaluation (IX) T. Sollier, D. Prémel and D. Lesselier (Eds.), Electromagnetic Nondestructive Evaluation (VIII) F. Kojima, T. Takagi, S.S. Udpa and J. Pávó (Eds.), Electromagnetic Nondestructive Evaluation (VI) A. Krawczyk and S. Wiak (Eds.), Electromagnetic Fields in Electrical Engineering J. Pávó, G. Vértesy, T. Takagi and S.S. Udpa (Eds.), Electromagnetic Nondestructive Evaluation (V)

Volumes 1–6 were published by Elsevier Science under the series title “Elsevier Studies in Applied Electromagnetics in Materials”.

ISSN 1383-7281 (print) ISSN 1879-8322 (online)

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Simulatiion an nd Mod deling g Relatted t Com to mputattional Science andd Roboticcs Tech hnolog gy Proceedin ngs of SiMCRT 2011

y Edited by

Fum mio Kojiima Futosshi Kobaayashi and

Hiroy yuki Nakaamoto

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Grad duate Schooll of System In nformatics, Kobe K Univerrsity, Kobe, JJapan

Amstterdam • Berrlin • Tokyo • Washington, DC

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

© 2012 The authors and IOS Press. All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without prior written permission from the publisher. ISBN 978-1-61499-091-8 (print) ISBN 978-1-61499-092-5 (online) Library of Congress Control Number: 2012943291 Publisher IOS Press BV Nieuwe Hemweg 6B 1013 BG Amsterdam Netherlands fax: +31 20 687 0019 e-mail: [email protected]

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Distributor in the USA and Canada IOS Press, Inc. 4502 Rachael Manor Drive Fairfax, VA 22032 USA fax: +1 703 323 3668 e-mail: [email protected]

LEGAL NOTICE The publisher is not responsible for the use which might be made of the following information. PRINTED IN THE NETHERLANDS

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Simulation and Modeling Related to Computational Science and Robotics Technology F. Kojima et al. (Eds.) IOS Press, 2012 © 2012 The authors and IOS Press. All rights reserved.

v

Preface

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

This volume includes a selection of papers presented at the International Workshop on Simulation and Modeling related to Computational Science and Robotics Technology (SiMCRT2011) was organized by Kobe University during November 1–3, 2011 at the Takikawa Memorial Hall, Kobe University Japan. This workshop was also cosponsored by AFOSR/AOARD under Grant No. FA23861111041. Historically, this workshop was the third international meeting for providing a forum for discussing recent developments in the growing field of engineering science and mathematical sciences. The first workshop took place in 1997 in Osaka Institute of Technology, Osaka, Japan. That was one of the satellite meetings of IEEE Conference on Decision and Control that was held in Kobe Japan. The second meeting was held in Kobe University in 2007 as the International Symposium on Mathematical Modeling and Computational Methods in Science and Engineering (MMCOM2007). At the course of continuing efforts on the international collaborations, those meetings have been extended to the academic research fields. In that respect, the objectives of this workshop are to i) bring together a diverse group of researchers in these areas in order to share and compare the different approaches to simulation and modeling for broad range of applications in computational science and robotics technology, ii) provide an evaluation of the state of the art in related computational methods and iii) present specific applications where there is a need to develop new computational methods in order to address important design and control problems. The workshop was also aimed at establishing collaborative links between engineering researchers of information and robotics technology (IRT) and applied mathematicians working in modeling and computational methods for design and control. In addition to the formal presentations, time will be allotted for researchers to discuss and evaluate possible applications of new design strategies. F. Kojima, F. Kobayashi and H. Nakamoto Editors

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

This page intentionally left blank

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

vii

SiMCRT – Kobe national Work kshop on Simu ulation and Modeling relateed to Computaational The Intern Science an nd Robotics Technology T Kobe, Jap pan, November 1–3, 2011 Organized by: • Kobee University Co-sponsored by: SR/AOARD • AFOS

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Internatio onal Organizzing Committtee: • F. Ko ojima, Kobe University, U Jap pan, Chairman n • H.T. Banks, B N.C. State S University, USA • J.A. Burns, B Virginiia Polytechnicc Institute and State Universsity • F. Kaappel, Universsity of Gratz, Austria A • M. Yamamoto, Un niversity of To okyo, Japan uo, Kobe Univ versity, Japan • Z. Lu

Secretary y: • Futosshi Kobayashi Gradu uate School off System Inforrmatics, Kobee University 1-1 Rokkodai, R Nad da 657-8501, Japan J Tel/Fax: +81-78-80 03-6489, E-m mail: futoshi.ko [email protected] •

Hiroyyuki Nakamoto Gradu uate School off System Inforrmatics, Kobee University 1-1 Rokkodai, R Nad da 657-8501, Japan J Tel/Fax: +81-78-80 03-6669, E-m mail: nakamoto [email protected]

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

This page intentionally left blank

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

ix

Participants of the SiMCRT 2011 Workshop, Kobe University, Japan H. Thomas BANKS, Professor North Carolina State University Center for Research in Scientific Computation USA [email protected]

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

John E. BANKS, Professor University of Washington Interdisciplinary Arts and Sciences USA [email protected]

Kazufumi ITO, Professor North Carolina State University Department of Mathematics USA [email protected] Kohji KAMEJIMA, Professor Osaka Institute of Technology Information Science and Technology Japan [email protected]

Van Phong DINH, Professor Hanoi University of Technology Science – Technology Office Vietnam [email protected]

Franz KAPPEL, Professor University of Graz Institute for Mathematics and Scientific Computing Austria [email protected]

Kenneth C. EVENSEN, Commander The U.S. Army Research, Development and Engineering Command ITC-PAC Japan [email protected]

Futoshi KOBAYASHI, Associate Professor Kobe University Graduate School of System Informatics Japan [email protected]

Wataru FUKUI, Doctoral Student Kobe University Graduate School of Engineering Japan [email protected]

Fumio KOJIMA, Professor Kobe University Graduate School of System Informatics Japan [email protected]

Kazuyuki HANAHARA, Associate Professor Kobe University Graduate School of System Informatics Japan [email protected]

Daigo KOSAKA, Instructor Polytechnic University Japan [email protected]

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

x

Shiro KUBO, Professor Osaka University Graduate School of Engineering Japan [email protected]

Naoshi NISHIMURA, Professor Kyoto University Graduate School of Informatics Japan [email protected]

Naoyuki KUBOTA, Associate Professor Tokyo Metropolitan University Graduate School of System Design Japan [email protected]

Kouichi NITTA, Associate Professor Kobe University Graduate School of System Informatics Japan [email protected]

Zhi Wei LUO, Professor Kobe University Graduate School of System Informatics Japan [email protected]

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Osamu MATOBA, Professor Kobe University Graduate School of System Informatics Japan [email protected]

Yusuke NOJIMA, Assistant Professor Osaka Prefecture University Graduate School of Engineering Japan [email protected] Masatoshi NOUMI, Professor Kobe University Graduate School of Science Japan [email protected]

Takao MIYATA, Director The U.S. Army Research, Development and Engineering Command ITC-PAC Japan [email protected]

Sigeru OMATU, Professor Osaka Institute of Technology Faculty of Engineering, Department of Electronics, Information and Communication Engineering Japan [email protected]

Shinichi NAKAGIRI, Professor Kobe University Graduate School of System Informatics Japan [email protected]

Takahide SAKAGAMI, Professor Kobe University Graduate School of Engineering Japan [email protected]

Hiroyuki NAKAMOTO, Assistant Professor Kobe University Graduate School of System Informatics Japan [email protected]

Hideki SANO, Associate Professor Kobe University Graduate School of System Informatics Japan [email protected]

Takao NAMBU, Professor Kobe University Graduate School of System Informatics Japan [email protected]

David SMITH, Professor The University of Western Australia School of Computer Science and Software Engineering Australia [email protected]

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

xi

Toshiyuki TAKAGI, Professor Tohoku University Institute of Fluid Science Japan [email protected]

Naoki YAMANAKA, Invited Professor Kobe University Center for Collaborative Research and Technology Development Japan

Teruo USAMI, Professor Kyoto Gakuen University Faculty of Economics Japan [email protected]

Daisuke YORIFUJI, Doctoral Student Tokyo Institute of Technology Japan [email protected] Masahiko YOSHIMOTO, Professor Kobe University Graduate School of System Informatics Japan [email protected]

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Masahiro YAMAMOTO, Professor The University of Tokyo Graduate School of Mathematical Sciences Japan [email protected]

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

This page intentionally left blank

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

xiii

Contents Preface F. Kojima, F. Kobayashi and H. Nakamoto

v

SiMCRT – Kobe, Conference Organization

vii

Participants of the SiMCRT 2011 Workshop, Kobe University, Japan

ix

Modeling and Simulation Simulation Algorithms for Continuous Time Markov Chain Models H.T. Banks, Anna Broido, Brandi Canter, Kaitlyn Gayvert, Shuhua Hu, Michele Joyner and Kathryn Link

3

Nonsmooth Optimization Method for the Elastic Contact Problem Kazufumi Ito and Sarah A. King

19

Population Models & Data in Applied Ecology: Surrogate Species John E. Banks, Azmy Ackleh and John D. Stark

34

A New Approach of Using Null Space of Jacobian Matrix in Simulation of Multibody Dynamics Van Phong Dinh and Hai Nguyen Nguyen

44

Inverse Problems

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Deformation Formulas and Inverse Problems for Advection-Diffusion Equations Shin-Ichi Nakagiri Crack Identification by the Passive and the Active Methods with the Use of Piezoelectric Film and Inverse Analysis S. Kubo, T. Sakagami and Seiji Ioka Forward and Inverse Simulations of Pipe Wall Thinning Using Pulsed Eddy Current Signals Shejuan Xie, Toshiyuki Takagi, Tetsuya Uchimoto, Zhenmao Chen and Li Wang Stochastic Inverse Methodologies for Structural Health Monitoring Using Electromagnetic Measurements Fumio Kojima and Mitsuhiro Kikuchi

61

79

85

100

Intelligent System Neuro-Control and Its Applications to Electric Vehicle Control Sigeru Omatu

109

Information Visualization in Intelligent Navigation for Multiple Mobile Robots Naoyuki Kubota and Yuichiro Toda

120

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

xiv

Parallel Distributed Genetic Rule Selection for Data Mining from Large Data Sets Yusuke Nojima, Shingo Mihara and Hisao Ishibuchi

140

Robotics On Comutational Robotics Zhiwei Luo

157

Teleoperation of Universal Robot Hand with Pinching Force Stabilization Futoshi Kobayashi, Hiroyuki Nakamoto, Fumio Kojima, Tadashi Maeda, Nobuaki Imamura and Hidenori Shirasawa

172

Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand Wataru Fukui, Futoshi Kobayashi, Hiroyuki Nakamoto and Fumio Kojima

185

Displacement and Force Measurement, Vibration Detection by Magnetic Type Tactile Sensor Hiroyuki Nakamoto and Satoru Takenawa

196

Applications Saliency-Based Geographics Annotation for Robotic Access to Naturally Complex Scenes Kohji Kamejima

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Monitoring Method for Underground Condition Teruo Usami and Fumio Kojima

213 228

Exact Pipe Wall Thinning Management with Flow Accelerated Corrosion Using Electro-Magnetic Acoustic Transducer Daigo Kosaka, Hiroki Tabata, Hiroyuki Nakamoto and Fumio Kojima

236

Subject Index

245

Author Index

247

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Modeling and Simulation

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

This page intentionally left blank

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Simulation and Modeling Related to Computational Science and Robotics Technology F. Kojima et al. (Eds.) IOS Press, 2012 © 2012 The authors and IOS Press. All rights reserved. doi:10.3233/978-1-61499-092-5-3

3

Simulation Algorithms for Continuous Time Markov Chain Models H.T. BANKS a,1 , Anna BROIDO b Brandi CANTER c Kaitlyn GAYVERT d Shuhua HU a Michele JOYNER c and Kathryn LINK e a Center for Research in Scientific Computation, North Carolina State University, Raleigh, NC 27695-8212 USA b Department of Mathematics, Boston College, USA c Department of Mathematics and Statistics, East Tennessee State University, USA d Department of Mathematics, State University of New York at Geneseo, USA e Department of Mathematics, Bryn Mawr College, USA

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Abstract. Continuous time Markov chains are often used in the literature to model the dynamics of a system with low species count and uncertainty in transitions. In this paper, we investigate three particular algorithms that can be used to numerically simulate continuous time Markov chain models (a stochastic simulation algorithm, explicit and implicit tau-leaping algorithms). To compare these methods, we used them to analyze two stochastic infection models with different level of complexity. One of these models describes the dynamics of Vancomycin-Resistant Enterococcus (VRE) infection in a hospital, and the other is for the early infection of Human Immunodeficiency Virus (HIV) within a host. The relative efficiency of each algorithm is determined based on computational time and degree of precision required. The numerical results suggest that all three algorithms have similar computational efficiency for the VRE model due to the low number of species and small number of transitions. However, we found that with the larger and more complex HIV model, implementation and modification of tau-Leaping methods are preferred. Keywords. stochastic simulation algorithm, explicit tau-leaping method, implicit tau-leaping method, VRE, HIV

1. Introduction Deterministic approaches involving ordinary differential equations to approximate large number discrete populations with a continuum, though widely used, have proven less useful when applied (often with little or no justification!) to small sample sizes. To address this issue, continuous time Markov chain (CTMC) models are often used when dealing with low species or population counts. There are a variety of stochastic algorithms that can be employed to simulate CTMC models. However, it appears that none of these algorithms is universally efficient in many problems of interest. There are a plethora of applications in which the questions we investigate here arise. In addition to the infection models we use for illustration, similar stochastic models 1 Corresponding Author: H.T. Banks, Center for Research in Scientific Computation, North Carolina State University, Raleigh, NC 27695-8212 USA; E-mail: [email protected]

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

4

H.T. Banks et al. / Simulation Algorithms for Continuous Time Markov Chain Models

arise in just-in-time production networks, manufacturing and delivery, logistic/supply chains, and multi-scale (large/small) population models as well as network models in communications and security. A typical example is the agricultural (pork) production system investigated in [2]. There a stochastic transport model was used to study the impact of disturbances (introduction of diseases and other disruptions in the network) in production systens such as that depicted in Figure 1.

Sows N1

Nursery N2

Finisher N3

Slaughter N4

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 1. Aggregated agricultural network model.

The schematic represents a simplified swine production network with four levels of production nodes: (i) growers/sows (N1), (ii) nurseries (N2), (iii) finishers (N3), and (iv) processing plants/slaughterhouses (N4). At the grower or sow farms (N1), new piglets are born and weaned approximately three weeks after birth. The three-week old piglets are moved to nursery farms (N2) to mature for another seven weeks. They are then transferred to the finisher farms (N3) where they grow to full market size. This takes approximately twenty weeks. Once they reach market weight, the matured pigs are moved to the processors (slaughterhouses) (N4). Such pork production industries operate under a “just-in-time” philosophy often employed in many manufacturing systems. In particular, feedstock and animals are grown in different areas and the animals are moved with precise timing from one farm to another, depending on their age. Any shocks propagate rapidly through such systems if conditions are conducive. For example, interruption to feed supply has much larger impact when farms have minimal surplus supplies. The maturity-based just-in-time movement of animals between farms serves as another vulnerability. Stopping movement (transportation disruptions due to weather, work stoppages due to illness, etc) of animals to and from a farm with animals infected by disease will have disruptive effects that quickly spread throughout the system. Other interruptions occur when nurseries supplying farms have nowhere to send animals as they mature if the farms have not cleared their current animals for some reason. This will cause finishers and slaughterhouses to have supply interrupted. Randomness seen in the stochastic network model originates from random movement of discrete “individuals” from node to node. Analysis (see [2]) shows that, due to an averaging effect, these random effects become less important as the system size (number N of “individuals”) increases. We observe that an application of such a stochastic transportation model to describe the system behavior should account for size of groups in which pigs are transported between nodes. If thousands of pigs are moved at a time, an appropriate notion of an “individual” in the context of the model might be a thousand pigs. Then treating each group of a thousand animals as unit would lead to a marked increase in magnitude of stochastic fluctuations seen at the “population” level. As a result, scaling in the model may result in vastly different stochastic fluctuations in model simulations. Thus one must exercise care in how data and “units” are formulated in modeling populations.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

H.T. Banks et al. / Simulation Algorithms for Continuous Time Markov Chain Models

5

To further illustrate these practical modeling concerns, we consider the results related to our investigations below of Vancomycin-Resistant Enterococcus (VRE) transmission among patients in hospital intensive care units. Figure 2 depicts the results obtained when comparing stochastic model simulations with corresponding deterministic ordinary differential equation formulations for the total number of beds N = 37 (left panel) and N = 3700 (right panel). The figure reveals that behaviors of the model simulations are quite different when treating one 3700 bed unit as 37 bed units in 100 hospitals if we could argue (not often plausible) that the units are similar in patient and health care worker routines. This offers rather clear warnings for the indiscriminate use of limiting deterministic ordinary differential equations in place of Markov chain models to study small population count systems.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 2. Results from VRE model simulations: Graphs in the left column are for uncolonized patients (U ) and colonized patients in isolation (J) and N=37 and the ones in right column are for uncolonized patients (U ) and colonized patients in isolation (J) and N=3700. The (D) and (S) in the legend denote the solutions obtained with the deterministic VRE model and stochastic VRE model introduced below, respectively, where the stochastic results are obtained with the SSA.

Fundamental to the investigation of such systems is the ability to efficiently simulate the systems in the context of inverse problems, parameter estimation, sensitivity, control, etc. Computational methods abound for the corresponding deterministic limiting (as population size increases) differential equations. While a number of stochastic simulation algorithms exist, they are often difficult to use in the contexts mentioned above. The goal of this paper is to illustrate how widely performances may vary for some stochastic algorithms when compared on two stochastic infection models and to demonstrate how one might perform computational studies to aid in selection of appropriate algorithms. Specifically, we examine three commonly used algorithms: a stochastic simulation algorithm (SSA), and explicit and implicit tau-leaping methods. One of the models used to demonstrate the efficiency of these three algorithms describes progression of a Vancomycin-resistant enterococcus (VRE) infection in a hospital unit. The other model describes the dynamics of HIV during the early stage of infection, in which the target cells are still at very high level while the infected cells are at very low level. The outline of the remainder of this paper is as follows. In Section 2 we give short descriptions of a stochastic simulation algorithm, as well as the explicit and implicit tauleaping algorithms. In Section 3 we apply these three stochastic algorithms to the VRE and HIV models and compare their computational efficiency. We conclude the paper in Section 4 with some summary remarks.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

6

H.T. Banks et al. / Simulation Algorithms for Continuous Time Markov Chain Models

2. Simulation Algorithms In this section, three computational algorithms for solving stochastic systems will be examined, the stochastic simulation algorithm, the explicit tau-leaping method and the implicit tau-leaping method. Outlines for implementing each algorithm will be given along with motivations for the algorithm and discussions about when one might want to use one algorithm over another. Unless otherwise indicated, a capital letter is used throughout to denote a random variable, a bold capital letter is for a random vector, and their corresponding small letters are for their realizations.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

2.1. Stochastic Simulation Algorithm The stochastic simulation algorithm (SSA), also known as the Gillespie algorithm [8], is the standard method employed to simulate continuous time Markov Chain models. The SSA was first introduced by Gillespie in 1976 to simulate the time evolution of the stochastic formulation of chemical kinetics, a process which takes into account that molecules come in whole numbers as well as the inherent degree of randomness in their dynamical behavior. However, in addition to simulating chemically reacting systems, the Gillespie algorithm has become the method of choice to numerically simulate stochastic models arising in a variety of other biological applications [1,2,12,13,16,17]. Two mathematically equivalent procedures were originally proposed by Gillespie, the “Direct method” and the “First Reaction method”. Both procedures are exact procedures rigorously based on the chemical master equation [8]; however, the direct method is the method typically implemented due to its efficiency. Likewise, this is the method employed in this paper. The direct method can be described for a general system by assuming X = (X1 , X2 , ..., Xn )T represents the state variables of the system where Xi (t) denotes the number in state Xi at time t (Xi may be the number of patients, cells, species, etc). Furthermore, it is assumed l transitions (often referred to as reaction channels in the biochemistry literature) are possible with associated transition rates (often referred to as propensity functions in the biochemistry literature) represented by λi , i = 1, ..., l. Given this terminology, the direct method for the Gillespie algorithm can be described by the following procedure: Step 1. Initialize the state of the system x0 ; Step 2. For the given state x of the system, calculate the transition rates λi (x), i = 1, ..., l; l Step 3. Calculate the sum of all transition rates, λ = i=1 λi (x); Step 4. Simulate the time, τ , until the next transition by drawing from an exponential distribution with mean 1/λ; Step 5. Simulate the transition type by drawing from the discrete distribution with probability Prob(transition = i) = λi (x)/λ. Generate a random number r2 from a uniform distribution and choose the transition as follows: If 0 < r2 < λ1 (x)/λ, choose transition 1; if λ1 (x)/λ < r2 < (λ1 (x) + λ2 (x))/λ choose transition 2, and so on; Step 6. Update the new time t = t + τ and the new system state; Step 7. Iterate steps 2-6 until t ≥ tstop .

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

H.T. Banks et al. / Simulation Algorithms for Continuous Time Markov Chain Models

7

2.2. Tau-Leaping Methods Since the SSA method keeps track of each transition, it can be impractical to implement for certain applications due to the computational time required. As a result, Gillespie proposed an approximate procedure, the tau-leaping method, which accelerates the computational time while only sustaining a small loss in accuracy [9]. Instead of taking incremental steps in time, keeping track of X(t) at each time step as in the SSA method, the tau-leaping method leaps from one subinterval to the next, approximating how many transitions take place during a given subinterval. It is assumed that the value of the leap, τ , is small enough that there is no significant change in the value of the transition rates along the subinterval [t, t + τ ]. This condition is known as the leap condition. The tauleaping method thus has the advantage of simulating many transitions in one leap while not losing significant accuracy, resulting in a speed up in computational time. In this paper, we consider two tau-leaping methods, an explicit and an implicit tau-leaping method. 2.2.1. An Explicit Tau-Leaping Method The explicit tau-leaping method is based on an explicit formulation for the update in number of species X at time t + τ , given X(t) = x. The basic explicit tau-leaping method approximates Kj , the number of times a transition j is expected to occur within the time interval [t, t + τ ], by a Poisson random variable Pj (λj (x), τ ) with mean (and variance) λj (x)τ . Once the number of transitions are estimated, the approximate number of species, known as the tau-leaping approximation, of X at time t + τ is given by the formula X(t + τ ) = x +

l 

Pj (λj (x), τ )vj

(2.1)

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

j=1

with vj = (v1j , ..., vnj )T where vij represents the change in state variable Xi caused by transition j [6]. However, as mentioned previously, the process for selecting τ is critical in the tau-leaping method. If τ is chosen too small, tau-leaping will essentially stop, leading to the standard SSA algorithm; on the other hand, if the value of τ is too large, the leap condition may not be satisfied, possibly causing significant inaccuracies in the simulation. In this paper, we use a τ -selection procedure based on the algorithm in [6]. For alternative procedures for selecting τ , we refer the reader to references [6,9,10]. Let ΔXi = Xi (t + τ ) − xi with xi being the ith component of x, i = 1, 2, . . . n, and  be an error control parameter with 0 <   1. In the given τ -selection procedure, τ is chosen such that    xi , 1 , i = 1, ..., n, (2.2) ΔXi ≤ max gi which evidently requires the relative change in Xi to be bounded by gi except that Xi will never be required to change by an amount less than 1. The value of gi in (2.2) is chosen such that the relative changes in all the transition rates will be bounded by . The tau-leaping method employed in this paper also includes modifications developed by Cao, et al. [5] to avoid the possibility of negative populations. When utilizing a tau-leaping method instead of the exact SSA method, as discussed previously, estimates

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

8

H.T. Banks et al. / Simulation Algorithms for Continuous Time Markov Chain Models

are made about how many times a transition has occurred during the leap-interval. From the estimate of the number of transitions and how each transition effects the state variables, an estimate is obtained for the number of species in each state, Xi , at the end of the leap-interval. In some instances, if a population or number of species is small at the beginning of the leap-interval, the estimate of the state variable after numerous transitions may result in a negative population. To avoid this situation, Cao, et al. [5,6] introduced another control parameter, nc , a positive integer (normally set between 2 and 20) which is used to separate transitions into two classes, critical transitions or noncritical transitions. A transition j is deemed critical if after nc of these transitions, there is a danger in one of the state variables involved in the transition reaching zero. An estimate for the maximum number of times Lj , j = 1, ...l that transition j can occur before reducing one of the state variables involved in the transition to 0 (or less) is calculated by  Lj =

min

{1≤i≤n;νij  ∑ ∑ M+ M Z  [  ⎪⎩ K= L= Z / =  M   M    : [ / = M   M    ;  M =    -

(28)

Furthermore, we define the unknown degree in the search,  / / / / P7PM OCR7PM (29) M Z  [  = M Z  [   M−  / / / / 7PM μ 7PM (30) M Z  [  =  − PM Z  [  We can reduce the search space for path planning by using the degree of uncertainty and exploration, and can obtain the smaller size of abstract map.

Fig.12. Membership functions used for evaluating multi-resolution map

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

134

N. Kubota and Y. Toda / Information Visualization in Intelligent Navigation

Fig.13. A problem of the low-resolution map 4.3. Initial localization We apply the multi-resolution map for initial self-localization for multiple mobile robots [36]. To begin with, we must discriminate occupied cells and empty cells on the multi-scale maps for self-localization. As mentioned above, a multi-scale map is represented by two different normalized values of {0, 1}. By using these values, we can obtain the state of the cell easily as the following equation, (31) / / 5VCVG ⎧ aaaaa KH aaaQM Z  [  > α  UM Z /  [ /  = ⎨ ⎪⎩aaaa QVJGTYKUG / / QM Z /  [ /  = PM Z /  [ /  + P7PM M Z  [ 

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

(32) where aState indicates the threshold value. If sk(xM,yM) is 1, then the state of the cell is occupied. Figure 14 shows the result of extraction the occupied cell on each resolution map.

(a) Original map

(b) k = 3

(c) k = 5

Fig.14. Multi-resolution maps in a laboratory room Next, we explain the initial self-localization method in detail. We used SSGA for SLAM in the previous section. SSGA is similar with the search based on hill climbing, because SSGA mainly uses elitist crossover. This approach is very useful in case of small or sequential update of self-position like SLAM, but SSGA is not suitable in case of global search in a large scale of search space. Therefore, we apply (µ+λ)-ES for initial self-localization. The µ indicates the number of parent population and the λ indicates the number of offspring population produced in a single generation. Algorithm 3 shows the procedure of initial self-localization where k indicates the level of the multi-scale maps; n indicates the number of steps; N indicates the threshold value of the steps. A candidate solution is composed of numerical parameters of revised values to the current position (gli,x , gli,y) and the current direction angle of the robot (gli,r). The fitness value fitli of the ith individual is calculated by the following equation, HKV KN =

JKV JKV + GTT

(33)

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

N. Kubota and Y. Toda / Information Visualization in Intelligent Navigation

135

where hit and err are the number of occupied cell and empty cells corresponding to the position measured by LRF, respectively. In the step 3 and step 7, the weight wli is calculated by the following equation, YKN =

z

HKVKN

∑ HKV L=

N L

(34) After the result of initial self-localization is satisfied with the predefined threshold, the number of λ is reduced step by step in order to conduct the local fine search. Finally, the number of λ is 1, and (µ+1)-ES is the same as SSGA. After initial self-localization, each mobile robot starts moving, and conducts self-location by SSGA.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Algorithm 3 Initial localization: Step1: Initialize µ parents and n = 0. Step2: Get the LRF data and calculate the fitness value fitli. Step3: Calculate the weight wli. Step4: Produce λ offsprings depending on wli Step5: Get the LRF data and calculate the fitness value fitli. Step6: The top µ candidates are selected as next parent. Step7: Calculate the weight wli. Step8: -If the best fitness value is higher than ah, then k = k – 1 and n = 0. -Otherwise, go to step 10. Step9: -If n > N and the best fitness value is lower than as, then k = k + 2 and n = 0. Step10: -If k = 1, then finish the initial self-localization. -Otherwise, go to step 4 and n = n + 1. We conducted an experiment using the initial self-localization method. The parameters used for self-localization are shown in the following. The numbers of parent candidates are 1000 and the numbers of offspring candidates are 500; the initial resolution level k of the multi-resolution map is 2. Figures 15 and 16 show the experimental results in a laboratory. In Fig.15 (a), the candidate solutions are randomly distributed all over the map in order to estimate the current robot position (The candidates are drawn by purple). However, the best fitness value is low, because the calculation result of fitness values is very sensitive to the estimation error of selfposition in the high-resolution map (k=2). Therefore, the robot updates the resolution level k = 4 in Fig.15 (b). When the resolution level is k=4, the best fitness value is higher than 0.9 in Fig.16. In the low-resolution map, it is easy to estimate the robot position because the low-resolution map has the wide acceptable error range. By estimating the self-position in low-resolution map, the best fitness value is high after downgrading the resolution level. In this way, the robot can estimate the current position (The best candidate is drawn by the red circle) in Fig.15 (c) by using the multiresolution map.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

136

N. Kubota and Y. Toda / Information Visualization in Intelligent Navigation

(a) t = 0

(b) t = 35

(c) t = 75

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Fig.15. Experimental result of initial self-localization

Fig.16. Change of resolution level and fitness value in the experimental results 4.4. Information Visualization by 3D Map building in Intelligent Navigation System This section explains how to integrate 2D map building and 3D environmental map building. In order to realize the remote monitoring and remote control, the self-position of each mobile robot should be estimated as precisely as possible. Therefore, we use 2D map building based on occupancy grid maps in SLAM using the LRF. According to the self-position, the robot performs 3D environmental map building using 3D distance images and camera images, because the preciseness of distance measurement using LRF is higher than that of 3D infrared distance sensor. Figure 17 shows an experimental result of information visualization on the iPhone.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

N. Kubota and Y. Toda / Information Visualization in Intelligent Navigation

137

Fig. 17 Experimental results of information visualization on iPhone

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

5. Summary This chapter discussed information visualization methods and simultaneous localization and mapping for intelligent navigation of multiple mobile robots for remote monitoring and control. First, we explained the total architecture of intelligent navigation system. In order to realize the remote monitoring and remote control, the self-position of each robot should be estimated as precisely as possible. Therefore, we used SLAM using the LRF. According to the self-position, the robot performs 3D environmental map building using 3D distance images and camera images. We applied (µ+λ)-ES and (µ+1)-ES (SSGA) for self-localization of mobile robots. We applied SIFTGPU, RANSAC, and quaternion for feature selection and matching of two camera images. The experimental results show that the proposed method can conduct selflocalization and 3D environmental map building. On the other hand, we proposed navigation systems based on touch interface [37], intelligent path planning methods [38], and formation behaviors of multiple mobile robots [39]. As a future work, we intend to integrate these systems to realize an intelligent navigation system of multiple mobile robots.

References [1] Naoyuki Kubota, Human-friendly tele-operation for robot partners, Proc. of the Institution of Mechanical Engineers, Part 1: Journal of Systems and Control Engineering, vol.225, no.3, pp361-336, May 2011. [2] L. E. Parker, Designing Control Laws for Cooperative Agent Teams, Proceedings of the 1993 IEEE International Conference on Robotics and Automation, 1993, pp. 582-587. [3] A. K. Das, R. Fierro, V. Kumar, J. P. Ostrowski, C. J. Taylor, A Vision-Based Formation Control Framework, IEEE Trans. on Robotics and Automation, vol. 18, pp. 813-825, 2002. [4] T. D. Barfoot, C. M. Clark, “Motion Planning for Formations of Mobile Robots”, Robotics and Autonomous Systems, vol. 46, pp. 65-78, 2004.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

138

N. Kubota and Y. Toda / Information Visualization in Intelligent Navigation

[5] P. Urcola, L. Riazuelo, M. T. Lazaro, L. Montano, Cooperative Navigation using Environment Compliant Robot Formation, Proc (CD-ROM) of IEEE/RSJ International Conference on Intelligent Robots and System, pp.2789-2794, 2008. [6] P. T. Szemes, H. Hashimoto, Fuzzy Neural Network based Mobile Agent Control for Intelligent Space, Proc. of SICE Annual Conference 2004, 2004, pp.1372-1377. [7] Dongbing Gu, Huosheng Hu, “Using Fuzzy Logic to Design Separation Function in Flocking Algorithms”, IEEE Trans. Fuzzy Systems, in press, 2007 [8] N.Kubota, M.Ogishi, and F.Kojima, Learning of Multiple Robots in Quasi-Ecosystem, Proc. (CD-ROM) of 2000 26th Annual Conference of the IEEE Industrial Electronics Society, pp.2105-2110, 2000. [9] J.A.Anderson, E.Rosenfeld, Neurocomputing, The MIT Press, 1988. [10] J.-S.R.Jang, C.-T.Sun, E.Mizutani, Neuro-Fuzzy and Soft Computing, Prentice-Hall, Inc., 1997. [11] S.J.Russell, P.Norvig. Artificial Intelligence, Prentice-Hall, Inc., 1995. [12] J. M. Zurada, R. J. Marks II, C. J. Robinson (eds.), Computational Intelligence - Imitating Life, IEEE Press, 1994. [13] N.Kubota, T.Narita, B.H.Lee, 3D Topological Reconstruction based on Hough Transform and Growing Neural Gas for Informationally Structured Space, Proc. (CD-ROM) of The 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.3459-3464, 2010. [14] N.Kubota, T.Obo, T.Fukuda, An Intelligent Monitoring System based on Emotional Model in Sensor Networks, The 18th IEEE International Symposium on Robot and Human Interactive Communication, pp.346-351, 2009. [15] H. Sasaki, N. Kubota, K. Taniguchi, “Topological Map and Cell Space Map for SLAM of A Mobile Robot”, GESTS International Transactions on Computer and Engineering, Vol.45, No.3, 2008. [16] J. Woo㧘N. Kubota㧘B.-H. Lee, ”Steady-state Genetic Algorithms for Growing Topological Mapping and Localization”, Pacific Rim International Conference on Artificial Intelligence (PRICAI 2010), LNAI 6230, pp.558-569, August 30-September 2, 2010. [17] Takenori Obo; Naoyuki Kubota; Kazuhiko Taniguchi; Toshiyuki Sawaya, Human Localization Based on Spiking Neural Network in Intelligent Sensor Networks, Proc. (CD-ROM) of IEEE Symposium Series on Computational Intelligence 2011 (SSCI2011), Paris, France, April 11-15, 2011. [18] Takenori Obo, Naoyuki Kubota, Toshiyuki Sawayama, Kazuhiko Taniguchi, A Fuzzy Spiking Neural Network Using Optical Oscillosensor and Pneumatic Sensor for Human State Estimation, Proc. (CDROM) of World Automation Congress (WAC) 2010, Kobe International conference Center Kobe, Japan, September 19-23, 2010. [19] Takenori Obo, Naoyuki Kubota, Lee Beom, Localization of Human in Informationally Structured Space Based on Sensor Networks Recent Advances on Fuzzy Robotics and Multi-robotics, 2010 IEEE World Congress on Computational Intelligence (WCCI 2010), Barcelona, Spain, July 21-23, 2010. [20] Naoyuki Kubota, Akihiro Yorita, Topological Environment  Reconstruction in Informationally Structured Space for Pocket Robot Partners, 2009 IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA 2009), pp.165-170 Daejeon, Korea, December 15-18, 2009. [21] N.Kubota, N.Aizawa, Self-Adaptation in Intelligent Formation Behaviors of Multiple Robots based on Fuzzy Control, 2009 IEEE International Conference on Fuzzy Systems, 2009. [22] SIFT on GPU, http://www.cs.unc.edu/~ccwu/siftgpu/ [23] B.Horn, “Closed-Form Solution of Absolute Orientation Using Unit Quaternions,” JOSA A, Vol. 4, No. 4, 1987. [24] S. Thrun, W. Burgard, and D. Fox, “Probabilistic Robotics”, The MIT Press, 2005. [25] S.Huang, G.Dissanayake, “Convergence and Consistency Analysis for Extended Kalman Filter Based SLAM”, IEEE Transactions on Robotics, vol.23 (5), pp1036-1049, Oct. 2007. [26] T.Bailey, J.Nieto, J.Guivant, M.Stevens, E.Nebot, “Consistency of the EKF-SLAM Algorithm”, Proc. of 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.3562-3568, Oct. 2006. [27] J. Folkesson, H.I. Christensen, “Closing the Loop With Graphical SLAM”, IEEE Transactions on Robotics, vol.23 (4), pp731-741, Aug. 2007. [28] M. Kaess, A. Ranganathan, F. Dellaert, “iSAM: Incremental Smoothing and Mapping”, IEEE Transactions on Robotics, vol.24 (5), pp1365-1378, Dec. 2008. [29] C.Kim, R.Sakthivel, W.K. Chung, “Unscented FastSLAM: A Robust and Efficient Solution to the SLAM Problem”, IEEE Transactions on Robotics, vol.24 (4), pp808-820, Aug. 2008. [30] J. Folkesson, P. Jensfelt, H. I. Christensen, “The M-space Feature Representation for SLAM”, IEEE Transactions on Robotics, vol.23 (5), pp1024-1035, Oct. 2007. [31] J.-L.Blanco, J.-A.Fernandez-Madrigal, J. Gonzalez, “Towards a Unified Bayesian Approach to Hybrid Metric-Topological SLAM”, IEEE Transactions on Robotics, vol.24 (2), pp259-270, April. 2008.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

N. Kubota and Y. Toda / Information Visualization in Intelligent Navigation

139

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

[32] K. Ni, F. Dellaert, “Multi-Level Submap Based SLAM Using Nested Dissection”, Intelligent Robots and Systems, 2010. IROS 2010, pp. 2558–2565, Oct. 2010. [33] Naoyuki Kubota, Yuichiro Toda, Shintaro Suzuki, Multi-scale Intelligent Information Processing for Multi-robot System based on Human-friendly tele-operation, 2011 International Symposium on MicroNanoMechatronics and Human Science, pp. 152-157, Nagoya, Japan, November 6-9, 2011. [34] S.Thrun, “Learning Occupancy Grid Maps With Forward Sensor Models”, Autonomous robots, Springer, Volume 25 (2), pp111-127, 2003 [35] K.Lee, W.K. Chung, “Effective Maximum Likelihood Grid Map with Conflict Evaluation Filter using Sonar Sensors”, IEEE Transactions on Robotics, vol.25 (4), pp887-901, Aug. 2009. [36] Yuichiro Toda, Shintaro Suzuki, Naoyuki Kubota, Self-localization of Multi-robot System based on Simultaneous Localization and Mapping, Proc (CD-ROM) of International Symposium on Advanced Intelligent Systems (ISIS2011), Suwon, Korea, September 28 – October 1, 2011. [37] Naoyuki Kubota㧘Yuichiro Toda㧘Beom Lee㧘Multifeatured Visualization and Navigation in Teleoperation of Mobile Robots, Proc. (CD-ROM) of IEEE Symposium Series on Computational Intelligence 2011 (SSCI2011), Paris, France, April 11-15, 2011. [38] Y.Toda, N.Kubota; N.Baba, Intelligent Planning Based on Multi-resolution Map for Simultaneous Localization and Mapping, Proc. (CD-ROM) of IEEE Symposium Series on Computational Intelligence 2011, Paris, France, April 11-15, 2011. [39] Naohide Aizawa, Naoyuki Kubota, Intelligent Formation Control based on Directionality of MultiAgent System, Proc. (CD-ROM) of IEEE Symposium Series on Computational Intelligence 2009 (IEEE SSCI 2009), Tennessee, USA, March 30 - April 2, 2009.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

140

Simulation and Modeling Related to Computational Science and Robotics Technology F. Kojima et al. (Eds.) IOS Press, 2012 © 2012 The authors and IOS Press. All rights reserved. doi:10.3233/978-1-61499-092-5-140

Parallel Distributed Genetic Rule Selection for Data Mining from Large Data Sets Yusuke NOJIMAa,1, Shingo MIHARAa, and Hisao ISHIBUCHIa a Department of Computer Science and Intelligent Systems, Osaka Prefecture University, Japan

Abstract. Genetic algorithms (GAs) have been successfully used for data mining thanks to their flexibility. Users easily incorporate their preference into objective functions to be optimized. Although GA-based data mining techniques are useful, there is a serious difficulty in the handling of data sets with a large number of patterns and/or attributes. That is, we need much long computation time for the evaluation of candidate solutions because all the patterns have to be classified by each candidate solution. To reduce the computation time of GA-based data mining, we propose a parallel distributed implementation of genetic rule selection. The main characteristic is to divide not only a population in GA but also a training data set into a number of sub-groups. Then a pair of a sub-population and a training data subset is assigned to a single CPU core. This approach can drastically reduce the computation time with no serious deterioration in the generalization ability of obtained classifiers. We demonstrate the effectiveness of our parallel distributed implementation through computational experiments on large data sets available from the UCI machine learning repository. Keywords. Genetic algorithms, data mining, genetic rule selection, parallel distributed implementation, pattern classification.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Introduction Association rule mining is one of the most frequently-used data mining techniques [1]. Application of association rule mining to pattern classification problems is often referred to as classification rule mining [2]-[4]. The task is to extract a set of if-then type classification rules from numerical data with class labels. This set of rules can be regarded as a classifier. By specifying rule evaluation criteria such as confidence and support, a large number of association rules can be extracted from a data set. An important issue is how to choose promising rules from the extracted ones. In [5]-[7], fuzzy if-then rules are extracted in a heuristic manner. Then a genetic algorithm (GA) [8] is used to optimize a subset of the extracted rules with respect to a user-defined fitness function. We can obtain a simple but accurate classifier by maximizing the classification rate and minimizing the number of fuzzy rules. This method is called genetic rule selection and currently one of the well-known approaches in the field of genetic fuzzy systems [9]-[11]. Although GA-based data mining approaches including genetic rule selection are very powerful and flexible in general [12], there is a scalable issue for large data sets 1

[email protected], Gakuen-cho 1-1, Naka-ku, Sakai, Osaka 599-8531, JAPAN

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

141

with many patterns and/or attributes. Since GA is a population-based stochastic optimization method, a large number of fitness evaluations are needed for optimization. For each fitness evaluation, a candidate solution (i.e., a classifier) has to classify all the training patterns. Thus, when we apply GA-based data mining to large data sets, we need much longe computation time. There are various techniques to reduce the computation time of GA and GA-based data mining methods. One popular approach is a parallel implementation of GA such as island models [13], [14]. In this approach, a population is divided into several sub-populations. Each sub-population is assigned to an island. Then GA process is performed in each island with the assigned subpopulation. This approach can reduce the computation time if we implement it to a cluster system or a workstation with multiple CPU cores. Another popular approach is data reduction such as instance selection [15]-[17]. We can reduce the computation time depending on the reduction rate of a data set. As a similar idea to instance selection, training data is divided into several sub-groups. Then a training subset is used for fitness evaluation. In [18], a different training data subset is used at every generation. This is called a windowing approach. In our former study [19], we combined the ideas of these two approaches and proposed a parallel distributed implementation of genetic rule selection for fuzzy classifier design from large data sets. The main characteristic is to divide not only a population but also a training data set into a number of sub-groups. Then a pair of a sub-population and a training data subset is assigned to a single CPU core. GA process is performed at each CPU core in order to optimize the sub-population with the corresponding training data subset. To avoid overfitting of the sub-population to the training data subset, we employ the periodical rotation of training data subsets. After a prespecified number of generations, a different training data subset is assigned to each sub-population. Through computational experiments on large data sets, we showed that our parallel distributed implementation can drastically reduce the computation time with no serious deterioration in the generalization ability of obtained fuzzy classifiers on test data sets. We also examined some extensions of the parallel distributed implementation. In [20], we divided a training data set into a larger number of subsets than the number of sub-populations for further reduction of the computation time. In [21], we chose fuzzy classifiers from all the sub-populations in order to generate an ensemble classifier. We also applied the idea of our parallel distributed implementation to hybrid fuzzy genetics-based machine learning [22], [23]. In this chapter, we use interval sets as antecedent conditions of if-then rules instead of fuzzy sets. While fuzzy sets used in the previous studies [19]-[21] are homogeneous partitions in order to give linguistic terms like small or big, interval sets used in this chapter are inhomogeneous ones which are well situated depending on the class distribution in the pattern space. Thus, there is a possibility that we can obtain more accurate classifiers using interval sets. Of course, there is another possibility that interval rule-based classifiers easily overfit to a training data subset and our parallel distributed implementation does not work well. So, the purpose of this chapter is to confirm the applicability of our parallel distributed genetic rule selection to the design of interval rule-based classifiers for large data sets. This chapter is organized as follows. First we explain interval rule-based classifiers and standard non-parallel genetic rule selection in Section 1. Next we explain our parallel distributed implementation of genetic rule selection in Section 2. The effectiveness of the parallel distributed implementation is examined through computational experiments in Section 3. Finally Section 4 concludes this chapter.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

142

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

1. Genetic Rule Selection for the Design of Interval Rule-based Classifiers 1.1. Interval Rule-based Classifiers Let us assume that we have m training (i.e., labeled) patterns xp = (xp1, ..., xpn), p = 1, 2, ..., m from M classes in an n-dimensional pattern space where xpi is the attribute value of the pth training pattern for the ith attribute (i = 1, 2, ..., n). We denote the set of these training patterns by D (i.e., D = {x1, ..., xm}). We also denote the set of training patterns from Class h as D(Class h) where h = 1, 2, ..., M. For our n-dimensional M-class pattern classification problem, we use if-then rules of the following type: Rule Rq: If x1 is Aq1 and ... and xn is Aqn then Class Cq with CFq,

(1)

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

where Rq is the label of the qth rule, Aqi is an antecedent interval (i = 1, 2, ..., n), Cq is a class label, and CFq is a real number in the unit interval [0, 1] which represents a rule weight (i.e., certainty grade). Each antecedent condition “xi is Aqi” means that xi is in [ AqiL , AqiU ]. We denote the antecedent part of the classification rule Rq in (1) by the interval vector Aq where Aq = (Aq1, ..., Aqn). Thus Rq is denoted as “Aq  Class Cq”. To generate antecedent intervals, we simultaneously use multiple partitions with different granularities as in Fig. 1. This is because we usually have no a priori information about an appropriate granularity of the discretization for each attribute. Since we simultaneously use multiple partitions with different granularities, we do not have to determine the number of intervals for each attribute. In our computational experiments, we use five granularities with K intervals where K = 1, 2, 3, 4, 5 (see Fig. 1). That is, each antecedent interval Aqi is chosen from 15 intervals in Fig. 1. Thus, the total number of combinations of antecedent intervals is 15n for our n-dimensional pattern classification problem. It should be noted that K = 1 in Fig. 1 corresponds to “don’t care” condition which plays a role in generalizing its rule.

K=5 K=4 K=3 K=2 K=1 0

Attribute value

1

xi

Figure 1. Five partitions with different granularities used in our computational experiments.

As shown in Fig. 1, the domain interval is divided into K intervals. In the area of machine learning, a number of discretization methods of continuous attributes into disjoint intervals have been proposed in the literature [24]-[27]. In this chapter, we use an optimal splitting method based on the class entropy measure [24] to specify (K  1) cutting points.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

| D j | M % | D jh | | D jh | " , & log 2  ## | D j | ! j 1 | D | h 1$ | D j |

143

K

H ( A1, ..., AK )   

(2)

where (A1, …, AK) is K intervals generated by the discretization of an attribute, Dj is the set of training patterns in the interval Aj, and Djh is the set of training patterns from Class h in Dj . Using the optimal splitting method [27], we obtain multiple partitions for various values of K for each attribute. Figure 1 shows an example of 21 patterns with three different classes (i.e., open circle, closed circle, and triangle). Since some intervals are identical to others, we regard such intervals as one interval. In the case of Fig. 1, we use nine individual intervals as antecedent conditions of this attribute. To determine the consequent class Cq and the rule weight CFq, first the confidence of the rule “Aq  Class h” is calculated for each class h. Let D(Aq) be the set of compatible training patterns with the antecedent part Aq: D ( A q )  {x p | x p1 ' Aq1 , ..., x pn ' Aqn } .

(3)

When D(Aq) is empty, we do not generate any rule with the antecedent part Aq. The confidence of “Aq  Class h” is calculated as follows:

c( A q  Class h) 

D ( A q )  D (Class h) D(A q )

, h = 1, 2, …, M.

(4)

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

The consequent class Cq is specified as the class with the maximum confidence. The consequent class Cq can be viewed as the dominant class among the compatible training patterns with the antecedent part Aq. The rule weight CFq is specified by the difference between the confidence of the consequent class and the sum of the confidences of the other classes as: CFq  c( A q  Class Cq ) 

M

 c( A q  Class h).

(5)

h 1 h * Cq

An interval rule-based classifier is constructed by a number of if-then rules of the above type. When a new pattern is to be classified by a classifier S (i.e., a rule set S), first all compatible rules with the new pattern are found from S. Then a single winner rule Rw with the largest rule weight is identified among the compatible rules. The new pattern is classified as the consequent class of the winner rule Cw. When multiple rules with different consequent classes have the same rule weight, the classification of the input pattern xp is rejected. The classification of xp is also rejected when there are no compatible rules with positive compatibility grades for xp. 1.2. Genetic Rule Selection This section explains how to generate a classifier by GA. We use genetic rule selection [5] which is a two-phase method. At the first phase, a number of if-then rules are

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

144

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

extracted in a heuristic manner. As explained in the previous section, we can generate a large number of classification rules by specifying the consequent class and the rule weight for each of the 15n combinations of the antecedent intervals. If n = 10, 5.7 x 1011 rules are generated. It is, however, very difficult for human users to handle such a large number of rules. It is also very difficult to intuitively understand long rules with many antecedent conditions. Thus we only generate short rules with a small number of antecedent conditions in the first phase of genetic rule selection. The number of antecedent conditions excluding “don’t care” conditions is referred to as the rule length. We only examine short rules of length Lmax or less (e.g., Lmax = 3). This restriction on the rule length is to find a small number of short (i.e., simple) rules that are easily understood by human users. We further decrease the number of rules by choosing only good rules. The goodness is defined by the confidence and support which are often used in the field of data mining as heuristic rule evaluation criteria [1]. The support of “Aq  Class h” is calculated as follows: s ( A q  Class h) 

D( A q )  D(Class h) D

, h = 1, 2, ..., M.

(6)

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

In association rule mining, the thresholds of the confidence and support are usually specified, and all the rules satisfying those thresholds are extracted. But, it is difficult to specify those thresholds beforehand. In an extreme case, no rule with minority class is extracted. To avoid this situation, in this chapter, we use the product of confidence and support as a heuristic rule evaluation criterion. We extract a prespecified number of rules per class with respect to this rule evaluation criterion (e.g., 300 rules per class). At the second phase of genetic rule selection, a combination of extracted rules at the first phase is optimized by GA. The GA procedure is as follows: Step 1: Randomly generate Npop binary strings of length N as an initial population. Step 2: Evaluate each string using a weighted-sum fitness function. Step 3: Iterate the following operations Npop times to generate an offspring population with Npop strings. 3.1: Select a pair of parent strings from the current population by binary tournament selection. 3.2: Recombine the selected parent strings to generate an offspring by the uniform crossover operation. 3.3: Apply a biased mutation operation to the offspring. 3.4: Calculate the number of correctly classified training patterns by the offspring and remove unnecessary rules from the offspring. 3.5: Evaluate the offspring using the weighted-sum fitness function. Step 4: Combine the current and offspring populations into the merged population, and select the best Npop strings from the merged population as the next population. Step 5: If a prespecified termination condition is not satisfied, return to Step 3. Otherwise, terminate the execution of the algorithm. Step 6: Select the best individual with respect to the weighted-sum fitness function from the final population.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

145

In Step 1, a subset S of the N extracted rules is denoted by a binary string of length N as S = s1s2s3 ···sN where si = 1 and si = 0 mean that the ith extracted rule is included in and excluded from the rule set S, respectively. Such a binary string is used as an individual (i.e., a classifier) of GA as in Fig. 2. Extracted Rules

Genotype

Phenotype

If-then rule A If-then rule B If-then rule C If-then rule D If-then rule E If-then rule F If-then rule G If-then rule H If-then rule I If-then rule J If-then rule K If-then rule L

1 0 1 0 0 0 1 1 0 0 1 0

If-then rule A If-then rule C If-then rule G If-then rule H If-then rule K

Classifier

Figure 2. A classifier represented by a binary string.

In Step 2, each string is evaluated by a weighted-sum fitness function composed of an accuracy measure and complexity measures as:

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

fitness ( S )  w1 & f1 ( S )  w2 & f 2 ( S )  w3 & f 3 ( S ) ,

(7)

where w1, w2, and w3 are non-negative weight values (e.g., w1 = 100, w2 = 1, w3 = 1 in our computational experiments). f1(S) is the number of correctly classified training patterns by S. f2(S) is the number of selected if-then rules in S. f3(S) is the total rule length (the total number of antecedent conditions except for don’t care conditions) in S. We maximize this fitness function to obtain an accurate and simple classifier. In Step 3.2, each bit value of the offspring is inherited from one parent randomly chosen from the two parents. This operation is applied with a prespecified probability (e.g., 0.9). When the crossover operation is not applied to the selected pair of parents, one of the two parents is randomly chosen and used as an offspring. In Step 3.3, the biased mutation changes a 0 to a 1 with a small probability and a 1 to a 0 with a large probability to decrease the number of 1’s (i.e., the number of selected rules) in the offspring. This bias intends to make the offspring simpler. In Step 3.4, only the first objective function f1(S) is calculated, while checking which rules do not become a winner rule. Some rules in S may be chosen as the winner rules for no training patterns. We refer to those rules as unnecessary rules. We remove unnecessary rules without degrading the accuracy. Then we calculate the second and third objective functions (i.e., f2(S) and f3(S)) in Step 3.5. We often use the total number of iterations of the algorithm (i.e., the total number of generations) as the termination condition.

2. Parallel Distributed Implementation of Genetic Rule Selection In this section, we explain our parallel distributed implementation of genetic rule selection. In this chapter, we implement it to a workstation with four CPU cores. We employ a server-client system. Three of four CPU cores are used as client CPU cores to

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

146

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

perform GA process. The other one is used as a server CPU core. Our parallel distributed implementation of genetic rule selection is written as follows:

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Step 1: Extract interval if-then rules at the server CPU core. Step 2: Randomly generate Npop binary strings of length N as an initial population using the server CPU core. Step 3: Randomly divide the current population and the training data set into subpopulations and training data subsets, respectively at the server CPU core. Step 4: Assign a pair of a sub-population and a training data subset to each client CPU core. Step 5: Execute GA for a prespecified number of generations at each client CPU core using the assigned sub-population and training data subset. Step 6: Systematically change the assignment of the training data subsets to the client CPU cores. Step 7: If a prespecified termination condition is not satisfied, return to Step 5. Otherwise go to Step 8. Step 8: Combine the sub-populations and the training data subsets into a population and a training data set, respectively. Step 9: Calculate the fitness value of each string in the population using the whole training data set at the server CPU core. Then choose the best string as the final solution. Figure 3 illustrates our parallel distributed implementation. A population is divided into three sub-populations of the same size (i.e., P1, P2, and P3). A training data set is also divided into three subsets of the same size while maintaining the class balance (i.e., T1, T2, and T3). Each CPU core performs GA using a sub-population and a training data subset (e.g., GA(P1, T1), GA(P2, T2), and GA(P3, T3) where GA(·) is the process of GA) for a prespecified number of generations (e.g., 50 generations). After that, the training data subsets are rotated clockwise as in Fig. 3. Then each CPU core performs GA again using the sub-population and the newly assigned training data subset (e.g., GA(P1, T3), GA(P2, T1), and GA(P3, T2)). This periodically rotation of training data subsets is used for avoiding the overfitting of each sub-population to only the corresponding training data subset assigned at first. CPU Sub-population

CPU Sub-population

CPU Sub-population

P1

P2

P3

T1

T2

T3

Data subset

Data subset

Data subset

Figure 3. Parallel distributed implementation of genetic rule selection with training data subsets rotation.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

147

After changing the assignment of training data subsets, each string is evaluated using the newly assigned training data subset. Of course, the best string at the previous generation is not always the best for the newly assigned training data subset. For fair comparison, we employ the number of fitness evaluations as the termination condition of our parallel distributed implementation. Our parallel distributed implementation theoretically decreases the computation time by the magnitude of the square of the number of client CPUs. Thus, it is nine times faster than the original non-parallel algorithm in Section 1 when we have three client CPU cores. This is because both the population size and the number of training patterns at each client CPU core are 1/3 of those in the original non-parallel algorithm. We also examine the effects of the number of training data subsets as in [20]. We divide a training data set into a larger number of training data subsets than the number of sub-populations (see Fig. 4). Whereas every training data subset is assigned to a different CPU core (i.e., a different sub-population) in the case of Fig. 3, some of the training data subsets are not assigned to any sub-populations in the case of Fig. 4. However, the periodical rotation of the training data subsets makes it possible for each CPU core to use all training patterns when specifying the rotation interval as a small number of generations. If we divide a training data set into six subsets, the GA process theoretically becomes 18 times faster than that by the original non-parallel algorithm.

CPU Sub-population

CPU Sub-population

CPU Sub-population

P1

P2

P3

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

T1

T2

T3

Td

T4

Figure 4. Parallel distributed implementation with a large number of training data subsets.

3. Computational Experiments Through computational experiments on three benchmark data sets in Table 1 which are available from the UCI machine learning repository, we examined the applicability of our parallel distributed genetic rule selection to the design of interval rule-based classifiers. Table 1. Data sets used in our computational experiments. Data name

Number of attributes

Number of patterns

Number of classes

Phoneme

5

5404

2

Satimage Penbased

36 16

6435 10992

6 10

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

148

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

We evaluated the generalization ability of the obtained interval rule-based classifiers by iterating the ten-fold cross validation procedure three times (i.e., 3 x 10 CV). That is, each result is the average over 30 runs. We used a workstation with two Intel Xeon 3.0 GHz dual processors (i.e., four CPU cores, in total). 3.1. Comparison with Non-Parallel Implementation First we compared the parallel distributed implementation with the standard nonparallel genetic rule selection. Experimental settings are as follows: , , , , , , , ,

Maximum number of extracted rules per class: 300, Maximum rule length of each rule: 2 for Satimage, 3 for the other data sets, Population size: 300 (i.e., sub-population size: 100), Weight vector in (7): w = (100, 1, 1), Crossover probability: 0.9, Mutation probability: 0.05 for 10 and 1/N for 01 where N is the total number of extracted rules, Termination condition: 300300 evaluations (i.e., 1001 generations in the case of the non-parallel algorithm), Number of training data subsets: 3.

We examined the effects of rotation intervals for training data subsets rotation. We examined the following five specifications:

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

,

Rotation interval (number of generations): 10, 50, 100, 200, none,

where “none” means that we do not change the assignment of training data subsets during the evolution. That case is the parallel implementation with instance selection. Table 2 summarizes the experimental results on the Phoneme data set. In the last row, we show the results by the standard non-parallel genetic rule selection. We performed statistical tests for examining the statistical significance of the difference between the standard non-parallel algorithm (i.e., “Standard” in Table 2) and the parallel distributed implementation with respect to the training data accuracy and the test data accuracy. We used a Wilcoxon signed-ranks test [28]. The results are highlighted in bold face when the results are not significantly different from the results by the non-parallel algorithm. We specified the significance level - as 0.05. We can observe from Table 2 that better test data accuracy was obtained than that by the non-parallel algorithm when the rotation interval was 10 generations. But there was no statistically significant difference in test data accuracy between all the specification of the parallel distributed implementation and the non-parallel one. The complexity (i.e., number of rules and average rule length) was similar to each other. From the last column of Table 2, we can see that the computation time was drastically reduced by the parallel distributed implementation. The reduction rate of the computation time was 8.03 (i.e., 35.72/4.45). Table 3 summarizes the experimental results on the Satimage data set. Regarding the training data accuracy, we can see the statistical difference from the standard algorithm. On the contrary, there was no statistically significant difference in test data accuracy. When we specified the rotation interval as 50, we obtained better test data accuracy than that by the standard algorithm. Furthermore its reduction rate of the computation time was 10.22 (i.e., 202.11/19.78). An interesting observation is that the

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

149

number of rules in the classifier obtained by the parallel distributed implementation was clearly smaller than that by the standard algorithm. This may be because some rules were well-suited for one training data subset but not for other training data subsets. Thus those rules might be removed just after changing the assignment of training data subsets. This means that the remaining rules are more general rules which cause better generalization ability of the obtained classifier. Besides, the decrease in the number of rules leads to the decrease in the computation time for evaluation. Table 4 summarizes the experimental results on the Penbased data set. We can observe almost the same effects of the parallel distributed implementation as the results on the Satimage data set in Table 3 except for the case where the rotation interval was 200 or none. This observation supports the importance of training data subsets rotation to obtain a classifier with high generalization ability.

Table 2. Experimental results on the Phoneme data set. Rotation interval 10

Training data accuracy

Test data accuracy 81.27

Number of rules 15.03

Ave. rule length 2.34

Computation time (min.) 4.45

82.42

50

82.30

81.25

15.57

2.40

4.42

100

82.30

81.01

15.80

2.29

4.34

200

82.30

81.12

14.70

2.19

4.35

None

82.16

80.92

15.40

2.14

4.31

Standard

82.42

81.26

16.30

2.25

35.72

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Table 3. Experimental results on the Satimage data set. Rotation interval 10

Training data accuracy 87.02

Test data accuracy 84.72

Number of rules 36.57

Ave. rule length 2.00

Computation time (min.) 20.13

50

86.80

84.96

34.53

2.00

19.78

100 200

86.62

84.78

36.03

2.00

19.44

86.48

84.69

35.40

1.99

19.80

None

86.18

84.50

36.57

1.99

20.55

Standard

87.30

84.84

45.20

2.00

202.11

Rotation interval 10

Training data accuracy 90.07

Test data accuracy

Table 4. Experimental results on the Penbased data set.

89.09

Number of rules 50.97

Ave. rule length 2.98

Computation time (min.) 51.09 49.70

50

90.11

89.15

49.00

2.99

100

90.09

89.15

48.77

2.98

48.62

200

89.98

88.90

48.50

2.98

49.17

None

89.60

88.69

49.97

2.98

50.88

Standard

90.37

89.35

65.40

2.99

493.77

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

150

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

3.2. Effects of the Number of Training Data Subsets

Test data accuracy (%)

83 82 81 80 79 10

50

83 82 81 80 79 10

100

200

6

3

Computation time [min.]

15 10 5 0 50

100

200

100

6 None 9

(c) Number of rules

200

6

3

None 9

(b) Test data accuracy

20

10

50

None 9

(a) Training data accuracy

Number of rules

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Training data accuracy (%)

We examined the effects of the number of training data subsets [20]. In this experiment, the number of training data subsets was specified as three, six, and nine. Figures 5, 6 and 7 show the results for 15 combinations of the rotation interval and the number of training data subsets. As in Subsection 3.1, we performed statistical tests for examining the statistical significance of the difference between the non-parallel algorithm (i.e., “Standard” in Table 2) and our parallel distributed implementation in the training data accuracy and the test data accuracy. We used a Wilcoxon signed-ranks test. The results are highlighted by dark color when the results are not significantly different from the results by the standard non-parallel algorithm. We specified the significance level - as 0.05. Figure 5 shows the results on the Phoneme data set. It is clear that the increase in the number of training data subsets led to the deterioration in training data accuracy (see Fig. 5 (a)). But the difference in test data accuracy between the standard nonparallel algorithm and the parallel distributed implementation was not statistically significant, when we specified the rotation interval and the number of training data subsets as 10 and 9, respectively (see Fig. 5 (b)). The increase in the number of training data subsets also led to the decrease in the number of rules (see Fig. 5 (c)). Figure 5 (d) shows that the computation time was directly related to the size of training data subsets.

3

6 4 2 0 10

50

100

200

6

3

None 9

(d) Computation time

Figure 5. Experimental results on the Phoneme data set by a large number of training data subsets.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

151

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

Test data accuracy (%)

Training data accuracy (%)

Under the condition that the difference between the standard algorithm and the parallel distributed one was not statistically significant in test data accuracy, the largest reduction rate of computation time was 22.38 (i.e., rotation interval: 10, the number of training data subsets: 9). That is, our parallel distributed implementation can be 22 times faster than the standard non-parallel algorithm and obtain a simpler classifier with no serious statistical difference in test data accuracy. Figure 6 shows the results on the Satimage data set. In Fig. 6 (a), the best specification of rotation interval seems to be 10 generations. But it did not always obtain good test data accuracy in Fig. 6 (b). The best specification may depend on the size of training data subsets for the Satimage data set. From Fig. 6 (b) and (d), the largest reduction rate of computation time was 22.45 (i.e., rotation interval: 50, the number of training data subsets: 6) under the condition that the difference was not statistically significant in test data accuracy.

88 86 84 82 10

86 85 84 83 82 10

50

100

200

6

3

Computation time [min.]

Number of rules

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

100

200

6 None 9

(c) Number of rules

200

6

3

None 9

(b) Test data accuracy

50 40 30 20 10 0 50

100

None 9

(a) Training data accuracy

10

50

3

24 18 12 6 0 10

50

100

200

6

3

None 9

(d) Computation time

Figure 6. Experimental results on the Satimage data set by a large number of training data subsets.

Figure 7 shows the results on the Penbased data set. We can observe the same effects of the rotation interval and the size of training data subsets as those on the Satimage data set in Fig. 6. However, the use of smaller training data subsets did not work well for the Penbased data set.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

Test data accuracy (%)

Training data accuracy (%)

152

91 90 89 88 87 10

90 89 88 87 10

50

100

200

6

3

Number of rules

Computation time [min.]

100

200

6 None 9

(c) Number of rules

200

6

3

None 9

(b) Test data accuracy

70 60 50 40 30 20 50

100

None 9

(a) Training data accuracy

10

50

3

60 40 20 0 10

50

100

200

6

3

None 9

(d) Computation time

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 7. Experimental results on the Penbased data set by a large number of training data subsets.

From the practical point of view, we can reduce the computation time to less than 1/10 by using smaller training data subsets if we accept slight deterioration in test data accuracy. A good thing is that we can obtain a simpler classifier with a smaller number of rules in that case. It should be noted that the reported computation time was only for the second phase of genetic rule selection (i.e., GA process). In this chapter, we ignored the computation time for the first phase (i.e., optimal splitting and rule extraction) because it was relatively small comparing with the second phase.

4. Conclusions In this chapter, we applied the idea of our parallel distributed implementation to the design of interval rule-based classifiers. Through the computational experiments, we examined the effects of the rotation interval and the number of training data subsets. We demonstrated that our parallel distributed genetic rule selection can obtain a simpler classifier whose test data accuracy is not statistically different from that obtained by the standard non-parallel algorithm. We also showed that the computation time can be reduced to less than 1/9 when we utilize only three CPU cores. As future research topics, we must find an appropriate rotation interval of training data subsets. It may depend on the characteristics of the data sets. We should also

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

153

consider an incorporation of genetic tuning for interval sets into the optimization process of GA instead of an optimal splitting method based on the class entropy measure. This is because the calculation of the class entropy measure also becomes a time-consuming task for large data sets which include too many patterns, attributes and classes. Furthermore we expect that the idea of our parallel distributed implementation is applicable to any GA-based data mining techniques. This is also our future research topic.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

References [1] R. Agrawal, H. Mannila, R. Srikant, H. Toivonen, and A. I. Verkamo, Fast discovery of association rules, In: U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, R. Uthurusamy (eds.), Advances in Knowledge Discovery and Data Mining, AAAI Press, Menlo Park (1996), 307-328. [2] B. Liu, W. Hsu, and Y. Ma, Integrating classification and association rule mining, Proc. of 4th International Conference on Knowledge Discovery and Data Mining (1998), 80-86. [3] C. C. Chiu and P. L. Hsu, A constraint-based genetic algorithm approach for mining classification rules, IEEE Trans. on Systems, Man, and Cybernetics: Part C 35 (2005), 205-220. [4] F. Thabtah, P. Cowling, and S. Hammoud, Improving rule sorting, predictive accuracy and training time in associative classification, Expert Systems with Applications 31 (2006), 414-426. [5] H. Ishibuchi, K. Nozaki, N. Yamamoto, and H. Tanaka, Selecting fuzzy if-then rules for classification problems using genetic algorithms, IEEE Trans. on Fuzzy Systems 3, 3 (1995), 260-270. [6] H. Ishibuchi, T. Murata, and I. B. Turksen, Single-objective and two objective genetic algorithms for selecting linguistic rules for pattern classification problems, Fuzzy Sets and Systems 89, 2 (1997), 135149. [7] H. Ishibuchi and T. Yamamoto, Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining, Fuzzy Sets and Systems 141, 1 (2004), 59-88. [8] D. E. Goldberg, Genetic Algorithms in Search Optimization and Machine Learning, Addison-Wesley, Reading, MA (1989). [9] O. Cordon, F. Herrera, F. Hoffmann, and L. Magdalena, Genetic Fuzzy Systems: Evolutionary Tuning and Learning of Fuzzy Knowledge Bases, World Scientific Publishers, Singapore (2001). [10] F. Herrera, Genetic fuzzy systems: Status, critical considerations and future directions, International Journal of Computational Intelligence Research 1, 1 (2005), 59-67. [11] F. Herrera, Genetic fuzzy systems: taxonomy, current research trends and prospects, Evolutionary Intelligence 1, 1 (2008), 27-46. [12] A. A. Freitas, Data Mining and Knowledge Discovery with Evolutionary Algorithms, Springer, Berlin (2002). [13] E. Alba and M. Tomassini, Parallelism and evolutionary algorithms, IEEE Trans. on Evolutionary Computation 6, 5 (2002), 443-462. [14] E. Cantu-Paz, A survey of parallel genetic algorithms, IlliGAL Report no. 95003 (1997). [15] H. Liu and H. Motoda, On issues of instance selection, Data Mining and Knowledge Discovery 6, 2 (2002), 115-130. [16] J. R. Cano, F. Herrera, and M. Lozano, Stratification for scaling up evolutionary prototype selection, Pattern Recognition Letters 26, 7 (2005) 953-963. [17] J. R. Cano, F. Herrera, and M. Lozano, Evolutionary stratified training set selection for extracting classification rules with trade off precision-interpretability, Data and Knowledge Engineering 60, 1 (2007) 90-108. [18] J. Bacardit, D. E. Goldberg, M. V. Butz, X. Llora, and J. M. Garrell, Speeding-up Pittsburgh learning classifier systems: Modeling time and accuracy, Proc. of 8th International Conference on Parallel Problem Solving from Nature (2004), 1021-1031. [19] Y. Nojima, H. Ishibuchi, and I. Kuwajima, Parallel distributed genetic fuzzy rule selection, Soft Computing 13, 5 (2009), 511-519. [20] Y. Nojima, H. Ishibuchi, and S. Mihara, Use of very small training data subsets in parallel distributed genetic fuzzy rule selection, Proc. of 4th International Workshop on Genetic and Evolutionary Fuzzy Systems (2010), 27-32. [21] Y. Nojima, S. Mihara, and H. Ishibuchi, Ensemble classifier design by parallel distributed implementation of genetic fuzzy rule selection for large data sets, Proc. of 2010 IEEE Congress on Evolutionary Computation (2010), 2113-2120.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

154

Y. Nojima et al. / Parallel Distributed Genetic Rule Selection for Data Mining

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

[22] Y. Nojima, S. Mihara, and H. Ishibuchi, Parallel distributed implementation of genetics-based machine learning for fuzzy classifier design, Lecture Notes in Computer Science 6457: Simulated Evolution and Learning, Springer, Berlin (2010) 309-318. [23] H. Ishibuchi, S. Mihara, and Y. Nojima, Training data subdivision and periodical rotation in hybrid fuzzy genetics-based machine learning, Proc. of 10th International Conference on Machine Learning and Applications (2011), 229-234. [24] J. R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers, San Mateo, CA (1993). [25] J. Dougherty, R. Kohavi, and M. Sahami, Supervised and unsupervised discretization of continuous features, Proc. of 12th International Conference on Machine Learning (1995) 194-202. [26] J. R. Quinlan, Improved use of continuous attributes in C4.5, Journal of Artificial Intelligence Research 4 (1996) 77-90. [27] T. Elomaa and J. Rousu, General and efficient multisplitting of numerical attributes, Machine Learning 36 (1999) 201-244. [28] D. J. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures (4th ed.), Chapman & Hall (2007).

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Robotics

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

This page intentionally left blank

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Simulation and Modeling Related to Computational Science and Robotics Technology F. Kojima et al. (Eds.) IOS Press, 2012 © 2012 The authors and IOS Press. All rights reserved. doi:10.3233/978-1-61499-092-5-157

157

On Comutational Robotics Zhiwei LUO Graduate School of System Informatics, Kobe University Abstract. This paper gives a comprehensive study on “computational robotics” motivated by the recent rapid development of supercomputer technologies. By applying advanced computational scientific approach, it is expected to realize deeper understanding of human as well as to solve more complicated computational problems for the next generation of robotics. Specifically, three main problems are discussed. The first one is about high dimensional superredundancy considering the full body human motor functions. With the rapid computation ability, it will be possible to realize simultaneous human movement measurement with dynamic human motion analysis and simulation within real time. The second topic is on the massive computation of a robot’s complex full body physical interactions with environment and/or manipulated objects such as a cared person. Here, how to simulate, plan and control the interactions with many kinds of frictions are discussed. The discrete feature of physical contacts together with continuous behaviors of motion/force control makes the problem as a hybrid dynamic system. Systematic approach to solve such problem requires massive computation. The final one simply studies on a robot’s cognitive motion in the real world so that to adapt to the unknown and/or uncertain environment. Such problem requires to process huge cognitive information inputs within real time so as to determine the robot’s environmental adaptive actions.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Keywords. Super-redundant mechanical system, Full body physical interactions, Cognitive motion, Computation, Robotics

1. Introduction History tells us that, development of robotics is highly relied on the development of computer engineering as well as Information science. Today, the world fastest supercomputer [K-computer] in Kobe, named for the Japanese word "kei" ( ੩ ) meaning 10 quadrillion, has reached the speed of calculation up to 10.51 PFLOPS, which strongly motivated the computational science to research and predict multi scale or multi physical complex nature through modeling and simulation in various scientific disciplines. By integrating studies of computational science with robotics, it is also expected that we can not only get better understanding of complex mechanisms of human, but also be helpful in the development of the next generation of robotics, such as more realistic computer simulation of the robot and its environment as well as real time sensing, planning and physical control of the flexible robots. There are many computation problem in robotics, rang from sensing, cognition, communication to motion planning, simulation and dynamic control in complex environment. In addition, by considering the real robot as a part of the human model, computation robotics may also open the new possibility to analyze and biofeedback of human motions.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

158

Z. Luo / On Comutational Robotics

In this paper, we only discuss three topics related to the computational problems of robotics. The first problem is on high dimensional super-redundancy, considering the full body human motor functions. The second problem is on the massive computation of a robot’s complex full body physical interactions with environment and/or manipulated objects such as a cared person. Here, how to simulate, plan and control the interactions with many kinds of frictions are considered. The discrete feature of physical contacts together with continuous behaviors of motion/force control makes the problem as a hybrid dynamic system. Systematic approach to solve such problem requires massive computation. The final problem studies on a robot’s cognitive motion in the real world so that to adapt to the unknown and/or uncertain environment. Such problem requires be solved with huge cognitive information input to determine the robot’s adaptive actions within real time.

2. Super-redundant computation

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Even for the easy task to reach the hand to the toy he/she saw is never simple in cybernetics, though it can be easily realized by a five months old baby. At least, it requires solve several nonlinear coordination transformations. The transformations may also contain the problem of redundancy. In this case, the inverse solution forms a solution manifold in the motor control space, which makes the solution not unique and not easy to be defined. Redundancy exists in a lot of levels of biological motor control systems.

Figure 1 A 3D computer simulation model of whole body human dynamic musculoskeletal system.

Figure 2 Whole arm cooperative manipulation of a ball.

Figure 1 shows a 3D computer simulation model of whole body dynamic musculoskeletal system of human we developed. As seen from this model, in order to realize the natural human motion, it is necessary to control more than 105 D.O.F by over 300 muscles. Human body is such a super-redundant system. Figure 2 shows at task level of another research example on how to control an object by whole arm cooperative manipulation under the influence from the external forces. Here, each arm has 4 D.O.F. and interacts with the object by all links not the end-effectors. From these

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

159

Z. Luo / On Comutational Robotics

two examples it is clear that, redundant D.O.F. provides the biological system with powerful hardware foundation to realize various smooth and delicate motions that have high reliability (reliable to the functional disability in some parts of the system) and adaptability (adapt to the environmental uncertainties, variations and different objectives). However, in order to organize the sensory-motor coordination, we have to overcome the ill-posed nonlinear problems that are not only come from the kinematics but also dynamics. 2.1.

Robotic researches of kinematic redundancy

Here, we first briefly review the redundancy problem and previous mathematical approaches to solve it. Without losing generality, we only consider the nonlinear relation between the task space and the joint space as x = f (θ ) (1)

[

where θ = θ 1 ,θ 2 ,...θ m

]T , x = [ x1 , x2 ,..., xn ]T , m > n , and

dx = Jdθ (2) where J is a n × m matrix. As shown in Figure 3, the range and null spaces of J are (3) R ( J ) = x ∈ R n : x& = J (θ )θ& for ∀θ& ∈ R m , N ( J ) = θ& ∈ R m : J (θ )θ& = 0

{

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

and dim R ( J ) + dim N ( J ) = m .

}

{

}

Figure 3 Nonlinear and redundant mapping If we assume the Jacobian J is known, then the following five typical approaches were proposed to solve the inverse: 1. Using the transpose of the matrix J to calculate the joints as

θ& = J T (x d − x)

where

(4)

x is the desired end-effector position [Chiacchio et al, 1991]. d

2. For the case when have

rank ( J ) = n , using J + , the pseudo-inverse of J , then we θ& = J + x& or θ& = J + x& + ( I − J + J )η

(5)

JJ + = I and vector ( I − J + J )η ∈ N ( J ) . When rank ( J (θ )) < n , then J is singular, the joint θ is the singular configuration [Klein et al, 1983].

where

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

160

Z. Luo / On Comutational Robotics

3. Specifying additional task constraints to extend J as a full rank square matrix

Je .

J e , we have [Baillieul, 1985] −1 θ& = J x&

(6)

Using this new matrix

e

4. Using regularization method to minimizes the cost function

dx − Jdθ + λ dθ . 5. Based on compliance control, by using the relations:

τ = Kθ dθ ⇔ F = K x dx;τ = J T F ,

T

T

−1

(7)

T

then we have Kθ = J K x J , dθ = ( J K x J ) J K x dx Approach 3 may closely related to the Bernstein’s concept of synergy [Bernstein,1967] considering the adding of additional task constraints. However, from the biological point of view, the main problem inherent in all above methods is the assumption that the system's Jacobian is known, which is impossible in biological system. In addition, the minimizations and task constraints considered here may be not realized in the biological system. These methods also suffer from the drawbacks that: (1) they need numerous computation of the Jacobian matrix and/or its pseudo-inverse; (2) the methods 2 and 3 may be numerically unstable; and (3) the quasicyclic problem [Lee et al, 1994]. 2.2.

Diffusion based computation algorithm Consider again the nonlinear and redundant unknown function

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

x = g(y ) ; x ∈ R n , y ∈ R m , m ≥ n . We try to obtain the inverse bounded task space:

(8)

y = g −1 (x) that minimizes a spatial criterion in a

1 ∂yT ∂y 2 ) + β (t ) A[x − g(y )] }dx. V (y ) = ∫ {α (t )tr ( x 2 ∂x ∂x

(9)

Using variational technic, it can be proved that the optimal inverse is the solution of the following PDE [Luo, 1998]:

y& = α (t )∇2 y + β (t ) A[x − g (y )]

(10)

This PDE has two terms. The first term is a diffusion term, that effects to interpolate the solutions of the y in the task space x, while the second term reduces the position errors. The discrete version of the equation is

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

161

Z. Luo / On Comutational Robotics

1 y ti ,+j1 = α (t )(y ti , j −1 + y ti −1, j + y ti , j +1 + y it +1, j ) 4 + β (t ) A tij (x id, j − g (y it , j )) where t is the evolution step, ( i ,

(11)

j ) are position in task space. As shown in Figure 4,

y ti ,+j1 is adjusted only depends on its four neighbor sides.

Figure 5 Resultant map

t +1

Figure 4, Adjust y i , j by its 4 neighbor sides

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

One of the main points in this approach is how to set the adjustment coefficients

α (t ) and β (t ) in the learning process. In our study, in order to learn the inverse Jacobian matrix A , we set the time functions α (t ) and β (t ) so that β (t ) = 1 − α (t ) . For example, initially we select coefficients α = 1 and β = 0 for only diffusion, after that, set α = 0 and β = 1 for error correction. Therefore, during diffusion process, the inverse matrix A can be obtained by A ti ,+j1 = A it , j +

1 Δx tij

T

2

(Δy tij − A tij Δx tij )Δx tij ,

(12)

considering the minimization of the cost function Ei , j =

1 Δy ti , j − A ti. j Δxti , j 2

2

(13)

where Δy ti , j = y ti , j − y ti.−j1 and Δxti , j = xit , j − xit ,−j1 are calculated during the two learning steps using forward relation.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

162

Z. Luo / On Comutational Robotics

The final computation algorithm is summarized as follows: 1. Given the forward function

x = g(y ) .

2. Select a boundary range in the task space x and divide it into a N˜N lattice. 3. Perform trial motions on the boundary and remember the corresponding y. 0

0

4. Set the initial condition y i , j and the initial inverse Jacobian Ai , j , respectively for

i, j = 1,2, K, N , and set the time functions α (t ) and β (t ) initially as α = 1 and β = 0 for only diffusion, after that, set α = 0 and β = 1 for error

all

correction. 5. Calculate Δy ti , j , Δxti , j , ∂Ei , j = −( Δy t − A t Δxt )Δxt T i, j i, j i, j i, j t ∂A i , j

for Ei , j = 1 Δy ti , j − A ti. j Δxti , j 2

2

t +1

6. Adjust y i , j and the inverse Jacobian matrix. Note that, for step 1, since x = g (y ) is a function from high to lower dimension, it is possible to be learned via general supervised learning, and if we already learned the system's forward relation in step 1, then during performing the learning steps of 5 and 6, we do not require the motor system to do the practical trial motions. Figure 5 shows the resultant map for a 3 link robot arm from above learning approach. It is clear that, the arm reaches its desired positions in all of the task space with spatial order. This approach has three advantages: (1). It does not require too many trial motions for the sensory-motor system. (2). During the map formation process, it requires only the local interactions between each node. Copyright © 2012. IOS Press, Incorporated. All rights reserved.

(3). It guarantees the final map's spatial optimality over all the bounded task space. The detailed proof of the above diffusion based learning algorithm using variational is given in [Luo et al, 1998]. The experiment of robotic application is also performed using FPGA.

2.3 Diffusion based generalization of optimal control Here, we consider the application of diffusion based computation in optimal motor control problem. Optimal control of the dynamic arm movement basically requires solve a nonlinear 2-point boundary problem, it takes enormous computation times. For any change of boundary conditions, it is necessary to perform the complex calculation again and hence is not suitable from motion generalization point of view. Here, we proposed to apply diffusion based algorithm in optimal motion generalization [Luo et al, 2001]. For example, as shown in Figure 6, we assume that, for the four initial S and terminal conditions T1 to T4, the optimal control inputs are already obtained, then, by using diffusion based algorithm, we can obtain all semi-

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Z. Luo / On Comutational Robotics

163

optimal controls for all initial and terminal conditions within a bounded work space as shown in Figure 7 without solving any nonlinear 2-point boundary equation.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 6 Diffusion based spatial generalization of optimal control

Figure 7 Compares the resultant semi-optimal solution with the optimal one.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

164

Z. Luo / On Comutational Robotics

2.4 Optimal motion formation Sensory-motor organization in above section solves the redundant relation from the sensory space input to the motor control output. In this section we consider how to solve the motion formation problem for the arm to move from one position to another one in the task space. Optimal formation of free motion For the simple human arm’s point to point (PTP) reaching movement in free motion space, it is found from experimental data that the hand path tends to be straight, slightly curved, and the velocity profile of the hand trajectory is smooth and bellshaped [Morasso, 1981; Abend et al, 1982]. These invariant features give us the hints about the internal representation of motor control in the central nervous system. One of the main approaches adopted in computational neuroscience is to account for these invariant features via optimization theory. Specifically, Flash & Hogan [Flash et al, 1985] proposed the minimum jerk criterion

J=

1 Tf T &Z&& &Z&&dt 2 ∫0

(14)

to show that the human implicitly plans the point to point movements in the task space based only on the arm’s kinematic model. Where, x is the position vector of the endpoint of human arm. The optimal trajectory with zero boundary velocities and accelerations can be obtained as

Z V = Z  + Z 6 H  − Z  U  −U  + U   ,

(15)

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

where s = t / Tf without considering the arm dynamics. Uno et al. on the other hand proposed to take into account about arm dynamics as a constraint condition when performing optimal motion planning. Based on this idea, the minimum joint torque-change criterion

J=

1 Tf T τ& τ&dt , 2 ∫0

(16)

is presented [Uno, et al, 1989], which implies that human implicitly plans the point to point reaching movements in the human body space based on the arm’s dynamic model. Where, τ is the combined vector of the joint torques. They also expanded this model to a muscle model [Uno, et al, 1989] and proposed minimum muscle force change criterion to show that CNS may generate unique hand trajectory by minimizing a global performance criterion,

J =∫

Tf

0

f& T f& dt , (17)

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Z. Luo / On Comutational Robotics

165

where f is the combined vector of the muscle forces. Kawato et al also presented a cascade neural network model that may possible for the nervous system to solve such a minimizing torque-change problem [Kawato et al, 1987]. As was pointed above, mathematically, optimal control of the dynamic arm movement has to solve a nonlinear 2-point boundary problem. For the nonlinear body dynamics, it is impossible to be solved within real time. Although Kawato’s approach can solve the problem numerically, for any changes in the boundary conditions of the reaching task, it requires enormous computation times to learn the optimal control again, which hence is not suitable from generalization point of view. Optimal formation of environmental constraind motion In the case of the simple point to point human arm movements in free motion space, it seems that the arm tends to minimize the joint torque change (or muscle force change). However, is it the universal criterion that can be applied even to the more complex constraint motions such as opening a door, turning a steering wheel, rotating a coffee mill, et al.? To ask this question, we performed experiments of crank rotation task as shown in Figure 8. In this task, as seen from the arm’s joint space, rotating a crank requires only one degree of freedom force, however we have to define the torques for the two joints of the arm. This is also a force redundant problem.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 8 Experiments of human motion in crank rotation task Using the measured position and force results of the movements, we performed many optimum calculations for many kinds of criterions including minimum jerk, minimum torque change, minimum muscle change, minimum end-effector’s interaction force change as well as our proposed criterion to minimize the combination of endeffector’s interaction force change and muscle force change [Ohta et al, 2004]. From Figure 9, it is clear that the predicted numerical result of contact force vector when using minimum torque-change (or muscle force change) criterion, which was proposed for P.T.P. motion in free motion space, is incorrect here. Instead, the human arm tends to minimize a combination of the change of hand interaction force and muscle force as in Figure 10. Therefore, it strongly suggests that human arm movement has different optimal criterions with respect to different task conditions as well as task requirements. The combined criterion captures well the muscle activity in the constrained multi-joint motions. It covers both the motions in free motion space as well as in constrained motion space since in the free motion space the interaction force at the end-effector is zero, therefore the combined criterion can be reduced to the same one as minimum muscle force change criterion.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

166

Z. Luo / On Comutational Robotics

(a)

(c)

(b)

Figure 9 Comparison of interaction force vector between human hand and crank in

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

experiment (a), and numerical simulations of (b) using minimum muscle force change criterion, (c) combination of the hand interaction force change and muscle force change criterion

c)

d)

Figure 10 Comparison of position responses of human hand (a), EMG (c) in experiment and numerical simulations (b), (d) combination of hand interaction force change and muscle force change criterion

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Z. Luo / On Comutational Robotics

167

Note that, in biomechanical researches of motion analysis, we may also face such redundant problem when we try to calculate the human muscle forces or joint torques from the body motion measurement. By now, most motion analysis system can only perform offline analysis after the motion measurements. We hope with the help of supercomputer, we can not only get simultaneous motion measurement and motion analysis, but also realize visual feedback to show the analyzed body activities to the subject during the motion.

3. Physical Human-Robot interaction

Figure 11 Human-robot interactions in care tasks

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

The second computation problem for the robot comes from the robot’s physical interaction with its environment or objects, such as a care person in Figure 11 considering the contact frictions and collisions between each other. Here , the

continuous dynamics of the robot and object can be formulated as

Mθ&& + h = τ + J T f M o q&& = G T f

(18) (19)

where G is the object holding matrix, J is contact point Jacobian, M and h stand, respectively, for the inertia matrix and velocity dependent forces of hand, M ois the inertia matrix of the object. At each contact point, the relative acceleration can be expressed as the following.

a = G T q&& − Jθ&& + G& T q& − J&θ& .

(20)

And by Coulomb's Law, the contact force lies within or on the boundary of its corresponding friction cone.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

168

Z. Luo / On Comutational Robotics

(μ 2 f

2

N

−f

[

2

T

−f

2

U

f = f NT f TT f UT

)T ≥ 0

(21)

]

T

If we distinguish the relative acceleration and contact force into rolling and sliding components, by considering the contact transitions within them, the following relations have to be satisfied.

( μ 2 R f 2 NR − f 2TR − f 2UR ) ⊥ (a 2UR + a 2TR ) μ 2 R f 2 NR − f 2TR − f 2UR ≥ 0

(22)

a 2UR + a 2TR ≥ 0 where the subscript R denote rolling contact. The equal mark is true only at the point where the rolling is switched into sliding. Practically, Coulomb friction cone can be approximated by a four-sided friction pyramid.

− μ f NR ≤ f TR ≤ μ f NR

(23) (24)

− μ f NR ≤ f UR ≤ μ f NR By introducing nonnegative slack variables, + − sTR = μf NR + fTR ≥ 0 , sTR = μf NR − fTR ≥ 0 + − sUR = μf NR + fUR ≥ 0 , sUR = μf NR − fUR ≥ 0 + − aTR = max(0, aTR ) , aTR = max(0, − aTR ) , + − aUR = max(0, aUR ) , aTR = max(0, − aUR ) ,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

(23) can be formulated as the following linear complementarily condition.

⎡a NS ⎤ ⎢ ⎥ ⎢a NR ⎥ ⎢a + ⎥ TR 0≤⎢ − ⎥⊥ ⎢aTR ⎥ ⎢ + ⎥ ⎢aUR ⎥ ⎢ − ⎥ ⎣aUR ⎦

⎡ f NS ⎤ ⎢ ⎥ ⎢ f NR ⎥ ⎢s + ⎥ ⎢ TR ⎥ ≥ 0 − ⎥ ⎢ sTR ⎢ + ⎥ ⎢ sUR ⎥ ⎢ − ⎥ ⎣ sUR ⎦

On the other hand, let

(25)

[

x = θ& T θ T q& T q T

]

T

,

u=M

−1

(τ − h ),

The dynamics (18), (19) subject to constraints (20), (25) is represented as a dynamic complementary system subject to linear complementary condition.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Z. Luo / On Comutational Robotics

x(t + 1) = Ax(t ) + B1u (t ) + B2 f s Bas as = Aps x(t ) + B ps f s − J p u (t ) 0 ≤ as ⊥ f s ≥ 0

169

(26) (27) (28)

The complementary condition (28) is equivalent to

{ a si ≥ 0, f si = 0} or {a si = 0, f si ≥ 0 }

(29)

By associating the Boolean expressions to a binary variable:

{ asi ≥ 0, f si = 0} ↔ {δ si = 0} {asi = 0, f si ≥ 0 } ↔ {δ si = 1}

(30)

the condition (30) can be equivalently expressed by the two mixed-logic linear inequalities:

0 ≤ f si ≤ diag ( max( f si ) ) δ si 0 ≤ a si ≤ diag ( max(a si ) ) (1 − δ si )

(31) (32)

Then the dynamical complementary system (26)㨪(28) can be translated into the following MLD model:

x (t + 1) = Ax (t ) + B1u (t ) + B2 f s Bas as = Aps x (t ) + B ps f s − J p u (t )

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

0 ≤ f s ≤ M fs δ s 0 ≤ a s ≤ M as (1 − δ s )

(33) (34) (35) (36)

Note that above MLD model is a combinatorial model for the hybrid plant, expresses both the continuous dynamics of each mode and the discrete configurations corresponding to active contact mode. Traditional robot control studies separated the optimal motion planning from real time feedback control. This approach doesn’t applicable for hybrid system, since it is not possible to do motion planning for the multiple manipulation tasks. Towards the realization of flexible and dexterous full body manipulation, the control while planning is more practical and attractive. In detail, predictive control method was used to solve the optimal control problem of MLD via mixed integer quadric programming MIQP algorithm. The optimal sequence of both continuous and logical variables were found simultaneously, which correspond to the optimal motion to realize dexterous manipulation. Hybrid system control represents a highly challenging research area. However, because of the existence of constraints and the combinatorial

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

170

Z. Luo / On Comutational Robotics

nature of hybrid systems, only optimal control approaches were proposed until now, based on heuristic rules inferred from practical plant operation. And only numerical solutions are pursued. Even though these, straightforward application of available frameworks faces the limitation of computational complexity which is expected to be solves via supercomputer in the future 4. Cognitive Motion

EnvironmentዘCared Person Cognition1

Intention

Action᧭

Cognition2

Value

Action 2

Cognition3 Sensing

᧪ ᧪

Evaluation Optimization

Modeling

᧪ ᧪

Action

Action m

Cognition n Data Structurize

Action 3

Data, Concepts

Models

Prediction

Imitation

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Robot Figure 12 A robot’s cognitive motion framework Finally, we simply review the computational problem in an autonomous robot’s cognitive motion in unknown environment. It is well known in artificial intelligence, there exist a famous frame problem, which expressing a dynamical domain in logic without explicitly specifying which conditions are not affected by an action. In order to realize environmental adaptive motion, a general framework is presented as in Figure 12, where the sensing from the left side are input to limited blocks 1 to n of cognition to recognize the geometric as well as dynamic structure and model of the environment/ required tasks. The cognition then leads to the aware of the robot’s intention, followed by adjusting its own value so as to process the next step of optimization. Based on the planning as well as internal model simulation, at the right side, the robot output its action by combining also limited actions of 1 to m to the environment. Though, Figure 12 gives the conceptual framework of a robot’s cognitive motion function, detailed mathematical formulation as well as analysis is under discussed. Also, this problem requires many computations from sensing to action in a parallel way.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Z. Luo / On Comutational Robotics

171

5. Conclusions Flexibility, diversity as well as environmental adaptability are highly expected for the next generation of robotics. For which, study of computational robotics becomes a more and more important approach. There are still many other problems in robotics such as intelligence, SLAM and multiagent system that may also be remarkably improved by computational approach. In addition, supercomputer based computation can lead to the novel interactive robot design and evaluation using 3D visualization. Acknowledgement: This research is partially supported by JSPS, MEXT (Grant-in-Aid for Scientific Research (A) No.2124009.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

References [1] Abend W., Bizzi E. and Morasso P., “Human arm trajectory formation”, Brain, vol.105, (1982), 331-348. [2] Baillieul J., “Kinematic programming alternatives for redundant manipulators”, Proceedings of IEEE Int. Conf. on Robotics and Automation, (1985), 722-728. [3] Bellman R., and Kalaba R., “Dynamic Programming and Feedback Control”, Proc. of 1st IFAC, (1960), 460-464. [4] Bemporad A., and Morari M., “Control of systems integrating logic, dynamics, and constraints”, Automatica, vol.35, no.3, (1999), 407-427. [5] Bernstein N., “The coordination and regulation of movements”, Pergamon Press, London, (1967). [6] Boltyanski V.G., Gamkrelidge R.V., Mishchenko E.F., and Pontryagin L.S., “The Maximum principle in the Theory of Optimal Processes of Control”, Proc. of 1st IFAC, (1960), 454-459. [7] Chiacchio P., Chiaverini S., Sciavicco L. and Siciliano B., “Closed-loop inverse kinematic schemes for constrained redundant manipulators with task space augmentation and task priority strategy”, Int. J of Robotics Research, vol.10, (1991), 410-425. [8] Flash T. and Hogan N., “The coordination of arm movements”, J. Neuroscience, vol.5, (1985),1688-1703. [9] Guez A. and Ahmad Z., “Solution to the inverse kinematics problem in robotics by neural networks”, Proceedings of IEEE Int. Joint Conf. on Neural Networks, vol.2, (1988), 617-624. [10] Kawato M., Furukawa K. and Suzuki R., “A hierarchical neural-network model for control and learning of voluntary movement”, Biological Cybernetics, vol.57, (1987), 169-185. [11] Klein CA. and Huang CH., “Review of pseudo-inverse control for use with kinematically redundant manipulators”, IEEE Trans. on on Systems, Man & Cybernetics, vol.13 (1983), 245-250. [12] Kuperstein M., “Neural model of adaptive hand-eye coordination for single postures”, Science, vol.239, (1988), 1308-1311. [13] Lee S. and Kil RM., “Redundant arm kinematic control with recurrent loop”, Neural Networks vol.7, (1994), 643-659. [14] Luo Z.W. and Ito M., “Diffusion-based learning theory for organizing visuo-motor coordination,” Biological Cybernetics, vol.79, (1998), 279-289. [15] Luo Z.W., Ando H., Hosoe S., Watanabe K. and Kato A., “Spatial Generalization of Optimal Control for Robot Manipulators”, Journal of Robotics and Mechatronics, vol.12, no.5, (2001), 533-539. [16] Morasso P., "Spatial control of arm movements", Exp. Brain Res., vol. 42, (1981), 223-227. [17] Mussa-Ivaldi F.A., and Gister S.F, “Vector field approximation: a computational paradigm for motor control and learning”, Biological Cybernetics, vol.67,(1992), 491-500. [18] Ohta K., Svinin M.M., Luo Z.W., Hosoe S., and R.Laboissie`re, “Optimal Trajectory Formation of Constrained Human Arm Reaching Movements”, Biological Cybernetics, vol.91, no.1, (2004), 23-36. [19] Uno Y., Kawato M. and Suzuki R., “Formation and control of optimal trajectory in human multi-Joint arm movement minimum torque change model”, Biological Cybernetics, vol.61, (1989), 89-101. [20] Uno Y., Suzuki R. and Kawato M., “Minimum muscle tension change model which reproduces human arm movement”, Proceedings of the 4th Symposium on Biological and Physiological Engineering, (1989), 299-302, (in Japanese). [21] Yin Y., Luo Z.W., Svinin M., and Hosoe S., “Hybrid Control of Multi-fingered Robot Hand for Dexterous Manipulation”, 2003 IEEE Conf. on Systems, Man & Cybernetics, (2003), 3639-3644.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

172

Simulation and Modeling Related to Computational Science and Robotics Technology F. Kojima et al. (Eds.) IOS Press, 2012 © 2012 The authors and IOS Press. All rights reserved. doi:10.3233/978-1-61499-092-5-172

Teleoperation of Universal Robot Hand with Pinching Force Stabilization Futoshi KOBAYASHI a,1 , Hiroyuki NAKAMOTO a Fumio KOJIMA a Tadashi MAEDA b Nobuaki IMAMURA c and Hidenori SHIRASAWA d a Dept. of Systems Science, Kobe Univ., Japan b Maeda Precision Manufacturing Limited Kobe, Japan c Dept. of Mechanics and Robotics, Hiroshima International Univ., Japan d The Advanced Materials Processing Institute Kinki Japan Abstract. The multi-fingered robot hand has much attention in various fields. Many robot hands have been proposed so far. We have also developed a small and fivefingered robot hand in order to carry out various tasks and the robot hand teleoperation system in order to utilize the human help. However, the operator cannot pinch and manipulate an object with complicated shapes stably because the operator cannot feel the pinching force when the robot hand pinches and manipulates the object in the teleoperation system. In this paper, we propose a pinching force stabilization method in the robot hand teleoperation system. Here, the operator does not teleoperate all robot fingers directly, but a part of fingers are autonomously controlled in order to stabilize the pinching force. The effectiveness of the proposed pinching force stabilization method are verified through some pinching experiments with various shapes.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Keywords. Robot Hand, Teleoperation, Pinching Force

1. Introduction Various humanoid robot hands have been developed so far. The Utah/MIT dexterous hand has four fingers with four joints driven by tendon cables and tactile sensors over the entire surface[1,2]. The Gifu hand has five fingers and 20 joints with 16 degrees of freedom (DOF)[3], and the KH hand type S has five fingers and 20 joints with 15 DOF[4]. More recently manufactured, robot hands incorporate multi-axis/force torque sensors and tactile sensors with conductive ink and are relatively lightweight. The TWENDYONE hand has four fingers and 16 joints with 13 DOF[5]. This robot hand is equipped with the six-axis force sensors and array-type tactile sensors. Honda Motor Co., Ltd. has developed a multi-fingered robot hand, which has five fingers and 20 joints with 13 DOF[6]. Each DOF is hydraulically actuated, and the robot hand has tactile sensors on the entire surface. AIST also developed a multi-fingered robot hand with 4 fingers and 17 joints with 13 DOF that are actuated by an electrical servomotor[7]. The AIST robot hand also employed multi-axis force/torque sensors in the fingertips. Many other robot 1 Corresponding Author: Futoshi Kobayashi, Dept. of Systems Science, Kobe University, 1-1 Rokkodai-cho, Nada-ku, Kobe 657-8501, JAPAN; E-mail: [email protected].

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

173

hands have been developed and researched[8,9]. We have also reported the universal robot hands I[10] and II[11]. A teleoperation system allows intuitive manipulation of a robot hand. Many teleoperation system for the robot hand have been developed so far[12,13]. Teleoperation of the robot hand requires a haptic device and many haptic devices have been developed. Most of these haptic devices are the body-grounded type, which are worn and mounted on a human operator[14,15]. However, the body-grounded type present only that contact force around the flexion/extension axis of the finger. Meanwhile, the grounded type of haptic devices can present three-dimensional contact force to the human operator[16,17,18]. However, this type is limited because it is fixed to a static object like a table or a floor. While a teleoperation is an excellent control method for the robot hand, it is not the most efficient technique for all operations. When the robot hand pinches and manipulates an object with complicated shapes, it is difficult for the human operator to pinch and manipulate the object stably through the robot hand teleoperation system. Moreover, workload of the human operator can be significantly increased by controlling the robot hand remotely. This paper proposes an automatic control method in the robot hand teleoperation. In this method, the robot fingers are controlled partly by the operator directly and partly by a automatic control method automatically. In direct control by the human operator, the human operator wears a motion capture date glove CyberGlove, which can measure joint angles of the human fingers. The measured joint angles are used for controlling the robot fingers. The references for controlling the robot hand are calculated by compensating the joint angles of the human fingers, because there is a difference in the structure of the robot hand and the human hands. Here, coefficients of compensating the joint angles are determined by the genetic algorithm(GA)[19,20]. In automatic control by the pinching force stabilization method, the robot hand is controlled according to contact force measured by multi-axis force/torque sensors. In this method, contact forces are maintained stably even if the human operator lose grasp on an object. Here, control commands for the robot hand are calculated by the GA with the model of the robot hand in realtime. The remainder of the paper is arranged as follows. The robot hand teleoperation system is introduced in Section 2. The autonomous control in robot hand teleoperation is described in Section 3. The experiments showing the performance of imitating the human hand posture are presented in Section 4.1 and the results of pinching experiments are shown in Section 4.1. Finally, the summary of this study is described in Section 5.

2. Robot Hand Teleoperation System 2.1. Universal Robot Hand II The overview and the DOF arrangement of the universal robot hand II are shown in Figure 1. The height from the bottom of the palm to the top of the middle finger is 290 mm, and the length from the thumb to the little finger is 416 mm when the hand is opened. The robot hand has 16 DOF. The thumb has 4 DOF (IP joint, MP joint, CM1 joint and CM2 joint). The other fingers have 3 DOF (DIP-PIP joint, MP1 joint and MP2 joint) and the PIP joint and the DIP joint synchronize like a human finger. This robot hand has the tactile sensors on the finger pads, and the multi-axis force/torque sensors in the fingertips as shown in Figure 2. The tactile sensor have 3

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

174

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

DIP

PIP IP MP1 MP2 MP

CM1 CM2

Figure 1. Robot Hand

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Multi-Axis Force/Torque Sensor

Urethane Gel Pressure Sensitive Rubber Electrode Pattern Sheet Array-Type Tactile Sensor Figure 2. Tactile and Force/Torque Sensor

layer structures (the electrode pattern seat, the pressure sensitive rubber, and the urethane gel), and this sensor can measure the pressure distribution by contact[21]. The multi-axis force/torque sensors are made by BL AUTOTEC, LTD. and can measure the force and torque applied to the fingertips.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

175

Robot Hand Sub-System

Universal Robot Hand II

Hand Control PC

Analog Voltage Direction&Speed Motor D/A Converter Driver Multi-Axis Analog Voltage Force/Torque A/D Converter Sensor Pulse Encoder Counter Board DC Motor

Ether Board

Array-type Tactile Sensor

...

FPGA Pressure Data

...

Analog Voltage

...

FPGA

LAN

FPGA Motion Capture Sub-System

CyberGlove

MotionCapture PC M

Ether Board

Posture Data

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 3. Robot Hand Teleoperation System

2.2. Teleoperation System with Universal Robot Hand II The teleoperation system with the universal robot hand II is shown in Figure 3. This system consists of two subsystems, a robot hand subsystem and a motion capture subsystem. The robot hand subsystem is driven on RT-Linux and is connected through local area network. This subsystem acquires force information from the multi-axis force/torque sensors, tactile information measured by array-type tactile sensors, and joint angles of the robot fingers from motor encoders. The control command is sent to the robot hand through motor drivers. The motion capture subsystem is used for operating the universal robot hand remotely. The human operator uses this subsystem and teleoperates the universal robot hand with the CyberGlove which is the well-known motion capture data glove. Here, the joint angles of the human fingers are sent to the robot hand subsystem through the local area network. By using the teleoperation system, the operator can operate the universal robot hand easily.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

176

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

Figure 4. Robot Hand v.s. Human Hand

3. Robot Hand Teleoperation with Autonomous Control

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

3.1. Joint Angle Compensation for Robot Hand The universal robot hand is teleoperated by a human operator with the robot hand teleoperation system. As mentioned before, the human operator controls the universal robot hand by using the CyberGlove. The CyberGlove is the well-known motion capture data glove and can measure the joint angles of human fingers. In the robot hand teleoperation system, the joint angles measured by the CyberGlove are used for controlling the robot fingers. However, if the robot hand is controlled according to the joint angles of human fingers directly, the trajectory of the robot finger is different from that of the human finger. Because there is the difference in the structure of the robot hand and the human hand as shown in Figure 4. In order to eliminate the structure difference, a joint angle compensation for compensating the references of the robot fingers is proposed in this paper. The joint angles of the human finger θij (t) are measured by the CyberGlove. Here, j represents the joint of the human finger i. The fingertip trajectory of the human finger hi (t) is calculated by using a model of human hand. In the joint angle compensation, the reference qij (t) of the joint j for the robot finger i is calculated as follows: qij (t) = cij · θi (t)

(1)

where cij represents the coefficients of compensating the joint j the robot finger i. Here, the coefficients is determined by the genetic algorithm. In the GA, an individual represents the coefficients of joints for all fingers and is evaluated by the fitness function expressed by Eq. (2).

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

177

F F0

Ptarget IP

3

M

Ptip

Z4

X4

2

Y4

L3 X3

MP Z3

1

Y2

Z2

Z0, Z1

L 1z

CM2

XW

L2

X2

CM1

YW

Y3

X1 0

ZW

L 1x

X0

Figure 5. Robot Thumb Model



 ¯ i (ri (t) − r¯i ) hi (t) − h

* f itness = * t  2 ¯ (hi (t) − hi ) (ri (t) − r¯i )2

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

t

(2)

t

where ri (t) represents the fingertip trajectory of the robot finger i which calculated by ¯ i and r¯i represent the average of using the robot finger model determined in advance. h hi (t) and ri (t), respectively. Once the coefficients are determined offline, the robot hand are controlled by using the fixed coefficients. 3.2. Pinching Force Stabilization As mentioned before, the universal robot hand is controlled by an operator with the motion capture data glove. Here, there is no a force feedback device in our teleoperation system. Because the conventional feedback devices have restriction on the robot hand teleoperation. So, it is difficult for the human operator to pinch and manipulate an object with complicated shapes through the robot hand teleoperation. Therefore, this paper proposes a pinching force stabilization for the robot hand teleoperation. The model for the thumb of the universal robot hand II is shown in Figure 5. In this figure, Ptip and F represent the contact point on the fingertip of the robot hand and the force vector on the contact point Ptip , respectively. Here, the contact point Ptip is measured by the tactile sensor, the force vector F is measured by the multi-axis force/torque

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

178

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

sensor, and F0 represents the initial force vector to be maintained. Then, θi represents the angle of the finger joint i. The target position of the fingertip is defined in order to eliminate the external force based on the contact point Ptip . The moving direction vector M is calculated as follows: M = F − F0 .

(3)

Consequently, the target position Ptarget is calculated as follows: Ptarget = Ptip + Kmov · M

(4)

where Kmov represents the gain of displacement. The fingertip position is moved to the target position Ptarget using position control method, the change of the fingertip force is decreased and the force F is controlled to the initial force F0 . The joint angles of the robot finger are calculated to satisfy the determined target position. These joint angles are used as the reference angles for the position control. However, it is difficult to calculate these joint angles from the target position. Therefore, the approximate solution for each joint angle is determined by using the Genetic Algorithm[19,20](GA) in this paper. In the GA, elements of the individual represent the joint angles. In the first step, individuals in the GA are generated randomly. In the second step, the target position Ptarget is calculated by the target position determination with the sensor values from the tactile sensor, the multi-axis force/torque sensor, and encoder. In third step, the fitness of each individual is evaluated as follows:

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

f itness = |Ptarget − PGA | ,

(5)

where PGA represents the fingertip position corresponding to the joint angles of each individual. That is, the fitness value represents the distance between the target position Ptarget to the fingertip position PGA . The individual with the lowest fitness value is used as the reference joint angles. Finally, the individuals of the next generation in the GA are generated with the genetic operators, such as the selection, the one point crossover, and the mutation. Considering the fact that the moving direction vector M is tiny, it is possible that the optimized solution in the next step is included with present individuals. Thus, the ranking selection is use as the selection operator. As mentioned above, the operator cannot feel the pinching force in the robot hand and cannot manipulate an object through the robot hand stably. Therefore, the pinching force stabilization is used for manipulating the object stably. Here, in the pinching task, the index finger of the robot hand is controlled directly by the operator and the thumb of the robot hand is controlled by the pinching force stabilization. By using the proposed method, the external force in the pinching task is cancelled by updating the target fingertip position.

4. Experiments 4.1. Experiment with Joint Angle Compensation Two experiments are implemented in order to show the effectiveness of the joint angle compensation. In the first experiment, the fingertip trajectories of human and robot finger

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

Table 1. Correlation Coefficient of Thumb

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Table 2. Correlation Coefficient of Index Finger

Thumb

Before

After

Index

Before

After

Xaxis

0.481709

0.867665

Xaxis

0.762605

0.788424

Yaxis

0.817539

0.882520

Yaxis

0.636498

0.931108

Zaxis

0.963414

0.982551

Zaxis

0.963760

0.992317

Figure 6. Trajectory of Thumb

179

Figure 7. Trajectory of Index Finger

Figure 8. Posture of Robot Finger without Joint Angle Compensation

Figure 9. Posture of Robot Finger with Joint Angle Compensation

are verified by using the robot hand model. Figures 6 and 7 shows the trajectories of the thumb and the index finger, respectively. In these figures, the upper figure shows the trajectory of the human finger, the middle figure shows that of the robot finger without Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

180

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

(a) Quadratic Prism

(b) Triangular Prism

Figure 10. Manipulated Object in Experiment

the trajectory correction, and the lower figure shows that of the robot finger with the proposed trajectory correction. Then, the red line, the blue dotted line, and the black broken line represent the trajectories on x, y, and z axes, respectively. Table 1 and 2 show the correlation efficient between the trajectory of the human and the robot hand. From these results, the robot hand can be teleoperated as imitating the movement of the human finger. Especially, the trajectory of the thumb is improved by the proposed trajectory correction. Thus, the operator can teleoperate the robot hand by the robot hand teleoperation system. The postures of the robot hand and the human hand are compared in the second experiment with the universal robot hand II. The posture of the fingers without the trajectory correction and that with the trajectory correction are shown in Figure 8 and 9, respectively. Form these figures, the robot hand can posture like the human hand by using the proposed trajectory correction.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

4.2. Teleoperation Experiments with Pinching Force Stabilization For showing the effectiveness of the pinching force stabilization, a pinching experiment with some objects is implemented. In this experiment, 2 types of objects, such as the quadratic prism and the triangular prism, are used as shown in Figure 10. The quadratic prism can be easily pinched and manipulated by the robot hand because that has the parallel surfaces, but the triangular prism cannot be easily pinched and manipulated because that does not has the parallel surfaces. Firstly, the operator pinches and manipulates the quadratic prism through the robot hand. Figure 11 shows the transition images in the first experiment. In the first experiment, the operator can pinch the quadratic prism as shown in the upper figure of Figure 11. Then, the operator manipulates the object as changing the posture of the object. The fingertip forces in the first experiment are shown in Figure 12. In this figure, the figures (a) and (b) show the fingertip force in the first experiment when the robot hand uses the pinching force stabilization or not, respectively. From these figures, the operator can pinches and manipulates the quadratic prism stably with the robot hand teleoperation system. Secondly, the operator pinches and manipulates the triangular prism through the robot hand. Figure 11 shows the transition images in the second experiment. In the second experiment, the operator can pinch the triangular prism and manipulates the object

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

181





         









 

 

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 11. Pinching and Manipulation of Quadratic Prism

















 

    



(a) with Pinching Force Stabilization

 

         





 

             

(b) without Pinching Force Stabilization

Figure 12. Fingertip Force for Quadratic Prism

as changing the posture of the object. The fingertip forces for the triangular prism are shown in Figure 12. As the same way, the figures (a) and (b) show the fingertip force in the second experiment when the robot hand uses the pinching force stabilization or not, respectively. From these figures, the operator can pinches and manipulates the triangular

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

182

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization





         









 

 

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 13. Pinching and Manipulation of Triangular Prism























        

(a) with Pinching Force Stabilization

         











        

(b) without Pinching Force Stabilization

Figure 14. Fingertip Force for Triangular Prism

prism stably. Consequently, in the developed robot hand teleoperation system, the operator can teleoperate the universal robot hand by using the pinching force stabilization. Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

183

5. Summary This paper proposed an autonomous control method for a robot hand teleoperation. Here, the robot hand is controlled partly by the human operator and partly automatically. In the direct control by the human operator, a joint angle compensation is proposed in order that the trajectories of the robot finger and the human finger become similar. Here, joint angles of the human fingers which compensated by the joint angle compensation are used as the references for controlling the robot fingers. In the joint angle compensation, coefficients for compensating the joint angle are determined by the genetic algorithm. In the autonomous control, a pinching force stabilization is proposed in order that the robot hand can stably pinch and manipulate objects with complicated shape. Here, in the pinching and manipulating task, the index finger of the robot hand is controlled by the human operator directly and the thumb of the robot hand is controlled by the pinching force stabilization. The proposed robot hand teleoperation with the joint angle compensation and the the pinching force stabilization was applied to some experiments with various shapes, the effectiveness of the proposed method were verified. The developed robot hand teleoperation system does not have a force feedback device for presenting force to a human operator. In the future, we will develop a robot hand teleoperation system with a force feedback device for intuitive manipulation.

References [1] [2]

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

[3] [4] [5] [6]

[7]

[8] [9] [10]

[11]

J.M. Hollerbach and S.C. Jacobsen, “Anthropomorphic Robots and Human Interactions”, Proc. of the 1st International Symposium on Humanoid Robots, 1996, pp 83-91. D. Johnston, P. Zhang, J. Hollerbach, and S. Jacobsen, “A Full Tactile Sensing Suite for Dexterous Robot Hands and Use in Contact Force Control”, Proc. of the 1996 IEEE International Conference on Robotics and Automation, 1996, pp 661-666. T. Mouri, H. Kawasaki, K. Yoshikawa, J. Takai, and S. Ito, “Anthropomorphic Robot Hand: Gifu Hand III”, Proc of International Conference on Control, Automation and Systems, 2002, pp 1288-1293. H. Kawasaki, T. Mouri, and S. Ito, “Toward Next Stage of Kinetic Humanoid Hand”, CD-ROM of World Automation Congress, 2004, pp 129-134. H. Iwata, and S. Sugano, “Design of Anthropomorphic Dexterous Hand with Passive Joints and Sensitive Soft Skins”, Proc. of the 2009 IEEE International Symposium on System Integration, 2009, pp 129-134. K. Takahashi, H. Waita, M. Kokusho, and M. Hayakawa, “Development of Dexterous and Powerful Multi-fingered Hand for Humanoid Robots”, Proc. of the 27th Annual Conference of the Robotic Society of Japan, 2009, AC1A1-01 (in Japanese). K. Kaneko, K. Harada, and F. Kanehiro, “Development of Multi-fingered Hand for Life-size Humanoid Robots”, Proc. of the 2009 IEEE International Conference on Robotics and Automation, 2009, pp 913920. S. Schulz, C. Pylatiuk, and G. Bretthauer, “A New Ultralight Anthropomorphic Hand”, Proc. of the 2001 IEEE International Conference on Robotics and Automation, vol. 3, 2001, pp 2437-2441. J. Jockusch, J. Walter, and H. Ritter, “A Tactile Sensor System for a Three Fingered Robot Manipulator”, Proc. of the 1997 IEEE International Conference on Robotics and Automation, 1997, pp 2080- 3086. H. Nakamoto, F. Kobayashi, N. Imamura, and H. Shirasawa, “Universal Robot Hand Equipped with Tactile and Joint Torque Sensors (Development and Experiments on Stiffness Control and Object Recognition)“, Proc. of the 10th World Multi-Conference on Systemics, Cybernetics and Informatics, Vol. 2, 2006, pp 347-352. W. Fukui, F. Kobayashi, F. Kojima, H. Nakamoto, T. Maeda, N. Imamura, K. Sasabe, and H. Shirasawa, “Development of Multi-Fingered Universal Robot Hand with Torque Limiter Mechanism”, Proc. of 35th Annual Conference of the IEEE Industrial Electronics Society, 2009, pp 2225-2230.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

184

F. Kobayashi et al. / Teleoperation of Universal Robot Hand with Pinching Force Stabilization

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

[12] M.A. Diftler, C.J. Culbert, R.O. Ambrose, R. Platt, Jr., W.J. Bluethmann, “Evolution of the NASA/DARPA Robonaut Control System”, Proc. of IEEE International Conference on Robotics and Automation, 2003, pp 2543-2548. [13] T. Mouri and H. Kawasaki, “A Novel Anthropomorphic Robot Hand and its Master Slave System”, Humanoid Robots, Human-like Machines, Matthias Hackel (Ed.), InTech, 2007. [14] M. Bouzit, G. Burdea, G. Popescu, and R. Boian, “The Rutgers Master II — New Design ForceFeedback Glove”, IEEE/ASME Transactions on Mechatronics, Vol.7, No. 2, 2002, pp. 256-263. [15] F. Kobayashi, G. Ikai, W. Fukui, and F. Kojima, “Two-Fingered Haptic Device for Robot Hand Teleoperation”, Journal of Robotics, Vol. 2011, Article ID 419465, 8 pages, 2011. doi:10.1155/2011/419465. [16] M. Monroy, M. Oyarzabal, M. Ferre, A. Campos, and J. Barrio, “MasterFinger: Multi-finger Haptic Interface for Collaborative Environments”, Proc. of EuroHaptics, 2008, pp. 411-419. [17] Y. Saitou, H. Nakamoto, F. Kobayashi, F. Kojima, N. Imamura, K. Sasabe, and H. Shirasawa, “Haptic Feedback in Universal Robot Hand Tele-Operation”, Proc. of Joint 4the International Conference on Soft Computing and Intelligent Systems and 9th International Symposium on Advanced Intelligent Systems, 2008, pp 1123-1128. [18] M. Mishima, H. Kawasaki, T. Mouri, and T. Endo, “Haptic Teleoperation of Humanoid Robot Hand Using Three Dimensional Force Feedback”, Proc. of 9th International Symposium on Robot Control, 2009, pp 565-570. [19] J.H. Holland, “Adaptation in Natural and Artificial Systems”, University of Michigan Press, 1975. [20] D.E. Goldberg, “Genetic Algorithm in Search, Optimization, and Machine Learning”, Addison Wesley, 1989. [21] H. Nakamoto, F. Kobayashi, and F. Kojima, “Shape Classification using Tactile Information in Rotation Manipulation by Universal Robot Hand”, Robotics 2010: Current and Future Challenges, 2010, pp 123-132.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Simulation and Modeling Related to Computational Science and Robotics Technology F. Kojima et al. (Eds.) IOS Press, 2012 © 2012 The authors and IOS Press. All rights reserved. doi:10.3233/978-1-61499-092-5-185

185

Object Manipulation based on Tactile Information of Multi-Fingered Robot Hand Wataru FUKUI a,1 , Futoshi KOBAYASHI b Hiroyuki NAKAMOTO b and Fumio KOJIMA b a Dept. of Computer Science and Systems Engineering, Graduate School of Engineering, Kobe University b Dept. of Systems Science, Graduate School of System Informatics, University Abstract. Many multi-fingered robot hands have been developed so far. However, there are no multi-fingered robot hands for actual use. One of the reasons is that the conventional control methods are not used for multi-fingered robot hands. A human manipulates an object by using tactile information (contact points, direction and severity of reaction force). In this paper, the tactile based control method for multi-fingered robot hands is proposed like a human manipulation. In the proposed method, 3 fingers are used for the object manipulation. By using this method, object grasping and manipulating is achieved. The proposed method is implemented in Universal Robot Hand II, and the effectiveness is verified.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Keywords. Multi-fingered robot hand, Manipulation, Tactile information

Introduction Expectations for automation by robots are raised in healthcare field, industrial field and so on. However the robots in actual use have only grippers or specialized tools, and are specialized to repeat just only a single task. These robots are not for the multiple uses. If the end-effectors are converted into multi-fingered robot hands, these robots can manipulate the existing tools and handle the multiple tasks. Additionally the initial cost can be cut down by using the existing tools. The automatic hand changer is also discussed, however waste of time in the changing is not negligible and the multi-fingered robot hands get a lot of attention. Many multi-fingered robot hands are researched and developed in research institutes and companies[1,2,3,4]. In the case of the grasping using multi-fingered robot hand, the target object must be grasped by more than 3 points[5]. If the target object is grasped by these 3 points by using position control method, the grasping points are fixed but the robot hand maybe drop the object down or break the object down. If by using force control method, the grasping points and the object position are not fixed and it is hard to manipulate it. Thus 1 1-1

Rokkodai-cho, Nada-ku, Kobe 657-8501, Japan; E-mail: [email protected]

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

186 W. Fukui et al. / Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand

both the position and the force must be controlled for the grasping using multi-fingered robot hands. As the control method for the position and the force, there are the stiffness control method[6], the hybrid control method[6] and the some improved hybrid control methods[7,8,9]. The strict position control cannot be achieved by the stiffness control method. The modelization is necessary for the hybrid control methods. However it is hard to modelize the grasping condition using the multi-fingered robot hand. These methods are not for multi-fingered robot hands and not for the object manipulation by using the multi-fingered robot hands. In addition to these, Tahara et al. have achieved the manipulation by using 3 fingered robot hand[10]. However the grasping force is not considered and it is necessary to analyze the target object and verify it. Thus it is necessary for multi-fingered robot hands to control by using a new control method. In the case of the manipulation to write "1" by a human, the pen is grasped by 4 points; the fingertip of the thumb, the fingertip of the index finger, the third dactylus of the index finger and the fingertip of the middle finger. At these points, these fingers perceive the contact points and the contact force. From the grasping posture, the index finger and the middle finger are moved for the nib position to follow the target trajectory. The thumb is moved to follow the behavior of the pen to keep the grasping force constant. This grasping is assumed a passive force closure[11] in the micro time. In the manipulation using the robot hand, it is reasonable that the index finger and the middle finger are controlled by using the position control method and the thumb is controlled by using the force control method like the human manipulation. After that, the contact points of the index finger and the middle finger are moved in steps of minutely small doses. In this paper, the method to grasp and manipulate an object by using the tactile information of the multi-fingered robot hand is proposed. In the proposed method, the index finger and middle finger are controlled by using the position control method, these fingers control the position of the grasped object and these fingers are named "master fingers." The thumb is controlled by using the force control method, the finger controls the grasping object and the finger is named "slave finger." The grasping force is gradual changed by the master fingers behavior and the slave finger stabilizes the grasping force passively by using the tactile information. The force control method in the slave finger is not the conventional force control method but a nonconventional force control method based on the position control method. By using the force control method, the behavior both as the force control and as the position control is achieved on the one control scheme. In other words, both the position control needed in the phase to approach the target object and the force control needed in the phase to manipulate it are treated on a control scheme. In the proposed method adopting the force control method, it is not necessary to analysis or model the grasping condition.

1. Universal Robot Hand System An overview image of the Universal Robot Hand II[4] and the arrangement of DOF are shown in Figure 1. "MP1 joint and MP2 joint" and "CM1 joint and CM2 joint" are twisted 90 times and arranged in the series as ball joints, "MP joint" and "CM joint." The PIP joint and the DIP joint synchronize like a human finger. The thumb has 4 DOF (IP joint, MP joint, CM1 joint and CM2 joint), the other fingers have 3 DOF (DIP-PIP joint, MP1 joint and MP2 joint) and Universal Robot Hand II has 16 DOF.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

W. Fukui et al. / Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand 187

DIP

PIP IP MP1 MP2 MP

CM1 CM2

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 1. Universal Robot Hand II

Figure 2. Tactile Sensor on Finger and Force/Torque Sensor in Fingertip

Universal Robot Hand II has Multi-Axis Force / Torque Sensor in fingertips and array-type tactile sensors on the finger pads (Figure 2). Multi-Axis Force / Torque Sensor is made by BL AUTOTEC, LTD. These sensors can measure the force and torque applied to the fingertips. The tactile sensors can measure the pressure distribution by contacts[12], and the contact coordinate in the fingertip frame is measured by using the pressure distribution (Figure 3). These force data and pressure distribution are defined as the tactile information in this paper. The control system constructed for Universal Robot Hand II is shown in Figure 4. This system is controlled by using Vine Linux and RT Linux.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

188 W. Fukui et al. / Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 3. Tactile Sensor and Fingertip Frame

Figure 4. Control System for Universal Robot Hand II

2. Object Manipulation Method 2.1. Position and Force Control Method for Multi-fingers The pen grasping by human hand is classified into a force passive force closure in the micro time. In this assumption, the index finger and the middle finger touch the pen body with the fixed contacts for determining the pen tip position, and the thumb controls the grasping force. This force passive force closure is applied to the multi-fingered robot hand. In this case, the index finger and the middle finger are controlled by using the position control method and the thumb is controlled by using the force control method.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

W. Fukui et al. / Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand 189

P1

P4 r P2

P3

z0 Σ0

y0

x0

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 5. Pen Grasping Model

By using this proposed method, the pen tip position and the grasping force are controlled more simply and more accurate than the case using the force control method or the stiffness control method. Additionally, the force control method applied in the proposed manipulation method is the force control method based on the position control. By using the force control method, the force or the position are controlled in one control scheme. The pen grasping model is shown by Figure 5. The contact points 0 Pi (i = 1, 2, 3, 4) in the absolute coordinate system Σ0 are calculated by the contact points tip Pi (i = 1, 2, 3, 4) in the fingertip coordinate system Σtip tip measured by using the tactile sensors, the joint angles measured by using the angle sensors and the designed link lengths. The object behavior is analyzed by using the Lagrange equation of motion generally. However the relative attitude between the pen coordinate system and the fingertip coordinate system is not decided. Thus, the transformation matrix is not decided, the motion equation is not formulated. In this paper, the pen attitude is analyzed geometrically. The line of the pen’s central axis in Σ0 is assumed as x + ay + bz − ω = 0.

(1)

The distance r from contact points 0 Pi (i = 1, 2, 3, 4) to the line (eq.1) is calculated from the following equation. r=

|xi + ayi + bzi − ω| √ (i = 0, 1, 2, 3). 1 + a2 + b2

(2)

The eq.2 has 4 unknown numbers (a, b, ω, r). However, 4 equations are developed with each contact point. Thus, these unknown numbers are specified. In the process of solving the system of equations, the absolute value sign must be solved. The solution space can be selected in the space where the pen exists. The space is specified roughly by using the force sensors or tactile sensors. As above, in the case of the pen with the unknown radius r is grasped by the 4 measured points, the pen central axis is decided. In the Figure 5, if the contact points of the master fingers (the index finger and the middle finger) are moved, the pen central axis is decided at any time.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

190 W. Fukui et al. / Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand

A

B -

F0

M

Kmov

+

P target

+

+

F

P

C

Pisition Control

Inverse θ target Servo Kinematics System Sover

τ

by Tactile Sensor

by Multi-Axis Force Sensor Universal Robot Hand II Figure 6. Block Diagram of Proposed Method

2.2. Force Control Method based on Position Control In the proposed manipulation method, the index finger and the middle finger are controlled as the master fingers by using the position control method, and the thumb is controlled as the slave finger by using the force control method. The conventional position control method is used for the master fingers. The force control method based on the position control method[13] is used for the slave finger. The control flow graph of the force control method is shown by the Figure 6.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

2.3. Application for grasping and manipulating The decision part of the target position (Figure 6 A) and the target angles (Figure 6 B) for the slave finger are implemented on Linux. The part of the servo system (Figure 6 C) is implemented on RT Linux. The part of the servo system for the master fingers are implemented on RT Linux as similar as the slave finger. This force control method is based on the position control method. If the target in Figure 6 is changed and the finger posture is changed, the robot hand can pre-postured and approach a target object by using the same control scheme. The robot hand cannot approach the target object by using the conventional force control method. The advantage is that the adopted force control method can be applied to both the approach phase and the manipulate phase. 3. Experiments 3.1. Simplified Pen Manipulation using 2 Fingers In this simplified experiment, it is shown that the pen object is moved and the grasping force is kept the constant. The experimental setup is shown by Figure 7. The pen object is fixed by using the rotating shaft by MP1 joint of the index finger. The pen is grasped and controlled by using the index finger and the thumb. The thumb (the slave finger) is controlled by using the force control method based on the position control method, and the index finger (the master finger) is controlled by using the PID position control method. The master finger is controlled at will and applies the displacement to the grasped object. The experimental procedure is shown by Figure 8. From the graspong posture (Figure 8 (1)), the index finger moved down (Figure 8 (2)-(3)). In this procedure, the thumb Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

W. Fukui et al. / Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand 191

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 7. Simplified Pen Manipulation

Figure 8. Finger and Object Postures in Experiment 3.1

is moved down and keep the grasping force constant. After that, the index finger moved up (Figure 8 (4)-(5)) In this procedure, the thumb is moved up and keep the grasping force constant. The graph of the experimental data is shown by Figure 9. In this experiment, the posture of the Universal Robot Hand II is changed for the pengrasping by using the position control and the pen is passed by the experimenter. The robot hand is controlled to keep the initial fingertip force. The initial fingertip force (the gray dot line) is (-8.70[N], -0.18[N], 3.10[N]). The fingertip force (the black line) around 19 [sec.] and around 24 [sec.] in tne X axis are differ from the initial fingertip force. The pen moved suddenly by the slip or scratch and the fingertip force is changed suddenly. The fingertip force is converged on the initial fingertip force after the diremption. The fingertip force can be

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

192 W. Fukui et al. / Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand

Fitness

150 100 50 0 -50 -100 -150 -200 150 100 50 0 -50 -100 -150 -200 100000 80000 60000 40000 20000 0

-0.18[N]

3.09[N]

0

5

10

15

20

25

30

Force [N]

0 -0.2 -0.4 -0.6 -0.8 -1.0 -1.2 -1.4 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 1.0 0.8 0.6 0.4 0.2 0 -0.2 -0.4

-8.70 [N]

Force [N]

300 250 200 150 100 50 0 -50 -100

Force [N]

Position [mm]

Position [mm]

Position [mm]

Fingertip Position (Index Finger) Fingertip Force (Thumb/ with control) Fingertip Force (Thumb / without control) Initial Fingertip Force

35

Time [sec.]

Figure 9. Experimental Result in Experiment 3.1

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Table 1. Fingertip Force in 2-Fingeered Manipulation Experiment X

Y

Z

Average [N]

-8.73

-0.17

3.04

Average [N]

1.32

0.18

0.53

kept around the initial fingertip force compared with the fixed thumb (the black dot line). In this result, the change of the fingertip force in the Y axis is smaller than the force in the other axis. The master finger is hard to move to the Y axis and the change of the force is also small. The average and the variance of the fingertip force are shown in Table 1. The fingertip force is kept around the initial fingertip force as shown by Table 1. As above, the robot hand can move the pen and keep the grasping force by using the proposed method. It is confirmed that the object can be controlled by using 2 fingers. 3.2. Pen Manipulation using 3 Fingers In this experiment, the proposed method is implemented on the thumb, the index finger and the middle finger of the Universal Robot Hand II. Experimental setup is shown by Figure 10. First, the robot hand is adopted a pre-shape by using the position control. Second, the robot hand makes to grasp a pen object by the experimenter. After that, the

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

W. Fukui et al. / Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand 193

Position Control

Fulcrum

Force Control

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 10. Pen Manipulation

(1)

(2)

(3)

(4)

(5)

(6)

Figure 11. Finger and Object Postures in Experiment 3.2

robot hand moves the index finger and the middle finger by using the position control and manipulates the grasped object. These fingertip positions and the fingertip force (thumb) are measured during the experiment. The process of the experiment is shown by Figure 11. The initial posture is shown in Figure 11 (1). The robot hand keeps the grasping force in this posture. The behavior of the holding down is shown in Figure 11 (1)-(3) and the behavior of the holding up is shown in Figure 11 (4)-(6). The graph of the experimental data is shown by Figure 12. The fingertip positions(the index finger / the green line, the middle finger / the blue line) in the X-axis and Z-axis are changed and the object is manipulated. On the other hand, the fingertip position (the thumb / the red line) is hardly changed, because of that the grasping position is near by the rotating center of the object (the third dactylus of the index finger).

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

194 W. Fukui et al. / Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand

X Y Thumb Index Finger Middle Finger Thumb

140 120 100 80 60 40 20 0

0.5 0 -0.5

-0.106[N]

Force [N]

1.0

X-Axis

-1.0 1.0

Y-Axis

0.5 0 -0.5

Force [N]

200 180 160 140 120 100 80 60 0 -20 -40 -60 -80 -100 -120 -140

: Fingertip Position : Fingertip Position : Fingertip Position : Fingertip Force

-0.070[N] -1.0 0

Z-Axis

-0.5 -0.245[N]

-1.0

Force [N]

Position [mm]

Position [mm]

Position [mm]

Z

-1.5 0

10

20

30

40

50

60

-2.0

Time [sec.]

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 12. Experimental Result in Experiment 3.2

The grasping force (the orange line) is kept around the target grasping force (the black dot-line). The grasping force in the Z-axis is changed the more than the others because of the stick-slip vibration between the urethan gel of the tactile sensor and the grasping object. As above, the robot hand can manipulate the pen object keeping the grasping force constant.

4. Summary In this paper, the object manipulation method has been proposed. This method is based on the tactile information (the contact position and the contact force) of the multi-fingered robot hand. In this proposed method, the grasping condition is assumed as the passive force closure in a micro time. The index finger and the middle finger are controlled by using the position control and control the object position. The thumb is controlled by using the force control and control the grasping force. By using this method, the object manipulation is achieved without analysis. In this paper, the fingertip position is controlled and the object is moved without the consideration of the object posture and position. For the future work, the trajectory of the working point such as the pen tip is controlled.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

W. Fukui et al. / Object Manipulation Based on Tactile Information of Multi-Fingered Robot Hand 195

References [1]

[2]

[3]

[4]

[5] [6] [7]

[8]

[9] [10]

[11] [12]

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

[13]

Kenji Kaneko, Kensuke Harada, Fumio Kanehiro: Development of Multi-fingered Hand for Life-size Humanoid Robots, Proc. of the 2009 IEEE International Conference on Robotics and Automation, (2009), 913-920. Tetsuya Mouri, Haruhisa Kawasaki, Keisuke Yoshikawa, Jun Takai, Satoshi Ito: Anthropomorphic Robot Hand: Gifu HandIII, Proc. of the 2002 IEEE International Conference on Control, Automation and Systems, (2002), 1288-1293. H. Liu, K. Wu, P. Meusel, N. Seitz, G. Hirzinger, M.H. Jin, Y.W. Liu, S.W.Fan, T. Lan Z.P.Che: Multisensory Five-Finger Dexterous Hand: The DLR/HIT Hand II, Proc. of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, (2008), 3692-3697. Wataru Fukui, Futoshi Kobayashi, Fumio Kojima, Hiroyuki Nakamoto, Tadashi Maeda, Nobuaki Imamura, Kazuhiko Sasabe, Hidenori Shirasawa: Development of Multi-Fingered Universal Robot Hand with Torque Limiter Mechanism, Proc. of the 2009 IEEE Industrial Electronics Society, (2009), 22252230. B. Mishra, J. T. Schwartz, M. Sharir: On the Existence and Synthesis of Multifinger Positive Grips, Algorithmica, (1987),541-558. M. H. Raiberd, J. J. Craig: Hybrid position/force control of manipulators, ASME Journal of Dynamic System, Measurement, and Control, 102, (1981), 126-133. Feng-Yih Hsu, Li-Chen Fu: Adaptive Fuzzy Hybrid Force/Position Control for Robot Manipulators Following Contours ofan Uncertain Object, Proc. of the 1996 IEEE lntemational Conference on Robotics and Automation, (1996), 2232-2237. Gilles Duchemin, Pierre Maillet, Philippe Poignet, Etienne Dombre, Francois Pierrot: A Hybrid Position/Force ControlApproach for Identification of Deformation Models of Skin and Underlying Tissues, Proc. of the 2005 IEEE Transaction on Biomedical Engineering, (2005), 160-170. Bin Yao, S. P. Chan, Danwei Wang: Unified Formulation of Variable Structure Control Schemes for Robot Manipulators, Proc. of the 2002 IEEE Transactions on Automatic Control, (2002), 371-376. Kenji Tahara, Suguru Arimoto, Morio Yoshida: Dynamic object manipulation using a virtual frame by a triple soft-fingered robotic hand, Proc. of the 2010 IEEE International Conference on Robotics and Automation, (2010), 4322-4327. Tsuneo Yoshikawa: Foundations of Grasping and Manipulation 1. Passive Closure and Active Closure, Journal of the Robotics Society of Japan Vol. 13 No. 7 (in Japanese), (1995), 950-957. Hiroyuki Nakamoto, Futoshi Kobayashi, Nobuaki Imamura, Hidenori Shirasawa, Fumio Kojima: Shape Classification inRotation Manipulation by Universal Robot Hand, Proc. of the 2008 IEEE/RSJ International Conference on Intelligent Robotsand Systems, (2008), 53-58. Wataru Fukui, Futoshi Kobayashi, Fumio Kojima, Hiroyuki Nakamoto, Tadashi Maeda, Nobuaki Imamura, Kazuhiro Sasabe, Hidenori Shirasawa: Fingertip Force and Position Control Using Force Sensor and Tactile Sensor for Universal Robot Hand II, Proc. of the 2011 IEEE Workshop on Robotic Intelligence in Informationally Structured Space, (2011), pp.43-48.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

196

Simulation and Modeling Related to Computational Science and Robotics Technology F. Kojima et al. (Eds.) IOS Press, 2012 © 2012 The authors and IOS Press. All rights reserved. doi:10.3233/978-1-61499-092-5-196

Displacement and Force Measurement, Vibration Detection by Magnetic Type Tactile Sensor a

Hiroyuki NAKAMOTO a,1 , and Satoru TAKENAWA b Graduate School of System Informatics, Kobe University, JAPAN b Kobe City College of Technology, JAPAN

Abstract. We propose a tactile sensor that uses variation of magnetic field. The sensor has a simple structure of two layers, an elastic layer and a substrate layer. The elastic layer is made of an elastic material and houses a cylindrical permanent magnet inside. When an object touches the surface of the elastic layer, the layer deforms. The deformation displaces the magnet and the magnetic field for the substrate layer changes. The substrate layer is a glass epoxy board, and has four giant magneto resistance (GMR) elements and four inductors. These elements detect changes of magnetic field strength from the magnet. It is possible to calculate three-axis displacement and force and detect slips from outputs of the elements. In this paper, we describe a structure of the sensor. Then, we confirm a fundamental performance of the sensor and show response character of the elements.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Keywords. Tactile sensor, Magnetic field, Three-axis force, Vibration

Introduction The human has a variety of senses and always uses them. In the senses, senses of vision and hearing are produced as vision sensors and microphones, respectively. They are embedded into video cameras, mobile phones, personal computers, and are used with displays and speakers every day. A tactile sense has an essential sense in our daily lives, and it seems a combination of artificial skin and tactile display contributes to our lives, too. Then, it is important to develop them that have sufficient functions, durability, and low cost. In this paper, we focused on developing such a tactile sensor. There are some commercially available tactile or force sensors. Pressure Profile Systems, Inc. produces a distributed pressure measurement system named TactArray [1]. This sensor has a thin structure, and can apply to a curved surface. Because of the thinness, the sensor cannot touch an object softly like human skin. Some 6-axis force/torque sensors that built into finger tips have been used for measurements of grasping force vector and dexterous manipulations [3]. These sensors have been miniaturized. However, they are too expensive to be distributed as the robot skin. 1 Corresponding Author: Hiroyuki NAKAMOTO, Assistant Professor, Kobe University, 1-1 Rokkodai-cho, Nada-ku, Kobe 657-8501, JAPAN; E-mail: [email protected]

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

197

There are various three-axis tactile sensors in some studies. Kobayashi et al. developed a prototype three-axis tactile sensor using a two-dimensional array of silicon structures with fixed strain gages [4]. It is difficult and expensive to manufacture a tactile sensor that has an array of strain gages. Ohka proposed a method by which to measure a force vector by utilizing the variation of refractive index in a fingertip-shaped optical waveguide [5]. Application of the optical-based tactile sensor to a robot finger is easy, but the sensor system requires an imaging device. Shinoda et al. proposed a tactile sensing element based on an ultrasonic cavity and verified the ability to detect the friction coefficient by measuring the resonant frequency of the cavity experimentally [6]. In their approach, the sensor enables only the coefficient of friction to be detected at the moment of contact. Slips are detectable by a piezoelectric device. Son et al. developed a tactile sensor with strips of piezoelectric film to detect incipient slips and occurrences of contact [7]. However, it is difficult to measure a force vector by the sensor. Jockusch et al. prototyped a tactile sensor using a piezoresistive and a piezoelectric device, which are stacked in layers, to perform simultaneous measurement of contact force and position, and slippage detection [8]. This hybrid sensor makes peripherals cumbersome and complicated. Cranny et al. have developed a piezoelectric thick-film sensor which is able to sense force and slip [9]. This sensor only measures force in a single direction via deflection of fingertip. Similarly, Wettles et al. have developed a sensor that measures force tri-axially via electrode impedance [10]. However, they just suggested various possibilities of the biomimetic tactile sensor. One of the earliest robot tactile sensors using magnetic transduction was suggested by Sato, Heginbotham and Pugh [11]. This sensor consists of a matrix of probes and measures the displacement of a probe by a sense coil. The structure is complex. As just described, tactile sensors that have sufficient functions, durability and low cost are being studying. To overcome these difficulties, we propose a magnetic type tactile sensor that can simultaneously measure a three-axis force. We had previously developed another magnetic type tactile sensor using a two-dimensional array of chip inductors [12]. Each inductor of the sensor detects a magnetic flux density while a deformation occurred. Thus, we were able to calculate a three-axis force from the inductors’ output voltages. However, this sensor was not able to calculate the force with sufficient accuracy, and it is necessary to improve it. We thus now propose a magnetic type tactile sensor that consists of giant magneto resistance (GMR) elements. The sensor is able to calculate a three-axis displacement and a three-axis force from output voltages of the GMR elements. The main features are reduced wiring, simple structure, and low cost.

1. Tactile Sensor using Magnetic Elements 1.1. Human Skin Human has soft skin over the body, and the skin has a lot of receptors. Tactile receptors in human skin are shown in Figure 1. Human skin has four receptors, Meissner’s corpuscle, Merkel cell, Ruffini ending and Pacinian corpuscle. These receptors are different structures. Therefore, they have different responses for a stimulus. Meissner’s corpuscle and Pacinian corpuscle are classified as rapidly adapting receptors which respond only to the onset and offset of mechanical deflection. On the contrary, Merkel cell and Ruffini end-

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

198

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

ing are classified as slowly adapting. It is important for tactile sense to have both rapidly and slowly adapting. We consider that human obtains a tactile perception by using two different receptors. Then, we propose a tactile sensor which has two different sensing elements in the next section. Meissner’s corpuscle

Epidermides

Merkel cell Ruffini ending

Dermis

Pacinian corpuscle Subcutaneous tissue

Figure 1. Receptors of Human Skin

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

1.2. Structure and Principle The proposed magnetic tactile sensor is mainly composed of two layers (Figure 2), i.e., an elastic layer and a substrate layer. The elastic layer is made of an elastic material, e.g., silicon or urethane rubber and houses a cylindrical permanent magnet inside. The substrate layer is made of a glass epoxy board; its surface side is flat and contains no electronic elements. Four GMR elements and four inductors are on the back side. The elastic layer is fixed to the substrate layer surface with an adhesive bond. Because these layers are not hard-wired, no wire breakages occur. The price of the sensor can be onetenth of that of three-axis force sensor [3]. If the elastic layer becomes worn after a long-term usage, it can be easily replaced with a new one. When the tactile sensor touches an object, the elastic layer surface is displaced. The inside magnet is also displaced depending on the degree to which the layer’s is deformed. This magnet displacement changes the magnetic flux density to the GMR elements on the substrate layer, and thus the outputs of elements are changed. The sensor calculates a three-axis displacement and a three-axis reaction force of the layer surface from the outputs. Thus, the GMR elements are used as slowly adapting receptors. An inductor generates an induced electromotive force depending on the variation of magnetic flux density. A rapid variation of magnetic flux density generates a significant induced electromotive force. This means that the inductors generate higher outputs when the elastic layer surface deforms rapidly. One example of high-speed deformation on the elastic layer surface is the stick-slip phenomenon in which sticks and slips repeat alternately. Thus, since inductors generate high outputs at the moment a slip occurs, they can be used to detect slips occurring on the elastic layer surface. The induced electromotive force are a rapid response, and the inductors are used as rapidly adapting receptors.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

199

1.3. Sensor Design We arranged positions of GMR elements to improve the signal-to-noise ratio of outputs. The electromagnetic field simulation software; Maxwel 2D, Ansoft products, estimates the magnetic field strength and the magnetic flux density distribution. Fig. 3(a) shows a simulation result. The magnet is a cylindrical neodymium with 6 mm in diameter and 1 mm in thickness and is placed at the distance of 10mm from the substrate layer. The substrate layer is a glass epoxy board with 1.6 mm in thickness. Similarly, Fig. 3(b) shows a simulation result. The magnet is 3 mm in diameter and 1 mm in thickness and the distance of 5mm from the substrate layer. In these figures, curve lines indicate the magnetic flux density distribution. Let us suppose that the GMR element has the singleaxis sensitivity in parallel to the plane of substrate layer, the ideal element position is parallel to the magnetic field line. Then, the element output has a high sensitivity for the magnet displacement. From Fig. 3(a), the position 10-15 mm away from just under of the magnet is suitable to mount the GMR element in case of the magnet with 6 mm in diameter. Similarly, from Fig. 3(b), the position 5-10 mm away from just under of the magnet is suitable to mount the GMR element in case of the magnet with 3 mm in diameter. Moreover, it is important that the magnetic field strength is within the range of the sensitivity of GMR element. Based on these simulation results, we have designed positions of the GMR elements on the substrate layer. Fig. 4(a) shows a layout drawing of the prototype φ20 sensor. The elastic layer is 20 mm in diameter, which houses the magnet with 6 mm in diameter and 1 mm in thickness. Fig. 4(b) shows a layout drawing of the prototype φ10 sensor. The elastic layer is 10 mm in diameter, which houses the magnet with 3 mm in diameter and 1 mm in thickness. Moreover, we have designed a bigger one. The elastic layer is 45 mm in diameter, which houses the magnet with 10 mm in diameter and 1.5 mm in thickness.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

1.4. Displacement and Force Calculation Three-axis displacement and force which act on the elastic layer surface are calculated from output voltages of four GMR elements. We set vi as the output of the GMR element, and x as the x-axis displacement of the sensor, where i(i = 1, ..., 4) is the number of GMR element. The GMR output is inversely proportional to the distance from the magnet. External force

Elastic layer

{

Object

Magnet Inductor Magnetoresistance element

}

Substrate layer

Figure 2. Sensor Structure

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

200

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

Magnet 6 1 mm

Magnet 3 1 mm Substrate

Substrate

5 mm

5 mm

(a) φ6×1 Magnet

(b) φ3×1 Magnet

Figure 3. Electromagnetic Simulation Results

The x-axis displacement follows the proportion to the square of the distance based on the geometric relationship. Then the x-axis displacement is proportional to the inverse square of the GMR output. We add terms of inverse of the GMR output, and define an equation. Δx, as the displacement of x axis direction, follows the inverse proportion to the square of vi : Δx = Cdx1 +

Cdx2 Cdx3 Cdx4 Cdx5 Cdx6 Cdx7 Cdx8 Cdx9 + + 2 + + 2 + + 2 + (1) v12 v1 v2 v2 v3 v3 v4 v4

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

where Cdxj (j = 1, ..., 9) is a coefficient of each term, and it is determined by multiple regression analysis. Furthermore, deformation of the elastic layer postulates a linear elasticity, the force of displacement for the elastic layer becomes, Fx = Cf x Δx

(2)

where Cf x is a constant of spring which depends on the material of an elastic layer. From Eqs. (1) and (2), set Cf xj = Cf x Cdxj and we can write. Fx = Cf x1 +

Cf x2 Cf x3 Cf x4 Cf x5 Cf x6 Cf x7 Cf x8 Cf x9 + + 2 + + 2 + + 2 + .(3) v12 v1 v2 v2 v3 v3 v4 v4

Displacement of y-axis direction and z-axis direction are Δy and Δz respectively, and components of a force vector of y-axis direction and z-axis direction are Fy and Fz respectively, which are calculated in a similar manner. Therefore, from Eq. 1 and Eq. 3, a three-axis displacement Δx, Δy, Δz and a three-axis force Fx , Fy , Fz follow ⎛ ⎞ Δx ⎜ Δy ⎟ ⎟ ⎜ ⎜ Δz ⎟ ⎜ ⎟ ⎜ Fx ⎟ = CV ⎜ ⎟ ⎝ Fy ⎠ Fz

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

Elastic layer position

201

Elastic layer position

20

20 15

20

4 8.5

10 5 Inductor

GMR

Inductor GMR Connector

Connector (a) φ20 sensor

(b) φ10 sensor

Elastic layer position 22.5

Inductor

GMR 10

Connector

100

100 mm

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

(c) φ45 sensor Figure 4. Sensor Layouts

⎞ 1 2⎟ Cdx1 Cdx2 · · · Cdx9 ⎜ ⎜ 1/v1 ⎟ ⎜ Cdy1 Cdy2 · · · Cdy9 ⎟ ⎜ 1/v1 ⎟ ⎜ ⎟⎜ ⎟ =⎜ . .. . . .. ⎟ ⎜ .. ⎟ . ⎟ ⎜ ⎝ .. ⎠ . . . ⎜ . ⎟ 2⎠ ⎝ Cf z1 Cf z2 · · · Cf z9 1/v4 1/v4 ⎛

⎞

⎛

(4) We applied to the solution Eq. (4) polynominal approximation using values of threedimensional reference points which are measured previously to decision of C; coefficient of each term. This procedure requires calibration values, which are the outputs of the GMR elements, three-axis displacements and forces of grid reference points Pl (l = 1, ..., L). In this case, the number of grid reference points is 77 (L=77).

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

202

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

Figure 5. Three Prototypes

1.5. Slip Detection The faster a variation of magnetic flux density is, the higher an induced electromotive force generated by an inductor becomes. At the moment a stick-slip phenomenon occurs, a deformed elastic layer rapidly is returned to a normal shape. Simultaneously, a highinduced electromotive force is generated. This electromotive force occurring during a stick-slip phenomenon is detected by three sigma limits. If the current output voltage vnow is in the following range, there is no possibility of a slip occurring. Conversely, if the voltage is outside this range, slips may occur.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

μ − 3σ ≤ vnow ≤ μ + 3σ where μ is the moving average of voltages and σ is the standard deviation. Because the tactile sensor has four inductors, the possibility of a slip is judged when all four inductors’ outputs vnow are outside three sigma limits. 2. Experiments 2.1. Prototype and Experimental Setup Figure 5 shows a prototype φ20 sensor, φ10 sensor, and φ45 sensor. The upper white parts are the elastic layers, and the substrate layer is under it. Each substrate is 1.6 mm in thickness. The GMR elements used for the prototypes are AA003-02 ( NVE Co.) and the chip inductors are ELJFB102JF (Panasonic Co.). Output voltages of GMR elements were amplified thirty-twofold by a differential amplifier and those of inductors were amplified ten-thousand-fold by a noninverting amplifier. Each output voltage was measured by a PC via an A/D converter board with a period of 1 ms. The experimental setup is shown in Fig. 6. Vertical and horizontal displacements were applied to the sensor surface by means of an aluminum part fixed to a three-axis

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

203

Figure 6. Experimental Setup

force sensor (BL Mini Sensor, B.L. Autotech LTD.) and the three-axis moving stage. This force sensor is used to precisely monitor the applied force and is calibrated so as to have a 0.06 N resolution in the -40 to 40 N range. The force sensor outputs are sampled by a PC via an A/D converter board with a period of 1 ms. Before the following experiments, we set 77 reference points within ± directions of the xy-axis and the - direction of the z-axis. The coordinate origin is at the center point of the elastic layer surface.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

2.2. Displacement and Force Experiments In the experiments, at first the prototype sensor was pressed in the vertical displacement and was moved in a circle on the horizontal plane by the three-axis moving stage. The vertical displacement is 1, 2, 3 mm and the diameter of the horizontal circle is 4 mm for the φ20 sensor and the φ45 sensor. Because the displacement range of the φ10 sensor is smaller than that of the φ20 sensor, for the φ10 sensor, the vertical displacement is 1, 2 mm and the diameter of the horizontal circle is 2 mm. Fig. 7 shows the results of the three-axis displacement and force by φ45 sensor. The left graph charts show the three-axis displacements, and the right charts show the threeaxis forces. The displacement and force results on xy-plane and xz-plane are shown in these graph charts. In each graph chart, the lower axis, right axis and left axis indicate x-axis, y-axis and z-axis, respectively. We see from Fig. 7(a), Fig. 7(c) and Fig. 7(e) that the z-axis displacements are calculated accurately. However, as the z-axis displacement becomes large, the y-axis displacement has a large margin of error. As an example, the sensor is pressed by 3 mm, and about 0.2 mm of error arises. Because of the large displacement for the sensor, the inside magnet moved a nonlinear relationship with the layer displacement. In Fig. 7(b), Fig. 7(d), and Fig. 7(f), there are a margin of error between the prototype sensors and the three-axis force sensor. These errors are about 5 % and can be offset easily.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

3

xy 2

2

0

-1

-1

-2

-2 xz

-3

1

z [mm]

0

y [ N]

y [mm]

xy - reference

xy - calculated

1

-3

-4

-4

-5 -3 -2 -1 0 1 x [mm]

-5

2

3

0

0

z [ N]

204

-2 -4

-1

-2 -2

-1

-6

xz - calculated

xz - reference 0

1

-8 2

-10

x [ N] (b) Force(z=-1mm)

(a) Displacement(z=-1mm)

3 xy

2

2

-1

-2

-2

-3

-3

xz

-4

-2

2

3

-2 -4

-1

-4 -1 0 1 x [mm]

0

-6

-5

-2 -2

-1

3

0

1

2

-10

x [ N] (d) Force(z=-2mm)

xy 2

2

xy - reference

xy - calculated

1

-1

-2

-2

-3

-3 xz

-4 -3

-2

-4 -1 0 1 x [mm]

2

3

-5

0

0

z [ N]

-1

1

z [mm]

0

y [ N]

0

-5

-8

xz - calculated

xz - reference

(c) Displacement(z=-2mm)

y [mm]

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

-3

0

z [ N]

-1

1

z [mm]

0

y [ N]

y [mm]

0

-5

xy - reference

xy - calculated

1

-2 -4

-1

-2 -2

-1

0

1

x [ N] (f) Force(z=-3mm)

(e) Displacement(z=-3mm)

-6

xz - calculated

xz - reference

Figure 7. φ45 sensor Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

-8 2

-10

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

205

Second, Fig. 8 shows the results of the φ20 sensor. A glance at Fig. 7(a), Fig. 7(c), and Fig. 7(e) will reveal that the calculated value of the x, y, z-axis direction of this sensor is high accuracy in the range within 3.0 mm of the z-axis direction, and do not have the error in the y-axis direction as the φ45 sensor. In the results of force vector, this sensor has a greater tendency to be smaller than the reference values by the three-axis force sensor like the φ45 sensor. On the other hand, the z-axis forces of this sensor has high accuracy, and margins of error are smaller than those of the φ45 sensor. Fig. 9 shows the results of the φ10 sensor. A glance at Fig. 9(c) will reveal the displacement has a margin of error on the xy-plane. In the force results of the xy-plane, this sensor has a tendency to be smaller than the reference values by three-axis force sensor. In contrast, the z-axis displacements and forces are calculated accurately. From Fig. 9(b) and Fig. 9(d), because the resolution capability of the three-axis force sensor which was used in these experiments is within 0.06 N in the z-axis direction, output values of the force sensor are oscillatory. However, the φ10 sensor forces are not oscillatory but regular in shape. Therefore, it is assumed that the φ10 sensor can output the three-axis forces with higher resolution than the reference force sensor.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

2.3. Slip Experiments Figure 10 shows the slip detection results of the φ45 and the φ20 sensors. In the experiments, an acrylic block loaded each sensor with a force of 2 N and slid horizontally three times. A laser displacement meter measured displacement of the block. The horizontal axis shows time steps of 1 ms. The left vertical axis shows the output voltages of the four inductors and those of the displacement meter, and the right vertical axis shows slip detections. In Fig. 10(a), the inductors’ output voltages become large or small at the three sliding moments when the laser displacement meter outputs changed voltages. Therefore, the φ45 sensor detected the three slips. Similarly, Fig. 10(b) indicates the φ20 sensor detected slips. In our algorithm, slips are detected only when all four inductors’ outputs are over three sigma limits. Naturally, there were the results that only outputs of an inductor are over the limits, in that case, the sensor don’t detect slips. This means that the sensor can detect slips precisely and is robust against partial slips. Repeated experiments showed that it had a 90 % detection rate. The three prototype sensor, incidentally, have no difference regarding detection rate. 2.4. Response to Vibration To confirm the response characters of a GMR element and an inductor, we had an experiment which produces a vibration to the sensor. Figure. 11 shows the experimental setup. The setup fixes the sensor to the stage. The stage has the vibration table touch the surface of the sensor. A vibrational frequency is decided by a function generator. An acceleration sensor which is fixed on the vibration table measures a vibrational acceleration. We measure outputs of the tactile sensor and vibrational acceleration in 1 kHz, and calculate vibration amplitude from a vibrational acceleration. Figure 12 shows a result under the condition that the vibration frequency is changed from 1 to 500 Hz. The right and left axes show voltage amplitudes of a GMR element and an inductor on the tactile sensor, and vibration amplitude by the vibrator, respectively. The bottom axis shows the vibration frequency. The amplitude of GMR is constant from

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

206

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

3

xy

2

2

-1

-1 xz

-2

-2 -3

-4

-4

-5

-5

-3

-2

-1 0 1 x [mm]

2

3

0

0 -2 -4

-1

-8 -2

-2

-1

xy - calculated

-10

-1

-1

-2

-2 xz

-4

-5

-5

-3

1

-3

-4 -2

-1 0 1 x [mm]

2

3

0

0 -2 -4

-1

-2

-2

-1

-6

xz - reference

xz - cal culated

0

1

-8 2

-10

x [ N] (d) Force(z=-2mm)

(c) Displacement(z=-2mm)

3

xy - reference

z [ N]

0

z [mm]

0

y [ N]

xy

2

2

1

xy - reference

xy - calculated

-1

-1

-2

-2

-3

-3 xz -3

1

y [ N]

0

z [mm]

0

0

0 -2 -4

-1

-6

-4 -2

-1 0 1 x [mm]

2

3

-5

-2

xz - calculated -2

-1

0

1

x [ N] (f) Force(z=-3mm)

(e) Displacement(z=-3mm)

-8

xz - reference

Figure 8. φ20 sensor Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

2

-10

z [ N]

y [mm]

2

2

-3

y [mm]

1

xy

1

-5

0

x [ N] (b) Force(z=-1mm)

2

-4

-6

xz - calculated

xz - reference

(a) Displacement(z=-1mm)

3

z [N ]

0

z [mm]

0

-3

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

xy - reference

xy - calculated 1

y [ N]

y [mm]

1

207

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

1.5

xy

1 0. 4

0.5

xy - calculated

xy - reference

- 0 .5

-1 -1.5

-1 xz

0. 2

-0.5

-1.5

-2

-0.2

-2

-2.5 - 1 . 5 -1 - 0 . 5 0

0.5

1

0

0

xz - reference

xz - calculated

-1.5

-0.4

-2.5 1.5

-0.4

x [mm]

-0.2

0

0. 2

0. 4

-2.5

x [ N]

(a) Displacement(z=-1mm)

1.5

z [ N]

-0.5

z [mm]

0 y [ N]

y [mm]

0

(b) Force(z=-1mm)

xy

1 0. 4

-0.5

-1 -1.5

-1 -1.5

xz

-2

0. 2

0

-0.5 -0.2

-2

-2.5 - 1 . 5 -1 - 0 . 5 0

0.5

1

-2.5 1.5

0

xz - reference

xz - calculated

-1.5

-0.4 -0.4

x [mm]

-0.2

0

0. 2

0. 4

-2.5

x [ N]

(c) Displacement(z=-2mm) Copyright © 2012. IOS Press, Incorporated. All rights reserved.

z [ N]

-0.5

xy - reference

xy - calculated

0

z [mm]

0

y [ N]

y [mm]

0.5

(d) Force(z=-2mm) Figure 9. φ10 sensor

1 to 10 Hz, and the vibration amplitude is also constant. Thus, the results reflect the character that the GMR outputs are proportional to the vibration amplitude. On the contrary, the amplitude of the inductor rises until 20 Hz. The reason is a character of the inductor that has high-sensitivity to a rapid change of amplitude. In case of using the experimental setup, the higher the vibration becomes, the smaller the amplitude becomes. Then, the amplitude of the inductor drops under the condition of above 20 Hz because the vibration amplitude and the variation of magnetic density become small. Under the condition of 300 Hz, the GMR has almost no change. However, the outputs of the inductor that have amplitude of 0.06 V are synchronized with the input vibration. Therefore, the inductor is more sensitive to vibration than the GMR. The result showes that the tactile sensor has two kinds sensing elements of different charactors. Using these elements, the tactile sensor has possibilities for sensing complex touch conditions that happen on the surface of the sensor.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

208

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

4

Inductor 1

3

Inductor 2

2

Inductor 3

1

Inductor 4

Slippage detection

Voltage [V]

5

0

Slippage detection -1 0 500

Displacement 1000 1500 Time [ms] (a) φ45 sensor

2000

Slip Not Slip

4 3.5 Inductor 3 Inductor 2

2.5

Slippage detection

Voltage [V]

3

Inductor 4

2

Inductor 1

1.5 1

Displacement Slip

0.5 Slippage detection 0 0

200

400 600 Time [ms]

800

Not Slip 1000

(b) φ20 sensor

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Figure 10. Response of stick-slip phenomena

2-axis stage (fixed) Tactile sensor Acceleration sensor Vibration table Vibration direction Vibration actuator

Figure 11. Setup for Vibration Experiment

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection 2

Inductor Magnetoresistance element Oscillation amplitude

4 3 2 1

1

0

Amplitude[mm]

Voltage [V]

1.5

209

0.5

0 1

10 Frequency [Hz]

100

500

Figure 12. Responses of Elements to Vibration

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

3. Conclusion and Future Works We introduced the magnetic type tactile sensor using four GMR elements and inductors. Its main features are reduced wiring, simple structure and low cost. It provides a main function, i.e., measuring the three-axis displacement and the three-axis force from the output voltages of four GMRs. The displacement and force were accurate values through the fundamental experiment. Additionally, the sensor detected stick-slip phenomena from inductors’ output voltages. We also vibrated the sensor surface at several frequency conditions, and confirmed that two elements in the sensor had different characters to vibrations. In this paper, we confirmed two elements on the sensor separately. Essentially, it is necessary to fuse the outputs of the elements as a tactile sense. We devise a new method to fuse the outputs at the next step. Many variations of our sensor can be created by changing its elastic layer hardness and shape. If the elastic layer is made of very soft materials, our sensor can measure reaction forces of some soft objects like a pudding. Furthermore, if we made the elastic layer of a hard material, the sensor is used as a force sensor at one-tenth price. However, we have to confirm the lifetime of the sensor. Then there is a problem that outer magnetic noises affect outputs of the sensor. Studying a method to cancelling the outer noises, we will apply our sensor to a variety of application, measuring equipment, robots, and so on.

References [1] [2] [3] [4]

Pressure Profile Systems, Inc.: TactArray distributed pressure measurement sysmtem, http://www.pressureprofile.com/products-tactarray. M.R. Tremblay and M.R. Cutkosky: Estimating friction using incipient slip sensing during a manipulation task, Proc. IEEE Int. Conf. on Robotics and Automation 1 (1993), 429–434. BL AUTOTEC, LTD.: FORCE TORQUE SENSOR, https://www.bl-autotec.co.jp/english/index.html. M. Kobayashi and S. Sagisawa: Three direction sensing silicon tactile sensors, IEICE Trans. Electron. J74-C-II(5) (1991), 427–433.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

210 [5] [6] [7] [8] [9] [10] [11]

M. Ohka, I. Higashioka and Y. Mitsuya: A micro optical three-axis tactile sensor (validation of sensing principle using model), Advances Information Storage Systems 10 (1999), 313–325. K. Nakamura and H. Shinoda: A tactile sensor instantaneously evaluating friction coefficients, Proc. 11th Int. Conf. on Solid - State Sensors and Actuators 2 (2001), 1430–1433. J.S. Son, E.A. Monteverde and R.D. Howe: A tactile sensor for localizing transient events in manipulation, Proc. IEEE Int. Conf. on Robotics and Automation (1994), 471–476. J. Jockusch, J. Walter and H. Ritter:A tactile sensor system for a three fingered robot manipulator, Proc. IEEE Int. Conf. on Robotics and Automation (1997), 3080–3086. A. Cranny, D. P. J. Cotton, P. H. Chappell, S. P. Beeby and N. M. White: Thick-film force, slip and temperature sensors for a prosthetic hand, Measurement Science and Technology 16 (2005), 931–941. N. Wettels, V. J. Santos, R. S. Johansson and G. E. Loeb: Biomimetic Tactile Sensor Array, Advanced Robotics 22(8) (2008), 829–849. H. R. Nicholls and M. H. Lee: A Survey of Robot Tactile Sensing Technology, The International Journal of Robotics Research 8(3) (1989), 3–30. S. Takenawa: A Magnetic Type Tactile Sensor using a Two-Dimensional Array of Inductors, Proc. IEEE Int. Conf. on Robotics and Automation (2009), 3295–3300.

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

[12]

H. Nakamoto and S. Takenawa / Displacement and Force Measurement, Vibration Detection

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

Applications

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

This page intentionally left blank

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Simulation and Modeling Related to Computational Science and Robotics Technology F. Kojima et al. (Eds.) IOS Press, 2012 © 2012 The authors and IOS Press. All rights reserved. doi:10.3233/978-1-61499-092-5-213

213

Saliency-based Geographics Annotation for Robotic Access to Naturally Complex Scenes Kohji KAMEJIMA 1-79-1 Kitayama, Hirakata 573-0196 Japan Abstract. A saliency-based approach is presented to the transfer of landmark information between robotic vehicles participating in over-thehorizon maneuvering processes. By identifying chromatic diversity of naturally complex scenes with degenerated version of fractal attractors, probe vehicles detect and upload as-is representation of scene specific primaries as annotation of the local geographics. In reference to the annotation, future visitors adaptively restore the saliency pattern to be matched with landmarks in encountered scenes. Through experimental studies, the saliency based geographics annotation was demonstrated to significantly reduce the complexity of noisy background. Keywords. Visual Saliency; Naturally Complex Scene; Chromatic Diversity; Multi-Fractal Articulation

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

1. Introductory Remarks The state-of-the-art space technologies for positioning, communication, and earth observation jointly provide infrastructure for networking machine vision systems. Within such an augmented scope of perception, robotic vehicles can retrieve a sequence of the scene images along the trajectory extended within the bird’s eye view [1]. This implies that we can apply various types of machine intelligence, including geometric path planning [2] and/or programming of autonomous systems [3], to as-is annotation of local geographics in terms of future scenes. Such an geographics annotation scheme [4] provides a logical basis of over-thehorizon maneuvering processes [5]; in the deployment of the multitude of robotic systems towards unstructured environment at the first visit [6], for instance, the geographics of the working space should be interactively annotated in accordance with the progress of missioned programs. To extend human’s capability of robot operation to the annotated geographics, the on-vehicle vision of future visitors are required to match the downloaded features within the imminent scene including unexpected distribution of corrupted objects. To facilitate such an adaptation process, on the other hand, each probe vision should generate transferable representation of notabilia marking the complex scenes to be encountered. In many practical situations, however, it is not easy to specify ‘what-to-be-observed’ prior to the completion of the missioned program. To maintain the integrity of robotic

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

214

K. Kamejima / Saliency-Based Geographics Annotation for Robotic Access

performance in the deployed environment, hence, the focus of features extraction and adaptation processes should be controlled in close cooperation with human’s inherent perception-decision steps. Through the iteration of selection pressure, the participants of the really existing world exhibit ‘chromatically attractive features’ to visualize individual intention [7]. As intentional participants of such a naturally complex world, humans have developed a not-yet-explicated capability to focus the vision towards the sensible features within the surroundings [8]. Such supervenience of inherent perception to naturally complex scenes implies that various types of landmarks should be designed to be acceptable as visual saliency in noisy background [9]. Noticing that the visual saliency can be detected via early processes [10] to jointly support subsequent decision making [11], the focus control process should nondeterministically associate the diversity of saliency distribution with symbolic structure underlying the first visit scene, as well. Following empirical knowledge of ecological optics [12] combined with neural sensitivity to fractal patterns [13], [14], the saliency distribution should be selforganized towards fractal attractors [15] via computational process of essentially finite complexity [16]; by invoking recent advancements in emotional- as well as computational-perception [17], [18], for instance, it has been demonstrated that scale shift in noise pattern can be articulated in terms of fractal codes spanning the expansion of a horizontal plane and an aggregation of boundary objects. Noticing that various types of roadway area can be segmented in terms of fractal attractors transferable between bird’s-eye views and scene images, in this paper, we introduce a new framework for geographics annotation in terms of focusing cue to possible landmarks. The problem is to transfer the control parameter for the localization of landmarks relative to the generic roadway pattern. Noticing the supervenience of the inherent perception to ‘naturally colorful world’, in what follows, the chromatic diversity of scene images is identified with a fractal attractor controlled by a set of scene specific primary. The transferability of the as-is primary is verified through the consistency analysis of associated saliency distribution within the perspective of scene images.

2. Chromatic Complexity Generator (up-link) The perception of chromatic diversity is essentially mental processes; for most human observers only three primaries are required to match a test light [19]. To manipulate the information conveyed by chromatic diversity, the coloring of the reflected light should be represented within the nonnegative subset of 3D Euclid space:  T fωRGB = Rω Gω Bω ,

0 ≤ Rω , Gω , Bω ≤ 1,

where Rω , Gω , Bω are the intensities of three primaries at the pixel ω in the image plane Ω. The color matching process is substantiated by stochastic capturing mechanism of a set of photopigments by neuronal system [20]; by indexing the performance of such a distributed stochastic process in terms of fωRGB , we have a

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

215

K. Kamejima / Saliency-Based Geographics Annotation for Robotic Access

R

γ [φω ]

G

Figure 1. Fractal Sampling of Naturally Complex Scene

B

Figure 2. Chromatic Diversity Representation via Fractal Sampling

precise representation of the information conveyed by the diversity of the incoming light. Define



T φω = fωRGB / fωRGB = φRω φGω φBω ,

φ(·) ω =

(·)ω . |fωRGB |

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

c 2 Since c∈RGB (φω ) = 1, we have a local representation of the probability for 2 capturing photopigments in terms of (φcω ) , c ∈ RGB. Thus, we have a distributed indexing of chromatic diversity for simulating saliency preference by inherent vision. Let the totality of the chromatic information φω be identified with the positive part of a unit sphere where the following density function is induced:   1 |φ − φω |2 . (1) gα (φ|φω ) = exp − 2πα 2α Following experimental studies using various types of roadway scene images including natural objects, the diversity scale α should be adjusted to 1/10 ∼ 1/100. For sufficiently small deviation |φ − φω |, the measure gα (φ|φω ) approximates the Gaussian distribution on tangential space at φω . By using the index gα , we can simulate precise color matching process by inherent vision. The gα -index is extended to the evaluation of the chromatic diversity of a set of color samples collected in a scene image; for the samples { fωRGB }, the diversity of the chromatic information s = { φ (fωRGB ) } is evaluated as follows: , + 2 σφφ 1 , (2) gα (s) = exp − 2πα 2α 2 σφφ =

1 × s(s − 1)



 RGB 

2

φ fi − φ(fjRGB ) ,

1≤i,j≤s i=j

where (·) is the size of the set (·). Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

216

K. Kamejima / Saliency-Based Geographics Annotation for Robotic Access

An example of the chromatic diversity arising in a naturally complex scene is captured on a fractal attractor covering the image plane Ω as illustrated in Fig. 1 where a set of chromatic information s = { φi , i = 1, 2, . . . } with size s = 995 is collected from the entire scene image. The complexity of the chromatic diversity is visualized within the planar color space Γ on which the information φω is mapped via the following projection:   Γ  γ = eRGB φω , eRGB = eR eG eB , (3)   T e(·) = cos θ(·) sin θ(·) , where θR = π/2, θG(B) = θR +(−)2π/3. Figure 2 shows a complex pattern generated by the set s. This figure demonstrates that we can analyze the diversity - chromatic .



of naturally complex scenes through 2D visualization Γs = γ (φω ) φω ∈ s : a connected complex pattern separated from the variation of the brightness level within linear color space Γ. Mathematically, the chromatic diversity conveyed by the samples s is identified with the aggregation of Dirac’s delta measure distributed on Γs . Consider mathematical model for simulating inherent saliency impression within the framework of the locally Gaussian color space. To evaluate the complexity of the chromatic diversity, let the distribution Γs be identified with a degenerated version of fractal attractor Ξs in the color space Γ. Suppose that the attractor is generated via the iterated function system [21]. This implies that the computational structure of Ξs can be restored through self-similarity analysis of the following measure:  χs = δγ(φ) .

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

φ∈s

Let the complexity of the chromatic diversity s be indexed in terms of minimal length of the ‘programs’ to regenerate the distribution Γs via possible selfsimilarity processes within the Γ space. By invoking a multitude of Brownian motion processes to simulate the uncertainty due to such a computational diversity, we have the probability distribution for capturing Ξs in terms of the solution to the following partial differential equation [22]: 1 Δϕρ (γ|s) + ρ [χs − ϕρ (γ|s)] = 0. 2

(4)

In this equation, the complexity factor ρ is adjusted to the essential length of the program for the classification of randomly sampled RGB photopigments, i.e., ρ = log RGB. Let gσ (γ) be 2D Gaussian distribution with zero mean and variance σ, i.e., 2

gσ (γ) =

e−|γ| /2σ . 2πσ

Noticing that the following approximation gσ (γ) − δ ∼

σ Δgσ (γ) , 2

γ ∈ Γ,

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

K. Kamejima / Saliency-Based Geographics Annotation for Robotic Access

217

induces the association ϕρ (γ|s) ∼ g1/σ (γ), we can extend the zero-cross method [23] to the smooth distribution ϕρ (γ|s) to yield the following Laplacian-Gaussian boundary [22]: ∂ g Γs =

-

.

1 γ ∈ Γ ϕρ (γ|s) = max ϕρ (γ|s) . e γ∈Γ

The attractor of the iterated function system is expanded nondeterministically towards the fixed points which should not be located interior nor exterior of the attractor [16]. This implies that we can identify the control parameter of the chromatic diversity, called as-is primary, through the allocation of the fixed point on the Laplacian-Gaussian boundary ∂ g Γs . The identification process is divided into the following three steps. First, a possible fixed point γ˜0f is located on the Laplacian-Gaussian boundary ∂ g Γs and expanded via the following successive scheme: - . ˜ ft ∪ dΓ ˜ ft , ˜ f = γ˜ f , ˜f = Γ (5) Γ Γ t+1 0 0 where the increment is selected by the following nondeterministic algorithm ˜ ft = dΓ

-

 . 

− − ˜ f ∀∂˜ ˜ ft ˜ ft ≥ ← η ∂˜ γ, Γ , γ:← η ∂˜ γf , Γ ∂γ

˜ f , ∂˜ ˜ ft , ∂γ γ ∈ ∂ g Ξs − Γ

− with respect to ← η (γ, Λ) = min |γ − λ|. Next, a subset { γˆk } satisfying the followλ∈Λ

ing conditions is selected as an estimate of the vertices

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

∀m, k :

θmk − θnk < π,



γˆ(·) − γˆk = γˆ(·) − γˆk ej (θ(·)k +θk ) ,

(6)

˜ f , γˆk = |ˆ γˆ(·) ∈ Γ γk | ejθk . ˜ is To simulate the counter-degeneration process, finally, the distribution of Γ expanded along the following repulsive force: dˆ γk =



(ˆ γk − γˆj ) gα (φk |φj ) ,

(7)

ˆ γ ˆj ∈Γ

within the possible coloring circle |ˆ γk | ≤ 1; in this equation, a the repulsive force is induced in the color space to separate the vertices { γ˜k } towards a Γ-space representation of the as-is primaries. The scheme (5) combined with (6) yields a set of fixed points to be associated with a set of contraction mapping for regenerating the distribution Γs . By adding the dynamics (7), we have a design of the iterated function system for the restoration of the attractor Ξs . Thus, we have the fractal model of the chromatic ˆ = {π diversity controlled by the as-is primary Π ˆi } represented in the locally Gaussian color space where

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

218

K. Kamejima / Saliency-Based Geographics Annotation for Robotic Access

γ ˆr

π ˆr

π ˆr

π ˆg π ˆr

γ ˆg

π ˆr

γ ˆb γ ˆgb

π ˆg

π ˆr

π ˆb

π ˆgb

π ˆg

π ˆb π ˆ gb

ˆ Figure 3. As-is Primary Π Figure 4. Complexity Reduction via ψˆω -filtering

π ˆi = π ˜i + π i 1RGB , 2  RGB T π ˜i = γˆi , e 3 2

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

πiT 1RGB · π i + |˜ πi | = 1. 3π 2i + 2˜ Figure 3 illustrates an example of the fractal modeling process; based on the ˆ are selected and expanded distribution Γs shown in Fig. 2, a set of fixed points Γ ˆ consisting of five instances: π to extract a system of the as-is primary Π ˆr , π ˆg , π ˆb to be associated with the trichromatic primary and scene specific π ˆgb . By using the as-is primary, we can simulate the adaptation process of the inherent saliency preference to naturally complex scenes. Let the probability of RGB selection be extended to the selection of as-is primary as follows:  T 2    2 φ π ˆi ˆ = Pω (ˆ Qω Π πi ) = ω , ˆi . (8) φTω π ˆ Qω Π ˆ π ˆ i ∈Π Since the probabilistic complexity arising in the adapted preference to the as-is primary can be indexed in terms of the following Shannon’s entropy:  ˆω =− Pω (ˆ πi ) log Pω (ˆ πi ) , H ˆ π ˆ i ∈Π

we have the evaluation of the chromatic saliency arising from the stochastic as-is ˆ primary selection: ψˆω = e−Hω . The pixelwise index ψˆω was applied to the scene image shown in Fig. 1 to extract the distribution of saliency patterns as indicated in Fig. 4. As exhibited in this figure, we have an enhancement of the saliency distribution; despite significant degeneration from original colors, various types of landmarks were detected based on the saliency index ψˆω . This implies that the as-is primaries provides robust cue to control the focus of on-vehicle vision in naturally complex scenes.

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

K. Kamejima / Saliency-Based Geographics Annotation for Robotic Access

(1)

(2)

(3)

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

(4)

(5)

(6) (a) Scene Image

(b) As-is Primary

(c) RGB Primary

Figure 5. Saliency Pattern Extraction

Simulation and Modeling Related to Computational Science and Robotics Technology : Proceedings of SiMCRT 2011, IOS Press,

219

220

K. Kamejima / Saliency-Based Geographics Annotation for Robotic Access

Table 1. Complexity Reduction via ψˆω -Filtering scene

Copyright © 2012. IOS Press, Incorporated. All rights reserved.

(1) (2)

dSG 1.109258 1.405944

dSH <