Chaos and Complexity: New Research [1 ed.] 9781608766765, 9781604568417

Citation preview

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

CHAOS AND COMPLEXITY: NEW RESEARCH

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

CHAOS AND COMPLEXITY: NEW RESEARCH

FRANCO F. ORSUCCI AND

NICOLETTA SALA

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

EDITORS

Nova Science Publishers, Inc. New York

Copyright © 2009 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. For permission to use material from this book please contact us: Telephone 631-231-7269; Fax 631-231-8175 Web Site: http://www.novapublishers.com NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Any parts of this book based on government reports are so indicated and copyright is claimed for those parts to the extent applicable to compilations of such works. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. LIBRARY OF CONGRESS CATALOGING-IN-PUBLICATION DATA Orsucci, Franco. Chaos & complexity : new research / Franco F. Orsucci, Nicoletta Sala. p. cm. ISBN 978-1-60876-676-5 (E-Book) 1. Chaotic behavior in systems. I. Sala, Nicoletta. II. Title. III. Title: Chaos and complexity. Q172.5.C45O77 2008 003'.857--dc22 2008026217

Published by Nova Science Publishers, Inc.

New York

CONTENTS

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Preface

vii

Chapter 1

Complexity and Chaos Theory in Art Jay Kappraff

Chapter 2

Pollock, Mondrian and Nature: Recent Scientific Investigations Richard Taylor

23

Chapter 3

Visual and Semantic Ambiguity in Art Igor Yevin

37

Chapter 4

Does the Complexity of Space Lie in the Cosmos or in Chaos? Attilio Taverna

49

Chapter 5

Crystal and Flame/Form and Process: The Morphology of the Amorphous Manuel A. Baez

53

Chapter 6

Complexity in the Mesoamerican Artistic and Architectural Works Gerardo Burkle-Elizondo, Ricardo David Valdez-Cepeda and Nicoletta Sala

73

Chapter 7

New Paradigm Architecture Nikos A. Salingaros

83

Chapter 8

Generation of Textures and Geometric Pseudo-Urban Models with the Aid of IFS Xavier Marsault

89

Chapter 9

Pseudo-Urban Automatic Pattern Generation Renato Saleri Lunazzi

103

Chapter 10

Tonal Structure of Music and Controlling Chaos in the Brain Vladimir E. Bondarenko and Igor Yevin

113

Chapter 11

Collecting Patterns That Work for Everything Deborah L. MacPherson

121

1

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

vi

Contents

Chapter 12

Stability Conditions in Contextual Emergence H. Atmanspacher and R.C. Bishop

131

Chapter 13

Contextual Emergence of Mental States from Neurodynamics H. Atmanspacher and P. beim Graben

143

Chapter 14

Phase Coupling Supports Associative Visual Processing: Physiology and Related Models R. Eckhorn, A. Gail, A. Bruns, A. Gabriel, B. Al-Shaikhli and M. Saam

161

Chapter 15

Processing of Positive versus Negative Emotional Words Is Incorporated in Anterior versus Posterior Brain Areas: An ERP Microstate LORETA Study L.R.R. Gianotti, P.L. Faber, R.D. Pascual-Marqui, K. Kochi and D. Lehmann

181

Chapter 16

"Cognitive Genes" Reveal Higher Codon Complexity than "Somatic Genes" C.S. Herrmann and W.S. Herrmann

205

Chapter 17

Mutual Phase Synchronization in Single Trial Data A. Hutt and M.H.J. Munk

217

Chapter 18

Ordinal Analysis of EEG Time Series K. Keller, H. Lauffer and M. Sinn

239

Chapter 19

Robustifying EEG Data Analysis by Removing Outliers M. Krauledat, G. Dornhege, B. Blankertz and K.-R. Müller

251

Chapter 20

Is Brain Activity Fractal? M. Le Van Quyen, M. Chavez, C. Adam and J. Martinerie

267

Chapter 21

The Timing of Neural Processes in Humans: Beyond the Evoked Potentials N. Mainy, J. Jung, G. Commiterri, A. Berthoz, M. Baciu, L. Minotti, D. Hoffmann, P. Kahane, O. Bertrand and J.-P. Lachaux

279

Chapter 22

Quantification of Order Patterns Recurrence Plots of Event Related Potentials N. Marwan, A. Groth and J. Kurths

293

Chapter 23

Neuronal Synchronization: From Dynamic Feature Binding to Object Representations A. Maye and M. Werning

307

Chapter 24

Testing for Coupling Asymmetry Using Surrogate Data M. Paluš, B. Mussiza and A. Stefanovska

319

Contents Chapter 25

What Can We Learn from Single-Trial Event-Related Potentials? R. Quian Quiroga, M. Atienza, J.L. Cantero and M.L.A. Jongsma

337

Chapter 26

The Internal Structure of the N400: Frequency Characteristics of a Language Related ERP Component D. Roehm, I. Bornkessel-Schlesewsky and M. Schlesewsky

357

Chapter 27

Measuring the Thalamocortical Loop in Patients with Neurogenic Pain J. Sarnthein and D. Jeanmonod

389

Chapter 28

Assessment of Connectivity Patterns from Multivariate Time Series by Partial Directed Coherence S. Wehling, C. Simion, S. Shimojo and J. Bhattacharya

405

Chapter 29

The Cortical Implementation of Complex Attribute and Substance Concepts: Synchrony, Frames, and Hierarchical Binding M. Werning and A. Maye

427

Index

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

vii

445

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

PREFACE This book presents new international research on artificial life, cellular automata, chaos theory, cognition, complexity theory, synchronization, fractals, genetic algorithms, information systems, metaphors, neural networks, non-linear dynamics, parallel computation and synergetics. The unifying feature of this research is the tie to chaos and complexity. As presented in Chapter 1, Kauffman and Varela propose the following experiment: Sprinkle sand or place a thin layer of glycerine over the surface of a metal plate; draw a violin bow carefully along the plate boundary. The sand particles or glycerine will toss about in a rapid dance, swarming and forming a characteristic pattern on the plate surface. This pattern is at once both form and process: individual grains of sand or swirls of glycerine play continually in and out, while the general shape is maintained dynamically in response to the bowing vibration. The abstract paintings of Piet Mondrian and Jackson Pollock are traditionally regarded as representing opposite ends of the diverse visual spectrum of Modern Art. Chapter 2 presents an overview of recent scientific research that investigates the enduring visual appeal of these paintings. Non-linear theory proposed different models perception of ambiguous patterns, describing different aspects multi-stable behavior of the brain. Chapter 3 aims to review the phenomenon of ambiguity in art and to show that the mathematical models of the perception of ambiguous patterns should regard as one of the basis models of artistic perception. The following type of ambiguity in art will be considered. Visual ambiguity in painting, semantic (meaning) ambiguity in literature (for instance, ambiguity which V.B.Shklovsky called as "the man who is out of his proper place"), ambiguity in puns, jokes, anecdotes, mixed (visual and semantic) ambiguity in acting and sculpture. Synergetics of the brain revealed that the human brain as a complex system is operating close to the point of instability and ambiguity in art must be regarded as important tool for supporting the brain near this critical point that gives human being possibilities for better adaptation. As explained in Chapter 4, the art of painting, as we have already known for a long time, is first and foremost an aesthetic inquiry on the nature of space. It’s easy to understand why. The state of being of an aesthetic experience such as a painting, always needs an extension, sometimes of a surface, often of a double dimension, always of some kind of phenomenology of space. Here is the ultimate reason why. In our modern times, even in the case of drawing the structure of a chip, or when we shoot a real event with a video camera, we use an extension as a support. So to say we are

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

x

Franco F. Orsucci and Nicoletta Sala

using an idea of space already known to us, in the same way in which we use the net. We can use it only because there’s an idea of pluri-dimensional space in it that we identified as fundamental: cyber-space, precisely/exactly. Chapter 5 presents the work and research produced through an on-going architectural project entitled The Phenomenological Garden. The project seeks to investigate the morphological and integrative versatility of fundamental processes that exist throughout the natural environment. Work produced by students in workshops incorporating educational methods and procedures derived from this research will also be presented. This evolving project is a systematic investigation of the versatile and generative potential of the complex processes found throughout systems in Nature, biology, mathematics and music. As part of the Form Studies Unit in the School of Architecture at Carleton University, the work seeks to investigate how complex structures and forms are generated from initially random processes that evolve into morphologically rich integrated relationships. The morphological diversity revealed by this working and teaching method offers new insights into the complexity lurking within nature’s processes and bridges the theoretical gap between Galileo Galilei’s conception of nature, as revealed above, and the modern theories of Chaos and Complexity as exemplified by Benoit Mandelbrot and Ilya Prigogine. This working process also offers insights into the conceptual and philosophical aspirations of such key central figures as Antoni Gaudi, Louis Sullivan, Frank Lloyd Wright, and Buckminster Fuller in the early formative period of modern architecture, and more recently, the architect/engineer Santiago Calatrava. The implications of these developments are relevant to the study of morphology as well as to the field of architecture at a time when it is addressing the concepts and themes emerging out of our deeper understanding of dynamic and complex phenomena in the physical world. It has been demonstrated that scribers, artists, sculptors and architects used a geometric system in ancient civilizations. There appears such system includes basically golden rectangles distributed in a golden spiral fashion. In addition, it is clear that we do not know the sequence in which the lines or pictures were originally traced or drawn. By this way, the artistic and architectural works can be considered as static objects and so they may be characterized by an inherent dimension. The aim of Chapter 6 is to introduce a description of the complexity presents in the Mesoamerican artistic and architectural works (e.g., tablets from Palenque and other sites, Maya stelae, Maya hieroglyphs, pyramids, palaces and temples, calendars and astronomic stones, codex pages, murals, great stone monuments, astronomic stones and ceramic pots). The authors’ findings indicate a characteristic higher fractal dimension value for different groups of Mesoamerican artistic and architectural works. Results could be suggesting that Mesoamerican artists and architects used specific patterns and they preferred works with higher (1.91) box and information fractal dimensions. As presented in Chapter 7, Charles Jencks wishes to promote the architecture of Peter Eisenman, Frank Gehry, and Daniel Libeskind by proclaiming it “The New Paradigm in Architecture”. Supposedly, their buildings are based on the New Sciences such as complexity, fractals, emergence, self-organization, and self-similarity. Jencks’s claim, however, is founded on elementary misunderstandings. There is a New Paradigm architecture, and it is indeed based on the New Sciences, but it does not include deconstructivist buildings. Instead, it encompasses the innovative, humane architecture of Christopher Alexander, the traditional humane architecture of Léon Krier, and much, much more.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Preface

xi

Geometric and functional modelling of cities has become a growing field of interest, raised by the development and democratisation of computers being able to support highdemanding graphics in real time. Actually, more and more applications concentrate on creating virtual environments. ARIA has been working for two years, within the DEREVE project (DER, 2000), on pseudo-urban textures and geometric models generation, by means of fractal or parametric methods. Chapter 8 explains the authors’ attempt to capture inner coherence of urban shapes and morphologies, by fractal analysis of 2D½ textures (top view + height) of real and synthetic city maps. The basic ideas lean on autosimilarity detection, fractal coding of regions, and processing with Iterated Function Systems (IFS). The authors introduce a genetic-like approach, allowing interpolation, alteration and fusion of different urban models, and leading to global or local synthesis of new shapes. Finally, a 3D reconstruction tool has been developed for converting textures to volumes in VRML, simplified enough for real time wanderings, and enhanced by some automatically generated garbage dump and decorated elements. Programs and graphic interface are developed with C++ and QT libraries. Chapter 9 aims to experiment automatic generative methods able to produce architectural and urban 3D-models. At this time, some interesting applicative results, rising from pseudorandom and l-system formalisms, came to generate complex and rather realistic immersive environments. Next step could be achieved by mixing those techniques to emerging calculus, dealing whith topographic or environmental constraints. As a matter of fact, future developments will aim to contribute to archeological or historical restitution, quickly providing credible 3D environments in a given historical context. Recent researches revealed that music tends to reduce the degree of chaos in brain waves. For some epilepsy patients music triggers their seizures. Loskutov, Hubler, and others carried out a series of studies concerning control of deterministic chaotic systems. It turned out, that carefully chosen tiny perturbation could stabilize any of unstable periodic orbits making up a strange attractor. Computer experiments have shown a possibility to control a chaotic behavior in neural network by external periodic pulsed force or sinusoidal force. In Chapter 10 the authors suggest that music acts on the brain near delta-,teta-, alpha-, and beta frequencies to suppress chaos. One may propose that the aim of this control is to establish coherent behavior in the brain, because many cognitive functions of the brain are related to a temporal coherence. Chapter 11 is focused on collecting patterns that work for everything. Would the authors even want a meta-methodology or collection such as “patterns that work for everything”? One simple evolving system of explanation and conceptual illustration? Where would these patterns reside? Who would interpret them? There are concepts being developed in the study of chaos and complexity that may help make arrangements for this collection. In particular, a glimpse at what the patterns might look like and act like. Maybe they also act like music, maybe we can discuss, present and interpret abstract information patterns the meticulous way we discuss, present and interpret abstract art. If you stuck a pin in today and drew back to the time when physics, chemistry and biology were one - what are we truly capturing about chaos and complexity for the corresponding point in the future? Are today’s algorithm writers yesterday’s alchemists and what is the best, least constrained and highest quality way to preserve the fundamental and esoteric qualities of this work for future studies? Can we imagine and develop an inherited collective memory for our machines, like language and culture are for us, to pass stories from one generation to the next? Even if they speak different

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

xii

Franco F. Orsucci and Nicoletta Sala

languages and live in different places as we do, something we can all measure may be generated by providing an unsupervised opportunity for our machines to create or illustrate patterns we have not thought about yet, noticed or engineered. There is a story in the study of chaos and complexity that may be able to tell itself. In Chapter 12, the concept of contextual emergence is proposed, as a non-reductive, yet welldefined relation between different levels of description of physical and other systems. It is illustrated for the transition from statistical mechanics to thermodynamical properties such as temperature. Stability conditions are crucial for a rigorous implementation of contingent contexts that are required to understand temperature as an emergent property. It is proposed that such stability conditions are meaningful for contextual emergence beyond physics as well. The emergence of mental states from neural states by partitioning the neural phase space is analyzed in Chapter 13, in terms of symbolic dynamics. Well-defined mental states provide contexts inducing a criterion of structural stability for the neurodynamics that can be implemented by particular partitions. This leads to distinguished subshifts of finite type that are either cyclic or irreducible. Cyclic shifts correspond to asymptotically stable fixed points or limit tori whereas irreducible shifts are obtained from generating partitions of mixing hyperbolic systems. These stability criteria are applied to the discussion of neural correlates of consiousness, to the definition of macroscopic neural states, and to aspects of the symbol grounding problem. In particular, it is shown that compatible mental descriptions, topologically equivalent to the neurodynamical description, emerge if the partition of the neural phase space is generating. If this is not the case, mental descriptions are incompatible or complementary. Consequences of this result for an integration or unification of cognitive science or psychology, respectively, will be indicated. Chapter 14 is a review on multiple microelectrode recordings from the visual cortex of monkeys and subdural recordings from humans - related to the potential underlying neural mechanisms. The former hypothesis of visual object representations by synchronization in visual cortex, or more generally, of flexible associative processing, has been supported by our recent experiments in monkeys. They demonstrated local coherence among rhythmic or stochastic γ-activities (30–90 Hz) and perceptual modulation, according to psychophysical findings of figure-ground segregation. However, γ-coherence in primary visual cortex (V1) of cats and monkeys is restricted to few millimeters, challenging the synchronization hypothesis for larger cortical object representations. The authors found that the spatial restriction is due to γ-waves (30-90 Hz), traveling in random directions across the object representations in V1. It will be argued that the observed phase continuity of these waves can support the neural coding of object continuity. Based on models with spiking neurons, potentially underlying neural mechanisms in visual cortex are proposed: (i) Fast inhibitory feedback loops can generate locally coherent γ-activities. (ii) Spike-timing dependent synaptic plasticity of lateral and feed forward connections with distance-dependent delays can explain the stabilization of cortical retinotopy, the limited cortical range of signal coherence, the occurrence of γ-waves, and the larger receptive fields at successive levels of visual cortical processing. (iii) Slow inhibitory feedback can support figure-ground segregation. (iv) Temporal dispersion in far reaching cortical projections destroys coherence of high frequency signal components but preserves low frequency amplitude modulations. In conclusion, it is proposed that the hypothesis of flexible associative processing by γ-synchronization in visual

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Preface

xiii

cortex, supporting perceptually coherent representations of visual objects, has to be extended to more general forms of signal coupling. In Chapter 15, the spatio-temporal organization of the neural activity that underlies perception of emotional valence was studied analyzing 33-channel event-related potential maps (ERP maps) from 21 normals while reading a sequence of emotional positive, negative and neutral words; subjects were asked to repeat the last word if a question mark followed; they were not informed that the study concerned emotions. Brain electric activity to emotional positive and negative words was compared. Microstate segmentation of the 113 ERP maps (corresponding to the 448 ms of word presentation) identified 14 microstates, i.e. putative steps of information processing. Three microstates, #4 (90-122 ms), #7 (178-202 ms) and #9 (242-274 ms) showed global map topography differences between emotional positive and negative words. During these three microstates, the involved brain areas were identified using Low Resolution Electromagnetic Tomography (LORETA). The results showed that the extraction of valence during the three emotion-sensitive microstates was incorporated in different brain areas: positive as well as negative emotional words caused predominant lefthemispheric activation in #4 and #9, but predominant right-hemispheric activation in #7. The striking communality across the three microstates however was that in each of them, positive words compared to negative words clearly evoked significantly more anterior brain activity. In Chapter 16 the authors want to apply the concept of complexity to the analysis and comparison of genes. A multitude of genes has been identified coding somatic function. Recently the analysis of mental disorders yielded insights about genes coding cognitive functions. According to the theory of evolution they evolved from other genes through mutation. Therefore, ‘cognitive genes’ and ‘somatic genes’ should differ in their coding reflecting these mutations. The authors investigated ‘cognitive gens’ and ‘somatic genes’ and demonstrated that their codon usage differs significantly. ‘Somatic genes’ are coded in accordance with the average codon usage of a species—‘cognitive genes’ differ from it, i.e. they reveal a higher codon complexity. This increased complexity might reflect the mutations which occurred during evolution. Analog signals of the cerebral cortex in behaving animals are characterized by strong oscillatory components. To investigate functional interactions among different areas of the cortex, it is biologically plausible to determine dependencies of oscillatory signals such as their phase relation both within and across areas. Chapter 17 proposes the application of a clustering algorithm to detect phase synchronization in local field potentials. The introduced synchronization index allows for the extraction of time windows, which exhibit strong phase synchronization in all examined time series. This kind of phase synchronization is highly non-stationary and is called mutual phase synchronization. The comparison of results obtained from single trials with averages of these results reflect the contributions and variability of the single trials at each time point in a frequency band. Therefore this study proposes a novel approach for the analysis of synchronization to be detected in single trials. The assessment of single trials with respect to the trial average revealed that a number of features in time-frequency space are common to different trials. A comparison to the results from applying the bivariate phase synchronization method elucidates the properties of the proposed algorithm. The results of this study show that short epochs of mutual phase synchronization can be detected at behaviorally relevant moments during single trials. Ordinal time series analysis is a new approach to the qualitative investigation of long and complex time series. The idea behind it is to transform a given time series into a series of

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

xiv

Franco F. Orsucci and Nicoletta Sala

ordinal patterns each describing the order relations between the present and a fixed number of equidistant past values at a given time. In Chapter 18 the authors consider ordinal pattern distributions and some measures derived from them in order to detect differences between EEG data. As explained in Chapter 19, biomedical signals such as EEG are typically contaminated by measurement artifacts, outliers and non-standard noise sources. The authors propose to use techniques from robust statistics and machine learning to reduce the influence of such distortions. Two showcase application scenarios are studied: (a) Lateralized Readiness Potential (LRP) analysis, where we show that a robust treatment of the EEG allows to reduce the necessary number of trials for averaging and the detrimental influence of e.g. ocular artifacts and (b) single trial classification in the context of Brain Computer Interfacing, where outlier removal procedures can strongly enhance the classification performance. There are recent evidences that human electroencephalograms (EEG) have long-range correlations over many time scales, suggesting a self-similar fractal behavior. Nevertheless, a strict fractality is often too restrictive to adequately characterize extremely inhomogeneous fluctuations. As observed in a variety of complex phenomena, properties higher than second order characteristics are often needed, in particular to characterize the broader class of multifractal processes. In Chapter 20, the scaling of structure functions is proposed in order to analyze the spontaneous EEG fluctuations. High order scaling exponents are calculated for long-term intracranial EEG recordings of 10 epileptic patients during seizure-free periods. The authors demonstrated that the scaling exponents of the energy fluctuations exhibit a nonlinear function which is incompatible with monofractal stochastic models. The reported nonlinearity suggests that brain fluctuations exhibit a more complex scaling behavior, consistent with a hierarchical and multiplicative generating process, and that the framework of multifractal provides a more complete and accurate statistical model of brain background fluctuations. Our understanding of the neural bases of mental processes in humans depend on our capacity to visualize the activity of focal neural populations with a great temporal precision. This spatio-temporal precision can only be accessed in certain patients implanted with intracerebral electrodes for therapeutical purposes. Many groups have used this exceptional opportunity to infer the time course of certain neural processes, almost exclusively through the study of intracerebral evoked potentials. One problem is that the evoked potentials reflect only the neural activities that are either of low frequency, or precisely phase-locked to sensory events. This technical limitation has moved the scope away from other, important, components of the neural activity, typically in high-frequency ranges, such as the gamma band (> 30 Hz). In Chapter 21, the authors use intracerebral recordings from a patient performing several cognitive tasks, to illustrate the temporal and functional differences between the evoked potentials and the non-phase locked, but task-related high-frequency neural activities, and advocate the use of the latter, in addition to the former, for the understanding of the human brain dynamics. Chapter 22 studies an innovative modification of recurrence plots defining the recurrence by the local ordinal structure of a time series. In this paper the authors demonstrate that in comparison to a recently developed approach this concept improves the analyis of event related activity on a single trial basis. In Chapter 23, using two different models of oscillatory activity in the primary visual cortex, the authors analyze the synchronization properties of the networks by an eigenmode

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Preface

xv

decomposition. Both models use clusters of feature-sensitive neurons representing local object properties like color and orientation. Whereas in the mean-field model oscillators communicate via their current amplitude, in the phase model oscillator interaction was controlled by phase difference. In both cases, eigenmode analysis decomposes the complex synchronization patterns into a time-invariant, spatial component, the eigenmodes, and characteristic functions describing their weight in network state over time. The authors find that characteristic functions can be associated with representations of objects in a visual scene, and eigenmodes represent different epistemic possibilities. Possible asymmetry in the coupling of complex oscillatory systems can, in principle, be inferred from experimental bivariate time series. However, the quantities estimated from experimental data can be severely biased. In Chapter 24, the authors propose to test the significance of the estimated asymmetry measures by statistical tests using surrogate data. In this way the authors will extend the usage of surrogate data fromtesting the existence of nonlinear dependence to testing whether the coupling is symmetric or asymmetric. The numerically generated surrogate data should mimic most of the statistical and dynamical properties of the tested data, except for the coupling asymmetry, and thus enable estimation of the bias and variance of the used asymmetry measures. “Ideal” surrogate data for testing coupling asymmetry would be series recorded from the same coupled systems when the coupling is symmetric, with its strength adjusted so that symmetric measures of coupling are the same as those in the tested data. Such surrogate data cannot, however, be obtained in usual experimental situations. In an extensive numerical study the authors compare the size and power of asymmetry tests using various types of surrogate data. The authors discuss the conditions under which Fourier phaserandomized surrogates can perform comparably with the “ideal” surrogate data with symmetric coupling. Chapter 25 presents a method for visualizing single-trial evoked potentials and show applications of the consequent single-trial analysis. The method is based on the wavelet transform, which has an excellent resolution both in the time and frequency domains. Its use provides new information that is not accessible from the conventional analysis of peak amplitudes and latencies of average evoked potentials. The authors review some of the applications of the single trial analysis to the study of different cognitive processes. First, the authors describe systematic trial-to-trial changes reflecting habituation and sensitization processes. Second, the authors show how an analysis of trial-to-trial latency variability gives new insights on the mechanisms eliciting a larger mismatch negativity in control subjects, in comparison to sleep deprived subjects when performing a pattern recognition learning task. Third, the authors show in a rhythm perception task that trained musicians had lower latency jitters than non-musicians, in spite of the fact that there were no differences in the average responses. The authors conclude that the single trial analysis of evoked potentials opens a wide range of new possibilities for the study of cognitive processes. Chapter 26 is structured as follows. In the first section, the authors will provide a brief introduction to the traditional ERP analysis methods and the assumptions on which this method is based. Section 2 discusses the frequency-based EEG analysis approach employed here and introduces the basic measures evoked power (EPow), whole power (WPow) and phase-locking index (PLI). The third section of the paper is empirical in nature and comprises three experiments that provide evidence for the two major claims outlined above. Finally, the authors conclude with a general discussion of the major findings and their consequences.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

xvi

Franco F. Orsucci and Nicoletta Sala

The dense and reciprocal connections between thalamus and cortical areas provide anatomical evidence for the importance of thalamocortical interactions in brain function. In Chapter 27, the authors studied the thalamocortical interplay in patients with neurogenic pain during therapeutic neurosurgical interventions by recording simultaneously single cell activity in the thalamus, local field potentials in the thalamus, and EEG on the scalp. In their patients, thalamic cell activity is dominated by Low Threshold Ca2+ bursts and the authors find high thalamocortical coherence in the 4-9 Hz theta frequency band. The authors also studied scalp EEG spectra before surgery and classified individual subjects into either patient or healthy control group. After surgery, the patient spectra approached that of the healthy control group. This report focuses on recording techniques, data preprocessing and multitaper spectral analysis. The three levels of recording shed light on brain function in general, on the genesis of scalp EEG, and on the pathophysiology of neurogenic pain in particular. The authors discuss the pathophysiology in the framework of a unified concept termed thalamocortical dysrhythmia. Embedded in this concept are our therapeutic approach and its clinical results. In nature connectivity between systems is a rule rather than an exception. Accordingly, for the analysis of complex physiological or biological systems, a crucial and important problem is to identify the underlying pattern of connectivity between constituent subsystems of a complex system or between multiple complex systems. In real-life this pattern may be transient, dynamic, non-random, or directed. These features cannot easily be detected by the linear measures of synchronization, such as correlation or magnitude squared coherence, nor by the nonlinear measures which are based on phase/generalized synchronization. In Chapter 28, the authors considered a measure, termed partial directed coherence (PDC), to investigate the connectivity pattern from multivariate signals. PDC is conceptually related to Granger causality, a statistical measure of causality based on comparative prediction. The authors initially evaluated the performance of PDC by simulated networks of linear and nonlinear chaotic systems and found that PDC is able to detect asymmetrical coupling and directional information flows in a network. The authors next applied the PDC measure on recorded multivariate EEG signals during an alternative forced choice task involving preferential or non-preferential decision. As compared to non-preferential decision, preferential decision showed a stronger long-range connectivity between posterior-anterior cortical regions at 800 ms before the decision, which is in accordance with a previous psychophysical study. Combining frame theory with the theory of neural synchronization, Chapter 29 proposes oscillatory networks as a model for the realization of complex attribute and substance concepts in the cortex. The network has both perceptual and semantic capabilities. Coherency chains and hierarchical binding mechanisms are postulated.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 1-22

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 1

COMPLEXITY AND CHAOS THEORY IN ART Jay Kappraff* New Jersey Institute of Technology, Newark, NJ 07102

Kauffman and Varela propose the following experiment: Sprinkle sand or place a thin layer of glycerine over the surface of a metal plate; draw a violin bow carefully along the plate boundary. The sand particles or glycerine will toss about in a rapid dance, swarming and forming a characteristic pattern on the plate surface. This pattern is at once both form and process: individual grains of sand or swirls of glycerine play continually in and out, while the general shape is maintained dynamically in response to the bowing vibration. Hans Jenny in his book Cymatics [1] has noted from this experiment:

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

“Since the various aspects of these phenomena are due to vibration, we are confronted with a spectrum which reveals patterned figurate formations at one pole and kinetic-dynamic processes at the other, the whole being generated and sustained by its essential periodicity. These aspects, however, are not separate entities but are derived from the vibrational phenomenon in which they appear in their unitariness.”

These are poetic ideas, metaphoric notions, and yet they have reflections in all fields from the wave/particle duality of quantum physics, to oscillations within the nervous system to the oscillations and distinctions that we make at every moment of our lives. Complexity and selforganization emerge from disorder the result of a simple process. This process also gives rise to exquisite patterns shown in Figure 1.

*

E-mail address: [email protected]

2

Jay Kappraff

Figure 1. a) Pattern formed by the vibration of sand on a metal plate.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 1. b) Vibration of a thin film of glycerine. From Cymatics by Hans Jenny.

Figure 2. A mark of distinction separating inside from outside.

Complexity and Chaos Theory in Art

3

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

G. Spencer Brown in his book Laws of Form [2] has created a symbolic language that expresses these ideas and is sensitive to them. Kauffman [3] has extended Spencer-Brown’s language to exhibit how a rich world of periodicities, waveforms and interference phenomena is inherent in the simple act of distinction, the making of a mark on a sheet of paper so as to distinguish between self and non-self or in and out (see Figure 2). There is nothing new about this idea since our number system with all of its complexity is in fact derived from the empty set. We conceptualize the empty set by framing nothing and then throwing away the frame. The frame is the mark of distinction. I have found that number when viewed properly reveals self-organization in the natural world from subatomic to cosmic scales. The so-called “devil’s staircase” shown in Figure 3 places number in the proper framework and reveals a hierarchy of rational numbers in which rationals with smaller denominators have wider plateaus and lead to more stable resonances. The devil’s staircase is a representation of the limiting row of the Farey sequence the first eight rows of which is shown in Figure 4. The n-th row is simply a list of all rational fractions with denominators n or less. Notice that row 8 on the interval from 0 to ½ contains all of the critical points on the Mandelbrot set, important for describing chaos theory, where the rationals are fractions of a circle when the Mandelbrot set is mapped from a circle (see Figure 5). On the other hand the interval from ½ to 1 contains many of the tones of the Just musical scale shown on the tone circle in Figure 6, including the tritone (5/7) and the diminished musical seventh (4/7) used in the music of Brahms. Only missing are the dissonant intervals of the semitone and the wholetone [4]. In Figure 7 the number of asteroids in the asteroid belt is plotted against distance from the sun in units of Jupiter’s orbital period Notice that sequence of gaps in the belt are at the rational numbers: 1/3, 2/5, 3/7, ½, 3/5, 2/3, ¾ and that these are consecutive entries to rows 6 and 7 in the Farey sequence. I have found (not shown here) that this same Farey sequence also expresses the hierarchy of phyllotaxis numbers that dictate the growth of plants from pinecones to sunflowers [4].

Figure 3. a) The devil’s staircase exhibited in the Ising model from Physics; b) The devil’s staircase subdivided into six self-similar parts.

4

Jay Kappraff

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 4. The first eight rows of the Farey sequence.

Figure 5. The Mandelbrot set showing critical values of the external angles at fractions from row eight of the Farey Sequence. The fractions determine the period lengths of the iterates zn for a given choice of the parameter c. The point “F” (Feigenbaum limit marks the accumulation point of the period-doubling cascade. A. Douday: Julia sets and the Mandelbrot set

Complexity and Chaos Theory in Art

5

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 6. The Just scale shown on a tone circle. Note the symmetry of rising (clockwise) and falling (counterclockwise) scales.

Figure 7. Number of asteroids plotted against distance from the sun (in units of Jupiter’s orbital period). Gaps occur at successive points in the Farey sequence. From Newton’s Clock by I. Peterson Copyright © 1992 by I. Peterson.

We see here that without a telescope or without a living bud or the sound of a musical instrument, our very number system already contains the objects of our observations of the natural world and is capable of reproducing phenomena in all of its complexity. How did this come to pass. Are we observing an objective reality or are we projecting our own organs of perception onto the world? These are deep questions for philosophical study.

6

Jay Kappraff

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

From the earliest times humans have tried to make sense of their observations of the natural world even though they often experienced the world as chaotic. Their very existence depended on reliable predictions of such events as the arrival of spring to plant, fall to harvest, the coming and going of the tides, etc. The movement of the heavenly bodies provided the first experiences of regularity in the universe and the application of number to describe these motions may have constituted the earliest development of mathematics. In ancient times astronomy and music were tied together. The earliest cultures were aural by nature and music played an important role as confirmed by the many musical instruments found in burial sites of ancient Sumerians from the third and fourth millennia B.C. There is evidence that the Sumerians were aware of the twelve tone musical scale in which tones were represented by the ratio of integers or rational numbers placed on a tone circle with 12 sectors similar to the positions of the planets in the zodiac [5]. In the East the pentatonic scale of five tones chosen from the twelve was prevalent corresponding to the five observed planets. In the West seven tones was the norm since the sun and moon were added to the planets. Expressing the musical scale in terms of rational numbers has certain problems associated with it. It was well understood that a bowed length of string has a higher pitch when it is shortened. For example, if a string representing the fundamental tone is divided in half it gives an identically sounding pitch referred to as an octave. Also the inverse of the string length gives the relative frequency, so that the octave has a frequency twice the fundamental. The key interval of the musical scale is the musical fifth gotten by taking a length of string whose tone represents the fundamental tone say D and reducing it to 2/3 or its length. A succession of twelve musical fifths placed into a single octave gives rise to the twelve tone chromatic scale known as “spiral fifths” as shown in Figure 8. Its serpent like appearance leads the ethnomusicologist, Ernest McClain to suggest that this scale lies at the basis of the many serpent myths in all cultures.

Figure 8. Serpent power: the spiral tuning of fifths. Courtesy of Ernest McClain.

On a piano which is tuned so that each of the intervals of the 12 tone scale are equal in a logarithmic sense (the equal-tempered scale), beginning on any tone and playing twelve successive musical fifths, results in the same tone seven octaves higher. Referring to Figure 8, the first and thirteenth tones in spiral fifths, Aflat and Gsharp, the tritone or three wholetones located at 6 o’clock on the tone circle, are the same tone in different octaves. However, in terms of rational fifths they differ by about a quarter of a semitone, the so-called Pythagorean comma. This is true because in order for (2/3)12 to equal (1/2)7 it would follow that 312 = 219

Complexity and Chaos Theory in Art

7

which is certainly false. Unless a limit is placed on the frequency of the tones, the use of rational numbers to represent tone would require an infinite number of tones. This presented ancient civilizations with a kind of 3rd millennium B.C. chaos theory. Similar problems faced early astronomers as they sought to reconcile the incommensurability of the cycles of the sun and the moon. The solar cycle of 365 ¼ days does not mesh with the lunar cycle of 354 days. A canonical year of 360 days was chosen as a compromise between the two. It turns out that the ratios 365 ¼: 360 and 360:354 are both approximately equal to the Pythagorean comma so that the musical scale had some roots in astronomy. Also if an octave is limited by relative frequencies of 360 to 720 eleven of the tones of the Just scale can be placed as integers within this limit missing only the tritone which you can verify by comparing the intervals of the following sequence with Figure 6 and 9 (the rational numbers represent relative string lengths):

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

D 360 1

Eflat 384 15/16

E 400 8/9

F 432 5/6

Fsharp 450 4/5

G 480 3/4

A 540 2/3

Bflat 576 5/8

B 600 3/5

C 648 9/16

Csharp 675 8/15

Figure 9. The Just scale shown as integers on a tone circle. Note the symmetry.

D’ 720 2

(1)

8

Jay Kappraff

All ancient scales were expressed in terms of integers with the integers of the Just scale divisible by primes 2,3, and 5 while the scale of “spiral fifths” were expressed by integers divisible by primes 2, and 3. Notice in Figures 6 and 9 that the tones of the Just scale are placed symmetrically around the tone circle. This is the result of symmetrically placed rational fractions in Sequence 1 being inverses of each other when factors of 2 are cancelled, e.g., 5/6 ≡ 5/3 as compared with 3/5. But factors of 2 result in the same tone in a different octave. Compare the limit of 360/720 with the limit of 286,624/573,268 required for spiral fifths. So the Just scale embodies the two great lessons of the ancient world, the importance of balance and limit in all things. Ernest McClain has traced the use of music as metaphor in the Rig Veda, the works of Plato and the Bible in his books and articles [6],[7],[8]. To ancient mathematicians and philosophers, the concept of rational number was thought to lie at the basis of cosmology, music, and human affairs. On the other hand, while the concept of an irrational number was not clear in the minds of ancient mathematicians, it was understood that rational numbers could be made to approximate certain ideal elements at dividing points of the tone circle into 12 equal sectors, what is now known as the equal tempered scale with 2 , 3 2 , 4 2 at 6, 4, and 3 o’clock respectively. The battle between rational and irrational numbers was dramatized by the imagery of the Rig Veda. Ernest McClain says [6]:

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The part of the continuum which lies beyond rational number belongs to non-being (Asat) and the Dragon (Vtra). Without the concept of an irrational number, the model for Existence (Sat) is Indra. The continuum of the circle (Vtra) embraces all possible differentiations (Indra). The conflict between Indra and Vtra can never end; it is the conflict between the field of rational numbers and the continuum of real numbers..

This battle between rational and irrational numbers continues into the present where it lies at the basis of chaos theory and the study of dynamical systems. In chaos theory no rational approximation to an irrational number is good enough in terms of yielding closely identical results as I shall demonstrate. Three decades ago scientists began to realize that many of the phenomenon that they thought to be deterministic or predictable from a set of equations were in fact unpredictable. Changing the initial conditions by as small an amount conceivable led to entirely different results. For example, a rational approximation to an irrational initial condition, no matter how good the approximation, would lead eventually to totally different results. The system of equations predicting weather was one such set of equations. In fact as soon as the equations were more complicated than linear, built into them was chaotic behavior. In other words the fluttering of a butterfly’s wings in Brazil could, in principle, over time affect the weather patterns in New York. The growth of plants is another natural system that appears to exist in a state of incipient chaos [4]. Notice that when the cells of a plant are placed around the stem successively at angles, known as divergence angles, related to the golden mean of 2π/φ radians the spiral forms reminiscent of sunflowers appear. Change the divergence angle to a close rational approximation of the golden mean and the spiral is lost and replaced by a spider web appearance (see Figure 10).

Complexity and Chaos Theory in Art

9

a)

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

b) Figure 10. a) A computer generated model of plant phyllotaxis with rational divergence angle 2πx13/21. Note the spider web appearance; b) irrational divergence angle 2π/φ2. Note the daisy-like appearance.

Consider the simple map governing the Mandlebrot set [9], z --> z2 + c for z and c complex numbers. Beginning with an initial point z0 and replacing this in the map leads to the trajectory z0, z1, z2, z3, … The Mandelbrot set constitutes all values of c that lead to bounded trajectories. This sensitive dependence on initial conditions holds for values of c outside of the Mandelbrot set. If the value of c is taken internally and away from the boundary of the Mandelbrot set the behavior of the trajectory is simple, leading either to a fixed point or a periodic orbit. The Julia set is the boundary of the set of points of the trajectory that do not

10

Jay Kappraff

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

escape to infinity. For example, when c = 0, the Julia set is a unit circle. Points outside the Mandelbrot set lead to chaotic behavior of the kind just mentioned. Points near the boundary of the set have the most interesting behavior. One such Julia set for a point near the boundary of the Mandelbrot set is shown in Figure 11. This is somewhat like the state of affairs that exists at the shoreline between land and ocean. The frozen character of the land as opposed to the chaotic nature of the ocean is mediated by the tide pools at the interface between the two. This is where life has its greatest diversity. Stuart Kauffman referred to this region of great differentiation as the “edge of chaos” [10].

Figure 11. A “dragon” shaped Julia set for a value of c at the boundary of the Mandelbrot set.

Complexity and Chaos Theory in Art

11

There is a strong relationship between chaos and fractals. In fact Julia sets generally have a fractal nature. The study of fractals had its beginning with the research of Benoit Mandelbrot into the nature of stock market fluctuations. However, such structures were noticed earlier by Lewis Richardson in his study of the length of coastlines. Richardson noticed that there was a power law relating the apparent length of coastlines when viewed at different scales. When viewed at a large scale such as the scale of a map, the coastline appears finite. But if the scale is reduced so that all of the idiosycracies of the coastline are evident, the ins and outs of the coastline have no apparent limit and its length is effectively infinite. Furthermore, a small stretch of coastline is similar to the whole when viewed in a statistical sense. Robert Cogan and Pozzi Escot have shown that music also has a fractal nature [11]. For example they show that musical structures appear and reappear throughout the musical score at different scales. This is the consequence of the music also satisfying a power law referred to as 1/f noise found in the structure of the music of Bach and Mozart [12]. 1/f noise has a spectrum of sound between the spectrum of Brownian motion in which the next note is completely determined from the previous notes resulting in a frozen quality in the music, and white noise in which the tones are randomly chosen leading to a chaotic sound. So we see that good music is again the result of finding the “edge of chaos.”

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

a)

b)

Figure 12. a) Van Gogh’s painting, “Starry Night”. About this painting Van Gogh wrote, “First of all the twinkling stars vibrated, but remained motionless in space. Then all celestial globes united into one series of movements…Firmaments and planets both disappeared, but the mighty breath which gives life to al things and in which all is bound up remain [13].”; b) a meandering stream winding through separate vortices. From Sensitive Chaos by Schwenk [14].

Good art also strives to incorporate the elements of self-similarity although this is generally done subtly. In a great work of art each image must related to the others in terms of its geometry and metaphorical themes. Artists and sculptors have always been inspired by the complex forms of nature. For example the vortices in Van Gogh’s famous painting, “Starry Night” in figure 12a appears to be taken directly from the meandering stream winding through separate vortices in Figure 12b. Trains of vortices also appear in the knarled cypress trees found in many of Van Gogh’s late paintings such as “St Paul’s Hospital, (1889)” of

12

Jay Kappraff

Figure 13a and perfectly embody the bark and knots of the cypress tree in Figure 13b. On the other hand, the design on a palm leaf from New Guinea represent yet another set of vortices shown in Figure 14a and b. Figures 12b, 13b, and 14b were taken from the beautiful photos of complexity in nature found in Theodor Schwenk’s book, Sensitive Chaos [14].

a)

b)

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 13. Van Gogh’s painting, “St. Paul’s Hospital, (1889)”. Van Gogh wrote, “ The cypress are always occupying my thoughts---it astonishes me that they have not been done as I see them.”; b) The bark and knots of a cypress tree from Schwenk [12].

a)

b) Figure 14. a) Design on a palm leaf (May River, New Guinea) Volkerkundliches Museum, Basel; b) A vortex train from Schwenk [14].

Complexity and Chaos Theory in Art

13

Manuel Baez (see this issue) creates sculptures reminiscent of complex forms from nature out of bamboo sticks and rubber band connectors [15] resulting in structures whose whole is greater than the sum of its parts. Baez describes his system as follows: “These dynamic processes are inherently composed of interweaving elemental relationships that evolve into integrative systems with startling form and structure generating capabilities”. Beginning with a simple shape such as a square or pentagon, a module is created which is replicated over and over. Since the sticks are flexible, the model inter-transforms into amazing shapes illustrating the order which exists within apparent chaos. Three structures from his “Phenomenological Garden” all made with 12” and 6” bamboo dowels and rubber bands are shown in Figure 15. They were all generated from a simple square pattern.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 15. The Phenomenological Garden of Manuel Baez.

Bathsheba Grossman invites scientists and mathematicians to send her complex images from their work such as proteins or globular clusters from astronomy or complex geometrical forms and recreates them as three dimensional sculptures in a variety of medias. Her “Cosmological Simulation” (see Figure 16a) was created from simulated scientific data and illustrates the fractal nature of the universe. “Ferritin Protein” (see Figure 16b) is a threedimensional model in laser etched crystal made from a protein data bank file. Her bronze sculpture “Metatron” is shown in Figure 17. It is made by a lost wax process and created from an operation upon a cube and an octahedron. It appears to be as a singular vortex fixed in time and is evocative to me of frozen music. Barnsley [16] has shown that fractal images can be created by subjecting an initial seed figure to the following transformations: contractions, translations, rotations, and affine transformations (transformations that transform rectangles to arbitrary parallelograms). For example, Barnsley’s fern is created by repeatedly transforming an initial rectangle to three rectangles of different sizes, proportions, and orientations and one line segment as shown in Figure 18. This approach to generating fractals is leading to revolutionary ways of understanding how complex structures arise from simple ones, and it is being applied to many

14

Jay Kappraff

applications from image processing to generation of fractal scenes for movie sets such as that shown in Figure 19 generated by Kenneth Musgrave.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

a)

b) Figure 16. a) Large scale model of a cosmological simulation; b) Ferritin, a symmetrical protein. Courtesy of Bathsheba Grossman.

Complexity and Chaos Theory in Art

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 17. The Metatron. Courtesy of Bathsheba Grossman.

Figure 18. Barnsley’s fern. Created by repeated transformation from a rectangular seed pattern.

15

16

Jay Kappraff

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 19. A fractal scene by Kenneth Musgrave.

a)

b)

Figure 20. a) Fractal simulation of Bamileke architecture. In the first iteration (“seed shape”) the two active lines are shown in gray. b) Enlarged view of the fourth iteration. From African Fractals by Ron Eglash [15].

Complexity and Chaos Theory in Art

17

Structures and designs with fractal properties appear quite naturally in many cultures. I will present two examples from Ron Eglash’s book African Fractals [17]. In the western part of the Cameroons lies the fertile grasslands region of the Bamileke. Eglash describes their fractal settlement architecture (see Figure 20). “These houses and the attached enclosures are built from bamboo—Patterns of agricultural production underlie the scaling. Since the same bamboo mesh construction is used for houses, house enclosures, and enclosures of enclosures, the result is a self-similar architecture—The farming activities require alot of movement between enclosures, so at all scales we see goodsized openings.”

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Many of the processional crosses of Ethiopia indicate a threefold fractal iteration (see Figure 21). Eglash suggests that the reason that the iteration stops at three may be for practical reasons. Two iterations is too few to get the concept of iteration across, while more than three presents fabrication difficulties to the artisans.

Figure 21. Fractal simulation for Ethiopian processional crosses through three iterations. From African Fractals by Ron Eglash [15].

The twentieth century was a revolutionary time in the history of mathematics and science. First the deterministic nature of physics was replaced by the strange world of quantum mechanics where the outcomes of an experiment depended on probability counter to the

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

18

Jay Kappraff

intuition of Albert Einstein that “God does not play dice.” Then the foundations of mathematics were shaken by Kurt Godel who showed that a mathematical system could not be both consistent and complete while Alan Turing discovered that there was no way of determining whether a computer program would halt once given some initial data. Mathematical and scientific theories are created by observing symmetries of all sorts. This enables the information inherent in the physical system to be compressed into a theory or set of equations. For example, all of the possible motions of celestial or earthbound bodies are governed by Newtons laws which is elegantly stated as F = ma. Knowing only a few facts about the initial motion, in other words only a few bits of information, the theory can predict the ensuing motion. What if the system exhibited no such symmetry? Then each specific instance would have to be observed in its entirety. In other words, no information would have been compressed for us to unlock by a theory. All we could do would be to observe each orbit and record what we saw. Systems generated by rules in which the next state is determined by the flipping of a coin is an example of a system devoid of symmetry. There is no way to determine the final state of the system except by following the coin flips to their conclusion. Similarly in mathematics, a mathematical system is generally compressed by stating several axioms representing a finite number of bits of information from which an unlimited number of theorems follow. Without axioms mathematics would not be concerned with judging truth or falsity but rather with generating patterns. G.J. Chaitin [18] has recently shown that rather than being an irrelevant curiosity, this state of affairs, reflected in Godel’s and Turing’s discoveries, is central to the representation of nature by mathematics and science. He created a number from number theory with the property that the determination of its digits was equivalent to flipping coins. We can now say that, it may be that only narrow islands of observation may be derivable from our standard equations and theories. As a result mathematicians have begun to realize that other approaches would be needed to characterize natural phenomena and to coax information from nature. One such program is being explored by Stephen Wolfram in his book A New Kind of Science [19]. Wolfram studied the behavior of a large class of systems governed by rules in which the next state of the system was determined by the previous state, so-called cellular automata. In response to simple rules and starting with simple initial conditions, complex forms would emerge such as the one in Figure 22a. Compare this with one of the network of veins of sand created by the interplay of sand and water shown in Figure 22b by Schwenk. Wolfram discovered that all such automata could be classified as being one four types and that naturally occurring systems of growth from plants and animals to blood vessels to crystals, some of which are shown in Figure 23, were themselves cellular automata exhibiting the same properties as the artificial ones he created. Furthermore he discovered an astounding principal which he refers to as the Principal of Computational Equivalence which states that all processes, whether they are produced by human effort or occur spontaneously in nature, can be viewed as computations. Furthermore, in many kinds of systems particular rules can be found that achieve universality, in other words, the ability to function as a computer in all of its generality, e.g., a universal Turing machine. The dramatic discovery of his book was to show that rather than being a rare event, such universality could be created out of simple rules.

Complexity and Chaos Theory in Art

a)

19

b)

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 22. a) An example of a system defined by the following rule: at each step, take the number obtained at that step and write its base 2 digits in reverse order, then add the resulting number to the original one. Dark squares represent 1 while light squares 0. For many possible starting numbers, the behavior obtained is very simple. This picture shows what happens when one starts with the number 16. After 180 steps, it turns out that all that survives are a few objects that one can view as localized structures. From A New Science by S. Wolfram [19]; b) A network of veins of sand created by the interplay of sand and water. From Schwenk [14].

Figure 23. A collection of patterns from nature suggesting natural cellular automata. From A New Science by S. Wolfram.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

20

Jay Kappraff

Figure 24. Cellular automata generated by simple rules with the appearance of Ethiopian crosses. From A New Science by S. Wolfram [19].

This new approach to science is an invitation for artists and scientists to draw closer to one another. After all, the examples of ornamental art have patterns similar to ones generated by cellular automata. For example, Figure 24 illustrates several eamples generated by cellular automoata reminiscent of the Ethiopian designs of Figure 20. Hans Jenny’s and Theodor Schwenk’s vibratory patterns offer another link between art, science and nature. Figure 25a from Jenny [1] shows particles of sand in a state of flow being excited by crystal oscillations on a steel plate. Compare this with Figure 25b from Schwenk [14] showing the ripple marks in sand at a beach.

Complexity and Chaos Theory in Art

21

a)

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

b) Figure 25. a) Particles of sand in a state of flow excited by crystal oscillations. From Jenny [1]; b) Ripple marks of sand on a beach. From Schwenk [14].

We are heading into an exciting new era of scientific and mathematical explorations in which artists, musicians and scientists will be joining hands to help each other and the rest of us to understand our universe in all of its complexity. More and more the question will be asked: Is it art or is it science? Mathematics will serve as the common language, scientists and engineers will create the technology, and artists and musicians will provide the spirit. These new approaches will suit our age and society much as ancient systems of thought met the needs of those cultures. Just as ancient systems of numerology were incorporated into the

22

Jay Kappraff

myths, religious symbolism and philosophy of those ages, the new science of complexity and chaos theory is certain to spawn its myths and metaphors for our age.

References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

[16] [17] [18] [19]

Jenny, H., Cymatics, Basel: Basilius Press (1967). Spencer-Brown, G. I, Laws of Form, London:George Allen and Unwin, Ltd. (1969). Kauffman, L.H. and Varela, F.J., “Form Dynamics,” J. Soc. And Bio. Struct.3 pp161206 (1980). Kappraff, J. Beyond Measure: A Guided Tour through Nature, Myth, and Number, Singapore: World Scientific (2003). McClain, E.G., “Musical theory and Cosmology”, The World and I (Feb. 1994). McClain, E.G., Myth of Invariance, York Beach, Me.:Nicolas-Hays (1976,1984) McClain, E.G., The Pythagorean Plato, York Beach, Me.:Nicolas-Hays (1978,1984). McClain, E.G. “A priestly View of Bible arithmetic in philosophy of science, Van Gogh’s Eyes, and God: Hermeneutic essays in honor of Patrick A. Heelan”, ed. B.E. Babich, Boston: Kluwer Academic Publ. (2001). Peitgens, H-O., Jurgens, H., and Saupe, D., Chaos and Fractals, New York: Springer (1992). Kauffman, S.A., The Origins of Order: Self Organization and Selection and Complexity, New York: Oxford Press (1995). Cogan, R. and Escot, P., Sonic Design: The Nature of Sound and Music, Englewood Cliffs, NJ: Prentice Hall (1976). Gardner, M., “White and brown music, fractal curves and one-over-f fluctuations,” Sci. Am., v238, No.4 (1978). Purce, J., The Mystic Spiral, New York: Thames and Hudson (1974). Schwenk, T., Sensitive Chaos, New York: Schocken Books (1976). Baez, M.A., The Phenomenological Garden, In On Growth and Form: The Engineering of Nature, ACSA east Central Regional Conference, University of Waterloo, Oct. 2001. Barnsley, M., Fractals Everywhere, San Diego: Academic Press (1988). Eglash, R., African Fractals, New Brunswick: Rutgers Univ. Press (1999). Chaitin, G.J. “A century of controversy over the foundations of Mathematics,” Complexity, vol. 5, No. 5, pp.12-21, (May/June 2000). Wolfram, S. A New Kind of Science, Champaign, IL: Wolfram Media, Inc. (2002).

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 23-35

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 2

POLLOCK, MONDRIAN AND NATURE: RECENT SCIENTIFIC INVESTIGATIONS Richard Taylor* University of Oregon, Oregon

Abstract The abstract paintings of Piet Mondrian and Jackson Pollock are traditionally regarded as representing opposite ends of the diverse visual spectrum of Modern Art. In this article, I present an overview of recent scientific research that investigates the enduring visual appeal of these paintings.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Introduction Walking through the Smithsonian (USA), it is clear that the stories of Piet Mondrian (18721944) and Jackson Pollock (1912-56) present startling contrasts. First, I come across an abstract painting by Mondrian called “Composition With Blue and Yellow” (1935). It consists of just two colors, a few black lines and an otherwise uneventful background of plain white (see Fig. 1). It's remarkable, though, how this simplicity catches the eye of so many passers-by. According to art theory, Mondrian’s genius lay in his unique arrangement of the pattern elements, one that causes a profound aesthetic order to emerge triumphantly from stark simplicity. Carrying on, I come across Pollock’s “Number 3, 1949: Tiger” (See Fig. 2). Whereas Mondrian’s painting is built from straight, clean and simple lines, Pollock’s are tangled, messy and complex. This battlefield of color and structure also attracts a crowd, mesmerised by an aesthetic quality that somehow unites the rich and intricate splatters of paint.

*

E-mail address: [email protected]

24

Richard Taylor

Figure 1. A comparison of Piet Mondrian’s “Composition with Blue and Yellow” (1935) with a painting by Alan Lee in which the lines are positioned randomly. Can you tell which is the real Mondrian painting?

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 2. Jackson Pollock’s “Number 3, 1949: Tiger.”

Both men reached their artistic peak in New York during the 1940s. Although Mondrian strongly supported Pollock, their approaches represented opposite ends of the spectrum of abstract art. Whereas Mondrian spent weeks deliberating the precise arrangement of his patterns [Deicher, 1995], Pollock dashed around his horizontal canvases dripping paint in a fast and spontaneous fashion [Varnedoe et al, 1998]. Despite their differences in the creative process and the patterns produced, both men maintained that their goal was to venture beyond life’s surface appearance by expressing the aesthetics of nature in a direct and profound manner. At their peak, the public viewed both men’s abstract patterns with considerable scepticism, failing to see any connection with the natural world encountered during their daily lives. Of the two artists, Mondrian was given more credence. Mondrian was a sophisticated intellectual and wrote detailed essays about his carefully composed works. Pollock, on the

Pollock, Mondrian and Nature: Recent Scientific Investigations

25

other hand, was frequently drunk and rarely justified his seemingly erratic motions around the canvas. Fifty years on, both forms of abstract art are regarded as masterpieces of the Modern era. What is the secret to their enduring popularity? Did either of these artists succeed in their search for an underlying aesthetic quality of life? In light of the visual contrast offered by the two paintings at the Smithsonian, it’s remarkable how the passers-by use similar language to discuss their aesthetic experiences. Both paintings are described in terms of 'balance,' 'harmony' and 'equilibrium.' The source of this subtle order seems to be enigmatic, however. None of the gallery audience can define the exact quality that appeals to them. It’s tempting to come away from this scene believing that, half a century after their deaths, we might never comprehend the mysterious beauty of their compositions. Recently, however, their work has become the focus of unprecedented scrutiny from an unexpected source - science. In 1999, I published a pattern analysis of Pollock’s work, showing that the visual complexity of his paintings is built from fractal patterns –patterns that are found in a diverse range of natural objects [Taylor et al, 1999]. Furthermore, in an ongoing collaboration with psychologists, visual perception experiments reveal that fractals possess a fundamental aesthetic appeal [Taylor, 2001]. How, then, should we now view Mondrian’s simple lines?

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

1. Pollock’s Dripped Complexity First impressions of Pollock’s painting technique are striking, both in terms of its radical departure from centuries-old artistic conventions and also in its apparent lack of sophistication! Purchasing yachting canvas from his local hardware store, Pollock simply rolled the large canvases (up to five meters long) across his studio floor. Even the traditional painting tool - the brush - was not used in its expected capacity: abandoning physical contact with the canvas, he dipped the brush in and out of a can and dripped the fluid paint from the brush onto the canvas below. The uniquely continuous paint trajectories served as 'fingerprints' of his motions through the air. During Pollock’s era, these deceptively simple acts fuelled unprecedented controversy and polarized public opinion of his work: Was he simply mocking artistic traditions or was his painting ‘style’ driven by raw genius? Over the last fifty years, the precise meaning behind his infamous swirls of paint has been the source of fierce debate in the art world [Varnedoe et al, 1998]. Although Pollock was often reticent to discuss his work, he noted that, “My concerns are with the rhythms of nature” [Varnedoe et al, 1998]. Indeed, Pollock’s friends recalled the many hours that he spent staring out at the countryside, as if assimilating the natural shapes surrounding him [Potter, 1985]. But if Pollock’s patterns celebrate nature’s ‘organic’ shapes, what shapes would these be? Since the 1970s many of nature's patterns have been shown to be fractal [Mandelbrot, 1977]. In contrast to the smoothness of artificial lines, fractals consist of patterns that recur on finer and finer scales, building up shapes of immense complexity. Even the most common fractal objects, such as the tree shown in Fig. 3(a), contrast sharply with the simplicity of artificial shapes. The unique visual complexity of fractal patterns necessitates the use of descriptive approaches that are radically different from those of traditional Euclidian geometry. The fractal dimension, D, is a central parameter in this regard, quantifying the fractal scaling

26

Richard Taylor

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

relationship between the patterns observed at different magnifications [Mandelbrot, 1977, Gouyet, 1996]. For Euclidean shapes, dimension is a familiar concept described by integer values of 0, 1, 2 and 3 for points, lines, planes, and solids respectively. Thus, a smooth line (containing no fractal structure) has a D value of 1, whereas a completely filled area (again containing no fractal structure) has a value of 2. For the repeating patterns of a fractal line, D lies between 1 and 2. For fractals described by a D value close to 1, the patterns observed at different magnifications repeat in a way that builds a very smooth, sparse shape. However, for fractals described by a D value closer to 2, the repeating patterns build a shape full of intricate, detailed structure. Figure 4 demonstrates how a fractal pattern’s D value has a profound effect on its visual appearance. The two natural scenes shown in the left column have D values of 1.3 (top) and 1.9 (bottom). Table 1 shows D values for various classes of natural form.

(a)

(b)

Figure 3. (a) Trees are an example of a natural fractal object. Although the patterns observed at different magnifications don’t repeat exactly, analysis shows them to have the same statistical qualities (photographs by R.P. Taylor). (b) Pollock’s paintings (in this case “Number 32, 1950”) display the same fractal behavior.

Pollock, Mondrian and Nature: Recent Scientific Investigations

27

The patterns of a typical Pollock drip painting are shown at different magnifications in Fig. 3(b). In 1999, my research team published an analysis of 20 of Pollock's dripped paintings showing them to be fractal [Taylor et al, 1999]. We used the well-established 'boxcounting' method, in which digitized images of Pollock paintings were covered with a computer-generated mesh of identical squares. The number of squares, N(L), that contained part of the painted pattern were then counted and this was repeated as the size, L, of the squares in the mesh was reduced. The largest size of square was chosen to match the canvas size (L~2.5m) and the smallest was chosen to match the finest paint work (L~1mm). For -D

fractal behavior, N(L) scales according to N(L) ~ L , where 1 < D < 2 [Gouyet, 1996]. The D values were extracted from the gradient of a graph of log N(L) plotted against log L (details of the procedure are presented elsewhere [Taylor et al, 1999]). Table 1. D values for various natural fractal patterns

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Natural pattern Coastlines: South Africa, Australia, Britain Norway Galaxies (modeled) Cracks in ductile materials Geothermal rock patterns Woody plants and trees Waves Clouds Sea Anemone Cracks in non-ductile materials Snowflakes (modeled) Retinal blood vessels Bacteria growth pattern Electrical discharges Mineral patterns

Fractal dimension 1.05-1.25 1.52 1.23 1.25 1.25-1.55 1.28-1.90 1.3 1.30-1.33 1.6 1.68 1.7 1.7 1.7 1.75 1.78

Source Mandelbrot Feder Mandelbrot Louis et al. Campbel Morse et al. Werner Lovejoy Burrough Skejltorp Nittman et al. Family et al. Matsushita et al. Niemyer et al. Chopard et al.

Recently, I described Pollock's style as ‘Fractal Expressionism’ [Taylor et al, Physics World, 1999] to distinguish it from computer-generated fractal art. Fractal Expressionism indicates an ability to generate and manipulate fractal patterns directly. How did Pollock paint such intricate patterns, so precisely and do so 25 years ahead of the scientific discovery of fractals in natural scenery? Our analysis of film footage taken in 1950 reveals a remarkably systematic process [Taylor et al, Leonardo, 2002]. He started by painting localized islands of trajectories distributed across the canvas, followed by longer, extended trajectories that joined the islands, gradually submerging them in a dense fractal web of paint. This process was very swift, with D rising sharply from 1.52 at 20 seconds to 1.89 at 47 seconds. We label this initial pattern as the ‘anchor layer’ because it guided his subsequent painting actions. He would revisit the painting over a period of several days or even months, depositing extra layers on top of this anchor layer. In this final stage, he appeared to be fine-tuning D, with its

28

Richard Taylor

value rising by less than 0.05. Pollock's multi-stage painting technique was clearly aimed at generating high D fractal paintings [Taylor et al, Leonardo, 2002].

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 4. Examples of natural scenery (left column) and drip paintings (right column). Top: Clouds and Pollock's painting Untitled (1945) are fractal patterns with D=1.3. Bottom: A forest and Pollock's painting Untitled (1950) are fractal patterns with D=1.9. (Photographs by R.P. Taylor).

Figure 5. The fractal dimension D of Pollock paintings plotted against the year that they were painted (1944 to 1954). See text for details.

Pollock, Mondrian and Nature: Recent Scientific Investigations

29

He perfected this technique over a ten-year period, as shown in Fig. 5. Art theorists categorize the evolution of Pollock's drip technique into three phases [Varnedoe, 1998]. In the 'preliminary' phase of 1943-45, his initial efforts were characterized by low D values. An example is the fractal pattern of the painting Untitled from 1945, which has a D value of 1.3 (see Fig. 4). During his 'transitional phase' from 1945-1947, he started to experiment with the drip technique and his D values rose sharply (as indicated by the first dashed gradient in Fig. 5). In his 'classic' period of 1948-52, he perfected his technique and D rose more gradually (second dashed gradient in Fig. 5) to the value of D = 1.7-1.9. An example is Untitled from 1950 (see Fig. 4), which has a D value of 1.9. Whereas this distinct evolution has been proposed as a way of authenticating and dating Pollock's work [Taylor, Scientific American, 2002] it also raises a crucial question for visual scientists - do high D value fractal patterns possess a special aesthetic quality?

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

2. Fractal Aesthetics Fractal images have been widely acknowledged for their instant and considerable aesthetic appeal [see, for example, Peitgen et al, 1986, Mandelbrot, 1989, Briggs, 1992, Kemp, 1998]. However, despite the dramatic label “the new aesthetic” [Richards, 2001], and the abundance of computer-generated fractal images that have appeared since the early 1980s, relatively few quantitative studies of fractal aesthetics have been conducted. In 1994, I used a chaotic (kicked-rotor) pendulum to generate fractal and non-fractal drip-paintings and, in the perception studies that followed, participants were shown one fractal and one non-fractal pattern (randomly selected from 40 images) and asked to state a preference [Taylor 1998, Taylor, Art and Complexity, 2003]. Out of the 120 participants, 113 preferred examples of fractal patterns over non-fractal patterns, confirming their powerful aesthetic appeal. Given the profound effect that D has on the visual appearance of fractals (see Fig. 4), do observers base aesthetic preference on the fractal pattern’s D value? Using computergenerated fractals, investigations by Deborah Aks and Julien Sprott found that people expressed a preference for fractal patterns with mid-range values centered around D = 1.3 [Sprott, 1993, Aks and Sprott, 1996]. The authors noted that this preferred value corresponds to prevalent patterns in natural environments (for example, clouds and coastlines) and suggested that perhaps people's preference is actually 'set' at 1.3 through a continuous visual exposure to patterns characterized by this D value. However, in 1995, Cliff Pickover also used a computer but with a different mathematical method for generating the fractals and found that people expressed a preference for fractal patterns with a high value of 1.8 [Pickover, 1995], similar to Pollock's paintings. The discrepancy between the two investigations suggested that there isn’t a ‘universally’ preferred D value but that aesthetic qualities instead depend specifically on how the fractals are generated. The intriguing issue of fractal aesthetics was reinvigorated by our discovery that Pollock’s paintings are fractal: In addition to fractals generated by natural and mathematical processes, a third form of fractals could be investigated – those generated by humans. To determine if there are any ‘universal’ aesthetic qualities of fractals, we performed experiments incorporating all three categories of fractal pattern: fractals formed by nature’s processes (photographs of natural objects), by mathematics (computer simulations) and by humans (cropped images of Pollock paintings) [Taylor, 2001]. Figure 4 shows some of the

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

30

Richard Taylor

images used (for the full set of images, see Spehar et al, 2003). Within each category, we investigated visual appeal as a function of D using a 'forced choice' visual preference technique: Participants were shown a pair of images with different D values on a monitor and asked to choose the most "visually appealing." Introduced by Cohn in 1894, the forced choice technique is well-established for securing value judgments [Cohn, 1894]. In our experiments, all the images were paired in all possible combinations and preference was quantified in terms of the proportion of times each image was chosen. The experiment, involving 220 participants, revealed a distinct preference for mid-range fractals (D=1.3 –1.5), irrespective of their origin [Spehar et al, 2003]. The ‘universal’ character of fractal aesthetics was further emphasized by a recent investigation showing that gender and cultural background of participants did not significantly influence preference [Abrahams et al, 2003]. Furthermore, based on experiments performed at NASA-Ames laboratory, our recent preliminary investigations indicate that preference for mid-range D fractals extends beyond visual perception: skin conductance measurements showed that exposure to fractal art with mid-range D values also significantly reduced the observer’s physiological responses to stressful cognitive work [Taylor et al, 2003, Wise et al, 2003]. Skin conductance measurements might appear to be a highly unusual tool for judging art. However, our preliminary experiments provide a fascinating insight into the impact that art can have on the observer’s physiological condition. It would be intriguing to apply this technique to a range of fractal patterns appearing in art, architecture and archeology: Examples include the Nasca lines in Peru (pre-7th century) [Castrejon-Pita et al, 2003], the Ryoanji Rock Garden in Japan (15th century) [Van Tonder et al, 2002], Leonardo da Vinci’s sketch The Deluge (1500) [Mandelbrot, 1977], Katsushika Hokusai’s wood-cut print The Great Wave (1846) [Mandelbrot, 1977], Gustave Eiffel’s tower in Paris (1889) [Schroeder, 1991], Frank Lloyd Wright’s Palmer House in Michigan (1950) [Eaton, 1998], and Frank Gehry’s proposed architecture for the Guggenheim Museum in New York (2001) [Taylor, 2001, Taylor, New Architect, 2003]. As for Pollock, is he an artistic enigma? According to our results, the low D patterns painted in his earlier years should be more relaxing than his later classic drip paintings. What was motivating Pollock to paint high D fractals? Perhaps Pollock regarded the visually restful experience of a low D pattern as too bland for an artwork and wanted to keep the viewer alert by engaging their eyes in a constant search through the dense structure of a high D pattern. We are currently investigating this intriguing possibility by performing eye-tracking experiments on Pollock’s paintings, which are assessing the way people visually assimilate fractal patterns with different D values.

3. Mondrian’s Simplicity Whereas the above research is progressing rapidly toward an appealing explanation for the enduring popularity of Pollock’s paintings, the underlying aesthetic appeal is based on complexity. Clearly, Mondrian's simple visual ‘language’ of straight lines and primary colors plays by another set of rules entirely. In fact, Mondrian developed a remarkably rigorous set of rules for assembling his patterns and he believed that they had to be followed meticulously for his paintings to display the desired visual quality. The crucial rules concerned the basic

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Pollock, Mondrian and Nature: Recent Scientific Investigations

31

grid of black lines, which he used as an artistic ‘scaffold’ to build the appearance of the painting. Mondrian used only horizontal and vertical lines, which he believed “exist everywhere and dominate everything.” In one of the more notorious exchanges in Modern Art history, he argued fiercely when colleague Theo Van Doesburg proposed that they should also use diagonal lines. Mondrian passionately believed that the diagonal represented a disruptive element that would diminish the painting’s balance. So strong was his belief that he threatened to dissolve the ‘De Styl’ art movement that had formed around his painting style. Mondrian wrote to him declaring, “Following the high-handed manner in which you have used the diagonal, all further collaboration between us has become impossible.” Although Mondrian’s theory of line orientation has legendary status within the art world, only recently have his aesthetic beliefs been put to the test. Whereas Pollock’s paintings are being used as novel test beds for examining peoples’ responses to visual complexity, scientists are becoming increasingly interested in Mondrian’s paintings because of their visual simplicity. In terms of neurobiology, it is well-known that different brain cells are used to process the visual information of a painting containing diagonal lines than for one composed of horizontal and vertical lines [Zeki, 1999]. However, as neurologist Semir Zeki points out, whether these changes in brain function are responsible for the observer’s aesthetic experience is “a question that neurology is not ready to answer” [Zeki, 1999]. In 2001, one of my collaborators, Branka Spehar, performed visual perception experiments aimed at directly addressing the link between line orientation and aesthetics. She used images generated by tilting 3 Mondrian paintings at different orientations [Spehar, 2001, Taylor, Nature, 2002]. The 4 orientations included the original one intended by Mondrian, and also 2 oblique angles for which the lines followed diagonal directions. Spehar showed each picture through a circular window that hid the painting’s frame. This removed any issues relating to frame orientation, allowing the observer to concentrate purely on line orientation. Using the ‘forced choice’ technique, she then paired the 4 orientations of each painting in all possible combinations and asked 20 people to express a preference within each pair. The results revealed that people show no aesthetic preference between the orientations featuring diagonal lines and those featuring horizontal and vertical lines. Spehar’s results clearly question the importance of Mondrian’s vertical-horizontal line rule. Mondrian’s obsession with the orientation of his lines extended to their position on the canvas. He spent long periods of time shifting a single line back and forth within a couple of millimetres, believing that a precise positioning was essential for capturing an aesthetic order that was “free of tension” [Deicher, 1995]. Australian artist Alan Lee recently used visual perception experiments to test Mondrian’s ideals [Lee, 2001, Taylor, Nature, 2002]. Lee created 8 of his own paintings based on Mondrian’s style. However, he composed the patterns by positioning the lines randomly. He then presented 10 art experts and over 100 non-experts with 12 paintings and asked them to identify the 4 of Mondrian’s carefully composed patterns and the 8 of his random patterns (see Fig. 1). Lee’s philosophy was simple – if Mondrian’s carefully located lines delivered an aesthetic impact beyond that of randomly positioned lines, then it should be an easy task to select Mondrian’s paintings. In reality, both the experts and non-experts were unable to distinguish the two types of pattern. Line positioning doesn't influence the visual appeal of the paintings! Could this surprising result mean that, despite Mondrian’s time-consuming efforts, his lines were nevertheless random just like Lee’s? To test this theory, I performed a pattern analysis of 22 Mondrian paintings and this showed that his lines are not random. For random

32

Richard Taylor

distributions, each line has an equal probability of being located at any position on the canvas. In contrast, my analysis of 170 lines featured in the 22 paintings show that Mondrian was twice as likely to position a line close to the canvas edge as he was to position it near the canvas center. In addition to dismissing the ‘random line theory,’ this result invites comparisons with traditional composition techniques. In figurative paintings, artists rarely position the center of focus close to the canvas edge because it leads the eye’s attention off the canvas. If Mondrian’s motivations were to apply this traditional rule to his line distributions, he would have avoided bunching his lines close to the edges. Another compositional concept applied to traditional artworks is the Golden Ratio (sometimes referred to by artists as the “Divine Proportion”). According to this rule, the aesthetic quality of a painting increases if the length and height of the rectangular canvas have the ratio of 1.61 (a number derived from the Fibonacci sequence). Whereas the shapes of Mondrian’s canvases don’t match this ratio, a common speculation is that he positioned his intersecting lines such that the resulting rectangles satisfy the Golden Ratio. However, this claim has recently been dismissed in a book that investigates the use of the Golden Ratio in art [Livio, 2002].

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

4. Discussion These recent scientific investigations of Mondrian’s patterns highlight several crucial misconceptions about Mondrian’s compositional strategies. According to the emerging picture of Mondrian’s work, the lines that form the visual scaffold of his paintings are not random. However, their positioning doesn’t follow the traditional rules of aesthetics, nor does it deliver any appeal beyond that achieved using random lines. The aesthetic order of Mondrian’s paintings appears to be a consequence of the presence of a scaffold and it’s associated colored rectangles, rather than any subtle arrangement of the scaffold itself. In other words, the appeal of Mondrian’s visual language isn’t affected by the way the individual ‘words’ are assembled! What, then, were his reasons for developing such strict ‘grammatical’ rules for his visual language? Mondrian wrote extended essays devoted to his motivations, and these focussed on his search for an underlying structure of nature [Mondrian, 1957]. This is surprising because, initially, his patterns seem as far removed from nature as they possibly could be. They consist of primary colors and straight lines - elements that never occur in a pure form in the natural world. His patterns are remarkably simple when compared to nature's complexity. However, his essays reveal that he viewed nature's complexity with distaste, believing that people ultimately feel ill at ease in such an environment. He also believed that complexity was just one aspect of nature, its least pure aspect, and one that provides a highly distorted view of a higher natural reality. This reality, he argued, "appears under a veil" - an order never directly glimpsed, that lies hidden by nature's more obvious erratic side. He believed that any glimpse through this "veil" would reveal the ultimate harmony of the universe. Mondrian wanted to capture this elusive quality of nature in his paintings. Despite the differences in their chosen visual languages, both Pollock and Mondrian aimed to capture the underlying structure of the natural world on canvas. Declaring "I am nature," Pollock focused on expressing nature's complexity. Remarkably, he painted fractal patterns 25 years before scientists discovered that nature's complexity is built from fractals. Furthermore, based on the fractal aesthetic qualities revealed in the perception experiments,

Pollock, Mondrian and Nature: Recent Scientific Investigations

33

current research is aimed at reducing people’s physiological stress by incorporating fractal art into the interior and exteriors of buildings [Taylor et al, 2003, Wise et al, 2003]. These scientific investigations enhance Pollock’s artistic standing in the history of Modern Art, with his work interpreted as a direct expression of nature’s complexity. Now that science has caught up with Pollock, how should we view Mondrian's alternative view of nature? The recent investigations of Mondrian’s patterns indicate that peoples’ aesthetic judgments of his visual language are insensitive to the ways that his language is applied. It’s tempting to conclude that Pollock succeeded in the quest for natural aesthetics and that Mondrian failed. However, this interpretation doesn’t account for the enduring popularity of Mondrian’s patterns. Perhaps he succeeded in glimpsing through nature’s "veil" with an unmatched clarity and was able to move his lines around with a subtlety well beyond our current scientific understanding of nature? Just as art can benefit from scientific investigation, so too can science learn from the great artists.

Acknowledgments I thank my collaborators B. Spehar, C. Clifford, B. Newell, A. Micolich, D. Jonas, J. Wise and T. Martin.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

References Abrahams, F.D., Sprott, J.C., Mitina, O., Osorio, M., Dequito, E.A. and Pinili, J.M., (2003), Judgments of time, aesthetics, and complexity as a function of the fractal dimension of images formed by chaotic attractors, in preparation (private communication). Aks, D, and Sprott, J, (1996), Quantifying aesthetic preference for chaotic patterns, Empirical Studies of the Arts, 14, 1. Burrough, P.A., (1981), Fractal dimensions of landscapes and other environmental data, Nature, 295 240-242. Cambel, A.B., (1993), Applied Chaos Theory: A Paradigm for Complexity, Academic Press, (London). Castrejon-Pita, J.R., et al, (2003), Nasca lines: a mystery wrapped in an enigma, to be published in Chaos. Chopard, B., Hermann, H.J., and Vicsek, T., (1991), Structure and growth mechanism of mineral dendrites, Nature, 309 409. Cohn, J., (1894), Experimentelle unterschungen uber die gefuhlsbetonung der farben helligkeiten, und ihrer combinationen. Philosphische Studien 10 562 Deicher, S., (1995), Mondrian, Taschen (Koln) Eaton, L.K.,(1998), Architecture and Mathematics (Ed. K. Williams). Family, F., Masters, B.R. and Platt. D.E., (1989), Fractal pattern formation in human retinal vessels. Physica D, 38 98. Feder, J. (1988), Fractals, Plenum (New York). Gouyet, J.P. (1996), Physics and fractal structures, Springer (New York). Kemp M. (1998) Attractive attractors, Nature; 394:627.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

34

Richard Taylor

Lee, A. (2001), Piet Mondrian: Buridan’s Ass and the Aesthetic Ideal, Proceedings of the International Conference on "The Art of Seeing and Seeing of Art," Australian National University, (Canberra). Livio, M. (2002), The Golden ratio, Broadway Books, (New York). Louis, E., Guinea F. and Flores, F., (1986), The fractal nature of fracture, Fractals In Physics (eds. L. Pietronero and E. Tossati), Elsevier Science 177. Lovejoy, S., (1982), Area-perimeter pelation for pain and cloud areas, Science, 216 185. Mandelbrot, B.B., (1977), The Fractal Geometry of Nature, W.H. Freeman and Company, (New York). Mandelbrot B. B. (1989) Fractals and an art for the sake of art. Leonardo; Suppl: 21-24. Matsushita, M. and Fukiwara. H., (1993), Fractal growth and morphological change in bacterial colony formation, In Growth patterns in physical sciences and biology, (eds. J.M. Garcia-Ruiz, E. louis, P. Meaken and L.M. Sander), Plenum Press (New York). Mondrian, P., (1957) Life and Work, Abrams (New York). Morse, D.R., Larson, J.H., Dodson, M.M., and Williamson, M.H., (1985), Fractal dimension of anthropod body lengths, Nature, 315 731-733. Niemeyer, L., Pietronero, L., and Wiesmann, H.J., (1984), Fractal dimension of dielectric breakdown, Physical Review Letters, 52 1033. Nittmann, J., and Stanley, H.E., (1987), Non-deterministic approach to anisotropic growth patterns with continuously tunable morphology: the fractal properties of some real snowflakes, Journal of Physics A 20, L1185. Peitgen, P.O., Richter, P.H. (1986), The beauty of fractals: images of complex dynamic systems. Springer-Verlag (New York). Pickover, C., (1995), Keys to Infinity, Wiley (New York) 206. Potter, J., (1985), To a violent grave: an oral biography of Jackson Pollock, G.P. Putman and Sons (New York). Richards, R., (2001), A new aesthetic for environmental awareness: Chaos theory, the beauty of nature, and our broader humanistic identity, Journal of Humanistic Psychology, 41 59-95. Schroeder, M., (1991), Fractals, Chaos and Power Laws, W.H. Freeman and Company, (New York) Skjeltorp. P., (1988), Fracture experiments on monolayers of microspheres, Random Fluctuations and Pattern Growth (ed. H.E. Stanley and N. Ostrowsky) Kluwer Academic (Dordrecht). Spehar, B., (2001), An Oblique Effect in Aesthetics Revisited, Proceedings of the International Conference on "The Art of Seeing and Seeing of Art," Australian National University, (Canberra). Spehar, B., Clifford, C., Newell, B. and Taylor, R.P., (2003), Universal aesthetic of fractals, to be published in Chaos and Graphics, 37 Sprott, J.C., (1993), Automatic Generation of Strange Attractors, Computer and Graphics, 17, 325 Taylor, R.P. (1998), Splashdown, New Scientist, 2144 30. Taylor, R.P., (2001), Architect reaches for the clouds, Nature, 410, 18. Taylor, R.P., (2002), Spotlight on a visual language, Nature, 415, 961. Taylor, R.P. (2002), Order in Pollock’s chaos, Scientific American, 116, December edition.

Pollock, Mondrian and Nature: Recent Scientific Investigations

35

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Taylor, R.P., (2003) Fractal expressionism-where art meets science, Art and Complexity Elsevier Press (Amsterdam). Taylor, R.P. (2003), Second nature: the magic of fractals from Pollock to Gehry, New Architect, July issue. Taylor, R.P., Micolich, A.P., and Jonas, D., (1999), Fractal analysis of Pollock's drip paintings Nature, 399, 422. Taylor, R.P., Micolich, A.P., and Jonas, D, (1999), Fractal expressionism, Physics World, 12, 25-28. Taylor, R.P., Micolich, A.P., and Jonas, D., (2002), The construction of Pollock's fractal drip paintings, 35 203 Leonardo, MIT press. Taylor, R.P., Spehar, B., Wise, J.A., Clifford, C.W.G., Newell, B.R. and Martin, T.P., (2003), Perceptual and physiological responses to the visual complexity of Pollock’s dripped fractal patterns, to be published in the Journal of Non-linear Dynamics, Psychology and Life Sciences. Van Tonder, G.J., Lyons, M.J., and Ejima, Y., (2002), Nature 419, 359. Varnedoe, K., and Karmel, K., 1998, Jackson Pollock, Abrams (New York). Wise, J.A., and Taylor, R.P., (2003), Fractal design strategies for environments, to be published in the Proceedings of the International Conference on Environmental Systems Werner, B.T., (1999), Complexity in natural landform patterns, Science, 102 284. Zeki, S., (1999) Inner Vision, Oxford University Press.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 37-48

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 3

VISUAL AND SEMANTIC AMBIGUITY IN ART Igor Yevin* Mechanical Engineering Institute, Russian Academy of Sciences, 4, Bardina, Moscow, 117324 Russia. 72

Abstract Non-linear theory proposed different models perception of ambiguous patterns, describing different aspects multi-stable behavior of the brain. This paper aims to review the phenomenon of ambiguity in art and to show that the mathematical models of the perception of ambiguous patterns should regard as one of the basis models of artistic perception. The following type of ambiguity in art will be considered. Visual ambiguity in painting, semantic (meaning) ambiguity in literature (for instance, ambiguity which V.B.Shklovsky called as "the man who is out of his proper place"), ambiguity in puns, jokes, anecdotes, mixed (visual and semantic) ambiguity in acting and sculpture. Synergetics of the brain revealed that the human brain as a complex system is operating close to the point of instability and ambiguity in art must be regarded as important tool for supporting the brain near this critical point that gives human being possibilities for better adaptation.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Non-Linear Models Perception of Ambiguous Patterns In perception psychology, multi-stable perception of ambiguous figures is often considered as a marginal curiosity. Nevertheless, this phenomenon is one of the most investigated in psychology. The first description of ambiguity was given by Necker in 1832. The most known examples of ambiguous figures are specially designed patterns such Necker’ cube, “young girl-old lady” and so on. But visual and semantic ambiguity is very often connected also with that the available visual or semantic information is not sufficient by itself to provide the brain with its unique interpretation. The brain uses past experience, either its own or that of our ancestors to help interpret coming insufficient and therefore ambiguous information. Many patterns in our every day life, in a way, are ambiguous patterns, but using additional

*

E-mail address: [email protected], Phone: (095) 57604

38

Igor Yevin

information, we usually resolve or avoid ambiguity [1]. Nikos Legothetis recently shown that resolution of ambiguity is an essential part of consciousness job [2]. This paper aims to review and to familiarize with the present state the phenomenon ambiguity in art and to show that the mathematical models of the perception of ambiguous patterns should regard as the basic models of artistic perception. Ambiguous patterns are examples of two-state, bimodal systems in psychology. When we perceive ambiguous figure, like the fourth picture in the row on Figure 1, the perception switches between two interpretations, namely “man’s face” or “kneeling girl” because it is impossible for the brain to recognize both interpretations simultaneously. Just like for any bifurcative state, it is impossible for ambiguous figure to predict what namely interpretation will appear first. G.Caglioti from Milan Politectic Institute firstly paid attention, that ambiguous figures are cognitive analogue of critical states in physics. Various authors pointed out that perception of ambiguous figures possess non-linear properties, and that multistabile perception could be modeled by catastrophe theory methods [3,4,5]

Figure 1. Ambiguous patterns are two-state systems. Their perception one can model by using elementary catastrophe "cusp"

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The switch between two interpretation could be described by elementary catastrophe "cusp"

x 3 − bx − a = 0 where a and b are control parameters and x is the state variable. The first parameter a is called the normal factor and quantitatively describes the change in bias in the drawing in a "shape space" from a man’s face to a woman’s figure. Because this model may be used for description of perception double meaning situations, it is reasonable to develop the idea of “shape space” on "meaning space" firstly introduced by Ch. Osgood [6]. The second parameter b is called the splitting factor or bifurcation factor and describes how much the amount of details is presented in the ambiguous figure.

Visual and Semantic Ambiguity in Art

39

The state variable x is presented as a scale from +10 ("looks a lot like a man's face") to  10 ("looks a lot like kneeling girl"). For this model we could formally represent potential function

V =

1 4 1 x + b x 2 + ax 4 2

which depicted on Figure 1, and consider catastrophic jump from one image to another as non-equilibrium phase transition. It is worth to note, that unlike to physical sciences, where potential function usually deduces from fundamental laws or standard theories, in mathematical models in psychology and others "soft sciences" potential function is hypothesized and really is considered as potential energetic function, which should be minimized. In this case it might be also considered as Lyapunov function in Hopfield’s model of pattern recognition. Actually, during the viewing of ambiguous figures, perception lapses into sequence of alternations, switching every few seconds between two or more visual interpretations. Ditzinger and Haken offered an approach to the description of such oscillation under recognition of ambiguous figures [7]. Each pattern is described in this model as a vector in the space of quantitative parameters. There is a procedure for selecting non-correlated parameters, which enable to reduce an information volume. The most informative parameters are the order parameters (all they peculiarities occur near critical points, as in the case of order parameters near phase transition [7]). Pattern recognition procedure is the following. First, pattern-prototypes are stored in the computer memory. Then, the pattern that should be recognized is inputted. The recognition dynamics is built in such a way, that its vector evolves in a parameter space to the most similar pattern stored in the computer memory. The prototype patterns are encoded by V i (i = 1,..., M ) . It is assumed that all these vectors are linearly independent. The components of every vector encode the features of the patterns. A pattern to be recognized is encoded by a vector Q (0) and is inputted in a computer

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

memory at t = 0 A dynamic of pattern recognition is constructed so that V i (i = 1,..., M ) , that is the initial vector Q(t), is pulled into one of prototype patterns Vk with which it mostly coincides. Recognized pattern is presented as the linear combination of prototype patterns M

Q (t ) = ∑ d i (t )Vi + ξ (t ) j =1

where di(t) is the order parameter, characterizing the degree to which a pattern is recognized, and ξ(t) is a residual, uncorrelated with Vi. The dynamic of pattern recognition is described as a gradient process in networks with only M neurons according to

40

Igor Yevin M

di (t ) = λd i − ( B + C )d i ∑ d 2j − Cd i3 , j ≠i

λi > 0, B > 0, C > 0, d i (0) = Vi ' Q(0) This system has only the attractors of the type (0, 0,..., dk ≠0,...0). It can be shown that they must be either saddle points or nodes, but not limit circles (oscillations).

Figure 2. Image ambiguity: "young girl" – "old lady".

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Ditzinger and Haken offered synergetic model of the perception of ambiguous patterns, describing dynamical features of such perception. It is based on the model of pattern recognition described above, and the model of the saturation of attention. The recognition of ambiguous patterns is reduced to inputting only two patterns-prototypes (e.g., "young girl" and "old lady") into computer memory with the order parameters d1 and d2. In this case the dynamics of pattern recognition is described in the following way:

Visual and Semantic Ambiguity in Art

where the overdot means

41

d , λ1 and λ2 are time dependent attention parameters, and A, B, dt

and g are constants. The last two equations describe the saturation of attention in the perception of prototype patterns. As analysis shows, the oscillation of perception occurs when the appropriate relations between constants are satisfied [7]. The recognition of ambiguous patterns has very profound and various analogies with numerous artistic phenomena. This model perception of visual ambiguous patterns also could be applied on the case of meaning ambiguity, because meaning perception also includes such phenomena as saturation of attention and the concept of the order parameter [8].

Visual Ambiguity in Art Let us first consider specially designed visual ambiguity in art. Painting by Giuseppe Arcimboldo “The Librarer” is one of the first examples of such type ambiguity in painting. At first sight we recognize face, but a closer look reveals just an arrangement of different books.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 3. Giuseppe Arcimboldo “The Librarer”

The most famous example of ambiguity in painting is, of course, Mona Lisa by Leonardo. In The Story of Art Ernest Gombrich said: "Even in photographs of the picture we experience this strange effect, but in front of the original in the Paris Louvre it is almost uncanny. Sometimes she seems to mock at us, and then again we seem to catch something like sadness in her smile." "This is Leonardo's famous invention the Italians call "sfumato" - the blurred outline and mellowed colors that allow one form to merge with another and always leave something to our imagination. If we now turn to the "Mona Lisa", we may understand something of its mysterious effect. We see that Leonardo has used the means of his "sfumato" with the utmost deliberation. Everyone who has ever tried to draw or scribble a face knows that what we call its expression rests mainly in two features: the corners of the mouth, and the corners of the eyes. Now it is precisely these parts which Leonardo has left deliberately indistinct, but letting them merge into a soft shadow. That is why we are never quite certain in which mood Mona Lisa is really looking at us. Her expression always seems just elude us" [9, p.228].

42

Igor Yevin

The ambiguity of Mona Lisa's smile one can compare with ambiguous images like "young girl - old lady". The oscillation in the perception of that painting can be described by Ditzinger-Haken's model.

Figure 4. Ambiguity of Mona Lisa’s smile.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure gives an example other kind of visual ambiguity, when the human face and part of his figure is designed from. An example of such ambiguity is Disappearing Bust of Voltaire by Salvador Dali.

Figure 5. Ambiguity of Voltaire bust in Salvador Dali's painting Disappearing Bust of Voltaire.

Visual and Semantic Ambiguity in Art

43

Semantic Ambiguity of Visual Scenes

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Let us consider the following painting by J. Vermeer [11]. Why depicted scene is semantically ambiguous? Because the available information is not sufficient and this scene offers huge amount of meaning interpretations. Undoubtedly, there is some relationship between the man and the woman. But is he her husband or a friend? Did he actually enjoy the playing or he think that she can do it better? Is the woman really playing - she is after all standing - or she is concentrating on something else, perhaps something he told her, perhaps announcing a separation or a reconciliation? All these and many others scenarios have equal validity. There is a humorous book called “Captions Courageous” by Reisner and Capplow attempting reinterpretation of famous masterpieces in painting – with more or less wit [12]. This possibility to create new interpretations for famous paintings which are perceived as comic is connected with insufficient information.

Figure 6. Jan Vermeer. A Lady at the Virginals with a Gentleman.

44

Igor Yevin

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Semantic Ambiguity in Plot Development and in Comic Situations A significant type of ambiguity in art means the possible existence in artwork (most often in position of main hero) of two different states, one of them may be hidden until a certain time. A commonplace example of this form of instability exists in numerous book and movie plots in which a spy or Secret Service agent is hiding his identity while maneuvering about in hostile camp. At any moment, he may be unmasked, and the agent’s task is to extend his secret identity as long as possible. In well-known American movie “ROBOCOP” the main character is simultaneously a robot, incarnating an idea pitiless and perfect machine of revenge, and a human being, capable on deep and tender feelings. Another, less- banal example, ambiguity of social nature - what V.B. Shklovsky describes as "the man who is out of his proper place" - is also widely presented in art [13]. The main character Hlestakov in the play by N. Gogol “Inspector General” obviously one may describe using this kind of ambiguity. In Apuleius’s "Golden Ass" the main character is, of course, out of his proper place because the ass in reality is a man.. The plots of such tales like "The Ugly Duckling" by H. Andersen and "The Beauty and the Beast" also are of the same type of ambiguity, sustained over the entire period of the plot. In the majority of the novels by Agatha Kristy we deal with semantic ambiguity, as almost any character of these novels could appear as the murderer. This state of semantic ambiguity is skillfully supported by the author down to an outcome of the plot: “You know, that I never deceive. I simply speak something such, that it is possible to interpret double” once confessed A. Kristy. Without ambiguity of natural languages, the existence of poetry is impossible. According to A.N. Kolmogorov, entropy of language H contains two terms: meaning capacity h1 capability to transmit some meaning information in a text of appropriate length, and flexibility of language h2 - a possibility to transmit the same meaning by different means [14]. Namely h2 is a source of poetic information, and the ambiguity of language is one of the causes of it’s flexibility. Languages of science usually have h2 =0, they exclude ambiguity, and cannot be used as a material for poetry. Rhythm, rhymes, lexical and stylistic norms of poetry will put some restrictions on a text. Measuring that part of the ability to carry information spent on those restrictions (denoted as β ), A.N. Kolmogorov formulated the law, according to which poetry is possible if β< h2 . If the language has β ≥ h2, than poetry is impossible. We know that the brain resolves a visual ambiguity by means of oscillation. A semantic ambiguity (the ambiguity of meaning) is a result of ambiguous words or whole sentence. Semantic ambiguity, wide spread in comic situations, also resolves by oscillations. Ambiguity of humor is often a clash of different meanings. It involves double or multiple meanings, sounds, or gestures, which are taken in the wrong way, or in incongruous ways. Here is D.D. Minayev's epigram: "I am a new Byron" - you proclaim yourself. I can agree with you: The British poet was lame. The rhymes of yours are also lame."

The method used in this epigram is connected with a comparison based on different distant meanings (Byron was the lame, and a vain poet was also a lame, but in his rhymes).

Visual and Semantic Ambiguity in Art

45

The situation described in this epigram is common to a lot of semantically ambiguous comic situations, which contain two states. One state we should call a state with high social status. This position is honorable and sometimes brings profit. The second state we should call a state with low social status. Everybody avoids occupying it. In the aforesaid example, the state with the high social status ("a good poet") we connect with words "a new Byron". Another poet is trying to get this state. But the author of the epigram unexpectedly transfers a poet to the second state with a low social status. This state we connect with the words "the rhymes of yours are also lame". Such an unexpected leap is achieved by using the same word ("lame") for totally different states. So, a feeling of comic is very often connected with sudden transition from a state of high social status to a state of low social status, or the other way round. Is it a single transition? Does it happens only once? Of course not. It is a multistabile perception of meaning. The rhythmical, repeating nature of laughter (ha-ha-ha, etc.) shows that such transitions are repeated. Evidently, a laughing person mentally oscillates every time from the state of high social status to the state of low social status and vice versa, by comparing them. As a result, the rhythmical laughter is generated by the nervous system. Let us consider also the following anecdote about Sherlock Holmes and Dr.Watson.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Sherlock Holmes and Dr. Watson are going camping. They pitch their tent under the stars and go to sleep. Sometime in the middle of the night Holmes wakes Watson up. “Watson, look up at the stars, and tell me what you deduce.” Watson says, “I see millions of stars, and if there are million of stars, and if even a few of those have planets, it’s quite likely there are some planets like Earth, and if there are a few planets like Earth out there, there might also be life.” Holmes replied: “Watson, you idiot, somebody stole our tent”.

We see, that Watson and Holmes offered two different semantic interpretations of the same visual picture of star sky and if Watson gave namely one of possible interpretation of picture of star sky, Holmes paid attention on semantic context of this picture and connected it with their rest position. The origin of the oscillatory character of laughter should be connected with the fundamental property of the distributed neuron set, i.e. as the oscillation occurring in the perception of ambiguous patterns. According to Ditzinger-Haken's model of recognizing of ambiguous patterns, stable limit cycles can be formed in systems of usual nonlinear differential equations for those variables, which describe the visual perception (e.g. attention). Evidently, this is the common characteristic of distributing neuron sets. That's why it is manifested not only in evolutionary low stages (the ancient visual-morphologic structure of nervous and psychological activity of a human being), but also in its latest stages as well (in the semantic-analytical structures of the left cerebral hemisphere). Comic situations are very often connected with polysemantic, i.e. semantically ambiguous, situations. Another situation of perception of ambiguous patterns occurs in a parody of a famous person by some actor. On one hand, we can recognize the manners, gestures, style and voice of that famous person. On the other hand, we see quite a different person. The same method is used in literary and poetic parodies. Every time we are dealing with a bimodal, double-meaning situation. As a result, we have the oscillation of perception, and laughter is one of the external manifestations of this oscillation.

46

Igor Yevin

One can assume that in ambiguous comic situations oscillations occur between two semantic images. The phenomena of synchronization are typical for a self-organizing process in an active medium (and the nerve substance is an active medium). From that, we can conclude that the period of oscillation between semantic patterns coincides with the period of outward macroscopic oscillations, manifested as laughter with the duration of about 0.1 sec. This value is much smaller than the oscillation period, which occurs when recognizing ambiguous figures (1-5 sec.). Why does laughter occurs in the perception of double-meaning situations, and not in the visual perception of ambiguous patterns? We can explain this by essentially different periods of the corresponding oscillations. In the visual perception this period is approximately equal to t=10 sec., and in the perception of the ambiguity of meaning this period is about t=0.1 sec. That difference could be explained by the fact that a much smaller mass of nerve substance is involved in creating semantic patterns, compared with constructing visual patterns. This is because visual information is processed in the massive and ancient visual cortex, and semantic patterns are interpreted in compact Broke-Vernike zone in the left brain hemisphere. Anecdotes, jokes and sketches deliberately are created as short as possible (laconic), in order to reduce the time needed for the saturation of attention in the process of recognition.

Mixed Ambiguity

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Ambiguity of Sculpture We have considered visual ambiguity in painting (see also [10]) and semantic ambiguity in jokes, anecdotes and puns. Let us consider mixed (visual and semantic) ambiguity, taking an example from sculpture art. Sculpture involves an ability to depict representatives of living nature (most often man and animals) from materials of inanimate nature (wood, stone, bronze, etc). In creativity of different sculptures can be observed a prevalence of one of these phase with respect to another. In Michelangelo's works we see triumph of alive and even spiritual under inert matter of stone. Gombrich wrote in book “The Story of Art”: “While in “The Creation of Adam” Michelangelo had depicted the moment when life entered the beautiful body of a vigorous youth, he, now, in the “ Dying Slave”, chose the moment when life was just fading, and the body was giving way to the laws of dead matter. There is unspeakable beauty in this last moment of final relaxation and release from the struggle of life - this gesture of lassitude and resignation. It is difficult to think of this work as being statue of cold and lifeless stone…”. It is interesting to note, that ambiguity of sculpture art influences on literature, because the plots of some works of arts in literature are based on the idea of animated statue - that is, the transition "inanimate-animated" (such as opera "Don Giovanni" by Mozart, "Bronzer Horseman", "Stone Guest" by A. Pushkin ) and of course in ancient legend about sculptor Pygmalion.

Visual and Semantic Ambiguity in Art

47

Ambiguity of Dolls In the essay “Dolls in system of culture” Yu. Lotman marks ambiguous (as well sculpture) nature of this cultural phenomenon closely connected to ancient opposition alive and dead, spiritual and mechanical. At the same time, as against a sculpture, the doll demands not contemplation but play. It serves as a certain stimulator provoking creativity[15].

Ambiguity of Acting Like any human being, an actor has in his everyday life some set of rather stable physiological and psychological personal properties: sex, appearance, timbre of voice, gait, temper, and so on. The acting involves it’s ability to create a second phase, a "role" phase, different from the original physiological and psychological nature of the actor. In other words, a bimodal "actor-role" state created may be compared with ambiguous patterns, for instance, the pattern where we see in turn "young girl" or "old lady". One may say that in this case young girl will "play the role" of old lady and vice versa. In acting, one can observe the existence of two polar types of actors:

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

1) An actor as a bright, brilliant individuality, eccentric person with the original appearance, and so on (Alain Delon, Arnold Schwarzenegger). It is rather easy to make a parody of such actors; 2) An actor with prominent outstanding abilities for transformation and reincarnation (Laurence Olivier, Alec Guiness). In that case, it is very difficult to make a parody. Yu. Lotman note, that in the cinema more, than at the theatre the spectator sees not only role, but also actor [15, p.658]. Observing play of the famous actor we alternately focus our attention or on guise (image) of actor familiar to us on other movies, or on peculiarities of a role, which the actor plays. Such oscillations of attention is the reason, that with the reference to acting we use a word “play”. In the case of acting the prototypes are, for instance, "Laurence Olivier" (the image of actor) and "Othello" (the image of character). Therefore, according to the common law of perception of ambiguous patterns, the oscillation of our attention takes place, and we see in turn either an actor or his role. Just as like bimodal nature of sculpture art begets plots about animated statue, bimodality of actor art gives a possibility to use a phase transition called "character invasion" for plot development [16]. The main hero of the film "A Double Life" plays the role of Othello for so long time that it begins to affect to his psychic activity, making him more and more jealous of his beloved, and like the stage character, he strangles her and then kills himself. In the film "Jesus of Montreal" the actor playing the role of Jesus Christ becomes transformed into a Christ-like figure [16]. As a rule, all bimodal metastable states in the end of movies turn into stable, onemodal states as a result of bifurcation.

48

Igor Yevin

Conclusion In ordinary speech, and especially in scientific communication, in general we try to avoid ambiguity. By contrast, in humor, one of the aims is to create ambiguous situations to provoke laughing. And in art as a whole ambiguity is an indispensable, necessary part. “…art is supposed to have multiple meanings. It self-defeating to increase one aspect of meaning. The more a single meaning dominates a work, the less it is a work of art. Something that has one and only one meaning – no matter how interesting or important that meaning is - is no longer a work of art” [17, p.46]

Synergetics and the theory of complexity revealed that the human brain operate near unstable point, because only near criticality the human brain could create new forms of behavior. Ambiguity in art is an important tool maintaining the brain near this unstable, critical point.

References [1] [2] [3] [4] [5] [6]

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

[7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17]

P. Kruse, M. Stadler, Ambiguity in Mind and Nature.: Multistable Cognitive Phenomena. Springer, Berlin, 1995. N.L.Legothetis. Vision: A Window on Conciousness. Scientific American. November, 1999 pp.69-75 T. Poston, I. Stewart, Nonlinear Model of Multistable Perception. Behavioral Science., 23 (5), 1978, 318-334. I.N. Stewart, P.L. Peregoy, Catastrophe Theory Modeling in Psychology. Psychological Bulletin, 94(21), 1983, 336-362. L.K. Ta'eed, O. Ta'eed, J.E.Wright, Determinants Involved in the Perception of Necker Cube: an Application of Catastrophe Theory. Behavioural Science, 33, 1988, 97-115 Osgood, Ch., Suci, G., Tannenbaum P., 1958, The Measurement of Meaning, University of Illinois Press H. Haken, Principles of Brain Functioning. Springer, Berlin, 1996. W. Wildgen, Ambiguity in Linguistic Meaning in Relation to Perceptual Multistability. In P.Cruse and M.Stadler [1]. E. Gombrich, The Story of Art. Phaidon, New York, 1995. G. Caglioti, Dynamics of Ambiguity. Springer Berlin, 1992. S.Zeki. Inner Vision. Oxford University Press. 1999 Reisner B. and Kapplow H. Captions Courageous. Abeland-Schuman, 1954 V.B.Shklovsky. Tetiva. Moscow, 1967.(In Russian) A.N. Kolmogorov, Theory of Poetry. Moscow, Nauka, 1968, 145-167 (in Russian) Yu.Lotman. About Art. St Petersburg, 1998 (In Russian) Neuringer C. and Willis R. The Cognitive Psychodynamics of Acting: Character Invasion and Director Influence. Empirical Studies of the Arts. v.13, N1, 1995 p.47 C.Martindale. The Clockwork Muse. Basic Books. 1990.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 49-51

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 4

DOES THE COMPLEXITY OF SPACE LIE IN THE COSMOS OR IN CHAOS? Attilio Taverna*

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Painter The art of painting, as we have already known for a long time, is first and foremost an aesthetic inquiry on the nature of space. It’s easy to understand why. The state of being of an aesthetic experience such as a painting, always needs an extension, sometimes of a surface, often of a double dimension, always of some kind of phenomenology of space. Here is the ultimate reason why. In our modern times, even in the case of drawing the structure of a chip, or when we shoot a real event with a video camera, we use an extension as a support. So to say we are using an idea of space already known to us, in the same way in which we use the net. We can use it only because there’s an idea of pluri-dimensional space in it that we identified as fundamental: cyber-space, precisely/exactly. But what is the space? Can we say that we know it for sure? Even Plato in the Timeo’s dialogue, the big Greek cosmogonic tale of 25 centuries ago, said that space has a bastard nature. He also admonished that space is the condition of possibility of being of all phenomena but at the same time it cannot become a phenomenon. That means that space is the conditio sine qua non for a phenomenon to appear but it cannot appear in the way phenomenon do. That’s the reason why it has a bastard nature: it allows appearance but doesn’t appear. So now, it becomes clear how the idea of space is something immersed in the ontological oscillation, which is something irrepressible. As we’ve already seen, it’s space’s own nature that allows the decline, in the visible manifestation, of the horizon of beings. This nature of space is the condition of possibility of every phenomenon’s apparition. 25 centuries after Plato, the German philosopher Immanuel Kant, attempted to solve the enigma of the nature of space in his "Critique of the Pure Reason" said: "the space is not an empirical concept, drawn by external experiences… it is, instead, a necessary representation a *

E-mail address: [email protected]

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

50

Attilio Taverna

priori, which serves as a fundament to all the other external intuitions". He would conclude by saying that the intuition of space is the original shape of sensibility. Space and time are the pure forms, a priori, of sensitivity. And when, at the beginning of the last century, space and time joined together thanks to physics-mathematics in only one quadri-dimensional being called spacetime, the aesthetical experience of painting became, as a result of physics, aesthetical search on the nature of spacetime. That’s all about philosophy. But talking also about science with an observation from Albert Einstein on the genesis of the theory of relativity, we can understand how the true nature of space is inevitably implicated with the formal and ideal systems which we call geometry. Albert Einstein, in fact, would say: suddenly I realized that geometry had a physical meaning…. After this consideration and intuition of the great physicist, who had revolutionized the knowledge of reality, how can we not ask ourselves about the meaning of the ideal forms of geometry, such as the curvature of spacetime, for instance, - which is a geometrical form produced by men- clash/coincide with one of the fundamental forces of nature, the gravitational force? Even better, gravity is the curvature of spacetime. And so? How can’t we wonder also about another question: What’s the form in ontology? Art is not, and cannot be considered, unrelated to this question. And its own history testifies and documents this fact. Art has conducted this query maybe since the beginning of man’s history. And painting realizes a vision of this possible question on the nature of spacetime, its possible form, before being any other form of aesthetic query, as we have already said. My aesthetical experience fed on this query as well. The nature of space, in my opinion, is a kind of chromatic polyphony of ideal and formal opportunities, not necessary axiomatic, as the systems of Euclidean geometries and not-Euclidean, but conceived as ideal opportunities of never-ending geometries existing in an unfinished space. We can’t forget that while Albert Einstein was conceiving the theory of relativity and made us aware of the physical meaning of geometrical forms, philosophy was analyzing with rigour the formal and primary idealizations of geometry. We have to remember Edmund Husserl’s studies. He is another popular German philosopher, who, at the beginning of last century, thought geometry was an ideal and eidetic dimension, defining it as the visual language of idealities non-in-chains concepts. So to say that the whole phenomenology was subjected to the causal principle, while this formal and ideal dimension called geometry was not subjected to it. From now on we could think of space as a dimension hanging on the greatest freedom of thinking and form joined together that man has ever possessed. The complexity of any possible notion of space becomes dizzing. And modernity took charge of this demonstration. Any other possible example would be superfluous. At the same time the mathematic notion of chaos contributed to change the idea of space which was crystallizing in the geometric systems consolidated/established axiomatically. We can add something else: if by chaos we mean the inability to foretell the future evolutions of every kinetic non-linear system, with the not completely known conditions of the initial system, we have to admit that the unpredictability of every future evolution of every system is totally open to a description made by endless ideal and unknown formality, from a formal and geometrical core of unpredictable descriptions.

Does the Complexity of Space Lie in the Cosmos or in Chaos?

51

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

So what comes to light as geometrical language, the "visual language of idealities non-inchains concepts ", is not a knowledge of the past, but is something ineluctable, a necessary knowledge of the future. We have also to underline, as useful indication, that in the theoretical contemporary physic some theories are elaborated – for instance the one of the superstrings - and these theories need many dimensions of space to explain their mathematical compatibility and their theoretic correctness. To say that the reality of spacetime is not exactly what happens in front of our senses and that we are used to see and express everyday. Even if art doesn’t want to find the foundation of the world, because this is not in his epistemic status and this result belongs to the purpose of hard sciences, physic for example, nevertheless art carries out the world as a fundament. That’s its vocation. That’s its destiny. And if the reality of spacetime gives up as foundation, so to say as the condition of possible apparition of any possible apparition, should art be excluded from this query on the foundation? No, centairly. That would be impossible. Great narrations of aesthetics would never stop to question everything, even better, on the everything, because the specific task of art is aesthetic query to the very limit of possibility. Since ever. Conclusion and question: if reality, the reality of spacetime, is possible to be described in the physic-mathematic sciences by an idea of a very complex space multi-dimensionality that escape any visibility and any chance of daily visibility, Who can see these possible concepts of space that are the real space described by science- if not an aesthetical experience that found its foundation on the artistic praxis right on this lyrical query on the nature of spacetime?

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 53-71

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 5

CRYSTAL AND FLAME/FORM AND PROCESS: THE MORPHOLOGY OF THE AMORPHOUS Manuel A. Báez* Form Studies Unit, Coordinator, School of Architecture, Carleton University, Ottawa, Ontario K1S 5B6 Canada

“Philosophy is written in this enormous book which is continually open before our eyes (I mean the universe), but it cannot be understood unless one first understands the language and recognizes the characters with which it is written. It is written in a mathematical language and its characters are triangles, circles, and other geometric figures. Without knowledge of this medium it is impossible to understand a single word of it; without this knowledge it is like wandering hopelessly through a dark labyrinth.”

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Galileo Galilei, “The Assayer” (1623) [1] “Why is geometry often described as “cold” and “dry?” One reason lies in its inability to describe the shape of a cloud, a mountain, a coastline, or a tree. Clouds are not spheres, mountains are not cones, coastlines are not circles, and bark is not smooth, nor does lightning travel in a straight line. More generally, I claim that many patterns in Nature are so irregular and fragmented, that, compared with Euclid ─ a term used in this work to denote all of standard geometry ─ Nature exhibits not simply a higher degree but an altogether different level of complexity. The number of distinct scales of length of natural patterns is for all practical purposes infinite. The existence of these patterns challenges us to study those forms that Euclid leaves aside as being ‘formless,’ to investigate the morphology of the ‘amorphous.’ Mathematicians have disdained this challenge, however, and have increasingly chosen to flee from nature by devising theories unrelated to anything we can see or feel.” Benoit B. Mandelbrot [2] “We are living in a world where transformation of particles is observed all the time. We no longer have a kind of statistical background with permanent entities floating around. We see that irreversible processes exist even at the most basic level which is accessible to us. *

E-mail address: [email protected]

54

Manuel A. Báez Therefore it becomes important to develop new mathematical tools, and to see how to make the transition from the simplified models, corresponding to a few degrees of freedom, which we have traditionally studied in classical dynamics or in quantum dynamics, to the new situations involving many interacting degrees of freedom.” Ilya Prigogine [3]

Abstract This paper presents the work and research produced through an on-going architectural project entitled The Phenomenological Garden. The project seeks to investigate the morphological and integrative versatility of fundamental processes that exist throughout the natural environment. Work produced by students in workshops incorporating educational methods and procedures derived from this research will also be presented. This evolving project is a systematic investigation of the versatile and generative potential of the complex processes found throughout systems in Nature, biology, mathematics and music. As part of the Form Studies Unit in the School of Architecture at Carleton University, the work seeks to investigate how complex structures and forms are generated from initially random processes that evolve into morphologically rich integrated relationships. The morphological diversity revealed by this working and teaching method offers new insights into the complexity lurking within nature’s processes and bridges the theoretical gap between Galileo Galilei’s conception of nature, as revealed above, and the modern theories of Chaos and Complexity as exemplified by Benoit Mandelbrot and Ilya Prigogine. This working process also offers insights into the conceptual and philosophical aspirations of such key central figures as Antoni Gaudi, Louis Sullivan, Frank Lloyd Wright, and Buckminster Fuller in the early formative period of modern architecture, and more recently, the architect/engineer Santiago Calatrava. The implications of these developments are relevant to the study of morphology as well as to the field of architecture at a time when it is addressing the concepts and themes emerging out of our deeper understanding of dynamic and complex phenomena in the physical world.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Introduction Through the aid of modern computer visualization and analyzing techniques, we have recently acquired deeper insights into the ways energy is interwoven into dynamic systems and structures of startling beauty and versatility that often recall the patterns and motifs found throughout the natural and man-made environment. The elemental cellular patterns that emerge from these processes inherently contain information and are themselves dynamic events-in-formation. An understanding and appreciation of our innate relationship with this phenomenon can be achieved through hands-on systematic “readings” of the complex characteristics of these emergent cellular units and their assemblages. These fertile, self-organizing and regulatory systems and patterns inherently exist within and generate the rich realm of natural phenomena. Simultaneously, they are also composed of and generate elemental inter-active relationships that gradually evolve into versatile integrative systems. When the versatility and generative potential of these systems and their interrelated cellular patterns are systematically analyzed, they can yield new insights into the emergence of complex morphological structure and form. The intrinsic nature of the patterns generated by these dynamic processes reveals that they are cellular configurations of highly ordered relationships. Through these apparently static patterns and stable forms flow the highly dynamic undulations of an energetic process.

Crystal and Flame/Form and Process

55

These emergent complex networks are fluently encoded patterns of potentiality offering a multitude of possible or alternative “readings.” The cellular units comprising these patterned morphogenetic inter-activities innately contain the intrinsic attributes of the versatile processes that generate them. We are inextricably part of and surrounded by this rich and dynamically complex matrix of natural phenomena. The probing of the inherent nature of this pro-creative matrix can lead to an insightful understanding of the reciprocal relationship between matter, developmental processes, growth and form. Rich and exciting educational methodologies are also offered through new procedures and techniques that would inherently allow for intuitive learning through self-discovery.

Background Galileo Galilei’s metaphor of the book of nature reflects the new philosophical direction of his time while, simultaneously, following an ancient tradition regarding the nature of the physical universe. He emphasizes the importance of understanding the nature of the characters through which the language of this book is written. At the time, it was believed that all-encompassing scientific knowledge could be achieved solely through the quantifiable and visual aspects of the material world and its organizing parts. Galileo’s vision reflects the influence of the work of Plato, most notably his Timaeus where we find an emphasis on the primary importance of the elementary geometric units or ideas behind the material world. This was in sharp contrast to the Aristotelian philosophy dominating the Western world up until the Scientific Revolution in the sixteenth and seventeenth centuries. Prior to this, the world was envisioned as a living organism where spirit, substance and form were inextricably interrelated. This new mechanistic vision culminates with René Descartes’ analytic method and eventually Isaac Newton’s grand synthesis of Newtonian mechanics. This vision would prevail and dominate Western science until the early part of the twentieth century. The first major influential challenge to this mechanistic vision came in the late eighteenth and nineteenth centuries from the Romantic Movement in literature, art and philosophy.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Primordial Seeds The Romantic Movement, as exemplified by J. W. von Goethe, had a profound influence on the American architect Louis Sullivan and, subsequently, Frank Lloyd Wright through the strong German cultural presence in late nineteenth century Chicago, the transcendentalism of Ralph Waldo Emerson, and the writings of the philosopher Herbert Spencer. For Sullivan and Wright, the creative process was seen as a transcendental experience similar to natural growth and development. Reminiscent of Goethe’s botanical observations, Sullivan made references to “the germ of the typical plant seed with its residual powers.”[4] In the primary geometric figures, Sullivan saw primordial seeds with “residual power” to grow and generate organic forms. He illustrated the development of his own ornament through the morphological transformations of these primary units (see Figure 1). To Sullivan these were the primary generative units of a “plastic” and “fluent geometry” containing “radial energy” and “residual power” capable of projecting outwards or inwards through the inherent “energy lines” or axes of the units.

56

Manuel A. Báez

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 1. Louis Sullivan [4], Manipulation of forms in plane geometry.

This dynamic, generative and comprehensive vision of nature inspired the work and ideas of both Sullivan and Wright. They both incorporated a basic unit system of working that would undergo systematic morphological permutations, limited only by the designer’s imagination. Wright would state: “All the buildings I have ever built, large and small, are fabricated upon a unit system—as the pile of a rug is stitched into the warp. Thus each structure is an ordered fabric. Rhythm, consistent scale of parts, and economy of construction are greatly facilitated by this simple expedient—a mechanical one absorbed in a final result to which it has given more consistent texture, a more tenuous quality as a whole.”[5]

Louis Sullivan and Frank Lloyd Wright both envisioned an organic, versatile, vibrant and integrative design process. Recent developments in modern science and in the early part of

Crystal and Flame/Form and Process

57

the twentieth century reveal a similar conception regarding the complex nature of the physical world.

A. Complex Tissue of Events During the early part of the twentieth century, a fundamental conceptual shift was underway regarding our comprehension of the physical world and the principles involved in its organizing and structuring processes. Fundamentally, the nature of matter was revealed to consist of an irreconcilable yet intrinsic paradoxical contradiction. At the heart of this dilemma was the nature of form, structure, developmental organization, and emergent patterns. Measurable or numerically quantifiable form and position were inextricably linked and reciprocally related to the highly complex behaviour of the dynamic inter-actions of energy. Subsequently, it was revealed that through these highly complex processes emerge three-dimensional networks or patterns of probable or possible alternatives. According to the German physicist Werner Heisenberg: "The world thus appears as a complicated tissue of events, in which connections of different kinds alternate or overlap or combine and thereby determine the texture of the whole." [6]

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The intrinsic nature of this dynamic conception consists of the realization and comprehension of patterns as highly complex networks of organizational texture and potentiality. Understanding these inherent characteristics would provide the necessary insights in order to probe deeper into this new paradoxical conceptualization. The contradictory nature of matter is a recurring theme that’s encountered when contemplating the relationships between substance and form, subject and object, as well as unity and multiplicity. In the history of biology, this ancient dilemma is found to be inextricably associated with the understanding of the forms of living organisms and their growth or developmental processes. In physics and biology, at the most elementary level, nature’s processes are essentially the inter-relationships between things in a myriad of different orders of magnitude. We are inextricably part of and surrounded by Heisenberg’s encoded “tissue of events.” The probing of the inherent nature of this fluently textured tissue, can lead to an insightful understanding of the nature of patterns and their correlation with matter, developmental processes, growth and form. In the words of Gregory Bateson: “We have been trained to think of patterns, with the exemption of those in music, as fixed affairs. It is easier and lazier that way but, of course, all nonsense. In truth, the right way to begin to think about the pattern which connects is to think of it as primarily (whatever that means) a dance of interacting parts and only pegged down by various sorts of physical limits and by those limits which organisms characteristically impose.” [7]

This dynamic conception envisions emergent networks as fluently encoded records or events that contain the in-forming and expressive potential of their generative processes. Modern computer visualization and simulation techniques are providing deeper insights into the richness of these networks that are embedded within Heisenberg’s “tissue of events” and Bateson’s “dance of interacting parts.” More profound fundamental insights are offered into the earlier developments regarding the nature of the physical world. Again, within the realm

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

58

Manuel A. Báez

of complex phenomena, we encounter “objects” or confined spatial forms that “attract” or resolve the dynamic inter-actions of energy. The emergent spatially confined activity is the mediation or resolution of the conflicting inter-actions. These processes reveal a wealth of detail and self-similarity at almost infinite scales of organization. Revealed in greater depth within this complexity is the fundamental role of the relationships between interacting parts in different orders of magnitude along with their emergent patterns and behaviour. In biology, a fundamental characteristic of these complex systems is that there is permanence to the overall macro behaviour while, simultaneously, the constituent parts are continuously dying out and being replaced. The human body is one of these complex systems similar to ant colonies or beehives. Hundreds of different cell types make up the overall complexity of the body. Approximately 75 trillion of these cells are actively at work in our body. In a matter of seconds, thousands of these cells have died and billions have been completely replaced within a week. This high turnover rate does not affect our overall conscious awareness of a “permanent” body. Contained within each cell nucleus is the entire genome for an organism with individual cells reading only a small portion of that information. The interactive context within which the individual cell finds itself, will determine the tiny portion of information that it will read. Through this multi-cellular communication process, cells self-organize into more sophisticated structures. Cells can detect the overall state of their surroundings as well as any changes within that state such as gradient fluctuations. Through this process, cells eventually self-organize into complex collectives leading to more complicated and sophisticated interactions. Throughout this decentralized process, local interactions and communication leads to the emergence of coordinated collective behaviour at different levels or scales of interactivity. We find other complex systems, forms and structures lurking within vastly differing scales of observation. Within the vast expanse of outer space, we encounter dynamically organized operations of light energy that remind us, through its spiral structures, of forms and patterns lurking within our immediate environment. The efficiency and incredible adaptability of this elemental form is further revealed through its use by nature in the highly versatile double-helix structure of DNA. Other dynamic and complex patterns can be generated through vibrations in a liquid or a fine powder and when a dense liquid is evenly heated in a pan. In all of these examples, the dynamic events activated within the medium resolve themselves or eventually mediate into resonant, highly charged and encoded networks of energy. Within these potent patterns of phenomenal inter-activity and their cellular units, we encounter a correlation between “stable” form and dynamic inner structure. This interrelationship between scales and between matter, process, and form, found both in physics and in biology, is not just encountered within the realm of appearances. D’Arcy Thompson was well aware of this and describes the quest to understand this interrelationship as “the search for community of principles or the essential similitudes.”[8] Most essential regarding such a quest, the anthropologist Gregory Bateson reminds us, is “the discarding of magnitudes in favor of shapes, patterns, and relations.”[9] Within the realm of the organizing principles of integrated, highly adaptable and structured relationships, we encounter scaleless order or, perhaps more significantly, a multitude of possible scales or magnitudes.

Crystal and Flame/Form and Process

59

Works-in-Process

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

My work and research has been inspired by the broad implications of the developments described above. Multiple-exposure photography was used in the initial phases of the work as a way of generating a series of images entitled Multiples. The resulting improvised images would emerge from the purely visual intermingling or blending of a repeated image or module (see Figure 2). Subsequently, a more physical, materially based and dynamic process was required and eventually conceived through the use of the rotary motion generated by a potter’s wheel. Intrinsic forms lurking within the spinning wheel’s spiral vortex were cast by securing a metal cylinder containing hot water and wax to the wheel. This process generated a series of forms reminiscent of seashells and biological shapes. Figure 3 shows two views of two of these wax forms. The potter’s wheel was also used to spin a suspended cotton string into initially stable and sequential wave-formations that become turbulent at higher speeds. This project, entitled Ariadne’s Thread/Rumi’s Ocean [10], was inspired by scientific investigations of dynamic phenomena. It was recorded from different vantage points, generating a wealth of morphological formations and generative working procedures, as well as insights into the correlation between reference frame and perception. Figure 4 shows several of the forms generated with the spinning string. The whirling string shown on the left is spinning at a rate whereby it casts shadows of itself on its generated surface.

Figure 2. Manuel A. Báez, Multiple #1.

60

Manuel A. Báez

Figure 3. Manuel A. Báez, Wax Forms cast with a potter’s wheel, 7" high x 3" wide.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 4. Manuel A. Báez, Ariadne’s Thread/Rumi’s Ocean, String & Potter’s Wheel, 1993-present. Left & upper right: String Formations; lower right: Collaged Motion Drawings; middle: Calligraphic String Drawing; middle right: Multiple Exposure String Drawing or “Ariadne’s Ball of Thread.”

Through extensive research and analysis of the work generated from the projects described above, and the conceptual developments that inspired them, the dynamic versatility of several elemental forms were explored by incorporating a flexible joint as part of an assembling process. These elemental relationships can be found within the inner structure of nature’s resolutions to dynamic phenomena. The underlying woven stress patterns found superimposed and interacting within the inner structure of bones, is a biological example of one way nature resolves a dynamically complex structural situation. Elemental shapes, such as a triangle, square, pentagon, etc., were considered as dynamic relationships instead of to static diagrams. The joints consist of two bamboo dowels joined together with rubber bands, thus allowing for a high degree of flexibility. Through a variety of different arrangements of these joints, very versatile cellular units have been conceived and their form generating potential explored through the construction of cellular membranes or fabrics. The flexibility of the joints and their three-dimensional relationships, both within an individual cell and throughout the cellular membrane, generates a wealth of forms and structures through the emergent transformative and organizing properties of the integrated assembly. These properties recall and re-generate the inherent characteristics of the natural phenomena that inspired their conception.

Crystal and Flame/Form and Process

61

The Garden of Phenomeno-logical Paths

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The most extensive exploration incorporated into the Phenomenological Garden project has been that of a square geometric relationship. Gradually, it becomes apparent that this is an extremely versatile relationship between joints. The cellular membrane is constructed with 12" and 6" bamboo dowels and rubber bands. The upper left-hand corner of Figure 5 shows the fabric along with several improvised studies. The upper right-hand corner shows an inherently coiling structure that’s approximately 30 feet in overall length and 2 feet wide. The forms and structures that can be discovered and developed through the process will be determined by how the initial fabric is probed and segmented into its inherent patterns. As stated above, the three-dimensional joint relationship, as an integrated assembly, contains and is inter-active in-formation. What can be revealed from this information depends on the methods and/or means of inquiry. The encoded information or potentiality has a multitude of possible readings or interpretations. Through ones increasing experience and familiarity with the working process, more expressive forms and intricate structures can be conceived. One literally feels the stresses being worked on and with, along with the inherent in-forming potential of the membrane. This is a random exploration of the interactions without any preconceived goals. This type of exploration allows for the discovery of unanticipated patterned arrangements and their resulting interactive emergent behaviour. The resulting pattern detection and subsequent “readings,” allow for the development of more sophisticated coordination and regulated structuring. Sensually fluid curves begin to emerge, as well as very organic or biological forms and structures. The experience is that of a process whereby one feels, follows and flows with, while guiding the versatile form generating properties of the dynamic relationship.

Figure 5. Manuel A. Báez, Suspended Animation Series, 1994-present. Form Studies with square cellular units, 12" and 6" bamboo dowels joined together with rubber bands. Upper left-hand corner shows a portion of the membrane used throughout all fabrications shown in figures 5, 6 and 7.

62

Manuel A. Báez

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The sculptural forms shown in the upper right-hand corners of Figures 5, and 6 are made from the same coiling structure. The inherent properties and versatility of this structure has been explored by uncoiling and re-arranging it into different configurations (see Figure 5, lower right). Again, the organic looking forms and structures are all generated from the emergent properties of the assemblies. Figure 6 shows an installation done at Cranbrook Academy of Art in Bloomfield Hills, Michigan, USA. By then, the fluent expressiveness of the fabric and working process, along with its possibly limitless capabilities, had become apparent. The installation was part of a symposium that I conceived and was invited to organize at Cranbrook Academy of Art for the Sybaris Gallery in Royal Oak, Michigan. The symposium, entitled Metaphoric Interweavings, explored the interrelationships and similarities between weaving, musical composition and architecture through the use of a modular compositional process: artist Lissa Hunter lectured on her work, basketry and weaving; classical pianist Marina Korsakova-Kreyn gave a lecture/performance on the intricate structure of musical compositions by J. S. Bach; and professor of architecture Gulzar Haider lectured on the use of muqarnnas as modules in spatial transformations in Islamic architecture. Mugarnnas is a system of projecting niches used for spatial transition zones and for architectural decoration.

Figure 6. Manuel A. Báez, Phenomenological Garden, Installation for the Metaphoric Interweavings Symposium at Cranbrook Academy of Art, Bloomfield Hills, Michigan, USA, 1998. Upper right: 4' 6" high sculptural form, lower right: reflected ceiling view of the installation through the mirrored central table (lower left).

The installation in Figure 6 initiated the Phenomenological Garden project. It was entirely constructed using the same square cellular unit and membrane shown in Figure 5.

Crystal and Flame/Form and Process

63

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Two supporting columns are gradually transformed into an intricately patterned ceiling structure. The majority of the patterns that emerged were unconsciously assembled and a rich variety of them are revealed as one walks around the installation or looks into the mirrored central table (Figure 6 lower right). A different vantage point will reveal an entirely different pattern, at times familiar, but quite often completely unexpected. As the project has evolved, the multiplicity of shadows cast by these constructions has become increasingly more relevant to the theme of the work. They have added another layer to the multiple readings and interpretations. Figure 7 shows an installation at the Network Gallery of Cranbrook Academy of Art. The shadows played a major role in this installation along with the three-dimensional sculptural possibilities of the working process. A series of improvised sculptural weavings and freestanding structures cast their shadows on the walls and floor of the Gallery. Again, different vantage points reveal different aspects of the woven structures.

Figure 7. Manuel A. Báez, Phenomenological Garden, Installation at the Network Gallery of Cranbrook Academy of Art, Bloomfield Hills, Michigan, USA, 1999. Improvised sculptural weavings and freestanding structures constructed with the membrane shown in Figure 5.

The Crossings Workshop The Phenomenological Garden is a project that has been evolving and will continue to do so as the explorations develop. Other cellular joint relationships have been studied along with their emergent properties. Figure 8 shows some of the work produced by students in my Crossings Workshop at Carleton University. The Workshop incorporates the educational

64

Manuel A. Báez

potential of the research and work as a way of introducing the students to the rich potential of the working process and the developments that have inspired its conception.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 8. Crossings Workshop Suspended Animation Series, Cellular Form Studies. Works by Carleton architecture students: Mariam Shaker, Diana Park, Sherin Rizkallah, Daniel Cronin and Sharif Kahn.

The left side of Figure 8 shows a structure constructed using a square cellular unit. By suspending it from the ceiling, the gradual effect of gravity is clearly demonstrated in the subtle, progressive undulations of the structure. To the middle and lower right of this structure are two different arrangements of the same structure constructed with a seven-sided (heptagonal) module. This structure is also shown in Figure 9 and is particularly interesting because, through different configurations of the same structure, the diversity of possible forms is clearly shown. Equally interesting and diverse are the organic looking shadow “drawings” shown in Figure 9. In the upper right-hand corner of Figure 8, are two other structures constructed with a square cellular unit and, again, they clearly demonstrate the different possibilities contained within the same cell. On the lower right-hand corner is a structure constructed using a five-sided (pentagonal) cellular unit. The numerous intrinsic assembling procedures lead to unexpected overall patterns and dynamic arrangements that generate new and diverse developmental directions for the assembling process.

Crystal and Flame/Form and Process

65

Figure 9. Crossings Workshop Suspended Animation Series, Cellular Form Studies and Shadow “drawings” (heptagonal cellular units). Work by Diana Park.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

An Intermingling of X, Y, and Z Co-ordination The cellular unit shown in Figure 10 is constructed with 12" and 5" bamboo dowels that are joined together, again, with rubber bands. The unit is composed of three surfaces (or planes) at right angles to each other with each surface being defined by four 12" dowels assembled into a grid of two pairs at right angle to each other and four 5" dowels, one at each end of the 12" pairs (Figure 10, lower right). The three surfaces have a high degree of transformability due to the flexibility of the joints and each surface defines one of the X, Y and Z coordinate directions in three-dimensional space. Each surface can fully collapse along the two orthogonal diagonals of the assembled grid. Individually, each surface can fully collapse along the two orthogonal diagonals of the assembled grid. Three-dimensionally, this cubic cellular unit (or module) is composed of multiple “interacting degrees of freedom” through the combination of 42 flexible joints. From another perspective, this complex intermingling is also the interactions of the three flexible hyperbolic paraboloids within the three-dimensional assembly. Figures 11 and 12 show several configurations that can be developed from this dynamic interplay.

66

Manuel A. Báez

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 10. Crossings Workshop Suspended Animation Series, views of X, Y & Z Coordinates Cellular Unit: Three intersecting planes at right angles to each other. Lower right: clearly shows one of the planes with the central diagonal edges of the other two. Upper right and lower left: show views through the four diagonals of the cubic assembly.

A

B

C

D

Figure 11. Crossings Workshop , X, Y & Z Coordinates Cellular Unit and several of its basic transformations. A: The Cellular Unit. B: Flattened assembly along one of the four diagonals of the cubic assembly. C: Collapsed assembly centered around one of the four diagonals. D: Collapsed X, Y and Z axes with 5" dowels removed.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Crystal and Flame/Form and Process

67

Figure 12. Crossings Workshop, different stages of the cellular unit shown in Figure 11 as it completely collapses into the X, Y and Z axes (upper left and right) and gradually expands into a tetrahedron (from left to right starting from the top).

Figures 13, 14 and 15 show several forms and structures that can emerge as the assembling process gradually evolves into more complex configurations. Figures 13 shows two axial views of the same construction. This particular assembling process generated a dodecahedron that was not preconceived nor initially anticipated. Cellular units (as shown in Figure 11, left side) were assembled together using their inherent interacting properties as the guiding principles. Within the resulting three-dimensionally dynamic pattern of the form one can discern the complex interweaving of the rich geometric properties of the dodecahedron:

68

Manuel A. Báez

cubes, tetrahedrons, octahedrons, icosahedrons and golden rectangles (to name a few) in a reciprocally complex relationship. Several of these shapes can be discerned in the two views provided. The left side of Figure 14 shows another construction generated through the same process as in Figure 13 and also reveals the same level of complex multilayering of forms. The different modifications to the original unit in Figure 10 lead to the emergence of totally different complex patterns and dynamic properties.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 13. Crossings Workshop Suspended Animation Series, two views of the same construction using the cellular unit shown in Figure 10. The construction is a dodecahedron that emerged from the assembling process. Throughout the structure and the generated patterns one can discern the cubes, tetrahedrons, octahedrons and icosahedrons that are intrinsically embedded within the dodecahedron.

Figure 14. Crossings Workshop Suspended Animation Series, Cellular Constructions. Left: Constructed with the same cellular unit as in Figure 13 and exhibits the same properties. Right: Constructed using a variation on the cellular unit used in Figure 13. Different patterns are revealed throughout these constructions from different points of view.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Crystal and Flame/Form and Process

69

Figure 15. Crossings Workshop Suspended Animation Series, Cellular Constructions. Upper left and right, by Dan Levin and Michael Lam, constructed with the cellular unit in Fig. 12; Upper left, with the units fully expanded and upper right, with the units almost fully collapsed. Middle left and right, by Michael Putman, Patrick Bisson and Rheal Labelle, with the cellular unit in Fig. 10. Lower left and right, by Ana Lukas, with the cellular unit in Fig. 10.

Conclusion In Six Memos for the Next Millennium, the Italian writer Italo Calvino offers us the following observations and advise:

70

Manuel A. Báez

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

“The crystal, with its precise faceting and its ability to refract light, is the model of perfection that I have always cherished as an emblem, and this predilection has become even more meaningful since we have learned that certain properties of the birth and growth of crystals resemble those of the most rudimentary biological creatures, forming a kind of bridge between the mineral world and living matter. Among the scientific books into which I poke my nose in search of stimulus for the imagination, I recently happened to read that the models for the process of formation of living beings "are best visualized by the crystal on one side (invariance of specific structures) and the flame on the other (constancy of external forms in spite of relentless internal agitation)." What interests me here is the juxtaposition of these two symbols, as in one of those sixteenth-century emblems . . . . Crystal and Flame: two forms of perfect beauty that we cannot tear our eyes away from, two modes of growth in time, of expenditure of the matter surrounding them, two moral symbols, two absolutes, two categories for classifying facts and ideas, styles and feelings. . . . I have always considered myself a partisan of the crystal, but the passage just quoted teaches me not to forget the value of the flame as a way of being, as a mode of existence. In the same way, I would like those who think of themselves as disciples of the flame not to lose sight of the tranquil, arduous lesson of the crystal.” [11]

The richness of nature’s processes challenges our imagination because of its complex simplicity. This paradox has inspired the work of J. W. von Goethe, Louis Sullivan, Frank Lloyd Wright and countless other creative individuals. Italo Calvino was also inspired by this tradition and was well aware of the modern developments in science. These developments, along with the history of science and its relationship with literature and philosophy, were a source of inspiration for his creative imagination. To Calvino, the Crystal and Flame symbolize the paradoxical and contradictory nature of matter as revealed to us in the twentieth century. This correlation between form and process, as well as, simplicity and complexity has been revealed to us periodically throughout history. “This is common to all our laws;” states the physicist Richard Feynman, “they all turn out to be simple things, although complex in their actual actions.”[12] Benoit Mandelbrot elaborates on this paradox and the complexity of fractal geometry: “The effort was always to seek simple explanations for complicated realities. But the discrepancy between simplicity and complexity was never anywhere comparable to what we find in this context.”[13] The work-in-progress presented here inherently addresses this fundamental paradox through an integrative working process. Such a process can offer new directions to the fields of morphology, architecture and other disciplines at a time when the ideas emerging out of our deeper understanding of complex phenomena are being embraced for conceptual inspiration. The way towards the rich realm of diversity, as nature shows us, is through simple fundamental rules that eventually lead to a paradox of constrained and versatile freedom.

References [1] [2] [3]

As quoted by Italo Calvino in (1999) Why Read the Classics?, New York: Pantheon Books. Mandelbrot B. B. (1983) The Fractal Geometry of Nature, New York: W. H. Freeman and Co. Buckley P. and Peat F.D., editors (1996) Glimpsing Reality: Ideas in Physics and the Link to Biology, Toronto: University of Toronto Press.

Crystal and Flame/Form and Process [4] [5] [6] [7] [8] [9] [10]

[11] [12] [13]

71

Sullivan L. (1924) A System of Architectural Ornament According with a Philosophy of Man’s Powers, New York: Eakins Press. “The Life-work of American Architect Frank Lloyd Wright,” (1965) Wendigen, New York: Horizon Press. Heisenberg W. (1958) Physics and Philosophy, New York: Harper Torch Books. Bateson G. (1980) Mind and Nature, New York: Bantam Books. Thompson D.W. (1992) On Growth and Form, Complete Revised Edition, New York: Dover Books. Bateson G. (1980) Mind and Nature, New York: Bantam Books. Ariadne is the mythological Greek guide to the labyrinth of chaos and the individual life. Jalai al-Din Rumi is the Great Persian mystic poet of the thirteenth century and the creator of the whirling, circular dance of the Mevlevi dervishes. Calvino I. (1988) Six Memos for the Next Millennium, Cambridge, Mass.: Harvard University Press. Feynman R. (1967) The Character of Physical Law, Massachusetts: The M. I. T. Press. Quoted in: Peitgen H., Jurgens H., Saupe D., Zahlten C. (1990) “Fractals: An Animated Discussion,” VHS/color/63 minutes, New York: Freeman.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Bibliography Bateson G. (1980) Mind and Nature, New York: Bantam Books. Buckley P. and Peat F.D., editors (1996) Glimpsing Reality: Ideas in Physics and the Link to Biology, Toronto: University of Toronto Press. Capra F. (1996) The Web of Life: A New Scientific Understanding of Living systems, New York: Anchor Books Doubleday. Calvino I. (1988) Six Memos for the Next Millennium, Cambridge, Mass.: Harvard University Press. Feynman R. (1967) The Character of Physical Law, Massachusetts: The M. I. T. Press. Heisenberg W. (1958) Physics and Philosophy, New York: Harper Torch Books. Johnson S. (2001) Emergence: The connected Lives of Ants, Brains, Cities, and Software, New York: Scribner. Mandelbrot B.B. (1983) The Fractal Geometry of Nature, New York: W. H. Freeman and Co. Peitgen H., Jurgens H., Saupe D., Zahlten C. (1990) “Fractals: An Animated Discussion,” VHS/color/63 minutes, New York: Freeman. Prigogine I. (1980) From Being to Becoming, San Francisco: Freeman. Prigogine I., Stengers I. (1984) Order out of Chaos, New York: Bantam Books. Sullivan L. (1924) A System of Architectural Ornament According with a Philosophy of Man’s Powers, New York: Eakins Press. Thompson D.W. (1992) On Growth and Form, Complete Revised Edition, New York: Dover Books.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 73-81

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 6

COMPLEXITY IN THE MESOAMERICAN ARTISTIC AND ARCHITECTURAL WORKS Gerardo Burkle-Elizondo* Universidad Autónoma de Zacatecas. Unidad de Postgrado II. Doctorado en Historia. Ave. Preparatoria s/n, Col. Hidráulica. CP 98060, Zacatecas, Zac. México

Ricardo David Valdez-Cepeda* Universidad Autónoma Chapingo. Centro Regional Universitario Centro Norte. Apdo. Postal 196, CP 98001, Zacatecas, Zac. México

Nicoletta Sala* Accademia di Architettura, Università della Svizzera italiana, largo Bernasconi 2 CH – 6850 Mendrisio, Switzerland

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Abstract It has been demonstrated that scribers, artists, sculptors and architects used a geometric system in ancient civilizations. There appears such system includes basically golden rectangles distributed in a golden spiral fashion. In addition, it is clear that we do not know the sequence in which the lines or pictures were originally traced or drawn. By this way, the artistic and architectural works can be considered as static objects and so they may be characterized by an inherent dimension. The aim of this paper is to introduce a description of the complexity presents in the Mesoamerican artistic and architectural works (e.g., tablets from Palenque and other sites, Maya stelae, Maya hieroglyphs, pyramids, palaces and temples, calendars and astronomic stones, codex pages, murals, great stone monuments, astronomic stones and ceramic pots). Our findings indicate a characteristic higher fractal dimension value for different groups of Mesoamerican artistic and architectural works. Results could be suggesting that Mesoamerican artists and architects used specific patterns and they preferred works with higher (1.91) box and information fractal dimensions.

Keywords: Archeology, Golden Figures, Mesoamerican Tablets, Stelae and Pyramids, Fractals, Fractal dimension. *

E-mail address: [email protected] E-mail address: [email protected] * E-mail address: [email protected] *

74

Gerardo Burkle-Elizondo, Ricardo David Valdez-Cepeda and Nicoletta Sala

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Introduction The scientific perception of reality has changed through the centuries. For example, the Baroque style liked a mathematical curve, the ellipse; in that time, the ellipse became popular and was used in physics, astronomy, engineering and art (Hilgemeier, 1996; Stierlin, 2001; Sala and Cappellato, 2003); so in the mind of a cultivated person, the planets traveled along perfect ellipses, Kepler’s laws, and people were certain about the stability of the solar system. However, it has been discovered that systems of orbiting bodies have rational proportions of orbital periods that become unstable sooner or later but this phenomena can be modeled for near future prediction taking into account our limited knowledge of the initial conditions. Contrary to this, with art produced by humans, there is no form to know the sequence in which the lines or pictures were originally traced or drawn. This means there are no equations or temporal information useful to characterize ancient artistic and architectural works when treated as complex systems. This means geometric analysis and mathematics used in art composition and design of buildings are not yet clearly elucidated, although at least some serious studies deserve be mentioned. Roman and Greek architects liked circles and golden rectangles Also, Egyptians used the an approximation of the golden rectangle in art, architecture and hieroglyphics (www.geocities.com/CapeCanaveral/Station/8228/arch.htm). Martínez del Sobral (2000) studied Mesoamerican art, sculptures, codex, and pyramids and urban architectural designs, and she have demonstrated the strong influence of golden measures on them, whilst de la Fuente (1984) pointed out Olmeca monumental heads were made under the basis of golden rectangles as harmonic units. These growing golden rectangles appear to be distributed following a golden spiral. In addition, both authors have demonstrated that in the prehispanic world, a system like this was used by scribers (named ‘tlacuilos’), artists, sculptors and architects making of it a standardized technique in composition, and these abilities and knowledge were transferred from one generation to another, like a tradition. By this way, the Mesoamerican artistic and architectural works can be considered as static objects (Miller, 1999; Stierlin, 2001), and so they may be having an inherent dimension. Therefore, the fractal dimension is an experimentally accessible quantity that might be related to the aesthetic of the pattern(s) of these works. Then it would be interesting to know if the artists and architects preferences were different for groups or types of work in the ancient Mesoamerican culture. In this paper, we present a fractal analysis of some Mesoamerican artistic and architectural works, and a comparison among them taking into account different groups or types of work.

Material and Methods To determine to degree of the complexity in the Mesoamerican arts, we collected 90 images (Table 1) of Mesoamerican artistic and architectural works by reviewing literature on archeology. From the 90 figures, 61 correspond to the Maya culture (MC) during late preclassic (300 b. C. to 250 a. C.), and early and late classic (250 to 700 b. C.) periods, developed at Mexican Chiapas and Yucatán states, and Guatemala and Honduras; 26 to the Aztec or Mexican culture (AC) during classic and epiclassic periods (300 to 1100 a. C.),

Complexity in the Mesoamerican Artistic and Architectural Works

75

developed at Mexican Central Highplains; two to the ancient Olmec culture (OC) developed from 1350 to 900 b. C., at Mexican (Veracruz, state); and one to the Toltec culture (TC), developed from 700 to 1100 a. C. corresponding to the first step of Nahua civilization, at Mexican Hidalgo State. All these 90 images have been digitized using a Printer-CopierScanner (Hewlett Packard®, Model LaserJet 1100A) and saved in bitmap (*.bmp) format on a Personal Computer (Hewlett Packard®, Model Pavilion 6651). Thereafter, these images were analyzed with the program Benoit, version 1.3 [9, 10] in order to calculate Box (Db), Information (Di), and Mass dimensions (DM), and their respective standard errors and intercepts on log-log plots. It was taken under consideration that the information dimension differs from the box dimension in that it weigths more heavily boxes containing more points. Figure 1 shows a partial fractal analysis realized by the program Benoit®.

Figure 1. Partial fractal analysis realized by the program Benoit® .

Box Dimension The box dimension is defined as the exponent Db in the relationship:

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

1 N(d) ≈ D d b

(1)

where N(d) is the number of boxes of linear size d (number of pixels in this study), necessary to cover a data set of points distributed in a two-dimensional plane. The basis of this method is that, for objects that are Euclidean, equation (1) defines their dimension. One needs a number of boxes proportional to 1/d to cover a set of points lying on a smooth line, proportional to 1/d2 to cover a set of points evenly distributed on a plane, and so on. Applying the logarithms to the equation (1) we obtain: N(d) ≈ −Db log(d).

Information Dimension In the definition of box dimension, a box is counted as occupied and enters the calculation of N(d) regardless of whether it contains one point or a relatively large number of points. The information dimension effectively assigns weights to the boxes in such a way that boxes containing a greater number of points count more than boxes with less points.

76

Gerardo Burkle-Elizondo, Ricardo David Valdez-Cepeda and Nicoletta Sala The information entropy I(d) for a set of N(d) boxes of linear size d is defined as N(d) I(d) = − ∑ m i log(mi ) i =1

(2)

where mi is: mi =

Mi M

(3)

where Mi is the number of points in the i-th box and M is the total number of points in the set. Consider a set of points evenly distributed on the two-dimensional plane. In this case, we will have Nd =

1 d2

(4)

and if it is considered that mi = d2. So equation (2) can be written as

[

( )]

I(d) ≈ − N(d) d 2 log d 2 ≈ −

[

]

1 = 2 d 2 log(d ) = −2 log(d ) . d2

(5)

For a set of points composing a smooth line, we would find I(d) ≈ −log(d). Therefore, we can define the information dimension Di as in:

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

I(d) ≈ −Di log(d).

(6)

In practice, to measure Di one covers the set with boxes of linear size d keeping track of the mass mi in each box, and calculates the information entropy I(d) from the summation in (2). If the set is fractal, a plot of I(d) versus the logarithm of d will follow a straight line with a negative slope equal to −Di. At the beginning of this section, we noted that the information dimension differs from the box dimension in that it weighs more heavily boxes containing more points. To see this, let us write the number of occupied boxes N(d) and the information entropy I(d), in terms of the masses mi contained in each box: N(d) = ∑ m i0 i

(7)

N(d) = −∑ m i log(m i ) i

The first expression in (7) is a somewhat elaborate way to write N(d), but it shows that each box counts for one, if mi > 0. The second expression is taken directly from the definition

Complexity in the Mesoamerican Artistic and Architectural Works

77

of the information entropy (1). The number of occupied boxes, N(d), and the information entropy I(d) enter on different ways into the calculation of the respective dimensions, it is clear from (7) that Db ≤ Di. The condition of equality between the dimensions is realized only if the data set is uniformly distributed on a plane.

Mass Dimension Draw a circle of radius r on a data set of points distributed in a two-dimensional plane, and count the number of points in the set that are inside the circle as M(r). If there are M points in the whole set, one can define the ‘mass’ m(r) in the circle of radius r as: m(r) =

M(r) . M

(8)

Consider a set of points lying on a smooth line, or uniformly distributed on a plane. In these two cases, the mass within the circle of radius r will be proportional to r and r2 respectively. One can then define the mass dimension DM as the exponent in the following relationship:

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

m(r) ≈ r D M .

(9)

In practice, one can measure the mass m(r) in circles of increasing radius starting from the center of the set and plot the logarithm of m(r) versus the logarithm of r. If the set is fractal, the plot will follow a straight line with a positive slope equal to DM. As the radius increases beyond the point in the set farthest from the center of the circle, m(r) will remain constant and the dimension will trivially be zero. This approach is best suited to objects that follow some radial symmetry, such as diffusion-limited aggregates. In the case of points in the plane, it may be best to calculate m(r) as the average mass in a number of circles of radius r. It can be shown that the mass dimension of a set equals the box dimension. This is true globally, i.e., for the whole set; locally, i.e., in portions of the set, the two dimensions may differ. Let us cover the set with N(d) boxes of size d, and let us define the mass, or probability, in the i-th box mi as: mi =

Mi M

(10)

where Mi is the number of points in the i-th box and M is the total number of points in the set. We can now write the average mass, or probability, in boxes of size d as m(d), the average mi in the N(d) boxes: m(d) =

1 n(d) 1 ∑ mi = N(d) i = 1 N(d)

(11)

78

Gerardo Burkle-Elizondo, Ricardo David Valdez-Cepeda and Nicoletta Sala

(the sum of all the masses mi is obviously one). As the operation of calculating the mass contained in a box of size d is the same as calculating the mass in a circle of radius r, we can write our definition of mass dimension (9) in terms of d rather than r: m(d) ≈ d D M

(12)

By using (4) and re-arranging terms, we obtain: N(d) =

1 d

DM

(13)

which is the definition of the box dimension; thus, the mass dimension equals the box dimension.

Results and Discussion

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In all the 90 cases a straight line was evidenced, so the three different approaches to estimate the fractal dimension works well. As an example, we show the plot to estimate the information dimension for ‘Coatlicue’, the Aztec god of life and death (shown in Figure 2).

Figure 2. Log-log plot for ‘Coatlicue’. It can be appreciated a straight line with a negative slope −Di = 1.906±0.006.

The calculated fractal dimensions are reported in Table 1. For all the 90 cases the fractal dimension values were high from a Db = 1.803±0.023 for the left and superior side of the ‘Vase of seven gods’ (MC, Group X), to a DM = 2.492±0.195 for the left side of the ‘Door to underworld of the Temple 11, platform’ at Copán (MC, Group I). This late case could be related to the Mayan vases, which are integrated in the Group X in Table 1, are less complex than the other figures and groups because they contain wider empty but painted rectangular or squared spaces. Certainly, there is unknown the sequence in which the lines were traced in those works having high DM values as the left and the superior side of the ‘Door to

Complexity in the Mesoamerican Artistic and Architectural Works

79

underworld of the Temple 11, platform’ at Copán (MC, Group I), which contains a lot of human like figures representing gods and ancestors but they are not concentrically distributed in a trapezoidal plane explaining its high DM value surpassing the dimension of the plane. In this figure the traces are in fact irregularly distributed which makes really a complex composition able to fill the trapezoidal plane, and this characteristic is common to other works from the same civilization an Aztec culture (Table 1) such as the whole and parts of the ‘Temple of foliated cross tablet’ (MC, Group I); the whole center east of the ‘Ball Game Tablet’ at Chichen-Itzá (MC, Group I); ‘Mural of the 4 Ages’ at Toniná (MC, Group I); the whole ‘Tablet of 96 Hieroglyphs’ at Palenque (MC, Group III); ‘Temple of the Cross, Door panel, Glyphs 2 and 14’ (MC, Group III); ‘Temple of the Sun, superior view’ (AC, Group IV); ‘Temple of the Sun’ at Palenque (MC, Group IV); ‘Pyramid of the Wizard’ at Uxmal (MC, Group IV); ‘Pyramid Temple’ at Tulún (MC, Group IV); ‘Palace of Hochob’ at Tabasqueña (MC, Group IV); ‘Dresden Codex, page 13b’ (MC, Group VI); ‘Borgia Codex, ritual 2, page 34’ (AC, Group VI); ‘Aztlán Annals’, page 3 (AC, Group VI); ‘Stela F’ at Quirigua (MC, Group VII); ‘Stela A’ at Copán (MC, Group VII); ‘Humboldt Disc’ (AC, Group VIII); ‘Huaquechula Disc’ at Puebla (AC, Group 8); ‘Jaguar, portico 10, jaguars joint, zone 2’ at Teotihuacan (AC, Group IX); ‘The Inferior Face of West Side of Chamber 1 of Murals’ at Bonampak (MC, Group IX); ‘Mural of the battle’ at Chichen-Itzá (MC, Group IX); ‘mayan vase with drawing of moon god with snake roll up’ (MC, Group X); ‘mayan vase’ of Naranjo (MC, Group X); and ‘disc of the Cenote sagrado’ at Chichen-Itzá (MC, Group X).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Table 1. Box (Db), information (Di), and mass (Dm) dimension, and their standard deviations (SD) for different Mesoamerican artistic and architectural work types. Work Type Group I. Tablets from Palenque and other sites Group II. Maya and other stelae Group III. Maya hieroglyphs Group IV. Frontal view of Maya pyramids, temples and other buildings Group V. Calendar pages (tonalamatl) from codex Group VI. Dresden and other codex pages Group VII. Frontal view of great stone monuments Group VIII. Circular astronomic and calendar great stones Group IX. Murals of Mesoamerica Group X. Maya vases (roll out) and other Overall average

n

Db±SD

Di±SD

Dm±SD

15

1.918±0.010

1.932±0.002

2.018±0.111

9 15

1.923±0.007 1.910±0.008

1.940±0.001 1.903±0.003

1.887±0.060 2.036±0.088

8

1.919±0.007

1.923±0.002

1.998±0.138

7

1.921±0.008

1.926±0.002

1.937±0.051

1.918±0.009

1.924±0.003

2.038±0269

8

1.917±0.009

1.914±0.003

1.954±0.053

7

1.900±0.006

1.877±0.003

1.975±0.047

9 12 90

1.919±0.006 1.883±0.013 1.912±0.009

1.929±0.002 1.888±0.003 1.916±0.002

1.964±0.058 1.966±0.214 1.983±0.117

Curiously, a few of the circular astronomic and calendar great stones from Aztec culture (Group VIII), which really contain a lot of information radially distributed are well characterized by DM values, that is, these values are similar to Db and Di values. Clearly, this occurs for ‘Aztec Calendar’ or ‘Sun Stone’ (Db = 1.92±0.005, Di = 1.9±0.005, DM 1.901±0.008); ‘Tizoc Disc’ (Db = 1.906±0.008, Di = 1.882±0.004, DM 1.866±0.008); and

80

Gerardo Burkle-Elizondo, Ricardo David Valdez-Cepeda and Nicoletta Sala

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

‘Chalco Disc’ (Db = 1.885±0.006, Di = 1.858±0.002, DM 1.842±0.01). What deserve be mentioned is that this approach, to estimate fractal dimension, works well in a few artistic or architectural works from the Groups I, IX and X. It is remarkable that Martínez del Sobral [5] has been described all these astronomic and calendar works by taking into account golden rectangles. Thus our result suggests the usefulness of DM when artistic and architectural works contain information radially distributed, so we prefer to use it on that type of works. Martínez del Sobral [5] pointed out that many pages from codices such as ‘Mendocino Codex’ ‘Borbonic Codex’, ‘Borgia Codex’ and ‘Dresden Codex’ are geometrically described by golden rectangles, and we find that codex pages (Groups V and VI) are well characterized by Db and Di. Examples are ‘1-wind 13th’ from ‘Borbonic Codex’ (Db = 1.932±0.004, Di = 1.931±0.001), ‘Page 1’ from ‘Mendocino Codex’ (Db = 1.938±0.004, Di = 1.926±0.003), ‘Page 13b’ from ‘Dresden Codex’ (Db = 1.909±0.004, Di = 1.908±0.0008), ‘Page 55’ from ‘Borgia Codex’ (Db = 1.949±0.01, Di = 1.940±0.008). From Group IV, Pyramids and Temples, Martínez del Sobral (2000) also described the following works through golden rectangles: ‘Temple of the Sun’ at Teotihuacan (Db = 1.913±0.003, Di = 1.93±0.0009), superior view of the ‘Temple of the Sun’ at Teotihuacan’ (Db = 1.923±0.004, Di = 1.913±0.003), superior view of the ‘Temple of Inscriptions’ (Db = 1.959±0.008, Di = 1.954±0.005), ‘Pyramid Temple I’ at Tikal (Db = 1.910±0.012, Di = 1.905±0.002), ‘Pyramid of 365 Niches’ at Tajín (Db = 1.914±0.004, Di = 1.935±0.001), superior view of the ‘Pyramid of 365 Niches’ at Tajín (Db = 1.926±0.007, Di = 1.91±0.002). From the Group VII, we characterize the following works. ‘Olmec Colossal Head, Monument 1’ at San Lorenzo (Db = 1.905±0.009, Di = 1.914±0.002), described by de la Fuente [1] and Martínez del Sobral [5] through golden rectangles. Also, Martínez del Sobral has characterized the following works by using golden rectangles: ‘Coatlicue’ (Db = 1.922±0.002, Di = 1.906±0.006), ‘Pacal Sarchophagus’ cover at Palenque (Db = 1.924±0.011, Di = 1.953±0.0003), ‘Stela A’ at Copán (Db = 1.937±0.006, Di = 1.934±0.003).

Figure 3. ‘Coatlicue’ the Aztec god of life and death as drew by León y Gama (from Martínez del Sobral [5]) (left) and architectural design of the ‘Pyramid of the Sun’ at Teotihuacan (right).

Complexity in the Mesoamerican Artistic and Architectural Works

81

In general, our results could be suggesting that Mesoamerican artists and architects used specific patterns and they preferred works with higher box (1.912±0.009) and information (1.916±0.002) fractal dimensions as appreciated in Table 1. In figure 3, we show two of the analyzed works for a readers’ best appreciation.

Conclusions Fractal geometry and Complexity are present in different cultures and in different centuries (Bovil, 1996; Briggs, 1992; Sala and Cappellato, 2003). Many of the Mesoamerican art and architectural works have an high fractal dimension. Meaningfully, Mesoasoamerican artistic and architectural works are characterized by a box fractal dimension Db = 1.912±0.009, and/or by an information fractal dimension Di = 1.916±0.002. There is a lack of studies to elucidate with a best precision the range for each type of fractal dimension to characterize the Mesoamerican artistic and architectural works once it has been discovered most of them are included in a series of golden rectangles that is connected to an aesthetic sense.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

References Bovil, C. Fractal Geometry in Architecture and Design. (Birkhauser, Boston, 1996). Briggs, J., Fractals - The Patterns of Chaos: a New Aesthetic of Art, Science, and Nature. (Touchstone Books, 1992) de la Fuente, B., Los Hombres de Piedra. Escultura Olmeca. (2nd Edition, Universidad Nacional Autónoma de México, Dirección General de Publicaciones. México, D.F. 1984). p.390 Hilgemeier, M., One metaphor fits all: a fractal voyage with Conway’s audioactive decay. In C. A. Pickover (ed.), Fractal Horizons: The Future Use of Fractals. (St. Martin’s Press. New York, USA. 1996). pp. 137-161 Martínez del Sobral, M., Geometría Mesoaméricana. (1st Edition, Fondo de Cultura Económica, México, D.F. 2000). p.287 Miller, M. E., The Maya Art and Architecture. (Thames and Hudson, London, 1999). Sala, N. and Cappellato, G., Viaggio matematico nell’arte e nell’architettura. (Franco Angeli, Milano, 2003). Stierlin, H., The Maya: Palaces and pyramids of the rainforest (Taschen, Köln, 2001). TruSoft Int’l Inc. Benoit, version 1.3: Fractal Analysis System. (20437th Ave. No. 133, St. Petersburg, FL 33704, USA). www.geocities.com/CapeCanaveral/Station/8228/arch.htm.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 83-87

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 7

NEW PARADIGM ARCHITECTURE1 Nikos A. Salingaros*

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Department of Applied Mathematics, University of Texas at San Antonio, San Antonio, Texas 78249, USA Charles Jencks wishes to promote the architecture of Peter Eisenman, Frank Gehry, and Daniel Libeskind by proclaiming it “The New Paradigm in Architecture”. Supposedly, their buildings are based on the New Sciences such as complexity, fractals, emergence, selforganization, and self-similarity. Jencks’s claim, however, is founded on elementary misunderstandings. There is a New Paradigm architecture, and it is indeed based on the New Sciences, but it does not include deconstructivist buildings. Instead, it encompasses the innovative, humane architecture of Christopher Alexander, the traditional humane architecture of Léon Krier, and much, much more. According to Jencks, the new paradigm consists of deconstructivist buildings, typified by the Guggenheim Museum for Modern Art in Bilbao, Spain, by Frank Gehry, and including other work and unbuilt projects by Peter Eisenman, Daniel Libeskind, and Zaha Hadid. Jencks has just revised his popular book “The Language of Post-Modern Architecture”, and has ambitiously re-titled it “The New Paradigm in Architecture” (Yale University Press, New Haven, 2002). Jencks bases his proposed new paradigm on what he thinks are the theoretical foundations of those buildings he champions. He claims that they arise from, and can be understood with reference to applications of the new science; namely, complexity theory, self-organizing systems, fractals, nonlinear dynamics, emergence, and self-similarity. In my own work, I have used results from science and mathematics to show that vernacular and classical architectures satisfy structural rules that coincide with the new science. Jencks claims a new paradigm with the opposite characteristics of living structure. That’s not what one expects from the new science, which helps to explain biological form. Trying to 1

This essay is a shortened version of "Cherles Jencks and the New Paradigm in Architecture", a chapter in the author's book "Anti-architecture and Deconstruction" (Umbau-Verlag, Solingen, 2004). Dr. Salingaros is considered as a leading theorist of architecture and urbanism, and an authority in applying science and mathematics to understand architectural and urban form. * E-mail address: [email protected], Homepage: http://www.math.utsa.edu/~salingar

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

84

Nikos A. Salingaros

get a perspective on this contradiction leads one to a witches’ brew of confused concepts and statements. Jencks does not provide a theoretical basis to support his claim of a new paradigm. An architecture that arises from the new science represents the antithesis of the deconstructivist buildings that are praised by Jencks. Clearly, we cannot have totally opposite and contradictory styles arising from the same theoretical basis. As a scientist who has taken an interest in architecture, I have worked with Christopher Alexander, and with coauthors who are scientists and mathematicians, some of them very eminent. Alexander’s new work “The Nature of Order” (Center For Environmental Structure, Berkeley, 2003) is an important and integral part of the new science. Our contributions to architecture are an extension of science into the field of architecture, beyond mere scientific analogies. The deconstructivists belong outside science altogether, and, despite their claims, do not come anywhere near to establishing a link with the new science. Instead, the deconstructivist architects draw their support from the French deconstructivist philosophers. Here we have two monumental problems: (i) deconstruction is rabidly anti-science, as its stated intention is to replace and ultimately erase the scientific way of thinking; and (ii) the spurious logic of French deconstructivist philosophers was exposed with devastating effect by the two physicists Alan Sokal and Jean Bricmont (“Fashionable Nonsense”, Picador, New York, 1998). How can we therefore accept claims for a new paradigm in architecture, based on science, if it is supported by charlatans who moreover are anti-science? A critical investigation into the pervasive and destructive influence of antiscientific thought in contemporary culture is now underway. It turns out that there is a basic confusion in contemporary architectural discourse between processes, and final appearances. Scientists study how complex forms arise from processes that are guided by fractal growth, emergence, adaptation, and self-organization. All of these act for a reason. Jencks and the deconstructivist architects, on the other hand, see only the end result of such processes and impose those images onto buildings. But this is frivolous and without reason. They could equally well take images from another discipline, for this superficial application has nothing to do with science. To add further confusion, Jencks insists on talking about cosmogenesis as a process of continual unfolding, an emergence that is always reaching new levels of self-organization. These are absolutely correct descriptors of how form arises in the universe, and precisely what Christopher Alexander has spent his life getting a handle on. Any hope that Jencks understands these processes is dampened, however, when he then presents the work of Eisenman and Libeskind as exemplars of the application of these ideas of emergence to buildings. None of those buildings appears as a result of unfolding, representing instead the exception, forms so disjointed that no generative process could ever give rise to them. It appears that perhaps the deconstructivist buildings Jencks likes so much are the intentional products of interrupting the process of continual unfolding. They inhabit the outer limits of architectural design space, which cannot be reached by a natural evolution. We have here an interesting example of genetic modification. Just like in the analogous cases where embryonic unfolding is sabotaged either by damage to the DNA, or by teratogenic chemicals in the environment, the result is a fluke and most often dysfunctional. Should we consider those buildings to be the freaks, monsters, and mutants of the architectural universe? Hasn’t the public been fascinated with monsters and the unnatural throughout recorded history as ephemeral entertainment?

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

New Paradigm Architecture

85

The key here is adaptation. I have looked into how Darwinian processes act in architecture on many distinct levels. A process of design that generates something like a deconstructivist building must have a very special set of selection criteria. No-one has yet spelled out those criteria. What is obvious, however, is that they are not adaptive to human needs, being governed instead by strictly formal concerns. Some factors responsible for the high degree of disorganized complexity in such buildings are: (i) a willful break with traditional architecture of all kinds; (ii) an expression of geometrical randomness and disequilibrium; and (iii) ironic statements or “jokes”. Trying to avoid the region of design space inhabited by traditional solutions, which are adaptive, pushes one out towards novel but non-adapted forms. By employing scientific terms in an extremely loose manner Jencks erodes his scientific credibility. As an example, he talks of “twenty-six self-similar flower shapes” used by Gehry in the Bilbao Guggenheim. As far as I can see, there are no self-similar shapes used in that building. As to resembling flowers, they don’t, because flowers adapt to specific functions by developing color, texture, and form, all within an overall coherence which is absent here. There is a tremendous difference between a mere visual and a functional appreciation of fractals. The Guggenheim Museum is disjoint and metallic, and as far removed from any flower as I can imagine. Jencks then refers to these non-self-similar shapes as “fluid fractals”. I have no idea what this term means, as it is not used in mathematics. A third term he uses for the same figures is “fractal curves”. Again, those perfectly smooth curves are not fractal. I was puzzled to read an entire chapter in Jencks’s book entitled “Fractal Architecture” without hardly seeing a fractal (the possible exceptions being decorative tiles). I can only conclude that Jencks is misusing the word “fractal” to mean “broken, or jagged” — even though he refers to the work of Benoît Mandelbrot, he has apparently missed the central idea of fractals, which is their recursiveness generating a nested hierarchy of internal connections. A fractal line is an exceedingly fine-grained structure. It’s not just zigzagged; it is broken everywhere and on every scale (i.e. at every magnification), and is nowhere smooth. Jencks himself admits that: “The intention is not so much to create fractals per se as to respond to these forces, and give them dynamic expression”. What does this mean? He refers to a building that has a superficial pattern based on Penrose tiles, and calls it an “exuberant fractal”. Nevertheless, the Penrose aperiodic pattern exists precisely on a single scale, and is therefore not fractal. Jencks discusses with admiration unrealized projects by Peter Eisenman, which both claim are based on fractals. But then, Jencks adds revealingly: “Eisenman appears to take his borrowings from science only half-seriously”. Science, however, cannot be taken only halfseriously; one can only surmise that we are dealing with a superficial understanding of scientific concepts that allows someone to treat fundamental truths so cavalierly. Jencks cites Eisenman’s Architecture Building for the University of Cincinnati as an example of what he proposes as new paradigm architecture. However, from a mathematician’s perspective, there is no evident structure there that shows any of the essential concepts of self-similarity, selforganization, fractal structure, or emergence. All I find is intentional disarray. As is admitted by its practitioners, de(con)struction aims to take form apart — to degrade connections, symmetries, and coherence. This is exactly the opposite of self-organization in complex systems, a process which builds internal networks via connectivity. For this reason, deconstructivist buildings resemble the severe structural damage such as dislocation, internal

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

86

Nikos A. Salingaros

tearing and melting suffered after a hurricane, earthquake, internal explosion, fire, or (in an eerie toying with fate) nuclear war. Architecture and urbanism are prime examples of fields with emergent phenomena. Cities and buildings with life have this property of incredible interconnectedness, which cannot be reduced to building or design components. Every component, from the large-scale structural members, to the smallest ornament, unites into an overall coherence that creates a vastly greater whole. Deconstructivist buildings, however, show the opposite characteristics where each component degrades the whole instead of intensifying the whole. This is easy to see. Does a structural piece intensify the other pieces around it? Is the total coherence diminished if it were removed? The answer is yes in a great Cathedral, but no in a deconstructivist building. I think that everyone will agree with me that each portion of today’s fashionable deconstructivist buildings detracts from and conflicts with every other portion, which is the opposite of emergence. Traditional architects such as Léon Krier and others have been using timeless methods for organizing complexity, and attribute their results to knowledge derived in the past. It is only very recently that we have managed to join two disparate traditions: (i) strands of various architectures evolved over millennia, and (ii) theoretical rules for architecture derived from a drastically improved understanding of nature. The new paradigm is a revolutionary understanding of form, whereas the forms themselves tend to look familiar precisely because they adapt to human sensibilities. Most architects, on the other hand, wrongly expected a new paradigm to generate strange and unexpected forms, which is the reason they were fooled by the deconstructivists. The buildings that Jencks prefers all have a high degree of disorganized complexity. This quality is arrived at via design methods mentioned previously. One can also include the use of high-tech materials for a certain effect, which is carefully manipulated to achieve a negative psychological impact on the user. This last feature is best expressed by Jencks himself in describing a paradigmatic building: “It is a threatening frenzy meant, as in some of Eisenman’s work, to destabilize the viewer …”. I don’t think that anyone is going to consider the common theme of disorganized complexity as constituting sufficient grounds for claiming a new paradigm. Jencks suggests that we are supposed to get excited because a computer program that is used to design French fighter jets is then applied to model the Bilbao Guggenheim. We are also expected to value blobs (which mimic 19C spiritualists’ ectoplasm) as relevant architectural forms simply because they are computer-generated. This fascination with technology is inherited from the modernists (who misused it terribly). When the technology is powerful enough, one may be misled into thinking that the underlying science can be ignored altogether. Most informed people know that one can model any desired shape on a computer; it is no different than sketching with pencil on paper. Just because something is created on a computer screen does not validate it, regardless of the complexity of the program used to produce it. One has to ask: what are the generative processes that produced this form, and are they relevant to architecture? We stand at the threshold of a design revolution, when generative rules can be programmed to evolve in an electronic form, then cut materials directly. There exists an extraordinary potential of computerized design and building production. Architects such as Frank Gehry do that with existing software, but so far, no fashionable architect knows the fundamental rules that generate living structure. A few of us, following the lead of Alexander,

New Paradigm Architecture

87

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

are discovering those rules, and we eventually hope to program them. Others working within traditional architecture have always known rules for generating living structure; now they are ready to generalize them beyond a specific style. When the scientific rules of architecture are universally adopted, the products will surprise everyone by their innovation combined with an intense degree of life not seen for at least one hundred years. Much of what I have said has already been voiced by critics of deconstructivism. And yet, like some mythical monsters, deconstructivist buildings are sprouting up around the world. Their clients, consisting of powerful individuals, corporations, foundations, and governments, absolutely want one of them as a status symbol. The media publicity surrounding deconstruction reinforces an attractive commercial image. I admit that the confused attempts at a theoretical justification, misusing scientific terms and concepts haphazardly, succeed after all in validating this style in the public’s eye. It appears that something is clearly working to market deconstructivism, and Jencks’s efforts help towards this promotion. Architects today are told that the new science supports and provides a theoretical foundation for deconstructivist architecture. Nothing appears to justify this claim. On the contrary, I believe the evidence shows that there does exist a new paradigm in architecture, and it is supported by the new science. Charles Jencks is in part correct (though strictly by coincidence, since his own proposal for a new paradigm is based on misunderstandings). Nevertheless, this new paradigm architecture does not include deconstructivist buildings. The new paradigm encompasses the innovative, humane architecture of Christopher Alexander, the traditional, humane architecture of Léon Krier, and much, much more.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 89-102

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 8

GENERATION OF TEXTURES AND GEOMETRIC PSEUDO-URBAN MODELS WITH THE AID OF IFS Xavier Marsault* UMR CNRS MAP, "Modèles et simulations pour l'Architecture, l'urbanisme et le Paysage" Laboratoire ARIA, Ecole d’Architecture de Lyon 3, rue Maurice Audin, 69512 Vaulx en Velin

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Abstract Geometric and functional modelling of cities has become a growing field of interest, raised by the development and democratisation of computers being able to support highdemanding graphics in real time. Actually, more and more applications concentrate on creating virtual environments. ARIA has been working for two years, within the DEREVE project (DER, 2000), on pseudo-urban textures and geometric models generation, by means of fractal or parametric methods. This paper explains our attempt to capture inner coherence of urban shapes and morphologies, by fractal analysis of 2D½ textures (top view + height) of real and synthetic city maps. The basic ideas lean on autosimilarity detection, fractal coding of regions, and processing with Iterated Function Systems (IFS). We introduce a genetic-like approach, allowing interpolation, alteration and fusion of different urban models, and leading to global or local synthesis of new shapes. Finally, a 3D reconstruction tool has been developed for converting textures to volumes in VRML, simplified enough for real time wanderings, and enhanced by some automatically generated garbage dump and decorated elements. Programs and graphic interface are developed with C++ and QT libraries.

Keywords: Fractal city, Urban pattern, IFS. Image, 2D½ and 3D model, Genetics, Fusion, Level of detail, Shape filtering, VRML

*

E-mail address: [email protected], Homepage: http://www.aria.archi.fr

90

Xavier Marsault

1. Introduction 1.1. Fractal Cities? Usually, geometric models of town patterns or whole cities can be generated with the aid of spatial growth simulators, or temporal simulators based upon a scenario (ex: Sim City), or by means of static shapes (Parish, Muller, 2001). Many related works deal with fractality: some of them use cellular automata (Torrens, 2000), other ones use DLA (diffusion limited by aggregation) (Bailly, 1998) or organic models inspired by physical laws (Makse, 1996). Indeed, some recent studies reveal the fractal nature of many urban structures at large scales and some architectural objects (Sala, 2002), (Batty, Longley, 1994), (Frankhauser, 1994). Focusing on the near scale of buildings and built patterns, we have shown that some urban shapes exhibit a local property of autosimilarity, while they lose it in a larger analysis. In this context, one way of research was to attempt to use IFS (that share this property) to analyse and generate new urban morphologies. Two cities belonging to the suburbs of Lyon (St Genis and Venissieux, fig 7a) and two synthetic maps (fig. 6) have been used during all developments and tests.

1.2. Iterated Function Systems (IFS) IFS theory is totally based upon the “scale change invariance” property (SCE), and thus allow the generation of fractal objects with a set of contractive functions showing this property, called Iterated Function System (IFS). It has been studied by Hutchinson within the mathematical frame of autosimilarity (a mathematical object is said to be autosimilar if it can be split into smaller parts calculated from the whole by a “similar transformation”) (Hutchinson, 1981), and by Barnsley within the frame of fractal geometry (Barnsley, 1988), leading to image compression applications (Barnsley, 1992-1993), (Jacquin, 1992).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Image Compression Since usually a given image is not a fractal object, it is unlikely to find a whole fractal generator of it. But there is an interesting application of IFS to image compression allowing fractal coding, where small regions are coded from contractive SCE transformations (called lifs, for “local ifs”) of other larger regions. The most common algorithm was developed by Jacquin (Jacquin, 1992). Given a square uniform pavement of N range blocks ri of size B and a pool of domain block di for matching research (Fig. 1), it tries to find a function i → α (i ) and N lifs ω i such as ri → rˆi = ω i (d α (i ) ) , so that

∑ ω (d α i

(i ) ) − ri

2

is minimum for each i

i

(local collage). The famous “collage theorem” ensures that the decoded image is an approximation of the original image, and gives a maximum measure of the error. Each ω i is set with a transformation projecting a domain block di of size D at the place and scale of the range block ri (decimation of pixels + isometry), followed by an affine transformation on grey levels of pixels : rˆi = iso i (σ i .ri + β i ) . The isometric transformation iso i is chosen among the

Generation of Textures and Geometric Pseudo-Urban Models with the Aid of IFS

91

8 possible transformations of a square block (identity, –90, 90 or 180 degrees rotations, x, y, or diagonal axis symmetries).

range bloc ri

Figure 1. Transformation from a domain block to a range block (lifs).

The image I is decoded by calculating the attractor of lifs wi from any random image I0. We note :

W (I ) =

N

N

∪ rˆ = ∪ ω (dα ) and W i

i =1

i

(i )

k +1

(

)

(I ) = W W k (I ) .

i =1

The attractor is then defined by : A( I ) = lim W k ( I 0 ) . k → +∞

The more local collages are better, the more the result of the attractor is a good approximation of the original image (collage theorem). A little number of iterations is needed for the decoding process to converge. In order to ensure the uniform convergence, we limit

σ i B} . Then, locally for each pixel p, we can define an average

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

autosimilarity measure : μ~( p ) =

1 minμ ( R, D) . But, this average can card {R, R ⊃ p} R ⊃ p D > B

∑

potentially mask an existing D block for which the appariement is exact, or almost exact. So, we also calculate (Fig. 3) the minimal measure : μ min ( p ) = min⎛⎜ min μ ( R, D) ⎞⎟ . When B is R⊃ p⎝ D>B ⎠ fluctuant, we could locally consider the higher value Bmax(p) of B for which the μ min ( p ) measure is minimum, and propose another measure (1- μ min ( p ) ).Bmax(p), which grows both

with B and the appariement quality. But such a task should require a tremendous computing time, even for a 256 x 256 image. One can also wander which information could be gathered from the study of the function D→ μ ( R, D) . For example, the decrease of this function could help characterizing a typical behaviour.

Generation of Textures and Geometric Pseudo-Urban Models with the Aid of IFS

part of Saint Genis

μ for B=5, D=20

μ for B=7, D=20

93

μ for B=8, D=20

Figure 3.

Our goal being the generation of urban shapes and structures that look like real ones, we decide to lean on real city plans, and use the IFS as an analysis and synthesis tool. Because IFS operate on a continuous space of shapes, allowing interpolation, alteration and fusion, and integrate as a whole approach analysis and generation of global or local new shapes, we expect them to produce good results.

2. From IFS to Urban Textures and Geometric Models

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

2.1. A Simplified Coding Method Urban scale concerns the spatial distribution of buildings within a certain piece of landscape. It can be described with a restrictive approach by a set of more or less simplified volumes, especially for fast rendering. The image compression technique described in 1.2 allows an approximated coding of an image from local transformations of parts of itself. It can be used to encode urban pattern with IFS if we decide to convert geometrical 3D volumes to images. In this 2dD½ approach, the grey levels represent the heights of buildings. Then, we use Jacquin’s fractal compression technique for coding the ground shapes and heights of buildings which populate a city map. We get an autosimilar approximation of the map, whose accuracy depends on the nature and the choice of the initial pavement of the map, and on the number of local lifs given to approximate the local diversities of the shapes. We have proposed some adaptations for urban pattern analysis: initial and static regular square pavement for the range blocks of size B, exhaustive research within the domain block pool (with varying size of D for each block B), pre-calculus of range blocks similarities, accelerated appariement by classification of range and domain blocks (uniform, outline), elimination of ground blocks and a « topological collage » for matching (see below).

Towards a Spatial-Coded Model When several domain blocks D are candidates to the best appariement for a given range block B, the question of choice appears, whereas it is not significant in the frame of image compression. In that case, the algorithm should select the D block whose neighbourhood if the closest to the one of the B block. It expands successively the outline of each block by on pixel until it finds the minimum among all the proposed range blocks. This option that we call « topological collage » slowers the processing time, but is the only one that can really take account of the topological links between B and D blocks. It has been successfully applied to our appariement algorithm used by the “asymmetric fusion” operator (see 2.3).

94

Xavier Marsault

2.2. General Processing Scheme Some pseudo-convex interpolation, mutation, fusion and filtering operators have been designed to generate new urban models leaning on existing ones (real or synthetic), or to add some modifications to them. For this purpose, we use the genetic analogy introduced by (Vences, Rudomin, 1997) (see 2.3). fusions 3D urban models

2D ½ images

IFS IFS

post-processing

alterations interpolations fractal filtering

Figure 4. General processing scheme.

Pseudo-Convex Interpolation of IFS Given two distinct images encoded by IFS1 and IFS2, our first idea was to define a λparametric convex IFS leaning on IFS1 and IFS2. Indeed, if the iteration semi-group is convex, its attractor is λ-continuous (Gentil, 1992). As this is not the case with the semigroup composed by the 8 isometries of the square, we should rather speak of pseudo-convex interpolation. And the awaited results are disappointing, since we get in fact the same as basic image interpolation. Nevertheless, depending on other suitable choices for the type of pavement used, IFS interpolation could become possible.

2.3. A Genetic-Like Formalization

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The genetic analogy proposed by Vences and Rudomin first in the frame of image compression, is very powerful for exploring new ways of creation, and let us envision applications to the generation and the alteration of urban geometric models. Assuming the notation IFS = (ω 1 , ω 2 ,..., ω N ) , where ω i are the lifs, we consider the IFS as a chromosome (genotype), the lifs as genes, and the attractor (image or 3D model) as a phenotype. This analogy can be justified in several ways. First, the information for decoding an image fragment is distributed among many lifs. Some lifs alterations can have consequences on numerous zones, or not. Moreover, the whole body of lifs represents a highly non-linear and complex system. Following this scheme, we apply the general fusion mechanism (Renders, 1995) which consists in generating a large population of IFS models sharing the same genes (inherited from two parents), while the mutation allows the alteration of genes during the crossing process or the exchange of genes along the same chromosome. The following paragraphs describe some ways we used to implement the fusion process.

Generation of Textures and Geometric Pseudo-Urban Models with the Aid of IFS

95

Direct or Asymmetric Fusion We follow the genetics analogy, where the fusion process, even highly combinatorial, does not take any genes at random. Since lifs are coding zones whose content may be very different, the process of fusion must be guided by an appariement step between IFS parents. Indeed, without any control, the direct fusion lifs to lifs or their copy from one zone to another (a kind of mutation), give very bad results. So, we decided to keep the original distribution of lifs and to attempt to group them, before fusion occurs between both IFS.

Figure 5. Principle of asymmetric fusion process.

A first step we proposed to take this mechanism into account is to calculate appariements of range blocks between both images, based on the images content. The process is asymmetric, since we associate to each image a list of range blocks and their lifs counterparts in the other one (fig. 5). A range block R1 from image 1 is linked (by appariement) to a range block R2 of image 2, which is encoded by an lifs based on domain block D2. The process of fusion consists in replacing lifs1 by lifs2, and leads to two fusion images : 1mod2 et 2mod1. One can observe on figure 6 the effect of asymmetric fusion : the generated distribution of shapes look like their two parents, modulated each one by the other.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

synthetic town model A

synthetic town model B

asymmetric fusion AmodB for B=4 and D=8 Figure 6. Continued on next page.

96

Xavier Marsault

asymmetric fusion BmodA for B=4 and D=8

asymmetric fusion AmodB for B=8 and D=16

asymmetric fusion BmodA for B=8 and D=16 Figure 6. 3D models of asymmetric fusion with synthetic towns A and B.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Pavement-Based Fusion Given a unique Jacquin square pavement for both IFS, and a size B for range blocks, we define square macro-blocks (or pavements) of size multiple of B, in order to group several connex range blocks. The pavement-based fusion consists in crossing spatially grouped sequences of lifs (rather than isolated ones) between both IFS, in order to preserve topology. The process alternately keeps some lifs from the first IFS and the second one. Possible discontinuities only appears at the borders of the macro-blocks. Our technique lets the algorithm first inject some macro-blocks of important size, and finishes with smaller ones, like a town planner who first deals with higher scales of the city before looking at the content of the neighbourhoods. Moreover, the fact of varying the size of injected macro-blocks allows the modulation of the crossing scale. While varying the minimum and maximum limits of the macro-blocks size, we modify the model topology by authorizing more or less discontinuities. The macro-block locations and the fill rates of each IFS are provided by the user or generated by a pseudo-random generator. The user also enters upper and lower limits for the macro-blocks size. The algorithms first fills up the entire image with one IFS. Then, it takes the other one, and will alternate until a break-test is verified. A margin is entered by the user to let the algorithm have a tolerance while matching the fill rate criterion. This margin progressively diminishes each time the filling IFS is changed, while the size of the macroblocks is lowered of one-pixel. This is a heuristic allowing to create on the fly a new genotype from both parents’ ones, guided by constraints depending on their phenotypes. For convenience, we added as fill rate criterion a fusion parameter λ, ranging from 0 to 1, allowing a sort of IFS interpolation between both models. We define a non-intersection criterion allowing to label as “admissible” each macroblock whose outline does not intersect buildings more than a tolerance threshold S, given by

Generation of Textures and Geometric Pseudo-Urban Models with the Aid of IFS

97

the user. This criterion is computed on the grey level differences around the outline. Moreover, it also takes account for the previous crossing steps of the algorithm, leading to a better continuity in the phenotype shapes. Nevertheless, this precaution does not guarantee that all domain blocks will belong to preserved zones, but this is a first and serious limitation to this problem. The S parameter plays an essential role in the appearance of new and original shapes. Pavement-controlled fusion gives some very good graphic, and provides new local or global shapes and distribution of shapes, whose details are the consequence of crossing models. The pavement choice is controlled by preservative criterions, and the number of map inputs is not limited for this process.

Figure 7. Example of pavement-controlled fusion (down) on real cities (Saint Genis and Vénissieux,up)

2.4. Shape and Detail Filtering Adjusting IFS Scale for Detail Filtering

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Since IFS coding does not take account for dimensions, it is easy to mathematically rebuild its attractor at any scale. This property leads to what has been called “fractal zoom”, that can be used with values greater than one (creation of fractal detail), or less than one (shape simplification). So we denote a correspondence between the fractal zoom and the generation of continuous “levels of detail” for objects, that can be used within a real-time wandering (fig. 8).

Figure 8. Two versions of the same buildings (before and after a 2x fractal zoom).

98

Xavier Marsault

Shape Filtering within the Domain Blocks Pool On the other hand, the encoding of range blocks from domain blocks being surjective, some domain blocks may be used more than other ones, and our experiments confirm this property, that gives an indication on the quantity of generative information used to approximate an urban fabric. Therefore, our idea was to implement a low-pass filtering on the IFS domain block calling frequencies, estimated for each range block containing the analysed pixel of the image, and then to recalculate the attractor of the IFS. This can lead to drastic geometric simplifications, depending on the value of the cut frequency fc (fig. 9).

Figure 9. Low-pass filtering with IFS (fc=1 ; fc=10 ; fc=100).

3. From Generated Scenes to Real World

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

3.1. Towards Urban Shapes Interpretation and Classification Because IFS do not take account for dimensions, it is first necessary to provide the correspondence between the pixel and grey level units and the size of the objects in the real world. Then, a general method of correspondence between virtual objects and real world ones has been proposed, based on the mixed criterion (surface, height), and allows a primary classification of generated urban objects, completed by some quick shape analysis to help identifying the type of a building, for example. This typology contains 7 types of objects : buildings and houses (blue grey), urban furniture (light orange), trees and vertical vegetation (light green), ground-levelled objects (swimming pools, parkings, lawns, ponds ; dark green), fountains (dark blue), shelters of garden (brown), electrical posts and public lamps.

Generation of Textures and Geometric Pseudo-Urban Models with the Aid of IFS

99

Instead of using raw objects as they are generated, we could operate some substitutions with other ones in typical libraries representative of certain urban atmospheres, for example. But this solution hasn’t been already implemented.

Figure 10. Two examples of classification of urban objects with a colour table.

3.2. Simplifying and Smoothing Generated Shapes

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Some algorithm developments have been required for smoothing irregular distribution of pixels, due to the jaggy appearance of vectorized pixels and the fractal nature of generation, and for obtaining simplified geometric shapes. Our work involves many existing simplification algorithms (Douglas-Peucker, characteristic vertex extraction), combined in a robust approach, introducing the notion of “significant geometric detail”, with a scale tolerance factor (Fig. 6). Another promising way of research concerns local adjustment of pre-defined configurations of “common angles” in architecture, up to global adjustment with constraints (for example, for placing roof shapes).

Figure 11. A noisy building – outline with details – simplified outline (tolerance = 1/20).

100

Xavier Marsault

3.3. Adding Automatic Garbage Dump to the Scenes Automated generation of streets and places graphs from urban imprints maps is an interesting research topic. This “raster approach” of the problem is rather new comparing to the one dealing with vector objects, and avoids the difficult task of shape vectorisation in noisy environments. Our still progressing work is done in three stages : •

• •

extraction of geometric structuring characteristics from the maps : graphs of streets and places, combined within a technique for spatially grouping houses and buildings in neighbourhoods ; geometric generation of corresponding smooth 3D objects in VRML ; search for heuristic methods to qualitatively identify plausible elements of garbage dump networks (ex : boulevards, avenues, alleys, water streams).

We’re still working on opened and closed connex graphs of street network. We apply some « mathematical morphology » basic tools for extracting homotopic skeletons of the ground zones and streets width (fig. 12). We obtain two types of graph, depending on the possibility to connect the city to its environment (opened city), or not (secluded city). To improve the quality of morphological processing, we work on super scanned images (a factor of 4 seems to be sufficient for 256 x 256 or 512 x 512 images).

Figure 12. homotopic opened skeleton (b )of part of Saint Genis city (a) and its street-width map (c).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

4. Conclusions and Future Works 4.1. Discussion We have shown in this paper how it is possible to encode simplified 2D½ city models using an IFS compression technique derived from Jacquin’s algorithm. Cutting urban maps with the aid of a square pavement allows local control on the content of the split zones. In this purpose, we initiated a genetic-like approach to share information between IFS coding two (or more) city models, in order to compose new urban and architectural shapes by fusion and mutation. The possibility to deal with real or synthetic urban fabrics opens a wide field of creation, and many ideas have been suggested for that. We mostly obtain orthomorphic geometric models, because of the approach of converting blocks of connex pixels as cubes with their borders. Recent references on city modelling use such objects (Parish, Muller,

Generation of Textures and Geometric Pseudo-Urban Models with the Aid of IFS

101

2001), where buildings are designed with the aid of L-systems. One can also observe the similitude between the geometric aspect of our models and the famous « architectones » of Kasimir Malevich (Figure 13). But, up to now, we’ve just used a uniform and general square pavement. The fact we did not consider other pavements more suitable for the process of isolated buildings or groups of buildings results in discontinuities and loss of topological identity, even if we strove to minimize them in our pavement-based fusion algorithm. Indeed, a related difficulty is the adjustment of the range size parameter B : if B is too small, only the outline of the objects are coded by lifs, and if B is too high, it can be very hard to find some blocks similarities. On the other hand, the IFS approximation quality requires a sufficient resolution for images. These two constraints result in higher computation time for lifs, even with the uniform square pavement.

Figure 13. A famous cubic architecton of Kasimir Malevich (1926).

4.2. Remaining Investigations

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

From a scientific point of view, several ways of research remain : Our experiments still suffer from a lack of theorical developments on IFS coding and partitioning for the use of fusion between several city models. It is important to search for better pavements of range and domain blocks, well fitted to match models and process properties. A semi-synthetic approach for IFS will be explored, in order to reproduce given models of spatial distribution of shapes, using “condensation IFS” that allow the import of extern objects in non coding blocks. Some experiments with genetic algorithms have to be done to optimise the fusion results, given some shape or statistical distribution criterions extracted from real cities, or by applying the famous “universal distribution law” (Salingaros, 1999). We also envision the study of location and recombinant mutations in order to increase the size of the IFS original extent, by distributing some lifs or groups of lifs to other places. From a simulation point of view, this would become a first step towards infinite generation of nonrepetitive urban fabric.

102

Xavier Marsault

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

References Bailly, E. (1998) Fractal geometry and simulation of urban growth, UMR CNRS Espace 6012, Nice. Barnsley, M. (1993) Fractal image compression, AK Peters, Ltd, Wellesley. Barnsley, M. (1992) Image coding based on a fractal theory of iterated contractive image transformation, IEEE transactions on image processing, 1:18-30. Batty, M., Longley, P.A. (1994), Fractal Cities: A Geometry of Form and Function, Academic Press, London and San Diego. DER (2000), Développement d’un Environnement logiciel de REalité Virtuelle Elaboré, Projet de recherche DEREVE de la région Rhône-Alpes, LIGIM, Université Lyon I. Frankhauser, P. (1994) La Fractalité des Structures Urbaines, Collection Villes, Anthropos, Paris. Frankhauser, P. (1997) L’approche fractale : un nouvel outil de réflexion dans l’analyse spatiale des agglomérations urbaines, Université de Franche-Comté, Besançon. Gentil, C. (1992) Les fractales en synthèse d’images : le modèle IFS, Thèse, LIGIM, Université Lyon I, Lyon. Hutgen, B., Hain, T. (1994) On the convergence of fractal transforms, Proceedings of ICASSP, 561-564. Hutchinson, J. (1981) Fractals and self-similarity, Indianna Universiry Journal of Mathematics, 30:713-747. Jacqui,n A.E. (1992) Image coding based on a fractal theory of iterated contractive image transformations, IEEE transactions on image processing, 1(1):18-30. Makse, H.A. (1996) Modelling fractal cities using the correlated percolation modeI, Fractal and granular media conference., session C18 Marsault, X. (2002) Application des Iterated Function Systems (IFS) à la composition de tissus urbains tridimensionnels virtuels, Autosimilarités et applications, Cemagref, Clermont Ferrand. Parish, Y., Muller, P. (2001) Procedural modelling of cities, SIGGRAPH. Renders, J.M. (1995) Algorithmes génétiques et réseaux de neurones, Editions Hermès. Sala, N. (2002) The presence of the self-similarity in architecture : some examples, in M.M.Novak (ed), Emergent Nature, World Scientific, 273-283. Salingaros, N. (1999) A universal rule for the distribution of sizes, Environment and Planning B : planning and Design, 26:909-923, Pion Publications. Torrens, P., (2000) How cellular models of urban systems work, CASA, Angleterre. Vences, L., Rudomin L. (1997) Genetic algorithms for fractal image and image sequence compression, Instituto Tecnologico de Estudias Superiores de Monterrey, Camus Estado de Mexico, Computation Visual. Woloszyn, P., (1998) Caractérisation dimensionnelle de la diffusivité des formes architecturales et urbaines, Thèse, Laboratoire CERMA, NANTES.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 103-112

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 9

PSEUDO-URBAN AUTOMATIC PATTERN GENERATION Renato Saleri Lunazzi* Architecte DPLG, DEA informatique et productique, master en industrial design. Laboratoire MAP aria UMR 694 CNRS – Ministère de la culture et de la Communication

Abstract This research task aims to experiment automatic generative methods able to produce architectural and urban 3D-models. At this time, some interesting applicative results, rising from pseudo-random and l-system formalisms, came to generate complex and rather realistic immersive environments. Next step could be achieved by mixing those techniques to emerging calculus, dealing whith topographic or environmental constraints. As a matter of fact, future developments will aim to contribute to archeological or historical restitution, quickly providing credible 3D environments in a given historical context.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

1. Introduction Since the end of the 70’s, the “fractality“ of our environment raised as an evidence, pointing some peculiar aspects of everyday phenomena. Some micro and macro-scopic internalarrangement principles appear to be similar or even auto-similar, leading the reasoning through general explanatory theories. Physicians and biologists regularly discover fractal processes through natural morphogeneses such as cristalline structures or stellar distribution. Human creations also seem to be ruled by fractal fundamentals and since 15 years, the “fractality measure“ of some human artefacts can be somehow achieved. Fractal investigation through urban patterns mainly focused on two subsequential aspects : the direct analysis of spatial organisation, and thus the formalization of selfgenerating geometrical structures. The growth of urban models is at this time fulfilled either by time-based spatial simulators or by simple static generators. Spatial simulators are usually based on simple “life-game“ (cellular automata) devices or even by “diffusion limited *

E-mail address: [email protected]

104

Renato Saleri Lunazzi

aggregation“ formalisms (DLA). In this paper, we will mainly focus on some generative techniques involved in 2D and 3D automatic builders.

2. Research Task Context The mainframe of this research task consists in real-time rendering of huge 3D databases. Different aspects of this goal have already been explored, considering from the top that rendering techniques should be optimal for a given applicative context. Therefore, the main aspect of MAP-aria participation in this project consists in building plausible urban structures related to some given historical or archeological context. Early stages of our investigation pointed the discontinuous properties of growth phenomena. In other words we barely believed in the existence of a possible continous morphological development model, according to the evidence of micro and macro-scopic observable morphological differences on one hand and through bidimensional and threedimensional topological discontinuities on the other. In other words, we focused some “scale-based formalisms“, related to specific urban scale-types, as listed in the following section.

3. Applications The description of the following formalisms is broadly summarized. Further refinement on geometrical models, architectural primitives and morphological break-down are under development...

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

3.1. Random 2D – 3D Generators Random or pseudo-random simple pattern generators applied to facades, according to buildings height or local floors indentations. Please note that the “hull filler“ generator, mentioned in this section, is shortly described in section 3.3 This very first applicative experiment was only acquired to test some early combinational conjectures. Some 3D “hull-filled“ objects are textured whith simple combinational patterns ensuring somehow an intrinsic global coherence in order to avoid 2D and 3D possible mismatch. This could be achieved by establishing for instance a common spatial framework, arbitrarly bounded here by 2,5 meters-sided cubes. As shown in the picture below, the intrinsic coherence of the texture itself depends on the pertinence of single texture patches positioning, known as inner, top, left, right and bottom occurrences : on the illustration, the gray-filled board zone invoke specific ledge-type instances as the inner white zones use generic tiles. Right underneath, some texture patches that come whith the 2D library and below, two facade variants. These examples are here intended as “ironic standalone designs“ : the (im)pertinence of these random objects is obvious. Meanwhile, if coupled whith accurately-sized 3D objects, the visual impression could be effective, as shown on Figure 3.

Pseudo-Urban Automatic Pattern Generation

105

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 1. The automatic facade builder and some architecural tiles.

Figure 2. Some “automatic“ facades.

We recently improved this application capabilities through some Maya© Embedded Language developments. The synchronous object-texture pattern generator produces “onclick“ 3D architectural-like objects and plots them over a simple 2D grid, The main controls provide some expansion parameters such as linear spread-out and rotation constraints. This very first MEL application deals whith a single-input façade library ; a very next step will consider a wider variety of morpho-textural relevant matchings.

106

Renato Saleri Lunazzi

Figure 3. Applying and rendering colored tiles on random-generated rule-based 3D objects.

3.2. Graphtal or L-System Generator Graphtal or L-System, Applied to Local Building and Block Propagation

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The L-System, or Graphtal, starts from a simple recursive substitution mechanism. This rulesbased generator, described in the late 60’s by A. Lindenmayer (Lindenmayer 1968) can quickly provide complex geometric developments. It’s charachteristic deal whith simple substitution rules, recursively applyed to a sprout, as shown below : All we need to start is an alphabet, listed hereby : 0,1,[ ,] In this example, 0 and 1 occurrences will “produce geometry“ while [ and ] will provide a simple affine transformation (rotation and/or translation). We can now describe simple substitution rules, applied to alphabetic elements : 0 : 1[0]1[0]0

1 : 11

[:[

]:]

If we recursively apply those substitution rules to an initial sprout (applied from the top to the rule of letter “0“) we obtain: 11 [ 1[0]1[0]0 ] 11 [ 1[0]1[0]0 ]1[0]1[0]0 Two “generations” or recursive steps later we obtain: 11 11 11 11 [ 11 11 [ 11 [1[0]1[0]0 ] 11 [ 1[0]1[0]0 ] 1[0]1[0]0 ] 1111 [ 11 [ 1[0]1[0]0 ] 11 [ 1[0]1[0]0 ]1[0]1[0]0 ] 11[ 1[0]1[0]0 ] 11 [ 1[0]1[0]0] 1[0]1[0]0 ] 11 11 11 11 [ 11 11 [11

Pseudo-Urban Automatic Pattern Generation

107

[1[0]1[0]0 ] 11 [ 1[0]1[0]0 ] 1[0]1[0]0 ] 1111 [ 11 [1[0]1[0]0 ] 11 [ 1[0]1[0]0 ]1[0]1[0]0 ] 11 [ 1[0]1[0]0 ] 11 [ 1[0]1[0]0] 1[0]1[0]0 ] 11 11 [ 11 [ 1[0]1[0]0 ] 11 [1[0]1[0]0 ] 1[0]1[0]0 ] 11 11 [ 11 [1[0]1[0]0 ] 11 [ 1[0]1[0]0 ] 1[0]1[0]0 ] 11[ 1[0]1[0]0 ] 11 [ 1[0]1[0]0 ] 1[0]1[0]0 The “trick” consists here in replacing the brackets by specific 3D operations – typically affine transformations, such as rotations or translations - and the “0“ and “1“ occurrences by 3D pre-defined objects. We notice how the transformations and object creations are invoked in the following source code (obviously part of the main program, implemented within a “switch“ JAVA object) The resulting output sourcecode is based on VRML 97, mimed with a CosmoPlayer© plug-in. Depending on initial rules, such a model can quickly “run out of control” and generate huge 3D databases. Its specific initial generative inputs are the only condition for the whole evolution process – which is meanwhile eminently determinist; nevertheless, geometry partial overlaps are frequent and due to concatenated affine transformations previously described. Hereby we show a four-steps generated VRML model, made of solely 2 architectural primitives. Some extra visual artefact is provided by the height change of the objects, depending on their distance to the first geometric settlement.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 4. A L-System-based growth engine.

Most of these generative models are developed whithin a web browser interface: a javascript code which dinamically generates a VRML source displayed by a CosmoPlayer plugin. We are studying by now other geometrical algorithms, in order to constrain these Lsystem, such as Voronoï diagrams or Delaunay triangulations.

3.3. Random or Pseudo-Random “Hull-Filler“ Random or Pseudo-Random “Hull-Filling” Generators for Single-Building Construction The “hull-filling” model offers by itself rather interesting investigative perspectives: in this model the specific positioning of architectural types or sub-types could be guided by a prior analysis that tends to break down or disassemble some historically-contexted architectural

108

Renato Saleri Lunazzi

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

types by a morphological factorization. The process is obvously reversible and could be achieved by a rules-based grammar. The amazing Palladio 1.0 Macintosh© Hypercard Stack (Freedman 1990) is a noteworthy example of such a morphological synthesis. We also must here quote the scientific goal of the research team “Laboratoire d’Analyse des Formes“ from the architecture school of Lyon that leads somehow this specific aspect of this research task (Paulin – Duprat 1991) Their aim is to identify major stylistics guidelines from distinct architectural families, dispatching them through pre-identified morphologic, functional, architectonic and compositional occurrences (Ben Saci 2000). A similar search will soon commence, leaning on Claude-Nicolas Ledoux (1736 – 1806) architectural production, whose factorizable characteristics appear as an evidence.

Figure 5. A graph-based morphological parser. Courtesy of “Laboratoire d’Analyse des Formes“

Pseudo-Urban Automatic Pattern Generation

109

Figure 6. Some “hull-filled“ objects.

At the moment, this complex formalism is barely drafted; it is therefore interesting to point out the relevant difference of the “ugly duckling“ bottom right object, that descends from the same construction formalism but differs from 1 single input attribute.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

3.4. Multi-scale Pattern Generator A “top of the heap” wide range concentric propagator, whose aim is to distribute, filter and drop geometric locators above a given terrain mesh. The deal is here to develop a “general land-scaled model“, mostly a variant of the Lsystem model depicted above. The initial distribution of locators basically follows a concentric distribution. Their final positioning can be meanwhile modified by some disruptive factor, mostly depending on simple angular non-overlapping constraints. The graph below shows three different steps of the computation: locators displacement, neighbourhood tracking and plot drawing.

Figure 7. Deployment of a 2D geometric model.

110

Renato Saleri Lunazzi

A local geometric transformation transforms the initial structure to a position-related “constructible zone”, starting from two initial input variables, named here d’ and d’’ At the moment, inevitable angular occlusions occur whith sharp and wide angles. This drawback should meanwhile be solved in a very next release of the applet.

Figure 8. Geometric deduction of “constructible zones“.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Extracting the n closest neighbours and drawing the respective bijective connexions leads the entire process, and we can finally hybrid this bidimensional mesh to allocation rules and topopgraphic constraints, to produce the models shown on the figures below : the skeleton and the final rendering.

Figure 9. The geometric skeleton...

Pseudo-Urban Automatic Pattern Generation

111

Figure 10. …and it’s 3D expression.

In this example, only four architectural primitives are distributed over the map ; a “hull – filling “ generator or som MEL-based architectural objects (both shortly depicted above) could be implemented to create a more realistic perceptive variety.

4. Conclusion Virtual reality hardware and software costs and means are still relevant today. Trying to partially solve this peculiar aspect of leading 3D rendering techniques is part of the regional DEREVE project, whose aim is to build a convergent know-how, trying to extend hardware and software intrinsec performances through methodological and algorithmic applications, in terms of modeling and rendering. As a matter of fact, the specific involvement of the “MAParia“ lab in this research task deals whith 3D scenes building, leaning on his specific architectonic culture and virtual reality previous experimentations.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

References Barber, C.B., Dobkin, D.P., and Huhdanpaa, H.T., (1996) "The Quickhull algorithm for convex hulls," ACM Trans. on Mathematical Software. Batty M., Longley (1994) P.A., “Fractal Cities: A Geometry of Form and Function“, Academic Press, London and San Diego, CA. Ben Saci, A. (2000) “Théorie et modèles de la morphose“, Thèse de la faculté de philosophie sous la direction de B. Deloche, université Jean Moulin. Frankhauser P. (1997) “L’approche fractale : un nouvel outil de réflexion dans l’analyse spatiale des agglomérations urbaines “, Université de Franche-Comté, Besançon. Frankhauser, P. (1994) La Fractalité des Structures Urbaines, Collection Villes, Anthropos, Paris, France. Freedman, R. (1990) “Palladio 1.0“, Apple Macintosh© Hypercard Stack. Heudin, J.C. (1998) “L’évolution au bord du chaos“ Hermès Editions. Horling, B. (1996) “Implementation of a context-sensitive Lindenmayer-System modeler“ Department of Engineering and Computer Science and Department of Biology, Trinity College, Hartford, CT 06106-3100, USA.

112

Renato Saleri Lunazzi

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Khamphang Bounsaythip C. (1998) “Algorithmes évolutionnistes“ in “Heuristic and Evolutionary Algorithms: Application to Irregular Shape Placement Problem“ Thèse Public defense: October 9, (NO: 2336) Lindenmayer, A. (1968) “Mathematical models for cellular interactions in development“, parts I-II. Journal of Theoretical Biology 18: 280-315. Paulin, M. and Duprat, B. (1991). “De la maison à l’école, élaboration d’une architecture scolaire à Lyon de 1875 à 1914“, Ministère de la Culture, Direction du Patrimoine, CRML. Sikora S., Steinberg D., Lattaud C., Fournier C., Andrieu B. (1999) “Plant growth simulation in virtual worlds : towards online artificial ecosystems. Workshop on Artificial life integration in virtual environnements“. European Conference on Artificial Life (ECAL’99), Lausanne (Switzerland), 13-17 september. Torrens, P. (2000) “How cellular models of urban systems work” , CASA.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 113-119

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 10

TONAL STRUCTURE OF MUSIC AND CONTROLLING CHAOS IN THE BRAIN Vladimir E. Bondarenko Department of Physiology and Biophysics, School of Medicine and Biomedical Sciences, SUNY at Buffalo, 124 Sherman Hall, 3435 Main Street, Buffalo, NY 14214, USA

Igor Yevin* Mechanical Engineering Institute, Russian Academy of Sciences, 4, Bardina, Moscow, 117324 Russia.

Abstract

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Recent researches revealed that music tends to reduce the degree of chaos in brain waves. For some epilepsy patients music triggers their seizures. Loskutov, Hubler, and others carried out a series of studies concerning control of deterministic chaotic systems. It turned out, that carefully chosen tiny perturbation could stabilize any of unstable periodic orbits making up a strange attractor. Computer experiments have shown a possibility to control a chaotic behavior in neural network by external periodic pulsed force or sinusoidal force. We suggest that music acts on the brain near delta-,teta-, alpha-, and beta frequencies to suppress chaos. One may propose that the aim of this control is to establish coherent behavior in the brain, because many cognitive functions of the brain are related to a temporal coherence.

1. Introduction Investigations of human and animal electro-encephalograms (EEGs) have shown that these signals represent deterministic chaotic processes with the number of degrees of freedom from about 2 to 10, depending on the functional state of the brain (awaking, sleep, epilepsy). Recent investigations [1,2] revealed that music tends to reduce the degree of chaos in brain waves. For some epilepsy patients music triggers their seizures. Loskutov [3], Hubler and co-workers [4] and others studied control of deterministic chaotic systems. It was found *

E-mail address: [email protected], Phone: (095) 5760472

114

Vladimir E. Bondarenko and Igor Yevin

that carefully chosen tiny perturbation could stabilize any of unstable periodic orbits making up a strange attractor. Computer experiments have shown the possibility to control chaotic behavior in neural networks by external periodic pulsed force or sinusoidal force [5,6]. We suggest that indeed the stable steps of music tonalities and appropriate chords are those tiny perturbations that control chaos in the brain. Any musical score might be considered as a program of controlling chaos in the brain. One may propose that the aim of this control is to establish coherent behavior in the brain, because many integrative cognitive functions of the brain are related to a temporal coherence [7].

2. Control of Chaos in the Brain by Sinusoidal or Periodic Pulsed Force The neural network model is described by a set of differential equations [5,6]: M

u i (t ) = −u i (t ) + ∑ aij f (u j (t − τ j )) + e sin ω e t , j =1

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

i, j = 1,2,..., M ,

(1)

where ui(t) is the input signal of the ith neuron, M is the number of neurons, aij are the coupling coefficients between the neurons, τj is the time delay of the jth neuron output, f(x) = c tanh(x), e and ωe are the amplitude and frequency of the external force, respectively. We studied the case when the all τj are constant (τj = τ). The coupling coefficients are produced by random number generator in the interval from –2.048 to +2.048, the coefficient c is used to vary coefficients aij simultaneously. The forth-order Runge-Kutta method, with the time step h = 0.01, is used for solution of equation (1). Small random values of ui(0) are chosen as the initial conditions. For the time t in the interval from −τ to 0, ui(t) are equal to zero. Time series of N = 100000 and N = 8192 points are analyzed after the steady state is reached. The frequency spectra are calculated using the ordinary digital Fourier transform. For the evaluation of the correlation dimension ν the Grassberger-Procaccia algorithm is used. According to this algorithm, the time series of single neuron's inputs are analyzed. The sampling frequency is chosen so that each significant spectral component should have at least 8-10 sample points on the time period. For calculation of the largest Lyapunov exponent in M-dimensional phase space, two trajectories are computed from the equation (1): unperturbed u0(t) and perturbed uε(t). For the calculation of perturbed trajectory after reaching the steady state, the small values εui are added to ui. Here ε is in the range from 10-14 to 10-6. The largest Lyapunov exponent is defined as

λ = lim lim t −1 ln[ D(t ) / D(0)] t → ∞ D ( 0 ) →0

Tonal Structure of Music and Controlling Chaos in the Brain

115

where ⎤ ⎡M D (t ) = ⎢∑ (u iε (t ) −u i 0 (t )) 2 ⎥ ⎦ ⎣ i =1

1/ 2

⎡M ⎤ D(0) = ⎢∑ u iε (0) − u i 0 (0)) 2 ⎥ ⎣ i =1 ⎦

1/ 2

are the distances between the perturbed and unperturbed trajectories at the current and the initial moments, respectively. The largest Lyapunov exponent λ is calculated from time series of N = 100000 points. We start from the case when the amplitude of the external force e = 0.0. Under this condition, the neural network produces chaotic output with the correlation dimension ν = 5.2 − 7.1 (depending on the ordinal number of the neuron) and the dimensionless largest exponent λ = 0.017. The peak frequencies in the cumulative spectra of 10 neurons are in the ratios of 0.12:0.28:0.46:1.04 (Fig. 1).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 1. Spectra of the outputs for all ten neurons without an external action: M = 10, c = 3.0, e = 0.0, τ = 10.0.

Similar ratios of main rhythms of the human EEG (delta-, theta-, alpha-, and beta rhythms) are observed in the experiments also: 2.3:5.5:10.5:21.5 [5]. Application of the external sinusoidal force to this neural network changes the output from relatively high-dimensional chaotic (ν ι 5 − 8, λ > 0) to low-dimensional chaotic (ν ≤ 3, λ > 0), quasiperiodic (ν ≤ 3, λ ≈ 0), or periodic ones (ν ≈ 1, λ ≈ 0) [6]. As a rule, the low-dimensional outputs are observed when the frequency of the external force is close to the eigenfrequency of self-excited oscillations in the neural network without an external action (Fig. 2). One may expect, therefore, that music acts on the brain near these eigenfrequencies or its harmonics, because considerably smaller amplitudes of the external forces are necessary to suppress chaos in the case of resonance, than without resonance.

116

Vladimir E. Bondarenko and Igor Yevin

But our neural network has only four eigenfrequencies whereas piano has over 80 keys producing more than 80 different frequencies. In order to resolve this contradiction, the attractor network model of music tonality is proposed that is based on Hopfield’s model of associative memory.

Figure 2. Correlation dimension ν (a) and the largest Lyapunov exponent λ (b) as functions of external force frequency ωe: M = 10, c = 3.0, τ = 10.0, e = 7.0.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

3. Model of Music Tonality Using Hopfield's model, we can consider pitch perception as a pattern recognition process. It gives us an ability to explain why notes with octave interval we hear as very similar. When we hear, for instance, note "C" in different octaves, we recognize very similar sound patterns, keeping in mind complex overtone structure of every musical note. In other words, sound patterns of notes divisible by octave are the most similar among all others notes and therefore belong to the same basin of attraction and precisely by this reason we hear notes divisible by octave as very similar. Tonality is a hierarchy (ranking) of pitch-class. If the only pitch-class is stressed more than others in a piece of music, the music is said to be tonal. If all pitch-classes are treated as equally important, the music is said to be atonal. Almost all familiar melodies are built around a central tone toward which the other tones gravitate and on which the melody usually ends. This central tone is the keynote, or tonic. Three stable steps of tonality: tonic, median, and dominant are prototype patterns or attractors

Tonal Structure of Music and Controlling Chaos in the Brain

117

of neural network model. Others steps of tonality: subdominant, submediant, ascending parenthesis sound, descending parenthesis sound play the role of recognizable patterns, gravitating to some or other prototype pattern [8,9].

Figure 3. Hopfield's potential function E for major tonality in Western tonal music.

The degree of instability (the degree of gravitation to appropriate stable state) depends on distances between unstable and stable sounds. The strongest gravitation of VII step to I step and of IV step to II step are observed (Fig. 3).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 4. Potential function E for minor tonality in pentatonic scale.

There are no semitone (half step) intervals between notes in music of some Eastern countries (for instance, in China, Vietnam, Korea) (Fig. 4). Such pitch organization is called pentatonic. Though pentatonic is more ancient than modern Western tonality system (Fig. 3), we can formally obtain major and minor tonalities in pentatonic by removing IV and VII steps from diatonic major and minor tonalities. For the lack of minor seconds intervals in a pentatonic scale there are not such strong gravitation as in a natural scale [9,10]. Because western and pentatonic systems of tonalities recognition have the same potential function, we may suggest that this potential function is formed not by music, but is an inherent property of brain functioning. It is reasonable to suggest that that all kinds of major tonalities gravitate to the one basin of attraction and all kinds of minor tonalities gravitate to the other basin of attraction.

4. Stable States of Tonalities and Resonance Action Because music acts on the brain as external force we may depict the action of major tonalities through the auditory nerve on neural network in the following way (Fig. 5):

118

Vladimir E. Bondarenko and Igor Yevin

Figure 5. Resonance action of major tonalities, ω is the frequency of spike trains in auditory nerve.

It means that the frequencies of spike trains, corresponding to tonic, mediant, and dominant in major tonalities in auditory nerve coinside with the frequences of delta-, alpha-, and beta rhythms of the brain, respectively. We hope this is a plausible assumption. The action of minor tonality on the brain we may depict as follows (Fig. 6):

Figure 6. Resonance action of minor tonalities, ω is the frequency of spike trains in auditory nerve.

In this case the frequencies of tonic, mediant, and dominant of minor tonalities coincide with delta-, theta-, and beta rhythms in the brain. The total action of music consisting of major and minor tonalities we may represent in the following way (Fig. 7):

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 7. Resonance action of major and minor tonalities

Hence, we have four different music frequencies acting as external forces on four different eigenfrequencies of neural network. As well known, interval structure of major and minor triads are the same as stable steps interval structure of corresponding tonalities. It means, that the action of these triads is reduced to simultaneous resonant action on delta-, teta-, alpha-, and beta frequencies.

References [1] [2]

N. Birbaumer, W. Lutzenberger, H. Rau, G. Mayer-Kress, and C. Braun, “Perception of music and dimensional complexity of brain activity,” International Journal of Bifurcations and Chaos, vol. 2, no. 6, pp. 267-278, 1996. A. Patel and E. Balaban, “Temporal patterns of human cortical activity reflect tone sequence structure,” Nature, vol. 403, no. 6773, pp. 80-84, 2000.

Tonal Structure of Music and Controlling Chaos in the Brain [3]

119

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

V.V. Alexeev and A.Yu. Loskutov, "The destochastization of a system with strange attractor by a parametric action" Moscow University Phys. Bull., vol. 26, no. 3, pp. 4044, 1985. [4] A.W. Hubler and E. Lusher, “Resonant stimulation and control of nonlinear oscillation,” Naturwissenschaft, no. 76, pp.67-74, 1989. [5] V.E. Bondarenko, “Analog neural network model produces chaos similar to the human EEG.” International Journal on Bifurcation and Chaos, vol. 7, no. 5, pp.1133-1140, 1997. [6] V.E. Bondarenko, “High-dimensional chaotic neural network under external sinusoidal force,” Physics Letters A, vol. 236, no. 5-6, pp. 513-519, 1997. [7] W. Singer, “Neuronal representations, assemblies and temporal coherence,” Progress in Brain Research, vol. 95, pp. 461-474, 1993. [8] I. Yevin and S. Apjonova, “Attractor network model and structure of musical tonality,” Abstracts of the 9th Conference Society Chaos Theory in Psychology and Life Sciences, Berkeley, CA, USA, July, 1999. [9] I. Yevin, What is Art from Physics Standpoint? Moscow: Voentechizdat, 2000 (in Russian). [10] I. Yevin, Synergetics of the Brain and Synergetics of Art, Moscow: GEOS, 2001 (in Russian).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 121-130

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 11

"Wavy Texture 2" Antelope Canyon USA photographed by Jin Akino

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

COLLECTING PATTERNS THAT WORK FOR EVERYTHING Deborah L. MacPherson* Independent Curator, 118 Dogwood Street, Vienna VA 22180-6394

Abstract Would we even want a meta-methodology or collection such as “patterns that work for everything”? One simple evolving system of explanation and conceptual illustration? Where would these patterns reside? Who would interpret them? There are concepts being developed *

E-mail address: [email protected] 703 242 9411 and 703 585 8924, www.accuracyandaesthetics.com, www.contextdriventopologies.org

122

Deborah L. MacPherson in the study of chaos and complexity that may help make arrangements for this collection. In particular, a glimpse at what the patterns might look like and act like. Maybe they also act like music, maybe we can discuss, present and interpret abstract information patterns the meticulous way we discuss, present and interpret abstract art. If you stuck a pin in today and drew back to the time when physics, chemistry and biology were one - what are we truly capturing about chaos and complexity for the corresponding point in the future? Are today’s algorithm writers yesterday’s alchemists and what is the best, least constrained and highest quality way to preserve the fundamental and esoteric qualities of this work for future studies? Can we imagine and develop an inherited collective memory for our machines, like language and culture are for us, to pass stories from one generation to the next? Even if they speak different languages and live in different places as we do, something we can all measure may be generated by providing an unsupervised opportunity for our machines to create or illustrate patterns we have not thought about yet, noticed or engineered. There is a story in the study of chaos and complexity that may be able to tell itself.

What Do We See and How Are We Telling This Story?

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Below is Robert May’s early glimpse presented in American Naturalist (1976). What if - even though so much high quality, rich and diverse information has been generated, presented and represented since the generation of this diagram - what if this is still an accurate portrayal of what we can see even with all of the new information? As the source of this diagram, we can assume that May cared deeply about this new science, that he was more convinced about his emerging ideas then anyone else could be, and he was committed to figuring these ideas out an accurate, arguable, mathematical way.

Figure 1. Bifurcations and Dynamic Complexity in Simple Ecological Models by Robert May in American Naturalist (1976).

Which elements of chaos and complexity studies are so fundamental and essential that together they sketch an overall? Which are the important intricacies? Each person will have a slightly different interpretation. These combinations and points of view about what is “important” are the never ending discussion and debate that signify progress in all domains. To capture legitimate progress and new ideas in the literal sense of preservation, we can also assume the most accurate record of chaos and complexity SCIENCE are the technical papers and any code we can still read. However, the literature alone does not completely describe

Collecting Patterns That Work for Everything

123

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

this branch or attempt to explain why people have dedicated such passionate thought to it. One reason Chaos, Making a New Science (Gleick, 1987) was a best seller so long is that this is also such a compelling STORY. The concepts did seem new, and obvious, which is rare. Stories are allowed to include pure commentary simply because these are interesting details, no other reason. A technical paper wants to eliminate unnecessary distractions. Methodology, related work and open questions are carefully annotated to form a context that justifies where the work belongs. Source code is now required with most journal submittals; words and .jpgs of algorithms and equations are no longer enough for a thorough review. The form of the continuous discussion and debate has changed as much as the topics being argued and we are not done yet. Any scientific body of work has always been an evolving web of interconnections that is very complex, today we just have more efficient ways of looking. For example, there have been massive improvements in navigating related work. Scientific digital libraries such as Cite Seer and ScienceDirect are not only thick with searchable publications, but customized alerts for topics of interest are available, users can access techniques and contact the authors with questions. Dealing with specific, complex and abstract information has become a much more interactive and precise process. Extracting a research thread from a digital library is like running on a hamster wheel, one piece of evidence leads to three more. Fortunately, the convenient units that research threads can be now broken into, away from entire books and journals, makes the content much easier to sift through when pulling together and justifying a new whole. Regardless of technical and communication improvements, the problem of deciding which work is related and why will never be “solved”. If systems and machines are to help us contextualize reasoning, presumably like journal referees, they would also insist on more to analyze than text in/text out. It would be progressive to engineer and be able to manipulate algorithms in/algorithms out, imagery in/imagery out, transformations in/transformations out and of course mixing and matching different proportions and hierarchies of the essential components. Specific hierarchies and combinations could only be recognized in context, the most useful metric would be proportion because proportion often indicates design.

Figure 2. a) “Delaware Gap” by Franz Kline b) “Pollen from Hazelnut” by Wolfgang Laib.

These artworks are being compared because one has no color, one is all color. One is fixed, the other could blow away. One is on the wall, the other on the floor. One is exactly the same as an archive, the other changes form completely. Their proportions are a similar scale

124

Deborah L. MacPherson

in relation to the viewer and they are both in the permanent collection of the Hirshhorn Museum and Sculpture Garden, Smithsonian Institution. How should they be digitized? Sometimes, regardless of which systems, machines or measurements are available, the concepts themselves may be so abstract or complicated that it becomes an extraordinary challenge just bringing a sensible group together. It is nearly impossible to be objective relating new ideas to familiar ideas, this is part of what is making it a new idea. At a Roundtable Discussion held at the Kreeger Museum, Olga Viso (2003), Deputy Director and Curator of Contemporary Art at the Hirshhorn Museum and Sculpture Garden, described evaluating contemporary art: “Sometimes you are not sure what you are looking at, so you need re-look at it, then look at it again”

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Olga Viso is not carefully examining these abstract complicated objects and ideas just to see or count them – her purpose is to make decisions and draw conclusions. Like theorists and detectives, a curator identifies or proposes new patterns, is engaged in a different kind of internal and external dialogue. Our new ability to share deeply interpretive information also gives us new reasons to look again and again at these circular patterns and dialogues. We are on the verge of a new way to discuss which patterns and dialogues have value; which objects, information and ideas we should provide care for; try to stop time and conserve so they can be interpreted again later with a fresh perspective and historical comprehension. A museum of any type has unlimited examples why critical selections and an interesting story are necessary with objects. Some objects museums are responsible for are quite fragile, it is safer to look at a copy, but there are already too many objects to look through let alone interpret, never enough resources to care for the originals not to mention the copies, therefore it serves very little purpose trying to “keep it all”. Like scientific ideas going in and out of favor, eventually museums can only focus on high quality originals, try to cover as much as possible, fill in gaps and build bridges between different aspects of the collection. The science and story of chaos and complexity is like a collection with many interpretations that would be very difficult to keep in one place. Decisions about scientific relevance, or exactly what constitutes proof, are made by huge numbers of people over time. Only the media and machines are fragile, yet there is no reason to care for or conserve them, our digital culture demands they improve.

Machines The Smithsonian Institution has consolidated a History of the Computer & Internet Resources. This “stuff” resides in many locations, is composed of a never ending diversity of encodings on various unstable materials and quaint artifacts that are not expected to perform. Many of the machine languages have been lost and most people do not miss them. The unwritten history also includes an astronomer’s “after paper” 999 dimension data array sitting in a drawer collecting dust, going obsolete. It includes someone curious playing around with genetic algorithms just to see what happens. There is an enormous portion of potentially relevant, interesting, complex information that is only partially interpreted and therefore may not be upgraded to meet new standards. The number of inspiring occurrences that were never

Collecting Patterns That Work for Everything

125

recorded is beyond measure. Does it mean this information, or potential information, is not valuable or possibly even important? So many finely detailed histories, new sketches and views have been enabled by our fickle relationship with machines. They can really spark our imagination but never ask “What are you measuring? Why are you measuring it? What is your method? Justification? Reason? Do you have funding? Has anyone else measured this? What can you show me?” They do not wonder what the best, most accurate interpretative record of emergence, chaos and complexity is. They have no collective memory or inside influences, they just perform. Which components of this now well established science cannot be recorded, preserved or represented without machines? Possibly none, but where is their voice in this democracy? As they evolve, are abandoned and replaced, most of their imagery is still limited to a backlit screen, their languages are illegible, they never get enthusiastic or bored yet they are also readers, recording our information patterns, always there. People talk about feedback loops, self similarity, unlimited variables and the effect of initial conditions but the encoding and representation of information patterns of all types feel like we are always starting in the same place, the transactions are constrained to equal packets working on a clock. Certain ways of thinking cannot be captured this fractured, regimented way. Maybe the patterns themselves can show us how to characterize this kind of information to help us to see new ways it is related.

Presentation and Representation People are always deliberately inventing new ways to express, figure out and present what we are thinking about. At "Look Up! "Chaos" Comes to New York" held at the CUNY Graduate Center December 2003, Jim Crutchfield and David Dunn described creating the Theater of Pattern Formation:

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

“… a comprehensive strategy for the visual and auditory articulation of scientific and mathematical research in the fields of complex systems and nonlinear dynamics or "chaos.” It explores naturally occurring patterns in nature and mathematics and how they can be seen within the aesthetic traditions of the arts.”

Figure 3. From the “Theater of Pattern Formation” by James P. Crutchfield and David Dunn (2003), a large-scale multichannel video-audio exploration of structure and emergence in the spatial and acoustic domains. The target venues are sensory-immersive all-digital dome theaters. Image used with permission from Dr. Crutchfield.

126

Deborah L. MacPherson

To get this presentation to work, not only were there the technicalities of getting the audio and visual patterns to influence each other, but also issues related to “stitching” together views, removing or faking distortions for the dome, the originals had to be developed in a round space, not on a computer screen. The results presented inside the dome sound like they will be effective. New kinds of presentations such as the Theater of Pattern Formation feel like they are getting more true to the form of certain patterns and are definitely more compelling both to people who understand the underlying mathematics and people it simply appeals to. These sounds and images are slowly entering our popular consciousness and how can that be turned into something useful? We didn’t have the search engine Google before, now people cannot imagine being without this way of looking. We are quick to learn a new way when it is useful. School groups go to into planetarium presentations and rip things apart with their enthusiasm and energy. Their adult counterparts do the same monitoring the literature. Our anthropocentric collective understanding is continually being clarified, explored, shredded, discarded, updated, and reflected through our modes of presentation and representation – these modes will not stay the same or ever be enough for developing and presenting new ideas.

Machine Aesthetics

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

How can machines help us ponder on and sort through patterns that might work for everything to help us establish standards and convenient units to interpret and preserve them in the future? We do not generate many tools to examine or establish overalls while we are still looking through little windows of order, generating and collecting pieces. How would a machine auto-measure context, conceptual relationships and overalls? To what extent are we comfortable with their style of brute force fussy dialogue going unsupervised? What might they notice and classify as interesting or relevant that is different than we would think of looking for?

Figure 4. “The Administration Building” by Michael Leyton.

Collecting Patterns That Work for Everything

127

If machines have some share in the responsibilities of cleaning our complex and chaotic information basement, deem something redundant and eliminate it, will it be that hideous sweater that truly, should never be worn again - or will it be a forgotten photograph? Can we trust them to consolidate what we are currently unable to perceive as either embarrassing or precious because we are in the middle of it and cannot see everything? What can they help us get rid of in a way we can accept?

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 5. Neolithic pottery from the Museum of Almería, Spain.

We cannot just “keep all of the patterns” nor does that serve any useful purpose. Even if we are not sure we are able to recognize fundamental or essential meaning in the data, information and patterns we have now, there is at least one time when one person and one machine evaluated something that looked interesting in the data. Maybe they were not even sure why, it just felt like it, maybe it was just easy for the machine to handle. We should protect these original combinations to look at again later with our new machines. A preservation effort of this type would not be to understand the past, but to participate in the future. The digitization and automated experiment craze presents a one-time opportunity to collect more now than will be proven to have value later when unfortunately, the traces we have left may be of such low quality that we accidentally infer the wrong things. We could put a broken piece of clay under sophisticated lighting pretending it is important only to discover later that more valuable works have been lost protecting this one. One fragile video tape by Pavel Hlava might have been the only imagery of the first plane hitting the World Trade Center, now there are so many copies and we understand what happened that it will be preserved by perpetuation. Data may be able to auto-perpetuate as it is distributed but it cannot auto-contextualize without characterization. There is no reason to limit salient data features to unique identifiers. There are variations in texture, density and alignment that machines can register more precisely than we can identify. Where can we establish boundless sets of endlessly intricate questions, experimental setups, data components, conclusions and patterns for curious creative people and our never ending parade of media and machines to fool around with just to see what might be sifted

128

Deborah L. MacPherson

out? If patterns that work with and supposedly represent everything were to be collected, analyzed, compared or just reflected upon, where could they be assembled or kept together in groups without generating too many copies? We could save only the context since most virtual information is a copy already. An image, description and measurements of a painting will never be as good as the real painting by itself. Source code that compiles very nicely does not put the reader inside the scientist’s head when the discovery was made. If information patterns that register this kind of thinking were anything like music, how can we use them to auto-eliminate noise aesthetically? Get machines to recognize the patterns we prefer, continue talking about and connecting with each other, get them to learn our aesthetics?

Redundant and Similar Information If it is even possible to have a comprehensive body of chaotic and complex patterns to represent all fields of inquiry it would need to be limitless, open and not restricted to certain languages. The system would be more similar to the act of translation than any sets of natural and machine languages. Scientists, scholars and the curious are actively generating a limitless collection of obvious or elusive relationships just by thinking about, categorizing and engineering their data, turning it into information, trying to add meaning to it. Then everybody starts to discuss and debate it. Maybe we can devise a mechanism to let this change the way data is perceived. Throughout the process of discovery, acceptance, rejection and perpetuation of information, there are many components that are similar. If we can use these similarities and conflicts to streamline and train the information space to automatically defer to the denser, higher quality, more original information and auto-delete the copy; this will not only protect the combinations that actually work, but will also help us to decide about and preserve what is actually important.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 6. Human chromosomes from www.nature.com.

Conclusions We have no current standards or shared systems to store and analyze unrelated chaotic or complex information in the partially interpreted state. Everyone is too busy, the patterns are confusing and there are too many. If we can get these patterns to play on their own, look at them again and again in a different relationship with our machines, maybe we can simplify them together. Patterns that work for everything are like intricate artifacts that will eventually become familiar. A collection of them might appear as mathematical patterns and metapatterns to machines but could be transformed and presented to us any way we prefer. A systematic logic of hierarchy and flow to interpret these patterns on abstract levels is dependent on cycles, fading away and replacement. We should not keep these perplexing records locked in the chunks of granite that are the current style of metadata. Modern

Collecting Patterns That Work for Everything

129

information patterns need to be more fluid and effect the other information around them. Like an artist working on a sculpture, as usual, there is too much there. Any system to collect patterns that work for everything would serve the explicit purpose of taking away, streamlining, making it elegant, beautiful, and not like something someone else already made. When complex or chaotic information qualifies for the last rounds of selection and we are left with only the context and essential components - each symbol, mark, word, arrangement, equation and level needs to count, be in their original state. There is no one “place” for context driven topologies, concept maps, or patterns that work for everything. They can only reside in our imagination, mathematical codes and communicative forms capable of binding these together. Techniques usually only improve, let us define a way for abstract information patterns to self-perpetuate, self-contextualize so we can keep only the highest resolution possible for the time when we are ready to see them.

Image Acknowledgements "Wavy Texture 2" Antelope Canyon USA by Jin Akino May 2001, courtesy of the photographer May RM and Oster G (1976) Bifurcations and Dynamic Complexity in Simple Ecological Models. American Naturalist 110, 573-599 “Delaware Gap” by Franz Kline (1958) and “Pollen from Hazelnut” by Wolfgang Laib (1998-2000)both from the permanent collection of the Hirshhorn Museum and Sculpture Garden, Smithsonian Institution “Sample Fractal” from The Theatre of Pattern Formation by James P. Crutchfield and David Dunn (2003) at the Art & Science Laboratory. Image and text used with permission of Dr. Crutchfield. “The First Administration Building” by Professor Michael Leyton, image provided by the artist “Argaric Neolithic Pottery” on display at the Museum of Almería by Manuel Salas Barón “Human Chromosomes” from www.nature.com

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

References Gleick J (1987) Chaos, Making a New Science Viking Penguin ISBN 0 14 00.9250 1 May RM and Oster G (1976) Bifurcations and Dynamic Complexity in Simple Ecological Models. American Naturalist 110, 573-599

Internet Jin Akino http://www.internetacademy.co.jp/~yesaki/ CiteSeer http://citeseer.ist.psu.edu/cis ScienceDirect http://www.sciencedirect.com/ Hirshhorn Museum and Sculpture Garden Smithsonian Institution http://hirshhorn.si.edu/

130

Deborah L. MacPherson

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Smithsonian History of the Computer & Internet Resources http://www.sil.si.edu/subjectguide/nmah/histcomput.htm Google www.google.com Theater of Pattern Formation http://atc.unm.edu/research/asl/asl.html Art & Science Laboratory http://www.artscilab.org/ Michael Leyton http://www.rci.rutgers.edu/~mleyton/homepage.htm Museum of Almería by Manuel Salas Barón http://members.tripod.com/~indalopottery/history.htm Human Chromosomes www.nature.com

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 131-142

ISBN 978-1-60456-841-7 c 2009 Nova Science Publishers, Inc.

Chapter 12

S TABILITY C ONDITIONS IN C ONTEXTUAL E MERGENCE Harald Atmanspacher1 and Robert C. Bishop2 1 Institute for Frontier Areas of Psychology Wilhelmstr. 3a, 79098 Freiburg, Germany and Parmenides Foundation Via Mellini 26–28, 57031 Capoliveri, Italy 2 Department of Philosophy, Rice University, Houston, TX77251, U.S.A.

Abstract

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The concept of contextual emergence is proposed as a non-reductive, yet welldefined relation between different levels of description of physical and other systems. It is illustrated for the transition from statistical mechanics to thermodynamical properties such as temperature. Stability conditions are crucial for a rigorous implementation of contingent contexts that are required to understand temperature as an emergent property. It is proposed that such stability conditions are meaningful for contextual emergence beyond physics as well.

1.

Introduction

A basic strategy for the scientific description of any system, physical or otherwise, is to specify its state and the properties associated with that state, and then introduce their evolution in terms of dynamical laws. This strategy presupposes that the boundary of a system can be defined with respect to its environment, although such definition is often seriously problematic. If a system can be defined reasonably, there is usually more than one possibility for specifying states and properties. The fact that states and properties can be formally and rigorously defined in fundamental physical theories, such as quantum mechanics, distinguishes the structure of such theories as particularly transparent. The situation is different in physical theories which are not regarded as fundamental (such as thermodynamics), or in descriptive approaches beyond physics (such as chemistry,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

132

Harald Atmanspacher and Robert C. Bishop

biology or psychology). For this reason, attempts have been made to relate descriptions of systems, which are not fundamental in the sense mentioned above, to descriptions which are fundamental in this sense. The usual (and often too simple) framework in which corresponding relations are typically formulated is that of a hierarchy of descriptions. In a hierarchical picture (which can be refined in terms of more complicated networks of descriptions) there are higher-level and lower-level descriptions. More fundamental theories are taken to refer to lower levels. In such a simple framework, reduction and emergence are relations between different levels of descriptions of a system, its states and properties, or the (dynamical) laws characterizing their behavior. In the philosophical literature, the usual guiding idea behind reductionist approaches is to “reduce” higher-level features to lower-level features. By way of contrast, emergentist approaches emphasize the higher-level features by stressing the irreducibility of at least some of their aspects to lower levels. In this way, the emergence of features at higher levels is related to the emergence of novelty. While reductionists would argue that both necessary and sufficient conditions for higher-level features are already embodied at the lower level, this is false in many of the more important examples (e.g. thermodynamics and statistical mechanics). An alternative kind of interlevel relation, contextual emergence, was recently proposed (Bishop and Atmanspacher, 2006) as a less rigid, more appropriate scheme, in which necessary but not sufficient conditions for higher-level features are provided by the lower-level description. Stability conditions are crucial guiding principles for contextual emergence, which might be helpful for applications beyond physics as well. Specifically, one may think of relations between different levels of descriptions in brain physiology, where one of the key questions is how properties of neuronal assemblies (i.e. populations of neurons) are related to properties of individual neurons and synapses. However, one may also think of relations between such neurobiological levels of description and their mental correlates at cognitive or psychological levels of description. An interesting candidate for interlevel relations of the latter kind will be presented elsewhere (Atmanspacher and beim Graben, this issue). Here we start with a brief introduction to the idea of contextual emergence and compare it with other kinds of interlevel relations in section 2. Section 3 outlines the general mathematical framework in which contextual emergence can be worked out for detailed examples. A particularly well-known example is presented in section 4, where we illustrate the formalism with details regarding the contextual emergence of temperature (and related thermodynamical properties) from a description in terms of statistical physics. The role of stability conditions for contextual emergence will be emphasized in section 5. Section 6 summarizes the basic arguments and results.

2.

Reduction and Emergence

Reduction and emergence are used in a variety of senses in the literature. In general terms, both concepts express ways to achieve a better understanding of some feature of a system in terms of other features which are assumed to provide such understanding. For the sake of simplicity, reduction and emergence schemes are typically organized in a hierarchical manner, such that levels of description or levels of reality are related to each other. As mentioned above, an analysis in terms of hierarchical levels often oversimplifies the picture.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Stability Conditions in Contextual Emergence

133

In general, non-hierarchical frameworks (cf. G¨unther 1976–1980) including other notions such as those of domains of description or domains of reality might be more appropriate. As indicated by the distinction between levels of description and levels of reality, there is a difference between epistemological and ontological frameworks for reduction and emergence. Broadly speaking, descriptive terms are subjects of epistemological discourse while elements of reality are subjects of ontological discourse. Both types of discourse are used in reductionist and emergentist approaches. The concept of reference establishes a connection between descriptive terms and described elements of reality (leaving aside difficult questions about reference itself). In addition to the distinction between epistemological and ontological discourse, one should distinguish between different types of features which are to be related to others. There are three main categories of relations: theories/laws to other theories/laws, properties to other properties, and wholes to parts. Clearly, relations between theories/laws are predominantly epistemological. The relation between wholes and parts, on the other hand, is primarily conceived ontologically insofar as it emphasizes elements of reality rather than their description. In the literature on property relations, both epistemological and ontological frameworks can be found. Property relations are sometimes meant ontologically (i.e., regarding properties of elements of reality) and sometimes epistemologically (i.e., regarding descriptive terms referring to properties of elements of reality). An ontological framework of discussion is usually employed in reductive approaches, where ontic elements are restricted to a fundamental level of description, at which those properties reside to which all other properties are regarded reducible and from which all other properties are regarded as exhaustively determined. An alternative idea of a tiered ontology, ascribing ontic elements to all levels of description, was proposed originally by Hartmann (1935). Quine (1969) has revitalized this idea with his notion of an ontological relativity. It was adopted by Putnam (1987) when he suggested his idea of internal realism, later denoted pragmatic realism. These philosophical frameworks of thinking were fleshed out by Atmanspacher and Kronz (1999) from a scientific perspective. This option presupposes a distinction between ontic and epistemic descriptions of the behavior of physical systems due to Scheibe (1973) and Primas (1990). A comprehensive review can be found in Atmanspacher and Primas (2003). Analogous to Hartmann’s and Quine’s approaches, this allows us to conceive ontic elements at each level of description. In addition, however, it allows us to formally propose ways in which interlevel relations can be designed. In a nutshell, an ontic description at one level serves as the basis for an epistemic description at a higher level, which can be “ontologized” and then provides the basis for proceeding to another epistemic description at yet another level. (For details see Atmanspacher and Kronz 1999). If one wants to have the option of ontic elements at each level of description rather than only at a fundamental one, a straightforward and strictly reductive scheme for interlevel relations becomes impossible and must be relaxed. The way in which ontic and epistemic descriptions are related to each other motivates contextual emergence as a viable alternative. In order to clearly distinguish between different concepts of reduction and emergence, it is desirable to have a transparent classification scheme, so that their basic characteristics can be discussed coherently. A useful approach toward such a classification is based on the role which contingent contexts play in reduction and emergence. More precisely, the way

134

Harald Atmanspacher and Robert C. Bishop

in which necessary and sufficient conditions are assumed in the relation between different levels of description can be used to distinguish four classes of relations: (1) The description of features of a system at a particular level of description offers both necessary and sufficient conditions to rigorously derive the description of features at a higher level. This is the strictest possible form of reduction. It was most popular under the influence of positivist thinking in the mid-20th century. (2) The description of features of a system at a particular level of description offers necessary but not sufficient conditions to derive the description of features at a higher level. This version is called contextual emergence, because contingent contextual conditions are required in addition to the lower-level description for a rigorous derivation of higher-level features. (3) The description of features of a system at a particular level of description offers sufficient but not necessary conditions to derive the description of features at a higher level. This version includes the idea that a lower-level description offers multiple realizations of a particular feature at a higher level, which is characteristic of supervenience.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

(4) The description of features of a system at a particular level of description offers neither necessary nor sufficient conditions to derive the description of features at a higher level. This represents a form of radical emergence insofar as there are no relevant conditions connecting the two levels whatsoever. For obvious reasons, class (4) is unattractive if one is interested in explanatory relations between different levels of description. Non-reductive property dualism (e.g., Davidson 1980) would be an example of radical emergence. By contrast, class (1) is extremely appealing if one is interested in simple explanations. The “received views” of reduction – as Batterman (2002) refers to them – fall into this class (e.g., Nagel 1961, Schaffner 1976). From a contemporary point of view, classes (2) and (3) are viable alternative schemes for analyzing relationships between different levels of description. Supervenience relations, generally belonging to class (3),1 have been extensively discussed on the basis of Kim’s proposals (Kim 1993). Interestingly, Kim himself has recently argued that supervenience may be inadequate for capturing relations in the sciences (Kim 1998, 1999). This development has led to an emphasis on realization relations (e.g., Kim 1998, 1999, Crook and Gillett 2001, Gillett 2002). In the remainder of this contribution we will focus our discussion on class (2), contextual emergence, which is less rigid than the strong form of reduction (1) on the one hand and provides more structure for interlevel relations than radical emergence (4) on the other. 1

Some versions of supervenience require that changes in lower-level descriptions are both necessary and sufficient to bring about changes in a higher-level description. Such versions are indistinguishable from reduction (Kim 1998) and fall into class (1).

Stability Conditions in Contextual Emergence

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

3.

135

Contextual Topologies and Asymptotic Expansions

A precondition for achieving a formal relation between descriptions at different levels is a well-defined concept of states and properties of the system considered at those levels. The algebraic approach in physics offers such well-defined concepts. For example, in algebraic quantum theory, properties are introduced as so-called observables forming a C ∗ -algebra2 A over the complex numbers which is not commutative in general. The associated concept of a state is introduced in terms of a positive normalized linear functional on A. The state space of a fundamental theory in physics is chosen such that only the most basic assumptions are required for its definition. In other words, the state space is chosen as context-independent as possible. Contexts are contingent conditions referring to the degree of “abstraction” at which a theoretical framework is formulated. Each description requires “abstracting from”, i.e. disregarding, those details of a given system (and its environment) which are to be considered irrelevant. Needless to say, declaring particular features as irrelevant is not a universally prescribed procedure, but must be tailored to particular purposes or interests. Features which are irrelevant in a particular context may be highly relevant in another. For instance, temperature is an example of a feature that is relevant in thermodynamics but irrelevant in Newtonian or statistical mechanics. Light rays are relevant in geometric optics, but they are irrelevant in Maxwell’s electrodynamics. The chirality of molecules is relevant in physical chemistry, but it is irrelevant in a Schr¨odinger-type quantum mechanical description. Nevertheless, there are strategies for implementing the contexts due to which temperature is relevant in thermodynamics, due to which rays are relevant in geometric optics, and due to which chirality is relevant in physical chemistry, at the level of statistical mechanics, of electrodynamics, and of quantum mechanics. A natural way to represent contexts of these kinds is the modification of the original topology of the lower-level state space to a contextual topology (Primas 1998). The finest topology corresponds to the most fundamental context,3 e.g. given by “first principles,” while coarser topologies represent an increasing amount of contextual information not encoded in first principles. The key idea of relating properties at different levels of description to each other is to specify the difference between the descriptions in terms of the contextual topologies of their corresponding state spaces. Implementing a particular set of contexts as a contextual topology is usually nontrivial. A powerful tool often used for this purpose are asymptotic expansions (see Friedrichs 1955, Dingle 1973, Berry 1994, Batterman 2002). In order to formulate such an expansion, a reference state, which represents essential features of the context, has to be specified in the lower-level state space of the fundamental description. If the expansion is singular (i.e., diverges) in the intrinsic, fine topology of the fundamental description as an appropriate parameter tends to some limit, this indicates the need for a change of topology. Examples for such parameters are the number of degrees of freedom for thermodynamics (thermody2

A ∗ -algebra is an algebra admitting an involution ∗ : A → A with the usual properties. A ∗ -algebra is normed, if there is a mapping ||.|| : A → R+ with the usual properties. A complete normed ∗ -algebra is a Banach ∗ -algebra. A C ∗ -algebra is a Banach ∗ -algebra A with the additional property ||x∗ x|| = ||x||2 for all x ∈ A (see Takesaki 2002, chap. I.1). 3 For instance, in conventional quantum theory states are standardly represented in a Hilbert space endowed with the norm topology. This topology derives from the C ∗ -property as expressed in footnote 2.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

136

Harald Atmanspacher and Robert C. Bishop

namic limit), the wavelength for geometric optics (short-wavelength limit), or the electron mass divided by nuclear mass for physical chemistry (Born-Oppenheimer limit). The crucial point then is to identify a new topology which regularizes the expansion such that it converges. This leads to a contextual topology of the state space which is coarser than the original finer topology of the fundamental theory. This contextual topology is contingent in the sense that it is not given by the original finer topology or any other elements of the fundamental theory. The closure of such new descriptions in the contextual topology generates new context-dependent features not defined in the original state space under the finer topology (see next section for an illustrative example). Invoking a new contextual topology, then, accommodates novel features within a higher-level description rather than approximating them as a limiting case in the topology of a lower-level description. It is important to realize that “the task of higher level descriptions is not to approximate the fundamental theory but to represent new patterns” (Primas 1998, p. 87). In general, these patterns are not reducible to a more fundamental level in the strict sense of class (1). Such reducibility would mean that only the first principles of the fundamental description are needed to describe new patterns exhaustively. If higher-level contexts in addition to first principles must be considered in order to rigorously derive descriptions of these new patterns, reduction according to class (1) fails. In such cases, contexts are at least as important as first principles. The procedure described so far provides a formal approach for a contextual emergence of higher-level features on the basis of contexts in addition to the terms of the lower-level description. Qualitatively new, emergent features, unavailable in the lower-level description, manifest themselves at the higher level. In somewhat different terms, the new state space with coarser topology can be considered to be partitioned (i.e. coarse-grained) in a way allowing the definition of new states, together with associated new observables, represented by the cells of its partition (cf. beim Graben 2004, Atmanspacher and beim Graben, this issue). In this terminology, the choice of a proper partition is not prescribed at the lower-level description, but depends on the purpose of the partitioning and is usually based on concepts that are foreign to the lower-level description. This is, for example, the core idea in the construction of a symbolic dynamics within the theory of dynamical systems. (See Lind and Marcus (1995) for a comprehensive introduction to the field of symbolic dynamics.) In this sense we suggest considering emergent features within class (2) of the proposed classification scheme. The contextual emergence of such features can be physically motivated and made mathematically rigorous via contextual topologies. While necessary conditions for emergent features exist at the lower level, sufficient conditions, represented by contexts, do not exist at the lower level. Hence, emergent features cannot be derived or predicted from the lower level alone, even if exhaustive information concerning this level is assumed. Only the lower-level description plus the appropriate contextual topology renders emergent features (e.g., emergent properties) derivable or predictable.

4.

Thermodynamic Equilibrium and Temperature

This section discusses an example of contextual emergence in enough detail to see how contexts can be introduced leading to contextual topologies and emergent properties. Moreover,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Stability Conditions in Contextual Emergence

137

it is shown how necessary conditions for the emergence of novel properties are related to lower-level descriptions, whereas contingent contexts, not available at the lower-level description, represent sufficient conditions leading to well-defined properties at higher-order levels of description. Our example is the often discussed reduction or emergence, respectively, of thermodynamic properties such as temperature to or from properties at lower-level descriptions. The lower-level descriptions in this context are statistical mechanics and point mechanics. How are these levels of description related to thermodynamics? First, the less controversial issue: The step from point mechanics to statistical mechanics is essentially based on the limit of (infinitely) many degrees of freedom. That is, particular properties of a system are defined in terms of a statistical description (e.g., for many particles) and make no sense in an individual description (e.g., for single particles). An example is the mean kinetic energy of a system of N particles, which can be calculated from the distribution of the momenta of all particles. Its expectation value is defined in the limit of infinitely many particles, assuming the applicability of limit theorems (e.g., the law of large numbers). Any thermodynamic property whose definition is based on a statistical description presupposes (infinitely) many degrees of freedom. This applies to several properties, and temperature is a paradigmatic example. The concept of temperature is meaningless for systems whose number of particles is too small. The more controversial issue in discussing the reduction or emergence of temperature refers to the step from statistical mechanics to thermodynamics (cf. Compagner 1989). In many philosophical discussions it is argued that the thermodynamic temperature of a gas is the mean kinetic energy of the molecules which by hypothesis constitute the gas. According to Nagel, this leads to a straightforward reduction of thermodynamic temperature to statistical mechanics (Nagel 1961, pp. 341-345). Such a rough picture, however, is a gross mischaracterization, based on a too generous treatment of important details. First of all, thermodynamic descriptions presume thermodynamic, or briefly thermal, equilibrium as a crucial assumption which is neither formally nor conceptually available at the level of statistical mechanics. Second, the very concept of temperature is fundamentally foreign to statistical mechanics and has to be introduced, e.g., on the basis of phenomenological arguments. Thermal equilibrium is formulated by the zeroth law of thermodynamics: if two systems are both in thermal equilibrium with a third system, then they are said to be in thermal equilibrium with each other. Based on this equivalence relation, the phenomenological concept of temperature can be introduced in the usual textbook way. Since thermal equilibrium is not defined at the level of a mechanical description, temperature is not a mechanical property but, rather, emerges as a novel property at the level of thermodynamics. In this sense, the concept of thermal equilibrium serves as a context providing conditions for a proper discussion of temperature. This context is available at the higher-level description of thermodynamics. It can be recast in terms of a class of distinguished thermal states, the so-called Kubo-Martin-Schwinger (KMS) states, at the lower-level statistical description. These states are defined by the KMS condition which is equivalent to a variational principle, representing the stability of a KMS state against local perturbations.4 Hence, the 4 For more details concerning the significance of the KMS condition see Sewell (2002, chap. 5). The stability requirement imposed by the KMS condition is discussed in detail in Atmanspacher and beim Graben (this

138

Harald Atmanspacher and Robert C. Bishop

KMS condition implements a higher-level context in terms of a lower-level stability condition distinguishing states that are thermal in the sense of the zeroth law of thermodynamics. The second law of thermodynamics expresses this stability criterion in terms of a maximization of entropy. (Equivalently, the free energy of the system is minimal in thermal equilibrium.) If a system is in a KMS state, then this state is the canonical Gibbs state, uniquely defining a parameter interpreted as a (inverse) temperature. In the framework of an algebraic statistical mechanics description, KMS states serve as reference states for a Gel’fand-Naimark-Segal (GNS) construction. Such reference states are functionals on a fundamental, lower-level, algebra of observables. The GNSconstruction gives rise to another, higher-level algebra of observables including thermodynamic temperature as a novel property of the system. Takesaki (1970) has shown that temperature emerges as a classical observable from an underlying quantum statistical description. Temperature is then an element of an algebra M of contextual observables, where the context is introduced by the KMS state as a reference state plus the contextual topology induced by this reference state. Since mechanical descriptions are given by type I W ∗ algebras and the contextual W ∗ -algebra M is of type III,5 temperature cannot be an element of a mechanical description (Primas 1998). Hence, temperature is not reducible to statistical mechanics in any straightforward sense. Thermodynamic temperature is an example of a contextually emergent property, which is neither contained in nor predicted by the exhaustive lower-level mechanical description alone. However, given the lower-level mechanical description and an appropriate contextual topology based on the KMS state, thermodynamic quantities can be rigorously derived. The contextual topology is a contingent condition not implied by the lower-level topology as is not the concept of thermal equilibrium applicable at the lower level. This fits precisely the conceptual scheme of contextual emergence, where the emergent property is the temperature (or other thermal features) of thermodynamics.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

5.

Stability Principles for Contextual Emergence

After the detailed discussion of thermodynamic properties as exemplars for contextual emergence, it is worthwhile to step back and look at its general principles. Repeating the characterization of contextual emergence as given in section 2, the description of features of a system at a particular level of description offers necessary but not sufficient conditions to derive features at a higher level of description. In logical terms, the necessity of conditions at the lower level of description means that higher-level features imply those of the lower level of description. The converse – that lower-level features also imply the features at the higher level of description – does not hold in contextual emergence. This is the meaning of the absence of sufficient conditions at the lower level of description. Contingent contexts issue). 5 A W ∗ -algebra is a ∗ -algebra which is isomorphic to a closed algebra of observables on a Hilbert space. A ∗ C -algebra M is a W ∗ -algebra if and only if it is the dual of a Banach space M∗ , where M∗ is the predual of M (see Takesaki 2002, Chap. III.3). W ∗ -algebras can be classified by their central decompositions, i.e. by factors. A factor is of type I if it contains an atom. It is of type III if it does not contain any nonzero finite projection. It is of type II if it is atom-free and contains some nonzero finite projection. For more details see Takesaki 2002, p. 296).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Stability Conditions in Contextual Emergence

139

for the transition from the lower to the higher level of description are required in order to provide such sufficient conditions. In the example of temperature, the notion of thermal equilibrium represents such a context. Thermal equilibrium is not available at the level of description of Newtonian or statistical mechanics. Using the KMS condition and the limit N → ∞, temperature can be obtained as an emergent property at the level of a thermodynamical description. It is of paramount importance for this procedure that the KMS state satisfies stability criteria6 that are induced by the contextual condition of thermal equilibrium at the level of thermodynamics and can be implemented at the level of statistical mechanics. Since the Newtonian and statistical mechanical levels of description are necessary to derive the higher-level property of temperature, principles or laws at these levels of description cannot be violated by any higher-level description incorporating temperature. That the Newtonian and statistical mechanical levels of description alone are not sufficient is recognized by the fact that they do not give rise to an algebra of observables including temperature unless additional contingent contexts are specified. The significance of contextual emergence as opposed to reduction in this example is clear. It would be interesting to extend the general construction scheme for emergent properties to other cases. More physical examples are indicated and discussed, for example, in Primas (1998) and Batterman (2002). But the concept of stability might be useful as a key principle for the construction of a contextual topology and an associated algebra of contextual observables in examples even beyond physics. One possible, and ambitious, case refers to emergent features in the framework of contemporary neuroscience. A particularly active field of research here is concerned with the emergence of new features at the level of neuronal assemblies from lower-level features of individual neurons. Particular interest in this issue derives from the fact that cognitive capabilities are usually correlated with the activity of neuronal assemblies, but detailed neurobiological knowledge refers mainly to the properties of individual neurons. Closing the gap in our understanding of the relation between neuronal assemblies and individual neurons could contribute significantly to understanding neurobiological correlates of consciousness. As a possible framework for research in this area, the scheme of contextual emergence might be fruitfully applied as follows. Novel features at the (higher) level of neuronal assemblies would have necessary but not sufficient conditions at the (lower) level of neurons. In order to identify contexts providing such sufficient conditions, those among the many possible assembly features which are relevant or interesting as emergent features must first be identified. Assuming that stability criteria play a role analogous to physical examples, techniques of modeling assemblies in terms of generalized potentials with particular stability properties and corresponding relaxation times or escape times suggest themselves. This can be implemented easily for powerful modeling tools such as neural networks (Anderson and Rosenfeld 1989) or coupled map lattices (Kaneko and Tsuda 2000). Contextual emergence might even be a viable scheme to address relations between the 6

The notion of the stability of a system as used here covers both structural and dynamical aspects, reflected by the invariant ergodic measure of the dynamics of the system and the existence of attractors, respectively. A more detailed account, which is beyond the scope of this contribution, can be found in Atmanspacher and beim Graben (this issue).

140

Harald Atmanspacher and Robert C. Bishop

neurobiology of the brain at various levels on the one hand and cognitive or psychological features – in other words: to address the relation between material (brain) and mental (consciousness) features. In another paper (Atmanspacher and beim Graben, this issue) concrete applications in cognitive neuroscience are elaborated in detail.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

6.

Summary

The goal of reduction is to derive the description of higher-level features of a system exhaustively in terms of the description of features at the most fundamental level of physical theory, no matter how remote the higher level is from that most fundamental level. The implicit assumption in this program is that the description of all features which are not included at the fundamental level can be constructed or derived from this level without additional input. However, many physical examples pose serious difficulties for this program. For instance, temperature is a novel property emerging from a more fundamental statistical mechanical description, but it is not derivable from this description alone. The concept of contextual emergence addresses such situations properly. Contextual emergence is characterized by the fact that the lower-level description provides necessary, but not sufficient conditions for higher-level descriptions. The presence of necessary conditions indicates that the lower-level description provides a basis for higher-level descriptions, while the absence of sufficient conditions means that higher-level features are neither logical consequences of the lower-level description nor can they be rigorously derived from the lower-level description alone. Hence, the notion of reduction is inapplicable in these cases. Sufficient conditions for a rigorous derivation of higher-level features can be introduced through specifying contexts reflecting the particular kinds of contingency in a given situation. Expressing these contexts in the lower-level description induces a change of the topology of the associated state space. There is a mathematically well-defined procedure for deriving higher-level features given the lower-level description plus the contingent contextual conditions. A key ingredient of this procedure is the definition of some type of stability condition (e.g., the KMS condition) based on considerations required to establish the framework of a higher-level description (e.g., thermal equilibrium). This condition is often implemented as a reference state with respect to which an asymptotic expansion is singular in the lowerlevel state space. Regularizing the expansion provides a novel, contextual topology in which novel, emergent features can be rigorously introduced. In the thermodynamic example, this procedure is represented by the GNS-construction. We propose that contextual emergence and the associated identification of appropriate stability conditions may have applications in other domains such as biology and psychology, and, ultimately, in the relationship between the physical and the mental. Concrete ways of how this can be achieved have been worked out elsehwere (Atmanspacher and beim Graben, this issue).

Acknowledgments We appreciate the helpful suggestions of two referees, due to which an earlier version of this paper has been improved.

Stability Conditions in Contextual Emergence

141

References Anderson, J. A., and Rosenfeld, E. (1989), Neurocomputing: Foundations of Research. Cambridge: MIT Press. Atmanspacher, H., and beim Graben, P. (this issue), “Contextual Emergence of Mental States from Neurodynamics”, Chaos and Complexity Letters. Atmanspacher, H., and Kronz, F. (1999), “Relative Onticity”, in H. Atmanspacher, A. Amann, and U. M¨uller-Herold (eds.), On Quanta, Mind and Matter: Hans Primas in Context. Dordrecht: Kluwer, pp. 273-294. Atmanspacher, H., and Primas, H. (2003), “Epistemic and Ontic Quantum Realities”, in L. Castell and O. Ischebeck (eds.), Time, Quantum and Information. Berlin: Springer, pp. 301-321. Batterman, R. (2002), The Devil in the Details. Oxford: Oxford University Press. Berry, M. (1994), “Asymptotics, Singularities and the Reduction of Theories,” in D. Prawitz, B. Skyrms and D. Westerstahl (eds.), Logic, Methodology and Philosophy of Science IX: Proceedings of the Ninth International Congress of Logic, Methodology and Philosophy of Science, Uppsala 1991. Amsterdam: Elsevier, North-Holland, pp. 597-607. Bishop, R. C., and Atmanspacher, H. (2006), “Contextual Emergence in the Description of Properties”, Foundations of Physics, in press. Compagner, A. (1989), “Thermodynamics as the Continuum Limit of Statistical Mechanics,” American Journal of Physics 57(2): 106-117. Crook, S. and Gillett, C. (2001), “Why Physics Alone Cannot Define the ‘Physical’,” Canadian Journal of Philosophy 31: 333-360. Davidson, D. (1980), Essays on Actions and Events. Oxford: Oxford University Press.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Dingle, R. (1973), Asymptotic Expansions: Their Derivation and Interpretation. New York: Academic Press. Friedrichs, K. (1955), “Asymptotic Phenomena in Mathematical Physics,” Bulletin of the American Mathematical Society 61: 485-504. Gillett, C. (2002), “The Varieties of Emergence: Their Purposes, Obligations and Importance,” Grazer Philosophische Studien 65: 95-121. beim Graben, P. (2004), “Incompatible Implementations of Physical Symbol Systems”, Mind and Matter 2(2): 29-51. G¨unther, G. (1976–1980), Beitr¨age zur Grundlegung einer operationsf¨ahigem Logik, Hamburg: Meiner. Hartmann, N. (1935), Zur Grundlegung der Ontologie, Berlin: deGruyter.

142

Harald Atmanspacher and Robert C. Bishop

Kaneko, K., and Tsuda, I. (2000), Complex Systems: Chaos and Beyond. Berlin: Springer. Kim, J. (1993), Supervenience and Mind. Cambridge: Cambridge University Press. Kim, J. (1998), Mind in a Physical World: An Essay on the Mind-Body Problem and Mental Causation. Cambridge, MA: MIT Press. Kim, J. (1999), “Making Sense of Emergence,” Philosophical Studies 95: 3-36. Lind, D., and Marcus, B. (1995), Symbolic Dynamics and Coding, Cambridge: Cambridge University Press. Nagel, E. (1961), The Structure of Science. New York: Harcourt, Brace & World. Primas, H. (1990), “Mathematical and Philosophical Questions in the Theory of Open and Macroscopic Quantum Systems,” in A. I. Miller (ed.), Sixty-two Years of Uncertainty: Historical, Philosophical and Physics Inquries into the Foundation of Quantum Mechanics, New York: Plenum, pp. 233-257. Primas, H. (1998), “Emergence in Exact Natural Sciences,” Acta Polytechnica Scandinavica 91: 83-98. Putnam, H. (1987), The Many Faces of Realism, La Salle: Open Court. Quine, W. V. (1969), “Ontological relativity”, in Quine, W. V. (ed.), Ontological Relativity and Other Essays. New York: Columbia University Press, pp. 26–68. Schaffner, K. (1976), “Reductionism in Biology: Prospects and Problems,” in R. S. Cohen et al. (eds.), PSA 1974. Boston: D. Reidel Publishing Co., pp. 613-632. Scheibe, E. (1973), “The Logical Analysis of Quantum Mechanics”, Oxford: Pergamon. Sewell, G. (2002), Quantum Mechanics and Its Emergent Macrophysics. Princeton: Princeton University Press.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Takesaki, M. (1970), “Disjointness of the KMS States of Different Temperatures,” Communications in Mathematical Physics 17: 33-41. Takesaki, M. (2002), Theory of Operator Algebras I. Berlin: Springer.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 143-160

ISBN 978-1-60456-841-7 c 2009 Nova Science Publishers, Inc.

Chapter 13

C ONTEXTUAL E MERGENCE OF M ENTAL S TATES FROM N EURODYNAMICS Harald Atmanspacher1 and Peter beim Graben2 1 Institute for Frontier Areas of Psychology and Mental Health, Freiburg, Germany and Parmenides Foundation, Capoliveri, Italy 2 School of Psychology and Clinical Language Sciences, The University of Reading, United Kingdom and Institute of Physics, University of Potsdam, Germany

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Abstract The emergence of mental states from neural states by partitioning the neural phase space is analyzed in terms of symbolic dynamics. Well-defined mental states provide contexts inducing a criterion of structural stability for the neurodynamics that can be implemented by particular partitions. This leads to distinguished subshifts of finite type that are either cyclic or irreducible. Cyclic shifts correspond to asymptotically stable fixed points or limit tori whereas irreducible shifts are obtained from generating partitions of mixing hyperbolic systems. These stability criteria are applied to the discussion of neural correlates of consiousness, to the definition of macroscopic neural states, and to aspects of the symbol grounding problem. In particular, it is shown that compatible mental descriptions, topologically equivalent to the neurodynamical description, emerge if the partition of the neural phase space is generating. If this is not the case, mental descriptions are incompatible or complementary. Consequences of this result for an integration or unification of cognitive science or psychology, respectively, will be indicated.

1.

Interlevel Relations

Knowledge of well-defined relations among different levels of descriptions of physical and other systems is inevitable if one wants to understand how (elements of) different descriptions depend on each other, give rise to each other, or even imply each other. The most

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

144

Harald Atmanspacher and Peter beim Graben

ambitious program in this regard is physical reduction in the sense that higher-level descriptions of features of a system are determined by the description of features at the most fundamental level of physical theory, no matter how remote the higher level is from that most fundamental level. This program assumes that the description of all features which are not included at the fundamental level can be constructed or derived from this level without additional input. However, already physical examples pose serious difficulties for this program. It has recently been proposed that the concept of contextual emergence (Atmanspacher and Bishop, this issue; Bishop and Atmanspacher 2006) addresses such situations more properly. Contextual emergence is characterized by the fact that lower-level descriptions provide necessary, but not sufficient conditions for higher-level descriptions. (Note that such a relation between descriptive levels does not necessarily entail the same relation between ontological levels.) The presence of necessary conditions indicates that lower-level descriptions provide a basis for higher-level descriptions, while the absence of sufficient conditions means that higher-level features are neither logical consequences of lower-level descriptions nor can they be rigorously derived from them alone. Hence, a full-blown reductive program is inapplicable in these cases. Sufficient conditions for a rigorous derivation of higher-level features can be introduced through specifying contexts reflecting the particular kinds of contingency in a given situation. A key ingredient of this procedure is the definition of some type of stability condition (e.g., the KMS condition, due to Kubo, Martin, and Schwinger) based on considerations required to establish the framework of a higher-level description (e.g., thermal equilibrium). This condition is often implemented as a reference state with respect to which an asymptotic expansion is singular in the lower-level state space. Its regularization defines a novel, contextual topology in which novel, emergent features can be rigorously introduced. There is, thus, a mathematically well-defined procedure for deriving higher-level features given the lower level description plus the contingent contextual conditions. Contextual emergence and the associated identification of appropriate stability conditions may have applications in other domains such as biology and psychology, and, ultimately, in the relationship between the physical and the mental. In this contribution we will address a situation which is particularly difficult because it exceeds the domain of material systems: relations between brain and consciousness. We will discuss the contextual emergence of mental states and related features (psychology, cognitive science) from brain states and related features (neuroscience). Using Harnad’s (1990) terms, this refers to the question of how mental symbols and cognitive computation can be grounded in neurodynamics. More specifically, mental representations will be considered as novel features at the (higher) level of cognition, which have necessary but not sufficient conditions at the (lower) level of neuronal assemblies. In order to identify contexts providing such sufficient conditions, those among the many possible cognitive features that might be relevant or interesting as emergent features must first be identified. Assuming that stability criteria play a role analogous to physical examples, techniques of modeling assemblies in terms of generalized potentials with particular stability properties and corresponding relaxation times or escape times suggest themselves. This can be implemented for powerful modeling tools such as neural networks (Anderson and Rosenfeld 1989) or coupled map lattices (Kaneko and Tsuda 2000).

Contextual Emergence of Mental States from Neurodynamics

145

Other interlevel relations in addition to contextual emergence are strong reduction, radical emergence, and supervenience (cf. Atmanspacher and Bishop, this issue). While we do not think that strong reduction or radical emergence provide clarifying insight for the relation between brain and consciousness, some comments about supervenience are appropriate here. The notion of supervenience characterizes situations in which lower-level descriptions contain sufficient but not necessary conditions for higher-level descriptions. This scenario has been employed for brain-consciousness relations in the sense that a conscious state with a particular phenomenal content can be multiply realized at the neural level (Kim 1992, 1993). For instance, Chalmers (2000) defines neural correlates of consciousness (NCCs) as neural systems that are correlated with conscious mental states and are minimally sufficient for the occurence of those states. In this definition, the notion of sufficiency rather than necessity takes into account that different neural states can be correlated with the same conscious state (multiple realization). Our notion of contextual emergence addresses the different question of how neural states are related to conscious states in each individual neural realization. Contextual emergence does not address the distinction between many-to-one and one-to-one relations but tries to elucidate principles which allow us to understand the relationship between mental and neural states itself, even in individual instantiations, in a more profound manner. In this way, supervenience and contextual emergence complement rather than contradict each other. Applying both concepts together may, thus, improve our insight into the nature of mind-brain relations.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

2.

Structural Stability in Symbolic Dynamics

The issue of stability plays a prominent role in statistical mechanics. Haag et al. (1974) have shown that Gibbs’ thermal equilibrium states are uniquely characterized by three stability conditions upon state functionals: (i) stationarity, i.e., expectation values of observables do not change in time; (ii) structural stability, i.e., stability of the dynamics against perturbations; and (iii) “asymptotic abelianness”, i.e., temporally distant observables become eventually compatible (see Bratteli and Robinson 1997). From these presuppositions, Haag et al. (1974) derived the KMS condition for thermal equilibrium states. In the following we will establish related stability criteria for symbolic dynamics. Consider a classical time-discrete, invertible dynamical system (X, Φ) given by a compact Hausdorff space as its phase space X and a map Φ : X → X. The flow of the system is generated by the time iterates Φt , t ∈ Z, i.e., t 7→ Φt is a one-parameter group for the dynamics. Then, the function space of complex-valued continuous functions over X, A = C(X), yields a C∗ -algebra of classical observables for that dynamical system. The states of such a C∗ -dynamical system are linear, positive, normalized functionals ρ : A → C. For classical dynamical systems they correspond to probability measures µρ R over the phase space X, such that ρ(f ) = X f (x) dµρ (x) for f ∈ C(X). While pure states can be identified with single points in phase space x ∈ X, non-pure states are statistical states given by measures µρ that are not concentrated on a single point. In its simplest sense, the stability of a dynamical system refers to the stability of a point x∗ ∈ X under the flow Φt : x∗ = Φ(x∗ ), i.e., x∗ is a fixed-point attractor. Limit cycles or higher-order tori as attractors can be related to fixed points by the technique of

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

146

Harald Atmanspacher and Peter beim Graben

Poincar´e sections. In general, attractors are invariant sets A ⊂ X, such that Φ(A) ⊆ A and Φ−1 (A) ⊆ A. This invariance property of A extends to probability measures µ according to µ(Φ−1 (A)) = µ(A) which are called stationary or invariant measures. Likewise, a statistical state ρµ over the algebra of continuous functions assigned to the measure µ has the invariance property. The invariance of thermal equilibrium states is the first postulate by Haag et al. (1974). Structural stability refers to perturbations in the function space of the flow map Φ. The system (X, Φ) is called structurally stable if there is a neighborhood N of Φ such that all Ψ ∈ N are topologically equivalent with Φ.1 As Haag et al. (1974) pointed out, the concept of structural stability is closely related to that of ergodicity. An invariant probability measure µ is said to be ergodic under the flow Φ if an invariant set A, has either measure zero or one: µ(A) ∈ {0, 1}. If µ is non-ergodic, there is an invariant set A with 0 < µ(A) < 1 corresponding to an accidental degeneracy. Such degeneracies are not stable under small perturbations. Hence, non-ergodic systems are in general not structurally stable (Haag et al. 1974). Thermal equilibrium states are given by invariant, ergodic measures over X. Beyond fixed points and limit tori, more complicated attractors are mixing in addition. Mixing refers to the loss of temporal correlations among the observables of a dynamical system. Formally, t→∞ a measure µ is called mixing if |µ(A ∩ Φ−t (B)) − µ(A)µ(B)| −→ 0 for all measurable sets A, B (Luzzatto 2006). This property can be rephrased by the correlation of observables f, g ∈ A at time t: Ct (f, g) = |ρµ (f · g ◦ Φt ) − ρµ (f ) · ρµ (g)| where ρµ is the statistical t→∞ state assigned to the measure µ. If Ct (χA , χB ) −→ 0 for characteristic functions χA , χB of the sets A, B ⊂ X, µ is mixing (Luzzatto 2006). Interestingly, Haag et al. (1974) derived this loss of correlations from a more fundamental, purely algebraic stability property called “asymptotic abelianness” (Bratteli and Robinson 1997). The mixing property of a state ρ follows from the asymptotic abelianness of the algebra under the assumption that ρ is relatively pure, i.e., ρ cannot be decomposed into a convex sum of invariant states (ρ might be decomposable into non-invariant states, however). Relatively pure states have sharp expectation values and correspond, therefore, to thermodynamic macrostates (Shalizi and Moore preprint). Stationarity (invariance), structural stability (ergodicity) and asymptotic abelianness (mixing) are important for the investigation of nonlinear dynamical systems. Many rigorous results are known for hyperbolic systems where either the whole phase space possesses a hyperbolic structure (Anosov diffeomorphisms) or there is a hyperbolic attractor. Anosov diffeomorphisms are known to be structurally stable (Robinson 1999), and systems with hyperbolic attractors have invariant, ergodic and mixing probability measures due to a theorem by Sinai, Ruelle and Bowen (Ruelle 1968, 1989). For non-hyperbolic systems, much less is known (cf. Viana et al. 2003). Now let us introduce the notion of epistemic observables. For this purpose, consider a piecewise constant function f over the phase space X. Such a function is generally not overall continuous and does therefore not belong to the C∗ -algebra A = C(X) of observables. Instead, it belongs to the larger W∗ -algebra2 of µ ˆ-essentially bounded epistemic 1

Two maps Φ, Ψ are called topologically equivalent, or conjugated, if there is a homeomorphism h such that h ◦ Φ = Ψ ◦ h. 2 The relationship between C∗ - and W∗ -algebras can be illustrated in the following way. Regarding a C∗ -

Contextual Emergence of Mental States from Neurodynamics

147

observables L∞ (X, µ ˆ) that are contextually defined by a reference probability measure µ ˆ on the phase space X used for a Gel’fand-Naimark-Segal (GNS) construction (Primas 1998, Atmanspacher and Bishop, this issue). Two states x, y ∈ X are called epistemically equivalent with respect to f if f (x) = f (y) (beim Graben and Atmanspacher 2006). Epistemically equivalent states are not distinguishable by means of the observable f . The classes of epistemically equivalent states partition the phase space X into disjoint sets Ai . S A finite partition of X, P = {Ai |i ≤ I}, Ai ∩ Aj = ∅ (i 6= j), i Ai = X, also called a coarse-graining, yields a symbolic dynamics (Lind and Marcus 1995) of the system (X, Φ) in the following way: Taking the finite index set of the partition as an alphabet A of cardinality I, one assigns to each initial condition x0 ∈ X a bi-infinite sequence s = . . . ai−1 ai0 .ai1 ai2 . . . of symbols aik ∈ A according to the rule x0 7→ s, if Φt (x0 ) ∈ Ait , t ∈ Z (the dot indicates the origin of the time scale). This mapping s = π(x0 ) is continuous in the topology of the space of sequences Σ = AZ . Accordingly, the first iterate x1 = Φ(x0 ) of x0 is mapped onto the sequence s′ = . . . ai−1 ai0 ai1 .ai2 . . .. Therefore, the sequence s′ is obtained by shifting all symbols of s one place to the left. A symbolic dynamical system is given by (Σ, σ) where σ(s) = s′ is the left-shift. Since the dynamics on Σ is trivially represented by the shift σ, all important information is now encoded in the structure of the symbolic sequences s. Therefore, symbolic dynamics deals with syntax and pattern analysis (Lind and Marcus 1995, Keller and Wittfeld 2004, Steuer et al. 2004, Steuer et al. 2001). The systems (X, Φ) and (Σ, σ) are related to each other by π◦Φ = σ◦π,

(1)

which can be represented diagrammatically as: x Φ π ?

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

s

- Φ(x)

π ?

σ σ(s)

where π : X → Σ acts as an intertwiner. If π is continuous and invertible and its inverse π −1 is also continuous, the maps Φ and σ are topologically equivalent. In this case the partition P is called generating. For generating partitions, the correspondence between the phase space and the symbolic representation is one-to-one: each point in phase space is uniquely represented by a bi-infinite symbolic sequence and vice versa. Additionally, all topological information is preserved. Generating partitions are generally hard to find. However, it is known that hyperbolic systems possess generating partitions for which the resulting symbolic dynamics algebra A as a complex vector space one can construct the dual A∗ of linear functionals containing the states over A. This is again a vector space that becomes a Hilbert space in the GNS construction and has a dual A∗∗ . The original C∗ -algebra A can be canonically embedded in A∗∗ by a(ρ) = ρ(a) where ρ ∈ A∗ , the right-hand side a ∈ A, and the left-hand side a ∈ A∗∗ . Hence, A∗∗ inherits the properties of A (including the C∗ -property). The fact that it has a Hilbert space as its predual turns it into a W∗ -algebra. The bidual A∗∗ is generally much larger than A and contains the epistemic observables.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

148

Harald Atmanspacher and Peter beim Graben

is a Markov chain (Sinai 1968a,b, Bowen 1970). The partitions that achieve this are so-called Markov partitions. The symbolic dynamics obtained from a Markov partition is a shift of finite type. This can be seen by defining an I × I stochastic matrix Pij = µ(Φ−1 (Ai ) ∩ Aj )/µ(Ai ) (Froyland 2001), where µ is a probability measure. The associated transition matrix Tij = sgn (Pij ) provides a subset ΣT ⊂ Σ of admissible sequences. The sequence s = . . . ai−1 ai0 .ai1 ai2 . . . belongs to ΣT if Taik aik+1 = 1 (i.e., the transition from aik to aik+1 is allowed). The left-shift restricted to ΣT yields then a subshift of finite type (ΣT , σ|ΣT ). Assume that a coarse-grained description of a dynamical system (X, Φ) is such a shift of finite type (ΣT , σ|ΣT ). Then we can distinguish two important cases. In the first case, either the matrix T itself or some power T l (l > 1) of T is diagonal. If T is diagonal, the cells Ai of the partition P are invariant sets under the flow Φ. That is, the partition is coarse enough to capture the asymptotically stable fixed points and limit tori together with their basins of attraction of a multistable dynamical system. Such systems are structurally stable unless they give rise to bifurcations. If the l-th power of T is diagonal, the admissible sequences of the symbolic dynamics are periodic and T is called cyclic. This means that the boundaries of the partition are transversally intersected by a limit torus, which is asymptotically stable as well. The space of symbolic sequences ΣT for these systems can be equipped with invariant, ergodic measures by taking Dirac measures for the periodic sequences. The second important case refers to an irreducible transition matrix T , i.e. there is a number l such that T l is positive. Then the corresponding shift of finite type (ΣT , σ|ΣT ) is an ergodic and mixing Markov chain where the eigenvector p∗ to eigenvalue one of the stochastic matrix P corresponds to a unique invariant, ergodic measure that is mixing in addition to the first case above (Ruelle 1968, 1989). A well-elaborated theory relates these measures to KMS states in algebraic quantum statistics (Olesen and Petersen 1978, Bratteli and Robinson 1997, Pinzari et al. 2000, Exel 2004), at least for structurally stable hyperbolic systems. Such systems have Markov partitions enabling the construction of thermal equilibrium KMS states (under certain conditions) which are also structurally stable (Robinson 1999). Furthermore, Markov partitions are generating and, thereby, admit a symbolic dynamics that is topologically equivalent to the underlying phase space dynamics. To conclude, subshifts of finite type (ΣT , σ|ΣT ), characterized by an I × I transition matrix T , are structurally stable if T is either cyclic (i.e. there is an l ≥ 1 such that T l is diagonal) or if T is irreducible. In both cases, the existence of invariant and ergodic measures ensure stability conditions as required for the contextual emergence of epistemic observables and associated states in a partitioned phase space.

3.

Contextual Emergence of Mental States from Neural States

Let us now consider a neurodynamical system N = (X, Φ) with phase space X described by neural observables fi : X → R (e.g. spike rates or action potentials or somato-dendritic membrane potentials of neurons) such that x ∈ X is a point or, likewise, an activation vector of a neural population given by the values (fi (x))i≤n ∈ Rn for n degrees of freedom. In the following subsections we address three different ways of introducing epistemic observ-

Contextual Emergence of Mental States from Neurodynamics

149

ables on such a phase space. The structural stability of their associated symbolic dynamics, which is of key significance for contextual emergence, will be emphasized in particular.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

3.1.

Neural Correlates of Consciousness

There is a great variety of conscious mental states forming a mental state space Y . Mental states range from coarsest-grained (“just being conscious”) to finer-grained states such as wakefulness versus sleep, dreaming, hypnosis, attentiveness, etc.3 Even more refined are states of consciousness associated with specific phenomenal content (Chalmers 2000). It is generally assumed that some neural system N with phase space X is correlated with particular mental states C ∈ Y . They can be related to epistemic observables p : X → {0, 1}, where p(x) = 1 if the activation vector x is actually correlated with the mental state C. A phenomenal family P = {C1 , . . . CI } is a Boolean classification of pairwise disjoint states that cover the whole mental state space Y (Chalmers 2000). In other words, P provides a partition of the mental state space Y into I states Ci . The whole mental state space can then be represented by a system of such partitions of different coarse grainings: At the lowest level there is a binary partition defining mental states of “being conscious” and “not being conscious”. At subsequent levels, there are more refined partitions defining, for instance, states of “wakefulness”, “sleep”, and altered states (e.g. hypnosis), again covering the entire mental state space Y . According to Chalmers (2000), a neural correlate of consciousness (NCC) can be characterized by a minimal sufficient neural subsystem N that is correlated with a conscious state C ∈ Y . This characterization refers to the interlevel relation of supervenience. The sufficiency of N means that the activity of N implies being in conscious state C. From the point of view of this contribution, however, it is also appropriate to look for necessary conditions for a neural subsystem N whose activation is correlated with the conscious state C in the sense of contextual emergence. Being in a conscious state C implies then the activity of N , so that this activity is a necessary condition for C. Suppose that N is an NCC for a conscious state Ci ∈ P with multiple realizations by different activation patterns of N . Then different neural states, x, y ∈ X, are sufficient for the conscious state Ci . Since pi (x) = pi (y), x and y are epistemically indistinguishable from one another and, hence, epistemically equivalent with respect to the observable pi corresponding to the mental state Ci ∈ P. In this sense, the partition P of the mental state space Y induces a partition Q = {A1 , . . . AI } of the neural state space X into classes of epistemically equivalent neural states. Labeling the cells Ai of Q by symbols ai of a finite alphabet A, we obtain a symbolic representation of the mental states, emerging from the neural state space by the mapping π : P → A, π(Ci ) = ai . The dynamics of states ai in A is a discrete sequence of symbols as a function of time, establishing a symbolic dynamics. If the transitions between states of consciousness can be described by an I × I transition matrix T , the mental symbolic dynamics is of finite type. A coarse-grained partition of X implies neighborhood relations between states in Y 3

A recent empirically based study concerning the relation between neural and mental state space representations for wakefulness versus sleep and other, subtler examples (selective attention, intrinsic perceptual selection) is due to Fell (2004). For alternative state space approaches see Wackermann (1999) and Hobson et al. (2000), and the following subsection.

150

Harald Atmanspacher and Peter beim Graben

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

that are different from those in the underlying neural phase space X; in this sense it implies a change of topology. Also, the algebra of mental observables differs from that of neural observables. Obviously, these two differences depend essentially on the choice of the contextual partition of Y , based on the choice of a phenomenal family, inducing the partition of X. We will now show that a particular concept of stability is crucial for a proper choice of such a partition and, thus, crucial for a properly conceived relation between X and Y . The crucial demand for contextual emergence is that the equivalence classes of neural states in X and, hence, the mental states in Y be structurally stable (in the sense of Sec. 2) under the dynamics in X. Consider, e.g., the partition of Y into the mental states “wakefulness” and “sleep” leading to two disjoint sets in X. Given an appropriate discretization of time, the transition matrix T is cyclic with T 2 = E (E denoting the 2 × 2 unit matrix). That is, the coarse-grained description provides a limit torus. By contrast, a sufficiently finegrained partition of Y into mental states of different phenomenal content would have to be described by a high-dimensional irreducible transition matrix T since any such state should be connected to any other state by a symbolic trajectory of sufficient length. In this case the resulting symbolic dynamics is an ergodic, mixing Markov chain with a distinguished KMS equilibrium state (Pinzari et al. 2000, Exel 2004). These stationary and structurally stable symbolic dynamical systems have strikingly different consequences (beim Graben and Atmanspacher 2006). While fixed points and limit tori do not allow for generating partitions (beim Graben 2004), aperiodic Markov chains can be obtained from Markov partitions which are generating. Generating partitions admit a continuous approximation of individual points in the neural phase space X by symbolic sequences in A with arbitrary precision. Hence, the neural description in X and the coarse-grained, mental description in Y are topologically equivalent. This shows that the generating property of a partition is an important constraint for a viable symbolic description of a system. Although this is a clear-cut criterion, generating partitions are notoriously difficult to find in practice, and they are explicitly known for only a few examples. Nevertheless, they are viable candidates for the implementation of a stability criterion appropriate for the contextual emergence of mental states. A related stability constraint has been proposed recently (Werning and Maye 2004, this issue). An alternative approach, focusing on information constraints rather than stability, is due to Shalizi and Moore (preprint).

3.2.

Macroscopic Neural States

Another approach, leading to coarse-grained neural states without involving mental states is based on mass potentials such as local field potentials (LFP) at the mesoscopic and the electroencephalogram (EEG) at the macroscopic level of brain organization. Let F : X → R be such an observable given by a mean field X F (x) = fi (x) , (2) i

where the sum extends over a population of n neurons and fi denotes a projector of X onto the i-th coordinate axis measuring the microscopic activation of the i-th neuron. Similar to the previous subsection, the outcomes of F have multiple realizations since the terms in the sum in Eq. (2) can be arranged arbitrarily. Therefore, two neural activation

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Contextual Emergence of Mental States from Neurodynamics

151

vectors x, y can lead to the same value F (x) = F (y) (e.g., when fi (x) − ǫ = fj (x) + ǫ, i 6= j), so that they are indistinguishable by means of F and, therefore, epistemically equivalent. If the equivalence classes of F in X form a finite partition Q = {A1 , . . . AI } of X, we can again assign symbols ai from an alphabet A to the cells Ai and obtain a symbolic dynamics. In this way, experimentally well-defined meso- and macroscopic brain observables, LFP and EEG, form a coarse-grained description of the underlying microscopic neurodynamics. It should be emphasized that related approaches do not involve any reference to concrete mental or conscious states. Whether or not one wants to relate corresponding coarse-grained neural states to mental states is left open (for attempts in this direction, see Fell (2004), Wackermann (1999) and Hobson et al. (2000)). Coarse-grainings based on the symbolic encoding of EEG time series became increasingly popular in recent years (Keller and Wittfeld 2004, beim Graben et al. 2000, Frisch et al. 2004, Frisch and beim Graben 2005, Drenhaus et al. 2006, Schack 2004, Steuer et al. 2004). Since such partitions are not induced by well-defined mental observables, it is unclear whether the stability conditions required for contextual emergence are satisfied. It is, thus, particularly important to check this carefully. One option to do this is to look for Markov partitions of the phase space which minimize correlations between their cells, thus creating a Markov process for the symbolic dynamics of the meso- or macro- observables if the dynamics in X is chaotic.4 Since Markov partitions are generating, they can be operationally identified by the fact that the dynamical entropy for a generating partition is the supremum over all possible partitions, the so-called Kolmogorov-Sinai entropy (see Atmanspacher (1997) for an annotated introduction). Iterative partitioning algorithms in this and similar contexts have been discussed by Froyland (2001): Starting with an initial partition, those sets which contribute to the greatest mass of the assumed invariant ergodic measure are refined iteratively. Optimal partitions are thereby generated by a dynamics in “partition space”. Alternatively, the measured meso- or macroscopic observables can be analyzed by segmentation techniques (Lehmann et al. 1987, Wackermann et al. 1993, Hobson et al. 2000, Hutt 2004, Schack 2004).5 A recent proposal to implement this is due to Froyland (2005): One tries to partition the space X into almost invariant sets such that trajectories spend most of the time within individual cells of the partition, and transitions between cells are likely at larger time scales. In this way, the dynamics on short time scales is described by cyclic transition matrices, whereas large time scales yield descriptions by Markov processes with irreducible transition matrices. The separation of time scales provides, then, a contextual criterion for properly defined macroscopic brain states. For a related approach see Gaveau and Schulman (2005).

3.3.

Remarks on Symbol Grounding

The symbol grounding problem posed by Harnad (1990) refers to the problem of assigning meaning to symbols on purely syntactic grounds, as proposed by cognitivists such as Fodor 4

Evidence for chaotic brain processes has often been reported (cf. Kaneko and Tsuda 2000, and references therein). 5 Lehmann et al. (1987) called the corresponding states “brain microstates” or “atoms of thought”, expressing the suggestion that they correspond to elementary “chunks” of consciousness.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

152

Harald Atmanspacher and Peter beim Graben

and Pylyshin (1988). This entails the question of how conscious mental states with phenomenal content can be characterized by their NCC. Chalmers (2000) defined an NCC for phenomenal content as a neural system N with “systematicity in the correlation”, meaning that the representation of a content in N is correlated with a representation of that content in consciousness. In other words, there should be a mapping from the neural state space X onto the space of conscious states Y such that regions in X are related to phenomenal contents in Y . This mapping differs from the mapping required for contextual emergence as discussed in Sec. 3.1. For a neural representation of content further constraints are crucial. First, all states in B ⊂ X representing the same content C should be similar in some respect: there should be a mapping g : X → X, let us call it a gauge transformation, such that B is invariant under g, g(B) ⊂ B. In this sense, g is a similarity transformation. On the other hand, graded differences in phenomenal similarity should be reflected by topologically neighboring regions in phase space. One would, therefore, require the mapping g to be a homeomorphism, leading to topographic mappings of contents (Chalmers 2000). A second requirement is the compositionality of representations (Fodor and Pylyshin 1988, Werning and Maye 2004). Compositionality refers to the relation between syntax and semantics insofar as the meaning of a composed (or “complex”) symbol is a function of the meanings of its constituent symbols and the way they are put together. A prerequisite for compositionality is the existence of syntactic rules determining which composites are constituents of a language and which are not. (In our approach, constituents are admissible (sub-)sequences in the corresponding symbolic dynamics (beim Graben 2004).) According to Harnad, these constraints need to be combined with his proposal that symbols must be grounded in embodied cognition. They represent objects or facts from the environments of physically embodied agents that collect information by their sensory apparatuses and act by their motor effectors. While Harnad (1990) suggests a hybrid architecture consisting of a neural network as an invariance detector and a classical symbol processor to meet the compositionality constraint, we shall discuss the alternative of a unified neurodynamical system. This can be achieved using the notion of conceptual spaces as discussed by G¨ardenfors (2004). A conceptual space is a vector space spanned by quantitative observables. The conceptual space for color, e.g., can be constructed as the three-dimensional RGB coordinate system or an equivalent representation supplied by the cones in the retina (Steels and Belpaeme 2005). According to G¨ardenfors (2004) a natural concept is then a convex region in a conceptual space such that all elements in that region are similar in a particular context. Implementing conceptual spaces by neural systems, we arrive again at partitions of neurodynamical phase spaces. The idea of gauge invariance yields partitions of finest grain, corresponding to “natural kinds” (Carnap 1928/2003, Quine 1969), that might be too refined for other contexts. Such contexts can be supplied by pragmatic accounts. Suppose a toy-world in which only orange objects are eatable, and all other objects are not (Steels and Belpaeme 2005). Then, a binary partition of color space into “orange” and “non-orange” will be sufficient for an agent to survive. Thus, survival (or successful communication) serve as contextual constraints for the emergence of cognitive symbols. Symbol grounding corresponds then to categorization of conceptual spaces driven by pragmatic goals. The contextual emergence of symbols in partitioned conceptual spaces raises the ques-

Contextual Emergence of Mental States from Neurodynamics

153

tion of the stability of the symbols. The dynamics that has to be taken into account now is, however, not neurodynamics but rather sociodynamics: the evolution of populations of cognitive agents. (Neurodynamically, concepts are static objects given by the cells of a partition.) An interesting approach in this sense has been developed within the framework of evolutionary game theory (Steels and Belpaeme 2005, J¨ager 2004, van Rooy 2004). In these models the phase space is spanned by the population numbers of agents with competing strategies. The outcome of the games is assessed by a utility function which in turn determines the number of offspring of the players. In cognitive applications of evolutionary game theory, offspring means adoption of the winning strategy by other players. If categories or concepts are given by partitions of conceptual spaces, competing strategies are different partitions of the same local conceptual spaces shared by different agents. Evolutionary game theory then describes a dynamics in partition space similar to the search for optimal partitions by iterative algorithms (Froyland 2001). Evolutionary stable strategies are asymptotically stable fixed points in evolutionary game theory (J¨ager 2004). This stability criterion means that cultural evolution grounds symbols in shared partitions of local conceptual spaces of cognitive agents. The structural stability of dynamically evolving partitions can be illustrated by the “naming game” (van Rooy 2004). When a categorization in conceptual space is fixed the cells are labeled by symbols of an alphabet A. This can be done arbitrarily by convention, or it can be achieved by another pragmatic game that optimizes the utility reward. For instance, assume that two meanings m1 , m2 assign two symbolic forms f1 , f2 , that m1 is less complex than m2 , and the same for the forms f1 , f2 . For such a scenario, van Rooy (2004) found only two evolutionarily stable strategies: the Horn strategy which assigns more complex forms to more complex meanings, and the anti-Horn strategy performing otherwise. Since the basin of attraction of the Horn strategy is larger than that of the anti-Horn strategy (J¨ager 2004), the Horn strategy provides a higher degree of structural stability.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

4.

Compatibility of Psychological Descriptions

It is an old and much discussed question whether and, if yes, how psychology could become a unified science, integrating the many approaches and models that constitute its contemporary situation. It is often argued that the largely fragmented appearance of psychology (and cognitive science as well) is due to the fact that psychology is still in a preparadigmatic, “immature” state. Some have even argued that this situation is unavoidable (e.g., Koch 1983, Gardner 1992) and should be considered as the strength of psychology (e.g., Viney 1989, McNally 1992) rather than an undesirable affair. From the perspective of the philosophy of mind, arguments against the possibility of a unified science of psychology have been presented as well. Most prominent are the accounts of Kim (1992, 1993) and Fodor (1997), both using the scheme of multiple realization in the framework of supervenience to reject unification. Shapiro (in press) has recently pointed out particular weak points in their arguments. On the other hand, there is a growing interest in articulating visions for a unified science of psychology, cognition, or consciousness (see, e.g., Newell 1990, Anderson 1996). Recently, various approaches have been proposed to reach a degree of coherence comparable to established sciences as, e.g., physics with well-defined relations between its different

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

154

Harald Atmanspacher and Peter beim Graben

disciplines. Examples are approaches such as “psychological behaviorism” (Staats 1996, 1999), “unified psychology” (Sternberg and Grigorenko 2001, Sternberg et al. 2001), and the “tree of knowledge system” (Henriques 2003). A key feature in the latter program is the commensurability of competing approaches in psychology, explicated by Yanchar and Slife (1997) and Slife (2000). This section presents a way in which the notion of commensurable models can be implemented in a formally rigorous fashion. A suitable way to formulate commensurability in technical terms is related to the concept of compatibility. Briefly speaking, two models are considered as commensurable if they are compatible in the sense that there exist welldefined mappings between them. If this is not the case, they are incompatible. It turns out that the scheme of contextual emergence provides some detailed and clarifying insights how to proceed in this regard. The two levels of description whose interlevel relations are significant for this purpose are those of neurobiology and psychology or cognitive science, respectively. Compatible and incompatible implementations of cognitive symbol systems have recently been discussed by beim Graben (2004). A key result of the work by beim Graben and Atmanspacher (2006) is that a nongenerating partition is incompatible with any other partition (even if this is generating) in the sense that there is no well-defined mapping between the partitions.6 As a consequence, models based on such partitions are incompatible as well. Since any ad hoc chosen partition is quite unlikely to be generating, it may be suspected that the resulting incompatibility of models based on such partitions is the rule rather than the exception. While incompatibility may admit the possibility of “partially coherent” models, the case of maximal incompatibility, also called complementarity, excludes any coherence between different models completely. At this point it should be clear that our notion of incompatibility is more subtle than a “logical incompatibility” (Slife 2000) in the sense that two models are simply negations of each other. Also, it would be interesting to compare Slife’s (2000) complementary models with our formal approach in terms of maximally incompatible models, which are basically incoherent only in a Boolean framework. From a perspective admitting non-Boolean descriptions, the notion of coherence acquires a more comprehensive meaning, including complementary descriptions as representations of an underlying, more general description (see Primas 1977). With these remarks in mind, incompatible models due to non-generating partitions represent a significant limit to the vision of a unified or integrative science of psychology. Or, turned positively, such a unification will be strongly facilitated if the approaches to be unified are based on generating, hence compatible, partitions that are structurally stable and induced by well-defined mental or cognitive states. As mentioned, it is a tedious task to identify such generating partitions. Nevertheless, the necessary formal and numerical tools are available today and can be implemented by the symbolic description using shifts of finite type and transition matrices. All one has to do is find transitions between mental states that are irreducible, yielding a stationary, ergodic, and mixing Markov chain with distinguished KMS states. 6

Two partitions P1 and P2 are (in)compatible if their σ-algebras are (not) identical up to µ-measure zero. Two partitions are maximally incompatible, or complementary, if their σ-algebras are disjoint up to the entire phase space X (cf. beim Graben and Atmanspacher 2006).

Contextual Emergence of Mental States from Neurodynamics

155

If there is a good deal of empirical plausibility for a particular partition, one might hope that this implies that such a partition is generating (at least in an approximate sense) and, thus, that the corresponding mental or cognitive states are stable (in the sense of the KMS condition). However, there may be cases of conflict between the empirical and the theoretical constraint on a proper partition. In such cases, one has to face the possibility that the “empirical plausibility” of cognitive states may be unjustified, e.g., based on questionable prejudices. If cognitive states turn out to be dynamically unstable, this theoretical argument against their adequacy is very strong indeed. Compatible partitions and, consequently, compatible psychological models show another important feature that is occasionally addressed in current literature: the topological equivalence of representations in neurodynamic and mental state spaces (cf. Metzinger 2003, p. 619, and Fell (2004) for empirically based examples). Topological equivalence ensures that the mapping between X and Y is faithful in the sense that the two state space representations yield equivalent information about the system (see Sec. 2). Non-generating, incompatible partitions do not provide representations in Y that are topologically equivalent with the underlying representation in X.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

5.

Summary

The relation between mental states and neural states is discussed in the framework of a recently proposed scheme of interlevel relations called contextual emergence. According to this proposal, knowledge of the neural description provides necessary but not sufficient conditions for a proper psychological description. Sufficient conditions can be defined by contingent contexts at the cognitive (phenomenal) level and implemented as stability criteria at the underlying neural level. This procedure has been demonstrated using the terminology of symbolic dynamics at the cognitive level. Equivalence classes of neural states are defined as neural correlates of mental states represented symbolically. Mental states are well-defined if criteria of temporal and structural stability are satisfied for their neural correlates. These criteria can be implemented either by generating or, more specifically, Markov partitions; or by partitions of systems with asymptotically stable fixed points or limit tori. This implies that proper mental or cognitive states must satisfy appropriate stability conditions. If this is not explicitly taken care of for chaotic systems admitting generating partitions, one has to expect that ad hoc selected partitions are not generating. As a consequence, models based on such partitions are incompatible. This may be a possible source of the long-standing problem of how to develop a unified science of psychology. Only for carefully chosen generating partitions it can be guaranteed that different cognitive models are compatible and, hence, can have transparent relations with respect to each other. Moreover, psychological (or cognitive) models are topologically equivalent with their neurobiological basis only if they are constructed from generating partitions. Without cognitive contexts serving as sufficient conditions for compatibility and topological equivalence, the neurobiological level of description provides only necessary conditions for psychological descriptions.

156

Harald Atmanspacher and Peter beim Graben

Acknowledgments We are grateful to Jiˇri Wackermann and three anonymous referees for their helpful suggestions how to improve an earlier version of this paper.

References Anderson, J. A., and Rosenfeld, E. (1989), Neurocomputing: Foundations of Research. Cambridge: MIT Press. Anderson, N. (1996), A Functional Theory of Cognition, Mahwah: Erlbaum. Atmanspacher, H. (1997), “Dynamical Entropy in Dynamical Systems”, in H. Atmanspacher and E. Ruhnau (eds.), Time, Temporality, Now. Berlin: Springer, pp. 327–346. Atmanspacher, H., and Bishop, R. (this issue), “Stability Conditions in Contextual Emergence,” Chaos and Complexity Letters. Bishop, R. C., and Atmanspacher, H. (2006), “Contextual Emergence in the Description of Properties”, Found. Phys., in press. Bowen, R. (1970), “Markov Partitions for Axiom A Diffeomorphisms,” Am. J. Math. 92: 725–747. Bratteli, O. and Robinson, D.L. (1997), Operator Algebras and Quantum Statistical Mechanics 2, Berlin: Springer. Carnap, R. (1928/2003), The Logical Structure of the World and Pseudoproblems in Philosophy, Peru (IL): Open Court.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Chalmers, D. (2000), “What Is a Neural Correlate of Consciousness?”, in T. Metzinger (ed.), Neural Correlates of Consciousness, Cambridge: MIT Press, pp. 17–39. Drenhaus, H., beim Graben, P., Saddy, D., and Frisch, S. (2006), “Diagnosis and Repair of Negative Polarity Constructions in the Light of Symbolic Resonance Analysis”, Brain and Language 96(3): 255–268. Exel, R. (2004), “KMS states for generalized gauge actions on Cuntz-Krieger algebras (An application of the Ruelle-Perron-Frobenius theorem)”, Bull. Brazil. Math. Soc. 35(1): 1 – 12. Fell, J. (2004), “Identifying Neural Correlates of Consciousness: The State Space Approach”, Consciousness and Cognition 13: 709–729. Fodor, J. (1997), “Special Sciences: Still Autonomous After All These Years”, Philosophical Perspectives 11: 149–163. Fodor, J. and Pylyshyn, Z. W. (1988) “Connectionism and Cognitive Architecture: A Critical Analysis”, Cognition 28: 3 – 71.

Contextual Emergence of Mental States from Neurodynamics

157

Frisch, S. and beim Graben, P. (2005), “Finding Needles in Haystacks: Symbolic Resonance Analysis of Event-Related Potentials Unveils Different Processing Demands”, Cogn. Brain Res. 24(3): 476 – 491. Frisch, S., beim Graben, P., and Schlesewsky, M. (2004), “Parallelizing Grammatical Functions: P600 and P345 Reflect Different Cost of Reanalysis” Int. J. Bifurcation Chaos 14(2): 531 – 551. Froyland, G. (2001), “Extracting Dynamical Behavior Via Markov Models,” in A.I. Mees (ed.), Nonlinear Dynamics and Statistics, Boston: Birkh¨auser pp. 281–312. Froyland, G. (2005), “Statistically Optimal Almost-Invariant Sets”, Physica D 200: 205 – 219. G¨ardenfors, P. (2004) “Conceptual Spaces as a Framework for Knowledge Representations”, Mind and Matter 2(2): 9–27. Gardner, H. (1992), “Scientific Psychology: Should We Bury It or Praise It?” New Ideas in Psychology 10: 179–190. beim Graben, P. (2004), “Incompatible Implementations of Physical Symbol Systems”, Mind and Matter 2(2): 29–51. beim Graben, P., and Atmanspacher, H. (2006), “Complementarity in Classical Dynamical Systems”, Found. Phys. 36(2): 291–306. beim Graben, P., Saddy, J.D., Schlesewsky, M., and Kurths, J. (2000), “Symbolic Dynamics of Event-Related Brain Potentials,” Phys. Rev. E 62: 5518–5541. Gaveau, B., and Schulman, L.S. (2005), “Dynamical Distance: Coarse Grains, Pattern Recognition, and Network Analysis”, Bulletin de Sciences Mathematiques 129: 631–642. Haag, R., Kastler, D., and Trych-Pohlmeyer, E. B. (1974), “Stability and Equilibrium States,” Commun. Math. Phys. 38: 173–193. Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Harnad, S. (1990), “The Symbol Grounding Problem,” Physica D 42: 335–346. Henriques, G.R. (2003), “The Tree of Knowledge System and the Theoretical Unification of Psychology”, Review of General Psychology 7: 150–182. Hobson, J.A., Pace-Schott, E.F., and Stickgold, R. (2000), “Dreaming and the Brain: Toward a Cognitive Neuroscience of Conscious States,” Behavioral and Brain Sciences 23: 793–842. Hutt, A. (2004), “An Analytical Framework for Modeling Evoked and Event-Related Potentials”, Int. J. Bifurcation Chaos 14(2): 653 – 666. J¨ager, G. (2004), Evolutionary Game Theory for Linguists: A Primer. Unpublished manuscript, Stanford University and University of Potsdam.

158

Harald Atmanspacher and Peter beim Graben

Kaneko, K., and Tsuda, I. (2000), Complex Systems: Chaos and Beyond. Berlin: Springer. Keller, K., and Wittfeld, K. (2004), “Distances of Time Series Components by Means of Symbolic Dynamics,” Int. J. Bif. Chaos 14: 693–704. Kim, J. (1992), “Multiple Realization and the Metaphysics of Reduction”, Philosophy and Phenomenological Research 52: 1–26. Kim, J. (1993), Supervenience and Mind. Cambridge: Cambridge University Press. Koch, S. (1993), “ ‘Psychology’ or ‘the Psychological Studies’?” American Psychologist 48: 902–904. Lehmann, D., Ozaki, H., and Pal, I. (1987), “EEG Alpha Map Series; Brain Micro-States by Space-Oriented Adaptive Segmentation,” Electroencephalogr. Clin. Neurophysiol. 67: 271–288. Lind, D., and Marcus, B. (1995), Symbolic Dynamics and Coding, Cambridge: Cambridge University Press. Luzzatto, S. (2006), “Stochastic-like Behaviour in Nonuniformly Expanding Maps”. In B. Hasselblatt and A. Katok (eds.), Handbook of Dynamical Systems, Amsterdam: Elsevier, pp. 265–326. McNally, R.J. (1992), “Disunity in Psychology: Chaos or Speciation?” American Psychologist 47: 1054. Metzinger, T. (2003), Being No One, Cambridge: MIT Press. Newell, A. (1990), Unified Theories of Cognition, Cambridge: Harvard University Press. Olesen, D. and Petersen, G.K. (1978), “Some C∗ -Dynamical Systems With a Single KMS State,” Math. Scand. 42: 111–118.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Pinzari, C., Watatani, Y. and Yonetani, K. (2000), “KMS States, Entropy and the Variational Principle in Full C∗ -Dynamical Systems,” Commun. Math. Phys. 213: 331–379. Primas, H. (1977), “Theory Reduction and Non-Boolean Theories,” Journal of Mathematical Biology 4: 281–301. Primas, H. (1998), “Emergence in Exact Natural Sciences,” Acta Polytechnica Scandinavica 91: 83–98. Quine, W. V. (1969) “Natural Kinds”, in Ontological Relativity and Other Essays, New York: Columbia University Press. Robinson, C. (1999), Dynamical Systems. Stability, Symbolic Dynamics, and Chaos, Boca Raton: CRC Press. van Rooy, R. (2004) “Signalling Games Select Horn Strategies”, Linguistics and Philosophy 27: 491 – 527.

Contextual Emergence of Mental States from Neurodynamics

159

Ruelle, D. (1968), “Statistical Mechanics of a One-Dimensional Lattice Gas,” Commun. Math. Phys. 9: 267–278. Ruelle, D. (1989), “The Thermodynamic Formalism for Expanding Maps,” Commun. Math. Phys. 125: 239–262. Schack, B. (2004), “How to Construct a Microstate-Based Alphabet for Evaluating Information Processing in Time”, Int. J. Bifurcation Chaos 14(2): 793 – 814. Shalizi, C.R., and Moore, C. (preprint), “What Is a Macrostate? Subjective Observations and Objective Dynamics”. Manuscript available at philsciarchive.pit.edu/archive/0000/898/. Shapiro, L. (in press), “Can Psychology Be a Unified Science?” Philosophy of Science. Sinai, Ya.G. (1968a), “Markov Partitions and C-Diffeomorphisms,” Functional Analysis and Its Applications 2: 61–82. Sinai, Ya.G. (1968b), “Construction of Markov Partitions,” Functional Analysis and Its Applications 2: 245–253. Slife, B. (2000), “Are Discourse Communities Incommensurable in a Fragmented Psychology?” Journal of Mind and Behavior 21: 261–271. Staats, A.W. (1996), Behavior and Psychology: Psychological Behaviorism, New York: Plenum. Staats, A.W. (1999), “Uniting Psychology Requires New Infrastructure, Theory, Method, and a Research Agenda”, Review of General Psychology 3: 3–13. Steels, L. and Belpaeme, T. (2005), “Coordinating Perceptually Grounded Categories Through Language: A Case Study for Colour”, Behavioral and Brain Sciences 28: 469–489.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Sternberg, R.J., and Grigorenko, E.L. (2001), “Unified Psychology”, American Psychologist 56: 1069–1079. Sternberg, R.J., Grigorenko, E.L., and Kalmar, D. (2001), “The Role of Theory in Unified Psychology”, Theoretical and Philosophical Psychology 21: 99–117. Steuer, R., Ebeling, W., Bengner, T., Dehnicke, C., H¨attig, H., and Meencke, H.-J. (2004), “Entropy and Complexity Analysis of Intracranially Recorded EEG”, Int. J. Bifurcation Chaos 14(2): 815–824. Steuer, R., Ebeling, W., Russel, D., Bahar,S., Neiman, A., and Moss., F. (2001), “Entropy and Local Uncertainty of Data from Sensory Neurons,” Phys. Rev. E 64: 061911. Viana, R.L., Grebogi, C., de S. Pinto, S.E., and Barbosa, J.R.R. (2003), “Pseudodeterministic Chaotic Systems,” Int. J. Bifurcation Chaos 13: 1–19.

160

Harald Atmanspacher and Peter beim Graben

Viney, W. (1989), “The Cyclops and the Twelve-Eyed Toad: William James and the Unity– Disunity Problem in Psychology”, American Psychologist 44: 1261–1265. Wackermann, J., Lehmann, D., Michel, C.M., and Strik, W.K. (1993), “Adaptive Segmentation of Spontaneous EEG Map Series into Spatially Defined Microstates,” International Journal of Psychophysiology 14: 269–283 Wackermann, J. (1999), “Towards a Quantitative Characterisation of Functional States of the Brain: From the Non-Linear Methodology to the Global Linear Description. International Journal of Psychophysiology 34: 65–80. Werning, M. and Maye, A. (2004), “Implementing the (De-)Compositionality of Concepts: Oscillatory Networks, Coherency Chains and Hierarchical Binding,” in S.D. Levy and R. Gayer (eds.), Compositional Connectionism in Cognitive Science, Menlo Park: AAAI Press, pp. 67–81. Werning, M. and Maye, A. (this issue), “The Cortical Implementation of Complex Attribute and Substance Concepts: Synchrony, Frames, and Hierarchical Binding,” Chaos and Complexity Letters.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Yanchar, S.C., and Slife, B.D. (1997), “Pursuing Unity in a Fragmented Psychology: Problems and Prospects”, Review of General Psychology 1: 235–255.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 161-179

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 14

PHASE COUPLING SUPPORTS ASSOCIATIVE VISUAL PROCESSING: PHYSIOLOGY AND RELATED MODELS R. Eckhorn, A. Gail, A. Bruns, A. Gabriel, B. Al-Shaikhli and M. Saam Physics Department, Neurophysics Group, Philipps-University, D-35032 Marburg, Germany

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Abstract This is a review of our work on multiple microelectrode recordings from the visual cortex of monkeys and subdural recordings from humans - related to the potential underlying neural mechanisms. The former hypothesis of visual object representations by synchronization in visual cortex, or more generally, of flexible associative processing, has been supported by our recent experiments in monkeys. They demonstrated local coherence among rhythmic or stochastic γ-activities (30–90 Hz) and perceptual modulation, according to psychophysical findings of figure-ground segregation. However, γ-coherence in primary visual cortex (V1) of cats and monkeys is restricted to few millimeters, challenging the synchronization hypothesis for larger cortical object representations. We found that the spatial restriction is due to γwaves (30-90 Hz), traveling in random directions across the object representations in V1. It will be argued that the observed phase continuity of these waves can support the neural coding of object continuity. Based on models with spiking neurons, potentially underlying neural mechanisms in visual cortex are proposed: (i) Fast inhibitory feedback loops can generate locally coherent γ-activities. (ii) Spike-timing dependent synaptic plasticity of lateral and feed forward connections with distance-dependent delays can explain the stabilization of cortical retinotopy, the limited cortical range of signal coherence, the occurrence of γ-waves, and the larger receptive fields at successive levels of visual cortical processing. (iii) Slow inhibitory feedback can support figure-ground segregation. (iv) Temporal dispersion in far reaching cortical projections destroys coherence of high frequency signal components but preserves low frequency amplitude modulations. In conclusion, it is proposed that the hypothesis of flexible associative processing by γ-synchronization in visual cortex, supporting perceptually coherent representations of visual objects, has to be extended to more general forms of signal coupling.

Keywords: phase coupling, feature binding, scene segmentation, coherence, traveling waves, gamma activity

162

R. Eckhorn, A. Gail, A. Bruns et al. .

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

1. Introduction In the proposed view of the visual system, temporal coding is intimately linked to the neural mechanisms of dynamic cortical cooperativity and flexible associative processing, including the largely unknown mechanisms of perceptual feature binding. How are local features flexibly grouped into actually perceived objects and events, and how do their current representations interact with visual memory and other higher-order processes? It has been proposed that binding of spatially distributed features and inter-areal cooperation are supported by the temporal code of fast synchronization among neurons involved in a common task, for example, the coding of a visual object (Reitboeck 1983, von der Malsburg & Schneider 1986). This hypothesis attracted attention when coherent γ-oscillations (oscillations in 30–90 Hz frequency band) were found in the primary visual cortex (V1) of anesthetized cats (Eckhorn et al. 1988, Gray et al. 1989; Eckhorn 1994) and awake monkeys (Kreiter & Singer 1992, Frien et al. 1994). Many subsequent experiments were supportive, some challenging with respect to binding of local features by γ-coherence (reviews, e.g., Gray 1999; Eckhorn 1999). For example, coherence of signals in the γ-range was found to be restricted to few millimeters in primary visual cortex, even with large coherent stimuli (Eckhorn 1994, Frien & Eckhirn 2000). According to a strict interpretation of the original synchronization hypothesis, this should result in locally restricted perceptual feature binding. But this is in contradiction to the capability of perceiving local features of large objects as coherently bound. However, the capability of long-range feature binding across the surface of a large visual object is probably due to a continuous binding among overlapping regions of locally restricted feature binding (as demonstrated by the perceptual laws of Gestalt psychology; e.g., Wertheimer 1923). This view is supported by our observation of γ-waves that propagate across the surface of the representation of visual objects in the primary visual cortex of awake monkeys. Accordingly, we suggest that the phase continuity of such γ-waves (by which we mean a continuum of overlapping, near-synchronized, patches as opposed to strict long-range synchrony), may be a basis of spatial feature binding across entire objects. Such locally synchronous long-range phase coupling has been found to cover larger cortical areas than γ-synchrony as it is measured with spectral coherence (Gabriel & Eckhorn 2003), and we will argue why it can fill the entire surface representation of visual objects in primary visual cortex. Such continuity may not be available between separate topographical maps (different visual cortical areas). However, γ-coherence has been found between neural groups with overlapping receptive fields in the adjacent primary (V1) and secondary (V2) visual cortical areas in cats (Eckhorn et al. 1988, Eckhorn 1994) and monkeys (Frien et al. 1994). It is probable that such coherence is also present among other visual areas when feed-forwardbackward delays are short, e.g., as between primary (V1) and medio-temporal visual cortex MT (Nowak & Bullier 1997). In contrast, when cortical areas are far apart, long conduction delays may cause cooperativity to be reflected in other forms of signal coupling which are less sensitive to any spatio-temporal restriction of coherence. Taking into account the timevarying amplitude (amplitude envelope) of γ-signals seems to be a particularly promising approach (Bruns and Eckhorn 2004). For our present work different types of neural signals have been recorded, and different forms of temporal coding have been investigated by means of various coupling measures. We

Phase Coupling Supports Associative Visual Processing

163

will demonstrate dynamic coupling of cortical signals in the form of local intra-areal phase coupling, and medium-range phase continuity of γ-waves. Our examples show that neuralsignal measures correlate with sensory events, and with perceptual and behavioral outputs in monkeys. In essence, we argue that the temporal coding hypothesis of binding-bysynchronization, initially restricted to γ-synchrony of oscillatory signals, has to be extended to more general forms of temporal coding, including non-linear signal coupling across the entire frequency range of cortical activity with phase- and amplitude-coupling among transient and stochastic (non-rhythmic) signals. On the basis of neural models with locally coupled spiking neurons we will discuss most of the physiological results and suggest potential neural mechanisms underlying the presented types of flexible temporal coding.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

II. Definition of Synchrony and Correlation Measures, and Characterization of Recorded Signals Synchrony. In the present context the terms synchrony and synchronization are related to temporal properties of neurons, in particular to postsynaptic integration time. In a first approximation the integration time can be estimated as the half-height duration of an average excitatory postsynaptic potential. If multiple presynaptic action potentials arrive within the integration time of a neuron, we will call them synchronized. Integration times can differ in different neurons, in a single neuron at different synapses, and they can change dynamically. The latter effects depend on postsynaptic membrane properties and on the temporal patterns of spike input (Häusser & Roth1997; Fox & Daw 1992; Agmon-Snir & Segev 1993; Nelson1994). In cortical neurons integration times can span the broad range of 2 to 100 ms. In the activated cortex of awake mammals, in which we are interested in the present paper, integration times are estimated as ranging within 5 to 50 ms (König et al 1996). Synchronized neural events may occur singly, or repetitively with a more stochastic or rhythmic (oscillatory) character. At a target neuron, synchronized excitatory input spike patterns produce higher and steeper membrane depolarization than temporally dispersed inputs of the same average spike density. Hence, synchronization increases the output spiking probability (König et al 1996; Volgushev et al 1998; Lumer et al 1997; Abeles 1982) and also facilitates preservation of transmitted temporal spike patterns. Depending on the temporal synchronization range imposed by the aforementioned integration times, synchrony can show in different frequency bands of physiological signals. Synchronization on a temporal scale of 5 to 15 ms, for example, corresponds to the so-called gamma (γ-) range (30–90 Hz), and will therefore be termed γ-synchronization or γ-coherence. Recorded neural signals. The neural signals in the present work were recorded extracellularly with multiple microelectrodes from awake monkeys. Data are based on local population activity mirrored in multiple unit activity (MUA), and local field potentials (LFPs). MUA and LFP were generally recorded by the same microelectrode. Broad-band signals were band passed (1–10 kHz) to extract the spike activities. Further processing of MUA consisted of full-wave rectification and low-pass filtering at 140 Hz to obtain a continuous signal proportional in its amplitude to the extracellularly averaged spike densities of neurons near the electrode tip (half decline radius about 50 µm; Legatt et al 1980; Gray et al 1995). For

164

R. Eckhorn, A. Gail, A. Bruns et al.

LFP only band pass-filtering (0.1–140 Hz) of broad-band signals was used to obtain extracellularly averaged postsynaptic potentials (half-decline radius about 400 µm; Mitzdorf 1987). Due to the extracellular superposition, MUA and LFP amplitude increases as synchronization of the contributing neural signals becomes more precise. This means MUA and especially LFP recordings are per se sensitive probes for synchronized activity in the neural population contributing to the signals. But they are also more sensitive to coupling among separated neural populations than are single unit spike trains. Even during states of high synchrony within a neural population, single unit spike activity is only weakly coupled to the group signals (Eckhorn 1991). Different forms of signal coupling quantified by appropriate measures. Cortical interaction has to serve various functions, and has to meet different spatial and temporal requirements. Therefore, cortical interaction will certainly be based on a variety of neuronal mechanisms. It would be naive to expect a single type of temporal coding, e.g. phase synchronization in the γ-range, to be indicative for any form of cooperativity. For the present paper, we used different coupling measures which are all selective for signal frequencies. The main difference between the measures concerns their sensitivity for the signals’ phase structure. To quantify synchrony we used spectral coherence (Glaser & Ruchkin 1976). High coherence requires a phase difference between two signals which has to be constant (but not necessarily zero) over time. Thus, synchrony in this context does not mean precise coincidence of events, but it still indicates a temporally stable phase relationship. Coherence determines linear coupling at each frequency independently. To quantify phase continuity, i.e., to detect traveling waves, we used a new and highly adapted multi-channel correlation method (Gabriel & Eckhorn 2003). This method, in contrast to coherence or cross-correlation, allows variable relative phases over time between two separated recording positions. But within each short time interval (20 to 40 ms) the phase gradient across space has to be constant. For example, signals obtained from linearly arranged, equidistant recording positions need to have phase differences proportional to the distance of the recording positions. Note that for model data we mimic the physiological signal types and use exactly the same measures for analysis of both types of data.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

III. Experimental Evidence A. γ-Activity in Monkey Primary Visual Cortex is Phase-Coupled within Representations of Scene Segments and Decoupled Across their Contours The binding-by-synchronization hypothesis suggests coupling among γ-activities representing the same object, or more generally, the same scene segment. Accordingly, neural groups representing different scene segments should decouple their γ-activities. Both predictions have been tested by investigating the effect of a static figure-ground stimulus on local field potentials (LFPs; see Appendix) in primary visual cortex (V1) of awake monkeys, recorded simultaneously from inside and outside a figure’s representational area (Figure 1A) (Gail et al. 2000). Time-resolved analysis of phase coupling by means of spectral coherence revealed: (i) γcoherence between neurons representing the same scene segment (figure or ground) is higher than for a homogeneous gray background of the same average luminance (Figure 1B,D); (ii)

Phase Coupling Supports Associative Visual Processing

165

stimulus-specific γ-coherence is strongly reduced across the representation of the figure-ground contour compared to a spatially continuous stimulus (Figure 1B,D); (iii) decoupling across the contour emerges with a latency of about 100 ms, and is absent in the earliest neuronal response transients (Figure 1D); (iv) coherence of low-frequency components does not show a difference between the figure-ground and the continuous condition (not shown; note that the receptive fields of the recording locations used for these analyses all did not border on the object’s contour, they were either at its surface or in its background). We propose that the increased γcoherence between neurons representing the same scene segment and the decoupling of γactivity at a contour representation are crucial for figure-ground segregation, in agreement with the initial binding-by-synchronization hypothesis.

A receptive fields

object

background with object

without object

Experiment

C

0.5

Model N = 10

N = 68

g-coherence

B

0.25

pre-stimulus

0

0

pre-stimulus without with object

without with object

D g-coherence

without object with object

0.4

Model 0.8 0.6

0.3 0.4

0.2

0.2

0.1 0.0

0.0 0

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

E

Experiment 0.5

200 400 time / ms

600

0

200 400 time / ms

600

Figure 1. Coherence of γ-activity is reduced across the representation of an object’s contour. A: Figureground stimulus and schematic positions of receptive fields. Stimuli were arranged in such a way that some of the receptive fields from the simultaneously recorded sites were located on the “object” surface (only present in the left condition), the others on the “background”, in both cases the receptive fields of the recording locations did not border on the object’s contour. B: A grating without object (right condition in A) induced a substantial increase in γ-coherence among local field potentials (LFPs) (light gray) compared to a blank screen condition (pre-stimulus: dashed line). Introduction of the object (left condition in A) reduced LFP γ-coherence between object and background representations almost to prestimulus level (dark gray) (Gail et al. 2000]. Coherence within each segment (object or background) remained high (data not shown). C: A network model (Figure 11) shows equivalent results. D,E: Time courses of coherence in the no-object condition (light gray) and across the object-background contour (dark) in the experiment and the model. Note that decoupling across the contour emerges about 100 ms after stimulus-onset. Data in B is taken from the time intervals with maximal decoupling for each monkey. (Modified from (Gail et al. 2000, Eckhorn et al 2001).)

166

R. Eckhorn, A. Gail, A. Bruns et al.

¶ γ-Phase Coupling in Monkey Extra-striate Cortex Correlates with B. Perceptual Grouping Are such synchronization effects correlated with perceptual feature grouping and figureground segregation? This was tested in a difficult figure-ground task in which a monkey indicated whether he perceived a figure composed of blobs among identical distractor blobs serving as background (Woelbern et al. 2002) (Figure 2). This task was sufficiently difficult such that about 25 % of responses were incorrect (failed figure detection). Pairs of local populations of figure-activated neurons in the secondary visual cortical area (V2) showed increased coherence within the γ-range in correct compared to incorrect responses during a short period before the monkey’s behavioral response (Figure 2). Other signal measures were unrelated to perception. These were the first indications that γ-coherence in visual cortex may not only represent physical stimulus properties but also supports perceptual grouping.

fixation point

2

3

frequency / Hz

100

10

5

50

0

0 –800

–600

–400

time / ms

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

significance level / %

1

–200

0

decision

Figure 2. A monkey’s correct perception of the orientation of a dual-inline row of dots within a set of distractors (stimuli 1 and 2) caused a short increase in coherence at about 80 Hz and 60 Hz in visual area V2, shortly before the monkey reported his perception (time t = 0 ms). The time-frequency map indicates the significance of increase in LFP coherence in trials with correct vs. failed detection of the figure. Three figure-ground stimuli are shown above, with dot rows being left-tilted (left), right-tilted (middle) or absent (right). (Modified from (Woelbern et al. 2002)).

C. Phase Continuity but not Synchrony of γ-Waves is Present across Medium Cortical Distances in Monkey Primary Visual Cortex Previous work demonstrated that the range of γ-coherence in primary visual cortex is limited to about 5 mm (e.g., (Frien & Eckhorn 2000, Steriade et al. 1996) and Figure 4A). Hence, objects with larger cortical representations can not solely be coded by γ-coherence within their representational area. One explanation for the limited cortical range of coherence lies in the spatio-temporal characteristics of γ-activity. In fact, wave-like phenomena defined by

Phase Coupling Supports Associative Visual Processing

167

coritcal distance / mm

4.5

model distance / space units

spatially continuous waves (γ-waves) do extend farther than 5 mm, but phase differences between any two sites change randomly already within 100 ms and also increase with cortical distance (Figure 3A) (Gail et al. 2000). Conventional pairwise coupling measures (cross-correlation, coherence) do not capture such non-trivial phase relationships across medium-range cortical distances, which explains the findings of restricted cortical ranges of coherence. To quantify those waves a new method has been developed in our group (Gabriel & Eckhorn 2003). It revealed that γ-waves travel at variable velocities and directions. Figure 3C shows the velocity distribution measured with a 4 × 4 microelectrode array in monkey primary visual cortex during retinally static visual stimulation. Note that this distribution is rather similar to the distribution of spike velocities of horizontal connections in this area (V1) (Nowak & Bullier 1997). We suggest that continuity of γ-waves supports the coding of object continuity, in which case their extent over object representations in visual area V1 and the related visual fields should be much larger than that covered by γ-coherence. We have indeed found that the cortical span of γ-wave fronts is much larger than the span of γ-coherence (Figure 4A,B) and that γ-waves are cut-off (damped down) at the cortical representation of object contours.

15

A: Monkey LFP

0

B: Model LFP

0

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

incidence of g-waves

200

0.05

300

400 time / ms

500

600

C: Velocity distribution (monkey LFP) 448 trials N = 22848

0 0.5

1.0 velocity / ms–1

1.5

2.0

Figure 3. γ-waves occur with fast and random changes of spatial phase relations in monkey primary visual cortex. A: Time-space maps of simultaneously recorded single-trial time courses of local field potentials (LFPs) from a linear array of 7 recording positions during sustained static visual stimulation. Gray scales give the instantaneous electrical potentials at the electrodes (in arbitrary units). B: ModelLFPs during presentation of an equivalent stimulus. (Modified from (Eckhorn et al. 2001).) C: Velocity distribution of traveling waves of γ-activity measured with 4 × 4 microelectrode arrays in monkey primary visual cortex.

R. Eckhorn, A. Gail, A. Bruns et al. 1 g-coherence

N = 325

A 00

6 2 4 cortical distance / mm

8

1

0

B 0

5 10 15 20 model distance / space units

g-wave probability

g-coherence

N=8

C 0

N = 325

6 2 4 cortical distance / mm

8

1

Model

0

0.6 g-wave probability

168

0 0

Model

N=8

D 5 10 15 20 model distance / space units

Figure 4. The spatial range of γ-waves is larger than that of γ-coherence. A: Coherence of local field potentials in monkey primary visual cortex is restricted to few millimeters (half-height decline 2.2 mm). B: The probability of occurrence of continuous γ-waves remains high across larger distances (estimated half-height decline: 9.5 mm). C, D: The model shows equivalent dependencies (4.1 vs. 12.8 space units). (Modified from (Eckhorn et al. 2001).)

IV. Potential Neural Mechanisms of Flexible Signal Coupling At present it is not possible to identify directly from experimental measurements the neural mechanisms underlying the above mentioned experimental observations of spatio-temporal processing in cortical sensory structures. We therefore use largely reduced model networks with spike-coding neurons to discuss potential mechanisms.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

A. γ-Oscillations and Coherence of Spike Density in Local Populations, Generated by Feedback Inhibition and Local Lateral Coupling ¶ How can the cortex generate γ-oscillations in local neural groups, as observed in the extracellularly recorded population spike densities (multiple unit activities, MUA) and local field potentials (LFP, 0.1 Hz to 150 Hz; e.g., Eckhorn 2000)? We argue (see Figure 5) that membrane potentials of local populations of excitatory neurons are simultaneously modulated by inhibition exerted via a common feedback loop (physiology: (McGuire et al. 1991, Bosking et al. 2002, Jefferys et al. 1996); models: (Chang & Freeman 1996, van Vreeswijk et al. 1994, Freeman 1996, Bush & Sejnowski 1996, Wennekers & Palm 2000); discussion in (Eckhorn 2000)). This loop can quickly reduce transient activations, whereas sustained input will lead to repetitive inhibition of the population in the γ-frequency range (Figure 5). In both modes – transient and rhythmic chopping – the common modulation of the neurons’ membrane potentials causes their spike trains to become partially synchronized, even if they fire at very different rates. The stronger a neuron is activated and depolarized, the earlier it will discharge its first spike during the common repolarization phase, whereby such a

Phase Coupling Supports Associative Visual Processing

169

population burst will be dominated by the most strongly activated neurons. As local cortical populations generally project to common targets (Braitenberg & Schüz 1991), temporally coherent spike densities (as they occur with population spike packages) will have stronger impact there than uncorrelated spike densities of equal average amplitudes, because they (i) appear quasi-simultaneously, and (ii) mainly comprise spikes of strongly activated neurons, which represent the stimulus at a better signal-to-noise ratio than the neurons that were less activated by the same stimulus. In addition to partial local synchronization by inhibitory feedback, the most relevant mechanism for explaining the generation of coherent signals in our models are lateral, activity-dependent, facilitatory connections. Local (instead of global) lateral connections are critically important for models of visual feature-binding by synchronization when pattern segmentation (desynchronization) is an important task (e.g. (Eckhorn et al. 1990, Wang 1995, König & Schillen 1991)). While Wang used lateral excitatory connections that were modulated in their efficacy by scene properties (in some respect similar to the facilitatory connections in our models (Eckhorn et al 1990)), others used lateral connections from excitatory to inhibitory neurons for synchronization (Bush & Sejnowski 1996, König & Schillen 1991). It is likely that a mixture of these local mechanisms is operative in the generation of rhythmic spiking activities and their partial synchronization. Future experiments have to answer this question. We can apply the discussed schemes to the primary visual cortex, where local neural clusters represent similar feature values (e.g., receptive field position, contour orientation, etc.). According to the synchronization hypothesis, partial synchronization of spike densities by a common inhibitory feedback means that currently present local combinations of visual feature values are systematically selected by their strength of activation and tagged as belonging together, which is reflected in single or repetitive population discharges. output spike density input spike density

common inhibition io tat

exci

n I

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

time

inhibition

time

Figure 5. Basic model of common spike density modulation in a local population of excitatory neurons by a common inhibitory feedback neuron. Note that first-spike latencies in each modulation cycle at the outputs (right) are roughly inversely proportional to the input spike densities (profiles at left), whereas the output spike rates are proportional to it (details in text).

Other models of visual feature binding use local oscillators, consisting of excitatory and inhibitory units with mutual feedback that generate local oscillations depending on a driving input (e.g. (Li 1998, Wang 1995, König & Schillen 1991)). In these models, the oscillatory signal of a local element stands for the spike density of a local group of partially synchronized spike-coding neurons. Thus, local inhibition in these models implicitly represents the synchrony of local populations of spike-coding neurons with similar receptive field properties, as has been explicitly modeled in our and other simulations (e.g., (van Vreeswijk et al. 1994, Bush & Sejnowski 1996)).

170

R. Eckhorn, A. Gail, A. Bruns et al.

level-2 constant constant

constant

velocity

delay

velocity

level-1 Scenario C

Scenario B

Scenario A

Figure 6. One-dimensional sketch of the initial connectivity in the model with spike-timing dependent plasticity (Hebbian learning) including distance-dependent lateral conduction delays. For a given level1 neuron (dark), the scheme shows lateral modulatory (facilitatory) connections (scenario A), and feedforward connections with either distance-dependent (scenario B) or constant (scenario C) conduction delays. (Modified from (Saam & Eckhorn 2000).)

B. Lateral Conduction Delays can Limit γ-Coherence to Few Millimeters ¶ in Cortex, Produce Wave-Like Phenomena, Stabilize Cortical Topography, and Lead to Larger Receptive Fields at Successive Levels of Processing The synchronization effect of fast orientation-specific inhibitory neurons is probably restricted to an area smaller than a single hypercolumn in primary visual cortex (Braitenberg & Schüz 1991). The most relevant mechanism for explaining flexible synchronization across several millimeters in the cortex in our (Eckhorn et al. 1990) and Wang’s (1995) model are the activity-dependent facilitatory connections. They are also highly useful for enabling fast desynchronization as is required for scene segmentation. Their putative physiological substrate in the primary visual cortex are the dense horizontal connections: they cannot directly excite their target neurons, but modulate their activities evoked from their classical receptive fields. The lateral connections project monosynaptically over a range of several hypercolumns (McGuire et al. 1991, Bosking et al. 1997,2002, Gilbert 1993), and models have shown that this type of connectivity is capable of synchronizing neural populations across larger distances (Eckhorn et al 1990, Wang 1995). Another type of local lateral connectivity enabling transient synchronization over larger distances was proposed in the model of König and Schillen (1991). They connected their oscillators by coupling the 20

B

20 0 225 neuron #

0 225 neuron #

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

A

0

0 0

time / ms

200

0

time / ms

200

Figure 7. Spatio-temporal properties of level-1 output activity in the learning model. A: Two events of spatially homogeneous, transient spike-rate enhancement (upper panel: total population spike density; lower panel: single spike traces). B: As in A, but with additional independent Gaussian white noise at the inputs. Note that the activity is spatially homogeneous in the sense that any two spike trains have the same weakly correlated temporal statistics. (Modified from (Saam & Eckhorn 2000).)

Phase Coupling Supports Associative Visual Processing

171

excitatory units via delay lines to the neighboring inhibitory units. However, it is difficult to show experimentally which mechanisms are operative in the visual cortex for synchronization across several hypercolumns.

rel. vertical pos.

10

0

0.025 synaptic weight

B

10 rel. vertical pos.

A

0

0.00

-10

-10 -10

0 10 rel. horizontal pos.

-10

0 10 rel. horizontal pos.

Figure 8. Spatial coupling profile between a single level-1 neuron (center position) and its neighbors. A: Before learning weights are randomly initialized. B: After learning synaptic strength decays with increasing distance. (Modified from (Saam & Eckhorn 2000).)

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In visual processing for example, one could suppose that neural populations representing the entire surface of a visual object might synchronize their spike packages via horizontal connections. However, γ-coherence is restricted to about 5 mm of cortical distance in area V1 of awake monkeys (corresponding to 5 hypercolumns), even if the cortical representation of a visual object is much larger (Eckhorn 1994, Frien & Eckhorn 2000). Hence, feature binding based on γ-coherence would also be restricted to visual objects being not larger in their cortical representations. In the following we will develop a concept of how distancedependent spike conduction delays can explain this restricted range of γ-coherence and the occurrence of wave-like phenomena in a network of spiking neurons. In addition, we will show that spike-timing dependent synaptic plasticity (Hebbian learning; Hebb 1949; Kempter et al 1999) combined with distance-dependent spike conduction delays leads to spatially restricted lateral connectivity within the same layer and restricted feed-forward divergence between different layers. Therefore, such a mechanism is also suitable to explain the emergence of larger receptive fields at successive levels of processing while preserving a topographical mapping. Note that these conditions are also present in topographically organized cortical areas of other sensory modalities, including auditory and somatosensory. 1) Model of Spike-timing Dependent Synaptic Plasticity with Finite Conduction Velocities. The local generation of γ-oscillations and their spatial coherence are two separate mechanisms. For the sake of simplicity, the following model solely investigates synchronization effects, thereby neglecting any inhibition and the generation of oscillations. The model (Saam & Eckhorn 2000) consists of spike-coding neurons (as in Figure 11B) at two successive, 2-dimensional retinotopic visual processing stages named level-1 (representing visual cortical area V1) and level-2 (V2) (Figure 6). Learning of lateral weights and level-1-to-level-2 weights is implemented using a spike correlation rule (spike-time dependent plasticity: Kempter et al. 1999). Feed-forward connections are additive excitatory and determine the properties of the classical receptive fields. Lateral connections are multiplicatory (with a positive offset of one), which means they cannot directly evoke spikes in a target neuron (as exitatory synapses can do), but require quasi-simultaneous feed-forward

172

R. Eckhorn, A. Gail, A. Bruns et al.

input to that neuron (model: (Eckhorn et al. 1990); physiology: (Fox & Daw 1992)). Spikes evoked by quasi-simultaneous feeding input to neighboring neurons can synchronize via their mutual lateral facilitatory connections because these spikes will often occur within the socalled capture range of the spike encoder’s dynamic threshold (Eckhorn et al. 1990, Johnson 1993, 1994). The lateral connections have constant conduction velocities, i.e., conduction delays become proportionally larger with distance. This reduces the probability of neurons becoming quasi-synchronized because constructive superposition of locally evoked and laterally conducted activities gets less probable for increasing delay. Hence, signal coherence is laterally restricted to a spatial range which is proportional to the conduction velocity of the lateral connections.

size of coupling kernel / space units

12

8

4

0 0.5

N=5 1.0 lateral condution velocity / ms–1

1.5

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 9. As a result of spike-timing dependent plasticity (Hebbian learning), the size of the synaptic weight profile (coupling kernel) of lateral linking connections within level-1 becomes directly proportional to the lateral conduction velocity. (Modified from (Saam & Eckhorn 2000).)

2) Spatio-Temporal Structuring of Lateral Connectivity with Spike Time Dependent Plasticity. The relation between conduction velocity and coherence range suggests an influence of temporal neighborhood (defined by the distance-dependent delays) on the ontogenetic, possibly prenatal formation of functionally relevant structures from an initially unstructured system (Trachtenberg & Stryker 2001, Ruthazer & Stryker 1996, Crair et al. 1998). This effect can be simulated with our model. In the beginning, neurons are fully interconnected within level-1 (Figure 6, scenario A). Feed-forward input spike trains have spatially homogeneous random patterns and are given a temporally confined, weak comodulation, mimicking activity before visual experience. This type of spike pattern appears, slightly modified by the connections, at the output of the level-1 neurons (Figure 7) and hence, is used for spike-timing dependent plasticity (Hebbian learning). The only topography in the network is given by the distance-dependent time delays of the lateral connections. During a first learning period, the homogeneous coupling within layer-1 shrinks to a spatially limited coupling profile for each neuron, with a steep decline of coupling strength with increasing distance (Figure 8). The diameter of the resulting coupling profile for each neuron is near the lateral range of coherence, and hence directly proportional to the lateral conduction velocity (Figure 9). 3) Spatio-Temporal Structuring of Inter-Level Connectivity. In a second learning period following the learning period within level-1, the excitatory level-1-to-level-2 connections are

Phase Coupling Supports Associative Visual Processing

173

adapted, also starting from full connectivity (Figure 6, scenario B). Again, as a result of Hebbian correlation learning (Kempter et al. 1999), the feed-forward divergence retracts to a limited spatial range which is given by the size of the level-1 “synchronization fields”, i.e., excitatory forward connections from neurons within a level-1 synchronization field (sending near-synchronized spike packages) converge onto one level-2 neuron (Figure 10). This convergent projection pattern even emerges if the feed-forward connections and the level-2 lateral connections are modeled with distance-independent constant delays (Figure 6, scenario C). The physiological interpretation of this result is that the size of level-1 synchronization fields (in visual area V1) can determine the size of level-2 receptive fields (in area V2). Indeed, synchronization fields in V1 and classical receptive fields in V2 of the monkey do have similar sizes. Since equivalent considerations should hold for projections from the retina to V1, the model accounts for the emergence not only of a spatially regular, but also of a retinotopically organized connectivity. average size of level-1-to-2 coupling profile / space units

12 10 8 6 4 2

3

5 7 average size of level-1 lateral coupling profile / space units

9

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 10. The size of level-1 “synchronization fields” determines the size of receptive fields of level-2 neurons. Synaptic weight profiles of level-1-to-2 feeding connections evolve correspondingly to synaptic weight profiles of level-1 lateral linking connections. (Modified from (Saam & Eckhorn 2000).)

4) Traveling γ-Waves with Altering Phase Relations. After the learning period, we wanted to investigate the network dynamics. To compare the results with experimental data, we added local inhibitory feedback neurons and provided a sustained spatially homogeneous input to level-1 neurons. This inhibition did not invalidate the previous results, because its dynamics rarely overlap with the learning process. This model reproduces the phenomenon of waves with jittering phase relations, traveling in random directions, just as it was observed in the primary visual cortex (Figure 3) (Gabriel & Eckhorn 2003, Eckhorn et al. 2001). Traveling waves of γ-activity, though concentrically expanding, have already been described in different cortical areas of different animal species (e.g., (Freeman & Barrie 2000)). The varying phase relations in our model as well as the more rapid spatial decline of γ-coherence (compared to γ-wave probability) are consistent with the experimental data (Figs. 4A,B, 5). Formation of γ-waves in the model results from the locally restricted inhibition, the lateral conduction velocity, and the steep spatial decline of coupling strength (Saam & Eckhorn 2000). It seems probable that cortical γ-waves are also strongly depending on the lateral conduction velocities, because the distribution of γ-wave velocities (Figure 3C) is similar to the distribution of spike conduction velocities of lateral connections in primary visual cortex.

174

R. Eckhorn, A. Gail, A. Bruns et al.

These velocities have been estimated in different preparations, including rat slices and in vivo recordings from cats and monkeys, to range between 0.1 and 1.0 m/s (review: (Nowak & Bullier 1997)). slow inhibition

A

fast inhibition

fast inhibition

+S +

0°

90° la te ra lc ou pl in g

lateral coupling

receptive field simple cells

B

receptive field

simple cells

+

lateral coupling

S

+

1

model neuron

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

slow shunting inhibition visual input excitatory feeding

+

fast inhibition

-

S

spike encoder

output

Figure 11. A: Simplified model of primary visual cortex with two retinotopic layers (left and right) of orthogonal orientation detectors. Each layer consists of an excitatory (simple cells at 0° and 90°) and an inhibitory sublayer forming fast, local feedback loops preferring the γ-frequency range (Figure 7). The cones between layers indicate direction and width of divergent projections of each single neuron. Modulatory (facilitatory) lateral coupling is confined to excitatory neurons of the same orientation preference with coaxially aligned receptive fields. Both orientation maps share an additional sublayer (middle) mediating slow, local shunting inhibition. To account for stochastic input from brain regions excluded from the model, all neurons receive independent broad-band noise. B: The spike-coding model neuron with dynamic threshold (Eckhorn et al 2000) is extended by inputs exerting shunting inhibition on the feeding pathway. Synapses are modeled by leaky RC-filters, lateral modulatory input is offset by a value of +1 and then coupled to the feeding pathway by multiplication.

In conclusion, the lateral conduction velocities in primary visual cortex, combined with Spike-timing dependent synaptic plasticity (Hebbian correlation learning), can explain the restricted spatial range of γ-coherence and the occurrence of traveling γ-waves with random phase relations. They can also account for the larger receptive fields at higher processing levels and for the emergence and stability of topographic visual (and other sensory) representations without the need for visual (sensory) experience. During visual experience, of course, similar influences on synchronization-field and receptive-field size and on topographic stability are probably operative at successive levels of processing, including other parts of the visual system. As the traveling waves can cover the entire representation of an

Phase Coupling Supports Associative Visual Processing

175

object’s surface in the primary visual area, we propose that phase continuity of γ-waves may constitute a mechanism that supports the coding of object continuity in visual cortex (Gabriel & Eckhorn 2003).

C. Model Explaining Figure-Ground Segregation and Induced Modulations at Lower Frequencies by Slower and More Globally Acting Feedback Circuit

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In a further approach the above model of the visual cortex has been expanded by using orientation-selective excitatory neurons and two types of inhibitory neurons with different spatio-temporal properties. However, the lateral connections among the excitatory neurons were modeled without delay and learning has been excluded in order to keep the complexity of the network within limits in order to understand its processing. Figure11A shows a simplified wiring diagram of this model. The spiking neurons (Figure11B) have linearly and nonlinearly acting synapses and are retinotopically arranged in two primary layers with receptive fields at perpendicular orientation preferences. Additionally to the fast inhibitory feedback loop, serving neurons with similar orientation preference and generating local γrhythms (see above and Figure 5), a slow shunting inhibition is added in this model that forms a feedback circuit among neurons with overlapping receptive fields and receive input from, as well as feed output to, excitatory neurons of all orientation preferences. 1) Decoupling of γ-Signals Across Figure-Ground Contour. In the figure-ground experiment, representations of different scene segments in primary visual cortex (area V1) were decoupled in their γ-activities (Figure 1B,D), while the same sites showed substantial γcoupling when representing one coherent scene segment. Analogous results are obtained with the model (Figure 1C,E). It explains the reduced γ-coherence as a blockade of lateral coupling at the position of the contour representation due to several effects: First, neurons responding preferentially to the horizontal grating are only weakly activated by the vertical contour. Second, their activity is even more reduced by the orientation-independent slow shunting inhibition that is evoked by the strongly activated vertically tuned neurons at the contour. As a consequence, neurons activated by the horizontal grating near both sides of the contour can not mutually interact, because the orientation-selective lateral coupling is interrupted by the inhibited horizontally tuned neurons at the contour representation. The resulting decoupling of inside and outside representations is not present during the first neural response transient after stimulus onset (Figure 1D,E) since the sharp, simultaneous response onset common to both orientation layers denotes a highly coherent broad-band signal which dominates internal dynamics. Note that orientation-selectivity was implemented for the sake of comparability with the experimental data. This does not limit the general validity of this model, since any object border constitutes a discontinuity in at least one visual feature dimension, and therefore an analogous argumentation always holds for other local visual features. 2) Spatial and Temporal Aspects of Object Representations. We have seen that within an object’s representation in primary visual cortex (V1), locally coherent γ-activations emerge that overlap in space and time and thereby support the formation of γ-waves traveling across the object's surface representation with random phase relations. When waves of γ-activity

176

R. Eckhorn, A. Gail, A. Bruns et al.

travel across V1, this is paralleled by quasi-synchronous activity of those neurons in area V2 having corresponding receptive field positions (Eckhorn et al. 1988, Eckhorn 1994, Frien et al. 1994), i.e., those receiving input from, and sending γ-feedback to, corresponding V1 neurons. Thus, adjacent V2 neurons, driven simultaneously by adjacent parts of a traveling wave, will also form a traveling wave of γ-activity (with similar extent, velocity and direction if projected to visual space). We expect such an argumention to hold for subsequent stages of processing, provided that they are retinotopically arranged, are activated by bottom-up input, and have fast inter-areal feedback (compared to a half-cycle of a γ-wave). Accordingly, quasisynchrony should generally be present among neurons with overlapping receptive field positions across cortical levels connected via fast feed-forward-backward loops (e.g., as among primary and secondary visual cortex (V1-V2) and among primary visual cortex and the next cortical area in the dorsal visual patway supporting visuo-motor tasks (V1-MT; Nowak & Bullier 1997, Girard et al. 2001). As the traveling waves are γ-activities and we observed γ-decoupling across the cortical representation of a figure-ground contour (explained in our model by the slow inhibition of neurons at the contour), we assume that the waves do not pass object contour representations with any figure-ground feature contrast. Object continuity across the entire surface may thus be coded by phase continuity of traveling γ-waves.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

V. Summary We investigated neural mechanisms of associative processing by considering a variety of flexible forms of signal coupling. In particular, we were interested in associations of local visual features into perceptually coherent visual objects by transient synchronization. In our recent experiments in monkeys, we have shown that local coherence among γ-activities in visual cortex correlates with perceptual modulation, which supports the hypothesis of object representation by synchronization in the visual cortex. The synchronization hypothesis for larger cortical object representations, however, has been challenged by our experimental finding that γ-coherence in primary visual cortex (V1) already drops to noise levels across few (4-6) millimeters of cortex. We can explain this restriction by the randomly changing phase relations among local patches of coherent signals, which, however, form continuous waves of γ-activity, traveling across object representations. Extending the initial synchronization hypothesis, we propose that phase continuity of these waves may support the coding of object continuity across intermediate and longer ranges within V1. We have discussed the different types and ranges of experimentally observed signal coupling on the basis of visual cortex models with locally coupled, spike-coding neurons. In these models, the lateral, activity-dependent facilitatory connections with distance-dependent delays are the most important feature for explaining coherent activities. They can account for local and medium-range γ-coherence, the occurrence of γ-waves and the limited extent of γsynchrony. Spike-timing dependent plasticity of these connections can explain the stabilization of cortical retinotopy and the larger receptive fields at successive levels of visual cortical processing. Fast local feedback inhibition in our models can generate local γoscillations and support their local coherence, while slow shunting-inhibitory feedback supports figure-ground segregation by decoupling activities within neighboring cortical representations of figure and background. In conclusion, we propose that the hypothesis of

Phase Coupling Supports Associative Visual Processing

177

associative processing by γ-synchronization be extended to more general forms of signal coupling.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

References Abeles M. (1982) Local Cortical Circuits. Springer, Berlin, Heidelberg. Agmon-Snir H., Segev I. (1993) Signal delay and input synchronization in passive dendritic structures. J. Neurophysiol. 70: 2066-2085. Bosking W. H., Y. Zhang, B. Schofield, D. Fitzpatrick (1997) Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. J. Neurosci. 17: 21122127. Bosking W. H., J. C. Crowley, and D. Fitzpatrick (2002) Spatial coding of position and orientation in primary visual cortex. Nature Neurosci. 5: 874–882. Braitenberg V., and A. Schüz (1991) Anatomy of the cortex. Berlin: Springer. Bruns A., and R. Eckhorn (2004) Task-related coupling from high- to low-frequency signals among visual cortical areas in human subdural recordings. Internat. J. Psychophysiol. 51:97-116. Bush P., and T. Sejnowski (1996) Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network models. J. Comput. Neurosci. 3:91–110. Chang H.-J., and W. J. Freeman (1996) Parameter optimization in models of the olfactory neural system”. Neural Networks 9: 1–14. Crair M. C., D. C. Gillespie, and M. P. Stryker (1998) The role of visual experience in the development of columns in cat visual cortex. Science 279: 566–570. Eckhorn R., Bauer R., Jordan W., Brosch M., Kruse W., Munk M., Reitboeck H. J. (1988) Coherent oscillations: a mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat, Biol. Cybernetics 60:121–130. Eckhorn R., H. J. Reiboeck, M. Arndt, and P. Dicke (1990) Feature linking via synchronization among distributed assemblies: simulations of results from cat visual cortex. Neural Computation 2: 293–307. Eckhorn R. (1991) Stimulus-specific synchronizations in the visual cortex: Linking of local features into global figures? In: Neuronal Cooperativity. Springer Series in Synergetics. J. Krüger (ed.). Springer, Berlin, Heidelberg, pp. 184-224. Eckhorn R. (1994) Oscillatory and non-oscillatory synchronizations in the visual cortex of cat and monkey, in Oscillatory Event-Related Brain Dynamics, C. Pantev, T. Elbert, and B. Lütkenhöner, Eds. New York: Plenum Press, pp. 115–134. Eckhorn R. (1999) Neural mechanisms of visual feature binding investigated with microelectrodes and models. Vis. Cogn. 6: 231–265. Eckhorn R. (2000) Cortical processing by fast synchronization: high frequency rhythmic and non-rhythmic signals in the visual cortex point to general principles of spatiotemporal coding. In Time and the Brain. R. Miller, Ed. Lausanne: Gordon&Breach, pp. 169–201. Eckhorn R., A. Bruns, M. Saam, A. Gail, A. Gabriel, and H. J. Brinksmeyer (2001) Flexible cortical gamma-band correlations suggest neural principles of visual processing. Visual Cogn. 8: 519–530. Fox K., Daw N. (1992) A model of the action of NMDA conductances in the visual cortex. Neural Computation 4:59-83.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

178

R. Eckhorn, A. Gail, A. Bruns et al.

Freeman W. J. (1996) Feedback models of gamma rhythms. Trends Neurosci. 19: 468–470. Freeman W. J., and J. M. Barrie (2000) Analysis of spatial patterns of phase in neocortical gamma EEGs in rabbit,” J. Neurophysiol. 84:1266–1278. Frien A., R. Eckhorn, R. Bauer, T. Woelbern, and H. Kehr (1994) Stimulus-specific fast oscillations at zero phase between visual areas V1 and V2 of awake monkey. Neuroreport, 5:2273–2277. Frien A., and R. Eckhorn (2000) Functional coupling shows stronger stimulus dependency for fast oscillations than for low-frequency components in striate cortex of awake monkey. Eur. J. Neurosci 12: 1466–1478. Gabriel A.,and R. Eckhorn (2003) Multi-channel correlation method detects traveling gamma-waves in monkey visual cortex. J. Neurosci. Methods. 131: 171-184. Gail A., H. J. Brinksmeyer, and R. Eckhorn (2000) Contour decouples gamma activity across texture representation in monkey striate cortex. Cereb. Cortex 10: 840–850. Gilbert C. (1993) Circuitry, architecture, and functional dynamics of visual cortex. Cereb. Cortex, 3: 373–386. Glaser E.M., Ruchkin D.S. (1976) Principles of neurobiological signal analysis. Academic Press, New York. Gray C.M., Maldonado P.E., Wilson M., McNaughton B. (1995) Tetrodes markedly improve the reliability and yield of multiple single-unit isolation from multi-unit recordings in cat striate cortex. J. Neurosci. Methods 63:43-54. Gray C. M. (1999) The temporal correlation hypothesis of visual feature integration: still alive and well. Neuron 24: 31–47. Gray C. M., P. König, A. K. Engel, and W. Singer (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties, Nature, vol. 338:334–337. Girard P., J. M. Hupé, and J. Bullier (2001) Feedforward and feedback connections between areas V1 and V2 of the monkey have similar rapid conduction velocities. J. Neurophysiol. 85: 1328–1331. Häusser M., Roth A. (1997) Estimating the time course of the excitatory synaptic conductance in neocortical pyramidal cells using a novel voltage jump method. J. Neurosci. 17:7606-7625. Hebb D.O. (1949) The first stage of perception: Growth of the assembly. Chpt 4, and: Perception of a complex:the phase sequence. Chpt 5. In: The Organization of Behavior. Wiley, New York, pp 60-106. Jefferys J. G. R., R. D. Traub, and M. A. Whittington (1996) Neuronal networks for induced ‘40 Hz’ rhythms. Trends Neuosci. 19: 202–208. Johnson J. L. (1993) Waves in pulse-coupled neural networks. In Proc. of the World Congress on Neural Networks, vol. 4, pp. 299–302. Johnson J. L. (1994) Pulse-coupled neural networks. In Proc. of the SPIE Critical Rev., 55: 47–76. Kempter R., W. Gerstner, and J. L. van Hemmen (1999) Hebbian learning and spiking neurons. Phys. Rev. E, 59: 4498–4514. König P., and T. B. Schillen (1991) Stimulus-dependent assembly formation of oscillatory responses: I. Synchronization. Neural Computation 3: 155–166. König P., Engel A.K., Singer W. (1996) Integrator or coincidence detector? The role of the cortical neuron revisited. Trends Neurosci. 19:130-137.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Phase Coupling Supports Associative Visual Processing

179

Kreiter A. and W. Singer (1992) Oscillatory neuronal responses in the visual cortex of the awake macaque monkey. Eur. J. Neurosci.. 4:369–375. Legatt A.D., Arezzo J., Vaughan H.G. (1980). J. Neurosci. Methods 2:203-217. Li Z. (1998) A neural model of contour integration in the primary visual cortex. Neural Computation 10: 903–940. Lumer E.D., Edelman G.M., Tononi G. (1997) Neural dynamics in a model of the thalamocortical system. II. The role of neural synchrony tested through perturbations of spike timing. Cerebral Cortex 7:228-236. McGuire B. A., C. D. Gilbert, P. K. Rivlin, and T. N. Wiesel (1991) Targets of horizontal connections in macaque primary visual cortex. J. Comp. Neurol. 305: 370–392. Mitzdorf U. (1987) Properties of the evoked potential generators: current source-density analysis of visually evoked potentials in the cat cortex. Intern. J. Neurosci. 33:33-59. Nelson, M. (1994) A mechanism for neuronal gain control by descending pathways. Neural Computation 6:242-254. Nowak L. G. and J. Bullier (1997) The timing of information transfer in the visual system. In Extrastriate Cortex in Primates. Cerebral Cortex, vol. 12, Rockland et al., Eds. New York: Plenum Press, pp. 205–240. Reitboeck H. J. (1983) A multi-electrode matrix for studies of temporal signal correlations within neural assemblies, In Synergetics of the Brain, E. Basar et al., Eds. Berlin: Springer, pp. 174–182. Ruthazer E. S., and M. P. Stryker (1996) The role of activity in the development of longrange horizontal connections in area 17 of the ferret. J. Neurosci.16: 7253–7269. Saam M.,and R. Eckhorn (2000) Lateral spike conduction velocity in visual cortex affects spatial range of synchronization and receptive field size without visual experience: a learning model with spiking neurons. Biol. Cybern. 83: L1–L9. Steriade M., F. Amzica, and D. Contreras (1996) Synchronization of fast (30-40 Hz) spontaneous cortical rhythms during brain activation. J. Neurosci.16: 392–417. Trachtenberg J. T., and M. P. Stryker (2001) Rapid anatomical plasticity of horizontal connections in the developing visual cortex. J. Neurosci. 21:3476–3482. van Vreeswijk C., L. F. Abbott, and G. B. Ermentrout (1994) When inhibition not excitation synchronizes neural firing. J. Comput. Neurosci. 1: 313–321. Volgushev M., Chistiakova M., Singer W. (1998) Modification of discharge patterns of neocortical neurons by induced oscillations of the membrane potential. J. Neurosci. 83:15-25. von der Malsburg C., Schneider W. (1986) A neural cocktail-party processor, Biol. Cybern. 54: 29–40. Wang D. L. (1995) Emergent synchrony in locally coupled neural oscillators,” IEEE Trans. Neural Networks 6: 941–948. Wennekers T., and G. Palm (2000) Cell assemblies, associative memory and temporal structure in brain signals. In Time and the Brain, Series: CABR: Conceptual Advances in Brain Research, vol. 3, R. Miller, Ed. Harwood Academic Publishers, pp. 251–273. Wertheimer M. (1923) Untersuchungen zur Lehre von der Gestalt: II. Psychologische Forschung . 4: 301-350. Woelbern T., R. Eckhorn, A. Frien, and R. Bauer (2002) Perceptual grouping correlates with short synchronization in monkey prestriate cortex. Neuroreport. 13: 1881–1886.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 181-203

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 15

PROCESSING OF POSITIVE VERSUS NEGATIVE EMOTIONAL WORDS IS INCORPORATED IN ANTERIOR VERSUS POSTERIOR BRAIN AREAS: AN ERP MICROSTATE LORETA STUDY Lorena R.R. Gianotti*, Pascal L. Faber, Roberto D. PascualMarqui, Kieko Kochi and Dietrich Lehmann The KEY Institute for Brain-Mind Research, University Hospital of Psychiatry, Zurich, Switzerland

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Abstract The spatio-temporal organization of the neural activity that underlies perception of emotional valence was studied analyzing 33-channel event-related potential maps (ERP maps) from 21 normals while reading a sequence of emotional positive, negative and neutral words; subjects were asked to repeat the last word if a question mark followed; they were not informed that the study concerned emotions. Brain electric activity to emotional positive and negative words was compared. Microstate segmentation of the 113 ERP maps (corresponding to the 448 ms of word presentation) identified 14 microstates, i.e. putative steps of information processing. Three microstates, #4 (90-122 ms), #7 (178-202 ms) and #9 (242-274 ms) showed global map topography differences between emotional positive and negative words. During these three microstates, the involved brain areas were identified using Low Resolution Electromagnetic Tomography (LORETA). The results showed that the extraction of valence during the three emotion-sensitive microstates was incorporated in different brain areas: positive as well as negative emotional words caused predominant left-hemispheric activation in #4 and #9, but predominant right-hemispheric activation in #7. The striking communality across the three microstates however was that in each of them, positive words compared to negative words clearly evoked significantly more anterior brain activity.

*

E-mail address: [email protected], Tel: +41-44-388-4939, FAX: +41-44-380-3043. Address of correspondence: Dr. Lorena R.R. Gianotti, The KEY Institute for Brain-Mind Research, University Hospital of Psychiatry, Lenggstrasse 31, CH-8032 Zurich, Switzerland

182

Lorena R.R. Gianotti, Pascal L. Faber, Roberto D. Pascual-Marqui et al.

1. Introduction Detecting emotionally salient cues in the environment is a fundamental skill that readies the organism for successful behavior [Damasio, 1999]. But, to be useful, detection of potentially salient cues of emotional valence (positive and/or negative) must be appropriate, automated, and rapid so that valence is assessed within useful time.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

1.1. Temporal Aspects of Emotional Processing The speed of valence processing for successful, real time interactions with the surround must be high, in fractions of seconds. Analysis of brain electric or magnetic measurements at present are the only approaches that offer the required time resolution and thereby make it possible to describe physiological correlates of very rapid processing. Indeed, in the last years, using brain electric and magnetic data analyzed with various techniques, many studies demonstrated that emotional stimuli are perceived and differentiated within the first 140 ms after stimulus onset [e.g., Esslen et al., 2004; Ortigue et al., 2004; Pizzagalli et al., 2002; Pourtois et al., 2004; Skrandies, 1998; Skrandies & Chiu, 2003]. As early as in 1979, valence effects have been reported in a reading study with event-related potential (ERP) waveshapes at about 140 ms after onset of word presentation [Begleiter et al., 1979], but this study was neglected until recently. ERP studies in all sensory modalities have shown repeatedly that within fractions of seconds, different processing steps follow each other and are putative “components” of the ERPs. In other words, processing of the incoming information is certainly not a homogenous activity over time, but successive sub-processes or steps are distinguishable. However, scalprecorded ERP waveshapes depend on the chosen reference (including recalculations to current source density) and therefore, conventional component latencies of ERP waveshapes are ambiguous. The present study utilized ERP microstate analysis to examine the temporal development of brain information processing. Microstate analysis assesses differences in spatial configuration of the multichannel ERP scalp fields instead of differences in ERP waveshapes at individual electrode positions: the ERP recordings can be viewed as a series of instantaneous maps in a millisecond-by-millisecond rendering of the brain electric field on the scalp. The potential landscapes of these instantaneous maps vary over time in a non-steady manner; they tend to remain quasi-stable for brief, fraction of second-periods, the (global) microstates, that are concatenated by relatively rapid changes of potential landscape [Lehmann & Skrandies, 1980]. Because different brain potential landscapes on the scalp must have been generated by differently active neuronal generator populations, and because it appears reasonable to assume that different active neuronal generator populations implement different functions, different microstates were conceptualized to reflect different functional steps in the stream of information processing [Lehmann, 1987; Lehmann et al., 1998]. Indeed, various studies showed that the topography, sequence and duration of ERP microstates were shown to reflect steps and types of information processing [e.g., Brandeis et al., 1995; Koenig et al., 1998; Michel et al., 2004]. Thus, the analysis approach permits to identify steps of brain information processing. Brain information processing is known to be massively parallel; hence, each microstate taking into account the entire brain electric field consists of very many

Processing of Positive versus Negative Emotional Words…

183

parallel local states. At the onset of a new microstate, the set of local states must have undergone some change: there might be at least one additional or omitted local state, or at least one of the local states may have become stronger or weaker. Also, there could well be some local states that continued across two or more microstates. The identified microstates can be tested for differences between stimulus conditions. Those microstates that differ in potential landscape between conditions then can be analyzed as to the intracerebral localization of the activity.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

1.2. Localization of Emotional Processing in the Brain The issue of localizing emotional processing in the brain has attracted attention for many years. Frequently discussed are two global models about the spatial organization of brain activity during emotional processing: the ‘valence hypothesis of hemispheric specialization’ and the ‘right hemispheric dominance model’. The first model posits that emotion processing is lateralized, with positive (or approach-related) emotions in the left, and negative (or withdrawal-related) emotions in the right hemisphere, often specifically in fronto-temporal regions. It is based on clinical data [e.g., Carota et al., 2001; Gainotti, 1972; Terzian & Ceccotto, 1959], EEG [e.g., Aftanas et al., 2001; Davidson et al., 1990; Graham & Cabeza, 2001], and functional imaging [Canli et al., 1998; Sutton et al., 1997]. However, many recent studies did not support this model [e.g. Cato et al., 2004; Fossati et al., 2003; Sander et al., 2003; Schupp et al., 2004]. The second model, right hemispheric dominance, posits that emotional processing is done in the right hemisphere [e.g., Borod et al., 1992; Spence et al., 1996]. Also in regard to this second model, several studies failed to support an exclusive right lateralization of emotion [e.g., Cato et al., 2004; Sander et al., 2003]. A quantitative metaanalysis of 65 neuroimaging studies on emotions [Wager et al., 2003] found no support for the right hemispheric dominance model, and limited support for the valence hypothesis model. A methodological caveat: Conventional localization in ERP work accepts the location of the waveshape of largest amplitude (or of the maximal or minimal value in a potential map) as source localizer, but the source is not necessarily perpendicular under the recording site of the largest potential, because electric sources have orientations. Recalculation of the scalpmeasured values to average reference or current source densities does not solve the problem. Intracerebral source modeling is needed. Contemporary studies of emotional processing project a very differentiated view of the brain areas involved in the various studied functions such as perception, experience, and expression of emotions, with an overwhelming amount of localization details, often in a bewildering variety of brain areas, and with a great richness of different experimental designs from viewing of emotional images to willful, self-implementation of emotions, and using several basically different techniques for physiological measurements (PET, fMRI, Infraredmapping, MEG, EEG, ERP) and within each of them, large and diversified arsenals of analysis methods. It appears that a global perspective is desirable to systematize principle rules of brain information processing of emotional valence. In the present study that used ERP field data as measurements and microstate analysis in order to identify the rapid steps of emotional processing, localization of the involved brain areas was done with Low Resolution Electromagnetic Tomography (LORETA, [Pascual-

184

Lorena R.R. Gianotti, Pascal L. Faber, Roberto D. Pascual-Marqui et al.

Marqui et al., 1999; Pascual-Marqui et al., 1994]). LORETA is an inverse solution method that estimates the putative neural sources of electrical brain activity recorded at the level of the scalp. LORETA was applied to those ERP microstates that differed in scalp-recorded field topography between positive and negative words.

1.3. The Present Study In sum, the aim of the present study was to examine the temporal and spatial organization of neuronal networks that process valence information during reading. ERP electric field data recorded during reading of emotional positive and negative words were analyzed using microstate analysis and LORETA functional tomography. We hypothesized that there are different steps of emotional processing that occur rapidly within fractions of seconds, and that assessment of emotional valence is incorporated by activity of neural assemblies with different spatial organization. Lateralized or non-lateralized valence effects were not additionally hypothesized, given the many inconsistencies in the published literature.

2. Materials and Methods 2.1. Subjects Subjects were 21 right-handed German or Swiss-German native speakers (mean age 23±3 years, 13 women), recruited among first-year psychology students at Zurich University, and remunerated with CHF 40. None of them had any history of psychiatric or neurological disorders, or alcohol or drug abuse. The study was approved by the Ethics Committee of the University Hospital Zurich, and subjects gave their written, informed consent for participation.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

2.2. Stimuli 27 emotional positive and 27 emotional negative words, together with 20 neutral words were used as stimuli, using a word list [Esslen, 1997] that was developed as follows: 116 preselected words were rated by 15 subjects for their emotional valence on a 1-7 scale (1=very negative; 7=very positive). The 27 most negative, the 27 most positive, and the 20 most neutral words constituted the three stimulus classes, making sure that there was no significant difference in word length (ranges were 3-6 letters and 1-3 syllables), frequency of occurrence in German texts (list F in [Rosengren, 1977]) and subjective rating of imagery propensity between the positive and the negative word classes. The semantic characteristics [Osgood, 1952] of ‘potency’ (weak-strong) and ‘activity’ (passive-active) were rated by 20 independent volunteers on a 1-7 scale. Valence of the utilized words was re-rated by the present subjects after the experiment. The list of the German stimulus words with their English translations is available upon request. The neutral word class was not used for the present analysis.

Processing of Positive versus Negative Emotional Words…

185

2.3. Procedure Subjects were seated in a comfortable chair in a sound, light, and electrically shielded EEG recording chamber. The experimenter in the adjacent recording room was in contact with the subject via intercom. During recording, the subject’s head was placed in a forehead-chin rest so that the distance between eyes and PC screen was constant (100 cm) and head movements were minimized. Stimuli were presented using the software ERTS (BeriSoft Cooperation, Frankfurt, Germany). Words extending in a 3.7 (±0.7) degree visual angle were sequentially presented for 450 ms each at the center of a PC screen at intervals of 2000 ms. During the interval, a fixation cross was displayed at the screen center. The 74 words were used repeatedly as stimuli, in 8 runs, for a total of 596 word presentations for each subject. In order to maintain some surprise in stimulus appearance while limiting the persistence of a given emotion, we chose a pseudo-random sequence of presentation where no more than 2 successive stimuli of the same class followed each other. For each subject and for each run, different pseudo-random sequences of the 74 words were used. Seventeen additional stimuli, inserted at random, were question marks. Words, fixation points and question marks were displayed in white on a dark grey background. Between runs, there was a 1 minute intermission. The entire recording lasted about 27 minutes.

2.4. Subject Instruction The subjects were instructed to fixate the cross at the center of the screen, and to read the words silently but attentively. When the question mark appeared, the subject had to repeat loudly the last word that was presented before the question mark (recall task). After the EEG recording, the subjects were asked to judge the emotional valence of all 74 stimulus words on a 1-7 scale from ‘very negative’ (‘1’) to ‘very positive’ (‘7’) as above.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

2.5. ERP Recording and Computation GRASS cup electrodes were placed according to the 10/10 international system [Nuwer et al., 1998] at the 33 positions Fp1/2, Fpz, AF3/4, F7/8, F3/4, Fz, FC5/6, FC1/2, T7/8, C3/4, Cz, CP5/6, CP1/2, P7/8, P3/4, Pz, PO3/4, O1/2 and Oz, with Cz as recording reference. Horizontal and vertical eye movements were recorded with electrodes at the outer left and right canthi and a left infraorbital electrode. Impedances were kept below 10 kOhm. The signals were amplified (bandpass 0.5-70 Hz) and digitized (250 samples/s) using a 64-channel EEG/ERP system (hardware: M & I Ltd., Prague, Czech Republic; software: Easys221, Neuroscience Technology Research Ltd., Prague, Czech Republic). Off line, all data epochs, starting at the onset of word presentation up to 113 timeframes later (=448 ms), thus covering the entire word presentation from onset to offset, were carefully examined on a computer display for artifacts (muscle, eye and head movement) before averaging. The mean number of artifact-free data epochs across subjects that were eventually available did not differ significantly between word classes (positive: mean=130.4, S.D.=43.1, negative: mean=129.1, S.D.=40.4).

186

Lorena R.R. Gianotti, Pascal L. Faber, Roberto D. Pascual-Marqui et al.

left

right ant.

post. 4μV 0 200 400 ms

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 1. 33-channel grandmean ERP waveshapes (across 21 subjects) for positive (thin traces) and negative (heavy traces) emotion words; the zero microVolt levels are offset for ease of visualisation of the corresponding waveshapes. Head seen from above, nose up, left ear left; waveshapes are localized in a semi-schematic electrode array where the electrode locations were a subset of the 10/10 locations (see methods).

For each subject, all available data epochs were averaged separately for the two word classes, and the resulting, average ERPs were digitally band passed (2-20 Hz) and recomputed against average reference. Grandmean 33-channel ERP waveshapes across all 21 subjects separately for the two word classes were computed and are shown in Figure 1 as overview of the data. In order to normalize the data, Global Field Power (GFP, [Lehmann & Skrandies, 1980]) for each given timeframe (map) was set to 1 by dividing the voltages at all electrodes by the GFP value of that map. GFP yields a single time series for all channels; as used in this study, GFP is equivalent to the spatial standard deviation of the instantaneous voltages at all electrodes of a given map. Grand-grandmean 33-channel ERP waveshapes were computed across the two word classes and the 21 subjects, and were transformed into a series of momentary ERP maps that display the potential distribution landscapes of the 113 timeframes (Figure 2).

Processing of Positive versus Negative Emotional Words…

187

0

8

16

24

32

40

48

56

64

72

80

88

96

104

112

120

128

136

144

152

160

168

176

184

192

200

208

216

224

232

240

248

256

264

272

280

288

296

304

312

320

328

336

344

352

360

368

376 R

L

A

P

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

384

392

400

408

416

424

432

440ms

Figure 2. The 33-channel ERP grand-grandmean map series (56 maps at 8 ms intervals) across the 2 word classes and 21 subjects. Head seen from above, nose up, left ear left; L/R = left/right, A/P = anterior/posterior. Isopotential level in arbitrary units. White=positive, black=negative potential versus average reference. Latencies in ms after stimulus onset.

2.6. Analysis 2.6.1. Microstate Analysis The grand-grandmean ERP map series was parsed into temporal microstates defined as brief sequences of successive ERP maps with quasi-stable potential landscape [Lehmann & Skrandies, 1980] using the global clustering approach [Pascual-Marqui et al., 1995]. The settings for this microstate analysis were: 20 random initializations with maximal 50 iterations, permitting between 2 and 12 clusters disregarding polarity. Based on the Global

188

Lorena R.R. Gianotti, Pascal L. Faber, Roberto D. Pascual-Marqui et al.

Map Dissimilarity (GMD, [Lehmann & Skrandies, 1980]), which is a global measure of ‘landscape distance’ between two maps, each ERP map was assigned to one of the obtained clusters. A microstate is defined as a sequence of successive ERP maps that are assigned to the same cluster. Every time an ERP map is assigned to a new cluster (or to the same cluster but with reversed polarity), a new microstate starts. Note that each of these clusters could occur several times with the same or opposite polarity during the analysis period. The microstate analysis was rerun 10 times. Because of the limited, even though randomly selected number of initializations and because of the limited iterations, in repeated runs of the analysis program, a certain variance of the timeframes that start new microstates ('start frames') is to be expected. In order to identify the most consistent start frames across the repeated analysis runs, their occurrence probability was tested. For each run, each of the 113 frames was assigned the value 1 if it was a start frame, and the value zero if not. In order to give equal weight to all runs regardless of the number of start frames in a given run, these assigned values of 1 or zero were normalized for each run using formula 1:

V wVi = 113 i ∑Vi i =1

(formula 1)

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

where wVi is the normalized value at the timeframe i, Vi is the original value at the timeframe i. Thus, 10 series were obtained, each consisting of 113 wVi –values (one for each timeframe). For each timeframe, the significance across the 10 runs of the deviation of the normalized values from zero was tested using t-statistics (n=10). Timeframes yielding Pvalues 0.5). For comparison, the same analyzes with shuffled data without long-term correlations beyond a lag equal of 0.1 sec (C) show slopes of β=2 and H=0.5, characteristic of a Brownian motion.

Figure 2. Long-range correlations in brain fluctuations.

To assess whether intracranial EEG exhibit self-similarity, we used here two methods, the power spectrum and rescaled range analysis (R/S) [30-31]. As an example, Fig. 2A shows this scaling behavior for a sample of 6 hours recorded in post-central opercularis (patient 1). The scaling properties are estimated from 39 non-overlapping intervals of 10 minutes (i.e. 200000 data points each). In accordance with previous studies on intracranial EEG [7], the power

Is Brain Activity Fractal?

271 β

spectrum of the cumulative energy reveals a clear power law scaling P( f ) ∝ 1 f with β>2 showing long-range correlation on time scale of minutes. The spectral method used alone can be susceptible to producing false results [30]. Another robust technique for quantifying this behavior is the use of R/S analysis to evaluate the Hurst exponent, H. For a time series of length n, X={Xt, t=1,…,n} with mean X ( n ) and variance S ( n ) , the R/S ratio is defined as: 2

R( n ) max( 0, W1 , W2 ,..., Wn ) − min( 0, W1 , W2 ,..., Wn ) = S ( n) S 2 ( n) where Wk = X1 + X 2 +...+ X k − k X ( n ) . For a sequence with long-range correlations, the

R( n ) →∞ ⎯n⎯ ⎯→ Cn H S ( n) with the Hurst parameter H>0.5 [30-31]. Applied to

ratio scales as the energy fluctuations of intracranial EEG, the R/S analysis confirms a Hurst exponent H>0.5, implying long range persistence (Fig. 2B). To test the robustness of these results, we compare them with surrogate data representing a realization of a shuffling process (Fig. 2C). In this procedure, the time series is divided into blocks and the blocks are shuffled [32]. Thus, the shuffling destroys long-term correlations from the series beyond a lag equal to the length of a block. We used block size of 0.1 sec for the shuffling. For comparison, the same analyzes with shuffled data without long-term correlations show slopes of β=2 and H=0.5, characteristic of a Brownian motion. Similar observations were made for all the 10 investigated patients.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Multifractal Analysis We next investigated higher order moments of spontaneous brain fluctuations through the structure function analysis [33]. Structure functions are essentially generalized correlation functions, and have been widely used in the study of fluid turbulence [34], with several inherent advantages, including greater accuracy for smaller data sets and the absence of transition regions between different scaling ranges. For a process Y(t), the structure function of order q is defined as:

S q (τ ) = Y ( t + τ ) − Y ( t )

q

. If the process is self-similar over

some range of time lags τ, then the function is expected to scale as: S (τ ) ∝ τ q

ξ(q)

[35]. The

process is monofractal if ξ ( q ) is a linear function of q and multifractal if ξ ( q ) is nonlinear. Therefore, the multifractal behavior can only be fully characterized by the whole spectrum of its exponents at various power q. For a monofractal process like the fractional Brownian motion [29], the q-th order scaling exponents follows exponent. For a Brownian motion, we have ξ ( q ) = q / 2 .

ξ ( q ) = qH where H is the Hurst

272

Michel Le Van Quyen, Mario Chavez, Claude Adam et al.

The stationarity of the time series is first checked by the mean slope of time lags over 0.1 sec,

S Xq (τ )

log S Xq (τ ) vs log(τ) (A).For

is approximately constant with zero slope for all q, showing stationarity.

Within this stationary range, the q-th order scaling exponents ξ ( q ) are estimated by the slope of

log SYq (τ ) vs log(τ) from 0.5 sec to 20 sec (B). The plot of ξ ( q )

vs q shows the scaling exponents

structure function (C, circles) and confirms the presence of long-range dependences (H= ξ (1) =0.72). Nevertheless for higher exponents, the curve is clearly nonlinear, compared to straight line of a fractional Brownian motion with same H (dashed line). The nonlinearity is further confirmed by the universal

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

multifractal function (thick continuous curve) fitting the data with H=0.72, C1=0.052, α=1.76. Furthermore, the scaling exponents of the shuffled data without long-term correlations are clearly different that the original data, showing a slope close to H=0.5 and characteristic of a Brownian motion (C, doted line).

Figure 3. Example of structure function analysis.

Following Ref. 36, we first evaluated the stationarity of the increments of Y(t), here the signal energy X(t) (see section III). Since the statistics of a stationary process are invariant d

under time expansion (i.e. stationary requires X (t ) = X ( at ) ), stationary data should exhibit no scaling, that is ξ ( q ) = 0 for all q. We estimated on which time scales the original series is stationary by checking where the mean slope of log S X (τ ) vs log(τ) is equal to zero for all q. Once the stationary data range has been determined (if it exists), the structure functions of cumulative time series are calculated [36]. In order to evaluate statistical bias, the mean of the q

Is Brain Activity Fractal?

273

structure function are estimated from 25-70 non-overlapping time records of 10 minutes. Fig. 3A presents a representative example of S X (τ ) vs τ for q =0.5, 1,..,5 and the time delay τ varying from 0.01 sec up to approximately 6 minutes. Note that the short time-scale regimes under 0.1 sec are largely determined by nonstationary fluctuations. For time lags over 0.1 sec, q

S Xq (τ ) have a zero slope. Within this stationary range, we calculated the q-th order scaling q exponents ξ ( q ) by the slope of log S Y (τ ) vs log(τ) from 0.5 sec to 20 sec (Fig.3B). In the

ξ ( q ) vs q confirms the presence of long-range dependences (H= ξ (1) =0.72). Furthermore, high order moments of the scaling exponents ξ ( q ) exhibit

example of Fig. 3C, the plot of

clear deviations from the linear development expected under mono-fractal behavior. This demonstrates that the generating process is more complicated than a fractional Brownian motion and is better characterized by a multifractal behavior. When the data are shuffled into blocks of length 0.1 sec, Fig. 3C shows that the scaling exponent of the raw data is clearly different beyond correlation horizon of the surrogate series. The scaling exponents of the

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

shuffled data exhibit monofractal behavior close to a Brownian motion).

The plot of

ξ (1)

ξ ( q ) = qH with H=0.5 (characteristic of

vs q shows the scaling exponents structure function and confirms the presence of long-

range dependences (H= ξ (1) = 0.73±0.05). Furthermore, the curves are clearly nonlinear, compared to straight line of a fractional Brownian motion with same H (dashed line). In order to extract generic properties, we fitted the data to an universal form for the structure function:

ξ ( q ) = qH −

C1 ( qα − q ) α −1

where H is the Hurst exponent, C1 is an intermittency parameter and α is

the Lévy index (thick continuous curve).

Figure 4. Structure functions of 10 subjects recorded in different brain structures.

274

Michel Le Van Quyen, Mario Chavez, Claude Adam et al. Finally, we confirmed the departure of the structure function from linearity in 10 subjects

(Fig. 4). In order to extract the deviations of

ξ ( q ) from a linear dependence, we use the

following universal form for the structure function [27, 37]: ξ ( q ) = qH −

C1 ( qα − q ) α −1

where H is the Hurst exponent, C1 is an intermittency parameter and α is the Lévy index. The parameter C1 distinguishes between a signal whose mean is dominated by a few localized intense peaks (large C1) and one where heterogeneity is very low (small C1, for non-fractal such as white noise C1 =0). The parameter α indicates the degree of multifractality and varies between 0 (monofractal process) and 2 (log-normal distribution). The agreement between the EEG fluctuations and the theoretical multifractal scheme is good (Fig. 3C). Suprisingly, although a considerable degree of inter-patient variability is to be expected from the different electrode implantations, the performed analysis shows that the different sets have parameters that behave quite closely. Using a nonlinear least square fitting technique, we estimate the following group-averaged values (Fig. 4): H=0.73±0.05, C1=0.09±0.05, α=1.73±0.29. All data show a low intermittency (small values of C1), corresponding to a relative homogeneity. On the other hand, the value of the α parameter near 2 shows a typical log-normal behavior [38]. Replicability of the results across subjects leads us to believe that a broader class of multi-fractal processes is needed to model EEG fluctuations. An exhaustive presentation of the results is presented elsewhere [39]

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Conclusion By means of the structure function, we show that the description of brain dynamics needs laws more complicated than simple 1/f scaling of monofractal models. In other words, second-order statistics like the power spectrum or the autocorrelation cannot quantify the full heterogeneity of the signal, only reflected by higher order moments. In this paper, we propose that the fluctuation of the data can be well fitted by a generic multifractal scheme [37], modeling the departure from linearity of the scaling exponents with two parameters C1 and α. Since these properties are shared by all the recordings, we propose that this description makes explicit generic structures of brain fluctuations, i.e. characteristics that are observed independently of the particular variation of the network under consideration. The results presented here have several implications for our understanding of brain dynamics. In particular, the observed multifractal statistics are reminiscent of the multifractal distributions found in turbulent flows [23, 34, 40]. As observed in numerical and experimental studies of hydrodynamic turbulence, there is strong evidence of deviations of

ξ ( q ) from a linear dependence, suggesting an irregular redistribution of the energy. This multifractality is an indication for a multiplicative cascade in where small eddies result from the breakdown of large ones [34, 40]. A multiplicative cascade is a process that fragments a given set into smaller and smaller sections according to a geometric rule and, at the same time, divides the measure of the set according to another (possibly random) rule [38]. Based on our findings, brain fluctuations might also carry an inherent multiplicative structures. A plausible physical explanation may be here that the brain fluctuations result of the nonlinear interaction of many physiological components operating on different time scales. Indeed the

Is Brain Activity Fractal?

275

brain is organized hierarchically from small neuronal populations to large scale networks. Each spatial level has its own intrinsic time scales [17]. In particular, a ‘macroscopic’ level has usually a slower time scale than the ‘microscopic’ levels below it. Furthermore, a macroscopic state defines an order parameter that strongly acts back onto microscopic states by top-down nonlinear loops [41]. Due to this hierarchically structured organization, macroscopic levels have a stronger influence on microscopic levels than conversely. This information flow from large to small spatial scale may generate a cascade from long-term to short-term time scale, leading to a possible multiplicative character of the EEG signals under consideration. Further investigations of the spatial characteristics of brain synchronization and studies at a cellular level [18] are necessary to validate these hypotheses. Applied to intracranial EEGs of epileptic patients during normal states, the multifractal exponents were in a narrow range, suggesting that a deviation from this range might be an indicator of dysfunction in large-scale brain synchronization implicated in epilepsy [7]. Recently, fractal scaling analysis has been applied to the human epileptogenic hippocampus but, in contrast to previous studies using nonlinear time series analysis [42-43 for reviews], energy fluctuation did not change as seizures approaches [7]. Motivated by our first results, we suggest that a multifractal approach may provide a refinement of the fractal measures, perhaps indicative of additional information on seizure anticipation. Our group is actively investigating this issue and it will be a focus for an upcoming study.

Acknowledgements Special thanks to Jean-Claude Bourseix for his technical support.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

References [1] Freeman, WJ. Sructure, and role of background EEG activity. Part 1. Analytic amplitude and Part 2. Analytic phase. Clin Neurophysiol., 2004 115, 2077- 2107. [2] Pereda, E; Gamundi, A; Rial, R; Gonzalez, J. Non-linear behavior of human EEG: fractal exponent versus correlation dimension in awake and sleep stages. Neurosci. Lett., 1998 250, 91–94. [3] Hwa, RC; Ferree, TC. Scaling properties of fluctuations in the human electroencephalogram. Phys. Rev. E, 2002 66, 021901-021905. [4] Linkenkaer, K; Nikouline, VV; Palva, JM; Ilmoniemi, J. Long-range temporal correlations and scaling behavior in human brain oscillations. J. Neurosci., 2001 21, 1370-1377. [5] Nikulin, VV; Brismar, T. Long-range temporal correlations in alpha and beta oscillations: effect of arousal level and test-retest reliability. Clin Neurophysiol., 2004 115, 1896–1908. [6] Nikulin, VV; Brismar, T. Long-range temporal correlations in electroencephalographic oscillations: Relation to topography, frequency band, age and gender. Neuroscience, 2005 130, 549-558.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

276

Michel Le Van Quyen, Mario Chavez, Claude Adam et al.

[7] Parish, LM; Worrell, GA; Cranstoun, SD; Stead, SM; Pennell, P; Litt B. Long-range temporal correlations in epileptogenic and non-epileptogenic human hippocampus. Neuroscience, 2004 125, 1069–1076 [8] Watters, PA; Martin, F. A method for estimating long-range power law correlations from the electroencephalogram. Biological Psychology, 2004 66, 79-89 [9] Mandelbrot, BB. The fractal geometry of nature. New York: Freeman; 1982. [10] Hurst, HE. Long-term storage capacity of reservoirs. Trans. Am. Soc. Civil Engineers, 1951 116, 770-799. [11] [11] Bunde, A; Kropp, J; Schellnhuber, HJ. The Science of Disasters: Climate Disruptions, Heart Attacks, and Market Crashes. Berlin: Springer; 2002. [12] Ashkenazy, Y; Ivanov, PC; Havlin, S; Peng, CK; Goldberger, AL; Stanley, HE. Magnitude and sign correlations in heartbeat fluctuations. Phys Rev Lett, 2001 86, 19001903. [13] Peng, CK; Buldyrev, SV; Goldberger, AL; Havlin, S; Sciortino, F; Simons, M; Stanley, HE. Long-range correlations in nucleotide sequences. Nature, 1992 356, 168–170. [14] Gabaix, X; Gopikrishnan, P; Plerou, V; Stanley, HE. A theory of power-law distributions in financial market fluctuations. Nature, 2003 423, 267-270 [15] Willinger, W; Govindan, R; Jamin, S; Paxson, V; Shenker, S. Scaling phenomena in the Internet: critically examining criticality. Proc Natl Acad Sci U S A, 2002 99, 2573-2580. [16] Teich, MC. Fractal character of the auditory neural spike train. IEEE Trans Biomed Eng, 1989 36, 150-160. [17] Segev, R; Benveniste, M; Hulata, E; Cohen, N; Palevski, A; Kapon, E; Shapira, Y; BenJacob, E. Long term behavior of lithographically prepared in vitro neuronal networks. Phys Rev Lett, 2002 88, 118102-118105. [18] Bhattacharya, J; Edwards, J; Mamelak, A; Schuman, E. Long-range temporal correlations in the spontaneous spiking of neurons in the hippoacampal-amygdala complex of humans. Neuroscience, 2005 131, 547-555 [19] Linkenkaer-Hansen, KM; Nikulin, VV; Palva, JM; Kaila, K; Ilmoniemi, RJ. Stimulus induced change in long-range temporal correlations and scaling behaviour of sensorimotor oscillations. Eur J Neurosci, 2004 19, 203–211. [20] Beggs, JM; Plenz, D. Neuronal avalanches in neocortical circuits. J Neurosci, 2003 23, 11167–11177. [21] Riedi, RH. Multifractal processes. In: Doukhan P, Oppenheim G,Taqqu MS, editor. Long-range dependence : theory and applications. Birkhäuser; 2002; 625–716. [22] Stanley, HE; Amaral, LA; Goldberger, AL; Havlin, S; Ivanov, PCh; Peng, CK. Statistical physics and physiology: monofractal and multifractal approaches. Physica A, 1999 270, 309-324. [23] Parisi, G.; Frisch, U. A multifractal model of intermittency. In: Ghil M, Benzi R, Parisi G, editor. Turbulence and predictability in geophysical fluid dynamics and climate dynamics. Amsterdam: North-Holland; 1985; 84-88. [24] Schmitt, F; Schertzer, D; Lovejoy, S. Multifractal analysis of foreign exchange data. Appl. Stochastic Models Data Anal., 1999 15, 29–53 [25] Feldmann, A; Gilbert, AC; Willinger, W. Data networks as cascades: investigating the multifractal nature of internet WAN traffic. Proc. ACM/Sigcomm, 1998 28, 42-55. [26] Arneodo, A; Bacry, E; Graves, PV; Muzy, JF. Characterizing long-range correlations in DNA sequences from wavelet analysis. Phys Rev Lett, 1995 74, 3293-3296.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Is Brain Activity Fractal?

277

[27] Ivanov, PC; Rosenblum, MG; Peng, CK; Mietus, J; Havlin, S; Stanley, HE; Golberger, AL. Scaling behavior of heartbeat intervals obtained by wavelet-based time-series analysis. Nature, 1996 383, 323-327. [28] Taqqu, MS; Teverovsky, V. On estimating the intensity of long-range dependence in finite and infinite variance time series. In: Adler RJ, Feldman RE, Taqqu MS, editor. A Practical Guide to Heavy Tails: Statistical Techniques and Applications. Boston: Birkhauser; 1998; 177-217. [29] Mandelbrot, BB; Ness, JWV. Fractional Brownian motion, fractional noises and applications. SIAM Reviews, 1968 10, 422-437. [30] Rangarajan, G; Ding, M.) Integrated approach to the assesment of long range correlation in time series data. Phys Rev E, 2000 61, 4991-5001. [31] Gilmore, M; Yu, CX; Rhodes, TL; Peebles, WA. Investigation of rescaled range analysis, the Hurst exponent, and long-time correlations in plasma turbulence. Physics of Plasmas, 2002 9, 1312-1317. [32] van Milligen, BP; Hidalgo, C; Carreras, BA. Comment “on the Hurst exponent and long-time correlation”. Phys Plasmas, 2000 7, 1181-1182. [33] Monin, AS; Yaglom, AM. Statistical Fluid Mechanics: Mechanics of turbulence. London: MIT Press, 1975. [34] Frisch, U. Turbulence, the legacy of AN Kolmogorov. Cambridge: Cambridge University Press; 1995. [35] Barabasi, AL; Vicsek, T. Multifractality of self-affine fractals. Phys Rev A, 1991 44, 2730-2733. [36] Yu, CX; Gilmore, M; Peebles, WA; Rhodes, TL. Structure function analysis of longrange correlations in plasma turbulence. Phy. of Plasmas, 2003 10, 2772-2779. [37] Schertzer, D; Lovejoy, S. Physical modeling and analysis of rain and clouds by anisotropic scaling multiplicative processes. J Geophys Rev, 1987 92, 9693-9721. [38] Muzy, JF; Delour, J; Bacry, E. Modeling fluctuations of financial time series: from cascade process to stochastic volatility model. Euro. Phys. Journal B, 2000 17, 537-548. [39] Le Van Quyen, M; Chavez, M; Adam, C; Martinerie, J. Long-range correlations in brain activity. Submitted 2006. [40] Kolmogorov, AN. A refinement of previous hypotheses concerning the local structure of turbulence in a viscous incompressible fluid at high Reynolds number. J Fluid Mech, 1962 13, 82-85. [41] Le Van Quyen, M. Disentangling the dynamic core: a research program for a neurodynamics at the large-scale. Biol Res., 2003 36, 67-88. [42] Le Van Quyen, M; Martinerie, J; Navarro, V; Baulac, M; Varela, F.. Characterizing the neuro-dynamical changes prior to seizures. J Clinical Neurophysiology, 2001 18, 191-208. [43] Le Van Quyen, M. Anticipating epileptic seizures: From mathematics to clinical applications. C. R. Biologies, 2005 328, 187–198

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 279-291

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 21

THE TIMING OF NEURAL PROCESSES IN HUMANS: BEYOND THE EVOKED POTENTIALS Nelly Mainy1, Julien Jung1, Giorgia Commiterri7, Alain Berthoz3, Monica Baciu6, Lorella Minotti2, Dominique Hoffmann5, Philippe Kahane2, Olivier Bertrand1 and Jean-Philippe Lachaux1,4* 1

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

INSERM, U280, Institut Fédératif des Neurosciences, Lyon, F-69000, France; Univ. Lyon 1, Lyon, F-69000, France 2 Grenoble Hospital, Department of Neurology, Grenoble, F-38000, France; INSERM, U318, Grenoble, F-38000, France 3 CNRS, Laboratoire de Physiologie de la Perception et de l'Action, Collège de France, Paris, France 4 CNRS, UPR640, LENA, Paris, France. 5 Grenoble Hospital, Department of Neurosurgery, Grenoble, F-38000, France; INSERM, U318, Grenoble, F-38000, France 6 CNRS, UMR 5105, Laboratoire de Psychologie et Neurocognition, Grenoble, France; Université Pierre Mendès-France, Grenoble, France 7 Laboratory of Functional Neuroimaging, Fondazione Santa Lucia, and Department of Psychology, University of Rome La Sapienza, Rome, Italy

Abstract Our understanding of the neural bases of mental processes in humans depend on our capacity to visualize the activity of focal neural populations with a great temporal precision. This spatio-temporal precision can only be accessed in certain patients implanted with intracerebral electrodes for therapeutical purposes. Many groups have used this exceptional opportunity to infer the time course of certain neural processes, almost exclusively through the study of intracerebral evoked potentials. One problem is that the evoked potentials reflect only the neural activities that are either of low frequency, or precisely phase-locked to sensory events. This technical limitation has moved the scope away from other, important, *

E-mail address: [email protected], Tel : +33- (0)4 72 13 89 13, Fax : +33- (0)4 72 13 89 01.(Corresponding author)

280

Nelly Mainy, Julien Jung, Giorgia Commiterri et al. components of the neural activity, typically in high-frequency ranges, such as the gamma band (> 30 Hz). In this paper, we use intracerebral recordings from a patient performing several cognitive tasks, to illustrate the temporal and functional differences between the evoked potentials and the non-phase locked, but task-related high-frequency neural activities, and advocate the use of the latter, in addition to the former, for the understanding of the human brain dynamics.

Keywords: evoked potentials, intracerebral EEG, induced gamma band responses, brain dynamics

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Introduction Functional neuroimaging techniques, such as fMRI and PET, have been used with tremendous success to associate precise mental processes with networks of well-defined brain areas in humans (Raichle, 2003). The acknowledged challenge is now to understand the dynamics of such networks, and this requires other human brain imaging techniques with a fine spatial and temporal resolution (Dale and Halgren, 2001). The advents of EEG and MEG have provided a way to estimate the latency of certain brain processes; however, their spatial resolution is not sufficient to observe when and for how long a specific brain region is active (Baillet, et al., 2001,Darvas, et al., 2004). Without fully effective multi-modal non-invasive imaging techniques combining the spatial resolution of fMRI and the temporal resolution of EEG/MEG (Dale and Halgren, 2001), the only way to obtain a millimeter/millisecond resolution in humans is to record the intracerebral EEG of patients implanted for therapeutical reasons (Penfield and Jasper, 1949), see (Bechtereva and Abdullaev, 2000) and (Lachaux, et al., 2003) for recent reviews. Intracerebral EEG recordings are obtained via electrodes placed either directly inside the brain tissue (depth electrodes) or onto the cortical surface (grids). They are mostly used to identify with precision the cortical origin of seizure onsets in epileptic patients candidate for surgery. The advantage of those electrodes is the high spatial resolution of their recordings, which is on the order of one cubic centimetre, that is, 10 to 100 times better than scalp EEG recordings. Using such rare recordings, several groups have been able to fine-probe the temporal dynamics of certain neural phenomena, involved for instance in language (Fernandez, et al., 2001) or object perception (Allison, et al., 1999,Halgren, et al., 1994). However, the dominant approach has been the computation of event-related potentials, that is, the averaging of multiple EEG signals recorded in response to repeated series of events. An often underestimated limitation of this technique is that it is blind to most of the neural response components that are not phase-locked to the events used for averaging (Tallon-Baudry and Bertrand, 1999). Recently, new techniques have been employed in intra-cerebral EEG to reveal induced, but task-related neural activities, invisible in the evoked potentials (Aoki, et al., 2001,Crone, et al., 2001,Crone, et al., 1998,Fell, et al., 2003,Halgren, et al., 2002,Howard, et al., 2003,Klopp, et al., 1999,Lachaux, et al., 2000). In particular, a strong association has been established between high-frequency EEG components (typically above 30 Hz, in the socalled gamma band) and several cognitive processes, such as short-term verbal memory (Sederberg, et al., 2003), visual attention (Tallon-Baudry, et al., 2005), object perception (Lachaux, et al., 2005) and motor control (Aoki, et al., 2001,Crone, et al., 1998).

The Timing of Neural Processes in Humans: Beyond the Evoked Potentials

281

The aim of this paper is to draw emphasize the potential interest for the study of the brain dynamics of such a dynamical spectral imaging (i.e. intracerebral imaging in time and frequency), in addition to the classical evoked potentials. This is done through a couple of illustrative examples that compare in intracerebral EEG recordings the timing of evoked potentials and high-frequency energy modulations across several cognitive tasks in the same patient.

Materials and Methods Subject This patient (P1) suffered from drug-resistant partial epilepsy and was candidate for surgery. Because the location of the epileptic focus could not be identified using noninvasive methods, she underwent intracerebral recordings by means of stereotactically implanted multilead depth electrodes (SEEG). Selection of sites to implant were made entirely for clinical purposes with no reference to the present experimental protocol. The patient had previously given her informed consent to participate in the experiment. She had normal vision without corrective glasses.

Electrode Implantation 14 semi-rigid electrodes were implanted in cortical areas adapted to the suspected origin of seizures. Each electrode (Dixi, Besançon, France) had a diameter of 0.8 mm and comprised 10 or 15 leads of 2 mm length, 1.5 mm apart, depending on the target region. The electrode contacts were identified on the patient’s individual stereotactic scheme, and then anatomically localized using the proportional atlas of Talairach and Tournoux (Talairach and Tournoux, 1988).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Paradigm Patient P1 performed three different experimental tasks in succession. In the first task (STORIES), she was presented with a succession of words, some written in green and the others in red (colors appeared in a random order). Words of each color told a different story, and the patient had to attend to and remember one of the two stories, the green one or the red one. There were 200 words per story, one attended story per block and 6 consecutive blocks. The attended color changed with every new block. The patient had to tell the experimenter the attended story after each block. The words subtended 2.1 degrees horizontally and appeared for 100 ms every 700 ms on average. This task was taken from (Nobre, et al., 1998) and adapted to use slightly longer interstimulus intervals. In the second task (STRINGS), the patient was presented with strings of characters that either formed words (ws condition), pseudo-words (ps), consonnant strings (cs) or false font strings (ff). False fonts were Karalyn Patterson characters (Baciu, et al., 2001). Each string was presented for 2s, before a 1.5 s (in average) period during which she had to use a response button to indicate whether a) the words were names of living entities or not

282

Nelly Mainy, Julien Jung, Giorgia Commiterri et al.

(condition ws), b) the pseudo-words ended with vowel sounds (like 'gurdo') or consonant sounds (like 'ziple') (ps), c) the consonant strings contained twice the same letter or not (cs) and d) the false font strings contained twice the same character or not (ff). The patient responded to 13 blocks, each consisting of one series of each stimulus type, with 20 consecutive stimuli per series. In the last experiment (SCENES), the patient was presented with pictures of a virtual environment taken from different points of view (see (Committeri and Galati, 2004)), in which three objects were located in front of a three-winged palace in variable positions: two trashcans (a big green one and a small blue one) and a ball. The patient had to respond according to the position of the trashcans, judging which was closer a) to her point of view, b) to the long side of the palace,c) to the ball, or d) she had to indicate which trashcan was laying on the ground (our 4 experimental conditions). Each image was presented for 1s followed by a 1.5 seconds fixation cross. Before each block of six images, instructions about condition appeared on the screen for 1.5 seconds, followed by a 1.5 seconds fixation cross. The patient viewed over a hundred of images in each experimental condition.

Recordings and Stimulation

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The SEEG studies were performed according to our routine procedure four days after the electrodes implantation. Intracerebral recordings were conducted using an audio-video-EEG monitoring system (Micromed, Treviso, Italy), which allowed the simultaneous recording of 63 depth-EEG channels sampled at 512 Hz [0.1–200 Hz bandwidth] during the experimental paradigm (and 256 Hz during the electrical stimulation session). One of the contact sites in the white matter was chosen as reference. Intracerebral electrical stimulations were performed a few days later under video-SEEG control, solely for the clinical purpose of mapping eloquent brain areas so that they would be spared by the surgery. Following our standard clinical practice (Kahane, et al., 1993), stimulations at 1 Hz (amplitude 3mA, pulse width = 3 ms) were applied between contiguous contacts at various levels of the electrodes axis. Bipolar stimuli were delivered using a constant current rectangular pulse generator designed for a safe diagnostic stimulation of the human brain (Micromed, Treviso, Italy), according to parameters proven to produce no structural damage.

Time-Frequency and Evoked Potential Analysis For each single trial showing no sign of epileptiform activity, bipolar derivations computed between adjacent electrode contacts were analyzed a) using the standard evoked potential procedure and b) in the time-frequency (TF) domain by convolution with complex Gaussian Morlet’s wavelets (Tallon-Baudry, et al., 1997) (for an outlook of the many applications of Time-frequency analysis, see (Akay, 1997)). This convolution with a signal s(t) provided for 2

each trial a TF power map, P (t , f ) =| w(t , f ) * s (t ) | where w(t,f) was for each time t and frequency f a complex Morlet’s wavelet w(t , f ) = A exp( −t / 2σ t ).exp(2iπ ft ) , with 2

2

A = (σ t π ) −1/ 2 and σ t = 1/(2πσ f ) and σ f a function of the frequency f: σ f = f / 7 .

The Timing of Neural Processes in Humans: Beyond the Evoked Potentials

283

The investigated frequency range was [1-200 Hz] (resp. [1 – 100 Hz] for signals sampled at 512 Hz (resp. 256 Hz). Those TF maps were then averaged across all trials in a given experimental condition. Averaged time-frequency maps were then normalized for visualization purpose. This normalization was done separately for each frequency, and consisted in a) substracting the mean power during a [-500ms : -100 ms] prestimulus baseline, and b) dividing by the standard deviation of the power during this same baseline EEG signals were evaluated with the software package for electrophysiological analysis (ELAN-Pack) developed at the INSERM U280 laboratory.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Results Figure 1a shows the average potential evoked by visually presented words in the left postcentral gyrus of patient P1 in the STORIES experiment in the attention condition (Brodmann area 3, site s’8, Talairach coordinates : -62, -13, 23). To check for consistency, we separated the trials in two arbitrary groups of equal number to produce two evoked potentials (ERP), the two traces overlap, and display a series of positive and negative peaks. How shall we understand such a succession of evoked components in terms of cognitive processes? Decades of successful research have established undeniable relationships between certain well-defined ERP peaks and specific cognitive processes. However, it is not completely clear whether every single peak of the ERP corresponds to an active neural process relevant for the cognitive task at hand. We suggest the possibility that this may not be the case for all the evoked components (even if this may be true for the majority of them). One possibility is that some components may just correspond to the passive propagation of neural activity through the brain dense synaptic networks. Under this possibility, a massive neural input could propagate passively throughout the network without involving an active neural 'process'. This flow of neural activity could eventually "rebound" to revisit a particular brain location several times and generate there a succession of ERP peaks. Such massive inputs can be partially simulated by focal electrical stimulations delivered to epileptic patients for presurgical evaluation. Figure 1b shows the potential evoked in the same postcentral site (s’8) by brief (3ms) electric pulses delivered in the left middle frontal gyrus. Forty pulses were delivered at a pace of one every second. We repeated the analysis used for the STORIES experiment (as in figure 1a) to compute two electrically evoked potentials for two arbitrary groups of 20 pulses. The response to the electrical stimuli is very reproducible and displays a complex succession of five negative and positive peaks (note that the effect is restricted to the low frequencies and does not affect frequencies above 40 Hz, fig1c). Since such electric pulses often produce inconsistent clinical symptoms except in primary sensory and motor areas, there is a strong possibility that some electrically evoked components are not associated with mental processes (although we cannot fully demonstrate it). We cannot rule out that similar effects could happen in response to sensory stimulations, meaning that local activations could indeed propagate passively through the brain synaptic network and generate evoked peaks not associated with cognition. Put another way, since sensory stimulations, like electrical stimulations, do produce large energy inputs in the massively interconnected brain, it is reasonable to suggest that if one kind of energy input does propagate passively through the system the other should do the same. In any case, this complicates the interpretation of certain components of the ERPs in cognitive protocols.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

284

Nelly Mainy, Julien Jung, Giorgia Commiterri et al.

Figure 1. Potentials evoked by a visual and an electrical stimulation. a) potential evoked by the brief (100 ms) visual presentation of attended words in the left postcentral gyrus of patient P1 (s'2) referenced to a neighbour site, 3.5 mm away. The trials have been divided into two equal but arbitrary groups to produce two evoked potentials (black and gray curves); b) potential evoked in the same site by brief electric pulses delivered directly in the left middle frontal gyrus (two arbitrary groups of 20 pulses). Potentials are in microVolts. c) corresponding energy modulation generated in the same site in the [40 Hz - 100 Hz] frequency range by the electric stimulations; s is the standard deviation of the [500 ms : - 100 ms] prestimulus baseline. The massive truncated bump around 0 is the trace of the pulse itself (the wavelet transform of the electric artefact, which can be modelled as a Dirac) and not a response to the electric pulse, d) corresponding Time-frequency map, c) is thus the average of the TF values of this map in the [40 Hz - 100 Hz] frequency range.

The Timing of Neural Processes in Humans: Beyond the Evoked Potentials

285

Functional and Spatio-temporal Dissociation between Evoked Components and Induced Gamma Responses

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Other EEG components can also be used as timed electrophysiological markers of cognitive events. The evoked potentials constitute just one part of the neural response to sensory and cognitive events; they occur together with induced components in several frequency ranges, that cancel out in the averaging procedure used to compute ERPs because of inter-trial phasejitter. Figure 2 shows an example of induced spectral modulation in the gamma range, in response to visually presented words in the intraparietal sulcus of patient P1 (STRINGS experiment). An interesting property of such gamma modulations is that their temporal profiles are often much simpler than the succession of evoked potentials. They are often characterized by one component, with an initial increase, followed by a plateau and a gradual return to a baseline level. This simplicity cannot solely be attributed to the fact that the temporal imprecision of the wavelet transform has a smoothing effect on the gamma band temporal profiles : with the parameters used in this study, the temporal resolution of the wavelet transform above 50 Hz is better than 80 ms (4 cycles of a 50 Hz oscillation), therefore, if the gamma band response was a succession of multiple peaks of activity over several hundreds of ms, the wavelet analysis would still allow to distinguish them. Figure 3 shows representative examples of gamma energy temporal profiles (together with the corresponding ERPs), in patient P1, across three cognitive situations (STRING, SCENES and STORIES) and in three anatomical locations corresponding to the left fusiform gyrus (Talairach = [-35, -57, -8]), the left intraparietal sulcus [-31, -65, 38] and the left parahippocampal gyrus [-37, - 58, 5] (from top to bottom).

Figure 2. Time-frequency map showing the average energy modulation in the left intraparietal sulcus of patient P1 (Talairach = [-31, -65, 38]), in response to visually presented words in experiment STRING. The map has been normalized frequency by frequency relative to a [ - 500 ms: - 100 ms] baseline (see methods). The words were all shown for 2 seconds.

The gamma energy modulations and the ERPs are strikingly dissimilar in all three sites either in their temporal profile or in their variations across experiments. For instance, in the parahippocampal gyrus (Fig3, bottom graphs), the evoked potentials have returned to baseline level after 300ms while the gamma energy goes on for a full second. Also, in the fusiform gyrus and the intraparietal sulcus (Fig3, top and middle graphs), the largest ERPs are

286

Nelly Mainy, Julien Jung, Giorgia Commiterri et al.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

observed in the STORIES experiment, while the strongest gamma energies are in the STRING experiment. Therefore, the two measures certainly correspond to different neural

Figure 3. Comparative time courses of gamma energy modulations vs. evoked potentials. The three graphs on the left show the time course of the energy in the [ 50 to 150 Hz ] frequency range (that is, the average in this frequency range of TF maps computed as in figure 2), while the three graphs on the right show the corresponding evoked potentials, computed from the exact same signals. Each graph displays the data from one recording site of patient P1 in three different experiments (STRINGS (black), SCENES (dark gray) and STORIES (light gray)), plots with the same color correspond to the different conditions of a given experiment. Units are in V2 for the gamma band graphs (the vertical line indicates the scale, while the horizontal line indicates the offset of the graphs relative to 0. Top graphs (a and d) are from the fusiform gyrus (f'2, [-35, -57, -8]), middle graphs (b and e) are from the intraparietal sulcus ([-31, -65, 38]) and bottom graphs (c and f) are from the parahippocampal gyrus ([37, - 58, 5]). In display c) the (x) sign points towards the graph corresponding to condition c in the SCENES experiment, which is the one where the patient have to estimate the distance between the target object and the palace wing. As explained in the text, this particular condition yields a stronger gamma band response than the other three conditions of the SCENES experiment.

The Timing of Neural Processes in Humans: Beyond the Evoked Potentials

287

processes (note that the fact that the two measures ‘see’ different frequencies was not sufficient to conclude that they quantify different neural processes, as a single neural process may generate a complex multi-frequency electrical activity). In addition, an important difference between the two measures comes from the fact that while ERPs are signed measures (where polarity depends on the spatial configuration of its neural sources relative to the recording probe), energy values are always positive. Therefore, energy decreases relative to a baseline level can be understood in terms of local neural deactivations or desynchronizations (as apparent in the bottom graph of figure 3 for the STRINGS experiment). In the evoked potentials, there is no way to directly understand negative and positive deflections relative to the baseline in terms of activations/deactivations or synchronization/desynchronization of neural sources. For this reason, ERP studies are mostly effective when contrasting two signals observed at the same latency in two different experimental conditions.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Functional Specificity of Gamma Band Energy Modulations As apparent in figure 3, gamma energy modulations vary greatly in amplitude and timing across experiments (as compared with weaker intra-experiment variability). For instance, in the fusiform gyrus, the SCENES experiment was characterized by a plateau lasting almost until the end of the one second of image presentation, while the response was shorter for the two experiments involving the simple reading of a character string, even though in STRINGS, the character strings stayed on-screen for two full seconds. In the intraparietal sulcus, the temporal profiles were similar to those in the fusiform gyrus for the STRINGS and for the STORIES conditions; however, they were markedly different for the SCENES condition, where the gamma power increase was much reduced, and slower than in the fusiform. Finally, in the parahippocampal gyrus site, the gamma power time course resembled the one in the fusiform gyrus for the SCENES condition, although the initial sharp peak was lacking; with a subsequent plateau which lasted for the duration of the picture presentation. Interestingly, the gamma response was stronger in this region when the patient extracted global spatial information from the scene to estimate the distance of the trash cans to the palace wing (condition b in the experiment’s description). This goes well with the role of the parahippocampal gyrus in the perception of scene layouts (Epstein, et al., 1999). In comparison, the gamma band response to words was much reduced in that region. This is a further indication that the level of gamma activation is a useful index to relate anatomical locations with mental functions (confirmed by two studies of the tasks STRINGS and STORIES in more than 10 patients, Lachaux et al. 2005, unpublished observations).

Conclusion Evoked potentials have been so far the preferred index to estimate the timing of focal neural processes in humans. We have underlined in this paper that high-frequency energy modulations may also reveal important aspects of the timing of the working human brain. Like evoked potentials, high-frequency energy profiles are exquisitely sensitive to the cognitive task at hand, but the two classes of electrophysiological responses sharply differ in

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

288

Nelly Mainy, Julien Jung, Giorgia Commiterri et al.

their anatomo-functional specificity, as already apparent at the intracerebral level in (Lachaux, et al., 2005,Tallon-Baudry, et al., 2005). Evoked potentials and induced gamma energy modulations also differ markedly in their spatio-temporal organization: highfrequency EEG power modulations have generally simpler temporal profiles than ERPs, and consisting of a single, but sometimes prolonged, block of activation (although one should obviously be careful that the data in this study come from a single subject and that our claims need to be confirmed in a large population of patients). The simplicity in temporal profile of high-frequency EEG energy modulations makes them suitable for latency and duration comparison across experimental conditions. We could show for instance that in the fusiform gyrus, those high-frequency activations last longer during the examination of a complex visual scene than during the reading of an isolated word, irrespective of the stimulus duration. In this sense, the study of such sustained energy modulations (not only in the gamma band, but also in lower frequency ranges, see (TallonBaudry, et al., 2001)) may be an appropriate way to approach the neural basis of prolonged cognitive processes, such as feeling an emotion or viewing a static visual scene, in addition to the already known sustained low frequency components of the ERP (Birbaumer, et al., 1990). For instance, Rols et al. (Rols, et al., 2001) had already observed in the primary visual cortex of monkeys, that the sustained presentation of colored figures produced gamma activations of equal duration, after a transient evoked response. Those findings were parallel to those of another study, that showed a sustained power increase in the gamma band in the monkey’s primary visual cortex induced by sustained visual stimuli (Logothetis, et al., 2001). Interestingly, this last study could relate this gamma band activation with a sustained BOLD signal, simultaneously recorded at the same location. This raises the critical question of which of the two functionally distinct systems of electrophysiological responses, ERPs or Gamma modulations, is more related with the fMRI signals. Recent studies have reported a lack of correspondence between the ERPs and the BOLD signal (Brazdil, et al., 2005,Huettel, et al., 2004). One possible reason is that in certain instances some ERP components may be due to the phase-resetting of ongoing brain oscillations without amplitude modulation (see (Makeig, et al., 2002), and (Shah, et al., 2004), (Makinen, et al., 2004) for qualifications), in which case the ERP production is not expected to vary the metabolic demand . In contrast several animal studies combining electrophysiological recordings and BOLD measurements have shown a good correlation between BOLD and gamma band modulations (Kayser, et al., 2004,Logothetis, et al., 2001,Niessing, et al., 2005). The extension of those last findings to other brain regions and cognitive situations in humans, would promote the functional mapping of electrophysiological high-frequency modulations as a natural partner of fMRI for fine-grained spatio-temporal brain imaging.

Acknowledgements We thank Jacques Martinerie, Laurent Hugueville, Valérie Balle, Patricia Boschetti, Carole Chatelard, Véronique Dorlin, Eliane Gamblin and Martine Juillard for their invaluable help. JPL was supported by the Fondation Fyssen. NM was supported by the French Délégation Générale pour l’Armement.

The Timing of Neural Processes in Humans: Beyond the Evoked Potentials

289

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

References Akay, M, ed. (1997) Time Frequency and Wavelets in Biomedical Signal Processing, WileyIEEE Press Allison, T; Puce, A; Spencer, DD;McCarthy, G. (1999). Electrophysiological studies of human face perception. I: Potentials generated in occipitotemporal cortex by face and non-face stimuli. Cerebral.Cortex (9), 415-430 Aoki, F; Fetz, EE; Shupe, L; Lettich, E;Ojemann, GA. (2001). Changes in power and coherence of brain activity in human sensorimotor cortex during performance of visuomotor tasks. Biosystems, 63 (1-3), 89-99 Baciu, M; Kahane, P; Minotti, L; Charnallet, A; David, D; Le Bas, J;Segebarth, C. (2001). FMRI assessment of hemispheric dominance for language in epileptic patients by using a phonological task. Epileptic Disorders, 3 (3), 117-124 Baillet, S; Mosher, JC;Leahy, RM. (2001). Electromagnetic brain mapping. IEEE Signal Process. Mag. (18), 14-30 Bechtereva, NP;Abdullaev, YG. (2000). Depth electrodes in clinical neurophysiology: neuronal activity and human cognitive function,. Int. J. Psychophysiol., 37, 11-29 Birbaumer, N; Elbert, T; Canavan, AG;Rockstroh, B. (1990). Slow potentials of the cerebral cortex and behavior. Physiol Rev, 70 (1), 1-41 Brazdil, M; Dobsik, M; Mikl, M; Hlustik, P; Daniel, P; Pazourkova, M; Krupa, P;Rektor, I. (2005). Combined event-related fMRI and intracerebral ERP study of an auditory oddball task. Neuroimage, 26 (1), 285-293 Committeri, G;Galati, G. (2004). Reference frames for spatial cognition: different brain areas are involved in viewer-, object-, and landmark-centered judgements about object location. Journal of Cognitive Neuroscience, 16 (9), 1517-1535 Crone, NE; Boatman, D; Gordon, B;Hao, L. (2001). Induced electrocorticographic gamma activity during auditory perception. Brazier Award-winning article, 2001. Clin Neurophysiol, 112 (4), 565-582 Crone, NE; Miglioretti, DL; Gordon, B;Lesser, RP. (1998). Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band. Brain, 121 ( Pt 12), 2301-2315 Dale, A;Halgren, E. (2001). Spatiotemporal mapping of brain activity by integration of multiple imaging modalities. Current Opinion in Neurobiology (11), 202-208 Darvas, F; Pantazis, D; Kucukaltun-Yildirim, E;Leahy, RM. (2004). Mapping human brain function with MEG and EEG: methods and validation. Neuroimage, 23 Suppl 1,S289-299 Epstein, R; Harris, A; Stanley, D;Kanwisher, N. (1999). The parahippocampal place area: recognition, navigation, or encoding? Neuron, 23 (1), 115-125 Fell, J; Klaver, P; Elfadil, H; Schaller, C; Elger, CE;Fernandez, G. (2003). Rhinalhippocampal theta coherence during declarative memory formation: interaction with gamma synchronization? Eur J Neurosci, 17 (5), 1082-1088 Fernandez, G; Heitkemper, P; Grunwald, T; Van Roost, D; Urbach, H; Pezer, N; Lehnertz, K;Elger, CE. (2001). Inferior temporal stream for word processing with integrated mnemonic function. Human Brain Mapping (14), 251-260

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

290

Nelly Mainy, Julien Jung, Giorgia Commiterri et al.

Halgren, E; Baudena, P; Heit, G; Clarke, JM; Marinkovic, K;Clarke, M. (1994). Spatiotemporal stages in face and word processing. I. Depth-recorded potentials in the human occipital, temporal and parietal lobes. J Physiol Paris (88), 1-50 Halgren, E; Boujon, C; Clarke, J; Wang, C;Chauvel, P. (2002). Rapid distributed frontoparieto-occipital processing stages during working memory in humans. Cereb Cortex, 12 (7), 710-728 Howard, MW; Rizzuto, DS; Caplan, JB; Madsen, JR; Lisman, J; Aschenbrenner-Scheibe, R; Schulze-Bonhage, A;Kahana, MJ. (2003). Gamma oscillations correlate with working memory load in humans. Cereb Cortex, 13 (12), 1369-1374 Huettel, SA; McKeown, MJ; Song, AW; Hart, S; Spencer, DD; Allison, T;McCarthy, G. (2004). Linking hemodynamic and electrophysiological measures of brain activity: evidence from functional MRI and intracranial field potentials. Cereb Cortex, 14 (2), 165-173 Kahane, P; Tassi, L; Francione, S; Hoffmann, D; Lo Russo, G;Munari, C. (1993). Manifestations électro-cliniques induites par la stimulation électrique intra-cérébrale par "chocs" dans les épilepsies temporales. Neurophysiol Clin, 22, 305-326 Kayser, C; Kim, M; Ugurbil, K; Kim, DS;Konig, P. (2004). A comparison of hemodynamic and neural responses in cat visual cortex using complex stimuli. Cereb Cortex, 14 (8), 881-891 Klopp, J; Halgren, E; Marinkovic, K;Nenov, V. (1999). Face-selective spectral changes in the human fusiform gyrus. Clin Neurophysiol, 110 (4), 676-682 Lachaux, JP; George, N; Tallon-Baudry, C; Martinerie, J; Hugueville, L; Minotti, L; Kahane, P;Renault, B. (2005). The many faces of the gamma band response to complex visual stimuli. Neuroimage, 25 (2), 491-501 Lachaux, JP; Rodriguez, E; Martinerie, J; Adam, C; Hasboun, D;Varela, FJ. (2000). A quantitative study of gamma-band activity in human intracranial recordings triggered by visual stimuli. Eur J Neurosci, 12 (7), 2608-2622 Lachaux, JP; Rudrauf, D;Kahane, P. (2003). Intracranial EEG and human brain mapping. J Physiol Paris, 97 (4-6), 613-628 Logothetis, NK; Pauls, J; Augath, M; Trinath, T;Oeltermann, A. (2001). Neurophysiological investigation of the basis of the fMRI signal. Nature, 412 (6843), 150-157 Makeig, S; Westerfield, M; Jung, TP; Enghoff, S; Townsend, J; Courchesne, E;Sejnowski, TJ. (2002). Dynamic brain sources of visual evoked responses. Science, 295 (5555), 690-694 Makinen, VT; Tiitinen, H;May, PJ. (2004). Auditory evoked responses are additive to brain oscillations. Neurol Clin Neurophysiol, 2004, 45 Niessing, J; Ebisch, B; Schmidt, KE; Niessing, M; Singer, W;Galuske, RA. (2005). Hemodynamic signals correlate tightly with synchronized gamma oscillations. Science, 309 (5736), 948-951 Nobre, AC; Allison, T;McCarthy, G. (1998). Modulation of human extrastriate visual processing by selective attention to colours and words. Brain, 121 ( Pt 7), 1357-1368 Penfield;Jasper. (1949). Electrocorticograms in man: effect of voluntary movement upon the electrical activity of the precentral gyrus. Arch. Psychiatr. Neurol., 182, 163–174 Raichle, ME. (2003). Functional brain imaging and human brain function. J Neurosci, 23 (10), 3959-3962

The Timing of Neural Processes in Humans: Beyond the Evoked Potentials

291

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Rols, G; Tallon-Baudry, C; Girard, P; Bertrand, O;Bullier, J. (2001). Cortical mapping of gamma oscillations in areas V1 and V4 of the macaque monkey. Vis Neurosci, 18 (4), 527-540 Sederberg, PB; Kahana, MJ; Howard, MW; Donner, E;Madsen, JR. (2003). Theta and Gamma Oscillations During Encoding Predict Subsequent Recall. Journal of Neuroscience (23), 10809-10814 Shah, AS; Bressler, SL; Knuth, KH; Ding, M; Mehta, AD; Ulbert, I;Schroeder, CE. (2004). Neural dynamics and the fundamental mechanisms of event-related brain potentials. Cereb Cortex, 14 (5), 476-483 Talairach, J;Tournoux, P. (1988) Co-planar stereotaxic atlas of the human brain. 3dimensional proportional system: an approach to cerebral imaging, Thieme Tallon-Baudry, C;Bertrand, O. (1999). Oscillatory gamma activity in humans and its role in object representation. Trends Cogn Sci, 3 (4), 151-162 Tallon-Baudry, C; Bertrand, O; Delpuech, C;Permier, J. (1997). Oscillatory gamma-band (3070 Hz) activity induced by a visual search task in humans. J Neurosci, 17 (2), 722-734 Tallon-Baudry, C; Bertrand, O;Fischer, C. (2001). Oscillatory synchrony between human extrastriate areas during visual short-term memory maintenance. J Neurosci, 21 (20), RC177 Tallon-Baudry, C; Bertrand, O; Henaff, MA; Isnard, J;Fischer, C. (2005). Attention Modulates Gamma-band Oscillations Differently in the Human Lateral Occipital Cortex and Fusiform Gyrus. Cereb Cortex, 15, 654-662

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 293-306

ISBN 978-1-60456-841-7 c 2009 Nova Science Publishers, Inc.

Chapter 22

Q UANTIFICATION OF O RDER PATTERNS R ECURRENCE P LOTS OF E VENT R ELATED P OTENTIALS Norbert Marwan1 , Andreas Groth2 and Jurgen Kurths1 ¨ 1 Nonlinear Dynamics Group, Institute of Physics, University of Potsdam, Potsdam 14415, Germany 2 Ernst-Moritz-Arndt University of Greifswald, Department of Mathematics and Computer Science, Greifswald 17487, Germany Abstract We study an innovative modification of recurrence plots defining the recurrence by the local ordinal structure of a time series. In this paper we demonstrate that in comparison to a recently developed approach this concept improves the analyis of event related activity on a single trial basis.

Keywords: Data Analysis, Recurrence plot, Nonlinear dynamics PACS numbers: 05.45, 07.05.Kf, 07.05.Rm, 91.25-r, 91.60.Pn

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

1.

Introduction

A basic research in cognitive science deals with the study of the behaviour of the brain after short, surprising stimuli. Such event related changes can be measured as changes in the brain potentials with electroencephalography (EEG), and are called event related potentials (ERPs) (Sutton et al., 1965). Traditionally, ERP waveforms are determined by computing an ensemble average of a large collection of EEG trials that are stimulus time locked. This is based on the following assumptions: (1) the presentation of stimuli of the same kind is followed by the same sequence of processing steps, (2) these processing steps always lead to activation of the same brain structures, (3) this activation always elicits the same pattern of electrophysiological activity, which can be measured at the scalp (R¨osler, 1982) and (4) spontaneous activity is stationary and ergodic.

294

Norbert Marwan, Andreas Groth and J¨urgen Kurths

EEG data contain a composition of different effects in the brain. Other signals not related with ERPs are regarded in this context as noise. In order to find characteristic ERPs in such strongly noisy EEG data, EEGs of a number of trials are measured. By averaging the data points, which are time locked to the stimulus presentation, it is possible to filter out the ERP signal of the noise (spontaneous activity). This way, a positive potential 300 ms after the stimulus (P300) was the first ERP discovered. It was inferred that the P300 component varies in dependence on subject internal factors, like attention and expectation, instead on physical characteristics (Sutton et al., 1965). The amplitude of the P300 component is highly sensitive to the novelty of an event and its relevance (surprising moment), so this component is assumed to reflect the updating of the environmental model of the information processing system (context updating) Donchin (1981); Donchin and Coles (1988). A disadvantage of the averaging is the high number of trials needed to reduce the signalto-noise-ratio. This disadvantage is crucial for example in clinical studies, studies with children and studies in which repeating a task would influence the performance. Moreover, several high frequency structures of the ERPs are filtered out by using the averaging method. Therefore, new methods for the analysis of event related activity on a single trial basis are highly desirable. A recently developed approach based on the recurrence quantification analysis has proven its ability to indicate transitions in the brain processes due to the surprising moment and to distinguish ERPs (Marwan and Meinke, 2004). In this paper we demonstrated an improvement in the analysis by an innovative modification of the recurrence plots, where the recurrence is defined by order patterns (Groth, 2005). This paper is organized as follows. First we briefly review the recurrence plots and its recurrence quantification analysis. Then, the modification of recurrence plots by order patterns is introduced. Finally, we compare both approaches on event related data from the Oddball experiment.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

2.

Recurrence Plots

We develop a recurrence quantification based on recurrence plots (RP). A RP is a N × N matrix representing neighbouring states ~xi in a d-dimensional phase space (Fig. 1) (Eckmann et al., 1987) ( 1 : k~xi − ~xj k ≤ ε ~xi ∈ Rd , i, j = 1 . . . N, (1) Ri,j (ε) = 0 : otherwise where N is the number of considered states ~xi ; ε is a threshold distance and k · k a norm. Hence, (1) is a pairwise test of the closeness of points on a phase space trajectory: points which fall in the neighbourhood of size ε are recurrence points. Another definition of RPs does not use such a fixed threshold ε: only the F nearest neighbours are considered to be recurrence points. This is the fixed amount of nearest neighbours (FAN) method and coincides with the original definition of RPs by Eckmann et al. (1987). The ratio F/N is the recurrence point density of the RP and we denote it as εF AN = F/N . In RPs we obtain different structures: If the phase space trajectory returns to itself and runs close for some time we obtain diagonal lines. Vertical lines or areas indicate phase

Quantification of Order Patterns Recurrence Plots

295

space trajectory which remain in the same area of the phase space for some time, and single dots indicate that the phase space trajectory heavily fluctuates. The phase space vectors can be reconstructed with the Taken’s time delay method ~xi = (ui , ui+τ , . . . , ui+(m−1) τ ) from one-dimensional time series ui (observation) with embedding dimension m = 2(d + 1) and delay τ (Takens, 1981; Kantz and Schreiber, 1997). B

A

24 22 45

Time

20 18 16 30

14 12

7 −25

10 10

20−30

12

14

16

18 Time

20

22

24

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 1. (A) Segment of the phase space trajectory of the Lorenz system (for standard parameters r = 28, σ = 10, b = 83 ; Lorenz, 1963) by using its three components and (B) its corresponding recurrence plot. A point of the trajectory at j which falls into the neighbourhood (grey circle in (A)) of a given point at i is considered as a recurrence point (black point on the trajectory in (A)). This is marked with a black point in the RP at the location (i, j). A point outside the neighbourhood (small circle in (A)) causes a white point in the RP. The radius of the neighbourhood for the RP is ε = 5. To characterize the dynamics of the underlying system several measures were introduced (Webber Jr. and Zbilut, 1994; Marwan et al., 2002; Marwan, 2003). Here we focus on the following four measures. We denote the frequency distribution of the lengths of diagonal lines by P (l) and that of vertical lines by P (v). The determinism is the amount of recurrence points forming diagonal lines with regard to the total amount of recurrence points DET (ε) =

PN

l P (ε, l) l=l . PNmin i,j Ri,j (ε)

(2)

Processes with stochastic behaviour cause none or very short diagonals, and thus we get low DET . Deterministic processes cause longer diagonals and less single, isolated recurrence points, and we get higher DET . The threshold lmin excludes the diagonal lines which are formed by the tangential motion of the phase space trajectory. For lmin = 1 the DET = 1, therefore lmin should be at least 2. To exclude the tangential motion, lmin can be, e. g., determined with the autocorrelation time (Theiler, 1986), but it has to be taken into

296

Norbert Marwan, Andreas Groth and J¨urgen Kurths

account that a too large lmin can worsen the histogram P (l) and thus the reliability of the measure DET . Diagonal structures indicate segments of the trajectory which are close to another segment of the trajectory at different time. Thus these lines are related to the divergence of the trajectory segments. The average diagonal line length PN l P (ε, l) l=l L(ε) = PN min (3) l=lmin P (ε, l) is the average time that two segments of the trajectory are close to each other, and can be interpreted as the mean prediction time. Although several authors stated that the inverse of the length of the diagonal lines correlates with the largest positive Lyapunov exponent (e. g. Trulla et al., 1996), it is important to note that this relationship is more complex. Analogous to the definition of the determinism (2), we define the ratio between recurrence points forming vertical structures and the entire set of recurrence points as PN vP (ε, v) v=v , (4) LAM (ε) = PN min v=1 vP (ε, v)

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

the laminarity. The computation of LAM is realized for those v that exceed a minimal length vmin in order to decrease the influence of sojourn points. For maps, vmin = 2 is used. LAM represents the occurrence of laminar states in the system without describing the length of these laminar phases. If the RP consists of more single recurrence points than vertical structures LAM decreases. The average length of vertical structures (cp. Eq. (3)) is defined as PN vP (ε, v) v=v T T (ε) = PN min , (5) v=vmin P (ε, v) and is called trapping time. With T T we measure the mean time that the system will abide at a specific state (how long a state will be trapped). The computation also uses the minimal length vmin as for LAM . Note that these measures can be computed from an entire RP or in moving windows (i. e. sub-RPs) covering the main diagonal of the RP. The latter allows us to study the change of these measures with time, which can reveal transitions in the system. Whereas the diagonal-wise defined measures are able to find chaos-order transitions (Trulla et al., 1996), the vertical-wise defined measures indicate chaos-chaos transitions (Marwan et al., 2002).

3.

Order Patterns Recurrence Plots

In (1) a recurrence is defined by spatial closeness between phase space trajectories ~xi or embedded time series ui . Now we neglect the norm k · k and define a recurrence by the local order structure of a trajectory. Given a one-dimensional time series (u1 , . . . , ui , . . . , uN ) we start to compare d = 2 time instances and define the order patterns as ( 0 : ui < ui+τ πi = (6) 1 : ui > ui+τ

Quantification of Order Patterns Recurrence Plots

297

Figure 2. Order patterns of dimension d = 3 (tied ranks ui = ui+τ are assumed to be rare). with the scaling parameter τ (tied ranks ui = ui+τ are assumed to be rare). Next, for d = 3 there are six order patterns between ui , ui+τ and ui+2 τ possible (Fig. 2). In general the d components in ~xi = (ui , ui+τ , . . . , ui+(d−1)τ ) can form d! different patterns. On systems with continuous distribution of the values the equality has measure zero and we neglect this. From these order patterns we get a new symbolic time series πi and define the order patterns recurrence plot (OPRP) as (Groth, 2005) ( 1 : πi = πj Ri,j (d) = i, j = 1 . . . N. (7) 0 : otherwise The order patterns decompose the phase space ~x into d! equivalent regions and recurrence is given if the trajectory runs throw the same region at different time. A main advantage of this symbolic representation is the well-expressed robustness against non-stationary data. The order patterns are invariant with respect to an arbitrary, increasing transformation of the amplitude. A common approach to overcome the problem of a non-stationary amplitude is the decomposition of a signal into instantaneous phase and amplitude, where only the phase is studied. In (Groth, 2005) relations between phase and order patterns are represented. Furthermore a robust complexity measure based on this symbolic dynamics was already proposed (Bandt and Pompe, 2002) and successfully applied to epileptic seizure detection(Cao et al., 2004).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

4. 4.1.

Event Related Potentials The Oddball Experiment

The Oddball experiment studies brain potentials during a stimulus presentation. In the present Oddball experiment accoustic stimuli were used. Test subjects were seated in front of a monitor and had to count tones of high pitch using the cursor keys of the keyboard. During these tests, the EEG of the subjects was recorded. The experiment was repeated in nine blocks containing at least 30 target tones. The blocks varied in the probability of occurrence of the higher tones from 10 to 90 %. The accoustic stimuli were computergenerated beeps of 100 ms length and of either high (1400 Hz) or low (1000 Hz) pitch. They were presented with an interstimulus interval of 1000 ms. The measurement of the EEG was performed with 31 electrodes/ channels (Tab. 1), where electrodes 26-31 were reference electrodes. The sample interval for the measurements was 4 ms (250 Hz).

298

Norbert Marwan, Andreas Groth and J¨urgen Kurths Table 1. Notation of the electrodes and their numbering as used in the figures (electrodes 26–31 are reference electrodes). # 1 2 3 4 5 6 7

Electrode F7 FC5 F3 FZ F4 FC6 F8

# 8 9 10 11 12 13 14

Electrode T7 CP5 C3 FCZ C4 CP6 T8

# 15 16 17 18 19 20 21

Electrode P7 PZ P3 CZ P4 P8 OZ

# 22 23 24 25

Electrode POZ PO3 CPZ PO4

We focus on the ERP data for an event frequency of 90% (ERP90) and 10% (ERP10). For ERP90, a set of 40 trials and for ERP10 a set of 31 trials are measured. The averaging of the potentials of ERP90 and ERP10 over the trials reveals the P300 ERP component, where its amplitude is higher for ERP10 (higher surprise moment, Fig. 3) (Marwan and Meinke, 2004). This confirms the knowledge about this ERP, that is related on subject-internal factors like attention and expectation instead of physical characteristics (Sutton et al., 1965) and its amplitude is sensitive to novelty of an event and its relevance (context updating, Donchin, 1981; Donchin and Coles, 1988).

−2

−2 FZ CPZ OZ

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

−1

−1

−0.5

−0.5

0

0

0.5

0.5

1

1

1.5

1.5

2 −200

0

200 400 600 Time (ms)

800

FZ CPZ OZ

−1.5

ERP

ERP

−1.5

2 −200

0

200 400 600 Time (ms)

800

Figure 3. Mean event related potentials for event frequencies of 90 % (left, 40 trials) and 10 % (right, 31 trials) at selected electrodes. The P300 component is well pronounced for the frequencies of 10 %.

Quantification of Order Patterns Recurrence Plots

4.2.

299

Recurrence Quantification

Recurrence quantification measures were already successfully applied to ERP data (Marwan and Meinke, 2004). In this work it has been shown that especially the measures DET , L, LAM and T T can be used for discrimination the events on a single trial bases. ERP90, trial 7 (CPZ)

ERP10, trial 24 (CPZ) −5 ERP

ERP

−5 0 5 −200

0

200 400 600 Time (ms)

0 5 −200

800

800

800

600

600

400 200 0 −200 −200

200 400 600 Time (ms)

800

ERP10, trial 24 (CPZ)

Time (ms)

Time (ms)

ERP90, trial 7 (CPZ)

0

400 200 0

0

200 400 600 Time (ms)

800

−200 −200

0

200 400 600 Time (ms)

800

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 4. Recurrence plots (RPs) for the ERP90 and ERP10 measured at the central-parietal electrode (CPZ). For the ERP10, more cluster of recurrence points occur around 300 ms. In order to uncover transitions in the brain processes during unexpected stimulation on a single trial basis, we firstly compute common RPs and their quantification similar as presented in (Marwan and Meinke, 2004). The quantification is applied on moving windows of size 240 ms (60 samples) with a shifting step of 8 ms, which allows us to study the time dependence of the recurrence measures. We use the embedding parameters m = 3 and τ = 12 ms and a neighbourhood criterion of εF AN = 10 % (fixed amount of nearest neighbours). The embedding parameters dimension and delay were estimated by the standard methods false nearest neighbours and mutual information, respectively (Kantz and Schreiber, 1997). The neighbourhood criterion of 10 % nearest neighbours was found heuristically to be reliable even for non-stationary data. The RPs of the ERP90 and ERP10 data sets contain diagonal lines and extended white areas (Fig. 4). One white band is located at the time of the stimulus. Other white bands which are located around 250 and 400 ms, occur almost only for ERP10 data and correspond with the P300 component. Moreover, clustered black points around 300 ms occur also only in RPs of the ERP10 data set. The application of the recurrence quantification measures to these ERP data discrimi-

300

Norbert Marwan, Andreas Groth and J¨urgen Kurths ERP10, trial 24 (CPZ)

0.8

0.8

0.6

0.6 DET

DET

ERP90, trial 7 (CPZ)

0.4 0.2

0.2 0

200 400 Time (ms)

0 −200

600

10

10

8

8

6

6

L

L

0 −200

4 2 −200

0

200 400 Time (ms)

2 −200

600

LAM 0

200 400 Time (ms)

0 −200

600

4.5

4

4 TT

LAM TT

600

0

200 400 Time (ms)

600

0

200 400 Time (ms)

600

0

200 400 Time (ms)

600

0.5

4.5

3.5 3 2.5 −200

200 400 Time (ms)

1

0.5

0 −200

0

4

1

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

0.4

3.5 3

0

200 400 Time (ms)

600

2.5 −200

Figure 5. RQA measures for selected single trials and the central-parietal electrode (solid line). The trial-averaged RQA measures for the same electrode is shown with a dashed line (the light grey band marks the 95 % significance interval). nates the single trials with a distinct P300 component resulting from a low surprise moment (high frequent events) in favour of such trials with a high surprise moment (less frequent

Quantification of Order Patterns Recurrence Plots

25

0.6

20

0.5

20

0.5

15

0.4

15

0.4

10

0.3

10

0.3

5

0.2

5

0.2

0.1

−200

0 200 400 600 L ERP10, trial 24

8

25

8

20

7

20

7

15

6

15

6

10

5

10

5

5

4

5

4

−200

0 200 400 600 LAM ERP90, trial 7

3

−200

0 200 400 600 LAM ERP10, trial 24

0.7

25

0.7

20

0.6

20

0.6

0.5

15

0.4

10

0.3

0.5

15

0.4

10

0.3 5

0.2 −200

0.2

0 200 400 600 TT ERP90, trial 7

−200

0 200 400 600 TT ERP10, trial 24

25

4

25

4

20

3.8

20

3.8

15

3.6

15

3.6

10

3.4

10

3.4

5

3.2

5

3.2

−200

0

200 400 Time (ms)

600

3

Electrode

Electrode

3

25

5

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

0.1

25

Electrode

Electrode

0 200 400 600 L ERP90, trial 7

Electrode

0.6

−200

Electrode

DET ERP10, trial 24

25

Electrode

Electrode

DET ERP90, trial 7

301

−200

0

200 400 Time (ms)

600

3

Figure 6. RQA measures for the same trials as in Fig. 5, but shown for all electrodes.

302

Norbert Marwan, Andreas Groth and J¨urgen Kurths

events). In a previous study it was found that LAM is the most distinct parameter for this discrimination (Marwan and Meinke, 2004). In the ERP data, the LAM reveals transitions from less laminar states to more laminar states after the occurrence of the event and a transition from more laminar states to less laminar states after approximately 350 ms. These transitions occur around bounded brain areas (parietal to frontal along the central axis). The comparable measures DET and LAM as well as L and T T reveal similar results, because extended black areas contain also a high amount of diagonal lines (Figs. 5 and 6). Next we compute OPRPs and quantify them by using the same moving windows as for the common RPs. We use the dimension d = 3, i. e. six order patterns and a delay of τ = 20 ms. The OPRPs are different in comparison to the common RPs (Fig. 7). They are more homogeneous and do not reveal such “disruption” as shown in Fig. 4. This is due to the robustness of OPRPs with respect to non-stationarity. All measures gained from OPRPs reveal significant differences between ERP90 and ERP10. For the same trial, we find a more distinct difference using OPRPs than common RPs (Fig. 8). The quantification measures for ERP10 reveal high amplitudes at approximately 300 ms after the stimulus, wheras for ERP90 they vary within their standard deviation. ERP90, trial 7 (CPZ)

ERP10, trial 24 (CPZ) −5 ERP

ERP

−5 0 5 −200

0

200 400 600 Time (ms)

0 5 −200

800

800

800

600

600

400 200 0 −200 −200

200 400 600 Time (ms)

800

ERP10, trial 24 (CPZ)

Time (ms)

Time (ms)

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

ERP90, trial 7 (CPZ)

0

400 200 0

0

200 400 600 Time (ms)

800

−200 −200

0

200 400 600 Time (ms)

800

Figure 7. Order patterns recurrence plots (OPRPs) for the ERP90 and ERP10 measured at the central-parietal electrode (CPZ). Their appearance differs from those of common recurrence plots (Fig. 4) In contrast to the analysis with common RPs, the measures based on OPRPs are more

Quantification of Order Patterns Recurrence Plots

303

different for different channels (Figs. 6 and 9). Electrodes in the frontal-central area (FZ, FCZ, CZ) reveal high amplitudes in DET , L, LAM and T T between 100 and 400 ms. Electrodes in the right frontal to parietal area (F4, C4, CP6, P4, PO4) reveal high amplitudes in these measures around 300 to 400 ms after the stimulus. ERP10, trial 24 (CPZ)

0.8

0.8

0.6

0.6 DET

DET

ERP90, trial 7 (CPZ)

0.4 0.2

0.2 0

200 400 Time (ms)

0 −200

600

10

10

8

8

6

6

L

L

0 −200

4 2 −200

0

200 400 Time (ms)

2 −200

600

LAM 0

200 400 Time (ms)

0 −200

600

10

8

8 TT

LAM TT

600

0

200 400 Time (ms)

600

0

200 400 Time (ms)

600

0

200 400 Time (ms)

600

0.5

10

6 4 −200

200 400 Time (ms)

1

0.5

0 −200

0

4

1

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

0.4

6 4

0

200 400 Time (ms)

600

−200

Figure 8. RQA measures for selected single trials and the central-parietal electrode (solid line). The trial-averaged RQA measures for the same electrode is shown with a dashed line (the light grey band marks the 95 % significance interval).

304

Norbert Marwan, Andreas Groth and J¨urgen Kurths DET ERP90, trial 7

15 0.4

10 5 0 200 400 600 L ERP90, trial 7

−200

0.2 0 200 400 600 L ERP10, trial 24

5

25

5

20

4.5

20

4.5

15

4

10

−200

3.5

0 200 400 600 LAM ERP90, trial 7

25

0.7

10

0.6

5 −200

3

0.8

15

3.5

0 200 400 600 LAM ERP10, trial 24

25

0.8

15

0.7

10

0.6

5

0 200 400 600 TT ERP90, trial 7

−200 6

3

0.9

20

0.5 0 200 400 600 TT ERP10, trial 24 6

25 20

5

15 10

4

5

Electrode

20

−200

4

10

−200

0.5

25

15

5

0.9

20

Electrode

0.4

10

25

5

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

0.6

15

5

0.2

Electrode

Electrode

−200

20 Electrode

0.6

0.8

25

Electrode

Electrode

20

Electrode

DET ERP10, trial 24 0.8

25

5

15 10

4

5 0

200 400 Time (ms)

600

3

−200

0

200 400 Time (ms)

600

3

Figure 9. RQA measures for the same trials as in Fig. 8, but shown for all electrodes.

Quantification of Order Patterns Recurrence Plots

5.

305

Conclusions

From these results we can infer that the application of order patterns is more appropriate in order to study event related potentials on a single trial basis. In comparison with the common recurrence plots, the transition to order patterns has the advantage to reveal more significantly the P300 component and, moreover, differentiates better between single electrodes. As already found in a previous work, the P300 component is related with specific chaoschaos transitions where laminar states occur (Marwan and Meinke, 2004). These transitions can also be detected with order patterns. Using OPRPs, these transitions can be localized in the frontal-central and slightly right frontal to parietal regions. The reliability of this method is currently tested by using EEG data of linguistic experiments Schinkel et al. (subm). A further improvement of this approach could be possible by using a spatio-temporal approach for the reconstruction of the phase space trajectory Mandelj et al. (2001).

Acknowledgments We are grateful to Anja Meinke (Research Centre J¨ulich) for providing the data of the Oddball experiment. This study was made possible in part by grants from the Microgravity Application Program/Biotechnology from the Human Spaceflight Program of the European Space Agency, project #14592 (N. Marwan, J. Kurths), and from the DFG priority programme 1114 (A. Groth, J. Kurths).

References Bandt, C. and Pompe, B. (2002). Permutation entropy - a complexity measure for time series. Physical Review Letters, 88:174102.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Cao, Y., Tung, W., Gao, J. B., Protopopescu, V. A., and Hively, L. M. (2004). Detecting dynamical changes in time series using the permutation entropy. Physical Review E, 70:046217. Donchin, E. (1981). Surprise! . . . Surprise? Psychophysiology, 18:493–513. Donchin, E. and Coles, M. G. H. (1988). Is the P300 component a manifestation of context updating? Behavioral and Brain Sciences, 11:357–374. Eckmann, J.-P., Kamphorst, S. O., and Ruelle, D. (1987). Recurrence Plots of Dynamical Systems. Europhysics Letters, 5:973–977. Groth, A. (2005). Visualization of coupling in time series by order recurrence plots. Physical Review E, 72(4):046220. Kantz, H. and Schreiber, T. (1997). Nonlinear Time Series Analysis. University Press, Cambridge. E.

306

Norbert Marwan, Andreas Groth and J¨urgen Kurths

Lorenz, E. N. (1963). Deterministic Nonperiodic Flow. Journal of the Atmospheric Sciences, 20:120–141. Mandelj, S., Grabec, I., and Govekar, E. (2001). Statistical approach to modeling of spatiotemporal dynamics. International Journal of Bifurcation and Chaos, 11(11):2731– 2738. Marwan, N. (2003). Encounters With Neighbours – Current Developments Of Concepts Based On Recurrence Plots And Their Applications. PhD thesis, University of Potsdam. Marwan, N. and Meinke, A. (2004). Extended recurrence plot analysis and its application to ERP data. International Journal of Bifurcation and Chaos “Cognition and Complex Brain Dynamics”, 14(2):761–771. Marwan, N., Wessel, N., Meyerfeldt, U., Schirdewan, A., and Kurths, J. (2002). Recurrence Plot Based Measures of Complexity and its Application to Heart Rate Variability Data. Physical Review E, 66(2):026702. R¨osler, F. (1982). Hirnelektrische Korrelate kognitiver Prozesse. Springer, Berlin. Schinkel, S., Marwan, N., and Kurths, J. (subm.). Towards single trial ERP analysis. Journal of Neuroscience Methods. Sutton, S., Braren, M., Zubin, J., and John, E. R. (1965). Evoked potential correlates of stimulus uncertainty. Science, 150:1187–1188. Takens, F. (1981). Detecting Strange Attractors in Turbulence. In Rand, D. and Young, L.S., editors, Dynamical Systems and Turbulence, volume 898 of Lecture Notes in Mathematics, pages 366–381. Springer, Berlin. Theiler, J. (1986). Spurious dimension from correlation algorithms applied to limited timeseries data. Physical Review A, 34(3):2427–2432.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Trulla, L. L., Giuliani, A., Zbilut, J. P., and Webber Jr., C. L. (1996). Recurrence quantification analysis of the logistic equation with transients. Physics Letters A, 223(4):255–260. Webber Jr., C. L. and Zbilut, J. P. (1994). Dynamical assessment of physiological systems and states using recurrence plot strategies. Journal of Applied Physiology, 76(2): 965–973.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 307-317

ISBN 978-1-60456-841-7 c 2009 Nova Science Publishers, Inc.

Chapter 23

N EURONAL S YNCHRONIZATION : F ROM DYNAMIC F EATURE BINDING TO O BJECT R EPRESENTATIONS Alexander Maye1∗ and Markus Werning2† 1 Zuse-Institute Berlin, Visualization and Data Analysis Group, Takustr. 7, D-14195 Berlin, current address: Institute of Neurophysiology and Pathophysiology, Center of Experimental Medicine, University Medical Center Hamburg-Eppendorf, Martinistr. 52, D-20246 Hamburg 2 Department of Philosophy, Heinrich-Heine-University, Universit¨atsstrasse 1, D-40225, D¨ usseldorf

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Abstract Using two different models of oscillatory activity in the primary visual cortex, we analyze the synchronization properties of the networks by an eigenmode decomposition. Both models use clusters of feature-sensitive neurons representing local object properties like color and orientation. Whereas in the mean-field model oscillators communicate via their current amplitude, in the phase model oscillator interaction was controlled by phase difference. In both cases, eigenmode analysis decomposes the complex synchronization patterns into a time-invariant, spatial component, the eigenmodes, and characteristic functions describing their weight in network state over time. We find that characteristic functions can be associated with representations of objects in a visual scene, and eigenmodes represent different epistemic possibilities.

1. Introduction Synchronization of neuronal activity is a recurring phenomenon in experiments analyzing cortical activity. In a basic explanation for a functional role it is seen as a mechanism for binding the responses of distributed neuronal populations responding to different properties of a visual stimulus (von der Malsburg, 1981; Gray et al., 1989; Engel et al., 1990). It can dynamically establish relations between neuronal populations (Singer, 1999), obviating the ∗ †

E-mail address: [email protected] E-mail address: [email protected]

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

308

Alexander Maye and Markus Werning

need for a combinatorially prohibitive number of specialized neurons (‘grandmother cells’) representing all possible feature combinations in visual stimuli. Despite the experimental evidence for the relevance of neuronal synchronization in generating perceptual states (Fries et al., 1997; Engel et al., 1999; Bhattacharya et al., 2001), the particular mechanisms still have to be revealed. In regard to perceptual binding of visual stimuli it is assumed that neurons responding to the shape and other properties of the same object tend to synchronize their activity while the activity of neurons activated by different objects remains uncorrelated. However, objects consisting of characteristic parts pose a dilemma for this simple binding mechanism. If synchronization is used to bind the properties within each of the object’s parts it cannot be used at the same time to bind the parts into a holistic perception of the object. Otherwise different assignments of the same set of properties to the object parts would not generate different object representations. For example, a brown cow with white legs would be indistinguishable from a white cow with brown legs. In an accompanying article by Werning and Maye and in (Werning & Maye, 2005) we show how this dilemma can be overcome by looking at synchronization as a mechanism for implementing frame theory. Here we analyze the temporal properties of a neural network using eigenmode decomposition. We argue that eigenmodes reflect different interpretations of a stimulus. The characteristic functions associated with each eigenmode can serve as object representations which take into account hierarchical relations between the different parts of a stimulus. Computing eigenmodes and characteristic functions yields a decomposition of oscillatory network dynamics into a spatial and a temporal component, respectively. Based on these components, in (Werning, 2005) a first order predicate language with identity P L= has been developed to show in detail how oscillatory networks can fully implement the semantics of concepts. There it was concluded that clusters of synchronously activated cells can be interpreted as compositionally structured conceptual representations of visual scenes. The explanatory power of the eigenmodes relies on the simultaneous analysis of a large number of neurons or neuronal populations. Currently, accordant experimental data are scarce. We therefore used two different models to simulate neuronal oscillatory networks. The mean-field model is biologically inspired and consistent with experimental findings on neuronal synchrony. By reducing this model to its basic functional principles we arrived at a network of phase coupled oscillators producing qualitatively the same results. The next section describes the models in detail. Section 3. explains eigenmode analysis, followed by a presentation of simulations and results in section 4.. The eigenmodes are interpreted and conclusions are drawn in section 5..

2. Network Models 2.1. Mean-Field Model The state variables in this model describe the average activity of a small population (≈ 100 . . . 200) of spatially proximal and physiologically similar biological neurons. Wilson and Cowan (1972) showed that two recurrently connected populations, one of excitatory and the other of inhibitory neurons, can generate stable limit cycle oscillations. In the

Neuronal Synchronization

309

following time, networks of coupled oscillators have been used to model multidimensional feature binding (Schillen & K¨onig, 1994), contour integration and enhancement (Li, 2000), and odor recognition (Li & Hertz, 2000). Equations describing the dynamics of a single oscillator have been derived using meanfield theory (Wilson & Cowan, 1972; Schuster & Wagner, 1990). Our model focuses on the interaction between coupled oscillators and does not use a mean-field approach in the strict sense. To distinguish this model which is based on equations derived by a meanfield approach from the phase-coupled oscillator model in the next section, we will call it mean-field model. In detail, the dynamics of a single oscillator (see Fig. 1a) with lateral coupling to other oscillators is given by: X τx x˙i = −xi − gy (yi ) + J0 gx (xi ) + Jij gx (xj ) + hi + ηx (1a) τy y˙i = −yi + gx (xi ) −

X

j

Wij gy (yj ) + ηy .

(1b)

j

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The activity of the excitatory and the inhibitory populations is denoted byx and y, respectively. Jij describes the coupling strength between the excitatory populations of oscillator i and j. Accordingly, Wij is the coupling between inhibitory populations. J0 models local self-excitation, η is white noise, and time constants τ can be used to match refractory times of biological neurons. For the transfer functions gα , α ∈ {x, y}, typically sigmoidal functions are used. The saturation property of this function prevents diverging network activity. Reaching saturating activity, however, would be disadvantageous for the neurons from a metabolistic viewpoint. In order to show that non-divergent network dynamics is possible without saturating activity we used semi-linear transfer functions with threshold, ( mα (x − θα ) if x > θα . (2) gα (x) = 0 else Conditions for non-divergent network activity have been derived analytically by Wersing et al. (2001). The external input hi was used to represent the output of feature-sensitive neurons in the visual pathway. Oscillators receiving unimodal feature information of a visual stimulus constitute a feature module. Within a feature module oscillators are arranged on a threedimensional grid. Neurons within a single layer respond to the same feature value at the respective location in the visual field. Different layers respond to different feature values of the same modality. Figure 1b shows the coupling scheme. Neighboring oscillators in a layer have connections between their excitatory neurons. These connections reduce phase differences between coupled oscillators, thereby synchronizing their activity. Neighboring oscillators in different layers have connections between their inhibitory neurons. These connections increase phase differences. We consider large phase differences to be equivalent to desynchronized states in biological oscillatory networks. The coupling strength falls off exponentially with increasing distance. The connection scheme can be seen as an implementation of

J0 h x

Orientation

Alexander Maye and Markus Werning

Color

310

y

J x Coupling

a)

y

W

y

b)

x

desynchronizing

0

synchronizing

Figure 1. a) A single oscillator. b) A feature module is composed of a three-dimensional arrangement of oscillators (represented by small cubes). Slices with grey levels visualize connection type and strength of the oscillator at the center of the feature module. Two feature modules have been used, one color module and one orientation module. The arrow between the feature modules symbolizes synchronizing connections between a single oscillator at the origin and all oscillators with the same receptive field position in the other feature module (visualized by a cylinder). This connection scheme is applied to all other positions √ as well. Parameters: τx = τy = 1, mx = my = 2, 2 θx = 2, θy = 1, hi = 2, J0 = 0.4, Jij = J/ 2πσ 2 exp(−rij /2σ 2 ) (Wij accordingly), J = 0.6 if i and j are in the same feature module and J = 0.08 if j is in a different module, W = 0.05, r = 4, η = 0.4.

two Gestalt laws of perception (Wertheimer, 1924/1950) according to which elements that are spatially proximal or share similar properties have the tendency to be grouped together. The model was set up to be consistent with anatomical findings in primary visual cortex. There, excitatory projection neurons in layer 2 and 3 are connected to local inhibitory interneurons in layers 2 to 6 (Thomson & Bannister, 2003). Multi-modal feature integration is attained by coupling different feature modules. Synchronizing connections between any single oscillator in one feature module and all oscillators with the same receptive field position in all other feature modules mediate the synchronization of oscillators activated by different properties of the same object.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

2.2. Model with Phase-Coupled Oscillators In order to verify the results obtained in the mean-field model we performed the same experiments with a model using an alternative description of neuronal oscillators. Models with phase-coupled oscillators do not consider the particular mechanism by which oscillations are generated. A single oscillator is assumed to generate asymptotically stable limit-cycle oscillations (Kuramoto, 1984; Sturm & K¨ onig, 2001) of frequency ω. Its current state is given by the phase φ(t) and the amplitude a(t). The output of the neuronal population modeled by a single oscillator is: xi (t) = ai (t) sin(φi (t)). Neighboring oscillators are coupled by synchronizing or desynchronizing connections. They can interact via their phase, their amplitude, or both. The current model is confined

Neuronal Synchronization

311

to phase interactions only. In the case of synchronizing connections, the phase difference between two oscillators is used to advance the phase of the lagging oscillator and to retreat the phase of the leading oscillator (vice versa for desynchronizing connections). In our model the dynamics of an oscillator is given by: φ˙i = ω −

X

sij aj sin(φi − φj ) + ηi

(3a)

j

a˙i = −ai + hi .

(3b)

Weights sij comprise couplings within a feature module as well as between feature modules. Synchronizing connections have sij > 0, whereas desynchronizing connections are given by sij < 0. The same connection scheme as for the mean-field model is applied. Again, η is a noise term and hi describes external input from the feature detectors. Phase-coupled oscillator models have been applied for reproducing experimental results from the visual cortex (Sompolinsky & Golomb, 1991). Apart from a more concise formulation and better analytical amenability, an important advantage of this model is the possibility to separately investigate the effects of phase and amplitude interaction between oscillators. In the current model we included phase interaction only. This is sufficient to reproduce qualitatively the same results as in the mean-field model. Amplitude interaction will be considered in future models.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

3. Eigenmode Analysis Upon stimulation the networks described in the previous section generate oscillatory patterns that simulate oscillatory activity of a larger number of neurons in primary visual cortex. In neurobiological experiments these patters are typically analyzed using pairwise measures like cross-correlation or coherence. Analyzing the temporal dynamics of a larger network using measures that take into account only pairs of oscillators is a very tedious task, though. What we want instead is a method for analyzing all oscillators simultaneously. From synergetics it is well known that the dynamics of complex systems is often governed by a few dominating states and can, therefore, be described by a small set of corresponding order parameters (Haken, 1990). These states are the eigen- or principal modes of the system, the corresponding eigenvalues designate how much of the variance is accounted for by that mode. In principle, eq. (1) can be solved analytically by linearization around the fix point and combination of both equations into a second order differential equation (which yields a vectorial form of the fundamental equation of a harmonic oscillator, for that matter). The eigenvectors of the solution constitute a set of eigenstates that characterize the dynamics of the network. Analytic determination of the fix point, however, is possible only for a small and rather uninteresting subset of stimuli, e.g., infinitely long lines or homogeneous activation. Another way of describing oscillatory network activity by superposition of eigenstates is to determine the principal components of the activity based on a numerical simulation of

312

Alexander Maye and Markus Werning

the network. This is possible for arbitrary stimuli. Computationally, the principal components are eigenvectors of the covariance matrix C:   x1 (t1 ) x1 (t2 ) · · · x1 (tm )  x2 (t1 ) x2 (t2 ) · · · x2 (tm )    D=  .. .. .. ..   . . . . xn (t1 ) xn (t2 ) · · ·

xn (tm )

C = DDT VΛV−1 = C Matrix D contains the activity of oscillators at equidistant time points. V is the matrix of eigenvectors and the diagonal matrix Λ contains the corresponding eigenvalues. The eigenmodes constitute an orthonormal coordinate system in which the variance of the network activity in each direction is determined by the magnitude of the respective eigenvalues. The network activity can be described by a superposition of the eigenmodesvi with time-dependent weights ci (t): X ci (t)vi x(t) = i

The weights ci (t) are determined by projecting the network activity on the respective eigenmode i: ci (t) = x(t)T vi . We will call the weights ci (t) characteristic functions because they correspond to distinct interpretations of the stimulus. If functions ci (t) have a sinusoidal time course they can be expressed by ki eλi t+φi . Here, ki is the amplitude of the oscillation and the imaginary part of the complex eigenvalues λi is its frequency. The network activity can then be written as X x(t) = ki vi eλi t+φi ,

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

i

which is isomorphic to the analytic solution. In general, the eigenvectors of the covariance matrix will differ from the eigenvectors of the analytic solution of eq. (1). However, if there are strong differences in the variances of the principal directions, they can be considered as approximations of the eigenmodes of the analytic solution. Strong differences of variances are given when the ordered sequence of eigenvalues λi is quickly decreasing, which was the case in all our experiments.

4. Results We investigated multi-dimensional feature binding in networks consisting of two feature modules, one responding to colors and the other to edge orientations. A number of simple (Maye, 2003; Maye & Werning, 2004) and more complex stimuli (Werning & Maye, 2005) have been applied. The network dynamics were determined by numerical integration of eqs. (1) and (3). Figure 2 shows a representative result for a stimulus consisting of a

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Neuronal Synchronization

313

red vertical and a green horizontal (Fig. 2a). Cross-correlation analysis revealed zero-lag phase synchronization between oscillators which are activated by the same object (Fig. 2b). We performed eigenmode analyses on the resulting network activity. The eigenmodes were ordered according to their eigenvalue and reshaped to visualize the contribution of the oscillators in different layers of the feature modules (Fig. 2c). Eigenmode analysis decomposes the network dynamics into a set of orthogonal states. In the following, we interprete the eigenmodes with the two largest eigenvalues. In the first eigenmode, i.e. the one which accounts for most of the variance, all oscillators activated by the stimulus make the same positive contribution. This corresponds to an oscillation pattern in which all oscillators are synchronized. This in turn can be interpreted as a representation of the stimulus as a single object. In the second eigenmode, neurons sensitive for red and for vertical orientation make a negative contribution, whereas neurons sensitive to green and horizontal make a positive contribution. This corresponds to an antiphasic oscillation between the oscillators activated by the red vertical and those activated by the green horizontal. This in turn can be interpreted as the network binding the properties red and vertical into one object and green and horizontal into another. Taken together the first two eigenmodes reflect two alternative groupings of the stimulus elements. Projecting the network activity into the eigenmode space yields the characteristic functions ci (t) (Fig. 2d). While eigenmodes depict the spatial distribution of activity, the associated characteristic functions show the contribution of each eigenmode to the network activity over time. From Figures 2d and 3b it can be seen that the characteristic function for the first eigenmode constitutes an envelope for the function of the second eigenmode. This result suggests a mechanism for the definition of hierarchical relations between different interpretations of the stimulus. The network dynamics is the representational basis of these interpretations and eigenmode analysis is just one way to extract a pictorial version. Selection of an interpretation would be possible by synchronization with network states which are dominated by the respective eigenmode, i.e. by synchronization with the corresponding characteristic function. In an accompanying article (see article by Werning and Maye in this issue) we show how characteristic functions can indeed serve as object representations. A Fourier transform has been used to analyze the spectral components of the characteristic functions (Fig. 2e). It shows that subordinate hierarchy levels are associated with increasing frequencies. The hypothesis that different frequency bands are involved in stimulus processing at different spatial scales is supported by experiments (Frien & Eckhorn, 2000). We tested if this effect is depending on the noise level in the network. Doubling the noise (η = 0.4) did not change the results qualitatively. Exactly the same analyses were applied to the dynamics generated by the phase model. The results are shown in Figure 3.

5. Conclusion The results demonstrate that eigenmodes can represent different binding solutions and, therefore, different interpretations of the stimulus. This capability is prerequisite for a hierarchical binding mechanism which allows recognition of an object as a single entity as

314

Alexander Maye and Markus Werning color module red layer

green layer

orientation module vertical layer

horizontal layer

correlation

a) 5 0 −5 −50

b)

0

50

c)

∆t

50 40

ci(t)

30 20 10 0 −10

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

d)

−20 800

c1 c

2

c3 c4 850

900 t

950

1000

e)

Figure 2. a) A stimulus showing a red vertical and a green horizontal (visualized by different shades of gray). b) The cross-correlogram shows zero phase-lag synchronization between the two oscillators at the position marked by white arrows in a). c) Four largest eigenmodes (from top to bottom) generated by the mean-field model. d) Characteristic functions associated with the eigenmodes. e) Fourier spectrum of the characteristic functions. Note the different scales on the y-axis. well as distinguishing its parts. Our results predict that this binding mechanism employs synchronization of neural activity in different frequency bands. A comparison of the two modeling approaches shows that, qualitatively, the results are model independent. It is known that nonlinear oscillators interacting through amplitude coupling can be reduced to a phase model in the condition of weak coupling (Kuramoto, 1984). Since amplitude interactions between oscillators were eliminated in the phase model we conclude that phase interactions are sufficient to generate the observed results. This leaves amplitude interactions with the possibility for another functional role which will be the focus of future investigations.

Neuronal Synchronization color module red layer

315

orientation module

green layer

vertical layer

horizontal layer

a) 15

ci(t)

10

5 c1 0

c

2

c3 c4

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

b)

−5 1900

1920

1940

1960

1980

2000

t

c) Figure 3. Eigenmode analysis of the dynamics of the phase model. a) eigenmodes, b) characteristic functions, c) Fourier spectrum of the characteristic functions. Note the different scales on the y-axis. Parameters: ω = hi = 1, sij = 0.625/ − 0.125/0.125 for synchronizing connections within a layer / desynchronizing connections to neighboring layers / synchronizing connections to other feature module,r = 1, η = 0.2

316

Alexander Maye and Markus Werning

Our results suggest that understanding the function of neuronal synchrony is likely to involve the analysis of temporal correlations between a large number of neurons. Application of our analysis to experimental data is limited by the low spatial resolution of current experimental techniques. The most advanced method in this respect uses voltage sensitive dyes in conjuction with optical imaging (Leznik et al., 2002; Jancke et al., 2004). It allows analyzing the activity of larger cortical patches with high temporal and spatial resolution. Together with the eigenmode analysis presented here, this might be a promising approach for studying the functional role of neuronal oscillations.

References Bhattacharya, J., Petsche, H., & Pereda, E. (2001, August). Long-range synchrony in the γ-band: Role in music perception. The Journal of Neuroscience, 21(16), 6329–6337. Engel, A. K., Fries, P., K¨onig, P., Brecht, M., & Singer, W. (1999). Temporal binding, binocular rivalry, and consciousness. Consciousness and Cognition, 8, 128-151. Engel, A. K., K¨onig, P., Gray, C. M., & Singer, W. (1990). Stimulus-dependent neuronal oscillations in cat visual cortex: inter-columnar interaction as determined by crosscorrelation analysis. European Journal of Neuroscience, 2, 588-606. Frien, A., & Eckhorn, E. (2000). Functional coupling shows stronger stimulus dependency for fast oscillations than for low-frequency components in striate cortex of awake monkey. European Journal of Neuroscience, 12, 1466–1478. Fries, P., Roelfsema, P. R., Engel, A. K., K¨onig, P., & Singer, W. (1997). Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proc. Natl. Acad. Sci. USA, 94, 12699-704. Gray, C. M., K¨onig, P., Engel, A. K., & Singer, W. (1989, March). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338, 334–337. Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Haken, H. (1990). Synergetik (3 ed.). Springer Verlag. Jancke, D., Chavane, F., Naaman, S., & Grinvald, A. (2004, March). Imaging cortical correlates of illusion in early visual cortex. Nature, 428, 423–426. Kuramoto, Y. (1984). Chemical oscillations, waves, and turbulence. Springer. Leznik, E., Makarenko, V., & Llin´as, R. (2002, April). Electrotonically mediated oscillatory patterns in neuronal ensembles: An in-vitro voltage-dependent dye-imaging study in the inferior olive. The Journal of Neuroscience, 22(7), 2804–2815. Li, Z. (2000). Pre-attentive segmentation in the primary visual cortex. Spatial Vision, 13(1), 25–50. Li, Z., & Hertz, J. (2000). Odour recognition and segmentation by a model olfactory bulb and cortex. Computational Neural Systems, 11, 83–102.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Neuronal Synchronization

317

Maye, A. (2003). Correlated neuronal activity can represent multiple binding solutions. Neurocomputing, 52-54C, 73–77. Maye, A., & Werning, M. (2004). Temporal binding of non-uniform objects. Neurocomputing, 58–60, 941–948. Schillen, T. B., & K¨onig, P. (1994). Binding by temporal structure in multiple feature domains of an oscillatory neuronal network. Biological Cybernetics, 70, 397-405. Schuster, H. G., & Wagner, P. (1990). A model for neuronal oscillations in the visual cortex. Biological Cybernetics, 64, 77–82. Singer, W. (1999, September). Neuronal synchrony: A versatile code for the definition of relations? Neuron, 24, 49–65. Sompolinsky, H., & Golomb, D. (1991, June). Cooperative dynamics in visual processing. Physical Review A, 43(12), 6990–7011. Sturm, A., & K¨onig, P. (2001). Mechanisms to synchronize neuronal activity. Biological Cybernetics, 84, 153–172. Thomson, A., & Bannister, A. (2003, jan). Interlaminar connections in the neocortex. Cerebral Cortex, 13(1), 5–14. von der Malsburg, C. (1981). The correlation theory of brain function (Internal Report No. 81-2). G¨ottingen: Max-Planck-Institute for Biophysical Chemistry. Werning, M. (2005). The temporal dimension of thought: Cortical foundations of predicative representation. Synthese, 146(1/2), 203–24. Werning, M., & Maye, A. (2005). Frames, coherency chains and hierarchical binding: The cortical implementation of complex concepts. In B. Bara, L. Barsalou, & M. Bucciarelli (Eds.), Proceedings of the twenty-seventh annual conference of the the cognitive science society. (in press) Wersing, H., Beyn, W.-J., & Ritter, H. (2001, August). Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions. Neural Computation, 13(8), 1811–1825. Wertheimer, M. (1950). Gestalt theory. In W.D.Ellis (Ed.), A sourcebook of Gestalt psychology (p. 1-11). New York: The Humanities Press. (Original work published 1924) Wilson, H. R., & Cowan, J. D. (1972, January). Excitatory and inhibitory interactions in localized populations of model neurons. Biophysical Journal, 12(1).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 319-335

ISBN 978-1-60456-841-7 c 2009 Nova Science Publishers, Inc.

Chapter 24

T ESTING FOR C OUPLING A SYMMETRY U SING S URROGATE DATA Milan Paluˇs1∗, Bojan Musizza2 and Aneta Stefanovska3 1 Institute of Computer Science, Academy of Sciences of the Czech Republic, Pod vod´arenskou vˇezˇ´ı 2, 182 07 Prague 8, Czech Republic 2 Department od Systems and Control, Joˇzef Stefan Institute Jamova 39, 1000 Ljubljana, Slovenia 3 Nonlinear Dynamics and Synergetics, Faculty of Electrical Engineering, University of Ljubljana, Trˇzaˇska 25, 1000 Ljubljana, Slovenia

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Abstract Possible asymmetry in the coupling of complex oscillatory systems can, in principle, be inferred from experimental bivariate time series. However, the quantities estimated from experimental data can be severely biased. We propose to test the significance of the estimated asymmetry measures by statistical tests using surrogate data. In this way we will extend the usage of surrogate data from testing the existence of nonlinear dependence to testing whether the coupling is symmetric or asymmetric. The numerically generated surrogate data should mimic most of the statistical and dynamical properties of the tested data, except for the coupling asymmetry, and thus enable estimation of the bias and variance of the used asymmetry measures. “Ideal” surrogate data for testing coupling asymmetry would be series recorded from the same coupled systems when the coupling is symmetric, with its strength adjusted so that symmetric measures of coupling are the same as those in the tested data. Such surrogate data cannot, however, be obtained in usual experimental situations. In an extensive numerical study we compare the size and power of asymmetry tests using various types of surrogate data. We discuss the conditions under which Fourier phaserandomized surrogates can perform comparably with the “ideal” surrogate data with symmetric coupling. ∗

E-mail address: [email protected], Phone: +420 266 053430, Fax: +420 286 585 789. Address for correspondence: Institute of Computer Science AS CR, Pod vod´arenskou vˇezˇ´ı 2, 182 07 Prague 8, Czech Republic

320

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

1.

Milan Paluˇs, Bojan Musizza and Aneta Stefanovska

Introduction

Cooperative behavior of coupled complex systems has recently attracted considerable interest from theoreticians as well as experimentalists (see e.g. the monograph [1]), since synchronization and related phenomena have been observed not only in physical, but also in many biological systems. Examples include the cardio-respiratory interaction [2, 3, 4, 5, 6, 7] and the synchronization of neural signals [8, 9, 10, 11, 12]. In such physiological systems it is not only important to detect synchronized states, but also to identify drive-response relationships between the systems studied. Several measures have been proposed for application to bivariate time series from coupled systems in order to establish possible asymmetry of coupling, and thus the causality in evolution of the interacting (sub)systems. Such measures, especially those applied to the instantaneous phases of interacting oscillators, are able to identify and quantify the coupling asymmetry in many numerical and experimental examples. Rosenblum et al. [13, 14] approximate functional relationships between the instantaneous phases of interacting oscillators and propose a normalized “directionality index” which should indicate the direction of coupling in the sense that it is ±1 in the case of unidirectional coupling, and 0 for symmetric coupling. The index is between 0 and ±1 for bidirectional but asymmetric coupling, while its negativity/positivity indicates which system dominates their mutual relationship, i.e. which system more strongly influences the other. Paluˇs and Stefanovska [15] proposed a way of capturing relationships between the phases of interacting systems by use of conditional mutual information, normalizing it (Sec. 2.) in order to obtain behavior comparable with the Rosenblum index [13, 14]. Both indices correctly indicate the coupling asymmetry and even reflect the ratio of coupling coefficients of asymmetrically coupled, structurally similar, dynamical systems [13, 15]. The situation can be quite different when processing real experimental data. Noise and the limited length of experimental time series can be sources of high variance; and different statistical and dynamical properties (distributions, dominant frequencies) of the two components of a bivariate series can cause severe biases in estimates of the directionality indices. In order to illustrate the approach and possible problems in its applications, we use data from an animal study in which we investigated cardio-respiratory coupling in anæsthetised rats (a detailed description will be published elsewhere). The directionality indices between respiratory and cardiac oscillations were computed within a window of 80 seconds, which was moved over the whole 75 minute record using a step length of 20 seconds. The two indices [13] and [15] give qualitatively consistent results but with different biases and variance, which can lead to differences in some situations. In Fig. 1, in the first part of the record (up to 45 minutes), the Rosenblum index [13](Fig. 1a) oscillates between positive and negative domains while the index based on mutual information [15](Fig. 1b) is confined to positive values, indicating the respiration is driving the cardiac activity, although the index variance is rather large. In the second part of the record (after the 45th minute) both the indices evidently decrease, the information-theoretic index oscillates between negativity and positivity while the Rosenblum index is negative, indicating driving of respiration by the cardiac activity. Which result is correct? None, since in this illustrative example we used the data of two different animals: the respiratory data are taken from the rat #7 and the cardiac data from the rat #9. We use this example to demonstrate how unreliable it

Testing for Coupling Asymmetry Using Surrogate Data

321

can be to draw inferences about causality when the coupling asymmetry is deduced from a directionality index alone. In this paper we propose to infer the coupling asymmetry using statistical testing based on the surrogate data approach. We extend the usage of the surrogate data from testing for nonlinear dependence into testing for coupling asymmetry. We demonstrate that, if we are not able to construct realizations of the adequate null hypothesis (symmetric coupling), even independent or bivariate linearly dependent surrogate data can perform satisfactorily for moderately noisy data of sufficient series length. In Section 2. the information-theoretic functionals used for computing the directionality indices are briefly introduced. More details can be found in [11, 12, 15]. The test data used in our numerical experiment are defined in Sec. 3.. Section 4. provides a short introduction to the concept of surrogate data, which is extended to the inference of coupling asymmetry in Sec. 5., with technical details on generation of the test data and related surrogate data given in Sec. 6.. The results are described in Sec. 7. and discussed in Sec. 8..

2.

Information Theoretic Approach to Inference of Coupling Asymmetry Using Phases of Coupled Oscillators

For detection and quantification of coupling (asymmetry) we use information-theoretic tools such as the well-known mutual information I(X; Y ) of two random variables X and Y , given as I(X; Y ) = H(X) + H(Y ) − H(X, Y ), where the entropies H(X), H(Y ), H(X, Y ) are given according to the usual definition by Shannon [16, 17, 18, 19, 20, 21, 11]. The conditional mutual information I(X; Y |Z) of the variables X, Y , given the variable Z, is defined using the conditional entropies [11, 21] as

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

I(X; Y |Z) = H(X|Z) + H(Y |Z) − H(X, Y |Z).

(1)

Consider two time series {x(t)} and {y(t)} regarded as realizations of two stationary ergodic stochastic processes {X(t)} and {Y (t)} representing the observables of two possibly coupled systems. Dependence structures between the two processes (time series) can be studied using the simple mutual information I(y; xτ ), where we use y for y(t) and xτ for x(t + τ ). I(y; xτ ) measures the average amount of information contained in the process {Y } about the process {X} in its future τ time units ahead (τ -future thereafter). This measure, however, as well as other dependence and predictability measures, could also contain information about the τ -future of the process {X} contained in this process itself if the processes {X} and {Y } are not independent, i.e., if I(x; y) > 0. For inferring causality relations, i.e. the directionality of coupling between the processes {X(t)} and {Y (t)}, we need to estimate the “net” information about the τ -future of the process {X} contained in the process {Y } itself using an appropriate tool – the conditional mutual information I(y; xτ |x). It has been shown [11, 12] that using I(y; xτ |x) and I(x; yτ |y) the coupling directionality can be inferred from time series measured in coupled, but not yet fully synchronized systems. Consider now that the processes {X} and {Y } can be modelled by weakly coupled oscillators and that their interactions can be inferred by analyzing the dynamics of their instantaneous phases φ1 (t) and φ2 (t) [13, 14]. The latter can be estimated from the measured

322

Milan Paluˇs, Bojan Musizza and Aneta Stefanovska 1

(a)

RDI(φR,φC)

0.5

0

-0.5

-1

(b)

0.6

RDI(φR,φC)

0.4 0.2 0 -0.2 0

20

40 TIME [MIN.]

60

Figure 1. (a) The relative directionality index (RDI) according to Rosenblum at al. [13, 14], and (b) RDI obtained from conditional mutual information. In each case the results were computed in a moving window of 80 second length moved by 20 second steps from the instantaneous phases of the respiratory rhythm recorded from rat #7 and the phases of the cardiac rhythm recorded from rat #9. time series {x(t)} and {y(t)}, e.g. by application of the discrete Hilbert transform [1, 31]. Rather than simply substituting the series {x(t)} and {y(t)} by the phases φ1 (t) and φ2 (t), which are confined in interval [0, 2π) or [−π, π), we consider phase increments ∆τ φ1,2 (t) = φ1,2 (t + τ ) − φ1,2 (t),

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

and the conditional mutual information I(φ1 (t); ∆τ φ2 (t)|φ2 (t)) and I(φ2 (t); ∆τ φ1 (t)|φ1 (t)), in a shorter notation I(φ1 ; ∆τ φ2 |φ2 ) and I(φ2 ; ∆τ φ1 |φ1 ). Now, in analogy with Rosenblum et al. [13, 14] we define the relative directionality index RDI(1, 2) =

i(1 → 2) − i(2 → 1) , i(1 → 2) + i(2 → 1)

(2)

where the measure i(1 → 2) of how the system 1 drives the system 2 is either equal to the conditional mutual information I(φ1 ; ∆τ φ2 |φ2 ) for a chosen time lag τ , or to an average of I(φ1 ; ∆τ φ2 |φ2 ) over a selected range of lags τ . In full analogy we define i(2 → 1) using I(φ2 ; ∆τ φ1 |φ1 ). RDI(1, 2) should be positive if the driving of system 2 by system 1 prevails, going to 1 for simple unidirectional driving, and negative, going to -1, for the opposite case. In the following we will also consider the absolute directionality index ADI(1, 2) = i(1 → 2) − i(2 → 1), where i(1 → 2) and i(2 → 1) are defined as above.

(3)

Testing for Coupling Asymmetry Using Surrogate Data

3.

323

The Numerical Test Data

The coupled R¨ossler systems x˙ 1 = −ω1 x2 − x3 + ǫ1 (y1 − x1 ), x˙ 2 = ω1 x1 + 0.15x2 , x˙ 3 = 0.2 + x3 (x1 − 10),

y˙ 1 = −ω2 y2 − y3 + ǫ2 (x1 − y1 ), y˙ 2 = ω2 y1 + 0.15y2 , y˙ 3 = 0.2 + y3 (y1 − 10), with ω1 = 1+∆ω, ω2 = 1−∆ω, ∆ω = 0.015, will be used here for generation of test data. Setting ǫ1 = ǫ2 we obtain symmetrically coupled systems, while setting ǫ1 = 0 we obtain asymmetric coupling in which the first systems evolves autonomously and also drives the second system.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

4.

Surrogate Data

The method of “surrogate data” [22] was introduced as a tool for identification of nonlinearity in experimental time series by testing the null hypothesis that the data were generated by a linear stochastic process [22, 24, 25]. Typically, in this approach, one computes a nonlinear statistic from the data under study and from an ensemble of realizations of a linear stochastic process, which mimics “linear properties” of the studied data. If the computed statistic for the original data is significantly different from the values obtained for the surrogate set, one can infer that the data were not generated by a linear process. For the purpose of such tests the surrogate data must preserve the spectrum, and consequently the autocorrelation function of the series under study [22]. In cases of data with distributions different from Gaussian, histogram transformations (see [25] and references therein) can be used for constructing the surrogate data with histograms approximately equal to the histogram of the test data. In the multivariate case, cross-correlations between all pairs of variables must also be preserved [23, 26]. Realizations of a linear stochastic process isospectral with the tested series, i.e. the surrogate data with the same sample spectrum as the tested time series, can be constructed using the Fourier transform (FT). The FT of the series is computed, the magnitudes of the (complex) Fourier coefficients are kept unchanged, but their phases are randomized. The surrogate series is then obtained by computing the inverse transform into the time domain. Different realizations of the process are obtained using different sets of the random phases. In the multivariate case, the cross-correlations can be preserved by preserving the original phase differences between the variables, i.e. the phases are randomized by adding random

324

Milan Paluˇs, Bojan Musizza and Aneta Stefanovska

numbers so that, for a particular frequency bin, the same random number is added to related phases of all the variables [23]. Recently, the surrogate data approach has been generalized to create statistical tool which turns the question of detection into a problem of hypothesis testing [27, 28, 29, 30]. Searching for the presence of a phenomenon, or its typical features in experimental data, one formulates a null hypothesis that the phenomenon is not present in the data under study. The null hypothesis is realized in the set of surrogate data which replicate all possible properties of the studied data but the features of the phenomenon sought. For instance, searching for oscillatory phenomena (quasiperiodic cycles) in a pink-noise background, an autoregressive process of order 1 is fitted onto the tested data, reproducing its correlation structure. It is, however, unable to support cyclic correlations [27, 28]. Then, a suitable statistic is chosen which quantitatively characterizes the phenomenon to be detected and is computed from the studied data and from the set of the surrogate data. If the value of the statistic obtained from the tested data is significantly different from the range of the values obtained from the surrogate data, the null hypothesis (non-existence of the phenomenon) is rejected. The rejection of the null hypothesis is usually considered as evidence for the presence of the phenomenon being sought in the analyzed data. The latter conclusion is not always correct, however, since departure from the surrogate range of the used statistic can also be caused by a different phenomenon than the one being tested for. Besides the sensitivity of the test used, its specificity should also be considered. The possibility of constructing multivariate surrogate data [23, 26] allows for testing for the presence of nonlinearity in bivariate and multivariate time series. Besides the general question of nonlinearity, surrogate data have also been used for testing coherence [32] and phase synchronization [31, 33]. The question of specificity in the latter case, i.e. the distinction between phase synchronization and simple association by linear correlations, is discussed in [33].

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

5.

Inference of Coupling Asymmetry

The topic of this paper is the detection of asymmetry in coupling. Can we use the approach of hypothesis testing based on the evaluation of surrogate data? The proper null hypothesis for such a test should be symmetry of the coupling. I.e. if we test bivariate data from a particular system, inference of possible asymmetry should be based on testing using surrogate data generated by the same system(s) in a state when the coupling is symmetric. Such “ideal” surrogate data for this purpose can easily be constructed in our “in silico” study using the above-defined coupled R¨ossler systems and, potentially, in some laboratory experiments e.g. using coupled electronic oscillators. In practice, however, the question of coupling asymmetry and corresponding causal relationships between systems is much more important in “in vivo” biomedical studies, when the only possibility of constructing surrogate data is through numerical manipulation of the tested data themselves. In such a way we can preserve the statistical properties (distributions of amplitudes and frequencies) and auto- and cross-correlations of the original data, but we cannot reproduce possible nonlinear properties and dependences, nor mimic possible nonlinear symmetric coupling. Therefore, for an asymmetry test, proper surrogate data usually cannot be constructed. In this numerical study we compare the performance of tests with “ideal” surrogate data with tests

Testing for Coupling Asymmetry Using Surrogate Data

I(φ1;φ2)

2

(a)

1

I(φ1;φ2)

0 2

(b)

1

I(φ1;φ2)

0 2

(c)

1

0 2 I(φ1;φ2)

325

(d)

1

0 0

0.01

0.02 COUPLING STRENGTH ε

0.03

Figure 2. A measure of phase synchronization – the (symmetric) mutual information I(φ1 ; φ2 ) of the phases of the symmetrically coupled R¨ossler systems (solid lines) and the surrogate I(φ1 ; φ2 ) ranges (mean±2SD, dashed lines) for (a) the “ideal” SRS and stochastic (b) 2FT2, (c) 1FT2 and (d) 1FT1 surrogate data. using standard stochastic (FT phase randomized) surrogate data which can, sensu stricti, be considered improper when the coupling asymmetry should be inferred.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

6.

The Test Data and the Surrogate Data

The test data for our study are obtained by integration of the coupled R¨ossler systems (Sec. 3.) in both the cases of symmetric (ǫ1 = ǫ2 = ǫ) and asymmetric (ǫ1 = 0, ǫ2 6= 0) coupling. The data, after they had been generated, were mixed with various amounts of noise. Independently generated realizations of Gaussian white noise were used, the noise content is given by the ratios of the standard deviations of the noise and the data. The instantaneous phases φ1 and φ2 were then estimated using the Hilbert transform [31], and the absolute and relative directionality indices were computed from phase time series of various lengths. The simple mutual information I(φ1 ; φ2 ) of the phases indicates the transition to phase synchronization by its steep increase [31]. The dependence of I(φ1 ; φ2 ) on the coupling parameter ǫ for symmetric and ǫ2 for asymmetric couplings is illustrated by the solid lines in Figs. 2 and 3, respectively. Since the directionality indices work for coupled but not yet fully synchronized systems [15], the test data were chosen for values of coupling parameters below the synchronization threshold. SRS (Symmetric R¨ossler Surrogates) – the “ideal” surrogate data for the test data generated by the symmetrically coupled R¨ossler systems were obtained by integrating the same systems but the coupling parameter was, for each realization of the surrogate data, changed

326

Milan Paluˇs, Bojan Musizza and Aneta Stefanovska (a)

I(φ1;φ2)

2 1 0

(b)

I(φ1;φ2)

2 1 0

(c)

I(φ1;φ2)

2 1 0

(d)

I(φ1;φ2)

2 1 0 0

0.05 COUPLING STRENGTH ε2

0.1

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 3. A measure of phase synchronization – the (symmetric) mutual information I(φ1 ; φ2 ) of the phases of the asymmetrically coupled R¨ossler systems (solid lines) and the surrogate I(φ1 ; φ2 ) ranges (mean±2SD, dashed lines) for (a) the “ideal” SRS and stochastic (b) 2FT2, (c) 1FT2 and (d) 1FT1 surrogate data. by a small random fluctuation around the value of ǫ of the test data. In the case of test data generated by the asymmetrically coupled R¨ossler systems, the “ideal” surrogate data were generated again by the symmetrically coupled R¨ossler systems with the value of ǫ chosen so that the symmetric dependence between the components of the (bivariate) surrogate data was the same as in the test data. In particular, I(φ1 ; φ2 ) of the surrogate data was close to I(φ1 ; φ2 ) of the test data. Note that the dependence of I(φ1 ; φ2 ) on the coupling parameter in the asymmetric case is different from that in the symmetric case, Figs. 2, 3. Variation of I(φ1 ; φ2 ) for the surrogate data, in particular, its mean±2SD (standard deviations) is illustrated in Figs. 2 and 3 by the dashed lines. Besides the “ideal” surrogate data, generated by the symmetrically-coupled R¨ossler systems, we also used the following types of the surrogate data, obtained by numerical manipulation (randomization) of particular test data: 2FT2 – Bivariate FT surrogates were generated by simultaneous randomization of Fourier phases of both components of the bivariate test series so that the individual spectra and autocorrelation functions, as well as the cross-spectrum and the cross-correlation of the test data, were preserved in the surrogates. 1FT2 – Independent univariate FT surrogates – FT phases of both components of the bivariate test series were randomized independently, so that the spectra (frequency content) of individual series were preserved in the surrogates, but possible mutual dependencies were destroyed. 1FT1 – FT “semi”surrogates – FT phases of only one of the two components of the test

Testing for Coupling Asymmetry Using Surrogate Data (a)

0.5 RDI

327

0 -0.5

(b)

RDI

0.5 0 -0.5

(c)

RDI

0.5 0 -0.5

(d)

RDI

0.5 0 -0.5 0

0.01 COUPLING STRENGTH ε

0.02

Figure 4. The relative directionality index RDI computed from the phases of the symmetrically coupled R¨ossler systems (solid lines) and the surrogate RDI ranges (mean±2SD, dashed lines) for (a) the “ideal” SRS and stochastic (b) 2FT2, (c) 1FT2 and (d) 1FT1 surrogate data. data were randomized, the other component being kept unchanged.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

7.

Measuring and Testing the Coupling Asymmetry

Dependence on the coupling parameter ǫ of the relative directionality index (RDI) for the test data generated by the symmetrically coupled R¨ossler systems (solid lines) as well as the surrogate ranges (mean±2SD, dashed lines) are illustrated in Fig. 4. As expected, RDI of the test data vanishes to zero, while the variance of the surrogates is larger. The 1FT1 surrogates have the largest variance. Here we can see an unfortunate property of the RDI – the normalization procedure (2) amplifies small departures from zero, leading to larger counts of false positive results, and attenuates large departures from zero, tending to increase the count of false negative results. In all the subsequent tests the absolute directionality index (ADI) has performed better. Therefore, for the sake of brevity, we report below only the results obtained by using ADI. The dependence of ADI on ǫ for the test data generated by the asymmetrically coupled R¨ossler systems (solid lines) as well as the surrogate ranges (mean±2SD, dashed lines) are illustrated in Fig. 5. The ADI of the test data rises at first as ǫ increases from zero, but then falls to zero at the synchronization threshold (cf. Fig. 3 and Fig. 5). As noted above, the coupling asymmetry cannot be inferred after the transition into the synchronized state, if the systems are not further perturbed. The variance of ADI of all the surrogates is in absolute terms very small, so there is no sense in formal statistical testing. This result (Fig. 5), however, was obtained from noise-free test data with a series of length 64k (65536

328

Milan Paluˇs, Bojan Musizza and Aneta Stefanovska (a)

ADI

0.4 0.2 0

(b)

ADI

0.4 0.2 0

(c)

ADI

0.4 0.2 0

(d)

ADI

0.4 0.2 0 0

0.05 COUPLING STRENGTH ε2

0.1

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 5. The absolute directionality index ADI computed from the phases of the asymmetrically coupled R¨ossler systems (solid lines) and the surrogate ADI ranges (mean±2SD, dashed lines) for (a) the “ideal” SRS and stochastic (b) 2FT2, (c) 1FT2 and (d) 1FT1 surrogate data. samples). The situation changes using noisy data – Fig. 6 presents the same results as Fig. 5, but with 60% of Gaussian noise added to the test data. Noise was added to SRS after their generation by integrating the symmetrically coupled R¨ossler systems. In the case of all the FT surrogates, the surrogate data were generated from noisy test data. We can see that, with such noisy data, the coupling asymmetry cannot be detected in most cases of coupling using the FT surrogates, nor using the ideal SRS in the cases of weak coupling. In order to evaluate quantitatively the performance of the various surrogate data used, we will use standard measures for evaluation of the statistical tests. The type I error measures the probability of false positive results – in our case it will give the relative number of tests in which the null hypothesis of symmetric coupling is rejected although it is true, i.e. when the test data were generated by the symmetrically coupled R¨ossler systems. A set of 100 realizations of test data were generated and tests performed for various series lengths and amounts of noise, using 30 surrogate realizations of each type of surrogate in each test. Using the nominal probability α = 0.05 for the significance criterion, considering the normal distribution of the statistic, we rejected the null hypothesis if the value of ADI for the test data was larger than mean + 1.96SD (or smaller than mean - 1.96SD) of the surrogate ADI. The size of the test, i.e., the actual probability p of false rejections given the nominal α=0.05, as a function of the series lengths for various amounts of noise in the data is presented in Fig. 7. The size of the test using the ideal symmetric R¨ossler surrogates (Fig. 7a) is independent of the amount of data and noise and a bit worse than the nominal α=0.05. This could possibly be improved by increasing the fluctuations of the coupling parameter in the generation of these surrogates, but this would be at the cost of test sensitivity (see be-

Testing for Coupling Asymmetry Using Surrogate Data

ADI

0.02

329

(a)

0.01 0 -0.01

ADI

-0.02 0.02

(b)

0.01 0 -0.01

ADI

-0.02 0.02

(c)

0.01 0 -0.01

ADI

-0.02 0.02

(d)

0.01 0 -0.01 -0.02 0

0.05 COUPLING STRENGTH ε2

0.1

Figure 6. The absolute directionality index ADI computed from the phases of the noisy (60% of noise given the ratio of noise/data standard deviations) asymmetrically coupled R¨ossler systems (solid lines) and the surrogate ADI ranges (mean±2SD, dashed lines) for (a) the “ideal” SRS and stochastic (b) 2FT2, (c) 1FT2 and (d) 1FT1 surrogate data.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

low). The test size for the bivariate 2FT2 surrogates (Fig. 7b) for short time series exceeds the nominal value 0.05, but for series length over 1000 samples the actual p falls sharply to zero. The other FT surrogates give similar sizes of tests. So the stochastic FT surrogates can prevent us from making false identification of coupling asymmetry, providing we do not use too-short time series for testing. The other question is the sensitivity of the test. The type II error characterizes the probability of obtaining a false negative result, i.e. that the actual coupling asymmetry is neglected. Again, we did the same testing as in the previous case, but the test data were generated by the asymmetrically coupled R¨ossler systems (ǫ1 = 0). For all types of surrogate data, and various amounts of data and noise, we counted the probability p of false negative results. The value 1 − p, known as the power of the test, is presented in Fig. 8, again as a function of the series lengths for various amounts of noise in the data. For short time series (128 samples) the tests usually fail to detect the coupling asymmetry, even using the ideal SRS surrogates (Fig. 8a). The test power improves with increasing the length of the series: the higher the amount of noise, the more data are required. For noise-free data, or data with 5–10% of noise, series of lengths exceeding 256 or 512 samples are sufficient; with more noise thousands or tens of thousands of samples are required. The performance of the bivariate 2FT2 surrogate (Fig. 8b) is, as could have been expected, worse. For very noisy data (50–60% of noise) the power of test is close to zero even for long time series. However, for moderately noisy data (10-30%) and series lengths of over a thousand samples, the test size quickly approaches the size of the tests with SRS. Comparisons of the power of the tests for all four types of surrogate data, and for 10% and 20% of noise in the test data, are presented

330

Milan Paluˇs, Bojan Musizza and Aneta Stefanovska

in Figs. 9a and 9b, respectively. Of course, the ideal symmetric R¨ossler surrogates have the best performance. From the stochastic FT surrogates the bivariate 2FT2 surrogates have the best performance, while the 1FT1 surrogates have the worst performance. Nevertheless, for data with 10% of noise, all surrogates give a test power of 1, i.e. the coupling asymmetry is detected in 100% of tests using the series length of 1k (1024) and more samples. 0.2 (a)

(b)

+√ ADI: SIZE OF TEST

0.15

0.1

0.05

0

h h h d√ d g f + h +○ g♦

h f

√

d

♦ g

+√ d

d h + ♦ + f f g ♦ d ○ ♦ ○ d h f ○ √ f + ○ d ♦ ♦ h + f ○ h f g g ○ h g d√ √ f ♦√ ○ g g +√ d + + f h ○ d ♦ ♦ +♦√ +√ √ d g d d d d g ○ h ♦ f g ○ f g h√ h ○ ♦ f ○ ♦ f g ○ ♦ f g ○ ♦ f g ○ + +√ h +√ h √

1000 10000 1000 10000 SERIES LENGTH [LOG-SAMPLES]

Figure 7. The size of the test – the probability of a false positive result using ADI and the “ideal” SRS (a) and stochastic 2FT2 (b) surrogate data in dependence on the series length (number of samples, logarithmic scale) and for different noise contents: 0% and 5% coinciding: solid line with plus signs, 10%: dash-dotted line with squares, 20%: dashed line with octagons, 30%: thin solid line with diamonds, 40%: dotted line with fancy diamonds, 50%: thin dash-dotted line with fancy squares, and 60%: thin dashed line with fancy crosses.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

8.

Discussion and Conclusion

In this numerical study we used test data generated by symmetrically and asymmetrically coupled R¨ossler systems in order to measure the performance of statistical tests used for detecting possible asymmetry in coupling. The tests are based on the rejection of a null hypothesis of symmetric coupling. The correct null hypothesis can only be realized in numerical or laboratory experiments. When processing experimental data such as those from physiological experiments or biomedical studies, using numerical manipulation – randomization procedures – we are able, in the better case, to reproduce linear relationships in the data. In many instances, however, only the distributions of values and frequencies of the data can be reproduced in the constructed surrogate data. By comparing the performance of the tests using the stochastic FT surrogates and the “ideal surrogates” representing the correct null hypothesis – realizations of evolution of the symmetrically coupled R¨ossler

Testing for Coupling Asymmetry Using Surrogate Data (a)

ADI: POWER OF TEST

(b)

+√ +d○√ d+○√ ♦○d+f√ gf♦○d+√ hgf♦○d+√ hgf♦○d+√ ♦ h d g f ♦ h g

1

+√ d+√ +d√ d+○√ +d√ ○d+♦√ ♦○d+√ ○ ○ f

0.5

○

○ h 0

gf♦ ○d+√

f h ♦ gh fh g

331

d g fh+ ♦○ d√

♦ gh gh g f g f♦○ hgf♦ f♦ hgf hf♦ hg h

1000 10000 1000 10000 SERIES LENGTH [LOG-SAMPLES]

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 8. The power of the test 1 − p, where p is the probability of a false negative result using ADI and the “ideal” SRS (a) and stochastic 2FT2 (b) surrogate data in dependence on the series length (number of samples, logarithmic scale) and for different noise contents: 0% and 5% coinciding: solid line with plus signs, 10%: dash-dotted line with squares, 20%: dashed line with octagons, 30%: thin solid line with diamonds, 40%: dotted line with fancy diamonds, 50%: thin dash-dotted line with fancy squares, and 60%: thin dashed line with fancy crosses.

systems, we demonstrated that for moderately noisy data and not too short time series the performance of the tests using the stochastic FT surrogates and the “ideal surrogates” are comparable. The size of the test can even be better in the case of stochastic FT surrogates. Thus the statistical evaluation of the coupling asymmetry measures is possible and necessary, since it can prevent us from making false detections of coupling asymmetry. Only in analyses of very noisy data sets of insufficient length the detection ability of our tests is lost. Therefore any inference of coupling asymmetry from experimental data should be accompanied by a surrogate data test. Coming back to our introductory example (Fig. 1), we present in Figure 10 the results of a part of the relevant surrogate data tests. Here, the phases of the respiratory and cardiac oscillations were obtained using the interbeat (RR) and interbreath intervals. These were randomly permutated prior to computing the surrogate phases (see also [15]). This type of surrogate data is roughly equivalent to the above 1FT2 surrogates. In order to have enough data, we perform the test in a large segment of approx. 30 minute length. Instead of rejection of the null hypothesis using the mean and SD of the surrogate data, we evaluated a larger number of surrogate realizations (2500) and estimated histograms of the distribution of the absolute directionality index ADI of the surrogates (in Fig. 10, the ADI values for the test data are marked by vertical lines.) In Fig. 10a, a segment 0–30 min. (cf. Fig. 1) using the respiratory data from rat #7 and the cardiac data from rat #9 is evaluated. Although the

332

Milan Paluˇs, Bojan Musizza and Aneta Stefanovska (a)

ADI: POWER OF TEST

1

0.5

d

d db db○♦ bd○♦ ○b♦d ○b♦d b ○♦

○♦bd ○b♦d ○b♦d ○b♦d ○b♦d ○b♦d

b ♦

0

(b)

○ db ○ ♦

d

♦ ○

○d b♦ ○b♦

1000 10000 1000 10000 SERIES LENGTH [LOG-SAMPLES]

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 9. The power of the test 1 − p, where p is the probability of a false negative result using ADI, applied on the data with 10% (a) and 20% (b) of noise in dependence on the series length (number of samples, logarithmic scale) using the ideal SRS (solid line with squares), bivariate 2FT2 (dash-dotted line with crosses), univariate 1FT2 (dotted line with diamonds) and 1FT1 (dashed line with octagons) surrogate data.

ADI for the test data is positive, it is well inside the distribution of the surrogate ADI, i.e. the ADI value is not significant and the null hypothesis of symmetry (or independence) is not rejected. Correctly, no coupling asymmetry or no causality was inferred in this case. In order to demonstrate also a correct positive result, the cardiorespiratory interaction in the data of rat #7 is presented in Fig. 10b. Here the ADI from the test data clearly departs from the ADI surrogate distribution into larger positive values, indicating the fact that the respiratory rhythm influences the cardiac oscillation. Note that neither of the surrogate ADI histograms is symmetrically distributed around zero but biased to positive values. This bias is caused by different dominant frequencies of the two processes under study – the asymmetry measure is biased in the sense that the slower process influences the faster one even if there is no real causality or no real interaction. Only the statistical evaluation of the asymmetry measure can prevent us from making false inference of causality on one side (Fig 10a), or confirm the reality of causality on the other (Fig. 10b), even though the surrogate data used are not proper in a strict sense, i.e. they realize independent, rather than symmetrically coupled processes. The most important fact is that the surrogate data mimic the statistical properties of the tested data which are the source of the bias and variance in the asymmetry measure estimation.

Testing for Coupling Asymmetry Using Surrogate Data

333

0.05 (a)

(b)

BIN PROBABILITY

0.04

0.03

0.02

0.01

0 0

0.001 ADI(φR,φC)

0

0.002

Figure 10. Inference of coupling asymmetry in cardio-respiratory interactions in the animal anæsthesia experiment. ADI computed from phases in 30 minute segments: the ADI value from the test data is marked by the vertical lines, the ADI surrogate distribution is estimated as 100-bin histogram using 2500 surrogate data realizations. (a): results from combination of the respiratory data of rat #7 and the cardiac data of rat #9 (cf. Fig. 1); (b): the actual test using the data of rat #7.

Acknowledgements

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

The authors would like to thank Prof. P.V.E. McClintock for careful reading of the manuscript. The study was supported by the EC FP6 project BRACCIA (Contract No 517133 NEST), by the bilateral Czech and Slovenian Ministries of Education and Science project KONTAKT No.1/2005-6, and in part by the Institutional Research Plan AV0Z10300504.

References [1] Pikovsky, A., Rosenblum, M. & Kurths, J. (2001). Synchronization. A Universal Concept in Nonlinear Sciences. Cambridge: Cambridge University Press. [2] Sch¨afer, C., Rosenblum, M.G., Kurths, J. & Abel, H.-H. (1998). Heartbeat synchronized with ventilation. Nature, 392, 239–240. [3] Sch¨afer, C., Rosenblum, M.G., Kurths, J. & Abel, H.-H. (1999). Synchronization in the human cardiorespiratory system. Phys. Rev. E, 60 857–870. [4] Paluˇs, M. & Hoyer, D. (1998). Detecting nonlinearity and phase synchronization with surrogate data. IEEE Engineering in Medicine and Biology, 17(6), 40–45.

334

Milan Paluˇs, Bojan Musizza and Aneta Stefanovska

[5] Braˇciˇc Lotriˇc, M. & Stefanovska, A. (2000). Synchronization and modulation in the human cardiorespiratory system. Physica A, 283(3-4), 451–461. [6] Stefanovska, A., Haken, H., McClintock, P. V. E., Hoˇziˇc, M., Bajrovi´c, F. & Ribariˇc, S. (2000). Reversible transitions between synchronization states of the cardiorespiratory system. Phys. Rev. Lett., 85(22), 4831–4834. [7] Jamˇsek, J., Stefanovska, A., & McClintock P. V. E. (2004). Nonlinear cardiorespiratory interactions revealed by time-phase bispectral analysis. Phys. Med. Biol., 49, 4407–4425. [8] Schiff, S.J., So, P., Chang, T., Burke, R.E. & Sauer, T. (1996). Detecting dynamical interdependence and generalized synchrony through mutual prediction in a neural ensemble. Phys. Rev. E, 54, 6708–6724. [9] Le Van Quyen, M., Martinerie, J., Adam, C. & Varela, F.J. (1999). Nonlinear analyses of interictal EEG map the brain interdependences in human focal epilepsy. Physica D, 127, 250–266. [10] Tass, P., Rosenblum, M.G., Weule, J., Kurths, J., Pikovsky, A., Volkmann, J., Schnitzler, A. & Freund, H.-J. (1998). Detection of n:m phase locking from noisy data: Application to Magnetoencephalography. Phys. Rev. Lett., 81, 3291–3294. ˇ erbov´a, K. (2001). Synchronization as Adjust[11] Paluˇs, M., Kom´arek, V., Hrnˇc´ıˇr, Z. & Stˇ ment of Information Rates: Detection from Bivariate Time Series. Phys. Rev. E, 63, 046211. ˇ erbov´a, K. (2001). Synchro[12] Paluˇs, M., Kom´arek, V., Proch´azka, T., Hrnˇc´ıˇr, Z. & Stˇ nization and Information Flow in EEG of Epileptic Patients. IEEE Engineering in Medicine and Biology Magazine, 20(5), 65–71.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

[13] Rosenblum, M.G. & Pikovsky, A.S. (2001). Detecting direction of coupling in interacting oscillators. Phys. Rev. E, 64, 045202(R). [14] Rosenblum, M.G., Cimponeriu, L., Bezerianos, A., Patzak, A. & Mrowka, R. (2002). Identification of coupling direction: Application to cardiorespiratory interaction. Phys. Rev. E, 65, 041909. [15] Paluˇs, M. & Stefanovska, A. (2003), Direction of coupling from phases of interacting oscillators: An information-theoretic approach. Phys. Rev. E, 67, 055201R. [16] Shannon, C.E. & Weaver, W. (1964). The Mathematical Theory of Communication. Urbana, IL: University of Illinois Press. [17] Gallager, R.G. (1968). Information Theory and Reliable Communication. New York, N.Y.: Wiley. [18] Khinchin, A.I. (1957). Mathematical Foundations of Information Theory. New York, N.Y.: Dover Publications.

Testing for Coupling Asymmetry Using Surrogate Data

335

[19] Billingsley, P. (1965). Ergodic Theory and Information. New York, N.Y.: Wiley. [20] Kullback, S. (1959). Information theory and statistics. New York, N.Y.: Wiley. [21] Cover, T.M. & Thomas, J.A. (1991). Elements of Information Theory. New York, N.Y.: Wiley. [22] Theiler, J., Eubank, S., Longtin, A., Galdrikian, B. & Farmer, J.D. (1992). Testing for nonlinearity in time series: the method of surrogate data. Physica D, 58, 77–94. [23] Prichard, D. & Theiler, J. (1994). Generating surrogate data for time series with several simultaneously measured variables. Phys. Rev. Lett., 73, 951–954. [24] Schreiber, T. & Schmitz, A. (2000). Surrogate time series, Physica D, 142(3-4), 346–382. [25] Paluˇs, M. (1995). Testing for nonlinearity using redundancies: quantitative and qualitative aspects. Physica D, 80, 186–205. [26] Paluˇs, M. (1996). Detecting nonlinearity in multivariate time series. Phys. Lett. A, 213 138–147. [27] Allen, M. R., Smith, L. A. (1996). Monte Carlo SSA: Detecting irregular oscillation in the presence of colored noise. J. Climate, 9(12), 3373–3404. [28] Paluˇs, M., Novotn´a, D. (1998). Detecting modes with nontrivial dynamics embedded in colored noise: Enhanced Monte Carlo SSA and the case of climate oscillations. Phys. Lett. A, 248, 191–202. [29] Paluˇs, M. & Novotn´a, D. (1999). Sunspot cycle: A driven nonlinear oscillator? Phys. Rev. Lett., 83(17), 3406–3409.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

[30] Paluˇs, M. (2001). Detection of a nonlinear oscillator underlying experimental time series: the sunspot cycle. In: Mees, A.I., editor. Nonlinear Dynamics and Statistics. (pp. 453-473). Boston: Birkhauser. Preprint available at: http://www.cs.cas.cz/˜mp/papers/palus.ps [31] Paluˇs, M. (1997). Detecting phase synchronization in noisy systems. Phys. Lett. A, 235, 341–351. [32] Faes, L., Pinna, G.D., Porta, A., Maestri, R. & Nollo, G. (2004). Surrogate data analysis for assessing the significance of the coherence function. IEEE Trans. Biomed. Eng. 51(7), 1156-1166. [33] Dolan, K.T. & Neiman, A. (2002). Surrogate analysis of coherent multichannel data. Phys. Rev. E, 65(2), 026108. [34] Hurtado, J.M., Rubchinsky, L.L. & Sigvardt, K.A. (2004). Statistical method for detection of phase-locking episodes in neural oscillations. J. Neurophysiol., 91(4), 18831898.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

In: Chaos and Complexity: New Research Editors: F.F. Orsucci and N. Sala, pp. 337-355

ISBN: 978-1-60456-841-7 © 2009 Nova Science Publishers, Inc.

Chapter 25

WHAT CAN WE LEARN FROM SINGLE-TRIAL EVENT-RELATED POTENTIALS? R. Quian Quiroga1*, M. Atienza2, J.L. Cantero2 and M.L.A. Jongsma3,4 1

Department of Engineering, University of Leicester, UK Laboratory of Functional Neuroscience, Universidad Pablo de Olavide, Seville, Spain 3 NICI - Department of Biological Psychology, Radboud University, Nijmegen, The Netherlands 4 Department of Cognitve Psychology and Ergonomics, University of Twente, Enschede, The Netherlands 2

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Abstract We present a method for visualizing single-trial evoked potentials and show applications of the consequent single-trial analysis. The method is based on the wavelet transform, which has an excellent resolution both in the time and frequency domains. Its use provides new information that is not accessible from the conventional analysis of peak amplitudes and latencies of average evoked potentials. We review some of the applications of the single trial analysis to the study of different cognitive processes. First, we describe systematic trial-totrial changes reflecting habituation and sensitization processes. Second, we show how an analysis of trial-to-trial latency variability gives new insights on the mechanisms eliciting a larger mismatch negativity in control subjects, in comparison to sleep deprived subjects when performing a pattern recognition learning task. Third, we show in a rhythm perception task that trained musicians had lower latency jitters than non-musicians, in spite of the fact that there were no differences in the average responses. We conclude that the single trial analysis of evoked potentials opens a wide range of new possibilities for the study of cognitive processes.

*

E-mail address: [email protected], Tel: +44 116 252 2314, Fax: +44 116 252 2619. Dept. of Engineering. University of Leicester, UK.( Corresponding author)

338

R. Quian Quiroga, M. Atienza, J.L. Cantero et al.

1. Introduction

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

It is common practice to study electroencephalographic (EEG) responses, recorded by scalp electrodes, to different types of sensory stimulation. These evoked, or more generally, eventrelated potentials (ERPs), are very small in comparison with the ongoing electroencephalogram (EEG) and are barely visible in the individual trials. Therefore, most ERP research relies on the identification of different waves after averaging several presentations of the same stimulus pattern. From the average responses it is possible to identify evoked components, whose amplitudes, latencies and topography have been successfully correlated with different sensory and cognitive functions in both the healthy and disordered brain (Regan, 1989; Quian Quiroga, 2006). Although ensemble averaging of individual cerebral responses improves the signal-tonoise-ratio, it relies on the basic assumptions that the evoked responses constitute an invariant pattern that is perfectly locked to the stimulus (assumption 1), laying on an independent stationary and ergodic stochastic background EEG signal (assumption 2) (beim Graben et al, 2000; Frisch et al, 2004). These assumptions are in strict sense not valid. In particular, it has been shown that the spectral content of the background EEG at the time of stimulation does have a strong influence on the ERP waveform (Jongsma et al., 2000a; 2000b). And, more importantly, averaging implies a loss of information related to systematic or unsystematic variations between the single-trials. These variations might affect the reliability of the average ERP as a representation of the single trial responses and such information may be crucial to study the time course of dynamic cognitive processes, simple and complex behavioral patterns and cognitive dysfunctions in pathological conditions. Growing evidence shows the important contributions of single-trial ERP analysis to cognitive neuroscience (see e.g. Quian Quiroga, 2006). From a physiological perspective, one might expect that neural responses are modified after several repetitions of the same stimulation pattern, or that they change during the emergence and consolidation of new brain representations, as occurs during learning processes. Single-trial analysis techniques are particularly suitable to gain insights into the time course of neural responses associated to cognitive acts. The present study is aimed at describing a denoising method that is applied to single brain responses. In the following sections, recent contributions of the subsequent single-trial analysis to cognitive processing will be showed, particularly stressing how the tracking of the single-trial responses allows the study of neural phenomena inherent to habituation, learning, and memory processes.

2. Wavelet Transform The wavelet transform of a signal x(t) is defined as the inner product between the signal and the wavelet functions Ψa,b (t)

Wψ x(a, b) = x(t ), ψ a ,b (t ) where Ψa,b (t) are dilated (contracted) and shifted versions of a unique wavelet function Ψ(t)  (usually called “mother wavelet”)

What Can We Learn from Single-Trial Event-Related Potentials?

ψ a ,b (t ) = a

−1 / 2

339

⎛t −b⎞ ⎟ ⎝ a ⎠,

ψ⎜

(a,b are the scale and translation parameters, respectively). In brief, the wavelet transform gives a time-frequency representation of a signal that has two main advantages over previous methods: a) an optimal resolution both in the time and in the frequency domains, adapted for each frequency; b) the lack of the requirement of stationarity of the signal. These advantages are particularly suited for the analysis of ERPs, since these brain responses show multiple frequency components with different time localizations (Quian Quiroga et al, 2001). In order to avoid redundancy and to increase the efficiency of algorithm implementations, the wavelet transform is usually defined at discrete scales a and discrete times b by choosing the dyadic a

j

= 2−j, b

= 2−jk

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

j ,k , for integers j and k. The wavelet transform gives set of parameters a decomposition of x(t) in different scales, tending to be maximum at those scales and time locations where the wavelet best resembles x(t). Contracted versions of Ψa,b(t) will match high frequency components of x(t) and on the other hand, dilated versions will match the low frequency ones.

Figure 1. Single-trials (bottom) and the average ERP response (top) to pattern visual stimulation. Note that evoked responses are clear in the average signal but are hard to be seen in the single-trials.

340

R. Quian Quiroga, M. Atienza, J.L. Cantero et al.

The information provided by the wavelet transform is structured according to a hierarchical scheme called multiresolution analysis (Mallat, 1989). This method gives a decomposition of the signal in different level of ‘details’ (i.e. components in consecutive frequency bands) and a final approximation or ‘residual’ that is the difference between the original signal and the sum of all the details. One main advantage of the multiresolution decomposition is that it can be implemented with recursive and fast algorithms (for details, see Quian Quiroga et. al, 2001 and references therein). Moreover, components corresponding to the different frequency bands can be reconstructed by applying an inverse transform. In the present study, a five level decomposition was used, thus having five scales of details (D1 to D5) and a final approximation (A5). Cubic bi-orthogonal B-Splines (Cohen et al, 1992) were chosen as the basic wavelet functions due to their similarity with the evoked neural responses (thus having a good localization of the ERPs in the wavelet domain), and due to their optimal time-frequency resolution (for more details see Cohen et al, 1992; Chui, 1992; Unser et al, 1992; Quian Quiroga et al, 2001).

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

3. Obtaining Single-Trial ERPs with Wavelet Denoising Figure 1 depicts an event-related cortical response evoked by a checkerboard pattern. This figure and the following ones are the output of a software package for denoising ERPs (EP_den) available from the internet (webpage: www.vis.caltech.edu/~rodri). These waveforms were obtained from a scalp electrode located over the left occipital lobe in response to 16 target stimuli within an oddball paradigm, in which infrequent target stimuli have to be detected within a sequence of frequent (non-target) ones. Non-target stimuli were color reversals of a checkerboard pattern and target stimuli were also color reversal of the checkerboard but with a small displacement. Subjects were instructed to pay attention to the appearance of target stimuli (Quian Quiroga and Schürmann, 1999). Note in the average ERP response the presence of a first positive deflection at 100 ms (P100) followed by a negative rebound at 200 ms (N200). At 400ms we observe a large and slower positive peak, the P300, which is usually elicited by the target stimuli. These evoked responses are clearly seen in the average signal but are hard to identify in each individual trial. Figure 2 shows the 5-scales wavelet decomposition of the average ERP of the previous figure. D1 corresponds to the highest frequency band and A5 to the lowest. Band limit values correspond approximately to: 63-125 Hz (D1), 31-62 Hz (D2), 16-30 Hz (D3), 8-15 Hz (D4), 4-7 Hz (D5) and 0.5-4 Hz (A5). Each coefficient shows the correlation of the signal with a wavelet function at different scales and times. Note that the P100-N200 response is mainly correlated with the first post-stimulus coefficient in the details D4-D5. The P300 waveform is mainly correlated with the coefficients at about 400-500 ms in A5. This correspondence is easily identified considering the following facts: 1) the coefficients appear in the same time (and frequency) range as the ERPs and 2) they are relatively larger than the rest due to phaselocking between trials (coefficients reflecting background oscillations cancel in the average). A straightforward strategy to avoid the fluctuations related with the ongoing EEG is to equal to zero those coefficients that are not correlated with the ERPs. However, the choice of these coefficients should not be exclusively based on the average ERP and should also consider the time ranges in which the single-trial ERPs are expected to occur (i.e. some neighboring coefficients should be included in order to allow for latency variations). The black

What Can We Learn from Single-Trial Event-Related Potentials?

341

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

coefficients are the ones used for denoising P100-N200 and P300 responses. Note that background EEG oscillations were filtered out in the final reconstruction of the average brain response. It is quite difficult to achieve this result by applying filtering approaches based on Fourier transform due to the different time and frequency localizations of the P100-N200 and P300 responses, and also due to the overlapping frequency components of these peaks and the ongoing EEG. Summarizing, the main advantage of wavelet denoising over conventional filtering is that one can select different time windows for the different scales. Once the coefficients of interest are identified from the average ERP, this same denoising can be applied to each single brain response, thus filtering the contribution of background EEG activity.

Figure 2. Wavelet decomposition of the average ERP from the previous figure. D1-D5 and A5 are the scales (i.e. frequency bands) in which the signal is decomposed (wavelet coefficients shown in grey). Note that ERPs are correlated with a few wavelet coefficients (in black), which can be used to denoise the signal. At the top, the original average ERP (grey) and the denoised reconstruction of the average ERP using only these coefficients (black) is shown.

342

R. Quian Quiroga, M. Atienza, J.L. Cantero et al.

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Figure 3 displays a contour plot of the 16 single trials after wavelet denoising. We observe a white pattern followed by a black one between 100-200 ms, corresponding to the P100-N200 complex. The more variable and wider white pattern at about 400-600 ms corresponds to the P300 component. Note that with denoising we can distinguish the P100N200 and the P300 in most of the individual trials. We remark that these responses are not easily identified in the original signal due to their low amplitude and their similarity with the ongoing EEG (Figure 1). This issue has been recently confirmed by using simulated datasets closely resembling real ERPs, which showed that wavelet denoising significantly improves the visualization of the single trial components (and the estimation of their amplitudes and latencies) in comparison with the original data and with previous approaches, such as Wiener filtering (Quian Quiroga and Garcia, 2003).

Figure 3. Single-trial denoising of the ERP shown in the previous figure. Note that all the components are now recognized in the single-trials.

What Can We Learn from Single-Trial Event-Related Potentials?

343

4. Neurophysiological Correlates of Habituation in Rat Auditory Evoked Potentials

Copyright © 2008. Nova Science Publishers, Incorporated. All rights reserved.

Auditory evoked potentials (click stimuli, 1 ms duration) were obtained from 13 adult male albino rats. For each rat, vertex EEG recordings were analyzed during 250 ms pre- and 250 ms post-stimulation in the first 100 trials (for details see Quian Quiroga and van Luijtelaar, 2002; de Bruin et al, 2001).

Figure 4A. Grand average auditory evoked potentials of 13 rats. B: Amplitude changes with trial repetition of the 6 components marked in A. Note the clear exponential decays in the first 4 components.

344

R. Quian Quiroga, M. Atienza, J.L. Cantero et al.

Figure 4A displays the grand average auditory evoked potentials of 13 rats. Note the presence of two positive components, at 13 and 20 ms, and 4 negative ones at 18, 24, 38 and 52 ms, respectively. For these components, amplitude and latency variations in the first 100 trials were further studied by implementing a denoising scheme, as explained above. For each rat, we further identified the peaks of Fig. 4A in the single trials. Amplitudes and latencies of each peak were automatically defined from the maximum (minimum) value within appropriate time windows. For the P13 the time window was defined between 10-15 ms, for the P20 between 17-23 ms, for the N18 between 15-20 ms, for the N24 between 20-25 ms, for the N38 between 30-40 ms, and for the N52 between 40-60 ms (for more details see Quian Quiroga and van Luijtelaar, 2002). Figure 4B shows the amplitude variations of the different ERP components of Figure 4A as a function of trial number. An exponential decay of the amplitude for the first 4 peaks was observed as the number of trials increased. This decrement in amplitude was completed after 30-40 trials, and it is functionally related to an habituation process. No systematic changes were determined in later trials. Moreover, the brain response to the first trial was smaller than the following ones for the P13, P20, N24, and most markedly for the N18. This is related to a sensitization process. Indeed, a one-way analysis of variance comparing the peak amplitude for the first 3 trials showed a significant increase for the N18 (p