Embodying Data: Chinese Aesthetics, Interactive Visualization and Gaming Technologies [1st ed.] 9789811550683, 9789811550690

Citation preview

Qi Li

Embodying Data Chinese Aesthetics, Interactive Visualization and Gaming Technologies

Embodying Data

Qi Li

Embodying Data Chinese Aesthetics, Interactive Visualization and Gaming Technologies

123

Qi Li Shanghai University of Engineering Science Shanghai, China

ISBN 978-981-15-5068-3 ISBN 978-981-15-5069-0 https://doi.org/10.1007/978-981-15-5069-0

(eBook)

Jointly published with Shanghai Jiao Tong University Press The print edition is not for sale in China (Mainland). Customers from China (Mainland) please order the print book from: Shanghai Jiao Tong University Press. © Shanghai Jiao Tong University Press 2020 This work is subject to copyright. All rights are reserved by the Publishers, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publishers, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publishers nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publishers remain neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Foreword

There are many histories of art, some that have already been formulated, others that are yet to be written, and some that remain hidden simply because the artistic imagination also encompasses the unimaginable. At the same time, art is in a constant state of its own re-invention, and that in turn re-frames the narratives of its potential histories. Over the last century the course of re-invention has been frantic, conjoined (in sympathy and/or antagonism) to deep-going societal and technological transformations. What also emerges more clearly in the development of a global consciousness is the vital heterogeneity of cultures, which of itself creates a world rich in parallel imaginaries about the nature of our being in the world. By examining the dichotomous parallelism of Western and Eastern representational aesthetics, Qi Li provides us with a penetrating understanding and a window of opportunity concerning one of the most fundamental challenges that artistic experience has always aspired to—the desire for embodiment. In our time this has led, for instance, to the development of kinetic art, performance art, participation art and more recently interactive art via computer-aided scenographies. Qi Li puts special emphasis on the advent of those technologies that can track and measure our body’s movements, gestures as well as biological processes, because these facilitate a new set of conjunctions between organism and imagination, between embodied experience and sensory perception. He elucidates how this condition lends itself to a Taoist interpretation that highly values the relationship between “…the organic microcosm of the body with the social macrocosm of humanity”. The research focus of Qi Li’s exegesis is the application of Taoist aesthetics to the burgeoning field of data visualization. He does this by means of an insightful theoretical analysis that informs and leads up to his art installation demonstrator Living Dream. A work like this expands and enriches the domain of the digital humanities—that emergent practice of inquiry and contemplation that is attempting to conjugate an ever-growing expanse of scientific knowledge with the venerable terrain—physical, metaphysical and pataphysical—of existential truths that govern our lives. The American visual theorist Johanna Drucker prioritizes aesthetic subjectivity over analytic objectivism and sees digital humanities projects as “…not

v

vi

Foreword

simply mechanical applications of technical knowledge, but occasions for critical self-consciousness”. While one of modern art’s great achievements has been its ability to free itself from the obligations of representation, it remains a proprioceptive medium no matter how emptied of matter and/or mind. Harald Szeemann’s 1969 Kunsthalle Bern exhibition When Attitudes Becomes Form was a seminal statement in this respect. Because innumerable artists have honed art’s multifarious capabilities over the course of art history, it’s always ready and thirsting to configure the representation of new knowledge wherever it finds it. And in doing so it itself configures new knowledge, as the Chilean filmmaker Raul Ruiz stated in his book Poetics of Cinema “…it is the type of image produced that determines the narrative, not the reverse”. The medieval philosopher Ramon Lull launched computational theory with his ars combinatoria—a system for producing knowledge via the recombination of ‘elemental truths’. In 1927, Aby Warburg’s iconographic compendium Mnemosyne Atlas was an aesthetic quantum leap in the form of a perceptual database of images that emphasized the visual as a means of interpretation. Today, computational alacrity is engineering a massively growing trove of information—a global database that brazenly aspires to materialize a Borgean scaled imaginary. If artists foresaw ‘data science’, both in its utopian and dystopian dimensions, then it can and should be artists that show it for what it is. Qi Li adds a vital new dimension to this undertaking by theorizing and demonstrating that a Taoist aesthetics makes this ‘showing’ not a return to mere representation but rather a practice of ‘embodied new knowledge’ that is informed by a holistic transdisciplinarity. Equipped now with this understanding, and by “…interpellating the subjective into computational activity” (Johanna Drucker) a new humanistic approach to data visualization becomes both possible and necessary. Jeffrey Shaw Artist and Chair Professor of Media Art School of Creative Media University of Hong Kong, Hong Kong

Preface

This book aims to investigate a possible new interactive aesthetic of data visualization that emphasizes how traditional Chinese aesthetics might be explored and examined in a contemporary digital technological context, specifically in combination with gesture-based gaming technology. Current research from the aesthetic perspective of data visualization identifies the potential to enhance emotional engagement in the perception of visual images but is primarily based in Western aesthetic concepts with little reference to non-Western aesthetics. The new aesthetic approach outlined in this research project draws from a non-Western aesthetic concept of the yijing and Taoist cosmological principles proposing the harmonious unification of the body, mind and technologies of visualization. The application of traditional Taoist philosophy associated with the Chinese yijing aesthetic to the design of data visualization emphasizes the experience of harmony and unity between a user’s body and the data associated with digital technology. By considering yijing as a possible aesthetic approach to data visualization, the concept focuses on enhancing the user’s perceptive experience in order to promote a fusion of the user’s feelings and the objective data in the digital technological context. This research project involved the process of creating two artefacts: Living Dream (2015) and Taiji (2015) using a sample of sleep electroencephalography (EEG) data. The artefacts were created using data-based digital practice and a theoretical framework. This framework articulates a data continuum, from information to aesthetics and then to Taoist principles. The two related artefacts were created in terms of the yijing aesthetic that emphasizes the complementarity of xu and shi, and Taoist data visualization that emphasizes the unification of the subjective body and the objective data. This book also presents a critique of the Western aesthetics underpinning data visualization, in particular, the Kantian sublime that emphasizes the experience of power over nature from a distance. The Kantian sublime encompasses the mathematical sublime, emphasizing a vastness that exceeds the comprehension of the viewers, and the dynamic sublime, emphasizing the attempts of the viewers to control nature. In data visualization, users perceive the magnitude of data from a distance, in which the users experience fear or awe over the power of the vii

viii

Preface

capabilities of technology. However, Taoist philosophy emphasizes the unification of the exterior of the body and the exterior of nature as a Taoist body, rather than emphasizing the opposition of mind and body. This research project aims to explore this Taoist body philosophy in the context of digital technology. It proposes a new concept and practice of Taoist data visualization involving notions of the digital yijing and the Taoist digital body. Gesture-based gaming technology, such as Kinect, facilitates embodied Taoist data visualization through interactivity and immersion. Thus, my creative practice explores the transformational relationship between the human body and technology, particularly in creating a new aesthetic approach combining traditional Chinese aesthetics and Taoist body philosophy through gesture-based technology. Shanghai, China

Qi Li

Acknowledgements

This book would not be written without support and assistance from Dr. John Ryan for his encouragement and support and Prof. Mark McMahon for his support and guidance in the development of this research project. I would also like to thank Associate Professor Rod Giblett for his guidance and especially for his Chinese aesthetic insights as a Western scholar. My thanks also to my student Yaqin Fu for her contribution to design of graphic and illustration for this book. I would like to thank all respective supporters who have in various ways supported and contributed to this project. This book was sponsored by Shanghai Pujiang Program (18PJC068).

ix

Contents

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

1 1 2 8 11 12 13 14 15

2 Overview of Data Visualization . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 The Concept of Data Visualization . . . . . . . . . . . . . . . . . . . 2.1.1 The Definition of Data Visualization . . . . . . . . . . . . 2.1.2 The Definition of Data and Information . . . . . . . . . . 2.1.3 The Forms of Traditional Data Visualization . . . . . . 2.1.4 The Characteristics of Traditional Data Visualization 2.2 Data Visualization and Human Perception . . . . . . . . . . . . . . 2.2.1 The Perceptual Process . . . . . . . . . . . . . . . . . . . . . . 2.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

17 17 18 20 21 41 42 42 44 45

3 Data Visualization and Aesthetics . . . . . . . . . . . . . . . . . . . . 3.1 The Concept of Aesthetics . . . . . . . . . . . . . . . . . . . . . . 3.1.1 The Definition of Aesthetics . . . . . . . . . . . . . . . 3.2 Aesthetics and Issues of Data Visualization . . . . . . . . . . 3.3 Aesthetic Approach to Data Visualization . . . . . . . . . . . 3.3.1 The Benefit of Aesthetics to Data Visualization . 3.3.2 Aesthetics and Utility . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

49 49 50 52 52 54 56

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Overview of Research Background . . . . . . . . . . . . . . . . . 1.1.1 Brief History of Data Visualization . . . . . . . . . . . 1.1.2 Visualization, Aesthetics and Chinese Philosophy 1.2 Significance of the Research . . . . . . . . . . . . . . . . . . . . . . 1.3 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Contents Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . .

xi

xii

Contents

3.4 Introduction of the Chinese Yijing Aesthetic . 3.4.1 Yijing and I-Ching . . . . . . . . . . . . . 3.4.2 The Concept of Yijing . . . . . . . . . . 3.4.3 The Forms of Yijing . . . . . . . . . . . . 3.4.4 The Characteristics of Yijing . . . . . . 3.5 The Concept of the Sublime . . . . . . . . . . . . 3.5.1 The Kantian Sublime . . . . . . . . . . . 3.5.2 The Technological Sublime . . . . . . 3.5.3 The Forms of the Sublime . . . . . . . 3.5.4 The Characteristics of the Sublime . 3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Data-Based Digital Art Practice . . . . . 4.1 Digital Art and Data Visualization . 4.1.1 Defining Digital Art . . . . . 4.2 Code as Creative Medium . . . . . . . 4.3 Data-Based Art . . . . . . . . . . . . . . . 4.3.1 Static Data Art . . . . . . . . . 4.3.2 Dynamic Data Art . . . . . . 4.3.3 EEG Data as Digital Art . . 4.4 Conclusion . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

57 58 59 62 63 65 66 67 68 69 70 70

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

73 73 74 75 76 77 78 82 84 85

...

87

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

87 89 89 90 91 91 93 96 97 98

. . . . .

. . . . .

. . . . .

102 103 105 108 108

5 Theoretical Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Developing a Theoretical Framework of Aesthetic Data Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Definition of Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 Data, Information and Knowledge . . . . . . . . . . . . . . . 5.2.4 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.5 The Kantian Mathematical Sublime and Anti-sublime 5.2.6 Computing Technology and the Kantian Sublime . . . . 5.2.7 The Yijing Aesthetic in the Framework . . . . . . . . . . . 5.2.8 Taoist Philosophy in the Framework . . . . . . . . . . . . . 5.2.9 Gesture-Based Gaming Technology and Taoist Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.10 Kinect and Natural User Interfaces . . . . . . . . . . . . . . 5.2.11 Kinect and Taoist Performance . . . . . . . . . . . . . . . . . 5.2.12 Moderators and Influence . . . . . . . . . . . . . . . . . . . . . 5.2.13 Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Contents

xiii

5.3 The Theoretical Framework of Aesthetic Approach to Data Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 5.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

115 116 116 118 121 122 123 125 127 130 132 133 133 135 136 136

7 Taoist Data Visualization . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Programming Processes . . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.2 Sleep EEG Data . . . . . . . . . . . . . . . . . . . . . . . 7.2.3 Methods of Visualizing Sleep EEG Data . . . . . 7.3 Conventional Visualization Technologies . . . . . . . . . . . 7.3.1 Sleep EEG Data Visualization as a Waveform . 7.4 A New Aesthetic Approach . . . . . . . . . . . . . . . . . . . . . 7.4.1 Code Implementation . . . . . . . . . . . . . . . . . . . 7.4.2 Emptiness . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.3 Color and Data . . . . . . . . . . . . . . . . . . . . . . . 7.4.4 Rhythm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.5 Transformation . . . . . . . . . . . . . . . . . . . . . . . . 7.4.6 Spirit Through Brushstrokes . . . . . . . . . . . . . . 7.4.7 Sleep EEG Data as Living Data . . . . . . . . . . . 7.4.8 Harmony . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Unification of the Body and Digital Technology . . . . . . 7.5.1 Sleep EEG Data as Performance . . . . . . . . . . . 7.5.2 The Digital Taoist Body . . . . . . . . . . . . . . . . . 7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

139 139 140 141 141 146 147 147 150 150 153 155 156 158 159 160 161 163 163 164 166 168

6 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Research Position . . . . . . . . . . . . . . . . . . . . . . . 6.2 Research Methodology . . . . . . . . . . . . . . . . . . . 6.2.1 Practice-Led Research . . . . . . . . . . . . . 6.2.2 Digital Practice Research . . . . . . . . . . . 6.3 Research Methods . . . . . . . . . . . . . . . . . . . . . . 6.3.1 Sketchbook . . . . . . . . . . . . . . . . . . . . . 6.3.2 Observation . . . . . . . . . . . . . . . . . . . . . 6.3.3 Building/Making . . . . . . . . . . . . . . . . . 6.3.4 Analysis and Interpretation . . . . . . . . . . 6.4 Research Design . . . . . . . . . . . . . . . . . . . . . . . . 6.4.1 Double Helix of Praxis-Exegesis Model 6.4.2 Praxis . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.3 Exegesis . . . . . . . . . . . . . . . . . . . . . . . 6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

xiv

8 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Summary of the Research . . . . . . . . . . . . . . . . . . . . . . . 8.2 Significance and Contribution . . . . . . . . . . . . . . . . . . . . 8.3 Research Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.1 Aesthetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.2 Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.3 The Theoretical Framework . . . . . . . . . . . . . . . 8.4 Limitations and Recommendations for Further Research . 8.4.1 Research Limitations . . . . . . . . . . . . . . . . . . . . 8.4.2 Recommendations for Further Research . . . . . . . 8.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Contents

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

171 171 173 174 174 176 176 177 177 177 180 181

Acronyms

2D 3D CLI CT DNA DV GPS GUI GVis HCI HMI ICA InfoVis MCM NUI RFID SciVis WIMP WYDIWYG WYSIWYG

Two-Dimensional Three-Dimensional Command Line Interfaces Computed Tomography Deoxyribonucleic acid Data Visualization Global Positioning System Graphical User Interfaces Geographic visualization Human Computer Interaction Human Machine Interaction International Cartographic Association Information Visualization Map communication model Natural User Interfaces Radio frequency identification Scientific Visualization Windows, Icons, Menus, Pointers What you do, what you get What you see, what you get

xv

List of Figures

Fig. 1.1 Fig. 1.2 Fig. 1.3

Fig. 1.4 Fig. 1.5 Fig. 1.6 Fig. 2.1 Fig. 2.2

Fig. 2.3 Fig. 2.4 Fig. 2.5 Fig. 2.6 Fig. 2.7

Pachmarhi cave paintings. (Image courtesy Wikimedia Commons) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Oracle bone script of the Shang Dynasty in the 1600–1046 BC. (Image courtesy Wikimedia Commons) . . . . . . . . . . . . . . . . . Chambers 1908 Hieroglyphic. Illustration from 1908 Chambers’s Twentieth Century Dictionary. (Image courtesy Wikimedia Commons) . . . . . . . . . . . . . . . . . The oval part in the Pamphilian Obelisk in the Piazza Novona at Rome by Pope Innocent the 10th in 1651 . . . . . . . Huangyu Quan Lan Tu (Map of China) (1708–1718). (Image courtesy Wikimedia Commons) . . . . . . . . . . . . . . . . . Plan de Paris, a detailed map of the city of Paris (1734–1736). (Image courtesy Wikimedia Commons) . . . . . . . . . . . . . . . . . The process of data visualization, illustrated by Yaqin Fu . . . The Tree of Architecture (1896), from the book of A History of Architecture on the Comparative Method. London: Athlone Press, University of London. (Image courtesy Wikimedia Commons) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Smartmoney.com, (n.d.), The status of companies in different fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Map of Ancient Greece, Graecia Vetus (Macedonia, Thessaly, Epirus, Achaia, Peloponnesus) in 1752 . . . . . . . . . . . . . . . . . . Map of live flight tracking, a screenshot from flight aware website . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . An exploratory scatter plot graphs the Babinet point against particulate concentration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . New York state total budget expenditures and aids to localities, New York Times, (1976) . . . . . . . . . . . . . . . . . . . . . . . . . . . .

..

3

..

4

..

5

..

5

..

6

.. ..

7 20

..

25

..

26

..

28

..

29

..

30

..

31

xvii

xviii

List of Figures

Fig. 2.8

Fig. 2.9 Fig. 2.10 Fig. 2.11 Fig. 2.12 Fig. Fig. Fig. Fig.

2.13 2.14 2.15 2.16

Fig. 2.17

Fig. 2.18 Fig. 2.19 Fig. 2.20

Fig. 3.1

Fig. 3.2 Fig. 3.3

Fig. 3.4 Fig. 4.1 Fig. 4.2 Fig. 4.3 Fig. 4.4

Fig. 5.1

The multivariate weather chart of Europe, Galton (1863); it shows barometric pressure, wind direction, rain, and temperature for afternoon and evening on each day during the month of December, 1861 . . . . . . . . . . . . . . . . . . . Province of ONTARIO-vital statistics, Hardy (1887) . . . . . . . Atlas de statistique graphique de la ville de Paris. I. Annee (1888) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Social network visualization published in Grandjean (2015) . . The map of Leeds (1833), showing the districts affected by cholera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . An examples of peripheral waveform (a) (2016) . . . . . . . . . . . An examples of generalized transfer function (b) (2016) . . . . An examples of central waveform (c) (2016) . . . . . . . . . . . . . The seismic waveform which recorded by GFZ German Research Center for Geosciences (2018) . . . . . . . . . . . . . . . . . Climate visualization of atmosphere (2007). (Image courtesy of Forrest Hoffman and Jamison Daniel of Oak Ridge National Laboratory) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A screen shot of climate change in 1950, it was created by NASA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A screen shot of climate change in 2013, it was created by NASA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Regional Networks Traffic in 1991. (Image courtesy of Merit Network, Inc., NCSA, and the National Science Foundation) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Napoleon’s march to Moscow. Charles Joseph Minard, (1845–1869), Tableaux Graphiques et Figuratives de M. Minard, 1845–1869. (Image courtesy Wikimedia Commons) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The scale of data and aesthetics, illustrated by Yaqin Fu . . . . Walking a mountain path in Spring. Ma Yuan, (Sung Dynasty), hand-scroll in ink on silk, 27.4  43.1 cm, collection of National Palace Museum, Taipei, Taiwan. . . . . . The sunset at Cottesloe beach, Perth, Western Australia. Photography by Qi Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A computer program showing two blocks of code . . . . . . . . . Last Clock. Ängeslevä and Cooper (2005), digital art. (Image courtesy Jussi Ängeslevä) . . . . . . . . . . . . . . . . . . . . . . Screenshot of Every Icon. Simon (2002), digital art . . . . . . . . The Listening Post, Mark Hansen and Ben Rubin, (2002–2005) digital art. (Image courtesy Wikimedia Commons) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Proposed theoretical framework . . . . . . . . . . . . . . . . . . . . . . .

.. ..

32 33

.. ..

34 34

. . . .

. . . .

35 36 36 37

..

37

..

38

..

39

..

39

..

40

.. ..

54 57

..

62

.. ..

65 76

.. ..

79 80

.. ..

81 88

List of Figures

Fig. 5.2 Fig. 5.3 Fig. 5.4 Fig. 5.5 Fig. 5.6 Fig. 6.1 Fig. 6.2

Fig. 6.3 Fig. 6.4 Fig. 6.5 Fig. 7.1

Fig. 7.2 Fig. 7.3 Fig. 7.4 Fig. 7.5 Fig. 7.6 Fig. 7.7 Fig. 7.8

Fig. 7.9

Fig. 7.10 Fig. 7.11

Wanderer above a sea of fog. Caspar David Friedrich, (1818), Oil painting, 95  75 cm . . . . . . . . . . . . . . . . . . . . . . Hall of the three pure ones in temple of eternal joy (Details), Shanxi, China, Yuan Dynasty, Wall Painting . . . . . . . . . . . . . Kinect device, a Microsoft product . . . . . . . . . . . . . . . . . . . . . An audience is interacting with Kinect in an international exhibition in Shanghai, 2018 . . . . . . . . . . . . . . . . . . . . . . . . . Health and longevity practice in Chinese ancient society . . . . Chinese ink paint on rice paper. Baishi Qi, (n.d) . . . . . . . . . . A photography of recording observation of a participant practicing taijiquan interacting with Kinect, Installation. Photographer: Qi Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Screenshot from a computer program . . . . . . . . . . . . . . . . . . . An Initial Design for Visually Representing Sleep EEG Data. Li, Photographer: Qi Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Double Helix of Praxis-Exegesis Model, which adapted from Marshall’s Double Helix Model.Li . . . . . . . . . . . . . . . . . Screen shot of the eight subject’s sleep EEG data. PhysioBank sourced from http://www.physionet.org/physiobank/database/ sleep-edf/ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The basic information of the subject. PhysioBank, the subject in file st7132j0_header.txt . . . . . . . . . . . . . . . . . . . The information of annotations. PhysioBank, in File st7132j0_annotations.txt . . . . . . . . . . . . . . . . . . . . . . . The information of the file. PhysioBank, it contains information in files st7132j0_data.txt . . . . . . . . . . . . . . . . . . . PhysioBank, Information in files st7132j0_data.txt . . . . . . . . . PhysioBank, Information in files st7132j0_data.txt . . . . . . . . . Screenshot of a PSG (polysomnography) of a person in REM sleep . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Hypnogram Composed of Four Signals: EEG Fpz-Cz, EEG Pz-Oz, Electro-Oculograophy (EOG) Horizontal and Submental Chin EMG, digital display on the screen, Li (2013) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Hypnogram Composed of Four Signals: EEG Fpz-Cz, EEG Pz-Oz, Electro-Oculograophy (EOG) Horizontal and Submental Chin EMG (Detailed with information of the signals, waveform and measurement unit), digital display on the screen, Li (2013) . . . . . . . . . . . . . . . . . . . . . . . Screenshot of the code by Processing software showing the first part of code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Screenshot of the code by Processing software showing two basic functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xix

..

94

. . 101 . . 103 . . 104 . . 107 . . 124

. . 126 . . 129 . . 131 . . 134

. . 143 . . 144 . . 144 . . 145 . . 146 . . 146 . . 148

. . 148

. . 149 . . 151 . . 151

xx

Fig. 7.12 Fig. 7.13 Fig. 7.14 Fig. 7.15 Fig. 7.16 Fig. 7.17

Fig. 7.18

Fig. 7.19 Fig. 7.20 Fig. 7.21 Fig. 7.22

Fig. 7.23

List of Figures

Screenshot of the code by Processing software showing readFile() function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Screenshot of the code by Processing software showing two functions and a class . . . . . . . . . . . . . . . . . . . . . . . . . . . . Screenshot of the code shows the implementation of Kinect . Screenshot of the code showing PolygonBlob class . . . . . . . . Screenshot of the digital space of emptiness. Qi Li, (2015), digital art. Photographer: Qi Li . . . . . . . . . . . . . . . . . . . . . . . . Screenshot of the work Living Dream representing data as lines, which symbolized the calligraphy. Qi Li, (2015), digital art. Photographer: Qi Li . . . . . . . . . . . . . . . . . . . . . . . . Screenshot of the work Living Dream (detail) representing data as colored lines, Qi Li, (2015), digital art. Photographer: Qi Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Screenshot of the work Living Dream (detail) creating a scene, Qi Li, (2015), digital art. Photographer: Qi Li. . . . . . The digital presentation of Taoist practice, photographer: Qi Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A visitor is playing with Kinect, photographer: Qi Li . . . . . . . The exhibition space. It presents the initial visualization of the work, in which emptiness of the space was displayed and the initial colored lines across the space, photographer: Qi Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The exhibition space; the image is continued visualization of the work. photographer: Qi Li . . . . . . . . . . . . . . . . . . . . . .

. . 152 . . 152 . . 153 . . 153 . . 154

. . 155

. . 157 . . 162 . . 165 . . 166

. . 167 . . 167

List of Tables

Table 2.1 Table 2.2 Table Table Table Table Table Table Table Table Table Table Table Table

2.3 3.1 3.2 3.3 3.4 4.1 5.1 5.2 5.3 5.4 5.5 6.1

Table 6.2 Table 7.1 Table 7.2

The forms of traditional data visualization. It shows the types and common forms of data visualization. . . . . . . . . The forms of data visualization. It lists the common and specific forms of data visualization . . . . . . . . . . . . . . . . . The characteristics of traditional data visualization . . . . . . . . The forms of yijing in visual art . . . . . . . . . . . . . . . . . . . . . . The characteristics of yijing in visual art . . . . . . . . . . . . . . . . The forms of the sublime . . . . . . . . . . . . . . . . . . . . . . . . . . . . The characteristics of the sublime . . . . . . . . . . . . . . . . . . . . . The data source for data art . . . . . . . . . . . . . . . . . . . . . . . . . . The forms and characteristics of information . . . . . . . . . . . . . Definitions of data, information and knowledge . . . . . . . . . . . The forms and characteristics of the yijing aesthetic . . . . . . . The forms and characteristics of Taoist data visualization . . . The comparison of CLI, GUI and NUI . . . . . . . . . . . . . . . . . Criterion for problem-solving practice research adapted from Scrivener’s table of norms of technology . . . . . . . . . . . The research methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The information of the EEG data signals. PhysioBank, it contains information in file st7132j0_signals.txt . . . . . . . . . Color representations in each signal of sleep data . . . . . . . . .

.. . . . . . . . . . . . .

22

. 23 . 41 . 63 . 64 . 68 . 69 . 77 . 91 . 92 . 98 . 102 . 106

. . 119 . . 123 . . 144 . . 156

xxi

Chapter 1

Introduction

This chapter is an overview of research background, significance and research questions of the book. It provides a discussion of traditional data visualization, which focuses on communication and understanding data; and aesthetic approach to data visualization, which enhances emotional engagement in the perception of visual images. Current research from the aesthetic perspective of data visualization identifies the potential to enhance emotional engagement in the perception of visual images, but is primarily based in Western aesthetic concepts with little reference to non-Western aesthetics. This book proposes the traditional Chinese aesthetic benefits to visual representation of data through Chinese aesthetic concept of the yijing and principles of Taoist cosmological philosophy. It investigates a possible new interactive aesthetic of data visualization that emphasizes how traditional Chinese aesthetics might be explored and examined in contemporary digital technological environment. The new aesthetic approach outlined in this book draws from a non-Western aesthetic concept of the yijing and Taoist cosmological principles proposing the harmonious unification of the body, mind and technologies of visualization. It concludes with an outline of the structure of the book.

1.1 Overview of Research Background Data visualization makes it possible to search for valuable insights within visual patterns of data. An established history exists of scientists and engineers using visualization technology to present data as visually understandable information. Images and pictures have contained valuables information that can be processed quickly by the human perceptual system. Data visualization can be found in our everyday life from a table in a newspaper, which presents data being discussed in a feature article to a map of a train or bus with information on time for determining train arrivals and departures. Research on data visualization has become an emerging field that including many sub fields, such as data visualization for internet security, medical analytics and business data visual analytics, traditional graphic design, computer © Shanghai Jiao Tong University Press 2020 Q. Li, Embodying Data, https://doi.org/10.1007/978-981-15-5069-0_1

1

2

1 Introduction

graphics displaying data with algorithms, semiotic design, and constructivist study of symbol structure. For example, it can be used in a weather diagram, which displays the movement of a storm front that may influence people’s life. Data visualization has been often used in business field, such as a graph of stock market activities that may predict an upswing or downturn in the economy; A CT scan is also data visualization, which creates a 3D reconstruction of people’s knee; An instruction manual for assemble a bicycle with visual represented each part to add together; A motorway sign is an example of data visualization indicating a curve, merging of lanes or an intersection; Data visualization can also be used in the following areas: a result of a financial market analysis; a design of bridge with a civil engineering analysis; a MRO scan for diagnosis and therapy; an analysis of human population consumption behaviors; a study of data for quantitative analysis; a marketing posters and advertising. Data visualization is often considered as communication between represented data and the viewer through graphic presentation of data (Evergreen, 2019). Users can easily understand the meaning of data and gain insight from the various graphics and charts. A chart or a graph refers to a type of information graphic representing numeric data and is often used to make it easy to understand large volume of data (Cardno, Ingham, Lewin, & Singh, 2018). Only when people understand the meaning behind the visualization, a decision can be made. Decision-making is one of two major reasons why data visualization is critical important (Ward, Grinstein, & Keim, 2010). For example, Linda Elting and her team investigate the decision-making process for doctors using different visualization techniques. The preliminary results from hypothetical clinical trials of a generic conventional treatment compared with a generic investigational treatment using four different visualization techniques were presented to 34 doctors for their decision-making. Four visualization techniques are Table, Pie chart, Bar graph and Icon. Two treatments are different and one treatment has clearly better than another. After seeing the visualization, the clinicians should decide to stop the trail. The results show that the choice of visualization impacted the decision process. It indicates that most clinicians preferred the table, and that several were contemptuous of the icon display. It emphasizes that the visualization is not only a key in presenting data well, but that user preferences are also important (Ward et al., 2010).

1.1.1 Brief History of Data Visualization Data visualization can be back to the 2nd century AD in Egypt, which was used to organize astronomical information for the navigation (Few & Edge, 2007). A table was used to present data, but visual attributes of alignment and space were used to arrange data into rows and columns (Few & Edge, 2007). The form of coordinates was used to surveying in ancient Egypt. However, it argues that the earliest visualization might be paintings created by Caveman in the Stone Age approximately 30,000 years ago. Ward, Grinstein and Keim (2010) demonstrate an example of the paintings found

1.1 Overview of Research Background

3

Fig. 1.1 Pachmarhi cave paintings. (Image courtesy Wikimedia Commons)

in Chauvet-Pont-d’Arc Cave, where located in the southern France, is considered as one of the earliest visualization. The purpose of these paintings is not for displaying in a gallery or museum for appreciation but to provide the essential information to children. Figure 1.1 presents a cave painting found at Panchmarhi Forest. Imagines and pictures are used to encode symbols and words in the ancient writing systems, which are coined as “logograms” (Crystal, 1987). For example, the oracle bone script of the Shang Dynasty is considered the earliest written document in China (Fig. 1.2). Chinese characters were used on oracle bones including animal bones or turtle plastrons in the late 2nd millennium BC and most of them are pictographic. Although the pictorial natures of the oracle bone script, it was a fully functional and grown writing system by the time of the Shang dynasty (Keightley, 1985). The oracle bone script record the ancient Chinese language in its entirety and not just isolated kinds of meaning. The oracle-bone inscriptions were used for divination in ancient China. These inscriptions were used to communicate with the supernatural power (Tseng & Zeng, 1993). In ancient China, the pyromancy was used in the divination, which “the diviner applied heat to the bone so as to produce stress cracks; the cracks were then ‘read’ to foretell the future” (Keightley, 1996, p. 68). In ancient Egypt, the earliest writing system is “hieroglyphics”, which means a large number of picture signs that consists of the various objects and animals (Budge, 1983; Ritner, 1996; Eyre & Baines, 1989). Figure 1.3 shows samples of picture signs, which are depicted as accurately as possible. It has three main groups: logograms (signs presenting morphemes), phonograms (signs presenting one or more sounds) and determinatives (signs helping join previous two groups together) (Ward et al., 2010). Figure 1.4 presents a part of obelisk on the Pamphilian Obelisk was brought to Italy by the

4

1 Introduction

Fig. 1.2 Oracle bone script of the Shang Dynasty in the 1600–1046 BC. (Image courtesy Wikimedia Commons)

Roman Emperors (Gliddon, 1847). In 1636, it has been translated by Father Kircher as the following: The beneficent Being, who presides over generation, who enjoys heavenly dominion, and fourfold power, coin. mite the atmosphere, by means of Moptha, the beneficent (principle of?) atmospheric humidity unto Ammon, most powerful over the lower parts (of the world,) who, by means of an image and appro-priate ceremonies, is drawn to the exercising of his power. (Gliddon, 1847, p. 3)

This writing system has been used by Egyptian for more than seven thousand years. However, some scholars argue that Egyptian may not invent it and there are evidence indicating the system that was brought into Egypt (Budge, 1983). The earliest graphical depictions of quantitative data were used in the 10th century. In the 14th century, a plotting and the logical relation between organizing value and plotting them first emerged (Oresme, 1968). By the 16th century, it is necessary to develop techniques and instruments for accurate observation and measurement, such as triangulation and measuring methods were generated. In the 17th century, a graph had been commonly used for representing quantitative data, which was

1.1 Overview of Research Background

5

Fig. 1.3 Chambers 1908 Hieroglyphic. Illustration from 1908 Chambers’s Twentieth Century Dictionary. (Image courtesy Wikimedia Commons)

Fig. 1.4 The oval part in the Pamphilian Obelisk in the Piazza Novona at Rome by Pope Innocent the 10th in 1651

invented by French philosopher and mathematician Rene Descartes (Few & Edge, 2007). There is an obvious change in presenting data. The theory and practical application in data visualization has gained a big expansion. In the 18th century, “with some rudiments of statistical theory, data of interest and importance, and the idea of graphic representation at least somewhat established” (Friendly, 2008, p. 7), it demonstrated the expansion of these facts to new domains and fields. By the end of the century, new graphic forms of data visualization (isolines and contours) were invented and thematic mapping of physical quantities appeared. At the time, statistical theory and systematic collection of empirical data have begun to use to design visualization with abstract graphs (Friendly, 2008). A coordinate grid has been

6

1 Introduction

used in maps and charts by the use of isolines to display contours of equal value. In ancient society, visualization was used for fundamental needs, such as travel, religion or communication. Maps is one of key tools for travelling as it can help people travel from one location to another quickly and also provides guideline to make travel planning. In 1708, Emperor Kangxi ordered the complete map of China (Huangyu Quan Lan Tu), which was used astronomical observation and triangulation methods shown in Fig. 1.4. It presents the map that emphasizes the mountains, rivers, islands and oceans with coordinated grid, which indicates the distance and directions. Figure 1.5 presents a map of Paris in the 18th century. It highlights the Seine crosses the city and streets, buildings and parks are all clearly represented in the map. Thus, a map has become a wildly-used visualization technique to show the geographical information. In 19th century, the statistical graphics and thematic mapping have been explosive growth. During this period, the common used visualization techniques have been invented, such as bar and pie charts, histograms, line graphs and time-series plots, contour plots and scatterplots. Symbolism has also been introduced as a novel form. The key to visualization is that human never stop to develop maps adopted new effective techniques or methods for more accuracy. However, modern data visualization began in the 19th century with witness of growth in statistical graphics. It includes “all of the modern forms of data display

Fig. 1.5 Huangyu Quan Lan Tu (Map of China) (1708–1718). (Image courtesy Wikimedia Commons)

1.1 Overview of Research Background

7

Fig. 1.6 Plan de Paris, a detailed map of the city of Paris (1734–1736). (Image courtesy Wikimedia Commons)

were invented: bar and pie charts, histograms, line graphs and time-series plots, contour plots, scatterplots, and so forth” (Friendly, 2008, p. 9) (Fig. 1.6). Most of data visualization including graphics, charts, histograms, line graphs, and tables were handmade by artists and designer using pen and paper in two dimension space before the invention of computer technology in the 20th century. In 1984, Apple computer has become a first computer to provide a professional graphic mode of interaction. By the end of the 20th century, data visualization has developed to “a mature, vibrant and multi-disciplinary research areas …”, which has become a dynamic and interactive data visualization (Friendly, 2008, p. 24). Initially applied within the scientific domain, visualization technology has been used to inform the public, engage users and acquire knowledge (Cox, 2006). Based on a cognitive system, visualization provides a human the maximum capacity to acquire more information through the use of vision. Data visualization provides valuable assistance for human to analysis data and decision making. Data represents “facts or ideas in a formalized manner capable of being communicated or manipulated by some process” (Press, 2013). In cognitive theory, it suggests that human factors contribute to the process of data visualization (Tory & Moller, 2004). With a pattern-finding mechanism, the complex

8

1 Introduction

brain contributes to the analysis of visual information (Ware, 2012). Cox (2006) outlines this concept in the broadest sense, suggesting that data visualization refers to “the process of making the invisible visible…the process of making the cognitive imagination visual using available and culturally dominant technologies is one of the most consistent behaviors of humankind” (p. 89). For a better comprehension, data visualization in this book refers to a general concept including information visualization, scientific visualization and data visualization using computer-mediated technology to “transform numerical data into a digital visual form” (Cox, 2006, p. 89). Due to the varieties of visual representation of data, this book attempts to avoid making a distinction between these concepts.

1.1.2 Visualization, Aesthetics and Chinese Philosophy Traditionally data visualization has applied to communicating and understanding data from research findings in the science or engineering disciplines (Jörgenson, Kriz, Mones-Hattal, Rogowitz, & Fracchia, 1995). In considering the purpose of discourse in this book, I describe this kind of data visualization as “traditional data visualization” in order to distinguish it from data visualization that emphasizes aesthetic value. In other disciplines or subjects, traditional data visualization focuses on understanding as an efficient means of communication. This focus is based on data structures, types and value that are inexplicable and monotonous for users who do not possess a scientific background. The technology of data visualization helps them to understand what can be considered tedious scientific data through reducing the mental burden on users (Cawthon & Moere, 2007), as visualization technology maps sets of discrete data to convey information in an understandable visual form (Manovich, 2010). This traditional data visualization enables users to communicate data accurately and efficiently, as the data is transformed into information, which is represented visually (Hagen, Barthel, Ebert, & Bender, 2003, Knupp, 1999). It is important that the design of traditional data visualization is able to privilege the end-user of the visualization, which assists in understanding the meaning of the data. In this sense, aesthetics is often excluded from the design of traditional data visualization or has not been seriously considered, in particular for scientific data visualization. However, recent research argues that an aesthetic approach has been identified as having value in contributing to the visual aspects of data visualization. It is considered as a parallel issue for visualization aesthetic and interactive design experimentation in the digital art community (Judelman, 2004). Artists and designers have developed guidelines for the visual representation, such as type, form and color to optimize visual effects of communication in the design of data visualization (Judelman, 2004). While architects use 3D visualization to enhance the design, interactive designers work in experimenting animation and interactive data visualization. Aesthetic study in data visualization has value in number aspects. Firstly, aesthetic visualization improves users’ engagement with the meaning of, or data contained

1.1 Overview of Research Background

9

within, images (Gaviria, 2008). For example, an appealing data visualization can attract users to engage more deeply with the meaning of the data presented. It is very important for visualization of scientific data, in which scientists are able to obtain the insight of complex scientific data. Secondly, it enhances perception of data and improves the accessibility of communication (Cawthon & Moere, 2007; Norman, 2004; Tractinsky, 2004). It enhances users’ ability to understand the meaning of date quickly. Thirdly, it promotes a subjective experience to design (Norman, 2004). More recently, the popularization and advancement of computer technology has raised debates about data visualization on a cultural level, where it is no longer considered as a technological tool for visually representing a substantial amount of data but as an independent medium with large expressive (communicative) potential (Rodenbeck, 2008). This extends to the field of visual art, with an increasing number of artists using various data as creative material for their digital art practice. Corby (2008) argues that data visualization is “capable of functioning as a sensual and critical art medium able to handle complex affective and emotive subject matter” (p. 462). Data-based artworks have been widely exhibited and recognized, such as the 2001 Data Dynamics exhibition at the Whitney Museum of American Art, New York, and a conference on visualization in the humanities at Massachusetts Institute of Technology (MIT), Massachusetts, in 2010 (Manovich, 2010; Viegas & Wattenberg, 2007). Therefore, an aesthetic approach to data visualization has become more and more prominent in emphasizing the role of aesthetics in improving data visualization and enhancing user engagement with data. It also involves digital art practices that use data for creative purposes related to the social, political or cultural contexts. These emerging data-based art practices often involve subjective experience in terms of the Western aesthetic approach, such as the Kantian mathematic sublime involving the experience of the magnitude of network data. However, little has been explored from non-Western aesthetic perspectives, which might promote different aesthetic experiences through either improving data visualization or stimulating creative experimentation and practice, whereby new knowledge might emerge. In this book, it proposes that traditional Chinese aesthetics have potential benefits to aesthetic data visualization through the application of concepts of harmony and unification. This involves two related aspects of traditional Chinese aesthetics, including the concept of the yijing and principles from Taoist philosophy. As a very traditional Chinese aesthetic concept and one of the highest criteria for Chinese art, yijing is considered a key to creating emotional harmony and overcoming the divide between subjectivity and objectivity (Li, 2009; Zhang, 2005; Liao, 2011). Yijing emphasizes the idea of the fusion of subjective feeling and the objective scene, through the expression of an artist’s personal emotion. Yijing involves an artist’s effort to achieve an embodied concept of “the enhancement of his mental energies into harmony with the world outside” (Osborne, 1968, p. 72). The consideration of yijing as a possible aesthetic approach to data visualization focuses on enhancing the user’s perceptive experience in order to promote a fusion of their feelings with the objective data. In particular, the two visual forms constituting yijing: xu (void) and shi (reality), work together to create a rhythm that evokes a harmonious experience

10

1 Introduction

(Fan, 2008). The two terms can also be translated as emptiness and fullness or emptying and filling. This complementary pair is deeply rooted in the Taoist philosophical concept of yin and yang, which refers to the indigenous philosophy in China. Taoist philosophy emphasizes a unity of the cosmos and humanity with the embodiment of the Tao (way) of nature that cultivates the creation of Chinese painting. In The Analects of Confucius, “The Master said, Set your heart on the dao, base yourself in virtue, rely on ren, journey in the arts” (quoted in Eno, 2015, p. 30). Paraphrasing the writings of Confucius, the unknown author of Xuan he hua pu, written in Song Dynasty (1119–1125), states that As to these arts, a scholar who is striving for the Tao must never neglect them, but he should merely take a delight in them and no more. Painting is likewise an art. When it has reached the level of perfection one does not know whether art is Tao or Tao is art. (cited in Osborne, 1968, p. 66)

This passage describes the importance of Taoist concepts and Confucianism in relation to Chinese thinking about artistic practices, reflecting ancient Chinese thoughts of art. Taoist cosmological philosophy, with its emphasis on cosmological concepts, differs considerably to Western aesthetics. The concept of the Kantian sublime highlights the perceptive experience of the power of nature by positioning a difference between the mathematical and the dynamic sublime. It posits a dualism between nature and humanity, in which the mathematical sublime emphasizes a vastness of nature that exceeds the comprehension of the viewer. Kant (2007) suggests that “mathematical estimation of magnitude understanding is as well served … the apprehension of which is indeed possible, but not its comprehension in an intuition of the imagination” (p. 84). In contrast, the dynamic sublime emphasizes the viewer’s attempt to overcome nature, which “elevates the imagination to a presentation of those cases in which the mind can come to feel the sublimity of its own vocation even over nature” (Kant, 2007, p. 92). In data visualization, the mathematical sublime is reflected in a viewer’s experience of the magnitude of the data or dataset. This research argues that a Taoist cosmological approach to data visualization could generate and promote new forms of aesthetic experience through the unification of a user’s body and the data, rather than separating them as two opposite objects as does the Kantian sublime. A Taoist approach to data visualization will be referred to in this exegesis as “Taoist data visualization”, and will require gesture-based gaming technology to generate the unification of body and data in the context of digital technology. Gesture-based gaming technology has the advantage of facilitating and enhancing interactivity and immersion, which are considered essential aspects of data visualization and databased digital art practice. The key difference to traditional interactive technology, such as the mouse and keyboard, is that gesture-based gaming technology focuses on interactivity and immersion through body movement. For example, Kinect, developed by Microsoft Corporation in 2010 as a peripheral for the Xbox 360 game console (Borenstein, 2012), allows users to control and interact with games through natural body movements (gestures) or voice commands, which are captured by sensors

1.1 Overview of Research Background

11

(Alexiadis et al., 2011). This type of interaction is different to the interactive technology used for traditional data visualization in which users interact with a computer system through a keyboard, mouse or tablet. Kinect device has been widely used in many research areas, such as controller-free exploration on medical data using the Kinect device (Gallo, Placitelli, & Ciampi, 2011). With its specific gesture-based interactive technology, Kinect is used in this research to generate interactive Taoist data visualization adopted sleep EEG data. Based on the discussion of these concept and theories, it proposes a conceptual framework of data visualization based on a continuum of communication and embodied interactive aesthetic perception, the yijing aesthetic, and Taoist philosophy. This framework focuses on aesthetic applications of traditional data visualization but also embodied aspects of gaming technology to construct and promote new aesthetic experience of data that exploits the inherent interactivity of these technological interfaces. This framework is put into practice through the process of designing visualizations using the example of sleep EEG data, which is developed through the process of praxis. The concept of praxis in this research will be discussed in the methodology chapter. This research uses the example of medical and scientific data in order to demonstrate the potential of this approach to enhancing user experience of data.

1.2 Significance of the Research This book is significant for the following reasons. Firstly, an aesthetic approach to data visualization is an emerging research area; it has been demonstrated that an aesthetic approach has contributed to the visual aspects of data visualization, which can enhance the effective communication. However most of the research is based on Western concepts of aesthetics, such as Kantian aesthetic concepts. As a traditional Chinese aesthetic, yijing creates visual harmony between the subjective and the objective in literature and art. Based on literature review, this book will demonstrate the possibility of its use in creating a harmonious aesthetic experience in visual representations of data that facilitate the user’s engagement. This research applies the yijing aesthetic to the contemporary visualization design that provides a new aesthetic approach to data visualization, which contributes to the theory and knowledge of both aesthetics and data visualization. In addition, the use of Taoist cosmological principles might generate a new aesthetic experience of data visualization, which emphasizes the unification of the user’s body and the technology used to visualize data. This Taoist approach to data visualization applies a “modernization” of traditional Taoist philosophy and aesthetics to the context of digital technology. Secondly, the current technology of data visualization has limitations for designers (Friendly, 2009). Gesture-based gaming technology has been recognized as having the potential to enhance the user’s perceptive experience through embodiment. The completely hands-free control technology of Kinect allows users to experience data visualization through immersion, which is unlikely to be experienced through traditional visualization technology. As an interactive technology, Kinect enhances the

12

1 Introduction

user’s motivation to participate in the representation (Hsu, 2011), which can enhance subjective experience during the process of data visualization. The Kinect technology thus has potential to contribute to data visualization technology. In particular it is used in this research project as a tool to generate Taoist data visualization. Thirdly, the aesthetic approach to scientific or medical data visualization is still a new area and little research has been conducted about it. For example, sleep EEG data are represented using traditional visualization technology as a waveform associated with charts or tables that lack certain aesthetic qualities. As EEG data refers to the condition of the human body, particularly the brain, its use suits an embodied approach to aesthetic visualization. An embodied Taoist approach will be employed over the scientific representation model. Thus, the research has the potential to contribute to an alternative way of linking scientific data to various contexts including social, political, cultural and artistic ones. Lastly, the proposed theoretical framework explores a relationship between communication-based and aesthetic-based data visualization. This framework proposes a continuum between technology, information, aesthetics and Taoist philosophy, which can be of benefit to the theoretical and practical design principles for data visualization users, producers, designers, artists or researchers.

1.3 Research Questions This book will examine the following two key research questions: (1) How can the Chinese aesthetic concept of yijing generate a new aesthetic experience of the visual representation of scientific data? This question investigates the aesthetic experience of data visualization, especially in analyzing how the yijing aesthetic enhances the emotional experience of data visualization. Within traditional methods and approaches, data are represented or interpreted in relation to the purpose of understanding. This question involves comparing traditionally communication-focused data visualization and aesthetically focused data visualization, the latter of which has an important role and value for the design of data visualization. This approach to data visualization investigates whether the yijing aesthetic, when applied to interactive technologies, such as Kinect, can generate new aesthetic experiences. To address this question, traditional data visualization methods from the literature will need to be reviewed and the forms and characteristics of the information and aesthetics of these methods analyzed. (2) How can traditional Taoist principles applied to interactive embodied data visualization through gesture-based gaming technology offer a basis for critiquing the Western aesthetic concepts typically underpinning data visualization? The transformation and reinterpretation of indigenous Chinese Taoist philosophy is an issue of concern, particularly in a digital context. It is important that this process of modernization remains largely independent of Western aesthetics, facilitating the

1.3 Research Questions

13

transformation of the thoughts, concepts and spirit of traditional Chinese aesthetics into new contexts or environments, leading to new interpretations. Accordingly, this question examines whether the key concepts of Taoist philosophy might be used in data visualization for a new aesthetic experience of knowledge generation. It focuses on the integration of traditional Taoist principles with digital technology rather than nature or natural objects. The key to my examination of the application of Taoist principles to contemporary digital technologies is the criticism of Western aesthetics that underpins data visualization, in particular in the Kantian sublime. This question further investigates what role gesture-based gaming technology can play in achieving this transformation of Taoist philosophy. This question is addressed through reviewing the literature in conjunction with the practice-led approach of engaging with gesture-based gaming technology. The two research questions are related to each other and both of them require digital practice as a research approach. The artefacts from the outcome of digital practice and the theoretical framework will demonstrate the possibilities of this new aesthetic experience.

1.4 Contents Structure This book consists of eight chapters. This chapter introduces the background, significance and questions of the research undertaken in this project. It outlines the key issues of an aesthetic approach to data visualization that involves adopting traditional Chinese yijing aesthetics and Taoist cosmological principles for data visualization through gesture-based gaming technology. Chapter 2 reviews the concept of data visualization and its key forms and characteristics of data visualization. Chapter 3 reviews the relevant literature to identify and analyze the main issues about the aesthetics and practical uses of data visualization, including possible benefits of aesthetics in data visualization. The chapter also discusses the key concepts of aesthetics including the Kantian sublime, the technological sublime and the Chinese traditional aesthetic yijing. Concepts of data visualization will be addressed in an analysis of the relationship between aesthetics and visualization. The key forms and characteristics of the yijing aesthetic and the Kantian sublime will also be explored. In Chap. 4, data-based digital artworks will be discussed, in particular in relation to the exploration of digital creativity through the visualizing of sleep EEG data as a practice. Chapter 5 outlines the theoretical framework for this research, which includes a project model. It addresses the factors of the framework related to the design of an aesthetic approach to data visualization. The principles of Taoist philosophy and the concepts of gesture-based technology will be discussed. Chapter 6 discusses the research methodology, methods and design for this research project. It establishes the use of practice-led research methodology as a vehicle to guide the process of the research through the discussion of digital practice. The research methods involve documentation (e.g. sketchbooks), observations, analysis and interpretation. The double helix of the praxis-exegesis model will be addressed in the

14

1 Introduction

section within this chapter on research design. This model describes the relationship between the praxis, documentation and the writing of the exegesis. Chapter 7 provides a critical commentary of the creative artefacts and how they demonstrate the project’s theoretical framework. It discusses the process of programming for the creation of the artefacts including the selection of software, selection of sleep EEG data as the sample, and methods used to implement the computer code to build a digital structure for the practice. A data continuum is defined as having three different levels: communication-based data visualization, the yijing aesthetic approach to data visualization and Taoist data visualization through Kinect. Chapter 8 summarizes the research, including the findings from the research and limitations of the thesis. The research findings section is grouped into the categories of aesthetics, technology, and the theoretical framework. This chapter also recommends further research in fields related to digital technology, digital creative practice, real time interactivity, 3D visualization, cultural promotion, the modernization of Taoist philosophy and the digital Taoist body.

1.5 Conclusion This book aims to investigate a Chinese aesthetic approach to data visualization for the generation of a new aesthetic experience. It integrates the Chinese yijing aesthetic, Taoist philosophy, and gesture-based gaming technology to visually represent scientific data with interactivity. In particular, it emphasizes the process of digital practice that demonstrates the theoretical framework, which was developed through reviewing relevant literature to identify the key forms and characteristics of data visualization and aesthetics. This includes the exploration of various forms of conventional data visualization, the analysis of the relationship between aesthetics and the utilization of data visualization, and the discussion of Chinese aesthetics in comparison to the Kantian sublime. This framework outlines a concept of a data continuum, from conventional information visualization to the yijing aesthetic and to Taoist data visualization. It offers design guidelines for artists or designers approaching data visualization based on information, aesthetics and Taoist philosophy. This book also demonstrates the use of data as a creative practice, involving the process of designing and creating two data visualization works based on the theoretical framework. The outcomes of this inquiry highlight that the artefacts reflect the spirit of Chinese aesthetics and Taoist cosmological principles. These artefacts offer a gateway to transforming traditional Chinese aesthetics and Taoist philosophy in the context of digital technology.

References

15

References Alexiadis, D. S., Kelly, P., Daras, P., O’Connor, N. E., Boubekeur, T., & Moussa, M. B. (2011). Evaluating a dancer’s performance using kinect-based skeleton tracking. In Proceedings of the 19th ACM International Conference on Multimedia (pp. 659–662). Borenstein, G. (2012). Making things see: 3D vision with Kinect, processing, Arduino, and MakerBot. Sebastopol, CA: O’Reilly Media. Budge, E. W. (1983). Easy lessons in Egyptian hieroglyphics (Vol. 3). Courier Corporation. Cardno, A. J., Ingham, P. S., Lewin, B. A., & Singh, A. K. (2018). U.S. Patent No. 9,870,629. Washington, DC: U.S. Patent and Trademark Office. Cawthon, N., & Moere, A. V. (2007). Qualities of perceived aesthetic in data visualization. In Proceedings of CHI Conference on Human Factors in Computing Systems. San Jose, USA. Corby, T. (2008). Landscapes of feeling, arenas of action: information visualization as art practice. Leonardo, 41(5), 460–467. Cox, D. (2006). Metaphoric mappings: The art of visualization. In P. Fisher (Ed.), Aestheticscomputing (pp. 89–114). Cambridge, MA: MIT. Crystal, D. (1987). Logograms. In The Cambridge encyclopedia of language (pp. 200–201). Cambridge: Cambridge University Press. Eno, R. (2015). The analects of confucius: An online teaching translation. Retrieved from http:// www.indiana.edu/~p374/Analects_of_Confucius_%28Eno-2015%29.pdf. Evergreen, S. D. (2019). Effective data visualization: The right chart for the right data. Sage. Eyre, C., & Baines, J. (1989). Interactions between orality and literacy in ancient Egypt. Paper presented in the Conference “From Orality to Literacy and Back”. Fan, L. (2008). An Analysis on Chinese traditional philosophy and the relationship between Reality and Void. Gansu Higher Pedagogical Education Journal, 13(1), 72–73. Few, S., & Edge, P. (2007). Data visualization: Past, present, and future. IBM Cognos Innovation Center. Friendly, M. (2008). A brief history of data visualization. In Handbook of data visualization (pp. 15– 56). Berlin, Heidelberg: Springer. Friendly, M. (2009). Milestones in the history of thematic cartography, statistical graphics, and data visualization. Retrieved from http://www.math.yorku.ca/SCS/Gallery/milestone/milestone.pdf. Gallo, L., Placitelli, A. P., & Ciampi, M. (2011). Controller-free exploration of medical image data: Experiencing the Kinect. In 2011 24th International Symposium on Computer-Based Medical Systems (CBMS) (pp. 1–6). IEEE. Gaviria, A. R. (2008). When is information visualization art? Determining the critical criteria. Leonardo, 41(5), 479–482. Gliddon, G. R. (1847). Ancient Egypt: Her monuments, hieroglyphics, history and archæology, and other subjects connected with hieroglyphical literature. Taylor. Hagen, H., Barthel, H., Ebert, A., & Bender, M. (2003). Component-based intelligent visualization. In F. H. Post, G. M. Nielson, & G.-P. Hsu, H. J. (2011). The potential of Kinect in education. International Journal of Information and Education Technology, 1(5), 365–370. Jörgenson, L., Kriz, R., Mones-Hattal, B., Rogowitz, B., & Fracchia, F. D. (1995). Is visualization struggling under the myth of objectivity? In Proceedings of the 6th IEEE Visualization Conference (VISUALIZATION ‘95) (pp. 412–415). Atlanta, USA. Judelman, G. (2004). Aesthetics and inspiration for visualization design: bridging the gap between art and science. In Proceedings in Eighth International Conference on Information Visualisation, IV (pp. 245–250). IEEE. Kant, I. (2007). Critique of judgement (J. C. Meredith, Trans.; N. Walker Ed.). New York, NY: Oxford University Press. Keightley, D. N. (1985). Sources of Shang history: The oracle-bone inscriptions of bronze age China. University of California Press.

16

1 Introduction

Keightley, D. N. (1996). Art, ancestors, and the origins of writing in China. Representations, 56, 68–95. Knupp, D. F. (1999). U.S. Patent No. 5,966,672. Washington, DC: U.S. Patent and Trademark Office. Li, Z. (2009). Chinese aesthetic tradition (M. B. Samei, Trans.). Honolulu, USA: University of Hawaii Press. Liao, T.-L. (2011). Yijing: Mr. Wang meng-ou’s aesthetic of language. Danjiang Chinese Journal, 25(1), 131-172. Manovich, L. (2010). What is visualization? Retrieved from http://manovich.net/content/04projects/063-what-is-visualization/61_article_2010.pdf. Norman, D. (2004). Emotional design: Why we love (hate) everyday things. New York, NY: Basic Books. Oresme, N. (1968). Nicole Oresme and the Medieval Geometry of Qualities and Motions: A Treatise on the Uniformity and Difformity Known as Tractatus de Configrationibus Qualitatum et Motuum. Madison, WI: University of Wisconsin Press. Osborne, H. (1968). Aesthetics and art theory: An historical introduction. London and Harlow: Longmans. Press, G. (2013). A very short history of data science. Retrieved June 24, 2018. Ritner, R. K. (1996). Egyptian writing (pp. 73–87). In Daniels & Bright (eds.). Rodenbeck, E. (2008). Information visualization is a medium. Retrieved from http://infosthetics. com/archives/2009/04/eric_rodenbeck_information_visualization_is_a_medium.html. Tory, M., & Moller, T. (2004). Human factors in visualization research. IEEE Transactions on Visualization and Computer Graphics, 10(1), 72–84. Tractinsky, N. (2004). Towards the study of aesthetics in information technology. In Proceedings of Twenty-Fifth International Conference on Information Systems (pp. 771–780). Washington, DC. Tseng, Y. H., & Zeng, Y. (1993). A history of Chinese calligraphy. Chinese University Press. Viegas, F. B., & Wattenberg, M. (2007). Artistic data visualization: beyond visual analytics. In Proceedings of the Second International Conference, OCSC 2007 Held as Part of HCI International 2007 (pp. 182–191). Beijing: Springer Berlin Heidelberg. Ward, M. O., Grinstein, G., & Keim, D. (2010). Interactive data visualization: Foundations, techniques, and applications. CRC Press. Ware, C. (2012). Information visualization: Perception for design. Burlington: Elsevier Science. Zhang, W. (2005). The inheritation and contribution of Zong Huabai’s Yijing theory. Retrieved from: http://www.kyoai.ac.jp/college/ronshuu/no-05/cho.pdf.

Chapter 2

Overview of Data Visualization

This chapter will first address data visualization and then discuss the relationship between data visualization and aesthetics. It discusses the definition of data and information and the forms and characteristics of traditional data visualization, emphasizes on understanding of meaning of data in effectiveness and efficiency. And then this chapter outlines some key data visualizations, which includes Trees, Scatter plots, Charts, Tables, Diagram, Graphic, Waveform, Simulation and Volume.

2.1 The Concept of Data Visualization The term “data visualization” has a long history dated back the 2nd century AD. At the ancient society, drawings and other visual representations were used to investigate the world and also to record the historical events. Data visualization has significantly contributed to invention and discovery throughout human history (Crapo, Waisel, Wallace, & Willemain, 2000). The invention of computer technology makes the huge change on the way of visual representation of data. Data analyst has become quicker and more accuracy using computer graphical data visualization. Data visualization has become an important part of research in many fields including algorithms, human perception, animation, computer vision and so on. Data visualization is usually associated with the field of computer science in the contemporary society. As an emerging field, it is considered a sub-classification of visualization and is regarded as “the science of visual representation of ‘data’” (Friendly, 2009, p. 2). The technology of data visualization has evolved from using hand drawing in the earliest stages, to “photo-etching”, to using computer technology, such as computing graphics and software (Friendly, 2009). In particular, the development of computer software has advanced the application of data visualization, allowing users to manipulate a substantial amount of data for exploration and analysis in an easier and more affordable way. In this book, data visualization that focuses on the communication for understanding data rather than any other approaches is considered as traditional data visualization. © Shanghai Jiao Tong University Press 2020 Q. Li, Embodying Data, https://doi.org/10.1007/978-981-15-5069-0_2

17

18

2 Overview of Data Visualization

Traditional data visualization has a number of advantages. To begin with, it has the ability to represent a vast amount of data immediately. Secondly, it enables viewers to identify emergent properties (e.g. patterns) in the data immediately for formulating new insights. The third advantage is that it can be used for product quality control where the immediate identification of problems is made possible through the data analysis. The fourth is that it enhances the understanding of large-scale and smallscale data. In this regard, Gray, Mayer, and Hughes (cited in Ware, 2012, p. 3) suggest that data visualization assists in the constructing of hypotheses. Data visualization often results in graphical images of data or concepts, which assists making decisions (Ware, 2012). The development of computing technology facilitates data visualization, identifying useful information or deriving insights from the graphical images. The importance of data visualization is described as follows: The success of data visualization is due to the soundness of the basic idea behind it: the use of computer-generated images to gain insight and knowledge from data and its inherent patterns and relationships. A second premise is the utilization of the broad bandwidth processes, and simulations involving data sets from diverse scientific disciplines and large collections of abstract data from many sources. (Post, Nielson, & Bonneau, 2003, back cover)

This statement emphasizes that data visualization as scientific research relies on computing technology and its utilization for the process of information. For this research, it is necessary to initially define the concept of data visualization in order to explore and identify the key forms and characteristics for designing a theoretical framework for Taoist data visualization, which will be discussed in Chap. 7.

2.1.1 The Definition of Data Visualization The term “data visualization” can be defined in several ways. Most definitions focus on the connection between data and computer technology in order to transform data into a visual or sonic form. Card, Mackinlay, and Schneiderman (1999) define data visualization as “the use of computer-supported, interactive, visual representations of data to amplify cognition” (p. 6). Manovich (2010) defines it as “a transformation of quantified data which is not visual into a visual representation” (p. 20). According to Friendly (2009), data visualization refers to “information which has been abstracted in some schematic form, including attributes or variables for the units of information” (p. 2). Data visualization involves an information exchange that includes the messenger, the receiver, and the message (Kirk, 2012). Kirk (2012) defines data visualization as “the representation and presentation of data that exploits our visual perception abilities on order to amplify cognition” (p. 17). It emphasizes that the design of data visualization requires representing data in an effective and efficient form. Visual representation of data is a key element for these definitions. The purpose of data visualization identifies patterns inside the graphic through exploring and analyzing data. Thus, Bikakis (2018) defines data visualization as the following:

2.1 The Concept of Data Visualization

19

Data visualization is the presentation of data in a pictorial or graphical format, and a data visualization tool is the software that generates this presentation. Data visualization provides users with intuitive means to interactively explore and analyze data, enabling them to effectively identify interesting patterns, infer correlations and causalities, and supports sense-making activities.

From mathematical perspective, it refers to an “understanding of the real number line, time, measurement, and estimation” as well as an “understanding of ratio concepts, notably fractions, proportions, percentages, and probabilities” (Reyna, Nelson, Han, &, Dieckmann, 2009). Numerous visualization software or applications have been developed and the main task is exploring, visualizing and analyzing data. In modern time, data visualization involves four aspects, which refer to real-time interaction, on-the-fly processing, visual scalability and user assistance and personalization (Bikakis, 2018). As Bikakis (2018) explains that real-time interaction requires efficient and scalable techniques that “should support the interaction with billion objects datasets, while maintaining the system response in the range of a few milliseconds.”(p. 2). For on-the-fly processing, the support of on-the-fly visualizations over large and dynamic sets of volatile raw (i.e., not preprocessed) data is required” (Bikakis, 2018, p.2). When addressing information overloading problems, effective data abstraction mechanism is necessary. A key feature of the modern visualization is user comprehension and customization capabilities to various user-defined exploration scenarios and preferences (Bikakis, 2018). Data visualization can be categorized into two major sub-fields: information visualization and scientific visualization (Marai, 2010; Post et al., 2003). Information visualization is used to visually represent abstract data, such as business data (Card, Mackinlay, & Schneiderman, 1999; Spence, 2007), while scientific visualization represents scientific data, which are usually physically based (e.g. the human body, the environment or the atmosphere) (Spence, 2007). Both information and scientific visualization focus on how to transform data into a visual form, to become understandable information for gaining insight and knowledge. Figure 2.1 presents the fundamental process of data visualization, in which data in any form can be transformed into graphical images. When a user reads or looks at a graphical image, the image is interpreted through the human cognitive system for the acquisition of insight or the apprehension of useful information. Central to the process of data visualization is the transformation of data into information. Understanding the differences between the concepts of data and information, as well as their relationship is crucial to understanding the process of data visualization. The difference between data and information is outlined in the next section.

20

2 Overview of Data Visualization

Fig. 2.1 The process of data visualization, illustrated by Yaqin Fu

2.1.2 The Definition of Data and Information The definition of data may vary depending on the context of the disciplines and the application of data. Data usually refers to raw, unprocessed information that is not recognized as having any meaning. Data can be organized in a variety of forms, such as name, numbers, age, signs, characters or symbols (Bellinger, Castro, & Mills, 2004; Yeung, 1998; Zins, 2007) and files, reports, graphs in a business context (“Concepts in information and processing,” n.d.). According to Ackoff (1989), data is symbols, while other researchers define data as “computerized representations of models and attributes of real or simulated entities” (Chen, 2005, p. 1). Data is an integral part of the culture in the contemporary information society. Human are confronted with the flood of data every seconds, such as World Wide Web, cellular calls, networks of smart home applications (Pedrycz, 2005). Data is raw and simply exists. Data “has no significance beyond its existence (in and of itself). It can exist in any form, usable or not. It does not have meaning of itself” (Bellinger et al., 2004). There are two types of data, which are primary data and secondary data (Agarwal, 2006). If the data are “collected from the units or individual respondents directly for the purpose of certain study of information” (Agarwal, 2006, p. 3), it is called primary data. For example, the primary data can be the data collected from a census study or an interview about enquiry made from tax payers. Many examples can be found in research, in particular in scientific study. The primary data can be numbers and people, for example, when “an experiment is conducted to know the effect of certain fertilizer doses on the yield or the effect of a drug on the patients, the observations taken on each plot or patient” are the primary data (Agarwal, 2006, p. 3). Agarwal (2006) defines the secondary data as the information contained in data, which “used to again from records, processed and statistically analyzed to extract some information for other purpose” (p. 3). It emphasizes that the secondary data is collected and statistically purposed by certain people or agency. Therefore, the secondary data can

2.1 The Concept of Data Visualization

21

be obtained from year books, census reports, research reports or survey reports for the scientific studies. In this sense, the primary data can be considered as raw data and the secondary data can be considered as information. In this book, “data” is defined as it is used in the fields of computing science and information science, as “unprocessed information” in the form of numbers or binary codes, which are the quantities stored and transmitted as electrical signals in computers. In this sense, data can be measured, explored, manipulated or retrieved. The data discussed in this book includes both primary data and secondary data. Data and information have a very close relationship as data is usually defined as “unprocessed information” and information considered “processed data” (Hey, 2004; Zins, 2007). Data itself has no value, significance or meaning until it has been processed into a form of information that can be understand by humans (Bellinger et al., 2004; Bernstein, 2011; Zins, 2007). Many different forms have been used to represent data. It depends on whether the data is geometric or symbolic, or whether it is static or dynamic. It also depends on the kinds of purposes the data visualization will serve. Similarly to data, the definition of “information” depends on the discipline and the context. In information science, the term normally refers to “processed data” that has meaning and can be understood by humans (Ackoff, 1989; Davis & Olson, 1985; “How to define data, information and knowledge,” n.d.). Information is considered “collections of data” (Zins, 2007, The Panel’s Definitions section, para. 32) that have been “collected and processed into numbers, artificial and natural languages, graphic objects that convey significance and meaning” (Zins, 2007, The Panel’s Definitions section, para. 32). Information can be data that “are processed to be useful, providing answers to ‘who’, ‘what’, ‘where’, and ‘when’ questions” (Chen, 2005, p. 1). Bellinger et al. (2004) argue that information is data that has been given meaning by way of relational connection. This “meaning” can be useful, but does not have to be. In computer parlance, a relational database makes for information from the data stored within it.

Therefore, information is considered as useful data, which people can recognize and understand it. Chen (2005) emphasizes that “data that represents the results of a computational process, such as statistical analysis, for assigning meanings to the data, or the transcripts of some meanings assigned by human beings” (p. 1).

2.1.3 The Forms of Traditional Data Visualization To understand traditional data visualization, it discusses the main forms of traditional data visualization in this section. The different forms of data visualization present a different visual effect, which can help to identify the problems or issues. It is also important to select the types of data visualization for a variety of disciplines or research fields. Although information visualization and scientific visualization are two different research fields, the forms of visualization are certain similarity.

22

2 Overview of Data Visualization

Table 2.1 The forms of traditional data visualization. It shows the types and common forms of data visualization Data visualization

Common forms

Information visualization: the process of representing abstract data in a visual way, which users can understand meaning of it

Tables; Charts; Trees; Maps; Scatter-plots; Diagrams; Graphs

Scientific visualization: the representation of data graphically as a means of obtaining comprehension and insight into the scientific data. It can also refer to visual data analysis

Simulations; Waveforms; Volume

The visual forms of two visualizations are often overlaps. Table 2.1 presents the main forms of data visualization. Those forms of representation are also called “visualization techniques” (Cawthon & Moere, 2007; Fayyad & Grinstein, 2002), which have been developed to optimize selecting and creating the most effective and efficient transformation of data into information (Senay & Ignatius, 1990). As represented within the table, the common forms of data visualization refer to trees, scatter plots, charts, tables, diagrams, and graphs. Selecting forms for data visualization is dependent on the type of data. Bertin (1977) suggests that data visualization primarily deals with two fundamental types of data, including data values and data structures. Ware (2012) proposes a similar suggestion that data can be split into entities and relations, in which entities refer to visualizing objects, and relations refer to structures and patterns relevant to entities. For example, people can be referred to as entities or a flock of birds can be referred to as a single entity. A house is built up by many different parts that have a structural relationship. Similarly, data usually has an attribute that is an emergent property of entities and may not exist independently. For example, the color of a car becomes an inseparable attribute of the car. Thus, traditional data visualization requires considering the type of data with regard to the entities, relation and its attributes. Information visualization, which listed in Table 2.1, has become an emerging area in the last decade. It is considered as information design, which is “about the selection, organization and presentation of information to a given audience” (Wildbur & Burke, 1998, p. 6). Information design aims to design the efficient communication of information as its foremost task, which requests the content needs accurate and unbiased in the visual presentation. It attempts to present all the objective data required to enable the users to make some kind of decision (Wildbur & Burke, 1998). According to Wildbur and Burke (1998), information visualization can be categorized as three parts. The first refers to information as an organized arrangement of facts or data. For example, a timetable, a signing system and most maps fall into this first category. This organized information can let users freely to extract only that information which they need for a given purpose. The second includes “information presented as a means of understanding a situation or process, such as a guide-book, a bar-chart or

2.1 The Concept of Data Visualization Table 2.2 The forms of data visualization. It lists the common and specific forms of data visualization

23

Common forms

Specific forms

Tables; Charts; Trees; Maps; Scatter-plots; Diagrams; Graphs; Simulations; Waveforms; Volume

Area Chart; Bar char; Bullet graph; Box-and-whisker Plots; Bubble Cloud; Cartogram; Circle View; Dot Distribution Map; Gantt Chart; Heat Map; Highlight Table; Histogram; Matrix; Network; Polar Area; Radial Tree; Streamgraph; Text Tables; Timeline; Treemap; Wedge Stack Graph; Word Cloud

a stage-by-stage description of how to get a machine to operate” (Wildbur & Burke, 1998, p. 7). The third includes the design of control systems. For example, an input or feedback controls of a product. The common forms for information visualization involve Tables, Charts, Trees, Maps, Scatter plots, Diagrams, Graphs, as indicated in the table. In scientific visualization, as indicated in the table, the forms of data visualization often refers to simulations, waveforms or volume, in which data is explored, transformed and represented as an image or simulation in order to obtain insight into the phenomena. It is quite common for scientific visualization to use 3D modelling to represent scientific data because data can possess various dimensions (Ware, 2012). In fact, there are many different types of forms for scientific visualization. There are more specific forms of data visualization, such as area chart, bar chart, bullet graph, Box-and-whisker Plots, Bubble Cloud, Cartogram, Circle View, Dot Distribution Map, Gantt Chart, Heat Map, Highlight Table, Histogram, Matrix, Network, Polar Area, Radial Tree, Streamgraph, Text Tables, Treemap, Wedge Stack Graph and Word Cloud (Data visualization beginner’s guide: a definition, examples, and learning resources, n.d.). These forms of data visualization are not common used in information visualization or scientific visualization. Table 2.2 presents the specific forms of data visualization along with the common forms, which have already listed in Table 2.1.

24

2 Overview of Data Visualization

By considering previous research, the common forms of scientific visualization will be identified and discussed in the following section (summarized in visualization section of Table 2.1). Trees Trees are considered one of the earliest visual illustrations of human thought systems (Lima, 2011). Its hierarchical structure has a great advantage in organizing, rationalizing, and illustrating information patterns and provides a key to interpret “evolving complexities of human understanding, from theological beliefs to the intersections of scientific subjects” (Lima, 2011, p. 21). Humans often use trees as a symbol to represent classification of the natural world (Lima, 2011). In visualization, tree can also refer to “Tree-Map” visualization (Johnson & Shneiderman, 1999). The hierarchical structures involve two types of information, which include structural information correlated hierarchy and content information correlated each node. For example, the branched schema of trees has been used for organizing family ties, social structures, and species evolution. The Tree of the Two Advents by Joachim of Fiore in 1202 is an example of trees that illustrates key characters and institutions of the Christian salvation history. This tree displays key figures from top to bottom, repeating Jesus Christ, Ozias the Prophet, Jacob the Patriarch, and Adam. The image of Christ dominates the whole trees and the lower branches link the image of Jacob to the twelve tribes of Israel. The Christ in the middle links twelve branches that represent twelve churches (Lima, 2011). Figure 2.2 presents a Tree-Map of architecture history in 1896. The Tree-Map was produced as a schematic diagram with what Banister Fletcher recognized as the tree branches of architectural styles. It was published in the first edition of Fletcher’s Architecture book. The author highlights a cross-cultural and historical evolution of architectural styles through a series of successive branches (Flood, 2012). The TreeMap shows the evolution of the various architectural styles in five periods including Peruvian, Egyptian, Greek, Assyrian and Chinese and Japanese. At the base of the tree trunk is a number of individual architectural styles and the crown of the tree several styles are repeated. It presents an overview of the entire hierarchy and makes the navigation of each node much easier. Gülsüm Baydar Nalbantoglu provided her comments on the diagram of The Tree of Architecture as the following: The “Tree of Architecture” has a very solid upright trunk that is inscribed with the names of European styles and that branches out to hold various cultural/geographical locations. The nonhistorical styles, which unlike others remain undated, are supported by the “Western” trunk of the tree with no room to grow beyond the seventh-century mark. European architecture is the visible support for nonhistorical styles. Nonhistorical styles, grouped together, are decorative additions, they supplement the proper history of architecture that is based on the logic of construction. (Morrison, 2015)

The visual metaphor of the tree is often used to represent a network of nodes that consists of root nodes and subordinate note within its branched schema in the field of data visualization. Trees are one of the most useful forms to depict data by starting from a root node and then tracing down to its subordinate node until all nodes have

2.1 The Concept of Data Visualization

25

Fig. 2.2 The Tree of Architecture (1896), from the book of A History of Architecture on the Comparative Method. London: Athlone Press, University of London. (Image courtesy Wikimedia Commons)

26

2 Overview of Data Visualization

Fig. 2.3 The Smartmoney.com, (n.d.), The status of companies in different fields

been explored (Spence, 2007). Trees can take various forms, such as TreeMap, SpaceTree, and StarTree, suitable for different data or information (Cawthon & Moere, 2007). The forms of trees have been used in many fields to represent data, including business and healthcare, amongst others. Spence (2007) presents a typical example of a tree map, which effectively displays a large amount of data for identifying patterns and tendencies (see the Status of Companies in Different Fields in the website of Smartmoney.com). Figure 2.3 shows a tree map that uses the hierarchical structure of the form of a tree—the roots, trunk, branches, but applies different visual forms. It presents clear information from financial markets that provide investors an instant overview of market activities. Maps A map is considered as a graphical depiction highlighting relationships between elements of space (objects or regions). The term of “map” comes from the Latin “mappa mundi”, in which “mappa” refers to cloth and “mundi” means the world. Therefore, a map can be defined as a 2D representation of the surface of the world. In fact, a map can present any space without consider context. For example, a DNA mapping or computer network topology mapping. However, a map often refers to geographic maps, which were used to help people define, explain and navigate through the world. As one of the most important inventions in human history, maps has been made and developed over centuries. Cartography is a term that study crafting representations of the earth upon a flat surface. Cartography refers to the production and study of maps. The term comes from

2.1 The Concept of Data Visualization

27

French cartographie, which was based on Latin carta. According to the International Cartographic Association (ICA), a map refers to “a symbolized image of geographical reality, representing selected features or characteristics” (Crampton, 2001, p. 240). This definition has clearly presented that the symbols need to match to the referents. The visual variables, which “use discrete symbols (e.g., choropleth maps) to show discrete data such as sales tax rates, and continuous symbols (e.g., isarithmic maps) to show continuous data such as temperatures.”, have been used to “a set of map graphic building-blocks which match spatial phenomena” (Crampton, 2001, p. 240). Robinson (1952) argues that using more scientific approach to cartography, which focuses on “function provides the basis for the design” (p. 13). Maps are considered as map communication model (MCM) in the opinion of Robinson’s review of cartography. He argues that “a map maker makes the map created by a cartographer who is supposed to be sensitive to the capabilities of his envisaged map reader” (Robinson, Morrison, & Muehrcke, 1977, p. 6). Robinson et al. (1977) further emphasizes that “corollaries of this view are a lessened concern for the map as a storage mechanism for spatial data and an increased concern for the map as a medium of communication …” (p. 6). This statement highlights a separation between the cartographer and the map users. The users can communicate information from the cartographer through maps. The MCM was considered to promote “the development of a philosophical and conceptual framework in cartography …” (Andrews, 1988, p. 185). The navigational maps are the most popular maps including road map, aeronautical and nautical charts railway or subway network maps, and even hiking or bicycling maps. For example, Google map is a web-based mapping service, which was developed by Google Company. It provides a range of services, including satellite imagery, aerial photography, street maps and so on. Most maps of the worlds are designed as four main maps, including political map that can show territorial borders of each country or city; physical map that presents features of geography, such as mountains, land, rivers, and also including roads, railways and buildings; topographic map that presents elevations and relief with contour lines; and geologic map that has a special purpose to display various geological features, which show rock units, bedding planes, structural features and plunge symbols. These different types of maps can service to different purposes. For example, a geological map can be used for geological exploration, while a political map may help people making plans to travel. Figure 2.4 shows a map of ancient Greece, which depicts the land of Greece surrounding by seas. It uses scale and coordinative techniques to present the geographical information in 1752. Maps were designed to static and fixed on paper or durable materials. The invention of computer technology facilitates the development of design of maps, such as dynamic and interactive maps. One of most important improvement use a variety of computer programs to generate a new form of maps. A basic function is interactive, which allows users to zoom in and zoom out the map. Users can easily search the name of locations they want to travel to in portable map device, which is known as a GPS (Global Positioning System). Geographic Visualization (GVis) which uses the map’s power to explore, analyze and visualize spatial datasets to understand patterns. These developments are key

28

2 Overview of Data Visualization

Fig. 2.4 Map of Ancient Greece, Graecia Vetus (Macedonia, Thessaly, Epirus, Achaia, Peloponnesus) in 1752

components of a “maps as social constructions” approach, emphasizing the genealogy of power in mapping practices, and enabling multiple, contingent and exploratory perspectives of data. Figure 2.5 presents real time flight information in Australian airports during Covid-19 coronavirus outbreak in the early January 2020. It shows that much less flights over Australian sky than that before. It is because strict restrictions on travelling cause the large number of flights being cancelled. Scatter Plots A scatter plot is considered as one of the earliest and most widely used data visualizations, which is based on the Cartesian coordinate system (Ward, Grinstein, & Keim, 2010). It is a very useful form when multivariate data is presented with x or y coordinates (Cleveland, 1993). Each plot presents data in 2D or variables. A scatter plot is critically important in scientific research as it often presents data supporting important findings. Research from Weissgerber, Milic, Winham, and Garovic, (2015) emphasizes the importance of a scatter plot in scientific publication. They found that a scatter plot can enhance the understanding of published data for readers and “the increased flexibility of univariate scatterplots also allows authors to convey

2.1 The Concept of Data Visualization

29

Fig. 2.5 Map of live flight tracking, a screenshot from flight aware website

study design information” (Weissgerber et al., 2015). It is very flexible used in small sample size studies. For example, Fig. 2.6 consists of a vertical scale and a horizontal scale that emphasizes that each of bivariate data (data for two variables) depends on the other, e.g. how Babinet points (the vertical axis) (the neutral points lie between 15° and 20° above the sun) depend on concentration (the horizontal axis) (Cleveland, 1993). Charts Charts are the useful forms to represent data in line, bar, or slice forms. Traditionally, charts represent data in the form of a table, graph, or diagram. For example, Fig. 2.7 presents a chart, which was published in the New York Times in 1976, displaying information about the budget expenditures in localities in New York State, in which data is organized as bars that display information by their sizes. This form may be very useful in representing quantitative data that convey information about sizes or dimensions. As an example of visually representing information using different visual characteristics, Tufte (1997) presents a chart of History of O-Ring Damage in Field Joints, in which each bar of the graph in a chart as a small rocket that reflects a popular, recognized object rather than abstract forms (bars, lines, and triangles). This chart is effectively showing information about temperature and damage, which could help to facilitate possible insights into the data. Another pioneer of exploring data visualization is Francis Galton, who made many contributions to data visualization and statistical graphics. It is well known for his role to develop the ideas of correlation and regression in graphic representation. He is also well known for his works on weather patterns began in 1861, which is shown in Fig. 2.8. The chart presents barometric pressure, wind direction, rain and

30

2 Overview of Data Visualization

Fig. 2.6 An exploratory scatter plot graphs the Babinet point against particulate concentration

temperature on afternoon and evening on every day in December 1861. Each day in the chart, “the 3 × 3 grid shows schematic maps of Europe, mapping pressure (row 1), wind and rain (row 2) and temperature (row 3)” (Friendly, 2008, p. 16). The chart presents weather information clearly, which people can easily understand “the series of black areas (low pressure) on the barometric charts for about the first half of the month, corresponding to the counter-clockwise arrows in the wind charts, followed by a shift to red areas (high pressure) and more clockwise arrows” (Friendly, 2008, p. 16). Tables A table is one of the most common forms to display data within many fields. The data table can be an efficient form to comparative data on categorical objects. It usually uses rows and columns to represent data in two dimensions in the meaningful way (Harris, 1996). The quantitative data are placed in the square sited at the intersection of the row and column (named Table cells). Traditionally in a table columns are used to represent dimensions or fields, and rows represent the actual data. It is an easier and simple visual form to represent data or dataset.

2.1 The Concept of Data Visualization

31

Fig. 2.7 New York state total budget expenditures and aids to localities, New York Times, (1976)

Compared to other visual forms, tables have a number of advantages including that they are “one of the best ways to convey exact numerical value” (Harris, 1996, p. 387), and “assist the viewers in making comparisons, determining how things are organized, noting relationships between various sets of data” (Harris, 1996, p. 387). Moreover, it can organize information where graphing might not be appropriate to the representation (Harris, 1996). Rows and columns of the table can use words, numbers or symbols. Figure 2.9 is a typical table to present the data of the population, number of death and ratio to population in Ontario in 1886. On the left side of the table, there are lists of counties, which include all county in Ontario in Canada. The columns on the right of the table are information presented six categories, which are “population in 1886”, “number of deaths”, “ratio to population”, “phthisis”, and “anemia”. The data presented in the table is clear and accurate. People are able to easy learn the number of population and number of death from phthisis in each county in 1886. A timetable is a type of table. It has been used to referring and managing tool for scheduled events, tasks and actions or appointment. The data is organizing within a table into chronological or alphabetical order helps users for quicker referencing. Timetables are basic way used to show the arrival and departure time, such as buses, trains and other forms of transportation. It is also an ideal tool for individual use for time management.

32

2 Overview of Data Visualization

Fig. 2.8 The multivariate weather chart of Europe, Galton (1863); it shows barometric pressure, wind direction, rain, and temperature for afternoon and evening on each day during the month of December, 1861

Diagrams A diagram can be used in business-system analysis that consists of rectangles and lines illustrating information flow. For example, The Diagram Displays a Relationship and Information Flow for a Business System Analysis by Gerald Lohse emphasizes that the addition of background shading could enhance the understandability of an image by differentiating groups of information from one another (Keller and Keller, 1993). Figure 2.10 presents two statistical diagrams of the wholesale of poultry and game at Halles Centrales. Color coded reference and legend are included in the diagram. It presents the different poultry and the price in each month in 1888. The color of each column presents six poultry (chicken, pigeons, ducks, geese, turkeys and guinea fowl) and readers can understand the price of each animal during the months in 1888. Before the computer technology is invented, most of graphs are competed by hand drawing, such as Fig. 2.11. Although it can accurately present the data or information in form of graph, it can be very difficult to handle a large amount data, such as internet data flow. Figure 2.9 shows social network data visualization through computer software.

2.1 The Concept of Data Visualization

33

Fig. 2.9 Province of ONTARIO-vital statistics, Hardy (1887)

Graphs A graph is similar to a diagram. In fact, the terms graph, diagram and chart are similar and can overlap. Graphics may have many purposes for presenting data. For example, a graph can help people to perceive and recognize the broad features of the data or information. It also can help people to look what are behind the data through

34

2 Overview of Data Visualization

Fig. 2.10 Atlas de statistique graphique de la ville de Paris. I. Annee (1888)

Fig. 2.11 Social network visualization published in Grandjean (2015)

2.1 The Concept of Data Visualization

35

the broad features (Anscombe, 1973). A graph may not present accuracy of data; however, it can provide the insight of data. There is a classic example of graphs Interest of the National Debt from the Revolution by William Playfair in 1786 representing the British government’s debt interest, from 1688 to 1784 (Tufte, 2001, p. 65). The use of the graphic clearly depicts the skyrocketing debt. Another example was the first case of cholera outbreak in Great Britain in 1831. The epidemic caused 52,000 people died within eighteen months. The water-born cause of the disease was not found until Dr. John Snow drew a famous map in 1855. It was considered as a landmark graphic discovery (Fig. 2.12). Waveform Waveform is a common form that is used in visual representation of the form of a signal, which refers to the shape of the time related to various quantities of data. It is often used in medicine, engineering, or earth science fields. For example, EEG data visualization is a typical waveform that is widely used in medical related research. Visualizing the stages of Non-Rapid eye movement sleep can be an example of waveform, which details the shape of a waveform with frequencies and bands showing electrical activity over time (Carskadon & Dement, 2011). There has an example of waveform, which has three waveform representing different studies. Figure 2.13a presents a peripheral waveform (e.g., from the brachial artery); Fig. 2.14b presents a waveform that is analyzed by a mathematical algorithm based on a generalized transfer function; a modulus amplification and phase lag characteristics are presented;

Fig. 2.12 The map of Leeds (1833), showing the districts affected by cholera

36

2 Overview of Data Visualization

Fig. 2.13 An examples of peripheral waveform (a) (2016)

Fig. 2.14 An examples of generalized transfer function (b) (2016)

Fig. 2.15c presents a central waveform that is reconstructed. From this waveform, the information about the central systolic (SBP) and diastolic blood pressure (DBP), and pulse pressure (PP) are derived. The central arterial systolic pressure (CASP) is defined in the peak. The information has been effectively displayed in the waveforms. A waveform is often used to visualize the data generated by earthquakes that are recorded and displayed as a seismic-wave form (Keller & Keller, 1993, p. 72). The waveform is an effective way to represent the magnitude of an earthquake. In Fig. 2.16, the horizontal earth movement is represented by dark color and the bigger waveforms present bigger magnitude of an earthquake. The left column presents the time in a day. This waveform is recorded in a station Jajag in Indonesia in 2018. It has effectively presented the data of earthquake magnitude and people can easy understand the size of magnitude.

2.1 The Concept of Data Visualization

37

Fig. 2.15 An examples of central waveform (c) (2016)

Fig. 2.16 The seismic waveform which recorded by GFZ German Research Center for Geosciences (2018)

Simulation Simulation is one of most useful forms of scientific visualization. One of the important advantages of simulation is that it enables viewers to understand and describe natural phenomena such as weather changes or the role of dark energy in the universe (Ahrens et al., 2010). Computer technology (CT) has been used to simulate natural phenomenon, such as a thunderstorm, global climate change (Thalmann, 1990;

38

2 Overview of Data Visualization

Fig. 2.17 Climate visualization of atmosphere (2007). (Image courtesy of Forrest Hoffman and Jamison Daniel of Oak Ridge National Laboratory)

Rosenblum, 1995). Figure 2.17 is an example that a computer simulates climate, which depicts the carbon dioxide from various sources that are adverted individually as tracers in the atmosphere model. Carbon dioxide from the land is shown as plumes during 1900. This visualization adopts a three-dimensional model to simulate the climate, which provided a realistic environment for effectiveness. Simulation is one of popular visualization of weather and climate related data. It creates the dynamic change of climate change. For example, NASA (The National Aeronautics and Space Administration) simulates a climate warming, which is shown in Figs. 2.18 and 2.19. This data visualization shows how global temperatures have risen from 1950 to the end of 2013. Figure 2.18 shows that the color in ocean area is almost blue and color in land is yellow and blue in the year of 1950, but it has changed to yellow and red throughout continents and much of ocean in the year of 2013 (Fig. 2.19). It is apparently that simulation can effectively present the nature phenomenon in a dynamic model, which has a significant different from other types of data visualization. In a simulation, people can see a process of the change throughout a certain period time. People may gain the insight or new finding from the simulation visualization. Volume Volume visualization is considered one of the most important forms of scientific visualization. It creates graphical representation of data set defined on 3D grids. Volume data set are multidimensional arrays of scalar data and vector data, which are typically defined on lattice structures representing values sampled in 3D environment (Knupp, 1999). Two basic types of volume data are scalar data and vector data (Bürger

2.1 The Concept of Data Visualization

39

Fig. 2.18 A screen shot of climate change in 1950, it was created by NASA

Fig. 2.19 A screen shot of climate change in 2013, it was created by NASA

& Hauser, 2007). Scalar data contains single values for each point while vector data contains two or three values for each point, defining the components of a vector. As the development of 3D data acquisition field, the volume dataset grows fast. Volume data can be captured by many technologies, such as CT or MRI scan technology. Volume visualization has been explored in many domains and it is often used in representing medical related data, such as functional MRI data from scans, data from confocal microscopy (Kaufman, 1996).

40

2 Overview of Data Visualization

Volume data visualization can be divided as five types, which are slice-based approach, emulation of other technology, volume rendering, indirect volume rendering and direct volume rendering (Tukalo, n.d.). Slice-based approach is a simple way to implement and has low computing complexity, however, the disadvantage of this visualization is that users need to restructure the whole object by the imagination as it only visualize a part of the object (Tukalo, n.d.). Most experts prefer to use emulation of other technology for visual analyses, as “the emulation allows experts to have a smooth transition to a modern technic from their old solutions” (Tukalo, n.d., p. 1). Volume rendering, the technique used in volume visualization, focuses on rendering a 3D data set into a 2D visualization. Indirect volume rendering has all typical features of 3D objects. Thus, this technology allows complex 3D analysis simpler. For direct volume rendering, it is considered as a powerful way to visualize volume data, as it has almost all the advantage of polygonal mesh models. It also can combine these models on the same scene (Tukalo, n.d.). For example, Fuchs, Levoy, and Pizer (1990) create a 2D image of a human skull, which is rendering by a CT scanner. It demonstrates how volume rendering is used to present a tumor in purple in a human head and a radiation treatment beam in blue using ray tracer technique. There is another example of volume rendering in Fig. 2.20, which is a visualization of inbound traffic measured in billions of bytes on the NSFNET T1 backbone for the month of September 1991. The volume range is displayed from purple (0 bytes) to white (100 billion bytes). It represents data collected by Merit Network Company.

Fig. 2.20 Regional Networks Traffic in 1991. (Image courtesy of Merit Network, Inc., NCSA, and the National Science Foundation)

2.1 The Concept of Data Visualization

41

The forms in Table 2.1 could be used in both information visualization and scientific visualization. Data and dataset in both information visualization and scientific visualization can be different. That selecting which type of visualization tools depends on the purpose of visualization. The forms of tables and charts can be found in both visualization fields. However, some forms are only commonly used in one or the other. For example, simulation is one of the main forms used to represent phenomena in scientific visualization but it is seldom used in information visualization. Many data visualization techniques adopt hybrids of various forms, such as the combination of a chart and graph. The forms used in Table 2.1 are the common techniques for data visualization. Although the visual effects differ, all forms have similar characteristics, which emphasize the represented data having readability, recognizability and meaning (Kosara, Drury, Holmquist, & Laidlaw, 2008). In the next section, it will discuss the characteristics of traditional data visualization listed in the Table 2.1.

2.1.4 The Characteristics of Traditional Data Visualization There are various visualization techniques are available for design a good data visualization, which can help people produce something with important insights. To create good data visualization, it should firstly understand what the common characteristics of traditional data visualization are. As data visualization as a topic has received attention in many fields in the last decade, it requires high skills in storing, managing and analyzing huge data flow. However, it also needs the ability to visualize data effectively for communication (Yarmuluk, 2019). Table 2.3 outlines the key characteristics of traditional data visualization, which can be used in both information visualization and scientific visualization. The review of the literature suggests the common characteristics of traditional data visualization including readability, recognizability and meaning. These characteristics are based on data communication for comprehension and knowledge acquisition. Data visualization may has more characteristics, such as graphical design, color and interactivity, but the essential and common characteristics focuses on readability, recognizability and meaning. The selecting of visual forms in Table 2.1 for the transformation of data to information facilitates these characteristics. Table 2.3 The characteristics of traditional data visualization Type

Characteristics

Information visualization Scientific visualization

Readability: the quality of being legible or decipherable Recognizability: able to be recognized or identified from previous encounters or knowledge Meaning: what is meant by a picture or a visual form

42

2 Overview of Data Visualization

Firstly, data visualization should be readable. Readability is one of the most important characteristics of data visualization, in which the visual representation is intelligible and can be understood easily. Secondly, users are able to recognize visualizing data. Recognizability is another important characteristic of data visualization that allows users to identify previous knowledge from the visual form. Lastly, data visualization should convey information people could understand. The visual representation of data assists users in understanding the meaning of data, which is an important characteristic for both information visualization and scientific visualization.

2.2 Data Visualization and Human Perception Human perception plays an important role in data visualization. Ware (2012) suggests that perception can remarkably improve both the quality and the quantity of information being displayed. Perception refers to the way sensory data is organized, interpreted and consciously experienced. Also, perception can be defined as “the process of recognizing (being aware of), organizing (gathering and storing), and interpreting (binding to knowledge) sensory information” (Ward, Grinstein, & Keim, 2010, p. 81). It is clearly that perception links to humane senses that produce signals from the environment through five senses including sight, hearing, touch, smell and taste. Ward et al. (2010) explain the notion of perception as the following: Simply put, perception is the process by which we interpret the world around us, forming a mental representation of the environment. This representation is not isomorphic to the world, but it’s subject to many correspondence differences and errors. The brain makes assumptions about the world to overcome the inherent ambiguity in all sensory data, and in response to the task at hand. (p. 82)

This explanation emphasizes that human perception is a subject mental activities, which interpret data or information we received around the environment. When quantitative data is presented in a graphical form, the users are asked to use their visual perception to make judgements about the visual form, to compare relative sizes, locations, orientations, colors, densities, textures of the elements of the visualization. In fact, human visual perception is a very complicated and subjective process, and that the effectiveness of the visualization for conveying objective understanding based on a wide range of subtle factors (Reuter, Tukey, Maloney, Pani, & Smith, 1990).

2.2.1 The Perceptual Process In psychological study, researchers and scholars have been looked at how the human visual system perceives and analyses images for many years. One of key findings from the study is that “the discovery of a limited set of visual properties that are

2.2 Data Visualization and Human Perception

43

detected very rapidly and accurately by the low-level visual system”, which was named as “preattentive” (Healey, 2007). The term preattentive is related to attention. Healey (2007) investigates four theories of preattentive processing, which are feature integration theory, texton theory, similarity theory, and guided search theory. Feature integration theory: – It is a theory of attention developed by Anne Treisman and Garry Gelade. Treisman provided an important insight into the preattentive processing (Healey, 2007). Firstly, Treisman tried to determine which visual properties are detected preattentively, which were named as “preattentive features” (Trick & Pylyshyn, 1994). Secondly, Treisman formulated a hypothesis about how the human visual system performs this processing (Rheingans & Tebbs, 1990). Treisman suggests when human perceives a stimulus, features are “registered early, automatically, and in parallel, while objects are identified separately”. It has been regarded as one of the most influential psychological models of human visual attention. Texton theory: – Bela Julész studies the texture patterns and attempts to determine if variations in a particular order statistic were seen by the human low–level visual system. He suggests that a group of features detected in the early visual system was named as “textons”, which can be categorized into three parts: a. Elongated blobs: specific properties such as hue, orientation, and width. For example, line segments, rectangles, ellipses; b. Terminators: ends of line segments; c. Crossings of line segments; It is believed that only a difference in textons or in their density can be detected preattentively (Healey, 2007). Similarity theory: – This theory involves search time for perception. Some researchers investigated conjunction searches by focusing on two elements: (1) Search time may depend on the number of items of information required to identify the object. (2) It may depend on how easily an object can be distinguished from its distractors, regardless of the presence of unique preattentive features (Healey, 2007). Search time is based on two criteria: T-N similarity and N-N similarity. Guided search theory: – A new visual search theory was proposed by Jeremy Wolfe. He coined it as “guided search”. Guided Search Theory is a two-stage model of visual processing in which initial parallel search mechanisms direct subsequent serial search mechanisms (Wolfe, 2014). One example of the Guide Search Theory is the process of visually diagnosing for red circles within green circles as well as red shaped squares. The theory suggests that there are two sensory scanners, one which is sensitive to the color red, while the other only detects circular shapes. The information from the

44

2 Overview of Data Visualization

two detectors amalgamates the collected information and figures out the target through analyzing for the most fitted suspect. If the target is incorrectly identified, the next most suitable candidate will be chosen. (Wolfe, 2014). Research on perception is also involves illusion, which is considered as a distortion of the senses. As the researchers attempt to find out how the human brain normally organizes and interprets sensory stimulation. This book does not attempt to address the theories of preattentive processing further, but it provides readers a sense of perception theories for understanding human visual system to data visualization. The human perception involves signals that go through the nervous system, which in turn result from chemical stimulation of the sensory system. For example, vision involves light striking the retina of the eye; smell is mediated by odor molecules; and hearing involves pressure waves. Data visualization focuses on vision perception, which is the primary human sense. To understand how human to perceive the images can help people to understand the importance of data visualization in our life and society.

2.3 Conclusion In summary, the forms and characteristics of traditional data visualization emphasizes comprehension of data by using related visual forms to represent information. Data visualization has many visual forms. The main visual forms include Trees, Scatter Plots, Charts, Tables, Maps, Diagram, Graphs, waveform, simulation and volume. Most of these visual forms can be used overlap. It emphasizes that good data visualization have the characteristics of readability, recognizability and meaning. This chapter also discusses how human visual system perceives data and information. It reviews that the definition of human perception and some important theories of preattentive processing, including feature integration theory, texton theory, similarity theory and guided search theory. However, these visual forms and characteristics do not necessarily involve aesthetic considerations or approaches to design visualization. Data visualization has ignored the contribution of aesthetics to enhancing understanding or insight. Nevertheless, aesthetics has more recently been identified as having value to data visualization. This reflects two aspects that include the relationship between aesthetics and usability in the design of human-computer interaction (HCI) as well as visualization considered as artistic creation that emphasizes the value of visualization “in its own right and for its own purposes” (Van Wijk, 2006, p. 428). The value of aesthetics within the HCI field was first introduced by Kurosu and Kashimura in 1995 (Tuch, Roth, Hornbaek, Opwis, & Bargas-Avila, 2012), demonstrating the importance of aesthetics in creating an interface that “significantly influences user’s perceived ease of use of the entire system” (Tuch et al., 2012, Introduction section, para 2). Visualization as art, which gained attention during the early twenty-first century, positions the value of aesthetics as not only concerning images and visual representations,

2.3 Conclusion

45

but also ideas, methods, and techniques. Van Wijk (2006) argues that in order for “aesthetic criteria for new methods to be effective guides…each link of the chain from idea, mathematical model, algorithm, implementation, to visual result is clean, simple, elegant, symmetric” (p. 428). The next chapter will address how aesthetics underpins data visualization. It includes recent research on aesthetic approaches to data visualization and the contributions of aesthetics. Chinese aesthetic concepts and Kantian sublime will be explored in visualizing data.

References Ackoff, R. L. (1989). From data to wisdom. Journal of Applied Systems Analysis, 16, 3–9. Agarwal, B. L. (2006). Basic statistics. New Age International. Ahrens, J., Heitmann, K., Petersen, M., Woodring, J., Williams, S., Fasel, P., et al. (2010). Verifying scientific simulations via comparative and quantitative visualization. Computer Graphics and Applications, IEEE, 30(6), 16–28. Andrews, S. K. (1988). Applications of a cartographic communication model to tactual map design. The American Cartographer, 15(2), 183–195. Anscombe, F. J. (1973). Graphs in statistical analysis. The American Statistician, 27(1), 17–21. Bellinger, G., Castro, D., & Mills, A. (2004). Data, information, knowledge, and wisdom. Retrieved from http://www.systems-thinking.org/dikw/dikw.htm. Bernstein, J. H. (2011). The data-information-knowledge-wisdom hierarchy and its antithesis. NASKO, 2(1), 68–75. Bertin, J. (1977). Graphics and graphic information processing. Berlin: de Gruyter Press. Bikakis, N. (2018). Big data visualization tools. arXiv preprint arXiv:1801.08336. Bürger, R., & Hauser, H. (2007). Visualization of multi-variate scientific data. In Eurographics (STARs) (pp. 117–134). Card, S. K., Mackinlay, J. D., & Schneiderman, B. (1999). Readings in information visualization: using vision to thin k. San Francisco: Morgan Kaufmann. Carskadon, M. A., & Dement, W. C. (2011). Monitoring and staging human sleep. In M. H. Kryger, T. Roth, & W. C. Dement (Eds.), Principles and practice of sleep medicine (5th ed., pp. 16–26). St Louis: Elsevier Saunders. Cawthon, N., & Moere, A. V. (2007). Qualities of perceived aesthetic in data visualization. In Proceedings of CHI Conference on Human Factors in Computing Systems. San Jose, USA. Chen, C. (2005, July/August 12–16). Visualization viewpoints. In IEEE Computer Graphics and Applications. Cleveland, W. S. (1993). Visualizing data. Summit, NJ: At&T Bell Laboratories. Crampton, J. W. (2001). Maps as social constructions: Power, communication and visualization. Progress in Human Geography, 25(2), 235–252. Crapo, A. W., Waisel, L. B., Wallace, W. A., & Willemain, T. R. (2000). Visualization and the process of modeling: a cognitive-theoretic view. In Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 218–226). Data visualization beginner’s guide: a definition, examples, and learning resources. (n.d.). Retrieved from https://www.tableau.com/learn/articles/data-visualization. Davis, G. B., & Olson, M. H. (1985). Management information systems. New York, NY: McGraw Hill. Fayyad, U., & Grinstein, G. G. (2002). Introduction. In U. Fayyad, G. G. Grinstein, & A. Wierse (Eds.), Information visualization in data mining and knowledge discovery (pp. 1–17). San Francisco, CA: Morgan Kaufmann Publishers.

46

2 Overview of Data Visualization

Flood, F. B. (2012). From the Prophet to postmodernism? New world orders and the end of Islamic art. In Elizabeth Mansfield (Ed.), Making art history: A changing discipline and its institutions. New York: Routledge. Friendly, M. (2008). A brief history of data visualization. In Handbook of data visualization (pp. 15– 56). Berlin, Heidelberg: Springer. Friendly, M. (2009). Milestones in the history of thematic cartography, statistical graphics, and data visualization. Retrieved from http://www.math.yorku.ca/SCS/Gallery/milestone/milestone.pdf. Fuchs, H., Levoy, M., & Pizer, S. M. (1990). Inteactive visualization of 3D medical data. In G. M. Nielson, B. Shriver, & L. J. Rosenblum (Eds.), Visualization in scientific computing (pp. 140–146). Los Alamitos, CA: IEEE Computer Society Press. Galton, F. (1863). Meteorographica, or, methods of mapping the weather: Illustrated by upwards of 600 printed and lithographed diagrams referring to the weather of a large part of Europe, during the month of December 1861. Macmillan. Grandjean, M. (2015). Introduction à la visualisation de données: l’analyse de réseau en histoire. Hardy, A. S. (1887). Report relating to the registration of Births, Marriages and Deaths in the province: of ntario for the year ending 31st December 1886. Printed by Warwick & Sons. Harris, R. L. (1996). Information graphics: A comprehensive illustrated reference. Oxford: Management Graphics. Healey, C. G. (2007). Perception in visualization. Retrieved February, 10, 2008. Hey, J. (2004). The data, information, knowledge, wisdom chain: The metaphorical link. Retrieved from http://www.dataschemata.com/uploads/7/4/8/7/7487334/dikwchain.pdf. How to define data, information and knowledge. (n.d.). Retrieved from http:// searchdatamanagement.techtarget.com/feature/Defining-data-information-and-knowledge. Johnson, B., & Shneiderman, B. (1999). Tree-maps: A space-filling approach to the visualization of hierarchical. Readings in information visualization: Using vision to think, 152. Kaufman, A. E. (1996). Volume visualization. ACM Computing Surveys (CSUR), 28(1), 165–167. Keller, P. R., & Keller, M. M. (1993). Visual cues: Practical data visualization. Los Alamitos, CA: IEEE Computer Society Press. Kirk, A. (2012). Data visualization: A successful design process. Birmingham: Packt Publishing. Knupp, D. F. (1999). U.S. Patent No. 5,966,672. Washington, DC: U.S. Patent and Trade-mark Office. Kosara, R., Drury, F., Holmquist, L. E., & Laidlaw, D. H. (2008). Visualization criticism. IEEE Computer Graphics and Applications, 28(3), 13–15. https://doi.org/10.1109/mcg.2008.63. Lima, M. (2011). Visual complexity: Mapping patterns of information. New York, NY: Princeton Architectural Press. Manovich, L. (2010). What is visualization? Retrieved from http://manovich.net/content/04projects/063-what-is-visualization/61_article_2010.pdf. Marai, L. (2010). Data visualization: Taxonomy. Retrieved from http://vis.cs.pitt.edu/teaching/ cs2620/lectures/L01_Taxonomy.pdf. Morrison, D. (2015). The tree of architecture. Retrieved from http://phylonetworks.blogspot.com/ 2015/07/the-tree-of-architecture.html. Pedrycz, W. (2005). Knowledge-based clustering: From data to information granules. Wiley. Post, F. H., Nielson, G. M., & Bonneau, G.-P. (2003). Data visualization: The state of the art. Dordrecht: Kluwer Academic Publishers. Robinson, A. H., Morrison, J. L., & Muehrcke, P. C. (1977). Cartography 1950–2000. Transactions of the Institute of British Geographers, 3–18. Reuter, L. H., Tukey, P., Maloney, L. T., Pani, J. R., & Smith, S. (1990). Human perception and visualization. In Proceedings of the 1st Conference on Visualization’90 (pp. 401–406). IEEE Computer Society Press. Reyna, V. F., Nelson, W. L., Han, P. K., & Dieckmann, N. F. (2009). How numeracy influences risk comprehension and medical decision making. Psychological Bulletin, 135, 943–973. Rheingans, P., & Tebbs, B. (1990). A tool for dynamic explorations of color mappings. In Proceedings of the 1990 Symposium on Interactive 3D graphics (pp. 145–146).

References

47

Robinson, A. (1952). The look of maps. Madison, WI: University of Wisconsin Press. Rosenblum, L. J. (1995). Scientific visualization: Advances and challenges. IEEE Computational Science and Engineering, 2(4), 85. Senay, H., & Ignatius, E. (1990). Rules and principles of scientific data visualization. Retrieved from http://www.siggraph.org/education/materials/HyperVis/percept/visrules.htm. Spence, R. (2007). Information visualization: Design for interaction (2nd ed.). Harlow: Pearson/Prentice Hall. Thalmann, D. (1990). Scientific visualization and graphics simulation. Chichester: Wiley. Trick, L. M., & Pylyshyn, Z. W. (1994). Why are small and large numbers enumerated differently? A limited-capacity preattentive stage in vision. Psychological Review, 101(1), 80. Tuch, A. N., Roth, S. P., Hornbaek, K., Opwis, K., & Bargas-Avila, J. A. (2012). Is beautiful really usable? Toward understanding the relation between usability, aesthetics, and affect in HCI. Computers in Human Behavior, 28(5), 1596. https://doi.org/10.1016/j.chb.2012.03.024. Tufte, E. R. (1997). Visual explanations: Images and quantities, evidence and narrative. Cheshire, Connecticut: Graphics Press. Tufte, E. R. (2001). The visual display of quantitative information. Cheshire, Connecticut: Graphics Press. Tukalo, A. (n.d.). Volume data visualization. Retrieved from https://www.arction.com/articles/ volume-data-visualization/. Van Wijk, J. J. (2006). Views on visualization. IEEE Transactions on Visualization and Computer Graphics, 12(4), 421–432. Ward, M. O., Grinstein, G., & Keim, D. (2010). Interactive data visualization: Foundations, techniques, and applications. CRC Press. Ware, C. (2012). Information visualization: Perception for design. Burlington: Elsevier Science. Weissgerber, T. L., Milic, N. M., Winham, S. J., & Garovic, V. D. (2015). Beyond bar and line graphs: time for a new data presentation paradigm. PLoS Biology, 13(4). Wildbur, P., & Burke, M. (1998). Information graphics: Innovative solutions in contemporary design. London: Thames and Hudson. Wolfe, J. M. (2014). Approaches to visual search: Feature integration theory and guided search. The Oxford handbook of attention, 11, 35–44. Yarmuluk, D. (2019). Discover the 4 key traits of a great data visualization. Retrieved from Microsoft Website. Yeung, A. K. (1998). Data organization and structure. NCGIA Core Curriculum in GIScience. Retrieved from http://www.ncgia.ucsb.edu/giscc/units/u051/u051.html. Zins, C. (2007). Conceptual approaches for defining data, information, and knowledge. Journal of the American Society for Information Science and Technology, 58(4), 479–493. https://doi.org/ 10.1002/asi.20508.

Chapter 3

Data Visualization and Aesthetics

This chapter addresses interdisciplinary research issue that draws on two main research areas: aesthetics and data visualization. In these two areas, the theories of data visualization, the concept of the sublime and the Chinese yijing aesthetic will be investigated. In particular, the key forms and characteristics of the yijing aesthetic and the Kantian mathematic and dynamic sublime will be discussed. In developing a new interactive aesthetic experience for the visual representation of scientific data, this chapter adopts these theories in order to promote a new aesthetic approach to traditional data visualization as well as outlining a Taoist philosophical approach to data-based digital practice through gesture-based gaming technology. While Taoist philosophy is a combination of religious and philosophical concepts, it aim to develop a philosophical discussion, addressing Taoist concepts in relation to gesture-based gaming technology using the theoretical framework, which will be addressed in the Chap. 5.

3.1 The Concept of Aesthetics Aesthetics represents a branch of philosophy concerning the nature of art, beauty or taste. It examines subjective value or judgement of sentiment. The concept of aesthetics was first inaugurated by the German philosopher Alexander Baumgarten in 1750. The term is used to examine taste in philosophical study, or the critical judgement of art. Baumgarten formally defined the notion as “the science of sensory knowledge” (Cooper, 1997, p. 1). Derived from the Greek work aesthesis, the term aesthetics refers to sensation, feeling, or perception (Cooper, 1997). The concept links human perceptual experience of objects in which pleasure or displeasure might be felt in a response to experiential qualities such as beauty, elegance, harmony, proportion, balance, color, depth and sublimity (Janaway, 2006). Aesthetic comprises the natural and artificial sources for aesthetic experience and judgment. Aesthetic refers to philosophy of art, which focuses on study on how artists imagine and © Shanghai Jiao Tong University Press 2020 Q. Li, Embodying Data, https://doi.org/10.1007/978-981-15-5069-0_3

49

50

3 Data Visualization and Aesthetics

create artworks and how viewers please and criticize the artworks. The ancient Greek philosophers Plato suggests that the beautiful objects involves proportion, harmony and unity while Aristotle argues that the universal elements of beautiful objects are order, symmetry and definiteness. For Hegel, art is the first stage in which the absolute spirit is immediately manifest to sense-perception, and is a subjective revelation of beauty but not an objective. As early as the 5th century, Chinese philosophers had already arguing on aesthetic issues, such as the role of the arts in human nature, in particular in poetry, music and calligraphy. Confucius argues that conducing ceremony with style similar to an artistic performance and music was primarily used for rites and ceremony (Aesthetics in Asia, n.d.). Since 20th century, aesthetics represents a theoretical research on principles of the production, transmission and reception of almost all forms of Chinese arts, including classic literature, poetry, painting, calligraphy and architecture (Jiang & Cai, 2018). In India, aesthetics highlights on inducing special spiritual stated in the audience. It involves two key concepts: rasa and kama, where the first concept emphasizes aesthetic pleasure and the second concept emphasizes the pursuit of love and enjoyment (Pollock, 2016). Aesthetic is of critical importance to art and artists, as Barnett Newman states that “Aesthetics is for the artist as Ornithology is for the birds” (Danto, 2003). Aesthetic examines topics involved aesthetic objects, aesthetic experience and aesthetic judgements.

3.1.1 The Definition of Aesthetics There are many definitions on aesthetics in published books, journals and dictionaries. Macmillan Dictionary defines aesthetics as “the appearance of something, particularly in relation to its beauty”. Aesthetics is considered as a term that primarily refers to visual elements and it can also be used as a noun to describe criteria by which an artistic movement is defined (e.g. the postmodern aesthetic). Cambridge Dictionary defines aesthetics as “the formal study of art, especially in relation to the idea of beauty”. Merriam-Webster Dictionary defines aesthetics as “a branch of philosophy dealing with the nature of beauty, art, and taste and with the creation and appreciation of beauty”. In the modern terms, aesthetics refers to “a particular theory or conception of beauty or art: a particular taste for or approach to what is pleasing to the senses and especially sight.” (Merriam-Webster Online Dictionary). Aesthetics can be defined as “the qualities and ideas in a work of art or literature that relate to beauty and the nature of art” in Oxford Learner’s Dictionary. It can also be defined as “the branch of philosophy that studies the principles of beauty, especially in art.” Collins Dictionary defines aesthetics as “the study or theory of beauty and of the psychological responses to it; specif., the branch of philosophy dealing with art, its creative sources, its forms, and its effects.” Those definitions from dictionaries emphasizes that aesthetics is actually related to a philosophical study to art, beauty and taste. The common questions of aesthetics often asked are: “What is meant when something is said to be beautiful?”; “What is art?”; “what is a work of art?”; “what

3.1 The Concept of Aesthetics

51

makes good art?” Some scholars define aesthetics in a different way. The encyclopedia of aesthetics edited by Michael Kelly provides a broad definition of aesthetics, which is “critical reflection on art, culture and nature” (Kelly, 1998, p. 48). Osborne (1986) proposes a concept of aesthetics. He argues that an aesthetic object can either be “anything upon which aesthetic attention is directed” or it can be “anything which is especially well adapted to evoke and sustain aesthetic interest.” (Osborne, 1986, p. 331). According to Harold Osborne, any artifact is an art work as long as it is adopted to evoke and sustain aesthetic attention. Monroe Beardsley defines that “any artifact which was produced with the intention of giving it the capacity to evoke and sustain aesthetic attention” (Osborne, 1986, p. 331). Therefore, according to the first definition, a beautiful scientific theory is a work of art. The value of aesthetics has two meanings. The first is that “it may refer to the value placed upon aesthetic experience-the appreciation of beauty in any of its manifold forms-in comparison with the values we ascribe to the other activities in which human beings indulge” (Osborne, 1986, p. 331). The second is “it may mean the value ascribed to the objects of aesthetic experience when we say, for example, that one work of art or one landscape is aesthetically better or worse than another” (Osborne, 1986, p. 331). These arguments emphasize on aesthetic experience to beauty in one work of art, in which it is what this book adopts for defining aesthetics. The experience of aesthetics involves the beautiful, the ugly, the sublime and the elegant and so on. Scruton and Munro (n.d.) ague that aesthetic experience can be applied to many realm of experience. They address it as the following: We are acquainted with an interesting and puzzling realm of experience: the realm of the beautiful, the ugly, the sublime, and the elegant; of taste, criticism, and fine art; and of contemplation, sensuous enjoyment, and charm. In all these phenomena we believe that similar principles are operative and that similar interests are engaged. (Scruton & Munro, n.d.)

Aesthetics initially deals with natural and natural objects, which are regarding as classic aesthetics. As it argues that it “deals not only with the nature and value of the arts but also with those responses to natural objects that find expression in the language of the beautiful and the ugly.” However, this definition on beautiful and ugly is too vague and too subjective in their meaning. There are three approaches to aesthetics. The first approach emphasizes on the study of aesthetic concepts. It argues that the “language of criticism”, which “particular judgements are singled out and their logic and justification displayed” (Scruton & Munro, n.d.). Edmund Burk, a well-known British philosopher, attempts to draw a distinction between the sublime and the beautiful, and “studying the qualities that they denoted, to analyze the separate human attitudes that are directed toward them” (Scruton & Munro, n.d.). Burk’s distinction is extremely “influential, reflecting as it did the prevailing style of contemporary criticism.” The second approach is a philosophical study of states of mind, in which “responses, attitudes, emotions - that are held to be involved in aesthetic experience.” The third approach is not related to art, but a study of aesthetic experience and “to find in them the true distinguishing features of the aesthetic realm” (Scruton & Munro, n.d.).

52

3 Data Visualization and Aesthetics

Therefore, aesthetic is related to study of art or beauty. However, data visualization presents abstract data in the graphical forms, in which pictorial images is similar to the paintings. In this sense, aesthetic apparently has value to the data visualization. In the next section, it addresses the issues of data visualization and aesthetic contribution to it.

3.2 Aesthetics and Issues of Data Visualization Aesthetics has come to be regarded as an important issue of data visualization research, especially in the information visualization field (Chen, 2005). In fact, aesthetics is considered a key element in information visualization, as exemplified by the field of ambient visualization that uses “aesthetics as a method to ensure displays remain unobtrusive in the physical settings in which they are placed” (Mankoff, Dey, Hsieh, Kientz, Lederer and Ames cited in Lau & Moere, 2007; Aesthetics & Information Aesthetics section, para. 1). Research into understanding the interaction between data interpretation and aesthetic presentation is considered one of top ten unsolved problems in the information visualization field (Chen, 2005). There is much research on aesthetics in information visualization, such as research focusing on graph-theoretical properties (Tufte, 2001), but a lack of research associated with data itself. In particular, there is little research into aesthetics in other visualization fields, such as scientific visualization, which still poses a challenge to aesthetic approaches. As this research aims to apply a non-Western aesthetic to data visualization, the following sections discuss the benefits of aesthetics for data visualization, especially in discussing the forms and characteristics of the Chinese yijing aesthetic. By considering a comparison of aesthetics in different cultures, the next section explores Western aesthetic concepts, particularly through a discussion of the concept of the sublime. The concept of the sublime is explored, as well as the forms and characteristics of the sublime, in particular focusing on a discussion of the Kantian sublime and the technological sublime.

3.3 Aesthetic Approach to Data Visualization An aesthetic approach to data visualization contains different values than other approaches, such as usability or functionality, especially in the domains of science or engineering. In data visualization, an aesthetic approach refers to an investigation of the judgement that examines the value of the visualization work according to the sensation it produces (Sack, 2011; Healy, 2018). The judgement about the value of visualization in scientific or engineering discipline depends on utilization, such as the speed, accuracy, or efficiency of the task (Card, Mackinlay, & Shneiderman, 1999; Brath, Peters & Senior, 2005; Tateosian, 2005). For instance, judgements about medical data visualization will depend on whether it helps doctors make a diagnosis

3.3 Aesthetic Approach to Data Visualization

53

more quickly and accurately. Similarly, scientists often use mathematical models to visualize the physical world for identifying the nature of physical issues. The process of discovery requires detailed observation, logical formalism (a term to describe proofing logic using the formal manipulation of symbols rather than its meaning) and repeatability with objectivity and reproducibility as the result. The aesthetics associated with, or resulting from, mathematical models are normally disregarded (Jorgenson, Kriz, Mones-Hattal, Rogowitz, & Fracchia, 1995). However, Jorgenson et al. (1995) suggest that aesthetic consideration of the visual representation of scientific problems before the development of mathematical models could contribute to the analysis and interpretation of massive scientific datasets. Previous research into aesthetic approaches to data visualization mainly focuses on questions such as what constitutes good data visualization and whether data visualization needs to be beautiful (Tufte, 1997; Herdeg, 1983). The questions explored in previous research are primarily focused on 2D graphics or diagrams. Edward Tufte, one of the pioneers in early research into data visualization, outlines the theory and practice of visually displaying quantitative data using various techniques. Tufte (2007) argues for the importance of aesthetics in information graphical design and examines the combination of words, numbers, and pictures for enhancing the visual quality of design. He compares “friendly” and “unfriendly” data graphics so as to guide the design of such data visualization (Tufte, 1983). For example, Charles Joseph Minard portrays data from Napoleon’s march to and from Moscow in 1869. In the visualization work, two simple bands in the middle of map represent the size of the army. The shape of the two bands adds effective visual impact by emphasizing the massive reduction in the size of the army in comparison of that at the beginning of the war. In a sense, this visual representation conforms to Tufte’s concept of the role of aesthetic theory in the design of a visualization, highlighting that the successful design of data visualization should be as follows: “graphical elegance is often found in simplicity of design and complexity of data” (Tufte, 2001, p. 177). It emphasizes the minimalist representation of data, in which visual elements that are unnecessary, useless and non-informative to information communication should be avoided (Tufte, 1997). The contrast between the widths of the bands highlights the massive devastation of the army, promoting a viewer’s stronger emotional engagement with the subject matter (Fig. 3.1). However, some scholars argue that data visualization cannot be simply considered as “a matter of competing standards of good taste” (Healy, 2018, p. 1). Healy (2018) suggests that aesthetic approach to data visualization needs to do more with how people’s visual perceptual works, thus it should considered “thinking about the relationship between the structure of your data and the perceptual features of your graphics” (p. 1). In addition, it needs to be noted that most of Tufte’s theories and practices, which focused on graphic visualization of statistics, have rarely been explored or engaged with using computer technology, including computer interactive techniques. In his book on information design, Tufte gives many examples of charts or diagrams that emphasize the objective characteristics of data. It shows the limitation of Tufts’s information design, which lacks a discussion of the application of computing technology to data visualization, including interactive techniques.

54

3 Data Visualization and Aesthetics

Fig. 3.1 Napoleon’s march to Moscow. Charles Joseph Minard, (1845–1869), Tableaux Graphiques et Figuratives de M. Minard, 1845–1869. (Image courtesy Wikimedia Commons)

In the digital age, most data visualization, particularly scientific and engineering visualization, relies more or less on computing technology.

3.3.1 The Benefit of Aesthetics to Data Visualization More recent research demonstrates that aesthetics has been gaining attention as a means to promote a positive effect on perception of visualization in order to enhance experience and amplify the ability to obtain knowledge (Cawthon & Moere, 2007). This has been recognized within the design discipline, where research demonstrates that visual attractiveness is a major factor in how users engage with data visualization (Gaviria, 2008). If a data visualization has a high aesthetic value, users may be encouraged to engage in finding more or richer meaning within images (Lang, 2008, Bennett, Ryall, Spalteholz, & Gooch, 2007). Applying aesthetics to visualization has the ability to invoke a sensation on two levels. It can firstly promote the traditional visualization focus of accuracy, efficiency and effectiveness, which is valuable in scientific data visualization (Card et al., 1999). Scientists are able to gain insight from the visual representation of data. It often leads to scientific discovery and contribution to knowledge. Secondly, aesthetics can be used to prompt a subjective experience in the form of an emotional response. Emotion refers to “a strong feelings from one’s circumstance, mood, or relationships with others” (The Oxford Dictionary). It has been expressed as consisting of a coordinated set of responses. The responses involve verbal, physiological, behavioral, and neural mechanisms (Fox, 2008). Emotions, such as pleasure, joy or a sense of fun, which are considered to be the prime motivators for human behavior, should have some bearing on the performance elements of a visualization e.g. its “usability, utility, and

3.3 Aesthetic Approach to Data Visualization

55

functionality” (Norman cited in Tractinsky, 2004, p. 14). Research demonstrates that visual analysis could be more effective and more productive if the artistic side of visualization is enhanced (Tateosian, Healey, & Enns, 2007). Tateosian et al., (2007) argue that effective visualization engages the user’s attention, whereby their gazes react and attend to a stimulus. Usability has been demonstrated as having value for data visualization. When designing data visualization, usability is often considered as a critical element for comprehension of complex data. However, aesthetics and emotion are also important. Aesthetics and emotion have been used in many design related fields, such as product design. Norman (2004) suggests that emotions and aesthetics play an important role in the product design for “attractive things work better” (p. 17), which departs from the traditional design principle of data visualization that claims functionality as the dominant element. Norman (2004) asserts that eliciting positive emotions can enhance communication elements for learning and creativity, as “positive emotions are critical to leaning, curiosity, and creative thought” and a happy emotion “broadens the thought processes and facilitates creative thinking” (Norman, 2004, p. 19). The key design theory in Norman’s book focuses on the human attributes governed by brain mechanisms at three different levels: the visceral, behavioral and reflective. The visceral level is the automatic response part of the brain. It requires design involving “immediate emotional impact … to feel good, look good” (Norman, 2004, p. 69). This emphasizes the importance of emotion and aesthetics that could attract customers from the design of appearance, such as advertisements and brochures. Norman argues that even highly rated products can be failed as long as they do not appeal to the aesthetic sense of the customers. The behavioral level involves “function, understandability, usability, and physical feel” (Norman, 2004, p. 70). Function is considered as the most important component in the behavioral design. The reflective level conveys a message and a culture, and its usability together. Each one has its own design principle. It often identifies an overall impression of a product (Norman, 2004, p. 88). The visceral level emphasizes the importance of physical features, such as visual appearance, sensation or sound. It plays a role in how people receive “emotional signals from the environment that get interpreted automatically at the visceral level” (Norman, 2004, p. 65). The behavioral level primarily deals with usability, in which performance, rather than appearance and rationality, is the main consideration. Based on Norman’s theory, traditional data visualization can be considered as a behavioral objective. The design of aesthetic data visualization should have visceral appeal. Reflective design helps to achieve a balance between the visceral attraction of people’s attention and behavioral usability, thus balancing aesthetics and functionality. Another important piece of research, which links aesthetics and data visualization, is the concept of aesthetic computing. The term, proposed by Fishwick (2006), integrates further the concept of aesthetics into mathematics and computing and, in particular, the role of aesthetics in broad computer-science related fields. Fishwick (2006) argues that art in an aesthetic sense is a broad concept combining computing and creativity, such that a computer program is a form of art. Aesthetic computing deals

56

3 Data Visualization and Aesthetics

with three activities, which are “(1) representing programs and data structures with customized, culturally specific notations; (2) incorporating artistic methods in typically computing-intensive activities, such as scientific visualization; (3) improving the emotional and cultural level of interaction with the computer” (Fishwick, 2006, p. 6). These three activities almost include all aesthetic benefits or contributions to data visualization as discussed above, such as emotion, enhancing user engagement and producing more effective and productive visual analysis. Assessment of aesthetic effects underpins the important role of aesthetics in data visualization. Cawthon and Moere (2007) conducted an online survey to test the aesthetic effectiveness of data visualization. Two methods, individual ranking and group ranking, were used to assess respondents’ subjective responses to aesthetics in visual form, including TreeMap, IcicleTree, SpaceTree, Windows Explorer, BeamTrees, StarTree, Dendogram Tree, Reingold Tilfer Polar View, StepTree, Botanical Viewer and SunBurst (Cawthon & Moere, 2007, p. 10). For individual ranking, participants were required to rate their perception of the visual effect using a slider interface to rank the ugliness or beauty of the visualization. Group ranking was conducted to rate the same set of seven visualizations. The results show that the SunBurst has a highest ranking in both individual and group rankings. SunBurst has a more harmonious visual form, which has the potential to raise delight and emotion (Cawthon & Moere, 2007). This assessment demonstrates that aesthetics improves the effectiveness of communication and evokes users’ emotional engagement.

3.3.2 Aesthetics and Utility The key issue identified in the research into data visualization and aesthetics is the focus on the utility and forms of the visualization. Friedman (2008) indicates that the successful design of data visualization often requires achieving a balance of aesthetics and utility. Depending on the context of the design, data visualization may emphasizes functionality or usability, may emphasizes aesthetic experience, or may totally ignore either usability or aesthetic issues. Lau and Moere (2007) propose a model of how aesthetics and functionality work together. Figure 2.16 (in Lau and Moere, 2007) illustrates how this model involves a scale of artistic freedom and data representation. In the model, artistic creation with a sublime quality and data representation focusing on readability and recognizability sit at the extreme opposite ends of the scale. The model demonstrates that functionality-focused data visualization usually limits the freedom of creativity, while data visualization with creative freedom usually provides a less utilitarian representation of data. Lau and Moere (2007) suggest a range of novel techniques that might be developed to support this model, such as museum technology designed for interactive installation in a museum (Fig. 3.2). This interactive installation attempts to serve both as “art” for the appreciation of collections from the museum and as a “tool” to perform different tasks for information about the museum (Boehner, Sengers, Medynskiy, & Gay, 2005). In addition, Kosara

3.3 Aesthetic Approach to Data Visualization

57

Fig. 3.2 The scale of data and aesthetics, illustrated by Yaqin Fu

(2007) suggests one possible method for combining aesthetics and data visualization. He proposes the term “interaction in particular” (Kosara, 2007, Navigating the Space in Between section, para. 1), in which the visual representation has the quality of the sublime (magnitude, grandeur, distance, scope, space, and height), and, at the same time, the user can interact with the work for generating specific patterns or images. Therefore, the user has the option to see which side of the visualization they would like to see: the artistic visualization (aesthetic effects) or the pragmatic visualization (empirical meaning). The discussion of aesthetics above primarily focuses on the principles of Western aesthetics (beauty, harmony, symmetry, color, and proportion), which emphasizes an attractive, beautiful appearance. As demonstrated, theorists argue that these could have benefits for data visualization. In contrast, the next section will explore a nonWestern aesthetic concept, the Chinese yijing aesthetic, in order to develop a framework for applying the concept of yijing to the design of data visualization. The concept of the yijing aesthetic is a key to creating a new aesthetic experience in my project. Compared to Western aesthetics, it could foster a different experience that focuses on harmony and unification. In particular, this invocation of the yijing aesthetic emphasizes embodiment, which is a commonplace theory used in the gesture-based gaming technology, as this research adopts the concept of the yijing to create embodied interactive data visualization through gesture-based gaming technology. Considering the complex and long history of yijing, this research explores the concept broadly in visual art, establishing its basic visual forms and characteristics from the following review of related literature.

3.4 Introduction of the Chinese Yijing Aesthetic As a classic Chinese aesthetic notion, yijing plays an extremely important role in traditional Chinese literature and art. Whereas Western traditional art emphasizes the realistic representation of objects (external realistic expression), traditional Chinese aesthetics places more emphasis on the emotional experience (internal spiritual expression) that creates a harmonious relationship between subjectivity and objectivity.

58

3 Data Visualization and Aesthetics

3.4.1 Yijing and I-Ching As there are insufficient translations and explanations of the yijing aesthetic in English, the concept of yijing could be confused with the concept of I-Ching, for which the pronunciation is based on the Wade-Giles Romanization system, used in the translations published in Western countries before 1979 (Krieger, Neill, & Jantzen, 1994). Under the current Chinese pinyin system, the two concepts have the same pronunciation: as yijing. Although yijing and I-Ching share some common characteristics, such as both being rooted in Taoism, they are two different concepts and have different Chinese characters. Recently, most English translations of Chinese texts have adopted the Chinese pinyin system. For example, Encyclopedia Britannica translates I-Ching as “yijing” that refers to “Classic of Changes” or “Book of Changes” emphasizing how the book is used for the purpose of divination. The concept of I-Ching, which is sometimes called Yi Ching (Cooper, 1972), is similar to the Western geomancy system (Karcher, 1999; “Encyclopedia Britannica”, n.d.). Geomancy, which is a method for divination, refers to earth divination and stems back to Ancient Greece. It usually involves a process of interpreting the patterns formed by tossed handfuls of rocks or sand. I-Ching, which is also named Zhouyi in Chinese, is considered to be the essence of Chinese traditional philosophy. It uses linear signs as oracles that can be combined as eight basic symbols, which have various meanings (Wilhelm, n.d). I-Ching is a traditional Chinese philosophy that emphasizes the dynamics between yin and yang as the basis for the whole world. This philosophy is depicted in Fig. 2.18 (in Cooper, 1972). It presents the bagua (trigrams) or the Pa Kua, as it is named in the Wade-Giles system, which has taiji (the great ultimate) or wuji (the great void) in the middle of the diagram (Cooper, 1972). The eight trigrams (in Cooper, 1972) can be expanded into the sixty-four hexagrams and each of them refers to each head of a chapter in the I-Ching (Cooper, 1972). It explains that every trigram “represents a force in nature and, necessarily partaking of the yin-yang principle, is either passive or active” (Cooper, 1972, p. 42). These trigrams will constantly interplay, act, and react due to the changes of the world, in which they reflect “in the whole realm of phenomena and all the antinomies” (Cooper, 1972, p. 42). The yijing and the I-Ching share some similarities, as both are deeply rooted in traditional Chinese philosophy. For example, the notion of the complementarity of yin and yang underlies the essence of the yijing as a relationship between xu (void) and shi (reality), which will be discussed in the next section. The yijing is a concept of aesthetics that an emphasis of artist’s emotional expression in the artworks. It has been extended to many art forms including paintings, calligraphy, poetry, literature and music. An ideal artwork in traditional Chinese art should contain the yijing experience, which the basic expression emphasizes as the artist’s inner emotion in relation to exterior natural objects and the achievement of harmonious experience between the subjective and the objective. The use of the term aesthetics needs to be defined. The Chinese translation of the term meixue, refers to the “study of beauty”. However this is not an exact equivalent in

3.4 Introduction of the Chinese Yijing Aesthetic

59

terms of the Western concept of aesthetics. In Western terms aesthetics refers to “sensual perception” that often involves the discussion of the beautiful, the picturesque and the sublime rather than just the study of beauty. The concept of aesthetics may also refer to the uncanny or ugly. However, in the Chinese tradition, the meaning of the beautiful is similar to the concept of “good”—both are interdependent (Pohl, 1996). In this research I adopt the Western term aesthetics to describe issues of art, art creation and art appreciation, rather than using meixue. Chinese aesthetics focuses on the fusion of Buddhism (Zen Buddhism) and the cosmology of Taoism, in which natural objects undergo a constant transformation in an attempt to comprehend the philosophy of life (Inada, 1997). Zen is a school of Mahayana Buddhism that began in China in the sixth century, and later spread to Vietnam, Korea and Japan. Thus, as a Chinese aesthetic concept, yijing is not only about emotion and spirit but is more focused on the expression of individual understanding of cosmological principles in Taoism. It is significantly different to the Western aesthetics because in the Chinese understanding life came not primarily from the activities of religion, philosophy or science, but more so from art (Rowley, 1959).

3.4.2 The Concept of Yijing The yijing aesthetic is an English translation, which is based on the Chinese pinyin system. It is an artistic concept that expresses the artist’s inner emotion through objects or scenes in the external world. The concept of yijing was first introduced in Wang Changling’s work Shige (The Poetic Style) in the Tang Dynasty in about the eighth century of the Common Era (Ye, 2005). Wang emphasizes d that the poetic styles have three jing (realms): wujing (related to natural scene), qingjing (related to emotion) and yijing (related to inner consciousness), which Wang has not provided a clear explanation for (Ruan, 1995). The three jings can be understood as three levels with the yijing as the highest level, building on the previous two levels. Since then, the concept of the yijing has evolved into many fields through the expansion and reinterpretation of its meaning—primarily in the areas of poetry, painting and calligraphy. Between the nineteenth and twentieth centuries, Wang Guowei, one of most important scholars on aesthetic study, expanded the concept of yijing to the liberal arts fields, such as opera and the novel. However, in the twentieth century, another influential scholar, Zong Baihua, extended the concept of yijing to all artistically related fields. In particular, he focused on the discussion of ancient Chinese paintings, calligraphy, architecture, sculpture and gardens. Most research and discussion of the concept of yijing is based on concepts of nature and the universe, which are rooted in Taoist philosophy, Confucianism and Zen Buddhism. As ancient Chinese people regarded nature with awe and reverence, nature was treated as a mysterious object above humans (Fan, 2001). In comparison, for Western thinkers, the size, mass or formlessness of nature prompted the development of sublimity as

60

3 Data Visualization and Aesthetics

a category of aesthetics, transcending the limits of the human comprehension and inspiring fear and awe (Kant, 2007). However, modern humanity has a much better understanding of nature and the universe than before. This has been achieved through scientific and technological inquiry. Knowledge of nature obtained by humanity becomes more complex, which results in nature becoming a less awe-inspiring object (and more a rationalized object) than ever before. Human power supplants natural awe through the construction of human-made buildings, such as dams, canals and railways, which some theorists have described as a “technological sublime” (Nye, 1994) (which will be discussed in Sect. 2.3.3.2). Unlike the sublime, there has not been much research into how the yijing goes beyond the traditional points of reference to the context of the contemporary technological age, in particular in relation to computer technology. Therefore, it is necessary to recognize the value of reinterpreting or restructuring the concept of yijing in the context of this digital technology, which is one of the aims of my research project. Although the concept of yijing has been active for over 1,000 years, there has been no agreed definition of the concept. It is often considered as one of the highest criteria for the evaluation of the traditional Chinese literature and art. It emphasizes personal emotion or perception and is defined or explained differently, according to different fields. For example, in language and literature, yijing can be understood as “the principles of illusion” (Liao, 2011, Abstract section, para 1). It can also be interpreted as an “artistic conception” (Liu & Bralewskiz, 2010; Chen, 2011). In the visual arts, yijing refers to the aesthetic quality of an artwork. It highlights the expression of an artist’s inner emotion through the representation of his or her subjective experiences of the extrinsic world. According to Li Zehou, one of the most important Chinese philosophers, yijing is similar to the Western concept of “empathy” consisting of “the melding of the appreciating (or creating) self with the appreciated (or created) object” (Li, 2009, p. 152). In Chinese aesthetics, this phenomenon of empathy is called qing jing jiao rong, referring to the fusion of feeling and scene, or the unity of self and object. It consists of two parts: qing, which refers to emotion; and jing, which refers to scene. The state of yijing therefore requires the fusion of feeling and scene. Li (2009) further explains that the fusion of feeling and scene is attained when: the appearance or action of the object calls forth my mental and emotional activity, which is subsequently dissolved in the full concentration of my faculties in the process of appreciation or creation, so that it is eventually replaced by the features and actions of the object, resulting in the unity of my own subjective emotions with the objective form. (p. 152)

Once the state described by Li is attained, “the beautiful” becomes a fusion of feeling and scene rather than a superficial or purely visual concept. Therefore, once yijing is attained, “reason dissolves completely into the emotions and imagination, and loses its independent character to become a sort of unconscious or nonconscious player” (Li, 2009, p. 153). Zong (2005) argues that the essence of yijing is the unification of xu (void) and shi (reality), which are two fundamental components in the yijing. These two components

3.4 Introduction of the Chinese Yijing Aesthetic

61

exist complementarily to generate one whole in a visual form, whose original source is the complementary concepts of yin and yang in Taoism. The concept of xu is explained as emptiness, nothing or void. It is composed of xu and wu, which are often presented together as xuwu (Fan, 2001). The concept of shi is explained as fullness, substance or reality. However, the concept of xu is considered more important than the concept of shi in the Chinese traditional thought system, as it was a key way to understand the universe for the ancient Chinese (Cheng, 1994). Xu is an essential principle of the Taoist ontology, and it is considered the original source for the creation of the whole universe. Cheng (1994) further explores the concept of xu in its practical application to the arts fields but also at the philosophical level, such as xu used in Chinese painting. In the process of making a painting, xu is required at every step including the basic brushstrokes and the overall composition—“it is a sign among the signs, providing the pictorial system with its effectiveness and unity” (Cheng, 1994, p. 64). In addition, rhythm or poetic rhythm is also important in traditional Chinese painting. The rhythm is constructed through the ideal employment of the complementarity of xu and shi as well as the arrangement of brushstrokes. It often involves poetry that is considered an artist’s emotional expression. The poem usually is written in the painting as a form of calligraphy. This kind of poetic expression in the painting requires reflection on matching the contents of poem to the contents of painting. For example, court painter Ma Yuan’s painting Walking a Mountain Path in Spring (Sung Dynasty) (Ma, Sung Dynasty), combines poetic rhythm and an ideal composition, specifically, diagonals with willow branches as the frame (Tregear, 1980). In the painting, the stroller’s contemplative stare and presence disrupts the birds and the mountains have faded into the fog. The artist emphasizes that xu (emptiness) describes the mountain in complementary composition to shi, which represents the stroller, the willow trees and others. These elements of the painting compose the scene with the effect of beauty, intensity and movement, as Emperor Ningzong inscribed in the poem in the painting: Brushed by his sleeves, wild flowers dance in the wind; Fleeing from him, hidden birds cut short their songs. (Kleiner, 2010, p. 66)

This poem reflects how the painting represents the scene of spring. However, the meaning can also be interpreted as signifying a woman in a palace stirred by the emperor’s presence through the painter and poet’s symbolical use of birds and flowers (Tregear, 1980). This painting expresses the experience of yijing very well, through depicting the harmony between the landscape of the image and the poetic calligraphy, and the fusion between the feeling evoked in the viewer/reader and the landscape. When an audience view the painting and read the poem, they will possibly feel the emotional intensity held by the artists, not just as expressed within the frame of the painting, but beyond the painting as well (Fig. 3.3). The aim of this research is not merely a study of Chinese aesthetics but the integration and application of this traditional Chinese aesthetic to creative practice

62

3 Data Visualization and Aesthetics

Fig. 3.3 Walking a mountain path in Spring. Ma Yuan, (Sung Dynasty), hand-scroll in ink on silk, 27.4 × 43.1 cm, collection of National Palace Museum, Taipei, Taiwan

in the contemporary context of digital technology. Therefore, a broad discussion of the concept of yijing and its background has been presented. The general visual forms of yijing and its characteristics will be explored in the following section.

3.4.3 The Forms of Yijing In this section, the forms of yijing will be discussed in terms of Chinese traditional painting, which reflects Taoist concepts. Traditional Chinese painting, which was mainly inspired by Taoist philosophy and later enhanced by Zen philosophy, emphasizes spiritual concepts rather than realistic representations of nature (Cheng, 1994). Its visual form is normally as a monochrome painting, with black strokes painted on white paper. The white space and black strokes are represented as the concepts xu and shi. According to Taoism’s dualism, traditional Chinese painting involved combining two fundamental elements: xu (void) and shi (reality), which exist as two complementary components. The aspect of black (shi) in the painting was “the visible, tangible and manipulable nature of things” (Inada, 1997, p. 125). The black strokes were created through the complementary juxtaposition of white and black elements, which are regarded as xu and shi and “a black-in-nonblack phenomenon” (Inada, 1997, p. 125). As discussed above, xu represents emptiness, nothingness or void. However, the concept of xu in

3.4 Introduction of the Chinese Yijing Aesthetic

63

Table 3.1 The forms of yijing in visual art Forms

Description

Xu

Emptiness/unclear; Nothingness or void; The concept includes whiteness, space, unlikeness and unrealistic objects

Shi

Tangible; Focus/clarity; Reality and fullness; The concept includes blackness, density; thickness and likeness

Unity

Complementary elements; Harmony/balance

Rhythm

Repeated visual patterns timed; Visual compounds

traditional Chinese painting also extended to other things, such as whiteness, space, unlikeness and unrealistic objects. The concepts of likeness and unlikeness are often used to describe objects in terms of Chinese aesthetics in traditional painting— “likeness” usually considered as realistic, and “unlikeness” as unrealistic or abstract. As Table 3.1 shows, shi is not only represented as reality and fullness, but also as blackness, density, thickness and likeness. All elements in the painting have been unified into a whole, including notions of polarity (high-low, far-near and big-small) through the mutuality and conjunction of xu and shi.

3.4.4 The Characteristics of Yijing Table 3.1 details the characteristics of the yijing aesthetic. In this research, the concept of yijing is considered an emotional harmony between subjectivity and objectivity. As a philosophical concept, subjectivity usually refers to a subject who possesses “particular perspective, feelings, beliefs, and desires” (Honderich, 2005, p. 900). It has also often been considered as the realm of experience since Descartes (Honderich, 2005). In contrast, objectivity involves “deternmining beliefs […] our judgments themselves can be objective” (Bunnin & Yu, 2004, p. 484). When objectivity relates to facts or involves being true or false, a judgment is considered objective (Bunnin & Yu, 2004). In this project, I consider data and technology as objective and the users’ experience as subjective. To achieve the state of yijing, artists or designers have to reach a feeling of fusion with the scene represented in the data visualization. In other words, when they are able to experience unity (achieve fusion) of their personal, subjective emotions (feelings) with the objective forms (scene) in the visualization works, the state of yijing is attained. Then the visualization reflects the state of yijing, evoking emotional harmony in users. In short, users will have a similar emotional experience to artists or designers. Xu is represented as emptiness, space or indistinct objects in an image.

64

3 Data Visualization and Aesthetics

Table 3.2 The characteristics of yijing in visual art Aesthetics

Forms

Characteristics

The yijing aesthetic: “artistic conception”; “the principles of illusion”

Xu (void, empitiness)

Xu represents unrealistic objects that include whiteness, spacing, fineness and unlikeness. Those objects are less important but are essential parts

Shi (reality)

Shi represents tangible objects such as main objects. It includes blackness, density, thickness and likeness

Unity

All elements or objects unite as one whole; It includes unity of color, shapes, and lines.

Rhythm

A strong, regular repeated visual pattern, which can be achieved through a good arrangement of xu and shi

Shi is represented as tangible objects that are focused with clarity, which is normally used to depict the main objects, while xu is used for less important objects. However, this does not mean shi is more important than xu as they are complementary concepts. In traditional Chinese painting, artists prefer themes of creativity that are related to landscape with elements “far”, “high far” and “deep far” (Ye, 1998). According to Ye (1998), objects such as water and mountains are tangible, but “far” extends past tangible objects, focusing the audience’s attention to distant areas in the composition, in order to shift from limited time and space to unlimited time and space. This is very important as this dynamic between near and far can stimulate the imagination through harmonious and balanced images, which evokes unity through all the elements achieving a rhythm (Table 3.2). Western aesthetics differs from the Chinese aesthetics focus on harmony, in its focus on concepts of the beautiful, the sublime and picturesque as applied to the appreciation of art or natural phenomena (Dickie, 1971). The concept of the sublime is one of the most important concepts in Western aesthetics, as artworks are usually considered having sublime qualities. In this sense, understanding the notion of the sublime facilitates the understanding of aesthetic approaches to data visualization as an expression of digital creativity rather than as a tool for objective communication of data.

3.5 The Concept of the Sublime

65

3.5 The Concept of the Sublime The sublime was originally used in rhetoric in literature. The Greek philosopher Longinus was the first scholar who used the sublime in his work On the Sublime (written around 1st century, CE), which focuses on the effect of powerful speaking and writing. In Longinus’s work, the sublime refers to a style of speaking and writing that has been elevated above the ordinary. It was used to describe language or thought that is great or elevated. In the latter, the term “the sublime” has been used to describe a special perception of human beings, generating a sensation of awe that exceeds the ability of comprehension. The concept of the sublime has been developed as an aesthetic quality of nature that is distinct from the concept of the beautiful, which “is small, contained, curved, delicate; that which is sublime is vast, unbounded, jagged, and harsh” (Edmund Burke as cited in Korsmeyer, 2005, p. 57). Kant (2007) distinguishes the sublime from the beautiful as follows: The beautiful in nature is a question of the form of the object, and this consists in limitation, whereas the sublime is to be found in an object even devoid of form, so far as it immediately involves, or else by its presence provokes, a representation of limitlessness, yet with a super-added thought of its totality. (p. 75) Fig. 3.4 shows a beautiful sunset in well-known beach in West Australia. Therefore, the “beautiful” is usually presented in a form having limits while “sublime” is represented as formless and limitless. For example, a flower as a form is

Fig. 3.4 The sunset at Cottesloe beach, Perth, Western Australia. Photography by Qi Li

66

3 Data Visualization and Aesthetics

called “beautiful” and the vast objects, such as mountains or oceans, can be described as the sublime. The sublime often involves “things that are powerful and terrifying to such a degree that they actually may threaten the annihilation of the spectator” (Korsmeyer, 2005, p. 57). This aesthetic concept is also considered as something that “inspires awe, grandeur, and evokes a deep emotional and/or intellectual response” (Kosara, 2007, One section, para. 1). According to the Kant’s definition, the sublime is developed to explore the limitation of human perception and a particular relationship to nature, which delimits the subjective boundaries of appreciation in relation to the vastness of the object.

3.5.1 The Kantian Sublime Kant, one of the most influential Western philosophers, distinguishes the beautiful from the sublime in his book Critique of Aesthetic Judgment. He analyses the beautiful in the First Book (Analytic of the Beautiful) through four “moments.” The first moment identifies the nature of pleasure or displeasure concerning how people feel about something, in which pleasure refers to a “disinterested” kind meaning “that it does not depend on the subject’s having a desire for the object, nor does it generate such a desire” (Ginsborg, 2013, What is a Judgement of Beauty? section, para 2). The second moment identifies judgement as a function of universal delight independent of conceptual reasoning, as “for beauty is not a concept of the object, and the judgment of taste is not a judgment of cognition” (Kant, 2000, p. 170). The third identifies finality in general, emphasizing the discussion of the purposiveness of that judgement of taste based on apprehension of the purposiveness of objects. Kant (2000) clarifies this condition as “the form of the purposiveness of an object” (p. 120). The fourth moment emphasizes the judgement of taste in which the beautiful is cognized “as object of a necessary satisfaction” (Kant, 2000, p. 124) apart from a concept. According to Kant cited in Cooper (2007a, p. 95): appreciation of beauty is tied to recognition of form and apparent design, it is especially important to distinguish and explain a different mode of appreciation appropriate to the ‘mathematical’ vastness of the unlimited heavens and the ‘dynamic’, formless might of oceans and volcanoes.

The object of this appreciation is considered “the sublime” (Kant, 1952). Kant further indicates that a judgement of taste is purely an aesthetic function only if it is “disinterested”, without desires or needs. He analyses beauty in nature and judgements of taste and concludes that “‘art can only be termed beautiful’ when it has ‘the appearance of nature’” (Kant, 1952, p. 95). Kant (1952) analyses the sublime in the Second Book (Analytic of the Sublime). He distinguishes the two forms of the sublime: the mathematical and dynamic sublime. Kant (2007) describes the mathematical sublime as follows:

3.5 The Concept of the Sublime

67

Sublime is the name given to what is absolutely great … [T]hat is sublime in comparison with which all else is small … We get examples of the mathematically sublime of nature in mere institution in all those instances where our imagination is afforded, not so much a great numerical concept as a large unit as measure (for shorting the numerical series). A tree judged by the height of man gives, at all events, a standard for a mountain; and, supposing this is, say, a mile high, it can serve as unit for the number expressing the earth’s diameter, so as to make it intuitable; similarly the earth’s diameter for the known planetary system; this again for the system of the Milky Way; and the immeasurable host of such systems, which go by the name of nebulae, and most likely in turn themselves form such a system, holds out no prospect of a limit. (p. 78–87)

Kant suggests that the mathematical sublime is unlimited in terms of measurement of the object. It can be understood in terms of units or sizes that are large enough to go beyond a human’s sensation or imagination. In the case of the dynamical sublime, Kant (2007) suggests “nature considered in an aesthetic judgement as might that has no dominion over us, [which] is dynamically sublime” (p. 120). Knowing they are safe helps humans feel less afraid even if they are in a fearful situation in nature. The power of the human intellect, in the dynamic sublime, overcomes the mathematical vastness of nature.

3.5.2 The Technological Sublime The notion of the aesthetics in the early period of European philosophy was inspired by nature. The sublime was initially used to explore the limitation of the relationship between humanity and nature that tests subjective capacities, as implicit to the Kantian sublime. In the nineteenth and twentieth centuries, however, the natural sublime experienced in landscape was gradually replaced in theoretical discussions by the technological sublime, which is an experience evoked by constructed artefacts, such as factories, aviation buildings, automobiles or computers (Mul, 2011). Cultural Historian Leo Marx coined the term “technological sublime” (later taken up by David Nye), which is considered to be an American version of the sublime (Marx, 2000). Both nature and technology were considered sources of the experience of the sublime by American setters in the early period of settlement (Nye, 1994). In particular, with the secularization of modern society, God no longer had an important role for humans (Giblett, 2008), and nature became less threatening and interesting to humanity. The technological sublime indicates humanity’s power to overcome and control the power of the natural world by building dams, canals and railways. Thus, the “power of divine nature has been transferred to the power of human technology” (Mul, 2011, para 12). Man-made objects, such as railways, dams and skyscrapers demonstrate the human ability to conquer nature, as modern society prefers to take technological command of nature instead of being overpowered by it (Mul, 2011). This particularly relates to contemporary digital technology, for example, video games can be considered a technological sublime, as aesthetics pervades the whole media (Quaranta, 2006). The technological sublime combines both Kant’s mathematical and dynamical sublime. Instead of self-reflection in the

68

3 Data Visualization and Aesthetics

mathematical sublime, the technological sublime deals with awe-inspiring size or complexity in a visual form. The affect from the technological sublime begins with the sensation of what Kant expresses an indirectly feeling a “momentary inhibition of the vital forces” which is then “developed as an amalgam of pleasure and pain, excitement and terror” (Shinkle, 2010, p. 4). Shinkle (2010) discusses the notion of the contemporary technological sublime in the context of digital technology with an example of video games, which are considered aesthetic forms. She suggests that the technological sublime is an affect incorporating awe and terror and inspiring emotion and banality together in which the subject encounters the limitation of the self. The technological sublime is associated not only with technology but also with banal affect, which is related to boredom and ennui. The notion of the banal reflects an infinite cycle of unfulfillable desire in post-industrial culture, and the technological sublime satisfies this desire. It could be argued that the combination of the sublime and the banal underlies many forms of entertainment and technology. A video game is related to the mathematical sublime, which represents the complexity of technology, particularly through the extent of coding. The complexity of the 3D technology in a video game is not easy to comprehend for most users. The game deeply involves players in a tedious engagement with a machine and tasks are often repeated again and again for the duration of the game. This kind of experience is described through the term of “stuplimity” (Sianne Ngai cited in Shinkle, 2010, Gameplay: Stuplimity or Flow? section, para. 2), which combines aesthetic awe interlaced with boredom. However, Shinkle (2010) argues that stuplimity cannot be represented as experience of the technological sublime, as it cannot “reveal and affirm the boundaries of the self, nor does it act to situate the subject in relation to the radically other” (Shinkle, 2010, Gameplay: Stuplimity or Flow? section, para. 5). Shinkle (2010) considers that stuplimity is just a superficial engagement that does not challenge “subjective boundaries and an affirmation of the powers of reason, the subjective experiences an attenuation of self in the guise of entertainment” (Gameplay: Stuplimity or Flow? section, para. 5).

3.5.3 The Forms of the Sublime Table 3.3 outlines the main forms of the sublime in terms of the Kantian sublime and the technological sublime. As the technological sublime is considered an extension Table 3.3 The forms of the sublime The sublime

Forms

The Kantian sublime

Any natural objects those appear incomprehensible in size or great in complexity

The technological sublime

Has a similar sublime effect to Kant’s sublime, but tends to be applied to digital technology contexts

3.5 The Concept of the Sublime

69

of Kant’s sublime, both forms of the sublime share the same characteristics. Kant’s sublime explores the relationship between humanity and nature, while the technological sublime focuses more on human achievement. It is not very clear what precise visual forms the technological sublime might take, but it involves the emotion of boredom and the performance of an endless round of unattainable desires. A game is an ideal application of such a sublime.

3.5.4 The Characteristics of the Sublime The sublime experience involves an affective mixture of positive and negative emotions, resulting in a complex subjective experience. For Kant’s sublime, natural objects with sublime qualities challenge the limits of human comprehension and sensibility and evoke awe, fear and a sense of greatness. As the technological sublime is drawn from Kant’s concept of the sublime, it combines understanding of Kant’s awe and terror with the experience of banality that is often semantically connected to boredom and ennui. Both banality and boredom or ennui are considered characteristics of the contemporary digital sublime, particularly in relation to gaming contexts (Shinkle, 2010), symbols of modernity “born out of shifting labour patterns and the novelty of unfilled time” (Shinkle, 2010, Aesthetics, the Subject, and the Technological Sublime section, para. 9). Banality relates to post-industrial culture that is “feeding on an endless cycle of unfulfilled and unfulfillable desire” (Shinkle, 2010, Aesthetics, the Subject, and the Technological Sublime section, para. 9). This banality is a way for humanity to attempt to satisfy this desire as well as attain the objects and rituals of their desire. Table 3.4 outlines the characteristics of the sublime in terms of both the Kantian and the technological sublime. The discussions are keys to exploring the aesthetic data visualization, which will address further in Chap. 5. The concept of the sublime has been used to describe data-based creative practice, specifically in terms of the technological sublime. In next chapter, data visualization and digital art practice will be discussed. As data visualization has been used as creative practice in the past few years, artists create data-based digital art practice in the various forms including static and dynamic forms. Table 3.4 The characteristics of the sublime The sublime

Characteristics

The Kantian sublime

Evokes awe and fear; Greatness

The technological sublime

A combination of elevated emotion (awe and terror) and banality (boredom and ennui)

70

3 Data Visualization and Aesthetics

3.6 Conclusion In summary this chapter reviews the key concepts of aesthetics and aesthetic underpin data visualization, the introduction of Chinese yijing aesthetic, and Western aesthetics with discussion of their fundamental forms and characteristics. Through the discussion of these concepts and theories, this book articulates the aesthetic value in design of data visualization for not just comprehension or more engagement but also on new understanding of data related to social, scientific and cultural transformation. This interdisciplinary research focused on the discussion of the relationship between data visualization and aesthetics with emphasis on aesthetic promotion of data visualization, which involves the discussion of traditional techniques for data visualization, Chinese and Western aesthetics, in particular in exploration of concept of the yijing aesthetic and the Kantian sublime. The review presents aesthetics in two different cultures that the yijing aesthetic emphasizes harmony with the complementary of xu and shi as visual form, while the Kantian sublime emphasizes experiencing awe or greatness of nature through mathematical sublime and attempting overcome the nature through dynamic sublime. The exploration of two different aesthetic concepts is needed for a further discussion of this book in the theoretical framework.

References Aesthetics in Asia. (n.d.). Retrieved from https://science.jrank.org/pages/8187/Aesthetics-in-AsiaChina.html. Bennett, C., Ryall, J., Spalteholz, L., & Gooch, A. (2007). The aesthetics of graph visualization. Computational Aesthetics, 2007, 57–64. Boehner, K., Sengers, P., Medynskiy, Y., & Gay, G. (2005). Opening the frame of the art Museum: Technology between art and tool. In Proceedings of the Digital Arts and Culture 2005 Conference. Copenhagen, Denmark. Brath, R., Peters, M., & Senior, R. (2005). Visualization for communication: The importance of aesthetic sizzle. In Ninth International Conference on Information Visualisation (IV’05) (pp. 724– 729). IEEE. Bunnin, N., & Yu, J. (2004). The Blackwell dictionary of Westerphilosophy. Oxford: Blackwell. Cawthon, N., & Moere, A. V. (2007). Qualities of perceived aesthetic in data visualization. In Proceedings of CHI Conference on Human Factors in Computing Systems. San Jose, USA. Card, S. K., Mackinlay, J. D., & Schneiderman, B. (1999). Readings in information visualization: Using vision to thin k. San Francisco: Morgan Kaufmann. Chen, C. (2005). Top 10 unsolved information visualization problems. Computer Graphics and Applications, IEEE, 25(4), 12–16. Chen, G. (2011). Landscape architecture: planting design illustrated. Irvine, CA: ArchiteG Inc. Cheng, F. (1994). Empty and full: The language of Chinese painting (M. H. Kohn, Trans.). Boston, MA: Shambhala Publications. Cooper, D. E. (Ed.). (1997). Introduction. Aesthetics: The classic readings. Oxford: Blackwell. Cooper, J. C. (1972). Taoism: The way of the mystic. Wellingborough: Aquarian Press. Danto, A. C. (2003). The abuse of beauty: Aesthetics and the concept of art. Open Court Publishing. Dickie, G. (1971). Aesthetics: An introduction. New York, NY: Pegasus. Fan, M. (2001). Ecological consciousness in traditional Chinese aesthetics. Educational Philosophy and Theory, 33(2), 267–270.

References

71

Fishwick, P. (2006). Aesthetic computing. Cambridge, MA: MIT Press. Friedman, V. (2008). Data visualization and infographics. Retrieved from http://www.smashingm agazine.com/2008/01/14/monday-inspiration-data-visualization-and-infographics/. Gaviria, A. R. (2008). When is information visualization art? Determining the critical criteria. Leonardo, 41(5), 479–482. Giblett, R. J. (2008). Sublime communication technologies. Basingstoke: Palgrave Macmillan. Ginsborg, H. (2013). Kant’s aesthetics and teleology. Stanford encyclopedia of philosophy. Retrieved from http://plato.stanford.edu/entries/kant-aesthetics/. Healy, K. (2018). Data visualization: A practical introduction. Princeton University Press. Herdeg, W. (1983). Graphis diagrams: The graphic visualization of abstract data: Die Graphische Visualisierung Abstrakter Gegebenheiten. Zurich, Switzerland: Graphis Press. Honderich, T. (Ed.). (2005). The Oxford companion to philosophy (2nd ed.). Oxford: Oxford University Press. Inada, K. K. (1997). A theory of oriental aesthetics: A prolegomenon. Philosophy East and West, 47(2), 117–131. Janaway, C. (2006). Reading aesthetics and philosophy of art—Selected texts with interactive commentary. Oxford: Blackwell. Jiang, C., & Cai, Z. (2018). Aesthetics. In Oxford Bibliographies. Jorgenson, L., Kriz, R., Mones-Hattal, B., Rogowitz, B., & Fracchia, F. D. (1995). Is visualization struggling under the myth of objectivity? In The 1995 6th Annual IEEE Conference on Visualization (pp. 412–415). Kant, I. (1952). The Critique of Judgement: Part 1, Critique of Aesthetic Judgment. Trans. by James Creed Meredith. Oxford: Clarendon Press. Kant, I. (2007). Critique of judgement (J. C. Meredith, Trans.; N. Walker Ed.). New York, NY: Oxford University Press. Kant, I. (2000). Critique of the power of judgement (P. Guyer & E. Matthews, Trans.; P. Guyer Ed.). New York, NY: Cambridge University Press. Karcher, S. (1999). Jung, the Tao and the classic of change. Harvest: A Journal for Jungian Studies, 45(2), 122–143. Kelly, M. (1998). Encyclopedia of aesthetics. Oxford University Press. Kleiner, F. S. (2010). Gardner’s art through the ages: Non-western perspectives (13th ed.). Boston, MA: Wadsworth. Korsmeyer, C. (2005). Terrible Beauties. In M. Kieran (Ed.), Contemporary debates in aesthetics and the philosophy of art (pp. 51–65). Oxford: Blackwell. Kosara, R. (2007). Visualization criticism—The missing link between information visualization and art. In Proceeding of the Information Visualization, 2007. IV ‘07, 11th International Conference (pp. 631–636). Zurich, Switzerland: IEEE. Krieger, L. S., Neill, K., & Jantzen, S. L. (1994). World history: Perspectives on the past. DC Heath & Company. Li, Z. (2009). Chinese aesthetic tradition (M. B. Samei, Trans.). Honolulu, USA: University of Hawaii Press. Lang, A. (2008). Aesthetics in information visualization. Trends in information visualization: Hauptseminar Medieninformatik WS 2008/2009. Retrieved from http://141.84.8.93/pubdb/pub lications/pub/baur2010infovisHS/baur2010infovisHS.pdf#page=16. Lau, A., & Moere, A. V. (2007). Towards a model of information aesthetics in information visualization. In Proceedings of Information Visualization, 2007. IV’07, 11th International Conference (pp. 87–92). Zurich: IEEE. Liao, T.-L. (2011). Yijing: Mr. Wang meng-ou’s aesthetic of language. Danjiang Chinese Journal, 25(1), 131–172. Liu, S., & Bralewskiz, T. W. (2010). Chinese Landscape Inspiration —“YIJING” (Special issue). Acta horticulturae et regiotecturae, 92–96. Marx, L. (2000). The machine in the garden: Technology and pastoral ideal in America. New York, NY: Oxford University Press.

72

3 Data Visualization and Aesthetics

Mul, J. D. (2011). Essay: The technological sublime. Retrieved from http://www.nextnature.net/ 2011/07/the-technological-sublime/. Norman, D. (2004). Emotional design: Why we love (hate) everyday things. New York, NY: Basic Books. Nye, D. E. (1994). American technological sublime. Cambridge, MA: MIT Press. Osborne, H. (1986). Aesthetic experience and cultural value. The Journal of Aesthetics and Art Criticism, 44(4), 331–337. Pohl, K. (1996). Chinese aesthetics and Kant. Twenty-first Century, 8(36), 84–89. Pollock, S. (2016). A Rasa reader: Classical Indian aesthetics. Columbia University Press. Quaranta, D. (2006). Game aesthetics: How videogames are transforming contemporary art. In M. Bittanti & D. Quaranta (Eds.), GameScenes: Art in the age of videogames (pp. 297–308). Milano, Italy: Johan and Levi. Rowley, G. (1959). Principles of Chinese painting. Princeton, NJ: Princeton University Press. Ruan, G. (1995). The disscussion of Wang Changling’s contribution to yijing theory. Journal of Guangdong Institute for Nationalities, 2(34), 1–8. Sack, W. (2011). Aesthetics of information visualization. In M. Lovejoy, C. Paul, & V. Vesna (Eds.), Context providers: Conditions of meaning in media arts. Intellect: Bristol, UK. Shinkle, E. (2010). Video games and the technological sublime. Retrieved from http://www.tate. org.uk/research/publications/tate-papers/video-games-and-technological-sublime. Tateosian, L. G. (2005). Characterizing aesthetic visualizations. North Carolina State University. Tateosian, L. G., Healey, C. G., & Enns, J. T. (2007). Engaging viewers through nonphotorealistic visualizations. In Proceedings of the 5th International Symposium on Non-photorealistic Animation And Rendering (pp. 93–102). New York, NY: Association for Computing Machinery. Tractinsky, N. (2004). Towards the study of aesthetics in information technology. In Proceedings of Twenty-Fifth International Conference on Information Systems (pp. 771–780). Washington, DC. Tregear, M. (1980). Chinese art. London: Thames and Hudson. Tufte, E. R. (1983). The visual display of quantitative information graphics press. Cheshire, Connecticut. Tufte, E. R. (1997). Visual explanations: Images and quantities, evidence and narrative. Cheshire, Connecticut: Graphics Press. Tufte, E. R. (2001). The visual display of quantitative information. Cheshire, Connecticut: Graphics Press. Tufte, E. R. (2007). Beautiful evidence. Cheshire, Connecticut, Graphic Press. Wilhelm, R. (n.d.). Introduction. Retrieved from http://www.iging.com/intro/introduc.htm. Ye, L. (1998). An analysis of Yijing. Research of Literature and Art, (1). Ye, L. (2005). A compendium of Chinese aesthetic history. Shanghai: Shanghai People’s Press. Zong, B. (2005). A walk of aesthetics. Shanghai: Shanghai People’s Press.

Chapter 4

Data-Based Digital Art Practice

This chapter is overview of data visualization as digital art practice. As an emerging phenomenon in digital art, data visualization has been used for creative practice in the last decade (Corby, 2008). Research has explored how data visualization benefits the contemporary understanding of visual arts practice and objects of creative work (Corby, 2008). Such data-based digital art emphasizes actual data instead of the surface appearance of the visualization or the application of metaphors (Viegas & Wattenberg, 2007; Manovich, 2015). This form of art is also termed “data visualization art” or “data art”. It usually uses computer technology to create artworks or artefacts, such as computer software to implement numeric data into programs, which convert data in visual or audio forms. During this process, data is reinterpreted according to the artist’s creative purposes through data existing on a disk, the Internet or generated by the users (Wands, 2006). To make data meaningful to users, it is essential to create data art through filtering or mapping information as part of the visualization process (Paul, 2003; Wands, 2006). This chapter provides an overview of digital art with the definition of digital art. It then discusses this concept of data art through the exploration of some key data-based digital artworks, which are related to this research. In particular, this section will examine the data art using sleep EEG data.

4.1 Digital Art and Data Visualization Digital technologies have becoming an important part in our society and an emerging research area on arts and humanities. The technology has been used to create a new form of art—digital art since the 20th century. The team of digital art has undergone several name changes, such as “computer art”, “multimedia art”, “cyberarts”, or “new media art” (Paul, 2016; Nadarajan, n.d.). Digital art has become an increasing role of contemporary artistic practice in the last decade. Digital art challenges traditional art historical terms and categories (Drucker, 2013). As any other contemporary © Shanghai Jiao Tong University Press 2020 Q. Li, Embodying Data, https://doi.org/10.1007/978-981-15-5069-0_4

73

74

4 Data-Based Digital Art Practice

artistic practice or moments, digital art refuses to accept artistic “objectification” and challenges traditional concept of the artistic objects. Comparing to traditional art, digital art is an inherently time-based, dynamic, interactive, collaborative, customizable, and variable art form (Zari¸na, 2014). The themes in contemporary society have constantly challenged human life and societies, such as genetic engineering (Hauser, 2008; Savini, 2017), post-human issues (Hershman-Leeson citedn in Grau, 2017) and ecological crises (Morgan, 2019). The traditional art forms, such as paintings or sculpture, have a very limited scope to create a visual representation of the contemporary issues. However, digital art, termed as “the art of our time”, has “more multifarious potential for expression and visualization; although underrepresented on the art market, which is driven by economic interests” (Grau, 2017, p. 23). Data visualization has become one of important themes for artists to represent data or dataset in the information age. Data artists like to deal with things that others overlook, and there are many original works based on their processing of statistical data or information (Colson, 2007). Colson (2007) suggests that the challenge “is to provide a suitable interface for these data and allow it to contribute effectively to the artist’s ideas” (p. 66).

4.1.1 Defining Digital Art To define the notion of digital art is not easy. Digital art can be defined as a term that “comprises all art using digital technologies at some point in the process of its creation, storage, or distribution” (Paul, 2016, “Introduction section”). According to Stanford Encyclopedia of Philosophy, digital art refers to “art that relies on computerbased digital encoding, or on the electronic storage and processing of information in different formats—text, numbers, images, sounds—in a common binary code” (Thomson-Jones & Moser, 2015). There many ways to create a digital art based on computer technologies. For example, a digital photography can be an artwork of “a manipulated sample of visual information captured with a digital camera from a “live” scene or captured with a scanner from a traditional celluloid photograph.” (ThomsonJones & Moser, 2015). Paul (2016) defines digital art as “born digital, and created, stored, and distributed via digital technologies, still is far from a unified category” (p. 2). Digital art can be categorized to numerous forms: “interactive or networked installations; software or Internet art without any defined physical manifestation; virtual reality or augmented reality; locative media art distributed via mobile devices” (Paul, 2016, p. 2). These include smartphones, or location-based technologies ranging from the global positioning system (GPS) to radio frequency identification (RFID). Therefore, it should be understand that a digital artwork is not just film or video, sound art or digital display a traditional painting, but the themes in digital arts have extended in many ways to the digital medium. Digital art in this sense focuses on the theme of artificial life and intelligence; telepresence and telerobotics, database aesthetics, mapping, and data visualization. Some other themes are also included, such as tactical media, gaming environment and narrative hypermedia environment

4.1 Digital Art and Data Visualization

75

and body and identity. Digital art relies on digital technologies, which can be used as a tool and as a medium. At the beginning of computer age, digital technologies have been commonly employed as a digital tool to produce a more traditional art form, such as a print. Digital technologies can also be used to store and deliver the works, which means they are digitized version of a painting on a DVD or a video format.

4.2 Code as Creative Medium A digital artwork is created through computer programming and encoding. Palyka addresses the issues of this digital create process as the following: Programming is a step-by step process where the programmer must understand what is happening at every step, but the images one makes using this process need not be totally ‘understood.’ The images themselves do not have a strict logical tie to the images which follow. The creation of images must have some logical visual flow from one to another to add order to the composition but, unlike the programming which generates it, the system of images will not collapse from the slightest deviation from logical conscious order. In fact, the deviations are what give emotion and tension to the piece. As in painting, with the computer art medium subconscious ‘not-understood’ constructions can co-exist with conscious ‘well-understood’ constructions”. (Crowther, 2018, p. 21)

This statement describes the creative experimentation and personal expression involved programming design. It highlights the importance of computer programming. Digital artists have not satisfied to use computer programming as a creative tool, but use programming as a medium for art practice. It has influenced not only the working methods but also the presenting forms of the work. The code has provided the artists “with a different register of sonority and texture, extending their palette of options in certain specific directions” (Colson, 2007). Marvin Minsky suggests that Programmers write – not “sequences” [of instructions] -but specifications for the individuals of little societies. Try as he may he will often be unable to envision in advance all the details of their interactions”. (Colson, 2007, p. 95)

This statement emphasizes that the works created by programming is unpredictable and no one knows the final outcomes of the code. Digital artist have worked with Java, C++, Visual Basic and Python and others. Figure 4.1 presents a code written in Processing language. It shows the setup function and the draw function. In the setup block, the size of sketch is defined as 700 × 700 pixels. In the draw block, it defines the background color by filling parameters, which are (32, 48, 65) following by noStroke function which means no outlines. After that there is a loop statement, which draws circles or rectangles according to the movements of a mouse. A code consists of functions and statements such as class, loops, variables, conditional statement, integers and float numbers.

76

4 Data-Based Digital Art Practice

Fig. 4.1 A computer program showing two blocks of code

4.3 Data-Based Art In the data-based digital art, data is an essential element of the works. There many different types of data available in our society, it could be very useful to consider them in different categories. Colson (2007) divides the common data used for data artists as three categories, which based on the location where the data stored. (1) “Local data from the immediate environment”. For example, data information can be available from a personal computer’s local hard drive, which may store numerous data or dataset. Data may also gain from surrounding, such as a webcam, responses, sound, tactile inputs, temperature, and movement and so on. Technologies required software includes PHP, MYSQL, which used for database enquiries and data collection. Hardware may include sensors, webcams, micro controllers. (2) “Data collected via the Internet using user inputs”. For example, database information that is dynamically updated remotely and made available over the website, including stock market values, weather report, news headlines, currency rates, online social media, and commodity prices. In this category, data collected at a remote location over the Internet and then sent to a remote server so that it can be used in somewhere else. The data is specifically used in online gaming or remote manipulation. Software required includes PHP, MYSQL for database enquiries and data collection with the Internet access. Hardware are required, such as sensors, webcams, protocol converters (HTTP to Serial, Serial to HTTP) micro controllers. (3) “Information collected from different places, and then made available to control events at one specific location”. In this category, data or information from GPS device have been used for tourists to visit a city or attend a conference. It requires a GPS receiver and transmitter with the Internet access and software used to program to interpret and use GPS data. It has been summarized in Table 4.1. Data-based digital art can be broadly divided into two categories: static data art and dynamic data art. While static data art often adopts digital technology as a tool

4.3 Data-Based Art

77

Table 4.1 The data source for data art Data category Local data from the immediate environment

Data location A computer’s local hard drive

Data collected via The Internet the Internet using user inputs

Information collected from different places used at a specific location

Data source

Technologies Hardware

Software

Webcam responses, sound, tactile inputs, temperature, movement

Sensors, webcams, micro controllers

PHP, MYSQL

Stock exchange values, weather report, news headlines, currency rates, online social media, commodity prices

Sensors, webcams, protocol converters, micro controllers

PHP, MYSQL; Internet access

GPS (global position system) GPS receiver and transmitter

Internet access; Application to interpret and use GPS data

Current location GPS data of a visitor

to create images, dynamic data art focuses on using digital technology as a medium to create interactive artefacts. It is established that dynamic data art represents the majority of data-based digital practice as it allows participants to “navigate visual and textural information and experience changes over time” (Paul, 2003, p. 177). Static data art is often adopted pre-recorded data to transform data to a graphical representation using algorithm. It does not require the Internet access or interactivity. Nevertheless, dynamic data art adopts online data to present an interactive representation of data in real time. It highlights the interactive with users or audience. The next section will discuss a number of data-based art examples including static data art, dynamic data art and a specific data-based art practice, which emphasizes the experience of the power of digital technology.

4.3.1 Static Data Art These static data visualizations involve the creation of digital artworks through digital compression technologies that enable complex data to be converted into compact forms for easy transfer or storage in digital media (Hope & Ryan, 2014). In visual art, compression technologies often use common file formats, such as JPEG, MP3 or MPEG (Hope & Ryan, 2014).

78

4 Data-Based Digital Art Practice

Jason Salavon’s digital artwork Homes for Sale, presents a series of photographs representing every home for sale in the USA (Viegas & Wattenberg, 2007). The artwork was created by averaging the color values of each image. The outcome presents a blurred image of a city’s pattern of weather and ghostly looking images of the houses. This work, which is based on numeric data, displays a sublime quality that has often been a criterion for the judgement of artworks, as suggested previously in this chapter. Jussi Ängeslevä and Ross Cooper’s digital artwork Last Clock (2005), investigates the visual representation of time and space (Viegas & Wattenberg, 2007, Fig. 4.1). The clock, which is similar to an analogue clock with second, minute and hour hands, was created with video footage from surveillance cameras around cities. It consists of three rings, in which the outside ring is assigned to the second hand, the middle ring is assigned to the minute hand and the inner ring is assigned to the hour hand. Each hand represents a slice of live video, when they spin around the clock “they leave a trace of what has been happening in front of the camera” (Virgas & Wattenberg, 2007, p. 8). This digital artwork “displays a space’s history and rhythm. Trails behind its hands paint the clock face with a video feed, creating a mandala of archived time” (Ängeslevä & Cooper, 2005, p. 20). The key to this work is its abstraction, its “aesthetically pleasing quality lends itself better to the background awareness of the remote place than a literal, direct video feed would” (Ängeslevä & Cooper, 2005, p. 20) (Fig. 4.2).

4.3.2 Dynamic Data Art John Simon’s data visualization Every Icon (2002), is considered a classic example for experiencing the magnitude of technology, which goes beyond human perception through the experience of the sublime. This work is presented as a grid of 32 × 32 squares that was designed online by systematically filling each square in every second in either white or black. Because there are 256 squares (32 × 32 = 256), there are 2256 possible icons, which will take billions of years to fill up. Figure 2.22 (in Sack, 2011) shows the current state of Every Icon on 10 November, 2012, 18:33:30. The result from this data visualization can cause difficulty in understanding its huge size and temporal framework (Fig. 4.3). George Legrady’s data artwork Pockets Full of Memories (2000), is interactive, involving participant contributions (Wands, 2006, p 175). This work was designed using an installation with a kiosk for data collection, a list of questions for audience members and a projection screen to display the outcome visually. It invited participants to contribute images of their personal possessions using digital scanning and to describe these objects with tags or keywords by answering the questions. A computer program was written to categorize these images of objects as a 2D map displayed on the screen, based on the description of these objects. Initially, the database of the work contained no data, but grew through participants’ input throughout the exhibition.

4.3 Data-Based Art

79

Fig. 4.2 Last Clock. Ängeslevä and Cooper (2005), digital art. (Image courtesy Jussi Ängeslevä)

Based on the participants’ contribution, it shows that a self-organizing map “determines the location of the objects by positioning them in a two-dimensional matrix according to similarities defined by the contributors” (Wands, 2006, p. 176). There is another type of data visualization art that focuses on visual representation as a specific kind of data or dataset, such as stock market or weather forecast data. John Klima’s data work Ecosystm (2000), is considered data art, involving a real-time visualization of the data of global currencies in 3D animation with a real-time weather report from John F. Kennedy (JFK) Airport (Klima, 2000). Ecosystm is considered a simulation in which the artist converted the financial market to ecosystem involved with behaviors of birds (Paul, 2003). The work shows that each flock of birds was represented as a country’s currency with branching trees representing the leading market index of the country. The tree will grow new branches if a market index grows, otherwise branches will fall off if the index drops (Klima, 2000).

80

4 Data-Based Digital Art Practice

Fig. 4.3 Screenshot of Every Icon. Simon (2002), digital art

The population and the behaviors of these birds were defined by the currency and its volatility (Paul, 2003). The territory of these birds was based on daily volatility that controlled the size of the territory according to the stability of the data. The weather of Ecosystm was decided by the weather report from JFK Airport where clouds affected the weather data. Garvin Baily and Tom Corby’s information visualization work Cyclone.soc, is a noteworthy example of exploring scientific data through aesthetic approaches (Fujisawa, Brown, Nakayama, Hyatt, & Corby, 2008, p. 393). This work, which is an immersive installation, makes use of data from USA satellite forecasting. The artists visualized this data as vector animations with information concerning depth and dimension (Fujisawa et al., 2008), then integrated textual data from Internet newsgroups and the isobars of satellite data to produce a “metaphoric waves of conversion” (Fujisawa et al., 2008, p. 393). The audience is able to interact with the work by

4.3 Data-Based Art

81

“reading and responding to…postings” (Fujisawa et al., 2008, p. 393). This interactive work represents newsgroup messages as weather patterns, exploring “a suggestive link between these extreme belief systems and their potential wider ecological impacts on the material world” (Fujisawa et al., 2008, p. 393). Jon Thomson and Alison Craighead created data visualization Decorative Newsfeeds in 2006. Decorative Newsfeeds is an installation work, which allows visitors to select from a number of soundtracks to interactive with CNN news and it presents as a series of curved texts moving across the exhibition display. The work can be displayed in the different forms, such as presented as a projection or shown on a screen. In the exhibition in Forest Hill in London, the work can be viewed from the street, “mocking converting this space of consumption into one of amusement and irony” (Jeffery & Minissale, 2009, p. 242). The artist is interested in how the work “on the way we receive what is inevitably biased reportage” and he argues that “news is always biased in some respects, wherever it comes from” (Colson, 2007, p. 73). The Listening Post is an installation data work created by Mark Hansen and Ben Rubin in between 2002 and 2005 (Fig. 4.4). The artwork composes of 231 small electronic displays with a grid of 11 feet high by 21feet wide, which installed onto a curved wall (Bullivant, 2005). It is a real-time visualization, which represents a quantity of data in the Internet chatroom or bulletin board conversations. On the screen, it displays random words from the online conversion. A software, which is a text-to-speech program, was designed to tell the words when they changed, and all these words are overlapped and strong sonic was created. The artists aim to “provide

Fig. 4.4 The Listening Post, Mark Hansen and Ben Rubin, (2002–2005) digital art. (Image courtesy Wikimedia Commons)

82

4 Data-Based Digital Art Practice

a sonic and visual environment through which the voice of the Web can express its contours and complexity” (Eleey, 2003). This piece of work involves people who are trying to connect through synthetic procedure, as the words are picked up from the various online resources, such as chat rooms, online newsgroup or forums. The work is also related to surveillance, privacy and data malleability, which shows “how our online activity is basically public, and available for virtually anyone to use for any purpose” (“The Listening Post”, n.d.). The visualization work highlights that the individuality and personality have lost in translation when the online life is mechanically programmed into data or dataset. Digital artists have a significant contribution to data visualization. They have demonstrated how artists explore the new way or methods to represent data or information and “making radical use of visual, tactile and auditory sensations” (Colson, 2007, p. 76). While Ecosystm, Cyclone.soc, Decorative Newsfeeds and the Listening Post adopted a specific source of data through the stock market and online news, my work adopted the data of sleep EEG as the data source to create visualizations demonstrating the proposed Taoist-based framework. These data-based artworks rely on the technological advances in data-mining techniques, in which artists are able to work with real-time data. The data in my work is not from the Internet, but visualization is interactive and dynamic. The next section will discuss EEG data-based art practices.

4.3.3 EEG Data as Digital Art Sleep has a natural connection with art and artistic practice. There are many artworks that use sleep as a creative theme and many works relate to dreams. Sleep has been linked with art and creativity through a number of themes (Shapiro, Boquiren, Boquiren, & Sherman, 2008). Many artists have a strong fascination with dreams or dream imagery. Themes in their artworks include religion, spirituality and the relationship between sleep and death. Since the Renaissance period, the theme of sleep has been an important source or inspiration for artists’ work. Sleep in an inactive state that “evokes a large array of emotions, and holds an abundance of internal activity” (Shapiro et al., 2008, para. 4). For example, Sandro Botticelli’s Mars and Venus (1483) present the god of war in the state of sleep, in which sleep is unpleasant as it lowers the conscious defenses of people who are as formidable as the war god. Whereas the Renaissance paintings express sleep in an external way, paintings of dreams offer a wonderful way to explore the internal world of sleep (Shapiro et al., 2008). Recently, sleep and EEG technology have been used by artists to explore the internal states of the human body in the context of digital technology (Wilson, 2002). These bodily experiments did not necessarily attempt to seek an improvement of existing knowledge, but to lead to novel directions in knowledge and understanding of issues concerning “the nature of the body in relation to social, political, scientific,

4.3 Data-Based Art

83

ethical, economic, and culture shifts in the world as we know it” (Wilson, 2002, p.178). Thus, my research aims to adopt sleep related data as an example for a new digital work and the putting into practice of the proposed theoretical framework. In this section, a number of artworks that specifically use EEG data as the creative medium have been discussed. These artistic works include using brain waves to create an interactive work and the visual representation of EEG data as colored abstracted images. Paras Kaul’s work Mind Garden (1997), a 3D game, explores a virtual garden using EEG signals involving digital brainwave analysis, and the World Wide Web (Kaul, 1997). In this work, participants were required to focus their attention to determine different forms and sounds; their brainwaves were recorded through EEG facilities. To explore the garden world, the brain theta and delta brainwave activity determined the course of the journey. The work aims to create a maximum level of experience based on one’s brain frequencies (Wilson, 2002). Bruce Gilchrist and Johnny Bradley’s work Divided by Resistance (1996), emphasizes interactive communication between participants and a sleeping performer experiencing REM (Rapid Eye Movement) sleep (Gilchrist & Bradley, 1996). Participants were invited to sit on a large vibrating chair named “the seat of consciousness” (Gilchrist & Bradley, 1996, Divided By Resistance section, para. 2). Its vibrating pads allowed participants to interactively communicate with sleepers’ brain states through electrodes that were connected to the sleepers. This work explored digital practice “into how signals from the external environment are symbolically incorporated and represented by the dreaming mind/brain” (Gilchrist & Bradley, 1996, para. 4). Daria Migotina, Carlos Isidoro, and Agostinho Rosa’s visual representation of sleep EEG data from the normal sleeping brain created in 2011 is another example of the visual representation of sleep EEG data as static images, exploring how sleep EEG data can be represented with aesthetic qualities, rather than as waveforms for communication only. In this work, the EEG data has been turned into abstract images (Migotina, Isidoro & Rosa, 2011, p. 148). The images represent the visual representation of sleep EEG data in two different modes. It shows a single image that combines five different images of five sleep stages. The image depicts the objects located in a spiral pattern. From the center of the image, the objects at first start to spiral outwards in a clockwise direction and in a chronological order. Through this method, the background color has been generated by the information in each sleep stage. The result of the second method shows a similar aesthetic quality as the one generated by the first method. These works present a series of color abstract images, which have different visual effects based on waveforms. Presumably, human subjects cannot understand these abstract images, in comparison to the shapes of waveforms. Although Migotina et al. (2011) claim that this abstract visual representation of sleep EEG data does not constitute an artwork but is only experimentation, the traditional data visualization (involving the communication of information, as explained previously in this section) is challenged. The final example is Nina Sobell’s Interactive Brainwave Drawing Game (1974–), an early project that explores how people’s brainwaves influence each other without

84

4 Data-Based Digital Art Practice

words (Wadeson, Nijholt, & Nam, 2015). The installation includes a monitor, whereby two people sit side by side in front of it, with EEG equipment. Their brainwaves were captured and displayed overhead with their real-time image on the monitor. These data-based artistic works explore data in the cultural realm that emphasizes aesthetic experience, rather than pure communication in the traditional sense of visualization with a scientific purpose of identifying patterns or relationships (Fujisawa et al., 2008). These works challenge traditional techniques in the fields of data visualization and enable people to engage more with the images “on emotional and conceptual levels by employing the visualization process to produce expressive works of visual art” (Fujisawa et al., 2008, p. 393). These data-based creative works indicate that data visualization is not only for communication and comprehension or identifying insight of data but also for enjoyment, user engagement or the production of sublime qualities in a digital artwork. However, current research on aesthetic approaches to data visualization in both theory and practice often emphasizes the visual effects of pleasure, engagement or the sublime experience, which are normally within the category of Western aesthetics. As the previous literature and artworks indicate, aesthetics in the design of data visualization focuses on the beautiful and sublime, which are the common categories of Western aesthetics concepts. However, this research proposes that non-Western aesthetic approaches to the design of data visualization have the potential to evoke a new aesthetic experience. This research aims to challenge the beautiful and sublime of Western aesthetic tradition through exploring traditional Chinese aesthetics. This new aesthetic is not only an innovation that extends knowledge of aesthetic experience of data visualizations, but also as a creative practice involving digital technology. More importantly, it is necessary to consider the data continuum that emphasizes the design of a data visualization from communication of data to aesthetic experience of it. This continuum should be considered a general design guideline for data visualization. Moreover, this continuum can incorporate different aesthetic modalities, including Taoist principles.

4.4 Conclusion This chapter has reviewed the concept of digital art and it explored a number of art practices in which data is considered a creative medium. Through the discussion of these concepts and theories, this chapter articulates the importance of aesthetic value in data visualization not only for comprehension or increased engagement, but also to promote new understandings of data as related to social, scientific and cultural transformations. These data based art practices present two categories. One is static and another is dynamic. Dynamic data visualization emphasizes interactivity, which is similar with my project that adopts interactive technology to visualize data. However, the difference between these art practices and my project is that mine adopts Chinese aesthetics

4.4 Conclusion

85

and embodied technology. My interdisciplinary research focuses on the discussion of the relationship between data visualization and aesthetics with emphasis on the aesthetic experience of data visualization. This has further involved the discussion of traditional techniques for data visualization, Chinese and Western aesthetics, and, in particular, the concept of the yijing aesthetic and the Kantian sublime. The review has presented aesthetics from two different cultural perspectives: the yijing aesthetic emphasizing harmony with the complementary of xu and shi as the basis of its visual forms, and the Kantian sublime emphasizing the experience of the awe or greatness of nature through the mathematical sublime and attempting to overcome nature through dynamic sublime. The further exploration of these two different aesthetic concepts is needed for a further elaboration of this research in the theoretical framework. As digital art is an emerging area involving many significant forms, such as network art, installation art, software art and data art created and exhibited in the last decade, it is not possible to explore all of these digital arts in this chapter. Instead, this chapter has outlined a number of key data based art practices that explore data or datasets for new understandings of culture, technology, aesthetics and society. Through analysis of the key issues of these data based digital art practices, this research proposes the potential for a new aesthetic approach to visually representing data through the concept of the yijing aesthetic and Taoist philosophy. In addition, the previous review of relevant literature has identified the key forms and characteristics of traditional data visualization, in relation to the yijing aesthetic that informs my theoretical framework. The next chapter will discuss the theoretical framework with further discussion of the yijing aesthetic, the Kantian sublime and Taoist philosophy, as well as gesture-based gaming technology.

References Ängeslevä, J., & Cooper, R. (2005). Last clock. Computer Graphics and Applications, IEEE, 25(1), 20–23. Bullivant, L. (2005). The listening post: Ben Rubin and Mark Hansen. Architectural Design, 75(1), 91–93. Colson, R. (2007). The fundamentals of digital art. Lausanne: AVA Publishing SA. Corby, T. (2008). Landscapes of feeling, arenas of action: Information visualization as art practice. Leonardo, 41(5), 460–467. Crowther, P. (2018). Digital art, aesthetic creation: The birth of a medium. Routledge. Drucker, J. (2013). Is there a “digital” art history? Visual Resources, 29(1–2), 5–13. Eleey, P. (2003). Mark Hansen and Ben Rubin. Frieze, 75. Fujisawa, N., Brown, K., Nakayama, Y., Hyatt, J., & Corby, T. (2008). Visualization of scientific arts and some examples of applications. Journal of Visualization, 11(4), 387–394. https://doi.org/ 10.1007/bf03182207. Gilchrist, B., & Bradley, J. (1996). Divided by resistance. Retrieved from http://www.artemergent. org.uk/dbr/dbr.html. Grau, O., (2017). Digital Art’s Complex Expression and Its Impact on Archives and Humanities. Museum and Archive on the Move: Changing Cultural Institutions in the Digital Era, 99–117. Hope, C., & Ryan, J. (2014). Digital arts: An introduction to new media. New York, NY: Bloomsbury Publishing.

86

4 Data-Based Digital Art Practice

Hauser, Jens (Ed.). (2008). Sk-interfaces. Exploding borders—Creating membranes in art, technology and society. Liverpool: Liverpool University Press. Jeffery, C., & Minissale, G. (Eds.). (2009). Global and local art histories. Cambridge Scholars Publishing. Kaul, P. (1997). Mind garden [digital work]. Los Angeles Convention Center. Klima, J. (2000). Ecosystm. Retrieved from http://www.cityarts.com/lmno/ecosystm.html. Legrady, G. (2000). Pockets full of memories. Retrieved from https://www.fondation-langlois.org/ html/e/page.php?NumPage=329 Manovich, L. (2015). Data science and digital art history. International Journal for Digital Art History, (1). Migotina, D., Isidoro, C., & Rosa, A. (2011). Brain Art: Abstract Visualization of Sleeping Brain. In Proceedings of GA 2011-XIV Generative Art Conference (pp. 141–153). Roma, Italy. Morgan, T. (2019). Sounding the unheard river: Reflections on an ecological sound art praxis as a response to ecosystem distress. Irish University Review, 49(1), 151–170. Nadarajan, G. (n.d.). Cyberarts: Intersections of art and technology. Retrieved from http://www.cyb erartsweb.org/nsa01/scene2.pdf. Paul, C. (2003). Digital art. London: Thames & Hudson. Paul, C. (2016). Introduction: From digital to post-digital-evolutions of an art form. In A Companion to Digital Art (pp. 1–19). Sack, W. (2011). Aesthetics of information visualization. In M. Lovejoy, C. Paul, & V. Vesna (Eds.), Context providers: Conditions of meaning in media Arts. Intellect Ltd: Bristol. Savini, M. (2017). Transgenic art: Creativity in the era of genetic engineering. Technoetic Arts, 15(2), 163–169. Shapiro, C., Boquiren, F., Boquiren, V., & Sherman, D. (2008). Sleep in art and literature. Retrieved from http://www.sleepontario.com/atlas_sleep_medicine_chapter1.pdf. Simon, J. F. (2002). Every icon. Leonardo, 35(5), 492–492. The Listening Post. (n.d.). Retrieved from http://www.digiart21.org/art/the-listening-post. Thomson-Jones, K., & Moser, S. (2015). The philosophy of digital art. Retrieved from https://plato. stanford.edu/entries/digital-art/. Viegas, F. B., & Wattenberg, M. (2007). Artistic data visualization: Beyond visual analytics. In Proceedings of the Second International Conference, OCSC 2007 Held as Part of HCI International 2007 (pp. 182–191). Beijing, China. Wadeson, A., Nijholt, A., & Nam, C. S. (2015). Artistic brain-computer interfaces: State-of-the-art control mechanisms. Brain-Computer Interfaces, 2(2–3), 70–75. Wands, B. (2006). Art of the digital age. New York, NY: Thames & Hudson. Wilson, S. (2002). Information arts: Intersections of art, science, and technology. Cambridge, MA: MIT Press. Zari¸na, S. (2014). The visual language of contemporary digital art and its collaborative aspects on science. Retrieved from http://doc.gold.ac.uk/aisb50/AISB50-S12/AISB50-S12-Zarina-paper. pdf.

Chapter 5

Theoretical Framework

This chapter addresses the theoretical framework of this research project based on reviewing relevant literature and current research on aesthetic approaches to data visualization. It includes a discussion of a project model that focuses on the continuum from communication-based data visualization to Chinese aesthetic approaches. This chapter outlines the aspects of the framework and discusses how these aspects work together to generate different data visualizations, which reflect traditional data visualization, the yijing aesthetic, Taoist cosmological philosophy, and gesture-based gaming technology. In particular, it develops a criticism of the Kantian sublime underpinning data visualization and proposes a new aesthetic of Taoist data visualization with the discussion of some key concepts, such as the anti-sublime, Chinese aesthetics, Taoist philosophy and gesture-based gaming technology.

5.1 Developing a Theoretical Framework of Aesthetic Data Visualization This section presents a conceptual framework that focuses on an approach in presenting data from information to aesthetics and then to Taoist philosophy, which underpin the production of data visualizations through three different influences (Fig. 5.1). This theoretical framework consists of seven categories of factors, including data, process, information, the yijing aesthetic, Taoism, moderators/influence and users. The factors of information, the yijing aesthetic, and Taoism are three different theoretical factors in the data visualization model it is developing. The three approaches have their forms and characteristics set along a continuum. The forms and characteristics draw from the discussion of traditional data visualization, the yijing aesthetic and Taoist philosophy in the Chap. 3. These three approaches are influenced by different factors. They are all at three different levels. The organizing data can be considered as traditional data visualization that emphasizes transferring data into © Shanghai Jiao Tong University Press 2020 Q. Li, Embodying Data, https://doi.org/10.1007/978-981-15-5069-0_5

87

Fig. 5.1 Proposed theoretical framework

88 5 Theoretical Framework

5.1 Developing a Theoretical Framework of Aesthetic Data Visualization

89

information for comprehension. In the design process, data can be presented through aesthetic approach, e.g. Chinese yijing aesthetic. In the process of creativity, it adopts Taoist philosophy to create embodied data visualization. The following sections discuss how these factors have been proposed for data visualization ranging from conventional communication-focused representation to aesthetic-focused visualization including the yijing aesthetic and Taoism.

5.2 Definition of Factors As some factors in the theoretical framework, such as data, information, and the concept of the yijing aesthetic have already been explored in Chaps. 2 and 3, these factors are only discussed here again in relation to this framework. All factors in the framework will be discussed and how these factors are linked together. In the discussion of the yijing aesthetic and Taoism, an analysis of Western aesthetics, in particular the Kantian sublime, and Chinese Taoist philosophy in the contemporary technological context is developed in the following sections.

5.2.1 Data As visualization is mainly deal with data that is to be visual represented, it is important to understand the data types and its characteristics. Data comes from various sources. For example, data can be collected from sensors or surveys, or it can be generated by simulations and computations. Data can be raw, which means not interpreted decoded, or it can be derived from raw data via some process, such as noise removal, scaling, or interpolation. In the computer science field, data has a number of types, including Integer, Floating point numbers, String, Boolean, which are common used in computer programming. Programming languages require defining the data type of a variable before assigning it a value. Some of programming language can automatically assign a variable’s data type if the initial data is entered into the variable. A typical data set can be divided as ordinal (numeric) or nominal (nonnumeric). Ordinal is the data taken on numeric values. It includes binary, discrete, and continuous. Binary number is assuming only values of 0 and 1; and discrete number presents only integer values, such as 1,3,5,7, 9; Continuous number represents real values, such as [0, 5]. Nominal refers to data taken on nun numeric values. It has three main characteristics: categorical, ranked and arbitrary. Categorical means “a value selected from a finite (often short) list of possibilities (e.g., red, blue, green)” (Ward, Grinstein, & Keim, 2010, p. 52); Ranked means that “a categorical variable that has an implied ordering (e.g.,small, medium, large)” (Ward et al., 2010, p. 52); And arbitrary means that “a variable with a potentially infinite range of values with no implied ordering (e.g., addresses).” (Ward et al., 2010, p. 52);

90

5 Theoretical Framework

Data in this book refers to scientific or medical data that is represented as “unprocessed information” in the form of numbers or binary codes. As a numeric form, data cannot be recognized or understood by humans. However, visualization is one effective way to represent data for comprehension. For example, sleep EEG data are often represented as waveforms, which provide valuable information for medical analysis. In addition to communication and analysis, data has also been used in art practices in the last decade, as discussed in Chap. 2. Thus, in this framework, data is considered “unprocessed information” and creative medium.

5.2.2 Information Information has a similarity with data, which usually refers to “processed data” that has meaning and can be understood by humans (Ackoff, 1989; Davis & Olson, 1985; “How to define data, information and knowledge,” n.d.). Information can be defined as data that is “information”, where data that has been stored, analyzed, and displayed, and is communicated through spoken language, graphic displays, or numeric tables (Dixon, 2000). It is considered as “processed data… formalized, capture and explicated; can easily be packaged into reusable form” (Tiwana & Williams, 2000). Information refers to a flow of message (Nonaka, & Takeuchi, 1995). According to Merriam Webster’s Dictionary, information refers to (1) (2) (3) (4)

“the communication or reception of knowledge or intelligence”; “knowledge obtained from investigation, study, or instruction”; “Facts, Data; quantitative measure of the content of information”.

In this framework, information is available through traditional data visualization. The conventional data visualization makes information available through the following main visual forms, such as in charts, tables, graphs, diagram, 2D animation, 3D animation and waveforms, outlined in Chap. 2 and presented in Table 5.1. As discussed in the literature review chapter, the purpose of these forms is to optimize selecting and creating the most effective and efficient transformation of data to information (Senay & Ignatius, 1990). Thus, the characteristics of the visual forms reflect the value of recognition, readability of information and the meaning of data. Table 5.1 describes traditional data visualization that focuses on conveying information. Yet this information-based data visualization has given minor consideration to aesthetic qualities. In particular, aesthetics has been excluded from traditional scientific or medical visualization. The rules and principles of visualization in science and medicine emphasize the mapping of data into a graphic representation that effectively convey the information encoded by the data (Senay & Ignatius, 1990). However, recent research argues that the effectiveness of information presentation and the value of aesthetics should not be considered independent. Thus, some theorists advocate the linking of data visualization techniques to artistic principles and the increasing of artistic influences on technical applications (Ursyn, 2010).

5.2 Definition of Factors Table 5.1 The forms and characteristics of information

91 Information Forms

Characteristics

- Charts - Tables - Graphs - Diagram - Dashboards - 2D animation - 3D animation - Waveform - Scatter Plot - Simulation - Volume - Maps - Trees - Infographics

- Recognition - Readability - Meaning - Rapidly

5.2.3 Data, Information and Knowledge Data, information and knowledge are three important terms that are often used in visualization in an interrelated context. In visualization, these terms are used to indicate different levels of understanding or truthfulness. Data visualization has been used for gain insight from data or information. In the process of visualization, the user’s knowledge is an essential element of visualization. For example, a user can choose a different position for revealing more meaningful information or a scenario required for further investigation (Chen et al., 2008). It argues that “knowledge” has been integrated into the visualization system, in particular in scientific visualization. Knowledge can be defined as information with meaning (Amidon, 1997). In additional, knowledge visualization has become a new field in business communities. It refers to apply visual presentation to transfer knowledge between two or more people (Burkhard & Meier, 2004). Therefore, it is important to understand three concepts in the process of designing visualization. Table 5.2 summarizes the definition of three concepts of data, information and knowledge.

5.2.4 Process Process in this framework refers to the transformation of data into a visual representation, consisting of three aspects: organization, design and expression. Data processing in terms of organization is one of the most common ways used in traditional data visualization, which focuses on the selection of representation of data according to different visual forms used for different purposes. Organization-focused processing of data involves selecting techniques or methods to achieve the most suitable ways to represent the meaning of data (Keller & Keller, 1993), and is often used in scientific

92

5 Theoretical Framework

Table 5.2 Definitions of data, information and knowledge Category

Definition

Data

Raw; Symbols (Ackoff, 1989) Data are elements of analysis (Amidon, 1997); Computerized representations of models and attributes of real or simulated entities (Chen et al., 2008) Refers to basic, unrefined, and generally unfiltered information (Liew, 2007)

Information

Useful; Processed data; or Refined fata; Information is data with context (Amidon, 1997); Give answers to “Who”, “What”,“When”,“Where”; (Ackoff, 1989) Data presenting the results of a computational process, such as statistical analysis, for assigning meanings to the data, or the transcripts of some meanings assigned by human (Chen et al., 2008) Information is data put in context; it is related to other pieces of data Information is about meaning, and it forms the basis for knowledge (Liew, 2007)

Knowledge

Application of data and information; Knowledge is information with meaning (Amidon, 1997); Give answers to “How” (Ackoff, 1989) Data presenting the results of a computer-simulated cognitive process, such as perception, learning, association, and reasoning, or the transcripts of some knowledge acquired by human (Chen et al., 2008) Knowledge is a body of information, technique, and experience that combines around a particular subject (Liew, 2007)

data visualization. Scientists usually select the best technique to reveal the structure of data according to the requirement of their research. For example, a waveform is usually selected as the most suitable technique to represent sleep EEG data through the conventions of sleep science. The second process of design processing refers to problem-solving considerations (Jirousek, 1995). The design process focuses on specified aesthetic issues in data visualization. Instead of selecting the most suitable technique in organization process, this process highlights aesthetic value in data visualization and seeks visual pleasure and emotional harmony, prioritizing the user’s experience. The third process focuses on the artist or designer’s emotional expressions. This expressive process maximally de-emphasizes the meaning and structure of data by avoiding traditional techniques for data visualization and creatively integrating new technologies into an aesthetic approach to data visualization. Data visualization associated with this process is often considered an artistic work, focusing on creative expression rather than pure communication. In particular, this process of expression emphasizes using digital technology to generate an environment for user interactivity.

5.2 Definition of Factors

93

5.2.5 The Kantian Mathematical Sublime and Anti-sublime In this framework, one of the important aspects is to compare and critique Western aesthetic approaches to data visualization using Chinese aesthetics. In particular, the Kantian mathematical sublime underpinning data visualization will be critiqued. My framework emphasizes the value of Taoist cosmological philosophy in contributing to data visualization that might be otherwise designed in accord with the Kantian sublime. The sublime has been described as the perception or experience of awe in relation to artworks in the Western philosophical system. The concept initially emphasizes d the experience of greatness or loftiness of thought or language, but later was used in the description of the grandeur of nature (Mul, 2011). As discussed in the previous chapter, Kant divides the concept of the sublime into two parts: the mathematical sublime and the dynamical sublime. The concept of the mathematical sublime involves extremely large magnitudes or vastness resulting in incomprehension for human viewers, pointing to “the infinitude of its scope or power of comprehension” (Guyer, 1996, p. 260). Kant (2007) elaborates the concept of the sublime: Now for the mathematical estimation of magnitude there is, of course, no greatest possible magnitude (for the power of numbers extends to infinity), but for the aesthetic estimation there certainly is, and of it I say that where it is considered an absolute measure beyond which no greater is possible subjectively (i.e. for the judging subject), it then conveys the idea of the sublime, and calls forth that emotion which no mathematical estimation of magnitudes by numbers can evoke… because the latter presents only the relative magnitude due to comparison with others of a like kind, whereas the former presents magnitude absolutely, so far as the mind can be grasp it in an intuition. (p. 82)

The painting Wanderer Above a Sea of Fog (1818) by the Romantic artist Caspar David Friedrich is one of classic examples of the sentiment of the Kantian sublime (Friedrich, 1818). In the painting, the Wanderer, appearing as a dominator, full of pride, stands firm, tall and victorious on the highest peak glancing down at the grandiose landscape in front of him (Fig. 5.2). His posture with his head and gaze looking down seem to show respect for the power of nature, while simultaneously suggesting a confidence to overcome and own it (Battersby, 2007). As Kant argues “we may become conscious of our superiority over nature within, and thus also over nature without us (as exerting influence upon us)” (Kant, 2007, p. 94). Here, Kant clearly indicates that nature and the body of the viewer are separated hence also describing the phenomenon of two oppositions or, more specifically, a concept of body-mind dualism. McCosker and Wilken (2014) extend the definition of the mathematical sublime to the context of contemporary computer technology, in which the increased big data flow “attempt[s] to comprehend human relations and activities at a large scale” (p. 157). The mathematical sublime refers to “estimation of magnitude by means of the concepts of number” (Kant, 2007, p. 254). Magnitude or vastness is possible for apprehension but not for comprehension, as Kant (2007) emphasizes:

94

5 Theoretical Framework

Fig. 5.2 Wanderer above a sea of fog. Caspar David Friedrich, (1818), Oil painting, 95 × 75 cm

In this mathematical estimation of magnitude understanding is as well served and as satisfied whether imagination selects for the unit a magnitude which one can take in at a glance, e.g. a foot, or a perch, or else a German mile, or even the earth’s diameter, the apprehension of which is indeed possible, but not its comprehension in an intuition of the imagination (i.e. it is not possible by means of a comprehensio aesthetica, though quite so by means of a comprehensio logica in a numerical concept). (p. 84)

Computing technology, for example, has been identified as an effective tool to calculate data through computing networks that consists of the World Wide Web and domain names. There are many visualization of networked social media interactions, such as network traffic and Internet Protocol (IP) addresses (McCosker & Wilken, 2014). The visual representation of networks refers to the metaphor as communication medium, which physically exist as communication channels and connecting points (King, 2000). These data visualizations can be understood as Kant’s mathematical sublime (McCosker & Wilken, 2014). Pillow cited in McCosker and Wilken

5.2 Definition of Factors

95

(2014) suggests that “a judgement of the mathematical sublime, in response to the question of the vastness and complexity of the internet, will seek (and fail) to unify as a whole the complexity of the phenomena represented” (p. 158). However, Manovich (2002) argues that the representation of data with aesthetic qualities actually involves what he terms the “anti-sublime”. The concept of the antisublime argues that the purpose in creating data visualization art is to attempt to make abstract data comprehensible for human beings. As Manovich argues that If Romantic artists thought of certain phenomena and effects as un-representable, as something which goes beyond the limits of human senses and reason, data visualization artists aim at precisely the opposite: to map such phenomena into a representation whose scale is comparable to the scale of human perception and cognition. (Manovich, 2002, Meaningful Beauty: Data Mapping as Anti-sublime section, para. 1)

Giving the example of Lisa Jebratt’s 1:1 (1999), Manovich (2002) attempts to demonstrate that the aim of data visualization is to create images of magnitude that influence human perception in technological environments. In Jebratt’s project, a dynamic database was created, which contained all the web addresses of the entire World Wide Web and all interfaces that were viewed or used. The size of cyberspace (as a perceptual phenomena) was reduced for formatting and transformed into a single image that can be viewed through a web browser (within the scale of human perception and cognitive). Manovich claims that the purpose of artistic data visualization is to represent quantitative data in a visual form that can be easily understood by users. Thus, “a good visualization should be that which can assist a user to perform a task more quickly or more efficiently than the user could do without the visualization” (Sack, 2011, Easy Information section, para. 7). Information-based data visualization in this framework can be considered anti-sublime, emphasizing the production of a “user-friendly interface” (Sack, 2011, Easy Information section, para. 7) for understanding data. However, the anti-sublime has provoked some criticism from researchers. Whitelaw (2004) does not totally agree with Manovich’s theory. She concedes that data visualization can “take the unimaginable ‘beyond’ of data, and make it available to experience” (Whitelaw, 2004, Sonification and the Data Sublime section, para. 7), but argues that the methods used for data visualization are different to those used with traditional art, in terms of how the sublime is evoked. She further explains her position through an example: A painting of a stormy sea never attempts to capture or entirely reproduce that sea or its dynamics, but to evoke an idea and instill a feeling; it makes a finite and specific impression of the vast beyond, and its own limits, its own static smallness, only adds to the pathos. So too, much of this work [data visualization] makes self-consciously limited but evocative impressions of the sublime of data. (Whitelaw, 2004, Sonification and the Data Sublime section, para. 7)

Similarly, McCosker and Wilken (2014) point out two disadvantages of the antisublime. Firstly, based on the concept of the sublime, they argue that visual representation of complex data, such as networked telecommunications traffic, should be

96

5 Theoretical Framework

considered a contemporary version of the mathematical sublime. This data visualization has parallels to earlier paintings evoking the mathematical sublime, such as Friedrich’s painting Wanderer above a Sea of Fog (1818), which has been discussed previously. Thus, data visualization cannot be considered as anti-sublime in its visual representation. In addition, data visualization performs another function that “further reinforces the sense of ‘unknowability’ that has become associated with the sublime experience of phenomena connected with ‘big data’” (McCosker & Wilken, 2014, p. 159). Secondly, McCosker and Wilken argue that it is difficult to distinguish the beautiful from the sublime, as the acceptance of various interpretations of the sublime can be useful in contemporary technological contexts, in which the concept “draws out the range of at times simultaneous and seemingly conflicting emotions and responses that we experience in the face of particular phenomena, including visualizations of big data projects” (McCosker & Wilken, 2014, p. 159). McCosker and Wilken (2014) emphasizes that the beautiful and the sublime are not “always readily separable or distinct categories” (p. 159).

5.2.6 Computing Technology and the Kantian Sublime The Kantian sublime is not only related to magnitude, vastness or greatness, but also involves fear, terror or awe produced by natural phenomena, such as storms or earthquakes. Data visualization in the late twentieth century is starting to adopt computing technology that provides a faster and more efficient environment to visually display a large volume of data. This could be considered as humans overcoming or conquering nature. This power of technology that has replaced the power of nature is termed the technological sublime, which has evolved from the Kantian dynamic sublime (Nye, 1994). Instead of experiencing fear or danger from nature, the technological sublime, however, evokes fear and danger from the technology for as “technology is an expression of the grandeur of the human intellect, we experience it more and more as a force that controls and threatens us” (Mul, 2011, para. 14). For example, atomic power stations have benefited humanity on the one hand, but have the potential to destroy humans on the other hand if they are not controlled well. However, most of technologies used in modern society are safe and the technologies of data visualization are not seen to threaten humans. For the extreme example of atomic power stations, the lack of control over technology makes them sublime. Nature is still more mystical and more powerful than the most technologies that give humans pleasure and that assist humans in overcoming nature. Although the Kantian mathematical sublime involves experiencing the magnitude or greatness of a huge amount of numeric data in a visual representation, it often lacks an aesthetic dimension that would focus on experiencing a transformation through computer communication in which a user may transcend time, space and the limits of his or her power (Mosco, 2004). As Giblett (2008b) points out the process of transformation of “solid matter into air” has been used as “a metaphor for mind-body dualism” (p. viii). The computer, as a technology of communication,

5.2 Definition of Factors

97

extends human’s bodily and mental capacities and mirror the power of machine, in particular digital computer systems (Giblett, 2008b). As an avatar of humans, the computer has the power of God and the human body, such as the hands and eyes, in which the “power summons from a distance” (Giblett, 2008b, p. 166). The idea of distance is often used in early Western aesthetics, as it suggests that nature can only be appreciated from a distance (Shinkle, 2010). Thus, the Kantian sublime emphasizes experiencing the power of nature at a distance, which separates the subjective body and the objective scene. Accordingly, in traditional data visualization, users perceive the magnitude of data or dataset from a distance that separates the body of users and the apparatus of the computer. In other words, users can experience fear or awe in relation to the almightiness of technology. The Kantian mathematical sublime differs significantly to the Chinese yijing aesthetic concept that emphasizes emotional harmony between subjective feelings and objective scenes (Li, 2009). It is also different from the Taoist philosophy of body and nature that emphasizes the unification of the body and the power of nature (Giblett, 2008a). These theoretical differences relate to artistic practice, as Pohl (1996) suggests. Considered from an artistic viewpoint, the inflexible and scientific methods associated with Kantian aesthetics are considered disadvantageous to the spirit of artistic creation. As a purely scientific framework, Kantian aesthetics, which lacks the vibrant and inspirational thinking considered a key to art practice, is more systematic and oriented towards modes of analysis than Chinese aesthetics. In the language of art expression, Pohl (1996) further argues that Western aesthetics emphasizes the importance of originality in artistic creation, while Chinese aesthetics focuses on gong that can be achieved through learning and perceiving the artistic spirit from predecessors’ artworks. Chinese aesthetics is non-systemic and linked to poetic expression, which is considered too subjective from the Western, Kantian viewpoint to be the basis of a system of perceptual analysis. However, Pohl (1996) argues that such an unclear and implicit Chinese aesthetics is in fact a perfect representation of the artistic creation, as the language of Chinese art is considered wufa (no rules) often referring to the nature of artistic expression. Therefore, in a more interactive and engaging aesthetic approach to data visualization, Taoist philosophy, one of key sources of Chinese aesthetics, should be considered a suitable basis for the artistic representation of data or datasets, rather than using Western inflexible aesthetic concepts, e.g. the Kantian mathematical sublime.

5.2.7 The Yijing Aesthetic in the Framework The concept of yijing is an artistic concept that focuses on the complementarity of xu and shi to create a harmonious visual experience between the subjective viewer and the objective scene. In the context of data visualization, the yijing could be evoked in the visual representation of data or datasets to create a harmonious interrelationship between users, data and the visualizing technology. Table 5.3, which is a part of the framework, proposes an aesthetic approach to data visualization that adopts the

98 Table 5.3 The forms and characteristics of the yijing aesthetic

5 Theoretical Framework The yijing aesthetic Forms

Characteristics

Xu (void) Shi (reality) Rhythm

– Emptiness or unclear – Focus or clarity – Unreadable – Unrecognizable – Complementary elements, harmony and balance

yijing aesthetic to visually represent data with aesthetic qualities. It presents the forms and characteristics of the yijing aesthetic in terms of its possible application to data visualization. In this table, xu and shi, unity, rhythm and evocative of imagination of nature are primarily visual forms. The characteristics of xu focus on emptiness in relation to xu while the characteristics of shi focus on focus or clarity in terms of shi. As data visualization using yijing focuses on visually harmonious effects and emphasizes the perceptual aspects of visualization, it is unlikely to be readable or understandable in a manner consistent with information-based data visualization. As yijing emphasizes harmony and balance, the characteristics of this data visualization includes the complementarity of xu and shi to achieve a harmonious and balanced visual effect. Responding to this lack of systematic study on the concept of yijing and its various explanations and applications in different contexts, my research does not attempt to explore all forms and characteristics but only focuses on ones which relate to Chinese painting. The concepts of xu and shi are deeply rooted into Taoist philosophy, which considers the cosmos as a composition of two complementary elements yin and yang.

5.2.8 Taoist Philosophy in the Framework “Taoist” and “Taoism” are English translations of the Chinese indigenous religion and philosophy of Tao, based in the Wade-Giles system (Komjathy, n.d). More recently, these terms have been translated as “Daoist” and “Daoism” according to the Chinese pinyin system. This research adopts the term “Taoist,” as it is a classic translation and many studies have adopted it in the discussion of the concept of Tao. As a classic Chinese philosophy, Taoism plays a fundamental role in the history of ancient Chinese thought and mainly adopts the thinking of philosophers Laozi and Zhuangzi. It has deeply influenced traditional Chinese art and culture, including the concepts of xu and shi in Chinese painting. Taoism is usually considered both a philosophy and a religion. My research focuses on its philosophical principles and concerns. Philosophical Taoism refers to “Lao-Zhuang” philosophy that comprises Laozi’s work the Daodejing (a classic text of Taoism) and Zhuangzi’s thoughts (Hall & Ames, 1998). The core tenets of Taoism relate to its cosmological philosophy that

5.2 Definition of Factors

99

emphasizes the concept of Tao, reflecting the xu of Taoist ontology. Taoists believe that the Tao is the origin of nature, which emphasizes cosmological attunement. Tao is considered “the process of the world itself, the ‘way’ of things” (Hall & Ames, 1998, Fundamental notions: dao section, para. 1). Taoists also believe that the highest realm of practice in Taoism is achieving the unification of cosmos (nature) and body as a whole. The fundamental doctrine of Taoism from Laozi is described as following: The original Tao gives birth to the one, The one gives birth to the two, The two gives birth to the three, The three produces the ten thousand existents, The ten thousand existents carry yin and embrace yang, Harmony is born with the breath of median emptiness. —Laozi (Cheng, 1994, p. 49)

This Taoist doctrine describes the essential conceptual structure of the Tao, reflecting cosmological principles that are considered inconceivable (to the rational mind). Taoism believes that everything in this world including humanity originally comes from the Tao, in which emptiness (xu) is a key to constructing harmony through the breath (qi). Fung (1960) avers that the concept of Tao refers to the unnameable and the “one” of the doctrine refers to the nameable, in which the nameable is created from the unnameable. The “two” and “three” are considered the beginning of eternity from where all beings come. This concept of the unnameable in the Tao often “lie[s] beyond shapes and features” (Fung, 1960, p. 94). However, Cheng (1994) replaces the concept of the unnameable by adopting a concept of emptiness (xu) and emphasizes the important role of yin and yang rather than the nameable and the unnameable. According to Cheng (1994), the Tao is considered the principal emptiness of the original state in the cosmos. The Tao creates the “one” that is primitive qi; this primitive qi then creates “two” that is considered the breath of yin and yang. The blend of yin and yang creates the “three”, consisting of three vital breaths including the breath of yin and yang, and a “median emptiness” that is considered a median qi. The function of the emptiness is to generate a harmonious relationship between the two opposite elements yin and yang. This harmonious relationship involves the internal transformation and unification of yin and yang. Three elements generate hundreds and thousands of beings (Cheng, 1994). Thus, xu (emptiness) plays a key role in the Taoist cosmological philosophy. The concept of emptiness involves formlessness and inconceivability. Fear or terror might be evoked from this uncanny emptiness in terms of Western aesthetics, but Taoist philosophy attempts to achieve apprehension through “practice” rather than to control or dominate experience of nature, in the Kantian sense. Based on the philosophical explanation of the cosmological principles of the Taoist religion, its purpose is to seek ways for humans to be healthy and achieve longevity. For this purpose, Taoism contains many different spiritual practices that include rituals, meditation (Kohn, 2001) and taijiquan. Taijiquan was designed for the purpose of seeking health and longevity. As Taoism emphasizes the unification of the body and

100

5 Theoretical Framework

the cosmos (or nature), the practice of taijiquan achieves harmony between all parts of the body with many benefits to human health. The Taoist viewpoint of the body, as Lévi describes, is “the cosmic body” that is as “perceived as a replica of the universe” (cited in Giblett, 2008a, p. 159). Both Taoism and traditional Chinese medicine position the human body as referring to a small cosmos (microcosm) while the universe refers to a big cosmos (macrocosm). The two cosmos exist equally. It is the most important that the body and cosmos are not separated and should not be placed in opposition. As Schipper argues, compared with Western theory that emphasizes the body “as something setting humanity apart from nature” (cited in Giblett, 2008a, p. 158), Taoism is understood as the embodied nature of humanity, in which the body is not separate from nature. The human body is considered as “the image of a country” (Schipper, 1978, p. 355) and Taoism treats the body as a landscape consisting of mountains, water and forest, defined as the “Interior Landscape” (Schipper, 1978, p. 355). For instance, a wall painting from the Yuan dynasty depicts a Taoist with a decorated cosmological robe, representing earth (mountains), water and heaven (stars) (Huang, 2012) (Fig. 5.3). The painting reflects the idea of the unification of humanity and nature. Therefore, dualism is not applied to the relationship between the human body and the cosmos (or nature) in Taoist philosophy. Kleinman highlights that the West positions the body and nature or matter as dualist: “the body is a discrete entity, a thing, and ‘it’, machinelike and objective, separate from thought and emotion” (cited in Giblett, 2008a, p. 160). Taoism believes that “the body is an open system linking social relation to the self, a vital balance between interrelated elements in a holistic cosmos” (Kleinman cited in Giblett, 2008a, p. 160). The body-self is an “organic part of a sacred, sociocentric world”, and it is “a communication involving exchanges with others” (Kleinman cited in Giblett, 2008a, p. 160). Zito and Barlow extend the notion of the relationship between the body and nature or mind to the relationship between “the organic microcosm of the body and the social macrocosm of humanity” (cited in Giblett, 2008a, p. 159). Based on the discussion above, my research argues that data visualization as a communication technology could be understood as an attuned relationship between the body of the user (as microcosm) and data and its visualizing technology (as macrocosm) that extends the Taoist viewpoint of the cosmological body. According to this theoretical framework, therefore, the body of the user attunes to the visual environment of the data through gesture-based interactive technology. The body and the data are treated as one body that is not separated into two opposing bodies. This embodied interactive data visualization is achieved through a Taoist theoretical framework (Table 5.3). The key forms and characteristics are presented in Table 5.4, in which the visual form of the data derives from gesture-based gaming technology in association with the yijing aesthetic. My theoretical framework builds on the yijing’s characteristics focus on embodied interactivity, pleasure, harmony and balance through the complementarity of all elements rendered unreadable and unrecognizable.

5.2 Definition of Factors

101

Fig. 5.3 Hall of the three pure ones in temple of eternal joy (Details), Shanxi, China, Yuan Dynasty, Wall Painting

This Taoist data visualization is different from the Kantian mathematical sublime. In the mathematical sublime, the body of users and the visual representation of data and its visualizing technology are separated and become two opposing bodies. Taoist data visualization aims to create a harmonious atmosphere involving the unification of the body of the users and the data as one whole, resulting in a Taoist body in a contemporary technological context.

102

5 Theoretical Framework

Table 5.4 The forms and characteristics of Taoist data visualization Taoist data visualisation Forms

Characteristics

Gesture-based gaming technology (Kinect) and – Embodied interactivity the yijing aesthetic – Evoke a deep emotion – Unreadable – Unrecognizable – Complementary elements, harmony and balance – Strong immersion

5.2.9 Gesture-Based Gaming Technology and Taoist Principle As a special form of game, gesture-based games are designed for players to interact with game devices more simply and intuitively using natural user interfaces, such as hands, voice or body. It has become an emerging area in last decade. A game has some specific characteristics, including entertainment (people play to amuse themselves); rules; locations; competitions; artificiality; cheating; and beginnings and endings (Berger, 2002). Among those characteristics, interactivity is one of the most important attributes. Berger (2002) suggests that “one of the most significant attributes of video games is that they are interactive — that is … the actions by players affect what happens in the game” (p. 12). It “allow[s] the audience to explore its nooks and crannies to let them generate causes and observe effects” (Berger, 2002, p. 15). Interactivity in games involves the player’s body in two ways. Ensslin (2012) implies that the bodies of players interact (embodied physically) with a computer and receive feedback (sensory stimuli) from the computer. Meanwhile, the players “are ‘re-embodied’ through feedback that they experience in represented form, through avatars [in the game]” (Ensslin, 2012, p. 125). The development of interactive technology has been determined as one of the key elements driving the development of gaming technology. In recent years, gesturebased gaming technologies have been developed to enhance body interactivity in gaming. Gesture-based technology is defined as a communication technology that positions “visible action as utterance” (Kendon, 2004, p. 7). It emphasizes communication through action that is, what Hansen and Morrison (2013) describe as, “bodymedia specific”. Farnell and Varela argue for the importance of body movement in visual studies and suggest that “dynamically embodied persons in action” (cited in Hansen & Morrison, 2013, p. 31) should replace “seeing the body as an object” (cited in Hansen & Morrison, 2013, p. 31). Hansen and Morrison (2013) argue that the body is endowed with the capacities for embodiment and communication, engaging in “complex interactions of personal and cultural values, beliefs and intentions, as well as numerous features of social organization” (Farnell, 1999, p. 148). These discussions of gesture-based gaming technology contain similar notions of embodiment

5.2 Definition of Factors

103

found in Taoist body philosophy. Both emphasize a person’s interactivity through bodily movement. While gesture-based technology is used as communication by body movement, Taoist body philosophy focuses on achieving the unification of the body and the external world as a Taoist body. In particular, the technology of Kinect makes it possible to achieve the unification of body and data visualization as through the Taoist body.

5.2.10 Kinect and Natural User Interfaces As an emerging gesture-based gaming technology, Kinect has revolutionarily changed the mode of traditional interactivity in gaming technology. As an input device with motion sensors initially designed for game consoles, Kinect allows users to control and interact with games through natural body movements (gestures) or voice commands, which are captured by sensors (Fig. 5.4). In 2010, Microsoft introduced the first generation of Kinect designed as a motion sensing input device for game controller of Xbox video games. The technology has a set of hardware including RGB camera, infrared projectors and detectors to allow the Kinect to perform realtime gesture recognition, speech recognition and body skeletal detection. The device was originally developed for Microsoft’s Xbox video game; however, it has been used for many research areas such as education (Haz, Molineros, Vargas, & Davila, 2019), medicine (Silverstein & Snyder, 2017), health care (Saini et al., 2019) and robotics (Chou & Lin, 2019), which are considered as non-gaming application. The advantage of Kinect enables users to interact with a computer system using a handfree “natural user interface” (NUI). A natural user interface refers to a user interface that goes beyond a command line interface (CLI) and a standard graphical user interface (GUI). The term “natural” is used in the context, as “most computer interfaces use artificial control devices whose operation has to be learned” (Kramer, 2012). The NUIs depend on a user being able to perform relatively natural gestures that can discover quickly to control the computing system/application or manipulate the onscreen content (Kramer, 2012). NUI is an emerging field in human-machine interaction (HMI), which includes three types of interfaces: Command Line Interfaces

Fig. 5.4 Kinect device, a Microsoft product

104

5 Theoretical Framework

(CLI), Graphical User Interfaces (GUI) and Natural User Interfaces (NUI). Figure 5.5 presents a comparison of three types of human computer interfaces. Command Line Interfaces use typing text to command and control computer application. It requests users to “follow a strict, text-based syntax, memorize the commands they needed to use and input those commands in a linear sequence” (Microsoft Surface User Experience Guidelines, 2008, p. 8). CLI uses keyboard as the method of input. However, it has a limitation on interacting with large amounts of text or data. In the 1960s, the mouse was invented to control the motion of a pointer on a computer screen. Meanwhile, computer graphics have obtained a big development and replace lines of text. The icons and graphics have been used to represent objects in real-world, such as Desktops, folders, files and trash cans. The application of computer graphics and mouse create an important type of interface: the Graphical User Interface (GUI). GUI adopts a new set of command to interactive with computers, which are Windows, Icon, Menus and Pointers (WIMP). The application of GUI has significance on the development of user experience, as users could see the objects they needed to interactive with on the computer screen. The experience of using computer became exploratory and users have been attracted to explore the objects. Nevertheless, with the introduction of NUI, the way that users interact with computers is replacing mouse and keyboard and toward the natural use of hands. The nature of GUI emphasizes “what you see is what you get” (WYSIWYG), while NUI emphasizes “what you do is what you get” (WYDIWYG). Research on natural user interface focuses on natural human ability, such as vision, speech, motion, touch and cognition (Hennig, 2016). The Kinect is considered as

Fig. 5.5 An audience is interacting with Kinect in an international exhibition in Shanghai, 2018

5.2 Definition of Factors

105

a classic example of natural users interface computing (Sanna, Lamberti, Paravati, & Manuri, 2013). Some research demonstrates that gesture-based NUIs will be the future of human-computer interaction, as it is very similar on how users intuitively communicate within the space (Smith, 2013). Kinect has been designed as tool that provides “a natural user interface (NUI) for controlling 3-D virtual globes such as Google Earth” (Boulos et al., 2011, Abstract section, para. 1). Kinect can also be used to navigating through virtual reality environment (Rahman, Clift, & Clark, 2019). It emphasizes that human bodily movement is a key to interactive with computer interface. This type of interactive technology is different to the interactive technology used for traditional data visualization in which users interact with a computer system through a keyboard, mouse or tablet. Kinect provides motion detection and human body tracking through a free-hands control function. Figure 5.5 presents an audience who is playing a Kinect game on an exhibition in Shanghai. The game is designed for user experience, in which audience could have experience of engagement and immersion. In the most interactive games, the user’s experience, in a simulating environment, is often termed “flow” (Csikszentmihalyi, 1990). It describes a state of total immersive experience in the game with a sense of there, which tests the boundary of the system creating a sense of presence in the environment. In this research, this immersion is considered an important moderator to generate a harmonious connection between the body of the user and the data visualization. In the proposed framework, it requires a fusion of the subjective body and the objective data that involves the emotional expression of the body-self as an image of data visualization. The potential applications of Kinect have been explored widely in the fields of teaching and learning (Hsu, 2011; Kandroudi & Bratitsis, 2012), physical therapy (Lange et al., 2011; Ferreira, Oliveira, & Postolache, 2017) and medical operations (Rouse, 2011), and military research. However, there has been little research on the application of Kinect to data visualization, whereby Kinect can be considered an interactive tool for data exploration or manipulation. There is an example of the application of Kinect to interactive data visualization for finding out the meaning of data. It depicts data visualization through Kinect, where the user is using his body movement, instead of a mouse or keyboard, to manipulate data (Table 5.5).

5.2.11 Kinect and Taoist Performance The body movement of Kinect is similar to the practice of taijiquan, which is based on Taoist philosophy. It requires performing body movements to generate the flow of energy (qi) that “connect[s] the interior of the body with the exterior” (Kaptchuk cited in Giblett, 2008a, p. 164). The performance of taijiquan is a slow and graceful body movement, which is understood as “an external and internal dance of bodily limbs, organs and fluids choreographed to the imagined shapes and actions of animal movement: stork cools wings, snake creeps down, golden cock stands on one leg, and

106 Table 5.5 The comparison of CLI, GUI and NUI

5 Theoretical Framework CLI (Command Line Interfaces)

GUI (Graphical User Interfaces)

NUI (Natural User Interfaces)

Textual

Graphical

Physical

Recall

Recognition

Intuition

Static

Responsive

Evocative

Strict textural syntax

Metaphors Mimics and represent real-world augment real-world objects objects

Controlled via keyboard

Controlled via a mouse or/and keyboard

Controlled via natural physical movements

so on” (Giblett, 2008a, p. 166). From a Taoist cosmological perspective, the practice of taijiquan is often required to be practiced in the natural environment and often involves the mimicking of animal behavior. It illustrates a performance of taijiquan as the mimicking of the movement of a white crane spreading its wings. When the body performs taijiquan, the flow of qi (breath) goes through the body as “the flow of movements [of body], and a body of depth, working on tendons, ligaments and organs, relaxing the minds, and harmonizing the spirit” (Giblett, 2008a, p. 165), which are the key steps to harmony and self-transcendence (Fulder, 1982). However, this kind of self-transcendence is different from the Kantian sublime that is a dualism of body and mind. Taoist self-transcendence is considered “a spiritualization of ch’i, of energy-matter itself” (Schipper, 1993, p. 41) that does not separate spirit from matter. Giblett (2008a) points out the difference between Western and Chinese Taoist philosophies, whereby the former “splits the body and the self and sets up the possibility of a narcissistic relationship between the self and the body” (Turner cited in Giblett, 2008a, p. 165), but the latter believes that “the self is embodied; the body/self is surface and depth, mass and flow” (p. 165). Traditionally, the practice of taijiquan was considered a teaching tool for health and a spiritual exercise, initially developed as a series of physical practices (Komjathy, 2013). Although it has not been widely accepted in Western societies, the practice of taijiquan has been demonstrated as beneficial to health (Giblett, 2008a). These health benefits include reducing high blood pressure, improving functionality, strength and flexibility, and fostering general wellbeing (Giblett, 2008a). The practice of taijiquan serves as “a moving meditation that reduces stress and provides a way to cultivate body and mind” (Giblett, 2008a, p. 168). It suits this research well because this study involves the application of health-related data for the production of visualizations (Fig. 5.6). This book aims to achieve Taoist data visualization through Kinect device by integrating the interior of the body and the exterior of the data through the flow of qi (energy). In such a sense, while Kinect serves as a technology for embodied interactivity, the practice of taijiquan serves as a vehicle for the unification of body, mind, technology and data.

Fig. 5.6 Health and longevity practice in Chinese ancient society

5.2 Definition of Factors 107

108

5 Theoretical Framework

5.2.12 Moderators and Influence As traditional data visualization focuses on how to communicate with users for recognizability, readability and meaning, the interplay between the user and the visual representation is portrayed through factors that moderate or influence perceptions. Such communication is primarily influenced by the need for effectiveness and efficiency, which promotes understanding. Effectiveness is an important criterion of whether visual forms satisfy a desired result. It often requires that visual forms accurately represent data and achieve the users’ specific tasks. Efficiency refers to using minimal resources to achieve tasks effectively, such as speed (Moere, 2007; Van Wijk, 2006). The three forms of data visualization are moderated or influenced by various elements in terms of communication and experience. In the process of informationbased data visualization, the potential moderators include the speed of comprehension in data visualization; the accuracy of representing structures of data; and the efficiency of visual representation of data including “the resources expended in relation to the effectiveness criterion, such as the required time or computational power” (Moere, 2007, p. 4). In the process of the Chinese yijing data visualization, the moderators refer to the fusion of the objective and subjective, which involves unifying the scene with feeling to achieve pleasurable experience. In data visualization, yijing generates the emotional experience of users with qualities of harmony and balance, rather than emphasizing pure communication for the purpose of data comprehension. In the process of Taoist data visualization, the process includes: the immersion of users into the data visualization; users paying attention to the visual effect or images; and users having a strong engagement with the data through their emotions and bodies. As it is based on the yijing aesthetic, the harmonious experience reflects the experience of pleasure and engagement. Both yijing and Taoist principles are considered as artistic expressions, rather than characteristics of speed, accuracy and efficiency.

5.2.13 Users In the process of data visualization, the users are primarily involved in either render visualization or observe/manipulate the data as end users. It often requires users have sufficient innate perceptual abilities for understand the three-dimensional objects. In some circumstance, the users need to have well trained for the tasks. Ward et al. (2010) categorize visualization to four aspects based on purpose of the visualization. The four main categories are exploration, confirmation, presentation and interactive presentation shown as the following:

5.2 Definition of Factors

109

Exploration: – The user wants to examine the data or data set to determine if there is a special feature or set of features is present. Most of the user attempt to explore data or data set for insights. Confirmation: – After data analysis by a computer, the user has “determined or hypothesized that a given feature is present in the data and wants to use the visualization to verify this fact or hypothesis” (p. 46). Presentation: – The user aims to convey a concept or fact to a viewer. The user can use labeling and various colors to highlight and support the conclusion. Most experience of visualization user has are presentation. In this case, a user or speaker gives a presentation using a chart or graph to indicate the values and the relationship, such as organizational charts. Interactive Presentation: – Interactive technologies enable the user to explore the presentation of data as above. Nevertheless, preparing an interactive presentation is time consuming. As there are “numerous constraints that have to be present so as not to allow the user to get lost, such as zooming out to a point where nothing is visible. Such interactive presentations are engaging when well done and are becoming more common as users are becoming more facile on the web” (p. 46). In the four categories, presentation visualizations are the primary visualization experiences for most users. Ward et al. (2010) highlight that “presentation visualizations are the primary visualization experiences for most people” (p. 46), because visualizations have been mostly static images requiring significant efforts to generate until recently. As its popularity, presentation visualizations have been used in a variety of fields including education and training. As this research investigates a new aesthetic experience of data visualization, the endpoint of the data visualization process often refers to the human user. Presentation visualization and interactive presentation will be main visualization techniques for the proposed theoretical framework. Thus, “users” in the framework refers to individuals including the general public audience and other data visualization users but not users who render visualization.

110

5 Theoretical Framework

5.3 The Theoretical Framework of Aesthetic Approach to Data Visualization This framework is an axiomatic approach in that it represents a vertical data continuum, from data through information, to the yijing aesthetic and to Taoist philosophy. Each is a higher level of processing undertaken, initially through organization, through to design and to expression. As the complexity of the data is reduced, the emotional experience of the user is enhanced, with the horizontal axis delineating the processes inherent in developing the two forms of representation on the left, with the user on the right. The interplay between the user and the representation is portrayed through the factors that moderate or influence perceptions. While communication is primarily influenced by the need for speed, accuracy and efficiency, the user’s experience is amplified by engaging with a more aesthetic representation. It is fair to say, therefore, that the quality of the experience varies due to the nature of the representation. Clear information promotes understanding while aesthetics promotes pleasure and engagement. Inevitably the relationship still exists between the need for communication and emotional engagement. However, I expect that the conventional communication elements might become increasingly ambiguous as an emotional response is elicited from the user. These statements are based on my observations of the exhibition. Key to this framework, too, is the differentiation of pure organization, design and expression as the main processes in generating the representation with the second and the third processes applying aesthetic characteristics. Information, the yijing aesthetic and Taoist principles are defined by recognizable characteristics, though the forms may vary. Some information is best organized in tables; for instance, EEG data is typically represented as a waveform. In this framework, the yijing aesthetic achieves the specified characteristics of unity, harmony through specific forms of shi, xu and rhythm, and unity. Shi can be understood as tangible objects, which are represented as realistic forms with characteristics of focus and clarity. Objects in shi are considered the most important parts in an image. Xu can be understood as emptiness or unclear objects, considered the less important parts in an image. Xu and shi have a complementary existence. However, these concepts cannot entirely be understood in objective language, that is, language that seeks to identify static objects in the world. In fact, these terms also relate to events, processes, and experience. This balance generates a visual rhythm that brings emotional harmony between subjectivity and objectivity, and evokes imagination. As an artistic practice, the yijing approach depends on each artist’s intention and experience. Taoist philosophy underlies a new aesthetic experience through the combination of the yijing aesthetic and gestured-based technology, e.g. Kinect. As a new interactive technology, Kinect allows users to interact with data and visualization through their body, but the resulting images are not readable and not recognizable. As Kinect has affordances for enjoyable and interesting interactivity (Hsu, 2011), users are able to experience pleasure. As the yijing aesthetic underpins the visual effect, Taoist data visualization has characteristics of harmony, balance

5.3 The Theoretical Framework of Aesthetic Approach to Data Visualization

111

and complementariness between all elements. This Taoist experience is influenced by how attention is gained and how users experience immersion. As initially applied to gaming, Kinect enhances the engagement of users with visualization by strongly embodied interactivity.

5.4 Conclusion This chapter has explored the proposed theoretical framework for this research with an explanation of its factors. This framework has demonstrated how data, information, the yijing aesthetic and Taoist philosophy operate along a continuum. It has indicated that visualization enhances communication through organization with a focus on speed, efficiency and accuracy while the yijing aesthetic creates an emotional relationship that engages deeply with users through the complementariness of xu and shi. This chapter also provides a critique of the Kantian sublime in the context of data visualization. The Kantian sublime was initially invoked through the power of nature with its magnitude or vastness of size. This perception of magnitude goes beyond a human’s comprehensive limitation. Furthermore, this kind of sublimity can be described in the visualization of “big data” (McCosker & Wilken, 2014), such as data from a computing network. However, Manovich argues that artistic data visualization is actually anti-sublime as data visualization artists or designers attempt to map data into a representation that can be understood by human perception and cogitation (Manovich, 2002). Nevertheless, this research argues that the Kantian sublime in data visualization ignores a dimension of aesthetics transcending time, space and power (Mosco, 2004). It highlights the dualism of Kantian sublime that separates the human subject from nature, which is different from the Taoist philosophy of body and nature. Taoism emphasizes the unification of body and nature, with the belief that the human body is a small cosmos (microcosm) and the universe (or nature) is a big cosmos (macrocosm). These two cosmoses have existed equally and have not been separated or counterpoised, which can be understood as the embodied nature of humanity. The unification of body and nature is achieved through the practice of taijiquan with the flow of qi through the body in order to relax body and mind (Giblett, 2008a). This book argues that Taoist data visualization develops an embodied relationship between the body of users (microcosm) and data visualization or technology (macrocosm). This approach departs from the Kantian sublime separation of the perceiver and perceived. This embodied interactivity can be achieved by gesture-based technology. Similar with the body movement in taijiquan, Kinect as an emerging gesturebased technology allows users to interact with data and visualization with natural embodiment, which enhances users’ engagement. In this framework, this gesturebased technology driving data visualization will be termed “Taoist data visualization” and will be based on the yijing aesthetic.

112

5 Theoretical Framework

The conceptual framework described in this chapter promotes a new aesthetic approach to data visualization that has the potential to enhance the user’s experience. By applying the forms of the yijing and the Kinect technology, Taoist data visualization is focused on experiencing immersion, attention and engagement with users. It not only generates an embodied interactivity but also has the possibility of producing health benefits, in particular for medical-related data and visualization. The practice of data visualization allows for the forms and characteristics of information, the yijing aesthetic and Taoist principles to be interrogated in a manner that identifies the key organizational, design and expression principles required not just to communicate concepts but to create a heightened experience for users. This identifies how human subjects can be embodied in data visualization technologies. The next chapter will discuss the research methodology used in this research including research methods and research design.

References Ackoff, R. L. (1989). From data to wisdom. Journal of Applied Systems Analysis, 16, 3–9. Amidon, D. M. (1997). Innovation strategy for the knowledge economy: The Ken Awakening. Routledge. Battersby, C. (2007). The sublime, terror and human difference. Abingdon: Routledge. Berger, A. A. (2002). Video games: A popular culture phenomenon. New Brunswick, NJ: Transaction Publishers. Boulos, M. N. K., Blanchard, B. J., Walker, C., Montero, J., Tripathy, A., & Gutierrez-Osuna, R. (2011). Web GIS in practice X: A Microsoft Kinect natural user interface for Google Earth navigation. International Journal of Health Geographics, 10(1), 45. Burkhard, R., & Meier, M. (2004). Tube map: Evaluation of a visual metaphor for interfunctional communication of complex projects. In Proceedings of I-Know (Vol. 4, pp. 449–456). Chen, M., Ebert, D., Hagen, H., Laramee, R. S., Van Liere, R., Ma, K. L. … & Silver, D. (2008). Data, information, and knowledge in visualization. IEEE Computer Graphics and Applications, 29(1), 12–19. Cheng, F. (1994). Empty and full: The language of Chinese painting (M. H. Kohn, Trans.). Boston, MA: Shambhala Publications. Chou, C. H., & Lin, S. W. (2019). Natural user interface design for control of robots by Hand Gestures. Journal of Computers, 30(1), 174–182. Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York, NY: Harper & Row. Davis, G. B., & Olson, M. H. (1985). Management information systems. New York, NY: McGraw Hill. Dixon, N. M. (2000). Common knowledge: How companies thrive by sharing what they know. Harvard Business School Press. Ensslin, A. (2012). The language of gaming. New York, NY: Palgrave Macmillan. Farnell, B. (1999). It goes without saying—But not always. In T. Buckland (Ed.), Dance in the field: Theory, methods, and issues in dance ethnography (pp. 145–160). Londo: Palgrave Macmillan Press. Ferreira, D., Oliveira, R., & Postolache, O. (2017). Physical rehabilitation based on kinect serious games. In 2017 Eleventh International Conference on Sensing Technology (ICST) (pp. 1–6). IEEE.

References

113

Friedrich, C. D. (1818). Wanderer above a sea of fog [Painting]. Hamburg, Germany: Kunsthalle Hamburg. Fulder, S. (1982). The tao of medicine. New York, NY: Destiny Books. Fung, Y.-L. (1960). A short history of Chinese philosophy (D (Bodde ed.). New York, NY: The Macmillan Company. Giblett, R. J. (2008a). The body of nature and culture. Basingstoke: Palgrave Macmillan. Giblett, R. J. (2008b). Sublime communication technologies. Basingstoke: Palgrave Macmillan. Guyer, P. (1996). Kant and the experience of freedom: Essays on aesthetics and morality. Cambridge: Cambridge University Press. Hall, D., & Ames, R. T. (1998). Daoist philosophy. In E. Craig (Ed.), Routledge encyclopedia of philosophy. London: Routledge. Hansen, L. A., & Morrison, A. (2013). Materializing movement—Designing for movement-based digital interaction. International Journal of Design, 8(1), 29–42. Haz, L., Molineros, Y., Vargas, E., & Davila, A. (2019). Multimedia System Kinect-Based. Learning Experience for Children of Primary School. Hennig, N. (2016). Natural user interfaces and accessibility. Library Technology Reports, 52(3), 5–17. Hsu, H. J. (2011). The potential of Kinect in education. International Journal of Information and Education Technology, 1(5), 365–370. Huang, S.S. S. (2012). Picturing the true Form: Daoist visual culture in traditional China. Cambridge, MA: Harvard University Asia Center. Kandroudi, M., & Bratitsis, T. (2012). Exploring the educational perspectives of XBOX kinect based video. Proceedings of ECGBL, 2012, 219–227. Lange, B., Chang, C.-Y., Suma, E., Newman, B., Rizzo, A. S., & Bolas, M. (2011). Development and evaluation of low cost game-based balance rehabilitation tool using the Microsoft Kinect sensor. In Proceedings of at the Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE (pp. 1831–1834). Boston, MA: IEEE Engineering in Medicine and Biology Society. Li, Z. (2009). Chinese aesthetic tradition. (M. B. Samei, Trans.). Honolulu: University of Hawaii Press. Liew, A. (2007). Understanding data, information, knowledge and their inter-relationships. Journal of Knowledge Management Practice, 8(2), 1–16. Jirousek, C. (1995). Creativity and the design process. Retrieved from http://char.txa.cornell.edu/ language/creative.htm. Kant, I. (2007). Critique of judgement (J. C. Meredith, Trans.; N. Walker Ed.). New York, NY: Oxford University Press. Keller, P. R., & Keller, M. M. (1993). Visual cues: Practical data visualization. Los Alamitos, CA: IEEE Computer Society Press. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press. King, A. B. (2000). Mapping the unmappable: Visual representations of the internet as social constructions. Retrieved from https://scholarworks.iu.edu/dspace/bitstream/handle/2022/171/ wp00-05B.html. Kohn, L. (2001). Daoism and Chinese culture. St. Petersburg, FL: Three Pines Press. Komjathy, L. (2013). The Daoist tradition: An introduction. New York, NY: Bloomsbury Publishing Plc. Komjathy, L. (n.d). The Daoist tradition. Retrieved July, 2018, from http://www.stpetersdelmar.net/ files/forums/Daoism_Presentation.pdf. Kramer, K. L. (2012). User experience in the age of sustainability: A practitioner’s blueprint. Elsevier. Manovich, L. (2002). Data visualization as new abstraction and anti-sublime. Retrieved from http:// manovich.net/content/04-projects/039-data-visualisation-as-new-abstraction-and-anti-sublime/ 37_article_2002.pdf. McCosker, A., & Wilken, R. (2014). Rethinking ‘big data’as visual knowledge: The sublime and the diagrammatic in data visualisation. Visual Studies, 29(2), 155–164.

114

5 Theoretical Framework

Microsoft Surface User Experience Guidelines. (2008). Retrieved from http://www.vassigh.com/ amv/wp-content/uploads/2011/10/Microsoft_Surface_User_Experience_Guidelines-v1-0.pdf. Moere, A. V. (2007). Aesthetic data visualization as a resource for educating creative design. In Computer-aided architectural design futures (CAADFutures) 2007 (pp. 71–84). Netherlands: Springer. Mosco, V. (2004). The digital sublime: Myth, power, and cyberspace. Cambridge, MA: MIT Press. Mul, J. D. (2011). Essay: The technological sublime. Retrieved January, 2019, from http://www. nextnature.net/2011/07/the-technological-sublime/. Nye, D. E. (1994). American technological sublime. Cambridge, MA: MIT Press. Nonaka, I., & Takeuchi, H. (1995). The knowledge-creating company: How Japanese companies create the dynamics of innovation. Oxford university press. Pohl, K. (1996). Chinese aesthetics and Kant. Twenty-first Century, 8(36), 84–89. Rahman, A., Clift, L. G., & Clark, A. F. (2019). Comparing Gestural Interfaces using Kinect and OpenPose. Rouse, M. (2011). Kinect. Retrieved from http://searchhealthit.techtarget.com/definition/Kinect. Sack, W. (2011). Aesthetics of information visualization. In M. Lovejoy, C. Paul, & V. Vesna (Eds.), Context providers: Conditions of meaning in media Arts. Bristol: Intellect Ltd. Saini, R., Kumar, P., Kaur, B., Roy, P. P., Dogra, D. P., & Santosh, K. C. (2019). Kinect sensorbased interaction monitoring system using the BLSTM neural network in healthcare. International Journal of Machine Learning and Cybernetics, 10(9), 2529–2540. Sanna, A., Lamberti, F., Paravati, G., & Manuri, F. (2013). A Kinect-based natural interface for quadrotor control. Entertainment Computing, 4(3), 179–186. Schipper, K. (1978). The taoist body. History of Religions, 17(3/4), 355–386. Schipper, K. (1993). The Taoist body (K. C. Duval, Trans.). Los Angeles, CA: University of California Press. Senay, H., & Ignatius, E. (1990). Rules and principles of scientific data visualization. Retrieved July, 2018, from http://www.siggraph.org/education/materials/HyperVis/percept/visrules.htm. Shinkle, E. (2010). Video games and the technological sublime. Retrieved March, 2006, from http:// www.tate.org.uk/research/publications/tate-papers/video-games-and-technological-sublime. Silverstein, E., & Snyder, M. (2017). Implementation of facial recognition with Microsoft Kinect v2 sensor for patient verification. Medical Physics, 44(6), 2391–2399. Smith, K. (2013). Digital outcasts: Moving technology forward without leaving people behind. Newnes. Tiwana, A., & Williams, M. (2000). The essential guide to knowledge management: E-business and CRM applications. Prentice Hall PTR. Ursyn, A. (2010). Aesthetic expectations for information visualisation. International Journal of Creative Interfaces and Computer Graphics, 1(1), 19–39. Van Wijk, J. J. (2006). Views on visualization. IEEE Transactions on Visualization and Computer Graphics, 12(4), 421–432. Ward, M. O., Grinstein, G., & Keim, D. (2010). Interactive data visualization: Foundations, techniques, and applications. CRC Press. Whitelaw, M. (2004). Hearing pure data: Aesthetics and ideals of data-sound. Unsorted: Thoughts on the Information Arts: An A to Z for Sonic Acts X Altena, A., Ed.

Chapter 6

Methodology

This book focuses on investigating a possible new aesthetic approach to data visualization through gesture-based gaming technology, rather than the conventional models of producing functional, visual understandings of data. In this book, data visualization is defined as the visual representation of numeric data, which is the foundation for a data-based creative practice. Recent research has shown that an aesthetic approach has benefits for data visualization (Lang, 2008). Such research argues that an aesthetic approach has the potential to enhance emotional engagement in user perception of visual images (Norman, 2004). The book aims to create a visual representation of numeric data based on principles of harmony and engagement by adopting the Chinese yijing aesthetic and Taoist philosophical tenets. This new aesthetic experience differs significantly to Western aesthetic concepts. It involves the design and development of Taoist-influenced visual representation of data through gesture-based gaming technology. Gesture-based gaming technology is a key tool to achieving Taoist data visualization, as its embodied interactivity is ideal for creating a Taoist embodied cosmos that emphasizes unification and harmony between the user’s body and the external world. In this research, gesture-based gaming technology involves the unification of the body of users and the numerical data. In particular, it enhances immersion and harmonious experience through its characteristics of enjoyment and entertainment. Firstly, this chapter will state the research position of this project. Secondly, it outlines the research methodology and methods employed in the project that underpin the data visualization as influenced by the yijing aesthetic and Taoist body philosophy. The chapter will then address the research methods used. Lastly, the research design will be discussed, including the double helix of the praxis-exegesis model.

© Shanghai Jiao Tong University Press 2020 Q. Li, Embodying Data, https://doi.org/10.1007/978-981-15-5069-0_6

115

116

6 Methodology

6.1 Research Position This research is positioned as creative practice. The researcher has a background in traditional Chinese philosophical aesthetics that engages with subjectivity and selfreflexive practice. Research based on self-reflexive practice involves an inquiry that emphasizes individual interest and creative intuition, which provides the researcher access to alternative conceptions and models (Sullivan, 2009). This inquiry involves creative practice and its relation to the subjective experience of visualization technologies. “Practice” is a term that is used to describe individual creative actions that are related to visual arts, design and new media (Gray & Malins, 2004). Here “practice” refers to creative action as a research method, in which knowledge is generated from artefacts or artworks through practice. The researcher’s background in and experience of traditional Chinese painting drives my approach to the visual representation of scientific data from the perspectives of Chinese traditional aesthetics and indigenous Taoist philosophy. This research aims to create a new aesthetic experience for users that differs significantly to Western aesthetic concepts, particularly in regard to the Kantian mathematical sublime that underpins data visualization practices (see Chaps. 2 and 3). The praxis of this research is based on contemporary digital practice in order to explore and reinterpret Taoist philosophical aesthetics in the context of digital technology. The definition of “praxis” can vary depending on the context. For this book the term is used as in the context of visual arts, to refer to a studio-based reflexive practice, which involves “a cycle of action-reflection-action” (Marshall, 2010, p. 78). More discussion of praxis will be addressed in Sect. 4.5.2, in which “praxis” refers to a key component in the double helix of the praxis-exegesis model. The researcher’s background in computing programming facilitates the praxis through the integration of traditional Chinese aesthetic concepts into technologybased data visualization. Digital practice often requires complex interactions between computer programmers and artists or designers (Paul, 2003). This background enables the researcher to build computer systems or write up programming code that has been a key to the creative artefacts of this research.

6.2 Research Methodology A research methodology considers any one of three basic paradigms that have guided research since Descartes in the seventeenth century (Guba, 1990): ontology, epistemology and methodology. Ontology involves assuming the nature of reality, and epistemology investigates the nature, source and limitations of knowledge. Methodology concerns how the researcher approaches knowledge (Guba, 1990). Methodology is the study of methods used as part of an overall strategy to construct knowledge within the research process. Depending on the nature of the research, the methodology is

6.2 Research Methodology

117

applied in order to answer different research questions. For example, in the conventional enquiry paradigm of qualitative and quantitative research, the methodology is used to answer questions, such as how about the researcher derives knowledge. However, in the positivistic paradigm, hypotheses are normally required prior to conducting research, including empirical testing within certain parameters. Initially, the research methodology planned for this project included qualitative and quantitative approaches, specifically interviews and questionnaires for data collection. Data collected from the interviews and questionnaires would be analyzed and used to answer the research questions. However, during the design and development of the data visualization works, I recognized that the research questions were raised through the action of creating the data visualizations. For example, the question that emphasizes the visual effect of the yijing aesthetic applied in data visualization required an enquiry-based in practice. The key to this research is its design praxis that uses the theoretical framework discussed in last chapter. This action of design praxis requires what Schön (1983) describes as reflecting in and on action where knowledge can be generated. I decided that qualitative or quantitative research paradigms would not fit well into this research. Thus, the research methodology shifted to a practice-led research strategy. In addition to a practice-led methodology, the researcher consider Chinese Taoist epistemological enquiry as part of the research strategy, due to the different epistemology of Taoist philosophy. Taoism does not tend to accept “the objectivity of knowledge in the sense of the independence of the world as known, from the knower” (Hall & Ames, 1998, Fundamental notions: de section, para. 6). Instead, Taoism emphasizes that the known and the knower are not independent from each other (Hall & Ames, 1998). The epistemology in Taoism suggests a dialectical concept between the knower and the known that “the more you know, the less you really know; and the less you know, the more you really know” (Shien, 1953, p. 260). As there exists a difference between the knower and the objects outside of the knower, the knowledge can only be received as “the appearance of things, the gross corporeal modality, while nothing is known of our true self or inner being” (Shien, 1953, p. 160). The knower knows things that focus only on the appearance of the objects not on knowledge of reality. Taoist philosophy positions that knowledge obtained from the exterior is the appearance of the objects, but not the genuine dialectic of reality (Shien, 1953). To know the reality of objects in the external world, Taoism suggests that people should know the “True Self” and “the principle of our birth and becoming which we share with all things in the universe” (Shien, 1953, p. 260). To obtain true knowledge, Taoist philosophy suggests the combination of the knower, knowing and known together, where true knowledge is generated through identification rather than duality that “is freedom from appearance, and awareness of true reality” (Shien, 1953, p. 261). This Taoist epistemology suggests that, as a researcher, I should not only focus on the appearance of the objects that I am investigating, but on the nature of reality itself. For example, in the process of creating the yijing aesthetic of data visualization, I did not concentrate on the visual appearance of traditional Chinese paintings, but focused more on exploring the spirit and philosophical concepts of Chinese painting with the yijing aesthetic, in particular, the linkage to Taoist cosmology.

118

6 Methodology

Therefore, this research adopts a Taoist epistemology that does not seek knowledge only from exterior appearances, but that in a way that might reveal a “true reality” through a practice-led research methodology that emphasizes knowledge generation from the practice.

6.2.1 Practice-Led Research Practice-led research is an established methodology in the visual arts and other creative fields, and has developed into a legitimate form of academic inquiry (Haseman, 2006). The term “practice” has a large range of definitions depending on different contexts. Gray and Malins (2004) explain the role and function of practice in the context of research. They argue that practice is considered “developing and making creative work as an explicit and intentional method for specific research purposes, For instance gathering and/or generating data, evaluation, analysis, synthesis, presentation, communication of research findings” (Gary and Malins, 2004, p. 104). This suggests that creative practice is a type of research method for data collection or evaluation and analysis of research findings, which is different to the common idea of practice as simply making something. Practice is often applied in the academic context. It requires that knowledge generation from practice is accessible, transparent and transferable. The artefact from practice embodies research concepts and provides visual evidence or documentation of the research process (Gary and Malins, 2004). Similar to Gary and Malins’s explanation of practice as a research method, Sullivan (2009) uses the description from the UK Arts and Humanities Research Council to discuss the role of practice-led research, whereby practice-led research refers to a specific characteristic of research activity in the visual arts fields: As with other research conducted by arts and humanities researchers, [practice-led research] involves the identification of research questions and problems, but the research methods, contexts and outputs then involve a significant focus on creative practice. This type of research thus aims, through creativity and practice, to illuminate or bring about new knowledge and understanding, and it results in outputs that may not be text-based, but rather a performance (music, dance, and drama), design, film, or exhibition. (p. 47)

This statement emphasizes the importance of creative products in a practice-led research methodology in which new knowledge is derived from creative practice. A practice-led research project might involve the same kinds of problem-solving found in other research methodologies, but the methods of the research rely on the production of artefacts that include performance, design and art exhibitions. In this research practice, the production of artefacts focuses on applying practice-led research to the production of digital artefacts in the context of digital practice. Scrivener (2000) distinguishes this problem-solving research from the research that he terms “creative-production projects” (Creative-production Projects and Problem-solving Projects Norms section, para. 1), which involve generating an artefact. He provides a broad concept of an artefact that includes a tool, technique or

6.2 Research Methodology

119

Table 6.1 Criterion for problem-solving practice research adapted from Scrivener’s table of norms of technology The criterion of problem-solving practice research 1.

Artefact is produced

2.

Artefact is new or improved

3.

Artefact is the solution to a known problem

4.

Artefact demonstrates a solution to problem

5.

The problem recognize–d as such by to others

6.

Artefact (solution) is useful

7.

Knowledge reified in artefact can be described

8.

This knowledge is widely applicable and widely transferable

9.

Knowledge reified in the artefact is more important than the artefact

product used to complete tasks. In this research, an “artefact” refers to the visual representation of numeric data created to evaluate and demonstrate the theoretical framework and apply the Kinect technology resulting in a Taoist data visualization. The difference between the two research forms (qualitative and quantitative on one side, and creative-production and praxis on the other) focuses on if there is a problem that needs to be identified and sorted out through the generation of artefacts and how knowledge generation relates to artefacts. Scrivener (2000) summarizes the criterion of problem-solving research as following (Scrivener, 2000, Table 6.1): The key criterion concerns the artefact as a solution to the problem and the knowledge reified in the artefact is transferable beyond the artefact itself and the context of its production. This criterion reflects the nature of my research. The artefacts that I created are considered exemplifications of know-how and as solutions to problems. For example, an aesthetic approach to data visualization is considered one of most important problems in data visualization research fields (Chen, 2005). The debate often focuses on the discussion of how aesthetic perception enhances the visual engagement of users in data visualization. My research aims to investigate a possible new aesthetic experience for traditional data visualization through a non-Western aesthetic basis, which potentially offers an additional solution to this aesthetic problem of data visualization discussed in the literature. My investigation involves a critique of the Western Kantian mathematical sublime underpinning data visualization through exploring the Chinese yijing aesthetic and indigenous Taoist principles improving aesthetic experience. The output of this investigation and research questions focuses on the creative practice of making Taoist data visualization. The artefacts of creative practice demonstrate and evaluate the framework that strives “to transform the world from what it is to something better…with intervention, innovation and change” (Scrivener, 2000, para. 1). The new aesthetic experience generated in the artefacts is applicable and transferable, and the proposed theoretical framework could be used to develop organizational and design principles that not only involve communicating concepts but also value creating a heightened aesthetic experience for users.

120

6 Methodology

Scrivener (2000) argues that problem-solving research shares characteristics with Schön’s model of reflective practice where reflection-in-action is considered a reflective form of knowing-in-action. In practice-led research, reflection plays a key role in how the final artefacts are created through the combination of various previous decisions and experiences (Daichendt, 2012). It allows artists or designers to discover the structure and growth of creative ideas. Schön (1983) explains how practice researchers use this concept in their practice for the generation of knowledge. He proposes that: when a practitioner reflects in and on his practice, the possible objects of his reflection are as varied as the kinds of phenomena before him and the systems of knowing-in-practice which he brings to them. He may reflect on the tacit norms and appreciations which underlie a judgment, or on the strategies and theories implicit in a pattern of behavior. He may reflect on the feeling for a situation which has led him to adopt a particular course of action, on the way in which he has framed the problem he is trying to solve, or on the role he has constructed for himself within a larger institutional context. (Schön, 1983, p. 62)

This quotation describes a number of different modes used by practitioners to deal with situations of uniqueness or instability. Schön (1983) argues that it requires an exterior position and goes on to criticize the practitioner’s “initial understanding of the phenomenon” (p. 63). This awareness leads the practitioner to “construct a new description of it, and test the new description [of the phenomenon] by an on-the-spot experiment” (Schön, 1983, p. 63). I adopt critical modes of framing and evaluating Taoist philosophical data visualization that reflects my individual experience and tacit knowledge. For example, the development of the yijing aesthetic, which reflects my knowledge of Chinese water-ink paintings in comparison to the works of Mondrian, was used as a problem-solving framework for this new aesthetic approach to data visualization. Frayling (1993) suggests three fundamental research frameworks for practiceled research, which include research into practice, research through practice, and research for practice. These three research modes involve different ways of generating knowledge and different methods for conducting research. The first category, research into practice, investigates historical, aesthetic or perceptual, and theoretical issues, in relation to social, economic, political, ethical, cultural and other concerns. Research into practice generates knowledge from the analysis of the artefacts; the methodology of this research usually focuses on anthropological, historical or psychological approaches (Daichendt, 2012). Research through practice aims to investigate any subject through creative practice with reflection, contextualization, and engagement (Daichendt, 2012). It emphasizes that knowledge is generated through the interaction between practice and the written document, both of which are important components in practice-led research. Research through practice includes materials research, development work, and action research (Frayling, 1993). The third category, research for practice, is more concerned with knowledge as “embodied in the artifact” (Frayling, 1993, p. 5) that is considered the end product of the research process. However, it focuses not on “communicable knowledge in the sense of verbal communication, but in the sense of visual or iconic or imagistic communication” (Frayling, 1993, p. 5).

6.2 Research Methodology

121

In this book, it defines as into, through, and for practice. It is research into practice, as this research focuses on the investigation of a Chinese aesthetic approach to data visualization in the context of digital practice. It requires me to investigate my creative research in relation to social, cultural and aesthetic contexts. Practice involves creating two data visualization works, which were based on the forms and characteristics of the proposed framework (Fig. 5.1). The artefacts were considered research “data”, and knowledge is generated from the analysis, evaluation and interpretation of the data. It is a research through practice, as knowledge in this research was generated through an interaction between creative practice and the written document, in which I adopted a praxis-exegesis research model for this research. It involves the process of creating Taoist data visualization through Kinect, which can be considered a “development work” (Frayling, 1993, p. 5). The development of gesture-based technology that generates a Taoist body is a process of innovation. Gray and Pirie (1995) argue that the potential of new media technologies should be used to “explore single and cross-cultural ‘human’ responses to visual stimuli, related to the digital control, manipulation and production of new product forms, color, patterns and texture” (p. 5). My research highlights the importance of developing new technology that is often identified as research through practice. It is also research for practice, as knowledge in this research is generated in the form of visual communication. The creative artefacts in this research become a key to generate knowledge. In this category, I considered myself a researcher, rather than an artist, whose work enables visual communication through the creative practice as the form of research. As a result, this research will have exhibited the artefacts and a written document as a commentary on the work and its broader context.

6.2.2 Digital Practice Research The practice of this research is framed as digital practice, which is a broad concept that often refers to any artistic works created by digital technologies (Paul, 2003). Brown and Sorensen (2008) suggest that digital practice includes building a digital system that can be used to explore ideas by modification of its programming code and the presentation of ideas “as formal expression in code” (p. 154). This is a key characteristic for research in digital practice. Mignonneau and Sommerer (2006) suggest that writing a programme is similar to writing a poem or article, which relies on the writer’s imagination and creative representation. It is similar to writing a programming code that has its own writing style, in which the artefacts derived from the programming really depend on the skills, experience and knowledge of the programmers (Mignonneau & Sommerer, 2006). The practice of this research is based on developing a computing system that builds visual and audio media artefacts, which were created by code in a computer programming language. I describe my work a creative digital practice, in contrast to a creative practice within a traditional media such as painting or music. Paul (2003) discusses the difference between the use of technology as a “tool” and a “medium”.

122

6 Methodology

This research applies digital technology “as its very own medium, being produced, stored, and presented exclusively in the digital format and making use of its interactive or participatory features” (Paul, 2003, p. 8). In comparison, other media work uses digital technologies as tools to create traditional artistic works, such as paintings, photographs or sculptures (e.g. digital images). Paul (2003) analyses the characteristics of digital practice and she suggests that digital works are usually interactive, participatory, dynamic, and customizable. Interactivity in digital systems can be considered one of key elements for most artists or designers. However, in the situation of artist as a researcher, digital practice provides extensive possibilities to explore various media, such as sounds, images or texts (Jefferies, 2010). These multiple modes form the origin of new knowledge (Jefferies, 2010). Paul (2003) suggests that “the possibilities of complex interaction in digital art go far beyond the simple ‘pointing and clicking’ that offers nothing more than a sophisticated form of looking at a work” (p. 67). This description enables me to explore the many possibilities for finding a suitable tool for interactivity that achieves Taoist data visualization with participants’ interactions in terms of Taoist body philosophy, in particular through the embodiment of data. Digital practice is often linked to participatory involvement that requires participants to contribute to the content of the work, and in which participants become “interactive within the parameters that have been set by the artists” or in which participants “set the parameters themselves” (Paul, 2003, p. 68). In Taoist data visualization, the interactivity involved two parts: the first was to set up sleep EEG data as “parameters” for participants’ interaction and the second was to create participants’ own parameters or, more specifically, the body shape of the participants. The two parameters or data blend together to create a Taoist body that emphasizes the unification of the body and the cosmos. Digital practice is dynamic and is able to change in response to data (Paul, 2003). Paul (2003) suggests that digital practice consists of a “back end” that is not visual and a “front end” that can be seen. The “back end” is often completed through code resulting in abstract communication whereas the “front end” can be a complex visual representation that can be experienced by the participants (Paul, 2003). These ideas relate to my digital practice, for example, in the development of the yijing aesthetic; the visual representation of sleep EEG data is the front end and the programming code that drives it is the back end. The front end is a dynamic visual representation that depicts the data as moving lines without repeating patterns.

6.3 Research Methods Guba (cited in Gray, 1996, p. 12) suggests that selecting a methodology is considered an outcome of ontology and epistemology. It requires awareness in the development of the methodology that the researcher should recognize the ambiguities of knowing and the value of partial knowledge. It includes what can be researched (or cannot be resolved through practice only) and “an awareness of the nature of the relationship between the researcher and the ‘knowable’” (Gray, 1996, p. 12). For example, the

6.3 Research Methods

123

Table 6.2 The research methods Methods

Description

1. Sketchbook

Hand drawing for development of documents; generating ideas

2. Observation

Planned and conducted in the way of systematically recording and interpreting

3. Building/making

Building an artefact including a physical artefact or a software program

4. Analysis and interpretation The process of interpreting data that reflects the original objectives of the research and produces explanations of the data

positivistic paradigm relies on a realist ontology, which points to the existence of objects, with an objectivist epistemology, which requires that the inquirer “must behave in ways that put questions directly to nature and allow nature to answer back directly” (Guba, 1990, p. 19). The constructivist paradigm is a relativist ontology, which acknowledges the existence of multiple realities and the subjective dimensions of epistemology. The different research paradigms require different methods to use in the process of enquiry, as they have different ontologies and epistemologies. For example, the methods used in the positivistic paradigm require experimental or manipulative practices in which “questions and/or hypotheses are started in advance in propositional from and subjected to empirical tests (falsification) under carefully controlled conditions” (Guba, 1990, p. 20). It requires experiments and empirical tests to evaluate research hypotheses or questions. However, creative practice paradigms (sometimes named artistic research paradigms) require “a pluralist approach and use of a multimethod technique, tailored to the individual project” (Gray, 1996, p. 15). Practiceled research adopts hybrid methodologies with various research methods and techniques. Haseman (2006) also argues that the dominant difference between the creative paradigm and other research paradigms is the way in which researchers express their research findings. The findings in the creative paradigm are often expressed “in forms of symbolic data other than words in discursive text” (Haseman, 2006, p. 6). It includes material forms of digital code. The key methods used in the creative paradigm are led by the requirements of the practice (Haseman, 2006). Thus, multiple research methods, which were used in this research, such as observation, sketches and building/making, include methods from other research paradigms, such as experiments or tests from the positivistic paradigm and software design techniques from the field of computer science. The research methods are listed in Table 6.2.

6.3.1 Sketchbook The sketchbook is a tool that provides documentation of the development of creative ideas from the beginning to the end of the creative practice. Although the impact

124

6 Methodology

of computer-aided illustration is far-reaching, hand drawing is still very important for many designers and practitioners (Schenk, 1991a). The use of drawing helps to achieve creative outputs and control the final artefacts. Through individual drawing experience and the environment in which drawing is performed, researchers “develop the type of visual understanding and knowledge that can be brought into use as required” (Schenk, 1991b, p. 170). The sketchbook used in my enquiry includes recording and analyzing the development of ideas and the visual representation of Taoist data visualization. It consists of two key functions, including analyzing the visual language of both classic Chinese paintings (Qi, n.d.) and Piet Mondrian’s painting, Broadway Boogie Woogie (1942–43) (Mondrian, 1942–43) as well as recording the development of the creative ideas leading to the development of Taoist data visualization associated with the yijing aesthetic (Fig. 6.1). Qi Baishi’s painting was chosen to analyses the visual forms and characteristics of the yijing aesthetic in order to seek possible application of Chinese traditional aesthetics to the context of contemporary technology. Qi expresses his artistic intentions through the small insects and objects from everyday life, which combine likeness and unlikeness. This genre reflects on concepts of emptiness, composition and poetic rhythm. I did sketch of the Chinese painting by Qi Baishi and it focused on visual balance. The key visual forms and characteristics of the painting were analyzed

Fig. 6.1 Chinese ink paint on rice paper. Baishi Qi, (n.d)

6.3 Research Methods

125

through my sketches of them, allowing me to contemplate the possible application of them in my data visualization works. It also assisted me in identifying the key characteristics of the yijing aesthetic in the visual arts. Sketching can also be considered a tool to record and develop creative ideas, as drawings represent ambiguous ideas and images held in the creative imagination (Goldschmidt, 1991). Goldschmidt (1991) argues that drawings can be used as study sketches where quick sketches are generated and are only understood by the producer of the sketch. Study sketches are often used in the early stages of a task for artists or designers. The initial ideas for the yijing aesthetic data visualization emphasize harmony through visual balance based on the complementarity of xu and shi and poetic rhythm. The ideas of the sketch combine the forms of the yijing aesthetic and Mondrian’s abstract visual elements with rich colors. The sketch has not been fully developed and is still rough. However, it provides me a basic visual form for the further development when I implement this into the computer programming code later. These exploratory sketches can be considered not only the forms of the yijing aesthetic but also abstract forms combining colors, texture, composition, and movement in the Western sense of aesthetics. In addition, sketches were used not only as drawn images but also as diagrams or charts. For example, sketching played a key role in the development of proposed theoretical framework that was discussed in the last chapter. The initial ideas of the framework, in which all concepts or theories are organized to present their relationships, were completed in the sketch. The sketch was a key for my analysis in generating the theoretical framework.

6.3.2 Observation Observation as a research method has been used in social science or art and design fields. It often involves “recording the behavioral patterns of people, objects and events in a systematic manner” (Collins, 2010, p. 132). It requires that the research should be planned and conducted in the way of systematically recording and interpreting (Foster, 2006). As a research method, observation has been used in various contexts. For instance Foster (2006) suggests that it could be used to “develop and test particular theories, and situations or cases may be deliberately selected for observation in order to facilitate this” (p. 58). I observed the practice of taijiquan as a data collection technique to facilitate the Taoist data visualization in terms of Taoist body philosophies. This involved an interactive and embodied environment in which to observe the behavioral patterns of the participants. There are two catalogues of observation, which includes non-participant observation (NPO) and participant observation (PO). The NPO requires researcher “remain[s] objective (neutral), and detached from the object/person/situation being observed” (Gray & Malins, 2004, p. 106), while the PO requires researcher participate the observation in the activities and be part of involvement, empathy, observation

126

6 Methodology

through the engagement (Gray & Malins, 2004). I selected non-participant observation to collect data. As the praxis of this book involves the practice of taijiquan that based on Taoist health philosophy, it requires participants with experience of practice of taijiquan. As a method of data collection, observation involves various techniques including one-way mirrors or video cameras for many different research purposes (Nachmias & Nachmias, 2000). In this book, video and photography were used as tools to record the participants’ body movements involved in natural activities, because natural observation requires behavior observation happening in one’s surroundings (Collins, 2010). These visual recordings capture direct visual experiences. Nachmias and Nachmias (2000) suggest that the first concern of observation is identifying what needs to be observed and which different types of behavior require different measurements. The different types of behavior include nonverbal, spatial, extra linguistic and linguistic behavior (Nachmias & Nachmias, 2000). These visual recordings capture the direct visual experience. Figure 6.2 is photographical recordings of taijiquan practice from a participant with many years’ experience on taijiquan practice. This experiment is designed for examining a user’s nonverbal behavior and how the user interacts with Kinect. The data in this experiment will be collected for further analysis. The surrounding was designed in a motion laboratory in Edith Cowan University, Western Australia. The whole process of observation involved only body movements and the response from the participant.

Fig. 6.2 A photography of recording observation of a participant practicing taijiquan interacting with Kinect, Installation. Photographer: Qi Li

6.3 Research Methods

127

Nachmias & Nachmias (2000) suggest that the first important concern of observation involves what need to be observed, which different types of behaviour require different measurements. The types of behaviour include nonverbal behaviour, spatial behaviour, extralinguistic behaviour and linguistic behaviour (Nachmias & Nachmias, 2000). This observation is identified as nonverbal behavior, which refers to “the body movements of the organism, also consist[ing] of motor expressions, though they may originate in various parts of the body” (Ekman, 1957, p. 141). Ekman (1957) indicates that motor expressions, including gestures, postures, and facial expressions, are forms of nonverbal behavior. He argues that the data generated from nonverbal behavior may “emphasize, contradict, aid in the interpretation of, or have little relation to a verbal statement” (Ekman, 1957, p. 142). Therefore, Nachmias and Nachmias (2000) suggest that nonverbal behavior observation can serve as a way to verify the participant’s responses. This research selected nonverbal behavior observation that allowed me to understand the natural body movement of the participant interacting with Kinect, in particular in reflecting on the integration of both numeric data and data from the body. It is used to validate a new aesthetic experience in Taoist data visualization, which is proposed in the theoretical framework.

6.3.3 Building/Making Computer programming is used as an important way to create Taoist data visualization through gesture-based technology. In the information society, programming has become a medium for the artists to create an interactive work. Colson (2007) highlights that programming “ceases to act to restrict idea development but allows artists to harness the processing power of the computer in order to achieve their aims” (p. 95). He argues that “the chosen media begins to influence not only the working methods but also the final form the work itself” (p. 95). To create Taoist data visualization, the researcher needs to adopt programming to achieve Taoist embodiment for generating experience of the strong engagement and immersion. Thus, a “build” methodology from computer science is selected for the programming language and code implementation. The build methodology refers to building an artefact including a physical artefact or a software program (Buro et al., n.d.). It requires that the artefact should be a new product or with new features that have never been used before. Build methodology involves designing the software system, reusing components, selecting an adequate programming language and testing on an ongoing basis, all of which have been used in the development of the programming for the Taoist data visualization based on the yijing aesthetic. For example, selecting a suitable programming language is a key to designing a reliable computer system. This selection needs to consider available libraries, required run-time speed, expressiveness and reliability (Buro et al., n.d.). The researcher selected Processing was selected to programme the data visualization, because it is ideal software for creative purpose.

128

6 Methodology

Processing is a computer programming language that was initially developed by Ben Fry and Casey Reas in 2002, which can be used in personal computers and Mac computers (Ramos, 2012). The software is an open source programming language, which was inspired by computer language, such as BASIC, but later it was developed based on Java language (Ramos, 2012). Processing was particular designed for creating visual, interactive media and ideal for art and design students and professionals. The works created by Processing have been exhibited at many renowned galleries and museums around the world, such as the Museum of Modern Art in New York, the Victoria and Albert Museum in London. It has been used to design projects, such as dance performance, interactive installations in the large public areas, export images for books and magazines and so on. Therefore, Processing has becomes a promising software for almost all digital related art practices. The next is a sample of Processing programming, which shows two basic functions: setup and draw.

void setup() { size(600, 480); } void draw() { if (mousePressed) { fill(0); } else { fill(255,0,0); } ellipse(mouseX, mouseY, 100, 100); } The code creates a window that is 600 pixels wide and 480 pixels high, and then starts drawing red circles at the position of the mouse, as the color of ellipse are filled with three parameters: 255, 0, 0 presenting red, green and blue. When a mouse button is pressed, the circle color changes to black. If a user runs the code, moves the mouse, and clicks to see what it does in Fig. 6.3.

6.3 Research Methods

129

Fig. 6.3 Screenshot from a computer program

Its small size makes it run quickly and reliably in most personal computers. In particular, it focuses on a text-based programming language for responsive images, instead of focusing on a visual programming language (Reas & Fry, 2006). Processing has the ability to bring programming into the context of art and design and to “open electronic art concepts to a programming audience” (Reas & Fry, 2006, p. 204). As a piece of open source software, Processing provides libraries on the Internet, which are contributed to by people around the world. For example, Beads was designed as a library for generating sounds or music for Processing, and is freely available on the Internet. It is also important to possess the knowledge of the capacity and inner structures of the programming language in order for artists or designers to create digital works (Mignonneau & Sommerer, 2006). It is also crucial to understand computer technologies including computer hardware and software in order to explore new artistic expressions that go beyond the current practices of digital art (Mignonneau & Sommerer, 2006). Thus, digital practices such as mine often require a comprehensive knowledge of computer technology in order to build computer systems for creating work. As part of my practice-led approach to digital making, I have had opportunities to “change, modify, and extend any part and use both hardware and software as flexible materials to express and shape one’s imagination and artistic vision” (Mignonneau & Sommerer, 2006, p. 180). It was central to this method of building/making, for me to become familiar with Kinect, which is a key in my digital practice. As discussed in the previous chapter, Kinect is a camera device with a depth sense that captures the distance of

130

6 Methodology

objects with an infrared light to generate a depth image (Borenstein, 2012). The key difference between Kinect and other gaming consoles, such as the mouse and keyboard, is that Kinect “enables users to control and naturally interact with games or other programs without the need to physically touch a game controller or object of any kind” (Boulos et al., 2011, Background section, para 1). Kinect can trace user’s bodily movement by enabling participants to use voice commend to control a game or program (Borenstein, 2012, p. 210). Kinect technology is supported by PrimeSense technology that “generates realtime depth, color and audio data of the living room scene” (Boulos et al., 2011, Background section, para 2). This is important as the user does not need to wear or hold anything but totally relies on their own body movement. Currently, there are two software development packages available for Personal Computer (PC) users. One is OpenNI and PrimeSense with open source drivers (“OpenNI 2 SDK Binaries & Docs,” n.d.), and another is Kinect Software Development Kit (SDK) developed by Microsoft. The first development software can use Processing software to write code building the Kinect applications and the second one uses C++ or C# programming language to develop the Kinect applications. Processing software was used to develop creative works for this research, as it enables the implementation of code easily and quickly.

6.3.4 Analysis and Interpretation Analysis is considered one of the key processes used to transform raw data into information, where knowledge is constructed based on the interpretation of human experience (Swift, 2006). It has been used in most research fields, such as education research whereby content analysis and document analysis used in action research methodology (McKernan, 1991). In the visual arts, a process of “data” analysis is used, that includes reading or reviewing the collected data, re-reading notes and interpreting artworks (Daichendt, 2012). Analysis is often considered the process of interpreting data that reflects the original objectives of the research and produces explanations of the data, requiring “taking a step back and reflecting on the process” (Daichendt, 2012, p. 101). Daichendt (2012) suggests that analysis in creative practice requires “summarizing the results, identifying any themes, recognizing patterns” (p. 101). As a result of analysis, a useful structure or flow may appear or a new understanding may be emphasizes d (Daichendt, 2012). Marshall (2010) suggests that artists or designers are often required to “keep, change, or delete elements of their work based on their observation and response to what they are doing” (p. 79), in which analysis is used to describe or evaluate the creative artefacts. Analysis was used in this research to interpret and evaluate the creative artefacts in terms of the yijing aesthetic, Taoist philosophy and the forms and characteristics of the proposed framework. It was important to organize and summaries the key conceptual issues at the beginning of my research, in particular in proposing the

6.3 Research Methods

131

theoretical framework that involved identifying a theme, theory or pattern, such as establishing a connection between communication-based data visualization and aesthetic approaches to data visualization. By this method of analysis, I have been able to evaluate the artefacts in terms of the process of creative practice that gradually allowed me to successfully test and evaluate the proposed framework through the digital artefacts. For example, during the development of the digital practice, I often used analysis and interpretation in “taking a step back” (Daichendt, 2012, p. 101) in order to evaluate the artefacts with the proposed framework and find out the meaning of the data. For example, I often asked myself what color I needed to use to represent sleep EEG data in order to create the yijing aesthetic experience; what kind of visual forms I should use to represent each of sleep EEG data in terms of Mondrian’s paintings and Qi’s paintings in the context of digital practice; and how the audio responses would enhance this aesthetic experience. Figure 6.4 shows an example of an experiment I undertook, in which sleep EEG data was visually represented as ellipses surrounded by lines with different colors. When I took a step back, and analyzed and evaluated this work in the terms of the yijing philosophy, I found that this experiment did not successfully represent the yijing aesthetic but was instead similar to the forms found in most scientific visualizations, such as Salt-dome Model by Victor Pereyra and Jose Rial in 1990 (Pereyra & Rial, 1990, p. 180). Interpretation is required as part of the writing and analysis process of research. It often involves the development of a dialogue between the findings, theories and expectations (Costley, Elliott, & Gibbs, 2010). As a part of practice-led research, interpretation enables data to gain meaning, and it is a key to “the process of attaching significance to the results and, if appropriate, theorizing from them and considering their implications” (Costley et al., 2010, p. 29). I used interpretation as a research

Fig. 6.4 An Initial Design for Visually Representing Sleep EEG Data. Li, Photographer: Qi Li

132

6 Methodology

method for this research, which links the project’s significance, the proposed framework as a set of expectations, the practical implications of the work, and the digital artefacts produced as a body of data. For example, I often asked how the digital artefacts compared to other existing digital practices of data visualization and how these digital artefacts could be evaluated or how they demonstrated the proposed framework in terms of the forms and characteristics. This method also enabled me to identify the implications of these digital artefacts for the conceptual basis of the project, specifically, Chinese aesthetics, Taoist philosophy and the Kantian sublime.

6.4 Research Design This section discusses the research design applied in this project. De Vaus (2001) suggests that research design involves logical considerations, which “ensure that the evidence obtained enables us to answer the initial question as unambiguously as possible” (p. 9). The general function of a research design is to address how research should be conducted in order to answer research questions. However, a research design in practice-led research often involves logical thinking through one’s praxis and reflection through and on the written document, or exegesis. Marshall (2010) suggests a research model of the double helix of praxis-exegesis especially for studiobased practice. The key to this double helix model focuses on the complementarity of the pairing between praxis and exegesis. The Double Helix of Praxis-Exegesis Model (in Marshall, 2010, p. 79) shows Marshall’s diagram of the double helix model with two spiraling bands, which represent praxis and exegesis. The two bands connect to rungs on a ladder and coils in the axis, forming a spiraling double helix. The praxis bands include creation, reflection, and response, while the exegesis band includes observation, description, and analysis (Marshall, 2010). When the helix spirals, two bands transform interactively to a new level. As a nonlinear research process, this model provides an inquiry combining both creative practice and written documentation (e.g. exegesis) as a way of undertaking synthesis and evaluation. A general principle of practice-led research, based on the research context and the researcher’s individual experience, links the practice and the written document. Praxis and written documents are two inseparable and interdependent parts of the creative process, in which art practice reflects criticism and theory (Marshall, 2010). This model of the double helix of praxis-exegesis however did not quite fit into my research design. For example, the praxis in my research involved collecting numeric data as a sample to demonstrate Taoist data visualization and the complex testing of code in the context of computer technology. Thus, I adopted Marshall’s double helix model as a research approach but needed to revise the key terms of Marshall’s double helix model, which could then be relevant to my research.

6.4 Research Design

133

6.4.1 Double Helix of Praxis-Exegesis Model In Marshall’s double helix model, praxis refers to creation, reflection, and response, and exegesis refers to observation, description, and analysis. In this double helix model (Fig. 6.5), I rephrased the terms “create, reflect, and respond” to “create & reflect, build, and test” in the praxis band, while in the exegesis band, “observe, describe, and analyse” were replaced with “observe, explain, and evaluate”. This modification of the double helix model is more suitable to my research focus on the creation of digital artefacts, which requires building a computer system involving a large amount of test and experiments. For example, a trial for testing software program, which is mentioned above, involved test the performance of the software. It is a basic process for a computer programme to test its code and stability. During the test, I wrote down any issues about the programme, and then I could modify the programme from these tests and experiments.

6.4.2 Praxis The term “praxis” can be explained in various ways. For example, Aristotle cited in Marshall (2010, p. 78) defines praxis as “an action done for its own sake”, which emphasizes the value of artistic creation. Kant (cited in Marshall, 2010, p. 78) considers praxis as an “application of theory to things we experience placing the practical above the theoretical”, which emphasizes the important role of practice. Praxis can be evaluated through a “research project demonstrat[ing] critical reflection and articulat[ing] visual thematic propositions, threads, or ideas that are consistent and that are technically, aesthetically, formally, or conceptually interesting or challenging” (Marshall, 2010, p. 78). This definition focuses on the process of critical reflection through a creative output that can be judged technically, aesthetically, or conceptually challenging as well inventive, innovative or skillful. However, Bolt (2007) argues that the process of praxis actually is a “material thinking” through which new knowledge emerges “in the involvement with material, methods, tools and ideas of practice” (p. 31). This material practice emphasizes the importance of artists’ tacit knowledge, which is not to be seen from self-conscious or rational perspectives but from handling tools, materials and ideas in the process of creative practice. Praxis in this research includes critical reflection and handling materials for the creative practice. For example, Kinect has been used to achieve Taoist data visualization, which is a key part of this creative practice. It can be considered a tool to facilitate the Taoist body interacting with numerical data, which was not through a purely self-conscious or perceptual basis. A new aesthetic experience emerged from actual physical engagement with Kinect as a tool revealing tacit knowledge. In addition, praxis in this research refers to the process of critical reflection and articulates

134 Fig. 6.5 The Double Helix of Praxis-Exegesis Model, which adapted from Marshall’s Double Helix Model. Li

6 Methodology

6.4 Research Design

135

or materializes the Taoist data visualization through the fusion of technological and aesthetic characteristics.

6.4.3 Exegesis The term “exegesis” derives from Greek, and refers to “interpret” or “to guide, lead” (“Oxford Dictionaries”, n.d.). Sullivan (2009) describes exegesis as the support material for use in exhibitions or other visual arts related research activities. It often takes the form of a written component in visual art research. Marshall (2010) suggests that writing a document about visual art enquiry is a reflexive approach to theorization. The written document should be developed simultaneously with the creation of artefacts. The exegesis records observations, descriptions, and analyses of developing artworks, and involving a complex interactive process between praxis and the written document. Brodie (cited in Barrett, 2007, p. 159) describes “exegesis” through the concept of the “meme”, which refers to “cultural replicators” or “the basic building blocks of culture”. Barrett (2007) suggests that the exegesis can be explained “both as a replication or re-versioning of the completed artistic work as well as a reflective discourse on significant moments in the process of unfolding and revealing” (p. 160). It can be understood that the exegesis possesses two purposes, including articulating more of the theoretical and practical significance of the work and illuminating the creative practice itself. Thus, the exegesis serves as a vehicle for validating creative inquiry and clarifying the value of the outcome (Barrett, 2007). Bolt (2007) suggests that the exegesis has a critical role in practice-led research, as tacit knowledge and the productivity of the process reveal new insights that can be expressed in the artefacts and clarified through writing. It emphasizes that the exegesis not only explains or contextualizes the praxis but also has a function of critical and complementary discourse about the artefacts. For this research, I applied the double helix of the praxis-exegesis model. I adapted the model to include the creation and reflection from Marshall’s double helix model, the evaluation of enquiry outcome from Barrett, and the explanation or contextualization of praxis from Bolt. The praxis and exegesis consist of two key components in the adapted double helix model (Marshall, 2010), which demonstrates the close relationship between creative practices and the written document, or exegesis. For example, for creating Taoist data visualizations, I built a digital system that involved testing this system for the implementation of code. Simultaneously, the observation of participants’ interaction with the digital system was recorded allowing an exploration and evaluation of this digital system. The adapted double helix model facilitated my enquiry of creative practice in a synergetic way, rather than treating the creative practice and written document as two separate parts. Thus, when developing the digital system, I did not consider digital practice as an independent element but instead combined digital practice with critical reflection and evaluation through the process of writing the

136

6 Methodology

exegesis. Creating digital artefacts often requires complex analysis and testing of the implementation of code, through which I considered praxis and exegesis as part of the process of synthesis and evaluation.

6.5 Conclusion The methodological approach to this research is framed as a practice-led enquiry with a theoretical framework applied to the analysis of my creative works. Practiceled research often requires knowledge generated from practice that is accessible, transparent and transferable. The nature of computing technology applied in this research required a mixed-method approach borrowing from digital practice and other fields. This research draws on conventional research methods of non-participant observation, sketchbook documentation, build/making, analysis and interpretation, and other methods including self-reflection and photography. These methods allowed me to explore the research issues throughout the process of creative practice in the development of Taoist data visualization through Kinect. The double helix of the praxis-exegesis model was adapted as a research design to connect creative practice and a written document recording the observations, explanations and evaluations of praxis. Moreover, this model is very important to validate the proposed framework emphasizing data continuum from information to the yijing aesthetic and then to Taoist data visualization, which was discussed in the last chapter. Having discussed the research methodology, methods, and research design, the process of developing the two data visualization artefacts will be addressed in the next chapter.

References Barrett, E. (2007). The exegesis as meme. In E. Barrett & B. Bolt (Eds.), Practice as research: Approaches to creative arts enquiry (pp. 159–163). London: I.B. Tauris & Co. Bolt, B. (2007). The magic is in handling. In E. Barrett & B. Bolt (Eds.), Practice as research: Approaches to creative arts enquiry (pp. 27–34). London: I.B. Tauris & Co. Borenstein, G. (2012). Making things see: 3D vision with Kinect, processing, Arduino, and MakerBot. Sebastopol, CA: O’Reilly Media. Boulos, M. N. K., Blanchard, B. J., Walker, C., Montero, J., Tripathy, A., & Gutierrez-Osuna, R. (2011). Web GIS in practice X: a Microsoft Kinect natural user interface for Google Earth navigation. International journal of health geographics, 10(1), 45. Brown, A. R., & Sorensen, A. (2008). Integrating creative practice and research in the digital media arts. In H. Smith & R. Dean (Eds.), Practice-led research, research-led practice in the creative arts (pp. 153–165). Edinburgh: Edinburgh University Press. Buro, M., Elio, R., Hoover, J., Nikolaidis, I., Salavatipour, M., Stewart, L., & Wong, K. (n.d.). About computing science research methodology. Retrieved from: http://webdocs.cs.ualberta.ca/~c603/ readings/research-methods.pdf.

References

137

Chen, C. (2005). Top 10 unsolved information visualization problems. Computer Graphics and Applications, IEEE, 25(4), 12–16. Collins, H. (2010). Creative research: the theory and practice of research for the creative industries. Lausanne: AVA Publishing. Colson, R. (2007). The fundamentals of digital art. Lausanne: AVA Publishing SA. Costley, C., Elliott, G., & Gibbs, P. (2010). Developing a methodology. In C. Costley, G. Elliott, & P. Gibbs (Eds.), Doing work based research: Approaches to enquiry for insider-researchers. London: Sage. Daichendt, G. J. (2012). Artist scholar: Reflections on writing and research. Bristol: Intellect. De Vaus, D. A. (2001). Research design in social research. London, UK: Sage. Frayling, C. (1993). Research in art and design. London: Royal College of Art. Ekman, P. (1957). A methodological discussion of nonverbal behavior. Journal of Psychology, 43, 141–149. Foster, P. (2006). Observational research. In R. Sapsford & V. Jupp (Eds.), Data collection and analysis (pp. 57–92). London: Sage Publications in association with the Open University. Goldschmidt, G. (1991). The dialectics of sketching. Creativity Research Journal, 4(2), 123–143. Gray, C. (1996). Inquiry through practice: Developing appropriate research strategies. No Guru, No Method, 82–95. Gray, C., & Malins, J. (2004). Visualizing research: A guide to the research process in art and design. Aldersho: Ashgate. Gray, C., & Pirie, I. (1995). ‘Artistic’ Research Procedure: research at the edge of chaos? Retrieved from http://design.osu.edu/carlson/id785/ead.pdf. Guba, E. G. (1990). The paradigm dialog. Newbury Park: Sage. Hall, D., & Ames, R. T. (1998). Daoist philosophy. In E. Craig (Ed.), Routledge encyclopedia of philosophy. London: Routledge. Haseman, B. (2006). A manifesto for performative research. Media International Australia Incorporating Culture and Policy, theme issue “Practice-led Research” (118), 98–106. Jefferies, J. (2010). The artist as researcher in a computer mediated culture. In H. Gardiner & C. Gere (Eds.), Art practice in a digital culture. Burlington, VT: Ashgate Publishing LTD. Lang, A. (2008). Aesthetics in information visualization. Trends in information visualization: Hauptseminar Medieninformatik WS 2008/2009. Retrieved from http://141.84.8.93/pubdb/pub lications/pub/baur2010infovisHS/baur2010infovisHS.pdf#page=16. Marshall, C. (2010). A research design for studio-based research in art. Teaching Artist Journal, 8(2), 77–87. https://doi.org/10.1080/15411791003618597. McKernan, J. (1991). Curriculum action research: A handbook of methods and resources for the refelctive practitioner. New York, NY: St. Martin’s Press. Mignonneau, L., & Sommerer, C. (2006). From the poesy of programming to research as art form. In P. Fishwick (Ed.), Aesthetic computing. Cambridge, MA: MIT Press. Mondrian, P. (1942–43). Broadway Boogie Woogie [Painting]. Museum of Modern Art, New York. Nachmias, C., & Nachmias, D. (2000). Research methods in the social sciences. New York, NY: Worth Publishers. Norman, D. (2004). Emotional design: Why we love (hate) everyday things. New York, NY: Basic Books. Oxford Dictionaries. (n.d.). Exegesis. Retrieved from http://www.oxforddictionaries.com/defini tion/english/exegesis. Paul, C. (2003). Digital art. London: Thames & Hudson. Pereyra, V., & Rial, J. A. (1990). Visualizing wave phenomenae with seismic rays. In G. M. Nielson, B. Shriver & L. J. Rosenblum (Eds.), Visualization in scientific computing. Los Alamitos, California: IEEE Computer Society Press. Ramos, E. (2012). Processing. In Arduino and Kinect projects (pp. 35–60). Apress, Berkeley, CA. Reas, C., & Fry, B. (2006). Processing code: programming within the context of visual art and design. In P. Fishwick (Ed.), Aesthetic computing. Cambridge, MA: MIT Press.

138

6 Methodology

Schenk, P. (1991a). The changing role of drawing with specific reference to the graphic design process. In Proceedings of the DATER 1991 Conference. Loughborough: Loughborough University. Schenk, P. (1991b). The role of drawing in the graphic design process. Design Studies, 12(3), 168–181. Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books. Scrivener, S. (2000). Reflection in and on action and practice in creative-production doctoral projects in art and design (Working Papers in Art and Design, Volume 1). Retrieved from https:// www.herts.ac.uk/__data/assets/pdf_file/0014/12281/WPIAAD_vol1_scrivener.pdf. Shien, G.-M. (1953). The epistemology of buddhism, taoism and confucianism. Philosophy, 28(106), 260–264. Sullivan, G. (2009). Making space: The purpose and place of practice-led research. Retrieved from http://artpracticeasresearch.com/wp-content/uploads/Sull_Making_Space_Practice_Led_ 2009.pdf. Swift, B. (2006). Preparing numerical data. In R. Sapsford & V. Jupp (Eds.), Data collection and analysis (pp. 153–183). London: Sage Publications in association with the Open University.

Chapter 7

Taoist Data Visualization

7.1 Introduction The concept of aesthetics has been shown to have benefits for the visual representation of data, which includes increasing user engagement and promoting subjective experience (Card, Mackinlay & Shneiderman, 1999). It also plays a key role in enhancing design for learning and creative thought (Norman, 2004). Existing research on aesthetics focuses on beauty with an emphasis on the visual pleasure of colors and shapes, and is often based on Western aesthetic concepts. Data visualization uses beautiful graphics to display data as a way of promoting clear communication and enhancing comprehension (Tufte, 2001). Edmund Burke distinguishes the clarity that is associated with beauty from the “obscurity” that is associated with the sublime (Burke, 1998), suggesting that a large size is powerful and evokes the experience of the sublime (Burke, 1998). For Kant, “the estimation of magnitude by means of numerical concepts (or their signs in algebra)” (Kant, 2000, p. 134) can be described as the mathematical sublime in which people experience the magnitude of data as incomprehensible. In the contemporary information age, the Kantian sublime has been used to describe the perception of the flow of a huge amount of data by “attempt[ing] to comprehend human relations and activities at a large scale” (McCosker & Wilken, 2014, p. 157), as Kant (2007) suggests: For in the estimation of magnitude by the understanding (arithmetic) we get just so far, whether the comprehension of the units is pushed to the number 10 (as in the decadic system) or only to 4 (as in the tetradic); the further production of magnitude being earned out by the successive aggregation of units, or, if the quantum is given in intuition, by apprehension, merely progressively (not comprehensively), moving across to an adopted principle of progression. (p. 84)

This indicates that the understanding of the mathematical estimation of magnitude is possible through apprehension, but not through comprehension. The key to the Kantian sublime is its emphasis on experiencing the power of nature from a distance, with the mathematical sublime emphasizing a vastness that exceeds the comprehension of the viewers and also the dynamic sublime emphasizing the attempt of the © Shanghai Jiao Tong University Press 2020 Q. Li, Embodying Data, https://doi.org/10.1007/978-981-15-5069-0_7

139

140

7 Taoist Data Visualization

viewer to control nature. In data visualization, users perceive the magnitude of data from a distance, in which the user experiences fear or awe from the capabilities of technology. While the Kantian sublime describes the phenomenon of the two oppositions of body and nature, Chinese Taoist cosmological philosophy of the body and nature emphasizes the unification of the body and the power of nature as one whole (Giblett, 2008). It is important for Taoist philosophy as embodied nature of humanity refers to “the image of a country” (Schipper, 1978, p. 355) or as an “Interior Landscape” with mountains, water and forest (Schipper, 1978, p. 355). In this research, this unification between body and nature has been used in representing a relationship between users and visualized data associated with technology, in which the body of the user attunes with the visual environment of data. Reflecting on Chinese traditional art often requires a fusion of feeling and scene or unification of self and nature through the visual complementarity of xu (void) and shi (reality) (Li, 2009), which is rooted in the Taoist epistemology of yin and yang. The characteristics of the yijing aesthetic emphasize harmony of the interior self and exterior environment. In data visualization, the yijing has the potential to enhance the user’s experience of harmony and pleasure through its specific complementarity of xu and shi in the visual forms (Li, McMahon, & Haddad, 2013). In digital practice, data visualization has been used as a creative medium and technology for artistic expression (Viegas & Wattenberg, 2007). It is different from traditional data visualization that emphasizes pure communication. It also has a different approach to existing discussions of the ability of aesthetics to enhance the understanding of the meaning of data. For the understanding of this data continuum from pure communication to an aesthetic approach in a cross-cultural context to data visualization as a creative medium to express an artistic theme, a theoretical framework has been developed and will be evaluated and demonstrated by the artefacts that were created from this digital practice. By adopting sleep EEG data as a sample, it visualized the data at two related approaches in terms of the yijing aesthetic and Taoist data visualization. Thus, creating Taoist data visualization is considered not only as a digital practice but also demonstrates the framework outlined in Chap. 5. The artefacts from this digital practice were created in two related processes of digital practice: data visualization with the yijing aesthetic, and Taoist data visualization engaging with gesture-based technology. Computer technology was important to creating these artefacts, including selecting the programming language and computer hardware, and writing code. This chapter will firstly discuss the process of computer programming in generating data visualization. Secondly, the artefacts from this digital practice, including two creative works, will be reflected on and interpreted in terms of the proposed framework.

7.2 Programming Processes In digital practice, computer programming is a crucial part of creative work. Computer code facilitates the artistic expression of creative practice, with computer

7.2 Programming Processes

141

programming considered a “form of creative writing” where artists’ expressions “exist as lines of code” (Wands, 2006, p. 164). It also refers to the “‘paint and canvas’ of the digital artist” (Wands, 2006, p. 164) who use computer programming as a creative medium, for example, artist Golan Levin cited in Wands (2006, p. 165) suggests that his digital practice is an “attempt to reclaim computation as a personal medium of expression”. Thus, programming in this research is a key to the expression of creative thoughts and a fundamental step for building the digital practice. This section discusses software, data selection, analysis, color modes and methods of visualizing data.

7.2.1 Software Digital artists do not normally create software for computer programming by themselves, but they often use off-the-shelf software, such as UNREAL for game engines or MySQL for accessing database (Wands, 2006). In this book, it used EDFbrowser software to demonstrate the communication-based data visualization and Processing programming language to create artefacts of data visualization with the yijing aesthetic and Taoist philosophical concept. EDFbrowser is an open source computer software that is designed for timeseries storage files, such as EEG, EMG or ECG (Van Beelen, n.d.). Processing programming language is easily accessible and its syntax is simple for implementing code more quickly when compared with other software. However, Processing software presents certain limitations for the praxis. As open source software it lacks technical support for implementation problems, and has limited graphics and visual effects. Although it includes functions for visualizing data, it can be slow when handling a very large amount of data, which was encountered in this research. Currently, Kinect implemented using Processing has problems working on some computer operating systems, such as the Windows 7 64-bit operation systems.

7.2.2 Sleep EEG Data Sleep EEG data was selected as the data sample for use in the traditional data visualization, making the yijing aesthetic approach to data visualization and Taoist data visualization through Kinect. It was sourced with permission from PhysioBank (Goldberger et al., 2000), an extensive archive of databases of physiological signals and data. The sleep EEG files have extensions .rec and .hyp, and are recorded in the European Data Format (EDF) (Goldberger et al., 2000). There are two reasons for the selection of sleep EEG data for this research. Firstly, sleep EEG data is typical of

142

7 Taoist Data Visualization

scientific data with signals that can be converted into numerical format. It is important that signals can be read and implemented by the programming that is used in this research. Sleep EEG data was available immediately, through an open-access Internet data bank. Secondly, as a distinct mode of human experience, dreaming often occurs during sleep. Dreaming has different meanings and qualities, including that of the “divinatory, deceitful, experiential, deriving from sensory experience, otherworldly, preposterous, mad or mirthful” (Parman, 1991, p. 1). Sleep is often considered as an enigmatic phenomenon “which aligns well with the ambiguity of aesthetics and is also often associated with creativity” (Li, McMahon, & Haddad, 2013, p. 8). In the art and design fields, sleep is considered one of the classic themes for discussing aesthetics (Shapiro, Sherman, & Kryger, 2013). According to the literature, human sleep is identified as two types: non-rapid eye movement sleep (NREM) and rapid eye movement sleep (REM). In psychological studies, it has been proved that dreams mainly occur in the REM stage of sleep, and that they are more memorable than dreams in other sleep stages (Hobson, 2009). It is established in consciousness theory that REM sleep, as a proto-conscious state, reflects the development and maintenance of waking consciousness through a model of “virtual reality” (Hobson, 2009). The data that was used in this research was recorded in the REM stage, which was edited from the original data files. This sleep EEG data has been used throughout this research, including visual representation of the data as waveform and Taoist data visualization associated with the yijing aesthetic experience engaging with Kinect.

7.2.2.1

Sleep EEG Data Information

The EEG data includes eight subjects, 21–35 year old Caucasian males and females not on any medication. The sleep EEG data files contain the submental-EMG envelope, oro-nasal airflow, and rectal body temperature, at 1 Hz. Each subject has two files: .hyp file and .rec file. The .hyp files contain information about the gender of the subjects, the times of the recordings, the lengths of the recordings. The .rec files contain information about the five stages of sleep (Fig. 7.1). The first four sc* files were obtained from volunteers in 1989. A modified cassette tape recorder was used to record twenty-four hours of the normal life of these subjects (Mourtazaev, Kemp, Zwinderman, & Kamphuisen, 1995). Then another four st* files were obtained from subjects who had normal health but mild difficulties falling asleep. The recordings were conducted during a night in a hospital with a miniature telemetry system with good signal quality (Kemp, Zwinderman, Tuk, Kamphuisen, & Oberye, 2000). In this book, the last subject was selected. The files were: st7132j0.hyp and st7132j0.rec. There were two reasons these files were used: – they have a better signal quality, and – they have a smaller file size when they are compared with other files, as the bigger file size may slow down the speed of visualization.

7.2 Programming Processes

143

Fig. 7.1 Screen shot of the eight subject’s sleep EEG data. PhysioBank sourced from http://www. physionet.org/physiobank/database/sleep-edf/

7.2.2.2

Conversion of Sleep EEG Data to Numeric Format

The files were the original recordings and their hypnograms were in EDF. A hypnogram is a graphic representation of the stages of sleep as a time function. It is considered as an easy way to present brain activities through EEG in a period of sleep. The Files in EDF format can only access through specific software, such as the Waveform Database (WFDB) Software Package. The WFDB software is used for viewing, analyzing and creating recordings of the signals. EDFBrowser software was used to represent the sleep EEG data visually, as it is easy to use, and can be used to edit the data.

144

7 Taoist Data Visualization

However, to visually represent sleep EEG data using Processing software for this research, it needed to be converted from EDF format to ASCII format, which can be read by a word processing software, such as WordPad. ASCII files contain the letters A–Z, numbers and punctuation that are readable. When the st7132j0.rec file was converted to ASCII format using EDFBrowser, there were four files available: st7132j0_header.txt, st7132j0_signals.txt, st7132j0_annotations.txt, st7132j0_data.txt. The st7132j0_header file contains information about the subject and the time the signals were recorded. This information shows that the subject was a 23-year-old male, the date of the recording was 8 August 1994 and that the recording lasted approximately ten hours (Fig. 7.2). Table 7.1 displays the information about the EEG data signals in file st7132j0_signals.txt, which includes the Hz frequency recorded and the type of the signals recorded in the data. The information in this table is clear, accurate and understandable. Figure 7.3 shows the annotations of the data. The values are onset, duration and annotation. In this file, there is no annotative information. The information of st7132j0_data.txt contains six columns (Fig. 7.4). The first column is the time of data recording; the second to fifth columns are four signals that

Fig. 7.2 The basic information of the subject. PhysioBank, the subject in file st7132j0_header.txt

Table 7.1 The information of the EEG data signals. PhysioBank, it contains information in file st7132j0_signals.txt Signal

Label

Transducer

Units

Min

Max

Dmin

Dmax

PreFilter (Hz)

Smp/Rec

1. EEG Fpz-Cz

AgAgCl

electrodes

uV

−3000

3000

−8.192

8.192

HP:0.03

1000

2. EEG Pz-Oz

AgAgCl

electrodes

uV

−3000

3000

−8.192

8.192

HP:0.03

1000

3. EOG horizontal

AgAgCl

electrodes

uV

−3000

3000

−8.192

8.192

HP:0.03

1000

4. EMG submental

AgAgCl

electrodes

uV

−3000

3000

−8.192

8.192

HP:0.03

1000

Fig. 7.3 The information of annotations. PhysioBank, in File st7132j0_annotations.txt

7.2 Programming Processes

145

Fig. 7.4 The information of the file. PhysioBank, it contains information in files st7132j0_data.txt

measure different parts of the brain, and the last column is information about ID + Sync + Error. The first step of conversion of sleep EEG data from numeric form into visual form is through the EDFbrowser software (Fig. 7.5). It presents five signals including the last signal ID + Sync + Error at 10 Hz. As the last column was not a part of the EEG signal and did not add to the hypnogram (Fig. 7.6), the waveform did not include the data in the last column; therefore, this study excludes the data in the last column of Fig. 7.1. Figure 7.5 presents the state of load the files into the program. The file of st7132j0_data.txt was used to creating data visualization with the yijing aesthetic and Taoist data visualization through gesture-based technology.

146

7 Taoist Data Visualization

Fig. 7.5 PhysioBank, Information in files st7132j0_data.txt

Fig. 7.6 PhysioBank, Information in files st7132j0_data.txt

7.2.3 Methods of Visualizing Sleep EEG Data Implementing code and building a computer system are fundamental to digital practice whereby the final artefact is generated in the form of a binary computer file that “exists as a collection of ones and zeros on digital storage media” (Wands, 2006, p. 14). In this research, the implementation of computer code was used to visually represent the sleep EEG data. To visualize the data in conventional form, the EDFbrowser was used to create a waveform of data. To visualize the data through

7.2 Programming Processes

147

the yijing as Taoist data visualization, Processing software, which involved writing code, was used. In designing the Taoist data visualization associated with the yijing aesthetic, I partly adopted the existing code that was available on the Internet (Owed, 2012). Two main classes “PolygonBlob” and “Particle” were borrowed from this existing code, as PolygonBlob class was an effective technique to catch body movement that was suitable for the Kinect technology. Particle class was used to generate line movement and form the shape of body. Both of the data visualization comprises the key part of the digital practice in this research.

7.3 Conventional Visualization Technologies According to the proposed framework, conventional visualization formats and technologies include charts, tables, graphs, diagrams, 2D or 3D animation and waveform. These visualization modes have been widely used in data visualization fields for the purpose of communication and understanding. As the first step of this praxis, sleep EEG data was visualized by the conventional technology based on the forms and characteristics of information in the proposed framework.

7.3.1 Sleep EEG Data Visualization as a Waveform Sleep is an important brain activity that refers to “a reversible behavioral state of perceptual disengagement from and unresponsiveness to the environment” (Carskadon & Dement, 2011, Sleep Definitions section, para. 1), which includes physiologic and behavioral actions. Normal sleep includes body recumbency (lying down), closed eyes and behavioral silence and other related behaviors. In abnormal sleep, some different behavior may occur, such as sleep talking or insomnia. Those sleep disorders can be diagnosed using medical techniques. EEG is one of main techniques used to explore sleep, assisting doctors to identify sleep problems (Migotina, Isidoro, & Rosa, 2011). Most outputs of EEG signals are represented as a waveform associated with a chart conveying key information about the brain’s activities (Carskadon & Dement, 2011). Waveform is typically applied to scientific data visualizations, including medical data visualization, in which users can recognize the visual form in order to obtain insights or understand the meaning of the data quickly. This technology is often used to assist doctors to visually represent brain structures as images for observation of neurochemical changes in the brain when it responses to different stimuli (Migotina et al., 2011). The key of its visual characteristic is its peak and trough where the biggest number (peak) represents the strongest signal and the smallest number (trough) represents the weakest signal in term of sleep. Figure 7.7 is a sample of REM data visualization as a waveform, which displays the REM stage of human sleep.

148

7 Taoist Data Visualization

Fig. 7.7 Screenshot of a PSG (polysomnography) of a person in REM sleep

The proposed framework suggests that designing a communication-based data visualization requires the organization of the data in the form of conventional technologies, such as diagram, graphs or waveforms. According to this, I selected the waveform as one of the conventional formats to represent sleep EEG data using EDFBrowser software. Figure 7.8 shows a screenshot of sleep EEG data visualization as waveform. This single waveform comprises four signals, which includes EEG Fpz-Cz, EEG Pz-Oz, Electro-Oculograophy (EOG) horizontal and submental chin

Fig. 7.8 The Hypnogram Composed of Four Signals: EEG Fpz-Cz, EEG Pz-Oz, ElectroOculograophy (EOG) Horizontal and Submental Chin EMG, digital display on the screen, Li (2013)

7.3 Conventional Visualization Technologies

149

Fig. 7.9 The Hypnogram Composed of Four Signals: EEG Fpz-Cz, EEG Pz-Oz, ElectroOculograophy (EOG) Horizontal and Submental Chin EMG (Detailed with information of the signals, waveform and measurement unit), digital display on the screen, Li (2013)

Electromyography (EMG) in 100 Hz frequency. According to the proposed framework, the data was organized as a waveform. As time elapses, the waveform changes to different wave shapes around the baseline in the middle. The change of the waveform was based on the value of four signals. Figure 7.9 shows detailed information about the waveform, which includes the name of the signals and measurement unit (uV). This information is presented clearly and can be recognized quickly. These medical visualization, which are represented as a waveform, focus on creating the most effective and efficient transformation of data to information (Senay & Ignatius, 1990). Both of them show a single waveform that represents one stage of sleep that indicates the state and quality of the sleep through measuring brain activity. Figure 7.7 conveys the same information to that of Fig. 7.4, which shows data in the different values. It shows the structure of the sleep EEG data as a waveform based on the value of each data. However, the data in the waveform is much more easily understood than the data listed in Fig. 7.4 as numbers. Thus, the waveform enables people to recognize data accurately and quickly. It demonstrates the proposed framework, in which data visualizing as information has the characteristics of recognition, readability and meaning depending on variables of speed, accuracy and efficiency. The aim of this data visualization emphasizes its communication with users or, in other words, that data has been transformed to useful information. However, the waveform does not attempt to visually represent the sleep EEG data with aesthetic qualities such as pleasure, harmony or the involvement of emotion. It is also unlikely to enhance the user’s engagement with the data or produce an emotional experience. This communication-focused artefact may be more suitably described as the antisublime, in which, as Manovich (2002) emphasizes s, the scale of the representation reflects the scale of human perception. This emphasizes the importance of comprehension, perception and cognition in the process of data visualization including organizing data as information. In the design process of visualizing data, aesthetics is considered as a key approach to its visual effect. The aesthetic approach can enhance

150

7 Taoist Data Visualization

a user’s engagement for comprehension, in particular in the information visualization field. However, this research attempts to generate a new aesthetic that adopts the Chinese yijing aesthetic in order to create harmonious experience and engagement but without the functional emphasis on data comprehension.

7.4 A New Aesthetic Approach This visual representation of sleep EEG data is a digital practice particularly focusing on experiencing the yijing aesthetic, which is considered a new aesthetic approach to data visualization based on an ancient philosophical system. The proposed framework highlighted the distinct visual forms of xu and shi, and rhythm with characteristics of emptiness, clarity, and complementarity of elements to produce a feeling of harmony. The work of Living Dream was created to evoke the yijing experience through the visual complementarity of xu and shi. Firstly, it presents the process of implementation of sleep EEG data. Secondly, it explores how the digital work evokes the yijing aesthetic experience. Lastly, Taoist embodied visualization will be address.

7.4.1 Code Implementation Processing software is used to create the works with a new aesthetic experience. Colson (2007) highlights the importance of code to data-based artwork. He states that “everything visible on the computer screen has been made by a statement in a computer program. The colour of the screen, the size of typeface or the shape of the cursor: each of these is addressable using code” (Colson, 2007, p. 96). Figure 7.10 presents code that creates a window of 1280 pixels wide and 720 pixels high. The program defines five ArrayList functions, which can assign the value of the numeric data in four sleep EEG signals and the value of speed. Figure 7.11 presents two main function of the code. The setup function creates the window in background color in (200, 30, 20) which shows in grey color. It calls a method of readFile, which read the EEG files and then setup visualization. In draw function, it creates a rectangle has the same size as the window, and then it starts to draw lines. In Fig. 7.12, a method (readFile) was implemented to read the EEG files. It firstly creates a table to store the data from the EEG files and then it removes all negative numbers using conditional statement. Figure 7.13 presents a method to represent EEG data as lines by loading Particle class. It shows lines crossing through the screen in different color and length heading to various directions. These code were implemented to create the yijing aesthetic experience. As yijing emphasizes the complementary of xu and shi, the next sections will discuss how the implementation of code presents the yijing experience. The code for Taoist data visualization was implemented based on the code for the yijing. The key difference is to import SimpleOpenNI library and then declare

7.4 A New Aesthetic Approach

151

Fig. 7.10 Screenshot of the code by Processing software showing the first part of code

Fig. 7.11 Screenshot of the code by Processing software showing two basic functions

the variable Kinect as showed in Fig. 7.14. In fact, there are many ways to Kinect programs, such as C++, Unity3D and Windows SDK (Cho & Lee, 2012). However, using OpenNI has the advantage of open source and many resources can be available online. Another key difference relies on class declared in the program (Fig. 7.15). PolygonBlob class is used to capture users’ body shape and motion (Pinedo Lopez,

152

7 Taoist Data Visualization

Fig. 7.12 Screenshot of the code by Processing software showing readFile() function

Fig. 7.13 Screenshot of the code by Processing software showing two functions and a class

7.4 A New Aesthetic Approach

153

Fig. 7.14 Screenshot of the code shows the implementation of Kinect

Fig. 7.15 Screenshot of the code showing PolygonBlob class

2017). These are allowed users be able to interactive with Kinect and the sleep EEG data, which could create a new Taoist embodied aesthetics.

7.4.2 Emptiness Emptiness or xu is considered a fundamental concept in traditional Chinese aesthetic such that it emerges as pivotal to the way the Chinese conceive the universe … emptiness governs the mechanism of a whole group of disciplines of meaning (painting, poetry, music, theater) and…the physiological domain (representation of the human body, the gymnastic practice of t’ai chi ch’uan, acupuncture, and so forth). (Cheng, 1994, p. 35)

In the process of making Chinese paintings, emptiness serves as a space (the unpainted area) for creative practice where artists express their spirit and thought through reality or fullness (shi). Emptiness has a fundamental function of structuring

154

7 Taoist Data Visualization

the use of reality or fullness that “makes up the visible aspect of structure” (Cheng, 1994, p. 47). In digital practice, emptiness is not referred to as a space in a traditional definition based on a rice paper medium, but actually is a “digital space” that usually refers to a computer monitor or display. Rice paper is termed xuan zhi and is one of the traditional mediums for Chinese painting. In contrast to rice paper, the digital space is composed “a grid of pixels” (Paul, 2003, p. 10) such that every pixel is designated as bits that are stored in a computer file in a binary form consisting of “ones” and “zeros” (Wands, 2006). Thus, in this praxis, the digital space initially was designed as a space of emptiness that just showed a grey screen without any objects (Fig. 7.16). This initial empty space enabled users to contemplate xu for stimulating engagement and imagination. Living Dream was designed as a form of animation, as Chinese aesthetics often requires a “vital quality” (qi) that “convey[s] a sense of liveliness in a work of art” (Pohl, 2009, p. 89). Taoism suggests the importance of qi in the close relationship between humans and the cosmos, in which the human “is a being not only of flesh and blood but also of breaths and spirits, who also possesses emptiness” (Cheng, 1994, p. 52). An ideal Chinese painting is required to link the painting of natural objects or the universe through the enigmatic Tao. It usually requires a criterion of “qi yun sheng dong” (vital resonance and live movement) (Pohl, 2009, p. 89). It is one of the most important principles in Xie He’s most influential work The Six Principles of Painting (Liufa) that describes the six fundamental principles in Chinese painting. In the principle of qi yun sheng dong, qi refers to spirit or vital quality; yun refers to resonance; sheng refers to life; and dong refers to movement (Bussagli, 1969). In the process of the practice, calligraphy plays a key role to create this vital quality through its calligraphic lines with black ink on a white surface that enables the brushstroke and ink to generate dynamic vitality (as well as an interplay between

Fig. 7.16 Screenshot of the digital space of emptiness. Qi Li, (2015), digital art. Photographer: Qi Li

7.4 A New Aesthetic Approach

155

Fig. 7.17 Screenshot of the work Living Dream representing data as lines, which symbolized the calligraphy. Qi Li, (2015), digital art. Photographer: Qi Li

xu and shi) (Pohl, 2009). The key to the interplay of white and black emphasizes movement and dynamism, which has more aesthetic potential than still images (Pohl, 2009). I adopted animation as a visual form that represents sleep EEG data, which generated data movement and dynamic vitality. The data was visually represented as lines, which symbolized the calligraphic lines of ancient Chinese painting (Li, 2015, Fig. 7.17). However, I did not adopt black as the only color of the lines, but used the value of the data itself to determine and compose the color of the lines. Color symbolized living nature with hundreds and thousands of objects. Color variation is also an important visual element in Western art.

7.4.3 Color and Data Color is one of the most important components of the visual arts in the West. It is also one of the important elements of aesthetic value, as Kant cited in Gage (1999, p. 8) suggests that “simple colors might be considered beautiful on account of their unmixed purity”. Color emphasizes subjective visual experience and objective stimuli (Gage, 1999), as it is considered as “the attribute of visual experience that can be described as having quantitatively specifiable dimensions of hue, saturation, and brightness” (Burnham, Haines & Bartleson cited in Gage 1999, p. 11). The color used in representing sleep EEG data as lines was inspired by the Dutch De Stijl movement (1917) that focused on creating visual movement using the natural color spectrum (Gage, 1999). In particular, Piet Mondrian’s abstract paintings symbolize three fundamental colors representing horizontal and vertical lines. His painting highlights a strong visual rhythm through primary colors with lines and squares, which generates movement and dynamism, especially in his painting Broadway Boogie Woogie (1942–43). The praxis in this research focused on color lines instead of black lines in the tradition of Chinese painting.

156 Table 7.2 Color representations in each signal of sleep data

7 Taoist Data Visualization First signal

Second signal

Third signal

Fourth signal

EEG Fpz-Cz

EEG Pz-Oz

EOG

EMG

Red (0–255)

Green (0–255)

Blue (0–255)

Speed

RGB color mode: White color represent as (255, 255, 255) Black color represent as (0, 0, 0)

Speed of movement

RGB color mode, which refers to Red, Green, and Blue as representing each pixel, is one of the most common color modes used in computer displays in digital practice. The principles of the RGB color mode involve using red, green and blue as the primary colors and generating secondary colors by combining red, green and blue, which is often used in computer programming (Kuehni, 2012). For example, yellow is generated by red added to green; purple is generated by red added to blue; and white is generated by red added to green and blue in the same amounts. In this research, RGB color consists of numbers in 24bit true color mode, in which each color ranges from 0 to 255. Number 0 represents black color and the number 255 represents white. Therefore, the different numbers in each color can generate many different colors. For example, a pure red color means red is set to 255 and green and blue are set to 0. It is represented as color (255,0,0). The four signals from sleep EEG data, which consisted of EEG Fpz-Cz, EEG PzOz, EOG horizontal and submental chin EMG, were visualized according to their values. Based on the RGB color mode, the first signal (EEG data) was visualized in red. The second signal (EEG data) was visualized in green. The third signal (EOG data) was visualized in blue. If the line is red, it means the first signal is of a greater number than the second and third signals. And then the three signals mix together to create a composite color. If the line is a grey color, it means that the values of the three signals were equal or that they had very close values. If the line is red, it means that the values of the first signals were largely greater than the value of two other signals. By mixing the values of three signals, color hue and saturability also become varied (Table 7.2). Figure 7.18 shows that each colored line is based on the value of each numerical data. These data-based lines, which are similar to the brushstrokes of Chinese calligraphy, are symbolized as the “digital brushstroke” in “digital space”. In addition to color, rhythm is another important element for the design of the yijing aesthetic according to the proposed framework.

7.4.4 Rhythm The composition of Chinese painting, in particular shan shui hua(mountains and water paintings), often requires a scattered perspective to create space that goes

7.4 A New Aesthetic Approach

157

Fig. 7.18 Screenshot of the work Living Dream (detail) representing data as colored lines, Qi Li, (2015), digital art. Photographer: Qi Li

beyond the Western linear perspective. It usually expresses this realistic space using the principle of qi fu (rising-falling) (Rowley, 1959), which refers to a concept of the “rhythmic sequence of landscape” (Cheng, 1994, p. 87). It is often used as qi fu bian hua that suggests changes, where qi or rhythm is created throughout the mountains and water painting, “suggesting the boundless infinity of the Tao” (Rowley, 1959, p. 64). This principle adopts a concept of the three depths to create a visual space consisting of a foreground, a middle ground and a far distance in the landscape (Rowley, 1959). Far distance, which is comprised of flat distance, high distance and deep distance (Cheng, 1994; Rowley, 1959), is a very important element for expressing the transformation of time and space. The objects, such as mountains and water, are tangible. The far distance breaks through the tangible objects, extending the viewer’s attention to a faraway area in order to move from limited space and time to unlimited space and time (Ye, 1998). It suggests the Taoist cosmological concept that a small painting reflects the magnitude of the universe. The grey lines in Living Dream symbolize various distances mixed with flat distance, high distance and deep distance that represent three different perspectives in Chinese landscape painting. Living Dream does not reflect the concept of three depths in terms of Chinese painting. Rather, it focuses on the spiritual movement of viewers carried along with Taoist qi or rhythm (Cheng, 1994). The xu or emptiness between these lines creates space that suggests immeasurability, which generates spirit and dream and the change of ink, including gradients between dark and light, also generates rhythm (Cheng, 1994). Of course, this space was generated within digital technology rather than created on physical rice paper, in the traditional sense. The colored lines and grey lines reflect the change of ink that transforms data to

158

7 Taoist Data Visualization

various digital colors. Thus, Living Dream focuses on the complementary movement of colored lines and grey lines in order to create rhythm and qi yun. The proposed framework suggests that an aesthetic approach to data visualization requires a design process working in accord with the user’s aesthetic experience. Instead of visually representing the data only as a beautiful form, the yijing aesthetic emphasizes the harmony of users and data through the complementarity of xu and shi. In Qi Li’s (2015) work, it shows a detail of Living Dream with grey lines and empty space that construct a digital space. The complementary existence of colored lines as shi (reality or fullness) and the digital space as xu (void or emptiness) along with grey lines generates qi and poetic rhythm with live movement. Qi can be understood as spirit in Taoist philosophy and it is an important criterion for Chinese painting. It highlights the importance of transferring nature to the practice of Chinese painting, as “the spirit of art was identified with the spirit of the universe” (Rowley, 1959, p. 34).

7.4.5 Transformation While the traditional Chinese aesthetic focuses on the discourse between nature and humans, the traditional Western aesthetics emphasizes the relationship between humanity and nature through the human perception of nature’s power, as we see in the notion of the Kantian sublime. However, how can traditional Chinese aesthetics find a place in digital practice? In other words, how can traditional Chinese aesthetics be reinterpreted and transformed in the context of digital technology? Or how does it achieve its “modernity” in the context of the information age? Chinese aesthetics differs from Western aesthetics that have been successfully modernized through the transformation of its tradition and in relation to technology (Pohl, 2009). For example, the technological sublime exemplifies the transformation inherent to the Kantian sublime, from experiencing the power of nature to the power of human-made artefacts (Nye, 1994). The concept of Western aesthetics has been expanded and extended to the context of technology, including railways, skyscrapers, and factories, in which “Kant’s sublime made the individual humble in the face of nature, the technological sublime exalted the conquest of nature” (Nye, 1994, p. 152). It emphasizes the transformation of subjective experience, which is very different from the Chinese aesthetics that emphasizes the experience of harmony and unification of subjective experience and objective scenes. The role of artists in painting in both traditions is also different. The traditional Western artist focuses on the expression of a realistic world with a photo-like painting that reflects the accuracy and clarity of natural science, such as artists of the Renaissance. However, in traditional Chinese painting, artists often express their ideas or emotion through the natural objects, such as mountains, water, trees and rocks. Cheng (1994) argues that artists did not just paint nature but painted themselves, for

7.4 A New Aesthetic Approach

159

to paint mountain and water is to paint the portrait of man – not so much his physical portrait…but more that of his mind and spirit, his rhythm, his gait and bearing, his torments, his contradictions, his fears, his peaceful or exuberant joy, his secret desires, his dream of the infinite, and so forth. (p. 84)

It can be described as “jie jing shu qing”. This idea becomes fundamental to evoking the yijing aesthetic as a fusion of artist’s feeling and the objective scene in order to harmonies the self (qi) and nature, through an inner transformation. Living Dream does not aim to express my individually specific emotion like that in the traditional expression of the yijing aesthetic, but, as a digital practice, it is more concerned with the evocation of the viewer’s harmonious perception of the power of digital technology through this process of creative practice. It is also considered a demonstration of the proposed framework (Fig. 5.1 in Chap. 5) in the aesthetic approach to data visualization in order to evoke harmonious experience between users and the data associated with technology. Thus, this research does not aim simply to apply a traditional Chinese aesthetic concept to digital practice with its focus only on the visual representation of natural objects. This research seeks a transformation of traditional Chinese creative materials such as rice paper, brush and ink to digital materials such as monitors, keyboard, mouse, and computer code with pixels. The questions raised here are: how can the yijing aesthetic are experienced in digital practice, where the computer monitor replaces rice paper, in particular in the concept of xu represented in the form of pixels? Living Dream also did not attempt to represent a virtual natural world reflecting traditional Chinese painting motifs, such as mountains, water, flowers and birds. Nor did it just mimic Chinese paintings with mountains surrounded by water in a digital form. In both cases, the digital technology is only considered as a “tool” but I have chosen to use the technology as a medium (Paul, 2003). Instead, the work emphasizes the characteristics of the yijing aesthetic as a way of exploring abstract visual forms through the symbolism of qi or spirit of the natural universe but in the digital macrocosm that I term “digital yijing”.

7.4.6 Spirit Through Brushstrokes Most of studies of traditional Chinese painting in the West are focused on the discourse of Chinese aesthetics with its creative principles. However, a few highlight the emphasis on materials and process of making Chinese paintings. The material used in Chinese painting requires higher skills to be familiar with ink and water in order to generate ideal visual effects, which is very different to oil painting in the West. Oil painting allows artists to change their works in terms of color, outline and shading many times, whereas with Chinese painting on rice paper it is much less possible to make these changes. It is also impossible to generate rich colors to create a photo-like image using the natural spectrum of Chinese ink, in the way that many Western artists working in the landscape tradition do. This limitation of material

160

7 Taoist Data Visualization

allows artists to focus more on individual spirit expression, with its requirements of ardent practice to master the painting skills that are freely controlled by artists. This highly developed painting skill often refers to a concept of gongfu (Pohl, 2009) or gongdi, which was borrowed from the tradition of martial arts gongfu in which strenuous practice is required to be a gongfu master. An artist with high painting skill is often considered a gongfu master in art. Once an artist obtains this high painting skill with a deep understanding of Chinese aesthetics, the artist is able to create his painting more freely. Shi tao (cited in Pohl, 2009, p. 90) suggests that “the highest rule is that of the rule of no rule”. This statement emphasizes that a free creative spirit has “‘perfect intuitive control’ over the artistic medium” (Pohl, 2009, p. 90). Shi tao, a painter in the Qing Dynasty, is famous for his essay “hua yu lu”, which emphasizes a concept of yi hua (one stroke) that is “an enigmatic concept without stylistic connotation and refers to the creative process of painting” (Loehr, 1980, p. 302). As one of the fundamental mediums, the brush has been used to create visual images on rice paper in traditional Chinese painting. Brushstrokes are considered a key to generating qi or spirit in traditional Chinese painting, as Tsung Ping cited in Cheng (1994, p. 63) suggests, the spirit has no form of its own; it is through things that it takes form. Thus, the idea is to trace the inner lines of things by means of brushstrokes inhabited by shadow and by light. When things are adequately caught in this way, they become the representation of truth itself.

In digital practice, technology has replaced brush, ink and rice paper with computer monitors and programming codes in an untouchable form. The moving lines in Living Dream symbolize brushstrokes to generate qi or spirit in the visual representation of sleep EEG data. By adopting Shi tao’s concept of one stroke, each of the sleep EEG data, which was represented as a colored line, is symbolized as one stroke. Each line in the visualization represents one sleep EEG data presented in numerical form. In this work, it is not a physical stroke but a “digital stroke”, which symbolizes the spirit or qi yuan generated in the digital context that emphasizes the superpower of digital technology.

7.4.7 Sleep EEG Data as Living Data Sleep EEG data, which has been visually represented in this work, is considered a living data that generates visual rhythm through data movement as lines rather than as static data that conveys information for comprehension. Indeed, the living data is neither recognizable nor readable, and it does little to identify any valuable insights by its visual representation. In contrast, according to the proposed framework, the work of Living Dream emphasizes users’ experience of a fusion between an individual’s subjectivity and the so-called objective world of Western epistemology. The value of pleasure is generated through the design process. In the six principles for the practice of Chinese painting, qi or qiyun is considered the most important characteristic (Arnheim, 1997; Bussagli, 1969; Pohl, 2009). The

7.4 A New Aesthetic Approach

161

sleep EEG data was visually represented as colored lines moving with various speeds, based on the value of the fourth signals (EMG). These colored lines appear as dynamic movement across the digital space in different positions in what initially appeared as an empty space. This dynamic movement creates qi through the transformation of xu and shi in a complementary way. The data constantly appeared across the space and vanished tracing the grey or low hue color of lines that consisted of an enriched emptiness. These dynamic lines and grey lines form the ambiguous shapes of mountains and water, which is similar to Chinese mountains and water painting. Nature and cosmology have always been selected as creative themes in traditional Chinese painting. Cheng (1994) suggests that “a painting does not aim merely at being an aesthetic object but rather seeks to become a microcosm that is itself creative in the manner of the macrocosm, an open space in which real life is possible” (p. 62). It highlights the importance of cosmology in Chinese painting, which requires capturing the spirit of the cosmos reflecting an individual’s inner ideas for painting. Thus, Chinese painting does not seek the realistic representation of the exterior of nature, but tries to capture the inner laws (li) that “hold fast the hidden relations between” (Cheng, 1994, p. 63) two polarities, such as mountains and water, man and woman, day and night, as a key concept of yin and yang. It is important in the cosmological theme related to Chinese painting, which often focuses on the combination of the two-opposite fundamental forces of yin and yang as one being, in particular as used in mountains and water paintings, in which mountains and water are two opposite objects. In Living Dream, colored lines and grey lines, in which data is represented as dynamic objects with rhythm through the creation of qi, form a relationship between two fundamental forces of yin and yang in terms of Taoist philosophy. The next section will discuss how Chinese aesthetics transforms two opposite natural objects into a harmonious whole through exploring Chinese classic painting of mountains and water.

7.4.8 Harmony Mountains and water paintings should be understood as a distinct artistic style that extends the expression of the artist’s thought and emotion in the landscape. In Chinese philosophy, mountains and water comprise two poles of nature that reflect on the heart (mountains) and spirit (water) of humanity (Cheng, 1994). Simultaneously, mountains and water represent the concepts of yin and yang, where mountains are symbolized as yang and water is symbolized as yin. The pairing of mountains and water presents a fundamental symbol in the transformation of cosmic processes, in which mountains or water is “perceived as a state that is constantly attracted and complemented by the other” (Cheng, 1994, p. 85). Mountains and water can be considered as ways for perceiving the magnitude of nature, as it is critically necessary to “have recourse to mountains in order to see the breadth of the world. Nothing less is necessary than to have recourse to water to see the immensity of the world”

162

7 Taoist Data Visualization

(Cheng, 1994, p. 86). It is a different aesthetic experience from the Kantian aesthetic that interprets the power of nature, in which the mathematical sublime concerns the extreme magnitude of nature that exceeds the viewer’s comprehension, including forms such as mountains or oceans, and the dynamic sublime concerns the attempt of viewers to control nature. By contrast, Chinese aesthetics emphasizes the transformation between two opposite objects in nature in order to achieve an integrated perception of nature. This transformation in mountains and water painting “catches the harmonious cosmological order of the world and its forces in a microcosmic way” (Pohl, 2009, p. 90). However, Living Dream does not attempt to present sleep data as a realistic landscape or a natural object, but expresses “a poetic quality” (Pohl, 2009, p. 89) in abstract form. In Chinese aesthetics, an ideal painting is usually considered within the realm of xiang wai zhi xiang (images beyond images) (Pohl, 2009, p. 89), which does not focus on realism like that used in traditional Western paintings. The nonrealistic and abstract representation of the data in this work conforms well to the notion of images beyond images, as it creates a scene evoking the user’s imagination of nature and humans (Fig. 7.19). This imagination in Chinese painting emphasizes that Beyond mountains there are more mountains; Although they appear to be disconnected, they are actually not, Beyond trees there are more trees; Although they appear to be joined together, they are actually not. (Rowley, 1959, p. 51).

This statement suggests the philosophical underpinnings of the work involving the imagining of nature through moving lines constructed through ideals of harmony.

Fig. 7.19 Screenshot of the work Living Dream (detail) creating a scene, Qi Li, (2015), digital art. Photographer: Qi Li

7.4 A New Aesthetic Approach

163

As the yijing aesthetic approach, Living Dream has achieved the harmonious experience of visual representation of sleep EEG data through its complementarity of xu and shi and rhythm with vital quality. The next section will discuss how Taoist philosophy achieves a new aesthetic of embodied data visualization through gesturebased technology.

7.5 Unification of the Body and Digital Technology Taiji was designed as an interactive installation that explores the concept of Taoist data visualization in the context of digital practice, in which it is possible to generate a new aesthetic experience for users. This second work is based on Living Dream, extending upon the engagement with embodied interactivity that was established by it. Gesture-based technology was used to achieve this new Taoist aesthetic. Kinect is considered an ideal gesture-based technology with its distinct interactivity that enables one to “capture, track and decipher body movement, gestures and voice. The auditory and visual information serve as commands to interact with digital contents presented in games or software programs” (Hsu, 2011, p. 365). Taiji was inspired by Taoist health philosophy that emphasizes the practice of taijiquan with the flow of qi moving through the body in order to achieve harmony and self-transcendence. Taiji is an outcome of this digital practice that attempts to represent sleep EEG data in the form of qi that users are able to interact with through a Taoist body. This Taoist body is a kind of embodied digital body that unifies the user’s body and data associated with the digital technology. Taoism often highlights that tian ren he yi (unification of heaven and human as one) is a fundamental philosophical concept for Chinese aesthetic thought (Li, 1999). Hence, the work of Taiji highlights the unification of body and digital technology as digital Taoist body (discussed further in Sect. 5.5.2) rather than the unification of heaven and the human. One member of the audience described the work as “interesting interactive art”, indicating that Taiji created an interactive experience based on digital technology to achieve Taoist body.

7.5.1 Sleep EEG Data as Performance As Taiji was based on the first work, Living Dream, sleep EEG data was initially visually represented according to the movement of colored lines. However, when a participant joins this work and appears in the front of the screen, these lines, including colored and grey lines, transform to shapes of the participant’s body. Kinect captures the shape of participant’s body and integrates the sleep EEG data and data from the participant’s body together as one whole as a Taoist body. This reflects the Taoist principle of the unification of the human subject and the universe. It allows participants to interact with the data through his or her body movements. One of the key characteristics of interactive technology is to generate immersion that enables the

164

7 Taoist Data Visualization

participant to engage deeply with visual images. Taiji enables participants to interact with the sleep EEG data while experiencing immersion. The lines keep moving across the shape of the virtual body combining sleep EEG data and the data of the living body (i.e. its movements, distance from the technology). It can be symbolized as a dynamic Taoist body, which embodies dream (a kind of subconscious state) and digital technology. The visualization is then transformed to different shapes when the participant moves his or her arms and body. The sleep EEG data was continually transformed in its visual form, such as ellipses, based on its values in response to the participant’s movements. The colors of these various shapes were based on the value of each numerical data, which is same as the color of the lines. It also enabled the participant to contemplate more than just visual forms that are generated by computer code. Instead of expressing an artist’s intention, this work was designed more for the participant to contemplate the myth and power of digital technology. Based on my observations of the exhibition, this contemplation is not only considered a visual experience, but also simulates the participant’s imagination, underscoring the relationship between digital technology and humans, as well as the experiencing of the power of computer technology. During the exhibition, some participants described the work as a “touching experience”, which highlights that “knowledge is proximity” in terms of Taoism (Hall & Ames, 1998, Fundamental notions: de section, para. 6). Knowledge generated from the work is a kind of proximate knowledge different from knowledge generated from conventional visualization technology, such as the waveform patterns discussed previously. Considering the format of the exhibition held at Edith Cowan University, I combined the two works Living Dream and Taiji into one work, which was displayed on the wall through a projector. The work in the exhibition is slightly different to the version tested at the motion capture laboratory, due to the limitation of the Kinect technology. For example, the work in the exhibition did not display different shapes as a response to the audience’s interactivity with data.

7.5.2 The Digital Taoist Body Taoism positions the human body as a microcosm that reflects the universe as a macrocosm. Often Taoist thought emphasizes the importance of a harmonious relationship between nature and the body, with such ideas extending to the attainment of health and longevity. The practice of taijiquan is considered one way to achieve the unification of body and nature for the purpose of health through its slow body movements mimicking animal behavior. In this research, the Taoist body can be understood as a digital Taoist body that transforms the traditional concept of the Taoist body as the unification of nature and viewer’s body to the unification of digital technology and the user’s body. This transformation is considered a transformation from the universe to the body, as Taoism argues that the creation of the universe could be understood through Laozi’s body transformation, which treated Laozi as a God in the Taoist religion.

7.5 Unification of the Body and Digital Technology

165

As Maspero (cited in Schipper, 1978, p. 359) translated, Laozi “transformed his body” and his “left eye became the sun, his right eye the moon, his head the K’unlun mountain, his beard the planets and the heavenly mansions, his bones dragons, his flesh quadrupeds, his bowels snakes…”. From this description, Laozi’s body integrates into the universe as the one. However, the body shape in Taiji differs to the traditional Taoist body that emphasizes “tending life” (Schipper 1978, p. 365). In Fig. 7.20, the body shape created by Kinect was not represented as a realistic body shape that can be identified; rather it shows an unclear “ghost-like” form of the body that symbolizes the incomprehensible macrocosm of Tao infusing the microcosmic body (Li, 2015). In this visualization, Tao was not considered the universe or cosmos in a traditional sense but as a concept of “digital technology” in which pixels comprise the basic elements to represent a digital cosmos. In this digital cosmos, while the Kantian sublime can be used to describe the experience of magnitude in the form or size of data, a Taoist philosophical approach attempts to build a new relationship between digital technology and human users, in which scientific, numeric data could be perceived, contemplated and imagined. Figure 7.21 shows a visitor is playing with Kinect at the exhibition in Spectrum Project Space, Perth, Western Australia. Aesthetic experience in Taiji differs significantly to the information-based data visualization and the work of Living Dream with the yijing aesthetic. The interactivity of Taiji brings about an uplifting experience, as comments from audience members emphasizes, describing this work as an “uplifting interactive artwork”. The new aesthetic of Taiji is central to the work being “uplifting”. Based on the proposed framework (Fig. 5.1), Taiji refers to an artistic expression that does not attempt to convey the meaning of the data; indeed it is neither recognizable nor readable in the

Fig. 7.20 The digital presentation of Taoist practice, photographer: Qi Li

166

7 Taoist Data Visualization

Fig. 7.21 A visitor is playing with Kinect, photographer: Qi Li

work. Instead, users were able to experience immersion or attention through interaction with the data. The key to understanding this work is that embodied interactivity enhances the user’s engagement through immersion (in the data, in the environment) and imagination (of the data, in the subjective perception). Imagination is a key in this work, as it is often been undervalued in conventional scientific visualization. Although science focuses on explanation of natural phenomenon, it is important to have imagination to “look beyond the limitations of what’s currently known, and see what could possibly be … and even more imagination to make sure this venturing beyond current understanding still stays within the bound of reason and known rules of science” (Plait, 2009, para. 12). Imagination generated from Taiji provides an alternative approach to scientific data visualization for data insights and discovery. Two works have exhibited in Perth’s Spectrum Project Space in Australia in 2015. Figures 7.22 and 7.23 are where the exhibition held.

7.6 Conclusion This praxis has employed an approach informed by the traditional Chinese aesthetic concepts and Taoist principles to the design of data visualization in generating a new aesthetic experience, which is different not only from communication-based data visualization but also from aesthetic approaches focused on the comprehension of data. Three artefacts of the visual representation of sleep EEG data have been considered in demonstrating the proposed framework: information, the yijing aesthetic and

7.6 Conclusion

167

Fig. 7.22 The exhibition space. It presents the initial visualization of the work, in which emptiness of the space was displayed and the initial colored lines across the space, photographer: Qi Li

Fig. 7.23 The exhibition space; the image is continued visualization of the work. photographer: Qi Li

168

7 Taoist Data Visualization

Taoist data visualization. Sleep EEG data visually represented, as a waveform is an example of information that can be recognized, read and understood efficiently. In contrast, Living Dream demonstrates the yijing aesthetic through the experience of harmony and pleasure through the complementarity of xu and shi; Taiji demonstrates Taoist data visualization through the experience of immersion and embodiment using the Kinect technology. Kinect is a key to generating an embodied interactivity in order to achieve Taoist data visualization. Taoist data visualization extends the original significance of the Tao from the enigmatic universe to the power of digital technology. This embodiment creates the Taoist body that unifies the user’s body and the data to generate new knowledge through digital practice involving the experience of immersion and harmony. This framework also highlights the data continuum from communication-focused to aesthetic approaches and then to embodied Taoist data visualization. As a new data-based digital practice, the production of the artefacts explored the yijing aesthetic and Taoist body principles in the context of digital technology. Living Dream explores the concept of “qi yun shen dong” with the complementarity of xu and shi reflecting the movement of lines formed by data in the digital space. Taiji highlights the relationship between the human subject and digital technology offering the possibility of modernizing Chinese aesthetics in digital environments.

References Arnheim, R. (1997). Ancient Chinese aesthetics and its modernity. British Journal of Aesthetics, 37(2), 155–157. Broadway Boogie Woogie (1942-43) is an artwork with the year the work created. Burke, E. (1998). A philosophical enquiry into the origin of our ideas of the sublime and beautiful. (A. Phillips, Ed.). Oxford: Oxford University Press. Bussagli, M. (1969). Chinese painting. New York, NY: Hamlyn. Card, S. K., Mackinlay, J. D., & Schneiderman, B. (1999). Readings in information visualization: Using vision to thin k. San Francisco: Morgan Kaufmann. Carskadon, M. A., & Dement, W. C. (2011). Monitoring and staging human sleep. In M. H. Kryger, T. Roth, & W. C. Dement (Eds.), Principles and practice of sleep medicine (5th ed., pp. 16–26). St Louis: Elsevier Saunders. Cheng, F. (1994). Empty and full: The language of Chinese painting (M. H. Kohn, Trans.). Boston, Massachusetts: Shambhala Publications. Cho, O. H., & Lee, W. H. (2012). Gesture recognition using simple-OpenNI for implement interactive contents. In Future information technology, application, and service (pp. 141–146). Springer, Dordrecht. Colson, R. (2007). The fundamentals of digital art. Lausanne: AVA Publishing SA. Gage, J. (1999). Colour and meaning: Art, science and symbolism. London: Thames & Hudson Ltd. Giblett, R. J. (2008). The body of nature and culture. Basingstoke: Palgrave Macmillan. Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P Ch., Mark, R. G., et al. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101(23), 215–220. Hall, D., & Ames, R. T. (1998). Daoist philosophy. In E. Craig (Ed.), Routledge encyclopedia of philosophy. London: Routledge.

References

169

Hobson, J. A. (2009). REM sleep and dreaming: Towards a theory of protoconsiousness. Nature Reviews Neuroscience, 10(11), 803–813. Hsu, H. J. (2011). The potential of Kinect in education. International Journal of Information and Education Technology, 1(5), 365–370. Kant, I. (2000). Critique of the power of judgement (P. Guyer & E. Matthews, Trans.; P. Guyer Ed.). New York, NY: Cambridge University Press. Kemp, B., Zwinderman, A. H., Tuk, B., Kamphuisen, H. A., & Oberye, J. J. (2000). Analysis of a sleep-dependent neuronal feedback loop: The slow-wave microcontinuity of the EEG. Biomedical Engineering, IEEE Transactions on, 47(9), 1185–1194. Kant, I. (2007). Critique of judgement (J. C. Meredith, Trans. N. Walker Ed.). New York, NY: Oxford University Press. Kuehni, R. G. (2012). Color: An introduction to practice and principles. New York, NY: Wiley. Li, S. (1999). Reflections on the concept of the unity of heaven and man (“Tian Ren He Yi”). In K.-h. Pohl (Ed.), Chinese thought in a global context: a dialogue between Chinese and Western philosophical approaches (pp. 115–128). Leiden, the Netherlands: Koninklijke Brill NV. Li, H. (2013). Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM. arXiv preprint arXiv:1303.3997. Li, Q. (2015). Taiji [Digital art]. Perth, Australia: Spectrum Project Space, Edith Cowan University. Li, R. Q., McMahon, M., & Haddad, H. (2013). An investigation into a new aesthetics of scientific data visualisation. In Paper presented at the Australian Council of University Art & Design Schools 2013: Locations of Practice, Sites for Creativity: From the Studio to the Cloud, Sydney, Australia. Retrieved from: http://acuads.com.au/conference/2013-conference/article/an-investiga tion-into-a-new-aesthetics-of-scientific-data-visualisation/. Li, Z. (2009). Chinese aesthetic tradition (M. B. Samei, Trans.). Honolulu: University of Hawaii Press. Loehr, M. (1980). The great painters of China. Oxford: Phaidon Press Ltd. Manovich, L. (2002). Data visualization as new abstraction and anti-sublime. Retrieved from http://manovich.net/content/04-projects/039-data-visualisation-as-new-abstraction-and-anti-sub lime/37_article_2002.pdf McCosker, A., & Wilken, R. (2014). Rethinking ‘big data’as visual knowledge: The sublime and the diagrammatic in data visualisation. Visual Studies, 29(2), 155–164. Migotina, D., Isidoro, C., & Rosa, A. (2011). Brain art: Abstract visualization of sleeping brain. In Proceedings of GA 2011-XIV Generative Art Conference (pp. 141–153). Roma, Italy. Mourtazaev, M., Kemp, B., Zwinderman, A., & Kamphuisen, H. (1995). Age and gender affect different characteristics of slow waves in the sleep EEG. Sleep-New York, 18(7), 557–557. Norman, D. (2004). Emotional design: Why we love (hate) everyday things. New York, NY: Basic Books. Nye, D. E. (1994). American technological sublime. Cambridge, MA: M.I.T. Press. Owed, A. (2012). Kinect physics tutorial for processing. Retrieved from http://www.creativeappl ications.net/processing/kinect-physics-tutorial-for-processing/. Parman, S. (1991). Dream and culture: An anthropological study of the western intellectual tradition. New York, NY: Praeger Publishers. Paul, C. (2003). Digital art. London: Thames & Hudson. Plait, P. (2009, April 06). Science is imagination [Web log post]. Retrieved from http://blogs.discov ermagazine.com/badastronomy/2009/04/06/science-is-imagination/-.VSYcKtwk-hM. Pinedo Lopez, R. (2017). Interaction for creative applications with the Kinect v2 device. Master’s thesis, Universitat Politècnica de Catalunya). Pohl, K. (1996). Chinese aesthetics and Kant. Twenty-first Century, 8(36), 84–89. Pohl, K.-H. (2009). Identity and hybridity—Chinese culture and aesthetics in the age of globalization. In A. V. d. Braembussche, H. Kimmerle & N. Note (Eds.), Intercultural Aesthetics (pp. 87–103). Netherlands: Springer. Rowley, G. (1959). Principles of Chinese painting. Princeton, New Jersey: Princeton University Press.

170

7 Taoist Data Visualization

Schipper, K. (1978). The Taoist body. History of Religions, 17(3/4), 355–386. Senay, H., & Ignatius, E. (1990). Rules and principles of scientific data visualization. Retrieved from http://www.siggraph.org/education/materials/HyperVis/percept/visrules.htm. Shapiro, C., Sherman, D., & Kryger, M. (2013). Sleep in art and literature. In M. Kryger (Ed.), Atlas of clinical sleep medicine (2nd ed.). London: Elsevier Health Sciences. Tufte, E. R. (2001). The visual display of quantitative information. Cheshire, Connecticut: Graphics Press. Viegas, F. B., & Wattenberg, M. (2007). Artistic data visualization: beyond visual analytics. In Proceedings of the Second International Conference, OCSC 2007 Held as Part of HCI International 2007 (pp. 182–191). Beijing: Springer Berlin Heidelberg. Wands, B. (2006). Art of the digital age. New York, NY: Thames & Hudson. Ye, L. (1998). An analysis of Yijing. Research of Literature and Art, (1).

Chapter 8

Future Research

This book explored the possibility of the transformation and reinterpretation of traditional Chinese aesthetics and Taoist philosophy into the context of contemporary digital technology, through which a new aesthetic experience of data could be established. This new aesthetic was located in the experience of the digital work through a gesture-based gaming technology environment in order to generate embodied interactivity. This digital practice involved creating two artefacts as guided by the theoretical framework, which serves as a general basis for designing the data visualization according to the questions and the purposes of the project.

8.1 Summary of the Research The praxis in this research reflects the two processes of individual subjective creative practice in relation to the enactment of the theoretical framework within the project model. Three levels in the proposed framework (information, the yijing aesthetic and Taoist data visualization) describe how data, information, aesthetics and Taoist body philosophy can be used along a data continuum in consideration of the factors of process, moderators or influence, and users. The process of the creative practice with two artefacts as the outputs, which are Living Dream and Taiji, has demonstrated different aspects of the theoretical framework. Sleep EEG data was used as an example to design a data visualization in which visual forms and characteristics were drawn from the framework. The process of data in organization, design and expression of the research influences users in three different perceptual areas: information, aesthetics and Taoist data visualization tenets that were overviewed from a review of relevant literature on data visualization, Chinese aesthetics and the Kantian sublime. In the design of the data visualization, the yijing aesthetic was adopted to generate emotionally harmonious experience between the subjective experiences of users and the visualization of data through two fundamental elements of xu and shi that exist complimentarily to generate rhythm. The concept of xu or emptiness is © Shanghai Jiao Tong University Press 2020 Q. Li, Embodying Data, https://doi.org/10.1007/978-981-15-5069-0_8

171

172

8 Future Research

more important than shi, because xu is considered as the original source in the world in terms of Taoist philosophy. Data visualization is not only considered as a form of communication, but also as a creative practice in the last few decades (discussed in Chap. 4). Data visualization is involved in artistic practices known as “data art”. In creative expression of data visualization within the framework, Taoist philosophy has been used to create Taoist data visualization as a digital practice through gesturebased technology embodied as a Taoist body (comprising the digital and the material bodies). Traditionally, Taoist philosophy highlights a unified relationship between humanity and the cosmos to attain harmony between them, which differs significantly to the Kantian sublime emphasizing the experience of the magnitude and power of nature from a distance. This often underpins the concept of dualism between body and mind. In this sense, this research provides a critique of the Kantian sublime underpinning data visualization through the development of Taoist data visualization. The latter achieves a digital Taoist body through the unification of the body of users and the data associated with the visualizing technology, engendering attention and immersion through gesture-based gaming technology, e.g. Kinect. A practice-led research methodology was adopted in this research project, as the nature of this research involved the process of design and creative practice involving data as information or as creative medium. It involves “developing and making creative work as an explicit and intentional method for specific research purposes” (Gray & Malins, 2004, p. 104), from which knowledge is generated from the artefacts as the outcomes of the creative practice. The process of creative practice emphasizes research into, through, and for practice that is considered a problem-solving practice adopting an aesthetic approach within the framework. This practice-led research was framed as digital practice that creates digital artefacts through computing technology, in which computer programming is a crucial part of the practice. Digital technology serves as either a “tool” to generate traditional artworks or a “medium” to create artworks that are interactive, participatory, dynamic and customizable (Paul, 2003). Through this practice-led research methodology, the artefacts created from this practice are considered outcomes of creative digital practice and demonstrate different forms of data that are data as information, data as aesthetics and data as Taoist data visualization, reflecting the categories of the framework. By using a practice-led research methodology, this research adopted Marshall’s double helix of praxis-exegesis model to synthesize praxis (digital practice) and the written document (exegesis) together as the process of the reflective practice. Drawing upon Marshall’s double helix model, I rephrased the key terms to suit the need of my research, in which I used “create & reflect, build, and test” to replace “create, reflect, and respond” on the praxis side and used “observe, explain, and evaluate” in place of “observe, describe, and analyses” on the exegesis side. The new double helix model conforms to my intention to create digital artefacts by building computer systems.

8.2 Significance and Contribution

173

8.2 Significance and Contribution The contribution of this research lies in the innovative application of traditional Chinese aesthetics to the visual representation of scientific data, generating a new aesthetic experience that is different to most of Western aesthetics. On the one hand, the digital artefacts contribute to the demonstration of the new aesthetic approach to data visualization in the theoretical framework; on the other hand, this research is opening a door to seek a way for transformation of the traditional Chinese aesthetics in relation to the context of contemporary digital technology. This transformation relies on the shift from traditional Chinese creative media to digital practice with the reinterpretation of the concept of traditional Chinese aesthetics in the relationship between the users and digital technology. The theoretical framework with the project model contributes to the general principle of designing data visualization, which engages with traditional Taoist philosophy and the gesture-based technology. It is important to realize the needs and suitability for designing numeric related data visualization in terms of communication or aesthetics. Serving as a design guideline to data visualization, this framework provides an additional approach to the aesthetic design of data visualization, as well as data-based creative practice. Depending on the need of their research, artists or designers are able to adopt either a method of representing data as information for maximal understanding through the selection of a suitable visual form; or data represented through the yijing for promoting the experience of harmony and balance; or data as creative practice reflecting the Taoist philosophy of unification of body and the data through gesture-based technology. While there is an increasing research about aesthetic approaches to data visualization, most of the research attempts to identify and develop the aesthetic underpinnings of enhanced comprehension of data, especially in the information visualization field that often emphasizes the role of aesthetics assisting users to understand information. However, more and more artists attempt to bring data into their practices in relation to social, cultural or aesthetic themes that attempt to avoid the great emphasis on the function of communication for understanding. This enquiry therefore provides a fundamental framework of synthetizing data visualization from visual representation for communication to creative expression of digital practice. Some scholars, such as Manovich (2002), argue that data based art is anti-sublime and that artists do not attempt to generate aesthetic experience beyond the user’s comprehension. However, Taoist data visualization in this theoretical framework does not emphasizes experiencing Western aesthetics, such as the Kantian sublime, but it focuses on the unification of users and data as one whole harmonious experience. Thus, it provides a non-Western aesthetic approach to data visualization emphasizing the experience of harmony and unification, which contributes to the aesthetic, embodied, immersive, emotional, and engaging aspects of data visualization. In the methodology, Taoist epistemological enquiry has contributed to the design of the data visualization. Taoist epistemology emphasizes seeking the reality of objects that are linked with nature. It requires designers to think about the self,

174

8 Future Research

data, its contents and constructs in a natural way that is seeking knowledge of one’s true self. Further exploration of the relationship between the designer and the data, for a deeper understanding of data where new knowledge can emerge, should be undertaken in further research. Additionally, the Kinect technology used to create the Taoist data visualization contributes to embodied interactivity that enhances user engagement and produces an enjoyable experience. Integrating the practice of taijiquan with Kinect offers a significant possible contribution to medical related data visualization, as Taoist body philosophy advocates the unification of the body with nature in order to achieve a manner of health as influenced by nature.

8.3 Research Findings The findings of this research lie in this creative practice primarily focusing on the following three aspects: aesthetics, technology and the theoretical framework.

8.3.1 Aesthetics The findings of this research in aesthetics focus on a new aesthetic experience that has emerged from the artefacts of Living Dream and Taiji with Taoist embodied data visualization. It is clear that both visualization works did not attempt to promote the communication of sleep EEG data for understanding or insight, but attempted instead to evoke emotional harmony and the unification of the body of users and the visualization of data. The key findings in aesthetics are discussed in the following: – The Taoist aesthetic lies in the digital representation of the concept of traditional Chinese aesthetics, which was generated as visual outcomes through the automation provided by computer code. It embodies the traditional Chinese aesthetic concept within digital technology. The digitalization of traditional Chinese aesthetics has the potential to shift its ancient concept of cosmology to addressing the relationship between humanity and digital technology in the context of contemporary digital environments, emphasizing characteristics of enigma and awe in the digital productions, as inspired by the Taoist epistemology. – It is critical to combine traditional Chinese aesthetic concepts with contemporary digital technology in order to open a door to a possible modernization of Chinese traditional aesthetics. Pohl (1996) suggests that Chinese aesthetic models need to have a role in the critique of Western aesthetics through cross-cultural dialogue with Western ideas of art and experience. It needs to be clarified that the Taoist data visualization clearly did not result in what Western aesthetics emphasizes as the “beautiful” achieved through harmony in the visual images or visualization. However, my Taoist visualization model has led to the spirit of the Chinese

8.3 Research Findings

–

–

–

–

175

aesthetic concept of qi yun sheng dong through the digital process of automating, continuing, and engaging with creative practice through computer programming code. The computer technology presents its power to transform unseen and highly abstract data into visual form, allowing people to appreciate or communicate. This power was not experienced as magnitude with awe or danger in terms of Western aesthetics, but created a harmonious atmosphere in one whole. Based on my literature review and preliminary participant’s feedback, it enabled people to think deeply in a different way about the nature and almightiness of the technology as well as its relationship to humanity. Future research could involve more significant participant engagement. Western aesthetics emphasizes the experience of awe or danger as a dualism of body and mind. This kind of opposition raises tension, which users may experience through the magnitude or vastness of the data with fear or terror, in particular, in medical related data visualization. For example, patients may experience fear or danger from the conventional visualization of sleep EEG data as a waveform. Based on the natural harmony of the body and the data, I find that this new aesthetic from Taoist data visualization emphasizes the unification of body and data in providing an alternative way to overcome the awe or danger experienced by the user, which could be applied in broader areas including medical and health related settings. In addition, the harmonious experience from the yijing could enhance this experience of unification of the subjective and the objective. Digital technology has the ability to mimic traditional Chinese paintings, such as the computer animation mimicking traditional Chinese painting on the theme of mountains and water paintings or birds and flower paintings. However, digital images that display computer code generated strokes are not the same as the free expression of traditional Chinese paintings with brushes, ink and rice paper. Thus, digital practice using traditional Chinese aesthetic concepts should not simply mimic the traditional Chinese painting. The modernization of Chinese aesthetics should go beyond the application of digital technology as a tool. In such a sense, the transformation requires a new interpretation of traditional Chinese aesthetics in terms of digital technology that emphasizes Taoist philosophical principles of focusing on the concepts of unification and harmony between human users and the digital technology. As the key principle of traditional Chinese painting, qi yun sheng dong is considered a criterion to create living visual forms in the yijing aesthetic through the complementary relationship between xu and shi. Concerning the digital artefact of living dream, the yijing aesthetic experience was constructed through visual representation of sleep EEG data as moving lines, which symbolize the strokes of Chinese paintings. This demonstrates the characteristics of aesthetic form according to the theoretical framework. Two artefacts have demonstrated the theoretical framework constructed with variables, whereas the proposed new aesthetic emerged from the creation of the digital artefacts.

176

8 Future Research

8.3.2 Technology In the process of praxis, Kinect plays a key role in creating immersion and more engagement that focuses on the embodiment of data through Taoist philosophical concepts. The key findings in relation to technology are discussed in the following: – Through the process of observation, I find that Kinect generated an embodied data visualization reflecting the traditional spirit of Taoist body philosophy and the concept of its health philosophy. This gesture-based technology integrates the practice of taijiquan with sleep EEG data promoting Taoist health concepts. – The digital technology provides a new environment and atmosphere for the reinterpretation of Taoist cosmological principles associated with the yijing aesthetic. I find that the transformation of Chinese aesthetics into the context of digital technology offers a new aesthetic experience, in particular, in substituting traditional Chinese painting medium with digital medium that consists of computer programming, code and technology. – In relation to Taoist philosophy, achieving health or longevity is often required through the practice of taijiquan by unifying the limited life of the body with the unlimited powerful cosmos, in a way. However, in the transformation of traditional philosophical concepts through the context of digital technology, do the aims and purposes of Taoist cosmological philosophy still remain the same? What is the implication of the transformation and interpretation of Taoist philosophy in the context of digital technology? Or what is new the relationship between the users and digital technology? How should we understand the concept of the digital Taoist body? By drawing from digital artefacts and Taoist philosophy, the digital Taoist body that unifies the body and the digital technology emphasizes the extension of the limited ability of human, including the limited span of his or her life, by integrating with the digital technology. The digital technology advances the ideas of integration. In this sense, I conclude that the Taoist body is digital technology, and the digital technology is Taoist body.

8.3.3 The Theoretical Framework This framework describes the details of visual forms and characteristics of each approach, and provides a potentially useful guideline for future digital practitioners from different fields. However, artists or designers should understand the moderators or influences of each approach along the data continuum from communication to aesthetics. It requires artists or designers to have some knowledge and ability with computer programming in order to operate the processes of each approach. The framework is considered as a broadly conceptual framework to guild the designing of data visualizations on three different levels (information, the yijing

8.3 Research Findings

177

aesthetic and Taoist data visualization), which emphasizes the exploration of the relationship between humanity and technology.

8.4 Limitations and Recommendations for Further Research There have been a number of limitations to this research project including programming software and discussion of aesthetic concepts.

8.4.1 Research Limitations The limitations in this research project have three following aspects: – Processing was selected for generating artefacts, but has certain limitations when it was used with Kinect. These relate to the ability of handle a very large amount of data, limitations of the computer operating systems, and the lack of suitable libraries to support more complex interactivity. From the observations made of the exhibition, the use of Processing software to drive Kinect presented certain instable features. For example, Kinect sometimes failed to capture the bodies of audience members. Restarting the program could solve this problem. Overall, however, it performed well during the exhibition and resulted in many positive comments from audience. However, the works could be improved more, including in terms of the programming and technology. – The discussion of this research only focuses on the Kantian sublime. However, aesthetics has been widely discussed and analyzed in the Western context and it is recognized that there are many different theories or ideas about aesthetics and data visualization. It was not possible to explore all aesthetic concepts due to the time and space restrictions of this project. – As the complex and non-systematic approach of Chinese aesthetics and Taoist philosophy has many different schools, it was not possible to explore all aspects of theories and thoughts. This research only provides a broad discussion about the key concepts of Taoist philosophy and aesthetics, as this research project aims to develop a Chinese aesthetic approach to data visualization in digital practice.

8.4.2 Recommendations for Further Research This research identifies a number of avenues for further research which are outlined below from cultural, philosophical, technological and creative perspectives:

178

8 Future Research

– Cultural promotion: The technological application of traditional Chinese principles could promote the understanding of Chinese art and culture for the West. The yijing aesthetic discussed in this research could enhance understanding of the Chinese aesthetic tradition particularly as it distinguishes the concept of yijing from I-Ching, which is the book of changes. Future research could include further exploration of the Chinese yijing aesthetic, which has not been addressed much in Western academic research. There is a broad research perspective of applying the yijing aesthetic in design and digital technology based art practice. Compared to the yijing aesthetic, the concept of I-Ching has been explored and discussed in the West since the beginning of last century. For example, artist and theorist Roy Ascott’s work Ten Wings adopted the I-Ching concept. This telematic artwork involves global consultation with the I-Ching. Ten participants were invited to represent “Ten Wings”, which correspond to ten parts of expressing seven commentaries (“Unveiling the mystery of I Ching,” n.d.; Ryan, 2015). Participants were asked to throw three coins by hand and then send information of the total sum of the coins to the terminal mailbox. Following responses, including a commentary and an image, were sent back to participants and this information was displayed in the Ars Electronica Centre (Linz, Austria) subsequently (Ascott, 2007). In this sense, the concept of yijing aesthetic could also have potential influence on technology-based art practice or development. Thus, exploring art practice with yijing enhances comprehension of traditional Chinese culture. – The theoretical framework: It can be applied to other fields that emphasize the harmonious emotional experience of Chinese aesthetics and the unification of the body and technology through Taoist philosophy, such as product design involving aesthetics and functionality. A product design initiative could apply this framework to design aesthetically focused products or functionally focused products. – Digital technology: As a digital art practice, the development of digital technology is considered as a key for further research. Instead of using gesture-based technology, further research may adopt other technologies to explore concepts of Taoist philosophy, such as wearable technology. Wearable technology can be traced back to the Enlightenment when eyeglasses were produced (Ryan, 2008). However, wearable technology was based on computing technology that was developed as a “body-based nature of wearable technology” (Ryan, 2008, p. 2). Mann (1998a) defines the concept of wearable technology as “a computer that is subsumed into the personal space of the user (wearer), controlled by the wearer, and has both operational and interactional constancy (i.e. is always on and always ready and accessible)” (p. 144). In this sense, it emphasizes the capability of “always ready” that generates “a new form of synergy between human and computer, characterized by long-term adaptation through constancy of user-interface” (Mann, 1998b, para. 1). According to Mann (1998b), wearable technology operates in three different modes: constancy that emphasizes interacting with users in a state of always ready; augmentation that enhances intellect or senses; or mediation that serves as encapsulation in the way of solitude and

8.4 Limitations and Recommendations for Further Research

179

privacy. By considering Taoist concepts of embodiment, these different modes of wearable technology could be an ideal digital technology for exploring Taoist philosophy in further developments of the Taoist digital body. – Modernization of Taoist philosophy: This research has discussed the modernization of traditional Chinese Taoism in the context of technology. Further research of this theme lies in exploring Taoist philosophy in new digital contexts. – Digital creative practice: As visualizing sleep EEG data in this research attempts to explore human consciousness and unconsciousness through digital technology, a further research direction may focus on the combination of art and science. Ascott (cited in Hope & Ryan, 2014, p. 198) suggests that digital art practice in the future “lies in the engendering of productive alliances between the arts and science”. Human consciousness, perception and creativity have been changed through digital technology, enabling humans to access innumerable states of conscious and unconscious awareness (Hope & Ryan, 2014). Gardiner and Gere (2016) argue that “the most important point is that, with the rise of digital or so called ‘new’ media, the means of experimentation, production, representation, distribution and consumption are all the same” (p.5). Although the technology used in digital art practice different from those in scientific works, it suggests that “… propose that all are different aspects of a culture in which the experimental is the dominant mode of engagement and production” (Gardiner & Gere, 2016, p.5) It emphasizes that the important role of experimentation in both digital art and science. Digital creative practice is necessary to adopt scientific methods, in particular in data visualization design. Therefore, further research on digital art practice necessitates reflecting on questions of immersion, interaction and transformation (Hope & Ryan, 2014, p. 198). – Real time interactivity and 3D visualization: A further research direction of EEG data visualization could include real time interactivity. Instead of using prerecorded and pre-selected EEG data, research projects could use technology to let participants’ interact directly with their own EEG data in real time. Although real time interaction is not a new approach to EEG data related visualization, as indicated by Paras Kaul’s Mind Garden (Kaul, 1997), which explores brain experience through EEG signals, Taoist data visualization with real time interactive EEG data could enhance aesthetic experience. Participants would experience their EEG data in real time, leading to multiple creative pathways and possible health benefits. Moreover, this book explores the concept of Taoist data visualization in 2D. However, a further research could extend it to 3D environments (Ramachandran & Varoquaux, 2011), which use 3D technologies to create 3D data visualization with a Taoist framework. For example, motion capture is an ideal device to create a 3D interactive environment with 3D representation of the body. Motion capture is a device that enables one to record live movement and to track key positional points, combining these points to generate a 3D representation (Menache, 2000; Moeslund & Granum, 2001). This system has been used mainly in medicine, games and feature films through three different systems: optical motion capture, electromagnetic trackers and electromechanical tracking systems (Menache, 2000). Research on 3D visualization would extend the Taoist

180

8 Future Research

digital body from 2D into 3D with more immersion and engagement, which could become a new application of Taoist philosophy in the context of digital technology. In addition, the technology of virtual reality providing virtual environments could enhance the sense of immersion by eliminating the distance between users and the visual representations. The use of virtual reality is a natural extension of the Taoist body and would assist the better realization of this body for future users and participants. For example, the iDome is a hemispherical dome creating a 3D digital environment (Champion, 2011; Bourke, 2009; Kuchelmeister, Shaw, McGinity, Del Favero, & Hardjono, 2009). It is often made with fibreglass or steel components that are able to form a smooth surface (Champion, 2011; Bourke, 2009). The iDome is an effective technology displaying panoramic imagery or movies with a high sense of immersion. However, it still could be developed more, especially in terms of interactivity, as Champion (2011) suggests that “…to fully utilize the dome I feel that standing and moving interaction without using a desk, or mouse or keyboard would be more interesting…” (p. 145). In this sense, an interactivity of body movement through Kinect or similar technologies would be useful to create an interactive and highly immersive 3D environment. Further research could focus on creating the environment of virtual reality that extends the Taoist body. – Exploring other types of data: The outcome of this research also can be applied to other types of data, such as medically related data or data of vehicular traffic in cities. It has significance to health related research from Taoist philosophy, which strives to unify the human body and nature emphasizing long life through natural harmony. The experience of immersive harmony through the digital Taoist body could evoke emotionally harmonious experience in the experience of medical data visualization, including visualizing the representation of blood pressure in a way of harmony that can be experienced with much less anxiety or nervousness. It will be especially important for children, as an embodied, harmonious and joyful visualization could eliminate or reduce the fear children may have during medical diagnosis.

8.5 Conclusion This chapter provides a summary of this research that aims to create a new aesthetic experience of data visualization by adopting traditional Chinese aesthetics and Taoist philosophy. The significance of this research reflects this new aesthetic experience, which was manifested through artefacts that harmoniously unified the body of users and the visualizing data through gesture-based technology. The outputs of this digital practice not only demonstrate the proposed framework but also are conceptualized as creative digital artefacts, which demonstrate the potential of transformation and interpretation of traditional Chinese aesthetics into the contemporary technological context. The key research findings of aesthetics, technology and the theoretical framework have been discussed in this chapter. It also indicates the limitations of this

8.5 Conclusion

181

research, which include the limitation of software, limitation of discussion of Kantian aesthetics and limitation of Taoist philosophy in the broad context. In particular, this chapter suggests directions for further research with cultural, philosophical, technological and art practice perspectives. It includes promoting cultural understanding of Chinese culture, modernization of Chinese traditional Taoist philosophy through digital technology, creative practice with immersion, interactivity and transformation, and real time interactive visualization and 3D virtual environments enhancing highly immersive representations of data. In addition, exploring other types of data would be also a further research direction. In conclusion, as a practice-led project, the praxis emphasizes the digital practice that involves data as creative medium for artistic expression. This research project has aimed to apply non-Western aesthetic concepts to the visual representation of scientific data, thereby demonstrating how Chinese philosophy and aesthetics can generate and enhance aesthetic experience and how such visualization can serve as digital practice rather than a mechanical means of communication and comprehension of traditional data visualization.

References Ascott, R. (2007). Ten wings. In E. A. Shanken (Ed.), Telematic embrace: Visionary theories of art, technology, and consciousness (pp. 184–185). Berkeley, CA: University of California Press. Bourke, P. (2009). iDome: Immersive gaming with the Unity3D game engine. Computer Games and Allied Technology, 9. Champion, E. (2011). Playing with the past. London: Springer. Gardiner, H., & Gere, C. (Eds.). (2016). Art practice in a digital culture. Routledge. Gray, C., & Malins, J. (2004). Visualizing research: A guide to the research process in art and design. Aldershot: Ashgate. Hope, C., & Ryan, J. (2014). Digital arts: An introduction to new media. New York, NY: Bloomsbury Publishing. Kaul, P. (1997). Mind garden [digital work]. Los Angeles Convention Center. Kuchelmeister, V., Shaw, J., McGinity, M., Del Favero, D., & Hardjono, A. (2009). Immersive mixed media augmented reality applications and technology. In Pacific-Rim Conference on Multimedia (pp. 1112–1118). Berlin, Heidelberg: Springer. Manovich, L. (2002). Data visualization as new abstraction and anti-sublime. Retrieved from http:// manovich.net/content/04-projects/039-data-visualisation-as-new-abstraction-and-anti-sublime/ 37_article_2002.pdf. Mann, S. (1998a). Headmounted wireless video: Computer-supported collaboration for photojournalism and everyday use. Communications Magazine, IEEE, 36(6), 144–151. Mann, S. (1998b). Wearable computing as means for personal empowerment. Paper presented at the 1998 International Conference on Wearable Computing ICWC-98, Fairfax VA. Menache, A. (2000). Understanding motion capture for computer animation and video games. San Francisco, CA: Academic Press. Moeslund, T. B., & Granum, E. (2001). A survey of computer vision-based human motion capture. Computer Vision and Image Understanding, 81(3), 231–268. Paul, C. (2003). Digital art. London: Thames & Hudson. Pohl, K. (1996). Chinese aesthetics and Kant. Twenty-first Century, 8(36), 84–89.

182

8 Future Research

Ramachandran, P., & Varoquaux, G. (2011). Mayavi: 3D visualization of scientific data. Computing in Science & Engineering, 13(2), 40–51. Ryan, J. C. (2015). Ecodigital art: Democratization, globalization, and interdisciplinarity. The International Journal of Environmental Sustainability. Ryan, S. E. (2008). What is wearable technology art. Intelligent Agent http://www.intelligentagent. com, 7–12. Unveiling the mystery of I Ching. (n.d.). Retrieved from http://www.iching123.com/brief_a_text. htm.