Language, Music and Gesture: Informational Crossroads: LMGIC 2021 9811637415, 9789811637414

Citation preview

Tatiana Chernigovskaya Polina Eismont Tatiana Petrova   Editors

Language, Music and Gesture: Informational Crossroads LMGIC 2021

Language, Music and Gesture: Informational Crossroads

Tatiana Chernigovskaya · Polina Eismont · Tatiana Petrova Editors

Language, Music and Gesture: Informational Crossroads LMGIC 2021

Editors Tatiana Chernigovskaya St Petersburg University Saint-Petersburg, Russia

Polina Eismont St Petersburg University Saint-Petersburg, Russia

Tatiana Petrova St Petersburg University Saint-Petersburg, Russia

ISBN 978-981-16-3741-4 ISBN 978-981-16-3742-1 (eBook) https://doi.org/10.1007/978-981-16-3742-1 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Organization

Chair Tatiana Chernigovskaya, St Petersburg University, Russia

Program Committee Svetlana Lavrova, Vaganova ballet academy, Russia Olga Manulkina, St Petersburg University, Russia Tatiana Petrova, St Petersburg University, Russia Elena Riekhakaynen, St Petersburg University, Russia Polina Eismont, St Petersburg University, Russia

Scientific Committee Apollinariya Avrutina, St Petersburg University, Russia Anthony Brandt, Rice University, USA Ian Cross, Cambridge University, UK Ekaterina Davidenkova-Khmara, St Petersburg State Conservatory, Russia Polina Eismont, St Petersburg University, Russia Elena Erofeeva, Perm National University, Russia Elena Falaleeva, St Petersburg State Conservatory, Herzen State Pedagogical University of Russia Andrey Filippovich, Moscow Polytechnic University, Russia Iiro Jääskeläinen, Aalto University, Finland Daniel Junqueira Tarquinio, Federal University of Brasilia, Brazil Jonah Katz, West Virginia University, USA Elena Khodorkovskaya, St Petersburg University, Russia v

vi

Organization

Andrey Kibrik, Institute of Linguistics of Russian Academy of Science, Moscow State University, Russia Anastasia Kolmogorova, Siberian Federal University, Russia Mikhail Kopotev, University of Helsinki, Finland Artemy Kotov, National Research Center “Kurchatov Institute”, Russia Svetlana Lavrova, Vaganova Ballet academy, Russia Mark Leikin, University of Haifa, Israel Olga Manulkina, St Petersburg University, Russia Elizabeth Margulis, Princeton University, USA Jakub Matyja, SWPS University, Poland Samuel Mehr, Harvard University, USA Andriy Myachykov, Northumbria University, UK, Higher School of Economics, Russia Isabelle Peretz, Montreal University, Canada Tatiana Petrova, St Petersburg University, Russia Elena Riekhakaynen, St Petersburg University, Russia Makiko Sadakata, University of Amsterdam, Netherlands Yury Shtyrov, Aarhus University, Denmark, Higher School of Economics, Russia Renee Timmers, University of Sheffield, UK Remi van Trijp, Sony CSL, France

Preface

The title of the conference, Language—Music—Gesture: Informational Crossroads, speaks for itself. This scientific forum is designed to bring together specialists working across different non-computational fields of modern cognitive science on one interdisciplinary platform. This conference features research investigating the interaction between language and music, as well as their acquisition, the interaction between verbal and nonverbal means of communication, body language as a way of transmitting information, and linguistic description of different semiotic systems. Brain science unites all these areas, combining different fields of the humanities and natural sciences. Recent advances in cognitive science have proven that brain activity is not limited to computational procedures, and its highest manifestations in both art and science are not algorithmic. Paradoxically for the natural sciences, the brain can only be understood by combining the tools of neuroscience, philosophy, linguistics, and the arts. The complexity of the interweaving of these spheres of human life transpired in the process of developing the conference logo. Finding and creating a logo is no easy feat. It began with creating a so-called mind map with the central words: music, word, and gesture. From that stemmed a huge variety of options for how interesting, non-standard and at the same time simple music, word, and gesture can look: Word: a letter, a hieroglyph, a book, a cloud of dialogue, an icon of a talking person, a microphone, a radio, a metaphorical pedestrian crossing between faces. Music: a note, a sound wave, various musical instruments, conductor’s baton, headphones, again the microphone and radio. Gesture: a dance, a hand movement, ballet dancer’s leg movement, “great!” gestures, warm touch signs, and an air kiss message. At this preliminary creative stage of developing a logo, the designers created many sketches for the figurative expression of each of the components, but the most difficult stage was the next one: combining these three images into one so that the new symbol was not complicated graphically, original but also understandable to the viewer. We ended up with 27 variants of the logo. When choosing the best one, we relied on the main goal of the conference: to show and discuss the existing “information intersections” between different semiotic systems. vii

viii

Preface

The authors of the articles presented in this volume solve the same problem that faced the developers of the logo: they all search for and discover the connections between the images of music, gesture, and word, penetrating into the very essence of the phenomena. The volume opens with the article by Tatiana Chernigovskaya, Baroque Brain: Science and Humanities in Interaction. The paper discusses the current neurophysiological approach to higher cognitive functions and tries to show its fundamental errors. It illustrates how important forethought and a philosophical foundation for interdisciplinary research are regarding the processes of brain and mind mechanisms. Unification of the efforts of various scientific domains provides qualitatively new knowledge. Brain and mind should be studied by interconnections of natural sciences, arts, and humanities. Cognitive sciences will never experience a sharp paradigmatic increase without looking at the problem from a different perspective— in the context of the products of the human genius. Human mind is not a Turing machine, and its principles are not based on the stimulus–reaction scheme. Rather it demonstrates the principles of Barocco: it extracts faces and objects, revealing specific and unusual features, it is not linear and stable. The brain is not just processing information—rather it creates it. The articles in the next two sections cover the issues raised by analyzing the interaction of language, music, and gesture, both from the point of view of theoretical justifications and from the point of view of their reflection in stage art forms. In her article Theory of Sign and Cognition: Literary Language and the Language of Music, Nina Scherbak compares the concepts of literary and musical languages. A rather abstract concept, the latter also consists of signs, although somewhat different from the language signs in the generally accepted terms. Focusing on the concept of Vygotsky’s inner speech, the author formulates the general principles of symbolization that operate in both literary and musical language. Giula Baijan Shamilli continues the comparative analysis of language and musical structures in the article Hierarchical and Non-Hierarchical Structures in the Language and Music, focusing on the formal analysis of music theory presented in the works of Lerdahl–Jackendoff. Expanding their proposed hierarchical structure, the author shows through the analysis of the maqam and its comparison with the Arabic syntax that a comprehensive analysis of the musical text structure requires to take into account the variability of the subject–predicate structure, which includes both hierarchical and non-hierarchical sequences. The first section concludes with Anastasia Gundorina’s article Metaphor as a structural principle of modern musical notation, which examines modern approaches to musical notation as one of the means of information transmission. Interpreting notation as a universal way of writing that reflects the peculiarities of human thinking, the author suggests analyzing the principles of its organization from the point of view of the metaphor theory. The articles of the second section, Language-Music-Gesture on Stage, analyze the interaction of verbal, musical, and choreographic texts in the works of modern composers. Olga Manulkina contributed an article Reading Opera: the Verbal vs the Non-Verbal in Opera and Music Theatre in the Late 20th and Early 21st Centuries. Analyzing the works by Steve Reich, Vladimir Rannev, and opera by Robert Wilson

Preface

ix

and Philip Glass, she shows that modern musical theater is a space for interdisciplinary creative experiments, in which it is difficult to distinguish different types and forms of information transmission and emotional impact. The author focuses on the methodological autonomy of modern musical theater research and offers tools for their comprehensive analysis. Svetlana Lavrova, in her article Music as “impossible experience”: multisensory perception in the composer’s practice of the 21st century, investigates the specifics of the listener’s perception of modern musical art. Based on her own analysis of the works of the Austrian composer Peter Ablinger, the author proposes to interpret perception as an “experience of the impossible,” when the listener needs to go beyond the usual and learn to perceive various noises as a piece of music. The article Kenneth King’s “Metapoiesis” as a form of intertext by Ekaterina Levina continues the theme of the multiplicity of modern musical representation and analyzes the work of the American choreographer through the concept of intertextuality and reflecting on the idea of “world as text.” The third section, Verbalization of Music and Gesture, deals with the description of music and gesture in natural and professional communication. The section opens with the article by Polina Eismont, Non-verbal behavior and its role in narrative production, which investigates the perception of gestures and other types of nonverbal communication by Russian-native children of different ages. The analysis of spontaneous narratives generated by children revealed that the successful interpretation and choice of means of describing the elements of non-verbal communication depends on their plot significance, expressed meaning, and the presence of a verbal context. The article by Leonid Pakhomov, Elena Erofeeva, and Tatiana Petrova, titled Metaphors in Vocal Training Discourse, explores the professional speech of vocal art teachers. Analyzing the metaphors used in teaching singing and modeling their structure, the authors identify the most typical frames and slots of the source and target domains (source and target regions) and the inter- and intraframe connections of the slots inside the metaphor. Elena Chistova’s article Non-verbal Semiotics in Dance Narrative and Simultaneous Interpreting: Cognitive Intersections is devoted to the research into types of cognitive intersections between dance and non-verbal behavior of simultaneous interpreters in the process of their work. Using the method of cognitive event analysis developed within the framework of cognitive ecology theory, the author shows that these activities converge based on the kinetic and rhythmic representation of information, including the psychoemotional state of the producer. The final article of the section, Exploring Gestures and Body Language in Professional Musicians during the Self-reflection Process on Technical Movement, by the Italian researcher Annamaria Minafra investigates the professional musician’s ability to recognize and describe their own unconscious non-verbal behavior during a performance. By analyzing the narratives of five professional musicians describing their movements during their performance in different conditions (from neutral to directed, when the participants were asked to focus on body movements), the author shows the process of developing body self-reflection and confirms the kinesthetic basis of thinking. The final two sections of the volume are devoted to the analysis of prosodics in language and music. The article Prosodic and musical prominence in Portuguese

x

Preface

folk verse by João Veloso is devoted to the comparison of speech and music metrics. Based on the analysis of Portuguese folk songs, the author shows that only the primary stress correlates with the strong musical beat, and the secondary stress is not reflected in the musical rhythm. The analysis of the folk music continues with the article by Ilia Saitanov, Evgenia Khazdan, and Sofia Vlasova titles Klezmer Music Modes: Two Approaches to Corpus Markup. Considering the various principles of organization and annotation of the klezmer music corpus, the authors emphasize the mode and present an example of creating such a corpus. The article by Vera Evdokimova, A comparative analysis of the spectral characteristics in dialogues, examines the changes in the speech spectral characteristics in twins. The author comes to the conclusion that the adaptation to the interlocutor is an individual trait and depends more on the speaker herself than on the degree of familiarity with the interlocutor. The dialogue analysis continues in the last section of this volume, Emotions in Linguistics and AI-Communication Systems, which discusses the perception of prosodic characteristics and expression of emotions when communicating with computer agents. The article by Artemiy Kotov, Nikita Arinkin, Alexander Filatov, Liudmila Zaidelman, and Anna Zinina, Emotional Balance Model for F-2 Companion Robot presents a computer model of emotional dynamics designed to simulate natural emotional reactions. This model provides for the possibility of simultaneous and sequential expression of several emotional states, which allows the robot to approach the natural human reactions as close as now possible. The article The Speech Corpus for Studying Phonetic Properties of Irony by Pavel Skrelin, Uliana Kochetkova, Vera Evdokimova, and Daria Novoselova investigates the perception of such a complex linguistic phenomenon as irony. The authors attempt a complete description of acoustic signals allowing us to distinguish an ironic statement from a non-ironic one, which is extremely important especially in the situation of communicating with computer agents. The article presents a phonetic corpus of ironic utterances in Russian and describes experiments aimed at studying the perception of individual elements of this corpus. The concluding article of the volume by Anastassia Kolmogorova, Alexandre Kalinin, and Anna Malikova, Part-Whole Relation in Emotional Investigation of Verbal and Musical Text: Two Ways of Emotional Dataset Assessment, is a study of recognizing emotions in verbal and musical texts. According to the results of the experiment on sentiment assessment, the authors conclude that for a verbal text, the difference between the emotional perception of the whole text and the emotional evaluation of individual fragments is significant, while for a musical text, the emotional perception is quite stable throughout the text. The editors are grateful to the Faculty of Liberal Arts and Sciences of St Petersburg University and its leadership, especially to Dean Alexei Kudrin, and to the leading organizer of SPbU Events & Partnerships Department Nadezhda Protsenko, to Vaganova Ballet Academy and its leadership, especially to Rector Nikolay Tsiskaridze and Vice-Rector Svetlana Lavrova, to the Institute for Cognitive Studies of St Petersburg University, especially to Alena Konina for the enormous work that they did to organize the conference. We are grateful to Saint Petersburg Academy of Talents and its leadership, especially to Inga Dorokhova for the development of

Preface

xi

the conference logo. We are also grateful to Springer Nature, Rebecca Zhu, Carolyn Zhang, and Saravanan Murugan for accepting this book and their help in the publication process. For any remaining errors and shortcomings, the editors alone carry the responsibility. Saint-Petersburg, Russia

Tatiana Chernigovskaya Polina Eismont Tatiana Petrova

Contents

Baroque Brain: Science and Humanities in Interaction . . . . . . . . . . . . . . . . Tatiana Chernigovskaya

1

Language–Music–Gesture as Semiotic Systems Theory of Sign and Cognition: Literary Language and the Language of Music . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nina F. Shcherbak

13

Hierarchical and Non-hierarchical Structures in the Language and Music . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Giula Shamilli

29

Metaphor as a Structural Principle of Modern Musical Notation . . . . . . . Anastasia Gundorina

39

Language–Music–Gesture Onstage Reading Opera: The Verbal versus the Non-verbal in Opera and Music Theatre in the Late 20th and Early 21st Centuries . . . . . . . . . . Olga Manulkina

55

Music as “Impossible Experience”: Multisensory Perception in the Composer’s Practice of the Twenty-First Century . . . . . . . . . . . . . . . Svetlana V. Lavrova

71

Kenneth King’s Metapoiesis as a Form of Intertext . . . . . . . . . . . . . . . . . . . Ekaterina V. Levina

81

Verbalization of Music and Gesture Non-verbal Behavior and Its Role in Narrative Production . . . . . . . . . . . . Polina Eismont

91

Metaphors in Vocal Training Discourse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Leonid Pakhomov, Elena Erofeeva, and Tatiana Petrova xiii

xiv

Contents

Non-verbal Semiotics in Dance Narrative and Simultaneous Interpreting: Cognitive Intersections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Elena V. Chistova Exploring Gestures and Body Language in Professional Musicians During the Self-reflection Process on Technical Movement . . . . . . . . . . . . . 139 Annamaria Minafra The Beat of Language Prosodic and Musical Prominence in Portuguese Folk Verse . . . . . . . . . . . 159 João Veloso Klezmer Music Modes: Two Approaches to Corpus Markup . . . . . . . . . . . 167 Ilya Saitanov, Evgenia Khazdan, and Sofia Vlasova A Comparative Analysis of the Spectral Characteristics in Dialogues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Vera Evdokimova Emotions in Linguistics and AI-Communication Systems Emotional Balance Model for F-2 Companion Robot . . . . . . . . . . . . . . . . . . 193 Artemiy Kotov, Nikita Arinkin, Alexander Filatov, Liudmila Zaidelman, and Anna Zinina The Speech Corpus for Studying Phonetic Properties of Irony . . . . . . . . . 203 Uliana Kochetkova, Pavel Skrelin, Vera Evdokimova, and Daria Novoselova Part-Whole Relation in Emotional Investigation of Verbal and Musical Text: Two Ways of Emotional Dataset Assessment . . . . . . . . 215 Anastasia Kolmogorova, Alexander Kalinin, and Alina Malikova

Editors and Contributors

About the Editors Tatiana Chernigovskaya is Professor of St Petersburg University, Head of the Department of the Problems of Convergence in Natural Sciences and Humanities, Director of Institute for Cognitive Studies, Doctor of Science in Human Physiology and in Theory of Language, Member of Russian Academy of Education, Member of the Norwegian Academy of Science and Letters, Honorary member of the Semiotic Society of Finland. Invited lecturer in many European and North American Universities. The author of more than 350 publications. Gold medal of Russian Academy of Sciences for the propagation of scientific knowledge winner (2017). Polina Eismont is an Associate Professor at the Department of General Linguistics, St Petersburg University. Her research interests include Language Acquisition and Psycholinguistics, Text Linguistics, Cognitive Linguistics, Event Structure, Music Semantics, Syntax of Nulls. She obtained her Ph.D. from St Petersburg University in 2008. She is the author of more than 40 papers in domestic and international journals and the co-editor of two CCIS volumes “Language, Music, and Computing” (Springer Verlag, 2015, 2019). Tatiana Petrova is an Associate Professor at the Department of Theory and Methodology for Teaching Arts and Humanities, and the leading researcher of the Institute for Cognitive Studies, St Petersburg University. Her research interests include Psycholinguistics and Neurolinguistics, Text Processing, Second Language Acquisition, Context Predictability, Reading and Visual Recognition. She obtained her Ph.D. from St Petersburg University in 2000. She is the author of more than 80 papers in domestic and international journals and the co-author of two online courses on Coursera. In the past 10 years, she has been coordinating the work of the St. Petersburg Seminar on cognitive Research. She also organizes St. Petersburg Winter Workshops on Experimental Studies of Speech and Language (2011, 2014, 2015, 2018, 2019), serving as a member of Program Committee.

xv

xvi

Editors and Contributors

Contributors Nikita Arinkin National Research Center, Kurchatov Institute, Moscow, Russia; Russian State University for the Humanities, Moscow, Russia Tatiana Chernigovskaya St Petersburg University, Saint-Petersburg, Russia Elena V. Chistova Siberian Federal University, Krasnoyarsk, Russia Polina Eismont St Petersburg University, Saint-Petersburg, Russia Elena Erofeeva Perm State University, Perm, Russia Vera Evdokimova St Petersburg University, Saint-Petersburg, Russia Alexander Filatov Russian State University for the Humanities, Moscow, Russia Anastasia Gundorina Gnessins Russian Academy of Music, Moscow, Russia Alexander Kalinin Siberian Federal University, Krasnoyarsk, Russia Evgenia Khazdan Independent researcher, Saint-Petersburg, Russia Uliana Kochetkova St Petersburg University, Saint-Petersburg, Russia Anastasia Kolmogorova Siberian Federal University, Krasnoyarsk, Russia Artemiy Kotov National Research Center, Kurchatov Institute, Moscow, Russia; Russian State University for the Humanities, Moscow, Russia; Moscow State Linguistic University, Moscow, Russia Svetlana V. Lavrova Vaganova Ballet Academy, Saint-Petersburg, Russia Ekaterina V. Levina Vaganova Ballet Academy, Saint-Petersburg, Russia Alina Malikova Siberian Federal University, Krasnoyarsk, Russia Olga Manulkina St Petersburg University, Saint-Petersburg, Russia Annamaria Minafra Independent Researcher, Florence, Italy Daria Novoselova St Petersburg University, Saint-Petersburg, Russia Leonid Pakhomov Perm State University, Perm, Russia Tatiana Petrova St Petersburg University, Saint-Petersburg, Russia Ilya Saitanov Higher School of Economics (HSE), Moscow, Russia Giula Shamilli State Institute for Art Studies, Moscow, Russia Nina F. Shcherbak St Petersburg University, Saint-Petersburg, Russia Pavel Skrelin St Petersburg University, Saint-Petersburg, Russia João Veloso University of Porto, Porto, Portugal

Editors and Contributors

xvii

Sofia Vlasova Independent researcher, Moscow, Russia Liudmila Zaidelman National Research Center, Kurchatov Institute, Moscow, Russia; Russian State University for the Humanities, Moscow, Russia Anna Zinina National Research Center, Kurchatov Institute, Moscow, Russia; Russian State University for the Humanities, Moscow, Russia; Moscow State Linguistic University, Moscow, Russia

Baroque Brain: Science and Humanities in Interaction Tatiana Chernigovskaya

Abstract The paper discusses current neurophysiological approach to higher cognitive functions and shows its fundamental errors. It illustrates how important philosophical forethought is for interdisciplinary research of brain processes and mind mechanisms. Joint efforts of various sciences provide qualitatively new knowledge. Brain and mind should be studied through interconnecting natural sciences, arts, and humanities. Cognitive sciences will never reach radically new results without looking at the problem from a different perspective—in the context of various products of human genius. Human mind is not a Turing’s machine, and its principles are not based on stimulus-reaction scheme. Rather, it demonstrates the principles of the Baroque: it highlights faces and objects, revealing specific and unusual features, it is not linear and stable. The brain is not just processing information—it creates information. Keywords Philosophy of Mind · Brain · Arts · Perception · Multi-disciplinarity

1 Introduction For decades, science has been looking at the brain as at a very sophisticated device which receives information from the world, processes it, and reacts like a biomachine. Facts are collected “from the bottom” with the hope that the building consisting of the existent set of “atoms”—neurons, their ensembles, zones with determined functions, and now built of individual neural circuities will erect itself. The failure of this approach is becoming more and more clear, but habits are stronger than reason. Of course, everything depends on the goal: if the goal is to understand who we are, and what and how our brain does exactly, then nothing is going to work. If we want to know how to make an artificial intelligence tool that encourages our laziness, is a different story.

T. Chernigovskaya (B) St Petersburg University, Universitetskaya nab., 7–9, Saint-Petersburg, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 T. Chernigovskaya et al. (eds.), Language, Music and Gesture: Informational Crossroads, https://doi.org/10.1007/978-981-16-3742-1_1

1

2

T. Chernigovskaya

Let us look at the brain through another lens—through art. What and how does the brain see, how has it create masterpieces of our civilization? How does it hear? How does it organize the world? Won’t it show us what it can do and how it operates? In other words, let’s turn the scheme familiar to science around… We have to recall what had been known to the ancients: we are dealing with human brain whose higher functions cannot be deduced by simple occumulation of data of the brain functions of relatively simpler organized animals. Moreover, it is now clear that cerebration is not limited to computational processes, and its ultimate manifestations, both in art and in science, are not algorithmic.

2 Neurovisualization of Qualia : Microscope or Telescope? Development of theoretical aspects of the problem of “the consciousness and the brain” is accompanied by scientific controversy, often demonstrating unbending stances. Theory of cognitome is being developed as an “architecture of mind”, a model designed to explain the emergence of the ideal from the material based on the dynamics of neural activity. K.V. Anokhin defines cognitome as a set of individual experience, including subjective reality (Qualia), and suggests reading it as a genome [1]. This raises the fundamental question of how exactly these models can be correlated with specific materialization of the higher functions (memory, attention, language, different modes of thinking, including metaphorical/analogous). In trying to understand the problem of consciousness, reflection and unconscious mechanisms in solving high-order tasks, cognitive sciences hope for increasingly subtle experimental data that will allow us to describe the brain codes. But is this the right way? What question will be answered by our knowledge that brain demonstrates certain patterns of brain activity when solving a certain problem? Is the question itself posed correctly? Moreover, is it posed at all? On top of that, since the chance of carrying out experimental studies on human brain is limited for ethical reasons, especially at micro-levels, the main experiments are performed on animals, and the results are extrapolated to the psychophysiological functions of a human being, based on the presupposition that basically physical and biological laws are unified, only in a human being, when it comes to macro-levels, everything is much more complicated. Neurovisualization studies of human brain provide a huge amount of ever-more subtle data, but this does not at all remove the main question. Moreover, “nature looks at itself with countless different eyes, either of which focuses on some other world... each world is locked by its own horizon, and what is visible is the only one visible” [2, p. 268], from which it follows that in our efforts to judge animals according to the laws of human world we are following the wrong scent and vice versa. Paradoxical as it may sound for natural science, brain can only be understood by combining neuroscience, philosophy, linguistics, and arts. They will show us what to look for... Owing to consciousness “… man acquires ... the ability to invent something in his imagination and in this way to build a completely new world” [3, p. 224]. This is intrinsic exclusively to consciousness intentional

Baroque Brain: Science and Humanities in Interaction

3

meaning-forming function, which cannot be reduced to any form of adaptive brain activity. Only in humans, thinking and being represent a continuum, and the text in its broad meaning is considered not as a composition of its parts, but as a partlesssign [4]. In this context, the question “Do animals think?” is not idle, and once again leads us into the trapfall of words... What does “think” mean? Ability to draw certain conclusions about the surrounding world with the help of higher (for them) psychophysiological mechanisms? Is it cogitation in its real sense? Do animals have some kind of nomination? if so, then it means that in world cognition they rely on certain universals, on “innate ideas”, so to say... But can we find it out? Of course not. And if not, then we cannot talk about the worlds of other beings, and extrapolation in both directions is a mistake. In order to discuss this, we must at least agree on the terms. Mamardashvili, discussing Descartes, calls this an outpouring of terms from one level to another. A more dangerous question also comes up: “Do all people think, and do they always do it?” If we are talking about the capability to make relatively simple assumptions and conclusions from apparent facts, then, of course, yes—all of us, the inhabitants of this planet, possess mentation. Or mind? Intellect? Finally, consciousness? Precisely these terms are used in the relevant discourse. But we are discussing the highest materializations of human abilities, the level that the best of us have reached in the process of biological and cultural evolution. Mamardashvili defines it as a hellacious work of thought: everything we deal with happens at the human ceiling; thought is available to a person at the limit of exertion of all of his abilities [5]. He keenly points out that in the portrait of Descartes the philosopher holds a book on which it is written: “Mundus est Fabula”–“World is a fairy tale.” If we, building our knowledge about the world, will rely only on the results (or products) of the nervous system, we will not be able to formulate any general laws. What is more, outside the topos of thought we will not notice them at all. According to Descartes, any consciousness, is self-consciousness. How can we study it using the modern neuroscience methods? This requires separate efforts and the work of philosophers. Studying the brain and its activity, one cannot neglect the culturally-specific features of cognitive processes, reducing the entire diversity to a kind of statistical average leads us to fundamental errors [6]. The article by Asmolov et al. accurately formulates an experimentally vicious reduction leading to the effect of “"simple living"– simplification of life—this is what is characteristic of modern natural science. This is precisely what the hypotheses of the identity of the physical and the mental is, on which explicitly or implicitly the computer metaphor is based, as well as metaphors that endow the brain with various energy, chemical and, especially, mental attributes: energy brain, heterochemical brain, wakeful brain, emotional brain, motivated brain, metaphorical brain, and finally the cognitive brain” [7]. Consideration of the ability to productively study higher cognitive processes, including the most complex of them—the consciousness, inevitably leads us to the same old story—to the problem of psychophysiological dualism as irreducibility of the physical and the mental. Understanding the psychophysical problem by itself is

4

T. Chernigovskaya

significantly different even within individual sciences, not to mention the difference in the discourses of different areas of knowledge [8–11]. Essentially all experimental neuroscience implies, though feeling a little awkward, following Crick and Koch [12], that if we could learn all the properties of neurons and interactions between them, we could have explained what spirit is. But, for example, J. Searle clearly states that consciousness is real and irreducible, and that the normal scheme of scientific reduction based on the model of physics presupposes a difference between illusion and reality, but such distinctions cannot be made for consciousness, since illusion for it represents reality [13]. V.A. Lektorsky formulated the paradox that brain exists in the world, and the world is in the brain [8]. The outer world is built from the inside [14], there is nothing outside of us that at the same time would not be in us, as Goethe said... There is no serious reason to deny that inexplicable psychophysically subjective states and all mental phenomena—conscious and unconscious—are generated by neural networks that have an obvious recipient who interprets their “texts” or at least simply reads them [11, 15, 16]. Who is “he”? The brain? Yes, the human brain. It is rather sad that all this is completely ignored in experimental research that treats it as irrelevant to the “case”. Man lives in a constantly changing world not only in the physical and social sense, but also in the multidimensional and dynamic contexts of personality, its states, specific interactions, goals and cultural environment. How does the brain manage to grasp ideas, process different levels of sensory and mental information and take decisions amidst such multifactorial instability? Moreover, by no means the whole volume of processing is provided by the mechanisms of consciousness. Solution of similar or even the same problems can happen in different ways at different times, and as a rule it is very difficult to reveal the transparency of algorithms or procedures of a different nature, and the danger of a biased interpretation is inevitable. How can one ignore such special, purely human and powerful phenomena like the insight? Human art can show us how this giant neural network functions, how it creates the world. In a way, such a path is analogous to archeology and cultural anthropology which judge the brain by the tools made by “it”. Following the old path accepted in neurophysiology, we go on making critical mistakes, and we will certainly not reach the goal.

3 Semiotics in the Brain Every creature has its own Umwelt (world), as Uexküll [2] wrote one hundred years ago, and the “texts” for its decoding are fundamentally different. Mamardashvili discusses it in this way: “After all, evolution could have followed completely different paths, and there exists the possibility for some completely different even unimaginable for us arrangement of nervous system, and a different basis of life, other than which is implemented on a protein basis, is possible. And then it would correspond to a completely different physical configuration, the visual material of which could

Baroque Brain: Science and Humanities in Interaction

5

be used to model the phenomenon of cognition. And the latter, since it exists, is the same in any possible consciousness—in the consciousness of a Martian, a worm, a creature with faceted eyes, in whom the change of visual states in time apparently occurs in a completely different unimaginable for us way” [5, p. 105]. The concept of brain as a “bee’s faceted eye” (metaphorically speaking) or as a supercomputer assembled using identical “pixels” and occupying the entire house will not in the slightest bring us closer to solving the most important question: How is the material being formed from the ideal and vice versa? It is impossible to escape the subject—it is the main one. Human brain is about a human being. Not about a dolphin, or a chimpanzee, or a crow. It cannot be calculated, much less understood by logically irreproachable complication of information about the brain of more “simple” species. And certainly, it cannot be understood on the basis of such extrapolations. Gathering information about the units and algorithms that, as we see it, underlie the brain mechanisms of mental activity of other beings, will not lead to a breakthrough in understanding the higher mental functions of a human being, the conscious and unconscious processes. The brain is not the sum of billions of neurons and their connections; there also is the individual experience, which has formed this tool—our brain—and tuned it. This is why you need to study it in a way different from studying the brain of other inhabitants of the Earth. Yes, the components are similar or even identical, but the networks they generate have not a quantitative, but a qualitative difference. Perception is active extraction of knowledge and construction of the world. As Lotman puts it, a human being has a symbolic consciousness, language, the ability to create, a semiosphere of a special character. Only man is the creator of his own worlds, music, mathematics, myths, dance, visual arts. But only a developed man... One cannot help but recall Mamardashvili, who spoke about millions of books filled with non-thinking...

4 Why Arts? Human brain creates meanings. Yuri Lotman writes that the need for art is obvious. It enables a person to “walk” the path that has never been taken, to experience what has not been experienced in real world, it gives the experience of what has not happened. In other words, art is a second life [17]. And, of course, art speaks its own language, it is not a copy of life, it is a separate world, or even a concept of world. At the extreme, there are experiments with abstract and conceptual art. Can an artificial brain, which creates paintings or music, form a concept of the world? Art is interpretation which is always different and belongs to its author. Can we talk about an interpretation coming from a program? Probably, we can, but it has nothing to do with human world. Lotman compares text to a grain containing the program of future development. The internal undeterminedness of its structure contains, in his opinion, a reserve for its dynamics. Such understanding of “concatenation with reality” clearly tells us: man lives in a qualitatively special dynamic world dependent on different types of contexts [18]. It is amazing how brain deals with an ever-changing world... Duality, if

6

T. Chernigovskaya

not polysemanticity of text (in the broad sense of the word), indicates: the text knows more than the author, and Lotman calls it self-growing logos [19]. The change in the meanings of the same object depending on the changing context (as in the wellknown psychological experiment with faces/vases) depends on the judgement, and the same exegetist may have a lot of them. There is enough space in the brain for everyone... And if the brain is in a pathological state, it can manifest itself in the “multiple personality” syndrome. One cannot help but ask the question: how can one fail to take all this into account when studying consciousness?

5 Why the Baroque? “The baroque” is an irregular pearl, it is uneven, unique and functions as a symbol and sign of the Baroque. The word “baroque” supposedly comes from the Portuguese perolabarroca– a weirdly shaped pearl or seashell; or from the Latin baroco—a mnemonic name for one of the types of syllogism in scholastic logic (it is noteworthy that similar Latin words barlocco or brillocco were also used in a similar meaning—a pearl of an unusual shape that does not have an axis of symmetry). Indeed, visual arts and architecture of this period are characterized by very sophisticated forms, by complexity, and dynamics. Alféred Schnittke wrote: “To form a pearl in a shell lying on the bottom of the ocean, a grain of sand is needed—something ’nonregular’, foreign. Just like in art, where the truly great is often born "against the rules". Now I understand that a "mistake" or handling a rule on the verge of risk is the area where vivific elements of art arise and develop.” A good brain, I think, is arranged by analogy with baroque. It, much like Caravaggio, snatches out faces and objects from the background. The brain highlights the unusual, non-trivial features of the world, fights against the darkness... When we look at baroque canvases, we see how we see any picture, registering the image using the modern eye-tracking technique, which records how our eye (more precisely, our attention) moves through the picture [20, 21]. In contrast to Renaissance in art and in our mechanistic ideas about the brain of the past (and partly of the present), we are now beginning to see the features of brain processes that are characteristic of the Baroque [22]: the Renaissance: static/correct planar sharp rational linear

the Baroque: becoming/moving in-depth/multidimensional blurred/unclear irrational holographic/vibrating

The Baroque turned art upside down, just as knowledge about the brain overturns science and not only natural science, but also anthropology and linguistics. The brain, like a baroque artist (for example, in Velazquez’s mysterious painting “Menina”),

Baroque Brain: Science and Humanities in Interaction

7

loves to play a mirror game: it even can, as it is suggested to the spectator, look from the inside of the mirror or even appear only in the mirrors without being “in the picture”, and gaze at us from there. Isn’t this exactly what happens in the case of a failure in the brain processes leading to psychotic disorders? In the baroque there are no voids, everything is filled up and gives a different impression depending on the breadth and depth of the gaze, and in this sense, baroque is always movement and transiency. A very contemporary style! Perhaps baroque is the future. Subtle researcher of this style, V. Markova, claims that the nature of baroque is contradictory and polyvalent. So is the brain, which, by analogy with speech genres in linguistics—for the sophisticated and the inexperienced—can use what she calls the problem of two manners: Connoisseurs and amateurs require different languages, different cognitive associations and techniques. Markova cites Pietro Liberi: “He painted for connoisseurs, as he spoke about it openly, with a light and unconstrained brush, retaining incompleteness; and for the ignorant—with the most diligent brush, which made it possible to see every part of the picture carefully painted, and the hair was painted in such a way that every hair could be counted... It is not the excessive level of detail, which forms the excellence of an artist: two titans of pictural art— Tintoretto and Giordano, the less they resorted to it, the more success they gained with connoisseurs” [23, 24]. We are talking about opposing the two most important concepts of art—improvisation and imitation. The further the viewer is from genuine penetration into the essence of the work of art, the sooner he is satisfied with the verisimilitude, the accuracy of the transmitted details. It is imitation that is a metaphor for those brain functions that are taken as a model for creating artificial brain. In the Baroque era, painting is increasingly becoming an elite art, the art for esthetes and connoisseurs, which exists along with church painting, oriented mainly towards the general public. This goes through centuries: for example, “citation” appears as a tool of new optics in perception and understanding (context is the main player in higher-level mental activity): the same object can be perceived as belonging to low style or, being placed in a different, for example, aesthetic context—as high or conceptual art. Cf. postmodernism, when all interpretations are not straightforward, but with quotation marks and, recalling Nietzsche, with winking. The Baroque is not just a change of style; it is a change in the picture of the world. The role of the baroque, its stylistics and special vision of the world, manifests itself in artistic creation down to our days. The cinema sector turned to this language and imagery in the twentieth century. In 1965 A. Moravia published his article “Federico Baroque”. Superfluity, grotesque, wandering in the world of phantasmagorias, illusions, desires and the atmosphere of the approaching apocalypse characteristic of the seventeenth century art—is Fellini, one of whose main contributions to the history of the world cinema is representation of the conscious and the unconscious. Fellini removed the usual boundaries between the real and the unreal, between what is actually happening and what the character sees with his inner vision—in his dreams, hallucinations and reveries. Such cognitive tools can demonstrate the work of human brain and the thinking process, based solely on Aristotelian logic... We cannot overlook how more than a century ago I.M. Sechenov wrote that there is no difference in

8

T. Chernigovskaya

the processes providing in the brain the real events, their consequences or memories of them. This is also shown by neurophysiological studies, including those of pathological processes in the brain. If we look at the brain with a different viewing system, the neurophysiological studies give us not just a map of activations and inhibitions of neural networks, but demonstrate the presence in the brain of mechanisms that ensure the stability of the world picture for a given subject at a given period of his life, his ability to remember where is the truth and where are the delusions, operate a complicated structure of mental vocabulary, distinguish reality from hallucinations, keep different layers of memory at different distances from consciousness, etc. There is one more obstacle for studying the brain within the framework of natural sciences: there are no two identical neural networks, each individual has his own cognitome, but in all of them neurons form into certain groups, which, being the basic functional brain units, are activated simultaneously. And, as it is now clear, the most important role is played by understanding of higher-level factors—concepts, which are the maps of maps, or networks of networks, which determine the second-order phenomena (concepts of concepts, language, awareness of the self, etc.). Moreover, all these phenomena are dynamic and only slightly (if at all) reflected by a human being. But art had discovered this ahead of science! [25, 26]

6 Capriccioso, Bizzarro, Stravagante Capriccioso, bizzarro, stravagante... – these are the words Heinrich Wölflin [22] uses to define the baroque. Human brain is designed to “feel the world”– space and time, with their clots and voids... And to place yourself in this world, looking in mirrors for checking purposes... Art answers the questions that have not yet been asked, and addresses them to the brain with its original potencies. Creativity, the ability to create something that has never existed before using only the power of thought and the spirit—this is what distinguishes us from our neighbors on the planet, and not at all the additional tens of billions of neurons as such. Hypernets, human cognitomes– these biological evolution peaks—cannot be studied by simple multiplication of the technical characteristics of units and rules specific to other biological species. Studying the brain is only possible by combining the means of neuroscience and humanitarian knowledge. Cognitive sciences will never make a paradigmatic leap if they do not look in a completely another direction—towards the highest manifestations of human genius. This is a thorny path: it is necessary to study not only the results of EEG and MRI, it is necessary to look at letters and diaries of creators, records of their conversations with each other, to peer into the paintings of great masters, to study drafts and sketches, because they allow you to peep how thought was developing, to listen and hear... Human brain is not arranged like a Turing’s machine and does not work according to the stimulus-response principle; rather, the principles of its work are similar to the principles of the Baroque: like Caravaggio, it snatches out faces and objects from

Baroque Brain: Science and Humanities in Interaction

9

the background, highlights the unusual features of the world... Brain doesn’t just process information, it creates it. Exploring the neural networks and hypernets in the hope to understand the meaning of what is happening in the brain and the problem of consciousness, we are hovering over the abyss, because we study only the body. Studying only the manifestations of our mind, we hoover over the same. The ability to play with light and darkness, with figure and background, with the chosen and the rejected is the main vital task of the brain which loves emphases, because it builds the world every second. A linear approach to the processes of such degree of complexity is futile. A paradigm shift is needed... The way of looking at the brain through the prism of higher human abilities—the art and the language— may change neuroscience. Baroque optics is an example of a different look, possibly reflecting the brain processes themselves... Acknowledgements The study was supported by the research grant no. ID 75288744 “Text processing in L1 and L2: Experimental study with eye-tracking, visual analytics and virtual reality technologies” from St Petersburg University.

References 1. Anokhin, K.V.: Kognitom: v poiskah fundamentalnoj nejronauchnoj teorii soznanija [Cognitom: in search of fundamental neuroscientific theory of consciousness] (in Russian). Journal of higher nervous activity, 71(1), 39–71 (2021). doi:https://doi.org/10.31857/S00444 67721010032 2. von Uexküll, J.: Wie sehenwir die Natur und wiesiehtsiesichselber? Die Naturwissenschaften, 10(12), 265–281 (1922) (in German). doi:https://doi.org/10.1007/BF01565504 3. Popper, K.R.: Znanie I psihofizicheskaya problema: V zashchitu vzaimodejstvija [Knowledge and the body-mind problem: in defence of interaction] (in Russian). Izdatel’stvo LKI, Moscow (2008). 4. Pyatigorsky, A.M.: Neprekrashchaemy razgovor [Continuous conversation] (in Russian). Azbuka-klassika, Saint Petersburg (2004). 5. Mamardashvili, M.K.: Kartezianskie razmyshlenija [Cartesian meditations] (in Russian). In: Gurvic, I. (eds.) Philosophical readings, pp. 507–816. Azbuka-klassika, Saint Petersburg (2002). 6. Kitayama, Sh.: Culture and basic psychological processes- toward a system of culture: Comment on Oyserman et. al. Psychological Bulletin, 128, 89–96 (2002). doi:https://doi.org/ 10.1037/0033-2909.128.1.89 7. Asmolov, A.G., Shekhter, E.D., Chernorizov, A.M.: Rodoslovnaya “zhizni soobshcha”: eshche raz o skachkah evolucii [Family tree of “life together”: once again about the leaps of evolution] (in Russian). Voprosy psikhologii, 4, 3–19 (2018). 8. Lektorskii, V.A.: Issledovanija intellektualnyh processov v sovremennoj kognitivnij nauke: filisofskie problem [Studies of intellectual processes in modern cognitive science: philosophical problems] (in Russian). In: Dubrovskii, D.I., Lektorskii, V.A. (eds.) Natural and artificial intelligence, pp. 3–16. Kanon+, Moscow (2011). 9. Chalmers, D.J.: The conscious mind: In search of a fundamental theory. Oxford university press, Oxford (1996). 10. Dubrovskii, D.I.: Soznanie kak zagadka i tajna: k paradoksam “radikalnogo kognitivizma” [Mind as enigma and mystery: to the paradoxes of radical cognitivism] (in Russian). Voprosy filosofii. 9, 151–161 (2017).

10

T. Chernigovskaya

11. Chernigovskaya,T.V.: Biology, Environment, and Culture: From Animal Communication to Human Language and Cognition.Vestnik Sankt-Peterburgskogo universiteta. Filosofiia i konfliktologiia, 36 (1), 157–170 (2020). doi:https://doi.org/10.21638/spbu17.2020.113 12. Crick, F., Koch, C.: A neurobiological framework for consciousness. In: Velmans, M., Schneider, S. (eds.) The Blackwell Companion to Consciousness, pp. 567–579. Blackwell (2007). doi:https://doi.org/10.1002/9780470751466.ch45 13. Searle, J.R.: The rediscovery of the mind. MIT press, Cambridge (1992). 14. Zinchenko, V.P.: Soznanie i tvorcheskiy akt [Consciousness and creative act] (in Russian). Iazyki slavianskikh kultur, Moscow (2010). 15. Chernigovskaya, T.V.: Yazyki razuma: kto chitaet teksty nejronnoj seti [Languages of mind: who reads texts of neural network] (in Russian). In: Avtonomova, N.S., Pruzhinin, B.I., Shchedrina, T.G. (eds.) Human in the world of knowledge: in honour of academician V.A. Lektorskii, pp. 403–415. ROSSPEN, Moscow (2012). 16. Chernigovskaya, T.V.: Cheshirskaya ulybka kota Shredingera: yazyk i soznanie [Cheshire Smile of Schrödinger’s Cat: Language and Mind] (in Russian). Iazyki slavianskoi kul’tury, Moscow (2017). 17. Lotman, Y.M.: O prirode iskusstva [On the nature of art] (in Russian). In: Articles on the semiotics of culture and art, pp. 265–271. Akademichesky prospect, St. Petersburg (2002). 18. Lotman, Y.M.: Kultura i vzryv [Culture and explosion] (in Russian). Gnozis, Moscow (1992). 19. Lotman, Y.M.: Mozg-Tekst-Kultura-Iskusstvenny intellect [Brain-text-culture-artificial intelligence] (in Russian). In: Lotman Y.M. Selected works in 3 vol, vol. 1, pp. 25–33. Aleksandra, Tallin (1992). 20. Chernigovskaya, T.V., Alekseeva, S.V., Dubasova, A.V., Petrova, T.E., Prokopenya, V.K., Chernova, D.A.: Vzglyad kota Shredingera. Registracija dvizhenij glaz v psiholinvisticheskih issledovanijah [The look of Schrodinger’s cat. Eye movement registration in psycholinguistic studies] (in Russian). Izdatel’stvo Sankt-Peterburgskogo universiteta, Saint Petersburg (2018). 21. Petrova, T. E., Riekhakaynen, E. I., Bratash,V. S.: An Eye-Tracking Study of Sketch Processing: Evidence from Russian. Frontiers in Psychology, 11, 297 (2020). doi:https://doi.org/10.3389/ fpsyg.2020.00297 22. Wölfflin, H.: Renaissance and Baroque. Azbuka-klassika, Moscow (2004). 23. Markova, V.E.: O dvuh manerah mastera barokko [On two manners of the baroque master] (in Russian). In: Polevoj, V.M. et al. (eds.) Soviet art history, vol. 22, pp. 226–237. Sovetskij hudozhnik, Moscow (1987). 24. Markova, V.E.: Karavadzho i posledovateli. Kartiny iz sobranij fonda Roberto Longi vo Florencii i v GMII im. Pushkina [Caravaggio and followers. Paintings from the collections of the Roberto Longa Foundation in Florence and the Pushkin State Museum of Fine Arts] (in Russian). The Pushkin State Museum of Fine Arts, Moscow, September 15, 2015–January 10, 2016. In: Markova, V.E. (eds.) Exhibition Catalogue, pp. 39–51. Art Volhonka, Moscow (2015). 25. Lehrer, J.: Proust Was a Neuroscientist. HMH, New York (2007). 26. Shelepin, Y.E.: Vvedenie v nejroikoniku [Introduction to neuroiconics] (in Russian). Troitskiy most, Saint Petersburg (2017).

Language–Music–Gesture as Semiotic Systems

Theory of Sign and Cognition: Literary Language and the Language of Music Nina F. Shcherbak

Abstract The article aims at showing certain similarities between language and the language of music. Language in the style of music is a highly abstract system. The correlation of literary language and language of music is related to the theory of sign, in which the sign appears sometimes as a unity of the signifier and the signified, which (as some theories argue) could sometimes split. The fact that we think in a particular language may often not be the only way to explain the process of cognition. A person can often think in images, which refers to the concept of “internal speech”, developed by Vygotsky. In contemporary research there is a distinct tendency to approach a fiction as a work of poetry, and compare poetry to music, which indicates a possible correlation and interchangeability of literary language and language of music. The relationship between psycholinguistic theories and a sign theory, the study of the “surface” level of writing provides necessary ground to identify traces of subconscious processes at the speech level, in some cases to correlate it with music. Symbolic aspects of the sign (in contemporary meta-modernist tradition hieroglyphs, the so-called “Vedensky’s hieroglyph as a concept”) allow to trace and address basic properties of the language, or the language of music. Keywords Postmodernism · Cognition · Internal speech · Difference and repetition · Deconstruction · Theory of sign

1 Introduction The aim of this study was to explain some of the factors that relate to the theory of sign, which allows to see differences and similarities between language and the language in music. Similarly, questions related to the nature of cognition were considered, as well as new trends in the development of modern prose and poetry (Anglophone literature).

N. F. Shcherbak (B) St Petersburg University, Universitetskaya nab., 7–9, Saint-Petersburg, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 T. Chernigovskaya et al. (eds.), Language, Music and Gesture: Informational Crossroads, https://doi.org/10.1007/978-981-16-3742-1_2

13

14

N. F. Shcherbak

Whether music is a language or type of speech is not a generally accepted statement. A complete materially ideal formation, music represents the unity of the signifier and the signified. Nevertheless, music is also an example of the most abstract system (as opposed to language), music is an example of a complex of different concepts, representations, and emotions that carry certain meanings. Language, in a similar way, has got an arsenal of properties that allows it to construct and render concepts while representing things. Moreover, for contemporary prose and poetry it is quite common “not to represent” events, actions, or states, but to “render the meaning” as if “through the word”: by means of the form and narrative, for instance by paying special attention to rhythmical patterns, motives, and allusions, which allows the readers to get the meaning (often not stated explicitly) in all its complexity. This tendency of twentieth-twenty-first century prose and poetry eliminates the border between literary language and language of music, bringing them closer to form one common foundation of perception, cognition, verbal, emotional, physical, and spiritual action.

2 Methodology and Sources Methodology of this study includes works by critics of music, linguistics, and literary theory specialists who developed ideas regarding the nature of language, sign system, and cognition [1–6]. Firstly, most theories regarding cognition and language; music and speech, have undergone certain transformations throughout time, depending on the aesthetic or scientific paradigm that was crucial at a particular moment in the science’s history.1 For instance, the key notion for conventional linguistics that the sign is “the unity of form and meaning” (de Saussure) was later challenged on a number of occasions. Certain changes, even in fundamental principles of the view of a semiotic system, are relevant when considering music, as well as comparing the language of music and literary language. The language world view that the structuralists have postulated (they believe that language study is descriptive, language is something that could be divided into elements) is different from that of transformational grammar developed by N. Chomsky. The general view of transformational grammar is about the similarity of many languages, the idea of Universal Grammar, surface and deep structure in a way substitute the structuralist point of view, concentrating on what unites languages rather than what the difference between them is. In quite the same fashion, the view about language learning changed at some point, the view that language is a behavior (Skinner), was challenged by the idea of Language Acquisition Device, introduced 1

What is meant here is a view proposed by M. Foucault about three epistemes (an episteme being a philosophical term that refers to a principled system of understanding; scientific knowledge). It also explains how “things” and “words” relate to each other depending on the historical period (episteme of the Renaissance era, episteme of the Classical era, episteme of the Modern era [7]

Theory of Sign and Cognition: Literary Language and the Language of Music

15

by Chomsky [8], which drastically changed not only the general view about language learning (from behaviorism to nativist), but also most Foreign Language Teaching Methods [2]. Therefore, there is always a paradigm change (in music, linguistic, and literary studies). Secondly, the idea about Universal Grammar and Language Acquisition Device brings into sharp focus the attention to the innate properties of the language, stressing the inherited, “human” view about the language skill, or, as some researchers famously call it, the language instinct (Stephen Pinker) [6]. Certain semantic and syntactic features are innate, and common for most people. This general capacity of humans to acquire a language is the very basis of one’s psyche and cognition, and this is definitely something common that language has with the language of music. Thirdly, regarding music, there have been a number of paradigm shifts through the centuries, in theory, as well as in music patterns. One of the many examples is, for instance, a major shift in the aesthetic paradigm that occurred with the appearance of New Music, or post-avant-garde music that brought about completely new rules for its composition [9–12]. The concept of rhizome, “difference and repetition” introduced by post-structuralist philosophy of the twentieth century (for example, G. Deleuze) coincided with and largely influenced the appearance and development of a new type of music in which completely new concepts appeared. The concepts that allowed to incorporate “noises” and “music sounds”, introduce (as a special entity) the notion of silence (John Cage) [13], allowed to develop the idea of “white noise”. The theory of noise sound was developed in the studies of the English electro-acoustic composer T. Wishart On Sonic Art [14]. Among other concepts one could also name “Vanitas”, the phenomenon of “network concept” (projected later by a contemporary composer Sciarrino into the concept of “shapes with windows” [15], the concept of disembodiment (Deleuze) and its further manifestation in New Music. Fourthly, it is important to mention that the previously discussed philosophical tradition falls under the umbrella term of “post-structural thinking”, a recent trend (among many others) may be called meta-modernist tradition, with its general principle of a “compression of sound”, “new simplicity”, “global quotation”, etc. Such concepts have appeared as a sign of major shift in the form of a piece of art or music, however, it does signify certain changes in the process of cognition as well, change of schemes, schemata, if not fundamental principles of composition itself [16]. Philosophical concepts that constitute a “school” or “tradition” are normally similar in all spheres of art, in literature, music, or architecture. In other words, the existence of an aesthetic paradigm (an episteme, in Foucault’s terminology) allows to unite music and language, and find similarities between the music of language and the language of music. Language and language of music have common ground, yet some of the features could be altered depending on the historical time, major aesthetic paradigm, or tradition.

16

N. F. Shcherbak

3 Results and Discussion 3.1 Music and Meaning, Theory of Sign and Iconicity If we take the statement about the meaningfulness of any musical activity as an axiom, it follows that any musical work (as an object of such an activity) is saturated with meaning, i.e., a certain spiritual (ideal) content, which is transmitted (broadcast) with the help of material means (sounds, acoustics), or with the help of a mental representation of real sound [17, p. 35]. In most cases, this meaning, the spiritualideal content, is evaluated as modeling, comprehension, generalization and, finally, communication of the phenomena of reality mastered in this way by music. We should not, however, lose sight of the fact that musical meaning is not only the assimilation of values, but at the same time, it is a new value created, which did not previously exist and does not exist outside of this musical work [17]. The attempt to transfer these ideas in relation to the language sign raises certain questions that do not have a straightforward answer. The principle of unity of the signifier and signified refers to the ideas of F. de Saussure, at the same time, in modern science there are researchers who believe that the plain of expression (the form) is separable from the plain of content (the meaning). In particular, Professor I. K. Arkhipov puts forward the idea that when transmitting a message, only the form of expression is transmitted by the speaker to the listener, and the meaning is interpreted by the listener accordingly [18]. The idea of a certain “division” possible in the relation between the form and the meaning in music, is very well seen, as the spectrum of the listener’s interpretation could vary dramatically. “The listener consciously (and more often, unconsciously) includes the fragment in a familiar context < … > , and then the signified is formed in the consciousness of the listener, associated with this fragment as part of a musical work. Or the listener, willingly or not, “draws up” a given piece of context in his mind, generates the context factors” [17]. The unity (and separability of the form and meaning) are in many ways consistent with research in the field of phono-semantics, and again draws our attention to the problem of the sign (as having a symbolic, index, iconic nature, according to Charles Pierce [19]). According to this semiotic system, it is common knowledge that most language signs are symbolic, that is, the sign refers to the concept conventionally, the sign has a certain content due to tradition. However, as research shows, a huge number of language signs have an iconic nature [20–26]. In other words, the sign relates to the concept in a certain way and then this connection can be designated and traced. In contemporary research it is widely shown [27] that the form of the sign does relate to its meaning. This could be manifested throughout languages in “open”- “closed” vowels option (the “form”), which corresponds to the meaning of “positive”—“negative”, correspondingly. The relation of the form to the meaning could be realized in what is called “iconic proximity”, or “iconic density” [28]. The former is the “closest” position of an adjective to the noun (which in the English language is fixed: the adjective that bears more “permanent” characteristics is closer

Theory of Sign and Cognition: Literary Language and the Language of Music

17

to the noun). The latter could be shown in more “densely” organized text, with many syntactic and semantic “aggravations”. Some approaches construe iconicity as a discrete property that is either present or absent; others treat it as involving semiotic relationships that come in kinds; and yet others see it as a gradient substance that comes in degrees. Researchers show the benefits and limitations that come with each of these construals and stress the importance of developing accounts that can fluently switch between them. This provision will speak both about the distinctive and similar features of literary language and language of music. It seems at first glance, that the idea of iconicity is less relevant for understanding similarities between language and language of music. However, there are many similarities. As Bonfeld writes [17], “summing up the consideration of the problem of differentiation of the musical sign as the unity of the signified-signifier, you can claim that this unity is realized only on the level of entire musical works (to a certain extent this applies to the finished parts); isolated from the context, the fragments of musical works cannot be considered as signs (words), for the meaning of these fragments is determined only and exclusively in the context which they are extracted from. At the same time, a finished fragment of the whole piece of music, extracted from the context, can function as a standalone piece if the fragment meets some conditions” [17]. If we take the notion of iconicity further, and look at meta-modernist music (the one that is characterized, among other factors, by a “compression” principle of organization) [29], then it is quite clear that “the compressed” sound in a way refers to a “universal kind of code”, almost “the basic sound of the Universe”, thus “rendering” the meaning and having a close relation to meaning. Such a connection is well seen in “Vvedensky hieroglyph”, a concept, which claims that a sign (a hieroglyph) refers to Logos, to the very original “word”: “The hieroglyph can be defined as the direct or indirect speech of the immaterial, that is the spiritual or the supersensible, addressed to me through the material or the sensuous.” [30]. This view is very similar to the definition given to the word “symbol” by a Russian classical poet Vyacheslav Ivanov—“the symbol is a word, dark in its last depth of meaning” [31]. Sound in New Music becomes something very special. “Spectral school” developed by Grize allows one to “hear the micro-world of sound by means of macrophonically” [32]. Taking the acoustics of each separate sound as a harmony material of the whole musical composition, French spectralism orientates towards the principles of “liminal writing” (limen—Latin, meaning “threshold”) that address the psychological aspects of one’s perception. “Threshold” or “liminal” states that appear when you feel the “threshold” of timbre or harmony, this is what constitute the inner sound space in spectralism [15]. The structure of inner sound space (its spectrum) is an important stage of development for contemporary composers. This attention to individual sound (that is possible to achieve due to technological development and discovery of electronic music) brings the music closer to its iconic properties allowing us to see the whole spectrum of meanings in one entity. The iconic property of language as well as the capacity of music to “address fundamentals” through discovering “the micro-world of sound” is what unites language and language of music, showing their similar foundation.

18

N. F. Shcherbak

3.2 Language and Thought, Internal Speech Another important point of “mutual ground” between literary language and language of music is the position of “inner speech”, developed by Vygotsky [33], and reflected in the study of such scientists as N. I. Zhinkin [34] and prof. A. G. Minchenkov [35], who postulated the notion of the “subject code”, or, respectively, the “language of thought”, different from the presumption accepted in modern linguistics (based on the assertion that we think only in a language). The idea of the “language of thought”, however, refers to an abstract-figurative thinking process, subject and semantic code, which postulates a completely new idea regarding the cognition process. These ideas correlate with the provisions of the cognitive theory of metaphor, which shows that a person thinks by means of metaphors. A language of music, similarly, is a semantic system in which different rhetorical figures were fixed in different eras. What is common for language of music and literary language, therefore, is the ability to maximize abstraction, which is realized through sound, harmony, or acoustic means. Music is a very abstract system of signs. Musical sounds are always a multidimensional non-linear “space". “The lack of a consistent unity of the signifiedsignifier in relation to fragments of musical works, a multi-dimensional nature of the musical space led to indiscreet (continuity) of musical statements which does not allow us to consider a piece of music as a combination of characters. This explains the absence (and, apparently, the impossibility of the appearance) of dictionaries covering “musical language". The existing “intonation dictionaries” represent nothing more than a register typical of the era, style, direction, or the individual composer, which does not assume any inherent vocabulary that could be accepted as a formal basis” [17]. However, what could be done to show the unity of the language of music and language is to show how both are related to existing concepts of philosophy.

3.3 Fiction as Poetry, Poetry as Prose One of the clearest manifestations of similarity between the language of music and literary language is the tendency of literary language to remind us of poetry, and for poetic language to remind us of music. In other words, in modern prose, there has been a tendency to concentrate not so much on the word and its meaning, but on the rhythm of the narrative, the organization of the narrative, the existence of various sub-systems with different spatial and temporal coordinates. The study of the heterogeneity of the text and attention to the smallest details of the narrative, allow us to trace the similarity of the narrative fabric and the musical work. The poetic language, in turn, gets closer to the musical one. For poetic creativity, rhythm and vocalization, the slightest nuances of sound realization, become the fundamental principles of the organization of verbal space.

Theory of Sign and Cognition: Literary Language and the Language of Music

19

Concepts of postmodernism (including that of rhizome, “difference and repetition”, “vanitas”, silence, general anamorphosis principle, and the “body without organs”) concept [39, 40] could be applied to explain the patterns observed in New Music. In a similar fashion, a great deal of post-modernist writing is developing along similar patterns to philosophical concepts, and, correspondingly, to musical concepts [36–38].

3.4 The Prose by Salinger and the Ecology of Listening The below mentioned research shows how the concept of rhizome, the concept of “difference–repetition” [39], and other post-modernist concepts (including that of silence, which is studied along the lines of “eco-listening”, observation of “spectrum of sound”) [40], are applicable to the study of the prose by Salinger, a famous American author. One example that could be given is the story Just Before the War with the Eskimos (1948–1953) written by Salinger, that follows post-modernist principles and well complies with postmodern philosophy. According to the opinion of researchers, the fact of placing 9 stories into one book, for Salinger, was to recreate the emotional, psychic, and spiritual way of an individual’s experience; reflecting the main spectrum of people’s sufferings and passions: that is comparing a piece of art with something that has organic nature (in terms of Ancient Indian philosophy). Zen philosophy was an attempt by the author to escape from reality, one of the ways to treat a psychic disorder that he suffered from after having participated in armed conflicts.2 The mirror-like isomorphic nature of the story is the basis of it, this is not just a compositional instrument or an aesthetic means, this is an intrinsic component of the meaning that is made up by the author on all possible levels. Symbolism and the semantics of mirroring have been studied carefully. For instance, for Bakhtin a mirror is a means of self-realization through your own reflection [41]. Other researchers

2

At the end of the war Salinger was in Normandy. In April 1945 he was among those who liberated Kaufering, one of the divisions of the concentration camp Dachau. The action in the story Just Before the War with the Eskimos is very simple, almost schematic. The action takes place in May 1945, immediately after the war. Ginnie tells her friend Selina that she doesn’t want to pay for the taxi herself. Though the story is schematic, at first sight, it gives a lot of insights. According to the opinion expressed by I. Galinskaya, “the poetic mood of the story is compassion, “the hidden meaning of compassion is felt apart from the direct meaning expressed in the plot”. The researcher points out that the parallelism method is simple: in the living room, in between Selina’s return and Franklin’s departure there appears Erich, Franklin’s friend who tells Ginnie about the homeless writer that he put up at his house feeling sorry for him and introduced to important people. It all ended up with the new friends stealing everything from Franklin and disappearing. The meaning of the parallel is in the movement of the feeling: from offence to compassion as experienced by Ginnie and feeling expressed towards, firstly, Selina and then—Franklin. In the other episode, it is the other way round—the feeling movement from compassion to offence as experienced by Franklin.

20

N. F. Shcherbak

consider a mirror to be the border (or the surface) between “yours-foreign”, “livedead” worlds, “inner-outer” [42].3 Salinger’s isomorphic mirror-like organization is manifested in the composition and is characteristic of his prose on the level of semantics, as well as on the syntactic and pragmatic level. In the story Just Before the War with the Eskimos there is the motif of offence and compassion that exist and don’t exist. There is no dynamism as far as the relations between the characters are concerned. What is there then? Endless numbers of mirror-like repetitions which create a crystal form that is pointing at the center, the center being the house of Seline that Ginnie (the main characters). In the space subsystem of the house, the virtual and real worlds become actualized. Coldness (a characteristic feature of every character), seem to crystalize the narrative (“friends are not paying back money”, is the complaint verbalized by Ginnie, “mother is ill” is Seline’s complaint, “a friend’s deceit” is Erich’s complaint, “wounded my finger” is what Franklin talks about). Apart from the above mentioned parallelism on the compositional level, in the story there are a lot of dialogues which at a closer look are interesting from the point of view of echo-repetitions and sound modifications. By echo-elements I refer to questions like (“do you?”, “is she”, that in the English language serve to establish contact with the speaker and are rhetoric, as they imply a positive answer). The end of the story is a series of repetitions that actualize the difference: the behavior of the main character in similar situations is different: in the first instance she wanted to do something but didn’t; in the second—she wanted to do something and did it. Thus, realization of the “difference-repetition” concept (well described by Deleuze, and so characteristic of New Music) is well seen in contemporary prose as well. For full understanding of the short story by Salinger it is crucial to take into account the susceptibility of the author to sound, its spectrum, and its different transformations. In the story Just Before the War with the Eskimos it is the unheard sound of a tennis ball hitting the ground, that creates a certain sound phantom, a virtual “bounce” that increases the illusion of the sound, giving it more volume. There are numerous forms of sound realizations in the text that are especially noticeable against the background of silence. The opposition of silence and speech are characteristic of postmodern writing and is derived from ideas expressed by post-structuralists, in general, and G. Deleuze, in particular, who wrote about the notion “emptiness” (notion similar to silence [39]). In the story Just Before the War with the Eskimos, Ginnie (one of the characters) pauses several times, indicating she is late with a reply cue. In another episode, in hostile silence, the girls stared out; in one more episode—the author says that “it was difficult for Erich to talk” and “he was out of breath” [43]. This sort of contrast between sound and silence creates a special sound 3

Organization of the narrative developed by Nabokov, for example, is based on the idea of multiple worlds, in which the reflected worlds are contrasted. The mirror, thus, in literary works denotes the image, the means, the symmetrical composition. If for some post-modern writers, for instance, for Nabokov, the principle of mirror or palindrome on the level of semantics and composition is more evident and frequent (that is the internal play of sounds and words that create the semantic seesaw principle, the simultaneous co-existence of many meanings, the co-existence of places and worlds).

Theory of Sign and Cognition: Literary Language and the Language of Music

21

space, in which voices of the characters are heard against the background of sudden pauses and silence. Another special feature of the story is the realization of different forms of perception including (a) by touch: Erich strikes Ginnie’s coat and talks about the quality of wool: “I love your coat,” he said, already out of his chair. He crossed over and took the lapel of Ginnie’s polo coat between his fingers. “It’s lovely. It’s the first really good camel’s hair I’ve seen since the war; (b) by smell: Franklin is inhaling a cigarette: tilting his head back, he slowly released an enormous quantity of smoke from his mouth and drew it up through his nostrils; (c) by sight: Ginnie is looking around the room assessing it negatively: “She looked around the room, mentally rearranging furniture, throwing out table lamps, removing artificial flowers. In her opinion, it was an altogether hideous room–expensive but cheesy”. All in all, the effect of the text “revival” is evident, a very rigid structure of the story (composition and plot) allow to accommodate fragments of real human experience and perception as if they were implanted into the text. This almost physical property of the text is very similar to the feeling one gets when listening to music.

3.5 J. Winterson’s Writing Another example of the realization of the fundamental concepts of postmodernism (and meta-modernism) in literary texts is contemporary women writing. The idea of women writing and its existence has been widely discussed not only by literary critics but by philosophers of language, who claim there is and there isn’t a difference between female and male writing [44]. Woman writing is very diverse, and at the same time shows certain degrees of similarity. A good example of the analysis is given by Showalter [45], who explains that women prose has had certain transformations, and at a later stage of its development it is not about “competing” with “male prose”, it is about establishing its own identity. J. Winterson’s prose is a good example of woman writing, and it is very special in its attention to wording, as well as to the sound of the narrative. Its rhythmical patterns and syncretism allow the author to develop writing that reminds us of music and is characterized by certain musical patterns. The process of vital activity of sounds and words in literature manifests itself when comparing the sound background and the color scheme of the surrounding world, then implementing this phenomenon into the text. With regard to the actualization of the motive of silence A. Veselovsky notes that in the poetic form, not only metaphor epithets are usually used, such as “the dead man is silent”, “silence is a sign of death”, but also other epithets explained by physiological syncretism [46]. Syncretic epithets correspond to the unity of sensory perceptions, which primitive man often expressed with the same linguistic indicators. The scientist draws attention to a number of Indo-European roots that correspond to the concepts of sound and light: the churchslavic “sharp” is used to denote both sound and light impressions; the French “voix somber” means “dark voice”, “bleu sourd” means “deep blue”, “voix blanche” means

22

N. F. Shcherbak

“white voice”; the German ‘heller Ton”—“light tone”, Latin “tacito … murmure” is “whispering silence”. Hugo writes about “a timid, dark noise made of shadows”, Dante writes about “the silence of the sun”, Virgil writes about “the friendly silence of the moon” [46]. Consider examples from Winterson’s prose in which the combination of sound and color chiaroscuro creates additional meanings, which, according to critics, “complicates and encrypts the plot” [47]: “from a distance only the light is visible, a speeding gleaming horizontal angel, trumpet out on a hard bend. The note bells. The note bells the beauty of the stretching train that pulls the light in a long gold thread. It catches in the wheels, it flashes on the doors, that open and close, in commuter rhythm” [48]. In the example you see a situation that describes the movement of a passing train. When describing the surrounding sounds (trumpet out, the note bells), a grammatical metaphor is used: nouns denoting musical instruments (trumpet, bell) are actualized as verbs. Color phenomena are described in terms of their color (golden thread), and energy characteristics: sparkles, flashes, turns on or off as a current, gleams; which makes the narrative multi-layered, voluminous, and intense. In the middle of the twentieth century, listening to ecology becomes an absolute value of literary creativity on par with musical creativity. The latter is characterized by the highest threshold of sound sensitivity and increased attention to microscopic sound elements, to the smallest nuances of changing sonority. Musical creativity is characterized by the fact that silence becomes a full-fledged sound material, acquiring aesthetic “legitimacy” [12]. In her later novel Frankissstein: A Love Story (1999) Winterson does not take music or art as the main subject, yet refers to poetry: one of the main characters in the book is Mary Shelley. In this way Winterson shows how writing a prose is similar to writing poetry. The title “Frankissstein” is a good manifestation of meta-modernist. On the one hand, it uses “compression” technique, manifested in “sss” sound in the title, on the other hand, it quotes previous material, without declaring the authorship: realized in “taking the story of Mary Shelley and changing it”. The concept of a Vvedensky’s hieroglyph here is the possibility of making a title uniquely reminiscent of the basics it refers to. The basics (or the referent) could mean (1) Frankissstein, the soulless monster, (2) a woman writer of the Romantic period, being suppressed, (3) a kiss, as a general metonymy of love of any kind, (4) Hertrude Stein—the reference is identified as the name Frankissstein rhymes with “Stein” and all the women who openly declared their independence and feminism. Winterson explores the possibility of language very thoroughly and in this way her writing is always reminiscent of music.

3.6 Poetry of Modernism and Meta-Modernism Poets have been dwelling on the similarity between poetic works and music throughout the centuries. In his major book on symbolism Andrey Beliy writes about music mind and quotes a famous German philosopher Schopenhauer, who contrasted

Theory of Sign and Cognition: Literary Language and the Language of Music

23

music with all the arts, saying that it expresses the will, that is, the essence of things. To comprehend the phenomenon, Andrey Beliy continues, “one has to listen to the music, here we are closest to the possibility of comprehension. < … > Music is the mathematics of the soul, and mathematics is the music of the mind. Nowhere do we have such an affinity and contrast at the same time as exists between the comprehension of phenomena (music) and the study of similarities and differences in the field of their quantitative change (mathematics)” [49]. In a way, Beliy writes about music being the mathematics of the soul, and mathematics being the music of the mind; poetry is almost always about music, as it renders a very complex view of the world, one that has many dimensions and the sound characteristic of it is of crucial importance. A good example in our study is the appearance of modernist poetry that was discussed by such poets as T.S. Eliot, who in his theoretical writing, explains that poetry, above all, is not about rendering the feeling (as it was during the Romantic era, with Coleridge or with Keats), but it is about transcending important notions of “the world”: [50]. For instance, in his essay on “Ezra Pound”. T. S. Eliot quotes Stray Document and writes that this is an advice he must have given himself: “Let the candidate fill his mind with the finest cadencies he can discover, preferably in a foreign language so as that the meaning of the words may be less likely to divert his attention from the movement; e.g. Saxon charms. Hebridian folk songs, the verse of Dante, and the lyrics of Shakespeare—if he can dissociate the vocabulary from the cadence. Let him dissect the lyrics of Goethe coldly into their component sound values, syllables long and short, stressed and unstressed, into vowels and consonants” [50]. Such an approach to poetry, allows to prove that the most significant asset in poetry is not the word meaning, but its form, and that is what T.S. Eliot’s and all modernist poetry is about. In 1909 Eliot use rhymes verse, but, as critics note, the referents are very vague and indefinite: This charm of vacant loss. The helpless fields that lie. Sinister, sterile and blind. Entreat the eye and rack the mind. Demand your pity. With ashes and tins in piles, Shattered bricks and tiles. And the debris of a city. [51] (Eliot, Second Caprice in North Cambridge).

In this poem one can observe a clear vowel opposition of the “short” “i” and the “long” “I”, their juxtaposition allow to create a very special mood when describing the landscape, as if in the text you get an enlargement of the picture (long, tense “I”), and then it is compressed (“shorter i”). This is an iconic way of showing nonlinearity, an uneven surface of the poem, which at the same time is very melodic

24

N. F. Shcherbak

and almost lyrical (the vowels change in the following words: pity, tiles, piles, city). Every element in this poem is very thoroughly chosen by the author: the foundation of writing is in its construction [52].

3.7 Surface Reading and Psycholinguistic Theories Versus Language of Music The most important issue in modern hermeneutical theory is the problem of “symptomatic” or “surface reading”, this approach allows us to analyze how the “surface level of a narrative” indicates the fundamental deep principles of text organization. This provision correlates and takes further the idea of the interaction of literary language and language of music, introducing a psychological component; since the analysis of the text, literary or musical, will include not only the study of the cognitive process, but also affect the subconscious. Perhaps, this is something that unites the language of music and language of words more than anything else, as it refers to language as a means to approach the subconscious, getting into the most fundamental entities or even archetypal areas of one’s psyche. If the philosophy of the subconscious was at some point ascribed to such figures as K. Hartmann [49, p. 87–88], the contemporary version of the psychoanalytical view of writing is associated with altogether different names and techniques. In their article on surface and symptomatic reading S. Best and Sh. Marcus [53] have quite recently explained the importance of the surface reading that gives insights into the most fundamental basic units of one’s psyche. Revolutionary at the time, it summed up most literary critical views on the subject, then continues with the tradition of psychoanalytical view of texts allowing to re-think literary works and look at them from a different perspective, as was done by Felman in his famous article on Henry James’s Turn of the Screw [54]. The latter explains down to the minutest detail a psychoanalytical perspective that the text of Henry James provides. Similar research has been carried out on most of J. Winterson’s works [55]. On the one hand, one can’t easily project certain psychological characteristics on a composer when listening to his music, though the language definitely provides good insights into the author’s psyche. On the other hand, certain techniques of contemporary music writing, like “liminal writing” or spectral music, (as was mentioned above) do allow to get a closer view of individual sounds and to study “threshold” states, and the psychological and psychoanalytic dimension.

Theory of Sign and Cognition: Literary Language and the Language of Music

25

4 Conclusion Firstly, the correlation of literary language and language of music is related to the problems of the sign theory, for which the sign appears sometimes as a unity of the signifier and the signified, and sometimes as having the ability to split this unity. Secondly, the ability to think in a language may often not be the only way to explain the cognitive process. A person can often think by means of images (also known as the “subject code”, or “inner speech”). Thirdly, there is a distinct tendency to approach a literary work and treat it as a poetic one (for twentieth-twenty-first century writer as well as critic), and, correspondingly, there is a clear tendency to treat a poetic work as a musical one, which indicates a possible correlation and interchangeability of a literary language and the language of music. Fourthly, the relationship between psycholinguistic theories and a sign theory, the study of the “surface” level of writing (symptomatic, surface reading) as well as “liminal reading” and, respectively spectral music, allow us to identify traces of subconscious processes at the level of speech, in some cases to compare language and music.

References 1. Berko G.J. The development of language. Allyn and Bacon, New York (2005). 2. Brown H.D. Principles of language learning and teaching. St. Francisco State University. St. Francisco (2006). 3. Clark, E.V. First language acquisition. CUP, Cambridge (2003). 4. Chomsky N. Aspects of the theory of syntax. CUP, MA, Cambridge (1965). 5. Ephratt, M. The functions of silence. Journal of Pragmatics, vol. 40, 1909-1938 (2008) 6. Pinker, St. The Language instinct, Penguin, London (2015). 7. Bernauer, James William. Michel Foucault’s Force of Flight: Toward an Ethics for Thought, Contemporary studies in philosophy and the human sciences. Humanities Press, Atlantic Highlands, NJ (1990). 8. Aitchison, J. Articulate Mammal: an introduction of psycholinguistics. Routledge, London, New York (2008). 9. Campbell E. Music After Deleuze. Bloomsbury Academic, London (2014). 10. Hulse C.B., Nesbitt N. Sounding the Virtual: Gilles Deleuze and the Theory and Philosophy of Music. Princeton, Princeton University, Princeton (2010). 11. Waterhouse B. Strati, piano, rizoma. John Cage e la filosofia di Gilles Deleuze e Félix Guattari. Tesi Università di Bologna, Bologna (2012). 12. Lavrova S.V. Logika smisla novoy muziki. Opit structurno-semioticheskogo analiza na primere tvorchestva H. Lahenmanna I Salvatore Sciarrino [Logic of sense of New Music] (in Russian). SPTSU, St. Petersburg (2013) 13. Cage J. Tishina. [Silence] (in Russian). Biblioteka Moskovskogo conceptualista Hermana Titova, Vologda (2012). 14. Wishart T. On Sonic Art. Routledge, London (1996). 15. Lavrova S.V. Salvatore Sciarrino I drugiye. Ocherki ot italyanskoy muzike kontza dvadzatogonachala dvadtzat pervogo veka. [Sciarrino and others] (in Russian). Academiya russkogo baleta imeni Vaganovoy, St. Petersburg (2019) 16. van den Akker, R., Vermeulen T., Gibbons, A. Journal of Aesthetics and Culture November (2010).

26

N. F. Shcherbak

17. Bonfeld, M. Muzika: yazik. Rech, Mishleniye. Opit sistemnogo issledovaniya muzikalnogo isskustva. [Music: Language. Discourse. Cognition. The experience of a systematic study of music] (in Russian). M Press, Moscow (1991) 18. Arhipov I. K. Polifonia mira, teksts i odinochestvo poznajushchego soznanija [Polyphony of the world, text and loneliness of the cognitive mind] (in Russian). Studia Linguistica XIII. Cognitive and communicative functions of language. St. Petersburg press, St. Petersburg, p. 7–18 (2005) 19. Pierce, Ch. Izbranniye philosofskiye proizvedeniya. [Collected works] (in Russian). Trans. from English by Golupovitch. Logos, Moscow (2000) 20. Attridge D. Peculiar Language: Literature as Difference from the Renaissance to James Joyce. Methuen, London, pp. 75–82 (1988) 21. Bolinger D.L. Forms of English: Accent, Morpheme, Order. Hokuon, Tokyo, pp.3–35, 447–448 (1965) 22. Croft W. Modern Syntactic Typology. In: Shibatani, Masayoshi, Bynon Theodora (eds.): Approaches to Language Typology. Clarendon Press, Oxford, pp. 85–144 (1995) 23. De Cuypere L. Limiting the Iconic. From the metatheoretical foundations to the creative possibilities of iconicity in language. J. Benjamins, Amsterdam (2008) 24. Dressler W.U. Word-formation in natural morphology. In: Štekauer, Pavol, Lieber, Rochelle (eds.). Handbuch of Word-Formation. Springer, Dordrecht (2005). 25. Haiman J. The iconicity of grammar: isomorphism and motivation. Language, 56 (3), 515–540 (1980). 26. Haiman J. Natural Syntax: Iconicity and Erosion. CUP, Cambridge, pp. 515- 540 (1985). 27. Nänny M., Fischer O. Form Miming Meaning. Amsterdam: John Benjamins, Amsterdam (1990). 28. Dingemanse M., Perlman M., Perniss P. Construals of iconicity: experimental approaches to form-meaning resemblances in language. Language and Cognition , 12 (1). Special Issue on Iconicity, March, 1–14 (2020). 29. Hrusheva N. Metamodern v muzyke i vokrug nee [Metamodernism in music and around it] (in Russian). Ripol-classic, St.Petersburg (2019). 30. Rimar, A. Ieroglifichesky tip simvolizatzii v hudozhestvennom tekste. Na material poetiki A. Vedenskogo. [Heroglyph symbols in literary text (A. Vedensky poetics)] (in Russian). Samarskiy Universitet Press, Samara (2004) 31. Ivanov, V. Poet I chern. [Poet and people] (in Russian). Completed works in 4 volumes. Vol. 1. Pres, Brussels. pp. 709—714 (1971) 32. Grisey G. Tempose ex machine: A composer’s reflections on musical time. Contemporary Music. Review, 2 (1), 239-275 (2008). 33. Vigotsky, L.S. Mishleniye I rech. [Cognition and speech] (in Russian). Moscow University Press, Moscow, pp. 292 — 334 (2007). 34. Zhinkin N.I. Yazik. Rech. Tvorchestvo. [Language, Speech, Art] (in Russian). Labirynth, Moscow (1998) (In Russ) 35. Minchenkov A.G. Kognitivno-evristicheskaya model perevoda: k postanovke voprosa [Cognitive and euristic model of translation (on the material of English language)] (in Russian). St. Petersburg University Press, Petersburg (2013) 36. Shcherbak N. F., Gerus A. I. Transcendentalism, Network Concepts and American Poetry. Discourse, 6 (1), 106–120 (2020) 37. Shcherbak N.F., Taunay A. The Prose by Salinger and the Metaphysics of Gilles Deleuze. Discourse, 6 (2), 29-43 (2020) 38. Shcherbak N. Breath of Sound, or Acoustic Alchemy: Pragmatic Functions of the Silencial Speech Act. NLO, no. 148 (6), (2017) 39. Deleuze, G. Différence et répétition (1968). Translated into Rus. by N.B. Mankovskaya. Petropolis, St.Petersburg (1998) 40. Deleuze, G. Cinema 2, L’Image-temps, Les Editions de Minuit, Paris (1985) 41. Bahtin, M. Estetika slovesnogo tvorchestva [Aesthetics of verbal art] (in Russian). Bahtin, Moscow (2002)

Theory of Sign and Cognition: Literary Language and the Language of Music

27

42. Vulis, A. Literaturniye zerkala. [Literary Mirrors] (in Russian). Sovetskiy pisatel, Moscow (1991) 43. Salinger, J.D. Nine stories, available at: https://materlakes.enschool.org/ ourpages/auto/2013/2/25/50973306/Nine_Stories_by_J_D__Salinger.pdf, last accessed 27.01.2020. 44. Derrida, J. Spurs: Nietzsche’s Styles. Éperons: Les Styles de Nietzsche. University of Chicago, Chicago (1979) 45. Showalter E. Towards a feminist poetics. In: K. Newton (ed.) Twentieth century literary theory. MacMillan Publishers Limited, pp. 216–220 (1997). 46. Veselovsky A. N. Iz vvedenija v istoricheskuju poetiku [Introduction to historical poetics] (in Russian). In: Veselovsky (ed.) Historical poetics. Moscow press, Moscow (1989). 47. Musatova, E. V. Poetika cveta i sveta v romane Dzhenet Uinterson “Art and Lies” [Poetics of light and colour in the novel Art and Lies] (in Russian) by J. Winterson. Bullitin of Novgorodsky State Universtiy, 51 (2009) 48. Winterson, J. Art and lies. Knopf, Canada (1994). 49. Beliy, A. Simbolizm kak miroponimaniye. [Symbolism as a world view] (in Russian). Respublika, Moscow (1994) 50. Sutton, W. Ezra Pound. In: W. Sutton (ed.) A collection of critical essays. Prentice Hall, N.J. (1963) 51. Eliot. T.S. Second Caprice in North Cambridge, available at https://songme.ru/song/3327005second-caprice-in-north-cambridge-by-t-s-eliot, last accessed 24/12/2020 52. Bernstein Ch. Content’s dream essays. 1975–84. Los Angeles: Sun and Moon (1986) 53. Best S. & Marcus Sh. Surface Reading. Representations, vol. 108 (1), 1–21 (fall, 2009) 54. Felman, S. Writing and madness. Literature. Philosophy. Psychoanalysis. Stanford University Press, California (2003). 55. van der Wiel R.C. Writing the Body: Trauma, Woolf, Winterson. In: Literary Aesthetics of Trauma. Palgrave Macmillan, London, pp.22–47, available at https://doi.org/10.1057/978113 7311016_2 (2014)

Hierarchical and Non-hierarchical Structures in the Language and Music Giula Shamilli

Abstract The study of verbal and musical structures shows that the unity of language and music is not limited by the hierarchically organized sequence (Lerdahl– Jackendoff) or the ‘hierarchy of events’ (Bharucha). Rather, it depends on the types of organization of the subject-predicate complex that represent hierarchical and nonhierarchical structures in both language and music. This idea is elaborated based on Near Eastern professional music of oral traditions, including Israel (the Art of maq¯am), on the one hand, and the classical Arabic language theory, which does not employ the concept of adequate to the ‘proposition’ in the Western sense, on the other hand. It is concluded that there are two parallel formulas of the process in music, depending on the quality of the subject-predicate linkage: (1) i → m → t (ac. B. Asafyev, 1926) and (2) i ↔ t = m—temporality (G. Shamilli, 2017). The possibility of non-hierarchical organization of the subject-predicate linkage in music is shown. The conclusion has profound strategic implications and points to a type of rationality that manifests itself in various segments of musical culture. Keywords Architectonic of Consciousness · Subject-Predicate Linkage · Proposition · Professional Music of the Oral Tradition

1 Architectonic of Consciousness (Instead of an Introduction) Does music obey the general regularities of thought, or does it operate with ‘particular’ logic, special functions that can only be described in the language of music theory? There is no contradiction between the two parts of this question, separated by the conjunction ‘or’. The thought expressed in a musical phrase, even in its most extravagant form, when the actual sound nature of art recedes into the background, proceeds according to the rules most clearly formulated today in the field of language theory. In other words, there is no reason to speak of a particular, specific

G. Shamilli (B) State Institute for Art Studies, Kozitski-5, 125009 Moscow, Russia © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 T. Chernigovskaya et al. (eds.), Language, Music and Gesture: Informational Crossroads, https://doi.org/10.1007/978-981-16-3742-1_3

29

30

G. Shamilli

logic inherent in music alone, if it performs the same fundamental thinking procedures that were discussed above. Of course, these fundamental procedures are carried out in music through their specific functions, and this must be taken into account. Musicology deals with this logic of the connection between harmonic (in a broad sense) and constructive means. However, this does not prevent us from seeing behind the specifics of the material itself the fundamental procedures of meaning-setting, the bearer of which is a person, his thinking, which he embodies in every act of his activity. In other words, to see the specific architectonics of consciousness and the types of rationality found in language and music.

2 Hierarchical Structures Searching for connection between language and music, based on the development of F. Lerdahl&R. Jackendoffs theory, [2] reveals the hierarchical organization of two ‘domains’. Examples of such organization on the scale of a sentence in musical and verbal speech, given, for example, by Patel [3, pp. 204–223], can be multiplied indefinitely. It is enough to analyze the statement ‘the Sun ascended over the sea’ and then the chorus ‘Glory’ from the Opera ‘Ivan Susanin’. In the first case (a) we have a hierarchical organization of the parts of speech, a nuclear formula of which is the ratio of the ‘sun—ascended’, in the second (b)—temporal periods of musical text, the reduction of which shows the hierarchy of important high-level melodic phrase, reducible in the end to subject-predicate linkage S = P. The hierarchical organization as a dependency structure is convincing for both verbal and musical text. It is known that the hierarchy in music is due to the difference in pitch, as well as time (rhythmic) levels of melody (F. Lerdal and R. Jackendoff). At the same time, the hierarchy of syntactic units is not taken into account. A. Patel, who develops the line of generative theory, believes that “music does not have the same kind of dependency structure as language, where an incoming element demands a cognitive connection with a specific distant prior structural element (as in the relation between ‘opened’ and ‘girl’ in the above sentence)”. [Ibid]. A. Patel thinks that in music ‘certain pitches in musical sequences are considered structurally more prominent than others, forming an ‘event hierarchy’ that defines the structural skeleton of a piece [1]’ [3, p. 204– 223]. In other words, the tones highlighted in the pitch hierarchy, organize events that seem to be as hierarchical in terms of altitude as the tones themselves. However, there is also a hierarchy of syntactic units in music that does not directly depend on either the pitch or event hierarchy. If we accept Patel’s point of view about the absence of subordinate structures in music, we risk forgetting about the concepts of musical language (spatial and temporal grammars) and musical speech (sound sequence or processivity). The second one implies strict subordination of syntactic units (motif, syntagma, phrase, super-phrasal unity). If syntactic units of different levels form a taxonomic series (complete or relatively complete), we can talk about hierarchy in both verbal and

Hierarchical and Non-hierarchical Structures in the Language and Music

31

musical speech. M. Schruder, referring to the work of Prince [4, pp. 19–100] emphasizes the presence of ‘building hierarchical structures’ both in verbal language and music (parallels between motif, phrase, and section, on the one hand, and word, compound, and constituent/phrases, on the other) [7; 3, 9].

3 Subject-Predicate Linkage The study of verbal and musical structures shows that the unity of language and music is not limited only by a hierarchically organized structures. Rather, it depends on the types of organization of the subject-predicate complex, which present both— hierarchical and non-hierarchical relationships—in the language and music. This idea is developed on the material of classical Arabic language theory, which is organic for the group of Semitic languages as a whole, which historically did not operate with a concept adequate to our ‘sentence’ (it appeared later), as well as for the professional musical traditions of the Near East and Israel. Meanwhile, the study of verbal and musical structures in the different cultures shows that the unity of language and music is not limited by the hierarchically organized sequence (the concept of Lerdal—Jackendoff) or ‘event hierarchy’ [1]. Rather, it depends on the types of organization of the subject-predicate linkage, which present both—hierarchical and non-hierarchical structures in language and music. In my report, this idea is developed based on unmetered musical structures in Near Eastern professional musical traditions including Israel, on the one hand, and classical Arabic language theory, on the other hand, which according to Frolov’s research [8, p. 17—34] did not operate with a concept adequate to our ‘proposal’. The subject-predicate complex (S-Pc) in music is a connectedness of selected tones that form the relation of identity and contrast as the main mechanism of a sense-generation. Connectivity is implemented in different ways and forms different types of S-Pc. Using the example of the organization of a verbal phrase, it can be explained as follows: if the noun (‘sun’) and the verb (‘rose’) form two equivalent supports on which the phrase rests, as an ‘event’ of the first level, then the other events of speech occupy the following levels. The two parts of speech that form the subject-predicate linkage are realized through a relationship that is also organic for thinking with ‘sounds’. Music is an information system and has a universal property of distinguishing sounds, which covers the processes of processing and storing information. Singleness is the relation of the subject (S) and the predicate (P). Thus, the musical phrase rests on the selected sound heights. In European medieval music theory, such ‘pillars of Hercules’ (E. V. Hertzman) were defined by the terms finalis and repercussa. In the later functional theory of fret, the ratio of the selected elements often coincides with the ratio of the tonic (T) and the dominant (D). If we translate these functions into logical ones, then the tonic, as a function of the fret, will become the subject approved by the entire logic of the development of a musical composition, and the dominant acts as a predicate, since it does not coincide

32

G. Shamilli

with the subject and thus implements the relationship of identity and difference. These functions are found as the nuclear structure of speech after the reduction of the surface layer of the musical text, which is symptomatic in general for Western classical music but is also found everywhere in the East. For example, in the compositions of the maqom genre model, which belongs to the Central Asian classical music tradition, two equivalent fret supports are designated by the terms sarparda and m¯ıa¯ nparda, which perform the function of limiting the fret plane. They constitute the nuclear formula of a proposition, like the subject-predicate complex of a verbal language. The melodic utterance is generated by the process of synthesizing units, in which the smaller units are absorbed by the larger ones. In this sense, it is extraordinarily simple, because it reveals a hierarchically-conditioned organization. The vocal melody is sung against the background of the instrumental accompaniment and is defined by the term ‘line’, or hat.t.. As a super-phrasal unity, ¯

Fig. 1 Dar¯amad section ‘Introduction’. Part of Sarah.bor ‘Main message’. The composition Buzurk ‘Great’

Hierarchical and Non-hierarchical Structures in the Language and Music

33

a) The first syntagma (mm. 1 4).

b) The second syntagma (mm. 5 8).

Fig. 2 a, b Temporary reduction of two syntagmas of the first phrase. Dar¯amad section ‘Introduction’. Part of Sarah.bor ‘Main message’. The composition (maqom) Buzurk ‘Great’

it is sung in a verse (bayt) of two half-verses (mis.ra‘) and contains two phrases (phr) or nimhat.t., as well as a final textless vocalization (xang), which prepares the ¯ next, higher-pitched compositional section of the Saraxbor-i Buzurk ‘Main Message’ (Fig. 1). Each phrase (phr) consists of two syntagmas (stg), also strictly hierarchical and organized by the subject-predicate linkage S(d1 ) = P(g1 ) (Fig. 2a–b). The first of them begins with the pillar tone d1 and soon reaches the second one g1 (Fig. 2a, mm. 1–4), which is held on the second syntagma, ending with a pause (Fig. 2b, mm. 5–8). It is obvious that the event hierarchy, conditioned by the relation of identity and difference, is formed strictly within the relation S (d1 ) = P (g1 ) at the time of the change of pitches from d1 to g1 , at the border of m. 2–3. This gives rise to smaller syntactic units of the motif type, despite the fact that the rhythmic factor plays a secondary role in the derivation process, in other words, it does not support the contrasting zones of the utterance, preserving or almost preserving its pattern (Fig. 2a, mm. 2–3). And, on the contrary, the zones of high-altitude identity, when the movement has not yet reached equilibrium and stability, are accompanied by a change in the rhythmic (time) pattern (Fig. 2a, mm. 4 and Fig. 2b, mm. 1). The events that occur during the first phrase, during the ascending, returning, and finally hopping movement on the ladder of the Buzurk d1 –e1 –f1 –g1 tetrachord, organize the hierarchical structure of summation by synthesizing smaller units into larger ones. The synthesis process is endowed with temporal semantics since it is

34

G. Shamilli

Fig. 3 Hierarchical structures (motif- syntagma-phrase-proposition) of the sound process

eventful. Each element of the nuclear formula S = P organizes an event that modulates time. It is also expressed by a specific syntactic structure. Consequently, the basic form of thinking and the syntactic structures themselves are organically fitted to each other. The movement here is a measure of musical (artistic) time. It can be continued indefinitely, but this will not change the nature of either the subject-andpredicate linkage or the empirically given whole. Events organized by this kind of relationship of identity and contrast, in the process of which smaller units are ‘absorbed’ and synthesized by larger ones, eventually draw a hierarchical line (motifsyntagma-phrase-proposition) of the sound process (Fig. 3). The utterance and the events taking place in it are subject to a strictly linear process, which is like a smoothly moving geometric ornament on the walls of architectural structures. The meaning of the statement is generated by the hierarchical and complementary nature of the fundamental ‘part-whole’ relationship.

Hierarchical and Non-hierarchical Structures in the Language and Music

35

4 Non-herarchical Structure Meanwhile, not every relation of subject and predicate is implemented by the type S = P. One should be prepared for this when studying non-Western culture and the question—whether musical speech can have a non-hierarchical character, as a result of a different basis of mental operations other than S = P-becomes particularly acute. It is known that the Arabic language reveals such a type of connection between subject and predicate, which was defined by classical Arabic grammar as the support (’isnad) of the leaning (musnad) on that-which-supports (musnad ilayy-hi). “For Indo-European languages, the core predication formula is ‘S is P.’ It is backed by a universal intuition of a closed space, as exemplified by Euler diagram. The formal logic based on that type of the subject-predicate linkage may be called substancebased logic. The ‘S is P’ construction is not possible in natural Arabic. This language uses an alternative predication formula for connecting subject to predicate, which in the terminology of traditional Arabic grammar is designated as ‘S isn¯ad P’ where isn¯ad literally means ’leaning.’ This Arabic elementary proposition reflects the basic intuition of the flux, the intuition of continuous process of flowing. The logic based on of this type of subject-predicate linkage is referred to as process-based logic” [7]. This connection pulls the acting and undergoing, or the ‘speaker’ and the ‘spoken’ into a process of ‘speaking’, devoid of a temporal characteristic. In thinking with sounds, this logical intuition of wholeness is expressed in the most general terms by the absence of two equivalent pillars supporting the construction of a melodic utterance and, as a result, encircling the border. On the contrary, the selected sounds being different pitch pulled together in syntagmatic axis attitude motion-and-rest time, you must separate from another moment by a pause. This property deprives the internal content of the moment of temporality, tying the selected tones to each other by an indissoluble connection of a single event that ends with a caesura. Time occurs between such moments, and not within them, which is why it ceases to be linear. The second type of subject-predicate linkage, which builds a melodic line so that it is compressed into a ‘point’-process, forming as a ‘leaning-and-supporting’ relationship. This concept is borrowed from the study of as-Sibawayhi (d. 796), the author of the first classical Arabic grammatical theory, in which the concept of almusnad wa-l-musnad ilayh (‘based’ and ‘that-on-what-is-based’) is justified as ‘a name for a predicative combination’ [7, p. 23]. As-Sibawayhi proceeded ‘from the principle that the units constituting it, are the words,’ but himself, analyzing ‘specific deployments of the speech circuit, was faced with several cases in the next step of deployment instead of single words appeared predicative complex, included in the circuit and functioning as a holistic unit, the positional equivalent of a single word [Ibid].’ In music, the leaning-and-supporting relationship is extra-temporal, it does not contain signs of change, because the movement of sounds does not cause a change of place. Sound process without a meter (no matter what, accentuated or non-accentuated), respectively, without a metric cycle, loses the most important cognitive basis that

36

G. Shamilli

gives time temporality. The only measure of time is no longer movement (!), but the breath as an exhalation-and-inhale of the performer. Such music provided that it is realized according to the nuclear formula S ↔ P, or ‘subject (process) predicate’, is an open form from the point of view of form-making. It includes compositions of the genre taqs¯ım, dastg¯ah, etc. ‘Exhalation’ makes the appearance—‘movement’ of stitches, and ‘inhale’—their disappearance a pause. Movement-and-rest is drawn together in the one atom of time, and the sounds move within it as if they are crushing it from within. They are not endowed with temporality, they do not have their existence, so movement does not measure time. The movement between the ‘pillar and the ‘leaning’ tones represents a process that does not master time and hierarchical events, while the time atom (the ‘pillarand-leaning’ relationship) performs the function of place. Consequently, it is not movement that is temporalized, but space: the time atom (‘exhale-and-inhale’) plays the role of place and replaces space, that is, it becomes a function of form. On the contrary, in ac. B. Asafiev’s understanding of movement, form becomes a function of time, and this is the most important contrast between the two types of thinking— substantive realize as ininium-movere-terminus or i → m → t (ac. B. Asafyev, 1926) and procedural i ↔ t = m—temporality [5, 6]. This is why the type of subject-predicate linkage S ↔ P is based on temporal rather than spatial intuition of whole. The melody, which represents a series of temporary atoms, moves like a Zeno arrow, simultaneously resting in each ‘now’ equal to a musical event. If in the first case considered, for a subject-predicate linkage of type S = P, each stitch forms an event, and their sum forms the event hierarchy of the melodic line, then here they are non-hierarchical, and the eventfulness is nonlinear; it has the character of coherence. The transition from the stable tone to the unstable tone itself, as a process, becomes the boundary separating one S-P-complex from another. The function of such a boundary does not consist in separating one structure from another, it consists in contracting the initial and resulting states of the process. Let us consider in more detail the procedural organization of the whole by the example of the melody Dar¯amad ‘Introduction’ in the genre model dastg¯ah (Fig. 4). The metrically free melody in Mahmoud Karimi’s version (rad¯ıf) and Masudiyyeh’s notation (2000) is a five-phrase textless chant. It is part (g¯uša) of the normative structure (rad¯ıf) of the Š¯ur ‘Excitement’ composition (dastg¯ah). During a concert performance of a melody (aw¯az), the number of its phrases can be increased at the performer’s request, but the fact of improvisation does not affect the type of linkage S ↔ P that organizes it, it also does not and cannot affect the generative formula—for this you need to completely transform the performer’s consciousness, which is not possible in the conditions of traditional music making. This fact significantly affects the duration of each section of the composition in real physical time. The linkage is determined by the ratio of two high—frequency centers-temporary /unstable (f1 ) and permanent / stable (d1 ), the first of which serves as a ‘leaning-on’ tone, and the second—as a ‘supporting’ (Fig. 5).

Hierarchical and Non-hierarchical Structures in the Language and Music

37

Fig. 4 The melody of Dar¯amad ‘Introduction’. The normative structure of the dastg¯ah Š¯ur composition

Fig. 5 The melody of Dar¯amad ‘Introduction’. The normative structure of the dastg¯ah Š¯ur composition. Third phrase

Thus, we have a temporary unstable pitch center (up-c) and a permanent stable pitch center (sp-c); the nature of their connectivity forms the type of subject-predicate complex. The analysis of the melody shows the non-linearity of the derivational process due to: (1) the procedural type of the subject-predicate linkage S ↔ P as the nuclear formula of the utterance; (2) the lack of a process for synthesizing smaller units into larger ones; (3) a series of non-hierarchical relations of musical speech units.

5 Conclusion It is concluded that there are two parallel linkage process for the music, one of which is type S = P and another—type S ↔ P that depend on the quality of the subjectand-predicate linkage and realizes as two formulas of prosses: (1) i→m→t (ac. B.

38

G. Shamilli

Asafyev, 1926) and (2) i↔t = m—temporality (G. Shamilli, 2017). In this paper, the subject-predicate construction in music is defined and the possibility of its nonhierarchical organization is proved. This conclusion has deep-reaching strategic implications and points to the type of rationality shown in different segments of culture.

References 1. Bharucha, J.J.: Event Hierarchies, Tonal Hierarchies and Assimilation: A Reply to Deutsch and Dowling. In: Journal of Experimental Psychology, General. № 113, pp. 421—425. (1984). 2. Lerdahl, F.: Jackendoff R. A. Generative Theory of Tonal Music. Cambridge, MA: MIT Press. (1983). 3. Patel, A. D.: Language, music, and the brain: a resource-sharing framework. In: P. Rebuschat, M. Rohrmeier, J.Hawkins, & I. Cross (Eds.), Language and Music as Cognitive Systems. Oxford: Oxford University Press. (2012). 4. Prince A.: Relating to the Grid. In: Linguistic Inquiry. №14. pp. 19—100. (1983). 5. Shamilli, G.B. Filosofiya muzyki. Teoriya i praktika iskusstva maqam [Philosophy of Music. Theory and Practice of the Art of Maqam] (in Russian). M.: YAzyki slavyanskih kul’tur; Sadra. — 552 s.: not., il. — Seriya «Filosofskaya mysl’ islamskogo mira: Issledovaniya». — T. 5. 6. Shamilli, G.B. Open form as a metaphor, term and phenomenon / “Scattered” and “collected”: cognitive techniques of Arab-Muslim culture: a collective monograph. - The series “Philosophy of the Islamic world”. Section “Research”. Vol. 10. - Ed. by ak. A.V. Smirnov. - M.: Languages of Slavic cultures, 2017. - pp. 95–220 - ISBN 978-5-906859-38-9. 7. Smirnov, A.V.: Proposition and Predication. In: Russian Studies in Philosophy, 56:3, pp. 156– 177, (2018). DOI: https://doi.org/10.1080/10611967.2018.1459375 8. Frolov, D.V.: K voprosu o ponyatii predlozheniya v arabskoj grammatike [About the question of sentence in Arab grammar] (in Russian). In: Frolov, D.V. Arabskaya filologiya: Grammatika, stihoslozhenie, koranovedenie: stat’i raznyh let. M.: YAzyki slavyanskoj kul’tury. pp. 17—34. (2006).

Metaphor as a Structural Principle of Modern Musical Notation Anastasia Gundorina

Abstract The article explores modern musical notation as a semiotic system with special principles of structural organization. Modern musical notation reflects today’s musical practice. Its primary function is communication, i.e., storage and transmission of information. We approach notation as a universal writing system that reflects principles of human thinking. The post-nonclassical system of perception of the world and its reflection is marked by a constant dialogue between the two global sociocultural paradigms. This creates the conditions where modern musical notation is considered as an intersection and complex interaction of structural mechanisms that include the formation of meaning and symbols inherent to canonical and noncanonical cultures. In this regard, we developed a hypothesis that musical writing reflects the universal laws of human thinking and follows a complex dialectic of conscious and unconscious, explicit and implicit, universal and individual. The principle of dialogue made it possible to adjust the research methodology—new graphic symbols are explored through their structural similarity with metaphor. The focus of modern scholars on the interactive dialogue between subject and object has shifted the momentum of psychological research towards metaphor. Taking a psycholinguistic perspective, metaphor reveals its meta-universal structural foundations since metaphor is a manifestation of polysemy common for all languages. The structural analysis of musical graphemes based on the methods of metaphorical modeling resulted in the typology of modern notation symbols. It also allowed to trace the genesis of graphic notation—a very special phenomenon in music of the second half of the twentieth century. The article highlights the anthropomorphic nature of modern notation and provides evidence that it has a deep level of symbolic meanings associated with thinking and the psychology of human perception. Keyword Modern musical notation · Symbols · Metaphorical thinking

A. Gundorina (B) Gnessins Russian Academy of Music, Moscow, Russia © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 T. Chernigovskaya et al. (eds.), Language, Music and Gesture: Informational Crossroads, https://doi.org/10.1007/978-981-16-3742-1_4

39

40

A. Gundorina

1 Introduction Musical thinking of the 20th-early twenty-first century composers triggered a dramatic change in music. No less dramatic was their influence on its visual representation. The system of musical notation developed a great many new notation signs (symbols). Apart from being a major add-on to the traditional European five-line staff, these changes also led to the emergence of new kinds of notation, e.g., graphic or verbal. Contemporary notography stands high on the research agenda in musicology. Such is the state-of-the-art of modern musical notation that it is no longer satisfied with mere observations or classifications. It is time to consolidate all the available data in order to identify the universal semiotic foundations of the emerging system. It is impossible to explain the essence of modern musical notation and its systemic features without taking into account the global cultural, cognitive and semiotic context. Therefore, at the outset, it is expedient to outline the reasons for notational changes that took place in the twentieth century. First, the twentieth century witnessed a dramatic change in the very core of culture—centricity, linearity, and genealogy lost their relevance. Instead, the twentieth century developed a new type of cultural entity that was bound to become one of the basic postmodernist concepts—the rhizome. Second, modern musical notation reflects conflicting and, at the same time, interdependent processes immanent to post-modern environment. The changes to the notation system are the outcome of the universal mindset of the era which is often referred to as “the end of the age” and the “neo-archaic”. This combination gives ground to regard current environment as a return to the early stages of human history of a completely different quality indicative of the development of the new planetary culture. Third, current environment adjusts the research perspective towards a more human-centered approach. This substantiates the methodological value of Yuri Lotman’s dichotomy of canonical vs. non-canonical cultures [9]. Modern cognitive science offers a corresponding dichotomy of models in musical thinking: discrete (unconscious) and non-discrete (rational, conscious). The two models laid the foundation for modern psycholinguistics. To us, the conscious/unconscious dichotomy is capable of providing a comprehensive and systemic exploration of what is going on in modern musical notation as well as revealing its structural framework as an order made of chaos. One of the most authoritative researchers of modern notation, Erhard Karkoschka, stated that: “The technical possibilities of a notation system influence the act of composing—the entire musical way of thinking of all musicians—so that the aural image of a musical work in every epoch is characteristically related to its visual configuration” [8, p. 1]. Elena Dubinets provides the following definition of modern notation: “In a broad sense, modern notation can be called a visual representation of the composer’s idea, since it is related to the composition methods used in the work” [6, p. 10].

Metaphor as a Structural Principle of Modern Musical Notation

41

Today, researchers tend to agree that notation, due to its specificity, has to be approached from two angles: as a written language of music and as a means of expressing musical thinking. One of the first experts to recognize that was a composer and a music critic Hugo Cole [4]. The main disadvantage of the existing classifications of modern notation as a semiotic system is their one-dimensionality due to the emphasis on the properties of the signified. This principle classifies legends according to the types of information they represent (pitch, rhythm, dynamics, accent, etc.). The primary goal of research in modern musical notation is to clarify the type of graphic symbols. This necessity arises due to a shift in the focus of attention that now concentrates on the properties of the signifier—the visual/graphic model of a symbol. High on the research agenda is the identification of the underlying principle between the signified and the signifier. For these reasons, new approaches to musical notation need exploration and clarification. A possible framework for this kind of analysis is offered by modern concepts of metaphor.

2 Metaphor in the Focus of Cognitive Research Modern semiotics that borrows insights from phenomenology and hermeneutics has taken a new, epistemological perspective on the semantics of a symbol, i.e., the perspective of a person’s inner world. Never before was metaphor in the spotlight of psychological exploration. The reason is the focus of modern scholars on the interactive dialogue between the subject and the object. Psychology and linguistics identified meta-universal structural foundations of metaphor. Metaphor is a manifestation of polysemy common for all languages [11]. Absence of polysemantic words would require a huge volume of memory to store an enormous number of words. The reason is simple—every object would be called a different name. From the modern linguistic perspective, metaphoricity is the most effective pattern of behavior due to its empirical foundations where the language is on constant watch for new, emerging phenomena. The principle of transferring meanings from the familiar to the unknown is a natural cognitive operation as well as a tool to understand new information. In this respect, of special importance are the contributions of Russian scholars engaged in interdisciplinary research with a special focus on the functions and interaction between left and right hemispheres of the brain. They regard the brain as a dual system that unities two opposite personalities engaged in a constant dialogue. The theory underpinning the concept of dialogical artistic thinking was substantiated by Mikhail Bakhtin and Lev Vygotsky. Significant contributions to the theoretical development of the concept were made by Vyacheslav Ivanov, Yuri Lotman and Vladimir Bibler [3]. Roman Jakobson’s neurosemiotics encompasses the advances of neurophysiology and neurolinguistics that study brain asymmetry and explore the internal

42

A. Gundorina

“dialogue of the brain” [7], then in musicology this approach was adopted by Vyacheslav Medushevsky who developed the concept of intonation-based dramaturgical form [10]. Another research avenue in brain asymmetry studies is a metaphorical transfer [3]. The hypothetical foundation for the metaphorical transfer is the disclosure of past and forgotten emotional and/or physical experiences. The unconscious foundations of metaphorical thinking are opened up by psychoanalysis. This draws special attention to the right-brain activity. Tatyana Chernigovskaya and Vadim Deglin found the following: left-brain and right-brain metaphor decoding analysis shows that, unlike idioms, metaphors are subject to decoding. Metaphor interpretation seems to be the responsibility of the left brain and its linguistic capabilities. However, its resource is not sufficient to perform the purely left-brain task of analyzing new textual information. This is when the right brain takes a lead. Interestingly, the right brain does not analyze metaphors element-wise. It uses a comprehensive, gestalt-based approach [3]. Research in neuropsychology and psycholinguistics shows that heterogeneity of thinking is related to functional specialization of brain hemispheres: “These semiotic poles create conditions where the dichotomy of poetry/science may be paraphrased as metaphor/ syllogism” [3, p. 235]. Research outcomes obtained by psycholinguists correlate with the basic concepts of philosophical anthropology, namely, the ideas of Aron Gurevich about archaic mentality. According to Gurevich, archaic mentality is cyclical and filled with particular events of the time (not time alone). Archaic mentality prefers old, tested, known experiences and has no interest in anything new, apriori, existing beyond and before any experience. This is a right-brain mindset [3, p. 249]. The left brain is the realm of scientific thinking and formal logic. It is a center for processing new data. Yet, no matter how hard it tries, it does not remember idioms and cannot decode metaphors unless the left brain comes to rescue. This is because finding solutions to these tasks is not about data structuring alone. However, the left brain is really good at solving syllogisms. Here, the solutions are based on a familiar (learned) algorithm. The left brain does not try to check if it is really so. Neither does it try to recall any past experiences. It is a formal solution with the focus on not simply new but also rational information. In its extreme form, the left brain engages in scholasticism and “the glass bead game”. To sum up, heterogeneity of thinking arises due to functional specialization of brain hemispheres. Right-brain thinking should by no means be considered ineffective or inferior. To state that would be equal to stating that traditional or child thinking is underdeveloped or backwoods. Like the left brain, and, at times, even more so, it is an agent of creativity. Its realm, as regards the dichotomy in question, is poetry [3, p. 249–250]. Below are the conclusions about metaphorical thinking that we made based on the research outcomes from psycholinguistics and psychophysiology:

Metaphor as a Structural Principle of Modern Musical Notation

43

– visual and conceptual memory is stronger than linguistic memory; – the meaning of the emerging information pattern reveals, or, rather, organizes itself in line with Hermann Haken’s view that synergetics is very closely related to brain and psyche as well as ideas and concepts of Gestalt psychology. We suggest taking a look at modern musical notation from the perspective of metaphor and its structural principles. However, given the dominance of the visual factor in the phenomenon under study, we will focus on a visual, not a linguistic metaphor. The feasibility of this approach is obvious. It allows to classify the existing array of modern notography into three major types.

3 The Principles of Generating Visual Metaphors in Modern Musical Notation 3.1 The First Type of Symbols A wide range of symbols used in modern musical notation has a subset that, presumably, developed due to the natural transformative power of note-taking. However, the underlying structural principle of modern notation is different. The transformation of traditional symbols results in the adjustment of visual images. It targets the performers’ instinct or intuition that allows them to capture the essence of the distorted original structure. Modern notography offers many examples where graphics dominates and, surprisingly enough, embraces the new concreteness and the old abstractness of symbols. Thus, the new symbols are based on metaphor as a structural principle which enriches idioms with a new, previously extrinsic meaning. At this point it is expedient to provide evidence in support of our hypothesis. The array of pitch symbols has not expanded dramatically due to sonoristic tools, yet, this area is not devoid of new developments. First, the sound system has expanded due to the introduction of quarter tones. The visual representation of microtonal notation has a large number of options. Figure 1 below shows a comparative table of microtonal notation found in the scores of K. Penderecki, Z. Penherski, E. Denisov and S. Gubaidulina [1, p. 168]: Apparently, the two systems have something in common. For example, a quartertone sharpening is the same in both approaches. Penderecki and Penherski alike use the “unfinished” sharp symbol to denote a higher pitch, while Gubaidulina and Denisov denote a three-quarters tone sharpening by an “overdone” sharp symbol that shows the necessary number of quarter tones. The logic behind symbols that denote a lower pitch in Penderecki and Penherski’s systems is as follows: a filled-in flat symbol is perceived as something smaller than a plain flat symbol due to the similarity with the time value of notes (Fig. 2).

44

A. Gundorina

Fig. 1 Comparative table of microtonal notation. © Copyright 2001 by E. I. Aronova

Fig. 2 Comparison by analogy: Note value and alteration

Fig. 3 Symbols of the highest and the lowest sound

Denisov’s system is based on the quantitative aspect of quarter tones. Through a metaphorical transfer, it combines traditional pitch symbols with rhythmic symbols. Another original sonoristic technique are the symbols denoting an extremely high or low sound (Fig. 3): The visual image is so vivid and convincing that its meaning is transparent and this transparency eliminates the chance of multiple interpretations. As the evidence shows, composers are unanimous about denoting the highest and the lowest pitch. The biggest changes in modern notation concern time values. Here, the visual images are diverse, yet, they show a common trend. These are metaphor-based images that use rhythmic symbols to denote a dynamic meaning. Similarly to different aspects of dynamics, these compound elements regulate agogics. These are the symbols of retardation or acceleration which have a visual association with the graphic symbols of opening up or closing. Different composers use different notation to gradually speed up or slow down individual sounds or motives. The examples below, found by Armin Köhler, are quite informative as regards a temporal aspect of performance (Fig. 4). The examples above reveal some universal constants. The diversity of images has a shared psychological and physiological foundation. It denotes an increase or a decrease of intensity. It is visualized as a change in the volume of energy by an opening-up angle, an upward-looking or a downward-looking line.

Metaphor as a Structural Principle of Modern Musical Notation

45

Fig. 4 Acceleration and deceleration of tempo in modern notation1 1

The examples (Fig. 4) are taken from the monograph by Ekaterina Aronova [cit. ex. 1, p. 129].

46

A. Gundorina

3.2 The Second Type of Symbols Another group of modern notation symbols is based on metaphorical transfer “by analogy". Their main purpose is to trigger a certain emotional or aesthetic effect. In other words, they are designed to generate a certain emotion in the performer. These symbols are formed by translating traditional musical terminology (for example, dolce, giocoso, ect.) into a graphic form of expression. One of the examples of such systems is Violin Concerto (1963) by Bogusław Julien Schaeffer. See Fig. 5 for some of the symbols used in his work. Next to each grapheme, the composer gives an explanation of the nature of the sound. When you visually perceive these signs, their meaning becomes clear even without knowing the author’s comments since they appeal to human universals—the memory of the race. Inside them is the figurative and objective memory of nature itself. The principles of perception are based on an unconditional response to the dichotomous differences between soft and rough, quiet and loud, high and low, etc. The second type of symbols also includes clusters with numerous ways of notation. The new musical notation system is by no means an abstract, rational invention of new graphic symbols. They are underpinned by universal laws of human reasoning and reflect the complex dialectics of conscious and unconscious, explicit and implicit, common and individual. This universal mechanism is metaphor. The above analysis shows that modern musical notation has a psycho-semiotic nature. The obtained evidence allows to draft an approach to the systemic exploration of musical notation which is immanent to its nature. The starting point of this approach is to identify the architypes of the collective unconscious within the structure of metaphor. The architypes may either refer to the natural, eidetic “protolanguage” or to the linguistic code of “memory of culture”. The former—the memory of life—uses an unconditioned response. These are the signs in Sect. 3.2. The latter— the memory of culture—a conditioned response / a reflex. The latter type embraces all the notational metaphors given above in Sect. 3.1. Moreover, there is another, third type of semioticity. It is very much in line with modern metaphor, its specific functional and structural features. Modern metaphors are marked by highly complex and intense processes taking place in the metaphorical semantic field. Hence, the new semiotic entity needs clarification and adjustment through a new concept—the rhizome. The rhizome is an artistic integrity inherent to the modern world order and, for this reason, to the dominant type of integrity.

Fig. 5 Bogusław Julien Schaeffer. Violin Concerto (1963). Graphical notation of sounds © Copyright 1965 by Polskie Wydawnictwo Muzyczne

Metaphor as a Structural Principle of Modern Musical Notation

47

3.3 The Third Type of Symbols Modern musical notation as well as modern music in general is marked by a phenomenon that, a priori, can only be explored within the synergetic methodological framework. This is the phenomenon of graphic music. It has already been subject to scrutiny within the scope of modern attitudinal issues. It has also encouraged self-sufficient development of spatial and visual aspects in music. Synthesizing music and visual art, the creators of graphic notation invented a fundamentally new semiotic system, a new type of language that tried to free the “written” music from the shackles of traditional musical notation, understandable only to musically educated people. The authors of such notation sought to make its “external” appearance less exotic for the average listener. The role of these scores is the same as that of abstract painting: to generate subjective associations. Graphic notation opens up numerous opportunities for improvisation. The audible outcome of graphic scoring is, in essence, an improvisation on the visual / plastic image. However, in this case the listener turns into a beholder. One of the first composers to introduce graphic notation was Earle Brown. His December 1952 (see Fig. 6) is a classic example. The composer gives a detailed description of how he created his work: “I had very much the impulse to do something in “our kind of music”, which would have to do with this highly spontaneous performing attitude—improvisational attitude, that is—from a score which would have many possibilities of interpretation. < … > I considered this kind of thing to be a mobility, which is to say a score that was mobile—a score that had more than one potential of form and performance realization” [2, p. 1]. A modern psychoanalyst will probably immediately pay attention to the homogeneity of geometric shapes—short lines, their accentuated rectangular conjugation, and interpret the content of the composition through the deep initial sense of the entire set of archetypes: courageous, rational, aggressively cold. Each performer/performers will create their own unique piece based on improvisation. The listener will probably look for metaphorical analogies to this work in surface structures of “cultural memory". In this case, the possible archetypes are the Gothic polyphony in the form of an isorhythmic motet, Paul Hindemith’s music, Anton Webern’s serial works, etc. Another illustrative example is Imaginary Music by Tom Johnson which comprises 104 illustrations featuring music notation symbols. One of the illustrations—Half-Note Field—is based on the graphic image of a half note that resembles a balloon or a flower on a long stem (Fig. 7). It embodies the principle of self-similar types of fractality. The image repeats itself over and over again and creates an immediate impression of harmony, tranquility, uniformity or eternity. The illustration may seem simple and unsophisticated, yet, it is rich in musical metaphors and multiple meanings. A longer and a more focused look will create energy vibrations typical of truly contemplative music from The Prelude in C Major by J. S. Bach and Robert Schumann’s Eusebius to patterns and additive structures typical for minimalists, including Tom Johnson.

48

A. Gundorina

Fig. 6 Earle Brown. December 1952. © Copyright 1961 by Wise Music Group

Fig. 7 Tom Johnson. Imaginary Music. Half-Note Field (1974). © Copyright 2001 by Tom Johnson. Editions 75

This type of graphic notation leads to a preliminary conclusion that it is reflective of a systemic character of the new music creativity. Fractality indicates the presence of certain universal meaning-making structures that are capable of self-organization and facilitate comprehensive perception. Along with that, such approaches to decoding new forms of music are referred to as special systems, i.e., non-linear, acentered, non-final, variable, etc.

Metaphor as a Structural Principle of Modern Musical Notation

49

It is impossible to grasp the essence of current developments and describe the new structural model without postmodern methodology, namely, the understanding of the updated dialectics of inside and outside in the new semiotic environment. Selforganization of human agency in the philosophy of the late 20th-early twenty-first centuries phases out the dichotomy of self/ other and us/them. It is replaced with the new opposition of exteriority and interiority. The former, according to Michel Foucault, is marked by permanent movement, while the latter resembles ripples on the water, it is the fold of the outside. According to Gilles Deleuze’s nomadology, the interior is the interiorization of the exterior through folding and reduplication. This explains why the development of subjectiveness is described by such notions as reduplication, double, and copy. This process is never-ending and its interpretation includes such notions as lot (dice), game, and lottery. Multiple connotations are reflective of a gradual phase-out of identity. Behind this process is depersonalization (almost complete) and The Death of the Author declared by postmodernists. This phenomenon is related to the reflection about the loss of the canon as an ontological prototype. The outcome is one of the basic concepts of postmodernism, i.e., the simulacrum2 (the copy of a copy). Apparently, this category also embraces graphic music compositions. This is so obvious that requires no further proof. No graphic music composition claims to be original. At the same time, it bears likeness to children’s drawings, graphs, and diagrams. The very notion of the simulacrum is crucial for our study because of the specific cognitive process which uses a self-explanatory algorithm. Modern philosophy treats simulacra as empty symbols that appear due to the radical denial of logocentricity and the very idea of reference. Hence, from this perspective, any graphic music composition has to be referred to as a play on an array of connotations, a goal in itself. We are in search of a logical foundation of current structural changes. In view of the modern dialogue between the implicit and the explicit, a more effective approach would to juxtapose European and non-European, or, Western and Eastern cultures. Postmodern philosophy noticed this juxtaposition a long while ago. This led to the emergence of yet another opposition—the tree and the rhizome.3 The tree is the embodiment of determinism, linearity, the only possible interpretation, while the rhizome is an acentered, nonhierarchical system marked by antigenealogy [5]. The basic quality of the rhizome is its heterogeneity without sacrificing integrity. It is defined as: “A semiotic chain is like a tuber agglomerating very diverse acts, not only linguistic, but also perceptive, mimetic, gestural, and cognitive: there is no language in itself, nor are there any linguistic universals, only a throng of dialects, patois, slangs, and specialized languages” [5, p. 7]. The rhizome is marked by polymorphy which arises due to the lack of a universal semantic center as well as the

2

From Latin simulacrum which means “image” and comes from the Latin verb simulo, “make a pretense”. 3 The term rhizome (from French Rhizome which means “subterranean stem”) was introduced by G. Deleuze and F. Guattari to denote an essentially non-structured (acentered) way of an entity’s being. The tree is a metaphor for the classical tradition.

50

A. Gundorina

centralizing universality of the (the General as metaphorically put by Gilles Deleuze and Pierre-Félix Guattari) [5, p. 21]. According to G. Deleuze and F. Guattari, the rhizome is modeled as a nonequilibrium entity very much similar to nonequilibrium media studied by synergetics. The rhizome is not subject to organization; it is marked by permanent creative movement. In such conditions, the source of transformation is not an external reason. It is the immanent instability (non-finality) of the rhizome rooted in the energetic potential of its self-variation. To G. Deleuze, the rhizome is neither stable, nor unstable, rather, it is metastable and contains potential. According to G. Deleuze and F. Guattari: “It has neither beginning nor end, but always a middle (milieu) from which it grows and which it overspills” [Ibid.]. Formally, the rhizome does not allow to draw a clear dividing line between the External and the Internal: “The rhizome operates by variation, expansion, conquest, capture, offshoots” [Ibid.]. The rhizome is, in essence, a model of the new type of symbology. What makes it fundamentally different from a symbol or metaphor is the absence of a deep structure or a genetic axis (canon, rule, prototype), i.e., its antigenealogy. The formation of the rhizome is related to such notions as deconstruction or dissemination/dispersal. This concept was introduced by Jacques Derrida in an attempt to adjust the popular term “pluralism”. He defines it as a play on the accidental formal resemblance related to the dissemination of semes—the smallest units or germs of meaning that generate polysemy. The rhizome may be compared to a hypermetaphor with a multi-rooted system that creates a problematic field determined by the phenomenon of chance. The phenomenon of chance reveals itself in such postmodern concepts as nonsense and a paradoxical element. Dissemination allows for multiple versions of any single experience. The use of different codes in an ironic way creates an array of varied textual interpretation. In the context of graphic music, this element of chance is the actual state of the recipient, i.e., a purely situational aspect of perception. Hence, the rhizome encodes the possibility of multiple contexts of perception due to the multiple ‘strings on the finger’ it contains. The actual psychological and emotional state helps to identify a specific point among all the versions of the rhizome environment in the array of possible textual meanings. This point is called a plateau, defined as a unilinear stage of the rhizome. The evidence for the foregoing discussion is easy to find. The simple graphic music piece Half-Note Field by Tom Johnson that we commented on earlier is a very telling example. The array of meanings that emerge at different moments of perception of a graphic construct is based on the paradoxical element that triggers the recipient’s (listener/viewer) perception. This element making possible connotations a matter of chance is pre-programmed in the postmodernist opposition of nonsense/meaning. The latter is a metaphor in all its glory with a dramatic mismatch of the apparent and hidden meaning. Nonsense manifests itself in the ostentatiously visual perception of, seemingly, auditory art with half notes depicted beyond the staff as balloons or dandelions on fragile stems. The experience of nonsense is a bifurcation point. It phases out identities through turning visual symbols into the copy of a copy—something that does not have a single prototype.

Metaphor as a Structural Principle of Modern Musical Notation

51

Consequently, on the one hand, the visual image of specific objects seems to be devoid of its own meaning of the word. The figurative meaning they pertain is perceived as a simulation, a mask. On the other hand, the problematic field generated by the opposition of nonsense/meaning is the result of the fluctuating influence of chance—ever-changing psychological and emotional states or different perception patterns. Every possible version of the emerging musical meaning is a plateau or a meaningful fragment of the rhizome available here and now. If the beholder of a music piece is in good spirits and a balanced state of mind, their focus will be on the modes of aeriality, lightness and euphoria. If the state of the beholder is quite the opposite—sadness, melancholy, frustration—their rhizome environment will trigger the opposite meaning, that of banality, commonness, sameness, monotony, etc. Moreover, a particular psychological and emotional state as a chance that facilitates a comprehensive perception of a graphic music composition may reverse the trajectory of similar images. One possible association with multiple half notes is eternity, with notes streaming beyond the horizon. Another association is an impending natural disaster—an avalanche or a mudflow. Yet another possible interpretation based on a random visual association is that of aeriality, lightness and weightlessness. At this point the combination of similar images may turn into a cloud that, in the twinkling of an eye, will develop a different interpretation.

4 Conclusion In conclusion, it should be emphasized that modern notation is part of the challenge posed by the language of modern culture, which requires a methodology that could adequately explain its internal systemic properties. It was found that at the heart of the new graphic language of modern music lies the structural principle of metaphor and numerous implementations of metaphorical transfer. The latter includes polysemy in the rhizome of graphic music compositions. These findings allowed us to substantiate the classification of the system of modern musical notation. Without rejecting the existing five-line staff, the modern notation system is gradually developing new approaches to graphic recording of music—they stand out in the chaos of modern graphic symbols as cosmogenic structures of the future.

References 1. Aronova, E. Graficheskie obrazy muzyki: kul’turologicheskij, prakticheskij i informacionnotekhnologicheskij vzglyady na sovremennuyu notaciyu. Sibirskoe universitetskoe izdatel’stvo, Novosibirsk (2001). [Graphic images of music: Culturological, practical and informationtechnological views on modern musical notation. Siberian University Publishing House, Novosibirsk (2001)]. (in Russian).

52

A. Gundorina

2. Brown, E. On December 1952. American Music. Vol. 26, no. 1, 1–12. [1970; Transcription of audio]. ([1970] 2008). 3. Chernigovskaya, TV. CHeshirskaya ulybka kota SHryodingera. YAzyk i soznanie. YAzyki slavyanskih kul’tury, Moskva (2017). [Cheshire smile of Schrödinger’s cat. Language and consciousness. Languages of Slavic culture, Moscow. (2017)]. (in Russian). 4. Cole, H. Sounds and Signs: Aspects of Musical Notation. Oxford University Press, London. (1974). 5. Deleuze, G. and Guattari F. A Thousand plateaus: capitalism and schizophrenia. Trans. by Brain Massumi. The. University of Minnesota Press, Minneapolis (1987). 6. Dubinets, E. Znaki zvukov. O sovremennoj muzykal’noj notacii. Gamayun, Kiev (1999). [Signs of sounds. About modern musical notation. Gamayun, Kiev (1999)]. (in Russian). 7. Jacobson, R. Poetry of grammar and grammar of poetry. Lingua. Vol. 21. pp. 597-609. doi.org/https://doi.org/10.1016/0024-3841(68)90079-X (1968). 8. Karkoschka, E. Notation in new music: A critical guide to interpretation and realization. Universal Edition, Tonbridge (1972). 9. Lotman, YuM. Kanonicheskoe iskusstvo kak informatsionny paradox. In: Problema kanona v drevnem i srednevekovom iskusstve Azii i Afriki. Moskva, pp. 436–441. (1973). [Canonical art as an information paradox. In: Canon and art in Antiquity and Middle Ages: Collected papers. Moscow. Pp. 436–441 (1973)]. (in Russian). 10. Medushevsky, V. Dvoystvennost muzykalnoy formy i vospriyatie muzyki. In: Muzykal’naya psihologiya. Pp. 84–91. Moskovskaya gosudarstvennaya konservatoriya imeni CHajkovskogo, Moskva. (1992). [The duality of musical form and the perception of music. In: Music psychology, pp. 84–91. Moscow State Tchaikovsky Conservatory, Moscow (1992)]. (in Russian). 11. Ullmann, S. Semantic universals. Universals of Language, 217–262. MIT Press, Cambridge, (1963/1966).

Language–Music–Gesture Onstage

Reading Opera: The Verbal versus the Non-verbal in Opera and Music Theatre in the Late 20th and Early 21st Centuries Olga Manulkina

Abstract Opera and music theatre in the last few decades with their great variety of composers’ and directors’ strategies should be regarded as a broad, interdisciplinary field within which the borders between different art forms and genres are often difficult to discern. Opera rejects the verbal libretto, the theatrical stage, musical instruments; the genre-specific definition of a work as “opera” is given to dramatic performances; “documentary music video theatre” arises. The issue of methodological autonomy in studies within this field is becoming ever apparent, making imperative the need for common analytical tools. This article attempts to find such tools. I examine ways of approaching the verbal text in works by Steve Reich, Vladimir Rannev and opera by Robert Wilson and Philip Glass, where the musical, verbal, and theatrical and/or video-texts were all created simultaneously, likewise opera productions by Dmitri Tcherniakov, Peter Konwitschny, and Anne Teresa de Keersmaeker. The works are placed on a spectrum ranging from strongly emphasized to weakly emphasized verbality. Examining new music theatre and Regieoper together allows us to isolate on inter-relations and common processes. This article relies on opera studies, new music theatre studies, and works by Hans-Ties Lehmann and Heiner Goebbels on postdramatic theatre. Roman Jakobson’s model of the functions of language is introduced as a means to better define the causes and effects and strongly or weakly emphasized verbality and the organization of different codes within a singular theatrical code. Keywords New Music Theatre · Regieoper · Postdramatic Theatre · Steve Reich · Dmitri Tcherniakov · Vladimir Rannev

1 Homo Legens, or Texts and Languages on Opera Stage The synthetic nature of opera and, generally speaking, of music theatre which unites different art forms and thus involves varying communication systems, is what determines a performance’s impact on its listeners and viewers and the complexity of its O. Manulkina (B) St Petersburg University, 7/9 Universitetskaya Emb., 199034 Saint-Petersburg, Russia © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 T. Chernigovskaya et al. (eds.), Language, Music and Gesture: Informational Crossroads, https://doi.org/10.1007/978-981-16-3742-1_5

55

56

O. Manulkina

reception. Scholars and practitioners, in trying to make sense of the diverse strategies in opera and music theatre employed by composers and directors in recent decades, work with a broad interdisciplinary perspective. Opera studies includes the theatre performance in its purview [8, 16, 17], while postdramatic theatre studies explore the process of “musicalization” [9, 13]; studies in the recently established sphere of “composed theatre” [18] adopt an interdisciplinary approach from the outset, while new music theatre studies [3, 24] demonstrate “a move from consideration of new music theatre as a closed genre with a distinct lineage and towards a more lateral and malleable interpretation open to volatile interdisciplinary relationships and contexts” [21, p. 79]. The necessity for further development of common methods, theoretical approaches [11, p. 73] and research tools within this interdisciplinary field though is still evident today despite the growing number of scholarly works. In this article I explore the ways music theatre today works with verbal texts, endowing them with additional functions compared with previous eras, or, conversely, depriving them of their traditional functions, and also the trends which these phenomena can be linked to. Throughout the twentieth century, and ever more dynamically during the latter half of the twentieth and in the beginning of the twenty first centuries, the nature, mechanism and degree of the listener’s understanding of the language(s) of opera have been changing: the original verbal language of a given opera (Italian, French, German) and the musical opera language in general, as well as operatic poetics and form in a broader sense [4]. The number of listeners and viewers who know these languages is naturally decreasing relative to the expanding global opera audience. The implication of this is that the relationship between music and words for this new and wider audience is different from it was for a nineteenth century audience [24, p. 78–95]. The modern listener is equipped with a libretto in a booklet (in the original language and in translation) and—what is more significant here—they are able to view subtitles with the translation.1 The text of the libretto intended for reading by the audience is included in the complex of elements of the opera performance, adding not only textual details that sometimes are indiscernible by ear even to a listener who knows the language, but also one more channel of information and communication. New technologies in opera recording and broadcasting have created a new level of accessibility of opera through the internet. Hence, new practices of watching and listening opera at home, on screen, in the same manner as films and TVshows (mandatorily featuring subtitles in the original language or in translation) have arisen and reached unprecedented levels of popularity in the year of the pandemic, and especially in the spring of 2020. 1

A note on terminology: in this article, I use “subtitles” both as a blanket term for any text that accompanies a film or theatre production and as a specific term where its specific meaning is appropriate. I employ the terms “surtitles,” “intertitles,” etc. where I mean to refer to a certain mode of such text reproduction. In general, the theatres use surtitles, while, for example, in the Metropolitan Opera there are subtitles for individual use; the device is built into the back of the chair in front of the viewer.

Reading Opera: The Verbal versus the Non-verbal in Opera …

57

The changes outside opera theatre should also be taken into account, and in particular, the quickly emerging and rapidly evolving practices of producing and consuming the written word in all sorts of gadgets: writing and reading texts in instant messengers and twitter, in chats, comments on social networks cannot but influence how film and theatre are produced. For example, on-screen chats that today appear in TV shows, were included by opera director Dmitri Tcherniakov in his staging of The Tsar Bride by RimskyKorsakov (Staatsoper Berlin, 2013). The chat of the Oprichniks, projected on the [stage] backdrop, became a way to explain the director’s plot to the audience, but more importantly, it was an element of the director’s concept—the power of the media in a modern society: “All the world’s a screen in this Tsar’s Bride, a society distinguished most by the ceaseless generation and consumption of ‘content’” [26]. The opera stage and the backdrop thus seem to have turned into a huge laptop, in front of which the viewers find themselves. When viewing a production on a real laptop, for a moment the illusion becomes reality, and the chat appears on one’s own computer screen. This is one of many examples of the inclusion of a verbal text intended for reading only both into a contemporary opera staging (in addition to subtitles) and into the new works for music theatre. The verbal text that the spectators read during the performance existed in opera and ballet performances throughout the whole history of both arts. The audiences of the first operas in Italy in the early seventeenth century read small books (libretti); the German public in Bayreuth at the end of the nineteenth century read guidebooks (Führer) to the Wagner’s tetralogy Der Ring des Nibelungen. On stage, subtitles were used most actively (until the end of the nineteenth century) in the ballet. During the twentieth century, subtitles, however, like many other things, moved to the cinematographic screen in the form of intertitles containing dialogues and defining the time and place of the action. Today, the very regularity and variety of techniques for incorporating verbal text in its written form into performance indicates a new trend. The relationship between music and text is the central theme of the history of opera, that to a large extent defined the poetics of various opera genres and opera reforms, composers’ styles and the “relief” of an individual work. It has also been the central subject of opera studies and critical debate for more than four centuries. Its importance has not diminished for newer genres and movements—one of the central principles of postdramatic theatre is the rejection of verbal text as a foundational part of a performance in favour of “musicalization” [13]. In this article, I examine varying degrees of emphasis on the verbal-textual aspect as a general principle, and consider modern works and performances, placing them on “verbality spectrum”. The article explores both the new works in music theatre and opera stagings by modern directors and choreographers.2 Thus I attempt to step outside a work-centered musicological approach, as well as to transcend the line between contemporary and traditional repertoire in order to touch upon Regieoper (Director’s opera). In my 2

For this article I chose the works that are representative of different shades of the spectrum; there are countless more examples that could be brought.

58

O. Manulkina

understanding, it would be more productive to examine these phenomena in a unified field, focusing on general techniques, tendencies and ways of interacting with the audience.

2 Words Without Music 2.1 Director’s Remarks, Surtitles, and Intertitles and Verbal Prologues in Dmitri Tcherniakov’ Opera Stagings At the verbal end of the spectrum, I will place works and productions that include a verbal text intended for the audience to read that does not simply duplicate the text being sung (it performs a different function than subtitles); a text, without which the concept of the performance cannot be fully understood or realized. As an example, here I will discuss productions by Dmitri Tcherniakov and Vladimir Rannev’s opera Prose (2017) with his own concept of the libretto and production. Several Tcherniakov’s productions in the 2010s feature his introductory texts, thus creating a conceptual framework for the opera. If these texts are pronounced by the starring singers from the stage (as, for example, in Rimsky-Korsakov’s The Tale of Tsar Saltan, Theatre de la Monnaie, Bruxelles, 2019) or from a video screen (as in Rimsky-Korsakov’s Sadko, Bolshoi Theatre, Moscow, 2020), the function of this part of the performance approaches the function of the allegorical prologue in baroque operas or, for example in Lulu by Berg. In contrast to the allegorical prologue however, it is not the performers who are then transformed into opera characters, but the very characters whom they played in the prologue. Such a prologue precedes the overture. The verbal remarks can also be projected onto the curtain or stage backdrop or appear on a creeping line (and this can be done during the overture). They establish the director’s plot of the performance: time, place, conditions, introduction to the events that do not coincide or partially coincide with the plot of the opera. In several productions of the last decade, the heroes (for Tcherniakov these are always our contemporaries) take part in role-playing games, in group psychoanalysis; director’s remarks clarify these contexts. For example, in Verdi’s Il Trovatore (La Monnaie, Bruxelles, 2012) the surtitles before the overture inform that the heroes of the opera are not dead, and after many years gather instead to sort out what happened; Prokofiev’s Betrothal in a Monastery (Staatsoper Berlin, 2019) opens with a projected line of commentary, “Opera-Addicts Anonymous.” In Sadko video interviews are given by three people who wished to participate in the quest “The Wish Fulfillment Park”. In the further structure of the performance such texts may be absent or may be included regularly, for example before each scene (Mozart’s Don Giovanni, Aix-enProvence Festival, Bolshoi Theatre, 2010), within scenes (Sadko), or even replacing a ballet scene (Betrothal in a Monastery). This technique can be associated with the abovementioned subtitles used with ballet performances or intertitles in cinema, on

Reading Opera: The Verbal versus the Non-verbal in Opera …

59

the other hand, with structures such as that of Stravinsky’s Oedipus Rex, where the Speaker informs the audience about the events that will be presented, before a given act or scene.

2.2 Two-Part Counterpoint of Sung and Drawn Texts and Plots: Vladimir Rannev’s Prose In his opera Two Acts (2012), composer Vladimir Rannev included a verbal prologue (a text displayed on a creeping line against a musical and noise overture) in the structure of the work itself, not only its production: it was a reflection on Faust and Hamlet, to which the whole opera is devoted. In its beginning we hear fragments of Goethe’s and Shakespeare’s texts pronounced in German and English. In the opera Prose (2017) Rannev again combined two texts (this time taken almost entirely): Chekhov’s story The Steppe and Yuri Mamleev’s story The Bridegroom. Chekhov’s text was set to music written for the vocal ensemble a cappella, mostly in a quasirecitative style. The singers remain almost invisible or semi-visible; owing to a special device with two mirrors they seem to float in the air behind a semi-transparent screen. What could be a necessary and sufficient text of an opera—verbal and musical—is seemingly left in the depths, in the background. Conversely, the second text comes to the fore, Mamleev’s story, and is not sung or recited, but only read and parsed through by the audience, since it is a comic (by artist Marina Alekseeva), projected onto a screen. Through two channels—audible and visible—two different texts are communicated to the public; the holistic effect is formed only in their combination. And again, this is not a fact of production, but rather of authorial text: “Theatrical text is not simply ‘sewn up’ inside the musical one, but it is an important prerequisite for its implementation” [20].

3 Words Eclipsed and Obscured 3.1 Music and Gesture, or no Need for Words: Dance/Choreographic Opera At the opposite end of the spectrum (works displaying weak emphasis on the verbal aspect) I place those performances and works for which non-verbal texts and languages are conceptually more important, where the verbal text relinquishes some of its functions. Here predominantly preside “dance operas” (Tanzoper) and “choreographic operas”: for example, Iphigénie en Tauride (1974) and Orfeo ed Euridice (Tanztheater Wuppertal, 1975) by Gluck choreographed by Pina Bausch, and Montevedi’s Orfeo choreographed by Trisha Brown (Theatre de la Monnaie, Bruxelles, 1998). However, some of these performances are shifting towards the

60

O. Manulkina

middle of the spectrum: in her production of Purcell’s Dido and Aeneas (Grand Théatre de Luxembourg, 2005), Sasha Waltz included not only additional musical fragments from other works by Purcell, but also the Prologue written by Purcell’s librettist Nahum Tate, the music of which has not survived; it is read by two actors. Another “dramatic intermission” is included between the two acts of the opera. Thus, Waltz brings Purcell’s only fully sung opera closer to his own operas in the more traditional English genre of semi-opera, which included musical and choreographic “masques” in dramatic plays. A particularly interesting example is Mozart’s Cosi fan tutte choreographed by Anne Teresa de Keersmaeker (Opéra de Paris, 2017). Mozart’s opera is distinguished from the above-mentioned operas by the source and nature of the plot (neither mythological, nor archetypal), by opera genre (Mozart’s opera buffa), by density (of both the actions and Lorenzo da Ponte’s libretto), by its positional entanglement and psychological states and situations, by the scale and complexity of the score, while Keersmaker’s staging, respectively, is distinguished from other stagings by the richness and detail of the choreographic score. As in the other choreographic operas, here a third text and language of the performance appears in addition to the musical and verbal ones — neither less important nor less intelligible — the choreographic and mimic language, realized by the dancers, the singers’ counterparts. However, the amount of information communicated to the spectator and listener is much higher than in Orfeo or Dido. And it can be assumed that for the listener/spectator, who does not speak Italian (as, perhaps, for the one who does), the verbal language in this performance surrenders its central communicative role to the gesture.

3.2 Not Meant to Be Understood: Libretti and Vocal Texts in Ancient or Non-International Languages At this end of the spectrum I will also place operas in dead or invented languages, such as Satyagraha (1980) and Akhnaten (1984) by Philip Glass. The model for them was obviously Stravinsky’s and Cocteau’s Oedipus Rex (1926), written in Latin, which, according to the authors’ plan, should not be comprehensible to the listener. The plot, as it was mentioned above, is announced by the Speaker (the subtitles to this opera-oratorio in contemporary practice violate this condition) in the language of the country where the opera is performed, while in the Latin libretto, free from this function, the very musical functions of the verbal text come to the fore: phonetic, rhythmic, structural. Latin also symbolizes antiquity and the mythological past. Sanskrit plays the same role in Satyagraha and so does the “hypothetical reconstruction of the sound of ancient Egyptian” [7, p. 178] in Akhnaten, where it is also commented upon by a narrator speaking a modern language.

Reading Opera: The Verbal versus the Non-verbal in Opera …

61

Along with operas, it seems logical to also cite mixed genres, employed in stage and concert and semi-concert performances, in which the phonetic, rhythmic, and structural properties of the language also are emphasised. Tehillim by Steve Reich (1981) belongs to the same group as Glass’ operas and is the direct heir to Oedipus Rex, but also to Stravinsky’s Symphony of Psalms. Reich extracted his texts from Psalms 19, 34, 18 and 150 in Hebrew; in the final part of Tehillim the last three of six verses of the 150th psalm are used, on which the final part of Stravinsky’s symphony was written. Reich “re-textualises” verbally (Hebrew instead of Latin) and musically the whole of Psalm 150. The words “Alleluja. Laudate Dominum”, which are set against the momentous, bell-like chords of the Symphony of Psalms, return to their unified and indivisible form in Hebrew: “Halleluyah” [16, p. 617]. This linguistic technique, however, is concealed from a listener who does not know the language; for him the text appears merely as a phonetic and rhythmic phenomenon; and he can hardly assume that “the musical rhythm comes directly from the rhythm of the Hebrew text and, therefore, the meter is constantly changing” [19, p. 101]. Rather, one will hear a similarity to scat-singing in the manner of The Swingle Singers, as it was in the case of Reich’s Music for Mallet Instruments, Voices and Organ (1973), where there was no text, only syllables. The sound image of Tehillim is also built on a combination of archaism and modernity, ancient ritual and high technology. In Ana Sokolovich’s opera Svadba (The Wedding, 2011) for six female singers a cappella, we hear the living languages of the Balkan nations, mainly Serbian, but one can hardly assume that they are known to an international audience. The study of the individual melody of the language is a subject of particular interest to the composer, but the ear picks out only individual words in singing, and only where the tempo (slow) and texture (sections of solo or in unison) allow it. As in the texts of any ritual folklore, the translation also does not bring any clarity; the listener reading the subtitles grasps only character and situation in general. Finally, the text sometimes becomes rather a purely rhythmic and phonetic game with otherwise meaningless, calculated rhymes. The composer picks up this game, amplifying and inventively expanding its arsenal of sounds and noises. Again, the model which is here obvious and emphasized is Stravinsky’s Les Noces (1923), including the way the composer works with the rhythmic and phonetic structures of a folk text.

4 Intermezzo: Verbal Texts’ Paradoxical Use or Effect 4.1 Verbal Text as a Staging To the center of the spectrum I designate the works and performances (or their significant fragments) that are very different in genre and structure, in which a reinterpretation and reevaluation of the relationship between the verbal and the non-verbal, the new tension between them is emphasized, and the transition from one to another, the connection and the separation determines the artistic strategy.

62

O. Manulkina

In the finale of Wagner’s Götterdämmerung in Peter Konwitschny’s production (Staatsoper Stuttgart, 2003), after Brünnhilde’s monologue, the black curtain is lowered, upon which subtitles are projected with Wagner’s final remark that flames should rage, the heavenly castle of Valhalla should perish, and the waves of the Rhine should roll—on stage. The text on a black background represents the only visual and theatrical solution to the final orchestral climax. It may seem that theatre here retreats in favour of music, but in fact, this is where it successfully resolves a difficult problem for theatre. Here the paradoxical nature of Konwitschny’s decision need be emphasised: it is the verbal text intended for reading only that marks the power of the non-verbal—musical and audible. It could be also seen as a commentary to Wagner’s complex “polyphonic narration” in the tetralogy, which has been analyzed by Carolyn Abbate in the following manner: “I suggest that the Ring contains musical narration that may speak both with and across the text, furnishing us with the opportunity to hear many narrating voices (both textual and musical), to separate the conflicting lines. Through repeated offerings of such polyphonic narration, Wagner undercuts the very notion of music as a voice whose purity is assured by virtue of its non-verbal nature” [1, XIV].

4.2 Verbal Text as a Musical Score The ballet Statement (2018) by choreographer Crystal Pite and playwright Jonathon Young seems also to hinge on a paradox: it is a ballet performance, where the verbal text plays the role of the music score. The critic Roísín O’Brien notes: While many choreographers might shy away from words, either out of a sense of loyalty to pure movement or an aversion to narrative, Crystal Pite and Jonathon Young happily collide the two worlds together [5].

Defining the ballet as “a kind of bodily lip-syncing to a spoken soundtrack” and outlining a summary (“the four dancers enact a high-tension script about a team tasked with inciting war in order to profit from it”), another critic Sanjoy Roy writes: The vocal rhythms, pitches and registers form an exacting template for their accusing fingers, nervous flinches, shocked about-faces and wily manoeuvres. It’s performed with the frameby-frame urgency of a graphic novel and—herein lies its brilliance—even as you are gripped by the drama you see its composition with startling clarity [22].

However, what remains unmentioned in both reviews (although Roy emphatically uses musical terminology in his description) is that the verbal text builds a template for movement and gesture by musical means, working with time, accents and structures. That is why this example is not placed at the verbal end of the spectrum; the verbal here replaces the implied musical score.

Reading Opera: The Verbal versus the Non-verbal in Opera …

63

5 Word as Music and Apart from Music: Wilson’s and Glass’ Einstein on the Beach 5.1 Non-verbal Libretto, Spoken Words, Sung Numbers and Solfège Syllables Among the many innovations of the opera Einstein on the beach (1976) by Robert Wilson and Philip Glass, which became a milestone in the development of postdramatic theatre, particularly important is the redefinition of the relationship between the verbal and non-verbal. A visual libretto (Glass wrote his score from Wilson’s visual sketches) was the fundamental change, but it did not exclude the possibility of writing the lyrics afterwards. The authors however chose a different path: a vocal ensemble sing either numbers (counting the beats or simply the number of repetitions of every sound), “which serve to highlight rather than constrain the music’s potential for metric unpredictability” [2, p. 165] or solfège syllables; both numbers and solfège were mnemonic devices that remained within the work. Robert Adlington observes “the difficulty of reconciling a narrative text with music that remains fundamentally non-narrative” in Philip Glass’ mature vocal works and calls Einstein a “relief” as a “remarkable concoction of text and music that works to solve the problem of their coexistence in a number of ways” [2, p. 165]. Any lengthy verbal texts in Einstein are recited “over” the sound of a vocal and/or instrumental ensembles. Their function is not coherent narration; what is important in these texts—and above all in the texts by Christopher Knowles—is the structure of repetitive patterns. Wilson inherited this approach to text from Gertrude Stein and John Cage, as well as the method of filling with verbal and musical texts of the so-called “empty containers” (as, for example, in Cage’s Lecture on Nothing). David Byrne, who worked with Wilson on CIVIL Wars (1984) described his process of composition: …He begins with structure, not narrative. Sometimes he’ll propose that a piece have five large acts, for example. Then he might say that the acts at the beginning and the end will be mirror images of each other, and the one in the middle will be the “axis.” He might set lengths for each act or scene, and these timings will be completely arbitrary, as there is no “content” for these scenes yet; but he will (sometimes) nonetheless adhere to that structure [6].

Many of these texts are recited in five knee-plays: before, after and between the four acts. Two actresses recite them in turn, but without entering into a dialogue: it is two parallel monologues, sometimes simultaneous, as if in opera ensemble, “which further minimises their narrative properties” [2, p. 165]. This is Wilson’s analytical (and not synthetic) method of disconnection at all levels. Refuting the assertion that Wilson connected “various disciplines into one unified whole”, Heiner Goebbels contrasts his concept with that of Wagner’s Gesamtkunstwerk: Because this unified whole — and this is the beauty of it — is just our respective own. And for every audience member it is a different one unity. To speak of unity, that already sounds much like ‘total work of art’. In Richard Wagner’s work all means of theatre are actually

64

O. Manulkina welded together into one unity, which tempted Brecht to speak of a ‘one washing-up’ and Heiner Müller to call it a ‘stew’, a ‘synthetic mash’ [9, p. 61-62].

With Wilson, Goebbels continues, all the disciplines on stage “never merge together, quite the opposite. He achieves unity due to the radical independence of all means of theatrical expression” [9, p. 62]. Wilson thus reveals that “we can hear texts differently and anew when they start to sound like litanies and turn into music without losing their meaning. And that this can transport us into a competent state of hearing language” [9, p. 63]. The verbal and the non-verbal are separated and reassembled again “vertically” (in simultaneity) and “horizontally” (in the temporal structure of the performance).

5.2 “The Union of Silent Film and Radio Play,” or Wilson’s Scenic and Audio Landscapes The disconnected elements exist in a landscape or “stillness” which, for the temporal arts, is produced by increasing the duration of a piece and “endless” repetitions (the qualities of minimal music). Hans-Ties Lehmann comments on this: “Only since Wilson’s ‘invention’ can we speak of a proper aesthetics of duration. The visual object on stage seems to store time in it. The passing time turns into a ‘Continuous Present,’ to use the words of Wilson’s role model Gertrude Stein” [13, p. 156]. In this landscape or, more precisely, at this pace, scale and number of repetitions, one can distinguish individual figures and actions on the stage, where something is constantly happening. The landscape principle as opposed to the linear unfolding of action is another important element inherited by Wilson from Stein and Cage. Critics and researchers have compared Einstein to the opera Four Saints in Three Acts (1928) by Stein and Virgil Thomson in which Stein suggested perception of the play as a landscape in which nothing moves or happens. Ana Vujanovich observes that “Stein’s interest in the materiality of language reappeared as the musicality of spoken text in Wilson’s early works” [25, p. 28], and introduces the term “landscape dramaturgy”. “Gertrude Stein’s texts will only find their congenial theatre aesthetics with Robert Wilson,” writes Lehmann, who characterizes the logic of these texts as such: The idea of an exposition of language seems paradoxical. Nevertheless, since Gertrude Stein’s theatre texts — if not earlier — we have the example of a language that loses its immanent teleological temporality and orientation towards meaning and becomes like an exhibited object. Stein achieves this through techniques of repeating variations, through the uncoupling of immediately obvious semantic connections, and through the privileging of formal arrangements according to syntactic or musical principles (similarities in sound, alliterations, or rhythmic analogies) [13, p. 147].

The audio environment of his performances Wilson calls “soundscape.” The compositions and happenings of John Cage are another model here. One is less capable, however, of “looking at” (by ear) simultaneously existing sound elements,

Reading Opera: The Verbal versus the Non-verbal in Opera …

65

one by one. In musical scores the very illusion of space is created by register, time, and rests. Wilson’s method is the opposite to Cage’s chance operation, and even more to happenings combining musical and non-musical layers. Wilson separates these elements, arranging them in the soundscape with strict distinction: reciting vs singing, one voice vs ensemble, verbal text vs numbers and solfège. All this, in its turn, is disconnected and placed in the landscape of the entire performance by actors and dancers on stage, by scenography and light. In addition to “landscape,” Wilson defines his ideal of theatre as “a combination of silent film and radio play” [13, p. 148]. Einstein seems to be quite close to this ideal. Vocal and instrumental ensembles, as well as speakers (“radio play”), do not accompany movement on stage (“silent cinema”). Lehmann mentions that Wilson calls his dramatic works “operas” in the context of his “tendency towards musicalization (not only of language)” in postdramatic theatre: An independent auditory semiotics emerges; directors also apply their sense of music and rhythm, which is influenced by pop music, to classical texts (Jurgen Krüse); Wilson calls his works ‘operas’. In the course of the dissolution of dramatic coherence the actor’s speech becomes musically overdetermined through ethnic and cultural peculiarities [13, p. 91].

The point, however, is not only in the musicality of the pronunciation of the text and of the impact, and not only in the auditory semiotics, but also in the poetics of opera genre itself. The fact that Wilson called his spoken performances “operas” even before Einstein points to a rethinking of the function of verbal text in dramaturgy. In Einstein, the play of the verbal and the non-verbal, the exchange of functions between them, their interplay and entanglement, constitute one of the principal subjects of this theatrical text as a whole. If we were to summarize what Wilson achieved in this regard, we could call it rejection of verbal text on several levels where it is supposed to exist, and, on the contrary, its inclusion where it is not,—and with such quality, which changes its traditional purpose, especially in the structure of a musical performance: i.e. a rejection of verbal text as the basis of a theatrical work and as an obligatory material for vocal parts; a rejection of narrative in the recited verbal text, which, as a result, cannot “move the action” between the vocal (musical) sections, as in most genres of music theatre; the transformation of knee-plays from musical intermezzos into verbal intermezzos between visual and musical acts.

6 “Typing Music” and Instumental Words 6.1 In the Beginning Was the Word: Reich’s and Korot’s The Cave The Cave (1990–1993) and Three Tales (2002) by Steve Reich and Beryl Korot are created in the genre which they called “documentary music video theatre” [19, p. 152]. In contrast to Einstein, these works exhibit a return to verbal text, which

66

O. Manulkina

Lehmann finds in the 1990s in postdramatic theatre in general (this does not mean, however, the emergence of a verbal libretto and a linear plot). In The Cave, the relationship between the verbal and the non-verbal, their alternation, the transition of the verbal (including the written text) to the non-verbal, defines the compositional methods and structures of each of three acts. Thus, form and structure work here for the content, since in the center of The Cave there are the sacred texts of the Bible, Torah and Koran. As Paul Griffiths correctly remarked, “The Cave is built as St John’s Gospel says, ‘In the beginning was the word’” [10, p. 294]. The second type of texts in The Cave are video interviews recorded by Reich and Korot; from the speech intonation of these interviews, the musical patterns of the instrumental ensemble are born (this method was first used by Reich in Different Trains, 1988). The text of Genesis is for the most part meant only for reading; it is shown on some screens in Hebrew, on others in English, French and German, and here it manifests itself gradually: we see how words, typed on a computer screen, appear; sometimes we see on video how the text is being written by hand. These moments are accompanied by “typing music”, for which a computer “typing instrument” was developed [19, p. 168–169]: written speech in The Cave also acquires sound. Such a method of combining the verbal (in reading) and the non-verbal (rhythm and sound) has not been encountered in the other examined examples (in addition, the text of the Book of Genesis is sung by duet of sopranos, it sounds also in the recording of the Torah cantillation in the act 1; cantillation of the Koran is included in act 2). Reading these texts is essential for the work’s holistic perception: like the subtitles in Tcherniakov’s productions they tell a story; like in Rannev’s Prose, they are placed into a musical and compositional whole.

6.2 Words and Music Go on Screen, or Technological Gesamtkunstwerk Reich’s and Korot’s Three Tales (2002) are realized completely on screen. Reflecting on Wilson’s paradoxical definition, one can say that the first parable, Hindenburg, is a silent film with music: documentaries from 1937 are separated from music by two-thirds of a century. What was presented in The Cave on a larger scale, slowly and step by step, is compressed and condensed in Three Tales. In the same way, the work differs from Wilson’s spacious landscape: everything here is brought together and acts simultaneously, there is no time to look at individual elements: this is exactly the “stew”, as Heiner Müller defined Wagner’s Gesamtkunstwerk. Texts of different origins and in different presentations are combined in a rather dense counterpoint. The very first minute of the first tale, Hindenburg, contains typing music (in the rhythmic canon of snare drums and kick drums) and, accordingly, the reading of the text (but now, unlike in The Cave, it is given in a variety of visual styles: headline from a New York Times column from 1937, words and phrases snatched from the

Reading Opera: The Verbal versus the Non-verbal in Opera …

67

column in a running line), singing (the same words and phrases are sung by tenors in three-part canon), while at the same time another text read from a running line, and documentary footage of the crash of the airship. Having reached the maximum density and speed for the very first minute, the movement then slows down, the energy decreases, the three tenors from the mobile, rhythmically active canon move to extended lasting chords on the words “it burst into flame, it flashed and it’s crashing”. Then a new element appears—voices as if from documentary recordings (of eyewitnesses gripped by the horror?), the words are blurred, it is impossible to decipher them: the technique that Reich used here is “freeze frame sound” (“a single vowel or consonant is extended for a long time leaving a kind of audible vapor trail behind each speaker that becomes part of the overall harmony” [19, p. 205]); this sound corresponds to the documentary footage, in which the huge airship is burning and falling in rapid speed. And in another layer, the same rhythmic pattern (drums) remains, which, as we remember, is typing music, that is to say, rhythm and timbre of the words from New York Times headline, and the tenors also sing words (already barely distinguishable) from that terrible news. All this is doubled by instrumental ensemble. In the second tale, Bikini, a countdown is amended to this set (constant metronome, numbers on the screen from 10 to 1, spoken numbers) and, reminiscent of The Cave, words from Genesis flash for a moment on the screen: “And God Created Man”. In contrast to other typing music they are accompanied by sonorous chords on pianos and vibraphones; that makes it impossible to imagine that they are also typed on computer, “as if they existed (as indeed they do) in another dimension” [19, p. 204]. The verbal elements here undoubtedly form a music drama.

7 In Quest for Research Tools The “verbality spectrum” proposed in this article, shows a more nuanced range and arguably gives a more detailed picture of how the elements of musical and theatrical action interact nowadays as opposed to any preexisting classification. It shows in particular that: – in opera, where the verbal was by definition less emphasised than in spoken theatre, one type of modern production reemphasises it (Tcherniakov), while another further diminishes its role (dance/choreographic opera); – a verbal text, when it ceases to be perceived as a necessary basis for librettos and performance, returns in the form of additions, explanations, independent sections or layers; – a reflection of today’s practices of reading and writing on screen informs both new music theatre and opera productions (Rannev, Tcherniakov), while the very writing of a verbal text becomes the subject of the composer’s and director’s analysis (Reich and Korot);

68

O. Manulkina

– dance/choreographic opera tends towards the opposite, where the verbal concedes to the gesture; works that explore words’ phonetic and rhythmic qualities similarly operate (Reich, Sokolovich). The need for tools to aid further research is evident. More than half a century ago, Yuri Lotman wrote about the problem of the “terminological isolation of [different fields in] the humanities” from each other [14 p. 28]; in another text he discussed a performance as “semiotic ensemble” with heterogeneous means of expressiveness and a combination of two types of sign systems, mobile and each time individual correlation of elements, different codes, different ways of communication with the spectator [15, p. 601–602]. In the field of contemporary music theatre (in the broad sence) the need for this interdisciplinary approach and common research tools and terminology is urgent. Works like Einstein, Prose, or Three Tales, as well as Tcherniakov’s stagings require analysis which combines the insights of musicology, opera and theatre studies, film and visual studies. Rannev’s Prose evidently belongs to the category of new music theatre, but besides this theatre (Reich’s, for example) his opera is clearly influenced by Wilson’s postdramatic theatre and composed theatre (Goebbels’, for example); however in its concept and theatre realisation the influence of Regieoper, and Tcherniakov’s opera stagings is likewise apparent. The opposition of the verbal vs the non-verbal is one of the approaches that could prove productive here. For further research of the causes and effects of weaker or stronger verbality as well as of the balance between different languages of a production, the model of language functions by Roman Jakobson could be applied3 (though the very possibility of applying it to the secondary modeling system needs to be separately considered): the communicative or referential function, the appellative or the conative function, the poetic function, the expressive or the emotive function, the phatic function, the metalingual function [12]. It may also help to discuss various approaches to theatre media, to define its new types and genres, as well as various ways of communication between the musical theatrical performance and its audience. For example, in traditional opera genres stronger verbality on the basic structural level (spoken dialogues) or on the level of words and music balance in the vocal parts (declamation) most often means a stronger communicative or referential function. However, in Einstein, the verbal texts and spoken monologues have the opposite effect: the stronger the poetic function in these texts, the weaker their communicative function; in the knee-plays the phatic function could also be noted: to start a dialogue with the spectator/listener and maintain it. The verbal text in the subtitles and “prologues” of Dmitri Tcherniakov’s stagings undoubtedly enhances the communicative function of his performances, and also contains a metalanguage function, setting the code for reading the director’s theatre text. While in Hindenburg from Three Tales the communicative function is just as strong at first, but then gradually the word dissolves into singing, and the musical and expressive essences come to the fore; by the nature of the relationship, it is the recitative and aria of traditional opera (with an emotional reaction to events), but 3

I am grateful to Igor Nemirovsky for suggesting Jakobson’s model to me.

Reading Opera: The Verbal versus the Non-verbal in Opera …

69

embodied in strikingly unique form of “a product of the late twentieth-century media age, and in particular the cultural form of the television documentary” [23, p. 9]. As was mentioned above, the relationship between verbal text and music has determined the poetics of opera genres as well as the dramaturgy of a particular work throughout opera history. Therefore, the methodological opposition put forth in this article (the verbal vs the non-verbal) can also be useful in analyzing traditional operas, especially concerning their modern productions. It helps in comprehending contemporary music theatre as a unified field and in contextualizing the most unusual and uncategorized works and performances in it. Today, opera can reject the verbal libretto, the theatrical stage, musical instruments, while spoken theatre performance can be defined as “opera”; Regieoper is as diverse as ever. Identifying the common principles and tendencies among this diverse and variable range of phenomena as well as the instruments and approaches for the research is, undoubtedly, an essential task.

References 1. Abbate, C.: Unsung Voices: Opera and Musical Narrative in the Nineteenth Century. Princeton University Press (1991). 2. Adlington, R.: Through a Glass Darkly. The Musical Times. Vol. 135, No. 1813 (Mar., 1994), pp. 164–165. 3. Adlington, R. (ed.): New Music Theatre in Europe: Transformations between 1955–1975. Routlege, Abington, New York (2019). 4. Adorno, T. W.: Introduction to the Sociology of Music, Ashton, E. B. (trans.). Seabury, New York (1976). 6. Byrne, D.: What’s a Knee Play? https://kneeplays.com/album/kp-essay-p2.shtml#top. 7. Colas, D.: Musical Dramaturgy. In: Greenwald, H. M. (ed.): The Oxford Handbook of Opera, pp. 177–205. Oxford University Press (2014). 8. Cooke, M. (ed.): The Cambridge Companion to Twentieth Century Opera. Cambridge University Press, Cambridge, New York (2005). 9. Goebbels, H.: Aesthetics of Absence: Texts on Theatre. Ed. by J. Collins, cons. ed. N. Till, trans. by D. Roesner and Chr. M. Lagao, Routledge, Abingdon, New York (2015). 10. Griffiths, P.: Reich. In: Griffiths, P.: The Substance of Things Heard: Writings about Music, pp. 292–302. Boydell & Brewer, Woodbridge, University of Rochester Press (2005). 11. Heile, B.: Recent Approaches to Experimental Music Theatre and Contemporary Opera. In: Music & Letters, Vol. 87, No. 1 (Jan., 2006), pp. 72–81. 12. Jakobson, R.: Linguistics and Poetics. In: Jakobson, R.: Language in Literature. Pomorska. Kr., Rudy, S. (eds.), pp. 62–94. The Belknap Press of Harvard University Press, Cambridge, Massachusetts, London (1987). 13. Lehmann, H-Th.: Postdramatic Theatre. Trans. by K. Jürs-Munby. Routlege, Abington, New York (2006). 14. Lotman Yu.: Lektsii Po Structural’noi Poetike [Lectures on Structural Poetics] (in Russian). In: Yu. M. Lotman i Tartussko-Moskovskaya Semioticheskaya Shkola, Gnosis, Moscos (1994), pp. 10–257. 15. Lotman, Yu.: Semiotika Szeny [Semiotics of Stage] (in Russian). In: Lotman, Yu.: Ob Iskusstve, pp. 583–603. Iskusstvo-SPb, Saint Petersburg (1998), pp. 601–602. 16. Manulkina, O.: Ot Ivesa do Adamsa: Amerikanskaya Musyka 20 Veka [From Ives to Adams: American Music of the Twentieth Century] (in Russian). Ivan Limbakh Publishing House, St Petersburg (2010)

70

O. Manulkina

17. Müller, U.: Regietheater/Director’s Theater. In: Greenwald, H. M. (ed.): The Oxford Handbook of Opera, pp. 582–605. Oxford University Press (2014). 18. Matthias Rebstock and David Roesner (eds.): Composed Theatre: Aesthetics, Practices, Processes. Intellect Ltd, Bristol (2012). 5. O’Brien, R.: Making a Statement: Crystal Pite’s The Statement for NDT. Fjord Review. https:// fjordreview.com/ndt-the-statement-crystal-pite/. 19. Reich, S.: Writings on Music, 1965–2000. Oxford University Press (2002). 20. Renansky, D.: Preodolenije Zvukovogo Barjera [Breaking the sound barrier] (in Russian). Kommersant (1 Dec 2017). https://www.kommersant.ru/doc/3475908. (Access 25 Mar 2021). 21. Rogers, H.: Audio-Visual Collisions: Moving Image Technology and the LaternaMagika Aesthetic in New Music Theatre. In: Adlington, R. (ed.): New Music Theatre in Europe: Transformations between 1955–1975. Routlege, Abington, New York (2019), pp. 79–100. 22. Roy, S.: Nederlands Dans Theater Review — a Superb Troupe of Dancers. The Guardian (27 Jun 2018). https://www.theguardian.com/stage/2018/jun/27/nederlands-dans-theater-review Accessed on 25 Mar 2021). 23. Rutherford-Johnson, T.: Music after the Fall: Modern Composition and Culture since 1989. University of California Press (2017). 24. Salzman, E., Desi, Th.: The New Music Theater: Seeing the Voice, Hearing the Body. Oxford University Press (2008). 25. Vujanovic, A.: Zusammen mäandern: Neue Tendenzen in der Landschaftsdramaturgie. In: Umathum, S., Deck, J. (eds.): Postdramaturgien, pp. 27–45. Neofelis Verlag, Berlin (2020). 26. Woolfe, Z.: A Modern Media Spin on Ivan the Terrible. In: New York Times (6 March 2014). https://www.nytimes.com/2014/03/07/arts/music/07iht-A-Modern-Media-Spinon-Ivan-the-Terrible.html (Accessed on 25 Mar 2021).

Music as “Impossible Experience”: Multisensory Perception in the Composer’s Practice of the Twenty-First Century Svetlana V. Lavrova

Abstract The aim of this article is to reflect the perceptual spectrum of new music media. At the center of the research is the so-called “experience of the impossible”. The recipient has to overcome all sorts of stereotypes of perception and reach threshold qualities within the individual, surpassing the standard inertia for listening. It is necessary to go beyond the usual, learn to perceive noise and surrounding sounds as musical material. Modern nostalgia for the living, elusive and infinitely changeable, conditioned by the development of digital technologies, becomes the basis for presenting music as an experience of the “impossible”. “Experience of the impossible” is an experience not limited by the external sides of being, which is defined in postmodern philosophy by the concept of transgression. The researcher bases her conclusions on the theoretical works of composers, in which they explain their approach to listening perception. In the work of Peter Ablinger, he is primarily interested in the maximum informative saturation: the so-called “white” all-frequency noise. The creative strategy of epiphany—the so-called “waiting horizon” in the concept of Salvatore Sciarrino becomes a reflection of the theoretical concept of David Huron and his idea of Sweet Anticipation. Keywords Experience of the Impossible · Psychology of sound perception · Multisensory perception · Composing

1 Introduction New technologies are quickly becoming part of the contemporary lifestyle. In our new reality that we find ourselves today, so unexpectedly, aura is changing into something impossible, and art belongs to the virtual more than to the “real”. In twenty-first century music, in particular, and in art, in general, different processes are taking place that are related directly to the problem of aura loss. This nostalgia for the live and stolen, while constantly changing, is the foundation to current music as the S. V. Lavrova (B) Vaganova Ballet Academy, Saint-Petersburg, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 T. Chernigovskaya et al. (eds.), Language, Music and Gesture: Informational Crossroads, https://doi.org/10.1007/978-981-16-3742-1_6

71

72

S. V. Lavrova

experience of the “impossible”. “The experience of the “impossible” is the experience which is not restricted by the outer limits of being, that is defined in post-modern philosophy as transgression. Transgression is a gesture that is addressed beyond the limits [6]. This notion allows to eliminate the opposition of the outer and inner, to retain the contradictions in their tension, which allows to open the space for the contamination of the paradoxical. According to Bataille, transgression takes away the “ban” but doesn’t eliminate it, allowing itself to be on the threshold. A researcher V.T. Faritov claims that music is a “pure movement, change and development, the change of the sound into another one, that is, “transgression”. In terms of the traditional view about musical themes, according to Faritov, the aim of music as a transgressive phenomenon is by removing one away from the state of a vicious circle and into a state of “fluidity” [5]. This is the way, from his point of view, music is different from mere noise. As an opposite of musical creativity, the researcher presents noise as a stable and non-changeable entity. Such a simplified view of the noise materials could be challenged by a wider spectrum of its wide application in new music, in which transgression is the “experience of the impossible”, or the fluidity of a different kind (fluidity of noises, a get through beyond the limits of previous understanding of “musical space”, play with perception on the threshold of “hear”—“don’t hear” (audible—non-audible), “perception of sound vibrations”, and treatment of sound as energy [19].

2 Methodology and Sources The research methodology included the theoretical works of composers of new music in which they postulated their creative concepts, as well as the writings of renowned musicologists, along with the works of specialists in psycho-acoustics such as Michel Imberty, Irene Delège, and David Huron [11]. Most of the theories are directly related to musical perception and have undergone various changes, depending on the aesthetic position of the researcher and their own perception of music [20].

3 Results and Discussion 3.1 Sound Perception and “The Structuring of Internal Experience» Introspective approach to sound perceptions focuses on the study of the elements that comprise consciousness [4]. The German psycho-physiologist Ernst Weber noted the basic fact concerning the psychology of perception is that it is independent from sensory modality. Man is not susceptible to absolute differences, but to differences of similar elements. That is, we understand that we could correlate tiny differences

Music as “Impossible Experience”: Multisensory Perception …

73

of noise streams and, comparing them, a listener could make a conclusion, or outline a meaningful “storyline”. The listener, as stated by Michel Imberty, through perception of music, structures his inner and subjective life experience as an experience of integration and decay. These are in a way schematic super-structures that have assimilated the organization of music temporality and the degree of consistency, or use mutual integration of the change of tension and stretching, creating emotional resonance in the time and space schemata [8, 13]. These views about the dynamic vector of the musical composition sustain the macro-structure of the musical work, giving it the direction of development, which is characterized by a series of tension and relaxation periods.

3.2 Noise and Music: Facets of the Possible The creative interaction of the Austrian composer Peter Ablinger with noise is unique. Acoustic integrity is not always presented as the main quality of sound. The understanding of the richness of the noise nature of the sound material was obtained by the composer through a listening experience: Once — I believe it was 1986, high summer — I came on something remarkable while on a walk through the fields East of Vienna near the Hungarian border and close to the birthplace of Haydn. The corn stood high and it was just before harvest. The hot summer east wind swept through the fields and suddenly I heard das Rauschen (noise/the sound). Although it was often explained to me, I can still never say how wheat and rye are different. But I heard the difference. I believe it was the first time I really heard outside an aesthetic circumstance (say, a concert). Something had happened. Before and after were categorically separated, had nothing more to do with each other. At least it appeared to me then that way. In hindsight I recognize/remember other comparable experiences that had to do with a jerking open of perception, but the walk through the corn fields was perhaps the most momentous. For one way or the other, it seems to me, all the pieces I’ve made since have to do with this experience. Even the pieces not dedicated to noise, or those played with traditional instruments, etc. [1]

In the noise sound, Ablinger is interested in its maximum informative density, the so-called “white” all-frequency noise. The composer doesn’t like when his music is referred to as “noise music”. For him the expressions “noise” and “music” that come together in this context, normally are exclusive. He prefers the German definition Raushen. In English, as the composer believes, there is no term for “white noise” or “static noise”. What he considers, is simultaneously, sound noise and sound monochrome qualities, as well as maximum density of sound information, as well as its abundance, above all, it is about individual perception and listening observation. The composer is one of the few, who uses noise not as a synonym for chaos and entropy, but in a different way, as transgression, or as a projection of the “experience of the impossible”. White noise is a summary of all the sounds, all-frequency noise. Tiny sound nuances and delicate situation of the sound perception allow to gain balance between noise and silence. Ablinger is one of the few contemporary painters who uses this noise. Studying the possibilities of manipulation of white noise and its use as a means of sound material, the composer was trying to materialize those

74

S. V. Lavrova

sound elements that were hardly perceivable. Originally the idea seemed to have no sense, as white noise is considered to be part of all frequencies, as a result of it, sound directors would have all experienced skeptical attitudes in the process of these experiments, evaluations, and assessments. However, in the end, they proved not to be pointless. The main element here was space or a concrete recipient’s position. In one of the sound installations of the composer, the white sound was played in a smaller space through two loudspeakers. With the help of filtration, the white noise is divided into two parts, each of which was projected by one of the loudspeakers. The summary of these two sound sources didn’t depend on whose channel chose the auditory analyzer. Such experiments changed the existing frequency spectrum, in any position this was the same “white” noise. However, each listener was able to understand the process of the sound stream division from one channel to the next, and this is that very “experience of the impossible” that so many sound directors opposed to. After such experiments Ablinger started working on a series of “White” compositions Weiss/Weisslich. In his extended study of noise in nature and in the natural world, as well as the limits of sense, sound, and noise, Ablinger solves a problem of the interaction between a double-edged romantic or postmodern irony. He creates music that no one hears. Only impulses of our consciousness make it sound. On the physical level, we feel the vibrations, and this is a far stronger feeling of waves and movements of air than our everyday “hearing sense”, in which waves have just stopped existing and have turned into “sound” and “echo”. This is how we hear sea waves, we don’t feel them, yet we feel the frequency that one wave follows another one. The world of sound, their systematization and order, that we call “music” makes us pay a higher price; we don’t notice what is happening around us. We hear only ourselves. What we perceive is only the result of our brain functioning. This is a limit that Ablinger tries to take us beyond. The imagined music and inner musical experience—this is a creation of the musical image in the absence of sound, spontaneous or voluntary. The imagined music could come from the initialized with mental effort, or could appear spontaneously, or, in some cases, with a contextual reference to other stimuli or situations, that we could appear in. Perhaps, it also happens as a support of other cognitive functions. In the piece by Ablinger of the 2012 (Das Wirkliche als Vorgestelltes), a persisting static noise complicates the understanding of the text. Ablinger uses the destructive potential of sound to sever the connection between the vocal representation and interpretation. To achieve the sound reality through the noise border, to represent the real as the imagined, and then to compare them: is this not the experience of the impossible?

Music as “Impossible Experience”: Multisensory Perception …

75

3.3 Conceptual Space and Experience of Turning Noise into Musical Sound In the short story of a science fiction writer Olaf Stapledon The World of Sound, the exact same idea as Tolkien is repeated, including that of Silmarillion. This is a dive into an extraordinary world of thinking “musical” beings of the wave nature. Imagining the sound world in this way, therefore, Stapledon materializes it and creates a special conceptual space [16], soundscapes of different landscapes from public places with the help of a computer are turned into quasi-noise material within musical structures in the framework of real time. A combination of different computer processes is the basis of the piece: one aims at forming a wide scope of formal harmonic structures from raw sound materials and the other focuses on the creation from these materials something like a musical phrase. The idea is that any recorded sound can be transformed into a series of chords alternating with quasi-melodic fragments, gradually transforming from a simple harmony to a more complex noise structure and then returning back to the sound. The material consists of two field recordings: one at Wilmersdorf in Berlin and the other at the Gare du Nord in Brussels. Moreover, the machine itself, by imposing large and small formal structures, serves to balance these sources, or at least allows the listener to find points of comparison between them. The transcription process is involved in the transformation of electronic sounds into traditional notes for a chorus and string quartets. This process is a kind of “science fiction”. It is “music” that is not related to the expectations of the listener, but rather generates a real-time “translation” of the music of the noise space into the “language” of traditional sounds. Crossing the boundaries of the worlds of the “noise space” and traditional “music composition” forms a “horizon of expectation” in the perceptual field of the listener. The sound appears from the background noise or landscape, and later disappears into a more complex and vague phase which is no less musical from the point of view of contemporary music perception.

3.4 “Epiphany of Sound” and the Possibility of Pre-Listening A different effect is produced by the moment of sound manifestation or the “epiphany of sound”: crossing an obscure phase of silence, the sound comes out of silence, but the moment of its appearance is felt differently by each listener. Thus, what exists as an imaginary sound, on the other side of sound reality, is an inner musical experience. Sound epiphany is a metaphor for the sound image in the music of the Italian composer Salvatore Sciarrino. Edmund Husserl represented the depth of the experienced (the so-called “inner” or “perceptual”) time as a “horizon of consciousness”, which is separated by a line of chronological time. Time horizons or horizons of the perceptual field are given the status of the original. Perception always occurs in both the external and internal horizons. “Behind” the inner horizon

76

S. V. Lavrova

the outer horizon of things unfolds, formed by their co-presence. All private horizons merge into a single total horizon, called the “outer world”. E. Husserl points out the prospects of this new synthesis. The idea of Husserl’s horizon of consciousness is in the title of S. Sciarrino’s composition Muro d’orizzonte (1997). “The preliminary sketches, which I often preface to editions of my compositions, could be called “sound cards”. Muro d’orizzonte—is the name of this work in the form of an oxymoron. An image, a riddle, something that creates spatial illusions in the mind… […] In our consciousness, the illusion of space and the space of illusions create certain representations. Sound cards create diagrams of circumnavigation around the archipelago of the composition and their creative abilities. Cards that are knocked out of the way, to reorient us in the right direction” [15]. “A slow move from silence to sound—material that all compositions are centered around»,—an Italian composer says, “the instability that connects and separates three moments, the generation of sound, its life and its fading. A sound is given birth to and it dies. We represent ourselves in a sound image, that becomes a philosophical conception” (..) When a sound comes from nowhere, it crosses a foggy phase. Where is the border that separates the sound from silence? Undistinguishable and impossible moment incorporates the last vibration of the previous sound and the first of the following one. Paradoxically, the perception starts the philosophy of silence. This event is called epiphany (epifanico). Created from materialized sound image and strengthened by “presence”, this kind of music aims at creating the phenomenology of the invisible and the disembodied. The sound and silence outline a hermeneutic circle that opens the horizon of expectation” [15, 16]. “Epiphany of sound”, is a phenomenon that forms an important moment in the aesthetic-philosophical system of S. Sciarrino. This metaphor was also a central concept in the early aesthetic theory of Joyce [14]. In the Character Stephen by J. R. R. Tolkien, Joyce defines what he calls epiphanies: “By epiphany, Stephen meant a sudden spiritual manifestation-whether in conversation, in gesture, or in the course of thoughts worth remembering. He considered it as a matter worthy of a literary man to register these epiphanies with extreme care, considering them to be very delicate and fleeting states of mind.” [14] Similar “sudden spiritual manifestations” are the “sound phenomena” by S. Sciarrino, where every moment is a perfection of form. The “horizon of expectation” and questioning are the possibility of illusory comprehension of meaning, which is actually the process of expectation, the question unanswered, an ellipsis. Epiphany is the focus in which the phenomenon of sound is presented to the listener. J. Joyce wrote about this in the words of Stephen: “Imagine my glimpses at that clocks the gropings of a spiritual eye which seeks to adjust its vision to an exact focus. The moment the focus is reached the object is epiphanised. It is just in this epiphany that I find the third, the supreme quality of beauty” [14]. The idea of the horizon of expectations in perception of music by a listener in the ‘50 s was developed in the works of a famous musical critic L. Meyer, in particular, in his book Emotion and Meaning in Music. In the glossary of the book Sweet Anticipation: Music and the Psychology of Expectation by Huron [12] the term “Sweet Anticipation” is defined by a famous Canadian researcher in the field of the

Music as “Impossible Experience”: Multisensory Perception …

77

psychology of music perception as a feeling that appears as a result of a reflection about forthcoming events. Huron in his understanding of the anticipation focuses on subconscious processes related to music expectation. He states that “Over the eons, brains have evolved a number of mechanisms for predicting the future. The biological purpose of these mechanisms is to prepare the body and mind for future events while simultaneously minimizing the consumption of metabolic resources” [12]. Nevertheless, his understanding is based on psychological theories, models, and methods. The instruments for understanding such music are based on a serious attitude to the subjective musical experience: modelling of musical perception in the process of music study on the basis of many cultures and understanding of cognition in evolutionary context. The object of anticipation, as Huron states, is “the event in time”: “The object of expectation is an event in time. Uncertainty accompanies not only what will happen but also when it will happen. Sometimes the when is certain but not the what. Sometimes the what is known, but not the when. Later, we will see how music manipulates both kinds of uncertainty, and how the different what/when combinations produce different emotional responses” [12].

3.5 The Experience of Multi-sensory Perception, Receptive Chain: Hear-See-Feel-Understand The experience of a multi-sensory perception, the possibility of light and darkness connotation and interpretation in the music of Swiss composer Heorg Fridrich Haas— this is not an extra effect, but something far more significant. When interpreting darkness and light we perceive a certain stimulus of music perception, as an attempt to consciously recreate what contemporary people have lost, that is complete darkness. In one memory of the composer, he describes a near death experience in childhood, related to a bright light. This influenced his musical interpretation of the after death. In the philosophical conception of Pascal, that one could interpret as a precursor of existentialism; death, eternity, fear are all related, and form a special existential knot: these are the notions that relate to every minute of a human experience. Heidegger doesn’t consider the source of light to be the highest point, but, what is hidden from light, and that is what allows the light to be visible [7] (Heidegger 1994, 17). Darkness, according to Heidegger [9], is not the absence of light, like Emptiness (Vanitas), a metaphysical analogous entity, similar to darkness, this is not just an absence of being. The touch or relation to Emptiness, according to Heidegger, happens in the state of horror, and is different from fear or anxiety [10]. This is existential fear or trembling are at the core of the music by Haas. Apart from the reflection of existential essence of the music by Haas, that is given through a number of feelings, an extra effect of the contrast of light and shade is the fact that you perceive the music more intensively if you are not distracted by visual effects. This could be different from the contemporary tendency usually opting for visual effects.

78

S. V. Lavrova

Turning off all the lights in an artistic representation may seem less radical to some of us than what actually happens. However, this seems to be hardly a feasible task, on the verge of the possible in the context of a concert of acoustic classical music, in contrast, for example, to electronic music recorded in advance. Performers must memorize very difficult sound material. There are listeners for whom the experience of being in complete darkness can be a difficult psychological test. Yet in most cases, the listeners claimed that the fear/discomfort/horror subsided as soon as the music started. Haas’ attraction to darkness stems from early childhood: he grew up on the “dark side” of a high valley in the Austrian Alps. His relationship with nature similarly reflects the complexity of his relationship with darkness. Darkness, like fire, hides many conflicting and parallel interpretations. It calms us, and frightens, and burns, purifies and eradicates the negative, darkness intensifies the feelings and terrorizes us. This is our basic condition and our worst fear, it symbolizes peace, calmness, but also it could be the symbol of regression and backwardness. The spaciousness of the music we experience with our sophisticated hearing aids is often confirmed by visual effects. The lack of the opportunity to see the performers who work on the creation of music, isolate the listener, and makes them experience a low level of security, an inability “to see with your own eyes”. In connection with darkness, Haas has an association with a phenomenon known as the Molyneux’s problem—a philosophical question of thought experiment [3]. So, if a person born blind can feel the shapes of objects, such as a sphere and a cube, can he, having received the ability to see (but without the help of touch), determine these objects only with the help of sight, correlating them with what he gets through tactile presentation? Can a listener who perceived music in complete darkness, feel it in the same quality in the light? There was such an experiment, when in 1950 the English zoologist and neurophysiologist John Zacharias Young [17] said in his radio lecture that when one person was shown an orange a week after his epiphany, he said that it was golden, and when he was asked what shape this object was, he answered: “Let me touch it and I will tell you!” The theory that blindness—or simulating it with darkness—serves to enhance our other senses is highly relevant today [4, 18]. The antonym of darkness is light. Darkness is interrupted by an aggressive burst of bright light. The light turns on and off abruptly. By juxtaposing darkness with such bright and artificial lighting, Haas demonstrates this opposition at its most extreme. Even during his studies, Haas overcame his dependence on rational carefully constructed forms, and focused his search for new musical content, projecting it onto a specific understanding of light and shade, which became for him not only a means of conveying a wide range of emotions and sensations, but also performed a thematic almost leitmotiv role, moving from an external level to an internal one. Hearing— Seeing—Feeling—Perceiving—Realizing—this semantic chain is projected into the receptive field that is formed in Haas’ music. The listener, as the final link in sound communication, is guided by this series of sensations, which (excited by the highest emotional intensity and intellectual tension of Haas’ music), set their creative imagination in an endless motion.

Music as “Impossible Experience”: Multisensory Perception …

79

3.6 Light as Sound Hong Kong-American composer Viola Yip creates silent pieces that are realized only with the help of light, but he is very rhythmic and thus can create music, even if there is no sound as such. The composer’s interest in unorthodox sound materiality, performativity, relativity, and an emotional aspect in music led her to create instruments. Creation of instruments out of everyday objects, encourages her to rethink the possibilities of modern musical instruments, offering an alternative vision and exploring the possibilities and abilities of the recipient in a new environment. For example, her cycle of audiovisual works Vibrations vivantes (2016–2017) for light bulbs and photobots, which explores the creative possibilities of the intersection of light and sound in a musical composition. How to use light as: an instrumental interface, a visual representation element, a shadow generator, or a musical medium? The tool interface performs a twofold function. At the beginning of the work light switches the tools. At the moment when performers create various percussive sounds using light switches, the light (when it is on) triggers electronic sounds through a light sensor and microcontroller, resulting in a change of pitch in sound, depending on the intensity of the light and its proximity. The on and off state of these light sources creates different color effects, and shadows of the performers are also formed, which creates a performative addition to the function of the light sources. Sounds and light elements are connected to each other. Light in this cycle acts as a musical medium. And, despite the fact that there are literally no sounds, the light creates the illusion of sound: it stimulates our visual receptors in the same way as sound waves. By creating this multisensory composition, in which sounds and light are incorporated into the musical field of the composition; understanding light as sound enriches our musical experience and transforms it into a multisensory one.

4 Conclusion Complex perception of music, as well as the perception of its separate elements is a complex process. This new music offers an experience that allows the recipient to leave the limits of stagnated and conservative ideas; instead address the transgression experience, which allows to reach what seemed to be impossible. By offering the listener the experience of the “impossible”, the composer could represent his own musical experience. With the philosophy of Maurice Blanchot, the idea of the impossible appears as a thinking process in the space of non-existence. Blanchot talks about “the meeting place”, when someone, disappearing, aims at meeting himself in the meeting point of his absence [2, 11]. This is a new multi-sensory and space experience that becomes a starting point to attain the “impossible” of the “beyond perception” in the works of music by the twenty first century composer.

80

S. V. Lavrova

References 1. Baier, C., Ablinger, P. English texts transl. by Bill Dietz / URL: https://ablinger.mur.at/engl. html. Accessed on 10 Jan 2021. 2. Blanchot M. The Most High. Transi. A. Stockl. Lincoln and London: University of Nebraska Press (1996). 3. Degenaar, M. Molyneux’s Problem: Three Centuries of Discussion on the Perception of Forms. Kluwer Academic Publishers (1996). 4. Deliège, I. La percezione della musica, in Enciclopedia della Musica, vol. IX, Torino, Einaudi, pp. 325–328 (2002). 5. Faritov, V. Transgressiya i transcentenciya v muzyke. Ontologichesky analiz [Transgression and transcendence in music. Ontological analysis] (in Russian) // Bulletin of the Tomsk State University. No. 370. pp. 52–553 (2013). 6. Foucault, M. “A Preface to Transgression,” in Language, Counter-Memory, Practice: Selected Essays and Interviews, ed. with an introduction by Donald F. Bouchard, trans. Donald F. Bouchard and Sherry Simon (Ithaca: Cornel University Press) (1977). 7. Heidegger, M. Heraklit. Gesamtausgabe 55. Frankfurt: Vittorio Klostermann (1994). 8. 8. Heidegger, M. The Essence of Truth: On Plato’s Cave Analogy and Theaetetus. Trans. by Ted Sadler. London and New York: Continuum (2002). 9. Heidegger, M. Off the Beaten Track. Ed. and trans. by Julian Young and Kenneth Haynes. Cambridge and New York: Cambridge University Press (2002). 10. Heidegger, M. Bremen and Freiburg Lectures: Insight Into That Which Is and Basic Principles of Thinking. Trans. by Andrew J. Mitchell. Bloomington and Indianapolis: Indiana University Press (2012). 11. Huron, D. Tone and voice: A derivation of the rules of voice-leading from perceptual principles. Music Perception, №19 (1), 1–64 (2001). 12. Huron, D. Sweet Anticipation: Music and the Psychology of Expectation. Cambridge, Massachusetts: MIT Press (2006). 13. Imberty, M. Les écritures du temps. Sémantique psychologique de la musique. Tome 2. SACCoM. Buenos Aires. p. 13 (2009). 14. Joyce, J. Stephen Hero, Theodore Spencer ed. New York: New Directions Press (1944). 15. Sciarrino, S. Carte da suono scritti (1981–2001). Palermo: CIDIM-Novecento, p. 201 (2001). 16. Stapledon, O. A World of Sound Lulu Press, Inc, 2016 (1936). 17. Young, J. Z. Reith lectures 1950: Doubt and Certainty in Science Lecture 4: The Establishment of Certainty TRANSMISSION: 23 November 1950 - Home Service URL: http://downloads. bbc.co.uk/rm, http://radio4/transcripts/1950_reith4.pdf. Accessed on 10 Jan 2021 (1950). 18. Young, J Z. A Model of the Brain. William Withering Lectures. Clarendon Press (1964). 19. Zhukova G.K. Veselaya nauka vs. zlaya mudrost’: o chelovecheskom imerenii muzykalnogo prostranstva [Gay Science vs Awesome Wisdom: to the Human Dimension of Musical Space] (in Russian). International Journal of Cultural Research 3 (28): 65–72 (2017). 20. Zhukova G.K. Reprezentaciya nacionalnogo v evropejskom musykalnom diskurse [Representation of the National in the European Musical Discourse] (in Russian). St. Petersburg State University Press, St. Petersburg (2014).

Kenneth King’s Metapoiesis as a Form of Intertext Ekaterina V. Levina

Abstract The article is devoted to the reflection of the poststructuralist concept of intertextuality in postmodern dance, or rather, in the choreography of the American choreographer Kenneth King. The first section deals with the concept of intertextuality in post-structuralism. According to the author, in art history the concept as an established phenomenon has not been defined yet. Despite this, the article goes on to consider the concepts of intertextuality and intertext. The poststructuralist idea of “the world as a text” was the dominant idea in the art of the second half of the 20th century. It is not only defined the direction of the art development, but also changed the artists’ consciousness. In section two the author analyzes the concept of intertextuality in postmodern dance. K. King can be considered to be one of its brightest followers, who creates metaphor productions, in which elements are texts, that interact with each other. One of his productions was designated by the choreographer as “a note to metapoiesis”. Subsequently, it was metapoiesis that became the subject of study for the author. The author offers understanding of metapoiesis as a transitional state of order from the original to the new. She analyzed the transition principle and the dialogue of various texts in such productions as “Cup/Saucer/Two Dancers/Radio”, “Blow-out”, “Print–Out”, “RAdeoA.C.tiv(ID)ty”, “Dance S(p)ell” and “PHI Project”. The author concludes that the choreographer’s metapoiesis is a multi–level code as a complex intertext. Keywords Poststructuralism · Concept · Intertext · Intertextuality · Game · Choreography · Postmodern dance · King

E. V. Levina (B) Vaganova Ballet Academy, 2 Rossi St, 191023 Saint-Petersburg, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 T. Chernigovskaya et al. (eds.), Language, Music and Gesture: Informational Crossroads, https://doi.org/10.1007/978-981-16-3742-1_7

81

82

E. V. Levina

1 The Poststructuralist Concept of Intertextuality 1.1 The Definition of the Concept Postmodern art reflects many concepts of the poststructuralist philosophy: spaces, structures, games, words, rhizomes, grids, etc. A concept is a mental entity in which all the necessary information is gathered together [1]. Stepanov [2] defines it as a “cell of a person’s mental culture”. In the article we do not set ourselves the task of offering the full range of definitions, but we intend only to list some of them, which will allow us to formulate the idea of a concept. In their monograph “Semantic and Cognitive Analysis of Language” Popova and Sternin use the concept as a discrete mental structure, which is the basic unit of the human thought code. The authors note that (1) it has a relatively ordered internal structure, (2) it results from a person’s and society’s cognitive activity, and (3) it also reflects information about the described object/phenomenon and on how the public consciousness interprets it and how it relates to the concept [3]. Telia stresses that the concept includes not only the essential features of the object (as the definition does), but also those, which for this very language group are filled with knowledge about the essence [4]. The term “definition” refers more to philosophy and logic, the term “concept” is widely used in cultural studies, Stepanov says [2]. The term “concept” is associated not only with cognitive linguistics (the earliest Russian studies date back to the beginning of the twentieth century), but also with philology, cultural studies and philosophy. The philosopher Neretina in her article “Concept” [5] emphasizes the importance of the concept for cognition of reality. The French postmodern philosophers Deleuze and Guattari defined philosophy itself as “the art of forming, inventing, and producing concepts” [6]. For them the concept is a multidimensional and multilevel phenomenon, which is built on a rhizomorphic basis. In art history the concept as an established phenomenon has not been defined yet. Nevertheless, we can get closer to understanding the concept by studying the works of famous art critics. In order to do this, we should refer to the studies of the modern art theorists—Greenberg, Krauss, Lehman (contemporary theater art), Fischer–Lichte (contemporary theater art) [7–9]. The structure of the concept is heterogeneous. The concept itself has no clearly defined boundaries. It consists of the core and periphery, which have different associations. It should be noted that it is a key nomination of a cultural fact, but is limited to the consciousness of the fact holder [2]. This is important to remember when further analyzing the concept under study. Stepanov divides the concept into 3 components: the essential feature, “passive” features and the internal form. While the first component exists for everyone, who uses the language, the second one exists only for some social groups [2]. The internal form of the concept is just becoming the subject of research. Popova and Sternin

Kenneth King’s Metapoiesis as a Form of Intertext

83

believe, that the structure of the concept comprises the core layer (core, image), information content and the interpretation field [10]. The system of concepts is fragmented: the structure of micro-concepts in its division into components is similar to macro-concepts. One can find many microconcepts in poststructuralism. For further research we must understand the microconcept of the game. The game is a significant phenomenon in the philosophy of deconstructivism. It is a competition between the subject and the text, no matter what this text is (a literary text, image, musical text or a dance) [11]. Derrida points out that “a game is always the game of absence and presence” [12].

1.2 Intertext and Intertextuality In this article we are going to analyze the concept of intertextuality and intertext. The term “intertextuality” was introduced by Kristeva. She defines it as a means of analyzing a literary text or describing the essense of literature [13]. Moreover, the term includes the definition later designated as a postmodern sensibility. When describing the concept of intertextuality, the researcher relied on the work by Bakhtin “The Problem of Content, Material and Form in Verbal Artistic Creativity” [14]. The author emphasizes that modern literature is in constant “dialogue” with that of the past. That idea of a “dialogue” formed the basis for the idea of a “dialogue” of texts as such. However, Kristeva perceives it exclusively as a literary idea, which was later studied in the works of structuralists and poststructuralists. Thus, there was established the idea of “the world as a text”, in which everything was a text: works of art, cultural phenomena, philosophical trends, the history and development of human society. From this perspective all cultural richness becomes a single intertext, serving as a pretext for new cultural texts. Kristeva writes, that this process is unconscious in its nature and is characterized by an autonomous existence [13]. In this context the human mind is completely immersed in various textual practices. This is done through the process of decentralizing a literary text, the dissolnig author in the citation writing and, in general, depriving him or her of the centering role, which reminds us of Barth’s essay “The Death of the Author” (1967) [15]. As Lavrova notes, the author, the text and the reader—in the process of reading a particular text - all become an endless field for the game of writing [16]. Within the framework of the idea of “the world as a text” the world is a variety of texts, that, combined with each other, create new texts. In Barth’s work, the text takes the form of an echo chamber [15], in Riffaterre’s work—of an ensemble of presuppositions of other texts [17]. In Gresset’s work, the concept of intertextuality is presented as a sign of literary activity and an integral part of culture as a whole [18]. In their work “Introduction to the linguistics of text” (1981) De Bogrand and Dressler reveal the relationship between textuality and intertextuality, which subsequently leads to the destruction of the very concept of a text [19]. Grivel also states,

84

E. V. Levina

that “there is no text except intertext” [20]. At the same time, Dellennbach and Van der Heuvel use intertextuality as a dialogue within a certain text, so they study the interaction between “one’s own” and “someone else’s”. In “Palimpsests: Literature in the Second degree” (1982), Genette defines the concept of intertextuality as “the co–presence of two texts”, transformation of one text into another, and offers his own classification of text interaction, in which intertextuality is one of the following types: 1. 2. 3. 4. 5.

intertextuality; paratextuality; metatextuality; hypertextuality; architextuality [21].

Broich, Pfister and Schulte–Middelich in their work “Intertextuality: forms and functions” distinguish between the following forms of intertexts: borrowings, plot developments, explicit and implicit citations, translation, plagiarism, allusion, staging, film adaptations, etc. [22]. They also deal with the functional significance of intertextuality. Thus, the concepts of intertext and intertextuality came as a result of philosophers’ reflection not only on the ideas of poststructuralism, but also on postmodern culture and art at large. They were seen as the answer to many questions, that appeared within the artistic development of the twentieth century, and influenced both, the artistic process and the consciousness of the artist.

2 Intertextuality in Postmodern Dance 2.1 Poiesis and Metapoiesis Based on the idea of “the world as a text”, dance can also be a form of intertext. Postmodern choreographers have considered the concept of intertextuality in their works. We are going to analyze the works of Kenneth King, choreographer, who, in our opinion, most clearly uses the concept in his performances. As Banes writes, for him a text is a kind of secret code, in which many meanings are hidden [23]. It could include sketchy outlines for metagrammar, metatheology, metamachine and metapoiesis. Poiesis in Greek (πo´ιησις) means creation, production. This is an archetypal structure, whose meaning lies in the primary emergence as a phenomenal pledge of the event. Poiesis becomes a proto–action of human creativity; whatever format it might be implemented in. Surina writes that poiesis is the original sense–forming assumption in the existential chaos, a breakthrough in the homogeneous infinity [24]. It is in poiesis, that we observe the dynamics of repetition and interaction between the reflecting and reflected, which allows us to perceive the world as a set of reflections multiplied by each other. Poiesis collects

Kenneth King’s Metapoiesis as a Form of Intertext

85

heterogeneous manifestations of various events, not allowing them to disintegrate, but, on the contrary, to find balance in the whole action. ´ meaning “between, after, through”, The part “meta–” from the Greek (μετα), stands for “intermediate, change of state, transformation”. Thus, we understand metapoiesis as a transitional state of order from the original to the new. It should be noted that in this case it is only the transition that can be developed; the new state in the homogeneous infinity ultimately turns out to be unpredictable.

2.2 Principles of K. King’s Choreography For King as an American postmodern dancer and choreographer of the Judson Dance Theater second generation dance is seen as a puzzle that must be solved after a thorough analysis. At the same time his productions are “part of a total theatrical experience” [23]. The metaphors, used by the choreographer, are complex, fragmented and unclear. He believed that postmodern dance, presented to the audience as pure dance, could become a platform for interaction between various dance techniques, which would create a synthetic form of dance theater that reflected the ideas of its time [25]. Anderson notes that the idea and thought became the main actors in choreographer King’s productions [26]. His early works were particularly imaginative and outrageous. In “Cup/Saucer/Two Dancers/Radio” (1964), dancers in their underwear performed a series of repetitive movements in front of the audience, using the objects mentioned in the title. In the audio recording, accompanying the production, King emphasized the equal importance of the dancers and the objects. Critics described the performance as a picture of the middle class family life, in which the man had a lustful mind, and the woman wanted to avoid hard work around the house, dreaming of becoming a ballerina [27]. Even in his earliest productions King strove for a “dialogue” of various texts— plastic, verbal, and figurative ones. We can consider “Blow-out” (1996) as an example. Like “Cup/Saucer/Two Dancers/Radio”, it involves two dancers. The woman tries to pick fruit glued to a Cubism style table, and the man, dressed as a biker, is tied up to the wall with elastic bands extending from his fingertips. Glass balls fall out of their mouths. It adds a new meaning, a new code to the existing picture. The entire performance is accompanied by Descartes’ arguments on sense perception, fragments of Vivaldi’s works and sounds of an electronic siren. For King the performer’s body becomes just another means of transferring an image in the production. Before him, it was the main intermediary between the choreographer and the audience He also actively uses words, objects and techniques, playing with them and creating his own information field, into which the viewer gets immersed. In this field improvisation becomes an important element for movement. Starting with a certain set of movements, the choreographer combines the flow of emerging ones according to the principle of a linguistic model. Subsequently, those movements build up choreographic phrases, sentences, reflections, like letters of an

86

E. V. Levina

alphabet. The choreographer is not interested in the result—but rather in the process of creating a choreographic text. Therefore, using the kinetic, scenic and poetic analysis of movement, he attempts to create metagrammatics of dance. He believes that “movements, the very act of dancing become a way of thinking” [28].

2.3 “Dialogue” of Texts in K. King’s Metapoetic Productions On the one hand, King’s performances are attempts to split the soul and body, language and gesture, but on the other hand they are metaphors of the mind, technology and information systems. In his metapoiesis we can find an infinite variety of secret codes that can’t always be deciphered. In his works, King transforms the real space with the help of various semiotic and technical means, only imagining how this can affect the audience. The result is unpredictable. The idea of intertext, a “dialogue” of texts is expressed in his productions more than just vividly; for King, all their components—body, plastique, voice, music, technique –are texts that, interacting with each other, reveal a new text to the world. Examples are such works as “Print–Out” (1967–1968), “The Dancing Jewel” (1972), “Metagexis” (1973). In the former we also find the subtitle “(CELE ((CERE))– BRATIONS: PROBELlems & and SOULutions for the dyEYEing: KING)”. The only movement In the performance was the choreographer’s entering the stage, wearing a hoodie and carrying a bucket, from which he took out a rubber snake and a lizard, all the rest was playing with texts and meanings. The small space did not imply any large-scale actions. The text was projected on a screen throughout the performance, with the voice-over reading it out. Thus, King offers us to plunge into a series of language adventures, with words and phrases splitting, combining, producing new unexpected combinations, generating nonsense, creating double or triple meanings. For a choreographer playing with words and a text is a process, showing that the word and text are another secret code, that can contain hidden multiple meanings of anything. He creates performances in which the language, plastique, sound are split into parts, supplemented with new elements, turned inside out, becoming a secret code, containing a huge number of different meanings. So in “RAdeoA.C.tiv(ID)ty” (King’s 1978 version) the title itself implies complexity of the structure. In the performance twelve dancers moved along geometric trajectories, emphasizing their body diagonals for the audience, studying gestures and isolated movements. While a female voice-over was reading excerpts from M. Curie’s “Radioactive Substances”, in the background the audience would see pictures, showing possible consequences Pierre and Marie Curie’s discoveries could have. The performers plastically duplicated the splitting of chemical elements, described in the text read. Symbolic choreography, isolation of movements combined with excerpts from “Radioactive substances” by Curie and observation of the scientists’ discoveries projected on the background—all of this is offered to the audience as a multi–level code that needs to be decyphered [23].

Kenneth King’s Metapoiesis as a Form of Intertext

87

The study of space, its subsequent encryption with new secret meanings can be observed not only in the work discussed above. King used a similar principle in “Dance S(p)ell” (1978). In this performance the study of space was dedicated to Nikola Tesla. Eight dancers in the same way as in “RAdeoA.C.tiv(ID)ty” move along geometric trajectories making swirling steps and light jumps. This performance succeeded in demonstrating the relationship between technology and the human body, expressed in interaction between various structures and actions, by using the pure movement, principles of classical dance, accompanied by electronic music by William-John Tudor, with whom the choreographer worked closely [23]. In “PHI Project” (1981), the choreographer continued to explore the thinking process, offering it as a code that the audience needs to unravel. The performers moved in semi-darkness, surrounded with slides and films with anatomical drawings, scientific diagrams, architectural facades and statements written in illegible small handwriting. Anderson suggests that the images shown could be metaphors for free associations of the mind, while the dancers depicted the process of free associating [26]. All of this created for the audience the image of the mind that dreams, plays and reflects. In “PHI Project” King was able to make thoughts visible by connecting them and creating a single organic plastic text through the human body. We can observe such coding in all King’s works. In conclusion we can say that the choreographer’s metapoesis is a complex intertext, offered to the audience as a multi–level code, even having solved which we will not be able to draw certain unambiguous conclusions opening the way to the knowledge of the universe as a macrocosm and the human body and mind as a microcosm.

References 1. Maslova, V.A. Problema natsional’nogo kharaktera skvoz’ prizmu yazyka. VGU im. P. M. Masherova, Vitebsk (2011). (Maslova, V.A. The problem of national character through the language prism. Monograph) (in Russian). 2. Stepanov, YU.S. Konstanty: Slovar’ russkoy kul’tury. Izd. 2–ye. Akademicheskiy Proyekt, Moskva (2001). (Stepanov, YU.S. Constants: Dictionary of Russian Culture) (in Russian). 3. Popova, Z.D, Sternin, I.A. Semantiko–kognitivnyy analiz yazyka. IS–TOKI, Voronezh (2007). (Popova, Z.D, Sternin, I.A. Semantic and cognitive analysis of the language) (in Russian). 4. Teliya, V.N. Russkaya frazeologiya. Semanticheskiy, pragmaticheskiy i lingvokul’tu–rologicheskiy aspekty. Shkola «YAzyki russkoy kul’tury», Moskva (1996). (Teliya, V.N. Russian phraseology. Semantic, pragmatic and linguoculturological aspects) (in Russian). 5. Neretina, S.S. Tropy i kontsepty. IF RAN, Moskva (1999). (Neretina, S.S. Tropes and concepts) (in Russian). 6. Deleuze, G., Guattari, F. Qu’est–ce que la philosophie ? Minuit, Paris (1991). 7. Krauss, R. Reshetki. Proyekt klassika 3, 23–28 (2002). (Krauss, R. The grids. Translated by D. Pyrkina) (in Russian). 8. Lehmann, H.–T. (2006). Postdramatisches Theater. Verlag der Autoren, Frankfurt am Main (1999). 9. Fischer–Lichte, E. Ästhetik des Performativen. Suhrkamp, Frankfurt am Main (2004). 10. Popova, Z.D., Sternin, I.A. Kognitivnaya lingvistika. AST: Vostok – Zapad, Moskva (2007). (Popova, Z.D., Sternin, I.A. Cognitive Linguistics) (in Russian).

88

E. V. Levina

11. Lavrova, S.V. Proyektsii osnovnykh kontseptov poststrukturalistskoy filosofii v muzyke postserializma. Kazan’ (2016) (Lavrova, S.V. Projections of major post-structuralist philosophy concepts in the post-serialist music) (in Russian). 12. Derrida, J. L’Écriture et la difference. Seuil, Paris (1967). 13. Kristeva, J. Semeiotikê. Recherches pour une sémanalyse. Seuil, Paris (1969). 14. Bakhtin, M.M. Problema soderzhaniya, materiala i formy v slovesnom khudozhestven-nom tvorchestve. In: Voprosy literatury i estetiki, pp. 6–71. Khudozhestvennaya literatura, Moskva (1975) (Bakhtin, M.M. The problem of content, material and form in verbal arts) (in Russian). 15. Bart, R. Izbrannyye raboty : Semiotika. Poetika. Progress : Univers, Moskva (1994) (Bart, R.Selected works: Semiotics. Poetics) (in Russian). 16. Perrone–Moises, L. L’intertextualite critique. Poetique 27, 372–384 (1976). 17. Riffaterre, M. La syllepse intertextuelle. Poetique 40, 496–501 (1979). 18. Gresset, M. Intertextuality in Faulkner. University Press of Mississippi, Jackson (1985). 19. Beaugrande, R.A. de, Dressier, W. Einfuhrung in die Textlinguistik. Niemeyer, Tubingen (1981). 20. Grivel, Ch. Theses preparatoires sur les intertextes. In: Dialogizitat, Theorie und Geschichte der Lit. und der schonen Kunste, ss. 237–248. Fink, Munchen (1982). 21. Genette, G. Palimpseste. Die Literatur auf zweiter Stufe. Suhrkamp, Frankfurt am Main (1993). 22. Broich, U., Pfister, M.: Intertextualität: Formen, Funktionen, anglistische Fallstdien. De Gruyter, Germany (1985). 23. Banes, S. Terpsichore in Sneakers: Post–Modern Dance. Wesleyan University Press, Hanover (1987). 24. Surina, T.V. Poezis kak arkhetip kul’tury. Vestnik Tomskogo gosudarstvennogo universiteta 316, 67–70 (2008) (Surina, T.V. Poiesis as a cultural archetype) (in Russian). 25. King, K. Toward a Trans–Literal and Trans–Technical Dance–Theater. In: The New Art, pp. 119–126. G. Battcock, ed. New York (1973). 26. Anderson, J. Kenneth King is a thinking man’s choreographer. https://www.nytimes.com/1981/ 05/10/arts/dance-view-kenneth-king-is-a-thinking-man-s-choreographer.html. Last accessed 14 Feb 2021 27. Johnston. J. Horizontal Baggage. Village Voice, 8 (1965). 28. Jowitt, D. Kenneth King: Our Only Dancing Philosopher? Village Voice. July 5, 118 (1976)

Verbalization of Music and Gesture

Non-verbal Behavior and Its Role in Narrative Production Polina Eismont

Abstract Mastering the skills of the interpretation of non-verbal behavior takes up all childhood. The factors that influence the explicit representation of characters’ non-verbal behavior in narratives were determined based on the analysis of unprepared oral stories by 125 Russian-native monolingual children aged from 4 years 7 months to 7 years 6 months and a control group of 19 Russian-native adults. These factors are the meaning of non-verbal acts, their importance for the plot progress and their dependence on the presence or absence of speech context. When representing a non-verbal act within the framework of an unprepared retelling, Russian speakers use any of the following strategies: describing the non-verbal act, interpreting it, identifying a situation that it can correspond to, or reproducing a possible speech context. Failure to perceive a non-verbal act results in either omitting or misinterpreting it. With the development of communication skills, the number of such failures clearly decreases. Russian-speaking preschool children recognize non-verbal behavior presented without a speech context, but tend to better interpret the episodes that are significant for the plot progress and, thus, perform a connecting function. In contrast to the narrative significant non-verbal acts, communicative non-verbal acts require a speech context and were less frequently present in the analyzed narratives, especially in the ones produced by younger children. Keywords Non-verbal behavior · Bodily expressions · Language acquisition · Narrative

1 Introduction Non-verbal behavior plays a significant role in the life of human society and its individuals. Facial expressions, body postures and bodily expressions, gestures and pantomime convey a variety of meanings that are significant both within the framework of specific communication and for the general development and socialization P. Eismont (B) St Petersburg University, Universitetskaya nab., 11, 190000 Saint-Petersburg, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 T. Chernigovskaya et al. (eds.), Language, Music and Gesture: Informational Crossroads, https://doi.org/10.1007/978-981-16-3742-1_8

91

92

P. Eismont

of a person. In the course of development, non-verbal behavior—the ability to reproduce it and to understand it in others—affects language acquisition [1–4] and science learning [5–7], socialization and academic performance in school [8–11]. As can be seen from the reviews in the cited works, the majority of studies analyze how children reproduce gestures in different situations, recognize and understand various co-speech gestures, and recognize individual facial expressions. At the same time, in recent years, many researchers have emphasized the need to study the entirety of non-verbal behavior and to shift from the analysis of individual gestures or individual facial expressions to the complex study of bodily expressions and pantomime [12–14].

1.1 Non-verbal Behavior: Nature and Function Non-verbal behavior can be directed towards both the recipient and the producer (cf. interpersonal vs. intrapersonal in [15]). In addition, it can be informative, that is, aimed at conveying some meaning, and communicative, that is, performing functions that are important for ensuring communication [16, 17]. Although some researchers suggest a more detailed gradation of its functions (cf. six different functions in [18]), it is not yet possible to determine the principal one, nor to reveal a clear correlation between a specific function and a certain type of non-verbal behavior. Understanding the non-verbal act (NVA), determining its meaning and function, as well as identifying the cognitive mechanisms that underlie this process, is far from being completed. By acknowledging non-verbal behavior as a multimodal phenomenon, researchers try to explain its role in communication as a result of the conceptual integration of mental representations [19], as a transformation of the visual input into the motor chain through mirror neurons [1], or as a replica of an action pattern typical for a particular frame [20]. In addition, understanding non-verbal behavior requires conceptual knowledge and symbolizing ability [21], a rather developed theory of mind [22] and imaginative thinking [6]. At the moment, there is no single classification of NVAs that make up non-verbal behavior. For a long time, facial expressions, gestures, pantomime and postures were studied separately, so each of these areas developed both their own terminology and classifications. Many researchers admit nevertheless that all these areas constitute a single continuum with vague boundaries within it [20, 23]. Gesture studies are especially diverse. Most often, the term gesture means only a certain movement of the hands (cf. a review in [5]), leaving the often-accompanying movements of the shoulders, head, and torso within the framework of bodily expressions research. The differences between these fields and types of gestures are most often described in terms of either the meanings they convey or the functions they perform. Among the gestures, emblems stand out with their clear fixed meaning being either international, or part of a specific cultural tradition. At the same time, many authors argue that such a stable meaning is also characteristic of bodily expressions [5, 22]. At the opposite end of the spectrum are the so-called adapters and conversational

Non-verbal Behavior and Its Role in Narrative Production

93

gestures, which ensure communicative effectiveness [15]. Gesture semantics and types of conveyed messages contribute to this discussion as well. Non-verbal behavior can convey a lot of information—semantic [15], grammatical [24], communicative [25], unintentional [26], emotional [12], etc. The option may depend on the context [27], but nonetheless all these data are still incomplete and not generalized. Intentionality is another aspect that distinguishes between the types of nonverbal behavior in general and types of gestures in particular. Emblems, iconic, representative gestures, although not always associated with a certain meaning by communicants, are more controlled and intentional NVAs than adapter gestures or beat gestures. Bodily expressions that reflect emotions and some internal intention or feeling that is often not even realized by the producer, and that, according to Witkower [14], can be attributed to biologically determined and innate ones, also include involuntary non-verbal behavior. This unintentional non-verbal behavior is poorly controlled, not discrete, and requires decoding. For instance, it is not clear if recipients comprehend it without such decoding by means of accompanying speech. Similarly, we do not know yet whether children perceive such biologically determined actions more easily than gestures conveying the meaning formed in the process of social interaction. Lately, pantomime piqued the interest of researchers investigating non-verbal behavior. The pantomime is mostly characterized by its enactment, robust iconicity, reference to events and whole-body movement [28]. Marentette [29] adds to this list the mandatory absence of accompanying speech, which leads some researchers to exclude pantomime from gesture-language studies [30]. At the same time, everyone agrees that, unlike other types of non-verbal behavior, pantomime corresponds to a separate independent event, which makes up a whole non-verbal text [20]. Pantomime is natural, spontaneous, capable of functioning as an independent communicative act and is largely metaphorical.

1.2 Non-verbal Behavior and Speech The majority of research on non-verbal behavior, especially on gestures, belongs to the field of gesture-speech interaction and concerns their common and distinct features and the relative weight of their contribution to the communication and information transmission. Most researchers believe that within the framework of communication, verbal and non-verbal components make up a coherent whole, which contributes to their successful interpretation [5, 6, 23, 25, 31]. Gesture and language work as partners, and the speaker can redistribute their roles even within one utterance to make one of the components more significant. At the same time, the authors note that the role of gestures is auxiliary, as they can clarify the content of the accompanying utterance to facilitate its perception or to emphasize something [24, 27] and change the literal content of speech, giving out the implicit intention of the speaker [25]. Speech is predominant, and the amount of information conveyed by gestures is apparently quite insignificant. According

94

P. Eismont

to Krauss [15], the information embedded in a gesture can be so insignificant that in the absence of speech, it is generally meaningless, and during natural verbal communication it reproduces the meaning of accompanying lexical units. Thus, co-speech gestures, which function as a supplementary channel in verbal communication and can convey some additional clarifying contextual or communicative (for example, the communication of speaker’s emotions) co-meanings or help in learning and education [3], are most widely researched. The gesture is multifunctional and can aim at both the speaker and the listener, making it easier for the latter to understand, and for the former to align her speech and to guide the listener according to her own logic [32, 33]. Other studies have shown that gestures, appearing in induced pauses when the speaker has difficulty finding a suitable word, serve as a linkage, demonstrating to the listener that the speaker has not yet passed the turn to her [15]. Studies of bodily expressions and pantomime show that a gesture may not be accompanied by speech, but convey some information perceived by the recipient. As a “non-verbal text”, pantomime also takes on the basic characteristics of a text, such as coherence and cohesion. It is yet to be established, though, whether the recipient perceives bodily expressions as a substantive element of non-verbal storytelling alone, or whether they also serve as a connecting element to fill in pauses. It is still unclear whether all bodily and facial expressions are equal for storytelling and plot comprehension.

1.3 Non-verbal Behavior and Language Acquisition Non-verbal behavior plays a significant role in the formation of social skills in children and in the development of their speech. The research on the perception and recognition of various types of non-verbal behavior (gestures, facial expressions, pantomime, and bodily expressions) by children in various situations, both with and without speech, shows that children react to non-verbal information from an early age. At the same time, despite the massive number of different tests and experimental techniques [8, 31], it is difficult to study how children process non-verbal information, since we cannot always say with confidence that children not just perceive non-verbal information, but also identify and understand it [5]. For example, studies by Bates and Dick [1] have shown that in the early developmental stages, children tend not so much to recognize the presented non-verbal information as to imitate it. In general, in typically developing children, the acquisition of verbal and non-verbal means of communication occurs in conjunction with one another [21]. However, children recognize gestures differently at various stages of development. Children gradually learn gestures based either on social interactions (cf. usage-based theory of acquisition [34]), or through unintentional reinforcement for some performed action [35]. Studies of children’s perception of different types of gestures show that before the age of four, children are better at recognizing more specific iconic and deictic gestures [33, 36], and later develop imaginative thinking that allows them to recognize abstract metaphorical gestures [37] and pantomime with imaginary objects

Non-verbal Behavior and Its Role in Narrative Production

95

[38]. Children reach an adult-like level of pantomime recognition by the age of six [39]. Furthermore, language acquisition plays a significant role in the perception of non-verbal information. The multimodality of communication leads to the fact that younger children experience difficulties when distributing attention between the verbal and non-verbal components, usually preferring one or the other. At a certain stage of development, children pay much more attention to verbal information than the visual one [40, 41]. Unfortunately, there are very few detailed studies of how the development of various language skills, the increase not only in the vocabulary size, but also in the variety of syntactic structures in the child’s lexicon, and the improvement of the cognitive mechanisms behind symbolization affect non-verbal behaviour recognition. It has been shown that the older the children become and the more their vocabulary grows, the more often verbal naming replaces non-verbal representations of an emotion or action [42]. According to many authors, children use bodily expressions and pantomime to compensate for language deficiencies, since non-verbal information can represent rather intricate information in an easier form and at a certain level of language development, children acquire it more easily than speech [43]. Nevertheless, the studies by Coletta et al. [18, 44] disprove this idea based on the analysis of such a complex speech product as spontaneous narrative and show that within narrative elicitation, both non-verbal and verbal development progress together.

2 The Role of Non-verbal Behavior in Narrative Production Thus, this study addresses the following research questions: – how important are bodily expressions as the elements of described action for event understanding, or are they perceived only as some means of cohesion between significant key points of a story? Are bodily expressions equivalent in terms of the role they play in the story? If such action elements are significant, then they will be mentioned during storytelling, and if they only perform a linking function, they will in most cases be omitted; – what means of verbalization do children use to reflect the bodily expressions in an unprepared spontaneous narrative? Do successful identification of bodily expressions and their interpretation depend on their importance to the narrative or on their meaning? – will bodily expressions as biologically determined emotions and meanings be recognized more easily at an early age? Or will the participants lack the speech component for their successful identification?

96

P. Eismont

2.1 Participants The analysis relies on data from a series of elicited storytelling experiments with Russian-native monolingual children and a group of adult Russian speakers. 144 volunteers participated in the study: 125 children of three age groups—4 years 7 months to 5 years 6 months (49 participants), 5 years 7 months to 6 years 6 months (41 participants) and 6 years 7 months to 7 years 6 months (35 participants)—and 19 adults (a control group). Including the control group in the analysis was necessary to understand whether the non-verbal behavior of the cartoon characters was understandable. All children went to preschool and had neither special language, nor narrative skills training. All children were typically developing. The experimental design was approved by the senior preschool teacher and the representatives of parents at an ethical commission. The parents signed a written informed consent before their children took part in the study. The control group consisted of the students from different Saint-Petersburg universities; they also gave their written informed consent before participating in the experiment.

2.2 Method and Procedure The cartoon “How to get big?” directed by Vladimir Degtyarev was created at the Soyuzmultfilm studio in 1967 using the technique of puppet animation. Not using still pictures or videos of non-verbal behavior as research material is a particular strength of the study. On the one hand, it makes the experimental task more natural, because children watch this type of cartoons from an early age, and on the other hand, it allows the readers to draw on these results for the research on children’s perception of animated computer agents [45, 46]. In the cartoon, there is one main character, a kitten, and four characters and groups of characters with whom it meets consequentially. All the events taking place in the cartoon may be divided into six episodes consisting of a different number of events. Ep.1. The kitten plays with cubes, but something does not go right, and it throws the toys around the room. Ep. 2. The kitten’s grandmother comes, she is unhappy with its behavior and the mess it has made, but she has to go to the store to get milk. Ep.3. The kitten is left alone and does not know what to do. Ep. 4. The kitten jumps out of the window and runs to the rabbits to jump over the rope, but the rabbits chase it away because it is too small. Ep. 5. The kitten sees the beavers who are building a dam and runs towards them, but they drive it away for the same reason. Ep. 6. The kitten meets a bear cub who advises it to climb a tree to become big, but this does not help; the bear cub leaves, and the kitten climbs down the tree. All of these episodes include different bodily and facial expressions that represent various narrative or communicative meanings. Since the cartoon is a sample of puppet animation, the characters have limited possibilities of conveying non-verbal information. Not every cartoon character performs non-verbal acts: rabbits and beavers do

Non-verbal Behavior and Its Role in Narrative Production

97

not make any gestures, bodily or facial expressions. The kitten, its grandmother and the bear cub use their hands that can move up and down, move forward, to the right or to the left. They can turn, tilt, lower and raise their heads. Their eyes can close and open, change gaze direction, and their mouths can open and close. When gesturing, the mouth is used as a base for the hand gesture (to cover the mouth, for example), and the mouth movements themselves are limited to the characters speaking. The kitten and its grandmother also have tails that they can wag. In total, 16 NVAs can be distinguished in the cartoon, matching some communicative action or emotion of the character (listed in Table 1). The experiment sessions took place at schools and at the university (with the adult control group). All sessions followed the same procedure: the participants were asked Table 1 List of non-verbal acts (NVA) in the cartoon Non-verbal act

Performing character

Bodily expression

Meaning

1

Kitten

Head up, mouth open, eyes winking, looks up

It is thinking

2

Kitten

Covers its mouth with its hand, eyes open, looks down

It is upset

3

Kitten’s grandmother

Shakes its head

It is displeased

4

Kitten’s grandmother

Swipes its hand down

It is discontent and in despair

5

Kitten

Lowers its head, eyes down, looks sullenly

It is ashamed

6

Kitten’s grandmother

Looks around

It is looking around searching for something

7

Kitten’s grandmother

Wags its hand (like a finger)

It is giving out instructions

8

Kitten

Head up, eyes open, covers its mouth with its hand

It is curious

9

Kitten

Head down, looks down

It is upset and hurt

10

Kitten

Raises its hand, looks around

It is in doubt

11

Kitten

Wags its tail

It is impatient

12

Beaver

Shakes its head, waves its It refuses hand

13

Kitten

Head down, tears drop from its eyes

It is upset and crying

14

Kitten

Shakes its head

It denies

15

Bear cub

Puts its hand to its mouth It is puzzled

16

Bear cub

Swipes its hand down

It gives up the communication, a kind of “forget it” meaning

98

P. Eismont

to watch the cartoon without any preliminary preparation and retell a four-minute fragment from it as it was happening on the screen. The experimenter turned her back to the screen and did not watch the cartoon with the participants to avoid influencing the responses [32]. The cartoon was shown in silent mode. Each session took place individually, and the responses were audiorecorded. Thus, I collected 144 unprepared oral narratives, all describing the same story from the cartoon.

3 Results and Discussion 3.1 Representation Strategies To answer the research questions, I determined the types of linguistic representation for each of the 16 NVAs in the narrative. In total, seven strategies were identified: description of bodily expression (GD), interpretation of bodily expression (GI), incorrect interpretation of bodily expression (GmI), imaginary bodily expression (IG), description of the situation corresponding to the given bodily expression (S), direct speech that could accompany this bodily expression (Sp) and omission of any description (Om) (the distribution of representation strategies between age groups is shown in Table 2). The description of bodily expression reflects the fact that the child noticed it, understood its possible importance for the story, but could not recognize its meaning (this corresponds to emotion perception [43]). The rest of the ways of explicitly presenting non-verbal behavior in the narrative show that the child not only saw a certain movement of the character’s body or facial expressions, but also tried to understand and explain it. The child did it either by conveying the meaning of this movement (GI), or by describing the situation to which it corresponds (S), or by restoring the accompanying utterance (Sp). The incorrect interpretation of the given bodily expression shows that the child understands that there is some meaning behind it and tries to interpret it, but for one reason or another, she has not yet succeeded. The same failed attempt to interpret the events in the cartoon leads the child to invent certain events to explain plot twists, resulting in imaginary bodily expression (IG). These five ways of bodily expression representation reflect the development of emotion cognition [43]. The absence of any mention of bodily expressions in the narrative suggests that the participant either did not notice it, or did not consider it important enough for the elicitation. The difference between the age groups in what strategies they chose to represent NVAs is statistically significant (χ2 = 172.853, df = 18, p < 0.001, Cramer’s V = 0.157). Most frequently, children of all age groups and the adult control group did not reflect bodily expressions in their narrative at all (as in [47]). However, the number of omissions noticeably decreased with age. This ignoring of NVAs may mean that the recipient misunderstood the meaning of bodily expressions, or decided that it is insignificant for the whole narrative (see the analysis of different NVA meanings

Total

Adults

3

2

1

Age group

94

4.04%

% within row

10.19%

Count

32

% within row

3.01%

% within row

Count

17

4.99%

% within row

Count

33

% within row

Count

12

1.52%

Count

GD

Representation strategy

7.95%

185

16.24%

51

9.57%

54

6.49%

43

4.69%

37

GI

Table 2 Total number of representation strategies in each age group

2.10%

49

2.55%

8

2.84%

16

1.96%

13

1.52%

12

GmI

0.69%

16

0.96%

3

0%

0

0.75%

5

1.03%

8

IG

79.25%

1845

60.19%

189

76.42%

431

79.76%

528

88.45%

697

Om

3.05%

71

1.59%

5

4.61%

26

3.63%

24

2.03%

16

S

2.92%

68

8.28%

26

3.55%

20

2.42%

16

0.76%

6

Sp

100 %

2328

100%

314

100%

564

100%

662

100%

788

Total

Non-verbal Behavior and Its Role in Narrative Production 99

100

P. Eismont

below). In addition, I could argue that these bodily expressions were too complex or atypical for a child [38]. The most frequent strategy of explicit representation of bodily expressions in all age groups was the interpretation of the characters’ movements (GI), although it should be noted that this choice of strategy visibly increased with age. This confirms that at the age of four, children already perceive bodily expressions as signs that correspond to a certain piece of information, and are capable of imaginative thinking and empathy. A complete understanding of the entire spectrum of NVA meanings has not yet been formed, but the process of symbolization and gesture formation is already underway [35]. Another possible explanation for these findings is that the speaker tends to insert into the narrative only those bodily expressions that are clear to her. Imaginary gestures or bodily expressions (IG) were the least represented strategy across all groups, although the appearance of such invented elements is characteristic of preschool children’s narratives [48]. This could be due to the experimental instruction (to retell the story while watching the cartoon), in which the children were not required to memorize the events and they did not have time for any reflection or processing of the plot. The same can explain the insignificant rate of bodily expression misinterpretation (GmI). In case of simultaneous retelling, a child does not have time to forget the action performed in the cartoon, and if she doubts the interpretation of an NVA, she would simply describe it (GD). In the control group (adult participants), one of the rarest strategies was found to be the representation of bodily expression through the description of the situation in which it is appropriate (S). For children’s narratives, this representation strategy was quite frequent. This tendency might be corroborated by studies showing that adults are better than children are at parsing continuous actions and situations into constituent elements [43], while children are more disposed to understand the action through its result, through the general scheme of situation. The second least frequent representation strategies deserve a separate discussion. In the youngest age group, this strategy was to restore the characters’ direct speech that could accompany the bodily expression (Sp). This result confirms the findings by McNeill et al. [49] suggesting that younger children struggle with combining information coming through different communication channels. In my experiment, the children watched the cartoon in silent mode, i.e., they had to invent or recall possible speech accompanying a bodily expression from their own communicative experience and switch from one communication channel (visual) to the other (auditory). The fact that children found it challenging may also cast doubt on the idea that children use gestures when it is difficult for them to find the appropriate language expression as a compensation for the developmental deficiencies in a certain communication channel. Perhaps, if the transition from one modality to another turned out to be a difficult task for preschool children, they would use gestures during pauses in their speech when they were struggling to find a lexeme for the same purpose as adults do, as an aid in generating coherent speech [15]. With age, the strategy of representing bodily expression in the narrative by means of character’s direct speech becomes more frequent. This finding confirms that for adult native speakers, gesture

Non-verbal Behavior and Its Role in Narrative Production

101

and speech are computed together, they are integrated into each other and determine one through the other (cf. interactive contribution hypothesis by [25]). This integration is also supported by the fact that adults (as opposed to children) often generate their narratives by taking on the role of the main character or performing all the characters in the cartoon. With character viewpoint perspective, direct speech is becoming the main way for comprehension and representation of all actions and bodily expressions in the cartoon [29, 50]. Table 3 shows correlations between the choice of the representation strategy and the age of the participants for each NVA. Only some NVAs show high contingency. NVA 3 is performed by the kitten’s grandmother and is her first appearance in the cartoon. Before that, the kitten, her grandchild, scattered things around the room, and the grandmother shakes her head in disapproval. Younger children hardly noticed grandmother’s non-verbal behavior: it was omitted in 84% of the retellings. With age, the number of omissions decreased to 15% in the narratives by adult participants. Children from the two older groups most often represented the grandmother’s actions by describing the situation, and the adult participants either interpreted it or reproduced the grandmother’s words that could accompany her non-verbal actions. NVAs 7 and 8 are situated at the junction of two episodes—the one in which the kitten’s grandmother is involved (at this point, she leaves the room and does not appear in the cartoon anymore), and the other episode in which the kitten is left alone and decides to run away from home. In NVA 7, the grandmother shakes her finger (with the whole hand) and, in the absence of any verbal context, it is Table 3 Chi-Squared test, contingency coefficient and Cramer’s V for each NVA χ2

df

P

Contingency coefficient

Cramer’s V

1

10.227

9

0.332

0.258

0.154

2

16.773

12

0.158

0.323

0.197

3

55.208

12