Explosions in the Mind: Composing Psychedelic Sounds and Visualisations (Palgrave Studies in Sound) 9811640548, 9789811640544

Citation preview

PALGRAVE STUDIES IN SOUND

Explosions in the Mind Composing Psychedelic Sounds and Visualisations

Jonathan Weinel

Palgrave Studies in Sound

Series Editor Mark Grimshaw-Aagaard, Musik, Aalborg University, Aalborg, Denmark

Palgrave Studies in Sound is an interdisciplinary series devoted to the topic of sound with each volume framing and focusing on sound as it is conceptualized in a specific context or field. In its broad reach, Studies in Sound aims to illuminate not only the diversity and complexity of our understanding and experience of sound but also the myriad ways in which sound is conceptualized and utilized in diverse domains. The series is edited by Mark Grimshaw-Aagaard, The Obel Professor of Music at Aalborg University, and is curated by members of the university’s Music and Sound Knowledge Group. Editorial Board Mark Grimshaw-Aagaard (series editor) Martin Knakkergaard Mads Walther-Hansen Editorial Committee Michael Bull Barry Truax Trevor Cox Karen Collins

More information about this series at http://link.springer.com/bookseries/15081

Jonathan Weinel

Explosions in the Mind Composing Psychedelic Sounds and Visualisations

Jonathan Weinel University of Greenwich London, UK

ISSN 2633-5875 ISSN 2633-5883 (electronic) Palgrave Studies in Sound ISBN 978-981-16-4054-4 ISBN 978-981-16-4055-1 (eBook) https://doi.org/10.1007/978-981-16-4055-1 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Palgrave Macmillan imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Acknowledgements

Thanks are due first and foremost to Rajmil Fischman, for his vision of what the future of electronic music and audio-visual composition might hold, and his openness and support for my ideas. I would like to acknowledge Keele University for supporting me with a grant to study my MRes from 2005–2006, and the AHRC for providing the funding that allowed me to study my Ph.D. at Keele from 2007–2010. Thanks are also due to the other friends and colleagues that have helped shape various aspects of my work over the years, especially Stuart Cunningham, Mark Grimshaw-Aagaard, and the teams at University of Greenwich and EVA London (Electronic Visualisation and the Arts). My friends Sol Nte and Lyall Williams deserve a special mention for their various recommendations of music, reading, and video games that I might have missed. Thanks also to those artists, musicians, programmers, and theorists, whose inspirational work I mention in this book. Lastly, thanks to my family, especially Jen for the green dots.

v

Praise for Explosions in the Mind

“Weinel’s book draws on his original and brilliant work as both a musician and visual artist of exceptional linguistic talent, which he uses to create trailblazing narratives worthy of defining the emerging field integrating digital sounds and images—his music and visualizations are wrought from expressive interpretations of psychedelic technicolor states of the inner mind. The reader’s journey is illuminating, exciting and scholarly, making this book a must-must have for musicians and visual artists.” —Tula Giannini, Professor, Pratt Institute, NYC, Museums and Digital Culture: New Perspectives and Research by Tula Giannini and Jonathan Bowen, Springer, 2019, Great Flute Makers of France: The Lot and Godfroy Families, 1650–1900 by Tula Giannini, Tony Bingham, London, 1993 “To the uninitiated, the term ‘psychedelic art’ may simply recall sixties tie-dye, or hippy nostalgia, yet Explosions in the Mind shows a way in which the topic can remain relevant. Through its use as the subject of ‘practice-based research’, Weinel’s output expertly straddles the divide

vii

viii

Praise for Explosions in the Mind

between underground electronic music and institutional electroacoustic music. The book provides detailed insight into his working methods which use both readily available digital tools and more traditional analogue means.” —Ryo Ikeshiro, Assistant Professor, School of Creative Media, City University of Hong Kong, Co-author of “Sound in Japan”, in Sound Art: Sound as a medium of art (ed. Peter Weibel), ZKM/MIT, 2019 “Weinel has been a leading light in the annual EVA London conferences on Electronic Visualisation and the Arts over recent years, especially in the musical aspects of these events. This book represents his thoughts regarding altered states of consciousness in relation to music. The book’s contents are founded on practice-based research, and demonstrate significant insights concerning music and digital culture.” —Jonathan Bowen, Emeritus Professor of Computing, London South Bank University, The Turing Guide (Oxford University Press, 2017), Bowen, J.P., Keene, S., and Ng, K., editors, Electronic Visualisation in Arts and Culture. Springer Series on Cultural Computing, Springer, 2013. ISBN 978-1-4471-5406-8

Contents

1

Introduction

2

Psychedelic Journeys in Sound: Electroacoustic Compositions

27

Melting in the Mind’s Eye: Real-Time Performances

55

3

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

1

81

5

Sensorial Apparatus: Interactive Projects

107

6

Optical Geometry: VJ Performances

131

7

Future Sound Dream: Virtual Reality Experiences

171

8

Conclusion: Design Frameworks

191

References

203

Index

225

ix

About the Author

Jonathan Weinel is a London-based artist/researcher, whose main expertise is in electronic music and computer art. His electronic music, visual music compositions, and virtual reality projects have been performed internationally, and he lectures in video games development at the University of Greenwich.

xi

List of Figures

Fig. 2.1 Fig. 2.2 Fig. 2.3

Fig. 3.1 Fig. 3.2 Fig. 3.3 Fig. 3.4 Fig. 4.1 Fig. 4.2 Fig. 4.3

Graphic score indicating the structure of Entoptic Phenomena Diagram indicating the compositional structure of Nausea Painting representing visual patterns of hallucination, as used for the label artwork of the Entoptic Phenomena in Audio vinyl EP Author’s sketches of Klüver’s (1971) form constants Atomizer Live Patch, main user interface Performance notes indicating the structure of Entoptic Phenomena in Audio User interface for the Bass Drum, Saxophone & Laptop Max/MSP application User interface and example visual noise output of the Atomizer Visual Max/MSP/Jitter application Graphic score indicating the structure of Tiny Jungle Still image from Tiny Jungle representing visual patterns of hallucination

42 47

51 57 59 65 70 85 86 89

xiii

xiv

List of Figures

Fig. 4.4

Fig. Fig. Fig. Fig. Fig. Fig.

4.5 4.6 4.7 4.8 5.1 5.2

Fig. 5.3 Fig. 5.4 Fig. 5.5

Fig. 5.6

Fig. 5.7

Fig. 5.8

Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig.

6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8

Various frames of animation showing visual artefacts and textures produced by direct animation on 8 mm film Spiral spatialisation patterns produced with PANcho A skull figure in Mezcal Animations Waves of triangles and fish in Cenote Zaci Skulls transfigure into candles in Cenote Sagrado Quake Delirium system diagram Quake Delirium in operation, with various graphical distortions representing a hallucinatory perceptual state Psych Dome system diagram Spiral animations based on visual patterns of hallucination and EEG signals in Psych Dome Screenshot from the ASC Sim project. Coloured boxes in a simple game scene provide sound sources located in 3D space for testing purposes. Metres for ‘attention’ and ‘enhancement’ (top-left) indicate the current values of these properties Diagram showing the ‘selective auditory attention’ mechanism. When the player attends an object, all unattended sound sources fade out Diagram showing the ‘enhanced sonic perception’ mechanism. As the enhancement value increases, sounds fade between ‘dull’, ‘medium’, and ‘bright’ versions of the source material Diagram showing the ‘spatial disruption of sound’ mechanism. Each sound source moves in oscillating spatial patterns around the object with which it is associated Various still images from the VJ loops ‘Trancecore’ sketch, digitally re-coloured Still from a practice VJ mix combining multiple loops Diagram showing the VJ setup used in the studio Still from the ‘Yes to Satan’ video Still from the ‘Wipe the Needle’ video Still from the ‘Acid Rain VIP’ video Diagram showing the live setup used for the VJ London performance

94 96 97 99 101 113 115 118 120

124

125

125

126 136 140 143 144 145 146 149 152

List of Figures

Fig. 6.9

Fig. Fig. Fig. Fig. Fig.

6.10 6.11 6.12 6.13. 6.14

Fig. 6.15 Fig. 6.16 Fig. 6.17 Fig. 6.18 Fig. 6.19 Fig. 7.1 Fig. 7.2 Fig. 7.3 Fig. 7.4 Fig. 7.5 Fig. 8.1 Fig. 8.2 Fig. 8.3

Performing as Soundcat at VJ London (New River Studios, 12 July 2018) (Photo credit: Laurie Bender [L’Aubaine]) Still from the ‘One Consciousness’ video Vortex, acrylic on canvas, 39.7 × 49.8 cm Trip at the Brain, airbrush on paper, 29.7 × 29.7 cm 31 Seconds, acrylic on canvas, 25.4 × 30.5 cm Bug Powder Dust, acrylic and collage on canvas, 25.4 × 30.5 cm Seasons in the Abyss, acrylic on canvas, 30.5 × 40.6 cm Holo Point Break, acrylic and collage on canvas, 50.8 × 76.2 cm Enter Soundcat, acrylic and collage on canvas, 30.5 × 40.6 cm Soundcat 2000, acrylic and collage on canvas, 30.5 × 40.6 cm Soundcat S-101, acrylic and collage on canvas, 30.5 × 40.6 cm Various still images showing scenes from Cyberdream GearVR Diagram showing the structure of Cyberdream, which resembles the form of a DJ/VJ mix with crossfades Controller configuration for Cyberdream Oculus Quest Still showing the ‘ZigZagToy’ (right) and ‘StreamerToy’ (left) sound toys in operation Step sequencer patterns used by the ‘ZigZagToy’ Framework for composing psychedelic journeys in sound Framework for designing altered states of consciousness simulations Framework for composing synaesthetic visualisations of sound

xv

153 155 156 157 158 159 161 162 164 165 166 178 183 184 185 186 195 198 200

List of Tables

Table 5.1 Table 8.1

Psych Dome compositional structure Summary of techniques used to represent ASC features in fixed-media electroacoustic compositions

119 195

xvii

List of Media

Media 2.1 Media 2.2 Media 2.3 Media 2.4 Media 2.5 Media 2.6 Media 3.1 Media 3.2

Media 3.3 Media 3.4

Night Breed , electroacoustic music, 6 minutes 23 seconds Surfer Stem, electroacoustic music, 7 minutes 9 seconds Night Dream, electroacoustic music, 8 minutes 0 seconds Entoptic Phenomena, electroacoustic music, 5 minutes 44 seconds Swamp Process, electroacoustic music, 7 minutes 35 seconds Nausea, electroacoustic music, 19 minutes 11 seconds Atomizer Live Patch, Max/MSP software Entoptic Phenomena in Audio (NYC 2010), recording of a real-time electroacoustic performance, 19 minutes 53 seconds Bass Drum, Saxophone & Laptop, Max/MSP software Bass Drum, Saxophone & Laptop (23 February 2010, Session 1), instrumental performance with live electronics, 11 minutes 5 seconds

33 36 38 40 43 45 57

63 68

76

xix

xx

List of Media

Media 3.5

Media 3.6

Media 4.1 Media 4.2 Media 4.3 Media 4.4 Media 4.5 Media 5.1 Media 5.2 Media 5.3

Media 5.4 Media Media Media Media

6.1 6.2 6.3 7.1

Media 7.2

Bass Drum, Saxophone & Laptop: Extract 1, instrumental performance with live electronics, 1 minute 0 seconds Bass Drum, Saxophone & Laptop: Extract 2, instrumental performance with live electronics, 1 minute 0 seconds Atomizer Visual, Max/MSP/Jitter software Tiny Jungle, audio-visual composition, 7 minutes 10 seconds Mezcal Animations, audio-visual composition, 4 minutes 0 seconds Cenote Zaci, audio-visual composition, 4 minutes 5 seconds Cenote Sagrado, audio-visual composition, 5 minutes 37 seconds Quake Delirium demonstration video, 6 minutes 0 seconds Quake Delirium EEG demonstration video, 2 minutes 36 seconds Psych Dome composition example, 2 minutes 4 seconds; and documentation video, 1 minute 36 seconds ASC Sim software and demonstration video, 3 minutes 5 seconds VJ Loops demonstration video, 6 minutes 30 seconds Soundcat DJ/VJ mix, 30 minutes 32 seconds AR Paintings demonstration video, 53 seconds Cyberdream GearVR demonstration video, 4 minutes 57 seconds and software Cyberdream Oculus Quest demonstration video, 4 minutes 40 seconds and software

77

78 84 86 92 97 100 112 116

118 123 135 151 160 174 182

1 Introduction

Imagine a trickling sensation at the back of the head, and a series of yawns, as if waking into a dream. The grain of the wood floorboards begins to move, flowing like water in one moment, fading back into place in the next. The shelves of a bookcase begin to resemble a grinning face. As water flows down the drain in the bathroom, it seems to take on a vocal quality, as if saying something in a recognisable, slightly humorous accent. Listening to music, the sound stage opens up—instruments seem more spatially separated and have a sparkling, high fidelity quality. Upon closing your eyes, you begin to see multicoloured geometric funnels, racing webs of repeating patterns, and animated caricatures that flicker and dance to the music. In one instance, these are like detailed pencil portraits, which morph into doves and take flight, before assembling into a gateway of skulls leading down into an underworld. Moments later cartoon dogs made of red, white, and blue electricity pick their noses and DJ, scratching records rhythmically to the beat. Now waves of intricate luminous patterns emerge from the doors of a cupboard. Like Persian rugs rendered in 3D, they cycle through complex iterative variations as they flow across the room, one after the other. Cryptic messages resembling a sixteen-segment LED display scroll across the carpet, while © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 J. Weinel, Explosions in the Mind, Palgrave Studies in Sound, https://doi.org/10.1007/978-981-16-4055-1_1

1

2

J. Weinel

blocky angular robots and circuit boards gyrate in mid-air. Gradually these impressions fade into soft flickering traces, as the distant sounds of farmyard animals and fresh countryside air punctuate the room in the first rays of the morning light. Episodes of hallucination such as these can be precipitated by psilocybin mushrooms, such as the ‘liberty caps’ (Psilocybe semilanceata), which can be found throughout the British Isles in the misty weeks of September before the frost sets in (Cooper 1977, pp. 16–17; Phillips 2006, p. 251).1 During such episodes, one may experience auditory or visual hallucinations. When visual hallucinations are formed in response to music, they can be understood as a form of synaesthesia, since the auditory sense acts as a stimulus for the visual impressions. It has often seemed to me that these ‘explosions in the mind’, in which the human psyche unfurls intricate webs of neon geometry, rapidly cycling animations, and cavalcades of absurd caricatures, would provide an excellent basis for the design of art and music. Furthermore, as a child of the 1980s raised on Atari STs and Betamax video cassettes, it is my view that the intense glow of computer pixels and crisp digital audio provides the ideal medium in which to sculpt such psychedelic sounds and visualisations. These technologies offer flexibility, whereby in principle, any visual or sonic impression, real or imaginary, can be designed, so long as one is able to translate it into a concrete form. How this can be achieved is the subject of Explosions in the Mind: Composing Psychedelic Sounds and Visualisations. The idea of representing psychedelic hallucinations became the seed for over a decade of creative practice, which I undertook across multiple areas, ranging from electroacoustic music to various areas of audiovisual composition, including video jockey (VJ) performances, interactive projects, and virtual reality (VR) applications. These works, and how they were realised, are the main focus of this book. In these pages, I will discuss compositional strategies for translating auditory and visual hallucinations into sound, and designing synaesthetic visual materials in correspondence with sound. These strategies are by no means trivial— after all, how exactly does one begin to design sounds or audio-visual materials that represent ephemeral subjective states, such as psychedelic experiences of synaesthesia? Visual and aural impressions that one sees

1 Introduction

3

and hears from a first-person perspective can be approximately represented using cameras and microphones, which may be used to capture similar patterns of light and sound to those that stimulate the senses. However, this approach cannot easily be used to represent hallucinations, where seemingly, the visual and aural perceptions one has do not originate in the surrounding physical environment. There is no ‘video capture device’ for the mind’s eye, yet, with an idea of what may typically be seen or heard in a hallucination, one can attempt to design sounds and images that represent these perceptual experiences. How this can be practically approached through electronic music and audio-visual composition will be explored in Explosions in the Mind. Through the course of the book, I will uncover a set of design frameworks that can be used for representing psychedelic states, ultimately revealing how one may compose psychedelic visualisations of sound across a variety of media. In doing so, my aim is to provide a useful resource for creative practitioners and researchers, especially those working in fields such as electronic music composition, sound design, audio-visual composition, video games, and VR development. To begin, in this chapter I will first provide a discussion of altered states of consciousness (ASCs), focusing in particular on the phenomenon of psychedelic hallucinations and experiences of synaesthesia that occur in response to sound. Following this, I will provide an overview of existing works of art and music that seek to either represent or induce psychedelic states in a variety of ways, thereby providing the backdrop and wider context for the compositional practices discussed throughout the book.2 Specific attention will also be directed towards existing psychedelic visualisations, which can be found in various forms of experimental film, VJ performance, music visualisations, video games, and VR applications. I will then proceed to outline the areas of electronic music and audio-visual composition that I have undertaken as practice-led research, which will be discussed in the upcoming chapters.

4

J. Weinel

Psychedelic Experiences Psychedelic experiences, such as those elicited by hallucinogenic mushrooms, can be understood as a specific form of ‘altered state of consciousness’ (ASC). The term ASC was coined in the 1960s and describes a range of perceptual or experiential states such as dreams, hallucinations, meditations, trances, or hypnotic states (Ludwig 1969). It can be difficult to define precisely what qualifies as an ASC, since we may presume that our conscious experience fluctuates constantly throughout the day, and also varies between individuals. Accepting these limitations, the ASC term is nonetheless useful as a general description that allows us to talk about points of significant divergence from a commonly accepted ‘normal-waking consciousness’. The caffeine in your morning cup of coffee probably induces something we might consider a very mild ASC, but generally speaking, when using this term we are talking about something a bit more out of the ordinary, which is unlikely to go as well with croissants and reading the newspaper. These extraordinary states can be induced through various means. For example, synthetic drugs or intoxicating plants found in nature may induce perceptual changes such as hallucinations. However, hallucinations may also occur without the use of drugs, as in cases of sensory deprivation, which can be elicited with sensory isolation tanks, which suspend the body in complete darkness. While reducing the senses in this way seems to elicit hallucinations, overloading them may also provide an alternative route for inducing ASCs. We see this in the various indigenous trance ceremonies found across the globe, where music, dance, and the spraying of liquids induce states of sensory overload in which participants believe they are possessed by spirits (Rouget 1985). Hallucinations can even occur due to extreme forms of hunger or fasting, such as may be experienced by explorers low on supplies in perilous environments in the extremities of the desert or Antarctic. While these are roads seldom travelled by most people, perhaps more familiar to the reader will be experiences of dreaming, which can also be viewed as a form of ASC analogous to a hallucination that occurs during sleep (Hobson 2003). Lucid dreams, in which one is aware that they are dreaming, can have particularly intense, hallucinatory qualities; while on the threshold of sleep it

1 Introduction

5

is also possible to have hypnogogic hallucinations, as in cases of ‘sleep paralysis’, where one may perceive strange terrifying beings or daemons (Hufford 1989). ASCs may be characterised by a range of effects. While these may vary between types, ‘ASC features’ may include disruptions to thought patterns and awareness, or distortions to time perception, in which moments may seem to pass more quickly or slowly than usual. The hippie on a deep, spiritual acid trip knows this at the peak of the experience, where the cosmic enormity of the pulsating universe weighs heavily and time dilates. In such experiences, one can expect visual hallucinations, as intricate geometric patterns unfold on the backs of closed eyelids. Though less common, auditory hallucinations or other sensory distortions to smell or taste may also be experienced. An altered sense of self may be described during some ASCs, so that one feels as though they have been overtaken by mystical forces, as in the case of possession trances in Haitian Vodou ceremonies, where spirits are believed to enter the dancing body of the trancer. Heightened emotions are often desired, and it is these that are sought by the raver, who hands over crumbled bank notes for some MDMA (3,4-methylenedioxymethamphetamine), so they can spend a few fleeting hours chasing waves of euphoria and ecstatic bliss, dancing to pounding techno music with saucer-wide pupils, until the inevitable crash the following morning. During some ASCs, the feeling of body weight may change, and this is something the psychonaut3 may try to engineer by combining their psychedelic trip with other drugs such as nitrous oxide, which have dissociative anaesthetic properties. In sleep states such as lucid dreaming, or hypnogogic hallucinations, it is even possible for one to experience weightlessness, levitation, or flight. Indeed, there is even suggestion that the legend of witches flying on broomsticks may be related to the effects of psychoactive compounds used in European witchcraft (Harner 1973). In general, ASCs can be varied and highly subjective, but there are often commonalities between experiences of a given type, which emerge from the many reports available in the surrounding literature.4 Various systems have been devised that allow us to classify different types of ASCs. For example, Fischer’s (1971) ‘cartography of ecstatic and meditative states’, provides a continuum ranging from ‘ergotropic’

6

J. Weinel

(promoting energy expenditure) to ‘trophotropic’ (promoting energy conservation) states. This model allows us to consider the energetic properties of ASCs, where deviation from normal-waking consciousness corresponds with points of extreme high or low activation. By focusing on energy levels, Fischer’s cartography has some similarities with Russell’s (1980) ‘circumplex model of affect’, a two-dimensional model of emotion often used in contemporary cognitive psychology, in which the x axis describes ‘valance’, ranging from pleasant to unpleasant; and the y axis describes ‘arousal’, ranging from high- to low-energy states. Fischer’s model also informed Rouget’s classic study Music and Trance: A Theory of the Relations Between Music and Possession (1985), in which the author investigated various global cultures where music was being used in shamanic or possession trance rituals. Here, Rouget proposed that music may be involved in the production of low-energy ‘ecstatic’ states (similar to meditation in his usage5 ), which are typically experienced in quietness and solitude; or high-energy ‘trance’ states, which may occur in situations of sensory overload where the various senses are bombarded through music, dance, and the spraying of liquids. Expanding on this topic, Fachner (2011) suggests that music can be used to structure the temporal experience of rituals that produce states such as trance. For example, in Haitian Vodou rituals, percussive drumming may contribute towards states of sensory overload, inducing trance states; while in Amazonian shamanism, sound-making instruments may focus the attention, or invoke spiritual symbols that shape the experience. These debates emphasise music as a signalling force that is used in specific cultural contexts, which according to Becker’s (2004) analysis, may trigger trance states in particularly susceptible individuals, whom she terms ‘deep listeners’. Rouget also recognises a general association between quickening tempos and states of trance , which corresponds with Gabrielsson and Lindstrom’s (2012) later meta-study of music and emotion, in which they found that quick tempos and percussive music were often associated with high arousal states, while slower tempos and sparse percussion were associated with low arousal. All of this points towards the idea that music can be designed in certain ways in order to promote high- or low-energy experiences, and this is an idea that will be discussed later on in Explosions in the Mind.

1 Introduction

7

Also of interest for our discussion in this book is Hobson’s (2003) ‘state-space’ concept of consciousness, which uses a neurologically based ‘activation, input, modulation’ model. According to this model, ‘activation’ describes states of high or low brain activity; ‘input’ relates to different sources of sensory experience, which may originate in the external environment, or arise internally in the brain; and ‘modulation’ describes how events are recorded to memory. According to this model, during a typical 24-hour cycle, a person undergoes various changes to activation, input, and modulation, and moves between different points on the model. For example, in normal-waking consciousness, activation may be high; input may arise predominantly from the ‘external’ surroundings; and events may be recorded to memory. In contrast, during sleep states, activation levels may vary; input is predominantly ‘internal’ while dreaming; and much of what is experienced is not recorded to memory and will be quickly forgotten. Throughout Explosions in the Mind , Hobson’s (2003) distinction between ‘internal’ and ‘external’ inputs will be particularly useful, because these terms allow us to distinguish between the unreality of dreams and hallucinations that arise ‘internally’, and sensory stimuli that originates in a real, ‘external’ physical environment. Some additional clarity will be useful here, since prima facie consciousness would seem to always involve processes of cognition internal to the individual. If we follow an enactive view of consciousness (Maturana and Varela 1998), an organism is systemically arranged with respect to its surrounding environment, and thus the internal must always be negotiated in relation to an external. From this perspective, even sounds heard in dreams or hallucinations might find their basis in an ‘external’ environment, for instance, if they are related to memories of past experiences. What is important to emphasise here, however, is that the ‘internal’ and ‘external’ labels refer only to the current inputs of the sensory information, and these will always be subject to further processing within the perceptual system. When understood in these terms, the ‘internal’ and ‘external’ classifications provide a useful means for distinguishing between the respective channels from which sensory experience emerges at a given point in time. For our purposes in this book, normal-waking consciousness is typically oriented towards ‘external’ sensory inputs that originate from patterns

8

J. Weinel

of light and sound in a person’s immediate physical surroundings, while dreams or hallucinations predominantly emerge from ‘internal’ sensory inputs that arise from memories and the imaginative faculties of the mind. These are not binary distinctions, so while sensory experience may emphasise one or the other at a particular point in time, there will usually be some form of negotiation happening between the two. While ASCs cover a variety of possible perceptual states, of particular relevance for many of the compositions discussed in this book are the psychedelic states produced by hallucinogenic substances such as LSD (Lysergic acid diethylamide), psilocybin mushrooms, or DMT (N,N-dimethyltryptamine). Although psychedelic experiences produce varied effects depending on the mind-set and situation of the individual (or, ‘set and setting’, to use the phrase popularised by Timothy Leary in the 1960s), there is some commonality with regard to the form of these experiences, which may be comparable between individuals. For example, psychologist Heinrich Klüver (1971, p. 66) carried out participant studies to investigate the effects of mescaline, a hallucinogenic compound that occurs naturally in cacti such as peyote (Lophophora williamsii), which can also be produced synthetically. Klüver described ‘form constants’: honeycomb, cobweb, funnel, and spiral forms, which were commonly perceived by his participants. According to his study, in the earlier stages of hallucination, one will see patterns related to the form constants, while later these may give way to figurative hallucinations of places, people, or animals. In later stages, as the effects wear off, one may also see visual impressions related to the form constants. Similar visual patterns were also reported in Strassman’s (2001) studies of DMT, in which his participants often described geometric patterns in the earlier stages of intoxication, before subsequently ‘breaking through’ to encounter hallucinations of strange beings, entities, and environments. During these episodes, auditory hallucinations were relatively less common, but are also sometimes described, and may include various forms of music, oscillating noises, high-frequency tones, or voices. For example, one participant in Strassman’s DMT studies referred to ‘high pitched’, ‘whining and whirring’, ‘chattering’, ‘crinkling and crunching’ sounds (p. 148).6

1 Introduction

9

Psychedelic hallucinations seem to promote experiences of synaesthesia, a phenomenon in which blurring across the sensory modalities occurs; for example, sounds may have a smell, or colours may have a taste (Cytowic 1989). There is a particular phenomenon, which is often reported during psychedelic experiences, of ‘sound-to-image’ hallucinations, in which sound triggers corresponding visual hallucinations (Bliss and Clark 1962, p. 97). This effect may be partly explained by recent neurological research, which suggests that psychedelic drugs such as LSD precipitate heightened states of interconnectivity in the brain, causing stronger associations between regions than would normally be present (Carhart-Harris et al. 2016). In normal-waking consciousness, sound may trigger associative memories, visual images, or emotions.7 We know this from research in sound and multimodality, which confirms that the auditory cortex provides various forms of multimodal integration (Purves et al. 2001). For example, the ‘McGurk effect’ (McGurk and MacDonald 1976), showed that seeing different mouth movements caused participants to hear different phonemes from the same acoustic stimuli; and various other studies have also shown that activity in the visual cortex can stimulate the auditory cortex (e.g. Calvert et al. 1997; Callan et al. 2003). In a psychedelic state, heightened interconnectivity in the brain may provide an increased sensitivity to these multimodal effects, so that associative properties of sound are manifested as visual impressions. The experience of psychedelic visual hallucinations in response to sound is a pivotal theme of this book, around which much of my compositional practice revolves. As we shall see through subsequent chapters, many of my earlier works such as electroacoustic compositions are explicitly based on the idea of representing visual or auditory hallucinations. These works use typical ‘ASC features’ described in the literature to inform the design of the compositions. Later however, the connection with psychedelic states becomes more implicit. For these works, I utilise a process in which the design of visual materials is related to the associative properties of sound, thereby adopting a similar mechanism to that which occurs in sound-to-image hallucinations, without necessarily referencing specific accounts of psychedelic states directly. While the design of these works depends less overtly on the idea of ASCs, they can be considered

10

J. Weinel

as ‘psychedelic visualisations of sound’, because they imitate the mechanism that occurs during psychedelic experiences of synaesthesia, where the manifestation of visual sensations occurs in response to sound.

Psychedelic Art and Music While elsewhere I have provided an extensive, wide-ranging analysis of art and music related to ASCs (Weinel 2018a), before we begin discussing compositional strategies, it will be useful to provide a brief tour through the wider cultural context for psychedelic art and music. ASCs have been present in human culture for thousands of years. For instance, Lewis-Williams and Dowson (1988) suggested that prehistoric shamanic rock art designs represented honeycomb, cobweb, funnel, and spiral patterns similar to the form constants described by Klüver (1971). Although this theory has been contested (Lewis-Williams 2007; Dowson 2007; Luke 2010), there are various other ancient cultures where ASCs seem to have played an important role. For example, Australian Aborigines have used the stimulant pituri for thousands of years as a means to access dreamtime (Schultes et al. 1996); the Ebers Papyrus indicates opium use in ancient Egypt (Merlin 1984, pp. 274–275); the Rig Veda, a sacred Aryan text of the Indus Valley, describes the ecstatic drink ‘soma’ (Wasson 1968; Flattery and Schwartz 1989; McKenna 1992, p. 120); and a hallucinogenic drink seems to have played a central role in the Eleusinian Mysteries, an initiation cult in Ancient Greece (Eliade 1978). For thousands of years, shamanic societies have used intoxicating plants to access visionary states in which a shaman communicates with a ‘spirit world’ (Eliade 1964; Vitebsky 1995). For example, Amanita muscaria mushrooms are used in Siberian shamanism; in North America, Native Americans use the peyote cactus in religious ceremonies; Psilocybe cubensis mushrooms are used by the Huichol and Mazatec peoples of Mexico; while the hallucinogenic brew ayahuasca (which contains an orally active form of DMT) is used in the shamanic practices of the Amazon rainforest. While Europe became oriented towards an alcoholic drinking culture that survives today (Sherratt 1995), there is also

1 Introduction

11

suggestion that intoxicating plants were once used as ‘hexing herbs’ in witchcraft (Harner 1973). In modern Western society, psychedelic hippie culture emerged in the 1960s. First synthesised by Albert Hoffman in 1938, LSD was initially used in psychotherapy trials, but soon found popularity in the 1960s counter-culture, due to its ‘consciousness expanding’ effects. These effects struck a chord with the youth culture of the time, who in the midst of various social, political, and sexual revolutions, were interested in altering perception to find different ways of thinking and being that challenged established societal norms. LSD was subsequently outlawed in the United States, with most other Western countries following suit. Other novel psychoactive compounds that were discovered by Alexander Shulgin in the 1970s, such as the euphoric stimulant MDMA, and the psychedelic drug 2C-B (2,5-dimethoxy-4-bromophenethylamine), were similarly banned. Nonetheless, these substances became available on the black market, and in the late 1980s, MDMA was widely used by revellers at all-night dance parties in rave culture. Though various substances wax and wane in their popularity over the years, underground use of drugs, psychedelic or otherwise, is ongoing. In recent years, however, there has also been a resurgence of interest in the therapeutic use of psychedelics for treating posttraumatic stress disorder (Mithoefer et al. 2010) and other conditions such as alcoholism (Bogenschutz and Johnson 2016). Across these various cultures of intoxication, perhaps not surprisingly, we find a great deal of art and music that responds to ASCs, or may even be used to induce them. Visual artefacts produced by shamanic cultures may represent visionary states, while music may actually be used to conduct the ritual. For example, Lewis-Williams (1996, p. 28) discusses San rock art (of South Africa), which he says depicts circles of figures taking part in trance rituals. Swan (1999) provides extensive documentation of Native American artefacts, many of which display patterns and spiritual symbols related to peyote rituals, in which traditional songs are sung, sometimes with percussive accompaniment (as documented on Harry Smith’s Kiowa Peyote Meeting, 1973). In the shamanic practices of Central America, the Huichol people create distinctive ‘yarn

12

J. Weinel

paintings’, which include colourful symbols related to visionary experiences (Berrin 1978); while Mazatec mushroom rituals (as documented on María Sabina’s Mushroom Ceremony of the Mazatec Indians of Mexico, 1957) may involve singing and clapping in which mushroom spirits are believed to speak through the shaman (Wasson et al. 1974). Among the Tukano people of the Amazon rainforest, zigzag, lattice, star, and spiral designs that are rendered on the walls of buildings are related to ayahuasca visions (Reichel-Dolmatoff 1978), in which the elders chant and hum (as documented on Brian Moser and Donald Tayler’s The Music of Some Indian Tribes of Colombia, 1972). Elsewhere in the Amazon, the Shipibo people make distinctive ronin quene (snake designs), which are based on ayahuasca visions (Gebhart-Sayer 1985, p. 162). Interestingly, these are understood as ‘design medicines’, and are also translated into songs by the shaman, in a manner that perhaps points towards the synaesthetic qualities that are inherent in visionary experiences. In Western culture, ASCs have provided a point of inspiration for various forms of visual art, literature, and film. For example, John Uri Lloyd’s Etidorhpa (1895) describes a hallucinatory journey based on a hollow-earth theory. In the early twentieth century, drawing on Freud’s (1899) theories of psychoanalysis, the surrealists were interested in making films and paintings that invoked a sense of dreams or the unconscious. A notable film from this period that remains a visceral viewing experience to this day is Un Chien Andalou (Buñuel 1929, made in collaboration with Salvador Dalí), which utilised shocking juxtapositions of imagery to produce an irrational experience for the viewer. In the 1940s and 1950s, avant-garde filmmakers such as Maya Deren created films that follow protagonists undergoing dreams, rituals, or possession trances, forming a category that Sitney (1979, p. 21) refers to as the ‘trance film’. Also around this time, Henri Michaux (2002) produced fascinating and highly detailed ink drawings based on his experiences of mescaline. Following this, with the arrival of the 1960s counter-culture, there was an explosion of psychedelic artwork, in which the effects of LSD seem to have influenced various areas of visual culture towards brightly coloured, mind-warping designs. Perhaps the best evidence of this can be seen in the melting neon letters and geometric patterns of poster design

1 Introduction

13

for psychedelic rock concerts in San Francisco and elsewhere (Grunenberg 2005). In the 1960s there were also numerous literary works that described ASCs, such as Paul Bowles’s A Hundred Camels in the Courtyard (1962), which uses a literary mosaic technique to reflect the effects of smoking hashish; or Carlos Castaneda’s The Teachings of Don Juan: A Yaqui Way of Knowledge (1968), in which the author describes visionary journeys and experiences of metamorphosis. This decade saw a great deal of experimentation in filmmaking, as seen in both mainstream ‘hippie exploitation’ films such as The Trip (Corman 1967) or PsychOut (Rush 1968), as well as underground productions by groups such as USCO, which sought to overload the senses with tape effects and spinning imagery, as seen on films such as Yalkut’s Turn, Turn, Turn (1968). As documented by Rubin (2010), psychedelic stylisations echo through various subsequent forms of visual art to the present day, and disperse into many strands of artistic work, from the hallucinogenic graffiti of Kenny Scharf to the visionary symbolism of Alex Grey’s paintings. Music was at the centre of psychedelic culture in the 1960s and 1970s. Bands of this era adapted the form of rock n’ roll by incorporating psychedelic themes and sounds, through lyrics, tape effects, and guitar pedals (such as flangers, wah-wahs, and fuzz boxes). In the studio, even more experimentation was possible through production techniques using tape loops. These techniques can be heard on recordings by ‘garage rock’ bands (such as those featured on the Nuggets: Original Artyfacts from the First Psychedelic Era, 1965–1968 compilation, Various Artists 1972); albums by artists such as The Jimi Hendrix Experience, The Beatles, Pink Floyd, Cream, The Rolling Stones, The Grateful Dead; and others. Johnson and Stax (2006) discuss these approaches, also noting the introduction of exotic sonic material derived from Eastern influences, which may support the ‘otherworldly’ quality of the music for Western audiences. Psychedelic forms of rock music can be traced through the decades that followed to the present day, via related genres such as space rock or stoner rock. There are also various other popular music genres that similarly incorporate hallucinatory stylisations, often through means of tape effects or electronic processing. For example, Michael Veal (2007) describes dub reggae as a form of ‘psychedelic Caribbean music’, and

14

J. Weinel

there are clear parallels to be found in this genre via the experimental use of tape, echo, and reverb. The approaches of psychedelic rock and dub were among those that were influential on the electronic dance music culture of the 1980s and 1990s (Collin 1998; Reynolds 2008), and a myriad of other associated genres such as drum & bass, trip-hop, ambient techno, and psy-trance. In various ways, these electronic dance music genres wrap forms of psychedelic sound design and sampling around energetic rhythms and beats, providing rave music that is tinged with acidic and hallucinogenic qualities. Electronic dance music producers do not always take drugs, and neither do the audiences of this music, but there is a proximity to drug use in these genres that means psychedelic themes are often close at hand. As St. John (2009) discusses, electronic dance music may well be used in combination with drugs, where these sounds are likely to be complementary, but above all, it is the music that appeals to audiences, and these sounds may even have the potential to elicit collective trance-like experiences of dance. Of special significance for this book is also those works of electroacoustic music,8 which connect with ideas of dreams, hallucinations, or unreality. Electroacoustic music is not usually seen as part of psychedelic culture, however, there are examples that relate to various concepts of ASCs, by using electronic manipulations of sound to elicit dreamlike aural experiences. A notable work that achieves this with striking success is Michael McNabb’s Dreamsong (1978), which transitions between recorded sounds that suggest a real-world location, and synthesiser sounds that indicate a dream world. Barry Truax has also created several significant works that traverse similar perceptual boundaries, such as Pendlerdrøm (commuter dream) (1997), which describes a travel experience in which a commuter lapses into a dream. In this case, the work was realised through various field recordings and computer-manipulated sounds, which allow Truax to move the listener between representations of normal-waking consciousness and dreaming, and thereby transitioning between ‘external’ and ‘internal’ points of Hobson’s (2003) ‘input axis’. Along similar lines, Truax’s piece The Shaman Ascending (2004–2005), takes the concept of an Inuit shamanic ritual, and uses this to inform the organisation of sonic materials within the piece, in

1 Introduction

15

which droning vocal sounds rapidly circle around the listener within the spatial field. Several other composers have also explored approaches such as these. For example, Gary Kendall’s Ikaro (2009–2010) is one of several compositions based on Peruvian shamanism, which incorporate soundscape materials to construct sonic experiences that are analogous to shamanic journeys. Mining similar territories, Adrian Moore’s Dreamarena (1996) and Åke Parmerud’s Dreaming in Darkness (2005) both use concepts of dreaming as points of creative departure. There is then a significant strand of electroacoustic work that explores notions of ‘reality’ and ‘unreality’ as a basis for musical composition. This area is important to highlight, because the compositional approaches that I initially explore in Explosions in the Mind emerge from the field of electroacoustic composition, and so these works have special contextual relevance.

Psychedelic Visualisations Of special importance for Explosions in the Mind are also those existing psychedelic visualisations, which can be found in various contexts ranging from experimental films and VJ performances to video games and VR experiences. In this section, I will outline some of the main examples from these areas that are particularly relevant for Explosions in the Mind , while acknowledging this is by no means an exhaustive account of audio-visual practices, which reflect an increasingly broad spectrum. ‘Visual music’ is an area of visual art and experimental film in which works are designed based on the form and structure of music (Brougher and Mattis 2005). Early examples of visual music include ‘colour organs’, which display lights in correspondence with sound (Moritz 1997), and the paintings of Wassily Kandinsky, which interpret music through abstract symbols and shapes. However, the term is now more strongly associated with the films of artists such as Len Lye, Normal McLaren, Oskar Fischinger, Harry Smith, John Whitney, James Whitney, and Jordan Belson. In the mid-twentieth century, these artists created striking short films in which shapes, patterns, and textures seem to move and dance around the screen in a way that reflects the rhythms, timbres,

16

J. Weinel

and melodies of music. As I discuss elsewhere (Weinel 2018b, pp. 127– 129), some of these works are explicitly psychedelic, reflecting themes of meditation and hallucination. For example, Jordan Belson’s films draw inspiration from meditation, providing cosmic journeys that contemplate the inner reaches of the human psyche and the outer limits of the galaxy. In his film LSD (1962), Belson uses various funnel patterns suggestive of Klüver’s (1971) form constants. Belson was also involved in a series known as the Vortex Concerts, which used multiple speakers and projections in a planetarium to provide immersive spatial performances (Molloghan 2015, pp. 72–74) that prefigure the audio-visual fulldome9 performances of today. An associate of the beatniks, Harry Smith’s work is also a particularly important reference point for Explosions in the Mind (see also Chapter 4), since his visual music films were partly inspired by experiences of synaesthesic sound-to-image hallucinations precipitated by psychedelic drugs and jazz music. Arguably ahead of his time, Smith even projected his films to live jazz performances and could be considered as a forerunner of today’s VJs. While visual music tends to be associated with fixed-media10 work, the psychedelic light shows of the 1960s and 1970s provided more live and performative approaches for visualising music. In New York the Joshua Light Show provided colourful projections to accompany rock concerts at venues such as the Fillmore East (Signore 2007; Zinman 2008); in London Mark Boyles and Joan Hills provided light shows for the UFO club (Robinson 2007); while the Manchester area was catered for by Nova Express (see Chapter 6, pp. 147–186). These light shows utilised technologies such as overhead projectors with glass clock faces filled with coloured oils, inks, and other chemicals; slide projectors; and 16 mm projectors. The groups manipulated these devices live, generating organic psychedelic textures and visual rhythms to accompany live performances by psychedelic rock bands of the day such as Pink Floyd. In the 1970s and 1980s, music visualisers began to appear, which could automatically drive light synthesis patterns in response to audio. For example, the Atari Video Music (1976) was a piece of home electronics that could be connected to a hi-fi and TV in order to generate analogue diamond patterns in response to an audio signal. Designed for home computers, Jeff Minter’s light synths such as Psychedelia (1984),

1 Introduction

17

Colourspace (1986), Trip-A-Tron (1987), and the Virtual Light Machine (VLM 1990–2003) provided more sophisticated forms of light synthesis that used FFT (Fast Fourier Transform) analysis (Bell 2019). Iterations of the VLM were also featured in the Atari Jaguar and X-Box 360 video games consoles, allowing users to insert any CD from their collection and generate psychedelic visualisations of sound. In the early 2000s, music visualisers like these became a common feature in home computers on Mac and PC, via plugins such as Milkdrop, which were used with media players such as Winamp and iTunes. The late 1980s and 1990s saw the emergence of electronic dance music culture, in which VJs provided a new kind of psychedelic lightshow that reflected the technological aesthetics of the music. Analogous to the role of the DJ, who mixes records to construct continuous aural journeys for all-night dance parties, the VJ mixes live visuals to accompany DJ performances (Faulkner 2006). In the 1990s, these visuals reflected the psychedelic imagery of electronic dance music culture, as seen elsewhere on record sleeves and rave flyers (Savage 1996). Videos of VJ mixes from this period include Dance in Cyberspace (Dr. Devious and the Wiseman 1992); Global Chaos (Hex 1993); Future Shock (Frost et al. 1993); and the X-Mix series (Studio !K7 1993–1998). Much as electronic dance music was facilitated in part by the democratisation of music production via low-cost home studio equipment, sequencers, samplers, and synthesisers, VJ culture benefited from affordable video editing technologies like the NewTek Video Toaster and ray-tracing packages that allowed 3D animations to be created on home computers. As I will discuss in Chapter 6, the technologies and visual aesthetics of VJ culture are in close proximity with the ‘demo effects’ (short realtime computer graphics demonstrations) produced in the ‘demoscene’ computer art subculture (Polgár 2005), and in some cases music visualisers were also used by VJs.11 Modern VJ culture is mostly computer-based. The VJs of today will typically use a laptop running software such as Resolume, VDMX, or Modul8, which allows live mixing of digital videos stored on hard disk drives. VJs can make their own video loops, rip them from any number of online sources, or purchase packs of pre-designed visuals online.

18

J. Weinel

Alternatively, they can use generative animations written in programming languages for visuals, such as Processing, Quartz Composer, or HLSL (High-Level Shader Language). Video can be mixed in realtime using MIDI controllers, but may also utilise audio amplitude or frequency analysis to generate oscilloscope patterns or automate certain visual effects. The typical function of the VJ remains the provision of live visuals to accompany DJs in nightclubs, however, these practices now extend into a wider field of audio-visual performance and media arts, and the boundaries between these areas are not always clearly defined. In London, this wider sphere of activity is represented by Splice Festival, which brings together various strands of audio-visual performance, visual music, VJ performance, live coding, and performance art (Weinel 2018b). Contemporary VJ work may also engage with videomapping technologies, using software such as Mad Mapper, which allows visuals to be projected across complex, irregularly shaped stage sets at music festivals. Video mapping can also be used to create VJ performances in fulldomes,12 as was showcased at the Fulldome UK 2016 festival in Leicester, which included several psychedelic works (Weinel 2018a, pp. 131–132). Also of relevance to Explosions in the Mind are representations of ASCs that occur in video games. A growing number of games include sequences in which the player character may undergo states of intoxication, psychosis, or other forms of non-ordinary sensory experience. As seen in titles such as Silent Hill (Konami 1999) and F.E.A.R. (Monolith 2005), it has become fairly common for psychological horror games to include representations of auditory hallucinations, as a means to support the narrative of the game, character development, and instil a sense of unease in the player (Demarque and Lima 2013). A more recent game that includes representations of auditory hallucinations is Hellblade: Seanu’s Sacrifice (Ninja Theory 2017), which used binaural recordings to give the impression of the protagonist hearing voices. This game received funding support from the Wellcome Trust, and these designs were informed by advice from mental health professionals. Other games incorporate sequences of intoxicated drug use, which are usually portrayed using various post-processing filters, DSP effects, and sonic atmospheres. For example, the neo-noir third-person shooter game Max Payne

1 Introduction

19

(Remedy Entertainment 2001) featured scenes in which the protagonist is intoxicated, wandering through the morbid corridors of his mind; while in Far Cry 3 (Ubisoft Montreal 2012), the player character experiences various sensory distortions after consuming magic mushrooms. Other intoxicated game sequences are more whimsical or comical. Grand Theft Auto V (Rockstar North 2013) features various humorous and ‘edgy’ representations of recreational drug use; while the ‘Lightbearer’ downloadable content for We Happy Few (Compulsion Games 2018) delivers a narrative filled with sex, drugs, and rock n’ roll escapades through colourful, surrealistic tongue-in-cheek portrayals of psychedelic pill-popping. Experiences like these are now being brought into VR, as seen in more meditative titles such as Jan Kounen’s Ayahuasca: Kosmik Journey (2019; see also Haridy 2019), which takes the viewer on a shamanic ayahuasca trip; or Soundself: A Technodelic (Andromeda Entertainment 2020), a voice-activated VR experience in which the user must create ‘om’ sounds that are transformed with DSP effects to embark on a voyage through waves of kaleidoscopic visuals. Other music-oriented video games may also include visualisations of sound, which to varying degrees may be considered ‘psychedelic’. The musical ‘on-rails shooter’13 Rez (United Game Artists 2001) is an important game in this area, in which events are quantised and occur in synchronisation with the electronic dance music soundtrack. The game notes inspiration from the visual music painter Kandinsky, and the updated VR version Rez Infinite (Monstars 2017) uses colourful particle effects which develop in tandem with the music, enhancing the impression of synaesthesia. Other shoot ‘em-up games that use various forms of psychedelic visual effects, audio reactivity, and rhythmic synchronisation include Polynomial (Lavrov 2010), Beat Hazard (Cold Beam Games 2010), and Lost Future Omega (Mebius 2018). The ‘rhythm game’ is an expansive genre, in which players are challenged to match the beat of the music, and these may include audio-reactive backdrops or environments that relate to the music. The most popular current example of this in VR is Beat Saber (Beat Games 2019). In Beat Saber, the player stands in a minimalist tunnel environment swiping at coloured blocks with lightsabers in time to the music. Other variations on the rhythm game formula situate the player as the pilot of a vehicle that

20

J. Weinel

must be precisely controlled in relation to musical timing, as seen in both Audio Surf (Fitterer 2008) and Thumper (Drool 2016), which has a VR version. There are also a number of music visualisations that do not have a ludic dimension, simply providing immersive audio-visual experiences of music that can be enjoyed in VR. For instance, MelodyVR (MelodyVR 2019) uses video footage to allow the user to watch concerts in VR, while Fantasynth (HelloEnjoy 2017) allows the user to fly through an illuminated environment that pulses in time to the soundtrack. Of course, there will always be many more works that one could mention in this section, but for our purposes, this overview is sufficient and outlines the wider context for the projects discussed in Explosions in the Mind.

The Chapters Through the course of this book, I will discuss the compositional methodologies used to realise various creative projects that represent ASCs and provide psychedelic sounds and visualisations. This work has been undertaken primarily as ‘practice-led research’ in an academic context. The works discussed in earlier chapters of the book were completed as part of my Ph.D. in music at Keele University, while many of the other projects covered later on were undertaken as postdoctoral research elsewhere. For readers unfamiliar with this term, ‘practice-led research’ is based on the premise that the production of the creative works themselves constitutes a contribution to knowledge, leading to innovations that could not otherwise be obtained through alternative means (Smith and Dean 2009). Practice feeds into the generation of theory, which in turn, informs practice.14 This methodology is often appropriate for academic research in areas such as sound design and music composition, because it allows researchers to gain new insights by developing new tools or compositional strategies that expand the repertoire. Practice-led research is also sometimes used in industry, and may be combined with interdisciplinary approaches, for example, by using empirical methods to test and evaluate outcomes (Weinel and Cunningham 2021). Practice-led research is sometimes described as ‘research through design’, emphasising the potential for innovation through the process of designing and making

1 Introduction

21

new things in relation to specific objectives. For our purposes here, those ‘new things’ are electronic music and audio-visual compositions, which explore possible strategies for representing ASCs and psychedelic visualisations of sound. These projects approach this area from various angles, and the continuity between works represents a journey through this subject across different media technologies. Chapters group the compositions thematically, and with a few exceptions made to accommodate the logical grouping of works, the discussion is also chronological. Chapter 2 begins by discussing fixed-media compositions of electroacoustic music composed between 2007 and 2011, which seek to represent ASCs. The initial works discussed in this chapter were composed by taking typical features of ASCs, such as visual patterns of hallucination, or distortions to time perception, and translating them into sound. Extending this idea, later works use the typical form of hallucinations to inform the structural organisation of materials, so that the composition as a whole becomes analogous to what one might see or hear in a psychedelic hallucination. The elaboration of this approach is explored through Nausea (2011), a long-form multichannel composition, which exhibits several distinct musical movements. Chapter 3 takes the discussion into the realm of real-time performances of electronic music. Several of my electroacoustic compositions were realised with a specially designed software tool: the Atomizer Live Patch, which facilitates the creation of sonic materials based on hallucinations, and can also be used for live performances. In this chapter, both the design of this tool and its use for a live performance in New York City are discussed. Following this, I also examine another piece of software: Bass Drum, Saxophone and Laptop, which provides a real-time performance system for live instrumentation and electronics, in which DSP is automated in order to suggest the shifting perceptual changes that one may experience during hallucinations. Chapter 4 moves into the area of audio-visual composition. First, I discuss Tiny Jungle (2010), a fixed-media piece, in which various materials were designed based on the concept of visual patterns of hallucination. This piece was also created with a bespoke software tool, the Atomizer Visual. Following this, I discuss a trio of fixed-media visual music compositions: Mezcal Animations (2013), Cenote Zaci (2014), and

22

J. Weinel

Cenote Sagrado (2014), which drew inspiration from a trip to Mexico, and are based on different concepts of hallucination and synaesthesia. These works incorporate hand-painted materials using direct animation techniques and explore the digital compositing of these materials with stop-motion animation and computer graphics. Chapter 5 looks at ways in which to ‘simulate’ psychedelic hallucinations through interactive projects. An early experiment in this area, Quake Delirium (2010), is a computer game modification, which automates game parameters, graphics, and sounds in order to represent fluctuations in perception, such as one might experience during an episode of hallucination. Following this project, Psych Dome (2013) is an interactive artwork designed for presentation in a mobile fulldome, which uses original software patches to generate visual patterns and corresponding sounds based on Klüver’s (1971) form constants. Through the use of a biofeedback electroencephalograph (EEG) headset, the piece provides a connection between the brain activity of the viewer, which is used to modulate the sounds and visualisation. Lastly, ASC Sim (2017) is a game-engine project that provides three prototype mechanisms to simulate auditory hallucinations. Chapter 6 focuses on a discussion of my VJ performances from 2018 onwards under the alias Soundcat. Building on techniques described in the earlier chapters, these performances were realised through the construction of various hand-produced and computer-generated materials, which were then mixed together in real-time, resulting in a series of original videos for existing music. In live performances, these videos are then combined to provide a DJ/VJ mix. In this chapter, I will also discuss various paintings that were completed alongside this work, which interpret sound and music through abstract and symbolic forms. These works can be understood as ‘sketches’ or companion pieces, which develop related synaesthetic visual ideas. Three of these pieces form a series in which augmented reality is used to bring still elements to life as VJ loops. Chapter 7 discusses Cyberdream (2019–2020), a VR project, which has been developed through distinct iterations for the Oculus Gear VR and Oculus Quest devices. This project extends many of the principles described in the earlier chapters, in order to provide a psychedelic visualisation of electronic music. The structure of the piece assumes the form

1 Introduction

23

of a DJ/VJ mix, where various scenes become analogous to music tracks, which the user moves through interactively. Cyberdream also provides a series of audio-visual ‘sound toys’, which allow the user to interact and creatively ‘paint with sound’, generating psychedelic visualisations of sound. Chapter 8 considers the various projects that have been discussed throughout the book, summarising the conceptual approaches, theoretical insights, and practical strategies used. Consolidating these outcomes, Explosions in the Mind will present a set of three design frameworks for composing works that represent psychedelic hallucinations and ultimately provide psychedelic visualisations of sound. These frameworks will allow the reader to consider new opportunities and approaches that might be used in their own electronic music, audio-visual compositions, or interactive projects, which will undoubtedly push this field of research and creative practice yet further into exciting, uncharted territories. For each chapter of Explosions in the Mind , supplementary materials have been provided online (http://www.jonweinel.com/exp_media/dow nloads.htm), which will allow the reader to listen to, watch, or experiment with software tools related to many of the electronic music and audio-visual compositions discussed. It is recommended that the reader refers to these in order to gain a more complete understanding of the work. For instances where the composition cannot be provided (as in the case of live performances), demonstration recordings and videos have been provided instead. ∗ ∗ ∗ From shamanic rituals to modern electronic dance music festivals and psychedelic VR experiences, human culture is interwoven with ASCs. Within this broad area, Explosions in the Mind focuses in particular on the representation of psychedelic hallucinations, which I have used as a basis for the design of electronic music and audio-visual composition for more than a decade. This book provides a comprehensive documentation of these works, ultimately leading to new strategies for composing psychedelic visualisations of sound. The journey presented herein traverses multiple creative approaches in practice-led research,

24

J. Weinel

and these are entwined with advances in immersive technologies. By providing a detailed account of my own compositional practices in this area, it is my hope that others will be able to draw upon and extend this work, allowing us to make the most of the opportunities that new audiovisual technologies provide for immersing audiences into synaesthetic virtual worlds of sound and image.

Notes 1. Possession of ‘magic mushrooms’ and many of the other drugs mentioned in this chapter is illegal in most countries. Many of the ASCs discussed in this chapter are also potentially dangerous, and are not recommended to the reader. In contrast, psychedelic visualisations of sound are generally safe, although some may use stroboscopic visual elements that should be avoided by anyone with photosensitive epilepsy. 2. Related topics are also explored in my book Inner Sound: Altered States of Consciousness in Electronic Music and Audio-Visual Media (Weinel 2018a). While the focus of Explosions in the Mind is on my own practice-led research in electronic music and audio-visual composition, Inner Sound provides a wide-ranging analysis of existing works related to ASCs. For an expanded discussion of the wider area, readers may also wish to refer to Inner Sound , which is complementary, and could be read either before or after Explosions in the Mind . 3. ‘Psychonaut’ is a colloquial term for a person who takes a particular interest in exploring the nethereaches of the human psyche through means of psychedelic drugs and other ASCs. 4. Accounts of ASC experiences are available in various books, scientific studies, and websites. For example, Hayes (2000) provides a collection of experience reports; Strassman (2001) documents participant studies with DMT; while the website erowid.org has a vast database of self-reports covering the effects of almost every known intoxicating plant or substance. 5. Rouget’s (1985) use of the term ‘ecstasy’ describes ASCs that occur in shamanic rituals characterised by quiet stillness, and should not be confused with those states produced by the euphoric stimulant MDMA, commonly known as ‘ecstasy’. 6. For a further discussion of auditory hallucinations, see also Weinel et al. (2014).

1 Introduction

25

7. Schafer (1994, pp. 148–150) discusses the associative properties of sound, which may give rise to psychoacoustic responses, semantics, and aesthetics. As Schafer acknowledges, these responses are not fixed, but may vary between individuals. 8. ‘Electroacoustic music’ is a form of Western art music originating in the mid-twentieth century via work such as Pierre Schaeffer’s ‘musique concrète’, and Karlheinz Stockhausen’s ‘elektronische musik’ (Manning 2004). See also Chapter 2. 9. ‘Fulldome’ is a type of immersive 360-degrees projection environment in which a video is projected across a dome-shaped ceiling. 10. Throughout this book ‘fixed-media’ refers to musical and audio-visual compositions created in the studio, resulting in audio or video recordings that may be played back at concerts or festivals. 11. For example, Jeff Minter’s VLM were also used to provide the visuals for dance bands such as The Shamen, Primal Scream, and The Orb (Minter 2005). 12. United VJs are a group who have been particularly active in this area, having created their own Blendy Dome VJ software. They have provided various international workshops on fulldome VJ performance (which the author has been fortunate to attend), and were behind the video mapping at various high-profile events such as the Rio 2016 Summer Olympics. 13. An ‘on-rails shooter’ is a type of video game in which the player must shoot at targets while moving along a predetermined animation path. 14. The integration of theory and practice in the arts is sometimes referred to as ‘praxis’. For a further discussion see also Liggett (2020).

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

Put on the headphones and lower the needle on to the record, which crackles faintly in anticipation as it makes its way into the side-A groove. It begins with a flickering high-frequency tone. Waves of noise gradually fade into the mix with sweeping filters, above a layer of throbbing percussive pulses, which gradually rise in amplitude. Now a brief phase of sonic reconfiguration, with crunchy 8-bit noise, echoing electronics, and a deep pounding bass that cuts through the mix, as fragmented hissing sounds splash over the top like purifying electronic waves breaking onto a shoreline of e-waste. The rhythmic pulsing shifts into a new gear, as the intensity ramps up and icy sounds shatter over the top. If you close your eyes, you can almost see them exploding in your mind’s eye, going off like electric-blue fireworks rendered in sharp fragments of crisp digital audio. Now an oscillator cycles up and down in frequency, as layer upon Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/978-981-16-4055-1_2. Where the icon is shown in the chapter, the reader should refer to the supporting media files.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 J. Weinel, Explosions in the Mind, Palgrave Studies in Sound, https://doi.org/10.1007/978-981-16-4055-1_2

27

28

J. Weinel

layer of tight rhythmic pulses are pressed on top of each other with the cold precision of technological automation. Sonic lasers fan out across the mix, as you are drawn deeper into a dense black sonic mass. Just as this audio system seems on the verge of collapse, the motors power down and the acoustic energy dissipates into the surrounding atmosphere. Sometimes episodes of listening can have a particular potency that stays with you. I can distinctly remember listening to ‘Il Etait… “Magnetique” / Une Possibilite’, by La Peste (Laurent Mialon) (2005) on headphones one afternoon in my university halls of residence. With vivid detail, I can recall the room I was in, the feeling of awe that came from the composition, and having almost visual or synaesthetic sensations that emerged from the sonic experience. Mialon described his music in terms of ‘sonic atoms’, where each individual unit of sound had capabilities for triggering neural impulses in the mind of the listener, giving rise to micro-responses at any given point in time (Weinel 2007). Individual pulses, encoded as analogue audio signals on the groove of the record trigger tiny fluctuations in the magnetic field of the needle. Translated into a varying electrical voltage, these pulses flow through the amplification system, driving moving coil headphone transducers that turn the analogue signal into acoustic sound waves. These sound waves propagate the electronic pulsing into the ears, activating vibrations on the eardrums, setting the hammer, anvil and stirrup in motion, causing the basilar membrane to quiver inside the cochlea. Different areas of this membrane trigger nerve impulses that stimulate the auditory cortex of the brain. From here, the brain makes sense of the stimuli, as differences in the incoming signals in the left and right ears give rise to spatial impressions of sound. As the brain interprets ambiguous shard-like sonic materials, it draws upon past recollections of familiar sounds and associated events, latching on to memories of shattering glass or ceramics. This leads to visual associations and emotions, as one imagines what the sonic event might look or feel like. In an instant, the dark plastic of the record transmits vivid impressions of sound, emotions, and associative images that can almost be seen, lending the music a synaesthetic, psychedelic potency. Electronic music has unique capabilities for realising almost any sound imaginable. Composers working with synthesis, sampling, and digital

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

29

signal processing (DSP) techniques can sculpt sound in a myriad of ways. By using these techniques, the pallet of possible sounds expands beyond those that can be achieved with acoustic instruments, to encompass all manner of noise-based sounds.1 These ideas are utilised in electroacoustic music, a field of music composition originating in the mid-twentieth century, as pioneered through Pierre Schaeffer’s ‘musique concrète’, and Karlheinz Stockhausen’s ‘elektronische musik’, in which manipulations of tape and electronic synthesis techniques were used to create new forms of music composition through organisations of sound (Manning 2004). The field of electroacoustic music subsequently developed through work at various other international studios, and new approaches made possible by computers, leading to its modern form, which is strongly connected with academic music departments and explored through many international concerts, festivals, and conferences dedicated to this art form. Such events may typically include compositions for live instruments and electronics; real-time performances created with laptops and various controller devices; fixed-media compositions, which are ‘diffused’ in loud speaker concerts through live mixing of the audio signal across large multi-channel arrays of loudspeakers that surround the audience; and audio-visual compositions, which combine sound with video elements. In the sphere of electroacoustic music, Smalley’s (1986) concept of ‘spectromorphology’ describes how the spectrum of sound is manifested in time. By manipulating spectromorphological properties, composers can construct sounds that resemble acoustic phenomena that we might encounter in the ‘external’ world around us (in terms of Hobson 2003), but we can also try to make other sounds that have indefinite reference points, or perhaps suggest the ‘internal’ worlds of dreams, hallucinations, and the imagination. In consideration of the associative properties of sound, Smalley’s (1997) concept of ‘source bonding’ describes the ‘natural tendency to relate sounds to supposed sources and causes, and to relate sounds to each other because they have shared or associated origins’ (p. 110). This phenomenon may also be understood through the lens of Schacter and Tulving’s (1994) concepts of memory, where ‘episodic memory’ describes specific autobiographical recollections that integrate multiple types of sensory information and emotion; while ‘semantic memory’ provides a general store of information and factual knowledge.

30

J. Weinel

When we hear a sound, this may trigger episodic memories if they cause us to think of a specific past event. For example, if we heard a recording of shattering glass, this might allow us to remember a specific time when we accidentally dropped a glass. However, more usually if we heard such a sound, we would probably access our semantic memory, which allows us to comprehend its associations in a more general sense. Hence, hearing shattered glass might not make us think of any specific episode, but rather we would grasp the significance of the sound in a more general, semantic way from past patterns of experience.2 These recollections are often ‘multimodal’, in that they may include not only sound, but also other modes of sensory information such as visual impressions, or emotions. By manipulating the spectromorphological characteristics of sound, composers can play with our experience of source bonding, and this in turn opens up many possibilities for electroacoustic music to trigger multimodal associations and visual images in ‘the mind’s eye’ (Taruffi and Küssner 2019; Trickett 2020). These associations might be used to conjure impressions of real-world environments; for instance, one could compose a soundscape3 with the sounds of buzzing insects, tall rustling grass and distant birdcalls, and this might paint a vivid mental image of a meadow in the height of summer. However, the illusory properties of sound may also allow the representation of imaginary or unreal places, as in compositions such as Barry Truax’s Chalice Well (2009), which takes the listener on a sonic journey from a well in Glastonbury into the spaces of a mythical underworld.4 Our journey in Explosions in the Mind begins within the field of electroacoustic music, understood from this perspective as an illusory medium that is capable of conjuring associative multimodal journeys that represent real or unreal places and spaces. In 2007, I began composing works of electroacoustic music that explored ways in which to represent altered states of consciousness (ASCs) through sound. In this chapter, I will discuss the design of these works and the compositional methodologies that were used to realise them. At first, the approach that I used was one of ‘adaptation’, in which the compositional form was modified by designing sonic materials based on typical features of ASCs. This was initially explored through three compositions: Night Breed (2008), Surfer Stem (2008), and Night Dream (2008). Later, the development of this led

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

31

to a ‘structural’ approach, in which sounds are based on hallucinations, and these are structurally organised based on concepts of how such experiences might progress over time. This technique was used to compose two pieces: Entoptic Phenomena (2009) and Swamp Process (2009), which take the listener on a sonic journey analogous to an experience of hallucination as it unfolds in time. The elaboration of this approach then led to the composition of a longer work: Nausea (2011), which develops this idea through several distinct movements based on ‘waves’ of hallucination. The various compositions discussed in this chapter were performed at a variety of events on the electroacoustic circuit of international festivals and conferences, and four were subsequently released on the Entoptic Phenomena in Audio (Weinel 2014) vinyl EP. Through a discussion of the compositional methodologies used to compose these works, this chapter aims to outline an approach for representing visual or auditory hallucinations through sound, which provides a foundation for the audio-visual and interactive works discussed later in the book.

An Adaptive Approach The first compositions that I created utilise an ‘adaptive’ approach, whereby the form is modified to incorporate features that are related to ASCs. As with many of the works discussed in this book, this approach emerged through the process of composition as a ‘bottom-up’ process. That is, the approach was not entirely pre-conceived at the beginning, but arose through iterative processes of composition and reflection.5 These works can be understood as ‘adaptive’, because they utilise existing approaches for composing electroacoustic music and electronic dance music, but modify the form of the composition to incorporate materials that relate to aspects of ASC experiences. This process of ‘adaptation’ can be explained by considering how one might render a watercolour painting that represents a hallway as one might see it during an episode of hallucination. To paint a hallway as it may appear during a state of normal-waking consciousness, we could use watercolour paints to render the scene, indicating the contours of the architecture, the colours and textures of the walls, floor, and ceiling. We could attempt to render

32

J. Weinel

the scene realistically, by making accurate use of perspective and appropriate colours. Suppose however, that we wanted to render the hallway as someone might see it during a hallucination. We could do this by using the familiar ‘toolkit’ of watercolour paints, but adapt the way the scene is rendered with these, based on typical features of hallucination. For instance, we could paint spiral patterns of dots in the scene, suggesting visual patterns of hallucination; present the hallway with warped lines to suggest perceptual distortions; or use stronger, brighter colours to indicate an increased visual sensitivity. By using such techniques, we would still be operating within the general domain of watercolour painting, with the standard toolkit of brushes and paints, but we would be adapting our approach by incorporating specific techniques related to typical features of an ASC experience. A similar adaptive approach can also be used in music, by working within an existing paradigm for musical composition, and modifying the use of this paradigm by taking ASCs into consideration. Psychedelic garage rock from the 1960s could be considered to exemplify such an approach.6 The songs operate within an existing framework provided by electric blues and rock n’ roll music. Hence, the artists use the familiar instrumentation of drums, bass, electric guitar, and vocals; verse-chorus structures; chord progressions; and the tonalities that we would usually expect to find in these genres of music. However, the music can be identified as ‘psychedelic’ because of the cultural context and historic period from which it emerges, and with this, certain psychedelic adaptations to the form of electric blues and rock n’ roll. For instance, the lyrics of songs such as The Electric Prunes’ ‘I Had Too Much to Dream (Last Night)’ (1966), Jefferson Airplane’s ‘White Rabbit’ (1967), or The Amboy Dukes’ ‘Journey to the Center of the Mind’ (1968) suggest themes of dreaming or hallucinating. In some cases, as heard on ‘I Had Too Much to Dream (Last Night)’, the productions also make significant use of effects processes such as echo, reverb, tremolo, and flanger, lending a warped, dreamlike quality to the sound, almost as if it were heard during an ASC. Another key work from this era, The Beatles’ ‘Tomorrow Never Knows’ (1966), makes use of droning tape loops and reversed materials, in correspondence with the theme of cyclical time and existence in the song; while The Jimi Hendrix Experience’s ‘And the

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

33

God’s Made Love’ (1968) also uses tape recordings with variable speeds, perhaps suggesting the distortions to time perception that may occur during an ASC (Ludwig 1969, pp. 13–14). Across these various examples, we find that familiar musical forms are modified towards various concepts of psychedelia. Furthermore, if we look more broadly at other ‘psychedelic’ genres of music—psychedelic folk, psy-trance, or stoner rock, for example—we would also often find similar adaptations, where the core approach owes a debt to established genre forms, but aspects of production and sound design modify these and orientate them towards tangible notions of psychedelia in cultural circulation. The electroacoustic compositions that I discuss in this section use a similar process of adaptation, but do so within the domain of electroacoustic music. These works are composed using the familiar toolkit of synthesis, sampling, and DSP techniques, including the use of processes such as granular synthesis, which are often used in this field of composition. However, the form of the work is adapted by designing various sonic materials based on features of ASCs. This process can also be understood in terms of Emmerson’s (1986) ‘mimetic discourse’, which describes the signifying potential of sound that results from referential or extrinsic qualities. By taking features of ASCs, such as concepts of visual or auditory hallucinations, these can be used to inform the design of mimetic sonic materials, which are then used to adapt the compositional form. This is the main approach that is used in the three fixed-media electroacoustic compositions that I discuss below.

Night Breed Media 2.1 Night Breed , electroacoustic music, 6 minutes 23 seconds Night Breed is the first composition that I created based on ASCs, and was performed at various electroacoustic events including the Royal Musical Association: Beyond the Dance (Keele University, 22 May 2008), and Energy Flash (Keele University, 5 May 2010) concerts. This composition utilises the typical approaches of electroacoustic music and electronic dance music,7 adapting these in accordance with various features of ASCs, which in this case were derived from Timothy Leary’s

34

J. Weinel

(1968) concept of ‘seven levels of energy consciousness’. According to Leary’s discussion, psychedelic drugs are able to elicit different forms of ‘energy consciousness’, each of which also corresponds with certain forms of art. Night Breed draws specifically on Leary’s concept of ‘cellular consciousness’, which he says can be experienced by consuming peyote or psilocybin mushrooms, and provides an awareness of the biological, cellular aspects of life. In a cellular psychedelic hallucination, concepts such as genesis, biology, evolution, and genetics might be experienced through hallucinatory manifestations of DNA coding, visualisations of cell structure and growth, biological processes, or evolution. Night Breed incorporates this concept of cellular consciousness by using various organic and morphing mimetic sonic materials.8 Following the previous discussion, Night Breed can be understood as an adaptation that modifies the approaches of electroacoustic music and electronic dance music towards a psychedelic aesthetic. While it is principally an electroacoustic composition, the piece draws significantly on forms of electronic dance music such as jungle/drum & bass, techno, and dubstep.9 The composition incorporates some familiar traits from these genres, but uses specific techniques to adapt the form in accordance with psychedelic ASCs. For example, at 1:00, 4/4 bass drum rhythms are introduced, which are similar to those used in techno music, while the section from 2:00 to 2:30 exhibits syncopated rhythms related to the patterns used in jungle. These are structurally organised as follows: {intro, rhythmic section 1, breakdown, rhythmic section 2, outro}; this is a pattern that is commonly used on many electronic dance music records. This provides the main form and structure of Night Breed , to which various adaptations are then applied in accordance with features of ASCs. As noted, the piece draws on Leary’s concept of cellular consciousness. Since this concept relates to cellular forms and organic matter, I attempt to incorporate ‘organic’ mimetic sonic materials in the piece. The interpretation of ‘organic’ that I use focuses specifically on natural dynamics of growth and decay, and complex variations in physical form, which are often characteristics of organic matter. Night Breed incorporates sounds that are made with organic materials, such as percussive sounds made by striking pieces of wood together. These sounds provide

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

35

a natural time-varying spectral form that is predictable, but also characterised by complex, subtle variations in timbre, which are caused by the interaction of the materials. The hands causing the collision of materials are also organic matter, and thus a natural process is embedded in the making of these sounds. A human vocal sample is also used, which can be considered an organic sonic material due to its biological origins. These various sounds can be heard throughout the piece from 0:45 onwards. The composition also utilises an organic approach to the transformation of sounds as they occur over time. For instance, as heard from 4:20 to 4:50, a pitch shifting effect is applied to the bass drum, which moves the frequency of sound up and down gradually. The design of the pitch envelope moves in a weaving organic motion following contours that we might find in nature if we were to view the way a river cuts through the landscape, or a snake as it moves through the grass. Rather than using mathematically perfect automation paths for the pitch-shift effect, these contours were achieved by recording parameter adjustments in realtime, thereby providing organic properties that arise from the imperfect motion of the human hand. In this way, the transformation of materials incorporates complex, non-linear variations that are mimetic of organic processes. Night Breed is also the first of many works that I will discuss in Explosions in the Mind , which uses the concept of visual patterns of hallucination as a basis for the design of mimetic sonic materials. As heard from 0:40 to 1:10, scattered rhythmic sounds are used in this piece, which were conceived in correspondence with visual hallucinations of cell structures. For example, one can imagine a hallucination of plant cells, which are scattered across the visual field as if viewed through a microscope. In order to approximately represent this visual impression through sound, organic rhythmic materials can be organised into a mimetic formation by using a sporadic distribution of their placement in time. Psychedelic hallucinations may also incorporate shifting changes to perception, such as colours seeming to appear to morph between brighter or duller hues, or a room seeming to grow smaller or larger at different points in time. To incorporate this idea of ‘shifting perception’ in Night

36

J. Weinel

Breed , the piece uses various filter, EQ, reverb, convolution, and amplitude envelopes to create an effect where sounds move in and out of the listener’s awareness, or morph into different versions of themselves. These shifting transitions can be heard from 1:50 to 2:10. In the section from 4:50 to 5:50, these morphing transformations were designed using organic contours, giving an aural impression of gradual submersion, almost as if the listener is moving underwater. In summary, Night Breed adapts typical approaches of electroacoustic music and electronic dance music, by using various techniques that are based on features of psychedelic ASCs. While the main form of the piece is derived from electroacoustic music and electronic dance music, the piece also incorporates various techniques that interpret features of hallucinations, focusing particularly on organic approaches to sound design, which are related to a cellular concept of hallucinations.

Surfer Stem Media 2.2 Surfer Stem, electroacoustic music, 7 minutes 9 seconds Continuing to explore Leary’s ‘seven levels of energy consciousness’, Surfer Stem uses the concept of ‘atomic consciousness’ as a basis for design. According to Leary, this level of consciousness is induced by drugs such as LSD, and causes hallucinations that enable the viewer to perceive atomic activity, such as the movement of atoms and electrons. Leary cites electronic music as an art form that is able to express this level. This concept speaks to the idea of electronic or digital hallucinations, which is something that William Gibson also describes in the classic cyberpunk novel Neuromancer (1984). Surfer Stem draws inspiration specifically from a passage of Neuromancer that describes a virtual reality (VR) hallucination, in which the protagonist finds himself trapped on a synthetic beach that extends to infinity. The composition itself was performed at various concerts including Sonic Fusion (University of Salford, 8 November 2013); the International Festival of Artistic Innovation (Leeds College of Music, 14 March 2014); and was included as part of an interactive sound installation for the Event Two exhibition of computer art (Royal College of Art, 15–17 July 2019).

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

37

In order to develop a hallucinatory electronic aesthetic, in contrast with the ‘organic’ approaches used for Night Breed, Surfer Stem prioritises the use of sonic materials with digital, synthetic qualities.10 The piece specifically draws upon the approaches of ‘flashcore’ music, a form of speedcore techno,11 which could be considered as a form of psychedelic cyberpunk music (Weinel 2007). In particular, I draw upon Laurent Mialon’s concept of ‘sonic atoms’, which describes micro-percussive units of sound. For Surfer Stem, I adapted this idea by designing rapid streams and clouds of percussive sounds, which I consider reflect patterns of atoms or electrons, such as one might perceive during Leary’s atomic consciousness. These sounds can be heard from 2:04 to 2:22, and were created through sequencing of percussive samples; granular synthesis techniques; and a Max/MSP12 patch that selects random percussive sounds from a bank of samples, and triggers them at random time intervals. This approach is one that I also develop in subsequent compositions, and the Max/MSP application used here was extended to provide the Atomizer Live Patch discussed in the next chapter. Drones created from vocal sounds can be heard from 3:20 to 3:45, and at various other points in Surfer Stem. These were created using samples derived from 1960s surf rock music,13 which were then processed using granular synthesis to create a time-stretching effect. Reverberation was also applied to these drones. By drawing on the sonic fingerprint or timbral characteristics of surf rock music, I intend these sounds to conjure a feeling or visual impression related to a sublime beach scene. Through the use of granular time-stretching, these sounds reflect a distorted perception of time, where an individual moment seems to extend to infinity.14 At the same time, the digital transformation of these materials is apparent, laying bare their synthetic and unreal qualities. Thus, the drones reflect a sonic impression of a virtual beach scene that extends infinitely, as in the passage of Neuromancer that the piece draws inspiration from. Surfer Stem also uses sonic materials that reference dub reggae and dubstep,15 as heard through the percussive guitar sound at 1:55; the slow rhythmic bass part from 2:04 to 2:50; and the simulated tape delay sounds from 3:01 to 3:20. I consider the echoing dub sounds as corresponding with perceptual distortions such as the visual trail effects

38

J. Weinel

that are often experienced during hallucinations, thus adding another psychedelic aspect to the composition. Overall, Surfer Stem uses juxtapositions of percussive rhythmic sounds and droning sounds to construct a sonic impression of a digital hallucination. The title of the piece is an adaptation of ‘SimStims’, the technology that William Gibson describes in Neuromancer, which broadcasts or records sensory inputs and experiences. Likewise, I conceptualise the piece as a sonic experience representing a digital hallucination of an infinite beach scene in VR, which pops and glitches with atomic visual hallucinations. The beach scene is primarily rendered with droning sounds derived from surf rock music, while the visual hallucinations are articulated with micro-rhythmic streams of ‘sonic atoms’.

Night Dream Media 2.3 Night Dream, electroacoustic music, 8 minutes 0 seconds Night Dream is the third electroacoustic composition that uses Leary’s ‘seven levels of energy consciousness’ as a basis for design. In this case, the concept used is ‘sensory consciousness’, which describes a heightened state of sensory awareness, as may be induced by marijuana, according to Leary. The piece continues to explore the use of drones, which are a primary feature, and are provided through several types of drone sounds and low-frequency sonic materials. Drawing upon ideas of ‘bass meditation’, which was popular in dubstep music at that time,16 bass is considered not only as a means of stimulating the ears, but also as a force with which to provide physical stimulation of the body. For Night Dream, this feeds into an overall concept of a sensory dream. The piece was first performed at MANTIS Festival (University of Manchester, 7 March 2009). Droning sounds are heard throughout Night Dream. These were technically produced through various forms of time-stretching, providing sonic textures that were then organised into layers, which fade in and out of the mix as it progresses in time. As with Surfer Stem, drones correspond with the ASC feature of distortions to time perception, and in this case are also underscored with low-frequency sinewave tones. These

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

39

synthetic tones are intended to enhance the sense of physical stimulation, especially when the piece is diffused at electroacoustic concerts where subwoofers are available. As heard from 0:20 to 1:20, the gradually evolving ‘macro’ drone sounds are processed with reverberation effects, and are presented with contrasting ‘micro’ rhythmic textures. The latter sounds do not have reverb effects applied, thereby providing a juxtaposing sense of open and closed spaces. As before, scattered rhythmic sounds are also used from 1:30 to 2:00, which are considered as corresponding with visual hallucinations. Completing this series of works, Night Dream is an electroacoustic composition based on ‘sensory consciousness’ and the idea of a dream. The piece explores this concept primarily through drones, and in doing so seeks to communicate distortions to time perception, while also incorporating sensory stimulation through the use of low-frequency materials. These are juxtaposed with micro rhythmic textures and scattered sounds, which are related to visual hallucinations. In summary, Night Breed , Surfer Stem, and Night Dream form a trio of works based upon Leary’s ‘seven levels of energy consciousness’. These pieces develop a variety of techniques that are used to adapt typical forms of electroacoustic composition, while also incorporating ideas from electronic dance music genres such as dubstep and flashcore. These forms were adapted by designing mimetic sonic materials, which are related to features of ASC experiences. Leary’s concepts inform the general approach that is taken for each piece: Night Breed interprets ‘cellular consciousness’ through organic sounds; Surfer Stem translates ‘atomic consciousness’ through a digital aesthetic; and Night Dream renders ‘sensory consciousness’ through low-frequency drones. Within these approaches, specific types of sonic materials are designed, which are considered to correspond with features of hallucinations. For instance, micro-rhythmic sounds are designed based on visual patterns of hallucination; while drone sounds are used to represent distortions to time perception. Various echo and reverb effects are also used to provide

40

J. Weinel

shifting perceptions of space, such as one might have during a hallucination. Through these various techniques, each compositional form is adapted towards specific notions of psychedelia.

Psychedelic Journeys in Sound The process of adaptation described in the previous section can be employed at various ‘micro’ and ‘macro’ levels of the composition; that is, it can inform the design of short individual units of sound, but can also be used to transform longer amplitude envelopes and DSP parameters. As a logical extension of this, we can allow ASCs to permeate the overall structural design of the composition at the top level. There is some precedence for this approach in works such as Barry Truax’s The Shaman Ascending (2004–2005), in which drone sounds are organised into rotating formations, which increase in pitch and velocity, in correspondence with the idea of an Inuit shaman ascending during a ritual. The works discussed in this section similarly organise electroacoustic materials in order to represent a hallucinatory journey or trip, so the progression of sounds in time reflects an imagination of what one might see or hear during such an experience on an approximately equivalent time frame. This can still be considered as an ‘adaptive’ approach to composition, but now ASCs inform both individual features as well as the overall structure. In what follows, I discuss three compositions that utilise this ‘structural’ approach.

Entoptic Phenomena Media 2.4 Entoptic Phenomena, electroacoustic music, 5 minutes 44 seconds The composition Entoptic Phenomena explores ways to represent visual patterns of hallucination through sound. The term ‘entoptic phenomena’ describes visual patterns of hallucinations that originate ‘internally’ within the visual system. According to Klüver (1971) the visual impressions seen during mescaline hallucinations may resemble ‘form

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

41

constants’: honeycomb, cobweb, funnel, and spiral forms. As discussed in Chapter 1 (p. 1), these geometric patterns may give rise to other visions derived from these basic forms; for example, a geometric funnel design could lead to impressions of tunnel environments in later stages of hallucination. While Klüver is inconclusive as to the origin of these forms, suggesting they occur within the eye, a subsequent study by Bressloff et al. (2001) suggests they are actually produced by patterns of connection between the retina and the visual cortex. The term ‘entoptic phenomena’ is also notably used in Lewis Williams and Dowson’s (1988) article ‘The Signs of All Times: Entoptic Phenomena in Upper Palaeolithic Art’, in which they argue that designs similar to the form constants indicate the presence of shamanic activity within the cultures from which various examples of rock art emerged. Taking the idea of visual patterns of hallucination as a point of creative departure, my composition Entoptic Phenomena develops the concept of a hallucinatory journey or narrative, which is used to inform the arrangement of sounds. The fixed-media version of the piece was performed at a departmental concert held at Keele University (4 November 2009), and a sixty second extract (under the title Atomizer ) was performed internationally as part of Robert Voisey’s ‘60 × 60: Sanguine mix’ and ‘60 × 60 Dance’ projects (e.g. Stratford Circus, London, 24 July 2010; for more information see: http://www.60x60.com). Entoptic Phenomena is based on an imagined DMT (N,N dimethyltryptamine; see Strassman 2001) hallucination that occurs in a sensory isolation tank,17 providing a conceptual narrative around which the piece is based. The narrative begins in the ‘external’ setting of a sensory isolation tank, in which an individual wearing a snorkel begins to hallucinate. As the experience ‘onsets’, waves of internal visual patterns of hallucination are experienced, which gradually increase in intensity. Following this, a ‘breakthrough’18 occurs and the experience moves into an intense ‘plateau’ phase of hallucination accompanied by a feeling of timelessness, in which individual moments seem to last much longer than usual, and the voices of strange entities are heard. Eventually, this gives way to another wave of visual patterns of hallucination, which gradually subsides as the experience ‘terminates’, returning us to the familiar

42

J. Weinel

external setting of the sensory isolation tank. As a powerful but shortlived hallucinogen, synthetic DMT hallucinations typically last between 5 and 15 minutes, and are sometimes referred to colloquially as ‘the businessman’s trip’. With a total duration of 5 minutes and 44 seconds, Entoptic Phenomena follows an {onset, plateau, termination} pattern on a roughly equivalent time frame, articulating what one might see or hear during such an experience. As indicated in the graphic score for Entoptic Phenomena (Fig. 2.1), each phase of the hallucinatory narrative is represented through corresponding sonic materials. In the opening, snorkel sounds suggest the experience of being in the familiar external setting of a sensory isolation tank. As the experience ‘onsets’, internal visual patterns of hallucination are represented through streams of micro-rhythmic percussion sounds (0:57–1:55). These sounds rotate in a circular motion within the stereo field, in correspondence with the funnel dot patterns of Klüver’s (1971) form constants, and were technically realised with The Atomizer Live Patch, a Max/MSP application that will be discussed in more detail in the next chapter. As heard prominently from 1:30 to 1:37, these rotating sounds utilise a Doppler effect. From 1:45 to 1:55 a crescendo is heard, accompanied by a whooshing sound, signalling the experience of ‘breaking through’ towards a deeper, more intense phase of hallucination. The piece then moves into the ‘plateau’ section from 1:56 onwards, during which droning sounds are used to suggest distortions to time

Fig. 2.1 Graphic score indicating the structure of Entoptic Phenomena

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

43

perception. Here, digitally transformed vocal materials are also used to suggest the voices of strange entities as auditory hallucinations. Next, in the final section of the composition, another wave of rapid percussion sounds in heard, in correspondence with visual patterns of hallucination (3:35–5:20), before the listener is returned once again to the familiar external snorkel sounds of the isolation tank, signalling the ‘termination’ of the hallucination. Entoptic Phenomena advances the previous compositional form, by not only using ASCs to inform the design of sonic materials based on specific features of hallucination, but also using them to inform the overall structural organisation of the piece. The idea of an imagined DMT hallucination that occurs in a sensory isolation tank provides a narrative concept, which the work is based upon. This narrative is developed through three main {onset, plateau, termination} sections (Fig. 2.1). Each of these phases of hallucination is provided through the use of corresponding sonic materials, taking the listener on a psychedelic journey through sound analogous to what one might see or hear as a hallucination unfolds.

Swamp Process Media 2.5 Swamp Process, electroacoustic music, 7 minutes 35 seconds John Uri Lloyd’s (1895) Etidorhpa: Or the End of the Earth is a ‘hollow earth theory’ novel,19 in which the protagonist undergoes a journey, traversing various caverns and underground lakes, encountering giant mushrooms, pointing hands, cubical crystals, and a strange being. In the story, drinking a ‘narcotic fungus’ precipitates these experiences, and so much like Samuel Taylor Coleridge’s ‘Kubla Khan’ (1797), the work can be read as an account of a drug-induced hallucination. Just as ‘Kubla Khan’ was informed by Coleridge’s actual experiences of opium intoxication, Terrance McKenna (1991, pp. 190–196) argues that Lloyd, a pharmacist and ethnobotanist by trade, may have drawn inspiration directly from his own experiences of psilocybin mushrooms. Whether by means of hallucination or imagination, Etidorhpa certainly emerges from

44

J. Weinel

the interior spaces of Lloyd’s psyche, providing an account of a hallucinatory journey, and in this regard it provides a point of inspiration for the next electroacoustic composition I will discuss: Swamp Process. Swamp Process is based on an imaginary hallucination that takes place in a gloomy swamp, within a cave environment. While primarily an electroacoustic composition, the piece also incorporates some approaches related to dub reggae and dubstep music, through subtle dancehall rhythms and basslines.20 The piece also utilises a subtractive approach similar to that used in dub music, where elements are composed, then subtracted, and brought fleetingly in and out of the mix. The fixedmedia version of Swamp Process was first presented at the 2012 edition of the International Computer Music Conference in Ljubljana, Slovenia (Institute for Sonic Arts Research [IRZU], 9–15 September 2012). Swamp Process develops a narrative concept where hallucinatory creatures lurk and emerge from the darkness of the cavernous swamp, spiralling around the listener, and retreating, before eventually engulfing us. These creatures are developed using an organic approach to sound design, as described earlier. According to Lewis-Williams and Dowson’s discussion, shamanic rock art renders animals seen in visionary experiences in dot formations that derive from Klüver’s (1971) form constants. Taking the idea of animals depicted through clusters of dots, Swamp Process uses clouds of percussive sounds to represent auditory ‘creatures’, which weave their way in and out of the mix, emerging and then retracting into the darkness of the cave. These sounds are first heard from 0:28 to 0:35, and occur repeatedly throughout the composition, eventually building towards a climax from 6:39 to 7:05, when the creatures engulf the spatial field completely. The composition also uses bass motifs, as heard from 1:12 to 1:44, which are associated with physical, bodily sensations of unease which may occur during hallucinations,21 developing the idea of low-frequency materials as a physical, sensory stimulus, as discussed in Night Dream. Similarly, Swamp Process also uses drones to suggest distortions to time perception from 1:25 to 1:46. In summary, Swamp Process continues to explore the idea of a hallucinatory narrative through the form of an electroacoustic composition with elements of dub music. The composition is based on the idea of a hallucination of a cavernous swamp environment inhabited by sonic creatures,

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

45

comprised of dot patterns related to Klüver’s (1971) form constants. These creatures are articulated with clusters of percussive sounds that move in and out of the auditory field, eventually engulfing the listener.

Nausea Media 2.6 Nausea, electroacoustic music, 19 minutes 11 seconds In a passage of Sartre’s (1938, pp. 182–183) novel Nausea, the protagonist Roquentin describes the onset of a strange experience, in which ‘existence… suddenly unveiled itself ’ (p. 183). His usual perception of the world is shattered. Things around him that were once innocuous now become terrifying. The way he perceives the roots of a tree, their form, and their usual meaning has ‘melted, leaving soft, monstrous masses, in disorder—naked with a frightening obscene nakedness’ (p. 183). In this passage Roquentin is experiencing an ASC comparable to the psychotic experiences of people with schizophrenia, or those under the influence of hallucinogens such as LSD22 In 1935 Sartre experimented with the drug mescaline, and this description is likely to have been based on his own experiences. Although the effects of this drug usually only last for a few hours, Sartre experienced further episodes for weeks afterwards and fell into a depression that lasted for six months (Drake 2005, p. 36). This literary passage informs the conceptual theme of the last composition that I will discuss in this chapter: Nausea. Realised in 5.1 surround sound, Nausea was included in the programme of the Internet Technologies and Applications 2013: Art Expo (Wrexham Glyndwr ˆ University, 11 September 2013), and an alternative version was included on the Mutable Audio compilation released on Anti Narcose Records (Various Artists 2018). The composition expands the ‘structural’ approach developed through Entoptic Phenomena and Swamp Process in order to provide a long-form composition with multiple movements. Nausea takes the idea of an ASC that occurs over a longer duration, and includes multiple ‘waves’ of hallucination. Each wave includes various combinations of visual hallucinations, distortions to timeperception, feelings of euphoria, anxiety, and physical sensations of unease or nausea. As with the previous compositions, sonic materials are

46

J. Weinel

designed in correspondence with various ASC features. Visual hallucinations are suggested through clusters of rhythmic sounds that rotate in the spatial field, in this case making use of the 5.1 surround sound format. Distortions to time perception are indicated through drones. Introducing a new idea, the concepts of euphoria and anxiety are suggested through ‘light’ and ‘dark’ sonic materials, respectively. Here, ‘light’ sounds refer to materials with a high spectral centroid, harmonic content, and slower attack and decay envelopes (e.g. 0:20–0:50); while ‘dark’ sounds refer to combinations of sonic material with fast attack and decay envelopes, and low spectral centroid, often processed with distortion effects (e.g. 16:30–16:50). The idea of physical discontent is indicated through dark, low-frequency bass materials, once again developing the idea of bass as a physical, sensory stimulus. The structure of Nausea is shown in Fig. 2.2. As with Entoptic Phenomena, the composition follows an {onset, plateau, termination} pattern, but this is now subdivided into six ‘waves’, which are expressed through corresponding musical sections or movements. The progression of these waves provides the overall structure and hallucinatory narrative of Nausea as follows: Wave 1 imagines a relaxed, euphoric phase of a psychedelic ASC. This begins with an initial breakthrough, signalling the onset of hallucination (0:02–0:12). Following this, from 0:10 to 1:45 we hear ‘light’, gently rotating ‘entoptic sounds’ (streams of rhythmic percussion based on visual patterns of hallucination), which gradually darken through means of pitch-shifting and filtering of the higher frequency content. Wave 2 moves into a darker, more uneasy phase of hallucination. From 1:40to 2:06, we hear reverberating metallic sounds and whispering voices, suggestive of perceptual distortions and auditory hallucinations. Reverberation suggests a large, cavernous space. From 2:06 to 3:25, bass sounds are introduced, corresponding with physical sensations, and a droning vocal sound is heard. Wave 3 begins at 3:25. We hear similar reverberating materials and entoptic sounds, which gradually increase in amplitude until a ‘breakthrough’ occurs at 4:26. Following this, sensory bass materials are used, and ‘dark’ rhythmic entoptic sounds are heard (e.g. 5:12), rotating with a

Breakthrough

Sensory bass

Female drone

Wave 3

Whispers

Sensory bass

Whispers

Female drone

Wave 4

Plateau

Fig. 2.2 Diagram indicating the compositional structure of Nausea

Breakthrough

Sensory bass

Dark

sounds

Light

Whispers

Yell drone

Wave 2

sounds

Wave 1

Onset

Sensory bass

Breakthrough

Dissonant drone

Wave 5 Yell drone

Wave 6 Female drone

Sensory bass

L Light ent so sounds

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

47

48

J. Weinel

higher velocity than before, reflecting intense visual patterns of hallucination. These sounds are processed without reverb, to indicate close visual proximity, and morph through successive layers using different frequency transformations, reflecting perceptual shifts. At 6:00, the mood turns towards euphoric sensations through the use of ‘light’ sonic materials. Sensory bass sounds are heard from 6:50 to 7:00, followed by entoptic sounds from 7:00 to 8:35. Wave 4 commences as the entoptic sounds of the previous wave subside, and we move into the plateau phase of hallucination, during which the listener is transported into strange, mystical planes of experience. Here the plateau is expressed through various reverberating materials, suggesting a vast unreal cavern (9:17), inhabited by whispering entities (9:28). Entoptic sounds are heard (e.g. 9:44), but now appear fleetingly as a subtle background feature. From 10:00 onwards, a vocal drone is heard, accompanied by sensory bass materials. Wave 5 indicates a rising sense of unease, as the plateau has passed and a sense of nausea is now growing (11:20–12:54). Here, dissonant droning sounds are heard, which gradually increase in amplitude and pitch. Wave 6 represents the last phase of hallucination and its final termination (12:54–19:12). A tearing sound is heard at 12:54, initiating another ‘breakthrough’, as successive layers of ‘dark’ entoptic sounds and drones are heard, which grow in intensity, peaking at 16:28. Rapid rhythmic sounds circle the listener furiously, with high amplitude levels, signalling intense visual hallucinations, accompanied again by sensory bass. This wave reflects an overwhelming experience of psychedelic nausea, but gradually the effects begin to subside; entoptic sounds are filtered to suggest their gradual dissipation from 18:10 to 18:28. Finally, we hear further bass sounds, corresponding with lingering physical sensations as the sonic hallucination draws to a close. In summary, Nausea develops the concept of a hallucinatory journey through sound, this time taking the idea of a powerful, quite unpleasant psychedelic experience, and using this to inform a longer piece which is structured through six waves of hallucination. These waves take us

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

49

through various intense phases of visual hallucinations, timelessness, and auditory hallucinations, before it all gradually subsides. One of the main innovations of this piece is the use of 5.1 surround sound for spatialising the rotating entoptic sounds, thereby exceeding the spatial possibilities of the earlier stereo compositions. Nausea also develops the idea of a psychedelic journey through a longer multi-movement work, providing 19 minutes and 11 seconds of nauseous psychedelia. The three compositions Entoptic Phenomena, Swamp Process, and Nausea adapt electroacoustic compositional forms towards notions of psychedelia. Extending the approaches used earlier, ASC features inform the design of corresponding sonic materials. Visual patterns of hallucination are interpreted through streams of percussive sound; drones are used to suggest distortions to time-perception; euphoria and anxiety are indicated through contrasting light and dark sonic materials; physical bodily sensations are depicted with bass sounds; and encounters with strange entities are rendered as vocal sounds or whispering voices, suggesting auditory hallucinations, as if the listener is hearing mysterious beings. However, the main development of these three pieces is the structural organisation of materials to construct imagined hallucinatory narratives. By organising sonic materials in correspondence with what one might see or hear during a hallucination, it is possible to create an ASC journey through electroacoustic music. These hallucinatory narratives derive inspiration from various literary sources, leading to three compositions that provide distinct psychedelic journeys through sound. ∗ ∗ ∗ In this chapter we have seen how electroacoustic music can be composed based on different concepts of psychedelia. Fundamentally, all the compositions discussed in this chapter utilise what I refer to as an ‘adaptive’ principle for composition. Using this approach, the standard toolkit of electroacoustic music, including synthesis, sampling, and digital transformations, is utilised to compose a sonic artwork. The first three works (Night Breed, Surfer Stem, and Night Dream) discussed in this chapter adapt the form of electroacoustic music by designing mimetic

50

J. Weinel

sonic materials based on specific features of hallucination, while also borrowing from other genres such as electronic dance music. The last three compositions (Entoptic Phenomena in Audio, Swamp Process and Nausea) then extend this idea by organising these materials to form hallucinatory narratives in sound, so that the composition as a whole, and its progress in time, becomes analogous to what one might see or hear in a hallucination. Before this chapter undergoes its own ‘termination’, and we move onto the next phase of Explosions in the Mind , there is one final wave of hallucination for us to discuss. As I will explore in later chapters, my work often traverses the boundaries of visual and sonic art, and when I compose music, these may be developed through ideas that are sketched visually, or vice versa. To provide an initial taste of this, the painting shown in Fig. 2.3 gives a visual impression of entoptic phenomena, corresponding with how I imagine it in these electroacoustic compositions. Circular funnel patterns are rendered in colourful ‘organic’ dots, which are suggestive of Klüver’s (1971) form constants. This image was used as the label artwork for the Entoptic Phenomena in Audio vinyl EP (Weinel 2014), which included Night Breed, Surfer Stem, Entoptic Phenomena, and Swamp Process, and was sold through Toolbox Records in Paris, an important record shop and distributor of underground electronic music. While the compositions on this record translate psychedelic visual hallucinations into sound, this painting translates those sounds back into a visual image.

Notes 1. The call for music made from noise-based sounds was made in the Futurist manifesto The Art of Noises (Russolo 1913), which is widely regarded as an important precursor to electroacoustic music. In this manifesto, Luigi Russolo famously proposed that “we must break at all cost from this restrictive circle of pure sounds and conquer the infinite variety of noise-sounds”.

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

51

Fig. 2.3 Painting representing visual patterns of hallucination, as used for the label artwork of the Entoptic Phenomena in Audio vinyl EP

2. Also of relevance to this discussion is DeNora’s (2010, pp. 21–45) discussion of how music may ‘afford’ certain interpretations, which are not fixed, but are established and reinforced over time through patterns of use. 3. Soundscape composition is a form of electroacoustic music, which prioritises the representation of recognisable sounds and places. For a further discussion, see Schafer (1994). 4. Truax (2002) describes his work in terms of ‘imaginary soundscapes’, which may explore or traverse the boundaries between recognisable sonic environments and unreal, dreamlike, or mythical spaces. 5. ‘Bottom up’ approaches to electroacoustic composition were discussed by Berezan (2010) in a lecture and concert at Keele University, who refers to Harrison (1999). 6. For an introduction to psychedelic garage rock, see the compilation by Various Artists—Nuggets: Original Artyfacts from the First Psychedelic Era,

52

7.

8.

9.

10.

11.

12. 13. 14.

15.

J. Weinel

1965–1968 (1972). For a further discussion see also Hicks (2000) and Weinel (2018). Hybrid combinations of electroacoustic music with various popular music and electronic dance music forms were also explored by several other composers working in the Keele University music studios at this time, for example see also Shave (2008, 2013) and Ratcliffe (2012). Cellular and organic forms are explored elsewhere in various forms of computer art. For example, see the morphogenetic 3D sculptures of Andy Lomas (2020); or the evolutionary computer graphics art of Stephen Todd and William Latham (1992). For an expanded discussion of electronic dance music genres such as jungle and techno, see Reynolds (2008). ‘Dubstep’ here refers to the genre of electronic dance music popularised by South London artists such as Skream, Loefah, Burial, and Digital Mystikz in the 2000s; see Walmsley (2009). In contrast with the ‘organic sounds’ of Night Breed , with Surfer Stem I wanted to achieve a more futuristic, digital sounding composition to express Leary’s atomic electronic level of energy consciousness. ‘Digital sounds’ can be achieved through various means such as emphasising the use of linear or stepped envelopes (as is possible with synthetic sound sources), and quantisation. ‘Speedcore techno’ is a form of techno music that uses fast tempos, typically above 200 bpm. ‘Flashcore’ is the term used by Laurent Mialon to describe work released on his label Hangars Liquides, particularly his own music as La Peste, which uses fast and irregular tempos, and dense waves of percussive electronic sounds. Flashcore extends the approaches of speedcore techno, while also taking influences from electroacoustic music. The term has since been adopted by other underground techno artists who have taken inspiration from these approaches; for instance, a search of the term on Toolbox Records (http://www.toolboxrecords.com/) will yield many results. For a further discussion of flashcore and Mialon’s music, see also Weinel (2007) and Migliorati (2016). Max/MSP is a visual programming language for sound and music. Surf rock is a specific genre of (often instrumental) rock n’ roll music that was popular in the 1960s; for a further discussion see Crowley (2011). This use of time-stretching can be understood in terms of Smalley’s (1986) discussion of continuant phases of sound, which can achieve dissociation from temporal notions of onset and termination. As discussed in Chapter 1 with reference to Veal (2007, pp. 209–210), dub reggae can be understood as a form of ‘psychedelic Caribbean’ music. In

2 Psychedelic Journeys in Sound: Electroacoustic Compositions

16.

17.

18.

19.

20.

21.

22.

53

some ways drawing parallels with electroacoustic music, dub uses experimentation with tape, audio effects, and other music technologies. As discussed by Jones (2017), reggae music is interwoven with multi-cultural British music culture, and signature traits from genres such as dub are often found in electronic dance music genres such as jungle/drum & bass (Belle-Fortune 2005) and dubstep. In the 2000s, dubstep music was often associated with the idea of ‘bass meditation’. For example, the DMZ club night was advertised with the phrase “come meditate on bass weight”, and the ‘bass meditation’ trope was used by various MCs and producers at this time. Sensory isolation tanks were used in combination with psychedelic drugs as a means to elicit hallucinations in the work of John Lilly (1972). This provided the inspiration for the movie Altered States (Russell 1980), and more recently, the character of Dr. Martin Brenner in the Netflix series Stranger Things (Duffer and Duffer 2016–present), which depicts various hallucinations in sensory isolation tanks. Various ‘breakthrough’ experiences are described by participants in Strassman’s (2001) DMT studies (e.g. pp. 179, 213), in which the hallucinations move beyond visual patterns and various encounters with entities may be experienced. ‘Hollow earth theory’ presumes that the earth contains substantial interior space. The theory has been used as a source of inspiration for many fictional novels, most famously Jules Verne, Journey to the Centre of the Earth (1864); and also films such as The Core (Amiel 2003) and Journey to the Center of the Earth (Brevig 2008). One of the main points of inspiration here is The Bug’s Pressure (2003) album, which uses syncopated dancehall rhythms throughout, and can also be linked with the emerging form of dubstep music in the early 2000s. For example, hallucinogens such as the peyote cactus or yagé (the hallucinogenic brew containing DMT, which is used in shamanic ceremonies of the Amazon rainforest) are known to cause physical discomfort and nausea. Bodily sensations in general may also be heightened during episodes of hallucination. Although psychotic experiences and LSD hallucinations are distinct forms of ASC, it is possible to draw some comparisons between the form of these experiences. For further discussion see Adams (1994), Hobson (2003, pp. 5–6), and Blackmore (2003, pp. 307–308).

3 Melting in the Mind’s Eye: Real-Time Performances

In the well-worn concert hall of the Camden Centre, in King’s Cross, London, a figure in black kneels before a huge bass drum hanging from a frame. Around it, various hand-made gongs and percussion constructed from pieces of metal are similarly suspended from frames. The audience sits attentively, as a ritual of trance-like drumming commences. With powerful precision, the drummer releases rhythmic streams of percussion, at once hypnotic, yet spontaneous and organic. Each pulse stimulates the auditory systems of those in attendance, inducing an almost dizzying energy that measures out, subdivides, and transforms their experience of time. The bass drum sways gently as the beaters strike it repeatedly, yet it remains under careful control. In combination with the large acoustic space, the drum emits deep reverberations, the sonic reflections seeming to activate every surface of the hall. You can feel the Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/978-981-16-4055-1_3. Where the icon is shown in the chapter, the reader should refer to the supporting media files.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 J. Weinel, Explosions in the Mind, Palgrave Studies in Sound, https://doi.org/10.1007/978-981-16-4055-1_3

55

56

J. Weinel

floorboards begin to vibrate as the room gradually accumulates acoustic energy. The percussionist seems to tune these effects to the characteristics of the acoustic space, manipulating the low-frequency vibrations to provide a deeper layer of sound that stimulates the whole body, not just the ears. Every so often one may attend a concert that transcends the everyday. Such was the case with this particular performance by Z’EV, and his collaboration later that day with John Zorn at the Equinox Festival of occult music and culture (London, 12–14 June 2009). Z’EVs trancelike rhythms demonstrated the potency that a live performance can have, and inspired me to make a percussion-based real-time performance of my own,1 which I will discuss later in this chapter. Various other parts of the Equinox Festival programme also turned out to be formative, since the festival featured films by Harry Smith, whose work I will discuss in the next chapter; and a performance by James Ferraro, who is often credited with pioneering the genre that became known as vaporwave2,3 which I explore in Chapter 7. The main focus of this chapter is on live performances—or rather, ‘real-time performances’, as they are often called in the domain of electronic music, where works may be performed using real-time audio mixing, synthesis, and sample triggering.4 Though rehearsed and prerecorded sound elements may still be included, real-time approaches can allow electronic music to be realised less predictably in live situations, allowing the performer to respond to the venue and the audience, much as Z’EV tailored his performance at Equinox Festival. In this chapter I will begin by discussing a software application: the Atomizer Live Patch, which I designed in Max/MSP to design sounds for various fixed-media compositions such as Entoptic Phenomena, which I discussed in the previous chapter. While this application was originally used for creating sonic materials for fixed-media works, I later expanded it to facilitate real-time performances of those compositions. The culmination of this was a laptop performance: Entoptic Phenomena in Audio, which I presented at the ISSUE Project Room in New York City (31 May 2010). In this chapter I will discuss the design of both the software, and its use for this performance. Following this, we will look at a subsequent piece of software: Bass Drum, Saxophone & Laptop, an

3 Melting in the Mind’s Eye: Real-Time Performances

57

automated real-time performance system that I designed to integrate psychedelic electroacoustic approaches with an instrumental composition. I will explain both the design of the software itself, and its use for three example recordings. Through the course of this chapter and supporting media examples, we will see how approaches for representing altered states of consciousness (ASCs) can be utilised for real-time performances, and how software can yield new forms of real-time psychedelic composition and performance.

Software: Atomizer Live Patch Media 3.1 Atomizer Live Patch, Max/MSP software The Atomizer Live Patch is a Max/MSP application that I originally designed in order to create sonic materials in correspondence with the visual patterns of hallucination described in Klüver’s (1971) mescaline studies. As shown in Fig. 3.1, we can readily design visual images and animations that correspond with Klüver’s form constants.5 However, for my electroacoustic compositions, I wanted to interpret these visual patterns of hallucination through sound. To do this, I drew partly on the approaches of La Peste’s concept of ‘sonic atoms’ (as discussed in the previous chapter), adapting this idea to imagine funnels of dot patterns as circular, rotating arrangements of rhythmic pulses. Each rhythmic pulse is imagined as a visual point of light, and by rotating these in the spatial auditory field, one can conceive of a formation in sound that corresponds with visual patterns of hallucination. A spiralling vortex of

Fig. 3.1 Author’s sketches of Klüver’s (1971) form constants

58

J. Weinel

visual points becomes a spinning matrix of percussive pulses, thereby translating psychedelic visualisations into sound. The Atomizer Live Patch is the software that I designed to facilitate the creation of these sounds, and later I expanded it to include several other types of sound generation, so it could be used for making drones and triggering samples in real-time performances. The first main iteration of the software was designed for use with the Novation Nocturn and Korg NanoKontrol MIDI controllers, and this version was used for several of the compositions discussed in the previous chapter.6 However later on I adapted the software for use with a CME Bitstream 3X MIDI controller, providing real-time 5.1 spatialisation and a more focused user interface. My discussion here will focus on this later version, which was used for my live performance in New York City. In what follows, I will describe the main functions of the Atomizer Live Patch.

Atomizer As seen in the top-left corner of the user interface (Fig. 3.2), the ‘atomiser’ is the main sound generation module of the software, which creates streams of micro-rhythmic sounds based on visual patterns of hallucination. The module provides sample banks of 10 short percussive sounds, which can be selected from a list of presets, or by loading other samples using the ‘custom’ button, which opens a sub-panel of the interface. These sample banks store variations of electronic pulses, organic sounds, metallic percussion, recordings of pieces of wood being struck, and cactus spines being plucked. When the module is triggered, either a specific or randomly selected sound is played. Sounds can be played as polyphonic ‘one-shots’, or as monophonic loops. The module can be triggered at random intervals by using the ‘random speed’ slider from the bank of controls directly beneath the module. Increasing this slider from the zero position activates triggers at random intervals with increasing regularity. This enables the streams of percussive sounds to be organised irregularly in time. Alternatively, the module can be triggered with a matrix sequencer. This is activated using the ‘sequence’ switch on the module, while

3 Melting in the Mind’s Eye: Real-Time Performances

59

Fig. 3.2 Atomizer Live Patch, main user interface

the ‘matrix’ button opens a sub-panel of the interface with a 128-step sequencer. This matrix allows rhythmic patterns to be designed that trigger any sample in the bank being used, specifically or at random. This sub-panel allows the user to select the BPM and also the number of ‘ticks’ (sequencer steps7 ) per beat. Patterns can be stored and recalled with a preset system, and on the main module, the ‘#’ button allows a pattern preset to be selected at random. Both the sequencer and the ‘random speed’ trigger can be used concurrently, interlacing regular and irregular triggers to form complex rhythmic patterns.

60

J. Weinel

The panel of controls beneath the atomiser module on the user interface provides various dials and sliders for manipulating the sound. A ‘loop point’ dial adjusts the end point of the loop, when the looping mode is switched on. ‘BPM’ changes the speed of the sequencer when active. ‘L/R’ pans the signal from left to right, while ‘F/B’ moves it from front to back in 5.1. These are applied additively with a Doppler shift based on a design by Rajmil Fischman, which provides a rotating effect. Using the ‘Doppler width’ and ‘Doppler pitch’ controls, the width and speed of rotation can be adjusted to elicit various circular and spiral spatialisation patterns that can be applied to the streams of percussive sound, thereby generating patterns that correspond with the form constants. A ‘deform’ control transforms the sound with a semi-random pitch bend effect based on the random interpolation of two internal breakpoint graphs. The ‘deform’ control adjusts the amount by which this value affects the pitch of the sample being played. The ‘speed’ control also affects the overall speed at which the sample is played, changing the pitch. The ‘atoms volume’ slider changes the amplitude of the samples, while the remaining dials and sliders provide controls for various effects: distortion, ring modulator, delay, reverb, and filter, which allow the rhythmic streams to be coloured in various ways. ‘A’ and ‘B’ buttons are also provided to quickly switch between preset effects configurations.

Atomizer Joystick/Ribbon The ‘atomiser joystick/ribbon’ (seen on the right-hand side of Fig. 3.2), provides a second atomiser module. This has a reduced set of controls, but allows the module to be manipulated with the x /y joystick and ribbon controllers of the CME Bitstream 3X. Touching the ribbon triggers the samples, allowing gestural bursts of sound to be produced. Meanwhile, the x /y joystick controller is mapped to affect various effect parameters, so moving the axis horizontally or vertically colours the sound in various ways. The joystick also allows these sounds to be spatialised in 5.1, where moving the joystick horizontally affects the left/right positioning, while moving it up and down affects the

3 Melting in the Mind’s Eye: Real-Time Performances

61

front/back location of the sounds. This is particularly effective for live multi-channel performances, where touching the ribbon while moving the joystick in a circular motion allows spiral formations of rhythmic sounds to be constructed ‘manually’ in real-time. This module also includes options for triggering sounds based on the matrix sequencer; switching looping on and off; and adjusting the reverb, distortion, and amplitude.

Drone Machine The ‘drone machine’ module (seen at the centre-top of Fig. 3.2) provides facilities for making drones, which correspond with distortions to timeperception (as discussed in Chapter 2). The module provides a granular synthesiser, which is configured specifically for making drones by means of granular time-stretching. A source sound can be loaded, from which the grains are extracted. The time value changes the point in the source sound that the grains are taken from, and moving this value slowly back and forth when using a continuous source such as a vocal sample, creates droning sounds that retain the sonic characteristics of the source. The ‘scrub’ slider on the control panel beneath the ‘drone machine’ allows this value to be controlled via MIDI, while the ‘scrub speed’ dial directly above modifies the rate at which this parameter changes. The ‘grain size’ dial changes the size of the grains, while ‘width’ adjusts their left/right spatial distribution. The ‘grain volume’ slider changes the overall amplitude of the drone, and the dials above provide spatialisation controls to pan the sounds between left/right and front/back positions. A reverb control is used to apply a plate reverb effect. Lastly, as discussed in the previous chapter, several of my compositions use low-frequency bass tones. These sounds can be generated with the sine-wave oscillator on this module, using the ‘bass’ (amplitude) and ‘frequency’ dials. This feature was partly inspired by Z’EV’s manipulation of low-frequency sounds with respect to the acoustic and resonant properties of the concert venue, as described in the opening of this chapter. By providing a sine-wave oscillator, it is possible to create droning sub-bass sounds that can be tuned to the performance space in live situations.

62

J. Weinel

DJ Mixer The ‘DJ mixer’ at the bottom-right corner of the user interface (Fig. 3.2) allows pre-planned sound materials to be triggered and mixed spontaneously, in the style of a continuous DJ mix. This provides two playback modules, ‘deck A’ and ‘deck B’, which are analogous to the two turntables of a typical DJ setup. Each ‘deck’ provides 3 sound file players, which allow several audio files to be triggered simultaneously. These are intended for use with either long pre-composed sections of music or short gestural sounds. Using the crossfader it is possible to fade between the sounds from ‘deck A’ and ‘deck B’. A ‘fade angle’ is provided, which provides different crossfade envelopes.8 In summary, drawing on my background as a DJ,9 the ‘DJ mixer’ module allows for the spontaneous live mixing of pre-composed sections of music, while various complementary drones and streams of rhythmic sounds can be improvised alongside these using the other modules.

Audio Output Finally, the ‘audio output’ module (visible in the top-right of Fig. 3.2) provides essential functions for selecting the soundcard, digital signal processing (DSP) status, and output levels. Amplitude meters are provided for 5.1 configurations. A ‘record’ module opens a sub-panel of the user interface, allowing the output to be recorded in real-time. This can be used to record whole performances; or improvised ‘jam sessions’, which may be used to generate sounds that are later edited for further use in fixed-media compositions. The former approach was used to make the recording discussed in the next section of this chapter, while the latter was used for many of the compositions discussed in Chapter 2. In summary, the Atomizer Live Patch is a bespoke Max/MSP application, which was designed to facilitate the production of sonic materials for fixed-media compositions, and for longer real-time performances related to these. Various sound generation modules are provided, enabling complex streams of rhythmic sounds, which rotate in the spatial

3 Melting in the Mind’s Eye: Real-Time Performances

63

field in correspondence with visual patterns of hallucination; and drones, which relate to the distortions of time-perception that may be experienced during ASCs. In real-time performance situations, these sounds can be combined with various pre-recorded sounds, which are mixed in a manner analogous to a DJ set. The software can be used with 5.1 multi-channel sound systems, and the output can be recorded. In the next section, I will discuss in further detail how all these features were used for a real-time performance in New York City.

Live in New York City In 2010, I performed a real-time set of my electroacoustic music at the Analogous Interactions concert at the ISSUE Project Room in New York City (31 May 2010), held as part of the International Computer Music Conference. The performance: Entoptic Phenomena in Audio, extends the concept of hallucination that occurs in a sensory isolation tank, using the Atomizer Live Patch to reconfigure multiple electroacoustic compositions as real-time ‘versions’.10 In this section, I will discuss this performance with reference to a recording of a rehearsal.11

Entoptic Phenomena in Audio Media 3.2 Entoptic Phenomena in Audio (NYC 2010), recording of a real-time electroacoustic performance, 19 minutes 53 seconds Entoptic Phenomena in Audio combines elements from Night Breed, Surfer Stem, Night Dream, Entoptic Phenomena, and Swamp Process, which were discussed in Chapter 2; and the audio track from Tiny Jungle (an audio-visual composition that will be discussed in the next chapter). Some pre-recorded saxophone sounds are also used, which were taken from the Bass Drum, Saxophone & Laptop (2010) project discussed later in this chapter. The overall performance was approximately 20 minutes in length, and this is similarly reflected in the rehearsal recording (Media 3.2).

64

J. Weinel

The overall structure of Entoptic Phenomena in Audio is based on an extended version of Entoptic Phenomena, following a similar {onset, plateau, termination} pattern. The piece imagines a hallucination that occurs in a sensory isolation tank, using the ‘onset’ and ‘termination’ sections from Entoptic Phenomena to ‘bookend’ the performance. Using this approach, the piece begins and ends with the snorkel sounds of a sensory isolation tank, and everything that happens in between during the performance is conceptualised as an extended sonic hallucination. This hallucination is constructed from various ‘waves’ made up of materials derived from Night Breed, Surfer Stem, Night Dream, Entoptic Phenomena, Swamp Process, Tiny Jungle, and Bass Drum, Saxophone & Laptop. The Atomizer Live Patch allows for the flexible organisation of materials at the ‘macro’ structural level, and so I was able to experiment with different arrangements through my rehearsals. Figure 3.3 has been redrawn from my rough notes, and indicates the approximate structure of the performance. The ‘Drone Machine’ row details the type of source sounds to be used for creating drones with the corresponding module of the Atomizer Live Patch. ‘Sax’ indicates drone materials from Bass Drum, Saxophone & Laptop; ‘Breed’ drones are from Night Breed ; and ‘Surf ’ drones are from Surfer Stem. The ‘DJ Mixer’ row shows elliptical shapes specifying pre-recorded sections of music to play with this module. These pre-recorded sections were taken from the fixed-media versions of the compositions, however some material such as drones and streams of percussion were removed, so that these sounds could instead be improvised in real-time with the ‘drone machine’ and ‘atomiser’ modules of the Atomizer Live Patch. ‘Ent01’, ‘Ent02’, and ‘Ent03’ are sections from Entoptic Phenomena; ‘Swamp01’ is from Swamp Process; ‘Breed01’ is from Night Breed ; ‘Tiny02’ and ‘Tiny03’ are from Tiny Jungle. It should be noted that the performance notes do not list everything that will be used, and some shorter sounds may also be triggered spontaneously during performances. The next two rows of the performance notes show which sample banks to use with the ‘Atomizer’ and ‘Atomizer Joystick’ modules. Here ‘Ent’ and ‘Blip Ent’ reference banks of electronic pulses used in Entoptic Phenomena; ‘Sax’ is a short bank of one-shot saxophone sounds from Bass Drum, Saxophone & Laptop; ‘Wood’ and ‘Bath’ are banks of

Ent

Atomizer Metal

Ent02

Sax

Bath

Wood

Sax Cactus

Breed 01

Breed

Swamp 01

Sax

Cactus

Sax

Sax

Bath

Metal (Loop)

Tiny01

Sax

Tiny03

Blip Ent

Surf

Fig. 3.3 Performance notes indicating the structure of Entoptic Phenomena in Audio

Ent01

DJ Mixer

Drone Machine

Bath

Blip Ent

Ent02

Sax

Ent03

3 Melting in the Mind’s Eye: Real-Time Performances

65

66

J. Weinel

percussion used in Night Breed and others; ‘Metal’ is a bank of metallic sounds used in Night Dream and others; and ‘Cactus’ is the bank of cactus spines being plucked, as used in Tiny Jungle. As the performance notes indicate, the arrangement mixes sounds from multiple compositions concurrently and these flow into each other, forming a greater whole. We may now turn to consider how this plays out on the rehearsal recording (Media 3.2). At the beginning we hear the opening section of Entoptic Phenomena, with snorkel sounds indicating the ‘external’ setting of a sensory isolation tank. Following this, we gradually transition into ‘internal’ sounds suggestive of hallucinations, as various ‘entoptic sounds’ are introduced, signalling the onset of visual patterns of hallucination, which gradually engulf the senses (0:30–2:00). At 2:00, a breakthrough is heard and we move to the next ‘wave’ of hallucination. From 2:00 to 5:25, we hear various droning sounds, some from Entoptic Phenomena, others made using saxophone sounds, and percussive streams. This section transitions into source materials from Swamp Process, during which organic cactus sounds and maniacal-sounding saxophone pulses are heard.12 At 5:25, a new ‘wave’ of hallucination begins, based on the rhythmic section of Night Breed . Here, the various entoptic sounds and drones are improvised with the Atomizer Live Patch, thereby creating a unique variation of this composition, while retaining recognisable characteristics. At around 8:15 the music diverges from Night Breed , and transitions into another drone section, recapitulating the saxophone materials heard earlier in the performance. Beginning at 9:12, we break through into a different ‘wave’ of hallucination, as the rhythmic track from Tiny Jungle begins. Metallic entoptic sounds are heard, some of which are created by switching to the ‘loop’ mode of the Atomizer Live Patch, as indicated in the performance notes (Fig. 3.3). The recurring saxophone drone is also heard at points during this section. At 11:28, a clicking sound signals a transition to the next ‘wave’ of hallucination. Entoptic sounds corresponding with visual patterns of hallucination give way to the emotive droning sounds from Surfer Stem, which represent a vision of a beach (as discussed in the previous chapter).

3 Melting in the Mind’s Eye: Real-Time Performances

67

This is the ‘eye of the storm’; a calm phase during the central ‘plateau’ of the hallucination. In terms of the performance techniques, here the drone sound is created by ‘scrubbing’ back and forth slowly with the 1960s surf rock vocal source sound from Surfer Stem, in such a way that the drone alternates between two notes, so that this single moment seems to trail on indefinitely. Beneath the drone, a throbbing low-frequency sub-bass from Tiny Jungle can be heard. At around 15:00 the beach vision begins to tumble away, as rapid pulsing sounds begin to overtake the spatial auditory field. An intense wave of sounds suggestive of visual patterns of hallucinations circle and spiral around the listener, created using the various ‘atomiser’ modules of the Atomizer Live Patch, while drones persist in the background. This section is essentially a reconfiguration of the closing section of Entoptic Phenomena, and also includes pre-recorded source materials from this composition. As in the fixed-media version of this composition, from 19:20 onwards, the hallucinations melt away, returning the listener from the ‘internal’ world of hallucinations to the familiar ‘external’ world of the sensory isolation tank as the experience ‘terminates’. The actual performance of Entopic Phenomena in Audio at the ISSUE Project Room was deeply satisfying, and benefitted from a loud P.A. system, powerful subwoofers, and the venue’s 15-channel hemispherical ‘floating points’ sound system.13 The 5.1 surround sound output of the Atomizer Live Patch was mapped across these channels, providing effective spatialisation of the circular and spiral rhythmic sounds, in a manner ideally suited to the piece. Likewise, the subwoofers enabled the sinewave bass drone oscillator to sit beneath the mix in key sections, and this could be tuned to resonate with different parts of the room in the live context as intended, by adjusting the frequency dial with the BitStream 3X MIDI controller. The performance seemed to be well received by the local Brooklynites in attendance, who expressed their gratitude with offerings of liquor and after parties!

68

J. Weinel

Software: Bass Drum, Saxophone & Laptop Media 3.3 Bass Drum, Saxophone & Laptop, Max/MSP software We may now turn to consider BassDrum, Saxophone & Laptop, a piece for live instruments and electronics. Drawing inspiration partly from Z’EV, the piece grew out of regular improvised14 jamming sessions with Sol Nte, in which I played a 26” marching band bass drum hung from a gong stand, and Sol played saxophone. For several years we explored variations of this setup through regular jamming sessions at Keele University. I decided to explore the augmentation of these performances through the design of another Max/MSP application: Bass Drum, Saxophone & Laptop. The software enhanced our instrumental improvisations by providing real-time triggering of sounds, and DSP effects, which responded organically to our performances, thereby eliciting psychedelically enhanced versions of them. The design of the Bass Drum, Saxophone & Laptop application adapts various modules from the Atomizer Live Patch for use in a performance for live instruments and electronics. As before, the design of the software was based on concepts of ASCs and features that may typically occur in these experiences. In particular, the software takes the idea of ASCs as providing temporal shifts in perception, where colours fade between lighter or darker hues, or may seem to morph over time. In a psychedelic ASC experience, such as one might have after consuming psilocybin mushrooms, if one were to gaze over a terracotta pot while hallucinating, the pink shades of the terracotta might be more noticeable in one moment, while the orange shades might become more prominent in the next. Colours may seem to glow with unusual high-contrast intensity in one instance, before fading to a dull, almost grey colour in the next, gradually changing and morphing over time. These effects can be interpreted in sound by changing spectral characteristics, ‘colouring’ the frequency properties through the use of DSP effects such as filters or ring modulators, and gradually modifying parameter envelopes in time. We can also think of these shifting perceptual changes in terms of space. Just as Alice, in her adventures in Wonderland (Carroll 1865– 1871), had experiences of growing and shrinking; during an episode of

3 Melting in the Mind’s Eye: Real-Time Performances

69

hallucination, a room may seem to expand in one moment and contract in the next. Of course, neither the size of the room or the individual really changes, but the perceptual system seems to become attuned to the ‘open’ and ‘closed’ aspects of space at different points in time. In sound, we can create corresponding sonic representations of this through means of spatial processes such as reverb, which allow us to transform sounds so that they appear as if heard in large spaces in one moment, and in close, compact spaces in the next. These concepts are particularly important for Bass Drum, Saxophone & Laptop, which facilitates these gradual shifting perceptual effects through automated DSP processes. Continuing to build on earlier approaches, the software also provides triggering of rhythmic sounds that relate to visual patterns of hallucination; delay effects that correspond with visual trail effects; and sensory derangement through rotating Doppler effects, which allow sounds to move erratically in the spatial field. These various processes are applied to audio signals from the bass drum and saxophone, which are captured through microphones. In what follows, I will describe the operation of each module of the software. In support of this discussion, Fig. 3.4 shows the user interface of the software.

Main Patch Controls/Presets At the top-left corner of the user interface we can see the main patch input controls and presets panel. This provides options for selecting the audio device, a bank of presets, and an ‘info’ button which opens an instruction manual. Much as one would find on any good keyboard, there is a ‘demo song’ button, which triggers pre-made recordings of a bass drum and saxophone, which are then processed in real-time, thereby demonstrating the patch in action for testing purposes. This panel also features a red light to indicate when the output is being recorded.

Fig. 3.4 User interface for the Bass Drum, Saxophone & Laptop Max/MSP application

70 J. Weinel

3 Melting in the Mind’s Eye: Real-Time Performances

71

Sax Input and Trigger/Drum Input and Trigger Moving right from the top-corner, the top two rows provide various modules related to the saxophone and bass drum instruments, respectively. The ‘sax input and trigger’, and ‘drum input and trigger’ modules receive microphone inputs from the respective instruments, which the patch will process with various DSP effects, and use to generate trigger messages. These modules allow the correct input channels of the soundcard to be selected and the amplitude of the incoming signals from the microphones to be adjusted. These modules activate triggers when the input signal reaches a certain level, which can be set with the ‘trigger level’ control. When a trigger is activated, it cannot be triggered again until the module reaches the ‘retrigger delay’ threshold in milliseconds. These modules also allow pre-recorded sound files to be loaded and used, instead of live microphone signals.

Sax Effects Rack/Drum Effects Rack The ‘sax effects rack’ and ‘drum effects rack’ modules provide various DSP effects, which are similar to those used in the Atomizer Live Patch. These include a rotating Doppler effect, distortion, ring modulator, delay, and a plate reverb unit. Various parameters and bypass options are available for each effect. These parameters can be set manually, however the patch also provides the option to automate these using control data from various other modules, received via the ‘patch bay’.

Sax Trigger Envelope/Drum Trigger Envelope The ‘sax trigger envelope’ and ‘drum trigger envelope’ modules receive trigger messages from the ‘sax input and trigger’ and ‘drum input and trigger’ modules. Here, these messages are used to trigger three different envelope generators. When a trigger message is received, three new envelopes are generated with different attack and decay properties. These can then be used as control envelopes, as selected with the ‘patch bay’

72

J. Weinel

module, so (for example) sounding the bass drum can be used to automate effects parameters of the saxophone, and vice versa.

Sax Scatter Envelope/Drum Scatter Envelope Along similar lines, the ‘sax scatter envelope’ and ‘drum scatter envelope’ modules provide another means of generating control envelopes that can be used to automate DSP parameters elsewhere in the patch, via the ‘patch bay’. When this module receives a trigger, it generates an envelope using an array of 50 slider values that can be manually adjusted. These slider values set where peaks of the envelope will occur in time; so each non-zero value will generate a new peak, thereby providing a complex envelope with multiple ‘scattered’ peaks each time a trigger is received. The overall time distribution of these peaks in milliseconds is effected by the ‘total scatter time’ parameter. These modules include options for ‘attack’ and ‘decay’ of the scattered envelopes; a ‘combine’ option that generates the scattered envelopes additively; and an ‘invert’ option which allows the peaks to be subtracted from a maximum value.

Sax Sustained Playing Envelope/Drum Sustained Playing Envelope Moving along to the right on the user interface panel, we find the ‘sax sustained playing envelope’ and ‘drum sustained playing envelope’ modules. These modules trigger envelopes only when sustained playing on the corresponding instrument is detected. This detection system works by receiving triggers from the input trigger modules. If the target number of triggers (x ) is reached within a specified period of time (y), then a ‘sustained playing envelope’ is activated. This sustained playing envelope remains active until a specified period of time (z ) passes without any triggers being heard. These modules provide an effective means of differentiating shorter bursts of sound from sustained playing, generating control envelopes that can be routed via the ‘patch bay’ module.

3 Melting in the Mind’s Eye: Real-Time Performances

73

LFO Bank/LFO Bank 2 While the various saxophone and drum envelope modules allow control envelopes to be generated based on the input signals from the corresponding instruments, the ‘LFO bank’ and ‘LFO bank 2’ modules each provide two low-frequency oscillators (LFOs), which operate independently. Different waveforms, frequency, amplitude, and offset properties can be selected for each LFO. The LFOs can also be routed via the ‘patch bay’ to control other parts of the patch.

Drunk Bank/Drunk Bank 2 The ‘drunk bank’ and ‘drunk bank 2’ modules each provide further means of generating two envelopes independently of the input signals. These modules generate randomised envelopes that wander ‘drunkenly’ between values. Parameters for ‘speed’, ‘range’, ‘set’, and ‘offset’ affect how these envelopes are generated. As before, each control envelope can be routed via the ‘patch bay’ module.

Atomiser Located at the bottom left of the user interface, the ‘atomiser’ module is an adaptation of the same module described earlier in the Atomizer Live Patch. This module triggers streams of sound in correspondence with visual patterns of hallucination. As before, this module can be used with various selectable sound banks, each of which contains 10 percussive samples. The module receives a trigger, either from the saxophone or the bass drum, which then triggers one of the percussive samples from the bank at random. Whereas the Atomizer Live Patch provided a ‘random speed’ slider, which triggered the module at random intervals, Bass Drum, Saxophone & Laptop replaces this with a ‘scatter’ function, which initiates a series of retriggers according to a pattern selected by the user (using a similar interface to the ‘sax/drum scatter envelope’ modules). When this option is selected, the module initiates a series of ‘scattered’ percussive sounds from a single trigger.

74

J. Weinel

This module also provides various other control parameters, which are similar to those found on the ‘atomiser’ module of the Atomizer Live Patch. Thus we have options for sample volume, panning, speed, pitch deformation, and a looping mode. Moving along the panel, we then also find DSP effects identical to those found on the ‘sax/drum effect rack’ modules, with Doppler, distortion, ring modulator, filter, delay, and reverb. Critically, as with the ‘sax/drum effect rack’ modules, all of these parameters can be exposed in the ‘patch bay’, so that the various control modules can be routed in a myriad of ways to automate the values.

Patch Bay Moving rightwards along the bottom of the user interface, the ‘patch bay’ module provides a modular system, which allows complex routing of data between modules. Input control envelopes can be selected from the various modules of the patch, and assigned to an output such as a DSP parameter on the saxophone, bass drum, or atomiser effects racks. This allows for many possible configurations of the patch, which produce different results during performances, whereby sounds of the saxophone can transform the bass drum and vice versa; or the various LFO and drunk modules can be used to change parameters independently of either instrument. This opens up many possibilities for organic, gradual transformations of sound, as effects morph between different settings which colour the sound, while spatial properties of reverb open and close on an on-going basis, thereby reflecting concepts of shifting perception in psychedelic experiences. Once effective patching configurations have been found, these can be saved using the ‘presets’ module.

Mixer & Audio Output Lastly, in the bottom-right corner of the user interface, the ‘mixer and audio output’ module provides a mixing desk, allowing levels to be balanced between the wet and dry signals from the saxophone, bass drum, and atomiser modules. An overall master output fader is provided,

3 Melting in the Mind’s Eye: Real-Time Performances

75

as are options for recording the output of the software in stereo or 5.1 multi-channel. At its core, the Bass Drum, Saxophone & Laptop application is a highly specialised multi-effects unit, providing various options for processing the sounds of live instrumental performances with DSP effects, while also triggering additional streams of percussive sound. In this way, the software continues to develop the electroacoustic approaches discussed earlier, adapting these for use in real-time performances. Using the patch, streams of percussive sound related to visual patterns of hallucination can be triggered by live instruments. Where before drones and sensory bass sounds were provided by the ‘drone machine’ module of the Atomizer Live Patch, these could now be performed using the bass drum and saxophone instruments. In particular, hanging the bass drum from a gong stand increases the acoustic resonance of the instrument, so that thunderous bass drones can be performed live; while the long notes performed on the saxophone can be extended with the delay, reverb, and feedback effects. The patching capabilities of the software allow various properties to be routed between instruments, so that the system seems to take on a life of its own. The laptop becomes an autonomous agent15 that initiates gradual shifting frequencies and spatial properties, in correspondence with experiences of shifting and morphing perception that may occur during psychedelic ASCs. Overall then, the software facilitates the psychedelic adaptation of live instrumental performances, and though conceived for bass drum and saxophone, it can potentially be used with one or two instruments of any type.16

Live at Keele University Having outlined the functionality of the Bass Drum, Saxophone & Laptop application, we can now discuss its specific use for live performance. Using a ‘bottom-up’ approach to composition and software design, Bass Drum, Saxophone & Laptop was designed through an iterative process of coding and performance.17 Once the software

76

J. Weinel

was complete, this shifted towards experimenting with different patch settings, so that performances could be based around these. In this way, the patch provides an organic, mutating sonic context within which psychedelic free improvisation can take place, where the software itself is inextricably linked with how the performers play their instruments, as they respond to the conditions of the patch and vice versa. The first live performance of Bass Drum, Saxophone & Laptop was carried out at the AutoPlay concert (Keele University, 29 September 2010). Since a recording of this concert is not available, in order to discuss the use of the software in practice, here I will refer to a recording of a practice jam session and two short demonstration extracts, which illustrate how the performance sounds when using different preset configurations. All examples were performed with myself on bass drum, and Sol Nte on saxophone.

Bass Drum, Saxophone & Laptop: 23 February 2010 Jam Session Media 3.4 Bass Drum, Saxophone & Laptop (23 February 2010, Session 1), instrumental performance with live electronics, 11 minutes 5 seconds The audio example (Media 3.4) provides a general illustration of the performance system in action, and is the best available recording from this period. In the opening of the recording we hear the bass drum, processed with a ring modulator sound, while the saxophone signal fluctuates in pitch via use of the Doppler effect. The ‘atomiser’ module is heard being triggered at 0.47 and 1:15, as the drum hits the target level, activating the scattered rhythmic sounds. For most of this section the drum is deliberately played beneath the trigger level, so activation of these sounds can be controlled, introducing the rhythmic sounds only when the performer intends. As before, these sounds are conceptualised in relation to visual patterns of hallucination, so as the performance level gradually increases, waves of rhythmic sounds engulf the auditory field, in correspondence with the ‘onset’ of a hallucination.

3 Melting in the Mind’s Eye: Real-Time Performances

77

From 1:26 to 2:35, it is possible to hear the ‘drum sustained playing envelope’ module being activated and used to control various parameters of the ‘sax effects rack’ module. As the dynamics of the drum performance increase in intensity, the ‘drum sustained playing envelope’ module comes into effect, increasing various effect parameters on the sax such as the amount of delay. The bass drum control envelopes are also shaping the saxophone reverb and Doppler speed in this section. As discussed earlier, the introduction of these effects can be understood in correspondence with ideas of shifting colour and spatial perception, giving the impression of hearing the saxophone as if in a warped or disorientated state of sensorial derangement. The composition builds dynamically, reaching various peaks in intensity, with the most significant one occurring from 9:00 to 10:00, where we hear various rhythmic entoptic sounds, as if in an intense phase of hallucination. Throughout the performance, the bass drum generates droning sounds, reflecting the concept of distortions to time perception. These drone sounds are produced through the combination of the resonant properties of the bass drum itself, and the tuning of the ring modulator patch on this particular preset (which does not change). At various points, the drum is played to provide repetitious hypnotic rhythms, building towards peaks and troughs of intensity in terms of loudness and tempo.

Bass Drum, Saxophone & Laptop: Extract 1 Media 3.5 Bass Drum, Saxophone & Laptop: Extract 1, instrumental performance with live electronics, 1 minute 0 seconds In this audio extract, we hear some of the different sounds that can be obtained with the bass drum, where distortion, ring modulator, and reverb effects are applied. Using this patch configuration, the ring modulator frequency of the bass drum is primarily controlled from envelopes generated with the saxophone. As this effect parameters changes, we hear the pitch of the bass drum rising and falling, in a striking manner slightly reminiscent of a didgeridoo. The complex interaction between the two

78

J. Weinel

instruments thereby produces fluctuations in pitch following organic contours.

Bass Drum, Saxophone, & Laptop: Extract 2 Media 3.6 Bass Drum, Saxophone & Laptop: Extract 2, instrumental performance with live electronics, 1 minute 0 seconds In this extract, we can hear how delay parameters of the saxophone are controlled using the internal ‘LFO bank’ and ‘drunk bank’ automation modules of the software. In this section, we can hear constantly shifting sound parameters, which are not dependent on what is being played on either instrument at that point in time. Although I consider the complex, organic interactions between the instruments and the patch as largely desirable characteristics, there are situations where these interactions become limited, as long sections of performance at high amplitude levels can ‘max out’ many of the modules, thereby reducing the amount of variation. The internal ‘LFO bank’ and ‘drunk bank’ automation modules can help to overcome this, since they are generated independently, producing constant variations, as heard in this extract. Taking Z’EV’s performance at Equinox festival as an inspiration, the Bass Drum, Saxophone & Laptop software develops the concept of psychedelic journeys in sound that was explored in the electroacoustic compositions, but here incorporates some of those ideas for a live instrumental performance, which is augmented by electronics. By running autonomously, the laptop becomes a third agent in the performance, creating a set of sonic conditions, around which the performers must construct an improvisation, where shifting DSP effects suggest perceptual distortions, while streams of rhythmic sound correspond with visual patterns of hallucination. ∗ ∗ ∗ Through the course of this chapter we have seen how the psychedelic approaches to electroacoustic composition discussed in the previous

3 Melting in the Mind’s Eye: Real-Time Performances

79

chapter were adapted in order to construct live performances. As Z’EV’s performance discussed in the opening of this chapter highlights, live performance can be exciting for both performers and audiences, offering something different from fixed-media compositions. With the aid of modern computers, real-time electroacoustic performances are pushing the art form into new and exciting places. For psychedelic music, the implications are significant, because in the past, producers have often had to rely on studio production techniques to warp and twist the sounds in interesting, disorientating ways. Where before these processes could not necessarily be used live, now there is much more that can be achieved with real-time DSP on the average laptop. With this comes not only the potential to recreate the psychedelic techniques of the recording studio, but also to go beyond them. In the future, I hope to see some of the tentative explorations discussed here expanded by others into psychedelic sonic territories as yet uncharted.

Notes 1. Z’EV kindly provided me with constructive feedback on the composition Bass Drum, Saxophone, and Laptop discussed in this chapter, and shared some of his writings on music and animism. Animism is the belief often held in shamanic societies, that forces and entities within nature have a soul (for example, see Eliade 1964; Vitebsky 1995). Z’EV’s writings discussed his practices making music and constructing percussive instruments using various ritualistic, animist practices. 2. As discussed by Tanner (2016), vaporwave is an Internet-based music genre and visual art style originating in the early 2010s, which remixes sound sources such as 1980s pop, elevator music and late night TV commercials, looping and slowing does these sounds ad infinitum. James Ferraro is often credited as one of the originators of the style; for example, see Kalev (2018). 3. The extent to which electronic music using pre-recorded sounds can be considered ‘live’ has been a subject of much debate; for example, see Sanden (2013). For our purposes here, it will be sufficient to acknowledge that ‘liveness’ in electronic music performances is often distinct from that of instrumental performances, but offers its own valid set of approaches. 4. For further visual representations and discussion of visual patterns of hallucination, see also Bressloff et al. (2001).

80

J. Weinel

5. Video evidence of these prototypes is available online: see Weinel (2009a, b). 6. The term ‘tick’ here is borrowed from music trackers to describe the smallest available unit of time in a sequence. A music tracker is a type of digital audio workstation, which reads music sequences from vertically scrolling lists of data (often using hexadecimal values), where each line is one ‘tick’. 7. DJ mixers, particularly those used by scratch DJs and turntablists often provide selectable crossfader angles, allowing the user to chose between various linear, high-shelf, or low-shelf amplitude envelopes. 8. In the 2000s I mixed DJ sets incorporating various styles of underground techno, performing sets at student nights; underground sound system ‘free parties’; and other events such as the Sonic Arts Network: Expo Plymouth in 2007. 9. Use of the term ‘version’ here, and later in this chapter, is a reference to the practice of making ‘dub versions’ (alternative mixes, often with elements subtracted), as used in dub reggae music. For a further discussion see Bradley (2001, p. 312) and Veal (2007). 10. Though it is possible to record the actual concert performances themselves with the software, I have usually avoided this to preserve the stability of the software, since recording places more demands on the resources of the laptop. 11. I find the pitched saxophone sounds here recall the opening of Geinoh Yamashirogumi’s ‘Dolls’ Polyphony’ (1993), from the Akira (Ôtomo 1988) movie soundtrack. In the movie sequence of Akira, pitched vocal sounds are heard with the onset of a delirious hallucination. In this sequence, the bizarre saxophone sounds communicate a sense of irrational madness. 12. The ‘floating points’ sound system consists of 15 speakers suspended from the ceiling of the concert venue. For more information see: ISSUE Project Room (2020). 13. The approach to improvisation discussed in this section can be associated with Derek Bailey’s (1993) ideas regarding ‘free improvisation’. 14. Autonomous agents in music are also notably explored in the work of Arne Eigenfeldt (e.g. 2017), who also performed at the Analogous Interactions concert at the ISSUE Project Room in New York (31 May 2010). 15. For example, to provide portable demonstrations to my students, I have often demonstrated the software with a melodica. 16. For an alternative discussion of processes where coding is interwoven with creative practices in digital painting, see Batchelor (2019).

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

At first you see a flickering noise pattern, which crackles with brilliant shades of red and green. Shapes begin to form and emerge from the chaos with increasing rapidity. First cell-like structures with nuclei that multiply, coalesce, and melt away before your eyes; then stroboscopic rectangles that engulf the visual field and dilating circular patterns. Nothing is static, everything is shifting and evolving, as colours and shapes distort, fade into each other, and fall apart. What you hear is jazz music. Hot saxophones flutter through the mix, dripping in sweat as punchy drum sounds bounce along amicably with a throbbing bass and percussive high-hats that mark out a quickening groove. Everything is playing and rolling together in sync. A cluster of notes explodes before you and suddenly there are circles everywhere; now angular rectangles Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/978-981-16-4055-1_4. Where the icon is shown in the chapter, the reader should refer to the supporting media files.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 J. Weinel, Explosions in the Mind, Palgrave Studies in Sound, https://doi.org/10.1007/978-981-16-4055-1_4

81

82

J. Weinel

that alternate in colour wildly as a drum break clatters through. A brilliant ball in burnt orange floats across a horizon, rising and setting like the sun, over and over again. It’s almost as though time itself has accelerated, but simultaneously there is a profound sense of eternity, as you are suddenly gripped by the gravity of the earth’s endless orbits and position as little more than a fleck of dust in the brief sneeze that is the universe. This is a description of Harry Smith’s experimental films (Early Abstractions, 1939–1957), though it might otherwise relate an experience of listening to some good jazz music on hallucinogens, where closing the eyes triggers wave after wave of intense synaesthetic visual hallucinations—forms emerging and dissipating from patterns of noise in the mind’s eye, in perfect synchronisation with sound. In the mid-twentiethcentury Smith painstakingly created a series of experimental films using a technique called ‘direct animation’, whereby the artist paints, stencils, and draws directly on to celluloid film in order to construct animations. The process is labour-intensive because the artist must render every single frame of animation on the tiny surface area of 16 mm film stock. The results create an effect that has a marked visual rawness. Shapes and colours can be designed to provide continuous animations, but the size at which the artist must work makes precision difficult, so the movement invariably ends up being abrupt and frenetic. Dust and paint particles are magnified during projection, so that there is always a background layer of noise that recalls the visual static (or ‘visual snow’) that some people may see when looking up at the night sky (Schankin et al. 2014). The manner in which Smith’s shapes emerge from these noise patterns and move across the screen like organic cells or cosmic bodies lends the films a definite psychedelic quality—they actually resemble the impressions one might see during an episode of psychedelic hallucination. Harry Smith’s work was partly inspired by the ‘visual music’ films of Oskar Fischinger. As discussed in the Introduction, ‘visual music’ is a term that was coined in the early twentieth century to describe paintings such as those by Wassily Kandsinky, in which visual forms were designed in correspondence with music (Brougher and Mattis 2005). Oskar Fischinger was among those who developed this form through films such as An Optical Poem (1938), where abstract visual shapes are

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

83

animated and move to music. Smith created his own visual music paintings and films, and his Early Abstractions (1939–1957) series clearly owes a debt to Fischinger’s work. Yet as discussed in the film American Magus (Igliori 2002), Smith’s work also drew inspiration from other sources such as the occult, and American folk traditions.1 In discussing his films, he interestingly claims they were inspired by experiences of sleep deprivation, intoxicated hallucinations (Sitney 1979, p. 233), and a Dizzy Gillespie concert, where he ‘had gone… very high, and… experienced all kinds of colored flashes’ in response to music (Sitney 1965, p. 270). His films can therefore be understood as psychedelic visualisations of sound, and were projected with jazz music2 as a soundtrack. The idea of creating films that represent the synaesthetic visuals that one might see during a hallucination is the main focus of this chapter. My work in this area falls under the category of ‘audio-visual composition’, a branch of electroacoustic composition closely related to visual music, where sonic artworks are designed with corresponding images or video elements.3 In what follows, I will explore strategies for composing fixed-media audio-visual works based on altered states of consciousness (ASCs) and psychedelic hallucinations. First, I discuss the Atomizer Visual, a Max/MSP/Jitter4 application designed for generating stroboscopic visual materials. I will describe how this was used to create Tiny Jungle (2010), in which electronic music is used in combination with hand-drawn materials and computer graphics to paint a psychedelic journey in sound and image. Following this, I will examine a series of three audio-visual compositions that were inspired by a trip to Mexico: Mezcal Animations (2013), Cenote Zaci (2014), and Cenote Sagrado (2014). Using direct animation, these compositions are visual music works with electroacoustic sound, which were also developed in relation to concepts of hallucination. These pieces were composed using various combinations of analogue and digital materials, including some elements made using Processing, a programming language for visual design. Through the discussion of these fixed-media audio-visual works, this chapter will explore possible methodologies for composing psychedelic sounds and visualisations.

84

J. Weinel

Tropical Hallucinations Extending the approaches developed through my electroacoustic compositions and transferring them into the audio-visual domain, Tiny Jungle was based on the idea of a psychedelic journey realised in sound and image. This audio-visual composition was performed at NoiseFloor festival (Staffordshire University, 20–22 September 2011), where the sound was spatialised via live diffusion on a multi-channel system. Sonically, the piece explores a similar ‘adaptive’ approach as was described in Chapter 2, where electroacoustic music and electronic dance music forms are modified to incorporate approaches based on features of ASCs. Visually, the work uses hand-painted materials in order to elicit organic and atomic forms and a journey through a forest-like environment, thereby representing visual hallucinations according to the concepts explored in the previous chapters. As with the electroacoustic compositions and realtime performances discussed earlier, the artistic process is interwoven with coding, and in this case some materials were designed using a bespoke software application: the Atomizer Visual.

Software: Atomizer Visual Media 4.1 Atomizer Visual, Max/MSP/Jitter software The Atomizer Visual (Media 4.1) is a Max/MSP/Jitter application that was used to design stroboscopic visual materials for Tiny Jungle in realtime. These materials were conceived in correspondence with the concept of sensory overload and trance states, while also drawing inspiration from VJ performances and the rapidly flickering animations of rave music visuals such as Humanoid’s Stakker Humanoid (1988) music video. By generating stroboscopic visual materials, the software provides a means to stimulate the viewer and create disorienting visual rhythms. The user interface for the Atomizer Visual is shown in Fig. 4.1. The software has four video channels, which can be controlled independently with the four sets of channel controls seen at the top of the user interface, or together using the ‘all channels’ panel. Each channel can receive a short video clip or still image as an input, which can be loaded via

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

85

Fig. 4.1 User interface and example visual noise output of the Atomizer Visual Max/MSP/Jitter application

the ‘drop file’ area of the patch. The four channels are mixed together to form a single video output. Each channel triggers a strobe effect by modifying the opacity of the source materials, which flicker rapidly at a tempo set by using the metronome controls. Images fade out when they disappear, according to specified time durations in milliseconds, which can be controlled using the ‘decay’ values. The blending of channels is defined using various mathematical expressions (e.g. addition, subtraction, multiplication, and division), which are selectable using the ‘exp’ menus beneath the channels. Depending on the source materials and combination settings, ‘visual noise’ effects can be produced (see Fig. 4.1). The video output of the software can be recorded using the controls at the bottom-right of the patch, and instructions are provided on the main user interface panel. The Atomizer Visual is a relatively simple Max/MSP/Jitter patch that was conceived as a tool for generating stroboscopic visual materials in real-time. While working in real-time comes with some constraints with regard to image quality,5 it also provides a more improvisational, performative workflow, and allows organic spontaneity to be incorporated in the design of the work.

86

J. Weinel

Tiny Jungle Media 4.2 Tiny Jungle, audio-visual composition, 7 minutes 10 seconds Tiny Jungle (Media 4.2) is based on the idea of a psychedelic vision of a flight through an imaginary forest. Figure 4.2 shows a graphic score of the piece. Musically the composition adapts electroacoustic music and jungle/drum & bass, interpreting the jungle visual associations of this genre to form a synaesthetic journey through a tropical environment. As indicated by the score, this journey takes us through visions of rocky landscapes, entopic phenomena, and a flight through a miniature forest to encounter various bizarre or mysterious forms. Considering first the sound design, Tiny Jungle is an electroacoustic composition that incorporates aspects of jungle/drum & bass, drawing in particular on the late 1990s output of labels such as Reinforced Records and Metalheadz.6 The piece utilises rhythmic patterns and structures that are typical of these genres, as heard in the long introduction section from 0:10 to 1:30; the syncopated rhythmic patterns that follow from 1:33 to 2:05; and bass sounds with pitch bend effects, which are first introduced at 2:26.

Fig. 4.2 Graphic score indicating the structure of Tiny Jungle

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

87

While these forms are borrowed from jungle/drum & bass, Tiny Jungle adapts their presentation based on the ASC concepts discussed in Chapter 2. For example, the long introduction section from 0:10 to 1:30 would usually be designed using synthesiser pads, but in this case it was realised using granular synthesis techniques that correspond with distortions to time perception. Along similar lines, the rhythms heard from 1:33 to 2:05 were constructed from organic source materials such as wood sounds, rather than using the sampled breakbeat7 loops that would be more common in jungle/drum & bass. The composition also uses sounds reminiscent of birdcalls, which emphasise the tropical theme. These birdcall sounds can be heard at 0:02–0:10, 1:21–1:31, and throughout. They were made by experimenting with pitch transformations (they are not actually recordings of birds), and were arranged into rhythm patterns, as heard from 2:00 to 2:20, where they fade into a wave of entoptic rhythmic sounds based on visual patterns of hallucination. These entopic sounds are also heard later in the piece at 4:45–5:00. Structurally the music of Tiny Jungle is based on concepts of energy levels, which relate to Fischer’s (1971) ‘cartography of ecstatic and meditative states’. As noted in Chapter 1 (p.1), Fischer describes ‘ergotropic’ states of energy expenditure and ‘trophotropic’ states of energy conservation, which in Rouget’s (1985) discussion correspond with states of trance and meditation, respectively. As indicated in Fig. 4.2, these concepts inform two distinct phases in the composition. The section from 1:33 to 3:47 uses fast, syncopated rhythms, which relate to ergotropic states of trance, while the section from 3:48 to 6:10 slows the tempo of the music to a throbbing bass groove, reflecting a trophotropic state of ‘bass meditation’. In this way the structure corresponds with the concept of an ASC that moves between ergotropic and trophotropic states, and this idea is also developed through the visual materials of the piece. We may now turn to consider the visual design of Tiny Jungle. The opening of the piece ‘onsets’ with a ‘landscape flight’ over rocky hills (0:00–1:00). During this section, the sky is depicted using digitally transformed footage of ink droplets falling into water, creating organic turbulence patterns. In the sky we see orbiting spheres reminiscent of Harry Smith’s Early Abstractions (1939–1957) suggesting planetary

88

J. Weinel

motion (0:25). During this section there are also faint traces of visual dot patterns suggestive of ‘entopic phenomena’, which were made by digitally scanning hand-drawn still images, and then processing them with the Atomizer Visual software. At 1:00 these effects begin to intensify, until 1:34 where a ‘breakthrough’ occurs and we move into a phase that is characterised by many dots representing visual patterns of hallucination. These were similarly created by processing hand-drawn images with the Atomizer Visual and combining multiple layers to form composites. This section includes mysterious hallucinatory forms such as an animated head, which appears from 1:33 to 1:55; three phallic or mushroom-like ‘weird sticks’, which emerge at 2:00; lizard-like creatures which crawl across the screen from 2:02; and flickering green triangle patterns which scroll vertically across the screen from 2:14. At 2:25 another transition occurs and a gyrating 3D atom appears. At 2:36, we fly through tunnels of spheres suggestive of Klüver’s (1971) form constants, this time rendered in 3D. We then see more atomic patterns (2:51) and various flickering visual strobe effects created with the Atomizer Visual, including fleeting impressions of a landscape with a forest in the distance (3:04). From 3:08 to 3:30, we then move into the ‘forest flight’ section, as various trees and a disorienting mass of branches whizz past us. Further hallucinatory patterns and forms are then seen, until a ‘breakthrough’ occurs at 3:47. With a clicking sound, at 3:47 we pop out of the previous wave of hallucination and the pace of the music changes, signalling a brief ‘plateau’ and the beginning of the ‘trophotropic’ section of the composition. In this section we see more ‘bizarre/mysterious forms’ suggestive of hallucinations. First, there is a strange rotating shape covered in spikes, which has four bone-like spokes. Then at 4:04 we move inside one of the spokes, entering an interior space with melting oily columns, before moving out again at 4:36. At 4:54 we drop through another of the spokes, where we see yet more visual patterns of hallucination (made with the Atomizer Visual) and 3D spheres (Fig. 4.3). A throbbing, trance-like bass sound is heard as a capsule floats past (5:12); discs pulse around a sun (5:21); and the orbiting spheres we saw at the beginning move across the screen (5:36).

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

89

Fig. 4.3 Still image from Tiny Jungle representing visual patterns of hallucination

Finally from 6:11, we once again find ourselves in a ‘landscape flight’ above the reddish rocks that we saw at the beginning of the piece. The sky is now filled with entoptic visual hallucinations, while orbiting spheres fly from left to right, and a luminous organism is born above us (6:43). The vision then melts away as the trip ‘terminates’ (6:48–7:10). Realised with the aid of the Atomizer Visual software, Tiny Jungle combines hand-drawn materials, computer graphics, and an organic electroacoustic soundtrack with jungle/drum & bass elements. These materials are structurally organised to provide an audio-visual experience analogous to a mystical flight through a hallucinatory forest. The theme of this composition relates to descriptions of visionary experiences that include extreme macro and micro perspectives. For example, one might have an experience of flight above a forest (macro perspective), or visualise cells or atoms (micro perspective).8 Extending the idea of psychedelic journeys in sound, Tiny Jungle provides a hallucinatory audio-visual journey that traverses these macro and micro perspectives.

90

J. Weinel

Synaesthetic Underworlds Continuing my explorations in the use of hand-drawn materials, I subsequently began making my own direct animation films in order to access the unique organic visual qualities that this technique provides. Direct animation was used not only by Harry Smith, but also various other visual music filmmakers. An early work of this type was Len Lye’s A Colour Box (1935), which was commissioned by the General Post Office Film Unit (Russet and Starr 1976, p. 65). On this film Lye used Dufaycolor (a film dying process) to apply various coloured patterns on to sections of film, which were then matched to ‘La Belle Creole’ by Don Baretto and his Cuban Orchestra (Horrocks 2001, p. 137), a dance piece popular in Paris at that time. Other films by Lye such as Kaleidoscope (1935), Trade Tattoo (1937), and Musical Poster #1 (1940) also utilise camera-less animation techniques. In a later work: Free Radicals (1958, 1979), rather than dying the film Lye scratched images on to black 16 mm leader film, producing abstract moving figures that dance to tribal drums in a manner perhaps suggestive of trance rituals (Lux 2020). Free Radicals was screened at the 1958 Brussels World’s Fair (Expo 58) where it won second prize in the International Experimental Film Competition (Len Lye Foundation 2020). In the 1940s, Norman McLaren also utilised direct animation to compose films such as Dots (1940) and Begone Dull Care (McLaren and Lambart 1949) (Russet and Starr 1976, pp. 116–128). These films both provide striking, close synchronisation between sound and image, so that visual animations express the timing of music, while different patterns, shapes, and colours also correspond with pitch and timbre. In the case of Dots, McLaren achieves these results in audio by drawing on to an optical sound strip (Peters 1951), thereby translating visual images directly into sound. Stan Brakhage is also well known for his direct animation work. His film Mothlight (1963) was made by attaching moths and various other debris gathered from a moth trap to 16 mm film stock. Though this piece is silent, Brakhage considered the piece to have various ‘musical’ aspects to its composition (Ganguly 1994), and thus it is an example of visual music where sound and musical qualities inform visual design, but are

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

91

not expressed acoustically. This piece exploits two significant functions of direct animation. First, the magnification of microscopic materials, since the moth wings, grass, and other debris are scaled-up when the film is projected; and second, the discontinuous progression in time that arises from successive frames containing completely different visual materials, which lack the smooth flow of motion that is possible with other techniques such as cel animation or computer graphics. In other works such as Persian Series (1999) Brakhage used hand-painted direct animation, yielding fantastic colourful results. As evidenced by this series, an appealing aspect of hand-painted direct animation is the way in which the magnification of paint amplifies the imperfections and textural properties of the materials. Every bubble, crack, or dust particle in the paint becomes visible. These imperfections are seen briefly while each frame plays for approximately 20–40 milliseconds (depending on the rate of projection). Similar features may be visible in the next frame, but will usually be seen in different places on the frame, so that there may be a general continuity in terms of colours or types of materials, but there is also a seemingly random, discontinuous placement of these imperfections. Through my own explorations of direct animation on 8 mm film, I have found that these magnified, discontinuous imperfections can produce moving patterns of visual noise. If one stares at blank film stock covered in dust particles, after a while these noise patterns begin to play tricks on the mind, which seeks to organise the visual noise into recognisable shapes, patterns, or faces. In psychology this effect is called pareidolia, and it is known to occur when looking at visual noise such as that produced by TV static (Barik et al. 2019). This might be an interesting technique in itself for artists interested in simulating psychedelic visual impressions to explore, however in my own work I am more interested in representing hallucinations, and so I compose visual forms that emerge from the noise. In this section I will discuss three audio-visual compositions that were created using direct animation. Unlike the analogue visual music films discussed above, my work combines direct animation on 8 mm film with digital methods of editing and compositing, allowing other types of visual materials to be introduced. Therefore, the final artefacts are

92

J. Weinel

not analogue celluloid films (though 8 mm films of the direct animation materials do exist), but rather digital video files. The three compositions: Mezcal Animations, Cenote Zaci, and Cenote Sagrado were all inspired by a trip to Mexico in 2012. Mezcal Animations imagines a hallucinatory state of alcohol-induced intoxication inspired by Oaxaca’s Feria Internacional del Mezcal (International Mezcal Festival)9 ; Cenote Zaci is based on my experiences swimming in a cenote (natural sinkhole) in Valladolid; while Cenote Sagrado (‘sacred well’ or ‘well of sacrifice’), is based on the cenote of the same name at the Chichen Itza archaeological site, which was a site of ritual Mayan sacrifice and a gateway to the underworld. Each piece interprets the given concept as a psychedelic audio-visual journey, building on the approaches discussed so far. These audio-visual compositions have been widely screened at various concerts and festivals including Seeing Sound (Bath Spa University, 23 November 2013; 10 April 2016); Sweet Thunder Festival (San Francisco, 27 April 2014), the joint International Computer Music Conference/Sound & Music Computing (University of Athens, 14–20 September 2014); the International Festival of Artistic Innovation (Leeds College of Music, 12– 13 March 2015); the High-Frequency concert at the The Yard theatre in Hackney (London, 18 January 2016); the Carbon Meets Silicon group exhibitions (Wrexham Glyndwr ˆ University, 8–11 September 2015; 11 September–8 December 2017); Daegu International Computer Music Festival (Daegu, 2018, 2019); and others.

Mezcal Animations Media 4.3 Mezcal Animations, audio-visual composition, 4 minutes 0 seconds Oaxaca’s Feria Internacional del Mezcal (International Mezcal Festival) is a celebration of mezcal, the distilled alcoholic beverage made from agave, which is traditionally made in Oaxaca, Mexico. In the city centre rows of stalls, many by family-run local businesses, provide samples and sell bottles of mezcal. Every kind of mezcal is covered, ranging from clear varieties to the flavoured ‘crema de mezcal’, as well as the ‘sal de gusano’ (worm salt, a spicy salt made from ground mezcal worms)

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

93

that complements the drinking of mezcal. The festival also includes various live music and performances. Taking the festival and the mezcal I purchased there as a point of creative departure, my audio-visual composition Mezcal Animations (Media 4.3) is a synaesthetic piece of visual music loosely based on the idea of a mezcal-induced hallucination. In relation to this theme, I also drew inspiration from the novel Under the Volcano by Malcolm Lowry (1947), which includes descriptions of mezcal, alcoholic delirium, and Oaxaca. Mezcal Animations was created using direct animation on 8 mm film stock, which was first bleached in a bucket in order to remove the emulsion.10 Once bleached, washed, and dried, the clear 8 mm film provides a blank canvas upon which paint or other materials can be applied in order to colour the film. For Mezcal Animations, I used various combinations of inks, marker pen, nail polish, and oil paint, in a variety of colours. As shown in Fig. 4.4, inks can be combined so that they flow into each other (a), brushes or other implements such as wooden sticks can be used to scratch patterns into paint (b), or designs may be drawn in pen (c), Hairs, particles or other debris may be attached to the film11 (d), and bubbles may form in the paint (e), adding visual noise. In some cases there may be sections of the film where parts of the emulsion are not fully removed by the bleach, leaving traces of the original footage (f ), Cracks in the paint (g), or newspaper (h), may also provide interesting textures. Once the paints and other materials have fully dried, the film can be projected. For this project I used a Eumig P8 standard 8 mm projector. Projecting direct animation films can be a laborious process since coatings on the film reduce the ease with which it will move through the projector mechanism. As a result, it will often get stuck, at which point the projector must be quickly switched off to prevent the film from melting under the heat of the bulb or causing a fire hazard. The process of projecting direct animation films changes them with each playback, as the heat from the bulb and movement of the film through the projector causes cracks in the paint, and the reels attract more dust particles. The process as a whole embeds a level of instability, amplifying the unpredictable, organic nature of the visual materials and generating various forms of visual noise. Like jazz improvisation, films made in this manner

94

J. Weinel

Fig. 4.4 Various frames of animation showing visual artefacts and textures produced by direct animation on 8 mm film

are unique in each performance and run the risk of failure at any moment. Even the speed of projection is unpredictable, as the film moves more slowly or quickly through the projector at different points in time. The results produced by projecting these reels can be incredibly exciting. Rapid bursts of colour, visual noise, and textures flash upon the screen

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

95

in quick succession, producing intense visual projections that acquire a luminous, hallucinatory quality. Once the 8 mm direct animation reels for Mezcal Animations had been completed and tested, they were then digitised. Digitisation allows the visual materials to be captured, after which an electronic soundtrack can be added, and the visuals can be subjected to further digital manipulation using modern video editing and compositing packages. The digitised version of the film loses some of the intense immediacy that the original projection reels have, but this is outweighed by the gains in stability and practical utility that a digital copy provides. While there are various solutions for digitising direct animation films, in this case a digital single lens reflex (DSLR) camera was used to make a video recording of the film as it was projected. Though higher quality results could be produced with more expensive equipment, this provided a simple, convenient, and inexpensive solution.12 A DSLR camera with a prime lens was used to record 1080p video in a dark attic space, with the camera located on a stand near the projector, slightly below the arc of projection. For Mezcal Animations the camera viewing angle crops the video to a 16:9 aspect ratio at the point of filming, though for other films I have often left a border to preserve the original aspect ratio of the film. The soundtrack for Mezcal Animations extends some of the compositional principles that were discussed in Chapter 2. As heard from 0:22 onwards, rotating streams of rhythmic sound were created, which were made by tapping various objects such as empty beer cans and a mezcal bottle procured from the aforementioned festival in Oaxaca. The streams of percussive noise made with these objects were captured with a condenser microphone, processed with an analogue delay, and then spatialised using the PANcho beta software by Rajmil Fischman. Figure 4.5 shows the animated spatialisation path that was designed using PANcho, which follows a spiral figure similar to one of Klüver’s (1971) form constants. The rhythmic rattling sounds correspond with rotating visual patterns of hallucination, and lend the piece a ritualistic quality that can be associated with the percussive music used in trance rituals to induce states of sensory overload. The droning sounds first heard at 0:44 were also made using transformations of electric guitar sounds, while the rhythmic bass pulses that are particularly prominent

96

J. Weinel

Fig. 4.5 Spiral spatialisation patterns produced with PANcho

around 2:34 were made using synthetic bass drum sounds processed with a guitar distortion pedal. As heard at the beginning (0:15), the visual noise arising from dust particles on the film is also matched with sounds of surface noise recorded from the run-in groove of a vinyl record. The audio-visual composition of Mezcal Animations organises the sonic materials in relation to the digitised direct animation footage. The video has three distinct movements, as follows: {I. Mezcal Reposado / Pensamiento; II. Mezcal Tobala / El Golpe; III. Sal de Gusano}. In the first movement (0:13–2:30) we hear the gradual onset of percussive patterns and drones, while corresponding visual textures and patterns unfold. The second movement (2:30–3:44) increases the intensity of sounds and visuals, and here various figurative images emerge from the visual noise, including mysterious symbols, geometric shapes, an eye, and a skull (3:14, Fig. 4.6). The final movement ‘Sal de Gusano’ (3:44–3:53), features short, shard-like audio-visual forms that relate to the sharp, spicy taste of worm salt, which is often consumed with a piece of citrus fruit after drinking mezcal.

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

97

Fig. 4.6 A skull figure in Mezcal Animations

Through the combination of digitised direct animation and electroacoustic music, Mezcal Animations places visual noise, colours, and textures in relation to sounds, forming an abstract synaesthetic whole. This collective whole is related to the concept of an intoxicated mezcal delirium, where sounds and images suggest hallucinations, reflecting the intense tastes and smells of mezcal, and more broadly, the hot, humid climate and pallet of burnt oranges, browns, and tropical shades of Oaxaca as a city.

Cenote Zaci Media 4.4 Cenote Zaci, audio-visual composition, 4 minutes 5 seconds If you take a walk on a summer’s afternoon down the dusty streets and baking hot pavements of Valladolid, Mexico, you may find Cenote Zaci slightly beyond the town centre—Valladolid’s answer to an outdoor swimming bath. For thirty pesos, you can purchase a paper ticket and find some relief from the heat in cool waters. As you punch a hole in the ticket and make your way down towards the swimming area, the faint

98

J. Weinel

echoes of splashing and excited children grow louder, until you turn a corner and see a vast hole in the rock filled with deep turquoise waters. A path around the outside meanders its way down towards the basin, the moist stone glistening in a deep brownish grey colour. Queues of children line up to jump in from a great height, dropping from the ledge one after the other, pausing only occasionally when a more tentative child has second thoughts. The line momentarily stalls, until finally, egged on by his friends, he too leaps into the deep cool waters below, with a loud splash that sends water cascading in all directions amidst cries of jubilation. For everyone else, dipping into the water happens at the bottom of the path, where one can slip gently from the edge of the rocks into the refreshing pool that is shared with the ‘lub’ (eyeless, charcoal-coloured fish). Taking this cenote as a source of inspiration, Cenote Zaci (Media 4.4) is an audio-visual composition based on the idea of an aquatic dream or hallucination of the cenote. Visually, the piece was made using a similar method of direct animation as that described for Mezcal Animations, where 8 mm film was first bleached and then painted with various inks. Rather than the orange tones of the former film, this piece uses predominantly green and turquoise shades, reflecting the deep waters of the cenote. From 0:27 and throughout the piece various fish are seen, rendered in a bright red colour. These were created by cutting out shapes of fish, which were then photographed in various positions over a lightbox to make stop-motion animation clips. These animation clips were then superimposed over the digitised direct animation footage. This method preserves the organic, textural qualities of the stop-motion animation, while adding other moving shapes that would be difficult to draw directly on 8 mm film with the same level of control. Along similar lines, Cenote Zaci also uses computer-generated animations, which were programmed in Processing. As first seen at 0:29, waves of triangle patterns scroll across the screen, which are reminiscent of the scrolling triangles used in Tiny Jungle, and similarly reflect visual hallucinations. Elsewhere (e.g. 0:40), a hypnotic diamond tunnel effect is produced in a similar way, which once again relates to Klüver’s (1971) form constants. In Processing, the code used to draw these shapes utilises

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

99

an animated colour, which is generated by assigning oscillating sinewaves to the red, green, and blue values. Sine functions were also used to arrange the triangles into wave formations. This simple programming technique, where oscillating values are used to animate properties, is also used in many other projects discussed later in this book. The audio track for Cenote Zaci was made using a Novation Super Bass Station monophonic synthesiser, running through a guitar distortion pedal; a bamboo flute, processed with a delay effect; and noise sounds created with an empty mezcal bottle. As with Mezcal Animations, these mostly analogue sonic approaches seemed appropriate as a means to complement the predominantly analogue visual materials. Sonically the composition aims to provide a hypnotic wall of pulsing electronic noise, which gradually builds in intensity. Popping out of the greenish visual noise, groups of fish dart back and forth, while brightly pulsing geometry engulfs the screen, reflecting visual hallucinations (Fig. 4.7). From 0:27 to 2:08 the composition builds in audio-visual intensity, by increasing the density of these various hallucinatory images, while also increasing filter cut-off frequencies on the synthesisers. At 2:08 there is then a brief pause in the composition, following which the final wave commences at 2:25, building intensity through rapid impressions of imagery and filters

Fig. 4.7 Waves of triangles and fish in Cenote Zaci

100

J. Weinel

that rise and fall in pitch, as formations of fish circle and dive on and off screen. Lastly at 3:56, the piece ends abruptly with a sweeping noise. In summary, Cenote Zaci is a direct animation composition that imagines a dream or hallucination of a cenote inhabited by various fish. This idea provides a point of creative departure leading to an abstract audio-visual composition that combines hand-painted materials, with colours characteristic of the cenote; stop-motion animations of fish; oscillating geometric patterns; and a pulsing electronic noise soundtrack that provides peaks and troughs in sonic intensity, lending the piece its structure.

Cenote Sagrado Media 4.5 Cenote Sagrado, audio-visual composition, 5 minutes 37 seconds The final work in this series of Mexico-inspired audio-visual compositions is entitled Cenote Sagrado (Media 4.5), after the sacred cenote of the Chichen Itza archaeological site in the Yucatán. The Chichen Itza site is most famous for El Castillo (the Temple of Kukulcan), a stepped pyramid notable for its transfiguration during the spring and autumn equinoxes, when shadows from the sun create the illusion of a serpent crawling down the side of the pyramid. A popular theory suggests that the acoustic reflections of the pyramid produce a ‘chirped echo’ sound, which resembles that of the sacred quetzal bird (Lubman 1998). If correct, we may consider that the architectural construction of the pyramid creates intricate illusions of sound and light that would have had spiritual significance for the Mayans. Experts in modern illusory computer technologies such as augmented reality and mixed reality take note—the Mayans beat you to it hundreds of years earlier! While the image of El Castillo frequently appears in the visual culture associated with psychedelic music,13 for Cenote Sagrado I looked instead towards the nearby cenote of the same name for inspiration. It is thought that Cenote Sagrado was once a site of ritual sacrifice that the Mayans believed was a gateway to the underworld, and into which humans, gold, and other treasures were cast (Phillips 2007, p. 101).

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

101

Drawing inspiration from my visit to Chichen Itza, Cenote Sagrado is an audio-visual composition based on the idea of a hallucinatory vision of a Mayan underworld in the dark depths of the cenote. Technically this piece continues to explore combinations of hand-painted direct animation on 8 mm film, stop-motion, and electronic sounds. Digitised direct animation materials were created using watery greens, blues, and blood red. Stop-motion footage, made using hand-painted frames and cut-out shapes, was overlaid on top of these visual textures. These include a Mayan face (0:11); blinking eyes that move around the edge of the frame (1:09); waves of triangles (1:34); and several long sections that feature moving cubes, rectangles, triangles, bats, and skulls, which dance around the screen (2:39–3:47). These sections were conceived as visual hallucinations of shapes performing mystical conjuring tricks, with candles and skulls symbolising the dead souls of human sacrifices14 (Fig. 4.8). Sonically the piece once again uses hypnotic electronic rhythms to provide a distorted, noise-based soundtrack that reflects ‘ergotropic’ states of trance-like sensory overload. The soundtrack was composed in the music tracker Renoise, using various electronic instruments and effects chains to provide rhythmic pulsing, percussion, and a noise track, which were then remixed in real-time to match the video using the

Fig. 4.8 Skulls transfigure into candles in Cenote Sagrado

102

J. Weinel

CME Bitstream 3X MIDI controller. The soundtrack therefore explores spontaneous, improvisational approaches in the studio. Turning now to consider the structure of the piece, at the ‘onset’ of Cenote Sagrado an explosion of triangles in the mind’s eye gives way to a Mayan face, which emerges from the visual noise at 0:11 and sticks its tongue out in thirst for human blood, as throbbing waves of percussion increase in amplitude. At 0:45, a family of cubes playfully bounces back and forth, magically disappearing and reappearing in different places. With another explosion of triangles, at 1:08 we see blinking eyes circle the screen ominously, and a wall of sonic noise erupts at 1:18. From 1:25 onwards, various geometric shapes pop across the screen, performing conjuring tricks and transfigurations, while oscillating bass tones are heard in the lower frequency range. At 3:18 bats descend, and three skulls dance around the screen while the rhythms cease and the soundtrack emits a piercing clicking sound—this section is the ‘plateau’, representing encounters with dead souls. Various hallucinatory dances and metamorphoses take place in this section, as the skulls become the flames of candles. From 3:43, the pounding bass drum sounds are reintroduced, but these now form a more hypnotic organisation, drifting repetitiously onwards for increasing lengths of time. At 4:42 the Mayan face greets us once again, pointing its tongue out. Finally, at 5:18, the tension of the hypnotic drum variations is released, as the familiar drum pattern from earlier in the piece resumes before the psychedelic journey abruptly ‘terminates’. Cenote Sacrado concludes this series of works by taking us on an audiovisual trip to a Mayan underworld, which is constructed through means of direct animation, stop-motion animation, and a pulsing electronic noise soundtrack. While the music is improvised, compositionally the piece continues to approximate the {onset, plateau, termination} pattern of earlier works, utilising percussion in correspondence with trance-like states of sensory overload. From the audio-visual noise various hallucinatory forms emerge, suggesting occult rituals and mystical encounters with

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

103

the dead souls of the Mayan underworld, who rest in the dark waters at the bottom of Cenote Sagrado. The three compositions discussed in this section form a series of works that construct synaesthetic, hallucinatory impressions in homage to Mexico. These pieces continue to explore many of the compositional approaches discussed in Explosions in the Mind so far. For example, materials are designed based on features of hallucination, and these are organised to form psychedelic journeys. As with Tiny Jungle, the designs are related to visual patterns of hallucination, and some similar motifs are used such as waves of triangles, mysterious forms, and encounters with strange entities and faces. Most significantly of course, Mezcal Animations, Cenote Zaci, and Cenote Sagrado are united by the use of direct animation, which in combination with digital techniques, allows the production of organic visual noise from which the hallucinatory designs can emerge. ∗ ∗ ∗ In this chapter we have seen how audio-visual compositions can be designed based on concepts of hallucinations. The four works discussed in this chapter can be seen as representations of synaesthetic soundto-image hallucinations. While the electroacoustic compositions of the previous two chapters construct psychedelic journeys on a purely sonic basis, the addition of visuals provides the possibility to represent what one might see during a hallucination. Visual materials like these can potentially be produced with almost anything at the composer’s disposal, and here we have seen how they may be designed using hand-made art, and computers graphics, with the aid of simple creative coding applications made in Max/MSP/Jitter and Processing. In these compositions, the role of sound shifts towards establishing energy levels of varying intensity, thereby providing affective journeys that are informed by concepts of trance and sensory overload. What we hear is electronic music, adapted towards various notions of psychedelia and trance. But what we see is a psychedelic visual journey, carefully interwoven with sound. The end-result is that we can begin to imagine that these works

104

J. Weinel

resemble the type of synaesthetic imagery that one might see exploding in the mind’s eye, as a psychedelic visualisation of sound.

Notes 1. For more information on Harry Smith including his visual music artworks and involvement with folk music, see also Perchuk (2010). 2. Harry Smith’s films were screened with various soundtracks (Singh 2010), and seem to reflect mutable relationships with music. In this regard his films might perhaps be understood as exploiting what Chion (1994) describes as ‘synchresis’, the natural tendency of the brain to integrate sounds and cinematic images. 3. For a further discussion of connections between electroacoustic music and visual music, see Garro (2012). 4. Max/MSP/Jitter is a visual programming language for sound and audiovisual design. 5. The output resolution of the Atomizer Visual software is 320 × 240 pixels. This is very low by modern standards, but reflects the capabilities of realtime video processing using Max/MSP/Jitter on a laptop with average specification in 2010. 6. For indicative examples of late 1990s drum & bass, see the Metalheadz compilation Platinum Breakz (Various Artists 1996), Arcon 2’s self-titled album (1997), and Alpha Omega’s Journey to the 9th Level (1999). 7. Jungle/drum & bass music typically makes use of sampled drum breaks taken from funk tracks by artists such as James Brown, Lyn Collins, or The Winstons, whose track ‘Amen Brother’ (1969) is the source of the famous ‘amen’ drum break used on countless tracks (Harrison 2004). The tempo of these breakbeats is usually increased to 160 bpm or higher, and they are often sliced, rearranged, and treated with various digital effects in order to construct complex rhythmic syncopations. 8. For example, in Jeremy Narby’s (1999) The Cosmic Serpent: DNA and the Origins of Knowledge the author describes visualisations of DNA structures while under the influence of ayahuasca. 9. More information about Internacional del Mezcal (International Mezcal Festival) is available online (https://www.oaxaca-mio.com/fiestas/feriadelm ezcal.htm). For a colourful, descriptive account see also Lizotte (2014).

4 Tune in, Turn Up, and Trip Out: Audio-Visual Compositions

105

10. This was accomplished using household bleach, following appropriate safety precautions. It is also possible to use clear leader film, but this is less readily available and tends to be more expensive. 11. Debris may become attached to the film accidentally or may be deliberated applied. For Mezcal Animations, I sprinkled dried coffee granules over the film to increase the amount of particle noise. 12. The process of digitization could alternatively be undertaken using professional digitization equipment such as an optical printer, allowing each frame to be photographed individually. For an expanded discussion of the processes for digitizing direct animation films, see also Weinel and Cunningham (2014). 13. For example, images of Mayan pyramids can be found on the cover of the psychedelic rock band Gong’s album You (1974), or more recently on artwork for the Mexican psy-trance compilation Mayan Loop (Various Artists 2013). 14. For other examples of ghostly and occult themes in visual music films, see also Harry Smith’s Early Abstractions (1939–1957) or Mary Ellen Bute’s Spook Sport (Bute et al. 1940).

5 Sensorial Apparatus: Interactive Projects

Imagine a device that allows you to play, record, and share sensory experiences. Placing a cap of wires and sensors on your head, suddenly your visual and auditory field changes to the sights and sounds of a deep canyon that you once visited. However, this is not like looking at a photograph, or watching a video recording—rather, it is a digital hallucination that makes you feel as though you are actually there. You can look and see the deep turquoise waters around you; hear birdcalls above and water lapping onto the rocks; and sense a gentle breeze and the sun beating down on your skin. Not only are your senses engulfed, but you can also feel the emotions you once felt on that day, a sense of quiet ecstasy and calm. Or perhaps, instead of the canyon trip, you could download something you’ve never experienced before, like taking a walk on the moon, or diving a sunken wreck forty fathoms deep. This device Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/978-981-16-4055-1_5. Where the icon is shown in the chapter, the reader should refer to the supporting media files.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 J. Weinel, Explosions in the Mind, Palgrave Studies in Sound, https://doi.org/10.1007/978-981-16-4055-1_5

107

108

J. Weinel

would not merely replicate the patterns of light and sound that would enter the eye during these episodes, but would also be capable of reproducing subjective sensory experiences and emotions. So if you fancied something really exotic, how about a shamanic peyote trip on a desert cliff-top? You could watch the mountains melt into black and white chessboards before your eyes, while the night sky above reorganises into a vortex of stars from which animals rendered in luminescent, pin-point dots emerge and whisper the secrets of the universe. Devices like this are depicted in various works of science fiction from the cyberpunk genre. In the movie Brain Storm (Trumbull 1983), scientists have invented a brain-computer interface (BCI) that can play and record experiences, which they demonstrate with a first-person perspective recording of someone rushing down a water slide. Elsewhere, in William Gibson’s Neuromancer (1984), the ‘SimStim’ is a sensorial apparatus that records and broadcasts a person’s sensory input, much like a video recorder, but for experiences. A similar device, the SQUID, was also depicted in the movie Strange Days (Bigelow 1995). Consisting of a network of sensors that fits on the head, the SQUID allows the wearer to play and record sensory experiences and physical sensations stored on minidisc. In the film, the device is used to do ‘playback’: street slang for illicit, addictive, virtual thrills, which leave people strung out with their brains fried from sensory overload. In the story, an underground economy has sprung up around the SQUID, in which crimes are being committed to record rushes of adrenaline, which can then be sold to addicts looking for their next vicarious fix. While the SQUID is used to capture normal-waking consciousness, elsewhere in the anime film Paprika (Kon 2006), a BCI called the DC Mini allows dreams to be recorded and played back on a laptop computer, providing a different take on a similar idea. These cyberpunk works highlight the ethical risks that the speculative sensorial technologies could bring. Yet fictional sensory recorders like the SimStim or the SQUID are logical extensions of what was already becoming possible with video technologies in the late twentieth century. For McLuhan (1964), television and video were extensions of the human nervous system, which allowed us to tap into hallucinatory global networks of sensory experience. Fictional sensory recording

5 Sensorial Apparatus: Interactive Projects

109

devices simply extend these possibilities, imagining what would happen if they were enhanced, so that video is no longer viewed through the rectangular screen of a cathode-ray tube television, but completely engulfs the senses, allowing users to jack into different conscious experiences. Cyberpunk works like Neuromancer or Strange Days paint a worrying picture of the dangers that such technologies might bring, but in doing so they also reflect the inherent dangers of the video technologies we already have, which are being used by some for various illicit purposes, surveillance, and control. Bleak, near-future cyberpunk dystopias resonate with audiences because they amplify the negative, emerging traits of present-day techno-capitalist societies, and warn us of the dark futures we may be heading towards.1 Though cyberpunk warns us about the dangers of these sensorial devices, the possibility to play and record experiences remains a tantalising possibility, and one that could also bring a variety of societal benefits. For instance, imagine the mental health benefits that could be produced if one were instantaneously able to induce states of complete relaxation; the educational possibilities of actually seeing and hearing what goes on in an operating theatre; or the sheer entertainment value that could be provided by being front row, watching that seminal performance by a rock n’ roll band that have long since disbanded. If such a device could simulate altered states of consciousness (ASCs), this too could have many benefits, perhaps by delivering rich cultural experiences, or even by providing new ways to access carefully designed psychedelic therapies.2 Though the sensorial apparatus described here may be speculative, such technologies could be available in the not-so-distant future. While the virtual reality (VR) of the 1980s and 1990s was somewhat limited by cumbersome hardware and relatively basic computer graphics, more recent technologies such as the Oculus Rift offer significantly improved possibilities for designing immersive experiences. One needs only to try Valve’s Half Life: Alyx (2020) for an hour or so to see the impressive graphical capabilities of current immersive technologies. Through VR, we can begin to conceive of experiences that allow us to see or hear what another person sees from a first-person perspective.3 Of course, replicating similar patterns of light and sound that one might sense in a

110

J. Weinel

given situation is not the same as simulating subjective perceptual experiences. Clearly, far more sophisticated technologies would be needed to do that.4 However, it is not beyond the realm of possibility for us to conceive of ways in which loudspeakers and video displays could be used to deliver virtual environments in such a way that they begin to resemble some subjective aspects of sensory experience. ASCs actually provide an interesting basis for this, due to the way in which they manifest ‘internal’ visual or auditory hallucinations, which are distinct from ‘external’ sensory inputs. As explored over the last few chapters, there are ways in which hallucinations might be represented through sound and computer graphics. If these audio-visual experiences are then created using interactive systems such as those provided by video game engines, we can begin to make immersive experiences that simulate the sensory inputs that one might have during an ASC. The main purpose of this chapter is to explore how interactive audiovisual systems can be used to represent psychedelic experiences. First, we will examine Quake Delirium (2010), a software-based project, which adapts the video game Quake (id Software 1996), modifying various graphical and game parameters in order to represent a delirious or hallucinatory state. This project was an early investigation into the use of game engines for providing interactive experiences that represent ASCs, and was realised through various scripts and an original Max/MSP patch, which allowed properties of the game to be remixed in real-time, while also generating a corresponding soundtrack. In a later project, Psych Dome (2013), I explored some related ideas, this time by generating psychedelic sounds and animations in an immersive fulldome environment. In this case, the audio-visualisation produced by the system was generated with Max/MSP and Processing, and could also be linked to the brain activity of a person inside the dome wearing an electroencephalograph (EEG) headset. Lastly, this chapter will also discuss ASC Sim (2017), a project created in the Unity game engine, which explored several possible interactive systems for representing auditory hallucinations in virtual environments. In different ways, each of these proof-of-concept projects illuminates possible approaches for designing interactive audio-visual experiences that represent subjective perceptual experiences such as psychedelic hallucinations.

5 Sensorial Apparatus: Interactive Projects

111

Delirious Hellscapes In the 1990s, video games were rapidly advancing. In a short period of time, the pseudo 3D graphics of first-person shooter (FPS) games like id Software’s Doom (1993) were quickly replaced by the true 3D of Quake (1996). Graphics accelerators like the 3dfx Voodoo Graphics card became essential hardware for anyone using a PC to play video games. 3D hellscapes could be rendered in exquisite detail like never before. Slotting a Voodoo Graphics—or better yet, two Voodoo2 cards in an SLI (Scan-Line Interleave) configuration—would allow the performance of games to skyrocket, while also unlocking a variety of enhancements including higher resolutions and OpenGL effects such as texture smoothing, transparent water, fog, and coloured lighting. When the first Unreal (Epic MegaGames and Digital Extremes 1998) game hit the shelves, it harnessed many of these possibilities in exciting new ways. I distinctly remember playing this game for the first time. In the opening sequence, your character (prisoner 849) must make their way out of a crash-landed prison-spacecraft. On the way out, you crawl through an air duct, which is filled with smoke and bathed in green light. What struck me most was the way in which you could almost smell and taste the smoke in this scene. Somehow the graphical effects, together with the immersive audio in this sequence, seemed to activate the multimodality of your senses, invoking past experiences of smoke. For me, the taste was synthetic smoke of the kind produced by fog machines, which I had probably encountered before at school discos or playing Quasar (a form of laser-tag) in Bournemouth. This is the real power of video game engines, and a big part of what makes 3D games appealing. With just a few textures, polygons, sounds, and visual effects, designers can paint rich, immersive worlds that activate the senses. Our multimodal interpretation of these environments completes the picture, filling in the blanks as we make sense of them in relation to our past interactions and memories of the real world. The environments may be virtual, but in some ways, we can experience forms of presence and embodiment that make us feel as though we are really there. Today, titles like Half Life: Alyx extend these ideas in VR, providing uncanny hyper-realistic audio-visual environments that one can almost taste, touch, smell, and feel.

112

J. Weinel

In order to explore ways in which video game engines can be used to represent ASCs, in the late 2000s, I began experimenting with the video game Quake. Quake is an FPS game set in a hellish world overrun with demons and experimental teleporters known as ‘slipgates’. The player must fight their way through various gothic caves and dungeons to save humanity. When I began this project Quake was already an old game, however, for many years it remained one of the most readily customisable FPS games, which made it an appropriate choice for modification work. The idea of the Quake Delirium project was to create a psychedelic ‘adaptation’ or remix of Quake, taking the existing game and changing various aspects of it in accordance with features of ASCs. The concept of remixing a video game in this way was partly inspired by other existing projects at this time, such as Street Fighter (an improvisational ballet), which hacked the video feed of Street Fighter II (Capcom 1992), generating original audio using a Max/MSP/Jitter patch to create an audiovisual artwork (Niv et al. 2006); or q3apd, which used Quake III Arena (id Software 1999) to synthesise audio in Pure Data5 based on spatial data from the game (Pickles and Oliver 2003).6 Along similar lines, Quake Delirium used a Max/MSP/Jitter patch to provide a psychedelic modification of Quake, which places various graphical and game parameters in a state of flux, while also generating a corresponding soundtrack. As we shall see, the result is still Quake, but it is adapted towards a more hallucinatory, delirious aesthetic, and this was later extended with the addition of a biofeedback device.

Quake Delirium Media 5.1 Quake Delirium demonstration video, 6 minutes 0 seconds The Quake Delirium project was realised with a Max/MSP application (Quake Delirium), which provides facilities for adjusting various graphical and game parameters, and generates a corresponding soundtrack in real-time. Figure 5.1 shows the system design including the flow of data and audio signals in relation to the game software. The system

5 Sensorial Apparatus: Interactive Projects

Keyboard and mouse control input

MIDI controller MIDI control data

Game control input

Quake with Fitzquake mod and QuakeC scripts

113

Control data

Quake Delirium

Log file

Game audio

Audio output

Altered graphics

Display output

Fig. 5.1 Quake Delirium system diagram

is based around a version of Quake running the Fitzquake modification and custom QuakeC scripts.7 The game is controlled using the typical keyboard and mouse combination used by PC-based FPS games. A two-way flow of information is provided between the game and the Quake Delirium Max/MSP application. The game writes messages to a log file based on events (e.g. when the player enters a level, picks up a health box, or collects ammo), and this is read by the Max/MSP application, allowing the patch to react to events in the game. Conversely, the Max/MSP patch sends control data to the game via virtual keystrokes, which are mapped to ‘cvar’ bindings specified in a QuakeC script. These keystrokes allow the following graphical and game parameters to be adjusted in real-time: • • • •

FOV (field of vision) Drunk mode (causes the camera to sway drunkenly) Fog density and colour Game speed

114

J. Weinel

• Stereo vision (a red and blue stereoscopic effect) • Gamma • Red hue. Using these graphical and game parameters, the Quake Delirium Max/MSP application makes adjustments while the game is played, thereby warping colours, perspective, and other properties of the game. These constantly changing effects reflect the shifting perception that may occur during an ASC. The parameters can be remixed in real-time using a Korg NanoKontrol MIDI controller, with the patch providing facilities to play and record automation paths. This allows different preset automation paths to be designed, or for one person to remix the game live while another person plays it. The patch also plays back a prerecorded electroacoustic music soundtrack, which incorporates various digital signal processing (DSP) effects. These were controlled using the same automation paths that modify the graphical and game parameters, thereby linking sound and image, so that the sonic and visual effects can be increased or reduced in correspondence with each other. The graphics output from the game via the display device, while sound from both the game and the Max/MSP application is routed to the audio output device. Media 5.1 demonstrates the Quake Delirium project in action using a preset automation path and soundtrack, which are included in the patch. In the video we see The Necropolis level of Quake (Episode 1: Dimension of the Doomed) being played with the software enhancements. On the audio track, we hear the normal sound effects of the game, while entoptic sounds are gradually introduced on the electroacoustic soundtrack. From 0:17 onwards we begin to see the ‘onset’ of shifting graphical effects, as the red and blue stereo vision effect blurs the image; purple fog fades in and out of the scene; and droning sounds are heard. As the droning sounds increase in amplitude, various graphical effects warp the colours between shades of red, blue, and purple (0:30–1:10). As prominently seen from 1:24 to 2:06, the game begins to speed up and slow down at points, reflecting distortions to time perception. During this phase the FOV shifts, gradually changing the perspective, as seen at 2:40. Throughout the video, all changes to visual properties correspond with DSP effects such as filter, echo, and reverb, so that the representation of

5 Sensorial Apparatus: Interactive Projects

115

a delirious state morphs organically as it unfolds in time. From 3:42 to 5:20, the visual and auditory effects reach a ‘plateau’ of intensity and we see bright colours, blurred with the stereo vision effect, while the FOV perspective is warped; time passes slowly; the camera sways drunkenly; and various entoptic and droning sounds are heard (Fig. 5.2). Finally, the psychedelic effects of Quake Delirium subside, ‘terminating’ at 5:30. While the demonstration video shows only one game level, Quake Delirium can be experienced with any section of the game. The log file reader triggers the various hallucinatory effects when the player first enters the level. The software also has an optional feature that allows the hallucinatory effects of the game to be cancelled out each time the player collects a health box. When this is enabled, the player can stave off hallucinations by collecting health, or seek more intense states of delirium by avoiding them! While at points the psychedelic adaptations make the game more difficult by obscuring what is happening on screen, at other times reductions in game speed give the player an advantage by providing more time to react. Negotiating the hallucinatory effects thereby adds a novel ludic dimension to the game, and allows the player to sculpt their experience of the delirious hellscape to their liking.

Fig. 5.2 Quake Delirium in operation, with various graphical distortions representing a hallucinatory perceptual state

116

J. Weinel

Quake Delirium EEG Media 5.2 Quake Delirium EEG demonstration video, 2 minutes 36 seconds A subsequent adaptation of the Quake Delirium project, Quake Delirium EEG , modifies the experience by replacing some of the automation paths with control data from a NeuroSky Mindwave consumer-grade EEG headset (Media 5.2). In this version, the values for red hue, fog density, fog colour, and gamma are controlled by the interpreted values for ‘attention’, ‘meditation’, ‘delta’, and ‘theta’ that the device provides; while field of vision, drunk mode, stereo vision, and game speed are controlled by the usual automation paths.8 In this way, the project links shifting psychedelic audio-visual effects to the brain activity of the person playing the game. As suggested through a complementary experimental study (Weinel et al. 2015a), the player does not have any significant volitional control over these parameters, but does gain a sense of ‘passive’9 connection between the simulation and the biofeedback signals, which is conceptually appealing since it links brain activity to the psychedelic effects. In summary, Quake Delirium and the subsequent Quake Delirium EEG extension sought to adapt an existing video game (Quake) towards various concepts of ASCs. This was achieved by using automated properties that allowed colours, perspective, and the speed of the game to shift over time extending the ideas of ‘shifting perception’ discussed in Chapter 3. The complementary soundtrack also integrated drones and sounds suggestive of hallucinations, utilising the electroacoustic approaches discussed in Chapter 2. The project was very much an experimental hack and lacked stability; however, it succeeded in providing an early proof-of-concept that allows us to think about how interactive game experiences might represent psychedelic experiences. An increasing number of video games would later use similar approaches to those I have described here. For example, the magic mushrooms sequence in Far Cry 3 (Ubisoft Montreal 2012), and the various visionary sequences of

5 Sensorial Apparatus: Interactive Projects

117

Hellblade: Seanu’s Sacrifice (Ninja Theory 2017) are just two examples of games that represent hallucinations through morphing graphical effects and corresponding sounds. Both these games use similar ideas to those outlined here, but do so with more sophisticated game engines, providing more polished results that readers interested in these techniques should experience. Certainly, there is a utility to these approaches then, which can allow games designers to enrich storylines through depictions of hallucinations and ASCs. In the case of Quake Delirium EEG, the use of biofeedback may also indicate exciting new ways to experience these psychedelic simulations interactively, and perhaps we will see developers exploring these ideas in the future too.

Psychedelic Apparatus Go to bed wearing an elegant headband, and in the morning wake up and access a video recording of your dreams, playing back any section you wish with transport facilities for play, pause, rewind, and fast forward. This is the alluring possibility suggested by the DC Mini in the anime movie Paprika (Kon 2006), a fictional BCI that records a person’s dreams. The concept of capturing such ephemeral experiences is an interesting one, since dreams are otherwise hard to recollect and explain or share with others. Yet it is not inconceivable that such a technology could one day allow this by using some form of neuroimaging, generating a visualisation from the data. With such a device, perhaps we could also record the visual and auditory components of sensory experience during ASCs, thereby generating visualisations that begin to resemble what one might see or hear during hallucinations. Taking inspiration from the DC Mini, Psych Dome is an interactive audio-visual project that uses a consumer-grade EEG headset to control a psychedelic visualisation with a corresponding soundtrack. The project was first presented in a mobile fulldome at Wrexham Glyndwr ˆ University (16 October 2013).

118

J. Weinel

Psych Dome Media 5.3 Psych Dome composition example, 2 minutes 4 seconds; and documentation video, 1 minute 36 seconds The Psych Dome project continues to develop various approaches for the design of audio-visual materials based on concepts of psychedelia, focusing in particular on Klüver’s (1971) form constants, which can be generated interactively using an EEG headset in a mobile fulldome. As indicated in Fig. 5.3, the project utilises various software running on a laptop inside the dome. The laptop receives input data from a Neurosky Mindwave EEG headset, which is interpreted as Open Sound Control (OSC)10 data using BrainWaveOSC by Trent Brooks.11The EEG headset provides filtered readings for alpha, beta, gamma, and delta brainwaves, and interpreted ‘attention’ and ‘meditation’ values.12 The OSC messages are then sent to a Max/MSP application, which uses the scaled values to control various sound generating processes based on ASCs, such as drones and rotating percussive sounds. These sounds are provided by an additive synthesiser; playback of various pre-recorded samples; and a range of DSP effects similar to those used in the Atomizer Live Patch (see Chapter 3), such as filter, delay, reverb, ring modulator and a rotating Doppler shift effect. The Max/MSP application passes the OSC messages to a Processing application, which generates realtime visualisations such as spiral dot patterns based on visual patterns of hallucination. Sound outputs to stereo loudspeakers via the audio

EEG signals

OSC

Processing

BrainWave OSC OS C

Sou

nd

Max/MSP

Fig. 5.3 Psych Dome system diagram

5 Sensorial Apparatus: Interactive Projects

119

device, while video outputs to a projector via the display adapter. For this project, an inflatable mobile fulldome was used, in which a single projector was directed at a convex mirror that reflected light onto a hemispherical projection screen inside the dome.13 The sounds and visuals generated by the respective Max/MSP and Processing applications follow a pre-determined structure. Each time the project runs, sounds and images are produced for 1 minute 40 seconds using a generative system, where the resulting audio-visual materials change based on the real-time input data from the EEG headset. Media 5.3 provides an example of the audio-visual materials that the project generates, and visual evidence of the project running in the mobile fulldome. Table 5.1 shows the compositional structure of the piece. From 0:00 to 0:24,14 we see the onset of spiral patterns based on Klüver’s form constants, while high-frequency rhythmic sounds are heard. Next, from 0:24 to 0:31, a tunnel of lines is seen, while we hear a bass tone and drone. From 0:31 to 0:41, the visualisation provides a different spiral pattern, accompanied by high-frequency percussive sounds. This is followed from 0:41 to 0:58 by another spiral variation and mandala, while rhythmic sounds and filtered noises are generated (Fig. 5.4). At 0:58–1:15, a tunnel of triangles is accompanied by a bass tone and drone; then at 1:15–1:40, there is a further spiral, and high-frequency rhythmic sounds, before the piece terminates. While the timing of each of these sections remains consistent between performances of the piece, EEG modulates aspects of the sounds and visualisations that are generated. Hence, the EEG signals affect oscillator frequencies and various DSP properties, while also transforming the visualisations by causing temporal shifts in colour, transparency, form, and size attributes of shapes Table 5.1 Psych Dome compositional structure Time

Visual material

Sonic material

0:00–0:24 0:24–0:31 0:31–0:41 0:41–0:58 0:58–1:15 1:15–1:40

Spiral Lines tunnel Spiral Spiral + mandala Triangle tunnel Spiral

Rhythmic high frequency Bass tone/drone Rhythmic high frequency Rhythmic high frequency + filtered noise Bass tone/drone Rhythmic high frequency

120

J. Weinel

Fig. 5.4 Spiral animations based on visual patterns of hallucination and EEG signals in Psych Dome

and lines. As before, these gradual changes reflect the shifting perception that one may experience during an ASC, and so Psych Dome provides an interactive, biofeedback-driven audio-visual experience that visualises psychedelic hallucinations with a corresponding synaesthetic audio track. Psych Dome advances the approaches discussed so far in Explosions in the Mind, by demonstrating how an interactive audio-visualisation using consumer-grade EEG can drive an immersive experience based on psychedelic visual hallucinations. As part of this project, the Affective Audio research team at Wrexham Glyndwr ˆ University also carried out an experimental study to investigate how users experienced the system, and the extent to which they were connected to the work or able to volitionally affect the artwork.15 As with Quake Delirium EEG, the outcomes of the study suggested that the EEG headset provides a ‘passive’ form of interaction in which the user feels connected to the artwork through their biofeedback signals, but does not have a strong sense of voluntary control over the visualisation. In this regard, Psych Dome does not record an ASC experience in the manner of the fictional DC Mini of Paprika, but rather

5 Sensorial Apparatus: Interactive Projects

121

it provides an audio-visualisation that is conceptually related to hallucinations, which is modulated by brain activity. The system may point towards some new ideas for using biofeedback to design psychedelic visualisations of sound that others may extend in the future. These extensions could be carried out by building directly on the system, as when Psych Dome was adapted by the Australian group Sacred Resonance for Noosphere: A Vision Quest at Adelaide Fringe festival (4–6 March 2016, Adelaide Planetarium),16 or by composing completely new works.

Altered Simulations The final interactive project that will be discussed in this chapter is ASC Sim, a prototype made with the Unity game engine, which explores approaches for representing auditory hallucinations through sound. The project was informed by the outcomes of an earlier empirical study carried out in the context of the Affective Audio research group at Wrexham Glyndwr ˆ University, in which nearly 2000 qualitative selfreports of intoxication were analysed in order to collect experiential accounts of auditory hallucinations (Weinel et al. 2014b; Weinel and Cunningham 2017). Using this dataset, ASC Sim takes three features of auditory hallucination described in these experience reports, and provides interactive designs based on these. In doing so, the project shows how collaborative, interdisciplinary work involving empirical studies may feed into the design of interactive systems that simulate psychedelic ASCs.

Three Features of Auditory Hallucinations ASCs can vary in their intensity, and the concept of structural dynamics that move through phases of onset, plateau, and termination (see Chapter 2) were utilised in many of the projects discussed so far. The Quake Delirium and Psych Dome projects utilised this idea as an underlying design principle, so that representations of hallucination onset gradually, increasing in intensity over time; reach some form of plateau

122

J. Weinel

or peak; and then gradually subside, eventually terminating. In terms of Hobson’s state-space concept of consciousness, these representations also traverse points from ‘external’ to ‘internal’ sensory input and back again. Within this framework, representations of psychedelic states may also correspond with a variety of features based on ASCs, which were derived from the studies discussed earlier, such as Leary (1968), Klüver (1971), Strassman (2001), and others. The ASC Sim project does not attempt to represent all of the features discussed in these studies, but instead focuses in particular on three specific features that emerged from the empirical study carried out by the Affective Audio research group: • Selective auditory attention • Enhanced sonic perception • Spatial disruption of sound. ‘Selective auditory attention’ describes a common feature in the reports of intoxicated auditory hallucinations, whereby attention to sound seems to be disrupted. Selective auditory attention already takes place during normal-waking consciousness; for example, cognitive psychology studies by Broadbent (1958) and Treisman (1960) suggest that our attention system filters our perception of sound. During auditory hallucinations, some reports seem to suggest that individuals become particularly susceptible to prolonged absorption in a particular sound source, or conversely may feel as though they have a broadened perceptual awareness of multiple sound sources at different points in time. ‘Enhanced sonic perception’ describes experiences where the individual perceives sound as seeming more interesting, detailed, or enjoyable than usual. Perhaps due to the euphoric effects of various intoxicating substances, reports describe experiences of sound and music which are enhanced, almost as though the spectrum has been modified, so that high- or low-frequency bands are more pronounced than usual, as if one were using higher quality audio equipment. Conversely, in other instances, reports describe the opposite, whereby sound is perceived as dull, almost as if frequency bands have been suppressed or played back on lower quality audio equipment.

5 Sensorial Apparatus: Interactive Projects

123

‘Spatial disruption of sound’ describes aural experiences where various spatial features of sound may be perceived differently during a hallucination. Sounds may seem as if they are either closer or more distant than usual. For instance, the sound of a car further down the street may be heard as if it is nearby, or vice versa. Alternatively, a person may become confused about the direction that sounds are coming from. In some cases, the acoustic properties of the environment one is in may seem to change, so that sound echoes in a way one would not expect in that context. These three features were not the only ones described, but were chosen selectively to explore in the ASC Sim project. In general, the experience reports suggested varying degrees of intensity in auditory hallucinations, which seemed to range from mild enhancements; to moderate distortions; and total hallucinations, in which one hears things that seemingly have no acoustic basis in the surrounding environment. The latter may consist of hallucinated noises, music, voices, or even silence, where expected experiences of sound seem to be absent. In the next section, I will describe how the three features of auditory hallucination outlined were used as a basis for interactive sound design in the ASC Sim project.

Software: ASC Sim Media 5.4 ASC Sim software and demonstration video, 3 minutes 5 seconds The three features of auditory hallucinations described in the section above provide the basis for ASC Sim. Using the Unity game engine, ASC Sim provides a basic first-person perspective configuration, consisting of a flat grassy plane, three coloured cubes, and a sky background (Fig. 5.5). Each cube emits unique droning and pulsing sounds, which are spatially located as 3D sound sources.17 In the virtual environment, the user operates a first-person player controller using the keyboard and mouse, which can be used to walk around, viewing and hearing the scene from different locations. This provides a deliberately simplistic setup, which is sufficient as a test bed for exploring three interactive sound design mechanisms based on selective auditory attention, enhanced sonic perception, and spatial disruption of sound. I will now describe each of

124

J. Weinel

Fig. 5.5 Screenshot from the ASC Sim project. Coloured boxes in a simple game scene provide sound sources located in 3D space for testing purposes. Metres for ‘attention’ and ‘enhancement’ (top-left) indicate the current values of these properties

these mechanisms, which are also demonstrated in operation by Media 5.4. ‘Selective auditory attention’ is represented in the project by first identifying an object that the user is paying attention to. In the project, ‘attention’ is based on which of the coloured cubes the player is currently looking at.18 When looking at one of the cubes, an ‘attention level’ numerical float value gradually increases from 0.0 to 1.0, as represented with an ‘attention’ metre on the user interface (Fig. 5.5). As the attention value increases, the amplitude of all unattended sound sources decreases, so that the user hears only the object that is currently the focus of attention (Fig. 5.6). When the player is no longer paying attention to the object, the ‘attention level’ decreases, and the sound levels of the unattended cubes return to normal. In this way, the project simulates a narrowing of attention on a particular sound source, such as may occur in a more pronounced way during an auditory hallucination. ‘Enhanced sonic perception’ is modelled in ASC Sim by modifying the frequency properties of all sounds in the virtual environment. This is technically accomplished using an ‘enhance level’ numerical float

5 Sensorial Apparatus: Interactive Projects

125

Fig. 5.6 Diagram showing the ‘selective auditory attention’ mechanism. When the player attends an object, all unattended sound sources fade out

value, which ranges from 0.0 to 1.0. ‘Enhance level’ is also represented with a metre on the user interface (Fig. 5.5). As indicated in Fig. 5.7, increasing the ‘enhance level’ causes the sound to crossfade between three different pre-recorded versions of the source sound. These provide ‘dull’, ‘medium’, and ‘bright’ variations, for which graphic equaliser DSP effects have been used to reduce or enhance the frequency content of the original source sound. Increasing the ‘enhance level’ crossfades between these, so the same sound source is heard, but it becomes much brighter, with more high-frequency content as the value approaches 1.0. This mechanism simulates the subjective experience of sounds being enhanced through a more detailed frequency spectrum. ‘Spatial disruption of sound’ is reflected in the virtual environment by manipulating the spatial location of sounds. This is achieved by using

Fig. 5.7 Diagram showing the ‘enhanced sonic perception’ mechanism. As the enhancement value increases, sounds fade between ‘dull’, ‘medium’, and ‘bright’ versions of the source material

126

J. Weinel

Fig. 5.8 Diagram showing the ‘spatial disruption of sound’ mechanism. Each sound source moves in oscillating spatial patterns around the object with which it is associated

oscillating sine-wave values, which change the location of the sound sources. As shown in Fig. 5.8, the sound source moves around the object it is associated with in an oscillating pattern.19 When the user is inspecting a sound cube with this effect applied, the cube remains in a static location, but the associated sound source is animated in oscillating patterns around the cube, thereby producing a disorienting aural experience. In this way, the mechanism represents experiences of auditory hallucination where sound sources may seem spatially disassociated from the emitting object. The three mechanisms described exemplify ways in which we can think about representing these specific features of auditory hallucinations. As can be seen in the game object hierarchy of the Unity editor in Media 5.4, these mechanisms were made using C# scripts,20 which provide an ‘ASC engine’, as part of the group of ‘first person controller’ objects that make up the player avatar. This structurally locates the code as part of the player avatar, in recognition that these features mediate the way the avatar represents the virtual environment to the player.

5 Sensorial Apparatus: Interactive Projects

127

An expansion of this ‘ASC engine’ could include other features for representing auditory or visual hallucinations using sound and graphics. In summary, the ASC Sim project provides a game-engine prototype that explores ways in which auditory hallucinations can be represented through interactive audio. The project was informed by a large-scale empirical study, which provided various accounts of auditory hallucinations.21 From these experience reports, three specific features of auditory hallucinations were identified, which were then used to inform the design of corresponding prototype mechanisms, which reflect various auditory enhancements and distortions. Of course, this project could be further developed to provide an expanded range of options for representing these ASC features, and various other features of auditory or visual hallucinations could also be implemented. For example, visual hallucinations could be modelled by animating various graphical properties using similar methods as those utilised in the Quake Delirium or Psych Dome projects. This could ultimately lead towards an expanded audio-visual ‘ASC engine’, which provides a comprehensive set of features modelling psychedelic hallucinations for player avatars. ∗ ∗ ∗ Through the course of this chapter, we have seen how three distinct projects, realised through various forms of creative coding and sound design, represent psychedelic hallucinations through interactive systems. These projects draw together the approaches in sound, interactivity, and audio-visual design that were outlined in the previous chapters. In doing so they provide interactive audio-visual forms that reflect shifting distortions to perception, and a variety of other features that resemble what one might see or hear during a hallucination. Of course, these are all very much prototypes, and perhaps what matters most here is not the end result, but the idea they all point towards: the possibility of representing subjective experiences, including ASCs, through specific uses of game-engine simulations. This idea can be described as ‘avatar-centred subjectivity’, since it modifies the way graphics and sounds are used to communicate the virtual, subjective experience of a game avatar from

128

J. Weinel

a first-person perspective.22 This concept has an immediate utility for designing experiences of intoxication in FPS video games, and looking more broadly, it could also be used for designing simulations of auditory hallucinations or other subjective states that may serve therapeutic or communicative functions.23 In this chapter we have also seen how biofeedback technologies could be used to connect users to these interactive systems, providing passive forms of interaction. These ideas are the seeds that could one day lead to something greater. Extending these principles, perhaps with improved biofeedback technology, could lead towards interactive systems that record, and playback the contents of sensory experiences, giving rise to the cyberpunk technologies of tomorrow.

Notes 1. For a further discussion, see also the Adam Curtis documentary HyperNormalisation (2016). 2. The potential of psychedelic ASCs for treating depression has been suggested (e.g. Carhart-Harris et al. 2016a), and it is possible that immersive technologies capable of inducing ASCs might also have therapeutic applications. 3. For further discussions regarding representations of subjectivity in VR, see also and Weinel et al. (2018); and Weinel and Cunningham (2019). Of wider relevance to these discussions are the debates surrounding presence and immersion (Slater and Wilbur, 1997), which address the ways in which users feel a sense of embodiment in VR (Sanchez-Vives and Slater 2005; Slater 2009; Landau et al. 2020); and the role of sound in providing this (Grimshaw-Aagard 2019). 4. While this may not yet be possible with available technologies, there is early research that explores the generation of images based on neuroimaging techniques using fMRI; see Nishimoto et al. (2011). 5. Pure Data is a visual programming language similar to Max/MSP (both languages were developed by Miller Puckette). 6. For a further discussion of these systems and the idea of remixing video games, see the original article on Quake Delirium (Weinel 2011).

5 Sensorial Apparatus: Interactive Projects

129

7. Fitzquake (http://www.celephais.net/fitzquake/) by John Fitz is a modification for Quake that provides various graphical improvements. QuakeC is a compiled programming language for Quake that allows various parameters to be scripted via cvar commands. 8. For more information on the Quake Delirium EEG system and parameter mappings, see also Weinel et al. (2015a). 9. Use of the term ’passive’ here refers to Zander et al.’s (2010) definition of ’Passive BCIs’: ‘a passive BCI is a BCI which derives its outputs from arbitrary brain activity without the purpose of voluntary control, for enriching a human–computer interaction with implicit information’ (p. 185). 10. Open Sound Control (OSC) is a computer music networking and communication protocol. 11. BrainWaveOSC is an open source software application designed by Trent Brooks in collaboration with George Khut, which passes EEG data as OSC messages for use with software such as Max/MSP (Brooks 2019). 12. The interpreted EEG outputs are provided by Neurosky’s proprietary technologies; for a further discussion of EEG signals see Sanei and Chambers (2007). 13. Other more sophisticated fulldome systems use rigid dome structures with multiple projections that are stitched together using software such as Blendy Dome VJ (http://www.blendydomevj.com/). 14. The time stamps used in this section describe the actual timings used in the software application, rather than the video examples included in Media 5.3, where there is an offset due to the inclusion of video titles. 15. For an overview of the Affective Audio research team, see Weinel et al. (2014a). Further discussion of the user study carried out with Psych Dome is available in Weinel et al. (2014b) and Weinel et al. (2015b). 16. For more information on Noosphere: A Vision Quest see the Sacred Resonance website (https://www.sacredresonance.com.au/projects) and Facebook page for the event (https://www.facebook.com/events/adelaideplanetarium-unisa-mawson-lakes/noosphere-a-vision-quest-at-adelaide-pla netarium-fringe-2016/209854189352393/). 17. In game audio 3D sound encompasses various features such as attenuation of amplitude based on the relative distance of the listening player avatar. For a further discussion see Horowitz and Looney (2014, pp. 129–131). 18. This is technically accomplished in Unity with ray casting. A ray is projected from the player avatar based on the direction the avatar is looking, and when the ray intersects with an object, that object is considered the current focus of attention.

130

J. Weinel

19. Dissociation between object and sound source is achieved in the Unity object hierarchy by creating a separate child object to the cube, which the sound source is then attached to. The child is animated with a C# script, while the parent object remains static. 20. While ASC Sim uses C# scripts and the Unity audio engine, similar mechanisms could alternately be devised using combinations of scripting and game audio middleware such as FMOD or Wwise. 21. Informing sound design through the use of empirical studies may provide a route towards more accurate representations of experiences such as auditory hallucinations; see also Weinel, Cunningham, and Griffiths (2014b); Weinel and Cunningham (2017). 22. The concept of ’avatar-centred subjectivity’ was first proposed in Weinel and Cunningham (2019). 23. For example, Weinel et al. (2018) discuss representations of autism through specific uses of sound design and graphics in VR, in order to raise public awareness.

6 Optical Geometry: VJ Performances

On the side of a dusty track somewhere in Croatia, I am waiting with a group of people in the sweltering afternoon heat for a shuttle to take us to Mo:Dem psy-trance festival. Eventually, we board a bus that has seen better days. It’s too hot inside—I leap into one seat and then crawl into another one that is out of the direct sunlight, in a bid to prevent myself from melting onto the faux-leather upholstery before we get to the festival. After what seems like an eternity, the bus leaves. Refreshing cool air rushes in through the windows as the vehicle speeds its way along winding roads deep into the countryside. At last over the brow of a hill, a sea of tents unfolds across the fields, as bronzed bodies in loose t-shirts and hemp clothing stroll along the side of the road, and others wait for the bus to give them a ride back into town. I wander down a muddy track with the other festivalgoers, heading towards the low-frequency Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/978-981-16-4055-1_6. Where the icon is shown in the chapter, the reader should refer to the supporting media files.

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 J. Weinel, Explosions in the Mind, Palgrave Studies in Sound, https://doi.org/10.1007/978-981-16-4055-1_6

131

132

J. Weinel

rumble of a sound system somewhere deep in the forest. Following the sound, I pick my way through a maze of trees, and hammocks that are occupied by the prone bodies of hippies who have already been partying here for days. Eventually, I reach a clearing, where swathes of animated figures dance relentlessly to the hypnotic beats of psy-trance music. The music itself is coming from a stage set that resembles a vast jagged sculpture. With sharp fins and an impossibly detailed mandala in the centre, the structure looks like a mysterious, crash-landed alien spacecraft from some strange science-fiction movie. As dusk falls, the pulsing throb of synthesiser basslines ramps up in intensity, while detailed layers of otherworldly sounds trickle, pop, and squirt over the top, as if the music is dripping in some kind of gooey neon hallucinogenic liquid. As the final rays of sunlight cutting through the tall branches high above fade to black, the stage comes alive with brilliant shades of purple, pink, green, and yellow. Snake-like forms creep across the sharp contours of the huge sculpture, illuminated by projections that pulse in time to the beat. In the hot sticky night, ultraviolet canopies surround us in the trees, while lasers and projectors cast tribal patterns and optical geometry across the scene, whipping the crowd into some kind of relentless, hypnotic frenzy of dance—truly a momento demento, or ‘moment of madness’. The psychedelic video projections at psy-trance festivals such as Mo:Dem are a form of VJ (video jockey) performance.1 As discussed in Faulkner (2006), VJing involves live mixing of video to music. The art form emerged primarily from electronic dance music culture in the 1980s, where VJs began mixing videos to complement DJ performances at nightclubs, raves, and festivals. While VJ performance has synergies with visual music, perhaps more importantly, antecedents can be found in the liquid light shows of 1960s rock concerts. Investigating the latter, in 2018 I interviewed Paul Brown, a member of Nova Express, a group which also included founder Jim MacRitchie and Les Parker, who performed psychedelic lightshows in Manchester and the North of England in the 1960s and early 1970s (Weinel 2018b). Nova Express provided light shows for bands such as Pink Floyd, The Who, The Nice, Canned Heat, Barclay James Harvest, Edgar Broughton, and others. As Paul explained to me in the interview, the group approached the light shows as improvisational performances. Using slide projectors, inks,

6 Optical Geometry: VJ Performances

133

paints, and chemicals in glass clock faces on overhead projectors, the group would energetically fade images in and out by moving their hands rhythmically in front of the beams, working in synchronisation with the music and creating their own forms of “visual syncopation’ where ‘one rhythm would be going into the eyes and another rhythm would be going into the ears’. The respective group members would even direct their visual performance to match the different musical instruments of the band, weaving patterns of light to match the drum, bass, and guitar parts of the music. Albeit with analogue projection technologies, Nova Express was essentially providing an early form of what might today be called a VJ performance. Of course nowadays at festivals like Mo:Dem or London’s audio-visual performing arts festival Splice,2 modern VJs utilise highspec laptops and MIDI controllers, which allow them to mix digital video in real-time. While Nova Express mixed turbulent oils and performed visual rhythms with their hands, the VJs of today can use audio analysis to map properties of frequency and amplitude to various visual filters and effects, transforming videos in real-time using software like VDMX, Resolume, or Modul8. They can also use video-mapping software such as Mad Mapper to project onto the contours of irregular surfaces, displaying their projections across complex stage sets like the ones used at Mo:Dem. While the technologies may have changed, the ultimate goal remains the same—to provide complementary synaesthetic experiences of rhythmic visual energy and psychedelic imagery, which reinforce the symbolic meaning of the music.3 There is an excitement that is brought by live visuals, regardless of whether they are created with 8 mm films, coloured liquids reacting under the heat of a lamp, or generative computer graphics animations that pulse in response to an audio signal. One advantage of digital approaches is that powerful computers are increasingly available at a relatively low cost, making VJ performance a much more accessible pursuit. As a natural extension of my previous DJing and my interest in audio-visual composition, in 2016 I began my own VJ experiments. In this chapter, I will discuss my work in this area, which explores

134

J. Weinel

various approaches for composing psychedelic visualisations in correspondence with sound. Here the main focus will be the construction of my VJ performances under the alias Soundcat. By designing VJ loops using direct animation, creative coding, and 3D animation, I created a DJ/VJ mix that combines 1990s breakbeat hardcore music4 with original visuals. Following a discussion of this project, I will also describe my parallel visual art practices, which explore related ideas through paintings, including three works that can be enhanced with an augmented reality (AR) application. Through the course of this chapter, we will see how VJ performance can provide an alternative means through which to compose psychedelic visualisations of sound based on symbolic associations with music.

Making the VJ Loops Extending my earlier work in audio-visual composition, from 2016 onwards I began making video loops for use in VJ performances. Initially, I created simple visual animations in Processing, often using oscillating colours and shapes. These designs extended concepts of psychedelic visual hallucinations and altered states of consciousness (ASCs) discussed earlier in this book, while also drawing significantly on the style of 1990s acid house VJ performances, as documented on videos such as Dance in Cyberspace (Dr. Devious and the Wiseman 1992); Global Chaos (Hex 1993); Future Shock (Frost et al. 1993); and the X-Mix series (Studio !K7 1993, 1998). These 1990s visuals are similar to the ’demo effects’ computer animations seen in the demoscene computer art subculture (Polgár 2005),5 which are also revisited in the aesthetics of the recent vaporwave music genre. While many of the loops I will discuss in this section were made in Processing, others were created using techniques such as stop-motion animation, direct animation, 3D graphics, or found video materials.

6 Optical Geometry: VJ Performances

135

Selected Examples Media 6.1 VJ Loops demonstration video, 6 minutes 30 seconds Media 6.1 provides a silent demonstration video with excerpts from a selection of VJ loops that I created for use in my performances. Still images from these loops are also shown in Fig. 6.1.6 These are the raw ingredients for my VJ performances, in which they are mixed together with additional visual effects and combined with music. Before describing these VJ performances in more detail, I will first explain the design of the individual VJ loops. ‘Sinewarp’ is one of the first VJ loops I made for this project using Processing (Fig. 6.1a). As with several other examples, the code generates oscillating float values using the sine function. These values are used to change the positions of two radial patterns, which intersect creating a moiré effect. Oscillating values are also used to control the red, green, and blue colours used for drawing the circles, causing them to change colour and flicker with a stroboscopic effect. ‘Rotwarp’ develops the ‘sinewarp’ code, providing oscillating colour effects that are used to draw radial patterns (Fig. 6.1b). Rectangles are drawn at various points of rotation using a for-loop, creating many radial patterns that vary depending on the angle of rotation. A fade effect is also created by drawing a transparent black rectangle on the canvas in each frame, thereby gradually erasing the contents of earlier frames, which persist for a short time. The results are similar to the roulette curves such as those produced by the Spirograph toy, and also relate to the concept of spiral patterns seen during visual hallucinations. ‘Skywarp’ is a related Processing sketch, in which animated geometric patterns are superimposed over rapidly cycling images taken from the window of an aeroplane (Fig. 6.1c). These aerial photos were taken during a trip to South America. The colour values of both the geometry and the photos oscillate. ‘Ghosty’ was also made with a Processing sketch that draws repeating patterns of animated sprites,7 which scroll horizontally with oscillating colours and vertical positions (Fig. 6.1d). The sprites were made by drawing individual frames of animation in a pixel art style,8 using a stylus and Inchworm Animation on a Nintendo 3DS portable games

136

J. Weinel

Fig. 6.1 Various still images from the VJ loops

6 Optical Geometry: VJ Performances

Fig. 6.1 (continued)

137

138

J. Weinel

console.9 These were exported and used in a Processing sketch, which plays the individual frames of animation on a loop. For ‘ghosty’, ghost and diamond sprites were used, however, several other adaptations of this sketch were also made by substituting different sprites into the sketch and changing various parameters. ‘Cycler’ was created using a 3D Processing sketch, which draws an array of spheres that oscillate in size and position (Fig. 6.1e). The overlapping spheres create a worm-like figure, which reflects the concept of ‘mysterious forms’ in hallucinations, discussed previously. A series of ‘cycler’ sketches were created based on the same code, providing various oscillating dot patterns using 3D spheres. These variations were created by experimenting with different parameters for the oscillating values, thereby repurposing the code to create a series of VJ loops. ‘Boxworld’ also uses the 3D functionality of Processing to make an endless tunnel of purple cubes (Fig. 6.1f ). This is technically accomplished by drawing rows of boxes that move forward in 3D space along the z axis with each frame. Boxes that pass the z position of the camera are returned to a point in the distance with each cycle (this programming technique is known as ‘object pooling’). A mirror post-processing effect is used to draw a ceiling of boxes, thereby creating the tunnel effect. ‘Lava’ was made using the direct animation techniques discussed in Chapter 4 (Fig. 6.1g). Hand-painted 8 mm film was projected, and digitally filmed. This provides colourful, organic psychedelic textures that can be combined with other materials in a VJ mix. The ‘lava’ title of the VJ loop reflects the bright orange colours that are seen in this clip, which were created using orange ink. As discussed earlier, various artefacts such as cracks in the paint add complexity and interest. ‘Blue flash’ was similarly made using direct animation (Fig. 6.1h). In this case, marker pens provide strong blue and orange shades and psychedelic textures that can be layered with other loops in a VJ performance. ‘Mirror Inktray’ is one of several VJ loops made using stop-motion animation (Fig. 6.1i). In this case, various inks, paints, and oils were mixed in a plastic tub, which was photographed, providing colourful abstractions reminiscent of the oil projections used in 1960s psychedelic lightshows.

6 Optical Geometry: VJ Performances

139

‘Leafwarp’ was similarly made using a stop-motion animation process (Fig. 6.1j). In this case, I experimented with leaves, Danish newspapers, and other found materials. I painted on to some of these materials, as seen in Fig. 6.1j, which exhibits a familiar spiral motif. ‘Scanshroom’ provides vertically scanning horizontal wave patterns (Fig. 6.1k). A Processing sketch draws horizontal white lines, which move downwards and gradually fade out. The resulting video can then be layered with a still image, using a blending effect. In this case, the still image used was a hand-drawn sketch based on visual hallucinations. To make variations I used photographs and other images. ‘Bspiral’ was made using a 3D Processing sketch, which draws oscillating cubes that rotate, expand, and contract around a central point (Fig. 6.1l). The Processing sketch uses the x and y coordinates of the mouse cursor as an input to modify parameters of oscillation and rotation, thereby changing the animation when the cursor is moved. By positioning the cursor at various coordinates and recording the output, a series of related VJ loops was quickly generated from the same sketch. ‘Plasma’ was made using a Processing sketch that combines multiple oscillating values, which are used to draw wave patterns on the screen (Fig. 6.1m). An array of cycling colour values is used to animate the colours. This creates the classic ’plasma’ effect seen in many demoscene videos.10 Variations were made by changing the oscillating colours, thereby producing a series of brightly coloured VJ loops. ‘Plasma bath’ is a 3D animation made in Cinema4D, which uses the ’plasma’ VJ loop described above to colour and displace the heights of polygons (Fig. 6.1n). This creates a dynamic 3D plasma effect, over which an array of rotating metallic spheres was superimposed. Various other clips like this were also created in Cinema4D using arrays of rotating geometric forms. ’Dreamscape’ is a 3D animation that references the visual imagery associated with rave culture (Fig. 6.1o). The VJ loop is based on a similar graphic used on the Dreamscape December 1991 rave flyer, and therefore provides a symbolic association with rave music. This loop was made using rotating 3D geometry in Cinema4D, which was rendered in green wireframe.

140

J. Weinel

‘Trancecore’ provides various oscillating 3D boxes as a camera moves rapidly through fields of red cuboid shapes (Fig. 6.1p). As I will discuss later in this chapter, while making my VJ loops, I also created various paintings and drawings. In some cases, visual motifs developed through paintings or sketches were translated into VJ loops and vice versa. Exemplifying this, Fig. 6.2 shows a digitally modified ink pen sketch

Fig. 6.2 ‘Trancecore’ sketch, digitally re-coloured

6 Optical Geometry: VJ Performances

141

depicting a strange landscape filled with fields of cuboid shapes. This sketch provided the basis for the ’trancecore’ VJ loop. In Processing, a similar impression was created in code using narrow 3D boxes, which move rapidly towards a camera (using similar approaches to ‘boxworld’). While the pen sketch includes an organic structure in the centre, in the Processing sketch, patterns of oscillating boxes rotate in the visual field. ‘Spinner’ is one of several geometric shapes rendered in Cinema4D (Fig. 6.1q). These shapes include an alpha channel, allowing them to be superimposed over different backgrounds and animated textures. ‘Plasma box’ was also made in Cinema4D, and reuses the ’plasma’ visuals to provide animated textures on a rotating cube (Fig. 6.1r). This design is a homage to a similar visual that appears in the Future Crew’s ‘Second Reality’demo effect (DemosceneVids 2015). ‘They live’ is a pastiche of a scene from the movie They Live (Carpenter 1988), reinterpreted as a vaporwave-style visual (Fig. 6.1s). In the movie, by wearing special sunglasses, the protagonist can read hidden messages that have been placed in advertisements, urging the public to ‘consume’, ‘obey’, ‘conform’, ‘buy’, and ‘watch television’. In my adaptation, these messages are displayed on Windows 95 pop-up advertisements. In addition to text such as ‘obey’, ‘consume’, and ‘no thought’, messages such as ‘take selfies’, ‘update status’, and ‘like sponsored link’ reflect the age of social media. ‘Purple interference’ is a Processing sketch, in which various oscillating waveforms are used to generate red and purple interference patterns (Fig. 6.1t). The resulting animation provides textural visual noise materials that are suitable for layering with other VJ loops. The selected VJ loops evidenced by Media 6.1 and Fig. 6.1 illustrate the type of materials used in my VJ mixes. Most of these loops were made using non-real-time procedures, in which sequences of frames are rendered that are then combined in post-processing software to make HAP11 video clips. Once the new VJ loops have been made they are added to a library of clips in VDMX, ready for mixing to music.

142

J. Weinel

As noted, the design of the VJ loops extends some earlier approaches to visual animation based on ASCs, while also drawing on the psychedelic computer graphics of 1990s VJ performances, demo effects, and vaporwave videos. In this regard, the VJ loops place a greater emphasis on exploring the ‘visual language’ associated with 1990s rave music, rather than basing the designs solely on reports of ASC experiences. For breakbeat hardcore music, this ‘visual language’ arises from the many techno-utopian/dystopian designs and fantasy images seen on rave flyers, record sleeves, and other visual artefacts from this era.12 Previous VJ mixes also serve to reinforce these audio-visual relationships. Because these visual associations already exist, we can imagine that a synaesthetic sound-to-image hallucination experienced in response to rave music might access them, manifesting visualisations like these in the mind’s eye. From this perspective, when the intended soundtrack is 1990s rave music, it is logical to design psychedelic visualisations of sound based on these kinds of images. While the above outlines the ‘representational properties’ used by these VJ loops, another significant aspect of design relates to the ‘affective properties’ of the video clips, which allow us to consider the extent to which they may induce states of high or low arousal for audiences.13 Since the intended usage for these clips is to provide corresponding visuals for high-energy breakbeat hardcore music, the VJ loops may complement and enhance this by also promoting states of high arousal for the viewer. Just as music communicates high arousal through the kinetic energy of rhythmic percussion, VJ loops can also generate a sense of visual arousal or sensory overload through movement, by using forms such as stroboscopic imagery, kinetic moving objects, rotating geometry, or animated tunnels. In this way, both sound and image may promote heightened states of arousal, perhaps even inducing states of sensory overload and trance. My VJ performance emerged from many hours spent iteratively making and experimenting with these VJ loops in VDMX. When mixing the loops together in real-time, I use a MIDI controller to adjust parameters spontaneously, blending, cross-fading, and applying further visual effects to the mix. Most visual effects have parameters that can either be controlled with MIDI or automated using values derived from frequency

6 Optical Geometry: VJ Performances

143

Fig. 6.3 Still from a practice VJ mix combining multiple loops

analysis or low-frequency oscillators. I experimented with creating VJ mixes to various existing music, recording some demonstration videos (Fig. 6.3).14 Around this time I also recorded a VJ mix for the Japanese psychedelic rock band Hibushibire using their album Freak Out Orgasm! (2017) as a soundtrack. This was undertaken with permission from the band and was used by Riot Season Records to promote the album.

Live in London Building on my library of VJ loops and initial experiments, I decided to create an audio-visual performance by mixing existing music in the form of a DJ mix, while simultaneously providing my own original visuals through a VJ mix. First, I recorded new visuals for existing pieces of music, essentially creating original music videos for each track. For the final performance, I then combined these videos together to make a DJ/VJ mix, while also triggering other audio-visual elements. This DJ/VJ mix was first performed under the alias Soundcat as part of a concert organised by VJ London15 held at New River Studios (London, 12 July

144

J. Weinel

2018). While this performance is an evolving project that can potentially incorporate different styles of music, my discussion here will focus on the iteration designed for the VJ London event, which used breakbeat hardcore music.

Recording the Videos For the VJ London performance, I used existing 1990s breakbeat hardcore music, and some other styles such as trip-hop and hardcore punk. To prepare for this performance, for each of the music tracks, a VJ mix was first recorded in the studio using my VJ loops. Figure 6.4 shows the technical setup used in the studio. VDMX runs on a laptop, with a Korg NanoKontrol and an Akai MPC Studio providing MIDI control data for manipulating video effects and triggering clips. The video signal outputs

Akai MPC Studio

MIDI Controller

Korg

MIDI Controller MIDI control data

MIDI control data

Laptop running VDMX

Audio-visual monitoring

Audio-visual outputs

Fig. 6.4 Diagram showing the VJ setup used in the studio

6 Optical Geometry: VJ Performances

145

to a Blackmagic Hyperdeck Shuttle and display, allowing the output to be viewed and recorded simultaneously. In what follows, I will describe the original music videos created with this setup. ‘Yes to Satan’ by New Atlantic (1992) begins with a brightly coloured blue alien landscape with a tumultuous red sky (Fig. 6.5). As the breakbeats begin, black and white patterns pulse across the screen, which are based on the concept of hallucinated cryptic messages resembling a sixteen-segment LED display (see also Chapter 1, p. [1]). This ‘cryptic messages’ motif is one that I have also used in various other visual artworks. In this case, the designs were made in Photoshop and triggered in time with the beat using the percussive pads of the Akai MPC Studio. At points, the video rhythmically cuts between mysterious rotating objects, including the ‘plasma box’ clip, which is processed with a feedback visual effect. ‘Nightmare’ by Kid Unknown (1992) uses a VJ loop of dancing alien figures, which was created with a 3D sculpting tool, Cinema4D, and the Mixamo animation tool. These figures were processed with various colour transformations and stroboscopic effects, which were mixed rhythmically in correspondence with the music, in order to provide a high arousal visual that reflects the energy of the drumbeats.

Fig. 6.5 Still from the ‘Yes to Satan’ video

146

J. Weinel

An inverted colour effect used in this video produces a result that I find reminiscent of the visual effects used in the hallucination sequences of the movie Altered States (Russell 1980), while the moving figures recall Hex’s Global Chaos (1993). ‘A London Sumtin’ by Code 071 (1992) uses scrolling patterns of acid house smiley faces, and a détournement of the London underground logo which I modified to read ‘Soundcat’. The mix cuts between this visual and the purple ‘boxworld’ tunnel, with the ‘cycler’ graphic superimposed over it. Here an oscilloscope effect is seen, which was generated using audio analysis in VDMX. Later in the clip, a fractal sequence designed in the software Mandelbulb 3D is also used. ‘Wipe the Needle’ by the Ragga Twins (1991) begins with a view of a Photoshop user interface, within which the ‘dreamscape’ visual is seen rotating, superimposed over the ‘boxworld’ tunnel (Fig. 6.6). Behind this, a scrolling graphic displays discordian symbols including an animated apple with the number 23 written on it.16 The background cuts to scanning tropical palm trees (photographed from a trip to Rapa Nui, and modified with a variation of the ‘scanshroom’ Processing sketch discussed earlier), and the animated ‘purple interference’ pattern. We then see an explosion of pop-up windows (‘they live’), which are

Fig. 6.6 Still from the ‘Wipe the Needle’ video

6 Optical Geometry: VJ Performances

147

synchronised with the beat of the music using VDMX’s beat detection functionality, and terminate with the ‘blue screen of death’ image of a PC crashing with a fatal system error. Taking inspiration from approaches used in the vaporwave genre, the various visuals of user interfaces, tropical palms, and failing computer systems expose the illusory mechanisms behind media fantasias (see also Chapter 7, p. 191). ‘Mil Vidas’ by Bixiga 70 (2015) also uses a 3D dancing figure. In the background, various red and yellow patterns move to the beat. These visuals were made using stop-motion animation of jagged pieces of paper over a light-box, resulting in short ‘one-shot’17 clips, which were triggered in time to the beat using the Akai MPC Studio. These were also processed with a mirror effect in VDMX. This background layer of the video was first recorded using the Blackmagic Hyperdeck Shuttle, and the footage was then recombined with the dancing character and other visuals, adding video feedback and other effects.18 ‘Animal’ by Jaguar (1998) uses a ‘cryptic messages’ effect similar to the one used in ‘Yes to Satan’, where horizontal patterns are triggered rhythmically in time with the beat. To produce more accurate timesynchronisation between the breakbeats and the visuals, I loaded the ‘Animal’ track into a music tracker (Renoise) and programmed a MIDI sequence that matches key percussive elements in the beat. This MIDI sequence was then used to trigger the one-shot ‘cryptic messages’ visuals in time with the beat, in real-time. Behind these, the VJ mix cuts between the ‘trancecore’ and ‘ghosty’ visuals. ‘Ape Shall Never Kill Ape (Twin Tower Mix)’ by U.N.K.L.E featuring Nigo & Scratch Perverts (1998) takes a different approach, using a ‘video mashup’ technique.19 The music track ‘Ape Shall Never Kill Ape’ uses various audio samples from trailers for the Planet of the Apes films (1968– 1973). The main visual for this track was made by lining up footage from the original Planet of the Apes trailers with the samples in the music, thereby providing images one might associate with these sounds. This video track was then mixed in VDMX, where it was overlaid with oscilloscopes, the ’plasma’ visual, scrolling patterns of animated ape heads, and the statue of liberty smoking a spliff (the latter an irreverent reference to the ending sequence of the first Planet of the Apes [Schaffner 1968]

148

J. Weinel

movie, which is in-keeping with the trip-hop style and pop-cultural sampling of the music). ‘Bed Jam Session’ by Tycoon To$h & Terminator Troops (1997) is based around a direct animation background, which is processed with a VHS effect. Over the top of this, we see brightly coloured orange and yellow lightning shapes, which were made using transformations of stopmotion animations made by moving and photographing pieces of paper over a light-box. These were mixed together with an oscilloscope graphic, in time with the beat. ‘LED Down’ by Tipper (2000) is a trip-hop track with a slower tempo, requiring less visually kinetic or energetic visuals. The main visuals were created using a recording of the PC video game Magic Carpet (Bullfrog Productions 1994), which is processed with various effects, including a brightness adjustment, which is tweaked in time with the music. At points, the VJ mix also cuts to footage of the M1 motorway heading into London, scrolling patterns of clouds, flying dolphins, and a rotating 3D vaporwave scene suggestive of a computer graphics demonstration. ‘Siren Bass’ by Aphrodite (2001) is also based around sampled footage of the Atari ST video game Lotus Turbo Challenge II (Magnetic Fields 1992). Rave music has associations with Atari ST and Commodore Amiga computers, which could be used for both gaming and home music production (Weinel 2018a). For example, Aphrodite is known to have used the music tracker OctaMED on the Amiga for many of his music productions. In my video for ‘Siren Bass’, sampled video game footage connects the music with computer games of the day. This footage is processed with video effects, and combined with palm tree animations, a ‘soundcat’ logo, and the ‘bspiral’ loop. ‘Acid Rain VIP (Breakage Final Chapter Mix)’ by Equinox (2006) is based around footage of a 3D object being squashed, unravelled, and exploded20 (Fig. 6.7). This footage was technically produced using 3D geometry in Cinema4D, which various animated deformation processes were applied on to. The deformations were designed to reflect the sounds of the processed drum breaks, which are the main focus of interest in this track. For example, the 3D object squashes resembling a cymbal as a crash sound is heard; shrinks into a ball in correspondence with a pitch-stop sound effect; and disintegrates as the drum breaks

6 Optical Geometry: VJ Performances

149

Fig. 6.7 Still from the ‘Acid Rain VIP’ video

are time-stretched. The rendered 3D footage was divided into several different video clips, which were then triggered using a MIDI sequence in Renoise, allowing precise synchronisation between the footage and the percussion. Visual filters were also applied in VDMX to give a striking electric-blue effect. ‘Message in a Bottle’, a cover of The Police song, by Excel (1989), uses various combinations of stop-motion animation (‘leafwarp’), oscilloscopes, direct animation (‘lava’), and Processing animations (‘sinewarp’ and ‘plasma’). These are rapidly cut-together in time with the music, in order to provide highly kinetic visual noise and patterns that match the energy of the music. While the process of creating all of these videos is somewhat labourintensive, the end result is a set of original music videos for existing pieces of music, which provide the building blocks for creating the final audiovisual performance. Each video becomes analogous to a ‘record’ that a DJ mixes, except in this case these ‘records’ also have a visual component, so the final performance becomes a DJ/VJ set. A limitation of this approach is that some aspects of video mixing and triggering, which are undertaken ‘live’ during the production process, become ‘fixed’ in the final performance. However, a benefit of this is that in the final

150

J. Weinel

performances more effort can be focused on manipulating sound and mixing the records together in time. This is significant because, in a solo performance, there is only so much that one person can do. In addition, accurate timing is also arguably more important for music than it is for visuals, since the ear is more sensitive to timing than the eye (Chion 1994, pp. 10–11). As a result of this phenomenon, if two rhythm tracks fall out of sync the musical experience as a whole will be derailed, whereas any imperfections in the timing of the visuals are more likely to go unnoticed. Moreover, I consider that the music is the most important, central element of the performance. The visuals must flow from and complement the sound, not the other way around. Prerecording visuals in this way also allows rapid transitions between many intricate configurations that would otherwise be difficult to recreate live, leading to more a diverse end result. Another practical advantage is that pre-rendering the visuals reduces the computational load on the CPU during the final performances, meaning the live performance system has improved stability. The workflow used to create these videos could be summarised as ’live VJ mixing in the studio’. Though the end result is a series of fixed-media videos, the process embeds aspects of spontaneous organic improvisation, allowing the videos to acquire qualities of musicality which differ from those that would likely be obtained through non-real-time processes (e.g. post-processing video software). Of course, this real-time approach means that a recording can go wrong, and in some cases, multiple takes were needed, from which either the best one was selected, or the two best were kept, thereby providing an alternative version that could be used if so desired in the final performance. Following the analogy of each video being like a ‘record’, these variations are like the alternate ‘dubs’ that a DJ might play. Since this workflow involves repeated real-time processes of sampling, remixing, and resampling of visuals, I also find the production process analogous to the ‘live mixing in the studio’ approach that dub reggae producers use. Just as the dub producer may play back prerecorded tracks in the studio, modifying and remixing them with filters, cutting sounds in and out of the mix, to make these videos I am using a similar approach with visuals. This allows special alternate versions of a track to be used in live performances. For example, an alternate video

6 Optical Geometry: VJ Performances

151

for ‘Acid Rain VIP (Breakage Final Chapter Mix)’ (which itself is a ‘dub version’ of the original Equinox track) was produced using a different set of visual filters to give an electric pink version of the video, which I can choose to use in some performances instead of the blue version.

Performing the DJ/VJ Mix Media 6.2 Soundcat DJ/VJ mix, 30 minutes 32 seconds For the final performance at VJ London, I utilised a DJ/VJ setup based on the Serato Scratch DJ software. This provides two virtual ‘decks’ with transport facilities for playing two music tracks simultaneously. As with a typical DJ system, pitch controls can be used on each deck to synchronise tracks, which can be blended with a DJ mixer. Adjustments to amplitude, EQ, filters, and a range of DSP effects can be applied to either deck. While a DJ typically works only with sound, the MixEmergency video plugin allows each track to have a video element. Fading between two music tracks therefore produces transitions between both sound and image, while the EQ and filter controls are also linked to visual effects parameters. The system also allows spontaneous live triggering of one-shot audio-visual materials. A diagram of the performance system is shown in Fig. 6.8. A laptop running Serato Scratch and MixEmergency is controlled by an Akai AMX DJ mixer. The compact Akai AMX outputs sound to the main PA (public address) system, while also sending a signal to a pair of headphones, which allows the performer to cue and preview the sound from either deck. In addition, an Akai MPC Studio controller is also used, providing 16 percussive MIDI pads, which can be used to trigger the one-shot audio-visual materials. Video outputs from the laptop to a projector via HDMI. Media 6.2 provides an example recording of a practice mix that is representative of the one performed at the VJ London event. The first half of the mix focuses on 1990s breakbeat hardcore music, beginning with the New Atlantic track ‘Yes to Satan’. During the intro, a oneshot element is triggered, consisting of pre-recorded synthesiser sounds created with a Korg MS-20 Mini; edited speech from the science-fiction

152

J. Weinel

Akai MPC Studio

MIDI controller

Laptop running Serato Scratch and MixEmergency

MIDI control data

Audio output (monitoring)

Akai AMX Mixer

Serato Scratch control data

Video output

Audio output

Audio system

Video projector

Fig. 6.8 Diagram showing the live setup used for the VJ London performance

TV show The Outer Limits (1963–1965); and psychedelic radial patterns (0:10). From 1:59–2:23, we hear the Kid Unknown track ‘Nightmare’ being introduced, and see the dancing alien figures of the video superimposed over the ‘Yes to Satan’ visuals while a transition between the two tracks occurs. From 3:38–4:44, the next transition begins, as ‘A London Sumtin’ fades in, with some rapid mixing techniques used to cut the sound and visual of ‘Nightmare’ in and out rhythmically. Launching with vaporwave visuals, the Ragga Twins’ ‘Wipe the Needle’ comes in from 6:07–6:42, accompanied by a one-shot audio-visual element that includes a time-stretched ‘s–o-u-n-d-c-a-t’ sample.21 Following this, Bixiga 70’s ‘Mil Vidas’ is dropped in at 7:58, as we see the dancing alien character after the ‘blue screen of death’ visual (Fig. 6.9). During this track at 10:47 and 10:57, two audio-visual one-shots are heard, one of which includes the sample ‘soundcat… he was born and raised in a hatchery just like an egg’, reflecting the irreverent humour that I incorporate in this performance. The crisp breakbeats of Jaguar’s ‘Animal’

6 Optical Geometry: VJ Performances

153

Fig. 6.9 Performing as Soundcat at VJ London (New River Studios, 12 July 2018) (Photo credit: Laurie Bender [L’Aubaine])

first cut through the mix at 11:08, merging with the rhythms of Bixiga 70, while also creating a rich composite of rhythmic visuals from the two videos. Punctuated by another audio-visual one-shot at 12:58, the two tracks are allowed to ride together until ‘Mil Vidas’ runs out at 13:13. ‘Animal’ ends with an abrupt ‘deck stop’ slow-down effect, which digitally simulates the effect of stopping a turntable. The second half of the mix drops down to a slower tempo, beginning at 14:06 with the sounds of U.N.K.L.E featuring Nigo & Scratch Perverts’ ‘Ape Shall Never Kill Ape’, with the new Planet of the Apes visuals. During this track, various audio-visual one-shots are used, and from 15:31–15:49 the mix cuts in with beats from ‘Bed Jam Session’.22 At around 18:18, a filter effect is applied together with a deck stop, as the mix transitions to Tipper’s ‘LED Down’, accompanied by video game visuals and motorway footage. At 19:52 the mix brings the tempo back up by bringing in Aphrodite’s ‘Siren Bass’, in double time relative to ‘LED Down’.23 From 21:48– 21:57, the drum & bass rhythms of the Aphrodite track temporarily drop out, which occurs due to a technical issue with the software, and is masked using an audio-visual one-shot. ‘Acid Rain VIP (Breakage

154

J. Weinel

Final Chapter Mix)’ drops into the mix at 22:38, superimposing the electric-blue 3D sculpture over the flickering video game visuals of ‘Siren Bass’. The two tracks ride together until 22:53, with emphasis gradually shifting on to ‘Acid Rain VIP (Breakage Final Chapter Mix)’ via a gradual crossfade and various rhythmic cutting effects on the mixer. During this track, we see various deformations of the 3D sculpture as it squashes and flies apart in correspondence with relentlessly deconstructed breakbeats, until 27:31 when another abrupt deck stop is used to transition to the final track. ‘Message in Bottle’ provides an irreverent finale, combining raucous punk rock with various collisions of psychedelic visual noise and sensory overload. As the track runs out, an audio-visual one-shot adds the ‘soundcat’ stamp and brings the DJ/VJ mix to a close. The Soundcat performance was well received at the VJ London concert, which also featured sets by Fabu, High Files, and Warehouse Animals. As noted, the approach of creating original music videos allows improvisational visual processes to take place in the studio, while the final performance shifts the emphasis on to the musical aspects. This allows each performance to be adapted by reorganising tracks spontaneously in the live situation, much as a DJ would do when adapting to the energy of an audience. Similarly, my performance can also be adapted or expanded by adding new ‘records’ (i.e. making new music videos). In a later performance at the Tˆy Pawb arts centre (Wrexham, 12 September 2018), I added four new videos using hard house, hard trance, and acid techno music. These included plasma visuals, direct animation, 3D geometry, funnels of spheres, horror movie clips, satirical animations, and more. Figure 6.10 shows a still from one of these videos, which was made to accompany the acid techno track ‘One Consciousness (Dynamic Intervention Mix)’ by Acid Ted (2006). Adding new tracks in this way allows the performance to evolve over time, and gives space to try out different ideas. More recently, a version of this set was also performed online at Art Futura (London, 17 April 2021).

6 Optical Geometry: VJ Performances

Fig. 6.10

155

Still from the ‘One Consciousness’ video

Visual Artworks My DJ/VJ performance as Soundcat developed from the work described earlier in this book, while also exploring some new directions. Alongside working on this project, I continued to compose various other works of electronic music and visual art. I have often worked on visual art in parallel with music, sometimes incorporating this into audio-visual works or using it as record sleeve artwork, as noted earlier for the Entoptic Phenomena in Audio vinyl (Weinel 2014; Chapter 2, Fig. 2.3, p. 56); or the Flood City (2015) 10 dubplate (acetate) records by Teknoshaman (2015), each of which featured unique hand-produced artwork. In order to further consider how work can traverse the boundaries between the sonic and the visual, in this section I will discuss some visual artworks I produced during this period, thereby illustrating the broader context from which the VJ work emerges. These synaesthetic visual artworks can be considered as complementary pieces, which inform the development of, and respond to, my VJ work, providing space to experiment with different symbolic forms and associations that can be formed between sounds and images.

156

J. Weinel

Synaesthetic Paintings Extending my explorations of Klüver’s (1971) form constants, Fig. 6.11 shows Vortex (2017), an acrylic painting that represents a tunnel impression, such as might be seen during a hallucination. While this painting is closely related to formal concepts of ASCs, with later visual artworks

Fig. 6.11

Vortex, acrylic on canvas, 39.7 × 49.8 cm

6 Optical Geometry: VJ Performances

157

I moved away from basing the designs on research regarding features of hallucinations, instead exploring imagery related to the symbolic visual languages associated with different styles of music. This led to a variety of paintings made in response to music using acrylic paint, airbrush, digitally cut stencils, and collaging of printed digital materials. Trip at the Brain (2017, Fig. 6.12) is based on a song of the same name by the hardcore punk band Suicidal Tendencies, and interprets the music as an ink pen sketch, which was then digitised, used to cut a stencil with a plotter machine, and rendered with an airbrush. The style of the piece draws upon the visual language associated with this style of music, as seen in various artwork created for the band by Ric Clayton (2018); and the aerosol artwork created by the band Excel (Ross and Clements 2018).

Fig. 6.12

Trip at the Brain, airbrush on paper, 29.7 × 29.7 cm

158

J. Weinel

31 Seconds (2017, Fig. 6.13) is one of several works that explores the use of typographic references to music. The airbrushed text ‘31 seconds’ references a sample used in the drum & bass track ‘Valley of the Shadows’ by Origin Unknown (1996). This was placed over an acrylic flow24

Fig. 6.13. 31 Seconds, acrylic on canvas, 25.4 × 30.5 cm

6 Optical Geometry: VJ Performances

159

painting with colours suggestive of the rainforest, thereby exploring tropical associations with jungle/drum & bass, as with Tiny Jungle (2010, see Chapter 4). Red ‘cryptic messages’ motifs are also used, which draw upon the style of visual art created by the Spiral Tribe rave collective.25 Bug Powder Dust (2017, Fig. 6.14) also uses text cut from a newspaper,

Fig. 6.14

Bug Powder Dust, acrylic and collage on canvas, 25.4 × 30.5 cm

160

J. Weinel

referencing a track of the same title by Bomb the Bass featuring Justin Warfield (1995). In this case, acrylic flow techniques and airbrushed stencilling were used to provide a cityscape with skeletons flying above. The gothic impression of flying skeletal figures draws influence from the ‘x-ray’ figures of William Burroughs’s ’shotgun paintings’ (Riflemaker 2005), in correspondence with the Bomb the Bass song, which is based on the Burroughs (1959) novel The Naked Lunch. Seasons in the Abyss (2017, Fig. 6.15) is based on the album of the same name by the thrash metal band Slayer. Acrylic flow techniques were used to create the main background for this painting, providing abstract white and dark reddish-brown shapes, from which it is possible to perceive forms in the manner one might do with a Rorschach painting. In this sense the painting may unlock the unconscious in the manner of surrealism, perhaps allowing one to see various screaming faces that reflect the hellish themes of Slayer’s Seasons in the Abyss (1990). On top of the abstract images, various red designs are rendered, which may suggest ‘cryptic messages’ of an occult nature. Holo Point Break (2018, Fig. 6.16) uses a multilayered approach that bears comparison with my VJ productions. The background layer is created with red and purple acrylic flow painting techniques, over the top of which we see wireframe geometric shapes, audio waveforms, and other fragmented elements. This provides a visual structure similar to my VJ productions, where shapes, waveforms, and other fragments are superimposed over abstract psychedelic textures. ‘Holo Point Break’ is one of several paintings that take inspiration from street art, such as the work of New York graffiti and hip-hop artist Rammellzee,26 whose work crossed boundaries between the sonic and the visual. The breakbeat hardcore discussed in this chapter is interwoven with hip-hop culture through the producers’ use of breakbeats and hip-hop samples, and so it is logical to explore visual connections of street art when visualising these musical forms.

Augmented Reality VJ Paintings Media 6.3 AR Paintings demonstration video, 53 seconds

6 Optical Geometry: VJ Performances

Fig. 6.15

161

Seasons in the Abyss, acrylic on canvas, 30.5 × 40.6 cm

Many of my paintings explore visual ideas of motifs that were later developed into VJ materials or vice versa. In three of my paintings: Enter Soundcat (2017), Soundcat 2000 (2017), and Soundcat S-101 (2017), I drew connections with VJing by integrating VJ loops, which can be

162

J. Weinel

Fig. 6.16

Holo Point Break, acrylic and collage on canvas, 50.8 × 76.2 cm

6 Optical Geometry: VJ Performances

163

activated using AR, when viewing the paintings through a mobile phone (Media 6.3). Enter Soundcat (Fig. 6.17) uses a background with yellow and red acrylic flow painting and torn newspapers. Hardcore punk imagery is placed over this background, including ‘soundcat’ written as a détournement of the Suicidal Tendencies band logo, and a mannequin head, which references the album artwork for The Joke’s on You by Excel (1989), which the song ‘Message in a Bottle’, discussed earlier, was taken from. The piece also includes typographic ‘cryptic messages’ related to visual hallucinations. Three rectangular prints of stills from my VJ loops are also used as collage elements, which are brought to life when viewed through AR, thereby integrating moving images into the painting. Soundcat 2000 (Fig. 6.18) takes its name from the Rainbow 2000 electronic dance music festival in Japan (1996, see Masaaki Kobari 2012), and the logo is a détournement of a design created for that festival by The Designers Republic.27 This element associates the painting with 1990s rave culture in Japan, and still images of VJ loops are included as collage elements, which are animated when viewed through an AR device. As before, these are placed over an acrylic flow background, and various other designs related to visual hallucinations are laid on top of this, including waves of triangles programmed in Processing, which were rendered in orange neon using digitally cut stencils and an airbrush. Soundcat S-101 (Fig. 6.19) also uses an acrylic flow painting background, over which the title of the painting is rendered in text. This text uses the same typography as the titles for the film Terminator 2: Judgement Day (Cameron 1991), and references the Cyberdyne Systems Model 101 terminator from the movie. The cinematic reference used in this painting is also a musical one, since science-fiction movies were widely sampled in 1990s hardcore rave tracks, such as Terminator by Metal Heads (1993). These visual elements can therefore be understood as a form of ‘visual sampling’ analogous to the use of audio sampling of film quotations in drum & bass tracks, like The Terminator. The film is also referenced through the use of airbrushed bullet holes across the painting, and a red wireframe animation of a Neural Net CPU (the Terminator’s CPU in the film) exploding. Beneath this, another image shows green ‘cryptic messages’, which were taken from an animated

164

J. Weinel

Fig. 6.17

Enter Soundcat, acrylic and collage on canvas, 30.5 × 40.6 cm

6 Optical Geometry: VJ Performances

Fig. 6.18

165

Soundcat 2000, acrylic and collage on canvas, 30.5 × 40.6 cm

Processing sketch.28 Both the disintegrating computer chip and the cryptic messages are AR elements. The printed still images from these two videos have painted cracks in them, suggesting damaged screens. The cracking motif is also used on the printed still images of ‘Enter

166

J. Weinel

Fig. 6.19

Soundcat S-101, acrylic and collage on canvas, 30.5 × 40.6 cm

6 Optical Geometry: VJ Performances

167

Soundcat’, and is intended to suggest fractures between the layers of painting, AR VJ materials, and conscious perception. The synaesthetic paintings discussed in this section were created in response to music, and were often made while simultaneously listening to music. The aim of this creative process is to channel the multimodal, affective, and associative properties of sound into visual forms. Using this approach leads to paintings that represent the types of imagery that are related to the idea of what one might see during synaesthetic soundto-image hallucinations. Each piece weaves together intricate symbolic visual references that correspond with music. These are not always rigidly controlled or determined in relation to individual pieces of music, but rather, motifs and approaches are developed, and allowed to bleed between the various paintings. This bleeding between works is cultivated beyond the paintings into the VJ work, so that the VJ loops are informed by the paintings and vice versa. The integration of visual art and VJ performance was also explicitly explored by incorporating VJ loops as AR elements in three of the paintings. ∗ ∗ ∗ Emerging from the psychedelic lightshows and electronic dance music culture of the late twentieth century, VJing is a dynamic and diverse form of performance that centres on visual representations of sound and music. This chapter has provided a detailed discussion of my personal explorations in this area,29 composing psychedelic visualisations through VJ performances, and paintings in response to music such as hardcore rave and punk. While these practices continue to utilise various forms based on specific reports of ASCs, the projects discussed in this chapter represent a shift towards basing the work on the visual languages associated with music. By mining the visual culture that surrounds music, it is possible to draw out the symbolic associations that one might have when listening, and use these to inform the design of imagery. The resulting compositions can be understood as psychedelic visualisations of sound, because they manifest synaesthetic visual imagery, making concrete the imaginary impressions or sound-to-image hallucinations that one might

168

J. Weinel

have in response to music. Yet just as languages evolve over time as they are continually used and revised, the language of VJing may not only restate existing visual associations, but also redefine them. In doing so, the VJ channels music, to reform and renew our multimodal visual interpretations of it, thereby eliciting the shape of synaesthetic hallucinations to come.

Notes 1. For a further discussion of VJ projections at Mo:Dem festival, see also Weinel (2018d, p. 131). 2. The audio-visual performing arts festival Splice is also discussed in Weinel (2018). 3. For a further discussion of the affective and representational functions of VJ performances in relation to music, see Weinel (2018d). Symbolic correspondences are also considered in Weinel (2020). 4. Breakbeat hardcore music (also ‘hardcore rave’, ‘old skool rave’, or ‘UK hardcore’) is a form of rave music popularised in the 1990s by artists such as The Prodigy and others, which is based around sped-up drum breaks sampled from funk and hip-hop tracks. For a further discussion see Weinel (2018c, pp. 86–87). 5. Many demoscene videos are available online, for a classic example see Future Crew’s ‘Second Reality’ (DemosceneVids 2015). 6. The VJ Loops demonstration video is provided for educational purposes only; please do not use these video clips in your own VJ performances without permission. 7. In computer graphics is a sprite in a two-dimensional bitmap image, as commonly used in video games to represent moving characters. 8. Pixel art is a style of video game art in which graphics are designed and edited at the pixel level. 9. This slightly unusual setup was used partly for convenience, since at the time I was regularly flying back and forth from Denmark, and the portability of this setup allowed me to work on animations while on the move. 10. For a technical description of how to program plasma effects, see Vandevenne (2004).

6 Optical Geometry: VJ Performances

169

11. HAP is a video codec for Mac OS X, which performs image decompression on the computer’s video card, thereby reducing the CPU usage when playing back the videos. At the time this was the preferred video codec to use for VJ performance in VDMX. 12. For examples and discussion of 1990s rave flyers and other imagery, see Savage (1996), Berlin (2018), and Tomlin (2020). 13. In this section, ‘representational properties’ refers to features of sound and audio-visual media that represent spatial locations, places, events, or concepts; while ‘affective properties’ communicate mood and emotion (see Weinel 2018c). For a discussion regarding the affective properties of motion graphics, see Bartram and Nakatani (2010). 14. For an example of one of my practice mixes, see my unofficial VJ mix for Paul Oakenfold’s ‘Goa Mix’ radio DJ set from 1994 (Soundcat VJ 2018). 15. VJ London (http://vjlondon.com/) are a London-based VJ collective. 16. Discordianism is a philosophical movement sometimes considered a parody religion, which is based on the worship of Eris, the goddess of chaos in ancient Greek mythology (see Hill and Thornley 1994). Both the apple and the number 23 are discordian symbols, and the latter has been used in rave culture by the Spiral Tribe collective. In an interview with Mark Harrison of Spiral Tribe, he attributes use of the number 23 to its significance as an ‘anti-icon icon’ (Transpontine and Harrison 2013). 17. The term ‘one-shot’ is borrowed from electronic music production, where ‘one-shot’ audio-visual samples are short sampled sounds that are not looped. In the context of this chapter ‘one-shot’ is used to describe short non-looping video samples. 18. The approach described here, whereby improvisational video mixing is undertaken in the studio, and the materials are iteratively reprocessed, was partly inspired by the studio techniques used by dub reggae artists. For example, Lee ‘Scratch’ Perry recorded his dub mixes by repeatedly mixing down (or ‘bouncing’) the tracks on four-track and two-track tape recorders in order to add more elements (Katz 2006, pp. 175, 330; see also Weinel, 2018c, pp. 78–79). 19. Video mashup is a style of audio-visual performance in which video music is constructed through rhythmic collaging of audio-visual samples. For example, see the music videos Timber by Coldcut and Hextatic (1997) or ‘The Wolf of Wall Street (Eclectic Method Chest Thump Mix)’ by Eclectic Method (2014). 20. The approach used in this video draws influence from the Autechre Gantz Graf video by Alex Rutterford (Autechre and Rutterford 2002).

170

21.

22.

23.

24.

25. 26. 27.

28.

29.

J. Weinel

Rutterford says his work on this video was partly inspired by geometric hallucinations seen on LSD (Kilroy and Rutterford 2010). This timestretching effect was made with the Akaizer (http://the-aka izer-project.blogspot.com/) application, which simulates the timestretching features of popular Akai samplers such as the S950/S1000/S2000/S3000 series, which were widely used in hardcore rave music during the 1990s. This section attempts to replicate scratch-DJ techniques, but this is difficult with the Akai AMX mixer and could perhaps be improved by exploring the use of alternative controllers such as turntables with control vinyl. DJs often move between tempos by mixing tracks that are half or double the tempo of each other. For an example, see DJ Food & DK’s (2001) transition between ‘Mirror in the Bathroom’ by The Beat and ‘Square Off ’ by Mask, on the Solid Steel—Now Listen mix. Acrylic flow painting (or ‘acrylic pouring’) is a technique where additives are mixed with acrylic paint to improve the flow properties of the paint. Multiple colours of paint can then be poured on to a surface resulting in interesting colourful patterns similar to those produced by marbling techniques. For example, see rave flyers for Spiral Tribe events and related sound systems, as documented in Seana Gavin’s visual diary Spiralled (2020). The work of Rammellzee was exhibited at Rammellzee: A Roll of the Dice (Laz Inc. 2018). The Designers Republic (https://www.thedesignersrepublic.com/) produced various graphic designs for electronic dance music artists, events, and record labels such as Warp. These designs draw upon approaches for designing abstract algorithmic computer text and music notation that are explored in Manfred Mohr’s computer artworks such as P-021 (1970–1976). This and other related works are included in the V&A’s collection of computer art. Of course, it should be acknowledged that this chapter reflects my own personal journey, and other VJs may use entirely different approaches and workflows that are equally valid. The discussion in this chapter should in no way be taken as a definitive ‘guide’ to VJing.

7 Future Sound Dream: Virtual Reality Experiences

Rotterdam, 1995. Twin brothers in sportswear with shaven heads are thrown out of a gabber rave in an old country mansion.1 While they ponder how to get back into the club without having any money, a young researcher invites them to participate in a sleep study, with the promise of one thousand gilders. Upon arriving at the research laboratory, the twins are strapped to beds and fitted with brain-computer interfaces, a torrent of wires flowing from their heads into various monitoring equipment. A hypnotic cassette plays, while behind a one-way mirror, the researchers begin to perform electronic music using keyboards, modular synthesisers, and computers running sequencer programmes and code. One of the performers plays an instrument that resembles a cross between an oboe and a Roland TR-909 drum machine, where the traditional keys of the reed instrument are augmented with a variety of switches, buttons, LEDs, and a trackball. As the performers begin to play, the trance-like Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/978-981-16-4055-1_7. Where the icon is shown in the chapter, the reader should refer to the supporting media files. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 J. Weinel, Explosions in the Mind, Palgrave Studies in Sound, https://doi.org/10.1007/978-981-16-4055-1_7

171

172

J. Weinel

melodies draw the twins into an altered state of consciousness (ASC). At once, they find themselves gliding across a glittering surface of artificial liquid in their Nike Air Max 90s. Drifting through the jaw-like doors of a virtual gabber club, they ascend a staircase of piano keys as the music accelerates and intensifies, and find themselves inside a synaesthetic hallucination where sound, melody, and visuals are one. These are the fictitious events that unfold in the animated short Culturesport: Rotterdam 1995 (Boling 2019), which is set in an alternate Dutch society, where rising water levels are driving technological innovations. Aspects of the narrative extend the imaginary, high-tech worlds depicted in actual electronic dance music culture of the 1990s. Looking through rave flyers from this era for events like Fantazia or Dreamscape (pastiche posters of which can be seen in the twins’ bedroom in Culturesport: Rotterdam 1995 ) reveals various techno-utopian and dystopian imagery. For example, on the surrealistic Fantazia New Years Eve 1991/1992 flyer, a woman gazes across a virtual reality (VR) landscape as moons erupt from her head, while the Dreamscape December 1991 flyer shows an artificial intelligence emerging from a 3D mesh. Like science-fiction movies such as Alien (Scott 1979), Blade Runner (Scott 1982), and The Terminator (Cameron 1984), which were often mined by music producers for audio samples,2 hardcore rave culture from the 1990s depicted the awesome, empowering potential of technologies, but also the dangers and dystopias that could arise from them. The music also explored these themes sonically, constructing portals into technologically enhanced worlds and paranoid cyberpunk nightmares, as heard on tracks like ‘Dreaming of a Better World’ by Exit EEE (1993) or ‘Kemistry’ by Metal Heads (on Terminator, 1992), in which rays of euphoric light filter through the cracks of tense kinetic breakbeats, anxious minor key melodies, and abrasive industrial textures. Mirroring these themes, in Culturesport: Rotterdam 1995 one of the rave posters reads ‘Future Shock’, referencing Alvin Toffler’s (1970) book on postindustrial societies,3 and the episode as a whole addresses speculative technologies that are both fantastical and disturbing. As a work of science fiction, Culturesport: Rotterdam 1995 imagines a speculative world in which the capabilities of the real-world technologies

7 Future Sound Dream: Virtual Reality Experiences

173

we have now are extended. For example, the cybernetic music hallucination sequence in the story is actually based on various real-world equipment, such as keyboards, modular synthesisers, electronic reed instruments, computer sequencers, biofeedback technologies, and music visualisations. Although these technologies haven’t yet been combined to generate synaesthetic hallucinations in quite the way that the episode suggests, they soon could be. Indeed, in Explosions in the Mind we have already looked at psychedelic electronic music; computer music systems that augment acoustic instruments; synaesthetic music visualisations; biofeedback-driven simulations; and VJ performances. It is not so difficult to imagine these forms being extended to provide immersive audio-visual experiences of music that surround and engulf individuals, giving them a feeling of presence as they drift through glistening virtual landscapes of sound and image. Moving towards the design of such immersive music visualisations, in this chapter I will discuss Cyberdream (2019–2020), a VR experience that extends the concepts of VJ performance described in the previous chapter, placing these compositional forms in an immersive, interactive context, which allows users to fly through synaesthetic virtual worlds of electronic music. First, we shall look at the initial iteration of this project for the Oculus GearVR headset, which provided a series of symbolic virtual environments accompanied by fragments of hardcore rave and vaporwave music. Following this, we will examine a later iteration of the project for the Oculus Quest, which provides various improvements, allowing a more seamless journey in which the user can also create parts of the experience by using the controllers to ‘paint with sound’. Through the discussion of Cyberdream, in this chapter, we shall see how electronic music, creative coding, VJing, and VR can be brought together to compose psychedelic visualisations of sound.

174

J. Weinel

Dreaming in Cyberspace Media 7.1 Cyberdream GearVR demonstration video, 4 minutes 57 seconds and software Cyberdream extends the approaches of VJ performance, bringing them into the domain of immersive technologies. While VJing allows audiences to watch visualisations of music via rectangular projections, VR holds the potential for immersive experiences that surround and completely engulf the user. The feeling of ‘being there’ in a virtual space is known as ‘presence’ (Slater and Wilbur 1997). Where it is provided, presence in VR may allow users to feel as though they are actually there inside psychedelic visualisations of music. Exploring this idea practically, Cyberdream aims to provide an immersive rave music and vaporwave experience for the Oculus GearVR. Conceptually Cyberdream incorporates many of the ideas discussed in the previous chapter, emerging as a visual synthesis of hardcore rave music, vaporwave, and 1990s-style VJ visuals. As discussed, the visual images associated with hardcore rave music in the 1990s were surrealistic techno-utopian and dystopian visions. We see these images in the various rave flyers of the era or VJ mixes.4 These images were often developed using various combinations of airbrushed art and/or computer graphics, which though cutting-edge at the time, may now seem primitive relative to newer forms of 3D rendering. The aesthetics of these visual images have more recently been revisited by vaporwave, an Internet-borne music genre emerging in the early 2010s. Sonically vaporwave uses loops of 1980s and 1990s corporate lounge music, advertisements, and banal pop as source materials,5 creating a soundtrack that is deeply nostalgic for the capitalist optimism of this period. Yet the loops are slowed down, repeating ad infinitum, and warped as if playing from an old cassette player in a broken-down hotel elevator. According to Tanner’s (2016) discussion, these flawed representations reflect a critical view of the capitalist excesses of this period. Drawing on Mark Fisher’s (2014) concept of ‘hauntology’, which suggests that the present day is haunted by lost futures that were once imagined, Tanner argues that vaporwave exposes the broken mechanisms

7 Future Sound Dream: Virtual Reality Experiences

175

of these capitalist illusions. Vaporwave is the sound of abandoned shopping malls filled with plastic palm trees, haunted by the sound of Toto’s ‘Africa’ (1982)6 on endless repeat. As a music genre and Internet meme transmitted on platforms like YouTube and Bandcamp, vaporwave is also usually accompanied by visual expressions of these themes such as video and album covers. In correspondence with the music, the visual style of vaporwave mines images of 1980s and 1990s shopping malls, skyscraper restaurants, and commercials depicting fantasies of tropical holidays, business trips, or home living enhanced by the latest appliances. In their original contexts, these alluring images might once have suggested the promise of fulfilling exotic lifestyles that could be obtained by purchasing products and services for business and leisure. Yet in vaporwave, these images are conspicuously dated and broken, rendered as fuzzy VHS recordings being played on analogue video recorders with bad tracking. As in the music, these images give a feeling of warmth and nostalgia, but also reveal the illusory, broken mechanisms behind them; like torn and faded billboards advertising tropical holidays in an apocalypse, what they represent is distant and unattainable, existing only in a memory of a dream. Vaporwave also draws extensively on computer graphics from the 1980s and 1990s, incorporating ray-traced images of statues from Ancient Greece, primitive 3D shapes, dolphins, and tropical palms. Like 1990s rave visuals, these images are suggestive of exotic technological utopias. In the 1990s, images like these were seen in the promotional graphics associated with software like Microsoft’s Encarta ’95 multimedia encyclopaedia, where leaping dolphins symbolised intelligence and ecological perfection.7 Vaporwave uses symbols like these extensively,8 yet images of the computer user interface are also frequently included, such as desktop elements or cursors from the Microsoft Windows 95 or Mac OS 7 operating systems. For example, one YouTube video offering a ‘1½ Hour Vaporwave Mix’ (Hany Tarek 2016) features a striking animation of a scene from a billion-dollar waterfront mansion, with a view gazing outwards across an impossibly pink tropical ocean. Yet the image is framed by the user interface from Microsoft Paint, the default graphics-editing package included with Windows 95. Showing the user interface of the graphics-editing software lays bare the artificial

176

J. Weinel

construction of the image; it shows us that the exotic illusion is not real, it is synthetic and computer generated. Vaporwave is a utopian vision of cyberspace rendered in low-polygon 3D, where an error in drawing gives rise to an infinite trail of replicated desktop cursors. Drawing these ideas together, Cyberdream is a virtual hallucination through the broken techno-utopias of cyberspace, set to a soundtrack of hardcore rave and vaporwave music. The project provides a journey through a series of symbolic VR scenes that draw upon the visual languages of these music genres in order to provide synaesthetic 3D spaces that correspond with the music. The scenes are dystopian in that they represent broken techno-utopian vistas in cyberspace, but they can also be read as euphoric deconstructions, where these fragments once liberated from their formal constraints become a playground of new possibilities. In what follows, I will discuss the design of the music and these 3D environments as they appear on the original version of Cyberdream for the Oculus GearVR.

Constructing the Soundtrack The soundtrack of Cyberdream consists of various electronic dance music and vaporwave compositions. The electronic dance music compositions were composed in Renoise, using a range of synthesisers and plugins. The styles range across various forms of hardcore techno and electro, using arrangements of rhythms, basslines, and synthesisers. The menu music incorporates synthesiser trance sounds and sampled materials from rave tapes, including an MC voice which says ‘if you look over there you’ll see there’s no DJ’. The latter sample reflects the hauntological theme described earlier—the VR experience is a memory or dream of a bygone era, and there are not really any DJs here, only ghosts. The vaporwave pieces are not original compositions, but instead are sonic extracts made with a plunderphonics approach (Oswald 2004), as is typically the case for most vaporwave music. The loops were taken from library music CDs, which provide stock music tracks intended for use on advertisements, corporate training videos, or other promotional films (Hollander 2018). Library music is appropriate source material for

7 Future Sound Dream: Virtual Reality Experiences

177

vaporwave. Library music CDs from the 1990s published by companies such as De Wolfe Music, Bruton Music, and Chappell Recorded Music Library have titles such as Neutral Atmospheres (Sinclair 1994), Success and Achievement (Scott et al. 1996), and Cyberscience (Wilson and Ransom 1996). The track titles reflect possible uses and sentiments, with titles like ‘World Class’, ‘European Partners’, and ‘Making the Deadline’ (Kiddy 1994). These discs can be understood as tools for designing the sonic landscape of the intensely capitalist 1980s–1990s era that vaporwave is concerned with, allowing companies to engineer the emotions of workers and consumers to increase positive feelings and productivity. Sonically the music was created by expert composers using samplers and synthesisers of the day, lending the music a clear, hi-fidelity quality of the type explored on vaporwave releases such as New Dreams Ltd. by Laserdisc Visions (2011). The album artwork of these CDs also corresponds with the types of designs that vaporwave references; for instance, Daytime Television (Kiddy 1996) uses gold 3D text and words like ‘fun’, ‘cookery’, and ‘relax’ over grinning faces and consumer products. The vaporwave tracks used on Cyberdream were made by taking samples from various library music, looping them, slowing them down, and applying various digital signal processing (DSP) effects.

Symbolic Environments The first version of Cyberdream was designed for the Oculus GearVR, an untethered VR solution that runs on a mobile phone placed inside a headset. As this device provides only limited controller facilities, Cyberdream was designed to provide an automated journey, in which the user flies through a series of synaesthetic environments based around the rave and vaporwave concepts described (Fig. 7.1). The project was created in the Unity video game engine. The menu screen of Cyberdream is based on the Fantazia New Years Eve 1991/1992 rave flyer described earlier. A giant face hovers above a wireframe 3D landscape beneath a pink sky. This landscape was technically constructed in Cinema4D and rendered as a skybox.9 At the top of the screen, titles and instructions tell the player that they can begin

178

J. Weinel

Fig. 7.1 Various still images showing scenes from Cyberdream GearVR

the experience by touching the button on the VR headset while looking at the face. The soundtrack for this scene includes the ambient trance composition (including the ‘if you look over there you’ll see there’s no DJ’ sample mentioned earlier). The next screen provides a green wireframe landscape with a bridge of tiles leading to a fractal pyramid of Sierpi´nski triangles (Fig. 7.1a).

7 Future Sound Dream: Virtual Reality Experiences

179

In this scene, the user flies across a bridge of oscillating tiles, which were created with a C# script that modifies the size of the tiles using a sine-wave equation. The location of the tiles changes the phase of the waveform, thereby producing a wave effect. Statues of mysterious entities are situated at either side of the bridge on top of checkerboards. The music in this scene is an acid techno/hard house track. As the camera flies into the pyramid, the scene transitions to an infinity pool hovering in the sky (Fig. 7.1b). Giant mannequin heads float in the pool, staring blankly into space. These symbols reference the vaporwave tropes of exotic capitalist lifestyles and high fashionistas. As we fly over the pool we hear a vaporwave track, before the camera disappears into the eye of one of the heads. The next scene revisits ideas developed through my VJ work discussed in the previous chapter (Fig. 7.1c). The scene is based on a handpainted background, which was made by digitally scanning a painting and manipulating it to produce a skybox. In this scene, a C# script is used to animate objects in spiral patterns that draw transparent trails behind them. This provides psychedelic patterns that relate to Klüver’s (1971) visual patterns of hallucination. Musically the scene is accompanied by a track that draws on instrumental styles of UK garage and grime.10 The scene fades into a field of pop-up computer windows suspended in a clear blue sky (Fig. 7.1d). This scene develops the same idea used in the ‘they live’ VJ visual described in Chapter 6, which was based on the capitalist advertisements depicted in the science-fiction movie They Live (Carpenter 1988). Pop-ups in the style of Windows 95 read ‘work’, ‘buy’, ‘watch TV’, ‘obey’, ‘consume’, ‘no thought’, ‘update status’, and ‘take selfies’. These windows open and close, an effect technically accomplished with a C# script, which modulates the size of the windows using float values that change following a sawtooth waveform. An easter egg11 in this scene is a window with text that references the cyberpunk movie Johnny Mnemonic (Longo 1995). The scene includes a techno soundtrack. Following this, we fly across a landscape of purple checkerboard mountains and Grecian statues (Fig. 7.1e). Giant 3D cursors rain from the sky, flickering black and white. These are based on the pixel art

180

J. Weinel

default cursor of the Atari ST operating system, and the colour oscillations are generated with square wave values. The cursors bounce around the scene chaotically, falling into the sea, where broken statues lie with their heads bowed mournfully. The music in this scene is a breakbeat hardcore track using ‘hoover’12 synthesiser sounds. The next scene finds us suspended in an artificial blue sky once again, surrounded by waves of brightly coloured cubes which flow across the screen. Symbolically this scene references the Windows 95 artwork, and is based on the idea of being inside a computer screen, where the individual red, green, and blue (RGB) pixels become enlarged. The soundtrack is one of the vaporwave tracks. This is one of the most striking scenes in Cyberdream, which users have often remarked upon during demonstrations of the piece. An aspect that seems to be particularly effective is the way in which the cubes move through where the body would be, giving the feeling of being waist-deep in digital waves. This seems to create a subtle physical sensation and slightly disorientating vestibular effect, which triggers traces of the sensations that one might experience standing in an actual sea at the beach. Technically the oscillating movement of the cubes is once again achieved with a C# script that generates sine-wave values, where phase is offset based on the 3D coordinates of each cube. This script uses combinations of sine-waves to change the size of the cubes, while also modifying the RGB colour values of them, creating a plasma effect. The scene after this is a variation of the previous one, in which the user is suspended in a room in the sky made of oscillating cubes (Fig. 7.1f ). As in the previous scene, the user does not move, but can look around and observe the wave patterns. The soundtrack consists of a techno beat that incorporates vaporwave samples via a kitsch synthesiser bell and a pitched-down vocal sample. With another transition, the scene then fades into a reddish room with a hand-painted background (Fig. 7.1g). As before, this was created by digitally scanning hand-drawn artwork. In this scene, metallic spheres move following Lissajous curves, which are generated using combinations of sine-wave values that move the spheres. Trail effects allow the spheres to draw arcs of electric blue across the room, providing a similar

7 Future Sound Dream: Virtual Reality Experiences

181

red/blue colour scheme to that used in Holo Point Break (Chapter 6, Fig. 6.16, p. 178). The design of this scene takes inspiration from demo effects (Polgár 2005), which often feature arrays of animated spheres and other 3D objects. Another fragment of vaporwave music provides the soundtrack for this scene. The final scene plunges us into a black void surrounded by stroboscopic arrows that move around maniacally (Fig. 7.1h). This aims to provide a feeling of visual noise and sensory overload, both through the flickering arrows, and the music, which consists of speedcore techno rhythms and distorted acid synthesiser sounds. Text suspended in this room reads ‘the future is lost, crash the system, back to the tribes’. Concluding the piece, this text can be read as a comment on the loss of a techno-utopian future that was once imagined, and a call to break down the digital structures of society and reformulate them in new ways. The comment ‘back to the tribes’ hints at the idea of technoshamanism,13 while also referencing free-party rave culture (e.g. Spiral Tribe). The first iteration of Cyberdream was a short audio-visual composition/software application for Oculus GearVR, which lasted approximately 5 minutes, taking the user on a synaesthetic journey through hardcore rave and vaporwave music. These musical forms were not visualised using the typical approach we might expect in a music visualisation, where the acoustic signal is used to generate moving patterns of light. Instead, the piece provides a symbolic psychedelic visualisation of music, allowing the user to feel a sense of presence inside 3D environments that correspond with the images and meanings inherent in these musical forms. A variation of the piece was also produced for VR cardboard,14 and it was demonstrated at a variety of events in 2019, including Cyberdelics Incubator Melbourne (Australia, 24 March 2019); the VR programme of Sci-Fi London film festival; the Event Two exhibition of computer art held at the Royal College of Art; and Audio Mostly 2019 at the University of Nottingham, where the demonstration received an award.

182

J. Weinel

Painting the Sound Dream Media 7.2 Cyberdream Oculus Quest demonstration video, 4 minutes 40 seconds and software The original Cyberdream was well received, however, despite being portable and convenient, the Oculus GearVR has a variety of technical limitations, and lacks the full interactive and immersive capabilities of other current VR systems. In order to address these limitations, in 2020 a revised version of Cyberdream was created for the Oculus Quest with the aim of providing improved interactivity and a higher quality VR experience.15 Building on the Unity project described in the previous section, the newer version of Cyberdream includes a revised soundtrack, which provides a more continuous experience between the 3D scenes that more closely resembles a DJ/VJ set. In addition, it also provides three interactive ‘sound toys’, which allow the user to interactively ‘paint with sound’ and intuitively create some aspects of the audio-visual experience.

Revising the Soundtrack One of the original ideas behind Cyberdream was the concept of creating a continuous audio-visual experience analogous to a DJ/VJ set in VR. The idea was to create an experience where each audio-visual scene would have its own musical soundtrack and be analogous to a record that a DJ would mix. Yet each record would have a synaesthetic visual component, and be rendered as a 3D space in VR, allowing users to feel as though they were actually ‘inside the music’. Following this metaphor, the project would be like a VR equivalent of a rave ‘tape pack’ such as the packs produced by Fantazia or Dreamscape. I was particularly interested in capturing the chaotic energy of Carl Cox’s DJ mixes (e.g. Fantazia: The Big Bang, 1993), where he rapidly blends tracks in quick succession across three turntables, making abrupt cuts between them. Although the original Cyberdream was partly successful in this regard, in the transitions between scenes the music simply fades out before the next track fades in, rather than cross-fading tracks in sync, in the manner of a DJ mix. To improve the sonic continuity between scenes, the revised

183

7 Future Sound Dream: Virtual Reality Experiences

Track 1

Track 2

Track 3

...

...

Fig. 7.2 Diagram showing the structure of Cyberdream, which resembles the form of a DJ/VJ mix with crossfades

version of Cyberdream provides a new soundtrack, which was implemented using the Wwise interactive engine for game audio. As shown in Fig. 7.2, this allows tracks to crossfade between scenes in the style of a DJ mix. While some of the same tracks from the earlier version of Cyberdream were used, a variety of new compositions were created at 140 beats per minute, using typical approaches of early 1990s breakbeat hardcore and ambient techno. Most pieces utilised sampled breakbeats and various Roland synthesiser sounds appropriate to the era. These were composed in parallel with the ‘sound toys’ that will be discussed shortly, thereby ensuring that both the background tracks and ‘sound toys’ work together harmoniously. As before, each track was paired with a different VR scene, so that the experience fades between music and visuals simultaneously. An additional feature in the project allows the order of these scenes to be randomised, thereby generating a different mix each time.

Painting with Sound The macro compositional structure of Cyberdream emerges from the series of symbolic VR scenes and electronic music, which unfold over the full duration of the experience. For the Oculus Quest version of Cyberdream, these are contrasted with micro compositional features provided by interactive audio-visual ‘sound toys’. The user can manipulate these audio-visual ‘sound toys’ using the hand-held Oculus Touch controllers, in order to generate various micro sonic materials that are complementary to the macro soundtrack of electronic music. Each audio-visual

184

J. Weinel

‘sound toy’ generates micro visuals that correspond with the macro visuals provided through the sequence of symbolic environments in Cyberdream, allowing the user to ‘paint with sound’ sonically and visually. In the current iteration of the project, there are three audio-visual sound toys. Each sound toy is manipulated using the left and right Oculus Touch controllers, as shown in Fig. 7.3. The ‘index trigger’ on each controller activates the sound toy currently selected. The ‘hand trigger’ modifies the sound toy, initiating a secondary function if one is available. On the top of the controller, moving the position of the ‘thumbstick’ applies various effects to the sound toy, changing the way it sounds and looks. Lastly, the A/B and X/Y buttons cycle up and down through a list of available sound toys, thereby allowing different combinations of toys to be used with the left and right hands. ‘ZigZagToy’ (Fig. 7.4, right) emits an orange laser beam that twists and produces percussive pulses in synchronisation with the music. The rhythmic patterns that the toy can generate are stored in 16-bit binary sequences as shown in (Fig. 7.5). Much like a drum machine, each

Reserved (Oc

sound toy (down) Select soun Select sound toy (up)

Secondary sound to Trigger the sound d toy

Fig. 7.3 Controller configuration for Cyberdream Oculus Quest

7 Future Sound Dream: Virtual Reality Experiences

185

Fig. 7.4 Still showing the ‘ZigZagToy’ (right) and ‘StreamerToy’ (left) sound toys in operation

binary value represents a ‘step’ in a sequence, where ‘1’ plays a percussive pulse and ‘0’ does not. The percussive pulses are triggered in time with the electronic music soundtrack. This is technically achieved using ‘callbacks’ from the Wwise audio engine to the game, which give information about the timing of the music. The pulses of the sound toy are then triggered pre-emptively, compensating for latency so that they are perfectly in sync. Using the hand trigger switches between different pattern sequences, which also vary between the left and right controllers. This allows different interlocking patterns to be produced with the left and right sound toys. The sounds are also slightly offset temporally between the left and right sound toys, so pulses triggered simultaneously with both produce a thicker sound.16 The orange beam connects a series of points with randomised coordinates that update each time a percussive pulse is triggered, thereby causing the beam to twist with each pulse. The beam can be pointed in different directions with the controller, and the sound is spatialised relative to the 3D location. Manipulating the thumbstick changes the sound by introducing various DSP such as filters and pitch shifters, while also changing the width and colour of the beam in correspondence with these effects.

186

J. Weinel

Pattern A: 1000100010001000 = 34953

Pattern B: 0010001000100010 = 8738

Pattern C: 1111111111111111 = 65535

Pattern D: 1001001010010010 = 37522

Pattern E: 1010001010000101 = 41605

Fig. 7.5 Step sequencer patterns used by the ‘ZigZagToy’

‘CircleToy’ emits asychronous pulsing sounds from a green beam that moves in a rotating arc. When the sound toy is activated, pulsing sounds are triggered according to a regular timing system that is independent of the music. This sound toy is visualised with a green beam that flows from the controller, employing a visual effect created by changing the beam width at different points using a sine-wave. Using the thumbstick changes the speed and sound of the pulses, while also modifying the width and colour of the beam. The spatial location of the audio is based on where the beam is pointing in 3D space, and so moving the beams

7 Future Sound Dream: Virtual Reality Experiences

187

around allows the user to intuitively create interesting spatialisations of sound. Sonically this sound toy bears some comparison with the rapid pulsing sounds produced by the ‘atomizer’ module of the Atomizer Live Patch (Chapter 3). ‘StreamerToy’ (Fig. 7.4, left) emits atonal drones and yellow lightning, which follow the movement of the hand controllers. Sonically, this toy functions by triggering droning sounds with longer attack and decay times than the pulsing sounds of the other toys. The Wwise audio engine is used to randomise the pitch of these sounds in each instance, thereby producing an atonal effect. Visually, yellow streams extend along a trajectory that is modified by the position of the hand controllers, producing complex organic contours that resemble lightning. The spatialisation of the drone is based on the end points of each lightning beam. In this way, the spatial position of the sound and its corresponding visual representation follow hand gestures made with the controllers, allowing the user to ‘paint with sound’. Using the thumbstick also allows the user to change the sound, width, and colour of the beam. The droning sounds fulfil a similar function as the ‘drone machine’ module of the Atomizer Live Patch. In summary, the Oculus Quest version of Cyberdream builds on the symbolic journey through hardcore rave and vaporwave music of the earlier version, while extending and improving the project. In particular, the revised version provides a new soundtrack that focuses on breakbeat hardcore music and improves continuity between scenes by cross-fading between tracks. The resulting journey through symbolic visual scenes and electronic music is analogous to the form of a DJ/VJ mix, and provides the piece with its macro compositional structure. In contrast, the three audio-visual sound toys allow the users to playfully experiment and improvise micro compositional elements. These are also synaesthetic, demonstrating various configurations where the generation of sound produces corresponding visuals, allowing the user to interactively ‘paint with sound’. The macro and micro compositional elements are designed so that both sounds and visuals are complementary between the respective layers. It can be observed that the sonic approaches used are similar

188

J. Weinel

to those of the psychedelic electroacoustic compositions discussed earlier in the book, while the visual approaches are related to those used in my earlier VJ performances and paintings. In many ways then, Cyberdream draws together approaches discussed throughout Explosions in the Mind , in order to provide interactive and immersive psychedelic visualisations of sound. ∗ ∗ ∗ The exciting potential of VR for composing psychedelic visualisations of sound emerges from the capabilities of these technologies for immersing users into synaesthetic wonderlands where music and visuals are intrinsically related. When we think of visualisations of music, we often think of audio-reactive effects that visualise acoustic signals in the manner of an oscilloscope. However, a key difference with the approach presented here is that the visualisations are primarily created by designing symbolic spatial environments that correspond with the genres of music in question. Using this approach, visual spaces are designed in response to the visual language of the musical forms—in this case hardcore rave and vaporwave music—allowing the user to journey through these spaces and feel as though they are ‘inside the music’. With the most recent iteration of Cyberdream, I have sought to take these ideas further by making the experience interactive through audio-visual sound toys, allowing users to ‘paint with sound’. In Rajmil Fischman’s (2011) work he proposes the idea of ‘music in the holodeck’, calling for accessible platforms for collaborative music making in virtual environments (like the ‘holodeck’ seen in the science-fiction TV show Star Trek). For Fischman, ‘music in the holodeck’ should be inclusive and collaborative, providing an electronic equivalent to ‘a weekend afternoon in a middle class parlour during the second half of the nineteenth century: a soiree, with music performed live by family members gathered around the piano’ (p. 53). In his own compositional practice with a data glove (e.g. Ruraq Maki, 2012), Fischman has sought to provide intuitive forms of electronic music making by exploring ways in which familiar hand gestures can be used to generate electroacoustic sound. This and other research in real-time computer music and VR (e.g. Serafin et al. 2016) may indicate possible ways in which we might realise his dream of

7 Future Sound Dream: Virtual Reality Experiences

189

‘music in the holodeck’. However, like the holodeck in Star Trek, immersive technologies like VR include both sonic and visual components, and we must therefore consider what ‘music in the holodeck’ might look like. It is my hope that Cyberdream may point towards some possible solutions, however, there is more work to be done. While I plan to extend this project, I also hope that others will forge new pathways through the frontiers of this exciting field.17 Towards this end, the final chapter will consolidate the approaches explored in Explosions in the Mind , with a view to supporting further advances in this field, drawing us ever closer towards the future sound dream.

Notes 1. The gabber nightclub in Culturesport: Rotterdam 1995 (Boling 2019) appears to be a simulacrum of the Club Parkzicht techno nightclub, which existed in Rotterdam during the 1990s (see Housenation 1992). 2. For example, ‘Terminator’ by Metal Heads (on Terminator, 1992) samples The Terminator (Cameron 1984); ‘Underworld’ by SP 23 (1993) samples Alien (Scott 1979); and ‘The Angels Fell’ by Dillinja (1995) samples Blade Runner (Scott 1982). 3. Toffler’s work has also been credited as inspiring Detroit techno artists, for a further discussion see Sicko (2010). 4. As discussed in Chapter 6, example VJ mixes include Dance in Cyberspace (Dr. Devious and the Wiseman 1992); Global Chaos (Hex 1993); Future Shock (Frost et al. 1993); and the X-Mix series (Studio !K7, 1993–1998). 5. The predominant use of sampled materials in vaporwave can be considered in terms of Oswald’s (2004) concept of plunderphonics. 6. Toto’s ‘Africa’ (1982) playing in an empty shopping mall was portrayed in a YouTube video by Cecil Robert (2017). For a further discussion see also Tolentino (2018). 7. For a related discussion of sonic symbolism see also Tagg’s (2016) ‘Intel Inside Jingle Analysis’. 8. For example, Eccojams Vol. 1 by Chuck Person (2010) references the video game Ecco the Dolphin (Appaloosa Interactive, 1992), and dolphins are also featured on GATEWAY 2000 by MindSpring Memories (2018), Saccharine Synergy by Vaperror (2020), and others. For a further discussion of the

190

9. 10. 11. 12.

13.

14. 15.

16.

17.

J. Weinel

symbols and iconography used in vaporwave graphics, see also Chandler (2016). In Unity, a skybox is a 6-sided cube that is drawn behind all graphics in the game, typically providing an impression of a sky. For indicative examples of instrumental UK garage and grime, see Bingo Beats Volume 3 by DJ Slimzee (2004). Easter eggs are hidden components that can be found in computer software and other media including music (Weinel et al. 2014). In electronic dance music production, the ‘hoover’ sound is a specific synthesiser sound originally created by Eric Persing for the Roland Alpha Juno, and popularised on tracks such as Mentasm by Second Phase (1991). Technoshamanism is a term associated with 1990s rave culture and cyberculture, where technologies such as VR may be considered as shamanic tools. See Weinel (2018, pp. 90–94). VR cardboard (or ‘Google Cardboard’) is a form of VR provided through a mobile phone mounted in a cardboard viewer. The Oculus Quest provides a better quality VR experience than the Gear VR due to a higher screen resolution of 1440 × 1600 per eye and six degrees of freedom (6DoF). This is similar to the ‘flam’ drumming rudiment, where two percussive strokes are played almost simultaneously to sound like a single broader note. There is already exciting work being produced by students exploring game engine technologies, for example see Holtum (2020).

8 Conclusion: Design Frameworks

Blood red lasers punctuate the darkness of a vast arena, drawing flickering patterns in the smoke. Thousands of bodies move to a pounding 4/4 techno beat, as a monotonous vocal sample repeats ‘we could go higher… higher… higher…’. The DJ is bathed in red light, and moves her body to the beat before an array of VJ projections showing sound waves and geometric patterns. A bubbling acid bass line weaves its way through the mix, the shifting filters of the Roland TB-303 synthesiser increasing the intensity in cascading waves of sound, as cycling spotlights creep across the ceiling of the arena. This could be a description of almost any acid techno rave from the past few decades, but the year is 2020, and the event is entirely virtual, existing only as a live stream designed for viewers to watch at home on TV. The DJ, Amelie Lens, is superimposed over a computer-generated dance tent, complete with a laser light show and VJ projections. As video cameras pan around the scene, it resembles what one might expect to see if a real music festival were televised. Yet the bodies dancing to the music are not real, instead, the DJ performs to a crowd of 3D avatars— automatons who dance monotonously with blank expressions. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 J. Weinel, Explosions in the Mind, Palgrave Studies in Sound, https://doi.org/10.1007/978-981-16-4055-1_8

191

192

J. Weinel

This was the scene at Tomorrowland 2020, one of the many virtual music festivals designed in 2020 in response to the COVID-19 pandemic.1 The social distancing measures needed to suppress the spread of the coronavirus meant that music festivals could not take place as in-person events. As a result, some promoters cancelled or postponed their festivals, while others moved to online alternatives in the hope that audiences could gain some enjoyment by partying in their living rooms. These virtual concerts typically worked by recording performances and presenting them as live streams. In some instances, this was achieved by broadcasting from a studio or empty theatre, as in the case of Alexis and VJ L’Aubaine’s performance (Resolution 2020), which was filmed in Venue MOT in London without an audience. In other cases, the artists were placed in computer-generated environments made using gameengine technologies. For example, Prospa (2020) presented their music in a virtual warehouse complete with smoke machines and strobe lights; rap megastar Travis Scott performed as an avatar in the multiplayer video game Fornite (Epic Games 2017), allowing players to attend a concert with psychedelic visualisations in the game world (Epic Games 2020; Webster 2020); while Lost Horizon allowed audiences to walk around a virtual festival including performances by DJs and VJs using a player avatar, and could be viewed with a virtual reality (VR) headset (Kocay 2020). As Hogan (2020) notes, these concerts were not without precedent— before the pandemic, Gorillaz, Bjork, and Hatsune Miku were among those artists who had already been exploring virtual modes of performance. MelodyVR had also been providing 360-degrees concerts for audiences to watch in VR, and Mbryonic had created Amplify VR (2018), an interactive VR music experience that allowed audiences to manipulate the music with game controllers. New ways to experience music in immersive spaces were not only dependent on computers, but were also being forged in exciting new ways through Unkle and Punchdrunk’s Beyond the Road (Saatchi Gallery, 12 June–8 September 2019) exhibition, which turned the band Unkle’s music into a multisensory experience that represented different tracks on the album through surrealistic, aromatic, smoke-filled neon-lit rooms. In different ways, these projects had already been pushing music into new immersive,

8 Conclusion: Design Frameworks

193

spatial, audio-visual contexts, but with the COVID-19 pandemic, live music events were suddenly forced to move online or else be cancelled altogether. Although it is certainly debatable if virtual events can truly provide an effective substitute for the atmosphere and social experience that audiences seek from real-world concerts, nightclubs, or music festivals, certainly these forms are capable of delivering interesting new ways of experiencing music. To make the most of this possibility, we must seek to understand how these experiences can be designed effectively. It is my hope that the approaches I have explored in Explosions in the Mind may contribute towards this understanding, and with this in mind, in this chapter I will consolidate the main compositional methods, thereby providing a set of design frameworks that can be used by artists, academics, and industry professionals for developing their own work. In this chapter, the compositional principles discussed in Explosions in the Mind will be summarised as three design frameworks. In Chapter 2 I discussed approaches for creating ‘psychedelic journeys in sound’. This involved taking various typical features of altered states of consciousness (ASC) experiences, and using them to inform the design of sonic and visual materials and/or structure. In this way, the compositions became analogous to psychedelic experiences. In Chapter 3 we saw how this idea could be utilised for real-time performances, while in Chapter 4 it was used to compose audio-visual works, which represented what one might see or hear during a hallucination. Chapter 5 extended this concept in order to design several prototype ‘altered states of consciousness simulations’, by using interactive audio-visual technologies such as video game engines to provide first-person perspective simulations of psychedelic experiences. Through VJ performances and VR, the projects discussed in Chapters 6–7 reflect a distinct progression of these ideas, while moving away from designs based on formal accounts of ASCs, in favour of an approach based on visual associations with sound and music. These projects sought to construct ‘synaesthetic visualisations of sound’ through the design of 3D environments and audio-visual sound toys. In what follows, I will summarise each of these design frameworks. There is of course significant overlap, and readers interested in utilising

194

J. Weinel

the frameworks will undoubtedly treat them as starting points for their own projects, making adaptations to further evolve their practices and find new directions.

Framework One: Psychedelic Journeys in Sound In Chapter 2, Explosions in the Mind explored how electroacoustic music could be composed based on ASCs. The approach I used can be understood as an ‘adaptive principle’ for composition. Genres such as psychedelic rock adapt typical genre forms towards various concepts of psychedelia, through lyrics that connect with psychedelic themes, or the use of effects pedals and production techniques that warp or colour the sounds so that they acquire hallucinatory qualities. In Chapter 2 we saw how this idea could be used in the field of electroacoustic composition. The typical tools of synthesis, sampling, and digital signal processing (DSP) were used to compose electroacoustic music, but the design of these compositions was ‘adapted’ based on concepts of ASCs. By considering a range of typical ‘ASC features’ that are described in the research literature, it was possible to devise mimetic sonic materials using ‘ASC techniques’. Table 8.1 summarises the main ‘ASC features’ and corresponding ‘ASC techniques’ that were used across the various electroacoustic compositions described in Chapter 2. Extending this methodology, we also saw how it was possible to organise these materials into structural forms that are analogous to the progression of hallucinations as they unfold in time. By doing this it is possible to compose electroacoustic music that provides a sonic journey that represents a psychedelic experience through sound. Figure 8.1 outlines this generalised approach for composing psychedelic works of electroacoustic music. ASC features are translated with corresponding techniques and organised into a structure that reflects the progression of a hallucination in time. Figure 8.1 provides an indicative example of a structural form similar to the ones used for compositions such as Entoptic Phenomena (2009) and Nausea (2011). Three main phases of ‘onset’,

8 Conclusion: Design Frameworks

195

Table 8.1 Summary of techniques used to represent ASC features in fixedmedia electroacoustic compositions Example compositions

ASC feature

ASC technique

Cellular consciousness

Organic sonic materials

Atomic consciousness Sensory consciousness

Synthetic sonic materials Physical bass drones and dub sounds Morphing textures Drone sounds

Shifting perceptual focus Distorted time perception

Micro–macro perception

Juxtaposed macro drones and micro rhythmic textures Scattered or rotating rhythmic materials

Visual patterns of hallucination

Strange voices or encounters with mysterious entities Euphoria or anxiety

Generalised approach

Processed vocal sounds

Light and dark sonic materials ASC features

ASC techniques

ASC journey

ASC structure

Onset

Night Breed, Swamp Process Surfer Stem Night Dream, Swamp Process, Nausea Night Breed Surfer Stem, Night Dream, Entoptic Phenomena, Swamp Process, Nausea Night Dream

Night Breed, Surfer Stem, Entoptic Phenomena, Swamp Process, Nausea Entoptic Phenomena, Nausea Nausea

Plateau Wave 1

Wave 2

Wave 3

Light

Whispering voices

Wave 4

Dark Drones Sensory bass Breakthrough

Breakthrough

Fig. 8.1 Framework for composing psychedelic journeys in sound

196

J. Weinel

‘plateau’, and ‘termination’ reflect the dynamic progression and intensity of the imagined hallucination that the work represents. Within this form, the work can be further subdivided into individual ‘waves’ of the experience. In my works, ‘onset’ is typically accompanied by the gradual introduction of materials based on visual patterns of hallucination, which increase in intensity. The ‘plateau’ phase may incorporate light and dark sonic materials representing euphoria and anxiety, sensory bass, drones that are suggestive of timelessness, or auditory hallucinations that reflect encounters with strange entities. The ‘termination’ phase usually includes representations of visual hallucinations, which decrease in intensity as the psychedelic experience draws to a close. Figure 8.1 illustrates this structural pattern to provide an indicative example, though many variations are possible. In Chapter 3 these ideas were developed in the context of real-time performances, and here our discussion shifted to consider how creative coding with programming environments such as Max/MSP could be used to design these. The Atomizer Live Patch was a software tool that I created to design sounds that fulfilled the requirements of these compositions, allowing sonic materials to be generated for the fixed-media works. This software was also created to realise a long-form live performance, Entoptic Phenomena in Audio (2010). This performance synthesised a hallucinatory structure similar to the one shown in Fig. 8.1 with the idea of a DJ set, integrating multiple compositions into a single continuous performance, where each individual composition becomes analogous to a ‘wave’ or phase of hallucination. These compositions always have an implicit ‘visual’ aspect, since they have sounds based on visual patterns of hallucinations or other features of hallucinations that could be articulated graphically, and so Entoptic Phenomena in Audio prefigures my later work such as the VJ performances and VR projects, which use related structural approaches. While Entoptic Phenomena in Audio is based on real-time synthesis, triggering of samples, and DSP effects, Bass Drum, Saxophone & Laptop (2010) showed how these compositional ideas could be used to realise a performance with acoustic instruments and live electronics, where real-time processes enable the sounds to undergo continuous changes, reflecting the shifting perceptual experiences that may occur during hallucinations.

8 Conclusion: Design Frameworks

197

While Chapters 2 and 3 explore how one might compose psychedelic journeys in sound, in Chapter 4 we saw how these approaches could also be used to compose audio-visual compositions. The audio-visual piece Tiny Jungle (2010) used similar ideas to those outlined in Fig. 8.1, except where ‘ASC features’ were identified, these were realised as audiovisual materials. Using this approach, visual patterns of hallucination or strange encounters could be depicted using animated visuals as well as sounds. Structural forms could then be developed in order to articulate psychedelic audio-visual journeys. In Chapter 4 these ideas were also explored through the audio-visual compositions Mezcal Animations (2013), Cenote Zaci (2014), and Cenote Sagrado (2014). Across these various works, analogue and digital approaches to synaesthetic animation were explored, as well as creative coding via the Atomizer Visual Max/MSP/Jitter software. Following these examples, composers working with forms such as visual music or experimental film may draw upon this first framework, in order to compose their own psychedelic audio-visual journeys.

Framework Two: Altered States of Consciousness Simulations If we extend the idea of representing ASCs with audio-visual media and consider how this might occur in the interactive contexts presented by game-engine technologies and VR, we arrive at the possibility of ASC simulations. In Chapter 5 I discussed a series of projects that exemplify forms related to this concept. Quake Delirium (2010) showed how a first-person shooter game could be modified in order to represent a psychedelic experience by automating various graphical and game parameters, while also generating a corresponding soundtrack. Psych Dome (2013) utilised Max/MSP and Processing in order to generate sound and a visualisation in real-time based on forms such as the visual patterns seen during hallucinations. Both of these projects also explored the use of biofeedback controllers in order to provide passive forms of interaction that link brain activity to the audio-visualisation. In addition, ASC Sim

198

J. Weinel

(2017) provided a prototype with three interactive mechanisms for representing aspects of auditory hallucinations in the context of the Unity video game engine. Across these projects we began to see how interactive audio-visual technologies such as game engines could be used to simulate ASCs. Figure 8.2 generalises the concept of ‘altered states of consciousness simulations.2 The design of interactive ASC simulations can be produced with an ‘ASC engine’. This is a system that generates graphics and sounds based on ASC features. For example, an ASC engine might generate visual patterns of hallucination, shifting forms of perception, or encounters with strange entities. These may be rendered with both graphics and/or sound, with the audio reflecting forms of auditory hallucination or distortions to spatial awareness. Typically these visual and auditory forms may be rendered from a first-person perspective, thereby representing what a player avatar might see or hear during a hallucination.3 Biofeedback technologies can be used to enrich these simulations, by linking the generation of sounds and visuals to aspects of the user such as brainwave activity or galvanic skin response. The concept of ASC simulations can be used for entertainment purposes in video games, or within the domain of ‘cyberdelics’,4 where consciousness expansion is sought through means of immersive technologies such as VR. Generalised approach ASC engine

Data inputs: Headtracking Gamepad Biofeedback

VR headset

User

Dynamic envelopes

Fig. 8.2 Framework for designing altered states of consciousness simulations

8 Conclusion: Design Frameworks

199

Framework Three: Synaesthetic Visualisations of Sound The projects discussed in Chapters 6 and 7 provide the basis for our third framework. While the first two frameworks rely significantly on formal aspects of ASC experiences, such as those described in studies of psychedelics, my later projects took a slightly different approach, whereby aspects of sound and music were translated into visuals. My VJ performances, paintings, and the Cyberdream (2019–2020) VR applications can be understood as synaesthetic, since they interpret sound through visual forms such as computer animations, paint, and 3D environments. Here the design of the work interprets symbolic and conceptual meanings that are embedded in music. This differs somewhat from the usual approaches to music visualisation, since instead of translating properties of an audio signal such as frequency or amplitude into visual images, the pieces look for representational symbols which can be rendered as animations or 3D environments. This opens up new worlds of possibility for visualising music, allowing us to conceive of music as imaginative, synaesthetic visual spaces that users can journey through. My works in this area translate individual music tracks into audio-visual designs, and then create journeys through multiple audio-visual tracks to form continuous experiences, which are analogous to those provided in a DJ mix. With the addition of the audio-visual sound toys, as explored in Cyberdream (2020), users can also take an active role in creating and composing audio-visual experiences of music themselves, providing new forms of creative composition and performance. Figure 8.3 summarises the framework for composing ‘synaesthetic visualisations of sound’, which could be used by designers making VJ performances, VR music visualisations, or other forms of visual art. Synaesthetic visualisations can be designed by incorporating features derived from the audio signal, such as properties of frequency or amplitude, or may draw upon the symbolic concepts that are embedded in the music. Projects using this framework should find ways to interpret and represent symbolic concepts, which may give rise to the generation of 3D spaces or environments. Although there are multiple ways in which this framework could be utilised, Fig. 8.3 provides an indicative

200

J. Weinel

Generalised approach

Audio signal analysis

Symbolic feature analysis

Track 1

Track 2

Track 3

Micro structure

Macro structure

Fig. 8.3 Framework for composing synaesthetic visualisations of sound

example similar to that used in Cyberdream, whereby audio-visual scenes are generated in correspondence with musical tracks, and these are organised to form a continuous journey analogous to a DJ set. This provides the macro structure, while the micro structure is provided by audio-visual sound toys. Since these designs replicate a synaesthetic process, which is similar to that found in sound-to-image hallucinations, we may also consider that these designs are psychedelic visualisations of sound. ∗ ∗ ∗ In the twenty-first century we are currently witnessing the expansion of networked technologies through new forms of immersive sensory interfaces. Sonic and audio-visual technologies such as VR are allowing us to design journeys through fantastic real and unreal environments rendered in light and sound. ASCs may provide a useful lens through which to consider the design of these journeys, by drawing our attention towards the myriad ways in which they affect our multimodal conscious experience, and the designs themselves may also provide us with interesting new ways to represent the sensory faculties of the human mind. Throughout Explosions in the Mind, I have explored a wide range of creative projects, which demonstrate practical approaches for doing this. These projects reflect my own personal journey as a creative practitioner, yet I hope the discussion presented in this book also reveals insights which have a utility beyond that provided by the experience

8 Conclusion: Design Frameworks

201

of the works themselves. We have seen how electronic music, audiovisual compositions, and creative coding can elicit psychedelic journeys; how interactive technologies might provide ways to simulate states such as hallucinations; and how VJ performances and VR applications can conjure voyages through imaginative synaesthetic worlds of music. In this, the final chapter of Explosions in the Mind , I have consolidated these approaches into three main design frameworks, which summarise the essential methodologies that I have used for creating my projects. These frameworks can allow other artists, designers, and programmers to develop their own projects, either by utilising the frameworks described, or by adapting, modifying, and extending them to forge new pathways into uncharted territories. Taken as a whole, these frameworks point towards an exciting new paradigm for composing psychedelic sounds and visualisations. Above all, Explosions of the Mind has expressed my vision of that paradigm, and yet, it is only as a collective endeavour that we can truly realise this dream—and so now I hand it over to you, the reader, to compose the psychedelic sounds and visualisations of the future.

Notes 1. At the time of writing, videos of Tomorrowland 2020 were available online via https://www.tomorrowland.com on a time-limited basis. A list of virtual concerts and music festivals are available online, see Stubhub (2020) and Billboard (2020). These events have also been discussed in various press, for example see Pollard (2020) and Kocay (2020). 2. The framework in this section continues the discussion of ’ASC simulations’ in Weinel (2018b). 3. As noted previously, the concept of representing the subjective perceptual experiences of avatars can be described as ’avatar-centred subjectivity’; see also Weinel and Cunningham (2019). 4. ’Cyberdelics’ is a portmanteau of cyberculture and psychedelics, and describes immersive technologies that seek to provide forms of consciousness expansion. For a further discussion see Weinel (2018a); Valentish (2019); and Filimowicz and Weinel (2020).

References

Acid Ted (2006) ‘One Consciousness (Dynamic Intervention Mix)’, One Consciousness. 12”, Acid Test Pressings. Adams, J.K. (1994) ‘Psychosis: “Experimental” and Real’, in Leary, T., Metzner, R., and Weil, G.M. (eds) The Psychedelic Reader. New York: Citadel Press, pp. 61–82. Alpha Omega (1999) Journey to the 9th Level. CD, Reinforced Records. The Amboy Dukes (1968) Journey to the Center of the Mind . 7”, Mainstream Records. Amiel, J. (2003) The Core [feature film]. Paramount. Andromeda Entertainment (2020) Soundself: A Technodelic [VR experience]. PC, Andromeda Entertainment. Aphrodite (2001) ‘Siren Bass’, Siren Bass/Playground Shades. 12”, Aphrodite Recordings. Appaloosa Interactive (1992) Ecco the Dolphin [video game]. Sega Megadrive, Sega. Arcon 2 (1997) Arcon 2. CD, Reinforced Records. Autechre (music) and Rutterford, A. (director) (2002) Gantz Graf [music video]. Warp. Bailey, D. (1993) Improvisation: Its Nature and Practice in Music. Cambridge, MA: Da Capo. © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 J. Weinel, Explosions in the Mind, Palgrave Studies in Sound, https://doi.org/10.1007/978-981-16-4055-1

203

204

References

Barik, K., Daimi, S.N., Jones, R., Bhattacharya, J., and Saha, G. (2019) ‘A Machine Learning Approach to Predict Perceptual Decisions: An Insight into Face Pareidolia’, Brain Informatics, 6(2). https://doi.org/10.1186/s40 708-019-0094-5 Bartram, L. and Nakatani, A. (2010) ‘What Makes Motion Meaningful? Affective Properties of Abstract Motion’, Fourth Pacific-Rim Symposium on Image and Video Technology (PSIVT), Singapore, 14–17 November 2010. https:// doi.org/10.1109/PSIVT.2010.85. Batchelor, S.F. (2019) ‘A Framework for Future Paintings’, EVA London 2019 (Electronic Visualisation and the Arts), pp. 310–317. https://doi.org/10. 14236/ewic/EVA2019.59. Beat Games (2019) Beat Saber [VR experience]. Oculus Quest, Beat Games. The Beatles (1966) ‘Tomorrow Never Knows’, Revolver. LP, Capitol Records. Becker, J. (2004) Deep Listeners: Music, Emotion and Trancing. Bloomington: University of Indiana Press. Bell, E. (2019) ‘Hacking Jeff Minter’s Virtual Light Machine: Unpacking the Code and Community Behind an Early Software-Based Music Visualizer’, Volume! 16(1). https://doi.org/10.4000/volume.7254. Belle-Fortune, B. (2005) All Crews: Journeys Through Jungle/Drum & Bass Culture. Vision Publishing. Belson, J. (1962) LSD [unfinished artist film]. Screened at: ‘Found: New Restorations and Discoveries from Center for Visual Music’, Tate Modern, London, 26 September 2013. Berezan, D. (2010) Lecture and Concert at Keele University Music Department, 5 November 2010. Berlin, C.L. (2018) Rave Art: Flyers, Invitations, and Membership Cards from the Birth of Acid House Clubs and Raves. London: Carlton Books. Berrin, K. (ed.) (1978) Art of the Huichol Indians. New York: Harry N. Abrams. Bigelow, K. (1995) Strange Days [feature film]. Lightstorm Entertainment. Billboard (2020) ‘Here Are All the Livestreams & Virtual Concerts to Watch During Coronavirus Crisis’, Bilboard , 30 July 2020. Online: https:// www.billboard.com/articles/columns/pop/9335531/coronavirus-quarantinemusic-events-online-streams (Accessed: 6 August 2020). Bixiga 70 (2015) ‘Mil Vidas’, III. LP, Glitterbeat. Blackmore, S. (2003) Consciousness: An Introduction. London: Hodder Education. Bliss, E.L. and Clark, L.D. (1962) ‘Visual Hallucinations’, in West, L.J. (ed.) Hallucinations. New York: Grune & Stratton, pp. 92–107.

References

205

Bogenschutz, M.P. and Johnson, M.W. (2016) ‘Classic Hallucinogens in the Treatment of Addictions’, Progress in Neuro-Psychopharmacology & Biological Psychiatry, 64, pp. 250–258. Boling, J.M. (2019) Culturesport: Rotterdam 1995 [animated short], YouTube. Online: https://youtu.be/UEcNEO3fVF8 (Accessed: 6 August 2020). Bomb the Bass Featuring Justin Warfield (1995) Bug Powder Dust. 12”, 4th & Broadway, Stoned Heights. Bowles, P. (1962) A Hundred Camels in the Courtyard , 2nd edn. Reprint, San Francisco: City Lights Books, 1986. Bradley, L. (2001) Bass Culture: When Reggae Was King. London: Penguin Books. Brakhage, S. (1963) Mothlight [artist film]. Available on: Criterion Collection: by Brakhage, An Anthology, Volume One. Criterion. Brakhage, S. (1999) Persian Series [artist film]. Available on: Criterion Collection: by Brakhage, An Anthology, Volume Two. Criterion. Bressloff, P.C., Cowan, J.D., Golubitsky, M., Thomas, P.J., and Wiener, M.C. (2001) ‘Geometric Visual Hallucinations, Euclidean Symmetry and the Functional Architecture of Striate Cortex’, Philosophical Transactions of the Royal Society B: Biological Sciences, 356(1407), pp. 299–330. Brevig, E. (2008) Journey to the Center of the Earth [feature film]. New Line Cinema. Broadbent, D. (1958) Perception and Communication. New York: Pergamon Press. Brooks, T. (2019) ‘BrainWaveOSC’ [Github code repository]. Online: https:// github.com/trentbrooks/BrainWaveOSC (Accessed: 19 July 2020). Brougher, K. and Mattis, O. (2005) Visual Music: Synaesthesia in Art and Music Since 1900. London: Thames & Hudson. The Bug (2003) Pressure. 2xLP, Rephlex. Bullfrog Productions (1994) Magic Carpet [video game]. MS-DOS, Electronic Arts. Buñuel, L. (1929) Un Chien Andalou [artist film]. Burroughs, W.S. (1959) The Naked Lunch. Reprint, London: Fourth Estate, 2010. Bute, M.E., McLaren, N., and Nemeth, T. (1940) Spook Sport [artist film]. Creative Film Society. Callan, D.E., Jones, J.A., Munhall, K., Callan, A.M., Kroos, C., and VatikiotisBateson, E. (2003) ‘Neural Processes Underlying Perceptual Enhancement by Visual Speech Gestures’, NeuroReport, 14(17), pp. 2213–2218.

206

References

Calvert, G.A., Bullmore, E.T., Brammer, M.J., Campbell, R., Mcguire, P.K., Iversen, S.D., and David, A.S. (1997) ‘Activation of Auditory Cortex During Silent Lipreading’, Science, 276(5312), pp. 593–596. Cameron, J. (1984) The Terminator [feature film]. Orion Pictures. Cameron, J. (1991) Terminator 2: Judgement Day [feature film]. TriStar Pictures. Capcom (1992) Street Fighter II [video game]. Super Nintendo Entertainment System, Capcom. Carhart-Harris, R.L., Bolstridge, M., Rucker, J., Day, C.M.J, Erritzoe, D., Kaelen, M., Bloomfield, M., Rickard, J.A., Forbes, B., Fielding, A., Taylor, D., Pilling, S., Curran, V.H., and Nutt, D.J. (2016a) ‘Psilocybin with Psychological Support for Treatment-Resistant Depression: An Open-Label Feasibility Study’, The Lancet Psychiatry, 3(7), pp. 619–627. Carhart-Harris, R.L., Muthukumaraswamy, S., Roseman, L., Kaelen, M., Droog, W., Murphy, K., Tagliazucchi, E., Schenberg, E.E., Nest, T., Orban, C., Leech, R., Williams, L.T., Williams, T.M., Bolstridge, M., Sessa, B., McGonigle, J., Sereno, M.I., Nichols, D., Hellyer, P.J., Hobden, P., Evans, J., Singh, K.D., Wise, R.G., Curran, H.V., Feilding, A., and Nutt, D.J. (2016b) ‘Neural Correlates of the LSD Experience Revealed By Multimodal Neuroimaging’, Proceedings of the National Academy of Sciences of the United States of America, 113, pp. 4853–4858. https://doi.org/10.1073/pnas.151 8377113. Carpenter, J. (1988) They Live [feature film]. Universal Pictures. Carroll, L. (1865–1871) Alice’s Adventures in Wonderland & Through the Looking-Glass. Reprint, Ware: Wordsworth Editions, 1993. Castaneda, C. (1968) The Teaching of Don Juan: A Yaqui Way of Knowledge. Reprint, London: Penguin Books, 2004. Cecil Robert (2017) ‘Toto- Africa (Playing in an Empty Shopping Centre)’, YouTube. Online: https://youtu.be/D__6hwqjZAs (Accessed: 3 September 2020). Chandler, S. (2016) ‘Escaping Reality: The Iconography of Vaporwave’, Bandcamp Features, 16 September 2016. Online: https://daily.bandcamp.com/fea tures/vaporwave-iconography-column (Accessed: 4 September 2020). Chion, M. (1994) Audio-Vision: Sound on Screen. New York: Columbia University Press. Chuck Person (2010) Eccojams Vol. 1. Digital audio, The Curatorial Club. Clayton, R. (2018) Welcome to Venice RXCX. Los Angeles: Ginko Press/KYI.

References

207

Code 071 (1992) ‘A London Sumtin’. Various Artists, Reinforced Anniversary Boxset. 5x 7”, 4x cassette, and USB stick with MP3, Reinforced Records, 2015. Cold Beam Games (2010) Beat Hazard [video game]. PC, Cold Beam Games. Coldcut and Hexstatic (1997) Timber [music video]. Ninja Tune. Coleridge, S.T. (1797) ‘Kubla Khan’, in Holmes, R. (ed.) Samuel Taylor Coleridge: Selected Poetry. London: Penguin Books, 1996, pp. 229–231. Collin, M. (1998) Altered State: The Story of Ecstasy Culture and Acid House. London: Serpent’s Tail. Compulsion Games (2018) We Happy Few [video game]. PC, Gearbox. Cooper, R. (1977) A Guide to British Psilocybin Mushrooms. London: Hassle Free Press. Corman, R. (1967) The Trip [feature film]. American International Pictures (AIP). Cox, C. (1993) Fantazia: The Big Bang. CD, Fantazia. Crowley, K. (2011) Surf Beat: Rock ’n’ Roll’s Forgotten Revolution. New York: Backbeat Books. Curtis, A. (2016) HyperNormalisation [documentary film]. First aired 16 October 2016, BBC. Cytowic, R.E. (1989) Synesthesia: A Union of the Senses. New York: SpringerVerlag. DeNora, T. (2010) Music in Everyday Life. Cambridge: Cambridge University Press. Demarque, T.C. and Lima, E.S. (2013) ‘Auditory Hallucination: Audiological Perspective for Horror Games’, SBC—Proceedings of SBGames 2013, São Paulo, Brazil. DemosceneVids (2015) ‘Future Crew—Second Reality (1993) [60fps]’, YouTube. Online: https://youtu.be/iw17c70uJes (Accessed: 29 July 2020). Dillinja (1995) ‘The Angels Fell’, on The Angels Fell. 12”, Metalheadz. DJ Food & DK (2001) Solid Steel—Now Listen. CD, Ninja Tune. DJ Slimzee (2004) Bingo Beats Volume 3. CD, Bingo Beats. Reichel-Dolmatoff, G. (1978) Beyond the Milky Way: Hallucinatory Imagery of the Tukano Indians. Los Angeles: UCLA Latin American Centre Publications. Dowson, T.A. (2007) ‘Debating Shamanism in Southern African Rock Art: Time to Move On...’, South African Archaeological Bulletin, 62(185), pp. 49– 61. Drake, D. (2005) Sartre (Life and Times). London: Haus Publishing.

208

References

Dr. Devious and the Wiseman (1992) Dance in Cyberspace. VHS, Parade Video. Drool (2016) Thumper [video game]. PC, Drool. Duffer, M. and Duffer, R. (2016–) Stranger Things [TV series], Netflix. Eclectic Method (2014) ‘The Wolf of Wall Street (Eclectic Method Chest Thump Mix)’, YouTube. Online: https://youtu.be/XR9-y4u9mew (Accessed: 12 August 2020). Eigenfeldt, A. (2017) ‘Musebots: Collaborative Composition with Creative Systems’, eContact! 20(2). Online: https://econtact.ca/20_2/eigenfeldt_crea tivesystems.html (Accessed: 16 April 2020). The Electric Prunes (1966) I Had Too Much To Dream (Last Night). 7”, Reprise Records. Eliade, M. (1964) Shamanism: Archaic Techniques of Ecstasy. Reprint, Princeton, NJ: Princeton University Press, 2004. Eliade, M. (1978) ‘The Eleusinian Mysteries’, in A History of Religious Ideas: Vol.1 From the Stone Age to the Eleusinian Mysteries. Chicago: University of Chicago Press, pp. 290–301. Emmerson, S. (1986) ‘The Relation of Language to Materials’, in Emmerson, S. (ed.) The Language of Electroacoustic Music. Basingstoke: Macmillan Press, pp. 17–39. Epic Games (2017) Fortnite [video game]. Playstation 4, Epic Games. Epic Games (2020) ‘Travis Scott’s Astronomical’. Online: https://www.epicga mes.com/fortnite/en-US/news/astronomical (Accessed: 6 August 2020). Epic MegaGames and Digital Extremes (1998) Unreal. Windows, GT Interactive. Equinox (2006) ‘Acid Rain V.I.P. (Breakage Final Chapter Mix)’, Acid Rain V.I.P. Digital audio, Planet Mu. Excel (1989) ‘Message in a Bottle’, The Joke’s On You. CD, Caroline Records. Exit EEE (1993) ‘Dreaming of a Better World’, Atrax. 12”, No Respect Records. Fachner, J.C. (2011) ‘Time Is the Key: Music and Altered States of Consciousness’, in Cardeña, E. and Winkelman, M. (eds) Altering Consciousness: Multidisciplinary Perspectives: Volume 1: History, Culture and Humanities. Santa Barbara: Praeger, pp. 355–376. Faulkner, M. (D-Fuse) (2006) VJ: Audio-Visual Art and Vj Culture. London: Laurence King. Filimowicz, M. and Weinel, J. (2020) ‘Altered States of Consciousness in Electronic Music and Audio-Visual Media: An Interview with Dr. Jonathan Weinel’, Medium. Online: https://medium.com/sound-and-design/altered-

References

209

states-of-consciousness-in-electronic-music-and-audio-visual-media-112d30 034ee (Accessed: 26 October 2020). Fischer, R. (1971) ‘A Cartography of the Ecstatic and Meditative States’, Science, 174(4012), pp. 897–904. Fischinger, O. (1938) An Optical Poem [artist film]. Fischman, R. (2011) ‘Back to the Parlour’, Sonic Ideas/Ideas Sónicas, 3(2), pp. 53–66. Fischman, R. (2012) ‘Ruraq Maki (Hand Made, Hecho a Mano) for Digital Glove’, Vimeo. Online: https://vimeo.com/55093629 (Accessed: 4 September 2020). Fisher, M. (2014) Ghosts of My Life: Writings on Depression, Hauntology and Lost Futures. Alresford: Zero Books. Fitterer, D. (2008) Audio Surf. PC, Dylan Fitterer. Flattery, D.S. and Schwartz, M. (1989) Haoma and Harmaline: The Botanical Identity of the Indo-Iranian Sacred Hallucinogen ‘Soma’ and Its Legacy in Religion, Language and Middle Eastern Folklore. Berkley: University of California Press. Freud, S. (1899) The Interpretation of Dreams. Reprint, Ware: Wordsworth Editions, 1997. Frost, M., Irwin, C., Jamieson, D., and Simpson, P. (1993) Future Shock. VHS, Prism Leisure Video. Gabrielsson, A. and Lindstrom, E. (2012) ‘The Role of Structure in the Musical Expression of Emotions’, in Juslin, P.N. and Sloboda, J.A. (eds) Handbook of Music and Emotion: Theory, Research, Applications. Oxford: Oxford University Press, pp. 367–400. Ganguly, S. (1994) ‘Stan Brakhage: The 60th Birthday Interview’, Film Culture 78 (Summer 1994). Garro, D. (2012) ‘From Sonic Art to Visual Music: Divergences, Convergences, Intersections’, Organised Sound , 17(2), pp. 103–113. Gavin, S. (2020) Spiralled. London: Idea Books. Gebhart-Sayer, A. (1985) ‘The Geometric Designs of the Shipibo-Conibo in Ritual Context’, Journal of Latin American Lore, 11(2), pp. 143–175. Gibson, W. (1984) Neuromancer. Reprint, London: Harper Voyager, 1995. Gong (1974) You. LP, Virgin. Grimshaw-Aagard, M. (2019) ‘Presence and Biofeedback in First-Person Perspective Computer Games’, in Filimowicz, M. (ed.) Foundations in Sound Design for Interactive Media. Abington: Routledge, pp. 78–94. Grunenberg, C. (2005) Summer of Love: Art of the Psychedelic Era. London: Tate Publishing.

210

References

Hany Tarek (2016) ‘A e s t h e t i c M e m e s 1½ Hour Vaporwave Mix’, YouTube. Online: https://youtu.be/3GDt3xNC_JQ (Accessed: 4 September 2020). Haridy, R. (2019) ‘Tripping on Virtual Reality: The Artists Trying to Replicate Psychedelic Experiences in VR’, New Atlas, 18 September 2019. Online: https://newatlas.com/vr/tripping-virtual-reality-ayahuasca-vr-psyche delic-experiences/ (Accessed: 1 July 2020). Harner, M.J. (1973) Hallucinogens and Shamanism. London: Oxford University Press. Harrison, J. (1999) ‘Sound, Space, Sculpture: Some Thoughts on the “What”, “Ho” and “Why” of Sound Diffusion’, Organised Sound , 3(2), pp. 117–127. Harrison, N. (2004) ‘Can I Get an Amen’ [video documenting an installation]. Online: http://nkhstudio.com/pages/popup_amen.html (Accessed: 7 July 2020). Hayes, C. (ed.) (2000) Tripping: An Anthology of True-Life Psychedelic Adventures. New York: Penguin Compass. HelloEnjoy (2017) Fantasynth [VR experience]. PC, HelloEnjoy. Hex (1993) Global Chaos. VHS, Ninja Tune. Hibushibire (2017) Freak Out Orgasm! Cassette, Riot Season Records. Hicks, M. (2000) Sixties Rock: Garage, Psychedelic, and Other Satisfactions. Urbana: University of Illinois Press, pp. 12–22. Hill, G. (Malaclypse the Younger) and Thornley, K.W. (Lord Omar Khayyam Ravenhurst) (1994) Principia Discordia or, How I Found Goddess, and What I Did to Her When I Found Her. Austin: Steve Jackson Games. Hobson, J.A. (2003) The Dream Drugstore. Cambridge, MA: MIT Press. Hogan, M. (2020) ‘Where Can Virtual Concerts Go After Travis Scott’s Fortnite Extravaganza?’, Pitchfork, 5 May 2020. Online: https://pitchf ork.com/thepitch/virtual-concerts-travis-scotts-fortnite-100-gecs-minecraft/ (Accessed: 6 August 2020). Hollander, D. (2018) Unusual Sounds: The Hidden History of Library Music. New York: Anthology. Holtum, C. (2020) ‘Simulacra: Using Game Development Software to Create Audio-Visual Performance Tools’, EVA London 2020 (Electronic Visualisation and the Arts), pp. 100–103. https://doi.org/10.14236/EVA2020.17. Horowitz, S. and Looney, S. (2014) The Essential Guide to Game Audio: The Theory and Practice of Sound for Games. Burlington: Focal Press. Horrocks, R. (2001) Len Lye: A Biography. Auckland: Auckland University Press.

References

211

Housenation1992 (2013) ‘Club Parkzicht Rotterdam, Full Documentary’, YouTube. Online: https://youtu.be/yEiVsNlEyno (Accessed: 6 August 2020). Hufford, D.J. (1989) The Terror That Comes in the Night: An ExperienceCentered Study of Supernatural Assault Traditions. Philadelphia: University of Pennsylvania Press. Humanoid (1988) Stakker Humanoid [music video]. Westside Records. id Software (1993) Doom [video game]. MS-DOS, List. id Software (1996) Quake [video game]. MS-DOS, GT Interactive. id Software (1999) Quake III Arena [video game]. Windows, Activision. Igliori, P. (2002) American Magus [film]. Inandout Digital Productions. Screened at: Equinox Festival, Conway Hall, London, 12–14 June 2009. ISSUE Project Room (2020) ‘Floating Points Residencies’. Online: https://iss ueprojectroom.org/program/floating-points-residencies (Accessed: 16 April 2020). Jaguar (1998) ‘Animal’, Animal. 12”, Thursday Club Recordings. Jefferson Airplane (1967) White Rabbit. 7”, RCA Victor. The Jimi Hendrix Experience (1968) ‘And the God’s Made Love’, Electric Ladyland . 2xLP, Polydor. Johnson, A. and Stax, M. (2006) ‘From Psychotic to Psychedelic: The Garage Contribution to Psychedelia’, Popular Music and Society, 29(4), pp. 411– 425. Jones, S. (2017) Black Culture, White Youth: The Reggae Tradition from JA to UK. Independently published. Kalev, M. (2018) ‘Into the Uncanny Valley’ [James Ferraro interview], The Wire, 416, October 2018, pp. 38–45. Katz, D. (2006) People Funny Boy: The Genius of Lee ‘Scratch’ Perry. London: Omnibus Press. Kendall, G. (2009–2010) Ikaro [electroacoustic composition]. Digital audio available online: http://garykendall.net/compositions-files/Ikaro.html (Accessed: 23 April 2021). Kiddy, C. (1994) Research & Achievement. CD, De Wolfe Music. Kiddy, C. (1996) Daytime Television. CD, De Wolfe Music. Kid Unknown (1992) ‘Nightmare’, Nightmare. Digital audio, Warp. Kilroy, N. and Rutterford, A. (2010) ‘Alex Rutterford on the Gantz Graf Video’, Warp. Online: https://warp.net/updates/alex-rutterford-on-the-cre ation-of-the-gantz-graf-video (Accessed: 12 August 2020). Klüver, H. (1971) Mescal and Mechanisms of Hallucinations. Chicago: University of Chicago Press.

212

References

Kocay, L. (2020) ‘Inside Lost Horizon: A Virtual Reality Music Festival’, Forbes, 1 June 2020. Online: https://www.forbes.com/sites/lisakocay/2020/06/ 30/lost-horizon-virtual-reality-music-festival/-cec5e7d39003 (Accessed: 6 August 2020). Konami (1999) Silent Hill [video game]. Sony Playstation, Konami. Kon, S. (2006) Paprika [feature film]. Madhouse. Kounen, J. (director) (2019) Ayahuasca: Kosmik Journey [VR application]. PC, Atlas V. Landau, D.H., Hasler, B.S., and Friedman, D. (2020) ‘Virtual Embodiment Using 180° Stereoscopic Video’, Frontiers in Psychology, 11, pp. 1229. https://doi.org/10.3389/fpsyg.2020.01229 La Peste (2005) Il Etait... ‘Magnetique’/Une Possibilite. 12”, Hangars Liquides. Laserdisc Visions (2011) New Dreams Ltd. Digital audio, Beer On The Rug. Lavrov, D. (2010) Polynomial [video game]. PC, Dmytry Lavrov Laz Inc. (2018) Rammellzee: A Roll of the Dice, 2 October–8 December 2018. Information available online: https://www.lazinc.com/exhibitions/ 1394/ (Accessed: 14 August 2020). Leary, T. (1968) ‘The Seven Tongues of God’, in The Politics of Ecstasy. Reprint, Oakland: Ronin, 1998, pp. 13–58. Len Lye Foundation (2020) ‘Free Radicals, 1958 (revised 1979)’. Online: http://www.lenlyefoundation.com/films/free-radicals/33/ (Accessed: 14 July 2020). Lewis-Williams, J.D. (1996) Discovering Southern A ican Rock Art. Claremont: David Philip. Lewis-Williams, J.D. (2007) ‘Shamanism: A Contested Concept in Archaeology’, Before Farming, 4, pp. 223–261. Lewis-Williams, J.D. and Dowson, T.A. (1988) ‘The Signs of All Times: Entoptic Phenomena in Upper Paleolithic Art’, Current Anthropology, 29(2), pp. 201–245. Liggett S. (2020) ‘Positioning the Arts in the Research Process: Perspectives from Higher Education’, in Earnshaw R., Liggett S., Excell P., and Thalmann, D. (eds) Technology, Design and the Arts—Opportunities and Challenges. Cham: Springer. Lilly, J.C. (1972) Center of the Cyclone: Looking into Inner Space. Reprint, Oakland: Ronin. Lizotte, C. (2014) ‘Oaxaca’s All-You-Can-Drink Mezcal Festival Is as Crazy as You’d Expect’, Vice, 6 August 2014. Online: https://www.vice.com/en_us/ article/jpawdx/oaxacas-all-you-can-drink-mezcal-festival-is-as-crazy-as-youdexpect (Accessed: 14 July 2020).

References

213

Lloyd, J.U. (1895) Etidorhpa: Or the End of the Earth—The Strange History of a Mysterious Being and the Account of a Remarkable Journey, Revised edition. Whitefish: Kessinger, 1992. Lomas, A. (2020) ‘Enhancing Perception of Complex Sculptural Forms Using Interactive Real-Time Ray Tracing’, EVA London 2020 (Electronic Visualisation and the Arts), pp. 206–211. https://doi.org/10.14236/ewic/EVA202 0.37. Longo, R. (1995) Johnny Mnemonic [feature film]. TriStar Pictures. Lowry, M. (1947) Under the Volcano. London: Penguin Classics. Lubman, D. (1998) ‘An Archaeological Study of Chirped Echo from the Mayan Pyramid of Kukulkan at Chichen Itza’, Acoustic Society of America, Norfolk, VA, 12–16 October 1998. Summary online: http://www.ocasa.org/MayanP yramid.htm (Accessed: 15 July 2020). Ludwig, A.M. (1969) ‘Altered States of Consciousness’, in Tart, C.T. (ed.) Altered States of Consciousness: A Book of Readings. New York: Wiley, pp. 9–22. Luke, D. (2010) ‘Rock Art or Rorschach: Is There More to Entoptics Than Meets the Eye?’, Time and Mind: The Journal of Archaeology, Consciousness and Culture, 3(1), pp. 9–28. Lux (2020) ‘Free Radicals, Len Lye’. Online: https://lux.org.uk/work/free-rad icals1 (Accessed: 14 July 2020). Lye, L. (1935) A Colour Box [artist film]. Available on: Len Lye: Rhythms, DVD, Revoir. Lye, L. (1935) Kaleidoscope [artist film]. Available on: Len Lye: Rhythms, DVD, Revoir. Lye, L. (1937) Trade Tattoo [artist film]. Available on: Len Lye: Rhythms, DVD, Revoir. Lye, L. (1940) Musical Poster [artist film]. Available on: Len Lye: Rhythms, DVD, Revoir. Lye, L. (1958, 1979) Free Radicals [artist film]. Available on: Len Lye: Rhythms, DVD, Revoir. Magnetic Fields (1992) Lotus Turbo Challenge II [video game]. Atari ST, Gremlin Interactive. Manning, P. (2004) Electronic & Computer Music, 2nd edn. Oxford: Oxford University Press. Masaaki Kobari (2012) ‘Rainbow 2000 富士1996’ [parts 1–5], YouTube. Online: https://youtu.be/C7RXIVZpqJM (Accessed: 14 August 2020). Maturana, H.R. and Varela, F.J. (1998) The Tree of Knowledge: The Biological Roots of Human Understanding. Boston: Shambhala.

214

References

Mbryonic (2018) ‘Amplify VR’. Online: https://mbryonic.com/portfolio/amp lify-vr/ (Accessed: 6 August 2020). McGurk, H. and MacDonald, J. (1976) ‘Hearing Lips and Seeing Voices’, Nature, 263, pp. 746–748. McKenna, T. (1991) The Archaic Revival . San Francisco: Harper Collins. McKenna, T. (1992) Food of the Gods. London: Rider. McLaren, N. (1940) Dots [artist film]. National Film Board of Canada. McLaren, N. and Lambart, E. (1949) Begone Dull Care [artist film]. National Film Board of Canada. McLuhan, M. (1964) Understanding Media. Reprint, London: Routledge, 2005. McNabb, M. (1978) Dreamsong [electroacoustic composition]. Available on: Dreamsong/Love in the Asylum/Mars Suite, CD, WERGO, 1993. Mebius (2018) Lost Future Omega [video game]. Nintendo Switch, Mebius. MelodyVR (2019) MelodyVR [VR experience]. Oculus Quest, MelodyVR. Merlin, M.D. (1984) On the Trail of the Ancient Opium Poppy. London: Associated University Presses. Metal Heads (1993) Terminator. 12”, Synthetic Records. Michaux, H. (2002) Miserable Miracle. New York: NYRB Classics. Migliorati, A. (2016) ‘Portrait #7: La Peste—“Para La Santísima Muerte Flashcore Live Set” [Previously Unreleased]’, Pay No Mind to Us, We’re Just a Minor Threat. Online: https://www.paynomindtous.it/portrait7-la-peste/ (Accessed: 16 April November 2020). MindSpring Memories (2018) GATEWAY 2000. Digital audio, Geometric Lullaby. Minter, J. (1984) Psychedelia. C64, Llamasoft. Minter, J. (1986) Colourspace. Atari ST, Llamasoft. Minter, J. (1987) Trip-A-Tron. Atari ST, Llamasoft. Minter, J. (1990–2003) Virtual Light Machine (VLM). Atari Jaguar/Nuon, Llamasoft. Minter, J. (2005) ‘VLM’, Llamasoft: Home of the Virtual Light Machine and the Minotaur Project Games. Online: http://minotaurproject.co.uk/vlm.php (Accessed: 1 July 2020). Mithoefer, M.C., Wagner, M.T., Mithoefer, A.T., Jerome, L., and Doblin, R. (2010) ‘The Safety and Efficacy of 3,4-MethylenedioxymethamphetamineAssisted Psychotherapy in Subjects with Chronic, Treatment-Resistant Posttraumatic Stress Disorder: The First Randomized Controlled Pilot Study’, Journal of Psychopharmacology, 25(4), pp. 439–452. Molloghan, A. (2015) The Visual Music Film. Basingstoke: Palgrave Macmillan.

References

215

Monolith (2005) F.E.A.R. [video game]. PC, Warner Bros. Monstars (2017) Rez Infinite [video game]. PC, Enhance Games. Moore, A. (1996) Dreamarena [electroacoustic composition]. Available on: Traces, CD, empreintes DIGITALes, 2000. Moritz, W. (1997) ‘The Dream of Colour Music and the Machines That Made It Possible’, Animation World Magazine 2(1). Online: http://www.awn.com/ mag/issue2.1/arti-cles/moritz2.1.html (Accessed: 1 July 2020). Moser, B. and Tayler, D. (Sound recordists) (1972) The Music of Some Indian Tribes of Colombia. 3x LP, British Institute of Recorded Sound. Narby, J. (1999) The Cosmic Serpent: DNA and the Origins of Knowledge. London: Phoenix. New Atlantic (1992) ‘Yes to Satan’, I Know. 12”, 3 Beat Music. Ninja Theory (2017) Hellblade: Seanu’s Sacrifice [video game]. PC, Ninja Theory. Nishimoto, S., Vu, A.T., Naselaris, T., Benjamini, Y., Yu, B., and Gallant, J. (2011) ‘Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies’, Current Biology, 21(19), pp. 1641–1646. Niv, Y., Seldess, Z., and Graydon, A. (2006) ‘Street Fighter (an Improvisational Ballet)’ [interactive sound artwork]. Information online: http://www.zachar yseldess.com/works.html (Accessed: 17 July 2020). Origin Unknown (1996) Valley of the Shadows. 12”, Ram Records. Oswald, J. (2004) ‘Bettered by the Borrower: The Ethics of Musical Debt’, in Cox, C. and Warner, D. (eds) Audio Culture: Readings in Modern Music. New York: Continuum, pp. 131–137. Ôtomo, K. (1988) Akira [feature film]. Toho. Parmerud, Å. (2005) Dreaming in Darkness [electroacoustic composition]. Available on: Nécropolis, CD, empreintes DIGITALes, 2016. Perchuk, A. and Singh, R. (eds) (2010) Harry Smith: The Avant-Garde in the American Vernacular. Los Angeles: Getty Publications, pp. 15–62. Peters, D. (1951) Pen Point Percussion [documentary film]. National Film Board of Canada. Phillips, C. (2007) The Art & Architecture of the Aztec & Maya. London: Southwater. Phillips, R. (2006) Mushrooms. London: Macmillan. Pickles, S. and Oliver, J. (2003) q3apd [interactive sound artwork]. Formerly documented online: http://www.selectparks.net/archive/q3apd. htm (Accessed: 9 April 2010). Current video documentation available on Vimeo: https://vimeo.com/1380686 (Accessed: 17 July 2020).

216

References

Polgár, T. (2005) Freax: A Brief History of the Computer Demoscene. CSWVerlag, Winnenden. Pollard, A. (2020) ‘“My Neighbours Think I’m Losing It”: From Livestreams to Virtual Reality Raves, How Musicians Are Coping Without Touring’, The Independent, 13 June 2020. Online: https://www.independent.co.uk/ arts-entertainment/music/features/musicians-coronavirus-lockdown-livest reams-raves-phoebe-bridgers-rufus-wainwright-tim-burgess-a9561946.html (Accessed: 6 August 2020). Prospa (2020) ‘Prospa—Rave Science Virtual Warehouse 001’, YouTube. Online: https://youtu.be/74hbr46dWK0 (Accessed: 26 October 2020). Purves, D., Augustine, G.J., Fitzpatrick, D., Katz, L.C., LaMantia, A.S., McNamara, J.O., and Williams, S.M. (2001) ‘The Auditory Cortex’, in Purves, D., Augustine, G.J., Fitzpatrick, D., Katz, L.C., LaMantia, A.S., McNamara, J.O., and Williams, S.M. (eds) Neuroscience, 2nd edn. Sunderland (MA): Sinauer Associates. Available from: https://www.ncbi.nlm.nih. gov/books/NBK10900/. Ragga Twins (1991) ‘Wipe the Needle’, Wipe the Needle. 12”, Shut Up and Dance Records. Ratcliffe, R. (2012) ‘New Forms of Hybrid Musical Discourse; an Exploration of Stylistic and Procedural Cross-Fertilisation Between Contemporary Art Music and Electronic Dance Music’ [unpublished PhD thesis], Keele University. Remedy Entertainment (2001) Max Payne [video game]. Sony PlayStation 2, Rockstar Games. Resolution (2020) ‘Alexis x L’Aubaine | Live at Venue MOT’, YouTube. Online: https://youtu.be/5rLk_cQonf4 (Accessed: 6 August 2020). Reynolds, S. (2008) Energy Flash: A Journey Through Rave Music and Dance Culture. London: Picador. Riflemaker (2005) Dead Aim: The Unseen Art of William S. Burroughs. London: Riflemaker. Robinson, J. (2007) ‘Tripping the Lights’, The Guardian, 14 April 2007. Online: http://www.theguardian.com/artanddesign/2007/apr/14/art. culture/print (Accessed: 25 October 2015). Rockstar North (2013) Grand Theft Auto V [video game]. Sony PlayStation 3, Rockstar Games. Ross, S. and Clements, D. (2018) Excel 1979–1992. New York: Paper Work NYC. Rouget, G. (1985) Music and Trance: A Theory of the Relations Between Music and Possession. Chicago: University of Chicago Press.

References

217

Rubin, D.S. (ed.) (2010) Psychedelic: Optical and Visionary Art Since the 1960s. Cambridge, MA: MIT Press. Rush, R. (1968) Psych-Out [feature film]. American International Pictures (AIP). Russell, J. (1980) ‘A Circumplex Model of Affect’, Journal of Personality and Social Psychology, 39(6), pp. 1161–1178. Russell, K. (1980) Altered States [feature film]. Warner Bros. Russet, R., and Starr, C. (1976) Experimental Animation: Origins of a New Art. New York: Da Capo Press. Russolo, L. (1913) ‘The Art of Noises: The Futurist Manifesto’, in Cox, C. and Warner, D. (eds) Audio Culture: Readings in Modern Music. New York: Continuum, pp. 10–14, 2004. Sabina, M. (1957) Mushroom Ceremony of the Mazatec Indians of Mexico. LP, Folkways. Sanchez-Vives, M.V. and Slater, M. (2005) ‘From Presence to Consciousness Through Virtual Reality’, Nature Reviews Neuroscience, 6, pp. 332–339. https://doi.org/10.1038/nrn1651. Sanden, P. (2013) Liveness in Modern Music: Musicians, Technology, and the Perception of Performance. New York: Routledge. Sanei, S. and Chambers, J.A. (2007) EEG Signal Processing. Chichester: Wiley. Sartre, J.P. (1938) Nausea. Reprint, London: Penguin Classics, 2000. Savage, J. (1996) Highflyers: Club Rave Party Art. London: Booth-Clibborn. Schacter, D.L. and Tulving, E. (1994) ‘What Are the Memory Systems of 1994?’ in Schacter, D.L. and Tulving, E. (eds) Memory Systems. Cambridge, MA: MIT Press, pp. 1–38. Schafer, R.M. (1994) The Soundscape: Our Sonic Environment and the Tuning of the World . Rochester, VT: Destiny Books. Schaffner, F.J. (1968) Planet of the Apes [feature film]. Twentieth Century Fox. Schankin, C.J., Maniyar, F.H., Digre, K.B., and Goadsby, P.J. (2014) ‘“Visual Snow”—A Disorder Distinct from Persistent Migraine Aura’, Brain, 137, pp. 1419–1428. https://doi.org/10.1093/brain/awu050. Schultes, R.E., Hofmann, A., and Rätsch, C. (1996) Plants of the Gods: Their Sacred, Healing and Hallucinogenic Powers, 2nd edn. Rochester, VT: Healing Arts Press. Scott, M., Nuttgens, S., Upton, A., and Platts, M. (1996) Success & Achievement. De Wolfe Music. Scott, R. (1979) Alien [feature film]. Twentieth Century Fox. Scott, R. (1982) Blade Runner [feature film]. Warner Bros. Second Phase (1991) Mentasm. 12”, R&S Records.

218

References

Serafin, S., Erkut, C., Kojs, J., Nilsson, N.C., and Nordahl, R. (2016) ‘Virtual Reality Musical Instruments: State of the Art, Design Principles, and Future Directions’, Computer Music Journal , 40(3), pp. 22–40. Shave, T. (2008) ‘Communicative Contract Analysis: An Approach to Popular Music Analysis’, Organised Sound, 13(1), pp. 41–50. Shave, T. (2013) ‘Hybridism; a Practice-Led Investigation’ [unpublished PhD thesis], Keele University. Sherratt, A. (1995) ‘Alcohol and Its Alternatives: Symbol and Substances in PreIndustrial Cultures’, in Goodman, J., Lovejoy, P.E., and Sherratt, A. (eds), Consuming Habits: Drugs in History and Anthropology. London: Routledge. Sicko, D. (2010) Techno Rebels: The Renegades of Electronic Funk, 2nd edn. Detroit: Painted Turtle. Signore, J.D. (2007) ‘Joshua White: The Joshua Light Show’, Gothamist, 2 April 2007. Online: http://gothamist.com/2007/04/02/interview_joshu.php (Accessed: 25 October 2015). Sinclair, M. (1994) Neutral Atmospheres. CD, Chappell Recorded Music Library. Singh, R. (2010) ‘Harry Smith, an Ethnographic Modernist in America’, in Perchuk, A. and Singh, R. (eds) Harry Smith: The Avant-Garde in the American Vernacular. Los Angeles: Getty Publications, pp. 15–62. Sitney, P.A. (1965) ‘Harry Smith Interview’, in Sitney, P.A. (ed.) Film Culture Reader. Reprint, New York: Cooper Square Press, 2000, pp. 260–276. Sitney, P.A. (1979) Visionary Film: The American Avant-Garde 1943–1978, 2nd edn. Oxford: Oxford University Press. Slater, M. (2009) ‘Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments’, Philosophical Transactions of the Royal Society B, 364, pp. 3549–3557. https://doi.org/10.1098/rstb.2009. 0138. Slater M. and Wilbur, S. (1997) ‘A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments’, Presence Teleoperators and Virtual Environments, 6(6), pp. 603–616. https://doi.org/10.1098/rstb.2009.0138. Slayer (1990) Seasons in the Abyss. CD, Def American Records. Smalley, D. (1986) ‘Spectro-Morphology and Structuring Processes’, in Emmerson, S. (ed.) The Language of Electroacoustic Music. Basingstoke: Macmillan Press, pp. 61–93. Smalley, D. (1997) ‘Spectromorphology: Explaining Sound-Shapes’, Organised Sound , 2(2), pp. 107–126.

References

219

Smith, H (1939–1957) Early Abstractions [artist film]. Available on: Early Abstractions & Mirror Animations, DVD, Harry Smith Archives. Smith, H. (Sound recordist) (1973) Kiowa Peyote Meeting. CD, Reissue, Smithsonian Folkways, 2009. Smith, H. and Dean, R.T. (2009) Practice-Led Research, Research-Led Practice in the Creative Arts. Stockport: Edinburgh University Press. Soundcat VJ (2018) ‘Paul Oakenfold - Goa Mix 1994 + 4K VJ Mix’, YouTube. Online: https://youtu.be/j1NNGF1ic5o (Accessed: 11 August 2020). SP 23 (1993) ‘Underworld’, on Network 23. 12”, Rabbit City Records. St. John, G. (2009) Technomad: Global Raving Countercultures. London: Equinox. Strassman, R. (2001) DMT: The Spirit Molecule. Rochester, VT: Park Street Press. Stubhub (2020) ‘How to Enjoy Your Favourite Festivals in Summer 2020’, Stubhub, 10 June 2020. Online: https://www.stubhub.co.uk/magazine/vir tual-festivals-summer-2020 (Accessed: 6 August 2020). Studio !K7 (Producer) (1993–1998) X-Mix [VJ mix series]. Studio !K7. Swan, D.C. (1999) Peyote Religious Art: Symbols of Faith and Belief. Jackson: University Press of Mississippi. Tagg, P. (2016) ‘Intel Inside Jingle Analysis’, Vimeo. Online: https://vimeo.com/ 150585474 (Accessed: 3 September 2020). Tanner, G. (2016) Babbling Corpse: Vaporwave and the Commodification of Ghosts. Zero Books, Alresford. Taruffi, L. and Küssner, M.B. (2019) ‘A Review of Music-Evoked Visual Mental Imagery: Conceptual Issues, Relation to Emotion, and Functional Outcomes’, Psychomusicology: Music, Mind and Brain, 29(2–3), pp. 62–74. Online: http://dro.dur.ac.uk/27482/ (Accessed: 13 April 2020). Teknoshaman (2015) Flood City. 10”, Hardcore Jewellery. Tipper (2000) ‘LED Down’, The Critical Path. CD, Higher Ground. Todd, S. and Latham, W. (1992) Evolutionary Art and Computers. London: Academic Press. Toffler, A. (1970) Future Shock. Reprint, London: Pan Books, 1972. Tolentino, J. (2018) ‘The Overwhelming Emotion of Hearing Toto’s “Africa” Remixed to Sound Like It’s Playing in an Empty Mall’, The New Yorker, 15 March 2018. Online: https://www.newyorker.com/culture/rabbit-holes/ the-overwhelming-emotion-of-hearing-totos-africa-remixed-to-sound-likeits-playing-in-an-empty-mall (Accessed: 3 September 2020). Tomlin, J. (2020) Junior Tomlin: Flyer and Cover Art. London: Velocity Press. Toto (1982) ‘Africa’, on Toto IV . Cassette, CBS.

220

References

Transpontine, N. and Harrison, M. (2013) ‘Spiral Tribe Interview’, Datacide 13. Also available online: https://datacide-magazine.com/spiral-tribe-interv iew/ (Accessed: 12 August 2020). Treisman, A.M. (1960) ‘Contextual Cues in Selective Listening’, Quarterly Journal of Experimental Psychology, 12(4), pp. 242–248. https://doi.org/10. 1080/17470216008416732. Trickett, T. (2020) ‘Experiencing Visual Music from an Artist’s and Listener’s Point of View’, EVA London 2020 (Electronic Visualisation and the Arts), pp. 85–90. https://doi.org/10.14236/ewic/EVA2020.14. Truax, B. (1997) Pendlerdrøm [electroacoustic composition]. Available on: Islands, CD, Cambridge Street Records, 2001. Truax, B. (2002) ‘Genres and Techniques of Soundscape Composition as Developed at Simon Fraser University’, Organised Sound , 7(1), pp. 5–14. Truax, B. (2004–2005) The Shaman Ascending [electroacoustic composition]. Available on: Spirit Journies, CD, Cambridge Street Records, 2007. Truax, B. (2009) Chalice Well [electroacoustic composition]. Available on: The Elements and Beyond , CD, Cambridge Street Records, 2014. Trumbull, D. (1983) Brain Storm [feature film]. MGM. Tycoon To$h & Terminator Troops (1997) ‘Bed Jam Session’. Various Artists, Major Force—The Original Art-form. 5x 12”, Mo Wax. Ubisoft Montreal (2012) Far Cry 3 [video game]. Sony PlayStation 3, Ubisoft. United Game Artists (2001) Rez [video game]. Sega Dreamcast, Sega. U.N.K.L.E Featuring Nigo & Scratch Perverts (1998) ‘Ape Shall Never Kill Ape (Twin Tower Mix)’, Ape Shall Never Kill Ape. 12”, Toy’s Factory. Valentish, J. (2019) ‘Acid Test: How Psychedelic Virtual Reality Can Help end Society’s Mass Bad Trip’. Online: https://www.theguardian.com/tec hnology/2019/mar/26/acid-test-how-psychedelic-virtual-reality-can-end-soc ietys-mass-bad-trip (Accessed: 26 October 2020). Valve (2020) Half Life: Alyx [VR experience]. Windows, Valve. Vandevenne, L. (2004) ‘Lode’s Computer Graphics Tutorial—Plasma’. Online: https://lodev.org/cgtutor/plasma.html (Accessed: 11 August 2020). Vaperror (2020) Saccharine Synergy. Digital audio, 100% Electronica. Various Artists (1972) Nuggets: Original Artyfacts from the First Psychedelic Era, 1965–1968. CD, Rhino, 1998. Various Artists (1996) Platinum Breakz. 2xCD, Metalheadz. Various Artists (2013) Mayan Loop. Digital audio, Apuruami Records. Various Artists (2018) Mutable Audio. Digital audio, Anti-Narcose. Veal, M.E. (2007) Dub: Soundscapes and Shattered Songs in Jamaican Reggae. Middletown: Wesleyan University Press.

References

221

Verne, J. (1864) Journey to the Center of the Earth. Reprint: Independently published, 2020. Vitebsky, P. (1995) The Shaman: Voyages of the Soul; Trance, Ecstasy and Healing from Siberia to the Amazon. London: Duncan Baird, 2008. Walmsley, D. (2009) ‘Dubstep’, in Young, R. (ed.) The Wire Primers: A Guide to Modern Music. London: Verso. Wasson, R.G. (1968) Soma: Divine Mushroom of Immortality. New York: Harcourt, Brace. Wasson, R.G., Cowan, G., Cowan, F., and Rhodes, W. (1974) María Sabina and Her Mazatec Mushroom Velada [including musical score and four cassette recordings]. New York: Harcourt Brace Jovanovich. Webster, A. (2020) ‘Travis Scott’s First Fortnite Concert Was Surreal and Spectacular’, The Verge, 23 April 2020. Online: https://www.theverge.com/ 2020/4/23/21233637/travis-scott-fortnite-concert-astronomical-live-report (Accessed: 6 August 2020). Weinel, J. (2007) ‘Flashcore: Earth Atomizer, Let’s Go’, Spannered , 20 May 2007. Online: http://www.spannered.org/music/1181/ (Accessed: 13 April 2020). Weinel, J. (2009a) ‘Dagon Live’, Vimeo. Online: https://vimeo.com/14004976 (Accessed: 16 April 2020). Weinel, J. (2009b) ‘Drone Wolf ’, Vimeo. Online: https://vimeo.com/14007001 (Accessed: 16 April 2020). Weinel, J. (2011) ‘Quake Delirium: Remixing Psychedelic Video Games’, Sonic Ideas/Ideas Sónicas, 3(2), pp. 22–29. Weinel, J. (2014) Entoptic Phenomena in Audio. 12”, Hardcore Jewellery. Weinel, J. (2018a) ‘Getting High on Virtual Reality’, OUPblog. Online: https:// blog.oup.com/2018/01/getting-high-virtual-reality/ (Accessed: 26 October 2020). Weinel, J. (2018b) ‘How Atari and Amiga Computers Shaped the Design of Rave Culture’, OUPblog, 5 April 2018. Online: https://blog.oup.com/2018/ 04/atari-amiga-computers-design-rave-culture/ (Accessed: 12 August 2020). Weinel, J. (2018c) ‘Improvising with Light: Nova Express Psychedelic Light Show’, OUPblog, 14 December 2018. Online: https://blog.oup.com/2018/ 12/nova-express-psychedelic-light-show/ (Accessed: 29 July 2020). Weinel, J. (2018d) Inner Sound: Altered States of Consciousness in Electronic Music and Audio-Visual Media. New York: Oxford University Press. Weinel, J. (2018e) ‘London Keeps It Psychedelic at Audio-Visual Performing Arts Festival’, OUPblog, 17 July 2018. Online: https://blog.oup.com/2018/

222

References

07/london-psychedelic-audio-visual-art-festival-splice/ (Accessed: 1 July 2020). Weinel, J. (2018f ) ‘Synaesthetic Overdrive’, in Weinel, J. (ed.) Inner Sound: Altered States of Consciousness in Electronic Music and Audio-Visual Media. New York: Oxford University Press. Weinel, J. (2020) ‘Visualising Rave Music in Virtual Reality: Symbolic and Interactive Approaches’, EVA London 2020 (Electronic Visualisation and the Arts), pp. 78–84. https://doi.org/10.14236/ewic/EVA2020.13. Weinel, J. and Cunningham, S. (2014) ‘Digitized Direct Animation: Creating Materials for Electroacoustic Visual Music Using 8mm Film’, eContact! 15(4). Online: https://econtact.ca/15_4/weinel-cunningham_directa nimation.html (Accessed: 14 July 2020). Weinel, J. and Cunningham, S. (2017) ‘Simulating Auditory Hallucinations in a Video Game: Three Prototype Mechanisms’, ACM Proceedings of Audio Mostly 2017 , Queen Mary University of London, UK. https://doi.org/10. 1145/3123514.3123532. Weinel, J. and Cunningham, S. (2019) ‘Designing Game Audio Based on Avatar-Centred Subjectivity’, in Filimowicz, M. (ed.) Foundations in Sound Design for Interactive Media. Abington: Routledge, pp. 59–77. Weinel, J. and Cunningham, S. (2021) ‘Practice-Led and Interdisciplinary Research Investigating Affective Sound Design’, in Filimowicz, M. (ed.) Doing Research in Sound Design. Abington: Taylor & Francis, pp. 167–184. https://doi.org/10.4324/9780429356360. Weinel, J., Cunningham, S., and Griffiths, D. (2014a) ‘Easter Eggs: Hidden Tracks and Messages in Musical Mediums’, International Computer Music Conference/Sound and Music Computing Conference (ICMC/SMC) 2014, 14– 20 September 2014, Athens, Greece. Online: http://hdl.handle.net/2027/ spo.bbp2372.2014.024 (Accessed: 4 September 2020). Weinel, J., Cunningham, S., and Griffiths, D. (2014b) ‘Sound Through the Rabbit Hole: Sound Design Based on Reports of Auditory Hallucination’, ACM Proceedings of Audio Mostly 2014, Aalborg University, Denmark. https://doi.org/10.1145/2636879.2636883. Weinel, J., Cunningham, S., Griffiths, D., Roberts, S., and Picking, R. (2014a) ‘Affective Audio’, Leonardo Music Journal , 24, pp. 17–20. Weinel, J., Cunningham, S., and Pickles, J. (2018) ‘Deep Subjectivity and Empathy in Virtual Reality: A Case Study on the Autism TMI Virtual Reality Experience’, in Filimowicz, M. (ed.) New Foundations in Third Wave Human-Computer Interaction: Vol.1—Technologies. Berlin: Springer, pp. 183–203.

References

223

Weinel, J., Cunningham, S., Roberts, N., Griffiths, D., and Roberts, S. (2015a) ‘Quake Delirium EEG: A Pilot Study Regarding Biofeedback-Driven Visual Effects in a Computer Game’, IEEE Internet Technologies and Applications 2015, Wrexham Glyndwr ˆ University, Wales. https://doi.org/10.1109/ITe chA.2015.7317420. Weinel, J., Cunningham, S., Roberts, N., Roberts, S., and Griffiths, D. (2014b) ‘EEG as a Controller for Psychedelic Visual Music in an Immersive Dome Environment’ [extended abstract], EVA London 2014 (Electronic Visualisation and the Arts), pp. 188–189. https://doi.org/10.14236/ewic/EVA201759 4.45. Weinel, J., Cunningham, S., Roberts, N., Roberts, S., and Griffiths, D. (2015b) ‘EEG as a Controller for Psychedelic Visual Music in an Immersive Dome Environment’, Sonic Ideas/Ideas Sónicas, 7(14), pp. 81–92. Wilson, P. and Ransom, G. (1996) Cyberscience. CD, Bruton Music. The Winstons (1969) Color Him Father/Amen Brother. 7”, Metromedia Records. Yalkut, J. (1968) Turn, Turn, Turn [artist film]. Screened at: ‘Head Trips: Abstraction and Infinity—Visual Music Films’, Barbican, London, 4 October 2015. Yamashirogumi, G. (1993) ‘Dolls’ Polyphony’, Akira—Original Motion Picture Soundtrack. CD, Demon Soundtracks. Zander, T.O., Kothe, C., Jatzev, S., and Gaertner, M. (2010) ‘Enhancing HumanComputer Interaction with Input from Active and Passive BrainComputer Interfaces’, in Tan, D., Nijholt, A. (eds) Brain-Computer Interfaces. Human-Computer Interaction Series. London: Springer, pp. 181–199. Zinman, G. (2008) ‘The Joshua Light Show: Concrete Practices and Ephemeral Effects’, American Art, 22(2), pp. 17–21.

Index

A

Acid techno 154, 179, 191 ASC Sim 121–124 Atomizer Live Patch 57, 58, 64 Atomizer Visual 84, 88 Attention 116, 118, 122, 124 Auditory hallucination(s) 5, 8, 9, 18, 33, 43, 121–124, 126, 196, 198 Augmented Reality (AR) 100, 163, 165, 167 ‘Avatar-centred subjectivity’ 127 Ayahuasca 10, 12, 19

B

Bass Drum, Saxophone and Laptop 68, 76 ‘Bass meditation’ 38, 87 Belson, Jordan 16

Biofeedback 116, 120, 121, 128, 197, 198 Brakhage, Stan 90 Bug Powder Dust 159

C

‘Cartography of ecstatic and meditative states’ 5, 87 Cenote Sagrado 100–102 Cenote Zaci 98–100 ‘Circumplex model of affect’ 6 Cyberdelics 198 Cyberdream 174, 176–178, 180–184, 199 Cyberpunk 36, 37, 108, 109, 128, 172, 179

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 J. Weinel, Explosions in the Mind, Palgrave Studies in Sound, https://doi.org/10.1007/978-981-16-4055-1

225

226

Index

D

H

Demo effect(s) 17, 134, 141, 181 Demoscene 17, 134, 139 Direct animation 82, 90, 91, 93, 95–98, 101, 134, 138, 148, 154 DMT 8, 10, 41–43 Dreaming 4, 5, 7, 14, 15, 32 Dreams 4, 12, 14, 38, 108, 117 Drum and bass 14, 34, 86, 87, 153, 158, 159, 163 Dub reggae 13, 37, 44, 150 Dubstep 34, 37–39, 44

Hauntology 174, 176 Hobson, J.A. 4, 7, 14, 29, 122 Holodeck 188, 189 Holo Point Break 160 Hypnogogic hallucinations 5

E

Electroacoustic music 14, 29, 63, 86, 97, 114, 194 Electroencephalograph (EEG) 116, 118–120 Enter Soundcat 163 Entoptic Phenomena 40–43 Entoptic Phenomena in Audio 63, 64 F

Films 13 Fischman, Rajmil 60, 95, 188 Flashcore 37, 39 Form constants 8, 10, 16, 41, 42, 44, 45, 50, 57, 60, 88, 95, 98, 118, 119, 156 Fulldome(s) 16, 18, 118, 119 G

Garage rock 13, 32 Granular synthesis 33, 37, 61, 87

I

ISSUE Project Room 63, 67

K

Kendall, Gary 15 Klüver, H. 8, 10, 16, 40, 42, 44, 45, 50, 57, 88, 95, 98, 118, 119, 156, 179

L

La Peste 28, 57 Leary, Timothy 8, 33, 34, 36–39 Literary works 13 LSD 8, 9, 11, 12, 36, 45 Lye, Len 90

M

Max/MSP 37, 42, 57, 68, 112–114, 118, 119, 197 Max/MSP/Jitter 83, 84, 112 McLaren, Norman 90 MDMA 5, 11 Meditation 16, 87, 116, 118 Mescaline 8, 12, 40, 45, 57 Mezcal Animations 93, 95, 96 Multimodal(ity) 9, 30, 111, 167, 168, 200 Music visualisers 16

Index

N

Nausea 45, 46, 48 Night Breed 33–36 Night Dream 38, 39 Nova Express 16, 132, 133 P

Painting(s) 12, 13, 15, 31, 50, 82, 83, 156, 157, 159–161, 163, 179, 199 PANcho 95 Peyote 8, 10, 11, 34 Plasma 139, 141, 154, 180 ‘Practice-led research’ 20 Processing 18, 83, 98, 118, 119, 139, 141, 146, 197 Psilocybin 2, 8, 34, 43, 68 Psych Dome 118, 120 Psychedelic light show(s) 16, 17 Psy-trance 14, 33, 131, 132 Q

Quake Delirium 112–115 Quake Delirium EEG 116

227

Shamanism 6, 10–12, 14, 15, 40, 41, 44 ‘Sleep paralysis’ 5 Smith, Harry 11, 16, 56, 82, 87, 90 Soundcat 2000 163 Soundcat S-101 163 Sound-to-image hallucination(s) 9, 16, 103, 142, 167, 200 Sound toy(s) 183–186 Spectromorphology 29 Speedcore techno 37, 181 State-space 7, 122 Strassman, R. 8 Surfer Stem 36–38 Surrealism 12, 160 Swamp Process 44 Synaesthesia 2, 9, 19 T

Technoshamanism 181 31 Seconds 158 Tiny Jungle 86, 87 Trance(s) 4–6, 11, 12, 84, 87, 90, 95, 101 Trip at the Brain 157 Truax, Barry 14, 30, 40

R

Rammellzee 160 Rave flyer(s) 17, 139, 142, 172, 174, 177 Rouget, G. 4, 6, 87

U

Unity 121, 123, 126, 177, 182, 198 V

S

Seasons in the Abyss 160 Serato Scratch 151 ‘Seven levels of energy consciousness’ 34, 36, 38, 39

Vaporwave 56, 134, 141, 147, 148, 152, 174–177, 179–181, 188 VDMX 17, 133, 141, 142, 144, 146, 147, 149 Video games 18, 19, 111, 116 Video mapping 18

228

Index

Virtual Reality (VR) 19, 20, 36, 38, 109, 111, 172, 174, 176–178, 181–183, 188, 189, 192, 197–199 Visual music 15, 16, 18, 19, 82, 83, 90, 93, 132 Visual patterns of hallucination(s) 32, 35, 39–43, 46, 57, 58, 66, 67, 69, 73, 76, 87, 88, 95, 118, 179, 196, 198 VJ culture 17

VJ London 143, 144, 151, 154 VJ loop(s) 135, 138, 139, 141 VJ mix(es) 17, 142–144, 148, 154, 174 VJ mixing 150 VJ performance(s) 18, 84, 132, 133, 142, 174, 199 Vortex 156

W

Wwise 183, 185, 187