Actionable Media: Digital Communication Beyond the Desktop 0190678089, 9780190678081

Citation preview

i

ACTIONABLE MEDIA

ii

iii

ACTIONABLE MEDIA Digital Communication Beyond the Desktop

John Tinnell

1

iv

1 Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark of Oxford University Press in the UK and certain other countries. Published in the United States of America by Oxford University Press 198 Madison Avenue, New York, NY 10016, United States of America. © Oxford University Press 2018 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, by license, or under terms agreed with the appropriate reproduction rights organization. Inquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above. You must not circulate this work in any other form and you must impose this same condition on any acquirer. CIP data is on file at the Library of Congress ISBN 978–0–19–067808–1 (pbk.) ISBN 978–​0–​19–​067807–​4 (hbk) 9 8 7 6 5 4 3 2 1 Paperback printed by WebCom, Inc., Canada Hardback printed by Bridgeport National Bindery, Inc., United States of America

v

CONTENTS

Acknowledgments

vii

Preface: Three Walks in Central Park

xi

Introduction: Making Media Actionable

1

1. The Invention of Ubiquitous Computing

24

2. Interpreting Post-​Desktop Practices

57

3. Futures of Computing via Histories of Writing

85

4. A Theory of Two Archives, from Cuneiform to Augmented Reality 108 5. Forms of Actionable Media

137

6. Creating Actionable Media

179

Epilogue: Kairotic Intellectuals

216

Bibliography

225

Index

239

vi

vi

ACKNOWLEDGMENTS

I’ve been very lucky. One of the luckier accidents of my life was bumping into Gregory Ulmer in a University of Florida elevator, just minutes after I checked out two of his books from the campus library. Reading the work of legendary thinkers is riveting; working with them is earthshaking. You get the chance to partake in their process and struggle through their leaps and bounds. You get to learn at ground zero. Greg’s teaching personified that most enigmatic ideal of the humanities: he taught me how to think. Whatever readers find most interesting in the following pages is sure to bear his influence. Much of this book was written at the University of Colorado Denver, where I have been spoiled with support of every kind. Foremost, the English Department brims with colleagues both brilliant and fun—​the bike rides, trivia battles, and karaoke nights have been just as vital to me as the more conventional forms of scholarly exchange. Mentorship from Nancy Ciccone and Pamela Laird continues to be enjoyable and ever valuable. I also benefited from several course releases and funding for research trips to Amsterdam, New York, San Francisco, and a stint at Stanford University’s Silicon Valley Archives to read Mark Weiser’s collected papers. From day one at CU Denver, I’ve gotten the chance to teach classes related to my research and to be fairly experimental. Doing projects with our students and exploring ideas with them afforded me a privileged, animate context for thinking about media technologies and digital practices that were, at the time, years away from entering the mainstream. Many thanks as well to all members of the CU Denver-​Boulder Media Studies reading group; about half the books we’ve discussed make appearances in this one. Several lines of inquiry emerged through conversations with fellow media scholars, artists, and researchers outside my field. At the University of Florida, I  studied amid a great network of peers theorizing digital culture:  Aaron Beveridge, Melissa Biachi, Kyle Bohunicky, Phil Bratta, Shannon Butts,

vi

v i i i   • 

Acknowledgments

Jacob Greene, Sam Hamilton, Madison Jones, Steven LeMieux, Jacob Riley, Caroline Stone, Joseph Weakland, and Jordan Youngblood. Robert Ray showed me the importance of prose style, and his scholarly writing remains a paragon. Talking with media artists Amir Baradaran, B.C. Biermann, and Conor McGarrigle, alongside interviewing technology experts/​entrepreneurs like Blair MacIntyre, Maarten Lens-​Fitzgerald, Christine Perey, and Jay Wright has broadened my mind considerably. In addition, I had pivotal discussions with and/​or received memorable feedback from David Blakesley, Jay David Bolter, Jim Brown, Michael Bernard-​Donals, Scott Campbell, Nicholas Carr, Maria Engberg, Amy Hasinoff, Remi Kalir, Michael Jenson, Sean Morey, Brett Oppegaard, Amy Propen, John-​Michael Rivera, Bob Stein, Annette Vee, and Tam Vu. My sincerest appreciation goes those interlocutors who also read and commented on various drafts of the manuscript: Michelle Comstock, Sidney Dobrin, Ulrik Ekman, Lori Emerson, Jason Farman, Laurie Gries, Rodney Herring, Elizabeth Kleinfeld, Jack Stenner, and Greg Ulmer. Three anonymous reviewers for Oxford University Press collectively delivered an inspiring blend of informed assessments, constructive challenges, and deft suggestions that propelled me through the final revisions. My editor at Oxford, Hallie Stebbins, was a pleasure to work with during each stage of the process. Hannah Doyle guided the book into production, and Michael Stein provided diligent copyediting. Thank you to the institutions and individuals who provided images and granted permissions. Portions of c­ hapter 5 have been adapted from my article “From WIMP to ATLAS:  Rhetorical Figures of Ubiquitous Computing,” in Computational Culture 5 (2016). Chapter  3 draws some material from “Grammatization:  Bernard Stiegler’s Theory of Writing and Technology,” Computers and Composition 37 (2015):  132–​146. The section on Monet in ­chapter 6, and a couple of paragraphs in ­chapter 5, have been condensed and revised since appearing as “Computing En Plein Air:  Augmented Reality and Impressionist Aesthetics,” in Convergence 20.1 (February 2014): 69–​84. A  brief section in c­hapter  4 pulls from “Techno-​Geographic Interfaces,” Design, Mediation, and the Posthuman, edited by Dennis Weiss, Amy Propen, and Colby Emerson Reid (Lanham, MD: Lexington Books, 2014), 69–​74. I am eternally grateful to Kim, who has been my everyday support system every day over the past decade. If there’s one person without whom this book would not exist, it’s her. She was the first person to fill me with the notion that I might like to be a professor, and she helps me through each definitive challenge and laughs off the occupational hazards.

ix

Acknowledgments  • 

ix

Book writing is an endurance sport, and my mom has long been a role model in that department. The way she pushes herself makes you want to pull yourself up to her level. Her example of habitual determination made me a believer that thousands of steps, with no finish line in sight, can one day amount to something. I was a late bloomer in all things scholastic, particularly reading. My formative experiences with words and meaning were nights staying up with my dad, as we whispered through several versions of every sentence for whatever paper I had due the next morning. We would mull over syntax and excitedly debate about how readers might react, until we forgot about the assignment and how low the stakes were. A lot of my ideas still begin in conversations with him. And, lastly, I thank Ms. Murphy, the teacher who changed my life. The thought of swinging by her classroom to drop off a copy of this book has motivated me as much as anything.

x

xi

P R E FA C E

Three Walks in Central Park

Since its opening in 1859, New York’s Central Park has become an unlikely incubator for unusual media projects. I have cherry-​picked three of them for this preface. The first project approximates the birth of a tradition of public signage that has, for all its appalling constraints, unwittingly enabled the emerging genres, rhetorical principles, and digital communication practices that Actionable Media sets out to examine and critically inflect. The second of these projects (which remains among the most contested art pieces of all time) foregrounds a deep-​seated set of attitudes and assumptions concerning authorship and interpretation that Actionable Media will challenge. The third is a beacon for the book, spurring its inquiry into contemporary mediascapes. Neither Central Park nor New  York command a special place in the chapters that follow. The Park and its media projects discussed here constitute a point of departure, yielding microcosmic impressions of a burgeoning phenomenon in digital culture:  the rise of ubiquitous computing (e.g., smartphones, smartwatches, smartglasses, smart cities, smart _​_​_​_​_​_​) and the demise of desktops, both as a piece of hardware and an interface metaphor. Actionable Media scales up to this broader scope. It examines how post-​ desktop interfaces stand to impact the creation, circulation, and consumption of digital content in various public settings. Historically, the capacity to inscribe urban locales and suburban plazas has largely been restricted to commercial advertising and municipal messaging. Recent literary, artistic, and civic experiments with nascent media forms signal the prospects of a radically different future. Where there have been only stop signs, billboards, and logos, there may be libraries of content. An ever-​widening array of everyday spaces now double as platforms, archives, and forums for multimedia authorship in the here and now. The evolution of media projects in Central Park—​from Fredrick Law Olmsted’s 1860s pedestrian sign network to Janet Cardiff ’s 2004 audio walk—​marks a prominent example, but it is one site among

xi

x i i   • 

Preface

many. Increasingly, intellectuals, artists, activists, and students, in cities and towns everywhere, are starting to author the built environment. Ultimately, this new wave of digital communication—​which will later I term “actionable media”—​ripples with provocative implications for readers keen to the interplay of technology and culture, interfaces and ideas, media and expression.

I. Fredrick Law Olmsted designed every nook and pathway of Central Park with people in mind. He had a vision for how visitors should use each area—​and how they should not. Speaking of his imagined audience, Olmsted insisted, “They will need to be trained to the proper use of it, to be restrained in the abuse of it” (Rosenzweig and Blackmar 239). This delicate balance between egalitarian access and utilitarian regulation proved difficult to maintain. Several cycles of trial and error ensued, and after two major initiatives failed, Park officials decided to try something new. Olmsted’s first regulatory measure had been to hire gatekeepers who stood at the Park’s entrances and repeated rules, directions, and reprimands to each batch of visitors. Within a year, as the crowds grew and the rules multiplied, these gatekeepers could not keep up. The Park’s Board of Commissioners turned to the written page; they bought expensive space in the city’s most popular newspapers and published a complete list of the Park’s regulations in hopes that the ten-​day print run would make lasting impressions on a critical mass of citizens (Henkin 66). Ensuing behavior modifications were inconsequential. Under this system, laws mandating the speed limits for horse-​riding had to be memorized and recalled on a path-​by-​path basis. Baseball games were permitted on certain lawns on certain days at a certain time. But without any signs to reference or follow, certainty was in the eye of the beholder. Park visitors violated the rules routinely without knowing they were doing wrong. Olmsted despised street signs. He wanted his park to be an unarticulated oasis from the newly-​minted billboards, window ads, and flyers that were starting to clutter Manhattan cityscapes. Nothing on Park grounds served to orient or inform visitors in the course of their actions. They were just supposed to know better. Clearly, something of Olmsted’s vision had to give. Two of his most adamant tenets were everywhere proving incompatible. If residents, newcomers, and tourists alike were to all follow a meticulous code of conduct, then they needed signs in and around the Park to guide them. And yet, no park in the country then played host to the networks of municipal signage that now blankets cities, towns, suburbs, trails, airports,

xi

Preface  • 

xiii

and highways. Central Park in the 1860s came to be the crucible of public communication via embedded signs. Reluctantly, Park Commissioners hired an official sign painter to infill the Park with a brand of strategically anchored, pedestrian-​facing texts modeled, in part, off of the billboards that commercial advertisers had installed along popular stretches of Broadway and Bowery thirty years earlier. From his archival research on Olmstead’s papers, historian David Henkin reports, “signs increased until they formed a comprehensive guide system, directing and regulating the movements of visitors” (66). From large posters listing all Park rules in full, the park-​to-​visitor monologue splintered toward a more distributed method of display: one directive per sign, with each sign posted at the setting most relevant to its message. For instance, signs discouraging stopped carriages from blocking a certain intersection were placed at that exact spot. General postulates forbidding acts of vandalism and graffiti became concise imperatives attached to individual statues and facades:  “DO NOT MARK OR DEFACE THIS STRUCTURE” (Henkin 67, my emphasis). Here, in the most robust network of municipal signage the world had ever know, writing performed a delivery function that had typically been performed by a speaker present at the scene, like the Park’s gatekeepers. Additionally, the signs brokered a very different manner of arranging texts:  sentences and phrases ran over the dynamic grounds of a park rather than the codified space of a bound page, as they had in the Park’s newspaper campaign. These alterations to the arrangement, style, and delivery of Park guidelines effectively modified the conditions under which visitors received the messages—​much to the Park Commissioners’ liking. The environment began to command itself with an air of authority. Written words in public spaces, addressed pointedly to whoever was there, impressed a palpable level of accountability upon visitors. The signpost in situ was a disciplining extension of state power, issued in the public interest, but sealed off from public authorship.

II. Restrictions limiting the authorship of public signage have their reasons. Consider what happens to community bulletin board spaces in college campuses or local coffeehouses. You post a flyer on the board promoting your upcoming event, only to discover the next day—​the next hour—​that it has been posted over by someone else’s flyer (and perhaps ten more). It’s a paper race only the most bullish can make worth their while. High-​traffic

xvi

x i v   • 

Preface

real estate is a coveted resource; the ratio reserved for citizen texts and community media remains infinitesimal. The problem of negation-​qua-​overaccumulation could be lessened by the addition of more such boards in more public spaces, but the absence of curation and moderation would only become more egregious. Why not, then, appoint qualified officials to curate works for parks and plazas in the way that museum professionals curate gallery walls? In 1967, this is exactly what the New York City Parks Department did. Their intent, as stated by then-​Parks Commissioner August Heckscher, was to let works of art “loose in the city, to set them under the light of day where they intrude upon our daily walks and errands” (“Forty Years of Public Art”). The first outdoor exhibition, Sculpture in Environment, debuted commissioned artworks in Central Park and dozens of other spots around the city; well over a thousand temporary exhibitions in the city’s open spaces have followed in its wake. The most famous one took place over two frigid weeks in February 2005, after having been planned and fought over for twenty-​six years. Christo and Jeanne-​Claude’s The Gates, a twenty-​three mile long installation of saffron fabric panels lining the Park’s paths, reportedly “brought over four million visitors from around the world to Central Park” during its brief display (Ferrera and Maysles). Christo and Jeanne-​Claude make it a point to call the work by its full name: The Gates, Central Park, New York, 1979–​2005. Doing so casts attention to the place, time, and long process informing the eventual installation. Let us take them at their word and take the work’s proceedings as its core. Making public installations at such a scale necessarily turns the creative process into a bureaucratic nightmare. Christo and Jeanne-​Claude combine the roles of artist, architect, community builder, social entrepreneur, and politician. They employ a legal team, public relations specialists, and construction crews. Their projects cost millions to put on and run only a few weeks, occasionally months. This makes for a challenging value proposition. Because construction time dwarfs exhibition time (the pieces are dismantled and recycled when the show ends), public officials are hard pressed to justify what net public benefit will follow from an artwork that “intrudes upon our daily walks and errands” not only in its finished state but also (especially) during its nuts and bolts formation. The matter is not softened by the artists’ tireless assertions that their displayed works contain “no purpose whatsoever . . . no message” ( Jeanne-​Claude qtd. in Becker 27). Shortly after The Gates proposal was initially rejected in 1981, Harvard sociologist and (in)famous New York intellectual Nathan Glazer published a scathing if well-​mannered rebuke of

xv

Preface  • 

xv

the project. The gap between his critique and the artists’ statements reveals a range of levels on which The Gates pushes against commonplaces concerning authorship and the nature of public spaces. Representative of many likeminded commentators, Glazer could not fathom why Christo and Jeanne-​Claude refused to scale back their vision and why they seemed so willing to persist in spite of local backlash. His essay issues a series of aesthetic correctives to Christo, such as shrinking the project to a mile and organizing it in conjunction with a major holiday or festival. This would give the work a purpose, as the colorful gates would “mar[k]‌the way to the event” (Glazer 76). Much of Glazer’s advice boils down to making the piece more socially acceptable: the gates should have a palpable instrumental function (like a directional signpost), they should ornament the environment without drawing undue attention, and they should clearly resonate in service of a familiar message, meaning, or event. As such, people would less likely make a fuss. Glazer writes in jest, “Had Christo spent the $50,000 the newspapers report as being the cost of the CPC&H survey [a study showing that most minority residents along the Park’s northern edge favored The Gates], he could have put up a hundred gates” (77). What Glazer failed to see, however, was that these performances of public opinion and deliberation (e.g., surveys, hearings, protests, debates) were foundational to the project. The sonorous flapping of The Gates’ saffron fabric was also, ultimately, an echo of local dialogues about expression, place, and meaning. Glazer assessed the proposed installation as if it were aiming after the same class of effects wrought by finished artworks housed in a museum. But Christo and Jeanne-​Claude’s projects aspire otherwise, ushering art-​making and authorship toward a broader canvas that is at once a forum: Nobody discusses a painting before it is painted. Nobody discusses a sculpture before it is sculpted. But everybody discusses the building of a new airport, a new bridge, a high-​rise building before it’s built. Our projects are argued, they’re discussed in public hearings and in meetings before they are built. (Christo qtd. in Becker 26) The project’s most outspoken critics mistook The Gates for a mere sculpture. Christo and Jeanne-​Claude were building civic infrastructure, and this aspect of their work would lend significance to but also outlast their resolutely ephemeral, admittedly whimsical art. Arguments against The Gates sought to preserve the Park’s integrity by keeping it pure of all artificial contrivances, be they artistic or not. Glazer’s

xvi

x v i   • 

Preface

essay, like Parks Commissioner Gordon Davis’ decision, cited (and sided with) a startling survey of public opinion conducted on the steps of the Metropolitan Museum of Art. Less supportive than any other surveyed group, only twenty-​ two of the 164 Met patrons voted to allow The Gates. Perhaps the project struck them as too lowbrow. Perhaps they shared Glazer’s sentiment: “Central Park is not a bland landscape that . . . needs to be dressed up” (76). Glazer pitched the issue as a conflict between art and nature, between Christo’s ambition and Olmstead’s legacy. To allow the former would trample upon the latter; the project was presumed destined to spoil “the feeling of being plunged into untouched nature” (Glazer 76). Art, after all, has a proper setting of its own in museums and galleries. For Glazer, The Gates would be best executed as a collage of drawings or photographs, hung up and framed, giving museum visitors a chance to imagine the Park anew without subjecting park-​goers to any material changes (76). Art and ideas should be placed in spaces we choose to enter into on our own time. At public hearings, detractors voiced similar concerns that permitting The Gates would thereby render Central Park vulnerable to a litany of other textual, sonic, and visual media interventions. That The Gates managed to so visibly probe such sentiments, to summon them up for philosophical scrutiny, remains its most poignant contribution. While speaking bodies preaching from soapboxes are heard beyond the bounds of objection, the slightest motion to inscribe the environment is readily combated. The hand of God and the hand of Olmstead may make their marks there, but the rest of us—​even to the minds of a good many intellectuals and art patrons—​ ought not to.

III. Whereas the crowds of people experiencing The Gates resembled a parade, the audience for Janet Cardiff ’s audio walk, Her Long Black Hair (2004), meandered solitarily through the Park in secret. If you visited Central Park during the piece’s first run (the summer before The Gates opened) or its revival (the summer after The Gates closed), you might have passed through unaware of its existence. The work’s initiates carried a Sony Discman, headphones, and five numbered photographs, which they had borrowed from an unassuming kiosk at the corner of Sixth Avenue and Central Park South. They all followed the same unmarked route, pausing occasionally to hold up a photo—​studying its relationship with the live scene surrounding them—​all the while listening. Listening to Cardiff ’s verbal narration playing in the headphones, but also to a track of recorded sounds and their coalescence with proximate noises

xvi

Preface  • 

xvii

breaching through the headphones:  car horns, fragments of conversation from passersby, thunder, music old and new. Along the walk, the boundary between Cardiff ’s work and ambient ephemera, between composition and sheer circumstance, wears thin and porous. Moving from the Park’s first signposts to a project like The Gates illustrates, in broad strokes, important themes accompanying the gradual movement from municipal signage to sculptural installation; Her Long Black Hair marks the sudden entry of electronic media authored to be encountered on the Park’s grounds. Distinct from both Olmsted’s behavioral imperatives and Christo and Jeanne-​Claude’s decorative structures, Cardiff ’s words, images, and soundtrack enmesh with the Park setting to impart critical, pensive postures among her audience. Her voice guides participants on a carefully charted passage through the Park, which intermittingly chronicles scenes focusing on an unnamed dark-​haired woman (she’s pictured in three photos along the walk), spliced together with bits of Baudelaire’s poetry, slave narratives, and lyrical descriptions of the setting. Engaging with the piece, as with other audio walks from Cardiff ’s prolific oeuvre, feels like reading a cerebral narrative essay, watching an experimental documentary, or taking in an aphoristic philosophy lecture. Except you’re not seated face-​to-​page, not gazing at a film screen, nor taking notes in a lecture hall. You are walking through Central Park. You are a pedestrian in pursuit of Cardiff ’s narrative arc, her speculative inquiry, interpreting it amid and in reference to everything you perceive in its path, as Figure P.1 illustrates. On experiencing the walk, arts critic Barbara Pollack wrote, “As we listen  .  .  .  [Central Park] becomes a living theater, as if the trees, bicyclists, panhandlers and pretzel vendors were placed there as performers on Cardiff ’s stage” (Pollack 58). Of course, Cardiff has staged nothing. The effect is cultivated through her weeks spent observing life in each setting and recording a wide spectrum of sounds emanating at various locations across the Park. As this media-​setting correspondence comes into play throughout the walk, the piece thrives on an acute sense of confusion it creates for participants. Because much of the ambient soundtrack is hard to place—​did that noise come from Cardiff ’s recording or the people around me?—​and because the narration resonates eerily well with objects, scenes, and events presently occurring in the participant’s perceptual field, Her Long Black Hair practically demands that we include our singular, live experience of the Park in our reading of the work. We are led, by Cardiff ’s design, to process the incidental as if it was choreographed and charged with hermeneutic implications.

xvi

x v i i i   • 

Preface

Figure P.1  Photo of Janet Cardiff ’s audio walk Her Long Black Hair (2005), in Central Park, New York. Image © Ray Weitzenberg, 2005.

Critics and fans of Cardiff ’s walks inevitably remark on how uncanny their experiences are. Contrary to more habitual modes of interacting with electronic media and digital networks, wherein the device is a portal to elsewhere or a content-​rich nowhere, these audio walks directly address the participant’s proximate surroundings. Soundtrack and setting are synchronized to trigger maximal rhetorical, aesthetic interplay between multimedia authoring and local action happening in the here and now. Granted, Cardiff is a world-​ renowned artist whose creative sensibilities far exceed most. Learning from her work and extrapolating from her approach (see ­chapter 6) is no paint-​ by-​number endeavor. Still, I cannot help but think that she has unearthed a fruitful stratum of digital writing—​a powerful, socially viable way to inscribe vibrant locales in the service of aesthetic, political, educational, or intellectual pursuits—​that the rest of us might do well to consider at a time when the lion’s share of media consumption (e.g., reading, viewing, clicking, commenting) takes place on handheld computers and, maybe soon, on headworn displays. Her Long Black Hair invokes the very affordances that quintessentially distinguish the new breed of media interfaces. Most notably, it foreshadows our now-​pervasive ability to access and deliver digital content at precisely

xi

Preface  • 

xix

any geographical point within wireless signal range, which in turn allows anyone with Internet access to effectively mark up public settings and craft media projects around them for a walking audience. This capacity to write upon shared environments, built and natural, has largely been off limits to citizens, activists, teachers, students, etc. Cardiff ’s audio walks, along with a growing list of work by other writers, artists, and researchers, clearly indicates that a change is in progress, or at least in potentia. The discourse conventions that have regulated print texts and sculptural interventions in public space, as well as the technological specifications that have kept desktops and laptops indoors, hold little sway in contemporary digital cultures. Indeed, recent advancements in mobile, Cloud, and wearable computing make the technical underpinnings of Cardiff ’s narratives much more accessible and easier to implement now than in 2004. Just as “Web 2.0” platforms have opened online publishing to masses of nonprogrammers, newer authoring programs are beginning to make the production of audio walks, environment-​situated annotations, and augmented reality projects as learnable as blogging. Setting up the technology has become the easy part. And yet, the formative excitement gathering around this mode of digital communication—​its prospects to facilitate diverse authorship amid public settings—​also harbors a formidable set of problems, rhetorical-​aesthetic challenges, and ethical considerations. These three walks in Central Park, from Olmstead’s signposts to Cardiff ’s soundtrack, give a faint map of the larger territory that Actionable Media embarks to study. Mediascapes emerging in today’s “smart cities” and Wi-Fi-​covered towns merit wide ranging investigations that frame millennial novelties in light of pre-​digital histories, that wander beyond current commonplaces to take up alterative theories and experimental practices. The most pressing concerns for communication scholars, media theorists, rhetoricians, cultural critics, and critical media producers loom larger than monthly software updates, “game-​changing” product launches, and the latest features hyped by tech journalists. Those of us devoted to humanistic analyses of emerging genres and also, perhaps, to authoring public intellectual work in such genres must attend to broader questions about what kinds of media forms and communication practices this rising tide of post-​desktop interfaces makes possible, and the kinds of dilemmas they pose for authorship, interpretation, cultural memory, and public life. How might we theorize, critique, and produce arguments and inquiries that are designed to occupy a park and be read by its inhabitants? Can intellectual discourse learn to live a double life in the streets? Can a ubiquitous public sphere emerge from the shadows of urban ads and their moving red neon? And if so, then to what

x

x x   • 

Preface

ends? Traditional media outlets and political forums remain adequate for many purposes. What is it, then, that makes a city block or town square, with all its hustle and bustle, a compelling publishing venue for essayistic reflection, democratic deliberation, and social commentary? Furthermore, by what measures shall we gauge the value or harm of various digital initiatives to author the built environment? What concepts and critical insights might inform, contextualize, or justify such media projects as well as humanistic efforts to analyze their effects?

xxi

ACTIONABLE MEDIA

xxi

1

INTRODUCTION

Making Media Actionable Enclosed in cars, often in headphones, seldom in places where encounters are left to chance, often opting out of face-​to-​face meetings, and ever pursuing and being pursued by designed experiences, postmodern posturban city dwellers don’t become dulled into retreat from public life; they grow up that way. The challenge is to reconnect. —​M a l c o l m M c C u l l o u g h , Ambient Commons

Every rhetorician has a favorite definition of rhetoric. While writing this book I  clung to Richard McKeon’s sweeping, audacious vision. McKeon is well-​known for his insistence that rhetoric’s scope extends beyond the construction of verbal arguments; it may also perform in the capacity of an “architectonic productive art” whenever scholars of communication, writing, media, or literature refashion disciplinary commonplaces into organizing principles that promote transdisciplinary synthesis. For McKeon, rhetorical concepts uniquely equip intellectuals of all kinds to bridge specializations in effort to jumble disparate traditions of inquiry, probe longstanding differends, and identify new agendas for experimental research and cross-​cultural development. (And he walked the talk, for example, through his formative involvement with the founding of UNESCO.) His notion of rhetoric’s copious value unfolds in leaps and bounds, in grandiose assertions like this one: “Rhetoric provides the devices by which to determine the characteristics and problems of our time and to form the art by which to guide actions for the solution of our problems and the improvement of our circumstances” (11). Here and elsewhere, McKeon sounds a little overzealous. More than a little. No rhetorician can fulfill all the promises he makes on our behalf. It is naïve to be inspired by him, but I am. His definition informs the broad contours of my inquiry, which sets out to discern a problem central to contemporary digital

2

2   • I n t r o d u ct i o n

cultures—​and to develop a theoretical framework that might help reorient some things for the better. McKeon’s remarks on the uses of rhetoric were, in his later essays, frequently accompanied by statements about the exigencies and challenges arising with modern technology. Though recent scholarship on new media and digital communication rarely mentions his name, vestiges of McKeon’s mentality abound among theorists who look at computation and see rhetoric all the way down, from pixels on a screen to bits of code.1 Rhetoric, conceived architectonically, can and should provide theoretical orientation to technical invention. Amid digital innovation, rhetorical theorists may aim to intervene in the cultural production of interfaces and devices, in addition to our more traditional focus on the persuasive content that circulates across a given medium. The constitution of such technologies, after all, levies a profound influence on the communicative flows they enable, constrain, and preserve. Digital interfaces, like them or not, collectively serve as the preeminent forum and most vital archive for an ever-​increasing portion of the world’s population. Few industries or professions even attempt to feign indifference toward this media seachange, least of all those concerned with communication. McKeon anticipated the societal ripples that would ensue once computers (or “calculating machines”) hit the mainstream. In the same breath, he marshals rhetoricians and humanists toward a new agenda:  “The architectonic productive art in an age of ‘technology’ is obviously technology itself given a rhetorical transformation” (52). When machines move into the sphere of knowledge production, from the factory to the office, the fabrication of technical systems acquires unprecedented rhetorical stakes, and thus merits close rhetorical analysis. As communication scholar David Depew suggests, McKeon was anxiously aware . . . that none of these new tools, techniques, and technologies is by itself an art, let alone an architectonic art. Their potential as sources of invention and tools of inquiry cannot be realized unless and until a universal rhetoric of communication that is as philosophically ambitious as Cicero’s mitigates, mediates, and ultimately resolves the conflict between the humanities and technosciences. (43) Cicero is one of McKeon’s most cherished figures, and since his call to theorize emerging technologies in rhetorical terms is issued parallel to a summary of Cicero’s objectives, we would do well to grasp the latter before embarking on the former.

3

Introduction  • 

3

In “The Uses of Rhetoric in a Technological Age,” McKeon characterizes Cicero’s achievements in majestic terms: Cicero embodies the architectonic rhetorician qua heroic problem solver. The central problem with Roman culture, as Cicero saw it, was that wisdom and eloquence had grown apart. Those who could speak clearly and convincingly “were ignorant of what they talked about”; those trained in dialectical reasoning often failed to communicate the weight of their insights to a broader public (18). (This may sound familiar.) While Cicero didn’t solve this problem once and for all, McKeon says he did help fix it for a while. How? By enlarging rhetoric into an architectonic productive art. For instance—​as we move from Aristotle to Cicero—​political rhetoric becomes “deliberative rhetoric,” forensic rhetoric becomes “judicial rhetoric,” and epideictic becomes “demonstrative.” In each case, these genres of rhetoric become recoded into transdisciplinary links. Cicero binds the scene of persuasion to a philosophical determination of causes and an ethical consideration of effects. As McKeon puts it: “the certainties and necessities of proof were merged with the estimations and necessities of action” (19). Under Cicero’s program, a central aim of rhetorical education was to (re)connect philosophy and politics. I mention McKeon’s summary of Cicero’s work because it serves as an analogy for framing a growing, contemporary problem and rhetoric’s relevancy to it. Whereas Cicero’s concern was the separation of wisdom and eloquence, the problem I’m getting at is the separation of networked multimedia and local action. Let me define these two terms. By “networked multimedia,” I am simply referring to all digital texts and audiovisual content that circulate by means of the Internet, and that we access via desktops, laptops, smartphones, tablets, etc. (So, while a website in a browser is networked, a print-​out of that website’s homepage is not—​at least not in this sense.) And by “local action,” I  mean words and deeds directed at what is here and now, along with the activities of nonhuman entities and material flows occupying a given proximity. (Tossing my banana peel into a garbage bin is a local action, as is the plastic bag blowing in the wind before me.) More pointedly, local action can be understood in opposition to what software scholar Lev Manovich calls “tele-​ action.” He claims that tele-​action is “acting over [a]‌distance in real time” (167). It is the capacity to affect reality at a location where you are not geographically present. More often than not, our interactions with multimedia networks prompt us to withdrawal from local action in favor of tele-​action, if only for moments at a time. A bias for teleaction has generally sway over human-​computer interaction ever since the introduction of desktops. But it has only been recognized as

4

4   • I n t r o d u ct i o n

a cultural problem fairly recently, with the proliferation of robust handheld computers. With headworn displays and computerized contact lenses on the horizon, we face a critical transition whereby post-​desktop media ecologies may increasingly intermingle with the built environments we cohabit and move through. Accordingly, forward-​leaning digerati from multiple fields are calling for concepts attuned to this unfolding scenario. MIT architect Carlo Ratti puts it thusly: “Today we need to find new ways to make sense of the increasing wealth of digital data layered over top of physical space . . . of bits layered on the top of atoms” (“Architecture that Senses”). New theories wanted. The rise of ubiquitous computing (ubicomp) prompts us to reexamine the relationship between digital communication and our surroundings and to remap the virtual public sphere. Actionable Media is my offering, from the humble quarters of the humanities.

Digital Distraction The problem that motivates Actionable Media—​the persistent disjunction of networked multimedia and local action—​has been circled by a swarm of critics, though they state the issue in different terms. Since 2010, technology and society writers have variously bemoaned the rise and spread of “digital distraction.” A few of their points are worth grappling with here because they imply the need for a theory like actionable media. The sting of their grim analyses (from which we may intuit a more informed basis for concept creation) should be felt in earnest rather than thumbed through and scoffed at—​ a reflex indulged by too many tech enthusiasts. A Pulitzer Prize finalist in 2011, Nicholas Carr’s The Shallows argues that the Web “promotes cursory reading, hurried and distracted thinking, and superficial learning” (116). Carr, in essence, appropriates Socrates’s famous critique of writing as a means to critique the Internet. In both cases the sheer breadth of the new archive threatens a thinker’s fragile capacity to sustain rigorous, deep inquiry. Readers may refer to a text without thoroughly internalizing subtleties of form and content (as memorization tends to enforce); Net surfers can instantly search a digitized book for keywords without having to tread through an argument’s linear development. The parallels binding these critiques, prescient as they are, reveal a shared limitation. Socrates and Carr each foretell the impending erosion of hard-​won cognitive postures wreaked by a new technology’s emergence, all the while foreclosing the likelihood that subsequent developments might mitigate the damages, let  alone summon new beats in intellectual life. But Carr reveals himself to be a proponent

5

Introduction  • 

5

and beneficiary of this latter tendency, evidenced by his own excursions into media history. The poignancy of Carr’s prognosis derives largely from his skillful juxtaposition of early chapters recounting centuries of minor technical and writerly advancements (pages, page numbers, tables of contents, headings and subheadings, etc.) that incrementally enabled the deep reading habits of modern print culture, which are followed by chapters packed with clinical studies and familiar anecdotes suggesting the ravage done to literacy conventions in just two decades of online developments. Carr’s Socratic reprimand of the Web’s misgivings is at once a corrective to Socrates’ original attack on writing. Eventually, after much trial and error, writing did become a medium synonymous with the cultivation of knowledge, wisdom, and every other virtue Socrates staked on the pre-​orthographic soul. The same will likely become true of digital media someday. Writing changed a lot from the time of cuneiform clay tablets to chapbooks penned in iambic pentameter, and computing technologies are changing at a much faster rate. This is not a loophole to the mournful, pessimistic stance. Rather, it is a generative caveat already at play in claims supporting that position. What we celebrate about book reading in the late age of print is a highly evolved set of practices that generational wisdom gradually prioritized over countless other textual experiments. What we condemn in online interface practices today will have proven to be a single paradigm of forms and conventions among other paradigms primed to manifest in the coming decades. Let us return to the discourse on digital distraction, then, with another analytical imperative: to disentangle the multiple streams of invention that often go undistinguished amid homogenous indictments of the Web or the Internet in general. Revered tech psychologist Sherry Turkle advances concerns about new media similar to Carr’s, though she turns more squarely to the antisocial side effects of social networking. Examining scenes from everyday life, she presents a wealth of interviews and case studies that all hold up a mirror to the underside of constant connectivity. It has become all too normal to experience what Turkle calls “being alone together”—​when mobile users habitually devote much attention to the constant feed of posts and updates streaming into their screens, leaving little left to give the people sitting next to them. It has become all too normal to have one foot in cyberspace while the other foot is on the gas pedal. Seemingly everywhere you look, there is this programmatic misalignment among multimedia, people, and their surroundings. In several publications and interviews, Turkle references observations made during her walks around the sand dunes of Cape Cod. “I’ve walked those dunes for decades,” she says, “everybody had their eyes up to the sand, to the sky, to

6

6   • I n t r o d u ct i o n

the sea, and this summer it hit me that so many people were walking them with a device in hand” (Turkle qtd. in Nolan 58). Scenes like these bother Turkle, in part because they contradict the optimistic view of digital media that she championed throughout the 1990s. Talking about her 2011 book, Alone Together, Turkle says: “[It] is a book of repentance, in the sense that I did not see this coming” (Nolan 55). Carr makes a similar confession. The Shallows begins with a literacy narrative, detailing his initial embrace of the Internet as “[his] all-​purpose medium”; as a writer, he reveled in the hallmark affordances of Web 2.0:  researching via search engines, publishing through blogs, and interacting with readers on social media (6). But then, Carr airs a “queasy feeling” he’s been having lately: “the Net seems to be . . . chipping away my capacity for concentration and contemplation” (6). When did this revelation occur to Carr? “Over the last few years,” he gestures, presumably referencing the period around 2007 to 2010. The timing of these revelations—​these widespread acts of repentance among technology critics—​is more significant than the authors themselves point out. Their attacks on digital distraction explicitly target the Internet at large, but so many of their examples imply a more specific root cause. Not the Internet in general, but a particular configuration of the Internet—​one that has taken hold in the wake of smartphones, over the last few years. That configuration can be critiqued, challenged, and changed. Indeed, the phrase “digital distraction” didn’t really exist before the iPhone’s release in 2007. Searching Amazon for books on digital distraction published before 2007 yields no relevant results. Why is that? Why was digital distraction a non-​issue for the desktop paradigm? Well, in order for something to be labeled a “distraction,” it has to divert some of your attention away from another task—​only some. If it commands all of your attention, pulling you away entirely, then it’s no longer a distraction. Then it becomes a diversion or simply an activity unto itself. Desktops and laptops are demanding like that. They need to be plugged in or near Wi-Fi hotspots; we can’t put them in our pocket, can’t use them on the go. These limitations prevent them from being distractions. They’re either on or off. We’re either using them or doing something else. Mobile and wearable devices are all about computing while doing. Global usage statistics plot our collective affinity for the freedom of anytime-​anywhere access. January 2014 marked a milestone; for the first time, Americans spent more time accessing the Internet via mobile apps than through traditional Web browsers. (The same thing happened in China six months later.) Whenever

7

Introduction  • 

7

people read, wrote, or viewed online content, 55% of the time, it was done with mobile devices (O’Toole). That percentage is only increasing, particularly in countries that lack the infrastructure needed to fully support desktop/​ laptop computing. Industry projections suggest that after 2017, “more than 90 percent of Internet users [worldwide] will access online content through their phones” (“Statistics and Facts on Mobile Internet”). Still, these minute alterations to computing hardware should not matter that much. Our screens are smaller. We touch them. Programs and websites are formatted a little bit differently. What has changed really? In most cases, mobile interfaces simply remediate and extend the desktop metaphor. The ways in which one must interact with a desktop have now become regular ways of behaving anytime, anywhere. And that is exactly where the problem lies. These devices allow for computing while doing, but they’re not yet designed for it. Furthermore, in the words of Atlantic editor Alexis Madrigal, “No one knows how to create words and pictures that are meant to be consumed out there in the world” (“The World is Not Enough”). The minor shifts in hardware actually demand a rhetorical transformation in the way we think about texts, media, and archives of all sorts. That transformation has not happened yet. For starters, we might reassess the language with which media scholars have been framing the issue.

The Trope of Hybridity The expression is always the same. You find it in tech journalism when pundits list key innovation trends. You find it in communication scholarship when researchers diagnosis the essential impact of mobile media. You find it in TED Talks—​you really find it in TED talks. The digital world and the physical world have converged, they say. “Cyberspace” is a term of the past. In its place, new spatial phrases now anchor the discourse: augmented space, hybrid space, scripted space, digital ground, information landscapes, etc. These concepts stipulate together that the digital and the physical are no longer spatially divided. The boundaries have thoroughly blurred, leaving us to inhabit a brave new ontology. This makes things interesting, because now we have two distinct groups of thinkers, each of whom are writing about the same object of study, and they clash at every turn. One group, the critics of digital distraction, tell us that the Internet and the lifeworld are set apart in the manner of a zero-​sum game. To engage with one is to disengage from the other; they’re separate, competing domains. The other group—​let’s call them “hybridity theorists”—​they

8

8   • I n t r o d u ct i o n

profess the opposite. To them, mobile practices like texting, Foursquare, or live-​tweeting signify a grand reconciliation of the digital/​physical binary. For instance, extrapolating from his analysis of smartphones, Jason Farman predicts that “emerging generations [will] no longer prioritize between material space and digital space since these spaces will simultaneously inform our experience of implacement” (42). Distraction critics see this as an urgent problem; to hybridity theorists, it is an intriguing phenomenon. I find both positions viable but also insufficient. Both fail to grapple with paradigmatic conflicts playing out in the post-​desktop turn and, consequently, neither offers an affirmative framework for reimagining digital culture beyond desktop-​oriented conventions. Nevertheless, they are enlightening and suggestive, especially if read as bridges leading toward a third position. Consider how the trope of hybridity functions in a few agenda-​setting statements. Adrianna de Souza ​e Silva, a leading scholar of mobile communication, relies on the trope of digital-​physical hybridity as she defines her influential notion of hybrid space. De Souza e​ ​Silva writes, “Hybrid spaces arise when virtual communities, previously enacted in what was conceptualized as cyberspace, migrate to physical spaces because of the use of mobile technologies as interfaces” (261). That phrasing should give us pause, namely this bit: “When virtual communities . . . migrate to physical spaces.” Where exactly were virtual communities before this migration occurred? Did users access them outside of physical space? None that I know of. The digital has quite frankly never existed outside of the physical. So-​called digital worlds and virtual realities have always owed their every pixel to the inorganic matter that technologists have organized into the hardware, satellites, and other infrastructure supportive of networked computing. I doubt that de Souza e S​ ilva would disagree. She formulates the idea of hybrid space as a corrective to the concept of cyberspace, which was evidently a misconception. But in reacting against this misconception, such pronouncements of digital-​physical hybridity breed another set of misconceptions. They attempt to reconcile an ontological illusion, all while retaining the binary terms that constituted the illusion in the first place. Note, further, how the trope of hybridity undergirds Lev Manovich’s definition of augmented space:  “The previous icon of the computer era—​a VR user traveling in virtual space—​has been replaced by a new image: a person checking his or her email . . . while at the airport, on the street, in a car, or any other actually existing space” (“The Poetics of Augmented Space” 221). There again, haven’t we always checked our email while inhabiting an “actually existing space”? Manovich’s underlying point, it seems, is that users are accessing

9

Introduction  • 

9

networked multimedia in more places than we used to. That part is valid, but a computer lab full of desktops is just as physical, actual, and existing as an airport or a street corner. Couching the discussion in this ontological binary—​ digital/​physical—​conceals another commonality, which also undermines his apparent distinction. To check one’s email, regardless of the screen size, is to “check out” of local action in favor of tele-​action. Both users in his statement can be said, in an attentional sense, to be “traveling in virtual space”; nonetheless, both remain situated in physical space the whole time—​their bodies do not suddenly vanish. Furthermore, another often-​cited distinction in Manovich’s essay does not hold up under rhetorical scrutiny. Technologies of “augmented space,” he claims, “deliver dynamic data to, or extract data from, physical space” (221). The same can be said for desktops. The desktop extracts data from me as I type into the keyboard or speak into the Webcam, and it delivers dynamic data to me at whatever physical space I’m sitting. What is different about mobile and wearable devices is that they can extract data from a much wider vocabulary of gestures than desktops can—​well beyond pointing and clicking. Take the simple case of GPS navigation. The navigation app generates multimedia in direct response to a driver’s otherwise non-​discursive gesture: the images on screen adapt to the turn of the wheel, in real-​time, without any explicit commands. Moreover, RFID tags make it possible for practically any object to store and deliver digital data. Hence, more and more physical objects are starting to perform like computers. But alas, computers were physical objects all along. Two themes have been lurking here. First, as can be gleaned in these critics’ definitions, the trope of hybridity mistakes differences in degree for a difference in kind. From the dawn of desktops to the spread of smartphones, there has been a multiplication of points from which to access digital networks, though all of those access points have always been physical, occupying actual existing spaces. Second, and most importantly, the trope of hybridity numbs us to the fact that these differences in degrees—​more points of access and circulation, more modes of delivery and extractions—​actually make a profound difference. The trope of hybridity also clouds garden-​variety definitions of augmented reality (AR). Here, too, one encounters a difference in degree being mascaraed as a difference in kind. Because of course, reality has always been augmented, at least since the invention of writing. What is revolutionary about these technologies, however, is their impact on how reality gets augmented, or, more precisely, on who has the capacity to augment it. What’s significant is that AR enables you and me to position texts and media in their most

10

1 0   • I n t r o d u ct i o n

kairotic environments—​the most opportune spacetimes for audiences to encounter our work. Historically, this capability has been restricted to commercial advertising and municipal messaging; the state and the corporation have hereto monopolized the right to author public settings. That monopoly is vulnerable now. Where there have been only stop signs, billboards, and logos—​there may be libraries of content. That is to say, the streets may host on-​demand texts, images, and audiovisual media on subjects as diverse as the holdings of any conventional library. In feral daydreams, I sometimes imagine what these street-​libraries will look like when I walk to and from my university campus in downtown Denver . . . the city as an open-​access canvas for digital projects of all kinds. Like a stop sign at an intersection, such media may exude significance on the spot, affecting the events and activities that unfold in its proximity. Unlike a stop sign, the domain of its signification need not be limited to banal utilitarian instructions. Walking the streets, I indulge a series of hallucinations: I see an overlaid image that casts new associations into my perception of a building and the institution it houses. I hear a counternarrative that defamiliarizes the scene before me. I post remarks onto pop-​up digital forums arranged at precise locations throughout the city, each positioned to correspond with a specific local issue, so that whenever citizens debate that problem, their discourse accrues in exactly the same spot where the problem occurs. Vague and wishful as I recount them here, my daydreams are actually quite plagiaristic. They’re little more than residual afterimages of some exemplary works and promising prototypes that I write about in this book. The projects discussed in ­chapters 5 and 6, for instance, show how a few bold artists, scientists, curators, and humanists have adapted the kairotic, in situ capacities of post-​desktop technologies for aesthetic expression, transdisciplinary research, civic engagement, and cultural criticism. Some cities are already overlaid with arrays of digital gloss, yet the early output has been deemed “gimmicky” in many cases. And with good reason. Merely consider the bulk of content now available on AR platforms: Wikipedia entries, real estate listings, Yelp entries, etc. All that these AR mediascapes do is simply redistribute information from the Web on a geographical basis. Rather than searching the Web for restaurant reviews, mobile users can, as Figure I.1 depicts, point their phone at a restaurant’s storefront to access reviews. While this phenomenon bears some interest from the standpoint of information architecture, the experience yields little aesthetic or rhetorical value aside from an initial shock and awe moment. A glaring dearth of

1

Introduction  • 

11

Figure I.1 Screen capture from Yelp Monocle iPhone application, overlaying the Financial District in San Francisco. Image © Yelp, 2009.

worthwhile content at this point plagues AR and related emerging platforms. This is excusable, given the fact that such mediums are still in their infancy, but it is also disconcerting. We are racing to adopt new information spaces, new archives, without giving much thought to the (unique) forms of expression they might enable and constrain. The lauded technical feats of digital-​ physical convergence do not come preinstalled with literary, artistic, or rhetorical innovations. If handheld devices and wearable displays are becoming the most frequently accessed reading and writing technologies of our time, then it behooves us to bring our conceptual resources to bear—​to inform and

12

1 2   • I n t r o d u ct i o n

orient the technology’s cultural development. From an arts and humanities standpoint, I see post-​desktop interfaces opening up vital modes of public deliberation, different ways to curate and engage with cultural heritage, and more venues through which intellectual discourse may intersect with everyday life. It is a promising sphere of practice in need of rhetorical insight (as well as pointed critique), and opportunities abound for teacher-​scholars across the humanities to augment the built environment in meaningful ways. For instance, in partnership with staff at the Denver Museum of Nature and Science, my students and I augmented existing text panels in the Museum’s Wildlife Exhibit (which convey zoological information about the animal’s habitat) with additional layers of multimedia that speak to ways in which these animals have inhabited the human imagination in mythology, art, and literature. The project enables museum visitors to encounter cultural histories of particular animals, in addition to the biological and ecological facts that traditionally comprise nature and science exhibits.2 Figure I.2 documents one student’s augmentation of the Museum’s echidna display, which links visitors to audio clips of three myths involving the echidna and to the student’s written commentary.

Figure I.2  Augmented Wildlife Exhibit, Denver Museum of Nature and Science. Photograph by the author, 2015.

13

Introduction  • 

13

Though far from groundbreaking, the results of our collaboration left an impression on everyone involved: if a random group of English majors can bring the humanities into a nature and science museum, then what else might be possible? Institutions around the world are wondering how post-​desktop technologies might enhance or degrade the embodied experiences that define their missions. Galleries, campuses, and public squares stand among the most fertile test sites (and battlegrounds) at which to experiment with the purposeful, rhetorical interplay of networked multimedia and local action. These emerging interfaces, if widely cultivated qua literacy, may democratize the means to transcode institutional spaces and city streets in an image of one’s making. Myriad built environments stand ready to double as ad h ​ oc classrooms, museums, forums, labs, stages, and studios—​or, conversely, just an endless bazaar. In coupling observation, imagination, and experience, I think the only certainty about these street-​libraries to come is that they will be voluminous. They might not be valuable—​not if intellectuals, artists, and teachers continue to concede the opportunities of in situ discourse to advertisers, and not if we don’t change the way we think about reading and writing under these unfamiliar circumstances. Increasingly, incessantly, people create and consume multimedia while doing something else. This is a cause for concern in many respects; it is also an indication that the new wave of interfaces instantiates a remarkably distinct sort of archive. Facing such conditions, critical media producers must learn to design and compose under a relatively unprecedented assumption: audiences will encounter and interpret their work in situ amid live, spontaneous, local action—​as opposed to more traditionally codified archival/​ communication spaces, wherein texts and audiovisual objects are dutifully framed, exhibited, and apprehended in a place set apart from the contingencies of daily life happening beyond the theater, the library, the gallery, the town hall, the margins of a page, or the virtual windows of desktops and laptops. Without conceptual interventions, the programmatic misalignment among multimedia, people, and their surroundings will grow more disparate. Today’s digital distractions will seem quaint. The digital and the physical will continue to converge in a spatial sense, but they’ll be alone together in a rhetorical sense. This problem, coupled with pending technological developments, beckons new theoretical frameworks for understanding media beyond the desktop and its relics—​for being digital in the here and now.

14

1 4   • I n t r o d u ct i o n

Defining Actionable Media The communication practices, cultural forms, and rhetorical-​aesthetic principles that I will come to associate with the term “actionable media” all contribute to a distinct agenda. They seek to mitigate the separation of networked multimedia and local action, and to ground meaning making at the interstices. As a critical-​creative endeavor, the overarching priority of actionable media is to theorize, analyze, and produce digital artifacts that are designed to resonate with what we’re doing while we’re doing it. In other words, media practices that conspire with the here and now in their very composition, rather than memes primed to circulate anytime-​anywhere. The locale in which audiences encounter the work becomes an integral contributor to the work’s form and content. As we observed in the Preface, Central Park is in no way extraneous to Janet Cardiff ’s Her Long Black Hair. The Park is not just an interchangeable backdrop for a reading experience that could be bookmarked and easily resumed elsewhere. You cannot really share the audio walk with your Facebook friends or Twitter followers; it’s not designed for boundless circulation. Cardiff ’s work, like all actionable media, utterly depends upon the spatiotemporal interplay of the audiovisuals she has composed and the dynamic urban environments onto which she grafts her narrative. Eschewing the engrained bias for tele-​action, actionable media projects gravitate around singular settings and finite materialities. “Going viral” is of little interest. The definitional scope of actionable media may be cast over and against what Henry Jenkins et al. call “spreadable media.” Jenkins et al. locate the exigence for this latter term (and the practices it signifies) in the rise of social networks and the subsequent explosion of peer-​to-​peer content sharing. For them, “spreadability” refers to the potential—​both technical and cultural—​for audiences to share content for their own purposes, sometimes with the permission of rights holders, sometimes against their wishes” (3). Spreadable media means to galvanize a certain type of sharing. While Jenkins et al. critique the epidemic connotations of viral metaphors (namely, the lack of human agency such terms imply), a basic thrust of their book is to elevate the analysis and creation of Internet memes into critical consciousness. Memes spread as a result of audience/​user appraisal, they stipulate, and not merely by means of top-​down infection (21). Their case studies suggest that meme campaigns, be they commercial or grassroots, are becoming a culturally valuable if not unique genre for negotiating one’s identity (or brand) and building new discourse communities. In every case, the memes funnel attentional flows around social networking feeds, updates, and posts. Spreadable

15

Introduction  • 

15

content spreads ever-​toward the distant-​now accruing atop so many real-​time scrolls (e.g., Facebook News Feed), each of which revolve around production features that “allow [users] to spread content well beyond their immediate geographic proximity” (2, my emphasis). Geographic proximity no longer constrains word of mouth and, indeed, no longer poses any special discursive weight on viral memes. Spreadable media, in this sense, eagerly pushes media scholars and digital creatives to embrace anytime-​anywhere models of circulation, which tend to value global reach and mass scale over local context. That currency is valorized in the authors’ motto, repeated throughout their book: “if it doesn’t spread, it’s dead.” The antigravitational pull of much online media research (not exclusive to Jenkins) appears increasingly unconcerned with particular acts of reception and the nuanced interpretation of content at a local, individual level. It is also at odds with the situational character of rhetoric emphasized by Lloyd Bitzer, James Kinneavy, Carolyn Miller, and other theorists since the 1960s, not to mention Aristotle. As such, the measure of digital communication risks becoming a function of quantitative transmission and sheer volume. A message that garners deep, local resonance unmeasured by data analytics carries less weight than a message that’s been casually clicked on and soon forgotten by thousands of “digital eyeballs.” At what point, I wonder, does the budding fascination with viral circulation and spreadability lead digital studies astray from the contextual imperatives that have anchored so much hermeneutic activity across the humanities, the arts, and the rhetorical tradition? The logic of “if it doesn’t spread, it’s dead” seemingly begets a categorical denunciation of work that adamantly grounds itself to certain spatiotemporal confines, as well as a denunciation of emerging technologies designed to foster such localization. I propose actionable media not as a corrective or reformist measure, but as an alternative line of critical inquiry and cultural expression. The exigence driving actionable media is less the rise of mobile and social media, and more the frustrations that those practices have come to engender. Critics of digital distraction, acknowledged above, speak to such frustrations, and their arguments have not been lost on leading technologists. Some of the most visible software engineers/​entrepreneurs in industry and academia have enlisted their research teams in the effort to rethink smartphones and the mobile Web. At the 2013 TED Conference, Google co-​founder Sergey Brin prefaced his demo of Google Glass by airing a series of disappointments and doubts about mobile devices and the withdrawn postures they so often incite. He and his staff at Google “question whether [the smartphone] is the ultimate future of

16

1 6   • I n t r o d u ct i o n

how you want to connect to other people in your life, how you want to connect to information . . . should it be by walking around and looking down?” (“Why Google Glass”). MIT interface designer David Rose more articulately channels Brin’s sentiment when he claims that “iThings” unilaterally “assert a cold, blue aesthetic into our world, rather than responding to our own” (21). Keen on the prospects of wearable computing, Rose visited the Googleplex to try out the Glass headworn display. Rose was, to his surprise, quickly and utterly disappointed in the Glass prototype, largely because (fashion and comfort aside) the device did not establish any sense of reciprocity between the graphics it projected and his live perceptual field (30). The only substantial difference was the position and size of the display. Rose’s shock is understandable. His expertise in burgeoning agendas in computer science, particularly ubicomp and the Internet of things, equipped him to expect a radically different experience than the flimsy approximation Google had managed. In his 2014 book Enchanted Objects, Rose recalls his initial anticipation of how Glass would work and how he, following his underwhelming demo, believes wearable technologies should work in the future: As I came into the city, an augmented reality layer would identify buildings that had once stood there before being destroyed by earthquake or fire, and also a vision of how planners and architects envisioned the city as it might look in the future . . . the real promise of Google Glass is to be a genuine augmented reality experience . . . To accomplish it, Glass will need to recognize and understand precisely where your gaze is directed and superimpose relevant information on the object, landscape, or person you’re focusing on—​and make it visually stick. When you move your head, the information needs to stay connected to the object or landscape or person, rather than move across the screen in the same position. (31) This passage holds glimpses into a conceptual paradigm that far exceeds Google Glass and comparable gadgets alternately hyped and ridiculed by tech journalists. We can already glean some basic themes from his passing remarks, which supply helpful premises for filling out the definition of actionable media. It is these conceptual themes (and not the fate of some product line) that are of most consequence to the cultural development of interfaces within which much reading, writing, communication, and archivization may take place.

17

Introduction  • 

17

Namely, media becomes actionable in Rose’s scenario to the extent that it circulates in concert with embodied action amid the built environment. The presentation of digital content processually aligns with (and changes in response to) one’s gaze relative to her surroundings. Data links to objects, landscapes, and people—​the stuff of our extracomputational lifeworld—​as opposed to virtual simulations of those things. Like many other facets in the history of computing, the desire to make media actionable has its precedents in military experiments and idioms. Military technologists pioneered applications, such as fighter jet interfaces shown in popular air combat films, that broker real-​time links between digital data and live events. Among military circles, officers have a special term for information arranged and delivered in this manner. They call it “actionable intelligence.” Actionable intelligence is information designed to help people (soldiers in the military’s case) make decisions about the situation at hand, in order to better inform their present actions. Ultimately, in coining actionable media, I  mean to marshal scholarly attention toward texts or audiovisual works that incorporate local action on both a formal and hermeneutical level. In their most overtly political manifestations, actionable media projects bring narrative, images, and testimony to bear on a particular site with an activist intent to instigate critical behavior at that spot, perhaps pushing citizens to raise questions, boycott a retail outlet, or regard institutions with greater scrutiny. (An array of such examples will be examined in ­chapters 5 and 6.) Crucially, though, not all actionable media practices aim to trigger immediate actions. On the more aesthetic and reflective end of the spectrum, actionable media grafts onto localities and materialities as a means to artistic or intellectual ends. The hustle and bustle of a city park becomes a poetic vehicle that productively complicates and enlivens artistic expression. The de facto arrangement of a museum exhibit becomes living fodder for in situ historiography, cross-​cultural juxtapositions, or deconstruction. In any case, proximate events actually (if not instantly) prompt rhetorical changes in the work’s composition, resulting in the invention of new content or processual alterations to the presentation of existing content. The structure of actionable media immixes, overlays, transcodes, or syncs with dynamic fluctuations occurring in one’s environment. Be they political, artistic, or intellectual, actionable media projects collectively aspire to more firmly tie the gestures and postures of critical thinking to local scenes of action and decision. Comprehending the technological initiatives that undergird actionable media practices is only a departure point. Computer science builds the

18

1 8   • I n t r o d u ct i o n

essential technical affordances, but it alone does not ferry us from computerized traffic lights to digitized public art. On a purely functional level, the movement of (non)human bodies through a college campus might alter the arrangement of a multimedia installation embedded on its grounds akin to the way vehicular traffic does in the case of motion-​sensitive traffic lights. In an engineer’s eyes, an actionable media work that senses and responds to its locality may appear no more remarkable than traffic lights. Traffic lights’ colors have long changed on the basis of changes in proximate activity, and those changes in color have long delivered actionable information to all drivers and pedestrians inhabiting the scene. A prototypical actionable system in this sense, the traffic lights adorning our intersections engender patterns of social organization that would not otherwise occur in their absence. They are also, of course, a very banal example. Traffic lights are boring. But that’s part of the point; in fact, it’s the key point from an arts and humanities standpoint. Actionable media, with the rise of ubicomp, need no longer be constrained to the banal, the municipal, and the commercial—​as it has long been. Current and projected trends in digital culture afford unprecedented access to the means to produce actionable systems in public settings. Transportation experts created a simple three-​color apparatus in order to facilitate orderly automotive flows. What kinds of actionable systems will artists, designers, poets, activists, intellectuals, and students create? All of these groups can now augment the built environment in ways much more complex than traffic lights do, and for purposes and outcomes much more varied than coordinating road congestion. And yet, the humanistic production and analysis of texts or audiovisual works that are rhetorically optimized to circulate in the capacity of actionable media—​to resonate with local action on a formal and hermeneutical level—​is not simply a cut-​and-​paste endeavor. We cannot expect to make an actionable impact by merely linking or geotagging our journal articles to park benches and busy intersections. Over the past two millennia, the vast majority of intellectual discourse, akin to artistic works, has been housed and experienced within distinct spaces, wherein audiences temporarily withdrawal themselves from the variable spontaneity of mixed-​use urban environments in order to concentrate on the speaker, the stage, the canvas, the page, or the screen. The prospects of actionable media, therefore, invoke different modes of knowledge production, theory-​ building, and aesthetics. One must compose inquiries and craft messages in conjunction with the ebbs and flows of local action, and no longer in spite of them. As well, the audiences of actionable media do not seclude themselves in codified spaces for immersive contemplation; instead, the work itself

19

Introduction  • 

19

meets them in the streets, so to speak: actionable media projects (composed of texts, audio, or images) prompt close readings of the live events occurring in one’s proximity. Actionable media, as such, does not appear to compete with or supplant the critical work occurring in books, galleries, and websites. Rather, it casts light onto a different set of opportunities and genres being put into play by the dynamic, in situ archival modalities associated with post-​desktop interfaces. These opportunities introduce a fresh set of challenges to authorship and analysis. They invite media scholars and digital communicators to become acquainted with a unique set of historical precursors, emerging forms, design philosophies, and rhetorical principles—​which the subsequent chapters endeavor to assemble.

Archival Traditions, Rhetorical Figures, and Digital Futures While the narrative thread of Actionable Media unfolds around technocultural developments of the past three decades, this temporal scope balloons out frequently in pursuit of historical resonances with ancient orthography, classical rhetoric, print culture, and modernist aesthetics. Transdisciplinary in its orientation, Actionable Media mobilizes rhetoric’s figurative tradition3 to bridge research agendas from rhetorical theory, computer science, mobile communication, continental philosophy, media history, and digital art. Rhetorical figures, as Nancy Christiansen has argued, are perhaps the most historically robust “heuristic for both composition and hermeneutics” (32). Likewise, Richard Lanham stipulates that rhetorical figures are “patterns of speech or writing that provide patterns for thought” (xiii). Rhetoric’s figurative imagination is, in this sense, devoted to the analysis and invention of attention structures and attendant principles, crafted and executed across various forms of media, which support higher cognitive processes like critical reflection, public deliberation, and creativity. Actionable Media accentuates the promise that figures hold for humanistic studies of ubicomp, both as an interpretive framework for analyzing digital communication and as a generative heuristic for creating actionable media projects. Chapters 1 and 2 probe the intellectual history of ubicomp, performing rhetorical analyses of the manifestos that sparked its invention, as well as the conceptual frameworks that have accrued in critical interpretations of post-​desktop practices among communications scholars, media theorists, and design researchers. Both the invention and reception of ubicomp jointly call for further engagements with grammatology—​the history and theory of writing.

20

2 0   • I n t r o d u ct i o n

A grammatological account casts a long view of ubicomp by situating it among contrasting traditions of writing, archivization, and cultural memory. Chapters  3 and 4 draw on Bernard Stiegler’s theory of grammatization to argue that ubicomp technologies and actionable media practices propagate a long-​marginalized form of the archive—​one that we construct and consult in media res amid the live contingencies of everyday life. Libraries, galleries, theaters, and desktop computers, on the other hand, tend to house media in codified spaces that, by design, differ from what surrounds them and prompt audiences to defer their involvement with local action happening outside the walls. What we are witnessing, then, in today’s post-​desktop initiatives is actually a much broader grammatological shift from deferred archives toward what I call actionable archives. Understanding the formal character of this shift via rhetorical figures, as a move from metaphor to metonymy, is a pivotal step toward critically engaging with digital projects circulating across the current wave of actionable archives and the modes of communication they facilitate. In ­chapter 5, we return to the contemporary milieu, ready to bring rhetoric’s figurative imagination to bear on nascent media forms, in the face of a twenty-​five-​year-​old lacuna. When interface designers first challenged the desktop metaphor, their initial concern was hardware. Research and development teams at Xerox PARC prototyped new device categories in 1988—​“tabs, pads, and boards”—​that clearly inform current smartphones and tablets. On the level of software, however, the ubicomp agenda has lacked a comparable set of forms. Since the 1980s, software developers have structured user experiences via the desktop design framework widely called “WIMP” (Windows, Icons, Menus, Pointers). Today, as actionable archives deconstruct the desktop metaphor, a new schema of interface forms is emerging: ATLAS (Apps, Tags, Layers, Actuators, Sensors). These respective acronyms propound contrasting rhetorical conditions for the invention, arrangement, and analysis of multimedia. Throughout ­chapter 5, I survey actionable media projects across digital art, citizen science, and public history, all the while emphasizing how apps, layers, and the other ATLAS forms may serve as critical-​creative categories for intellectual discourse, aesthetic expression, and democratic deliberation. The objective is to theorize ATLAS as an original set of rhetorical figures fit to orient diverse inquiries into modes of scholarly, artistic, and political communication developing within ubicomp cultures. Building on these rhetorical figures, ­chapter  6 outlines some generative principles for media scholars, digital humanists, and research-​creation teams looking to create actionable media projects for ATLAS interfaces. These principles, illustrative of Roland Barthes’s theory of figuration, are extracted from

21

Introduction  • 

21

the creative practices of three artists: Claude Monet, Augusto Boal, and Janet Cardiff. Their respective oeuvres, which together comprise a hundred-​year span of (post)modernist innovations in visual art, theatrical performance, and sonic storytelling, variously fashion techniques for channeling the contingencies of local action into their invention processes and/​or their works’ reception contexts. Breaking from the deferred archives conventional to their disciplines, Monet, Boal, and Cardiff ’s methods blaze trails for critical media producers that parallel the rhetorical-​aesthetic quandaries associated with ATLAS-​enabled genres of cultural expression. Throughout the book, I examine actionable media practices apparent in ubicomp artworks and urban installations, as well as collaborative projects initiated by cultural institutions and transdisciplinary groups. These contemporary examples are contextualized through connections with rhetorical theory, cultural history, and aesthetics, including continental philosophies of action, histories of urban signage, and the creative practices of relevant avant-​ garde artists and writers. Monet’s en plein air painting methods shed light on stylistic considerations intrinsic to writing and designing for augmented reality. Boal’s conception of the citizen as “spect-​actor” and his systematic fusion of politics and theater suggest prospects for deliberative rhetoric pertinent to creating public interactive installations. Hannah Arendt’s remarks on vita activa become a lens for envisioning the impacts of actionable media on intellectual discourse. In the Epilogue, I propose that actionable media production invites scholars and students to undertake such work in reverence to a new ideal. Whereas newspapers, magazines, radio, blogs, podcasts, and social media have progressively conditioned the rise of public intellectuals, actionable media may conjure up another figure: the kairotic intellectual. Finally, I must confess that the conceit of my book—​to spur a rhetorical transformation of ubicomp culture à la the arts and humanities—​is taken, more or less, from the mandates of a computer scientist. His name is Mark Weiser, the man widely remembered as “the father of ubiquitous computing.” A vast majority of the literature on ubicomp pays cursory homage to Weiser’s vision for a post-​desktop future, which he outlined in a series of essays during the 1990s, while serving as Chief Technologist at Xerox PARC before his much too early death in 1999. Weiser’s influence in technical and theoretical circles makes his writing a rich transdisciplinary link connecting lab research with new media scholarship. In addition, his manifestos carry suggestive points of departure for recasting digital communication beyond the desktop. I purport to read Weiser’s texts as if he were an important philosopher of technology. At the very least, he was a technologist whose design

2

2 2   • I n t r o d u ct i o n

theories bear enormous intellectual and cultural weight, especially when one presses upon his frequent allusions to existential phenomenology and the history of writing. The fact that Weiser, a computer scientist by training, spent years devouring Martin Heidegger’s oeuvre is but one of many facets that make his technical work so intriguing to scholars across the humanities. Indeed, Weiser had a maxim he repeated time and again to his colleagues at Xerox PARC. “Start from the arts and humanities,” he exclaimed. In rooms packed with software engineers and technicians of all sorts, he would underscore the point in presentations: “This is the most important part of the talk” (“Building Invisible Interfaces”). It wasn’t lip service. The arts and humanities were the lifeblood of Weiser’s vision. Actionable Media picks up where he left off, in some respects, though several measured objections and critical departures ensue. The book begins by interrogating the conceptual underpinnings and rhetorical implications of Weiser’s manifestos; in them, the prerogative to make media actionable lies in wait.

Notes 1. The digital turn within rhetorical studies, especially widespread over the past decade, has indeed contributed several distinct lines of inquiry to the broader, interdisciplinary study of new media. Many rhetorical concepts have been with us for over 2,000 years, and this persistence may be attributed largely to their adaptability in the face of technocultural change. Digital rhetoricians variously appropriate classical terms, refashioning them as conceptual lenses for examining how new media authoring platforms afford, constrain, or otherwise shape acts of digital writing and online identity/​community formation. Such inquiries often invoke the canons, the appeals, or the rhetorical triangle. Jenny Rice (2005) as well as Jim Ridolfo and Dànielle Nicole DeVoss (2009), for instance, respectively deconstruct sender-​receiver models of communication to account for greater degrees of fluidity among authors and audiences emerging with hypercirculatory, remix-​oriented composition practices. The canon of delivery prominently serves as an orienting lens for recent books in the field such as Ben McCorkle’s 2012 book Rhetorical Delivery as Technological Discourse and Sean Morey’s 2016 book Rhetorical Delivery and Digital Technologies. For good examples of appeal-​based inquiries in digital rhetoric, see James Brown Jr. (2009) and Collin Brooke (2009), each of whom examine how ethos is performed and assessed on Wikipedia. Read Douglas Eyman’s Digital Rhetoric: Theory, Method, Practice (2015) for the most authoritative survey of the field to date. 2. Several other augmented reality projects at museums have been collaboratively developed by faculty, graduate students, and curatorial staff in recent years. For noteworthy examples, see the project website for “Art++” at Stanford University’s

23

Introduction  • 

23

Cantor Arts Center, as well as the online documentation of “Metaverses” at the Metropolitan Museum of Art. 3. My push to reframe the study of digital communication around rhetorical figures is not without precedent in other humanities fields. Indeed, outside rhetorical studies, a notable slew of historians, philosophers, and linguists have long been appealing to and appropriating specific figures of speech as essential terms for articulating new theories and analyzing various objects of study. Hayden White’s influential book Metahistory posited that modern historians routinely gather evidence and write narratives in one of four distinct figurative modes, comprising a spectrum from metaphor, metonymy, and synecdoche to irony. Proceeding White, Kenneth Burke and Roman Jakobson also appropriated these figures into linchpins for their respective theses on the nature of thought and language. These transdisciplinary conceptions of figures and tropes contrast profoundly to traditional connotations in popular discourse and even some landmark treatises on rhetoric. For instance, when the infamous sixteenth c​ entury French pedagogue Peter Ramus reduced rhetoric to an art of style and delivery (reallocating the remaining canons to other disciplines), he instituted a legacy of attributing primacy to “plain style” (Ong 273). Ramus and his disciples taught generations of students to regard figurative language as a sparingly applicable means to engage the most uninterested or unruly of audiences—​those who need a little sugar to make the medicine go down, as it were (Ong 254). By and large, contemporary rhetorical theorists (De Man; Lanham; Christiansen) renounce this ornamental view in favor of more expansive perspectives that accord a constitutive role to rhetorical figures in verbal expression, human cognition, and cultural development.

24

1

THE INVENTION OF UBIQUITOUS COMPUTING When computers are all around, so that we want to compute while doing something else . . . [then] we must radically rethink the goals, context, and technology of the computer. —​M a r k W e i s e r a n d j o h n s e e l y b r o w n , “The Coming Age of Calm Technology”

Ubiquitous computing (ubicomp) is not simply a set of technical problems for computer scientists to solve. Like personal computing before it, ubicomp entails the involvement and transformation of many disciplines, professions, and institutions as the technologies it introduces facilitate new practices that reshape information and communication around the world. Ubicomp is a technocultural paradigm—​the emerging third stage in the history of digital media. The term, as I will clarify below, denotes scenarios in which personal computing conventions bifurcate in the face of devices and networks that are mobile, locative, wearable, projected, embedded, and implanted. Scholarship in digital rhetoric and multimodal composition—​from Kathleen Blake Yancy’s trailblazing 2004 essay “Made Not Only in Words” to Collin Brooke’s crucial 2009 book Lingua Fracta and Dennis Baron’s A Better Pencil—​has directed much needed critical and pedagogical attention toward now-​quintessential online genres:  wikis, blogs, webpages, search engines, Cloud-​ based authoring software, media sharing sites, social media, etc. Without denying the foundational importance and continuing relevancy of such work, I  contend that the rise of ubicomp beckons scholars to consider another set of emerging genres, media ecologies, and rhetorical challenges. The questions formulated and taken up here, in this encounter with ubicomp’s conceptual beginnings, suggest points of departure for recasting digital communication beyond the desktop. For instance, how might the affordances and problems that characterize this wave of technical invention provoke parallel innovations in media theory, rhetorical analysis, digital humanities,

25

The Invention of Ubiquitous Computing  • 

25

and public intellectual discourse? What if humanists and rhetoricians were to variously grapple with and contribute to the mission driving leading technologists today? That is, how might we inform (and become informed by) the endeavor to impart ever-​greater degrees of resonance and synchronicity between global computational networks and local live actions? Humanities perspectives on media, communication, and culture provide critical lenses for envisaging ubicomp beyond the commercial motivations (e.g., enhancing workplace productivity) that surrounded early experiments at Xerox PARC, and beyond several interpretations recently popularized by media pundits and designers. The present chapter’s examination of texts and interfaces prototypical of ubicomp’s technological imagination—​as well as the principles, limitations, and enigmas that have accrued in the wake of its transdisciplinary reception—​will prefigure and propel inquiries into actionable mediascapes across subsequent chapters. Ubicomp technologies condition the possibilities of actionable media production. Post-​desktop interfaces can potentially raise our collective capacity to digitally author built environments—​to appropriate parks, plazas, and city streets as dynamic canvases for aesthetic expression, public argumentation, cultural criticism, and democratic deliberation. At the same time, current innovations portend some disconcerting implications with regards to privacy, access, and technocratic hegemony. The global proliferation of ubicomp invites us to make the most of its powerful rhetorical affordances, but it also demands that we identify budding social ills, which are already manifesting in problematic uses cases among early adopters. Actionable media, then, encompasses both a novel mode of digital communication emerging with ubicomp as well as a theoretical framework intended to bring critical, humanistic insights to bear on post-​desktop ideals, prototypes, and practices.

“A Radically New Kind of Computer” First, a caveat. The rise of ubicomp does not foretell the end of desktops, virtual windows, or mice and keyboards. The Web as we know it now will remain, evolve, and thrive. As ubicomp development continues, however, one will gain a clearer sense of how the forest exceeds the trees—​how the Internet and networked multimedia as a whole exceed the particular form of their most predominant configuration to date (i.e., the browser-​based Web). In order to access the Internet or create digital content, we can sit down at a computer in a library or a coffee shop and open up a Web browser, but we do not have to do it that way anymore. Increasingly, most people are opting not to.

26

2 6   • Act i o n a b l e   M e d i a

Consumer reports since 2012 reveal a dramatic decrease in worldwide PC sales, notably among the young, who gravitate toward mobile devices (Wohlsen). 2010 industry projections have proven correct: smartphones and tablets have become the primary devices for Internet access, surpassing desktops and laptops (Anderson and Wolff ). This is especially true in so-​called developing countries, many of which lack the infrastructure to support robust landline connections and abundant Wi-​Fi hotspots. To most citizens of high tech societies, smart mobile devices represent the apex of digital innovation. As a relatively new class of technology, they may seem like the least likely form of computing to resolve the digital divide. Recent mobile adoption trends in developing countries belie that assumption. A desktop made in 2015 is more of a toy for the rich, so to speak, than a brand new smartphone. “The mobile Internet has enabled some developing countries to leapfrog ahead, letting telecommunications companies and consumers skip over fixed-​line broadband” (Pfanner). The Pew Research Center reports parallel findings about Internet access in the United States:  “Those with relatively low income and educational attainment levels, younger adults, and nonwhites are especially likely to be ‘smartphone dependent’ (i.e., the phone is their primary or only means of Internet access)” (“US Smartphone Use in 2015”). Regardless of the mode of computing you or I prefer for research and writing, many young people, low-​income families, and citizens in developing countries rarely use desktops and laptops. A new model has already taken hold around the world, reaching a broader and more diverse population than desktops ever have. Indeed, Mark Weiser predicted that “ubiquitous computing, phase 1” would reach a critical mass by 2011. Pioneering ubicomp in the late 1980s, Weiser and his colleagues at Xerox PARC prototyped entirely new categories of mobile, wearable, and embedded devices designed to “draw computers out of their electronic shells,” in order to bring the virtuality of computation “into the physical world” (“The Computer for the 21st Century” 98). For the members of Weiser’s team, this initial phase of ubicomp had been well underway since the early ‘90s. They tested wearable devices that tracked and broadcasted their location to others within the building. They sent messages to each other via networked handhelds while roaming the halls. They enjoyed on-​the-​go access to shared documents, anytime-​anywhere, within the space of the PARC campus. At present, these and other “phase 1” capabilities supported by networked handhelds have spread to digital cultures at large, and for the most part, we remain stuck at this impasse. That is, while our devices hint at a new paradigm, the concepts and practices by which we engage with, think about, and create

27

The Invention of Ubiquitous Computing  • 

27

multimedia for those devices remain unattuned to emerging technocultural conditions. For instance, while the Web has become mobile, smartphone usage largely mimics interaction modalities native to the desktop. (This is why pedestrians sometimes bump into things while composing a text message or scrolling through Instagram.) Weiser anticipated the paradoxical character of our transitional moment, which is why he termed it “phase 1.” His objective for this phase of ubicomp research was quite clear. Weiser wanted ultimately to make computers integrate more meaningfully with everyday life, but he realized this ambition demanded a prerequisite step: Could I  design a radically new kind of computer that could more deeply participate in the world of people? As I began to glimpse what such an information appliance might look like, I saw that it would be so different from today’s computer [i.e., the desktop PC] that I could not begin to understand or build it. So I set out, instead, to build some things that my colleagues and I could put in use, things as different as we could imagine from today’s computers, yet using technology that could be made solid today. Using these things would then change us. From that new perspective, I would then again try to glimpse our new kind of computer and try again. (“The Technologist’s Responsibility and Social Change”) This passage, published in 1995, forecasts that the value of networked handhelds lies primarily in the insights researchers might gain from experimenting with them. In other words, they are to be approached as transient bridges enabling theorists and designers to depart from personal computing in search of ubicomp. For Weiser, “mobility is the nearer-​term focus, prior to true ubiquity” (“Building Invisible Interfaces” 21). Outside of Xerox PARC, however, the current iterations of these networked handhelds have come to play altogether different roles in popular culture and technology discourse. By revisiting Weiser’s writing, alongside recent work that challenges or builds on his initial vision, this chapter assembles a critical thread of ideas about ubicomp that remained largely dormant during the first decade of smartphones and tablets. These devices have often served as little more than convenient portals to familiar desktop services. Email, text messaging, social media, and Web browsing constituted the lion’s share of US mobile usage from 2008 to 2014, for example (Anderson). While on-​the-​go access to our digital networks is hardly lamentable, the widespread tendency to see and use post-​desktop interfaces in the desktop’s image is fraught with contradictions.

28

2 8   • Act i o n a b l e   M e d i a

Withdrawn postures synonymous with “texting while driving” or “texting while walking” epitomize an underlying irony evident daily in high tech societies. Mobile technologies that were initially developed to enrich our contextual awareness, to augment our environments with only the most pertinent bits of digital content, tend more often to keep us fixated on pixels. In moving out from the experimental labs of Xerox PARC to the next-​big-​thing factories of Apple and Samsung, handhelds and wearables seem to progress further afield from the ideals that animated their invention. Popular mobile interface designs have, in this sense, doubled down on the initial building blocks of Weiser’s vision, only to shortchange its more radical horizon. And yet, the ubicomp research agenda continues to thrive and evolve in progressive R&D labs where technologists remain committed to making computational systems that operate in tandem with local action. This contemporary research is, as much as Weiser’s formative work, a part of the story of ubicomp and its ongoing invention, which I delve into below. In addition to prototyping the smart mobile devices that so many of us carry at all times, Weiser’s writings and subsequent ubicomp initiatives also imply prescient tasks for media theorists, digital rhetoricians, and communication scholars that are crucial for conceptualizing digital culture in an increasingly post-​desktop world.

Mark Weiser’s Intellectual Ethic When you picture a computer scientist in your head, whatever you are picturing is bound to be a poor approximation of Weiser. He was chief technologist at the legendary Xerox PARC—​the place where the desktop was invented—​ and he loathed desktops. Xerox aside, Weiser was a computer science professor who read Heidegger, was a drummer in a rock band, and wanted above all else to reimagine the future of computing by revisiting the history of writing. Every time he imagines and describes a new kind of interface, Weiser makes an analogy to some form of writing. Moreover, in setting an agenda for ubicomp research, Weiser instructed his colleagues to “start from [the] arts and humanities” (“Building Invisible Interfaces” 10). His prose comprises words, not acronyms. His articles feature what must be the most poetic sentences in computer science discourse: “computing should be like a walk in the woods,” or, “our computers should be like our childhood:  an invisible foundation that is quickly forgotten but always with us, and effortlessly used throughout our lives.” Given the legacy of his publications, one could argue that he was, in effect, the lead architect of our recent iRevolution (iPhones, iPads, etc.), though he died in 1999, at the age of forty-​six.

29

The Invention of Ubiquitous Computing  • 

29

And yet, as careful readers of Weiser’s oeuvre will recall, the popular success of smartphones and tablets marks only the onset of the ubicomp paradigm. To Weiser’s mind, these mobile devices—​modeled off what Weiser termed “tabs” and “pads”—​represented the enabling conditions within which to invent more radical forms. “Embodied virtuality” was not Weiser’s only objective. The second of ubicomp’s twin pillars was to create “context-​aware” computers that could interface with the multitude of autonomous, noncomputational fluxes that Weiser grandly referred to as “the infinite richness of the universe” (“The Computer for the 21st Century” 94). Thus far, humanistic studies of mobile and locative media have done relatively little to account for this latter dimension of ubicomp, which is now taking shape in new media forms within and beyond the mobile Web. Reassembling the conceptual underpinnings of Weiser’s vision will prepare us to theorize recent ubicomp initiatives in a manner that acknowledges the full spectrum of rhetorical affordances and social implications suggested by the paradigm’s most transformative capacities, beyond the basic features of mobility and location-​based data. This broader characterization of ubicomp foregrounds theoretical principles and ethical imperatives that routinely elude critics’ cursory depictions. Thinking from the position of the forgotten passages in Weiser’s work seeds a framework for digital communicators and designers who aspire to produce multimedia projects optimized for ubicomp platforms. In addition, emerging media scholars might move beyond the trope of spatial hybridity—​beyond a general recognition of digital-​physical convergence—​and toward rhetorical considerations of specific forms that underwrite acts of communication, deliberation, invention, search, and memory in ubicomp-​saturated societies. Ultimately a deeper engagement with Weiser’s oeuvre will provide theorists and practitioners alike with a fruitful base to reckon with and build upon as we endeavor to make media actionable in the wake of post-​desktop interfaces. Nearly every book and article dealing with ubicomp begins with a brief sketch of Weiser’s vision for a post-​desktop future, which he articulated in a series of articles during the 1990s.1 Over twenty years later, leading researchers in computer science and interaction design still credit his writing for blazing vital paths of research in both fields. Widespread innovations in mobile devices, as well as many cutting-​edge experiments in wearable computing and smart environments, can all be traced rather directly to concepts or prototypes Weiser introduced (Want 32–​33). For these reasons, Weiser’s ubicomp manifestos merit a closer reading than has been offered in the arts and humanities. His essays belong in the tradition of canonical texts

30

3 0   • Act i o n a b l e   M e d i a

written by twentieth ​century technologists, such as those featured in Noah Wardrip-​Fruin and Nick Montfort’s massive New Media Reader, wherein Weiser receives no mention. Just as Vannevar Bush’s “As We May Think,” Ted Nelson’s Computer Lib/​Dream Machine, and Brenda Laurel’s Computers as Theater proved essential to the technocultural development of hypertext and Web design, Weiser’s writings constitute a suggestive basis for theorizing ubicomp at a paradigmatic level (i.e., transversal to feature-​level innovations in a given product area). Conceptual work that strives to think at the level of the paradigm carries a key benefit: one does not willfully subject the rhetorical life of her insights to the planned obsolescence intrinsic to capitalist economies of permanent innovation. A promising way to read Weiser is to patch together his design philosophy according to the logic and context of its invention. In doing so, the critical focus remains intently centered on the “blue sky” ideational core of ubicomp, rather than on building interpretations that appeal to Weiser’s ideas only through a filter of now-​popular applications and consumer products. The mission of the present chapter can also be summarized another way. I aim to articulate the intellectual ethic of ubicomp. In Nicholas Carr’s estimate, any technology that targets acts of communication, navigation, memory, empirical measurement, or some other cognitive process inevitably imparts an “intellectual ethic” that people consciously or unconsciously come to adapt, in varying degrees, over the course of prolonged engagement with that technology. Carr defines the term in the following passage: As stories of the map and the mechanical clock illustrate, intellectual technologies, when they come into popular use, often promote new ways of thinking or extend to the general population established ways of thinking that had been limited to a small, elite group. Every intellectual technology, to put it another way, embodies an intellectual ethic, a set of assumptions about how the human mind works or should work. (45, my emphasis) Popularized by Carr, this conception of technology and its cultural significance is by no means unique to his work. A  distinguished list of landmark studies2 attest to the constitutive roles played by various writing and media technologies in the shaping of mental habits and patterns of social organization throughout history. Still, much technology criticism presumes the technologists themselves to be unaware of these broader ontological ramifications. Carr claims, “The intellectual ethic of a technology is rarely

31

The Invention of Ubiquitous Computing  • 

31

recognized by its inventors” (45). Weiser is one of the exceptions. Weiser’s texts aim to invent ubicomp in both a technical and a theoretical sense. In fact, his remarks on ubicomp invoke cultural history more frequently than the technological present. His reputation as “the father of ubiquitous computing” perhaps owes more to the concepts he formulated in writing than to the prototypes he built in the lab. While hardware and software has advanced well beyond what the available means of computing were during the 1990s, today’s digital innovators still aspire to actualize the interaction modalities and smart environments capable of supporting the epistemology that Weiser described. More than anything else, he is the inventor of an intellectual ethic that might inform interface design and digital communication well into the twenty-​first century. “Ultimately,” Carr writes, “it’s the invention’s intellectual ethic that has the most profound effect on us” (45). In the case of Weiser’s ubicomp manifestos, readers encounter a set of assumptions about how the mind should work in concert with one’s surroundings, and how computational media, in turn, should better correspond with local actions occurring in the extracomputational lifeworld. What is at stake, here, is an entirely different way to triangulate rhetorical relationships among people, multimedia, and environments. Whereas personal computers, the desktop metaphor, and early iterations of the Web collectively amplified the global village (which privileges infinite connections between people and documents regardless of geography), ubicomp privileges transductive relations between humans, media, and environments—​such that all three affect and become affected by one another. While both of these paradigms clearly have a future in digital cultures around the world, each entails a respective intellectual ethic. And while desktops and headmounted displays (e.g., Google Glass, Magic Leap, Microsoft HoloLens) will cohabit the same media ecologies, the theoretical principles underlying their design and criticism, along with the communication practices animating their cultural vitality and political effectivity, will remain quite distinct, if not fundamentally disparate. Just imagine what would become of the world were we to engage headmounted media (VR goggles aside) through the same kind of immersive postures with which we browse the Web today. Popular parody videos (in which a barrage of error messages and pop-​up ads suddenly overwhelm the headmounted display) make it clear that simply extending the desktop metaphor to smartglasses or digitally augmented car windshields would be laughable at best and dangerous at worst. More to the point, though, we might ask: Do “breakthrough” mobile apps such as the record-​setting Pokémon Go model a promising actionable media ecology, or do they further distort public

32

3 2   • Act i o n a b l e   M e d i a

perceptions of the ubicomp research agenda? What sort of digital experiences should we create for people who interact with multimedia while walking around a busy city? Digital networks are becoming pervasive, whether we like it or not. You can choose to disconnect, but people in your vicinity may opt to stay connected, check social media, send messages, or “catch ’em all” (as the Pokémon Go slogan encourages). Ubiquity has already entrenched its way into personal habits and societal expectations, and it has incited as many calls for concern as for celebration. The question now is not whether ubiquity will occur. Instead, the question—​the opportunity for in(ter)vention—​concerns which set of premises, concepts, and norms will come to orient the production and consumption of multimedia amid anytime-​anywhere access scenarios. Given that so many mobile media practices transport a user’s attention elsewhere, can we imagine futures for handhelds and wearables in which local contingencies and present proximities serve to anchor, organize, and purposefully inflect our encounters with digital content? In turn, how might we conceive of and better create media projects that do their rhetorical, aesthetic, or political work in situ—​media that meets ubicomp users when and where they are? With these questions in mind, I posit that the tasks before us, to interpret Weiser’s intellectual ethic and to theorize digital communication beyond the desktop, are as much ethical endeavors as intellectual ones.

Inventing Ubicomp Contemporary theories of invention are ever more reluctant to locate creativity in the individual. Karen Burke LeFevre, one of the first rhetoricians to champion a collective conception of invention, insisted that, “Invention is better understood as a social act, in which an individual who is at the same time a social being interacts in a distinctive way with society and culture to create something” (1). LeFevre’s emphasis on invention’s circulatory nature has been amplified recently by theorists drawing upon new materialism, distributed cognition, actor-​network theory, and science and technology studies. For instance, Laurie Gries urges scholars of rhetoric and communication to become more attuned to how texts and images spread, mutate, and evolve “in the strands of time beyond the initial moment of production and delivery” (Still Life with Rhetoric 14). The work of invention extends much further than an author or inventor’s finite labor. Whenever ideas and artifacts circulate among discourse communities and diverse publics, they “must be studied as divergent, unfolding becomings in order to account for their unique,

3

The Invention of Ubiquitous Computing  • 

33

distributed rhetorical ontology” (19). Moreover, emergent perspectives on invention widely embrace posthumanism. Thomas Rickert corroborates Gries’s new materialist position as he attributes rhetorical agency to what he terms “ambience,” that is, “an ensemble of variables, forces, and elements that shape things in ways difficult to quantify or specify” (Rickert 7). Thus conceived, every act of invention is fundamentally social, circulatory, and posthuman. To study an inventor properly is to examine an expansive network of relationships, eventualities, bidirectional influences, and intersecting milieus. In light of these compelling approaches to invention, a preliminary stipulation is in order regarding Weiser’s role in the invention of ubicomp. While Weiser’s texts are clearly the primary subject of my analysis below, his vision “interacts in a distinctive way” with existential phenomenology, histories of alphabetic writing, and the social and material conditions driving human-​ computer interactions in America since the 1970s. Weiser, or the intellectual ethic apparent in his writing, signifies a unique constellation of suppositions, concepts, models, critiques, and imperatives that—​while they emerge from and speak to different historical moments—​are combined around the task of reorganizing the flow of digital information at the turn of the millennium. If an individual is only ever a conduit for collective invention over time, Weiser is to be highlighted as a catalytic node nonetheless. He remains an exemplary touchstone for the ubicomp paradigm in its most ambitious constellation. Nascent post-​desktop initiatives, whether they channel or challenge Weiser, continue to situate their objectives in relation to still unrealized goals extolled in Weiser’s writing. Tracing the conceptual formation, cultural context, and interdisciplinary reception of Weiser’s expansive texts entails many detours. His work binds the structure of the present chapter, but my inquiry willfully transgresses the boundaries of the individual, in keeping with contemporary rhetorical theories of invention. In order to approach Weiser’s writings as conduits for the ongoing invention, circulation, and appropriation of ubicomp, I have adopted a methodology developed by Gregory Ulmer called “heuretics.”3 Hence, I structure my analysis in the sequence of Ulmer’s “CATTt” acronym, a fivefold framework devised to—​among other things—​identify and probe key conceptual undercurrents operative in avant-​garde manifestos (Ulmer, Heuretics 8). Accounting for Weiser’s CATTt—​the generative resources that function as Contrasts and Analogies for his formation of ubicomp, the Theories he draws upon, the domain of application that his work Targets, and the “tale” through which ubicomp disseminates (emphases mine)—​can yield a more nuanced understanding of the ideational network that Weiser configured to imagine a new

34

3 4   • Act i o n a b l e   M e d i a

research agenda, a new way of envisioning the place and purpose of computers. This ideational network is of ultimate interest from a heuretic standpoint. Heuretic scholars analyze avant-​garde discourses with an eye to inhabit the ideational network themselves, often critically inflecting it, then extending a given stream of invention into new domains. As Ulmer makes clear, heuretics “does not stop with analysis or comparative scholarship but conducts such scholarship in preparation for the design of a rhetoric/​poetics leading to the production of new work” (4). Heuretics differs sharply from the hermeneutic reading strategies more typical among intellectual historians. Rather than (or in addition to) asking, “What might be the meaning of an existing work?” the heuretic scholar foregrounds another question: “Based on a given theory, how might another text be composed?” (5). In my case, based on the logic of invention evident in ubicomp discourses and technologies, how might a new theory of media inform the critical production and humanistic study of digital communication beyond the desktop? Furthermore, how might cultural critics, digital humanists, and other intellectuals adapt ubicomp into a set of rhetorical platforms for delivering public arguments, interactive narratives, and social commentary? Thus, in addition to serving as a lens for examining Weiser’s work, a heuretic CATTt-​style reading will also furnish building blocks in service of this book’s core objectives (pursued in later chapters):  to pinpoint emerging forms, articulate theoretical principles, as well as synthesize relevant archival, rhetorical, and aesthetic traditions that provide insight into the complexities of post-​desktop mediascapes. In examining the design philosophy and transdisciplinary resonance at play in Weiser’s texts and subsequent ubicomp research, I aim to strike a generative critical posture, unearthing points of departure for further theoretical development, in much the same way as Ulmer describes a heuretic study of Freud: To study Freud as an inventor, for example, is to observe the gradual dissemination of his ideas through a range of disciplines and institutions and into daily life . . . To point out that a given theory lacks a politics (as has been said of deconstruction) or an aesthetics (as was said of Marxism) is not to refute that theory but to call for an invention. (5) Such will be the manner with which I engage Weiser’s vision, its legacy and its lackings, before critically inflecting ubicomp discourse to account for rhetorical challenges and possibilities concerning the production and analysis of multimedia qua actionable media. Attending closely to the invention of

35

The Invention of Ubiquitous Computing  • 

35

ubicomp throws into sharp relief the need for adjacent innovations among those who study and create digital content. While ubicomp has profoundly altered research in human-​computer interaction (HCI), communication and media scholars are only beginning to conjure the magnitude of ubicomp’s implications for writing, public discourse, memory, archivization, and questions of access and power. Weiser’s ideational network, his CATTt, provides a fertile starting point.

Contrast: Personal Computers The most obvious element of Weiser’s CATTt, personal computers (PCs) stand out as a constant object of distain in his texts on ubicomp. Weiser’s frustrations with PCs, however, were not the kind that culminate in tempered expletives or a desperate banging on the machine. It was a matter of principle, not faulty performance. Weiser believed that PCs, firmly rooted in the desktop metaphor as they were, imposed fundamental limitations on HCI that no amount of system upgrades could mitigate. His frustrations with PCs culminated in a decisive, self-​serving schema for charting the broad contours of modern computing’s evolution, whereby he forecast the demise of desktops. This boisterous history of the present hypothesizes a three-​wave progression of succeeding modalities:  mainframe computers, personal computers, and ubiquitous computing. As the terminology implies, there is a shift from computers to computing, from rooms full of clunky machinery then to desktops and laptops and now to an omnipresent array of smart objects and smart environments. What motivated Weiser and his colleagues to push computing relentlessly in the direction of these latter manifestations? Why was he so quick to condemn desktop PCs? After all, they had only hit the mass market a few years before his first manifesto, “The Computer for the 21st Century,” headlined the September 1991 issue of Scientific American. Weiser’s arguments against PCs revolve around four major objections: (1) PCs require too much specialized, technical knowledge to operate; (2) PCs monopolize users’ attention and arrest their movement; (3) PCs are too isolated from one another; and (4) as a general result, PCs tend to alienate or at least temporally bar people from one another, their surroundings, and noncomputational activities. In raising the first point, Weiser maintains that the modalities of HCI typical to PCs broker a persistent disconnect between the proscribed gestures required to engage computation and the more traditional actions that people perform while at work, home, and other everyday environments. In spite of PCs’ early market success during the late

36

3 6   • Act i o n a b l e   M e d i a

1980s, Weiser claims, “the computer nonetheless remains largely in a world of its own . . . It is approachable only through complex jargon that has nothing to do with the tasks for which people use computers” (“The Computer for the 21st Century” 94). Aside from a technical elite, who had learned programming languages and/​or adapted their skills to the demands of the PC’s input mechanism (e.g., artists who learned to draw with the mouse and keyboard), users had to devote the bulk of their awareness to translating their actions into commands or movements that the PC could recognize. Of course, the Graphical User Interface (GUI) of early PCs like the Apple Macintosh transformed the HCI modalities of command-​line interfaces, such that computers could be operated by clicking on windows, icons, and menus displayed visually on a virtual desktop. And yet, in making basic computational operations easier to learn, the GUI still did not solve what Weiser took to be a larger problem: the underlying fact that PCs restrict one’s attention and movement to the virtual space of an isolated box/​screen and its limited set of arbitrary interaction modalities. Noting the spaces in which computation could take place via PCs, Weiser advances several descriptions of desktop users that collectively sharpen his central image of “people holed up in windowless offices before glowing computer screens” for whom “the outside world and all its inhabitants effectively cease to exist” (104). Crucially, Weiser adds that the development of portable PCs (“laptops machines, dynabooks, [and] netbook computers”) does little to help people integrate computing into the rhythm of their broader workflow and everyday lives. Making the computer mobile (à la smartphones) effectively multiplies the spaces in which PC interaction modalities take place; portable, wireless devices alone do nothing to alter character of the interaction modalities themselves. People equipped with mobile computers may be able to engage computation amidst otherwise noncomputational, pedestrian settings (e.g., parks, subways, sidewalks, etc.), but mobile devices that retain the desktop’s basic design principles harbor more or less the same degree of neglect toward a user’s immediate surroundings. No matter how tiny the screen is, navigating a GUI still requires users to disengage, if only momentarily, from local actions happening in their proximity. According to Weiser, “ ‘Ubiquitous computing’ . . . does not mean just computers that can be carried to the beach, jungle or airport. Even the most powerful network computer, with access to a worldwide information network, still focuses attention on a single box” (94). Today, amidst the proliferation of smart mobile technologies (many of which “still focus attention on a single box”), it is illuminating to remind ourselves of this critical aspect of Weiser’s vision.

37

The Invention of Ubiquitous Computing  • 

37

I carry this theme into ­chapter 2 when considering recent debates in mobile media scholarship, which make clear that the tensions animating these competing design philosophies (desktops vs. ubicomp) continue to play out in the varied affordances of smartphones, tablets, and the communication practices they facilitate or hinder. Moving beyond the desktop is, quite crucially, more than a matter of portable hardware. In their most salient point of contrast to the PC paradigm, Weiser and his colleagues sought to circumvent virtual reality. Weiser believed that virtual reality—​of which the desktop GUI is a basic model—​created maps that excluded territories (94). The folders and files of the desktop interface, as Steven Johnson has explained, created a visual metaphor that used the familiarity of a typical office environment as a vehicle to help people navigate a seemingly infinite and incoherent array of binary code and electrical currents (48). Rather than incorporate familiar settings as a strictly imaginary vehicle, Weiser envisioned the opposite scenario. He wanted to install computing power into everyday objects and existing environments, so that people could engage computation without disengaging from the things that surround them—​the things they already lived among. In doing so, the ubicomp paradigm would not turn these things into apparatuses for simulating the world, but would instead aim to weave computation into things, infusing them with task-​specific functionality, always with the intent to “invisibly enhanc[e]‌the world that already exists” (“The Computer for the 21st Century” 94). Instead of continuing to build “single boxes,” Weiser challenged engineers and designers to construct a network of “hundreds of computers per room,” wherein each smart object would communicate with other smart objects to perform certain operations in response to proximate movements and behaviors as they occurred throughout a given setting (98). Emphasizing this relational mode of operating, Weiser draws a further distinction between PCs and ubicomp when he suggests that, in the latter paradigm, computing power comes “not from any one of these devices—​ it emerges from the interaction of all of them” (100). In other words, one designs and evaluates the computing power of a smart object in proportion to the relations it sustains within smart environments. Smart environments sustain a real-​time, actionable feedback loop based on the sensing and actuating capacities afforded by smart objects relating to each other. For example, when a sensor embedded into a door identifies authorized personnel by tracking a microchip embedded in their wearable ID badge, the data processed by the sensor prompts a programmed action: unlocking and opening the door when authorized personnel approach. In such cases, computational media syncs

38

3 8   • Act i o n a b l e   M e d i a

with the activities of humans and other entities, and software runs in response to various local actions within a given proximity—​not (only) in response to explicit commands issued by users tending to their glowing screens. Weiser’s sentiment is now channeled, even embellished, as a matter of ingrained truth among leading interface designers who champion the nimble ambience of wearable devices and the Internet of things. Sergey Brin’s coy mockery of the desktop-​oriented “looking down at my phone” posture on stage at TED2013 was a performative allusion to “The Computer for the 21st Century,” indicating in a single gesture the prescience of Weiser’s criticisms. Likewise, famed designer and author Amber Case laments that “we live in an age of intrusive media;” her 2016 book titled Calm Technology (a term borrowed from Weiser and John Seely Brown) pushes fellow digerati to revisit the tenets of ubiquitous computing pioneered by Xerox PARC researchers during the 1990s: “They were so far ahead [of their time] that their work is in danger of being forgotten . . . precisely at the time we most need it” (viii). On the crest of computing’s third wave, we should nonetheless temper the value hierarchies that this discourse invokes. Interacting with a screen in your eyeglasses is not categorically better or more ethical than mousing around on a desktop. Moreover, much of the content and software comprising any given ubicomp experience was likely created by someone sitting at a keyboard. Ubicomp is neither a golden age nor a technological epoch unto itself. But it is a distinct, burgeoning design philosophy ripe with unique communicative affordances and emerging cultural practices, as well as potentially disconcerting societal impacts. The mounting antagonism between ubicomp and personal computing, in final assessment, might be best understood and valued in the manner of Wittgenstein’s ladder: essential for the new perspective its ascent yields, even if its basis may seem a bit unfounded and disposable afterwards.

Analogy: Writing and Literacy Weiser’s critique of personal computers assailed the conventions and assumptions that had, during the 1980s, successfully driven industrial production and consumer expectation. Because his vision for ubicomp called for such a stark departure, Weiser needed to relate his fantastical image of “hundreds of computers per room” to a more established tradition of cultural practice. If PCs demanded too much of people and alienated them from the surrounding environment, then why burden us with more computers? Ubicomp no doubt calls for a tenfold increase. When grasped as a new design paradigm, however,

39

The Invention of Ubiquitous Computing  • 

39

ubicomp is not just about living with more computers, but about inventing whole new forms of computing to live with. Searching for alternatives, in contrast to the desktop logic of PCs, Weiser repeatedly consults the history of writing. For Weiser, computing needs to become more like writing, which he classifies as “perhaps the first information technology” (“The Computer for the 21st Century” 94). We need digital displays of all shapes and sizes that we can easily embed into our everyday environments. In fact, each of the three categories Weiser formulates to guide the production of ubicomp hardware—​“tabs, pads, and boards”—​are inspired and defined by analogies involving forms of writing commonly found in modern offices: post-​it notes, pocket-​size notebooks, sheets of paper, dry-​erase boards (98–​99). Many of the paradigmatic differences between ubicomp and personal computing stem from this basic maneuver: to envision computing in the image of writing, and no longer in the image of furniture. In this sense, Weiser redirects the tradition of constructing interface metaphors drawn from the modern office. He looks beyond the scene of the desktop, toward the writing that circulates at its margins. More generally, Weiser evokes the long history of alphabetic writing in order to situate current iterations of computing in relation to relevant patterns evident in the progression from scribal, manuscript, and print cultures. After he depicts writing as the ideal medium of seamless human-​technology interaction, Weiser projects the future of computing parallel to writing’s evolution in Western civilizations. He notes that writing has evolved in immense, unforeseen ways from the clay tablets of ancient Sumer to the experimental novels of James Joyce (102). Weiser pins PCs toward the early end of this spectrum: “The state of the art [of PCs] is perhaps analogous to the period when scribes had to know as much about making ink or baking clay as they did about writing” (94). Such analogies become a rhetorical apparatus for persuading his readers to see the history (and future) of computing in successive stages, culminating in ubicomp—​which, by virtue of its position in the analogy, shines as the most democratic, sophisticated, convenient, and fruitful stage. In short, the ubicomp research agenda intends to develop computing in the same fashion that enlightened societies nurtured orthographic writing: from specialized usages among elite groups and professions toward mass adoption and fluid integration into everyday life. In the lived experiences of literate populations throughout twentieth century America, written text had long since achieved ubiquity, thanks in no small part to consumerism. Weiser explains, “Not only do books, magazines

40

4 0   • Act i o n a b l e   M e d i a

and newspapers convey written information, but so do street signs, billboards, shop signs and even graffiti. Candy wrappers are covered in writing” (94). What Weiser calls “the real power of literacy” is not some special attribute of books; to fully appreciate the inescapable influence of writing within contemporary life, one must note the more subtle written texts spread across our built environments (94). For instance, imagine trying to find your way through a large international airport without any writing—​no gate numbers, no arrival/​ departure boards, no written flight information, etc. Entertaining this thought experiment, applied to any complex site of social interaction (and even relatively simple venues), brings into focus what Weiser took to be writing’s most salient feature. In text-​laden societies, engaging with writing “does not require active attention, but the information to be transmitted is ready for use at a glance . . . you absorb [written] information without consciously performing the act of reading” (94). As one readily gathers from this passage, Weiser’s conception of writing remains anchored in sender-​receiver, signal-​transmission models of communication. This may raise little alarm among network engineers, but it is a bona fide red flag for humanities scholars who have come to embrace more dialogic, intertextual, and embodied theses on communication. In ­chapter 3, I explore the prospects of theorizing ubicomp from a different understanding of writing and media, namely the grammatological perspectives set forth by Jacques Derrida and critically inflected by Bernard Stiegler. Quite notably, the qualities of writing and literacy that Weiser highlights differ from the phenomenon of “deep reading” that many literary theorists and historians of literacy hold in such high regard. Since the rise of poststructuralism and reader-​response criticism, humanists have tended to approach reading as a productive act, whereby readers inevitably co-​create the text from a network of associations weaved from enculturated beliefs and past experiences. A signal’s meaning varies quite widely depending on each receiver’s perspective. The notion of signal transmission, shrouded in technical blinders, does not acknowledge hermeneutic complexities. More in vogue theories of (hyper)textual interpretation are indebted to Roland Barthes’s concept of “the writerly text,” which seems to be a latent quality of almost any text: “this text is a galaxy of signifiers . . . it has no beginning, it is reversible; we can gain access to it by several entrances, none of which can be authoritatively declared to be the main one” (S/​Z 5). Hence, the fact that Weiser valorizes texts that can be absorbed “at a glance”—​texts that essentially fit Barthes’s (pejorative) definition of “the readerly text”—​is likely to strike humanists as simplistic, naive, or utterly boring. No street sign or candy wrapper has ever found a place in the literary canon. Books are privileged, not because they deliver relevant

41

The Invention of Ubiquitous Computing  • 

41

information to us in moments of need, but because they support focused contemplation (Wolf and Barzillai). Books, like the libraries that traditionally house them, establish a separate, silent space—​set apart from environmental distractions. Individually bound books are materially and symbolically optimized for sustained, immersive engagement with long chains of thought, argument, narrative, etc. Since the dawn of silent reading, as Carr explains, book readers have resolutely “disengaged their attention from the outward flow of passing stimuli in order to engage more deeply with an inward flow of words, ideas, and emotions” (65). This breed of deep reading entails a virtual reality of sorts, and recognizing this dynamic helps elucidate Weiser’s peculiar decision to locate the “real power of literacy” beyond books. Books share much in common with personal computers, whereas ubicomp aspires, in one way at least, to be more like street signs and candy wrappers—​present at the scene of action and embedded in everyday objects. Predictably, critics often speak glibly about our relationship with the fragmented texts (instructions, menus, headlines, labels, signage, etc.) that we glance at during a routine day: “These forms of reading tend to be shallow and of brief duration” (Levy 104). And yet, as such remarks imply but fail to note, the apparent triviality of these mundane genres has more to do with political constraints than stylistic ones. The right to create textual and multimedia interventions into public and private spaces has long been intertwined with power relations. Censorship is the de facto law of the land when it comes to urban and suburban terrains, even wilderness areas. Unsolicited inscriptions by rogue “artists” at US National Parks are combated with criminal sentencing, social media shaming, and immediate restoration efforts (Paul). Most signage that occupies built environments is made and controlled by companies or government organizations with the intent to promote corporate or state interests. Posters are typically restricted to ephemeral bulletin-​board spaces. Graffiti remains illegal in many countries. Weiser’s vision of ubicomp implies a future in which some of these political constraints might be thoroughly disrupted and potentially reconfigured. With more objects turning into smart objects that spawn ad ​hoc digital networks, the city (or the suburb) can become an open canvas, a living game board, or a museum without walls. As such, the texts and multimedia that occupy scenes of everyday life—​mundane as they have been—​are set to balloon in scope and proportion. Akin to the democratization of digital publishing occasioned by the spread of blogging, social networks, and media sharing hubs, newer technologies such as low-​cost sensors (e.g., iBeacons) and augmented reality platforms have drastically lowered traditional barriers for creating public, in

42

4 2   • Act i o n a b l e   M e d i a

situ actionable media projects. While the sheer emergence of these authorship capabilities is a historic event, it should not be mistaken as the dawn of a revolutionary aesthetic utopia. If the history of literacy and the past two decades of the Web have taught us anything, it’s that entrenched media conglomerates often manage to maintain their dominance, technological divides renew with every touted innovation, and advertising will always find new ways to wedge itself between us and the content we’re actually looking for. Furthermore, while ubicomp is widening the production and dissemination of actionable media, genuine questions about access remain. What social consequences brew when digital writing gets inscribed everywhere but is legible only to people with the right app? Both the authorial affordances and the prospective problems suggested by ubicomp discourses make it incumbent for new media scholars to probe, complicate, and deepen the analogy with writing. Histories and theories of writing, which I take up in earnest across ­chapters 3 and 4, stand to help us better foreground the rhetorical stakes of contemporary ubicomp developments, particularly the implications they bear from the stances of cultural memory, public communication, and comparative media studies.

Theory: Readiness-​to-​Hand “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it” (Weiser, “The Computer for the 21st Century” 94). As we have seen, writing sets Weiser’s standard for “profound technologies.” Of course, written texts, in many forms of iteration, demand active attention. Regardless of their literacy levels, readers of Being and Time or Of Grammatology, for example, will not get very far if they do not enter into the text’s virtual reality and disregard their surroundings while reading. Clearly, Weiser’s appeals to writing emphasize one of its circumstantial capacities, not its essence. From this circumstantial capacity—​exhibited most widely in public texts embedded at particular scenes of decision (street signs, ads, product packaging, etc.)—​Weiser pontificates on the nature of technology, and more to his point, on the question of what makes a tool a “good tool” (“The World is Not a Desktop” 7). To Weiser’s mind, good tools and profound technologies effectively recede into the background of human activities all the while enhancing them. He believes: A good tool is an invisible tool. By invisible, I mean that the tool does not intrude on your consciousness; you focus on the tasks, not the

43

The Invention of Ubiquitous Computing  • 

43

tool. Eyeglasses are a good tool—​you look at the world, not the eyeglasses. The blind man tapping the cane feels the street, not the cane. Of course, the tools are not invisible in themselves, but as part of a context of use. (7) This theory of good tools prefaces Weiser’s call to imagine new interface metaphors for ubicomp that should be, above all, premised upon invisibility qua integration. As this passage indicates, his usage of “invisible” here is figurative—​“the tools are not invisible in themselves.” Computing will “disappear” into objects and settings in the same way that writing disappears into street signs, which is to say it will hide in plain sight. Invisibility is simply a provocative way to describe the integrative, actionable character of ubicomp. Like the blind man’s cane, ubicomp becomes operative through movement and contact; both afford their beholders a mode of perceiving and knowing that is utterly dependent on environmental input. The cane without the street is just an expensive stick. A mobile device without contextual awareness is a portable desktop. Good tools, in Weiser’s approximation, are not (only) extensions of the human mind or body. More profoundly, technologies at their best may serve as liaisons linking us to subtleties in the dynamic lifeworld we each inhabit from moment to moment. The myth of technological invisibility has been oversold by big tech firms; parades of new product advertisements willfully fudge the ways in which Weiser conflated invisibility with actionable integration. Weiser’s figurative play with the term “invisible” has also been taken literally by entrepreneurs chasing the dream of a completely natural or transparent interface that would be almost telepathic in its ease of use. Mark Zuckerberg revealed in 2015 his hope that Facebook will one day be able to read users’ minds, effectively transcribing and uploading new posts on the basis of our passing thoughts (Trotman). Humanistic critics indubitably shudder at ambitions of telepathic transparency. Amy Kimme Hea warns that a totally transparent medium would make “critique of technological practice nearly impossible or irrelevant,” and render “agency in relation to technology . . . unnecessary and undesirable,” etc. (201). Observing the history of computing, Lori Emerson rightly points out that the more “user-​friendly” interfaces ostensibly become, the more elements of computation get folded into a black box removed from user intervention (3). These ideals are synonymous with tech companies’ perennial mission to further optimize “frictionless” digital experiences for blasé consumers, a strategy which yields shorter product cycles and more frequent sales (Emerson 8–​11). While product advertisements often tout

4

4 4   • Act i o n a b l e   M e d i a

post-​desktop devices as being “interface-​free,” I contend that media scholars may counter this ideology by accounting for a nascent set of interface forms at play in ubicomp cultures (see ­chapter 5). After all, devices such as headmounted displays and smartwatches, even smart environments, are far from invisible. But they do beckon us to articulate a new conceptual schema, a new critical lens for examining configurations of networked multimedia that are difficult to piece together from a PC-​oriented perspective. In contrast to interface-​free marketing campaigns, designers, artists, and researchers heeding Weiser’s notion of good tools strive to facilitate context-​awareness and maximal user agency. Referencing the work of current HCI researchers such as Yvonne Rogers, Ulrik Ekman insists that ubicomp designers may even be abandoning the very idea of making technology invisible, which was more or less misleading all along: “It may thus well be that ubicomp culture is not primarily a question of leading calm lives, but rather (also) a question of emboldening the human side of context-​aware invention and play” (44). Rogers urges her fellow technologists to integrate computation into everyday life on a highly noticeable, more interactive basis than Weiser lets on in his articles: “Weiser’s idea that technologies be designed to be ‘so embedded, so fitting and so natural’ that we use them without thinking about them needs to be counterbalanced; we should also be designing them to be exciting, stimulating and even provocative—​causing us to reflect upon and think about our interactions with them” (412). While many critics recite valid concerns about surveillance and privacy, it is also imperative for theorists to imagine how smart environments might become self-​reflexive breeding grounds for emergent genres of cultural expression, civic engagement, and intellectual exchange. In any case, changes in interface design do not make technology altogether disappear; rather, the shift toward ubicomp means that we must learn to create and analyze multimedia via a different set of forms, in addition to those already conventional to desktop frameworks. Wishing to integrate computation with local action, Weiser objects to the common metaphors associated with desktop GUIs (e.g., virtual windows, intelligent agents, television) because each of them makes computers the gravitational center of attention (“The World is Not a Desktop” 7–​8). Indeed, the rise of ubicomp may warrant skepticism about the inherent privilege with which radio-​film-​television scholarship has claimed to study digital networks, as if those three media held an undisputed parentage over computers. Leading scholars of cinema and visual culture, including N.  D. Rodowick and Lev Manovich, have argued that contemporary computing is

45

The Invention of Ubiquitous Computing  • 

45

but the virtual life of film, to such an extent that Manovich proclaims “what was cinema is now the human-​computer interface” (Language of New Media 86). Manovich insists further that, on the heels of film’s popularity during the twentieth century, cinema has become the dominant cultural logic and interface metaphor of our time—​“a toolbox for all cultural communication, overtaking the printed word” (86). When read against Weiser’s ubicomp manifestos, however, this cinematic understanding of interfaces appears well suited to the virtual reality of PCs, but not necessarily conducive for grasping alternative paradigms of computing. Manovich’s recurrent suggestion that writing is being obsolesced by cinematic computing thus seems less applicable to the twenty-​first century at large and more wedded to the thirty-​year reign of the desktop metaphor. The theories of technology that dot ubicomp discourse have roots in continental philosophy. Five years prior to Weiser’s Scientific American article, Terry Winograd and Fernando Flores published Understanding Computers and Cognition, a widely read book that introduced computer scientists and Silicon Valley to the philosophy of Martin Heidegger. The book’s launch also happened to coincide, fortuitously and fittingly, with Weiser’s arrival at Xerox PARC. He had been grappling with Heidegger’s work since his college days in Sarasota, Florida. Weiser’s collected papers (now housed at Stanford University’s Silicon Valley Archives) include graded essays he wrote as a teenager about Heideggerian notions of “Dasein” and “Being-​with.” Though Weiser cites Heidegger only rarely in his published articles, the decisive influence of the philosopher’s “readiness-​to-​hand” distinction shows up in Weiser’s stated preferences about tools, technologies, and interface metaphors. Briefly put, in Heideggerian terms, readiness-​to-​hand signifies a mode of encountering entities as equipment; that is, Dasein takes up the entity (a pencil, a screwdriver, a gun, etc.) in order to achieve certain tasks. Ontologically prior to scientific reflection, readiness-​to-​hand names an instrumental orientation toward entities in the world. Heidegger elaborates on this concept through his famous example of using a hammer: The less we just stare at the hammer-​thing, and the more we seize hold of it and use it, the more primordial does our relationship to it become, and the more unveiledly is it encountered as that which it is—​as equipment. The hammering itself uncovers the specific ‘manipulability’ of the hammer. The kind of Being which equipment possesses—​in which it manifests itself in its own right—​we call ‘readiness-​to-​hand.’ (Being and Time 98)

46

4 6   • Act i o n a b l e   M e d i a

By contrast, “un-​readiness-​to-​hand” occurs whenever any sort of malfunction disrupts this “primordial” relationship (i.e., Dasein using tools to achieve a task). Weiser’s critique of the PC paradigm essentially depicts desktops and the HCI conventions associated with them as a set of un-​readiness-​to-​hand circumstances to which people have been forced to succumb whenever they wanted to compute. To Weiser’s Heideggerian mind, the normative functionality of PCs amounts to, at best, a socially-​accepted dysfunctional system: PCs are pieces of (un)equipment that, being veiled from contexts of local action, require us to “stare at” them in order to execute screen-​based tasks; we cannot simply “seize hold of it and use it,” like taking a hammer to a nail. Weiser, however, did not only want ubicomp platforms to be easier to use than PCs. The ubicomp paradigm in its most radical iterations crafts modes of HCI whereby people no longer “use” computers in a unilateral master/​ slave manner. Accordingly, Weiser’s “good tool” ideal belies typical tool use cases in which a tool at rest stays at rest. In addition to being readily available for intermittent use, ubicomp platforms perform their work in the midst of one’s other work or play. Today’s ubicomp-​inspired Fitbit wristbands process data and render visualizations while their wearers sleep. When computational power gets installed in everyday objects, settings, and worn accessories, software can run in step with the established rhythms of daily life, apparently dissolving—​but not completely disappearing—​into so-​called natural behaviors and environments. While a certain instrumentalism is undeniable in Weiser’s tone, his underlying desire is to make the instrument so useful that one no longer has to use it. His stance is one of hyperinstrumentalism that, in effect, founds (perhaps unwittingly) an approach to interface design that points beyond instrumentalism. Paul Dourish, a former colleague of Weiser’s at Xerox PARC, has written extensively on the philosophical underpinnings of ubicomp and commented on its resonance with Heidegger’s existential phenomenology. Though the connection is largely implicit in Dourish’s scholarship, we can view Heidegger and Weiser as parallel figures, each of whom advanced his field in similar directions in response to roughly equivalent problems. Breaking from the Cartesian dualism of his mentor, Edmund Husserl, Heidegger rejected the mind/​body split conventional to Western metaphysics. Rather than reflecting on mental phenomena within the brackets of the cogito, Heidegger believed that “thinking is derived from being” (Dourish, Where the Action Is 107). In order to think, first I must be, and being is always being-​in-​the-​world. More generally, phenomenology sought to (re)assert the primacy of our everyday “actions in the world,” with the insistence that these localized, embodied actions are prior

47

The Invention of Ubiquitous Computing  • 

47

to and inseparable from scientific reflection and its “abstract and idealized realm of dimensionless points and frictionless surfaces” (Dourish, “Seeking a Foundation” 6). Akin to Heidegger’s philosophy, Weiser’s work in computing rejected the virtual/​physical split (i.e., digital dualism) that still structures popular conceptions of digital media and its relation to “real life.” Weiser’s goal was to bring computers “out of their electronic shells” and “into the physical world,” so as to make computing function in accordance with a variety of situated, everyday activities (“The Computer for the 21st Century” 98). If phenomenology’s emphasis on being-​in-​the-​world effectively promoted a reconciliation of embodied action and abstract meaning, as Dourish suggests, then we might understand the ubicomp research agenda—​propelled by this theory—​in terms of a related drive to reconcile or sync computational operations with enculturated rhythms of human action. Just as existential phenomenology theorizes the meaning of being on the basis of being-​in-​the-​world, ubicomp deploys computing on the grounds of local activity. And so, as I will argue in ­chapters 5 and 6, media projects that circulate among ubicomp platforms stand to encounter, affect, and be affected by the actions of proximate actants in ways that are both new and rhetorically salient.

Target: The Flow of Digital Information How do technologists begin to make computers function more in tune with local action? What are the sites of intervention where this paradigm shift unfolds? Producing more computers that are smaller and capable of networking with one another is necessary but insufficient. Weiser and his colleague John Seely Brown insist, “When computers are all around, so that we want to compute while doing something else . . . we must radically rethink the goals, context, and technology of the computer” (“The Coming Age of Calm Technology” 3). More than a matter of hardware, ubicomp targets the general flow of information across workplaces, the home, and other everyday settings. A goal of ubicomp is to make these flows of information calm, a word Weiser privileges in his later writings. “Calmness,” Weiser and Brown proclaim, “is the fundamental challenge for all technological design of the next fifty years” (3). As evident in his critique of PCs, Weiser believes that desktops and other electronic boxes corral data such that accessing and manipulating information requires our utmost attention. Further, due to the alleged un-​readiness of this interaction modality, attending to a desktop’s virtual GUI via windows, icons, menus, and pointers (WIMP) means disengaging from one’s surroundings in the manner of a zero-​sum game.

48

4 8   • Act i o n a b l e   M e d i a

In pursuit of calmness, the ubicomp paradigm aims to restructure the flow of digital data circulating among networked devices. Prior to the invention of ubicomp, the development of the GUI and its WIMPs in the 1970s restructured the command line interface into a set of visual cues and symbols. In contrast to the command line, the desktop GUI “relies on recognition but not recall” (Quigley 250). The PC paradigm was revolutionary in that it made computers user-​friendly; today, the revolutionary value of the ubicomp paradigm lays in its capacity to sync computing and media with a multitude of dynamic objects and live actions in various locales. With most PCs, the keyboard and mouse are my only representatives—​the computer recognizes me only in so far as I type, point, and click. Plus, in order to do these three things, I  remain stationary at my computer, and often near a power outlet or a Wi-​Fi connection. Ubicomp technologies recognize a much broader range of movements by, first and foremost, introducing new input techniques. A more expansive HCI scheme allows for a wider economy of contribution, which conditions the possibility of interaction modalities through which flows of information become generated, presented, and organized in concert with extracomputational actions happening offscreen. At present, two related strands of innovation strive toward interfaces that render digital data tangible and ambient. In the case of tangible interfaces (sometimes referred to as “physical computing”), one no longer clicks on keyboard and mouse buttons to manipulate virtual elements on the screen. Instead, the presentation of graphics and the user’s capacity to manipulate them are built into commonplace artifacts turned smart objects (Rieder). What happened with phones is continually happening with other items. The Moleskine “Smart Writing Set,” released in 2016, establishes a real-​time, wireless, indexical connection between a camera laden pen, a smart paper notebook, and a Moleskine companion app. As the writer puts pen to paper, every mark is replicated in pixels on the app. The drawn image and its digital double match exactly. Computer scientist Aaron Quigley describes the novelty of tangible HCI modalities as follows: A user manipulates a physical artifact with physical gestures, this is sensed by the system, acted upon, and feedback is given . . . The artifacts provide sites for both input, using well-​understood affordances, and can act as output from the system. Unlike the GUI, a classical TUI [i.e., tangible user interface] device makes no distinction between the input devices and the output devices.” (253)

49

The Invention of Ubiquitous Computing  • 

49

Tangible interfaces transcribe bodily gestures into software moves and acts of media production, and they register a wider vocabulary of gestures than those registered by PC input mechanisms. We have moved from typing remembered codes into a command line, to double-​clicking recognizable visual cues, and now to manipulating (and receiving real-​time haptic feedback from) smart objects via a range of spontaneous or, at the very least, non-​proscribed bodily gestures. Much prototyping in the arena of ambient interfaces continues to explore interaction modalities that circumnavigate the PC’s reliance on intentional human input. Ambient interfaces are designed, not unlike kinetic sculptures, to register and respond to a diverse mix of dynamic variables in a particular setting. An early case in point is Urban Botanical Garden (2006), a bus stop prototype created by the MIT Design Lab, which “detect[s]‌motion and the ambient sound to generate and position the [digital] plants and make them reflect the changes happening around them” (Telhan). Scores of installation artists have since used this technology to create multimedia displays (inside and outside of museums) that respond directly to viewers’ motions as well as pedestrians passing by. On the infrastructural basis of low-​cost sensors, which can be embedded in built structures or attached to bodies and objects, ambient interface design radically redistributes the boundaries of digital interaction. As such, the flow of information among ubicomp platforms may correspond to whatever live actions emerge from a proximate economy of actants, many of which would remain structurally excluded from the single-​user feedback loops at the core of PCs. Ultimately, in syncing computing with everyday actions and environments, post-​desktop systems aim to deliver small batches of relevant functionality and timely content, and to do so in ways that do not require users to attend—​ constantly and conspicuously—​to those information streams. Computing should “increas[e]‌our knowledge and . . . our ability to act without increasing information overload” (Weiser and Brown 4, my emphasis). Ubicomp seeks to fragment and decentralize all the computation that a typical desktop houses, making computational media more pervasive and localized, but also more marginal. In contrast to PCs, ubicomp interaction often occurs alongside or at the periphery of otherwise nondiscursive action (Weiser and Brown 3). We already see all sorts of scenarios, absolutely impractical if not impossible with PCs, in which people engage with computing while they remain engaged in another activity. A runner wearing an Apple watch, for example, generates data as she runs; each stride produces online records of her vital signs, calenders of calories burned, and line graphs of steps taken. She runs,

50

5 0   • Act i o n a b l e   M e d i a

and her running is computing. The Uber driver, too, is at once a motorist traversing the streets and a clickable icon poking along on the apps of local Uber customers. The live digital cartography on screen exudes a calm appearance while the app’s algorithms toil away, churning fresh data in a circling pattern, so as to be of service when the decisive moment comes. Embodied actions like running and driving, occurring in traditionally noncomputational environments, feed steadily into ubicomp systems and shape the flow of digital information every passing second. Meanwhile, the flow of information and the available means of interaction bifurcate into their most elemental configurations, becoming more and more discrete, until they are concise enough to reside in the periphery of one’s perception. Standing at the ready, but not in the way. The desired effect in theory is to heighten contextual awareness: to deliver information or commentary about the scene that is not apparent otherwise, to stimulate close readings of subtleties in our surroundings that we tend to neglect, or to enable forms of in situ content creation that thrive on documenting things on the spot. The pursuit of calmness and the attendant push toward what Clay Shirky calls “situated software” can, however, become a double-​edged sword in practice. Adopting Shirky’s term, urbanist Anthony Townsend cites Uber as a commendable example of situated software that has successfully “connected the Web to the physical world” by meeting the needs of “people in close proximity” as opposed to the global village (233). The Uber app (and its ilk) deals in digital cartography; drivers and prospective passengers receive a carefully curated, live streaming map containing geospatial updates and user ratings meant to inform their present transportation decisions. The map is highly accurate and highly selective, though the source of its appeal for users unwittingly causes frustrations for nonusers. For instance, Uber’s app does not show you the traffic hazards that your driver (through no fault, really) must incite and endure while you finish up your drink at that hip restaurant on a busy two-​lane street with no parking. There’s no icons on screen representing the cars backed up helplessly behind your idle Uber. There’s no user profile for the mother who must skillfully swerve around your Uber and into the oncoming lane. On the Uber costumer’s app, the process is calm and seamless, but it can make for disarray on the road. Connecting “the Web to the physical world” is an inherently messy business. In bringing digital communication to bear on the here and now of our surroundings—​in making media actionable—​we must strive to be cognizant of the broader, offscreen consequences triggered by so-​called digital-​physical convergence. Convergence is just a euphemism for colonization in cases

51

The Invention of Ubiquitous Computing  • 

51

where tech conglomerates impose their maps onto neighborhoods irrespective of established norms. When an app or an artwork or a game calls for actions amid public spaces, each of those actions occurs in a dynamic setting and stands to affect and be affected by a chain of (un)related activity occurring there. The seat of an actionable media project’s rhetorical power, its political effectivity to prompt collective action, can thus also be a harbinger of moral dilemmas. In ubicomp-​saturated societies, for better and for worse, flows of digital information are wedded to overlapping spheres of local action. It is critically and ethically imperative, then, that the analysis and production of actionable media should not take place in a sender-​receiver style rhetorical vacuum that privileges intended users, readers, or viewers to the detriment of the social ecologies they inhabit.

Tale: The “Sal” Anecdote The final slot of Ulmer’s CATTt framework, the “tale,” considers the concrete form(s) through which an avant-​garde discourse represents and primes itself for “circulation and dissemination [as] a cultural invention” (Heuretics 4). In the case of ubicomp, Weiser’s manifestos point beyond the scene of his writing as he alludes to technologies in the making at Xerox PARC, which were being designed to facilitate the use cases his texts portray. Mentioned above, the “tabs, pads, and boards” that Weiser outlines merit further attention here for the part they have played in ubicomp’s dissemination. We should note, too, the distinction Weiser makes between his near-​term expectations and his far-​future ambitions. In order to apprehend the scope of the latter, we must also attend to Weiser’s “Sal” anecdote and its resonance with contemporary ubicomp developments. Four years after publishing “The Computer for the 21st Century”—​the 1991 article that introduced tabs, pads, and boards—​Weiser referred to these three classes of devices collectively as “ubiquitous computing, phase 1” (“The Technologist’s Responsibilities and Social Change”). Tabs, pads, and boards are the computers one needs to build and tinker with in order to then rethink computing for the twenty-​first century. As such, “phase 1” creates new ports that effectively scatter established PC information flows across a multitude of inch-​scale (tabs), foot-​scale (pads), and yard-​scale screens (boards) in the hopes of making computation more amenable to real-​time tracking, rapid prototyping, and spontaneous collaboration. Tabs, pads, and boards—​as Weiser pitches them—​create the conditions for ubicomp experimentation. He does not present them in the manner of a product demonstration, as is

52

5 2   • Act i o n a b l e   M e d i a

most often the tale through which lab-​based technical inventions are put on display for general audiences. Of course, a variety of commercial products—​ from first-​generation PDAs to iPhones and iPads—​have since proven the value and appeal of these three hardware genres as mobile personal computers. As I will argue in ­chapter 2, such devices now play host to a tug of war in which competing design philosophies attempt to push and pull users toward disparate sets of media practices. Today’s tabs, pads, and boards support digital practices that advance ubicomp ideals as well as, far more commonly, use cases whereby mobile hardware serves as a smaller, lighter platform for familiar desktop activities. Perhaps the closest Weiser ventured to giving a product demo was the “Sal” anecdote he sketched at the conclusion of “The Computer for the 21st Century.” This narrative depicts a day in the life of a representative working professional (named Sal), who lives during some unspecified period in the future, when computation has become thoroughly integrated into the social fabric of everyday life. Hence, the Sal anecdote imagines some ways in which mobile interfaces might operate in step with Weiser’s ubicomp vision, rather than serving as miniaturized desktops. From the moment Sal wakes up, she glides through easy encounters with a series of computers, each built discreetly into her home and work environment, which offer up information and initiate actions in response to her movements, speech, and gestures, all the while accommodating an array of custom settings reflective of her preferences. Her coffee maker syncs with her alarm clock; some of the house’s windows act as interfaces, displaying spatially accurate “electronic trails” of any recent pedestrian and vehicular traffic that occurred near her property (Weiser, “The Computer for the 21st Century” 102). Whenever she highlights passages from a print newspaper with her smart pen, the highlighted text is sent to her office computer. During the drive to work, we learn that Sal’s car windshield shows her real-​time navigational information, even recommending an optimal parking spot. It seems every pane of glass anywhere is capable of displaying graphics. As soon as Sal enters the office building (wearing her microchipped ID badge), all of the electronics at her workspace boot up while she chats with a few colleagues. At work Sal uses tabs and pads to screencast her digital workflow, share her location with collaborators, and edit documents projected onto office walls. With so many devices active, whole new kinds of data get recorded without “user input” and become automatically stored in search-​friendly formats, just in case Sal ever needs to access a fractional bit of this information in the future.

53

The Invention of Ubiquitous Computing  • 

53

At the same time, however, it is painfully obvious that every single one of Sal’s interactions with ubicomp platforms reinforces her role as a corporate employee within a postindustrial capitalist society. The alarm clock and the coffee maker envelope her first waking moment, priming her for the workday without a second wasted. At the breakfast table, the texts she highlights from the newspaper streams directly to the computer at her office, along with her stream of consciousness. In every case, the digital traces she makes and stores—​as well as the traces presented to her at home, in the car, and the office—​all relegate her attention to information, objects, and actions that are conducive to “getting things done” with maximum efficiency and with an eye toward securing private property (e.g., the “electronic trails” tracking recent activity outside her house). Suffice to say, Sal’s morning is quite different from the mornings Thoreau celebrates in Walden: The morning, which is the most memorable season of the day, is the awakening hour. Then there is least somnolence in us; and for an hour, at least, some part of us awakes which slumbers all the rest of the day and night. Little is to be expected of that day, if it can be called a day, to which we are not awakened by our Genius, but by the mechanical nudgings of some servitor. (115–16) In Weiser’s Sal anecdote, ubicomp manifests as little else than a vehicle for such “mechanical nudgings.” Whereas his design philosophy is radical in a technical-​theoretical sense, Weiser’s gestures toward application here certainly leave much to be desired and even departed from. For all the autonomy and agency that mobiles and wearables afford to knowledge workers like Sal, the social shaping of these technologies in contemporary workplaces suggests that being “always on” entails added pressures and constraints. The spatial freedom brokered by anytime-​anywhere Internet access concurrently renders one’s time more vulnerable to interruption (if not surveillance). Referencing empirical studies of white-​collar professionals, Judy Wajcman claims, “While individuals derive substantial benefits from their mobiles . . . the shared practice of continual connectivity fosters expectations of accessibility, escalating engagement with work at all hours of the day and night” (103). Indeed, this dynamic can extend beyond office life and into municipalities at large. Even more disconcerting is the potential for smartphone-​enabled power relations to fuel what Evgeny Morozov calls “algorithmic regulation.” Because “so much of our everyday behavior is already captured, analyzed, and nudged,” Morozov forewarns against data-​driven

54

5 4   • Act i o n a b l e   M e d i a

hegemonies that may grow hand in glove with so-​called smart city projects (e.g., Rio’s/​IMB’s massive Operations Center), which aim to extend state/​ corporate monitoring toward a point of omniscience (“The Rise of Data”). While it remains imperative to argue against emerging technocratic agendas, the problems endemic to those practices should not blind us to the democratic promise that post-​desktop interfaces show in community-​driven advocacy campaigns. For example, Georgia Tech professors Christopher Le Dantec and Kari Watkins partnered with local citizens and city officials to launch CycleAtlanta, an app that equips cyclists with the capacity to trace their routes, note problematic infrastructure, and share their data with the city’s transportation planners. Each user’s ride data co-​creates an actionable map of electronic trails that planners consult “to make future decisions about where infrastructure is needed to create bike-​friendly routes throughout Atlanta” (“Georgia Tech Cycling App”). Cyclists using the app essentially vote with their pedals. While more popular cycling apps like Strava or MapMyRide measure riding vis-​à-​vis recreational metrics, CycleAtlanta frames cycling as a political act in a grassroots, rhetorical attempt to influence public policy and alter infrastructure to better accommodate the environmental, public health, and affordable mobility benefits that bikes deliver. A notable foil to the electronic trails in Weiser’s Sal anecdote, CycleAtlanta foreshadows how actionable media producers can appropriate ubicomp tracking devices into a platform for aiding political arguments. Digital traces of local action can be authored to inform community advocacy, impact civic deliberation, and shape future policies on a range of social issues. What remains important about the Sal narrative—​setting aside its evident corporatism—​is that it charts the crucial transition from smart mobile devices to smart environments, whereby the embodied interactions among devices and bodies in motion become the crux of computation and ad hoc networking. Obviously, this day-​in-​the-​life overview provides only the slightest, offhand glimpse of potential ubicomp scenarios. The examples it features reflect Weiser’s everyday life as an employee at Xerox PARC, where research revolved around issues of workplace productivity.4 (One might make similar claims of Vannevar Bush’s Memex use scenarios in “As We May Think,” which are imagined purely from the standpoint of a research scientist.) Still, the vision sketched out in Weiser’s manifestos has become a vital source of inspiration for leading computer scientists and interface designers. The global proliferation of smart mobile devices, recent advancements in urban informatics, and other ubicomp prototypes currently in development all stand as strong

5

The Invention of Ubiquitous Computing  • 

55

indicators that “many aspects of their vision [i.e., Weiser’s team at PARC] have been realized, at least from a technological perspective” (Dourish and Bell 3). Observing trends in popular computing and lab research, Paul Dourish and Genevieve Bell reflect insightfully on the rhetorical life of ubicomp discourse and its ongoing circulation—​from undercurrent to new wave—​across a number of intellectual milieus: [Ubicomp] has come to be broadly recognized in academic, commercial, and government settings worldwide as one of the key agendas for information technology research. And it has held sway, in a range of sites and guises, for more than twenty years. Influencing more than two generations of scholars, it has become a foundational story, a technomyth, in computer science and allied fields and as a result has shaped the kinds of technologies that have been made and also made possible. (3) But more work remains to be done, particularly in fields beyond computer science where scholars have just begun to think seriously about the philosophical, communicative, and social impacts of ubicomp. In what ways, then, might ubicomp become meaningful for considerations other than workplace productivity, corporate advantage, and utilitarian chores? Clearly, if ubicomp technologies are to become a vibrant cultural platform for actionable media, rhetoricians and media scholars have much work ahead of them. While post-​ desktop devices have sprung swiftly from research labs to consumer markets, we are only at the formative onset of ubicomp cultures. Those of us tasked with critically evaluating the affordances and effects of these new communication technologies have pivotal roles to play.

Notes 1. The weight of Weiser’s posthumous influence is also immense at professional conferences; HCI researchers Genevieve Bell and Paul Dourish found that “almost one quarter of all papers published in the Ubicomp conference between 2001 and 2005 cite Weiser’s foundational articles, a remarkable number of publications to cite a single vision as fundamental for their own work over a decade later” (“Yesterday’s Tomorrows” 133). 2. Notable scholarly works in this tradition, to name only a few, include Walter Ong’s 1958 examination of the spatial-​visual epistemology that emerged hand in glove with

56

5 6   • Act i o n a b l e   M e d i a

early print presses (qua Peter Ramus’s diagrammatic pedagogy), Jack Goody’s 1977 thesis contending that the literate revolution in Ancient Greece was prefigured by and intimately related to the categorical thinking techniques made apparent through the pre-​alphabetic invention of “figures of the written word” (e.g., the list, the formula, etc.), and Elizabeth Eisenstein’s 1980 analysis calling attention to the standardization of cultural memory (and the amplification of national identity) that accrued as manuscript culture gave way to print. 3. Though Gregory Ulmer’s heuretic methodology predates rhetoric’s recent turn toward circulatory, new materialist, and posthuman theories of invention, Heuretics:  The Logic of Invention is heavily cited by newer books (e.g., Hawk’s Counter-​History of Composition, Rice’s Digital Detroit, Rickert’s Ambient Rhetoric, Gries’ Still Life with Rhetoric) that each champion heuretics as an especially innovative framework for invention today. 4. HCI scholar Shaowen Bardzell argues that “every design projects its own ‘ideal user,’ ” which makes for outcomes where certain subject positions are catered to more than others (1307). That Weiser’s “The Computer for the 21st Century” privileges corporate, upper-​middle-​class professionals like “Sal” is clear. Drawing on feminist standpoint theory, Paul Dourish and Scott Mainwaring call for ubicomp researchers “to find alternatives that embrace diversity, polyvocality, and difference” (137).

57

2

I N T E R P R E T I N G P O S T - ​D E S K T O P P R A C T I C E S As the world itself becomes replete with media, Mark Weiser’s original notion that we will live in an age of ubiquitous computing needs to be amended: our world will be, indeed is now, one of ubiquitous media. — ​J a y D a v i d b o l t e r e t   a l . , Throughout

While technologists have been grappling with technical challenges unique to ubicomp since the 1990s, the arts and humanities disciplines—​which Weiser thought so vital—​have been slower to interpret post-​desktop practices apart from more established digital genres and online activities. Nevertheless, the commercialization of ubicomp qua smartphones has put the paradigm on media studies’ cognitive radar. The cultural study of mobile interfaces has ramped up in proportion to iPhone sales. As the above epigraph attests, media and communication scholars now recognize that the technological sea change initiated by ubicomp’s adoption posses major questions about electronic textuality, digital expression, and the fluidity of multimedia artifacts. Ubicomp, according to Bolter et al., begets cultures of “ubiquitous media” whereby Internet access and content delivery far exceed Web browsers, game consoles, or Wi-Fi hotspots: “the whole world becomes available for the reception and production of mediated performance” (332). If all the world’s becoming a stage for media production and digitally inflected interactions, then how might we distinguish among the myriad ways in which media (are designed to) inhabit the shared environments that ground our copresent encounters? This chapter amasses an answer in the form of a three-​pronged spectrum. While there’s no denying the escalating amount of everyday objects, actions, and transactions now being digitized, the utter diversity of HCI modalities and media effects apparent in disparate mobile communication practices drowns out the resonance of a catchall term. Using a phone to text while driving differs immensely from using the same device for GPS navigation. Digital networks are a click away; however, one click may prompt

58

5 8   • Act i o n a b l e   M e d i a

audiovisual cues related to our present activity and another click might open an app that beckons attention to its rolling scroll of incoming messages. In ubicomp-​saturated societies, online media are becoming ubiquitous, but they are not homogeneous. The versatility of tablets, smartphones, wearables, and nanoelectronics renders hardware categories fallible as criteria for delineating post-​desktop practices. Only critical concepts will do. The following section assembles three conceptions of media emerging with ubicomp:  mobile media, locative media, and actionable media. When grasped together, they constitute a robust framework; alone, each phrase can seem like it applies to everything and nothing. The latter concept—​actionable media—​is obviously the one I foreground throughout much of this book. Building upon ­chapter 1, I argue here that actionable media production pursues and manifests the most ambitious horizon of outcomes envisioned by the ubicomp research agenda. That is, actionable media producers author digital content and arrange networked capacities with the resolute intent to enhance contextual awareness, inform present actions, generate data from proximate actant-​networks, and frame lived perceptions of a dynamic locale. The bonds between networked multimedia and local action are much tighter, both technologically and rhetorically, in actionable media practices than in mobile or locative media. Mobile and locative practices usefully connect users to information repositories, geospatial notification systems, and to (a)synchronous communication channels on an anytime-​anywhere basis. Indeed, they bring media to us wherever we happen to be, but often do so in ways that account for our surroundings only loosely or not at all. To move back and forth from mobile media and actionable media is to slide along a spectrum, not a hierarchy. If actionable media aligns with the bleeding edge of current ubicomp innovations, mobile media pertains largely to use cases whereby portable devices serve as miniaturized desktops. The former is not progressive or better in every case simply because it’s more forward-​looking. PC-​oriented genres of mobile communication such as texting, email, and social networking command a stronghold on global smartphone usage, owing surely to their profound utility and vast appeal. My purpose in articulating these concepts side by side is not to denigrate one in order to celebrate the other. I aim to emphasize their respective affordances and constraints as theoretical lenses, while remaining sensitive to the technohistorical moments that have informed each concept’s formation. After outlining these salient transmutations in humanistic scholarship and interface design over the smartphone’s first decade, I proceed to case analyses that help juxtapose

59

Interpreting Post-Desktop Practices  • 

59

mobile, locative, and actionable media as distinct perspectives available to scholars, designers, and general users. Those examples spotlight in-​vehicle digital communication practices, since much technical invention and public debate still revolves around the scene of driving and the place of post-​desktop interfaces therein. The chapter then concludes with reflections about how we might further theorize actionable media practices—​their potential to engender new forms of public authorship and deep attention—​at a time when computational capitalism appears poised to co-​opt all things digital.

From Mobile Media toward Actionable Media Coined in the wake of Weiser’s manifestos, field-​defining terms such as mobile media and locative media are not merely matter-​of-​fact descriptors. They emphasize certain practices, properties, and characteristics to the neglect of others. They are interpretations of the ubicomp paradigm and its cultural potency. Without doubt, the idea of mobile media stands as the preeminent interpretation of the post-​desktop era from 2007 to 2017; on-​the-​go access to commonplace Web platforms remains a perennial focal point for many users, software developers, and critics. Mobile media scholarship delivers valuable insights about the social impacts of our contemporary electronic boxes. In doing so, such work tends to examine post-​desktop interfaces that reflect or remediate personal computing, the desktop metaphor, and browser-​based services. Cultural inquiries on mobile media effectively reveal the extent to which popular smartphone practices short-​circuit the central contrast underlying Weiser’s design philosophy. Smartphones qua mobile media embody no qualms with the desktop’s virtual reality, so long as files, windows, and pages load quickly. I understand the mobile media stance as one of interpretive meiosis—​the classical figure of speech denoting instances of intentional understatement. Mobile designers strategically bracket the notion of smart environments, opting to concentrate instead on how people can use their portable devices without having to rely on peripheral technologies that were, especially a decade ago (and even at present), “still in beta” (e.g., sensors networks, computer vision, ambient intelligence). Likewise, mobile communication scholars gravitate toward widespread digital practices so as to study computation’s most pronounced and consequential effects, which are far less measurable in the embryotic phases when a technology might be taken up only in lab prototypes and experimental artworks. Mobile media research gravitates, with some exceptions, toward interfaces that have already reached a critical mass. To be clear, I am

60

6 0   • Act i o n a b l e   M e d i a

not claiming that mobile media is a fallacious critical/​creative lens; rather, harking back to the spectrum mentioned above, I only mean to point out that its conceptual fidelity to established desktop schemata conversely downplays the ubicomp mandate to make media actionable. My analysis revolves around mobile media’s scope and not the validity of its findings. Indeed, mobile media scholarship provides deft indications as to how Weiser’s vision of ubicomp has become adapted, abated, and sometimes compromised in early edition smartphones and popular applications to date. From Studying Mobile Media to Theories of the Mobile Internet and Foundations of Mobile Media Studies, numerous edited collections over the past five years have put forth comprehensive portraits of the mobile research agenda. Surveys on this growing field abound. Foregoing a summary of these summaries, I want to accentuate two conceptual tenets that buttress mobile design and its attendant discourse. Namely, notions of personal ownership and sheer portability. To the extent that mobile media practitioners exercise interpretive meiosis with respect to ubicomp, they do so under these two auspices: when they posit device ownership as a signifier for ubicomp access, and when they take movability to be the defining trait of post-​desktop interfaces. It is on these grounds, in other words, that mobile media narrows the theoretical scope of ubicomp technologies to their operational capacity to function as tiny PCs, as handheld electronic boxes cut off from the smart environment ideal. These meiosis-​style maneuvers occur in formative works of mobile scholarship, though I should stipulate that bearing the mark of meiosis does not diminish the immense value of such work so much as it serves to time stamp it. Theorists are bound more or less to the examples available at the time of their writing; different focal points emerge from different technohistorical moments. Jason Farman’s 2012 Mobile Interface Theory wagers that the more radical aspects of Weiser’s vision may amount to an unrealistic technological utopia, or at least a highly exclusive luxury reserved for a privileged elite. Writing during the dawn of smartphones, Farman predicts that smart objects, let alone smart environments, “[will] remain accessible to only a fraction of the world’s population” (9). On the basis of this claim, Farman proceeds to justify his methodological decision (and his field-​level recommendation) to limit the ubicomp paradigm to practices already supported by mass consumer electronics. He suggests that theorists who associate ubicomp with broader developments beyond mobile may be living on a prayer, so to speak, and that, in any case, the “massive potential for ubicomp . . . is already integrated into digital cultures: our mobile devices” (8). While I agree that certain iterations

61

Interpreting Post-Desktop Practices  • 

61

of mobile media do act as intriguing ciphers unto the ubicomp paradigm—​ and I reference many examples throughout this book—​I find Farman’s reasoning here consistent with the PC paradigm in that he takes personal ownership of a codified device to be a criterion for the arrival (or lack of arrival) of ubicomp technologies. Granted, elaborate “smart homes” enjoyed by the likes of Bill Gates will be no more widespread than multimillion dollar mansions are now (though products such as the Nest thermostat and Amazon’s “Echo” aim to bring some smart home functionality to the middle class). Likewise, most small businesses cannot afford to upgrade their workspaces to mimic the Googleplex. But ubicomp is not about personal computing. Ownership of a smart mobile device is not always a prerequisite for engaging with ubicomp interfaces, especially those that are designed to be ambient, embedded facets of public infrastructure. The viability and vibrancy of ubicomp infrastructure projects is, to be fair, easier to observe in 2017 than it was around 2010. Much to his credit, Farman’s theoretical insights stretch far beyond the study of mobile devices qua electronic boxes, and his prescient use of the term “information landscapes”1 remains an essential concept for making sense of emerging smart environments. Cities all over the world fund and manage publically accessible smart environments designed to foster a variety of civic and cultural initiatives. These projects are not as widespread as consumer smartphones, but they have existed in urban locales for over a decade and they attract the efforts of leading architects, planners, and interaction designers. For example, in 2007, the Spanish city of Zaragoza commissioned architect Carlo Ratti and his SENSEable City Lab at MIT to build a public smart environment titled Digital Water Pavilion (see Figure 2.1). Fountains and other water fixtures have long been crucial to the design of public squares around which citizens gather for communal events and collective lingering. The Mayor of Zaragoza asked Ratti to imagine how embedded computing might enhance the circulation of water in such gathering spaces, in order to amplify the traditional fountain’s social function. Digital Water Pavilion reconceives the abstract patterns characteristic to most fountains and turns the flow of recycled water into a system for crafting alphabetic letters. Ratti’s design treats water as an array of pixels. Water becomes a digital writing material, and the pavilion senses dynamic variables in the surrounding environment and actuates structural and rhetorical changes in response to local activity. Liquid letters can drop from the ceiling one word at a time, in rapid succession, making for a rather dramatic construction of sentences and lists in the service of public communication. Citizens may also send text

62

6 2   • Act i o n a b l e   M e d i a

Figure 2.1  Water shaped into patterns and words at the Digital Water Pavilion, debuted at the 2008 Zaragoza World Expo in Spain. © Carlo Ratti Associati, 2007.

messages to the pavilion, which publishes their texts in water for everyone to read. More than a setting for communal exchange, the fountain qua hydroponic, crowdsourced printing press supports a networked, yet hyperlocal public sphere wherein smart materiality initiates, reflects, and contributes to dialogues occurring in its proximity. A related installation project, The Adaptable Bus Stop (2006), extends the reach of Ratti’s design methodology to additional elements in the urban fabric. The Adaptable Bus Stop converts the wall of a typical bus stop into a massive touch screen. As Figure 2.2 demonstrates, bus patrons may consult real-​time data feeds pertinent to their commutes. Many mobile apps perform similar functions, of course. The more noteworthy aspect of the project is that it showcases a model for democratizing the power-​knowledge of smartphones by installing relevant app functions into public infrastructure. Device ownership is not required. Moreover, beside the real-​time map, the screen’s right half (less apparent in the photo) acts as a moderated bulletin board where locals can “post ads and community announcements” (“The Adaptable Bus Stop”). Dormant nodes in the city’s transportation network thus acquire an additional discursive dimension. The bus stops facilitate and punctuate a local social network grounded

63

Interpreting Post-Desktop Practices  • 

63

Figure  2.2  Researchers interacting with The Adaptable Bus Stop. © MIT Senseable City Lab, 2006.

in urban flows at the street level, which is perceptible to small crowds and passers-​by, not just smartphone owners. Public smart environments, in this sense, may potentially supplement current urban renewal efforts by remaking traditionally neglected/​blighted waiting areas into curated sites for actionable media production, civic engagement, and collective deliberation. Accounting for the growth of public ubicomp projects like these allows us to engage a wider range of the paradigm’s cultural potential, and, in turn, to resist claims that reduce ubicomp to the medium-​specificity of the latest product line of electronic boxes. Recalling the generative interplay evident in Weiser’s work among smart objects and smart environments—​rather than postulating a division between them—​is paramount to cultivating a more robust understanding of the post-​desktop spectrum. If a smart object or portable device is not operating relationally as part of a smart environment, then “true ubiquity” is not occurring. Media is mobile but not yet actionable. Mobile media analysis directs much critical attention to the ways smartphones, tablets, and wearables variously extend the reach of online activities to an unprecedented array of everyday settings. At the turn of century, Lev Manovich aptly foresaw the everyday scenes that have come to characterize mobile media societies:

64

6 4   • Act i o n a b l e   M e d i a

The previous icon of the computer era—​a [virtual reality] user traveling in virtual space—​has been replaced by a new image:  a person checking his or her email or making a phone call using a PDA/​cell phone combo while at the airport, on the street, or any other actually existing space. (“Poetics of Augmented Space” 221) This “new image” (which Manovich pinpoints but doesn’t necessarily endorse), blatantly contradicts the ideals of environmental knowing and contextual awareness at the core of Weiser’s intellectual ethic. The sheer portability affordance of anytime-​anywhere Internet access fosters online practices that, more often than not, prompt users to temporarily divert their attention away from the immediate proximity. Quintessential mobile practices, such as checking one’s email while walking the streets or an airport terminal, constitute nothing less than the retrofitting of ubicomp technologies by PC-​oriented conventions. In other words, while mobile software clearly brings the virtuality of computation out of stationary computers, the models of computing that they mobilize still pay homage to the desktop metaphor. Designing for portability without regard for proximity divides the action on screen from local action occurring in one’s surroundings.2 While ubicomp research agendas clearly aim to broaden the domain of digital networks, readers who isolate this element (taking it for the whole of Weiser’s philosophy) fabricate an understanding of ubicomp that could more aptly be described as Weiser’s worst nightmare. The most problematic meiosis occurs when critics base their interpretations of ubicomp on little else than the etymology of the word “ubiquity.” For example, rhetorician Amy Kimme Hea discusses ubicomp as if it were strictly a matter of manufacturing wireless, mobile hardware. Treating laptops and pocket-​PCs as emblems, Kimme Hea equates ubicomp with “the idea that place and time can be transcended,” which she fears might, in turn, “lead to an erasure of history, place, context, and agency” (204). This sort of technologically conditioned alienation from one’s spatiotemporal bearings is, as c­ hapter 1 stressed, exactly what Weiser and his colleagues objected to in their critiques of personal computing, virtual reality, and the desktop metaphor. Because Kimme Hea does not acknowledge Weiser’s mission to rethink computing and the entire flow of digital information in order to promote context-​awareness, she regards post-​desktop mediascapes purely in terms of a multiplication and miniaturization of PCs. From this standpoint, scholars might be led to conclude that the primary way in which ubicomp impacts communication is by providing a means to

65

Interpreting Post-Desktop Practices  • 

65

work on digital documents in more places. Writers can start composing a document in their office and, smart device in tow, continue composing later in a bus or at a picnic table. If our interpretations of post-​desktop affordances stop at sheer portability, then ubicomp’s emergence appears to bear minimal rhetorical implications. Only when we grasp ubicomp at a paradigmatic level do we realize the more profound stakes at hand. Ubicomp is not only about expanding the territory in which acts of media production can occur; more profoundly, it correlates with the founding of new forms and publication spaces through which texts, images, audio, and video can circulate. In doing so, ubicomp enables the rhetorical constitution of proximate smart environments in real-​time, whereby digital authorship may accrue dynamically from transductive relationships among composed multimedia, local actants, and other material flows. These emerging media ecologies present complex rhetorical situations that foreground unique opportunities and challenges for all digital creatives, whether they are typing words on a smartphone or designing images on a desktop. Increasingly, multimedia is being read, heard, and viewed within ubicomp-​enabled contexts of reception. These new contexts of reception, in turn, reshape the very exigencies and objectives that motivate acts of media production and digital communication. In spite of its strong emphasis on smartphone practices that favor portability over proximity, the scope of mobile media analysis has steadily diversified to account for popular devices’ expanding feature sets. Location has become a privileged variable in mobile communication research on alternate reality games, digital wayfinding, and location-​sharing apps such as Foursquare and Glympse. These case studies show a general trend: whereas so much early work in the field hinged upon mobile computing’s apparent detachment from material bindings (with keywords like “wireless,” “untethered,” “anytime-​anywhere”), many scholars now devote their attention to post-​desktop practices that connect us to places in unprecedented ways. The term “mobile communication” began to evoke very different activities in 2013 than it did a decade prior. Reflecting on this shift, Scott Campbell casts an established, all-​encompassing definition of mobile media that pertains to “devices and services that support [or at least enable] mediated social connectivity while the user is in physical motion” (9). Campbell quickly stipulates, however, that mobility becomes most meaningful when “it allows for flows of information and communication to be more seamlessly weaved into the rhythms of everyday life” (10). Recent turns in mobile scholarship endeavor to study handhelds and wearables as interfaces that integrate with users’ spatiotemporal contexts, in addition to their capacity to broker portable access to

6

6 6   • Act i o n a b l e   M e d i a

online commonplaces. Further stretching mobile media’s more familiar connotations, Rita Raley’s literary look at mobile media poetics pushes the phrase to its limits. Raley’s notion of mobile applies strictly to artworks and narratives (including Cardiff ’s audio walks) wherein “content responds dynamically to the place of the reader/​participant” (303). The widening range of perspectives that employ the mobile media moniker attest to its amenability; at the same time, while contrasting usages of shared terms do cultivate productive tensions, this heightening divergence also serves to fracture the conceptual consistency of mobile media, rendering it a less precise critical lens in need of attendant modifiers. Enter “locative mobile media,” “geomobile Web,” and “location-​based mobile social networks.” Pivoting upon such terms, the locative turn in mobile communication studies has ushered the field toward post-​desktop practices that more closely reflect Weiser’s ideals and subsequent ubicomp innovations. With the rise of geospatial data and location tracking, the scope of scholarship has broadened to consider smartphone apps in terms of their evolving capabilities to help people navigate places, store multimedia at particular coordinates, and to broadcast the position of users within location-​sharing networks. Several media researchers, including Alice Crawford and Gerard Goggin, invoke Henri Lefebvre’s conception of space as a social production, along with Situationist practices such as détournement and dérive, as theoretical resources for understanding the potential of locative practices to reconstruct both our experience of everyday places and the arrangement of media on the Web.3 Whereas early commercial iterations of location-​based services were limited simply to consumerist wayfinding (e.g., helping tourists find the nearest Starbucks or ATM), Crawford and Goggin identify hopeful trends in “user-​generated geomobile content that promises to . . . elaborat[e]‌‘fields of care’ based on community, history, and individual stories about the places in which we live” (105). Locative interfaces theoretically empower citizens to produce and share their own digital cartography, which may impart subversive or at least spontaneous mappings of space, suggesting tactics for resisting the generic place(less)-​making strategies of transnational corporations. Another strand of inquiry examines how Web 2.0 giants such as Wikipedia, Facebook, and Yelp are evolving to include location-​based features, many of which allow users to rearrange the display of content via geographical metrics (de Souza e Silva and Sutko 2011; Humphreys 2013; Frith 2015). A key difference, then, between mobile media and locative media—​as interpretations of the ubicomp paradigm—​is that locative theories and practices underscore the merging of digital media and physical locations. This affordance owes its

67

Interpreting Post-Desktop Practices  • 

67

existence to mobile technologies, of course, but the sheer portability defining conceptions of mobile media is, in locative frameworks, no longer a chief focal point orienting design, user experience, and critical reflection. Being online anytime-​anywhere gives way to augmenting what’s here and now. Actionable media shares a degree of conceptual overlap with locative media. Just as locative practices emerged from mobile devices (leading many writers to use “mobile” and “locative” interchangeably), actionable media projects owe the conditions of their possibilities to location-​aware interfaces, be they portable devices or sentient structures. Location-​awareness enables digital data to be shared and accessed on the grounds of a specific place. This capacity is foundational for the creation of smart environments that support ad hoc networks among proximate constituents. Location-​awareness remains, however, a technological step down from context-​awareness. The terms of this distinction elucidate the gradient contrast between locative and actionable media. Ubicomp luminary Adam Greenfield observes that even interaction designers often confuse location and context (“Worth a Thousand Words”). As an HCI modality, context-​awareness more acutely accounts for the live actions taking place in a location. Location can be measured in binary fashion (present/​absent). Context-​awareness involves more dynamic means for processing ensembles of objects and activities in real-​time. In order to be deemed context-​aware, an interface’s functionality must “largely if not entirely [be] determined by the other networked objects around it” (Greenfield). Stepping up from location-​awareness requires the capability to process and track not only the geographical coordinates of a single entity, but also the capacity to constellate the relative positions of multiple objects in motion. It’s the difference between a push notification system and an augmented reality app. Representative of the former, locative software developed by the likes of Groupon and Taco Bell sends digital coupons to users when they traverse certain geographical boundaries. One’s basic presence triggers the push notification. The same message transmits to each user, and the advertisers hope that reading the text (e.g., “Get your free Doritos Locos Taco with purchase today”) in the vicinity of a Taco Bell restaurant will prompt users to swing by. While this example is culled from the floor of locative media, it does illustrate how location-​based services approximate rhetorical relationships between the electronic texts they deliver and the contexts in which those texts (or audiovisuals) are received. If a user passes over location X, then send said user content Y. Locative media pulses to geographical beats. Stipulated coordinates broker data transmission. Meanwhile, whatever local actions surround

68

6 8   • Act i o n a b l e   M e d i a

the reception of locative media may affect one’s reading in roughly the manner any given environment envelops an encounter with any book, or presumably more so, if the text has been composed to relate to that context. But text and context remain divisible in locative practices that dot much communication scholarship. At the very least, to extrapolate an example from Adriana de Souza e Silva and Daniel Sutko, a Wikipedia entry about the Washington Memorial that gets “pushed” to WikiMe users standing in the tower’s shadow is, syntactically speaking, the same piece of text I can read on Wikipedia from my office computer. The location-​based content is merely redistributed as is from the Web. The only notable rhetorical decision, in cases such as this, is the selection of a “physical location [that] provides the context” for the recycled text (de Souza e Silva and Sutko 26). Location and content must be toggled between; they correlate in the manner of a hyperlink that, when clicked, opens a website in a new window. Locative media artifacts are stored on the spot, but they do not cohabit the scene. Actionable media—​the critical lens situated at ubicomp’s context-​aware horizon—​casts attention toward practices and projects whereby text and context are fundamentally indivisible and processually co-​constitutive. When multimedia is composed to be porous and sensitive to its surroundings, so-​called “contexts” actively (re)shape the composition of the work on a variety of formal levels. This transductive fluidity among texts and contexts is an axiomatic principle developed more fully in my efforts to theorize actionable media vis-​à-​vis the history of writing, rhetorical analyses of contemporary projects, and avant-​garde aesthetics across c­ hapters 4, 5, and 6. Nevertheless, basic themes have already been broached in the descriptions of actionable practices earlier in the book. Digital traces charted by the CycleAtlanta app, referenced in ­chapter 1, exceed traditional location-​based services. The network of cyclists’ routes doesn’t just indicate where users are or were; it reveals to the city’s transportation planners the precise infrastructural contexts that motivate, alter, or constrain the cyclists’ collective actions. The electronic trails generated and displayed in the app are, borrowing from Greenfield’s definition of context-​awareness, “largely if not entirely determined” by proximate objects, local actions, and built structures surrounding each cyclist as she rides from block to block. Actionable media projects of this ilk deploy the context-​ aware affordances of ubicomp platforms to digitally write the actant-​networks that constitute specific localities, often with the intent to advocate for change. Conversely, augmented reality artworks and electronic audio walks (such as Janet Cardiff ’s Her Long Black Hair in Central Park) create multimedia that enmesh with the urban scenes they inhabit. Listeners hear her recorded voice

69

Interpreting Post-Desktop Practices  • 

69

and the voices of passersby in the same breath. In the case of Cardiff ’s work and many other actionable projects I examine later, meaning-​making happens at the interstice of text and context, composition and circumstance, content and commotion. At the same time, it’s imperative to recall that technology alone does not underwrite the spectrum shift from locative to actionable media. Not all augmented reality apps, for instance, facilitate environmental knowing or bolster one’s capacity to perceive and reflect upon the situation at hand. Software categories are just as amendable (and fickle) as hardware categories. Consider the 2016 global phenomenon, Pokémon Go. Touted as the first “killer app” of augmented reality, Pokémon Go broke mobile gaming records for downloads and revenue in its first two months, while also surpassing Twitter in daily active users. The app aims to extend the popular transmedia franchise and its fan base by mapping the gaming experience across cities and towns, inviting smartphone users to “catch Pokémon in the Real World” rather than on their home televisions, desktops, or Nintendo consoles. The fast and loose manner through which this mapping of AR content occurred, however, clearly resembles the approximate rhetorical relationships characteristic of location-​ based services. The app’s geospatial interface, which mimic’s Uber’s virtual-​ first design, encourage players to experience the world through the lens of the game, which is not inherently problematic, but it can be. In one highly publicized case, hundreds of New Yorkers suddenly “stampeded” into Central Park at 11:00 PM on a Thursday, in search of the character Vaporeon, an especially rare and elusive catch (Price). Pictured in Figure 2.3, video footage of the event captured by stunned onlookers shows how surreal it was—​and how powerfully the game can propel its most passionate users into a single-​ minded, stop-​at-​nothing pursuit of location-​based virtual targets. At its most frenzied, the app’s ludic world reduces locales to a series of checkerboard-​like spaces to be traversed by the mandates of gameplay. Never mind the car you’ve left running in the middle of the road; there’s no time to look both ways before crossing this street. The game beckons, and heeding its call requires travel to particular geographical coordinates, but attending to the particularity of a place is optional. Catching a Pokémon creature in Central Park involves the same process as catching one inside a shopping mall. Places are intermittently privileged points on the app’s virtual map, meaningful only to the extent that they mark the spot of the present hunt. None of this is to deny or trivialize the social, emotional, and aerobic benefits that early adopters of Pokémon Go so frequently cited. Aside from the rare outcomes that made headlines shortly after its launch, the game is a

70

7 0   • Act i o n a b l e   M e d i a

Figure  2.3  Image from video capturing a Pokémon Go “stampede” in Central Park, New York. © Jukin Media, 2016.

largely benign form of leisure that a record-​setting amount of people enjoy. That enjoyment, though, springs from the thinnest of ties between digital content and local settings (e.g., water-​faring creatures are placed near bodies of water). Programmers working at Niantic Labs in San Francisco can, equipped with a database of crowdsourced nominations, dole out digital creatures to landmarks, intersections, and parks around the world without leaving their desks. Views ascertained through Google Earth are the seat of the app’s invention. The map, not terrain, drives rhetorical decision-​making about the arrangement and delivery of game content. Suffice to say, this is a far cry from the approaches modeled by Janet Cardiff ’s audio walks, the CycleAtlanta app, and the actionable media projects I examine at greater length in subsequent chapters. Pokémon Go, whatever else it may be, is not a strong model of actionable media.

Modeling Actionable Media Further degrees of difference between mobile, locative, and actionable practices manifest in the contrasting media ecologies at play when people compute in cars; namely, in three fundamental use cases: distracted driving, GPS navigation, and driverless vehicles. Automobiles have long been an important, incubative venue for emergent forms of wireless communication and

71

Interpreting Post-Desktop Practices  • 

71

networked media consumption (Goggin “Driving the Internet”). From dashboard radios to in-​vehicle Internet, enterprising technologists perennially target the hours that citizens of car-​driven societies spend behind the wheel. In-​vehicle media practices can be read to reveal much about the paradoxical effects and contradictory outcomes associated with smartphone habits in other settings as well. Computing in cars showcases (at their most extreme) the capacity of post-​desktop practices to withdraw attention from our surroundings and, in other cases, boost contextual awareness. To engage with a handheld or wearable interface is to be always in possession of these distinct HCI modalities and the disparate modes of attention they foster. Though distracted driving and GPS navigation may seem mundane (at least from an aesthetic perspective), I see theoretical value in examining the basic ways in which multimedia circulates and gets generated within each scenario. Ultimately, by foregrounding the actionable aspects of GPS navigation, we might regard its media ecology as a prototypical model for understanding an array of nascent rhetorical situations that invite writers, researchers, artists, and designers to create actionable media projects intended to be experienced by pedestrians and passengers. The objective is to extrapolate themes from these commonplace activities, along with speculations about the future of vehicular media consumption in driverless cars, in order to imagine further possibilities for digital texts and audiovisual works that fully exploit the context-​aware affordances of ubicomp platforms. Moreover, reflecting on the outcomes associated with these three modes of digital driving underscores the acute ethical concerns one should be mindful of when conceptualizing and authoring post-​desktop mediascapes that inhabit dynamic social settings. In situations where people are engaging with networked multimedia in the midst of performing an extracomputational activity, digital creatives cannot approach their work as if it were going to be accessed in a cyberspace vacuum. The wisdom of Weiser and Brown’s foresight bears repeating here: “when computers are all around, so that we want to compute while doing something else . . . we must radically rethink the goals, context and technology of the computer” (“The Coming Age of Calm Technology” 3). In valorizing anytime-​ anywhere connectivity or sheer portability as the quintessential capacity of post-​desktop interfaces, many designers and users of smartphones struggle to negotiate a discord that is programmatically installed:  smartphones enable people to “compute while doing something else,” but the vast majority of current applications bring the virtuality of computation out into the physical world without giving heed to Weiser’s second objective to make computers interface with the “infinite richness of the universe.” As such, mobile practices

72

7 2   • Act i o n a b l e   M e d i a

often extend the realm of personal computers and the browser-​based Web without radically rethinking computing and the flow of digital information. Nowhere are the unintended consequences of this halfway adaptation more evident than in the phenomenon of “texting while driving,” or distracted driving, which has revved up since the early days of smartphones. According to the US National Safety Council, one-​fourth of all automotive crashes (about 1.2  million) in 2010  “[were] the result of distracted driving” (“Annual Estimate”). Early legislative attempts to ban or regulate smartphone use while driving have failed to slow the steady increase of crashes, especially among teenage drivers—​nearly half of whom admitted, in a 2012 NHTSA survey, they send texts and emails while driving (Halsey). Another 2012 study by the US Department of Transportation highlights the severity of the issue, indicating that texting while driving involves a greater sense of withdrawal from the act of driving—​apparently six times greater—​ than drinking and driving does (“Blueprint for Ending Distracted Driving”). While most initiatives endeavor to document, condemn, and cope with the outcomes of distracted driving, communication scholars Scott Campbell, Joseph Bayer, and Rich Ling point out that “very little research has been done to explain this behavior” (28). Campbell et  al. argue that expectations of anytime-​anywhere connectivity and a growing acceptance of divided attention have burrowed deep into the subconscious of mobile users. Being always at hand, mobile devices “become an important part of not only the user’s identity, but also the social and environmental ecology” (Campbell et al. 30). Routine users begin to take for granted the ability to text and participate in mobile social networks at a moment’s notice. Interlocutors expect each other to be reachable and responsive, and their devices’ default settings offer up incessant notifications on a rolling basis. The social shaping of mobile devices in this vein has tended to normalize divided attention. Concerned technologists tout hands-​free wearable devices as a technical solution, insisting that headworn gadgets do not require drivers to glance downward, as they must do when texting on a smartphone. Referencing Google Glass, Campbell et al. maintain a sense of skepticism, quite rightly I think, about whether the rise of this new form factor will actually “mitigate absent presence” or “if it might just mitigate the appearance of absent presence” (33). Texting via a headmounted display and speech-​to-​ text software will hardly “fix” distracted driving. Technical bells and whistles do not in themselves negate the fundamental division between screens and surroundings that the practice brokers.

73

Interpreting Post-Desktop Practices  • 

73

Of course, not all digital communication harbors divided attention. So many “live” events now rely on electronic mediation at their very core. The liveness of a rock concert, as Philip Auslander demonstrates, is certainly not diminished by the amplifiers and audio systems that project sound throughout the venue; media technology in such cases is in no way secondary to an otherwise nontechnical event (97). The same could be said for the role of instant replay at a sporting event, or even museum exhibits that feature audio tours. If these examples illustrate how “the mediated can actually constitute the live event,” then texting while driving (or social networking during meetings) constitutes another end of the spectrum whereby “the intimacy fostered by asynchronous mobile communication . . . functions as the main, embodied space for social interaction” (Farman 101). Texting compromises the live event of driving. Though texting is theoretically asynchronous, users’ expectations for availability and responsiveness render it a more or less synchronous mode of communication in practice. Mobile social media have clearly intensified the sense of urgency already evident in text messaging practices. On social networking threads, one is always on the verge of being too late to read, post, and comment. Every new post drives previous posts further down into an abyss of data that loses social currency with each passing hour. The kairos of mobile social media is now—​always now—​and the pull of collective synchronicity, the “fear of missing out,” can compete all too well our perceptual awareness of local action. The weight of this pull is evident in Farman’s description of a commonplace example from his own mobile media use. The scene: Farman and his colleague are carpooling on their morning commute to campus. His colleague is driving while Farman rides shotgun, checking his phone every few minutes to read messages from his students as part of a text messaging project he had recently assigned. Eventually, the colleague asks Farman to “stop doing that,” to which Farman replies: “Doing what?” (113). What does this reply signify? Is Farman’s habit of checking his phone in the presence of others so natural that he doesn’t pay it much attention? Is he so absorbed in the distant-now of his mobile social network that he just needs a second to shift his focus back to the here inside the car next to his colleague? Does the response serve as a playful lead-​in to Farman’s explanation of the class assignment (which aims at cultivating “collective intimacy” among a remote group via texting)? In any case, after listening to Farman’s rationale, the colleague says, “Well, it feels like two people are whispering back and forth right next to me and I’m not included in the conversation” (113). When we prioritize anywhere-​anytime

74

74   • Act i o n a b l e   M e d i a

instant messaging over and against local co-​presence, we normalize moments like these in the manner of a sacrifice to the distant-​now. Farman realizes this: In this moment, the obvious tension between the mobile device as an individual interface and as a tool for connecting socially with others finally made itself very clear. In the car, I felt a distinct sense of connectivity with my class; however, the mode of connectivity was simultaneously an isolating and excluding activity. (113) While the spread of distracted driving (and comparable mobile practices that habituate divided attention) may be symptomatic of PC-​oriented approaches to ubicomp, we can find in the margins of this same scene an alternative, actionable model: GPS navigation. GPS navigation provides a basic starting point, a generative prototype, from which theorists might articulate more compelling interpretations of post-​desktop interfaces in order to help reorient multimedia production, digital communication, and other technocultural practices. The aforementioned turn to locative media and location-​aware computing, which much of Farman’s work contributes to, marks a productive step on the way to a more comprehensive framework. With the rise of geoinformation and the democratization of GPS over the past decade, scholars have increasingly discussed mobile devices in terms of their evolving capabilities to help us locate places, attach digital data to particular locations, and trace the current position of users within location-​based social networks. Still, while GPS navigation is clearly mobile and locative, these qualities alone do not account for the technology’s more radical, most important affordance: the capacity to sync with (i.e., to invent or arrange multimedia in response to) the live actions of human, non-​human, and environmental actants in the driver’s proximity. Tracing the media ecologies of GPS navigation in a manner that foregrounds the rhetorical involvement of local actions/​actants will, in turn, foreground an alternative path for theorizing post-​desktop practices and articulating principles suited to composing actionable media designed to circulate amid the built environment. When examined rhetorically, the media ecologies characteristic to GPS navigation make it clear that some of Weiser’s loftiest ambitions have already manifest in commonplace technologies since the early 2000s. Humanities scholars have simply not yet articulated a robust theory that identifies the possibilities GPS raises for the future of multimedia writ large. First, consider its basic functions as a writing machine. GPS navigation software generates

75

Interpreting Post-Desktop Practices  • 

75

audiovisual traces and directional cues in direct response to drivers’ otherwise non-​computational gestures and movements. The images on screen adapt to the turn of the wheel in real-​time. The audio narrative of turn-​by-​ turn directions is paced to coincide with relevant moments of action or decision, referring to street names that are likely to match up with signs present in the driver’s field of vision. As opposed to a map or a list of directions pulled up from the Web, GPS software delivers actionable audiovisual fragments to drivers such that they need not disengage from the act of driving in order to engage with computation. The act of driving becomes an HCI modality. GPS navigation models rhetorical processes at the conceptual core of actionable media: it (re)arranges flows of digital information to account for incidental fluxes, delivering multimedia at the periphery of proximate events in a timely fashion, with the intent to compliment or contextualize local action rather than compete with it. In addition, GPS navigation records and displays the action of autonomous entities in real-​time (e.g., changing traffic patterns), thereby creating an indexical representation of social and environmental ecologies unfolding across a specific locale. This affordance complicates the user-​centered logic whereby individuals issue overt commands to their PCs via keyboard and mouse. From her empirical research on GPS usage, Amy Propen concludes that “the GPS and its user co-​construct an interactive agency that is malleable and involves a process of ongoing exchange, negotiation, and sometimes resistance” (141). A notable example crops up in her interviews with GPS users who described their tactics for altering the software’s driving directions. The interviewees said they would regularly and purposefully disobey certain turning instructions, knowing that the device would adapt and calculate a new route (123). Counterintuitively, the drivers’ agency to command the software in these cases derived from the device’s ability to maintain precise surveillance over them. Drivers “write” within the software insofar as the device “reads” their every move. The redistribution of rhetorical agency in GPS media production suggests a mode of environmental knowing that can at once be “understood as contributing to posthuman, embodied ways of knowing” (Propen 123). This insight may prompt some confusion, especially if we recall the anthropocentrism on display in early ubicomp research. If the intent of ubicomp was/​is to make computation more integrated with human action, then how does one account for this contribution to posthuman ways of knowing? As today’s digital innovators aim to craft interfaces that operate in concert with human action, they inevitably assign computational functions to nonhuman

76

7 6   • Act i o n a b l e   M e d i a

entities and ambient flows, ostensibly as a means to achieve anthropocentric ends. And yet, in spite of this human-​centered design bias apparent in technologists’ theoretical discourse (including Weiser’s manifestos), the actionable practices emerging with ubicomp platforms suggest a kind of rhetorical-​ontological flattening occurring within twenty-​first century media environments. Among context-​aware use cases, digital communication no longer circulates unto itself, impervious to its surroundings. Actionable media ecologies bond texts, software, and events together in a bilateral, enunciative assemblage. The composed work is irrevocably hospitable to proximate flux. By contrast, book pages and many webpages will retain the relative permanence of their initial form and format regardless of where they are read. When a reader moves or the milieu fluctuates, the text stays still on the page. With GPS navigation, on the other hand, the array of material flows that dynamically constitute a locale effectively (re)compose the text. The invention, arrangement, and delivery of multimedia in GPS navigation accrues through human-​ automotive movements amid lively terrains. In striving to bolster real-​time environmental knowing, a crucial element of GPS navigation is this capacity to trace flows of activity relevant to the present task, the situation at hand, etc. As such, the virtuality of computation is not only brought out into the physical world; digital networks come to interface with the richness of local action. That GPS navigation functions in real-​time relative to dynamic, geographical variables also challenges some longstanding epistemological norms and temporal conventions associated with print-​based literacies. When reading a book or magazine, for instance, people can generally distinguish between their perception of live events and their engagement with forms of technical memory. Granted, one’s engagement with written artifacts alters, even furnishes, the criteria that shapes his or her lived perceptions. Nevertheless, this influence occurs over time—​it is deferred—​on account of the technics (i.e., techniques and technologies) of alphabetic writing. Philosopher Bernard Stiegler argues that, because audiovisual media are overshadowing the archival modalities of linear print, the literate experience of historical consciousness is becoming supplemented by the real-​time of contemporary technics (Technics and Time, vol. 3, 136). Streams of real-​time information events—​ to the extent they are made actionable, as with GPS navigation—​conjoin lived perception and technical memory, a phenomenon I discuss at length in ­chapter 4. One hears and sees digital content that variously informs, inflects, or otherwise frames her capacity to perceive, reflect upon, and act into the surrounding environment.

7

Interpreting Post-Desktop Practices  • 

77

While GPS navigation has channeled the flow of in-​vehicle multimedia strictly around the act of driving, what’s the use of turn-​by-​turn directions in a driverless vehicle? What kinds of media experiences will be had in self-​driving cars replete with passengers? Multiple scenarios come to mind. Quite feasibly, akin to the transportation scenes in the dystopian Disney film Wall-E, automated cars may double as home theaters on tires: tinted windows, surround sound, interactive small screens for each passenger. Relieved of driving duties, ex-​drivers who used to text at the wheel could text at will and much else. We may come to pass through city streets and country roads as if aboard a subway, submerged not underground but in a backlit glow. That’s one future on the driverless spectrum—​mobile media at 60 miles per hour—​for those who prefer the isle seat and don’t get queasy while reading in cars. To the window-​seat minded, for whom movement through landscapes holds a rather cinematic quality, driverless vehicles promise much as an emerging venue for actionable media display.4 As navigational prompts become less relevant to passengers in driverless cars, the media ecology underlying GPS navigation may accommodate and deliver actionable fragments of narrative, argument, history, journalism, or social commentary instead of turn-​by-​turn directions. The basic mechanics would remain: bits of audiovisuals paced to correspond with a series of lived scenes unfolding during the ride; context-​aware features would alter the presentation of content to account for dynamic variables in real-​time. But the scope of in-​vehicle actionable media will have ballooned out, beyond the stuff of street signs, to become amenable to genre experiments and readerly experiences of all sorts. Furthermore, as recent iterations of GPS have demonstrated, in-​vehicle actionable media projects may, in driverless cases especially, utilize the car’s windows as a translucent canvas for screening graphics in line with passengers’ perceptions of the surrounding environment. What has taken place on the smartphone may now be projected onto the car’s windows and broadcast through its speakers. Venerable literary and filmic traditions (e.g., travelogues, road movies) could evolve to literally be experienced over the course of a road trip—​like listening to an audiobook that was written to proceed in sync with the historic terrain one witnesses along, say, US Route 15 (aka “The Old Carolina Road”) from Gettysburg to Manassas. Actionable media narratives may cast a new metaphor over driverless automotive journeys:  the car becomes a vehicle for twenty-​first century theoria,5 and not exclusively a mobile home theater. In-​vehicle actionable media practices may also factor into our everyday commutes. With major magazines like The Atlantic already mapping out

78

7 8   • Act i o n a b l e   M e d i a

editorial strategies for emerging geopublishing platforms, one can envision partnerships among digital publishers and driverless transportation services whereby passengers encounter city-​specific stories that pertain to the areas they traverse (Madrigal). Open-​source platforms could also furnish vehicles with hyperlocal libraries of actionable content created by regional news outlets, community activist groups, and urban universities. Though driverless cars are a relatively speculative and contested endeavor, they remain a subject of immensely funded research and development. Their potential to alter societies for better or worse vastly exceeds matters of media and communication, but they do stand to affect digital culture in serious ways. Today’s actionable media projects are generally designed for pedestrians; given the state of self-​ driving cars, it is worth anticipating that tomorrow’s projects could also be experienced by passengers. At the very least, the disparate HCI modalities and modes of production that now characterize the present range of smartphone practices, from mobile media to actionable media, seem likely to inform the emerging spectrum of use cases at hand in driverless vehicles.

Actionable Media, Computational Capitalism, and the Need for Critical Invention More than mere trends in transportation, the media ecologies developing around in-​vehicle digital communication practices are early indicators of new interaction modalities among multimedia, humans, and environments. GPS navigation—​as pervasive as it has become—​is only a subset of a much broader array of actionable practices emerging with the spread of smartphones, tablets, wearable computers, augmented reality, urban informatics, and nanotechnology. Certainly, like print and film have been, all of these technologies have been and will continue to be pressed into the service of authoritarian regimes, corporate advertising, and other problematic forces that demand fervent critique. While ideological resistance is of the utmost importance, theorists and teachers across the humanities may also strive to discern and experiment with the context-​aware affordances ubicomp raises for cultural expression, civically engaged pedagogy, and public intellectual discourse. Without intending to, Walter Benjamin supplied an impetus for such endeavors: “What, in the end, makes advertisements so superior to criticism? Not what the moving red neon says—​but the fiery pool reflecting it in the asphalt” (86). Here and elsewhere, Benjamin’s statements about the decay of criticism seed ripe prospects. His famous image emphasizes the interplay of the sign and the surrounding scene. Not the red neon sign’s message nor its manic flashiness, but the reflection

79

Interpreting Post-Desktop Practices  • 

79

it cast upon the dampened street. Words projected across a puddle become actionable. With ubicomp—​and the scope-​broadening of actionable media production that it makes possible—​perhaps criticism, too, might inscribe itself over the asphalt, so to speak, in an effort to inhabit the built environment as much as advertisements do, toward radically different ends. Of course, this move is predicated upon the realization that we may occupy critical, authorial positions in relation to the multitude of ubicomp technologies entering our lives, which have more often been associated with passive consumption, mundane utility, and soft surveillance. Decades of dystopian sci-​fi narratives (e.g., The Terminator, 1984; They Live, 1988; Minority Report, 2002; Rainbow’s End, 2006) have cast characters as mere consumers (if not victims in the crosshairs) of augmented reality data, wearable interfaces, and smart environments. Scene after scene emotes feelings of inefficacy, disorientation, and entrapment; agency is limited to technocratic agencies, military personnel, wealthy corporations, and a few muscle-​bound rebels. Slowly but steadily, many ubicomp platforms are becoming as open to diverse public authorship as Web 2.0 genres. Technical and financial barriers to actionable media production are rapidly diminishing. Anyone who can create a blog or wiki on the Web possesses enough know-​how to create an augmented reality project in the locales of their town or city, for example. Now, the more pressing and profound challenges entail considerations that are rhetorical, aesthetic, ethical, and political. The democratization of ubicomp devices by no means guarantees a thriving new era for public authorship amid local settings. There are at least two ways to integrate digital networks with extracomputational activities occurring in built environments. Post-​desktop interfaces may be tucked into the nooks and crannies of urban infrastructures and interior spaces so as to be active behind the scenes, processing data and performing operations without drawing attention to themselves. Such is the modus operandi of fast spreading smart city initiatives that, well intentioned or not, collect big data about citizens on a top-​down, unilateral basis. Thus engineered, actionable information extracted from everyday social flows stands to reinforce the power-​knowledge of elites in marketing, security, and management. If companies’ abilities to track consumer behavior wasn’t alarming enough, executives have entered into cost-​benefit discussions about the prospects using biometric wearable systems to track their employees’ job performance (Green). Technocratic, corporate coercions confirm the gravest fears about emerging media, but these initiatives are rooted far more in the psychology of their proponents than in the technology itself. Alternatively, these same technologies can also

80

8 0   • Act i o n a b l e   M e d i a

be mobilized to engage people through direct address and open dialogue. We may craft them into communication platforms for producing, curating, and delivering actionable media to and from diverse arrays of audiences and contributors. Drawing contrasts between ubicomp’s rhetorical affordances and the politics of public authorship characteristic to print cultures, Malcolm McCullough claims, “New forms of digital tagging create new roles in who has the right to mark up the city” (453). Technologies such as RFID tags and geotags allow digital creatives to embed networked multimedia into objects and settings in ways that support greater and more varied plentitudes of (counter)public messaging than comparable print genres (e.g., flyers, signs, pamphlets). Hence, ubicomp-​enabled modes of “embellishing everyday space provide remarkable range for cultural critique . . . [and] can allow information media to provide orientation rather than escape from urban environmental awareness” (McCullough 453). Contra to Big Brother-​style data analytics, open networks of actionable media may bring digital content to built environments in a grassroots fashion conducive to free expression, civic debate, and public knowledge building. Theorizing this alternative potential for diverse authorship of built environments does not discount the likelihood that more mundane, passive, or blasé practices may prevail in popular usage. Big tech companies gravitate toward the latter. From Xerox to IBM to Google, the primary exigencies that have spurred ubicomp’s industrial development (and public perception) are overwhelmingly tied to neoliberal ideals. These sentiments are especially obvious in “demos” of various digital projects and services, as noted in the above commentary on Weiser’s Sal anecdote, and as can be observed in just about any Google promotional video. For instance, a video introducing Google Maps Navigation celebrates GPS technology on account of the fact that “finding the most convenient burger joint has never been easier” (Google 2009). Of all the ways to introduce GPS navigation, Google decides to define it as a better way to connect people to McDonalds, and it’s not just Google. The visual search engine Blippar uses computer vision to recognize objects and display overlays of related online content. To showcase the app, which bills itself as the augmented reality successor of browser-​based search engines, the company’s “proof of concept” promos revolve around branded mega products: Pepsi cans, Kit Kat bars, and Heinz ketchup bottles. Blippar’s quest to “make objects come to life” resolutely begins with the stuff of multinational advertising. Adopting a pair of Stiegler’s terms, it is quite reasonable to associate these marketing campaigns and limited applications of ubicomp with the broader liquidation of existence in the name of subsistence, the erosion of

81

Interpreting Post-Desktop Practices  • 

81

citizenship in the wake of consumerism. As computing moves from our desks to pockets to our eyes, the computer’s contextual awareness and optical preferences stand to play a more definitive role in our lived experiences and social relations. The existential and political stakes of digital networks are only getting higher. As Stiegler’s writing suggests, “computational capitalism” amplifies the reduction of existence to subsistence by creating and maintaining pervasive feedback loops whereby desire is perpetually aroused and instantly linked to arrays of products and services that can be browsed and purchased anytime-​anywhere in a few clicks (Decadence 37). At the onset of desire, one is driven to consume or to work in order to buy. Digital intensifications of neoliberalism portend the “decadence of industrial democracy”; in response, Stiegler believes that critical resistance is necessary but not sufficient. He calls for critical invention: “One must struggle against this tendency by inventing rather than by resisting” (Decadence 37). The scholarly condemnation of certain iterations of ubicomp, which merits volumes, will continue to fill many worthwhile pages. There is no shortage of emerging applications that invite skepticism, concern, and outrage. But I opt to devote more space, over the remaining chapters, to trying the approach suggested by Stiegler’s mandate. There is a glaring shortage of post-​desktop practices that support public deliberation, cultural criticism, and other genres of deep attention. To date, few theorists have endeavored to galvanize the potential of ubicomp technologies to “become objects of culture, that is, of practices, and not merely of usages” (Stiegler, Decadence 144). Context-​aware, real-​time technologies stand to affect knowledge building in profound ways. All information, communication, and memory practices that generate, preserve, and circulate knowledge are of technics and therefore not outside of time and its technical evolution (Technics and Time, vol. 3, 150). Within the actionable media ecologies of real-​time systems, digital texts may occupy different places than those conventional within the spacetimes of literate cultures. In the latter scenarios, one often engages with the cultural archive in places explicitly devoted to that purpose (libraries, museums, classrooms, offices, theaters, etc.); time spent in such spaces no doubt furnishes us with an expanding universe of referents that (re)shape how we experience events beyond these archival spaces. As Derrida famously suggested, the archive produces the event—​but we might add that, in the case of literate archives, this production occurs in deferred time. “This learned sense of disconnection,” which Brian Greenspan believes to be at play in our encounters with paper books and e-​books alike, “encourage[s]‌ mobile readers to decontextualize and ‘leave behind’ even more real-​world locations” (“New Place of Reading”). The iconic places and postures of close

82

8 2   • Act i o n a b l e   M e d i a

reading constitute inviting, temporary cocoons through which we can better commune with bound pages of text in relative seclusion from environmental noise. With actionable media projects, texts and audiovisuals are encountered amid the proximate present, often while we are doing something else (in addition to reading); hence, their cultural value, as well as the rhetorical principles informing their production and analysis, will differ radically from more pronounced traditions involving words, sounds, and images (be they oral, print, or digital) that have been crafted for the deferred time of conventional archival spaces.6 While mobile media multitasking divides attention in ways that appear to fray the conditions of close reading, I venture that actionable media ecologies may foster an evolutionary species of deep attention, whereby text and context cohabit one another. More typically, deep attention and close reading are presumed to be under threat by digital interfaces; even the most sympathetic commentators characterize the redemptive prospects of electronic textuality in terms of “hyper reading” and “machine reading” (Hayles, How We Think 69–​70). Assessed as such, digitally inflected modes of reading can add tactics to one’s analytical toolbox, but they are taken to cultivate breadth rather than depth. Here again I derive a dictum from Stiegler, who so routinely breaches the party lines of media theory and technology criticism. Amid the digitization of archives and attendant shifts in information consumption, Stiegler’s concerns about the fate of deep attention culminate in a prospect for critical invention: One must both rejoice and worry about such a state of affairs . . . because this new pharmacology . . . can lead to either the destruction of [literate] attention and the individuation resulting from this deep attention, which has been cultivated through the text since the beginning of the great civilizations—​or to the production of a new kind of deep attention, closely connected to new attentional forms. (56) These two scenarios map out neatly over the fault line that divides media ecologies emblemized by distracted driving and others that might be modeled off of GPS navigation. The former outcome (the destruction of deep attention in favor of multitasking, divided attention, etc.) follows from the fetishization of the distant-​now, while the latter outcome (a new kind of deep attention) may emerge when digital networks are designed to resonate with local action in real-​time—​when mobile media becomes actionable media. In bringing networked multimedia to bear on one’s surroundings, ubicomp platforms and

83

Interpreting Post-Desktop Practices  • 

83

actionable practices afford the potential to reinvent deep attention outside of the deferred spacetimes of traditional archives and textual forms. Actionable media producers may aim to author the built environment so as to stimulate deep attention in situ. Theorizing digital communication beyond the desktop, then, is not a matter of willy-​nilly content creation for post-​desktop interfaces. It is about analyzing and inventing actionable mediascapes that contribute to existence (rather than subsistence), citizenship (rather than consumerism), social practices (rather than proscribed usages), public authorship (rather than unilateral data flows), individuation (rather than standardization), and deep attention (rather than divided attention). Such an ambitious project, which this book can only hope to initiate, requires that we appropriate Weiser’s conceptual relays, so as to broaden the scope of ubicomp by envisaging its historical significance and cultural potential from vantage points unique to the arts and humanities. Stiegler’s concept of grammatization provides the first of these vantage points.

Notes 1. Farman’s notion of information landscapes cues mobile media scholars to consider how location-​aware apps may bypass the desktop metaphor and arrange digital data on a geospatial basis. He writes, “Landscapes, it can be said, have become information interfaces much like the graphical user interface of a computer screen. The landscape around us can serve as a type of interface where data of all types can reside, from the quotidian rankings of various restaurants to the mobile mapping of crisis zones after a major natural disaster” (43). 2. It would be rash, though, to say that this division is an absolute one in practice. I may well access a website on my smartphone that adds to my environmental awareness in that moment, and at any rate, a dialogical co-​influence always exists between my geographically situated experience and whatever media I  am accessing on the go, regardless of its relevance. The mobile Web can, with enough diligence on the part of users, be made to bear relevance to their present locale in spite of browser-​based designs. 3. Mobile communication research derives the notion of locative media from digital arts and urban informatics, where the term has been circulating since 2002. In general, the artists and urbanists who work on locative media are more interested in staging political interventions amid everyday spaces and less interested in studying popular mobile apps that utilize geospatial data. To get a further sense of how locative media differs in these fields, see the 2006 special issue of Leonardo Electronic Almanac devoted to the topic (Vol. 14, no. 3).

84

8 4   • Act i o n a b l e   M e d i a

4. The impending rise of driverless cars is, of course, riddled with unsolved technical problems and worrisome social implications. My speculations about the future of in-​ vehicle media experiences (and the potential for driverless cars to become venues for actionable media) are not meant as endorsements in favor of automated vehicles. At the time of writing, however, there is clear and overwhelming evidence that driverless cars are primed to become, despite valid objections, a significant part of the transportation ecosystem in high-​tech societies (likely as branded public services more so than individually owned goods). Hence, my critical interventions concern the range of media practices that may follow from the technology’s adoption. 5. Often summarized as “intellectual seeing,” theoria designates an ancient Greek practice that combined sight-​seeing with critical inquiry. For a comprehensive discussion of theoria and its modern legacy, see E. V. Walter’s book Placeways: A Theory of the Human Environment. 6. Much more is made of this distinction in ­chapter 4, where I argue that the differences defining these two computational research and development agendas extend much further back into the history of writing and archivization. Going beyond Weiser’s contrasts from desktops and his peculiar analogies with embedded texts (e.g., streets signs and candy wrappers), I posit that what we are witnessing in the rise of ubicomp is in part the revival of a previously marginalized archival modality—​what I will call “actionable archives” versus “deferred archives.”

85

3

FUTURES OF COMPUTING VIA HISTORIES O F W R I T I N G During [the] period that gave birth to the West, therefore, the question was to know what interpretation to give to that form of grammatization that was unfolding at that time . . . Today, this question remains intact. — ​B e r n a r d St i e g l e r , The Decadence of Industrial Democracies

Having taken stock of Mark Weiser’s manifestos and critical discourses surrounding post-​desktop practices, I  move to isolate, appropriate, and reinvigorate two of the conceptual wellsprings that underlie ubicomp’s rhetorical and technical invention. Weiser coaxed his lab colleagues to “start from the arts and humanities”; now that ubicomp has leaped out from research labs into mass circulation, it’s a good time to up the dosage. I want to remix the notion of ubicomp with further insights from the arts and humanities. My objective is to galvanize the cultural development of actionable mediascapes that foster public authorship and deep attention amid built environments. In this chapter, I take Weiser’s reliance on the history of writing and continental philosophy as inlets through which newer humanistic theories of media and communication may join the discourse and, ultimately, enable us to see current innovations in light of a broader logic that Weiser could not fully articulate. The roots of actionable media run deeper than digital technology. Weiser’s talk of candy wrappers and street signs only begins to broach the ancient impulse to lace everyday objects with text and to inscribe messages on the spot. While actionable media constitute only a burgeoning subset of post-​desktop use cases at present, continental inquiries on inscription and archivization supply a comparative method for tracing a continuum of actionable practices across history. Bernard Stiegler’s recent philosophical treatises on writing and technics offer a particularly generative counterpoint to Weiser’s project in this regard. Stiegler’s theory of “grammatization” locates writing at the epicenter of historical and contemporary

86

8 6   • Act i o n a b l e   M e d i a

technological, ontological, and social transformations. Grammatization also compels studies of (pre)alphabetic text and audiovisual media to prioritize questions concerning bodies, gestures, objects, and environments. With grammatization and ubicomp alike, there is a resounding effort to couple meaning and materiality, symbolic activity and local action. Rethinking ubicomp as a process of grammatization (and as a relatively untapped archival modality) better positions theorists, designers, and critics to understand the emerging forms that are orienting actionable media production. From this vantage point, we may discern more acutely the opportunities and challenges these actionable forms and practices raise for the future of digital communication in civic, artistic, and public intellectual contexts. Such investigations are precisely what insights gleaned from Stiegler’s work will enable us to pursue in ­chapters 4, 5, and 6. Following the invention of ubicomp and its scholarly reception, this turn to grammatization lays a methodological bridge for connecting nascent quandaries in actionable media to pertinent histories, theories, and traditions of writing, rhetoric, and aesthetics.

Into the Dustbin of Media Theory Prior to unpacking the idea of grammatization, I should note how Stiegler and Weiser’s parallel angles of attack distance their respective projects from enthymemes that undergird conversations about digital culture in many academic and popular circles. Like Weiser, Stiegler methodologically consults the history of writing as a privileged source of generative analogies with which to frame research agendas, create concepts, envision practices, and to cultivate an ethos for proposing and evaluating digital futures. Against the currents of leading scholarship on new media, theirs remains an unusual, aberrant stance. Normative descriptions of media evolution invoke a lineage on par with the following: first came writing; then photography, film, radio, and television; and now digital media. Writing seems antiquated within this progression—​ a distant ancestor whose legacy persists, but whose high seat has long since been taken by media presumed to be categorically different and more immediately akin to twenty-​first century advancements. One readily notices this sentiment at play in theses put forth by Vilém Flusser and Lev Manovich, for example, who each wrote off writing during the heyday of personal computing. In order to take seriously Weiser and Stiegler’s respective appeals to histories of writing, one must rescue the idea of writing from these dismissals. First, consider Flusser, who is quick to forecast the demise of writing in his 1987 book Does Writing Have a Future? He issues this prognosis on the

87

Futures of Computing via Histories of Writing  • 

87

basis of a fundamental distinction between writing and programming. For Flusser, writing is essentially defined by four attributes:  (1)  writing is the “linear alignment of signs”; (2) “written signs are, directly or indirectly, signs for ideas” (i.e., writing is secondary to thinking); (3) writing is “outwardly directed” toward one’s fellow human beings; and (4) writing is synonymous with historical consciousness (or, more exactly, “history is a function of writing”) (6–​8). Programming, by contrast, marks an ideographic gesture of “mathematical consciousness” that is directed toward machines and largely composed of “if-​then” propositions, which represent a purely functional or “value-​free” way of thinking (60–​61). This latter aspect decisively separates programming from writing in Flusser’s schema. Whereas written commandments, laws, and instructions address the Other in political or ethical terms—​in the form of mandates or arguments appealing to the readers’ sense of judgment—​computer programs (while informed by such genres) interpret data and execute commands (or fail to) without question, without negotiating between behaviors and values. Programming “change[s]‌the imperative propositions (‘thou shalt’) to functional if-​then propositions” and consequently promotes a “depoliticization” of behavior (Flusser 57). This epoch-​ level distinction would prove compelling if it could hold, but it fails in the face of two major observations. Software is rhetorical through and through. In the decades following Flusser’s argument, scholars across the digital field have constructed frameworks for identifying and critiquing the very elements of computation that Flusser alleged it did not possess. Chiefly, Mark Marino’s proposal for a new line of research called “Critical Code Studies” has gained traction among media critics and digital humanists, who have delivered many case analyses in service of Marino’s basic call: “to analyze and explicate code as a text, as a sign system with its own rhetoric, as verbal communication that possesses significance in excess of its functional utility” (“Critical Code Studies”). Even if code were to eclipse prose and become our lingua franca, we would still be dealing in argumentation, politicized statements, and the free play of interpretation. Programming is a form of writing best understood as an outgrowth from centuries of literacy (Vee). Ian Bogost’s Persuasive Games also demonstrates the persistence of classical rhetoric through compelling analyses of the persuasive appeals that underlie videogame play. Videogames—​the computer program par excellence in games studies scholarship—​become recast as an assembly of rules, functions, and processes that constitute a game’s worldview, inevitably assigning certain values to the human-​computer interactions a game makes possible. If-​then propositions amount to arguments whenever users perform

8

8 8   • Act i o n a b l e   M e d i a

or play out the functions that each game necessarily arranges and privileges according to a designed set of hierarchies, incentives, and affordances. In addition to being rhetorical in nature, digital interfaces and multimedia artifacts are thoroughly grammatological; that is, they can be aptly regarded as new gramme (the Greek work for “written marks”) within the context of histories and theories of writing. Doing so does not simply mean foregrounding the role of alphanumeric texts in digital environments to the neglect of audiovisual and procedural aspects, which Bogost takes to be a problematic tendency apparent in early composition scholarship on new media (“Rhetoric of Games”). From grammatological vantage points, as I will explicate below, the notion of writing encompasses audiovisual inscription, machinic automation, and genetic modification, extending well beyond the alphabet-​bound (artif )acts within which scholars may otherwise confine it. Novel HCI modalities flowering with ubicomp (e.g., speech-​to-​text software, image recognition, gesture-​based computing) are sometimes, in the spirit of Flusser, purported to leap past writing. These input mechanisms do make typing unnecessary, but writing still occurs whether users type or not. Words spoken to Amazon’s Alexa or Apple’s Siri, gestures directed toward Microsoft Kinect, are only processed to the degree they may be translated into codes that the software can read. Likewise, when Blippar’s visual search app performs feats of computer vision, all it’s really doing is composing keywords on the user’s behalf. Scanning everyday objects with a handheld or wearable device outsources alphanumeric production. The work of writing may be relocated, but it is not obsolesced. Electronic textuality exceeds typing and, in any case, a grammatological approach to digital communication presupposes that writing has always already been (extra)textual, (non)linear, (meta)linguistic, and (post)human. Manovich casts writing aside in a different way than Flusser and his heirs. Arguably the most influential monograph on new media to date, Manovich’s 2001 book The Language of New Media, presents striking affinities between cinema and computation. Cut-​and-​paste functions and the “aesthetic of compositing” fundamental to many digital authoring programs are shown to have conceptual ties to the avant-​garde editing practices of Sergei Eisenstein and Jean-​Luc Godard (151). Experimental films by Dziga Vertov and Peter Greenaway appear formative of the “database imagination” (239). Manovich briefly acknowledges histories of writing qua print as “rich sets of metaphors” for thinking about digital interfaces (particularly the notion of the page); however, turning quickly from print to film, he explicitly privileges cinema as a superior comparative lens for contextualizing design trends in virtual

89

Futures of Computing via Histories of Writing  • 

89

reality, computer games, and database aesthetics during the 1990s: “cinema is now becoming the cultural interface, a toolbox for all cultural communication, overtaking the printed word” (86). Serving as more than a metaphor, “cinema’s aesthetic strategies have become basic organizational principles of computer software” (86). New media inherits its language and its logic from cinema. And yet, as I alluded to in c­ hapter 1, the ontological claims implicit in Manovich’s genealogy (e.g., that new media is cinematic by nature) should be critically appreciated as historical observations that are incredibly resonant with—​but also limited to—​the personal computing era, or what Manovich terms “new media during its first decade” (8). His groundbreaking effort to connect work in HCI and media theory established a vital reciprocity between software and aesthetics, though it remains electronic boxes and virtual windows all the way down. In many ways, his book’s cinematic emphasis primed readers for a digital culture populated by graphic avatars traversing virtual landscapes such as in World of Warcraft or Second Life. Virtual reality was the holy grail during that “first decade,” and the most popular Internet genres then (e.g., chat rooms, online games, Web browsers) appeared to project a path toward total immersion. But even in the 1990s, as we have seen, a budding class of computer scientists and interface designers were already mapping escape routes away from the holodecks and “CAVEs” championed by cyberspace pioneers. Now in the midst of its third decade, new media simultaneously inherits and rebels against the ideals and conventions that drove its first decade. Cinema is no longer the default arbiter of new media histories, theories, and practices. Indeed, the invention and dissemination of ubicomp proceeds from a principled rejection of the virtual window. This is not to suggest that histories and theories of film have no place in discussions about post-​desktop futures; for instance, the cinematic notion of indexical recording—​film’s lauded capacity to capture what D.W. Griffith affectionately called “the wind in the trees”—​seems immensely relevant to scenarios wherein emerging digital genres (e.g., augmented reality, public interactive installations, audio walks) aggressively admit live, unscripted environmental processes into their textual, visual, and sonic frames. Moreover, anyone who studies the circulation of networked multimedia amid built environments will derive much insight and inspiration from a divergent vein of scholarship on film, television, and music exemplified by discipline-​expanding books like Anna McCarthy’s Ambient Television and Anahid Kassabian’s Ubiquitous Listening. Both authors—​in their respective analyses of site-​specific TV programming and background music in airports, bars, shopping malls, elevators, and subways stations—​bring

90

9 0   • Act i o n a b l e   M e d i a

into critical focus an array of actionable media practices that predate the popularization of ubicomp. These vanguard works deftly capture how audiovisual inscriptions, like printed signage before them, have subtly inhabited urban spaces and institutional settings in ways that guide, nudge, or critique local action. If cinema furnished the cultural logic of personal computing, it is also incumbent to note how the pending coalescence of computation and cities are extending the place and purpose of audiovisuals. Cinema is itself mutating in the wake of ubicomp—​the digital future that Weiser and others envisioned through the looking glass of writing. Hence, I move to probe what Manovich discarded, guided by a belief that the rising technocultural paradigm beckons attention to histories and theories of writing as much as personal computing, the early Web, and authoring software of the 1990s merited Manovich’s cinematic approach. As ubicomp interfaces outstrip the desktop metaphor, the interdisciplinary, multimodal study of writing stands to become more central to digital inquiry, interaction design, and critical media production.

Grammatology and Grammatization Reacting against the marginalization of writing apparent in much early work on new media, several theorists in rhetoric and media studies, including Richard Lanham and Lydia Liu, have issued calls for more expansive considerations of writing and its vitality today.1 These calls, as I see them, at once beg for a resurgence of grammatology, a (trans)discipline that professes the centrality of written marks and other gramme to the ontological leanings, cultural habitus, communication practices, and forms of social organization at play in various historical moments. In her discussion of grammatology, Liu references Stiegler’s assertion that modern informatics constitutes a kind of writing, only to then classify (if not dismiss) the entirety of his oeuvre as a vague philosophical gesture in route toward a more prescient task, which she summarizes as follows: “we need . . . a global and integrated concept of writing that is simultaneously historical and theoretical to guide us toward a richer knowledge of these extraordinary processes” (18). Actually, I cannot think of a better way to describe Stiegler’s project. My aim in explicating his theory of grammatization is, first of all, to show how such a “global and integrated concept of writing” has already ripened across Stiegler’s texts. I posit that Stiegler’s work may be regarded as the latest advance in the field of grammatology, which has most recently been taken up by media scholars (e.g., Matthew Kirschenbaum, Jeff Rice, Federica Frabetti) aiming to theorize emerging practices parallel to the history and theory of writing (broadly conceived to include virtually

91

Futures of Computing via Histories of Writing  • 

91

any act of technical inscription). Secondly, I intend to derive a methodology from Stiegler’s analyzes of writing systems and digital networks. This critical method—​which I term “tracing orthographic continuums” below—​will then serve in ­chapter 4 as a means to construct a long view of ubicomp in relation to expansive, contrasting traditions of archivization and cultural memory. To forecast:  theorizing ubicomp qua grammatization spotlights how emerging interfaces and actionable practices appear to be breaking away from a more predominate archival tradition that, in large part, has informed the evolution of discourse across print genres and personal computers alike. Consequently, the spread of ubicomp stands to cast alternative, historically marginalized archival tactics and rhetorical strategies into unprecedented cultural prominence. Conceptualizing digital communication beyond the desktop at once entails ancillary queries into counterhistories of writing. At one level, Stiegler’s Technics and Time series effectively revives central concerns held by a litany of grammatological thinkers during the 1960s to the 1980s: classicists and historians of writing (Gelb, Havelock, Goody), French philosophers and literati associated with Tel Quel (Derrida, Barthes, Kristeva), and North American media theorists (Ong, McLuhan, Ulmer). As such, Gregory Ulmer’s succinct account of grammatology’s emergence as a transdisciplinary research agenda provides an initial basis for outlining the nature of Stiegler’s intellectual contributions, which will set us on the path toward parsing out key insights that his concept of grammatization poses for our understanding of actionable mediascapes. According to Ulmer, grammatology developed in three phases—​historical, theoretical, and applied—​all of which remain in progress. First, the historical phase featured a variety of archeological and paleontological investigations into the evolution of writing systems. These historians attempted to gather evidence about the invention of writing in ancient civilizations. A chief contributor, Sir Godfrey Driver devised elaborate taxonomies for categorizing the world’s writing systems, almost as if taking inventory of different species of plants or animals. Racing to amass data surrounding the origins of particular writing systems, early historians of writing scarcely pondered the ontological significance of writing, nor did they question inherited assumptions about which activities and artifacts counted as writing. For this reason, Derrida—​the first theoretical grammatologist—​embarked on a “point-​by-​point repetition of the history of writing into a theory of writing” (Ulmer, Applied Grammatology 17). As he deconstructed the metaphysical opposition of speech and writing, Derrida assembled a collection of liminal examples, wherein nonphonetic systems like hieroglyphics functioned as emblems through which he

92

9 2   • Act i o n a b l e   M e d i a

theorized writing in general (i.e., “arche-​writing”), beyond the limits of phonocentric discourse. Stiegler’s project may be described as a complex variation of grammatology that, while centered on contemporary technics, routinely enacts a point-​ by-​point techno-​historicization of the theory of arche-​writing and the logic of supplementarity. In this sense, Stiegler negotiates historical and theoretical grammatology, which is also to say that he, like Ulmer, forges a wealth of connections between Anglo-​American and French grammatologists whose texts rarely cite one another. Moreover, Stiegler and Ulmer both appropriate and extend Derrida’s work, but each does so in a different way. Ulmer explores how Derrida’s writing process might serve as a stylistic model for developing “a mode of writing, and ultimately of pedagogical practice” to inform humanities research and teaching amid the rise of networked audiovisual media (Applied Grammatology 5). Throughout Applied Grammatology (and in several later books), Ulmer’s primary mission is to build a poetics extracted from the textual strategies of poststructuralist philosophers and the aesthetic tendencies of avant-​garde artists, many of whom, he argues, anticipated or suggested something fundamental about thought and expression in the age of digital media (i.e., “electracy”). Derrida’s theory of the signature, for instance, propels Ulmer’s invention of the mystory genre. Crucially, as Ulmer explicates Of Grammatology with unparalleled precision, his underlying interest in doing so typically relates to inventing new academic practices. After citing Derrida’s exhaustive redefinition of writing, Ulmer stipulates the analytical focal point through which his inquiry will apply the insights of this all-​embracing (trans)discipline: All these manifestations of writing, so visibly different, share an irreducible and invisible element—​the gramme . . . A grammatologist may be able to bring this range of materials together within a field of study, but my concern in this book is with grammatology’s own compositional practices. (Applied Grammatology 10, my emphasis) Over the two past decade, Stiegler has emerged to play the role of this nameless “grammatologist” against which Ulmer defines his own agenda. Stiegler contrives a distinctly hybrid historical-​theoretical grammatology from which we can learn and apply critical methodologies in addition to those Ulmer finds in Derrida’s prose. Stiegler, even more than Derrida, professes to see life in the world—​human becoming, historical change, social organization—​as the evolution and play of gramme. What atoms are for physicists, gramme are for Stiegler.

93

Futures of Computing via Histories of Writing  • 

93

The Technics of Writing In order to theorize ubicomp and actionable media as processes of grammatization, we must establish a working knowledge of what grammatization means. This is no casual task. The term’s boundaries seem infinite, and they are densely rooted in Stiegler’s first principles concerning techno-​human relations. One immediate scholarly benefit of his philosophy is that it seems to reconcile a dilemma that has hampered the humanistic study of technology ever since the inception of media history, electronic textuality, computers and writing, and digital communication. Those writing about technology in such fields have, at least until recently, had to preemptively guard themselves against fast and loose accusations of technological determinism.2 Stiegler’s work remaps the dividing lines upon which this habitual objection has been staked. He postulates that to study emerging media, writing systems, and communication technologies is to attend to the constitutive forces that condition the possibilities of human becoming, which are always shifting from epoch to epoch, especially in permanent innovation societies. Accordingly, his realignment of techno-​human relations puts writing, or, more precisely, the technics of writing at the forefront of cultural and ontological transformations.3 Informed by philosopher Gilbert Simondon, Stiegler routinely defines technics as “organized inorganic matter.” The term refers both to the history of fabricated objects (e.g., flint, hammers, pencils, computers) and to the domain of techne: the techniques and practices involved in making (something with) technology. Technics are more than merely a part of the environment humans inhabit. As a generative tissue of social becoming, technics constitute—​not determine—​the fabric our experiences on many level, from retention to anticipation, and from cultural history to genetics. Under this perspective, humans were not born into the world already equipped with mature cognitive capacities. These capacities developed over time in a transductive relationship with Neolithic technics, and they are still developing today hand in glove through our collective play with contemporary technics. Quite uniquely, Stiegler refuses to conserve any afterimages of a master-​ slave dialectic wherein humans still assert control over technologies through their habits of usage. In fact, he insists that the very question of techno-​ human relations must be posed differently: it is not a matter of asking whether humans control technology or whether technical evolution determines human evolution. Theories of co-​evolution, after all, profess nothing but a mutual determinism. Following Derrida (who was his teacher), Stiegler asserts that “the static oppositions of Western metaphysics must be replaced by dynamic

94

9 4   • Act i o n a b l e   M e d i a

compositions: one must think in terms, not of hierarchies or totalizing systems, but of processes.” (“Memory” 69). Beyond the dialogic ideal of coevolution, Stiegler circumvents persistent oppositions between humanity and technology. He starts by studying the constitutive processes that underwrite so-​called humans and technologies—​before considering them as separate, already-​constituted entities. These constitutive processes, in Stiegler’s lexicon, are called processes of grammatization. On the basis of Derridean grammatology, we can gain a partial sense of what grammatization signifies in Stiegler’s work. (But keep this in mind: the full weight of Stiegler’s theory registers only after taking into account his measured departure from several Derridean premises discussed in the following section.) Any reader of Stiegler’s texts will have no doubt encountered several shorthand definitions of grammatization, which tend to be more puzzling than illuminating. For example, Stiegler writes that grammatization “is the production and discretization of structures,” or “grammatization is the discretization of the continuity of movement” (Decadence 173). Making sense of the concept requires a liberally assembled collage of passages and examples taken from disparate nooks of his prolific output. First of all, grammatization is not the “production and discretization” of just any structure; the kind of structures it concerns are gramme, which, as noted above, derives from the ancient Greek word for written mark. Second, grammatization is a process term. Like augmentation, remediation, or convergence, grammatization theorizes a process allegedly orienting and informing new forms of media and technological innovation. In particular—​and this where I locate its prescience for ubicomp cultures—​grammatization pertains to processes by which a material, sensory, or symbolic flux becomes a gramme, which, broadly conceived, can include all manners of technical gestures that maintain their iterability and citationality apart from their point of origin. Grammatization, in other words, provides a comparative lens for analyzing how otherwise nondiscursive objects, bodies, or gestures become endowed with new capacities in the order of technical memory, mechanical reproduction, orthographic inscription, and networked communication. For Stiegler, the shift from cuneiform to phonetic symbols is a process of grammatization, the shift from hand-​tools to factory machines is a process of grammatization, and so is genetic engineering—​cells and organs become replicated and revised like a kind of alphabet. In each case, a continuous flux (e.g., speech, the body, the genome) becomes broken down into a system of discrete, signifying elements (e.g., alphabetic characters, mechanical systems, recombinant DNA sequences). And, in every case, the latter’s emergence always disrupts, transforms, or reconfigures the former. Interpretations of grammatization processes

95

Futures of Computing via Histories of Writing  • 

95

revolve, therefore, around two basic questions: What established flux gets broken down? And what new set of gramme emerges? Stiegler posits that the formalization of new gramme accrues through metonymical invention, whereby a part becomes detached from the whole of a continuous flux. This metonymical orientation contrasts markedly with Jay Bolter and Richard Grusin’s theory of remediation, which describes media evolution in terms of metaphorical representation of prior media (45). Furthermore, while remediation (akin to Henry Jenkins’s notion of convergence) identifies the reciprocal flow of forms, techniques, and content among various media technologies, grammatization foregrounds the breakdown of continuous flows and the sets of discrete elements that crystalize in the process. Any medium can only be distinguished as a medium to the extent that it develops a unique gramme or set of gramme. Whenever one medium represents another, the newer medium develops by means of breaking down the flow of information organized by the older medium and, in so doing, reconfigures that flow around new units of production and analysis. Remediation is only half of the story, or one way of telling it. Jason Farman turns to remediation in his explication of Weiser’s objectives for ubicomp (Mobile Interface Theory 7). Grammatization yields another story theorized through a different lens. To conceptualize ubicomp development as a process of grammatization, rather than a matter of remediation, is to locate its novelty and significance not merely in the strategies by which the emerging paradigm represents more established media technologies—​but in the unique set of cultural forms/​ gramme it founds through metonymical invention. Grammatization traces the breakdown of the desktop metaphor, not its extension. Consider a few examples of grammatization, from alphabetic writing to ubicomp. Alphabetic writing breaks down the flux of speech into a finite system of recognizable characters that are, on the one hand, iterable and modular, and on the other hand, capable of orthographic stability (Stiegler, “Memory” 70). Writing does not simply hold a mirror up to speech; systems of writing exceed their apparent “representation function” (Sanchez 85). Writing forges a new relationship to verbal language that effectively reorients the flow of speech as much as it represents it. The rise of a new set of gramme alters the basic conditions of oral communication in literate societies. Forms intrinsic to writing, such as the list or the paragraph, have informed the cultivation of speech patterns and habits of mind that were not widely practiced before the spread of handwritten manuscripts and, much later, printed pages (Goody 16–​18). The development of these literate interface forms, which Goody called “figures of the written word,” furnished the ancient Greeks with

96

9 6   • Act i o n a b l e   M e d i a

a new medium for education and the transmission of disciplinary knowledge. Eventually, the scholars and students of early manuscript cultures (an elite group, to be sure) could inscribe the word into relative permanence in a space apart from lived memory, and do it without emotional identification, regular recitation, or a harmonious and mimetic style—​the essential technics of oral poetry (Havelock 47). Sharing the burden of retention with alphabetic writing materials, they were able to move from acts of recitation to exercises in abstraction (Havelock 254). In this sense, gramme act as the generative nexuses between techne and episteme, marking the ebb and flow of basic forms through which memory, communication, and knowledge building tend to occur in a given society. Parallel transitions can be observed in the history of modern computing and its digital gramme. Prior to ubicomp, the development of the graphical user interface (GUI) in the 1970s—​so vital to the desktop’s subsequent popularity—​effectively broke down (i.e., grammatized) the command line interface into a new set of images, prescribed options, and standard protocols. Users could identify an option (e.g., save, quit, etc.) from a drop-​down menu and simply click on it. They no longer needed to remember (or cumbersomely retrieve) precise bits of arbitrary command language in order to perform even the most minute operations. The transition to GUIs mirrored the alphabetization of speech to a degree. Both cases established a new relation between hypomnesis (i.e., the technical exteriorization of memory) and anamnesis (i.e., the embodied act of remembering). Whereas speech and the command line require people to perform anamnesis, alphabetic writing and GUIs have language and commands structurally installed into them. To engage with alphabetic characters and desktop icons is to mobilize those signifying functions of phonetic language and computer code, respectively, which have already been rendered discrete, iterable, modular, and visible. One can signify via these entities—​these gramme—​without having to precisely recall, let  alone consciously engage with, the intricacies of pronunciation or programming. The concept of grammatization, furthermore, stipulates the necessity of thinking technologies and techniques together. And this is where Stiegler’s philosophy really starts to illuminate current ubicomp initiatives. As processes of grammatization break down an otherwise continuous flux, certain gestures or traces become detached from the initial continuity and (in)form a technology capable of performing specific techniques or functions independently of any supposed point of origin. In this sense, the technics of writing—​the technologies and techniques it makes manifest—​supply the basic operating logic that continues to orient industrial innovation, from nineteenth ​century

97

Futures of Computing via Histories of Writing  • 

97

factories to the latest advancements in wearable computing. Whereas Ulmer draws upon Derrida’s grammatology in order to craft a poetics and pedagogy of “picto-​ideo-​phonographic writing,” Stiegler stretches grammatological theory into another dimension, adapting it into a basis for understanding all manner of technical systems beyond those traditionally regarded as writing or communication technologies. This move is incredibly pertinent at a time when high tech societies are delving ever further into the Internet of things. Everyday objects are becoming “smart objects,” social practices are becoming digitally supplemented, and technics in every domain are becoming mnemotechnics (i.e., systems capable of capturing, storing, and sharing information). Not only does writing happen in audiovisual modes alongside alphabetic texts; increasingly, we experience the decoupling of the act of writing from human rhetors, such that a multitude of objects, machines, and buildings are capable of registering or documenting symbolic activities, as well as producing utterances in response to a rhetorical situation. Gartner, Inc. calculates that by 2020 the world will play host to 25 billion “connected things”: objects capable of recording, processing, and sharing digital data (“Gartner”). Adding to the broadened notion of writing qua multimodality, Stiegler motions toward another scope expansion as he prompts us to recognize the growing array of contemporary (artif )acts that now double as gramme. This suggestive aspect of Stiegler’s thought stems from his belief in a general interplay of gramme and gestures: “To separate the question of language, alphabet, photograph, cinematograph, audio recording, and so on from all the questions of gestures is a very big error” (“Cultural Technologies”). According to Stiegler, some of the most important processes of grammatization—​historically and especially with the rise of ubicomp and nanotechnology—​involve the becoming-​gramme of territories, bodies, and even cells. In addition to writing through discursive gestures (e.g., handwriting or typing), our apparently nondiscursive bodily gestures can—​by virtue of various machines—​produce writing of sorts, to the extent that such gestures are being broken down from the flow of muscular continuity, metonymically detached from our bodies, and made iterable in their absence. For instance, the act of hammering a nail can be performed by a discrete and replicable machinic assemblage, just as alphabetic letters on a keyboard may be pressed to record a speaker’s every word. In “grammatizing” the factory laborer’s gestures, industrialization precipitates proletarianization, or “the loss of know-​how” on the part of laborers (Stiegler, “Memory” 71). For Stiegler, industrialization thus constitutes a pivotal stage of grammatization, which becomes intensified by digitization as industrial resources and investments

98

9 8   • Act i o n a b l e   M e d i a

center on the programming of behavioral models engendered through the production and commoditization of audiovisual media. Whereas prior grammatization processes dealt primarily in the “sphere of language, of logos” throughout much of writing’s history,” industrialization “came to invest [in] the sphere of bodies,” and the age of mass media turned mechanization toward “apparatus-​dependent reproducibilities of the visible and the audible” (Stiegler, “Memory” 70). Defining all writing technologies as pharmakon, Stiegler warns that hyperindustrial investment in digital machines is contributing to a general proletarianization of the consumer’s existence, perhaps to an even more pervasive extent than the industrial investment in factory machines initiated a proletarianization of the worker’s labor. This general proletarianization of existence portends a loss of knowhow with regards to political participation, namely of “symbolic activities . . . which, since the Greeks, have been conceived as the rights and duties of the citizen” (Stiegler, Decadence 35). The best way to combat such erosion, according to Stiegler, is not through resistance—​it is delusional to think that humans could abstain from consumption or technics—​but by inventing cultural forms and rhetorical practices capable of sustaining civic symbolic activities outside the restricted economy of mass entertainment, corporate imperatives, and “pure and simple utility” (Decadence 35). In addition to a range of disconcerting social ramifications, the pervasive networks of gramme and gesture emerging with wearable devices, smart environments, and the Internet of things mark new rhetorical ecologies that introduce unusual and promising affordances for public discourse and cultural criticism (which I explore in ­chapters 5 and 6). In addition to more traditional mnemotechnics, grammatization qua ubicomp now engulfs all manner of objects and machines as computation regularly becomes installed into things and onto bodies. Digital text, audio, and video can be invented anytime-​anywhere, delivered on almost any surface, and be arranged to correspond with local action. Ultimately, Stiegler’s analysis of (hyper)industrial activities suggests an altogether different account of media evolution and techno-​human relations than the familiar notion of augmentation, which models all technologies as prosthetic enhancements. Lacing everyday life with computation is not (only) about augmenting physical entities with digital bells and whistles. Conceived as processes of grammatization, the organization of inorganic matter that technologies are results not from the extension or representation of an innate human capacity but from an appropriation, a writing, defined by its disjunction from the continuity of a flux or movement, human or otherwise. Once disjointed and inscribed autonomously as inorganic matter—​having become

9

Futures of Computing via Histories of Writing  • 

99

a figure of the written word, or even a figure of the industrial factory or a figure of the digital interface—​the movement becomes a gramme that embodies the logic of supplementarity.4 Theories rooted in augmentation cannot account for the autonomous iterability of technics and their originary, constitutive force within the development of cultural practices, disciplinary knowledge, social organization, and even the human brain. Rather than regard media as forms of re-​presentation that extend the presence of interiority—​and thus contrary to all media theorists who subscribe to Marshall McLuhan’s “media as the extensions of man” maxim (and Douglas Engelbart’s goal of “augmenting human intelligence”)—​Stiegler asserts that mnemotechnical systems come into being precisely by breaking down (meta)physical gestures into gramme. Comprised of gramme, technical ensembles like the typewriter or the factory machine operate in the ways they do because of their “capacity to be formed and to function as a reference that is empty or cut off from its referent” (Derrida, Limited Inc. 11). Just as Derrida posits that writing’s functioning is contingent upon its capacity to remain readable or iterable apart from the originary circumstances of its production, Stiegler defines industrial machines and computational programs in terms of what he calls “the indetermination of the machine’s functioning” (Technics and Time, Vol. 1, 168). The emphasis he puts on accidentality, dissemination, and indetermination also distinguishes Stiegler’s perspective on techno-​human relations from media theory’s machine-​fearing faction. Where other philosophers of technology (e.g., Friedrich Kittler, Martin Heidegger, etc.) see only imposing, inhumane mechanisms of precisely calculated automation, Stiegler regards computational machines as being radically open to and generative of multiple, accidental becomings qua the technics of writing.

Tracing Orthographic Continuums There are, however, two strands of grammatization at play in Stiegler’s texts. One unfolds in correspondence with Derridean grammatology (as outlined above), while the other emanates from Stiegler’s critique of Derrida in Disorientation, the second volume of Technics and Time. Both strands yield insights into the rhetorical dimensions of ubicomp cultures, though discerning the latter strand will push us to reevaluate some basic tenets of the former. Grammatization in its Derridean semblance has furnished three working premises:  (1)  communication technologies and the gramme that structure them are constitutive forces of cultural development; (2)  new gramme emerge through metonymical invention, whereby a (set of ) discrete

01

1 0 0   • Act i o n a b l e   M e d i a

element(s) becomes detached from a continuous flux and rendered iterable as such; and (3) contemporary processes of grammatization (e.g., ubicomp) cast the technics of writing into increasingly vast domains of life, spheres of practice, and areas of lived experience—​all technics are becoming mnemotechnics, all types of objects are becoming smart objects, multimedia is being superimposed over the lens of perception, and every surface may serve as a surface for digital inscription. Whereas the Derridean inflection of grammatization specifies relevant principles for understanding ubicomp initiatives, Stiegler’s swerve away from Derrida models a methodology that will prove essential for unearthing a pre-​digital genealogy of actionable media practices. It may be fruitful, moving forward, to regard Stiegler as a kind of postphenomenologist, with the emphatic caveat that he posits the technics of writing as an originary condition for the (re)shaping of structures of consciousness.5 In other words, where traditional phenomenologists set out to describe the lived experience of perception, memory, imagination, and the like; Stiegler maintains that all of these so-​called nontechnical, human capacities are (and have always been) constituted through the experimental organization of inorganic matter. Consciousness develops during a stage in the history of life “out of which emerges the possibility of making the gramme as such” (Stiegler, Technics and Time, Vol. 1, 138). History, for Stiegler, is most aptly characterized as the “history of the supplement,” in which each period becomes distinguished by its processes of grammatization (“Memory” 71). While archival/​ writing systems manifest the logic of supplementarity, each does so in different ways.6 The postphenomenological orientation of Stiegler’s work also serves as a cipher into his departure from Derrida—​a departure that may revitalize the relevancy of grammatology to contemporary research in rhetoric, communication, and media studies. Quite crucially, the qualities that set Stiegler’s grammatization apart from Derridean grammatology impart a more nuanced framework for situating ubicomp among contrasting traditions of cultural memory whose fault lines have not been properly discerned in prevailing theories of the archive. If Derrida’s logic of supplementarity furnishes the general insight that technics (techne) condition the possibility of theory (episteme), then Stiegler’s history of grammatization processes ventures that we can build knowledge of technics in general by comparing the emergence, materiality, and circulation of gramme across different media ecologies. Don Ihde’s description of postphenonomology’s objective in relation to technology is quite resonate with Stiegler’s methodology. Ihde claims that a postphenomenologist “finds a way

10

Futures of Computing via Histories of Writing  • 

101

to probe and analyze the role of technologies in social, personal, and cultural life that [s/​he] undertakes by concrete—​empirical—​studies of technologies in the plural” (23). It is precisely this posture toward technics that sparks tension between Stiegler and Derrida. For example, during their conversation in Echographies of Television, Derrida voices skepticism about Stiegler’s entire enterprise when he intimates that it may be impossible to study technics as an object of theoretical knowledge, given the premise—​which they both share—​that technicity conditions the very possibility of critical reflection. How can one presume to know the very conditions that make knowledge possible? Noting that Stiegler fails to respond to Derrida’s provocation in the Echographies interview, Ben Roberts claims that Stiegler’s writing also does not face up to this issue: “Technics and Time never really explicitly poses the question of how the theory of technics or a history of the supplement is possible” (9). Noting Roberts’s assertion, I contend that the long opening chapter of Technics and Time 2 could be read as Stiegler’s most poignant response to the aporia with which Derrida confronts him. Stiegler’s answer may not be as direct as Derridean scholars wish, but to answer a Derridean aporia with a succinct refutation would be futile. Indeed, Stiegler’s response takes the form of a Derridean deconstruction of Derrida’s own statements about writing in Of Grammatology. This rift between the two philosophers carries profound implications for the study of cultural memory and archival technologies, from clay tablets to tablet computers. Taking the Derridean critique of phonocentrism as a focal point for his own critique of Derrida, Stiegler outlines several investments evident in Derrida’s project that lead Derrida to neglect the specificity of linear writing. For instance, Stiegler finds in Derrida’s grammatological work an “essential tendency” to guard against the “always-​immanent return” of phonocentric impulses, which in turn creates the impression that it is “impossible to state any specificity that would not immediately claim superiority [for the phoneme over the gramme]” (Technics and Time, Vol. 2, 30). Stiegler then extracts what he calls the “heuristic principle” driving Derrida’s grammatology, which can be broken down into a pair of interdependent imperatives: 1) To establish “the question of arche-​writing . . . beyond the restricted concept of writing” 2) To “disturb and destabilize linear writing’s specificity” in an effort to “efface all metaphysical privileging accorded to speech, through the very writing that is truest to it [i.e., the phonocentric, restricted concept of writing as alphabetic writing]” (30)

012

1 0 2   • Act i o n a b l e   M e d i a

Derrida is eager to squint when he regards linear, phonetic writing. Distorting its specificity correlates with his intent to decenter phoneticization, as he associates the phoneme’s reign with the metaphysics of presence. He wants to unseat the metaphysics of presence by theorizing arche-​writing and the logic of supplementarity. Stiegler, by contrast, takes the logic of supplementarity as a starting point for understanding how episteme is variously manifested and reiterated by different techne throughout history. The very differences, in other words, that Derrida seems to efface in his tendency to see all traces, all life, in the image of différance. The omni-​concept of arche-​writing reveals a peculiar restriction of its own, alas, when Derrida repeatedly states that the trace, the gramme, and the written sign are necessarily iterable and citational apart from the circumstances of their initial production. As a result, Derrida’s grammatology exiles from its scope those gramme whose very legibility are premised upon attachments to a singular, formative context—​which is clearly the case with archival/​writing systems including cuneiform, urban signage, and augmented reality, as I argue in ­chapter 4. Phoneticization is more an object of inquiry in Stiegler’s texts and less a target for deconstruction. He commits himself to discerning the techn(ont) ological specificity of linear writing and the cultural impacts of its phoneticization. It is positioned as a process of grammatization studied alongside others in a manifold history of orthographic supplements, any of which may serve as an analogical lens for identifying new gramme today. To Stiegler’s mind, Derrida’s resolution to critique phonocentricism effectively overdetermines his interpretations of writing systems. Derrida equates alphabetic writing strictly with the linearity and presence of the voice, while he casts other forms like dreams (i.e., “psychical writing”) in an altogether different scene of writing “which puts words on stage without becoming subservient to them . . . a model of writing irreducible to speech which would include, like hieroglyphics, pictographic, ideogrammatic, and phonetic elements” (Writing and Difference 209). In Stiegler’s estimation, greater fluidity exists between these kinds of writing, between arche-​writing and “vulvar” systems of notation. Stiegler mobilizes the history of alphabetic writing in his philosophy by privileging its “orthographic character” over its phonetic dimension, insisting that “it is a matter of recording rather than the voice” (Technics and Time, Vol. 2, 13). This shift in emphasis links alphabetic writing with subsequent recording technologies, reframing it as a paradigmatic initiator of exact recording and not (only) as a phonetic monopoly over nonlinear writing and “the pluridimensional character of symbolic thought” (Ulmer, Applied Grammatology 8). By stressing the exact recording of the voice (rather than the

013

Futures of Computing via Histories of Writing  • 

103

exact recording of the voice), one can trace orthographic continuums between different processes of grammatization evident in early picto/​ideo-​graphic writing, alphabetic writing, industrial machines, photography, cinema, digital networks, and biotechnologies. Stiegler’s orthographic orientation toward the grammatization of phonemes is indicative of his general methodology, which he also brings to bear on contemporary authoring/​archival platforms such as YouTube. For example, in Stiegler’s essay “The Carnival of the New Screen,” YouTube functions as a node with which to establish an analogical network for tracing the emergence of techniques that are more enduring than any single brand of hardware or software. To study platforms like YouTube qua grammatization is to study a momentary “stabilization of technical evolution,” an iteration of arche-​writing (Stiegler, Technics and Time, Vol. 1, 31). As such, Stiegler treats YouTube as a cipher into the broader evolution of technocultural forms, that is, the metonymical invention of new gramme accruing through the breakdown of more established gramme. For our purposes, this essay on YouTube is notable because it concisely models grammatization as a critical method applicable to all manner of media practices. The eventual conclusions that Stiegler reaches about YouTube are interesting but not groundbreaking. He begins by wagering that the grammatization process culminating in YouTube is altering our relation to moving images as substantially as the invention of alphabetic characters changed early literate societies’ relation to spoken language. In establishing this parallel, Stiegler is not imploring us to see YouTube in the image of phonetic alphabets; rather, it is imperative that we keep the invention, circulation, and evolution of the alphabet in mind as we theorize the cultural development of digital video. For three reasons: 1) The invention of writing serves as a rich source of precedents to help us identify and prioritize tasks for critical invention relative to the rise of networked multimedia (which is presumably in its infancy) 2) This analogy provides a broader historical basis on which to assess present attitudes about digital communication practices 3) Oscillating between each of these moments in this orthographic continuum enables us to more readily grasp the most consequential novelties manifest in emerging mediascapes Stiegler thus extends the scope of his YouTube inquiry to include three early forms of writing: the advent of cuneiform in Mesopotamia, Egyptian hieroglyphics, and alphabetic writing in Greece. With these comparisons, he traces

014

1 0 4   • Act i o n a b l e   M e d i a

two distinct orthographic continuums linking, first, the scribal cultures of prealphabetic writing with the culture industries of broadcast media and, second, the dissemination of print-​based literacy with the development of cloud-​ based video software. The former is organized around “flows of programs” (which do not readily permit end-​users to manipulate content), whereas the latter forms emerge by breaking existing flows down into “stocks of footage”—​a new gramme that renders audiovisual media available for general interlocution (“The Carnival of the New Screen” 52). Moreover, the bottom-​ up production of metadata characteristic of video-​sharing websites displaces the cultural industries’ “calendar organization for program access,” which to a large extent set the schedule and viewership parameters for broadcast media (52). From the introduction of VCRs to today’s online video networks, flows of programs have incrementally become stocks of footage, as more people are afforded the capacity to break down, manipulate, annotate, and disseminate audiovisuals in increasingly substantial ways. Prior to the culmination of this process in YouTube, the audiovisual temporal flux was experienced largely as a medium without gramme, much like writing during the conditions of scribal culture. Productive and reflective participation was, in both cases, extremely restricted due to the complexity of codes, limited means of intervention, and unilateral modes of transmission. The technocultural development of a new gramme (e.g., alphabetic characters, stocks of footage) is what catalyzes the medium into becoming a diversified venue for everyday creativity, a site of common knowledge building, and an important vehicle of deliberative democracy.

Grammatization as a Comparative Method More generally, as the YouTube example indicates with its discussion of “flows of programs” and “stocks of footage,” grammatological analyses of particular digital platforms can produce concepts and insights that resonate beyond the hype cycles of those platforms. Grammatization foregrounds pervasive units of analysis and production over and across product variations. If we recall an implication of Stiegler’s work—​that gramme mark the intersection of techne and episteme—​then these fundamental units can be seen as cardinal categories that variously orient communication theories and practices. In other words, while technology does not determine rhetoric, we are inevitably positioned by the basic terms and forms (e.g., units of production, gramme, etc.) through which we conceptualize (artif )acts of writing and communication. The notion of orthography founds a broad basis for

015

Futures of Computing via Histories of Writing  • 

105

sketching continuums along which to compare the emergence of gramme among disparate kinds of media, which are otherwise routinely set against each other under the auspices of oppositions such as verbal vs. visual, symbolic vs. indexical, communications vs. manufacturing, vulgar vs. arche (in Derrida’s case), or analog vs. digital. Stiegler’s methodology foregrounds analogies and contrasts that accrue over long throughlines in time. This aspect, too, distinguishes it from media histories ventured on the basis of neatly divvied, ostensibly homogenous epochs that reduce societies and centuries to a simple dichotomy (e.g., orality vs. literacy, print vs. electronic). Tracing orthographic continuums makes for more varied, less predictable paths through the history of supplements: the grammatization of a labor’s gesture, a speaker’s voice, or a programmer’s code may prove analogous to the grammatization of anything else. Communication practices, therefore, are not grouped by defaulting to medium or chronology. Orthographic continuums span multiple eras of media. In every case, the initial recognition of semblance among various writing systems and rhetorical affordances ultimately doubles as a register in which to theorize unforeseen techno(ont)ological threads that link and differentiate various modalities of technical retention, inscription, and archivization. Such is the heuristic principle driving Stiegler’s work, which we can adopt in effort to discern the orthographic traditions that ubicomp builds on and breaks from. Starting from distinctions between ubicomp and personal computing—​and proceeding to work backwards into the history of writing and archivization—​what continuums might we assemble to lend insights into the present paradigm shift and the depth of the disruptions it may pose for cultural memory, rhetorical thinking, and public discourse? If post-​desktop interfaces give shape to new gramme, then what modes of communication, analysis, and expression do these new gramme appear to be making available for general interlocution? When pressed into the service of such questions, Stiegler’s critique of Derrida’s refusal to engage with the specificity of linear writing signals a further shortcoming in the latter’s conception of the archive. Derrida’s “essential tendency” to guard against the hierarchies he finds in phonocentric or “vulgar” notions of writing leads him, in turn, to establish hierarchies of his own, which effectively marginalize all inscriptions that are hermeneutically bound to a single context. Against the grain of Derridean logic, ubicomp propagates an archival modality—​a process of grammatization—​that eludes his so-​called “nuclear traits” of arche-​writing, as well as the kinds of temporal deferrals and spatial differings that characterize more traditional, privileged forms of the archive.

016

1 0 6   • Act i o n a b l e   M e d i a

While Stiegler’s method adheres to Derrida in its refusal to treat writing technologies in the tradition of augmentation (as secondary support systems or mere tools), Stiegler nevertheless endeavors to generate an episteme of techne relative to technologies in the plural—​directly challenging Derrida. Amid permanent innovation, one can approach changes in techne from the standpoint of Stiegler’s episteme (i.e., grammatization), in order to identify which processes of change actually disrupt established gramme and, in so doing, contribute to the emergence of new gramme. Such becomes the criteria by which to evaluate and describe the (lack of ) significance that an apparent advance in digital innovation bears for the history and future of writing and computing. If “the next big thing” does not markedly disrupt the history of supplementarity, or extend its logic to new domains, then it might not have a lasting impact on digital communication or media theory. Ubicomp does both. Now, let us put Stiegler’s methodology to new use by excavating a historical chasm that predates our present paradigm shift and illuminates the course of actionable media.

Notes 1. After praising The Language of New Media, Richard Lanham chides Manovich for his neglect of writing:  “Manovich fails to include one element that has remained surprisingly persistent on the digital screen: the written word. He just doesn’t consider it. We are to suppose that it doesn’t matter any more . . . But perhaps there is something to be said for text after all, something that would fit between writing it off and digging one’s heals in around the printed and bound book” (128). 2. For a detailed critique of technological determinism as a problematic disciplinary speech-​act, see the video recording of John Durham Peters’ presentation “Two Cheers for Technological Determinism” at the 2011 Conference on Media Histories. 3. As one of the earliest American readers of Stiegler, Mark Hansen has gone so far as to claim that the reach of Stiegler’s research “has the consequence of transforming cultural studies into technocultural studies” (“Realtime Synthesis”). And yet, because many of Stiegler’s major texts have only recently appeared in English, his work remains an underreferenced resource for contemporary media theory in the United States, and scholars in communication and rhetoric have published very little about him at the time of writing. 4. Stiegler’s theory of grammatization is, in many ways, profoundly indebted to Derrida’s early grammatological texts on writing. From a grammatological stance, writing—​understood as arche-​writing—​is always already “orphaned and separated” from any one hermeneutical authority, and it functions via spacing, iterability, citationality, dissemination, and difference—​which Derrida refers to collectively as

017

Futures of Computing via Histories of Writing  • 

107

“nuclear traits” of all writing (Limited Inc. 8). No signification, not even communication via spoken language, would be possible without functioning as writing in this sense—​without being capable of becoming orphaned and separated from a single signified referent or the horizon of a single author’s/​reader’s experience (Limited Inc. 9). For an extensive discussion of arche-​writing in Derrida’s work, see Raul Sanchez’s The Function of Theory in Composition Studies. 5. See c­hapter 1 of Don Ihde’s book Postphenomonology and Technoscience for a thorough explication of how postphenonomology departs from the traditional, Husserlian practice of phenomenology. 6. The issue of supplementarity has become a pivot point in commentators’ debates about Stiegler’s intellectual relationship with Derrida. Federica Frabetti suggests that Stiegler’s “fundamental point of departure from Derrida’s theory” is the former’s tendency to discuss supplementarity in terms of its historical specificity (15). Like Frabetti, Mark Hansen offers a positive assessment of Stiegler’s historical, materialist appropriation of Derrida’s generalization of writing. In Hansen’s view, “Stiegler relativizes what he calls the ‘quasi-​transcendental’ field of différance or arche-​writing in relation to the material infrastructure of its appearance and efficacy in the world at any given moment in time” (“Realtime Synthesis”). On the other hand, Derridean loyalists such as Geoffrey Bennington cite this crucial dimension of Stiegler’s work as its tragic flaw (see Bennington’s 1996 article “Emergencies”).

018

4

A THEORY OF TWO ARCHIVES, FROM CUNEIFORM TO AUGMENTED REALITY The technical structure of the archiving archive also determines the structure of the archivable content even in its very coming into existence and in its relationship to the future. — ​J a c q u e s D e r r i d a , Archive Fever

Chapters  1 and 2 highlighted major points of contrast distinguishing ubicomp interfaces from personal computing and the desktop metaphor. What I  wish to suggest now, by mobilizing Stiegler’s theory of grammatization from ­chapter 3, is that the differences defining these two computational research agendas and the digital communication practices they foster extend much further back into the history of writing and archivization. The cultural and rhetorical significance of ubicomp’s break from personal computing acquires more weight when we understand the desktop as the latest manifestation of a long-​prevalent orthographic continuum, a tradition I will discuss below under the name of the “deferred archive.” In mounting an alternative orthographic continuum, ubicomp and the actionable media practices to which it gives rise are not only meaningful for computational developments considered within the context of digital studies. Conceived most broadly, this burgeoning technocultural paradigm harbors elements from previously marginalized writing systems and media displays. In doing so, it sounds the echoes of another archival modality—​the “actionable archive”—​that has largely been suppressed and underdeveloped within digital and print cultures. The archival modalities and media ecologies emerging with ubicomp, therefore, presage a significant break not simply from the interface conventions of the last three decades, but also from those of the past 2,500 years.

019

A Theory of Two Archives  • 

109

The Deferred Archive Archives, taken in a broad sense of the historical record, are not just passive receptacles for storing accounts of past events. Writing, media technologies, and other “chains of differential marks” constitute the basis of all our temporal relations—​even the “now” of the present moment (Derrida, Limited Inc. 10). Orthographic mnemotechnologies preserve the exact records of a past we have not lived, and they also underwrite our own sense of futurity, by which we may anticipate the legacy of the events we do live and the circulation of the writing and media we generate (as its producer or its subject matter). We relate to our lived experience by the technical retention of a historic heritage and by the anticipation that the record of our own activities may be adopted by future generations to some extent, and that those retentions, in their circulation apart from us, may come to belong to what will have become traditions for people yet to come: our technical recordings will mark parts of the past they have not lived. Hence we are variously conditioned both by the historic modes of recording that broker our access to preserved cultural heritages and by the contemporary modes of recording by which we anticipate (and often participate in) the archiving of our own lives. Interface forms that constitute the composition of archives “are not only the manifestations of thought, invention, creativity, they also shape its future forms” (Goody 9). The archive produces the event, as Derrida proclaimed; however, archives come in radically different configurations. Upon this conception of the archive, I  propose to draw a new distinction between two archival traditions traceable across the history of grammatization, both of which have evolved—​in conjunction and contrast with one another—​from the earliest writing systems to contemporary initiatives orienting digital innovation. The desktop metaphor and the ubicomp paradigm are, respectively, the most recent expressions of these two traditions:  the deferred archive and the actionable archive. Within a deferred archive, texts occupy a codified space set apart from (different than) local circumstances, which the reader also withdraws from for the sake of textual immersion, ostensibly deferring her involvement with the extratextual lifeworld. Actionable archives house and present texts on the basis of overt links with local action; texts invented for and arranged according to this archival modality confront the reader in medias res, while she is doing something else in addition to reading (or while perceiving phenomena not enframed within

10

1 1 0   • Act i o n a b l e   M e d i a

an opaque page or screen), in wherever place that “something else” occurs. I will now trace processes of grammatization at play in each archival tradition. Both imply a respective orthographic continuum that yields key movements in their evolution qua various “figures of the written word” and other pivotal interface forms. The distinction between deferred archives and actionable ones does not correspond to any purported opposition between print and digital, or page and screen. Each tradition manifests itself at both ends of these spectrums. If ubicomp appears to recede from view momentarily here, it is only to better situate the post-​desktop milieu within a broader historical matrix. Theorizing the actionable archive will give way, in ­chapters 5 and 6, to more pointed theses about the criticism and creation of actionable media projects in contemporary rhetorical situations. Because deferred archives have been privileged throughout the development of literacy, I start by elaborating their characteristics and their decisive influence on conceptions of textuality and communication that remain conventional today.

Cuneiform and Context The transition from pictograms and cuneiform to the Greek alphabet occupies a formative role in Western histories of writing. Cuneiform’s demise and the alphabet’s rise may also serve, I venture, as a sort of primal scene that initiates an underlying rift between actionable archives and deferred archives, and it foreshadows why the later has been privileged ever since. (As we shall see, actionable media projects and ancient cuneiform script share important commonalities.) According to French historian Jean Bottero, an expert on Mesopotamian civilization, “true writing” did not begin until the Greeks added vowels to the Phoenician alphabet; prior to that, no methods of inscription had achieved a sufficient correspondence between spoken language and written marks, and so they did not properly count as writing (81). What interests me about Bottero’s otherwise typical phonocentric stance is his notion of a minimum threshold by which he assesses proto-​writing systems. He labels the Greek alphabet the first true writing system because it allows readers to make sense of a written text without having to know anything other than the language in which it has been written. Failing to meet this standard, cuneiform and pictograms rely on a more ephemeral set of allusions. They allude not (only) to language, but to singular events which one may or may not have remembered, let alone perceived. Cuneiform functions at best like a blueprint that triggers the rich remembrances of a building one has lived or worked in;

1

A Theory of Two Archives  • 

111

however, to those who’ve never seen that building, the blueprint offers mere approximations. It evokes hollow structures that can only be accurately filled in by personal memory. The reader of such prealphabetic texts must be already attuned to the particular context that each text conjures, for the text signifies metonymically in relation to that context. Bottero exclaims, “it is the context that permits the reader to choose the right reading: this is why in our foreign and removed eye a cuneiform text can never be simply read, it has to be deciphered” (86). This allusive attachment to a singular context, endemic to prealphabetic writing systems, limits their efficacy through time and space. Their hermeneutic threshold is steep: the cuneiform narrative remains fundamentally ambiguous and practically illegible in the eyes of all but those who happened to live through the described events. Consequently, the prealphabetic text can never stray too far from its author. Bottero thus contends that the quintessential purpose of most prealphabetic texts “could not have been more than a mnemonic device” (79). Looking over one’s own vowelless, gestural markings reminds the author of an event s/​he lived or an observation s/​he once made, so that s/​he might now speak to others about it. In order to achieve the minimum threshold definitive of so-​called true writing, texts had to be made iterable apart from any necessary attachments to a singular, idiosyncratic context. Texts needed to simulate, signify, or evoke such contexts by means of shared, relatively stable reference points. Meaning could no longer be premised exclusively upon one’s lived experience or personal memory, not if the writer wanted to give readers a fair chance of interpreting the text in his or her absence. By making it feasible to separate knowledge from the knower/​speaker in precisely this manner, alphabetic writing institutes an unheard of opposition between text and context precisely because, structurally speaking, alphabetic writing—​through its high fidelity to a shared spoken language—​preserves traces on more accessible hermeneutic grounds. Meaning accrues on the basis of collective signs that maintain their legibility apart from an individual’s living memory, without relying on allusions to ephemeral circumstances one had to live through. Invoking Derrida’s idea of iterability, Stiegler asserts that phonetic writing founds “a new relationship of statements to their context” (Technics and Time, Vol. 2, 55). Namely, an alphabetic text remains structurally consistent apart from the circumstances of its production and despite the absence of the writer or speaker presumed to already know it. As a process of grammatization, the Greek alphabet founds a system of abstract yet discrete characters that are iterable in textual space, without requiring any necessary relation to a particular place.

12

1 1 2   • Act i o n a b l e   M e d i a

Fixating on this capacity, we might say that alphabetic writing systems found contexts, and this contextuality—​constitutive of deferred time—​ comes to constitute the deferred archive. In support of this notion, Stiegler often summarizes the effects of alphabetization as a matter of “contextual wrenching.” He notes that, in cases of alphabetic writing, “textuality presents itself as a deferred time” and, we should add, within a differed space—​ the space of the page (or scroll, tablet, etc.) circulates in-​itself, remaining orthographically intact, in spite of environmental fluctuations (Technics and Time, Vol. 2, 57). Moreover, like the page, the iconic archival space of the library also acts as a temple for textual immersion that is at once a bordered fortress against contextual distractions. Différance is a condition that emanates from the materiality of contexts. Here, Stiegler’s work affords us the leverage to cast another measured critique of Derridean grammatology: différance is a nuclear trait not of writing in all its iterations, but of writing that is produced and encountered à la deferred archives. When viewed this way, Derrida’s “arche-​writing” bears a surprising and profound commonality with Bottero’s notion of “true writing.” While Derrida’s term clearly acknowledges a much wider range of writing systems than Bottero’s does, both thinkers define their respective theories of writing in opposition to textual forms that attach themselves to a particular lived context. In other words, if a text stakes its iterability, its very legibility, on ephemeral circumstances outside of the textual field, then that text is neither true writing nor arche-​writing in a strict sense. It gets cast into the margins of the margins of grammatology. From traditional phonocentric typologies of writing to their most renowned critiques, this assumption that textuality is essentially contextual appears to hold sway. If unchecked, it stands to muddle our conceptions of actionable archives as well as the actionable media practices distinct to ubicomp cultures. Consider the text/​context dynamic at play in Derrida’s often cited remark about a “nuclear trait” of writing:  “a written sign carries with it a force that breaks with its context, that is, with the collectivity of presences organizing the moment of its inscription” (Limited Inc. 9). Let us probe this premise by assessing how it applies to the everyday, embedded forms of texts and signage referenced throughout Weiser’s ubicomp manifestos. Granted, a text belonging to an actionable archive—​say, a stop sign cemented at a busy intersection—​can be extracted from its context and continue to signify something (else) elsewhere, even in the presence of new contexts unrelated to “the moment of its inscription.” (Such is the basic gesture of Andy Warhol’s Campbell’s Soup Cans from 1962; after Warhol reproduces the can’s

13

A Theory of Two Archives  • 

113

appearance and relocates it from grocery stores to a museum, the can still signifies and signifies anew in spite of, because of, the change in context.) Still, I counter: this is an act of translation. To relocate or “wrench” a stop sign from the built environment—​writing under erasure its initial inscription as an arbiter of local action at that intersection—​is to transpose its mode of signification (in addition to its location). If severed from the kairotic scene of its actionable inscription and put on exhibit inside a library or museum, the stop sign (or any once-​actionable sign) becomes subjugated to différance. That is, the relocated sign is now made to signify in the manner of a deferred archive—​local action is bracketed from hermeneutical attention—​in accordance with the chronos of deferred time and the differed space of the page, the book, the gallery, the screen, or whatever virtual window enframes its contextual presentation. And yet, just because an actionable sign can be transposed from its kairotic context and made into a context within a deferred archive does not mean, in turn, that all writing and reading is inherently contextual. Any text can be more or less translated into English, but not every text is written and read in English. A stop sign, like Duchamp’s urinal, can be taken from one context to the next, flipped upside down, and hung on a wall—​but, in the process, we have altered the archival modality underwriting its signification. If we purport that all writing obeys the logic of différance simply because it can be made to perform in this capacity, then we privilege one set of textual forms, one archival tradition, at the expense and suppression of others. We privilege deferred archives as “true archivization” and neglect actionable archives. Surely, however, just as any text can retain its iterability when detached from a particular context, any text may also be remade to premise its signification upon structural allusions to singular lived experiences or a specified field of local action occurring in one’s proximity. Among other possible examples, artist Teri Rueb’s project Itinerant (2005) is a case in point illustration (examined later in this chapter) of how the contexts of a deferred archive can be remade into actionable media. Indeed, the new set of gramme emerging with post-​desktop interfaces may support this latter mode of translation to an unprecedented degree.

Lists, White Cubes, and Virtual Windows Deferred archives exercise a further dimension of “contextual wrenching” traceable in the cognitive affordances associated with a continuum of interface forms, starting with the written list and evolving through the desktop

14

1 1 4   • Act i o n a b l e   M e d i a

GUI. In his account of writing and cognition in ancient societies, Jack Goody highlights the emergence of prealphabetic lists, insisting that the list was the most common and characteristic kind of writing during “the first fifteen hundred years of man’s [sic] documented history” (80). A mundane administrative genre, the proliferation of written lists nevertheless created the technocultural conditions necessary for the more storied intellectual achievements that fellow classicist Eric Havelock describes in his account of the “literate revolution” in ancient Greece. Speaking of the period, Goody claims that the list “is something that rarely occurs in oral discourse at all”; reading and writing lists demands (and thus cultivates) entirely different forms of cognition than speech does (80). For instance: The list relies on discontinuity rather than continuity . . . it encourages the ordering of the items by number, by initial sound, by category, etc. And the existence of boundaries, external and internal, brings greater visibility to categories, at the same time as making them more abstract. (Goody 81) As list writing brings conceptual categories to the forefront of one’s attention, Sumerian scribes begin to posit and recognize unspeakable relationships among the items they inventory. Lists allow scribes to reflect upon and rearrange items apart from particular circumstances—​and in accordance with more complex and systematic notions of order, resemblance, and differentiation. Note how Goody’s language prefigures Stiegler’s understanding of alphabetic grammatization when the former declares that lists “stand opposed to the continuity, the flux, the connectedness of the usual speech forms” (81, my emphasis). Lists, like many forms of writing, break down the flux of speech (i.e., spatialize its temporal flow), yet lists can be distinguished from other orthographic techniques. In contrast to written dialogues or treatises, for example, lists “substitute an arrangement in which concepts, verbal items, are separated not only from the wider context in which speech always, or almost always, takes place, but separated too from one another” (Goody 81). The list breaks down a writing surface into discrete columns, short-​circuiting the formation of sentences, in a manner most conducive to systematic classification via conceptual categories. Lists isolate individual symbols (often nouns) from the prepositional phrases that would, in the grammatical flux of sentences, ostensibly locate the nouns in time and space. A  noun in a list presumably undergoes an even greater degree of contextual wrenching than a noun in a sentence.

15

A Theory of Two Archives  • 

115

Deferred archives of many sorts organize and present their contents in the manner of a list: each new item becomes indexed as a thing-​in-​itself that may be grouped with other items deemed to belong to the same category or subcategory. Libraries and museums often institutionalize what Goody takes to be one of literacy’s latent tendencies (initiated by early list writing): “to freeze a contextual statement into a system of permanent oppositions, an outcome that may simplify reality for the observer but often at the expense of a real understanding of the actor’s frame of reference” (71–73). Hence after a phonetic text comes into being as an iterable structure of non-​presence—​ deferring and differing from any particular context through alphabetic grammatization—​that text enters into another set of second-​order oppositions imposed by the archive, which structures access to its holdings. Library books indexed in this familiar fashion are either Fiction or Nonfiction, British or American, and so on; their placement in relation to other books is categorically determined on the basis of schemas involving alphabetical order (by title or author), family resemblances, genus-​species, genre approximations, etc. Items occupying a deferred archive are delimited as contexts and then sorted out for access according to a meta-​contextual logic devised by the host institution. In other words, the contextual statement—​having already undergone an initial break from “the actor’s frame of reference,” as alphabetic writing structurally obsolesces cuneiform’s dependence on its readers’ acquaintance with a particular lived experience—​becomes subject to a further performance of contextuality when housed in a traditional library. The categories governing the arrangement of achieved items reflect their deferred status as items of an abstract list rather than as actants beholden to dynamic ebbs and flows of proximate activity. Twice severed, the signifying chains linking texts to their evoked contexts, media artifacts to particular material locales, must be reassembled anew in readers’ imaginations. We read and exchange interpretations on the basis of approximated representations of contexts, and no longer on the common grounds of settings we each co-​inhabit together here and now. We should cherish deferred archives no less for this, of course, but appreciating their value should not preclude us from also cultivating actionable archives and the alternative forms of deep attention, cultural memory, and public authorship they afford. The categorical impulses of the deferred archive extend far beyond libraries and written words. Similar modalities of indexing and exhibition impart terms and conditions structuring the reception of visual art and film. A triumph of modernism, the so-​called “white cube” has come to dominate the construction of gallery space in most art museums throughout the world. In

16

1 1 6   • Act i o n a b l e   M e d i a

his seminal essay on twentieth century gallery design, artist and critic Brian O’Doherty satirizes the mandates underlying the white cube: A gallery is constructed along laws as rigorous as those for building a medieval church. The outside world must not come in, so windows are usually sealed off. Walls are painted white  . .  .  The art is free, as the saying used to go, ‘to take on its own life.’ The discreet desk may be the only piece of furniture. In this context a standing ashtray becomes almost a sacred object, just as the firehose in a modern museum looks not like a firehose but an esthetic conundrum. Modernism’s transposition of perception from life to formal values is complete. (2, my emphasis) The white walls—​“untouched by time and its vicissitudes”—​aspire toward the atemporal, such that any work of art “exists in a kind of eternity of display . . . giv[ing] the gallery a limbolike status; one has to have already died to be there” (O’Doherty 2). Or, less dramatically stated, one has to defer involvement with ‘the outside world,’ which every aspect of the gallery’s design conspires to bracket off from the artworks. (The fact that museums are still among the most ill-​connected buildings for cellphone reception and wireless signal seems fitting in this sense, true to white cube form.) “The ideal gallery,” according to the ideology propagated by the white cube, “subtracts from the artwork all cues that interfere with the fact that it is ‘art’ ” (O’Doherty 2). Innovative curators and museum directors have experimented with approaches that challenge the white cube’s isolationism. Bartomeu Marí summarizes the spirit of some such efforts in his call for more porous exhibition tactics:  “the [gallery] doors must be flung open so that the work is executed alongside and in tandem with the other stimuli generated by the outside world” (15). Ultimately, though, in order for works to “blend [themselves] into the fabric of our more general sensory experience,” public artists (and public intellectuals) must enlarge their conception of a work to include the conditions of reception in which an audience views their images, reads their texts, or watches their films (Marí 15). The work is not only the context, readymade for circulation into whichever deferred archives agree to exhibit it in their privileged confines. Rather, under this broadened model (whose aesthetic precedents I examine in c­ hapter 6), artistry and authorship entail selecting scenes that are germane to the work’s production and reception, as well as cultivating rhetorical-​aesthetic techniques for grafting composed multimedia onto a particular field of local

17

A Theory of Two Archives  • 

117

action. This selection-​g rafting process is fundamental if not unique to actionable media. Meanwhile, despite alternatives modes of exhibition being tested in the contemporary art scene, cinema and television (not to mention “black box” theaters) have popularized deferred viewing experiences that vastly extend the ideology and reach of the white cube. Numerous film theorists argue that, upon sitting down to watch movies or television shows, viewers largely withdraw their attention from local action in order to resituate themselves “in a subjective elsewhere, in a virtual space, a virtual time” (Friedberg 178). Anne Friedberg groups cinema, television, and computer screens under the heading of the “virtual window”; she believes that these screens have together become “a component piece of architecture” comparable to traditional glass windows and the roles they have played in the built environment (138). Unlike glass windows, however, “a virtual window is reliant not on its transparency but on its opacity; its highly mediated modulation of light provides an aperture: not to a reality, but to a delimited virtuality” (Friedberg 138). It is precisely through this resolute adherence to “delimited virtuality”—​vistas of data which “remov[e]‌our experience of space, time, and the real to the plane of representation”—​that the desktop metaphor establishes a simplified, “user-​ friendly” digital interface (Friedberg 138). Adopting the virtual window as their axiom, desktop computers (and virtual reality headsets) climax the tradition of the deferred archive, already so deeply entrenched in the conventions associated with interface forms such as the list, page, book, library, white cube, cinematic theater, and televisual screen. In each case, the textual/​visual field privileges the content fixed within its own frame, all the while short-​circuiting attention to external variables that might otherwise occupy “the actor’s frame of reference”: the local action surrounding the reader at the moment of reading, the viewer at the moment of viewing, and the user at the moment of human-​computer interaction. Alas, desktop-​based “interactions” may seem an overstatement in retrospect. In the case of personal computing, keyboards and mice are the only registers of end-​ user activity. Desktops recognize me only insofar as I type, point, and click upon its windows, icons, and menus. The archetypal interface design framework known as “WIMP” (Windows, Icons, Menus, and Pointers) mimics the contextuality of alphabetic literacy, both in its strict adherence to differed (“virtual”) space and in its total embrace of categorical thinking. Interacting with a desktop is a process of filtering the breadth of one’s being down to fit the input categories proscribed by an operating system. The computer only processes actions that correspond to the categories it displays, within the guise

18

1 1 8   • Act i o n a b l e   M e d i a

of drop-​down menus (i.e., digital lists) and other WIMP elements. Desktop icons equate select images and words with select arrays of computer code; computational operations ensue on the basis of these structural couplings, much like alphabetic grammatization equates select written marks with select phonemes, so as to interlock the textual field with a shared spoken language. Amid both systems, the meaning of every trace is strictly contextualized (i.e., detached from local circumstances and made iterable through categorical correspondences). The desktop metaphor begets user-​friendliness through the execution of what J.C.R. Licklider called “predetermined programs.” That is, programs which require (and indeed permit) very minimal or very simple input from end-​users, whose pointing and clicking remains within the bounds of categorical procedures that have been thoroughly anticipated and likely suggested by the software’s developers. The recorded index of the PC user’s broader capacity for action (let alone those of other actants)—​the ratio of gestures that can be registered as gramme—​remains infinitesimal. This low fidelity between the interface’s enframed referents and the actor’s frame of reference is as tenuous for papyrus scrolls, manuscript pages, and print books as it is for desktop computer screens and WIMP-​designed mobile media. In every case, a restricted categorical economy—​persistently rooted in some notion of context—​parses out and elevates intelligible essences from the contingent accidents of the sensible. Archived as such, be they on page or screen, texts and multimedia proceed hermeneutically sealed in containers set apart, spatially differed and temporally deferred, from whatever local circumstances happen to surround them at a given moment.

The Actionable Archive Actionable archives depart from the contextual forms intrinsic to the constitution of differed space and deferred time, within which the most traditionally privileged artifacts of writing are preserved, organized, and displayed for readerly immersion. Because of this divergence from contextuality, actionable archives entail rhetorical-​aesthetic considerations that are downright antithetical to basic assumptions that orient textual analysis and communication practices across many common genres. For instance, a novel set in New York’s Central Park does not take place there, not in a material sense. Its paragraphs are not inscribed over the actual space of the park. Such a scenario seems almost farcical to imagine: the novel’s opening paragraph inscribed on that park bench, the next paragraph at the base of this fountain, another floating

19

A Theory of Two Archives  • 

119

above those trees. In the case of contexts, setting is metaphorically constructed apart from any necessary allusions to activities happening live in a singular proximity. While the material setting in which one reads any text inevitably exerts a minimal, probably subconscious influence on one’s interpretation, the text qua context retains its orthographic character in all weathers. A Tale of Two Cities is just as legible in Australia as it is in England or France; one need not read its pages amid the ruins of the Bastille in order to claim to have read it. The same could be said for a film shot in Rome, a painting of the Nile, or a Wikipedia entry about Shanghai. Of course, none of these claims could be made about texts or multimedia that inhabit an actionable archive. Early literary, artistic, and civic experiments with ubicomp platforms indicate a return to modes of production and interpretation initiated by prealphabetic pictograms and cuneiform inscriptions. But the nature of this return is easy to botch. To be clear, the orthographic continuum I posit here is not the same as the one Derrida emphasizes in his discussions of ancient hieroglyphs. Whereas Derrida appeals to prealphabetic systems to evoke a range of writing beyond phonocentrism—​before and after the letter, before and after the book—​the parallel I  am drawing strictly concerns the phenomenon whereby symbolic meaning, even basic legibility, is premised upon and interlocks with a particular lived experience: a past event retained within one’s living memory (in the case of cuneiform/​pictography), or a finite sphere of local action that is readily perceptible within the reader’s proximity (in the case of ubicomp). This latter distinction between cuneiform and ubicomp merits further elaboration, but consider first the metonymical logic that both of these actionable archives share. Lacking a coherent set of vowels, cuneiform marks out words in a partial manner. Likewise, ancient pictograms convey nouns and facts but generally do not enunciate an argument or story that would impart a definitive, authorial stance in regards to the scene they sketch. The consonant letters conjure up the whole word for the native speaker; the symbols incite a narrative in the mind of a confidant. In both cases, the written trace must be filled in or added to just to establish legibility, let alone meaning. In fact, because of this fundamental metonymy, the perception of legibility always already entails an act of interpretation. The figure of a cuneiform letter or a pictogram is only capable of prefiguring a proper signifier. The process of signification it sets into motion requires that—​before a signifier becomes linked to a signified—​ readers first decipher which word the consonants appear to be approximating (but do not wholly spell out). As Bottero aptly explains, the metonymical markings of prealphabetic systems (which he refuses to call writing) demand

210

1 2 0   • Act i o n a b l e   M e d i a

that readers decipher these essentially insufficient signifiers by identifying a point of resonance between the orthographic script and some related event or observation recalled from their own living memory. Only after this resonance is established does the prealphabetic trace become a signifier. “In order to understand [a cuneiform script] well one has to know already” (Bottero 86). As such, metonymical markings become recognized as signs only after they have signified. Ubicomp channels this metonymical mode of signification into the realm of interface design. Whereas the transition from cuneiform to the Greek alphabetic resulted in “contextual wrenching,” ubicomp works against the decoupling of written texts and local contexts. If phonetic alphabets facilitated the erasure of context from the scene of writing, ubicomp revives something of cuneiform’s reliance on experiential contingencies. Metaphor splinters under the pull of metonymy and the desire to signify in concert with the singularities of being. Hayden White defined this pair of rhetorical figures thusly: “Metaphor is essentially representational, metonymy is reductionist” (Metahistory 32). We flock to metaphor in the face of information overload, and we revel in metonymy when metaphor overdetermines meaning. Having made computers much less daunting, with its graphic representations of modern office artifacts and so on, the desktop metaphor has since come to engulf much earthly existence in the backlit glow of virtual windows unto elsewhere. As a process of grammatization, ubicomp may be aptly characterized as a matter of desktop metonymy. Its defining imperative is, as White said of metonymical historiography, “to reduce the whole to one of its parts” so that a now-​iterable part may relocate HCI into the periphery of the extracomputational activities one wishes to engage in (33). As a result, the scene of computing is no longer differed and deferred from the rest of the lifeworld, no longer enframed within the desktop’s simulations or filtered through its restricted economy of keyboard and mouse. Metonymical reductions are not reductive in the same way that metaphorical representations are; it is not a question of reducing the complexity of the tenor (binary code) to the familiarity of the vehicle (file, folder, trashcan). Rather, desktop metonymy performs a phenomenological reduction, in the sense that Maurice Merleau-​Ponty stipulated: as a return to the lifeworld that proceeds and exceeds “every scientific schematization” (Phenomenology of Perception ix). Though not an escape from schematization as such, metonymical reductions mean to bracket metaphorical representations in hopes of getting back “to the things themselves,” as Edmund Husserl put it. Ubicomp design whittles away at the desktop, carving it into

12

A Theory of Two Archives  • 

121

smaller parts for the sake of inciting actionable relationships between computational networks and proximate contexts. Versatile desktop applications, for instance, are broken down into task-​specific apps streamlined around a single action or exigence. Extracted from virtual windows, transposable layers of multimedia may be overlaid against the ground of users’ present surroundings rather than the backdrop of a Web browser. Just as windows, icon, menus, and pointers function as the gramme or interface forms constitutive of the desktop metaphor, a new set of design tropes and rhetorical figures is accruing amid the turn to desktop metonymy, and in c­ hapter 5 I will explicate these figures with consideration to the implications they bear for digital studies, rhetorical theory, and media criticism. First, it is crucial to specify the ways in which desktop metonymy reworks the modes of production and interpretation that it seems to inherit from the metonymical markings of pictography and cuneiform, as well as another important precursor (urban signage). Stiegler’s grammatological analysis of twentieth ​century audiovisual media and industrial machines advances two pertinent theoretical models, which we may develop further as a means to understand the actionable archives surrounding us today. Perhaps the greatest difference between pictography/​cuneiform and ubicomp involves their respective temporality. With regards to the former, “in order to have clear access to a pictographic inscription’s signification, one must have lived the event it purports to record” (Stiegler, Technics and Time, Vol. 2, 55). With ubicomp interfaces, as I have been suggesting, multimedia projects interlock with a finite sphere of local action that is readily perceptible within a specific proximity. The perceived form of an augmented reality work, for example, is indissociable from the set of presences which organize the moment of its reception. Desktop metonymy underwrites the potential for actionable media that is premised not upon allusions to one’s living memory but upon allusions to one’s live perception of proximate events as they unfold. Digital content qua actionable media signifies with/​in the here and now. Stiegler’s notion of “real-​ time information events” will serve as an inroad to this temporal dynamic. In addition, his spatial term “techno-​geographic milieus” (borrowed from Gilbert Simondon) brings to attention an uncanny mode of production clearly at play in media ecologies emerging through desktop metonymy. Produced and arranged on a techno-​geographic basis, contemporary actionable archives instantiate a logic of invention and circulation that bears little in common with the categorical procedurality that structures deferred archives and the contexts they house. To give a sense of the rhetorical-​aesthetic charge these two concepts carry, I will supplement my explications with succinct

21

1 2 2   • Act i o n a b l e   M e d i a

analyses of Teri Rueb’s Itinerant—​a digital literary work that manifests actionable media tactics.

Real-​Time Information Events Changing at a retrospectively slow rate until the nineteenth century, writing and communication technologies—​which have traditionally been at the heart of educational and cultural institutions—​are now thoroughly under the sway of industrialization. The high-​tech companies that Stiegler calls “international programming industries” have “ubiquitously installed themselves in our daily life and have re-​defined calendarity [and cardinality]” (Technics and Time, Vol. 3, 146). For Stiegler, calendarity and cardinality “constitute the primordial interlinking elements of societies” (134). This pair of terms signifies the collective techniques by which we orient or navigate through time and space, as embodied by a monthly calendar and the cardinal directions on a compass, for instance. “Today,” Stiegler claims, “cardinality and calendarity have been deeply disturbed”; this disruption is the source of the “disorientation” he associates with contemporary technics (134). Disorientation is brought about by the industrial production of real-​time, which Stiegler regards as “perhaps the fundamental trait of contemporary technology” (Technics and Time, Vol. 2, 63). Real-​time enables archival modalities that may outstep spatiotemporal orientation techniques and cognitive postures that have been cultivated on the basis of deferral. Stiegler insists, “Techniques are needed to help us navigate, no longer through past experience handed down by history, but through the real-​time of information events that occur on this planet, by the hundreds of millions, with every second that passes” (“Our Ailing Educational Institutions,” my emphasis). We know how to think within deferred archives; now we must also learn how to think critically amid the real-​time of actionable archives. But what exactly does “real-​time” mean in Stiegler’s writing? His usage gets at something other than the more familiar connotations. In computing scenarios, real-​time is typically evoked as a descriptor of systems that respond to input in milliseconds or less. It applies to software that enable instantaneous manipulation of computational objects, or to “real-​time search engines” that monitor and aggregate the latest activity (i.e., up to the second) happening across various social networks and blogging websites. Stiegler’s broadened conception of real-​time, while not unrelated to phenomena of this sort, unfolds across an orthographic continuum from photography to live broadcasting to GPS navigation. Beyond its status as a touted feature in

213

A Theory of Two Archives  • 

123

recent software, Stiegler intimates that real-​time may incite a subtle revolution in the phenomenology of memory and perception. Stiegler’s profound concern over the emergence of real-​time stems from his engagement with Husserl’s schema concerning the phenomenon of retention in time-​consciousness. In fact, Stiegler’s definition of real-​time and its ontological stakes are difficult to articulate without Husserl’s vocabulary, which Stiegler adopts and adds to. Husserl postulates two categories of retention: primary and secondary. Primary retentions are synonymous with “immediate” perception and are regarded as properties of whatever phenomena are present to consciousness right now (Stiegler, Technics and Time, Vol. 3, 14). Stiegler takes special interest in Husserl’s phenomenological description of listening to a melody. As one hears a melody, in Husserl’s example, the echo of a prior note is retained as a residue against which each succeeding note is heard. This cumulative resonance is what distinguishes melody from sheer noise. Grounded resolutely in perception, primary retentions are never imaginary; they are always presented to consciousness by an object or flux that is present (16). Secondary retentions are activated by intentional or involuntary recall. Consciousness projects to itself the properties of a past-​present phenomenon that has been stored in one’s memory. Because Husserl concerns himself only with lived experience, he willfully neglects technics. He does not address any retentions that accrue outside of human perception and personal memory—​retentions which Stiegler believes play a constitutive role in the activities of consciousness (21). Stiegler thus introduces a third term—​“tertiary retention”—​to theorize the interplay of recording technologies and so-​called immediate perception and living memory. When we listen to a melody that has been recorded (and view a photograph or film), the separation between perception and imagination central to Husserl’s framework proves false. In such cases, the melody remains exactly the same, yet one inevitably has “two different musical experiences” (21). The retained phenomena of a past-​present experience (distinct from the “just-​past”) supply consciousness with criteria that (re)orients perception qua primary retention, which is always a process of selection due to retentional finitude. In contrast to Husserl’s schema, Stiegler derives his fluid theory of perception by paying attention to technics: “the intervention of the imagination at the heart of perception, is only made obvious by tertiary retentions—​by a phonogram [in this case]” (18). Furthermore, twentieth c​ entury recording technologies and audiovisual media seed another mode of temporal experience, which, in turn, anticipates the perceptual conditions unique to contemporary actionable archives.

214

1 2 4   • Act i o n a b l e   M e d i a

For Stiegler, the historical progression from photography to film to television marks the emergence of real-​time. The real-​time of contemporary technics, beginning with photography, can be distinguished from the deferred time of orthographic writing. In the contexts of alphabetic literacy, people can easily differentiate between their perception of live events and their engagement with forms of tertiary retention; granted, one’s engagement with any tertiary artifact can shape the criteria (i.e. secondary retention) that directs his or her live perceptions (i.e. primary retention). Nevertheless, this impact occurs over time—​it is deferred—​because the event “precedes its input into a system, and this input precedes its dissemination—​its reception” (Stiegler, Technics and Time, Vol. 3, 134). Stiegler traces the dawning of real-​time to what he calls (following Roland Barthes) the “reality-​effect” of photography. That the photograph, at least in theory, manifests a “conjunction of the past and of reality” makes for a unique synchronicity: “The instant of the capture coincides with the instance of that which is captured” (Stiegler, Technics and Time, Vol. 2, 16). Film then transforms the photographic image into a temporal flux, so as to render audiovisual objects that correspond to the stream of human consciousness—​such is Stiegler’s explanation for the immense global appeal of cinema, television, and digital video (Technics and Time, Vol. 3, 9–​12). This genealogy of real-​time reaches its crescendo in the live feeds of televisual news broadcasting. Live television incorporates the consciousness-​like audiovisual flux of cinema and amplifies photography’s reality-​effect, such that the instant of projection also syncs up (more or less) with the instant of recording. Live footage is promoted and archived as an event before the broadcast actually takes place. After singling out live broadcasts of the first moon landing as a prototypical real-​time information event, Stiegler delivers the key line: “If a distinction between primary and tertiary memories remains possible  .  .  .  here it has nonetheless become absolutely formal and empty” (Technics and Time, Vol. 2, 242). Hence, the real-​time of live broadcasting scrambles Husserl’s schema of retentions even further, only to be outdone by current actionable media practices. The real-​time information events of ubicomp signal a general synchronization of live perception and technical memory, of primary and tertiary retentions. The “here and now” of live perception becomes, also at once, a gaze into the already-​there of digital archives. Real-​time interfaces do not constitute a departure from technological memory, but rather the arrival of a new paradigm in memory’s exteriorization (qua actionable modalities of archivization). With smartphones, tablets, and especially headmounted displays, the enmeshing of primary and tertiary retention intensifies to unprecedented

215

A Theory of Two Archives  • 

125

degrees of reach and consequence. Once accustomed to GPS devices, for instance, drivers tend to make navigational decisions on the basis of real-​time information events (i.e., audiovisual cues that correspond to the scene before them), and not primarily on the basis of their own historical consciousness or living memory. Transportation scenarios are just the humble beginnings, foreshadowing the more gradual impact real-​time practices may have on perception, search, and deliberation in various other domains. Here again, real-​time signifies not only a system’s response-​ability to its users, but also the condition whereby primary and tertiary retentions synchronize with each other and with proximate fields of local action. Real-​time is the temporal rhythm constituted by this triangulation:  lived perception, networked multimedia, and local action all interact at the same instant without overtaking each other. If we emphasize this reciprocal triangulation as a basic condition of actionable media ecologies, then urban signage arguably makes for a more appropriate precursor to the real-​time information events emerging with ubicomp than do the recording practices assembled in Stiegler’s orthographic continuum (e.g., photograph, film, television). Urban signage—​prevalent in most suburbs and small towns, too—​is not necessarily indexical and its “instant of capture” (or production) typically precedes its dissemination and reception. It is altogether another characteristic that qualifies street signs and candy wrappers as real-​time information events: their content pertains to the scene of action/​decision to which they are attached; we read them as part and parcel of the proximate setting, often to inform our present activities. Live perception takes cues from tertiary retentions that have been arranged to orient local action. According to historian David Henkin, the quintessential forms of urban texts (signs, posters, handbills, etc.) came into their own in New York City during the antebellum period. And yet, in spite of their centrality to modern life, genres of “city reading” remain largely unaccounted for in period studies of the era and many others since. Acknowledging the “persistently powerful image of the private reader,” Henkin argues that this “familiar model  .  .  .  locates [reading] indoors rather than outdoors and in seclusion rather than in the company of strangers” (6). Historians of reading and writing have commonly set their scope around deferred archives and contexts such as the novel, when in fact, “books of prose fiction accounted for but a small fraction of the reading matter of everyday life” (6). Henkin’s period-​specific observations are poignant cases in point indicative of the long-​standing neglect of actionable archives. His work also helps us understand why such attitudes have held sway, especially in the humanities (and the digital humanities, too).

216

1 2 6   • Act i o n a b l e   M e d i a

When pressed enough, few would deny the claim that urban signage plays an important role in engendering and sustaining the collective rhythms of everyday city life. Objections more likely arise at the insistence that signage constitutes a worthwhile object for textual analysis—​it’s not true writing, it’s not serious reading. Perhaps its status seems inferior not because of the same failings Botterro spots in cuneiform, but because the subject matter of urban signage is generally banal, superficial, utterly conventional, and meaningless compared to the contextual genres more readily deemed worthwhile in the arts and humanities. Collectively, the signage one encounters around any given city speaks to an infinitesimal faction of the breadth of topics and depth of expertise one finds in deferred archives such as a library. Henkin’s study elucidates the reason for this great disparity. Fodder for city reading came into being, in large part, as a manifestation of commercial interests and corporate power. Henkin writes, “The most prominent and imposing words in the Manhattan cityscape inscribed claims of private property and reinforced the spectacle of competitive commerce” (40). Eventually, on the heels of messaging conventions established by urban advertising, municipal entities installed robust systems of street signs and instructional/​regulatory signage designed to solve utilitarian problems and promote polite behavior in public settings. If collected and catalogued into its own library, the texts and audiovisual media occupying any city’s streets (public art aside) would almost invariable be tagged “commercial” or “municipal.” Imagine how sparse the shelves would be for “literature,” “history,” “science,” and other cornerstone subject areas that comprise a conventional library’s holdings. The fact that the capability to position texts and multimedia in their most kairotic environments has long been restricted to commercial advertising and municipal messaging should be understood in light of its material causes. Actionable print documents (even graffiti inscriptions) necessarily occupy real estate. Real estate is a scarce resource and conceived by law as private property. The public-​facing façade of a store or office building remains under the authority of those who own it. Public infrastructure is open to the public, but it has not been open for public authorship. The actionable achieves of ubicomp, on the other hand, position digital content for kairotic reception amid all of these environments, often without intervening in the material makeup of a place. The fragmentary and superimpositional character of actionable media allows writers to circumvent the authorities that have heretofore restricted the authorship of urban signage and, indeed, actionable archives of all sorts. On account of this disruption, deferred archives need no longer be the default destination and venue for digital cultural production. Suddenly,

217

A Theory of Two Archives  • 

127

for the first time, actionable archives may become a repository for all kinds of knowledge and forms of expression. The built environment now doubles as an archive, canvas, and networked forum for general interlocution. Where there has been only commercial signage and municipal messaging, there may soon be street-​libraries replete with actionable media. Even during the heyday of personal computing, several artists and writers hacked portable electronic devices into working prototypes for actionable storytelling. Teri Rueb’s project Itinerant (2005), for example, “invites people to take a walk through Boston Common and surrounding neighborhoods to experience an interactive sound work that reframes Mary Shelley’s Frankenstein” (“On Itinerant”). The “reader” carries a handheld media player and GPS device as she walks around the park. Whenever she traverses one of the project’s narrative territories, a designated audio file plays: either a specific except from Frankenstein or a para-​text Rueb has written about one of the novel’s minor characters. Of particular interest, for our purposes, is the fact that Rueb takes as her starting point a context qua deferred archive (i.e., Shelley’s classic novel). Itinerant refashions fragments of the novel into an actionable narrative, transforming selected passages into real-​time information events by transposing them onto urban locales. Instead of zeroing in on the page or the screen, the audience conducts their (aural) reading of the story as they attune themselves to their embodied experience of dynamic material surroundings. Consequently, the narrative becomes “shaped by all the complex indeterminacy of the participant’s agency and place within an actual and discursive milieu of social, cultural, physical, and natural forces” (Rueb, “Shifting Subjects” 132). The project encourages its texts to be read with (not in spite of ) the variability of local action perceptible in one’s proximity. On a more specific level, Itinerant experiments with real-​time in its ambition to foster a hybrid stream of consciousness enmeshed from the technical recording of its narrative and the live perceptions of readers. A literary cousin to GPS navigation, the project hinges upon audio cues that resonate with or complicate the present scene through which readers pass. According to Rueb, “The [reader’s] own point of view as a walker in the city becomes conflated with the wandering characters whose voices fill her head as she walks” (“On Itinerant”). These basic aesthetic conditions, which underlie each reader’s encounter with Itinerant, provide a sufficient basis for appreciating some stark differences that distinguish actionable narratives from the contextual orientation of deferred storytelling. Here, point of view is not a prerogative of the author/​narrator. The first-​person narrator addresses a first-​person reader whose observations in the here and now maintain a generative, perhaps

218

1 2 8   • Act i o n a b l e   M e d i a

reversible figure/​ground relationship with the composed fiction. In contrast to the opaque, differed space of a (web)page, actionable narratives—​like actionable arguments—​unfold amid and alongside whatever readers happen upon on their way. Rueb composes the work with approximations of readers’ potential experiences of Boston Common at the forefront of her authorial projections. The world of the narrative acquiesces with (and brushes against) the material flows of the lifeworld.

Techno-​Geographic Milieus Composed around real-​time information events, Rueb’s Itinerant project can be regarded as an artistic parallel for another one of Stiegler’s theoretical models. In addition to constructing point of view at the interstice of recorded sound and live events, Rueb’s narrative—​its geospatial organization—​adopts the landscape as its structuring principle. Audio clips are anchored within finite fields of local action. Selecting a scene is like choosing a group of words: both carry contingent semiotic networks that exceed prediction. Within the permeable real-​time frame of the work, unsuspecting pedestrians, straying birds, and the wind in the trees may variously inflect the scene’s signifying chain. Words and things intermingle. Electronic utterances are condemned to meaning-​with the environment. The writing space has become, in Stiegler’s lexicon, a techno-​geographic milieu. Geographical flows are no longer represented, captured, or bracketed from attention qua deferred archives; rather, they become axiomatic to the work’s rhetorical-​aesthetic effects. For Stiegler, contemporary techno-​geographic milieus harbor a process of grammatization that is becoming elemental to digital cultures. He charts its emergence along an orthographic continuum analogizing twentieth century industrial machines and twenty-​first century geoinformational networks. The idea of a techno-​geographical milieu, however, begins with Gilbert Simondon. By revisiting the characteristic examples that link his work with Stiegler’s, we can acquire some additional terms helpful for conceptualizing the techno-​ geographic modes of production, arrangement, and circulation at play in the actionable archives of ubicomp. Simondon, a student of Merleau-​Ponty’s during the 1950s, was one of the first French philosophers to assert the cultural and ontological import of modern technology. His influential doctoral dissertation, On the Mode of Existence of Technical Objects, presents elaborate descriptions of machines integral to industrial production throughout the twentieth century: steam turbines, automobile engines, boilers, electron tubes, power plants, etc.

219

A Theory of Two Archives  • 

129

These meticulous accounts all lead to philosophical reflections. Simondon treats technical objects as if they constitute an important class of beings. The rise of industrial machines ushers a new era of techno-​human relations; henceforth, “human reality resides in machines as human actions [become] fixed and crystallized in functioning structures” (Simondon 13). Simondon introduces the term techno-​geographic milieu to designate an especially unique kind of machine. Generally put, these machines interlock intensely with their surrounding environments. They assign a multitude of technical functions to certain geographical flows, which they have been designed to accommodate. Such machines are distinct from older windmills and watermills in that they link wind, water, sunlight, and so on with electrical networks. Simondon’s primary example of a techno-​ geographic milieu is the Guimbal turbine and its role in wave power stations. The Guimbal turbine stands as a technical ensemble “constituted by [the] oil and water in motion within and around [it]” (50). The turbine, reminiscent of cuneiform in this regard, stakes its very functionality on the presence and movement of contextual variables, namely seawater, without which it does not operate. In frequent references to Simondon’s analysis of the Guimbal turbine, Stiegler summarizes the ways in which seawater becomes a “polyfunctional element” in the production process: [T]‌he [techno-​geographic] milieu structurally and functionally associates the energies and natural elements composing this milieu, such that nature becomes a function of the technical system. This is the case of the Guimbal turbine, which assigns to saltwater (the natural element) a triple technical function: to furnish energy, to cool the structure of the turbine, and to catalyze the waterproofing of the stages. (Technics and Time, Vol. 3, 83) This passage may call to mind Heidegger’s concern that modern technology renders nature into a standing reserve of energy, reducible to a reservoir of resources available for (and subservient to) human use. And so, by turning saltwater or other ecosystems into polyfunctional elements, do techno-​ geographic machines necessarily reinforce instrumental approaches to nature? No, not if we adopt Simondon and Stiegler’s general views about how machines operate. What’s more, understanding industrial machines in this manner prefigures the techno-​geographic orientation of augmented reality applications.

310

1 3 0   • Act i o n a b l e   M e d i a

Both Simondon and Stiegler share Heidegger’s contempt for a world reducible to calculation and utility; so much so, in fact, that Simondon and Stiegler constantly refuse to discuss technics as a mere tool or instrument. Whereas Heidegger associates modern technics largely with a technocratic and inauthentic way of dwelling, Simondon locates the ultimate power of machines in their “margin for indetermination . . . a margin that allows for the machine’s sensitivity to outside information” (13). In the case of techno-​ geographic milieus, then, what is most significant—​at least in terms of Simondon and Stiegler’s emphasis—​is the fact that this machinic margin of indetermination syncs with and becomes sensitive to proximate geographical flows, which retain some degree of autonomy and accidentality even as they perform specific functions assigned by the technical system. As such, we might say, “the two worlds act on one another” (Simondon 46). The ideal techno-​geographic system works in relative harmony with established ecological rhythms, without indulging in the sort of unilateral extractions that characterize unsustainable energy practices. Machines that cultivate techno-​ geographic milieus do not enframe nature in a captive sense; rather, they graft onto existing geographical flows as constitutive forces, admitting ecosystems into the site of production processes by which we make (sense of ) things. Stiegler extrapolates from Simondon’s writing on techno-​geographic milieus as he expands the concept to account for ways in which digital networks have begun to converge with geographical flows. According to Stiegler, many current technological and industrial transformations are related to “the fact that, with digital technology, and particularly networked technology, the technical milieu is becoming . . . a techno-​geographic milieu” (“Technics, Media, Teleology” 334). Throughout Technics and Time 3, Stiegler probes what he calls “the digitization of territories” now occurring with the spread of geoinformation. By “geoinformation,” Stiegler means to indicate all manners of digital simulations and reproductions of earthly locations, as well as all mobile devices that operate in accord with georeferenced data. Akin to the Guimbal turbine, geoinformation assigns a “technical navigation function” to territory (Technics and Time, Vol. 3, 138). Territories effectively enter into geoinformatic systems as a generative rhetorical force, playing a heuristic role as “a vehicle for action and information in a three-​way [transductive] relationship involving man, the machine and the world” (Simondon 68). Through digitization qua geoinformation, territories become a constitutive element in the essential technicity of perception, memory, and decision making—​ and not merely a standing reserve to be seen, retained, and acted upon by an already constituted subject.

13

A Theory of Two Archives  • 

131

We may regard the phenomenon of geoinformation in terms of a broader strand of technical evolution giving rise to a “geo-​graphical” gaze, in which perception and cognition fuse with the writing (graph) of the Earth (geo). Attending to both senses of the phrase, the writing of the Earth signifies the act of making inscriptions upon the Earth as a (digital) writing surface, as well as the modalities through which the Earth’s geographical flows become translated (i.e., sensed and actuated) into digital traces, or at least affect the arrangement and/​or delivery of data. No longer restricted to turbines and power plants, techno-​geographic modes of operating have contributed to a definitive class of software, which many citizens of digital cultures engage with on a daily basis. On the heels of GPS navigation and Pokémon Go, developers caught up in the spirit of desktop metonymy are racing to distill browser-​based experiences into their most actionable essences, such that the retained bits may be animated by techno-​geographic bonds occurring in the here and now as each user lives it. In the case of post-​desktop social networks, smartphone apps such as Highlight (now defunct) and Blippar’s “Face Profiles” (launched in 2017) both assign a polyrhetorical role to human geography. Each app’s margin of indetermination grafts onto users’ bodies, linking their status updates and online bios to their actual faces. In contrast to the Foursquare app, Highlight recorded and shared user information on an ambient basis. Rather than “checking in” to a location, Highlight users’ projected their activity at all times. Likewise, if a curious bystander wanted to know more about a fellow Blippar user, s/​he could open the app and scan the person’s face to access a self-​designed augmented reality profile, as depicted in Figure 4.1. While Highlight’s launch predated the release of this Blippar feature by only four years, the former was derided by press for its apparent threats to individual privacy and the latter app, which entails more or less the same outcomes, has been warmly reviewed by tech journalists who tout Face Profiles as a boost to personal expression. The disconcerting rapidity with which facial recognition practices seem to have become normalized may be critically apprehended as a symptom of collective desires, even social expectations, for every material to conduce to digital interaction—​like the toddlers who habitually try to zoom in and swipe over printed photographs. What we observe in such cases is an actionable archive that brokers access to traces (user profiles) arranged on a techno-​geographic basis by live, local flows (bodies in motion). “The device user,” as Stiegler foresaw, “becomes a datum circulating in a ‘data stream’: electronic data physically localized” (Technics and Time, Vol. 3, 138).

312

1 3 2   • Act i o n a b l e   M e d i a

Figure 4.1  Screen capture demonstrating Blippar Augmented Reality Face Profiles. © Blippar, 2017.

Having translated moving bodies into networked gramme, facial recognition apps aim to layer social network functionality into face-​to-​face encounters. Blippar privileges and presents information according to proximity, as if it were a library shelf that processually changed its contents relative to live events taking place around the user. Of course, once one’s face becomes a rhetorical node in various techno-​geographic networks, a politics of free rein, grassroots actionable media production may be tougher to stomach. Fair use of another’s photographic image has become a justified and normal practice in cultural criticism, but extending the fair use doctrine a person’s face—​such that people may digitally pin critiques and parodies to your forehead—​seems an altogether different and dicey matter. As more surfaces become a surface for digital inscription, the question of which surfaces should remain off limits for public authorship will merit much needed debates. While Stiegler’s turn to techno-​geography foregrounds the polyrhetorical role of human bodies in contemporary geoinformatics, his work deals less explicitly with a wider array of posthuman entities that are becoming gramme through the rise of actionable media. In spite of his ongoing demonstration of humanity’s essential technicity, Stiegler has not yet formulated a term that concisely sidesteps the anthropocentric connotations his work critiques at length. When technologists design interfaces that function more in sync with human action, they often assign computational functions to nonhuman variables.

31

A Theory of Two Archives  • 

133

In ubicomp-​saturated settings, practically any body or thing (e.g., bodies in motion, inorganic matter, atmospheric effects) can impact the computational operations that regulate the production, organization, and circulation of multimedia content. Already, we see experimental projects whereby animals get equipped with wearable computers, which gather data and produce digital visualizations. In such cases, animals’ movements transmit (or at least become translated into) semiotic values, and the digital traces they “produce” carry a potential to influence social action. For Fish Communication (2005), an early project of this sort, researchers attached waterproof sensors to fish in order to create an interface that visually projected the fishes’ movements up toward the water’s surface; upon realizing this, some bystanders tossed food in the water, toward the real-​time traces of the fishes (Rod). Subsequent research projects across the American West, including the Wyoming Migration Initiative, have similarly employed wearable technologies to better identify barriers to seasonal migrations, in hopes of alleviating those barriers. As a corollary to Stiegler, Bruno Latour’s theory of action offers a way to account for the ontological flattening inherent in (and extensible beyond) the techno-​geographic operations of actionable media. Actions are never solely the work of a human actor, according to Latour. Action is a fundamentally a relational capacity engendered through contingent chains of associations among humans and nonhumans (“Technology is Society Made Durable” 109). Latour’s concept of an “actant” denotes any contributing source of action, and he applies the term liberally to humans, animals, weather, technologies, raw materials, fabricated artifacts, etc. Latour refuses to attribute a heightened or privileged degree of agency to any one species of actants. Each and every kind of actant may, at a given moment, make a pivotal difference that affects the actant-​network (or “actor-​network”) with which it associates. Reflecting on Latour’s actant-​network model in the context of political philosophy, Jane Bennett insists that we need to more acutely consider the vibrancy of material flows and nonhuman entities in our assessment of social issues and public policies. Like Bennett, Laurie Gries draws on Latour’s discussion of actants in order to think beyond the humancentricity bound up in conventional theories of communicative agency. Coining the notion of “rhetorical actancy,” Gries ventures that “humans and other actants work together to construct rhetoric” and that “discourse is a vital, material force.” (“Agential Matters” 87–​88). Both Bennett and Gries pinpoint scenarios in which human rhetors and an audience’s interpretations of their discourse are influenced by encounters with material entities, and Bennett celebrates texts that “direct sensory, linguistic, and imaginative attention toward a material vitality” (19).

314

1 3 4   • Act i o n a b l e   M e d i a

In actionable media ecologies, as my analyses of numerous projects in ­chapter 5 will illustrate, rhetorical and political activity becomes distributed or flattened in yet a further sense. Actionable texts not only draw attention to vibrant matter; (non)human material flows occupy the textual field, produce traces, and (re)structure the work. The actant-​network doubles as a networked gramme. Hence, in addition to the grammatization of human gestures and human bodies, and in addition to the grammatization of geographical data (qua the digitization of territories), actionable media assign polyrhetorical roles to proximate arrays of nonhuman actants that constitute chains of association in which communicative activity is necessarily situated. Put another way, actionable media arrange and deliver digital content in concert with the posthuman writing of local actant-​networks. While more emphatically posthuman examples come up in the next chapter, we can glimpse the techno-​geographic dimension of actionable media at play in Rueb’s aesthetic. Reflecting on Itinerant and her other soundscapes, Rueb claims, “the narrative and its structure and meaning emerge from and are dependent on the interaction of participant, place, time, and social context” (“Shifting Subjects” 130). When multimedia projects become techno-​ geographic, readers and writers come to interact with texts, images, and sounds in new ways. Non-​discursive actions—​walking, in the case of Rueb’s projects—​become a source of structural agency for readers, while nearby actants exhibit agency by assuming a polyaesthetic function as a constitutive narrative vehicle. Moreover, like the Guimbal turbine, actionable narratives craft themselves around a margin of indetermination, heightening the reader’s “sensitivity to outside information.” Consider how Rueb envisions the acts of reading that her project conditions: [E]‌ach participant becomes an “actor” or “character” uniquely written into the narrative, but ever unable to fully enter it as in a traditional narrative or through “suspension of disbelief.” Furthermore, the participants exist as characters that are never fully knowable or accountable by the author, as in traditional texts. Yet this variable condition is heightened as the context in which the participant experiences the work constantly changes within a range of parameters determined only by the limitless space-​time of the participant, the city, and her journey through it. (“On Itinerant,” my emphasis) Alongside Rueb’s soundtrack, one can imagine the landscape of Boston Common playing a triple narrative role: (1) to geospatially anchor the

315

A Theory of Two Archives  • 

135

reception of each audio clip; (2) to furnish the narrative with dynamic sensory stimuli; and (3) to complicate or inflect the assertions and themes proffered in the audio clips. In sum, Rueb’s piece introduces some of the spatiotemporal leaps that rhetors, critics, and designers take when they work outside of deferred archives. From Itinerant, we will move to examine in the remaining chapters a wide range of recent projects that reveal further challenges and considerations intrinsic to the analysis and creation of actionable media across various genres.

From Actionable Archives to Actionable Media That artists are pioneering much of the innovative work in actionable media should come as no surprise, especially when we recall Marshall McLuhan’s observation that “it’s always been the artist who perceives the alterations in man [sic] caused by a new medium, who recognizes that the future is the present, and uses his [or her] work to prepare the ground for it” (“The Playboy Interview”). As we move back and forth between deferred archives and actionable ones, exigencies characteristic to the latter beckon us to conceive of media production/​dissemination in techno-​geographic terms and to regard moments of reception as real-​time information events. Since 2010, the ubicomp authoring systems that Rueb and other artists had to fabricate themselves in previous decades have become readymade and as approachable as Web 2.0 mainstays like WordPress, Wikipedia, and YouTube. As a result of this increased accessibility, a wider array of writers and communicators may create projects designed to circulate on actionable media platforms, whose social adoption can only be expected to increase when more robust wearable devices reach a mass market and digital public installations spread across more cities. If artists were the only ones bold enough to experiment with actionable narratives in the decades before smartphones, recent years have seen the launch of trendy, headline-​ grabbing startups such as Asunder that deploy iBeacon and Bluetooth technology in the service of place-​ based storytelling. Students now compose augmented reality projects in addition to research papers. The street-​libraries to come have already opened their doors, but such technical victories give way to pressing challenges in the areas of technocultural analysis, media criticism, and critical media production. New gramme raise new problems for theory and practice. A broad understanding of actionable archives, their linage and the place of ubicomp therein, is necessary but not sufficient.

316

1 3 6   • Act i o n a b l e   M e d i a

While grammatization provides a theoretical-​historical framework for discerning this watershed transformation in archival modalities, the pragmatic discourse on rhetorical figures, spanning classical and contemporary rhetoric, offers ideational scaffolding more suitable for specifying post-​desktop interface forms as basic units of analysis and communication available for general interlocution. Grammatologists drift among the epochs. Rhetoricians deal in situations and decisions from which individual projects are forged. The present chapter cast a bird’s-​eye view of ubicomp by situating it among contrasting traditions of archivization. Having distinguished the actionable archive and some of its variations over time, we move now into more pointed questions: what new cultural forms (i.e., rhetorical figures) appear to be taking hold in contemporary actionable media ecologies? In other words, what shapes are networked multimedia assuming in the wake of desktop metonymy and how do these nascent rhetorical figures stand to reconfigure (artif ) acts of digital authorship, cultural memory, deep attention, and public intellectual discourse?

317

5

FORMS OF ACTIONABLE MEDIA In the new millennium, and notably after 2005, as a citizen of more or less massively networked information societies, one has already been interacting enough beyond command-​lines, menus, desktops, and GUIs to have realized that another set of models is operative, and that there is at this point an obvious need to pursue analyses and critical evaluations of these models. — ​U l r i k E k m a n , “Interaction Designs for Ubicomp Cultures”

From speech to writing to photography, every new medium for technocultural expression inspires rhetorical invention in the figurative imagination. The figurative imagination cuts across all modes of cultural production, examining communication and design practices in order to theorize forms, patterns, and strategies that garner suasive power or hold an aesthetic charge. Chapter 4 ruminated on conflicting figures of thought at work in current strands of digital innovation and long-​standing orthographic continuums. Metaphor and metonymy were suitable for the purpose there, which was to understand the post-​desktop turn on the basis of its family resemblances and fundamental differences from prior iterations of the archive. Now that we have elaborated some theoretical models for distinguishing ubicomp platforms as actionable archives, we are positioned to discern more specific conventions, genres, and figures accruing in the wake of those broader historical considerations. Here, attention turns from the grammatization processes and figurative postures of archival traditions toward a closer study of contemporary projects that are proving formative of ubicomp cultures and emerging actionable practices. Let us progress, then, from an analysis rooted in established tropes toward the critical invention of rhetorical figures yet to be articulated from an arts and humanities standpoint. The need to conceptualize new media forms consonant with ubicomp interfaces has been signaled by leading thinkers in a variety of fields, including digital humanities, interaction design, software studies, and media aesthetics. From Mark Weiser’s manifestos

318

1 3 8   • Act i o n a b l e   M e d i a

to recent transdisciplinary research on digital cultures, a lacuna resounds in technologists’ and humanists’ calls to one another, still unanswered. Recall that Weiser’s preliminary concern was to create hardware beyond the desktop; his Xerox PARC team invented new device categories—​“tabs, pads, and boards”—​that prefigured current smartphones and tablets. On the level of software, Weiser’s vision lacked a comparable set of forms. Meanwhile, ever since the 1980s rise of personal computers, interface designers have been structuring user experiences around the WIMP framework: Windows, Icons, Menus, and Pointers. WIMP design undergirds the virtual reality of the desktop metaphor that Weiser so vehemently critiqued. In the absence of ubicomp-​oriented frameworks, many mobile use cases remediate these design elements from the PC era. Modes of interacting with desktops have thus become regular ways of behaving anytime-​anywhere. Networked multimedia has become mobile, but all too often it is not yet actionable. The WIMP framework tends to rei(g)n in post-​desktop practices. This chapter builds a generative matrix between interface theory, rhetoric’s figurative discourse, and novel digital works in order to propose answers to the following questions. Which figurative elements apparent in current interfaces might constitute the ubicomp equivalent of WIMP? That is, what are the rhetorical figures and design tropes of actionable media? How might we begin to theorize their heuristic roles in the production and analysis of actionable media projects, as well as the disruptive implications these nascent forms pose for commonplaces in digital studies and rhetorical/​ design thinking? In what follows, I demonstrate how a new set of rhetorical figures—​ which I will outline under the acronym “ATLAS”—​may help orient critical engagements with actionable media projects and the unique communication spaces they engender. (The ATLAS framework brings to the fore these five elements: Apps, Tags, Layers, Actuators, and Sensors.) Moreover, just as figures like metaphor and metonymy have proven useful to poeticians and poets, ATLAS offers genre footholds for critical media producers creating actionable content, as well as a basic rubric for humanistic designers making decisions about end-​user interactivity. Rhetorical figures have been ideational devices for writers, artists, and designers working across a variety of media, digital and otherwise, throughout history.1 At the same time, while many creative practices have been spurred by classical figures of speech, theorists also coin new figures to account for major changes in media technology and cultural production. I lean on this lineage of figurative invention in my formulation of ATLAS. Once identified and elaborated, these nascent

319

Forms of Actionable Media  • 

139

figures, like the WIMP schema before them, will serve as critical-​creative categories meant to inform case analyses of digital artifacts and (in c­ hapter 6) the development of rhetorical-​aesthetic principles for making multimedia. This shift from WIMP to ATLAS—​consistent with earlier distinctions between PCs and ubicomp, deferred archives and actionable ones—​bears paradigmatic weight. Hence, the compelling notion that new media must be studied and composed in terms of interface (Brooke 25–26) needs to be unpacked through rhetorical explorations into more particular interface units that are forming heterogeneously across increasingly diverse mediascapes.

Beyond “Post-​WIMP” Discourse Latent connections in the writings of technologists, rhetoricians, and designers have left a placeholder for figurative invention in the age of ubicomp. Around the turn of the century, computer scientists such as Andries van Dam employed the term “post-​WIMP” to call for design principles and interface innovations fit for a post-​desktop world. Post-​WIMP discourse, like many “post-​” constructions, rallied around repudiating an established order. No unified recognition of a clear alternative was touted from the onset. Indeed, the WIMP framework is hard to dislodge, and poking holes in it is no easy task. Windows, icons, menus, and pointers remain the basic building blocks for constructing and navigating an array of digital environments. The software in which media critics locate cultural logics, the websites that rhetoricians claim are challenging disciplinary assumptions, the authoring programs at the foundation of digital humanities research and multimodal pedagogies—​so many of these objects of study and tools for production were built by designers working within the WIMP schema. As such, WIMP interfaces created the enabling conditions for digital activity since the 1980s, helping computation become a mass medium for communication, governance, education, commerce, and so on. En route to new schemas, some useful insights can be gleaned by comparing the WIMP framework with the design logic gestured at in post-​WIMP discourse. Quintilian’s conception of rhetorical figures provides an apt lens for examining this rift; he identified the figurative imagination to be at work whenever language was “poetically or rhetorically altered from the simple and obvious method of expression” (9.1, 13). Over the past three decades, the breakthroughs of the desktop metaphor have become commonplaces, and WIMP interface design is now “the simple and obvious method of expression” that post-​WIMP technologists set out to alter or bypass. With few exceptions,

410

1 4 0   • Act i o n a b l e   M e d i a

WIMP interfaces generally mandate that users allocate their attention in the manner of a zero-​sum game:  to engage with the personal computer screen means disengaging from one’s surroundings, with varying degrees and durations (Rieder). This consequence of WIMP design resonates ironically well with common dictionary definitions of the word “wimp” when it is used as a verb (e.g., to “wimp out”). When used as a verb, wimp means “to withdraw from a course of action”; that meaning is so frankly revealing of WIMP interface design and its persistence today. More often than not, popular use cases supported by mobile interfaces still require people to wimp—​to withdraw from a course of local action in favor of tele-​action. Proof of this withdrawal is evident in research studies of texting while driving cited in ­chapter 2. Beyond the road, of course, citizens living in emerging ubicomp cultures constantly inhabit settings in which people engage with mobile media that diverts their attention away from the here and now. In contrast to popular mobile use cases, post-​WIMP design aims to deliver or generate multimedia at the periphery of otherwise nondiscursive actions, so that one need not oscillate between the screen and the surroundings. Post-​WIMP interfaces, then, are not (only) designed around a user, a willful individual subject whose every interaction with computers is a manifestation of conscious personal commands. Conversations about post-​WIMP prototypes exude contempt for anthropocentrism. Working in different fields but referencing similar projects, rhetorical theorist Thomas Rickert and interaction designer Jan Rod lend apt posthuman perspectives to this decentering of the user. Principally, Rickert interprets ubicomp’s mandate to embed computation into everyday objects and architectural structures in terms of an ontological flattening variously signaled in philosophical discourse (e.g., Bruno Latour’s actant-​networks, Graham Harman’s object-​ oriented ontology, Jane Bennett’s political ecology of things). As such, rhetorical activity exceeds intentional human subjects more explicitly than ever before. No longer regarded as an inert stage or mere tool to be taken up by human rhetors, ambient environments and smart objects each become “an active player in their own right,” registering fluctuations in the surroundings and generating “adaptive responses to evolving situations” (29). Intentionality and agency become distributed to the point where subject/​ object dualities are conceptually untenable, if not imperceptible. Rod’s work complements Rickert’s Ambient Rhetoric by showing how ubicomp breeds parallel disruptions in design thinking. For Rod, smart objects incite a general erasure of the subject/​object dualism on which popular user-​centered

41

Forms of Actionable Media  • 

141

HCI theories have been based. He argues that anthropocentric approaches do not account for the fact that “subjectivity is being subsumed into objects and creates hybrid entities that break out of these dualistic categories” (Rod). As a class, post-​WIMP interfaces register a much wider array of variables and gestures than the mechanics of pointing and clicking associated with user-​centered inputs (i.e., a mouse and keyboard). Consequently, in ubicomp-​saturated milieus, any actant (e.g., bodies in motion, inorganic matter, atmospheric effects) can impact the computational operations that arrange and deliver multimedia. Whereas “user-​friendliness” was the highest value of the personal computing era—​achieved largely via WIMP GUIs—​post-​WIMP designers aim after a further goal: “to minimize the mechanics of manipulation and the cognitive distance between intent and the execution of that intent” (van Dam 2). The key objective, put more simply in terms of actionable media, is to impart greater degrees of synchronicity between computation and action. Post-​ WIMP digital interactions should dance in step with what the user is doing, while she’s doing it, and where she’s doing it. (Google’s 3D sensing platform “Project Tango” is aptly named.) With such trends gaining traction, Malcolm McCullough foresees a “huge cultural shift” looming in the way we deploy digital technologies—​“away from a means to overcome the world toward a means to understand it” (Ambient Commons 13). Post-​WIMP initiatives are largely a matter of designing more nimble interfaces that, in turn, enable new ways to circulate data out of cyberspace and into various locales. At its limits, post-​WIMP discourse beckons the invention of new figures. Without a pronounced and formalized set of concepts to organize our experience and channel reflections on emerging mediascapes, theorists and practitioners lack the cognitive interstices needed to exchange general insights derived from particular cases and ephemeral circumstances. The ubicomp counterparts to WIMP are not “post-​icons” or “post-​menus.” But the ground cleared by post-​WIMP scholarship is catalytic for discovering more assertive terms. Just as WIMP taught a critical mass of people to productively engage with computation (without having to face complicated arrays of programming languages and binary data), a new schema of forms has begun to emerge in actionable media ecologies. These figures are poised to transgress the bounds of the desktop metaphor and reconfigure many online networks. The main concern here lies no longer in creating simulations of territories and behaviors for a virtual map, but in deconstructing the map to more profoundly integrate its components with lived territories and a wider range of actants.

412

1 4 2   • Act i o n a b l e   M e d i a

ATLAS: Apps, Tags, Layers, Actuators, Sensors First, a statement of the obvious: I did not invent these words. The five terms that comprise my ATLAS acronym (Apps, Tags, Layers, Actuators, Sensors) already circulate as keywords in computer science, engineering, and tech journalism—​just as the words “network,” “database,” and “hypertext” were in regular use before media theorists in the 1990s infused them with unprecedented conceptual, social, and semiotic weight.2 In order to close “the distance between the interface design community and that concerned with critical theory,” Johanna Drucker urges humanities scholars to raid their own intellectual traditions for alternative conceptual resources to describe contemporary interfaces and media effects (12). I mean to imbue these five terms with rhetorical, hermeneutic, and aesthetic significance, in an effort to foreground the actionable modalities of communication, expression, and interactivity that they make available. To the extent that the circulation of digital content continues its post-​desktop migration; apps, layers, and the like may become key media forms by which many of us read, write, design, and think—​in concert with our surroundings. Whereas WIMP prompts users to withdraw from local action in favor of tele-​action, ATLAS aims to forge links between local action and networked media. To manifest this design philosophy, each ATLAS figure works in coordination with the others, which is why it makes sense to discuss them acronymically as a set of rhetorical figures. At the same time, each ATLAS figure marks a unique mode of configuring bonds between multimedia and locales, just as classical figures of speech distinguish among different formalized structures of verbal expression that variously remix conventional usage. We can distinguish between tags and layers, for instance, in much the same way we differentiate alliteration from anaphora, or hyperbole from allegory. Related figures are grouped together when they appear to produce similar rhetorical effects; still, rhetoricians have always recognized the respective means by which each figure achieves its ends. ATLAS, the elemental forms of actionable media, each emphasize a distinct way of breaking down (i.e., grammatizing) the flux of the Web and/​or a distinct way of transcoding actant-​networks into networked traces. Apps, layers, and actuators break down the flux of the Web to suit experience economies specific to particular sites of action. Tags and sensors attach and generate multimedia feeds from assemblages of actants, effectively assigning (in the manner of Stiegler’s techno-​geographic milieus) a variety of rhetorical-​ aesthetic functions to the real-​time movements of humans, nonhumans, and

413

Forms of Actionable Media  • 

143

geographical flux. Each of these ATLAS figures constitutes an important seat of digital communication beyond the desktop. Table 5.1 lists characteristics unique to each figure, as a basis for establishing initial definitions before we delve into close analyses of each figure in contemporary mediascapes. In addition to providing a means to conceptualize rhetorical figures consonant with ubicomp cultures, the figurative tradition also harbors a central distinction that critically attunes us to the poetics at play in the shaping of actionable media forms. The logic of desktop metonymy articulated in ­chapter  4 underwrites not only the technical emergence of ATLAS genres but also the character of their rhetorical power. Table 5.1  ATLAS: Apps, Tags, Layers, Actuators, Sensors ATLAS Apps

Basic Principle

Streamline the arrangement, production, and circulation of networked media around a single embodied activity Tags Transform vast arrays of objects and places into surfaces for inscribing and projecting digital content Layers Present fragments of pertinent texts and/​or audiovisual media amid one’s current sensory perceptions Actuators Perform computational and operations in response Sensors to dynamic data gathered in situ via embedded sensors, which perpetually scan local surroundings and convert environmental “noise” into meaningful signals to be actuated upon

Common Practices Exemplary Projects Crowdsourcing for iSeahorse; “Citizen Science”; SeeClickFix Civic Forums for “Government 2.0” Place-​Based or Textopia; Object-​Oriented Trash|Track Micronarratives; Tracking for Data Visualizations Street Archives; Streetmuseum; Augmented Manifest.AR @ Reality Artworks ZERO1

Image Recognition AR|AD Takeover; Interventions; David’s Way Public Interactive Installations

41

1 4 4   • Act i o n a b l e   M e d i a

Whereas WIMP-​use cases are rooted in metaphor, metonymy is the figurative mode that drives the formation and development of ATLAS scenarios. In each iteration of the latter, a discrete part becomes detached from the desktop GUI, and that part stands in for the whole of post-​desktop computing. Note Hugh Bredin’s distinction between these two important rhetorical figures: “Metaphor creates the relation between its objects, while metonymy presupposes that relation” (45). As a metonymy-​oriented enterprise, ubicomp presupposes our familiarity with the virtual space of the personal computer and its core operations. ATLAS-​style interfaces found new configurations of multimedia that refer to—​but at once transform—​those of the PC era. Software applications comprised of intricate menus and multiple windows become streamlined down to apps, which typically fragment larger programs, parsing out small doses of functionality that are highly specific to one task. Tags are lifted from their backstage function in computer code (e.g., HTML) and promoted from their supporting role in Web 2.0 (e.g., tag clouds, hashtags), such that Web 3.0 tags (e.g., QR, RFID) now circulate freely with material objects and moving bodies, variously reconfiguring the delivery and composition of digital content around local action. In each case, the ATLAS figures are becoming primary vehicles for perceiving, interacting with, and making sense of the extracomputational lifeworld. If the desktop metaphor brought the office into the machine, then desktop metonymy transcodes the entire networked world in the machine’s fractured image. Consider further the metonymical evolution of layers over the past thirty years, from the PC era to the dawn of ubicomp. One way to comprehend the post-​desktop life of layers is to trace the lines of flight by which they transgress the WIMP framework. Layers have existed since the PC era in two predominate forms: (1) as dynamic menus constitutive of and subordinate to WIMP interface logic; and (2) as modular components of audiovisual media files produced and edited within GUI authoring software. In his study of Adobe Photoshop, Lev Manovich explains that layers have long been a commonplace form essential to “pre-​digital” media production techniques including multitrack audio recording and cell animation in filmmaking (“Inside Photoshop”). With Photoshop, as with these production techniques that precede it, media are rendered into discrete “layers” (or “tracks”) that can be manipulated independently. In each case, we encounter “[t]‌he same idea of treating an image [or audio composition] as a collection of elements that can be changed independently and reassembled into new images [or new audio compositions]” (“Inside Photoshop”). As such, these iterations of layers present transformative implications for content creation: “what used to be an indivisible

415

Forms of Actionable Media  • 

145

whole becomes a composite of separate parts” (“Inside Photoshop”). Whereas a painter’s every brushstroke alters the appearance of their composition as a whole, the multimedia producer working with layered files can experiment with, oscillate among, and indefinitely bracket an immense range of audio/​ visual properties and file combinations. With the ongoing proliferations of professional and “freemium” WYSIWYG authoring software (WIMP design par excellence), the modularity and variability of computation continue to transform the rhetorical and aesthetic processes by which writers and designers invent, arrange, and stylize all manner of texts, images, and audiovisual media. These programs, in short, make for compositional environments that differ profoundly from more archetypical scenarios, wherein one puts pen or brush to a blank white surface, doing so without a database of media effects standing in reserve at the margins, courtesy of WIMP design. So what becomes of layers when they migrate into ubicomp interfaces? First, they are no longer subordinate to WIMPs; rather, they operate in conjunction with apps, tags, actuators, and sensors to orchestrate a new relationship between multimedia and local, live action the lifeworld. Beyond the desktop, layers are media files (texts, images, audio, video) that reside, appear, and function amid the contingencies of everyday life, the infinite richness of the universe, or the spontaneous actions of actant-​networks. This heightened degree integration with extracomputational activities is a direct function of the metonymical mode. Reducing the desktop metaphor to its most atomic elements filters the scene of computing into the site of one’s present surroundings. While ubicomp layers may still be composed via WIMP interfaces like Photoshop, they are designed to circulate on post-​WIMP platforms—​on head-​mounted wearable displays, for example, rather than the browser-​ based Web. The transformation of layers beyond the desktop is conditioned, in this sense, by the evolution of tags. Whereas HTML tags often stipulate the arrangement of media files within conventional websites, Web 3.0 tags sync digital content with the movement of (non)human bodies, geographical coordinates, and architectural structures in the built environment. Here, one no longer designs a menu-​layer for navigating webpages; digital content exists as networked layers outside of conventional Web browsers. And these layers may, at least in some cases, obsolesce menu-​filled websites and become an anchor of deep attention in actionable media ecologies such as augmented reality and urban installations, as we shall see below. Having outlined basic principles that distinguish each ATLAS figure, and having noted the metonymical nature of their emergence, I now move to perform closer readings of ATLAS at play in public art, science, and history

416

1 4 6   • Act i o n a b l e   M e d i a

projects that variously instantiate the affordances and prospects of actionable media. In doing so, I will specify some cultural implications and rhetorical challenges made apparent by works that deploy an ATLAS figure toward suggestive if not exemplary results. Each section also probes early criticisms of these forms put forth by media scholars, developers, and tech journalists. Indeed, there is already a widespread tendency to interpret instances of ATLAS through the lens and values of the desktop metaphor, as if it were the transcendental signified of all digital inquiry. Instead, we might learn to regard ATLAS as signifying chains that desktop metonymy puts into freeplay. When conceived as nascent rhetorical figures—​as constitutive forms of actionable media and ubicomp cultures—​apps, layers, and the like stand to supplement and possibly disrupt frameworks in new media, interface design, and digital communication that remain conceptually bound to personal computing, the browser-​based Web, and other electronic iterations of the deferred archive.

Apps January 2014 marked the first month in which Americans started accessing the Internet via smartphone/​tablet apps more often than through the browser-​based World Wide Web (O’Toole). The trend has since intensified and been mirrored in other countries. Statistics tracing the rise of apps spur ongoing debates among tech journalists and media professionals, many of whom interpret this trend as a threat to the more conventional websites, search engines, and wikis signified by the term Web 2.0. Controversy ensued when Wired published a 2010 feature article by Chris Anderson and Michael Wolff called “The Web is Dead. Long Live the Internet.” After cataloguing a typical day in the life of an “iConsumer,” Anderson and Wolff note with palpable shock how mobile computing has, within just a few years, led online traffic away from the World Wide Web and its apparent gift economy of open-​source collaboration. Contrary to the Web, mobile apps house silos of digital content that do not readily link to other networked documents. If the Web remains the same decentralized utopia it was for its early proponents, then apps, according to Anderson and Wolff, represent a kind of capitalistic colonization of the virtual frontier. They believe Internet users have turned to apps “not because they’re rejecting the idea of the Web but because these dedicated platforms often just work better or fit better into their lives” (Anderson and Wolff ). The neatly bundled character of apps, that is, makes them more consumer-​friendly than the malleable and messy artifacts enabled by the Web’s hacker ethos.

417

Forms of Actionable Media  • 

147

Following Anderson and Wolff ’s article, hoards of follow-​up essays and blog posts appropriated their title in efforts to rebut or affirm the authors’ thesis. While the affirmations predictably reinforce and extend Anderson and Wolff ’s proclamation in the context of more recent usage trends, the rebuttals are far more interesting for the anti-​app sentiments they express. Whereas Anderson and Wolff describe the open Web as a waning “adolescent phase” in the history of the Internet, others tech industry writers bemoan the rise of apps as a recurring nuance that must be critiqued to death, for the sake of preserving uninhibited connectivity among documents, devices, and people. Dave Sikora traces the structure of mobile apps back to the walled garden platforms of the early 90s such as AOL and CompuServe, and he defines walled gardens as “a closed-​loop system for engagement” (Sikora). In addition to social networks, Ryan Holmes insists, “Mobile apps . . . are another challenge to the open Web—​self-​contained islands that lock off their data from the wider net” (Holmes). The pull of this sentiment is also evident in electronic publisher Bob Stein’s adoption and eventual rejection of apps, both as a term and a form of software. In 2010, Stein proposed that apps might be the next big evolution in the history of media, that apps may become as pervasive and fundamental as books have been (“The Future of the App”). Websites, the analogy implies, will go the way of the ancient scroll. Given the boldness of this claim, Stein’s 2012 dismissal of apps is rather surprising. This time, he reformulates the relationship between apps and books in terms of reproduction rather than evolution: “apps . . . reproduce the separation of one book from the next . . . they just replace one fixed bit of content with another fixed bit of content” (“SocialBook”). Browser-​based websites, on the other hand, support seemingly infinite arrays of linkage among dynamic content. One can access the Web’s information within a single window. On the heels of this distinction, Stein adds that apps are “reactionary”—​nothing more than a “transposition of the mentalities and paradigms of print into the electronic space” (“SocialBook”). But a transposition of another set of mentalities reveals itself in Stein’s logic. His evaluation of apps adopts global village virtues as a criterion for assessing technologies that were designed to serve radically different ideals. From the standpoint of ubicomp, the browser-​based Web is the ultimate walled garden. Documents link to one another ad infinitum; duly surfing websites effectively walls one’s being from the world not simulated within windows, icons, menus, and pointers. Apps are ubicomp’s first antidote (bearing poisons of their own, of course) to the structural divisions inherent in a system that privileges global circulation at the expense of local context.

418

1 4 8   • Act i o n a b l e   M e d i a

Communication scholar Gerard Goggin traces the arrival of apps as a bona fide cultural platform to the lines of research initiated by Weiser’s ubicomp manifestos (149–​150). So concerned with the commercial basis of Apple’s App Store is Goggin’s analysis, however, that his urgent plea for a “politics of openness” on the part of all app stores distances him from grappling with his more fundamental question of “how [apps] operate as new circuits for culture” (155). Accounting for the grammatological constitution of apps can channel insights into their rhetoricity. For starters, we can rework the above notions of openness and closure. Apps must be “closed” in order to let in the extracomputational lifeworld. Their fragmentary design makes for a more permeable scene of computing. Whereas apps are “self-​contained” in relation to the larger Web, their crafted arrangements encourage implicit links to embodied activities. If we account for the world beyond the Web, then app development is not (only) about blocking but about bonding. Apps lay a foundation for ad ​hoc networks—​loose bonds—​to emerge among networked multimedia and local action. This is their basic purpose in actionable media ecologies, and attuned digital creatives recognize that purpose when composing apps and app-​based content. The practices I discuss below provide case studies of apps operating in this sense and, more to the point, they specify some of the opportunities and challenges apps pose for writers and designers of all sorts, from researchers to lay publics. I have chosen examples to edify the thesis that apps may serve as a rhetorical figure “for tuning in rather than out” (McCullough, Ambient Commons 210). Angry Birds did not make the cut. In other words, many popular mobile apps today still revolve around virtual worlds that steer toward trivial diversions. Worse still, apps built in the service of major global brands amplify the encroachment of corporate advertising into everyday life, especially through the use of daily “deal alerts” and “push notifications,” which seem to “replac[e]‌the sensible experience of singularity with the aesthetic conditioning of consumer behavior” (Stiegler, Decadence of Industrial Democracies 85). My characterization of apps as an emerging cultural form privileges projects that turn multimedia networks toward local action in attempt to foster crowdsourced knowledge ​ building, informal learning, grassroots place making, civic engagement, and social critique. From the perspective of rhetorical invention and democratic deliberation, these apps are provocative for the kinds of forums they establish and for the modes of discourse and discovery they initiate. Two burgeoning areas of app development, at the time of writing, offer promising models for analysis and elaboration: citizen science apps and civic apps.

419

Forms of Actionable Media  • 

149

Citizen science apps aim to unite professional researchers with lay publics who have shared interests or at least engage in activities related to the researchers’ agenda. An avian specialist might, for example, enlist the help of casual bird watchers around the world to extend the scope of data to be gathered for a given study. Field researchers have employed such crowdsourcing methodologies well before the Internet, albeit on smaller scales, through open calls for volunteer contributions or sponsored competitions. The significance of citizen science apps, for our purposes, lies in the collective mode of discovery they suggest for all knowledge ​building communities in the age of ubicomp. These apps streamline the arrangement, production, and circulation of networked multimedia around a specific embodied activity. Take the citizen science app iSeahorse as a representative example. A collaboration between marine biologists at the University of British Columbia and iNaturalist.org, iSeahorse adds a vital public dimension to the reseachers’ ongoing efforts to study seahorses in the wild and identify endangered populations most in need of conservation attention. Because seahorses are notoriously elusive objects of study—​most varieties are considered “data deficient”—​the app-​equipped masses will likely stumble upon them with much greater frequency than a few determined specialists. The iSeahorse app is “designed for people to quickly log seahorse sightings whenever they encounter an animal in the wild” (“Crowdsourcing Seahorses”). In essence—​ as a “closed-​loop system for engagement”—​iSeahorse’s interface restricts the scope of production to photos and textual descriptions of seahorses, and it confines their circulation to the project’s website. The content presented to users is also constrained. Observations added by other users are not accessible within the app. The only texts in the app remain static: the “About” page and the “Seahorse guide,” both of which are centrally authored and administered. In this sense, the app seems to foreclose precisely those affordances most celebrated by tech pundits and digital rhetoricians in their commentaries about wikis, blogs, and social media. Why abandon the open coauthorship model so common to online discourse communities? Because opening the interface in a Web 2.0 manner would likely short-​circuit the collective process of discovery the app seeks to facilitate. iSeahorse contains only that content which provides actionable intelligence to people searching for seahorses in situ. After snapping a photo of a seahorse, the app prompts users to determine the species by consulting its seahorse guide, which features a photo of each species and brief descriptions about each species’ normal habitat, distinct traits, etc.—​all to help citizen scientists make an informed judgment at the moment of discovery. The app,

510

1 5 0   • Act i o n a b l e   M e d i a

through its actionable design, frames the encounter in a scientific lens, supplying lay publics with a lean database of knowledge that adds a researcher-​ curated feedback loop to each user’s available means for negotiating his or her sensory experience. In turn, users engaged in the app-​aided process of discovery generate a networked trace of their individual activity, as any photos and texts they produce become uploaded to the project’s website. The website, of course, embraces the Web 2.0 conventions that we find on many blogs and media sharing platforms. Registered users post comments about each other’s observations, occasionally debating the species identity or asking the observer questions about an encounter. The app, when we take these online conversations into account, thus feeds into the same general media ecology wherein Web browsing and social networking take place; its so-​called “self-​contained” nature is somewhat illusionary, and this is the true of most apps. The closed-​ loop structure furnishes actionable links between an app’s networked content and the user’s perceptions of the extracomputational lifeworld, but—​at the same time—​many of the app-​enabled actions that occur become recorded or transcoded as digital data that makes rhetorical contributions to a corresponding website or social media feed. Citizen science apps of this sort do not wall off a writing space entirely separate from the Web. Instead, the app flows into the Web, altering the character of online forums by establishing a new basis for peer-​to-​peer commentary and debate. Website conversations about seahorses, in this case, take anchor in the indexical traces of curated discovery gathered in the app: an individual’s local action becomes the talk of the global village, as it simultaneously informs and is informed by the iSeahorse network of researchers and volunteers. Civic apps, another emerging genre, extend the logic of citizen science networks to social problems and political concerns shared by a city population. Whereas citizen science apps center on gathering data to inform research, most civic apps are created from existing data sets compiled by local governments and other community organizations. Originating in cities such as Portland, Chicago, and San Francisco, civic apps have quickly become a highly promoted agenda item in cities around the world endeavoring to make their records not only more accessible but also more actionable for people on a day-​to-​day basis. By and large, though, early iterations address fairly mundane matters: bus schedules with real-​time route tracking (allSchedules), data visualizations of recent criminal activities in a neighborhood (SpotCrime), notification systems that send personalized reminders if your car is parked in a zone soon destined for street sweeping (SweepDodger). Without denying the utility evident here, the rhetorical value of civic apps—​ubicomp

15

Forms of Actionable Media  • 

151

software developed for the sake of enriching public life—​remains to be realized. A suggestive minority do, however, provide intriguing case studies for further cultural development. They inch toward a renewal of small forums for deliberative rhetoric at the city level and yield glimpses into how local politics might be performed in the near future. Just as citizen science apps involve lay publics at the ground zero of professional research, some civic apps channel citizens’ observations about their neighborhoods into a networked forum that government agencies monitor, evaluate, and act upon. While popular services like 311 and Nextdoor accomplish this to some degree, the lesser-​known app SeeClickFix provides a more fitting example. SeeClickFix positions the citizen as an agent capable of identifying problems and discussing solutions with concerned peers and relevant authorities. The citizen, in other words, is regarded as a citizen and not simply a consumer of municipal information and services. The latter assumption underlies the bulk of so-​called “civic hacking” efforts, wherein developers work with city officials to package data in user-​friendly formats. But even the most sophisticated data visualizations seem trivial next to the promising affordances of SeeClickFix, despite its lackluster usage in most cities and towns to date. The app’s interface resembles iSeahorse in design. Upon encountering an exigency, citizens open the app to capture a live action shot (photo or video) of the problem and add a description to establish context and urgency. This basic capacity holds serious potential to impact local politics in twenty-​first ​century cities. It makes the same difference that having a cellphone makes when one’s car dies out on a desolate road. Having the cellphone ties you directly to people with resources, skills, and experience necessary to help you better understand the problem, or to request their immediate service. SeeClickFix conjures up similar affordances for ad h ​ oc bonds between citizens, experts, and organizations on a much grander scale. When citizens witness something in their city that they deem socially problematic or otherwise worthy of collective attention, what lines of communication exist for them to share their impressions? The options vary greatly across different regional contexts, of course, but common possibilities can be listed: call the police, family, or friends; notify strangers in the vicinity; directly intervene, verbally or physically; go home and write a letter to the newspaper or elected officials; contact a local reporter and ask them to look into it; take a photo or video of the incident and post it to Facebook, Instagram, etc. This list is not exhaustive, though I am already exhausted by it. The former options seem effective enough for dealing with isolated,

512

1 5 2   • Act i o n a b l e   M e d i a

unequivocal emergencies—​but they rarely spur collective debate or extended reflection. The latter options are variously removed from the time and place of the event and the sphere of local action. When SeeClickFix is added to the available means of communication, a citizen’s impression can be almost instantly transcoded into multimedia documentation on an actionable forum open to all interested citizens. The most promising faculty of such apps lies not (only) in mobilizing quick fixes to public infrastructure, but in fostering a mode of deliberative rhetoric wherein local action sets the agenda and persists as the primary object/​site for ad ​hoc citizen networking. Whenever citizens voice concerns about a social problem, their discourse accrues on exactly the same grounds where that problem occurs. This emphasis on proximity can be the way we come to understand and attempt to reconcile community issues, pushing civic utterances to anchor themselves in local singularities and, in turn, eroding the ethos of rhetors who couch their appeals in readymade partisan positions handed down from national debates. Moreover, because such citizen networking occurs in a space where relevant institutions can follow the conversation in real-​time, discussions held among one group do not have to be held ad infinitum up different rungs of a bureaucratic ladder. One can imagine SeeClickFix serving as more than a platform for reporting potholes. The app (or others in its wake) could be easily renamed and reframed to encourage a broader agenda of civic issues, prompting in situ dialogues about sources of social injustice and environmental pollution, in addition to quick fixes to public infrastructure. Nevertheless, interface design has its limits. The affordances of a platform wither without rhetorical participation; its social value is only hypothetical until conversations ensue. Therefore, a pressing task for rhetorical and political theory, with regards to civic apps and ubicomp cultures, is to model compelling futures for actionable forums like SeeClickFix—​to see beyond what is and prototype what could be, for better and for worse. Indeed, if we imagine scenarios where such forums may become more substantive, we should also acknowledge how the municipality’s capacity to respond to citizen observations could breed societies of control just as easily as deliberative democracies. Already evident among SeeClickFix participants is a tendency to use the app as a means to spy on one’s enemies, monitoring them for the slightest wrong-​doing and reporting it out of spite or revenge. To empower citizens with a direct link to state power is, potentially, to foster a neighborhood-​police state. Citizens can no doubt use the app in a manner whereby exercises of individual agency inadvertently disenfranchise the autonomy of every individual.

513

Forms of Actionable Media  • 

153

Both of these horizons of social becoming, respectively constitutive of deliberation and control, are induced by the grammatization processes characteristic to desktop metonymy. Apps, in their varied iterations, ferry digital content toward all sorts of everyday scenes of decision, and they institute concentrated arenas of production around a closely related scope of embodied activities. They are the gateway figure propelling the continued growth of actionable media, marking the most basic modality through which sensory experience becomes infused with computation. Apps are the most basic ATLAS figure because, in many cases, they still require individual users to open them up and consciously activate their various features. As such, the bonds they establish between networked multimedia and local action are largely implied. When apps interlock with tags, layers, actuators, and sensors—​as is increasingly the case on handheld devices and especially newer wearable and embedded interfaces—​one’s visual or sonic field plays host to multimedia inhabiting her surrounding environment (shared with others) as well as data feeds to which she (and possibly only she) subscribes at that moment. Historically, this mode of perception has been marginal and occasional; henceforth, it may no longer be the exception but the rule. Grammatology shows us that such fundamental techn(ont)ological changes inevitably affect the rhetorical practices and social conditions underlying political life. The appearance of actionable forums for knowledge b​ uilding and civic engagement (qua citizens science apps and civic apps) begs theorists to speculate about the virtues and vices potential to the niche research collectives and hyperlocal political networks emerging with the ATLAS figures. Let us continue to mull over this theme in the analysis of tags.

Tags New modalities of tagging have underwritten each major evolution of the Web. Since its inception, the Web has been authored entirely between the tags that make up HTML, CSS, and other Web markup/​programming languages. Prior to “Web 2.0,” if you did not surround your texts with these tags, then your online activities would leave a webpage’s form and content more or less unaltered. Tags developed through newer Web programming languages such as PHP initiated a pivotal change, whereby dynamic websites could generate HTML tags on-​the-​fly, in response to the otherwise noncoded pointing, clicking, and typing that users perform on a given website. This advancement in tagging is what enables so-​called user-​generated content; users can create a Facebook profile without inputting HTML tags because the site’s PHP tags

514

1 5 4   • Act i o n a b l e   M e d i a

automatically add the appropriate tags to the text one types into its textboxes. Of course, further iterations of tagging formed with (and catalyzed) the rise of Web 2.0 and social media sites. Tag clouds in WordPress, photo tags in Facebook, keyword tags in YouTube, and hashtags in Twitter, etc. These quintessential Web 2.0 tags established a different way to arrange, deliver, and discover online content. Instead of relying primarily on search engines to sift through the metadata offered by webpage authors, Web 2.0 users create metadata on-​the-​fly and this metadata occupies prominent positions within the body of webpages, no longer relegated to a backstage, reference-​only position in source code. The index becomes the table of contents. Citing the rise of ubicomp, pundits and theorists now bespeak of a “Web 3.0” and an Internet of things. The meaning of these buzzwords can be most acutely derived by surveying the tags which anchor this nascent reconfiguration of the Web. Whereas Web 2.0 tags made media “spreadable” in the sense that Henry Jenkins et al. argue, Web 3.0 tags are making media actionable. The latter establish links, as all tags do; what distinguishes their manner of linking from previous iterations, however, is the capacity Web 3.0 tags have for supporting ad h ​ oc networks of links among online content, local action, everyday objects, and geographical flows in real-​time. Early Web 3.0 tags, such as QR coding and geotagging, map digital data over two-​dimensional surfaces and geographical coordinates. While QR codes and geotags are pervasive if unglamorous, more robust manners of actionable linking have emerged. In particular, radio-​frequency identification (RFID tags) and image recognition add further technical and rhetorical dimensions to these earlier modalities, as I will discuss below. Web 3.0 tags collectively transform almost any surface—​car windshields, building facades, books, tattoos—​into a surface for inscribing and projecting digital content. Hence, many more materials and places stand as computing and writing spaces in potentia, as open to access and authorship as the browser-​based Web. Rather than structure their narratives or arguments over the space of a page (print or digital), writers and researchers currently experimenting with the grammar of Web 3.0 tags endeavor to arrange their work over the space of a city park, a college campus, or a historic landmark. We might say, rifting on Shakespeare in the age of ubicomp, that all the world’s a link. As audiences of Macbeth have learned to view life in the image of theater, so too will twenty-​first c​ entury digital cultures learn to live among countless objects that each harbor and generate curated arrays of images, texts, audio, or video. These novel affordances of Web 3.0 tags also put them at the center of recent controversies over the political implications of ubicomp. More than

51

Forms of Actionable Media  • 

155

any other ATLAS figure, tags exasperate “big brother” anxieties. Tags make it possible for governments and employers to embed a wireless signal into the identification cards carried by their citizens and employees. Pet owners install microchips into the bodies of their cats and dogs. Parents attach “Tile” trackers to their child’s backpack. With Web 3.0 tags, one can track or trace anything with gear purchased from Best Buy or Amazon. Peer-​to-​peer surveillance has become a mass consumer market. The things we carry, the things we buy or even pass by, may be collecting or sharing data about us, often in too subtle a way. Meanwhile, popular location-​sharing features on mobile social media manufacture consent, a collective will-​to-​be-​tracked. To participate on such networks is to broadcast your whereabouts to varying degrees as well as to subscribe to the real-​time movements of others. We become the targets and the arbiters of pervasive tracking, from top-​down to bottom-​up. Such are the sentiments that drive humanities scholarship on tags. A dominant inference path has been forged. Tagging is tracking is surveillance. Revelations of covert mass surveillance initiatives (e.g., evidence of the Stasi’s domestic wiretapping of East German homes during the Cold War, Edward Snowden’s leaked documents exposing the NSA’s omniscient data-​mining of global telecommunications) provide eminent justification for resisting newer tagging technologies. At the same time, the politics exercised through one domineering set of practices is not the only politics that may be served or constituted by a given technology. All technologies may be regarded as pharmakon, which is not to say they are neutral, but that they are multidimensional. Web 3.0 tags stand as remedies that are also poisons, and they engender multiple possibilities. Some demand ideological critique, others merit cultural development. In addition to identifying the poisons, rhetorical theory can also clue us into the enigmas and opportunities that tags introduce for digital inquiry and critical media authorship. As a multitude of everyday objects become tagged with computation, what will they say and do as rhetorical actants in various digital networks? How will these networks (and the everyday activities they support) change as a result of that participation? In other words, how might Web 3.0 tags turn or figure digital discourse toward local action? Rickert suggests that the Internet of things promises to extend the scope of rhetoric in two respects, which are worth mentioning here. First, the kinds of persuasive appeals and value judgments at play within speeches and texts will enter into everyday objects; in addition to their industrial design, smart objects are rhetorically attuned to gather data, perform interpretations about that data, and to deliver actionable statements to relevant audiences. Consequently, to the extent that smart

516

1 5 6   • Act i o n a b l e   M e d i a

objects “think” and “reply” in this manner, it becomes increasingly untenable to regard rhetoric as a humans-​only domain. Rhetoric in the age of ubicomp, according to Rickert, should be reconceived as the study of the ways in which the world becomes revealed in manners that inspire action (29). Mobilizing Web 3.0 tags to craft (a network of ) smart objects stands to become a fundamental gesture for digital rhetoric in this milieu, and the political efficacy of these technotexts, already apparent in early iterations, merits close rhetorical analysis. We may begin to glean the rhetoricity of smart objects (enabled by Web 3.0 tags) by looking first at a basic thought experiment Rickert sketches in Ambient Rhetoric, then moving on to consider how this ontological/​ rhetorical flattening unfolds in an ambitious project executed by researchers at MIT’s SENSEable City Lab. Rickert’s thought experiment involves “a medicine bottle connected to a database that monitors usage patterns”; this networked medicine bottle, tagged via RFID, serves as a rhetorical keystone in a “dynamic material-​informatic ecology” (32). In practice, whenever the medicine bottle’s contents fluctuate, a corresponding database records the data in real-​time and, if the proscribed threshold gets crossed, the software promptly notifies relevant authorities and supplies them with pertinent information needed to take actions appropriate to possible overdose emergencies. Whereas the text printed onto the medicine bottle serves to remind the patient of the doctor’s proscription, the RFID tag extends the doctor’s gaze to encompass the patient’s interactions with the medicine bottle. The patient acts, hence, not only in relation to the doctor’s words but also under the semblance of a watchful eye. The tagged bottle links the patient’s actions to a network of institutions, protocols, and personnel who (in Rickert’s example) take the bottle at its word. The ethos programmed into this “material-​informatic network” is such that decisions and actions are based upon empirical data processed in the abstract. In this example, the consequences do not appear particularly controversial, but Rickert’s point comes through. The medicine bottle is the guard in the high tower. Every smart object is at once (potentially) a panopticon. Such conclusions are familiar enough. As humanities scholars, why are we so adept at projecting critique qua digital dystopia and so reluctant to envisage how our disciplinary recourses might contribute to more desirable outcomes?3 Clearly, in Rickert’s thought experiment, Web 3.0 tags serve to advance the societies of control that Gilles Deleuze aptly vilified. Critiquing “the values and decisions that emerge from and are built into” such scenarios is gravely important as ubicomp increasingly enters public infrastructures

517

Forms of Actionable Media  • 

157

(Rickert 32). Still, this is low-​hanging fruit. Rickert, like many recent critics writing about ubicomp, gravitates toward examples that cast the public merely as witnesses or subjects of surveillance systems designed and implemented by higher authorities. He never makes the heuretic leap to consider how fundamental disruptions to anthropocentric agency may inform future acts of multimedia production, or how rhetors might deploy Web 3.0 tags to craft networks of smart objects that further democratic values. In addition to becoming aware of how ambient interfaces redistrubute rhetorical activities, we also need theoretical frameworks that enable us to situate and negotiate our agency in relation to other rhetorical actants with whom we partner. Web 3.0 tags emerging with ubicomp may be conceived as platforms for ubicomposition, and not solely an object of critique forced upon us by the technocratic powers that be. As N.  Katherine Hayles rightly asserts, “The challenge RFID presents is how to use it to re-​think human subjectivity in constructive and life-​ enhancing ways without capitulating to its coercive and exploitive aspects” (“RFID” 48). This is a call to rhetoric, issued from the limits of ontology. What is needs to be more eloquently and ethically developed. As more objects become smart objects, the substance of objects and our relations with them will change. But the precise nature of that change is not predetermined. Where ontologists speculate, rhetoricians strategize. Let us, then, analyze the rhetoricity at play in exemplary projects and extrapolate from them some insights for writing with tags and for theorizing the kinds of communication that tags make available. The examples described in the next two sections will emphasize how tags, when combined with layers and sensors, break down the flux of the browser-​based Web (and WIMP GUIs) in order to project and display networked multimedia in sync with and peripheral to the local actions of (non)human bodies via mobile, wearable, and embedded interfaces. The first example I consider here, Trash|Track, illustrates a different rhetorical affordance. Public interactive installations like David’s Way, as we will see below, embed tags to fixed structures to deliver already-​composed layers to a scene according to the rhythms of actions occurring there (as read via sensors)—​doing so in order to engender (new) patterns of social organization through dynamic environmental cues. Other kinds of tag-​driven projects, like MIT’s Trash|Track, are designed to provoke critical reflection about deeply entrenched cultural practices on a much grander spatiotemporal scale. Through action-​oriented tracking, they trace the actant-​networks that underwrite some of the most intricate ecological processes that societies rely upon but also take for granted.

518

1 5 8   • Act i o n a b l e   M e d i a

The Trash|Track project, launched by MIT’s SENSEable City Lab in 2009, aims to raise public awareness about the circulation of garbage. Essentially, it endeavors to render visible the actant-​networks that condition acts of recycling in America. Researchers attached traceable microchips (i.e., RFID tags) to “different types of trash so that these items c[ould] be followed through the city’s waste management system, revealing the final journey of our everyday objects in a series of real time visualizations” (“Introduction to Trash|Track”). Items of trash are pivotal actants in the environmental issues about which we propose public policies. And yet, we know very little about the afterlife of our things, once we have thrown them away. Where exactly does each trash-​thing end up and how exactly does each item reach its final destination? Trash|Track translates the circulation of actual trash items into a real-​time data visualization (Figure 5.1). RFID tags are mobilized to generate/​invent networked multimedia from trash-​things, which each become cast in the role of a rhetor. Every pixel that is composed on the screen has been produced via the actions of various actants as they intersect with and propel the movement of the trash. These actants include the machines and employees at waste management facilities, the vehicles that transport trash, the highways and airports that structure their journey, etc. Each one of these actants, consciously or not, becomes embellished with a coauthorial function by which they shape the (re)composition of the real-​time data visualization. The visualization is the writing of the actant-​network. More than a laboratory experiment, Trash|Track

Figure  5.1  Data visualization mapping the circulation of garbage from the project Trash|Track. © MIT Senseable City Lab, 2009.

519

Forms of Actionable Media  • 

159

visualizations have been set up as installations at several museums and libraries to address public audiences. Commenting on the results of the project, Team Leader Dietmar Offenhuber noted, “The extent and complexity of the network of waste trajectories was quite unexpected” (“MIT Researchers Map the Flow of Urban Trash”). For instance, the most surprising (and disconcerting) discovery came when his team tracked a printer cartridge that traveled 3,823 miles in order to reach the recycling facility to which it had been sent. Traces like these complicate societal inclinations to regard recycling as an intrinsically eco-​friendly practice. The wasteful means—​3,823 miles of gas-​g uzzling transport—​sometimes contradict the well-​intentioned ends. Visualizing the actual outcomes of existing actant-​networks—​which are simply illegible without action-​ oriented tracking techniques—​encourages us to revise those actant-​networks and, more broadly, to reflect on the (dis)connections between apparent values and evident results. Placing RFID tags onto garbage is, in this sense, an act of rhetorical invention that bears political weight. The discarded printer cartridge becomes a digital rhetor, producing traces of an entirely different order than the ink marks it once made. When it was part of a printer, hooked up to a desktop or laptop, the cartridge fulfilled the commands of a human user, impressing onto paper whatever onscreen marks had been created, saved, and verified within the parameters of a word processor, image editor, or spreadsheet program. After consumption, now equipped with tags, the cartridge acts as the Rosetta Stone of a posthuman writing machine. By tracking cartridges, tags reveal the path of each cartridge, but also—​more importantly—​ they map the actant-​network that conditions and ferries every inch of the cartridge’s journey. In ever-​globalizing societies, such cartographies are invaluable and increasingly untenable to achieve through conventional, anthropocentric means. Whereas critics tend to see actionable tracking purely in terms of its hegemonic potential for surveillance, Trash|Track demonstrates promising affordances of tags that encourage heightened collective awareness and robust forms of deliberative rhetoric. The project “allows individuals to monitor and describe their environment, while also providing an insight into the impact of their own actions” (“MIT Researchers Map the Flow of Urban Trash”). As an innovative case of actionable media production, Trash|Track orchestrates a rhetorical-​aesthetic-​political performance on the basis of the real-​time actions of actant-​networks. Here, nonhuman actants are not only the subjects of texts composed to draw attention to the vibrancy of inorganic matter; rather, things are themselves made to double as digital rhetors. Actionable

610

1 6 0   • Act i o n a b l e   M e d i a

tagging practices, when conceived critically and ethically executed, afford the parliament of things a more prominent (and indexical) place in deliberative democracy. Though visualizations like Trash|Track, traces of an actant-​ network may be transcoded to exert a voice in public policy debates. Imagined from this angle, Web 3.0 tags can become more than just a strongarm method for soft surveillance. They also represent a medium through which to widen and flatten the domain of rhetorical agency in practice, not only in theory. But there is a further, equally profound manner in which ATLAS figures assign rhetorical-​aesthetic capacities to vibrant matter in motion. This second dimension comes to the fore as we turn attention toward tagging techniques associated with image recognition, which is quickly becoming the preferred method for arranging layers of digital content amid the lifeworld.

Layers Web 3.0 tags carry a rhetorical capacity to transform objects into sites of inscription that transcode actant-​networks into networked traces. Tags also condition the production and circulation of layers beyond the desktop. That is, they give actionable media producers the ability to present texts and audiovisuals in alignment with one’s current sensory perceptions of her extracomputational surroundings. Unlike RFID tags (which support tracking and tracing), tags based on image recognition transform objects into surfaces for storing and projecting layers of digital content in situ. A vital area of practice where such layers are manifesting is augmented reality (AR). AR layers depict a further way in which media is becoming actionable. Chapter 2 discussed how turn-​by-​turn GPS navigation devices prefigure actionable media in that they generate a running text which effectively senses drivers’ nondiscursive movements, translates them into data, and actuates pertinent audio/​visual cues in real-​time relative to precise locations. GPS navigation is about more than site-​specificity, for it does more than just attach texts or media to a given place. Likewise, AR is not only about creating multimodal texts for users to access at a particular place; more crucially, recalling Stiegler’s terminology, it marks a form of techno-​geographic textuality. AR layers incorporate geographic flows as a vital element in the ongoing composition of the textual field. Whereas GPS navigators, akin to Trash|Track, produce virtual maps that trace bodies in motion, AR layers envelop the local action of proximate materialities, cosigning to them a sense of rhetorical agency as in-​text occupants intrinsic to the very structure of a composed work.

16

Forms of Actionable Media  • 

161

Those who have set out to create or analyze layer projects will recognize how disruptive this emergent form is to conventional notions of text and context, which have generally served as a crux for rhetorical thinking relative to all sorts of print and digital exigencies. As such, AR layers clue us into an important aspect of the paradigm shift from personal computing to ubicomp, one that suggests a broader departure from deferred archival traditions. For all their differences—​and digital rhetoricians have aptly pointed out many—​desktops, laptops, and the browser-​based Web inherit some fundamental assumptions about reading and writing from books, magazines, and scrolls. Namely, all of these interfaces enframe texts and detach them from a given, originary context. They bind texts together into a cohesive and iterable format. Within the frame of such interfaces, texts are no longer bound to a specific environment. Indeed, their unbounded-​ness to a particular context becomes a great source of value. They circulate freely and retain their basic orthographic structure. On the other hand, the ubicomp research agenda takes its cue from billboards and street signs. It is in these genres that Weiser locates “the real power of literacy.” Such texts are materially bound to specific contexts, and they interlock with local action, often influencing it in subtle, yet decisive ways. Contrary to books and websites, their bounded-​ness to particular settings is the source of their rhetorical power. Similar to these marginalized genres, layers—​the basic unit or figure of AR textuality—​are fundamentally indissociable from the real-​time activity of nearby geographical flows. Local surroundings are not just the context in which we experience AR texts. Once an electronic file enters the actionable media ecologies of AR, that file becomes part of a dynamic techno-​geographic milieu. AR textuality lies definitively in excess of the digital content it delivers; any attempt to isolate and privilege the electronic file as the text subjugates the medium to the logic of the page or the screen. Writing and reading qua AR does not occur on a screen. It occurs on a screen-​camera. The digital content included in an AR layer occupies only a portion of the screen-​ camera, which always distributes the composition of the textual field to other sources of agency and action: the kinetic movements of “cinematographer” (not just a viewer) holding/​wearing the screen-​camera and the autonomous flux of proximate actants that occupy the transparent textual field. Scholars writing about AR, however, often fail to acknowledge this crucial point of convergence between screens and cameras, between networked digital writing and live indexical recording. This omission is surprising, given the fact that the appearance of layers over a transparent writing space seems unprecedented, or at least conspicuously odd. Throughout the history of

612

1 6 2   • Act i o n a b l e   M e d i a

manuscript and print culture and even the Web, the overwhelming majority of texts and images have rested on an opaque writing space. To experience multimedia via a transparent lens constitutes a departure from readerly experiences associated with the static, virtuality-​oriented surfaces of inscription that have dominated the history of writing and computing. Book pages, canvases, and even websites employ a fixed background color or image to establish a perceptible contrast with the inscribed text. More often than not, writers hope that readers will become immersed in their writing, regardless of where the reader is and in spite of whatever happens around him. Museums typically keep gallery walls as sparse as possible, in order to emphasize the artworks inside the frames. As readers and viewers, we often try to block out the surrounding environment, regarding it as a source of potential distractions. In short, much of our reading and writing aspires toward total immersion, not unlike virtual reality. It has taken place (as if ) in the library, the office, the study, the classroom à la the logic of the studio—​not en plein air. When critics endeavor to understand AR as a genre of locative media, they emphasize the situated quality of networked digital writing, but neglect the live indexical recording aspect inherent to AR’s screen-​camera. This screen-​ centric neglect of the camera can lead to unwarranted dichotomies or red herrings, and it reinforces the need to recognize the figurative, actionable character of layers in our analyses and compositions. Writing about locative narratives, which sometimes involve AR, Christine Paul cautions that “very rich media content and traditional narration . . . might run the risk of competing with the audience’s awareness of and focus on the site itself ” (410). Writers, artists, and designers should aim, then, to produce texts that do not block out the contexts of their reception. And yet, backing off from her own guideline, Paul wonders if ubicomp users’ ability to identify and concentrate on individual texts will not be helplessly compromised by what she calls “contextual noise” (415). She takes this dilemma to be a dire challenge to artistic and cultural expression in the age of ubicomp. We might imagine a layer of electronic poetry grafted across the space of a busy playground: children screaming and crying and laughing, helicopter parents barking out orders, blinding rays of sunlight glaring off a metallic slide, etc. If our objective is to read the text in a traditional figure-​ground sense, then this would not be an ideal reading environment. There is too much contextual noise, according to Paul’s logic. On the other hand, what if we read this hypothetical layer project the way we read a street sign? Approached in this manner, the layer is not a cocoon for textual immersion; rather, it serves as a literary lens for reading the present

613

Forms of Actionable Media  • 

163

scene and reflecting upon the action taking place there. Layers offer up texts for reading in order to facilitate a deeper reading of locales and local actants perceptible within and around the screen-​camera. The “text” can never be the sole object of contemplation because it is resolutely written with/​in “contextual noise.” So-​called contextual noise is as much a part of the text (and as much a stimulus for deep reading) as the digital elements. With Paul’s analysis in mind, I contend that—​as long as we maintain strict text/​context oppositions—​we will be caught in the catch-​22 toward which her argument points. The more fundamental challenge is to learn how to conceptualize, design, and analyze layers without recourse to text/​context oppositions. Akin to the notion of kaleidoscopic composition evident in French Impressionism (which I describe in ­chapter 6), AR layers beckon theorists and practitioners to consider the integral role of sensible plenitude in actionable media projects. Layers, outside of WIMPs, are more than a category of electronic textuality. Layers comprise a new attentional form wherein proximate arrays of contextual noise become rhetorically adopted as a constitutive element within the work’s visual, sonic, or textual frame. Therefore, let us attempt to describe the rhetoricity of some layer projects without relying on traditional text/​context barriers. Though I will emphasize visual images in examples below, layers clearly embody what Maria Engberg terms a “polyaesthetic” design space: “a combination of sight, hearing, touch, and proprioception” (Bolter et al., “Media Studies” 44). Consider, for instance, the polyaesthetic role of sensible plenitude in the following sound installation. Sonic artist Abby Aresty’s installation Paths II: The Music of Trees (2012) creates an embedded sound layer throughout Washington Park Arboretum in Seattle. Aresty spent roughly a year recording audio at specific spots in the Arboretum, then composed seven tracks comprised of noises from the Arboretum’s soundscape. For the project’s exhibition, seven speakers were installed in the Arboretum at the respective spots from which each track was drawn. Visitors heard Aresty’s compositions amidst whatever noises (and other sensory elements) happened to occur on the spot. Aresty’s aesthetic task, as she describes it, was to “bring these simultaneous yet distinct layers of sound into dialogue with one another.” The phenomena that Paul would have us regard as contextual noise (e.g., sounds, sights, tactile sensations) permeate the audio of each sound layer, as well as Aresty’s own conception of her composition process. More precisely, the sound layer is composed of the audio Aresty edited plus the range of noises she learned to anticipate. The sound layer that each piece is is a singular performance that accrues at the interstice of the composed audio and the spontaneous aural-​visual-​tactile sensations

614

1 6 4   • Act i o n a b l e   M e d i a

of blowing leaves, wildlife, nearby auto traffic, air traffic, fellow park visitors, etc. Aresty insists, “The pieces bring attention to the Arboretum’s dynamic soundscape, but also depend on it to bring new life to each listening.” Closing one’s eyes and ears to the contextual noise—​straining to listen exclusively to the electronic audio—​would be the equivalent of bringing sheet music and earplugs to a concert hall. By the same token, Aresty’s reflections suggest that the local actions and geographical flows characteristic to the scene of a layer should be keenly observed during the creative process. At the time of writing, some of the most ambitious AR layers have been created by museum professionals and digital artists whose work hinges upon historical representation. Collectively, these works harbor a distinct sense of temporality that is predicated on the way layers reconfigure and disrupt text/​ context relations. Not unlike Paul, Jason Farman draws text/​context distinctions in his analysis of historical AR projects such as the Museum of London’s Streetmuseum (2010), and he portrays the medium as a digital extension of text panels that provide commentary at landmarks and museums. The key rhetorical affordance of mobile AR, Farman argues, is the medium’s capacity to display site-​specific information that affects users’ sense of implacement (39–​4 0). He claims that the aim of historical layers like Streetmuseum is to lend “further context” to our encounters with places (44). AR layers do inflect perception in this manner. More radically, though, layers—​in conjunction with the other ATLAS figures—​also transform the very constitution and composition of textuality. They don’t just amplify our ability to consult online texts; they’re not simply a means to add contextual information to the margins of our lived experiences. As we endeavor to produce and analyze layer projects, the rhetorical vitality of context at play within the text—​the ways in which texts involve contexts in the hermeneutic event—​beckons a more fluid, nondualist approach. Thus, as an alternative to the text panel metaphor implicit in Farman’s prose, AR layers might be conceived as brokering a real-​time, singular documentary that each viewer-​cinematographer lives as they inhabit locales with their screen-​camera. The latter emphasizes the live action occurring around and within the visual frame, the textual field, the aural milieu, etc. We may employ this metaphor to generate different descriptions of the actionable media ecologies at hand in historical layers. Consider, for example, the screenshot of Streetmuseum shown in Figure 5.2, which was taken from the perspective of an AR user standing in front of the Museum of London Docklands. What interests me most about such examples is not site-​specificity—​ not space but time. Granted, layers of this sort attach digitized archival

615

Forms of Actionable Media  • 

165

Figure  5.2 Screen capture from Streetmuseum iPhone application, overlaying the Museum of London Docklands. © Museum of London, 2010.

photographs to precise geographic coordinates; in doing so, however, I posit that layers also conjure a peculiar modality of time in which to exhibit, encounter, and study the historical past. When archives take to the streets, they alter the lived present, too. This is what makes looking at AR layers different from (and more rhetorically significant than) holding up a photograph of the same image. Owing to a layer’s transparency and boundedness to local action, the archival image and the viewer’s contingent surroundings are both positioned on equal terms within the visual frame. Layers extend the frame beyond the contours of the photograph, such that the image bleeds into whatever is happening around it, and vice versa. A real-​time documentary unfolds for each viewer-​cinematographer. In the screenshot above, the man standing on the cart gazes at the girl leaving the museum; the woman seated to the far left watches the crew lower (or raise?) bags of sugar to/​from the cart. When layered via AR, the image and the scene begin to cross-​pollinate one another. What is it about the girl that holds the man’s gaze, as she walks out of the museum and through the sugar warehouse, toward us? Remember, the scene is animate: we witness the image as a part of the unfolding present. Every passing moment brings new actants into the textual field—​a looming cloud, a shadow, a woman seated at a table—​and they become cast into the historical layer by virtue of their position within the frame. An ever-​changing

61

1 6 6   • Act i o n a b l e   M e d i a

cast of actants passes into the image like a live stream, brushing associations onto the image, the work of the crew, the man’s gaze. A dramatic network of relations between past and present ensues. In pointing to a few of these relations, it is beyond my purposes here to pursue an interpretation of London Docklands. I merely wish to indicate how historical layers impart new conditions for encountering archival materials. Bearing witness to the past through AR furnishes a spacetime that differs from traditional forms of historiography, whereby the historian enters into the (deferred) archive, makes connections and inferences among primary sources, and reports back to her readers/​ viewers qua a text or film that is structurally unbound from the actual scenes and artifacts it describes, narrates, and explicates. The porous frame of the screen-​camera transposes the archival photograph into a documentary performance. The past radiates out from the image and the present infiltrates the image. Time passes in both directions all at once. While local, live action grafts onto the past, the present scene becomes held under the surveillance of history, such that current activity can be seen and judged in immediate relation to an actionable archive of particular historical events. More politically charged historical layers might employ this tactic to highlight the persistence of systemic injustice among institutions that would rather showcase themselves as engines of progress. Historical layers foreground our responsibility to the past, and may counteract attempts to revise, whitewash, or diminish history. Popular modes of representing past events (e.g., timelines in textbooks, museum exhibits, texts panels or photographs encased at the base of a landmark) do not prompt the same degree of temporal fluidity, or at least they do not conventionalize it. The same photograph of the work crew, were it encased at the museum’s entrance, would lend itself to another way of seeing, another way of gauging time. Inhabiting a frame unto itself, the image would demarcate a point in time formally removed from the present scene. Any relationships to be constructed between the framed historical event and the present scene must be fabricated in deferred time. They would contextualize each other in the manner of footnotes. They would not hold a conversation in real-​time. From Lewisburg, PA to Silicon Valley (2012)—​a collaboration between artists John Craig Freeman and Lily and Honglei documented in Figure 5.3—​ mobilizes a matrix of history and geography that is decidedly more complex and presumably more self-​aware than Streetmusuem. Streetmusuem takes a literal, paint-​by-​number approach to arrangement and delivery:  the image layers are positioned so as to overlay the buildings they document. From Lewisburg, PA to Silicon Valley, as the title indicates, enacts figuration through

617

Forms of Actionable Media  • 

167

Figure  5.3  Screen capture of the augmented reality artwork From Lewisburg, PA to Silicon Valley, overlaying the Apple Campus in Cupertino, California. © John Craig Freeman and Lily & Honglei, 2012.

montage: its visual layers assert a metaleptic correspondence between the history they represent and the present scene they occupy. The juxtaposition between composed multimedia and the indexicality of the scene is, in this case, visually striking but conceptually puzzling. Sketched images of plainly dressed young men and women float in the air outside of Apple Inc. headquarters. Their bodies are upside-​down and contorted. Lacking any notable facial expression, they appear to be drifting whimsically through the scene like plastic bags caught in the wind. In notes on the piece, Freeman et al. describe the city of Lewisburg’s golden years as a manufacturing mecca during the postwar era. Starting in the 1970s, however, a critical mass of American companies began to move their facilities away from unionized towns like Lewisburg to “right-​to-​work” states, then internationally to Mexico and China, in pursuit of ever-​lower operating costs. Coupling this labor migration with the fact that China is now the world’s leading manufacturer of consumer electronics, one sees in these floating teenagers an intricate constellation of political and economic outcomes in which companies like Apple—​and digital media consumers like us—​are thoroughly implicated. As the user holds her iPhone up to the Apple headquarters building, she beholds the displaced American workers whose factory jobs have been outsourced—​ and whose relatively high quality working conditions have been sacrificed—​ for the sake of higher profits and lower prices. The layer creates an exemplary critical collage of global capitalism that enmeshes here with there, me with

618

1 6 8   • Act i o n a b l e   M e d i a

them, and now with then. This affect transpires in the simultaneity of layered time, not in linear time. Connecting the emergence of layers with persistent themes in media history lends theoretical weight to the proposition that this technocultural form conditions and conventionalizes an otherwise marginalized modality of time. The introduction of the line qua linear writing serves as a relevant analogy. Referencing ancient Greece, Vilem Flusser ventures that the formation of historical consciousness was tied to the development of contrasting discursive geometries across oral to literate cultural practices. The difference between history and prehistory is a matter of lines and circles: “Only one who writes lines can think logically, calculate, criticizes, pursue knowledge, philosophize . . . Before that, one turned in circles” (Flusser 7). The orthographic line permits re-​reading, reflection, and revision, whereas the self-​referential circles of oral memory cannot be stepped out of without dissolving. In referring to these “dizzying circles of preliterate consciousness,” Flusser is no doubt indicating the habit of immersive recitation that was/​is the cornerstone of collective memory in oral societies, as well as the cyclical patterns of the seasons that structured ancient agrarian life (7). The transcoding of the line from the ancient workplace (the farm) to the page parallels the transcoding of the desktop from the modern workplace (the office) to the screen.4 Having become a prominent figure of literate cultural archives, the symbolic power of the line reaches beyond the functionality of the page, becoming the basis for linear processes of thought, temporality, and social organization. Experiments in wearable computing make it viable to imagine futures wherein layers of multimedia experienced via head-​mounted displays could become as axiomatic to actionable media as lines of words have been to the written page. If one entertains this analogy further, then it follows that the cultural production of layers might someday instantiate new variants of temporal experience and historical consciousness. Learning about history through chronologically structured textbooks reinforces linear time. Learning about historical events through AR layers breeds layered time. In linear time, the past leads to the present and events occupy distinct points on a timeline, abstracted to some degree from the circumstances in which they occurred. Layered time binds the representation of an historic event to the place of its literal occurrence (or a place of figurative correspondence) mediated through the present moment. Whereas linear time tends to encourage histories structured by causality, progress, and evolution, layered time favors the resonance and juxtaposition of collage and montage. Layered time is coordinated through associative leaps rather than successive days and years.

619

Forms of Actionable Media  • 

169

Actuators and Sensors If tags are the figure constitutive of smart objects, actuators and sensors jointly underwrite ad ​hoc networks among such objects, to the extent that they may be regarded as the key figures of smart environments. All of the ATLAS figures work in conjunction with one another, but actuators and sensors are practically an indivisible pair. The dynamic between them resembles the letters “Q” and “U”: like Q, actuators tend to appear only when coupled with sensors; like U, sensors may be used apart from actuators, in which cases they simply collect data without implementing any actions in response. My analysis here privileges examples of the former instance, whereby sensors sense and actuators act on the basis of those senses. More precisely, sensors scan their surroundings for specified variables, perpetually filtering through local action in search of meaningful signals. Whenever a sensor perceives meaningful activity, that information is sent to actuators, which perform computational operations selected from the array of options stipulated by a predetermined program, or acquired over time through exposure and repetition via machine​ learning techniques. Accounting for the rhetoricity of actuators and sensors, I admit, leads us into descriptions that seem unduly technical or mechanistic. And yet, this impression may be a symptom of insight. With actuators and sensors, artificial intelligence happens at street level. The built environment starts to read complex situations, write changes to its own structure, and interpret phenomena by means of an enculturated subconscious riddled with associations, memories, and values. Experientially speaking, humans become the objects of rhetorical thinking that emanates from machine-​to-​machine conversations. Alien as they may be, these ambient discourse networks are made legible (and even show themselves to be within the purview of our rhetorical prowess) when we grasp the figurative gestures characteristic to actuators and sensors and map their performances in conjunction with apps, tags, and layers. Actuators and sensors enable actionable media producers to establish smart environments in two main ways. They can be embedded in handheld or wearable devices, in which case users will perceive the changes wrought to their proximity on the device’s screen-​camera (qua a looking-​glass unto otherwise invisible data). In other cases, actuators and sensors are embedded directly into public infrastructure, such that people does not need a particular device to experience the rhetorical activity these figures perform. The changes wrought are written into the extracomputational lifeworld. Both of these modalities pervade everyday life in ubicomp cultures. In keeping with the

710

1 7 0   • Act i o n a b l e   M e d i a

previous section, AR provides a salient arena for tracing basic implications that actuators and sensors carry for communication and perception in handheld/​wearable scenarios. After that, I  will turn to public interactive installations, in order to examine the rhetoricity of actionable practices whereby actuators and sensors inhabit material structures anchored in urban locales. AR layers—​ accessed via smartphones, tablets, and head-​ mounted displays—​reveal themselves to the human sensorium only after being registered and processed by a device’s sensors and actuators. This technical circumstance bears an important rhetorical if not ontological consequence, which we can surmise only by working through the essential functions of sensors and actuators. Optical sensors, in the case of AR, convert images and other perceptual data into electronic signals that, if recognized by the app, become interpreted as a command, thereby prompting a software-​based actuator to execute a programmed action in direct response to data gathered by the sensor. This series of action only occurs, however, after AR writers have arranged content for delivery via quintessential Web 3.0 tags, and will vary depending on which type of tag they choose. Geotags enable us to host networked multimedia at precise longitudes and latitudes; in such instances, the sensors on our devices work with GPS satellites to triangulate our position relative to geotagged content. Another technique for tagging, computer vision, requires AR writers to upload images into app-​based authoring programs and link their digital compositions to those images. When an AR app recognizes these images amid the built environment (in the form of signs, posters, pages, etc.), the sensors on the device will scan the image in order to detect any data that has been linked to the image. Sensors are constantly scouring the surrounding environment via GPS and/​or computer vision. If the sensor perceives X phenomenon, then the actuator will perform any actions associated with X. If the sensor perceives Y phenomenon, then the actuator will perform actions associated with Y, and so on. Together, sensors and actuators constitute the machinic reading and writing processes always active at the periphery of one’s attention (whenever AR apps or wearable displays are activated). The underlying role played by sensors and actuators in actionable media ecologies cast the if/​then logic of computer programming over our engagement with what is here and now. To see with computer vision is to perceive the world at least partially through the lens of if/​then logic. Digital creatives who design layers for AR, therefore, must arrange and deliver their compositions on the basis of embedded tags that address the computer’s eye and mind—​sensors and actuators—​in route to the human sensorium. Re+Public’s AR|AD Takeover (Figure 5.4), an AR

17

Forms of Actionable Media  • 

171

Figure  5.4 Screen capture of the augmented reality project AR|AD Takeover by Re+Public, overlaying Times Square in New York. © BC Biermann and Jordan Seiler, 2011.

project rooted in image recognition, converts existing advertisements around Manhattan’s Times Square into sites for triggering and showcasing street art. Walking around Times Square without an AR-​enabled device (or without knowing about this project), one inevitably entertains a series of associations and inferences upon seeing a movie billboard or product ad. Streams of consciousness activated by texts and images set in city space are not private in every sense; the stimuli are in plain view and the language of our interior monologues is socially inflected. The right to display urban signage, however, remains tightly guarded. But Re+Public’s AR layers, which hover over the billboards when activated, add a new rhetorical dimension into the perceptual-​ cognitive mix. Seeing with sensors and actuators, one views the billboard in the image an association, an inference, or a subversion that Re+Public has crafted and made visible for others who are equipped to view it. The privately controlled ads are appropriated into sites for triggering and exhibiting digital public art. Early experiments with computer vision invite the reckoning of possible futures wherein we may read, write, and revise multimedia amid built environments on a self-​selecting if/​then basis. If I see ads, then show me artworks instead; if I see a company’s logo, then pull up their Twitter feed also; if I see a monument, then play an audio clip that has been tagged to it by an artist, student, politician, or historian (whomever I select). This capacity will amplify current anxieties about the so-​called echo chambers already associated with social media feeds. Still, it warrants further inquiry and experimentation across

712

1 7 2   • Act i o n a b l e   M e d i a

the arts and humanities; corporate-​driven urban mediascapes already propagate echo chambers of their own. Projects such as AR|AD Takeover—​coupled with similar pieces done by Janet Cardiff and Rem Koolhaus described in Manovich’s “The Poetics of Augmented Space”—​collectively imply a spectrum of rhetorical-​aesthetic opportunities for immixing cultural heritage, artistic expression, and intellectual discourse with everyday moments of action, recurring scenes of decision, and public streams of consciousness. In other words, we might strive to regard such artworks as incubators of avant-​ garde interaction modalities ripe for development or appropriation by cultural, civic, and educational institutions looking to make their media projects actionable. ATLAS figures, as we have seen, create conditions whereby an increasing array of objects, beings, and surfaces may operate in the manner of an if/​then hyperlink. This not only multiplies the stages on which media content and online networks can perform, but—​as actuators and sensors make clear—​ it also supports new modalities of interaction among humans, computers, and environments. Layers emerging with the ubicomp paradigm are often accessed through device-​mounted actuators and sensors, which supplement pointing and clicking through positioning, movement, and computer vision. Consequently, the medial surfaces and material structures that comprise a built environment also function as servers for hosting (in)visible digital content. Handheld/​wearable AR aside, other forms of post-​WIMP layers have been cropping up in public interactive installations. David’s Way Plaza in Dallas—​described by its creators as an “urban musical instrument”—​is a network of embedded actuators and sensors located at a high-​traffic intersection along the city’s most popular running/​cycling trail. David’s Way is a representative actionable media project in that it plays or actuates audiovisual layers in direct response to the movements of pedestrians who pass within the range of its sensors, which are housed in metal poles that line the trail. The poles thus function in the capacity of tags, as they furnish sites where digital content is stored and projected. As a commissioned piece of public art, David’s Way also illustrates the multiple purposes typical of actionable installations that inhabit urban spaces. In his proposal, artistic director Christopher Janney describes the audio layers that get generated from the project’s database as containing “a mix of melodic tones and environmental sounds, possibly also texts spoken or whispered” (“David’s Way”). The visual layers, more visible at night, are simply programmed sequences of LED lights that punctuate the audio. In addition to its aesthetic dimension, the project serves a utilitarian interest: the sonic

713

Forms of Actionable Media  • 

173

character of the audio layers cue the trail’s runners and cyclists to the busy street they are approaching. The audio layer, which in most cases evokes a jungle vibe, delivers a gentle shock that—​through cognitive dissonance (why I am hearing jungle noises in the middle of Dallas?)—​can snap people out of a “runner’s high” and provoke them, almost instinctually, to inspect the surrounding environment ahead. Most crucially, from the standpoint of actionable media, public interactive installations of this sort configure multimedia to sync with one’s live actions and, in so doing, strive to engender a certain attunement to the activity of the actant-​network in which s/​he participates at that moment. Clearly this effect could not be triggered were it not for the actuators and sensors embedded at the scene. Bonding audiovisual layers with the flow of pedestrians, actuators and sensors establish a subliminal system for imparting a state of mind to the proximate audience, instantaneously and without words. McCullough and Hayles have each written insightfully about the ways embedded sensors, actuators, and tags are subtly altering the relationship between humans and built environments. For McCullough, situated or ambient technologies hold a significant advantage over personal mobile devices, at least for civic purposes. Because the former are “physically inscribed” into the public infrastructure rather than sent to-​and-​fro on tiny screens, they support the production of media events that occur in a shared perceptual field and contribute to the local cultural commons (McCullough, Ambient Commons 111). Hayles adds two further themes to ponder, which we may consolidate by stitching together her analysis of telegraphy with her remarks on the “technological unconscious.” Hayles locates the digital legacy of telegraphy in the “machine-​centric” view of communication it conjured, largely by implicating the human as operator and no longer as author: “fewer sending and receiving skills were located in humans, and more were located in machines” (How We Think 146). As I suggested above, this tendency has intensified to great extents in ubicomp cultures, such that people routinely become objects of machine-​to-​machine communication, wherein even the role of conscious operator appears to be designed out of the equation. However, unlike telegraphy and so much computation developed in its wake—​whereby “codes grew more procedural and decontextualized”—​the actuators and sensors that underwrite public interactive installations reconfigure the production of code and computer programs on a more performative and localized basis (How We Think 142). They seem to be writing at and to the present scene, rather then just sending messages through it. Building on Nigel Thrift’s research into contemporary built environments,

174

1 7 4   • Act i o n a b l e   M e d i a

Hayles asserts that the technological unconscious (i.e., either bodily or computational actions that have become automated or simply habitualized as “routines carried on without conscious awareness”) has become unusually adaptive (96–​97). Architectural structures, which have always shaped or affected collective/​social routines of this sort, now stand to become affected by the variety of social, geographical, material, and discursive flows that traverse territories within the reach of a given sensor-​actuator network. The actions of proximate actants, whose behaviors may be more or less cued by a building’s arrangement, send cues back to the building, effectively altering some aspects of the building’s composition (e.g., the multimedia it projects, the arrangement of its wall, the shade of its lighting). Hayles writes, “A change in the environment results in a change in the technological unconscious . . . and that in turn creates the possibility for a change in the content of attention” (98). Such references to a technological unconscious are quite suggestive, and the rhetoricity of actuators and sensors provides an occasion to probe the concept further. Taking language as our oracle, the terms “sensor” and “technological unconscious” point to an intriguing parallel in Freudian psychoanalysis—​ one that makes for an illuminating contrast against which to outline a general outlook on the machinic reading/​writing of sensor-​actuator networks. Freud theorized what he called a “dream censor” in his later work on the unconscious. In A General Introduction to Psychoanalysis, he introduces this new figure of the dream censor by means of personification. The dream censor, Freud tells us, is like a newspaper editor whose sole job is to mark out any passages that might be “not to the liking of the censoring authorities,” which was common around Europe in the World War years (113). During a dream, the dream censor performs this same editorial task on behalf of the conscious mind while one sleeps. It is because of the dream censor—​this elusive “author of dream distortion”—​that some of our dreams are strange and incomprehensible. Freud catalogues the dream censor’s favorite rhetorical strategies: displacement of emphasis, allusions, innuendoes, modification, omission, and the substitution of murmurs in place of contemptible statements (114). Ultimately, the censor intervenes whenever the most taboo of wishes (“the expressions of a boundless, reckless egoism”) would otherwise surface: incest, murder, etc. While dreamers tend to dismiss these distorted dreams, refusing to claim agency for them and foreclosing an analyst’s motion to interpret them, Freud admonishes his professional peers to read into these dreams with the utmost vigor: “If you reject the unpleasant, you are repeating the mechanism of dream construction [the dream censor, in this case]

715

Forms of Actionable Media  • 

175

instead of understanding and mastering it” (119). Hostile to free associate and interpretation, the censor inhibits the unconscious from fully manifesting in the dream state by obscuring the fulfillment of forbidden wishes. How, then, does the censor of the Freudian unconsciousness position us to further describe the work performed by the sensors and actuators of the technological unconscious? Both the dream censor and sensor-​actuator networks play upon the same axis: awareness and repression. Their respective rhetorical inclinations relative to this axis, however, are exactly inverse. The dream censor reinforces enculturated inclinations, repressing “unconscious tendencies in [one’s] psychic life” during sleep (Freud 118). Sensor-​actuator networks incite awareness of phenomena that might otherwise be repressed or neglected on account of normative patterns of attention that correspond with social habitus, or one’s particular routines developed relative to constrains and affordances of the technological unconscious intrinsic to a given built environment. Designing a sensor-​actuator network to be embedded at a particular scene, therefore, entails grappling with questions such as: What patterns of attention and behavior do people tend to exhibit at this scene? If there is a pervasive natural attitude (phenomenologically speaking), then what potential perceptions or actions remain dormant at the margins of those dominant tendencies, automated actions, etc.? What are the stakes of this collective repression or neglect? And what kinds of changes to the environment might rewire the technological unconscious so as to spark different habits or at least reroute attention? These questions are no longer exclusive to architects and urban planners. The democratization of actuators and sensors qua ubicomp technologies opens the technological unconscious for general inscription. To write at the level of the technological unconscious is an attempt to intervene in the collective habits that constitute basic forms of social organization. Such patterns of automated action, conditioned by the composition of built environments, are elemental to modern urban life and influential in shaping the flows and exchanges that characterize particular spaces. Infrastructural media systems have throughout the twentieth century limited the production of embedded sensor-​actuator networks to strictly utilitarian concerns. Electric traffic lights provide a representative example. Invented in 1914, the first electric traffic lights were wired to switches that city employees operated manually from a control booth (“First Electric Traffic Signal”). Major cities began to computerize traffic lights during the 1960s, and we have since witnessed a gradual transition from automated programs of “fixed-​time signals”

716

1 7 6   • Act i o n a b l e   M e d i a

(e.g., timers that switch lights at regular intervals) to adaptive “semi-​actuated signals” (e.g., sensor-​actuator networks that switch lights by detecting the presence of vehicles) (“Toronto Traffic FAQ”). As drivers pull up to a high-​traffic intersection, they are experiencing the execution of computational scripts. Traffic light systems are visual media, situated at and responsive to crucial scenes of decision. They remain one of the most widespread ways in which actuators and sensors have produced and automated consequential rhythms of everyday life. The system is simple. A column of three colors arbitrates the flow of traffic, effecting patterns of social organization that almost never emerge spontaneously when drivers are left to their own devices. Traffic lights, emblematic of other infrastructural media, are indeed a top-​down, state-​controlled affair. The recent spread of public interactive installations, however, attests to the prospects of more diverse infrastructural mediascapes, which will likely manifest in more urban spaces as the sphere of authorship for sensor-​actuator networks continues to widen. Residing at important scenes of decision and action, new sensor-​actuator networks may pave inroads to the technological unconscious, whereby actionable media projects, created with arts and humanities interests in mind, may forge alternative patterns of social becoming—​well beyond utilitarian outcomes. Recent advancements in ubicomp technologies such as AR, the Internet of things, and headworn displays—​the stuff of sci-​fi films and lab experiments—​are now positioned for mass adoption and cultural impact. Amid rampant innovation, HCI researcher Jonathan Grudin insists, “Our best chance to anticipate change is to find trajectories that extend from the past to the present” (xxvii). Rhetorical figures offer a pliable source of theoretical scaffolding for assembling such trajectories. In describing the media ecologies evident in actionable practices throughout this chapter, I have hoped to clarify how ATLAS might serve as a figural framework for analyzing the rhetoricity of post-​desktop mediascapes. Moreover, I have hoped to conjure an initial sense of how one might begin to design, communicate, and write with these figures. Chapter 6 delves further into challenges and affordances associated with ATLAS forms, placing greater emphasis on the perspective and process of production. As the page differs from the screen, so too do the writing/​reading spaces of ATLAS differ from the desktop’s WIMP-​laden authoring platforms and browser-​based reception contexts. Facing this transition, media scholars, public intellectuals, and digital humanists who aim to create actionable media projects must identify and develop stylistic principles suited to emerging exigencies. To which traditions, then, might we turn for rhetorical-​aesthetic guidance?

71

Forms of Actionable Media  • 

177

Notes 1. While the desktop metaphor provides an obvious recent example, Renaissance painting offers another case in point where figures of speech constituted “a system of thought” through which artists approached their work (Lanham 117). Renaissance painters read Quintilian’s Institutio Oratoria and extrapolated from his discussion of rhetorical figures new techniques for depicting human bodies in unconventional ways (Baxandall 122). One can also see the legacy of this figurative logic at play in Cubist portrayals of the body, which are riddled with visual iterations of metonymy, metalepsis, and synecdoche. 2. Throughout the 1990s, leading technocultural theorists identified many important elements characteristic of early PC interfaces and the browser-​based Web. Manovich’s discussion of the database as a dominant cultural form is exemplary in this respect (see The Language of New Media). Jay David Bolter and Steven Johnson each historicized different figures from the WIMP framework; Bolter discussed icons as an element of writing foundational to the graphical logic of desktops (see Writing Space), and Johnson chronicled the role of Windows and Menus as “basic building-​block metaphors” that facilitated the breakdown and transformation of the command-​line (see Interface Culture). These are but a few examples of the larger trend. Many other technical terms and popular buzzwords associated with the Internet and computing got picked up and appropriated by new media scholars throughout the decade (and ever since): protocol, algorithm, glitch, etc. 3. I ask this question out of allegiance to what I believe is a more balanced and constructive stance, which N. Katherine Hayles outlines in her article “RFID: Human Agency and Meaning in Information-​ Intensive Environments.” Hayles writes, “Realizing the utopian possibilities of RFID and minimizing its dystopian features thus requires more than regulation, evasive tactics, and progressive legislation, useful as these may be in the short term. Also necessary are analyses that probe the technology’s deep entanglement with economic structures and political ideologies, and strong counter-​visions that articulate a future worth fighting for. Without such inspirations, we are left in defensive postures that can respond to the technology’s abusive uses but are helpless to imagine how it might be directed in other, more positive ways” (61, my emphasis). 4. In Of Grammatology, Derrida comments on Rousseau’s account of ancient farming, foregrounding a curious relationship between farming and the development of the line in writing: “It is a matter of writing by furrows. The furrow is the line, as the ploughman traces it . . . The furrow of agriculture, we remind ourselves, opens nature to culture . . . writing is born with agriculture” (287). The line had been a structural element of the farm since the dawn of agrarian societies; hence, from a purely geometric perspective, the line is not new when it manifests on the page in the form of written lines. (Lines had long shaped the way people went about farming.) The cognitively transformative dimension of lines did not unfold until they became

718

1 7 8   • Act i o n a b l e   M e d i a

incorporated into one of the primary cultural technologies for communication, expression, memory, numeration, and economic exchange. As soon as it becomes an axiomatic constituent of the written word, the line starts to shape and orient the bulk of human activities, eventually disrupting the circular habits of thought so prevalent in oral societies.

719

6

C R E AT I N G A C T I O N A B L E   M E D I A No one publishes a city, they publish a magazine or a book or a website. If we’ve thought about our readers reading, we’ve imagined them at a breakfast table or curled up on the couch or sitting in their office. No one knows how to create words and pictures that are meant to be consumed out there in the world. — ​A l e x i s M a d r i g a l , “The World is Not Enough”

All of the ATLAS projects discussed in c­ hapter 5 have something in common: none of them were created by humanities scholars or by students studying a humanistic discipline. Museum professionals, city archivists, fieldwork-​driven scientists, tech-​savvy architects, and public artists have proven to be the early adopters of actionable media production. Collectively, they command sufficient skills with digital authoring software as well as a desire to make their work more public and, in some cases, to make their research/​creative processes more collaborative or crowdsourced. Most humanities fields dealing with media and communication now play host to similar initiatives calling for computational methods, open access, and collective knowledge b​ uilding. Why, then, do we seem slower to attend to these new directions in digital cultural production? At the very least, why do we harbor a reticence (relative to other disciplines and institutions) to experiment with ATLAS forms in our research, writing, and teaching? Certainly, it is not for a lack of interest in digital production. Many new humanities graduate programs founded in the twenty-​ first century bill themselves as “theory-​ practice” hybrids. For instance, the University of California Santa Cruz’s Ph.D. program in Film and Digital Media “challenges the traditionally conceived borders between creative and critical practice  .  .  .  The program enables potential dialogue between creative practice and theoretical knowledge as related forms of intellectual work” (“Ph.D. Program in Film & Digital Media”). In addition to regarding communication technologies as objects for academic study, the Ph.D. in Communication, Rhetoric, and Digital Media at North Carolina

810

1 8 0   • Act i o n a b l e   M e d i a

State University prepares its students “to actively engage digital media through research, criticism, production, and practice” (“Communication, Rhetoric, and Digital Media”). Similar doctoral programs are thriving at Clemson University, Georgia Tech, and the University of Colorado-​Boulder. This trend in graduate education reflects budding changes in scholarly communication aptly theorized in Kathleen Fitzpatrick’s Planned Obsolescence, and exemplified by prominent online journals (e.g., Vectors and Kairos) that invite scholars from various fields—​not just new media studies—​to make their arguments in nonlinear, multimedia formats. So-​called “born-​digital” book series, managed by major university presses including Michigan and Stanford, challenge authors to reimagine the shape of their lengthier pieces. And, of course, the growth of digital humanities continues to add to scholars’ arsenals new genres of critical expression, often supported by national grants. In addition to writing books and articles, digital humanists build digital archives, annotated virtual maps, and tools to aid online research.1 Canadian universities have been especially quick to institutionalize this “research-​creation” paradigm; the past decade has seen the launch of Carleton’s Hyperlab, Concordia’s SenseLab, Toronto’s Critical Making Lab, and Waterloo’s Critical Media Lab. Moreover, as more scholars test such genres in their own work, a push toward digital humanities pedagogy has some undergraduate students creating digital projects about as often as they write papers. Rhetoricians teaching in departments of English and Communication routinely offer required courses in multimodal composition, media authorship, and information design. From professional research to classroom pedagogy, the humanities are gradually becoming a critical and creative enterprise. Advances in electronic publishing invite us to experiment with intellectual discourse. A growing number of humanities scholars seem willing to adopt a degree of stylistic daring long championed by the likes of Roland Barthes: “intellectual discourse . . . is compelled to consider itself, in a certain fashion, as an object of poetics . . . to abolish the hierarchical distance between ‘creator’ and ‘commentator’ ” (The Rustle of Language 173). Traditional formats are still paramount and irreplaceable in many cases, but they will likely become less of a disciplinary compulsion as entrepreneurial-​minded scholars continue to demonstrate the viability and merits of emerging, born-​digital genres. For twenty-​first c​ entury humanists, especially those trained within this milieu, the cross-​platform production of scholarship will be riddled with questions of media aesthetics.2 When asked “What are you working on?” they will describe both their topic and their chosen medium, as there will likely be several bona fide forms of scholarly communication to choose from.

18

Creating Actionable Media  • 

181

Actionable Media in the Humanities While the majority of digital projects built across the humanities remain anchored in the desktop metaphor and contemporary iterations of the deferred archive (e.g., the Web qua virtual library), a handful of scholar-​ designers have ventured into the realm of actionable media production. The in situ digital archives and geospatial annotations created by this latter group employ ATLAS interfaces to arrange and deliver multimedia on the grounds of historical monuments, cultural heritage venues, and tourism destinations. Such compositions reside upon these public sites as if they were translucent pages, stages, or canvases. Funded by an NEH grant, new media professor Jay Bolter headed up a cross-​disciplinary effort to create Voices of Oakland in 2005, an augmented reality layer project that laces Atlanta’s Historic Oakland Cemetery with actionable monologues. Cemetery visitors use their smartphones to access audio clips and images that have been tagged to specific graves. For the most part, these audio clips perform excerpted passages from letters, diaries, and other historical records (think Ken Burns) with the intent to “educate visitors about historically and culturally significant events related to the deceased inhabitants of the cemetery” (Dow et al. 1). Communication professor Brett Oppegaard’s Fort Vancouver Mobile App (2011), a similar project also funded by the NEH, houses a wide range of audiovisual materials (e.g., original maps, archived photographs, filmed reenactments, park ranger tours) pertinent to the Fort Vancouver National Historic Site. Drawing on notions of kairos as opportune proximity, Oppegaard locates the success of this award-​winning app in its rhetorical capacity to filter and present content relative to one’s surroundings: “we can give [visitors at the site] any kind of media that we possibly could create at the moment that they’re most interested in receiving it” (“The Fort Vancouver Mobile Project”). This actionable relation to electronic textuality differs markedly from more celebrated digital humanities projects (e.g., The Walt Whitman Archive or Index Thomisticus), which essentially convert a print-​based deferred archive into a digital one that is more amenable to machine-​aided “distant reading” methods. Projects like Voices of Oakland and Fort Vancouver demonstrate the viability and potential merits that may follow from the actionable composition of humanities research that meets public audiences where they are, be it for purposes of historical exposition, informal education, theoretical speculation, or social critique. Additional projects in-​progress at the time of writing pursue a more subversive path. For example, Brian Greenspan’s app Lansdowne Revived aims to galvanize local resistance against the recent commercialization

812

1 8 2   • Act i o n a b l e   M e d i a

of Ottawa’s historic Lansdowne Park; the app will reframe the Park’s “revitalization” in light of the destruction that new developments have wrought, with the intent to “reveal how diminished our own present reality is compared with these forgotten and repressed alternatives” (“Hyperlab Projects”).3 The University of Florida’s TRACE innovation initiative is spearheading the creation of “augmented reality criticisms,” which “use AR to modify and transform dominant narratives about objects, subjects, sites, and historical moments” (“ARCs”). SeeWorld, a TRACE project created by Melissa Bianchi, Sidney Dobrin, and Jacob Greene, infiltrates the space of SeaWorld-​Orlando in an effort to clue visitors into the problematic animal captivity practices occurring throughout the park. Grafting its audiovisual layers onto specific exhibits, the SeeWorld app effectively localizes broader polemics about the unethical treatment of marine species (e.g., the 2013 documentary Blackfish), as it buttresses its rhetorical appeals upon evidence accruing in the here and now (Greene). Whereas newspapers, magazines, radio shows, documentary films, blogs, podcasts, and social media have all ushered the rise of public intellectuals, ATLAS interfaces beckon a new ideal: the kairotic intellectual. Kairotic intellectuals, as I will suggest in the Epilogue, create work in the capacity of actionable media, such that their humanistic inquiries not only address a general audience, but also interface with local action amid highly public settings. While artists and curators may lean on a lineage of actionable practices from kinetic sculpture to earthworks, the humanistic disciplines lack comparable, homegrown reference points for authoring texts anchored in locales. But a groundswell in actionable media production is nonetheless rippling across the humanities. In the interest of cultivating humanistic approaches to actionable media production, this chapter mines some foundational stylistic principles from the creative practices and aphorisms of artists and writers who took to the streets, as it were. They adopted actionable modes of invention and dissemination over and against the default—​deferred—​venues within which work in their disciplines conventionally gets produced, performed, and exhibited. From a long list of candidates, I have picked three figures: Claude Monet, Augusto Boal, and Janet Cardiff. Their respective oeuvres, which span from painting to theater to sonic storytelling, variously fashion techniques for channeling the contingencies of proximity into their invention processes and/​or their works’ reception contexts. As such, they provide digital humanists, critical designers, and research-​creation teams with provocative starting points for thinking through key rhetorical-​aesthetic issues such as scene selection, audience analysis, and the strategic arrangement of multimedia within particular ecologies of local action.

813

Creating Actionable Media  • 

183

Principles of Figuration The following analyses of Monet, Boal, and Cardiff are motivated by two ideas from Barthes. First, his conception of style (outlined over a series of essays that helped spur the 1960s revival of rhetoric in France); second, his enigmatic distinction between figuration and representation. (Monet, Boal, and Cardiff are masters of figuration.) In “Style and its Image,” Barthes proposed a view of authorship that anticipates the ethos of remix culture and its now-​familiar critiques of romantic individualism. Whereas his intertextual readings of literature stress the profound degree to which a text’s meaning is enmeshed with its relations to other texts, Barthes also posits a similar process occurring at the level of form, in the act of speaking or writing. Stylistic innovation is resolutely a matter of navigating, negotiating, and appropriating “the depositaries of culture”—​a question of cultivating influences rather than shedding them off in pursuit of one’s own style (Rustle of Language 99). Barthes thus charts a course for rhetoricians and aestheticians who study style: what should govern the stylistic task  .  .  .  is the search for models, for patterns . . . and what should animate this task is the conviction that style is essentially a citational procedure, a body of formulas, a memory (almost in the cybernetic sense of the word), an inheritance based on culture and not on expressivity. (Rustle of Language 99, my emphasis)

The literary critic, following Barthes, no longer attempts to locate the unique or personal elements that constitute an author’s style; instead, one asks after the ways in which writers disrupt or refashion established formal patterns that inevitably haunt the scene of writing. “To write,” Barthes insists, “is to let these [stylistic] models come to one and to transform them” (97). Creating actionable media across the humanities requires a robust set of stylistic models with which to work—​models culled from traditions quintessential to our disciplines. Otherwise, we tend to adopt the stylistic conventions of our “closest neighbors,” to use Barthes’s phase (97). Humanities discourse begins to dilute as it breaches past the book and the essay, taking on the default tones of journalists, activists, tour guides, advertisers, museum educators, or whatever profession appears to hold discursive jurisdiction over the genres we wish to venture into. Such stylistic assimilation is evident above in the otherwise successful apps and layers that humanists have created in collaboration with

814

1 8 4   • Act i o n a b l e   M e d i a

informal learning institutions. While the public vitality of these projects is very encouraging, they seem hampered by a tone of “edutainment” characteristic to American cultural heritage sites. (Indeed, I would apply the same critique to layer projects I have done with my students in partnership with museums.) Without a set of stylistic principles, modeled from exemplary humanistic objects of study, I worry that actionable media in the humanities will too readily assume modes of discourse lacking a genuinely critical edge needed to incite deep reflection beyond the passing knowledge one gains from a gallery text panel, a visitor’s brochure, or text posted at a National Park trailhead. Bait-​and-​switch tactics that drive in situ advertising are even further afield. Granted, the creative concerns of en plein air painters and theatrical directors do not mirror those of digital humanists, but I’d much sooner take cues from artists than from marketing gurus. Before discussing Monet, Boal, and Cardiff as stylistic models for humanistic actionable media production, I should also indicate how Barthes’s notion of figuration provides a unifying lens for seeing their oeuvres in an actionable light. Figuration occupies Barthes’s thought most prominently as a term of contrast against which he defines “representation” in The Pleasure of the Text. Although his definitions are fairly bizarre, they become intriguing when he distinguishes the two terms on the basis of temporality and framing. Figuration erects sites of textual pleasure for the reader in the moment of reading; representation, on the other hand, alludes to distant pleasures recounted after the fact. Representation imitates desire, mimetically referencing past events from which the reader’s presence is forever barred, while figuration produces desire within the reader on the basis of his or her present situation. In spite of Barthes’s erotic terminology (desire, pleasure, etc.), this temporal distinction between representation and figuration maps reasonably well over the two archival traditions I traced in ­chapter 4. Deferred archives operate qua representation. Actionable archives signify through figuration. Moreover, presuming that the logic of figuration transcends Barthes’s immediate project (i.e., an erotics of literary reading), we can supplement “pleasure” and “desire” with words like understanding, reflection, deliberation, and critique, in order to imagine figuration as a rhetorical-​aesthetic concept pertinent to humanistic actionable media genres. Barthes attributes the respective forms of desire that characterize representation and figuration to the contrasting ways through which each stylistic modality relates a text to its frame. Under the auspices of representation, the circulation of desire associated with a text or image “remains interior to the fiction . . . nothing leaps out of the frame: the picture, the book, the screen”

815

Creating Actionable Media  • 

185

(Pleasures of the Text 57). Figuration breaches the fourth wall—​indeed, all walls—​from within and without. It not only leaps out at the world but also lets the world in. In defusing the textual frame, figuration thus infuses the text with the performative force of proximate objects and signifies in conjunction with them. Signification à la figuration produces desire or pleasure (and, for that matter, self-​reflection, public deliberation, historical awareness, or environmental knowing) on the basis of present stimuli (no longer) exterior to the frame. Figuration punctures the borders that keep works of representation hermeneutically sealed off from their surroundings. Figuration is techne with/​in the world. Figuration, as a stylistic modality, is intrinsic to actionable media. Readers familiar with Barthes will notice that his distinctions between representation and figuration bear much in common with his more famous theoretical pairings (e.g., work vs. text, readerly text vs. writerly text). Drawing on Barthes’s concepts, my procedure will be to triangulate points of correspondence between this notion of figuration, the creative practices of the artist/​writer in question, and some rhetorical-​aesthetic exigencies relevant to humanistic actionable media production. The latter includes emerging genres for intellectual discourse such as public interactives, augmented reality layers, civic apps, tag/​sensor-​based data visualizations, and critical audio walks. Monet, Boal, and Cardiff each exude stylistic principles that may orient scholars, designers, and students as they create actionable projects.

Kaleidoscopic Composition: Claude Monet’s Way of Seeing Claude Monet may seem an odd stylistic model for actionable media production. Given what I have written about figuration, his celebrated oeuvre hardly poses any challenges to conventional framing or deferred archivization. Monet, like his fellow Impressionists, exhibited his work on traditional gallery walls and his canvases, framed or not, all draw clear borders that distinguish the world of each painting from whatever surrounds it. But Impressionists painters—​spurred on by mobile inventions like paint tubes and the box easel—​did develop the first aesthetic and methods of composition systematically tailored around painting en plein air. That is, while they were not the first artists to leave the studio, the Impressionists were among the first to refashion the act of painting in light of newly feasible circumstances associated with “open air” scenes of production. Monet practiced principles of figuration at the level of perception, invention, and style (but

816

1 8 6   • Act i o n a b l e   M e d i a

not delivery). Accordingly, I focus on the early, incubatory phases of his creative practices: his way of seeing, selecting scenes, and composing images. This requires attending to his painting as a verb and not just a noun, a rhetorical-​ aesthetic process of production in addition to a series of finished products. It is precisely at the work-​in-​progress stage where Monet’s practice resonates with Barthes’s theory. By forcing their intersection, I intend to generate several stylistic principles relevant for humanistic producers of actionable media, who seek to create digital projects with ATLAS interfaces that flourish outdoors amid the built environment, en plein air. Let us adopt one of Barthes’s theses on figuration, then, as an entry point into impressionism. “Interpretation must gradually give way to a new discourse, whose goal,” Barthes exclaims, “is not the revelation of a unique and ‘true’ structure but the establishment of an interplay of multiple structures” (Rustle of Language 154, my emphasis). Here again, porosity is the path to plurality. Figuration is the art of poking holes in a work (in Barthes’s sense of the term), in order to inaugurate “a serial movement of disconnections, overlappings, variations” (Image-​Music-​Text 158). To the long list of innovations attributed to the Impressionists, we should add another: they discovered an uncanny threshold where visual signifiers radiate significations without ever closing upon one signified. Gaze long and hard at almost any Monet painting. You will not find any discrete objects, not if you look closely. Monet’s tree is a tree and not a tree. As Steven Ungar’s commentary on Barthes suggests, “figuration involves a turning, bending, or deflecting of meaning, replacing it with a mobile space or field of action” (69). Monet poked holes into the act of painting, pushing it toward a new relation with surrounding fields of action, and his actionable techniques lend insight into how intellectual discourse might be similarly pursued today. When asked to describe his approach to painting, Monet once replied: “Merely think, here is a little square of blue, here is an oblong of pink, here is a streak of yellow; now paint, just as it looks to you, the exact color and shape until it gives you your own naïve impression of the scene before you” (cited in Schapiro 49). The quintessential Impressionist, Monet did not paint streetlamps, plane trees, or ships. He painted shapes of color. The primary task of painting, as Monet reframed it, was to see the world in a kaleidoscopic manner. That is, one needed to quite willfully privilege the ongoing play of shapes of color and, in so doing, resist or suspend the ancient compulsion to categorize things—​to behold the perceived scene as a collection of codified objects bearing essential properties. Philosopher Hélène Cixous summarized Monet’s challenge aptly: “to no longer ‘paint water lilies’ while painting water

817

Creating Actionable Media  • 

187

lilies” (589). Working with each signifier’s singularity, painting this water lily (without subordinating it to “the water lily” qua intelligible signified) entails a process of optical fragmentation. How did Monet cultivate this way of seeing? Most critics’ answers point to photography, but their explanations diverge from there. Moving from the more canonical explanations to a relatively overlooked one reveals a latent dimension of the en plein air analogy, unearthing the pertinence of Monet’s observational techniques for actionable media producers. Familiar accounts of modern art generally begin with Impressionist painting and early photography. The two seem pinned against each other in a competition for the cultural eye. Most narratives characterize the Impressionists as a band of romantic drifters who, at the onset of photography’s ascent, willed imaginative feats of perception that embraced the radical subjectivity of an individual’s visual experience, and thus defied the normative, mechanical realism captured by photography. In other words, the advent of photography freed painters from the medium’s traditional investments in mimesis. When the artists we now call Impressionists (e.g., Monet, Renoir, Pissarro, etc.) came to Paris in the 1860s, they arrived as realist landscape painters amid the first wave of photography’s industrialization (Scharf 125; Venturi 35). In 1960s Paris, for the first time, professional photographs and stereoscopic images circulated as popular spectacles readily available to a mass viewership, though amateur cameras would not be marketed until the turn of the century. The spread of photography incited debates among painters who felt themselves torn between imitation and expression. Many artists began working from photographs, though they rarely admitted to it for fear of losing credibility with dealers and patrons (Scharf 125). It was a decade in which all painters, Impressionists included, had to reconsider the very purpose and future of painting. If photography could represent the world better, faster, and cheaper, then painters had better try something different. Art critic Jonathan Crary problematizes such narratives, insisting that this “myth of modernist rapture” is premised upon an ill-​informed “binary model of realism vs. experimentation” (4). In his rebuttal, Crary shifts the temporal focal point. He contends that Impressionism and photography in the 1870s were the symptoms of a more significant technocultural development that occurred during the 1820s: the formation of what he terms “the observer” (5). The perceptual techniques of the observer—​possibilities conditioned by optical technologies like the stereoscope—​rest upon the notion that visual experience is subjectively constructed and, in some cases, technically manufactured. Impressionism and photography, to Crary’s mind, both serve as mature expressions of this techn(ont)ological reconfiguration, whereby

81

1 8 8   • Act i o n a b l e   M e d i a

people begin to conceive of perception as a contingent activity that is open to (if not inherently interrelated with) various intellectual, psychological, and technical processes. Both Crary’s alternative account and the conventional narratives he critiques, however, largely overlook a third position argued by Aaron Scharf in his 1974 book Art and Photography. For our purposes, Scharf ’s thesis is immensely provocative and merits reflection. Scharf posits that the most novel stylistic elements of Impressionist painting were in fact inspired by the painters’ exposure to early photography—​its accidents and glitches, in particular. The canonical narratives of modern art’s origins downplay the fact that photography in 1960s Paris was not an exact science. Failed photographs (those that did not capture the exact likeness of things) achieved a more radical way of seeing than any painter could have conjured up. Monet did not declare war against photography or flee from it. He embraced the bizarre hallucinations in photographs gone awry, learning to see and paint in their image. If we can understand Monet’s way of seeing as a technique for observing the world in the image of media effects, then we will have established a productive basis for further analogical insights relevant to actionable media production, particularly with regards to emerging genres like augmented reality layers, pop-​up forums, and ATLAS-​based data visualizations. To date, no biographical documentation confirms the notion that Monet painted directly from photographs; in any case, it is a mystery of little aesthetic importance. As Scharf suggests, the paintings themselves attest to a greater, figurative dimension of photographic influence at the level of perception, which furnished the Impressionists with techniques of observation as they painted en plein air. For instance, Scharf presents striking parallels between Monet’s brushwork and 1860s photographs of Parisian settings, which had been on exhibit throughout the city. Whereas portrait and still life photographs of the period could capture images that passed as exact likenesses, early photographs of modern urban life were far from realistic. Due to slow exposure speeds (by current standards), photographic images tended to blur whenever pictorial subjects were in motion: pedestrians walking the sidewalks were rendered into “anonymous smears,” flags in the wind lost their angular contours, and horse-​drawn vehicles left only “ghostly vestiges” (Scharf 129). These surreal distortions, which photography’s technologists regarded as a problem to be solved, contain the same stylistic tendencies that came to define Impressionist painting. While such features were common in early urban photography, Monet’s portrayal of pedestrian figures, for example, “seems entirely new in art [i.e.

819

Creating Actionable Media  • 

189

painting]” (Scharf 129). Further stylistic commonalities—​intense contrast of light and dark areas, elevated viewpoints, diffusion of details—​collectively support Scharf ’s claim that “[the Impressionists’] desire to record the transitory character of natural light and shade, amounted to a kind of perceptual extremism which was germane to photography itself ” (126). And yet, the first art critics to write about Monet and other Impressionists failed to note this transmedia influence, which appears undeniable from the side-​by-​side image comparisons collected in Scharf ’s book. When revisited in the wake of Scharf ’s analysis, Lionello Venturi’s influential assessment of the nature and significance of the Impressionism seems deeply flawed. According to Venturi’s canonical account, “What the Impressionist painters actually accomplished was the finding of a form closer to the first impression of the appearance of things than other painters had” (38). Venturi credits this innovation to the artists’ “vivid sensibilities” and insists that “their mind[s]‌[were] sufficiently free of traditional principles of abstract form to undervalue their impressions” (38). And yet, the notion of an “innocence of the eye,” which Venturi borrows from critic John Ruskin, is conceivable only in the wake of nineteenth century advancements in optical technology (Crary 66). Ways of seeing, like theories of perception, are enmeshed in technics. The techniques of observation that propelled Impressionism were not the equivalent of pressing the reset button on Western artistic consciousness. Rather than learning to see like a child or a blind man “suddenly gifted with sight” (Ruskin 22), Scharf maintains that Monet, consciously or subconsciously, painted scenes “not as the eye would see them but as they might be recorded by a camera” (131). From this angle, the Impressionist aesthetic is not one of transcendence but of transcoding. That is, instead of cultivating a natural or “naïve” subjectivity in opposition to what Venturi calls the devices of the “mechanical arts,” Monet’s innovation lies, at least partially, in his ability to simulate the visual logic of photography in his lived perceptual experience. In this sense, the exigencies of actionable media harbor questions similar to those that occupied Monet at his mobile easel. Consider, for the sake of comparison, the act of painting en plein air and the basic rhetorical conditions involved in creating augmented reality projects. Both cases demand the artist/​rhetor to project a transmedial hallucination onto the perceptual field. Monet beheld the world through his eyes and also his mind’s eye, the latter applying photographic distortions to the scene. The subject matter is glimpsed in this kaleidoscopic fashion—​local action percolating through imagined media overlays—​just before his brush returns to the canvas. Of

910

1 9 0   • Act i o n a b l e   M e d i a

course, when the painting was finished and hung inside a gallery, viewers encountered the traces of Monet’s visual transcoding process, but they were removed (both temporally and spatially) from the moment and place of figuration. His figurative act of observing the world in the image of media effects became framed and delivered in deferred spacetimes. By contrast, with augmented reality, the figuration process unfolds in real-​time within the subject matter’s proximity. The moment of transmedial hallucination, wherein the artist/​rhetor’s kaleidoscopic imagination transfixes the scene, remains open for viewing. AR layers are archived expressions composed in the manner of Monet’s fleeting impressions. The dynamic meeting grounds between local action, atmospheric flux, and media effects that inspired Monet’s painting are precisely the phenomena that transpire in AR layers. Producers of actionable media may adopt Monet’s way of seeing, viewing the scene before them with the interchangeable lenses of all the available means of digital transcoding (no longer limited to photographic media effects). Indeed, more than Monet could say of his audience, the viewers of actionable media projects, in our era of so-​called “digital-​physical convergence,” stand privy to experience the world kaleidoscopically by default. Impressionism was, of course, more than a way of seeing. The transcoding of photographic media effects spread from lived perception into pictorial composition. And while the technics of early photography clearly inform Impressionist aesthetics, the encounter was by no means deterministic or unilateral. Monet’s style was more than the sum of photography’s impact upon his gaze. Had he not developed a loose set of painterly aesthetic principles, Monet would have been remembered as little more than a gimmick artist, whose novelty would have worn thin once photography became acculturated as a mass medium. Milestones in the arc of Monet’s career indicate distinct evolutions in his brushstroke and, intricately related, his approach to selecting scenes and subject matter. After accounting for the interplay between style and topoi in Monet’s aesthetic, I will again turn to glean the transdisciplinary rhetorical insights his creative practices cast over actionable media exigencies. One of the most amusing events in the story of Impressionism lies in the 1874 critical reviews of what historians now call the “First Impressionist Exhibition.” These critics were blind to the trace of photography, as noted above, and several did in fact dismiss the Impressionists’ style as gimmicky. Most importantly, the critics—​whose reviews read like a contest amongst themselves to see who could best ridicule the paintings—​unwittingly identified one of Impressionism’s most generative stylistic principles: the notion of the unfinished.

19

Creating Actionable Media  • 

191

Shown at the 1874 exhibit, Monet’s Impression, Sunrise emerged as an emblem for the movement during its early years. In addition to inspiring critic Louis Leroy’s coining of “Impressionism” (which he used scornfully), this painting was the object of Leroy’s quintessential punch line: “Wallpaper in its embryonic state is more finished than that seascape” (1874). Reviews by Ernest Chesneau and Jules Castagnary also single out this lack of finish in their backhanded compliments of the exhibition. Chesneau labeled Monet’s Boulevard des Capucines “a marvelous sketch,” before revealing his true opinion: “Clearly, this is not the ultimate statement of art in general, nor of this art in particular. This sketch must be transformed into a finished work” (1874). Despite a lavish assessment of several artists’ potential, Castagnary nonetheless dismissed Impressionism as a flimsy, unfounded exaggeration of a minor tendency in more established styles of painting (what he termed “the unfinished”), and he concluded that the impressionists are “soon to perish where they stand” (1874). In each review, the mockery, the satire, and the insults all hinge upon a shared assumption that fine art entails a high degree of finish; figures should be well defined and every aspect of the scene rendered in painstaking detail. Not until roughly a hundred years later would continental philosophers champion the unfinished in their reflections on modern literature, music, and visual art (e.g., Barthes’s “writerly text,” Eco’s “open work,” and Cixous’s “works of being”). Stylistically, Impressionism emerged in point-​by-​point contrast to the then critically acclaimed standards of academic art, and in many ways this dichotomy recalls critical differences between the desktop metaphor and ubiquitous computing. Academic painters (who dominated the annual Salon) typically portrayed subject matter associated with “history, myth, and imagined worlds” (Schapiro 10). Virtual realities and differed pasts, as it were. Even in portraits of their contemporaries (e.g. William Bouguereau’s Breton Brother and Sister), academicians contrived the scene so as to immobilize their human subjects in a classical pose, whereby all is clear and categorically discrete, without a hint of movement or change. According to these conventions, a painting is not finished until no apparent trace of the artist’s brushstroke remains. Monet’s paintings are “discernibly constituted by the stroke,” and this, for art historian Meyer Schapiro, is their key feature—​one that is all the more evident when seen in person (Schapiro 51). Whereas most prior styles of Western painting work to conceal artifice, the Impressionists emphasize materiality and movement, both in their composition of the landscape and in their kinetic engagement with the paint itself. Paradoxically, Monet’s resolutely unfinished composition process renders “both the illusive

912

1 9 2   • Act i o n a b l e   M e d i a

image-​appearance of a scene in deep space and the tangible substance of the painting as effects produced by the artist on the framed surface of the canvas” (Schapiro 52). That is, the twofold style conjures a representational image, but does so in a blatant painterly and materialist manner, à la figuration. The image leaps out of representation; the paint leaps out of the image. But images hover over the paint nonetheless. Unlike academic artists, Monet’s paintings present scenes of a world constantly in flux. The mechanics of his brushstroke carry serious metaphysical challenges to the Cartesian worldview. Monet’s objects (or quasi-​objects) appear rough and permeable, not due to his uncertainty, but because nothing alive is ever finished. Identity is presented as a passing aftereffect of difference, the ongoing play of what philosopher Karen Barad calls “intra-​ actions.”4 Monet sees and composes at the level of intra-​activity. This metaphysical leap, more apparent in his later works such as the Water Lilies series, follows from his progressive embrace of nonlinear composition, though not in the temporal sense critics usually employ the term. The founding gesture of Impressionism, deployed most emphatically by Monet, is “the dissolution of the line” (Schapiro 9). Visually speaking, lines stabilize the play of phenomena (i.e., intra-​actions) into defined objects with essential properties. To delineate objects via lines is to finish the impression and, in metaphysical terms, to institute a hierarchy of the intelligible over the sensible. Or, borrowing from Deleuze and Guattari, it is to experience nature as a product rather than as “a process of production” (3). Impressionism, as an aesthetics and even a metaphysics, is not rooted in intelligible essences of defined objects; rather, it is processually uprooted by the accidents of the sensible. Monet’s Water Lilies usher the attentive viewer away from “water lilies” apprehended by the natural (Cartesian) attitude and toward a more phenomenological encounter with intra-​actions: the accidental play of light and shade and color, accruing in the sensible plentitude of a local scene. Photography shares this dynamic. As an indexical sign, the photograph measures the singularities of light exuded by specific geographical flows. The real-​time capture of photography (what Barthes called its “reality effect”) allows a photograph to record what might otherwise remain unnamable in the image, owing to its irreducibility to verbal cultural codes—​codes that have themselves accumulated from the deferred economy of linear writing, which can no more capture the singularity of an instant than the rhythmic economy of living, oral memory could retain abstract, prosaic statements (Havelock 25–​26). If Monet’s engagement with photography’s glitches models a way of seeing for actionable media producers, then the metaphysical implications of

913

Creating Actionable Media  • 

193

his composition techniques suggest another prescient stylistic principle: one must learn to write, communicate, and design with the accidents of the sensible, no longer in spite of them. Recall Cixous’s incisive remark about Monet’s method: to no longer “paint water lilies” while painting water lilies. Not only does the verbal code “water lilies” fail to speak to the singular qualities of each particular water lily; more generally, the words we ascribe to things can shape our perception in the same way that lines effect a painting. Western metaphysics—​from Plato to Aristotle to Descartes—​has typically maintained a fundamental opposition between the intelligible and the sensible. Ancient and modern philosophers habitually abstracted “essential” properties from barrages of sensible matter. Plato infamously defined “man” as a featherless biped, rooting the identity of our species in this essential property. Though this definition failed (Diogenes demonstrated that the same description could apply to a plucked chicken), the method of defining things by deducing their “essences” clearly prevailed, most astoundingly in the pedagogical legacy Peter Ramus shed over Renaissance Europe.5 These cognitive norms at the core of Western literacy pull consciousness in the opposite direction of Monet’s kaleidoscopic gaze. Monet’s creative practice, his fidelity to sensible plentitude, exhibits a conception of topos much different than Aristotle’s, for example. While Aristotle’s topoi designate commonplaces for finding arguments, his usage of the term rarely deals with actual places. Rhetors can pursue Aristotelian topoi without leaving their writing desk or the podium. Aristotelian topoi (e.g., genus/​species, whole/​part, cause/​effect) are intelligible categories for probing the available claims that one might advance about virtually any subject whatsoever. Monet’s topoi are literally material, geographical places. He refashions the scene of painting, as well as his composition process, so that he may foreground the contingencies of local action occurring before him—​ accidents of the sensible otherwise marginalized through studio painting, deferred writing spaces, and habits of mind intrinsic to Western literacy. In fact, art historian John House plots Monet’s artistic evolution around a pivotal shift in topoi. As Monet’s career progressed, sky, water, fog, and intense patches of sunlight became “the keynotes of [his] paintings,” forming intra-​ actional spaces in which the identity of any object diffuses (House 19). According to House, [Monet] was insisting that the significance of the objects he painted lay in the relationships between them, in the multifarious elements which together went to make up the modern scene, and not in any

914

1 9 4   • Act i o n a b l e   M e d i a

external ordering process imposed by the artist in order to elevate one aspect above the others. (17) For Monet, notably in his London landscapes (e.g., the Houses of Parliament series), sensation and mood intensify around places that generate their own abstractions and afterimages qua aquatic reflections, weather glosses, and atmospheric effects. Accidents of the sensible became ever more consequential. “Without the fog,” Monet wrote, “London would not be a beautiful city” (cited in House 29). He composed images with/​in the fog. Fog served as a passageway from representation into figuration, precisely in the sense that it dispersed familiar perceptions “into a more generalized and heterogeneous symbolic activity that tempers intentional meaning with its opposite” (Ungar 156). Or, we might say, fog tempers the intelligible filters by which we tend to apprehend—​to constrict and overlook—​sensible plentitude. By selecting scenes that played host to such intra-​actional geographical flows, and by cultivating those flows as aesthetic topoi, Monet put local action at the generative core of his invention process. Here, then, is a further stylistic principle that actionable media producers can derive from Monet’s en plein air composition practices: find your keynotes and learn how to compose with/​in them. For Monet it was fog, but for a scholar of environmental communication, for instance, the keynote might be smog. This scholar might adopt smog as a topos, crafting and delivering her discourse (e.g., institutional critique, content analysis) around the smoke-​ producing sites in a given metro area, or even geotagging and layering her text along persistent flows of smoke so as to foster an intra-​actional resonance between her intentional theses and the live events occurring in the viewer’s perceptual field. The scene of production/​pollution doubles as the project’s reception context. One must scour the reception context, observing all its actual and potential contingencies, in search of a sensory keynote. The reception context enters the text as a privileged, autonomous force of invention; it shares in the ongoing (re)composition of the text, crowding around and seeping through the permeable frame. Actionable media is always unfinished in the sense that it, by definition, accommodates the flux of proximate contingencies rather than bracketing them from attention. Signification accrues through contextual intra-​actions, not contextual immersion. Accordingly, creating actionable media (e.g., audio walks or more image-​driven AR layers) involves decisions about subject matter in two co-​constitutive registers:  (1)  multimedia files; and (2)  the places—​or, more specifically—​the accidental, intra-​actional flows that constitute the opportune proximity at

915

Creating Actionable Media  • 

195

which audiences will encounter those files. The rhetorical-​aesthetic challenge, therefore, is to forge bonds between networked multimedia and local action—​to write without the enclosure of document-​to-​document bindings, to paint without lines. Of course, new publishing outlets will be needed to solicit, review, and circulate this prospective mode of actionable criticism. Editors would deal in layers rather than articles, volumes, or websites. Layers would be published at relevant locales (or circulate upon specified objects) on a regional, national, or global scale. These scenarios are, I concede, difficult to imagine, and they may excite only a faction of scholars at present. And yet the technical groundwork for such models of intellectual discourse has already been laid and may reach mass adoption quicker than many of us presume. Some prominent writers, editors, and publishers are pushing their professional peers to rethink the shape of magazines, newspapers, and online networks in light of emerging “geo-​publishing” platforms. Alexis Madrigal, a senior editor for The Atlantic, tested Google’s Field Trip app and imagined how its geospatial methods for delivering content might be experienced through the lens of headworn displays. Noting critical differences between geospatial AR apps and standard Web browsers, Madrigal believes the wearable future of AR invites “a new kind of media” that will challenge traditional assumptions about genre, setting, and audience: No one publishes a city, they publish a magazine or a book or a website. If we’ve thought about our readers reading, we’ve imagined them at a breakfast table or curled up on the couch or sitting in their office. No one knows how to create words and pictures that are meant to be consumed out there in the world. (“The World is not Enough”) This passage levels me every time I read it. It is as if Madrigal points out the edge of a cliff, revealing an abyss . . . gaps in the history of writing, reading, and textuality that I never knew were there. Cities—​or any built environments within the reach of wireless networks—​are becoming a viable outlet for intellectual discourse alongside monographs, journals, newspapers, podcasts, and blogs. Cities have indeed served as repositories for public-​facing texts since ancient Athens, but still, “no one knows how to create words and pictures” with/​in the city. Our stylistic models are either the corporate-​ municipal signage that blankets most urban corridors or the more experimental “arts of the street” to which Monet’s en plein air practices made a formative contribution.

916

1 9 6   • Act i o n a b l e   M e d i a

Meanwhile, digital-​public artists—​today’s en plein air avant-​garde—​have just started to pitch cultural critiques in this manner. Severalprojects make the prospects of actionable criticism more readily discernable. AR Occupy Wall Street (2011) and The Invisible Artist (2010), for example, suggest compelling avenues for scholarly appropriation, if not artist-​scholar collaborations. As a crowdsourced initiative, AR Occupy Wall Street is a wide-​ranging collaboration between artists and activists. Some of its most intriguing layers are set around the New York Stock Exchange. For example, one piece transforms the building’s grand columns into a slot machine, while another (pictured in Figure 6.1) blankets the street with a streaming NYSE stock ticker, such that pedestrians walking on the sidewalk, in the background of the frame, are obscured by flows of financial data. In both cases, the metaphors occur at the interstice of live urban geography and composed multimedia. When layered over actual pedestrians, the streaming stock ticker may be read as a hyperbolic manifestation of their stream of consciousness, or as a tragic testament to market forces that cosign

Figure 6.1  Screen capture from the project AR Occupy Wall Street in New York, organized and documented by Mark Skwarek. © Mark Skwarek, 2011.

917

Creating Actionable Media  • 

197

their socioeconomic exclusion. The effect ultimately fluctuates depending on the apparent class status of the pedestrians who occupy the frame, whether it is a group of slick investment bankers or a malnourished homeless person. Because the streaming stock ticker binds with a wide range cinematography potential to the geographical flows of Wall Street, those flows (especially the pedestrians) become topoi initiating new lines of association and interpretation. Still, the rhetorical power generated by these image layers could become all the more salient if they were coupled with audio or textual commentaries. This is where actionable criticism may step in, commencing to theorize, historicize, or analyze the affects that variously charge these scenes. The Invisible Artist could be read as a more direct attempt at actionable cultural criticism, in that a composed audio layer compliments the piece’s imagery. This London-​based AR project, created by John Goto and Matthew Leach, poses as an innocuous guide to the city’s top art galleries. Approaching the entrance of each gallery, users are greeted by a fashionably dressed, headless 3-​D figure who provides historical commentary about these institutions (see Figure 6.2). The commentary, in spite of the figure’s pleasant tone of voice, advances a discerning assessment of the galleries’ curatorial practices, effectively “satirizing the bureaucracy and lack of diversity of London’s contemporary art

Figure 6.2  Screen Capture of “Tate Modern” from The Invisible Artist series. © John Goto and Matthew Leach, 2010.

918

1 9 8   • Act i o n a b l e   M e d i a

scene” (Goetz). In turn, the figure’s missing head emblematizes all the artists whose work is routinely marginalized for failing to meet the standard tastes of appointed Academicians who curate for the Royal Academy of Arts. For our purposes, this image marks a fitting last link in the en plein air analogy; it is a twenty-​first century homage to Monet, the Salon des Refuses, and the will to experiment in the face of institutional doxa.

The Spect-​actor: Augusto Boal’s Ideal Audience The title of Augusto Boal’s 1974 masterwork of dramatic theory, Theatre of the Oppressed, alludes to the influence of another revolutionary text: Pedagogy of the Oppressed, published by fellow Brazilian Paulo Freire in 1968. The two books mirror one another in chapter structure. Freire starts with a historical critique of how political oppression informs and is reinforced by the teacher/​ student divide in schools, and Boal angles a similar attack on the actor/​spectator divide in theater. Freire, the more widely read of the two, is famous for the intellectual massacre he waged against “the banking concept” of education. Under this model, teachers and students behave according to a clear epistemological hierarchy:  “the teacher issues communiqués and makes deposits which the students patiently receive, memorize, and repeat . . . the scope of action allowed to the students extends only as far as receiving, filing, and storing the deposits” (Freire 72). Freire believed that any liberatory pedagogy had to begin by recasting these conventional roles played by teachers and students, which he took to be dehumanizing. Because knowledge “emerges only through invention and reinvention,” everyone who occupies a classroom should be “simultaneously teachers and students” (Freire 72). Teachers around the world have since adopted Freire’s maxims as battle cries to promote education reform; in the context of American higher education especially, one would be hard pressed to name a text or thinker who garners so much acclaim and so little resistance in scholarly conversations. Nevertheless, the banking concept continues to manifest in practice. Large lectures are still far and away the most popular format in many disciplines on most campuses, and lecture-​ style MOOCs are sometimes touted as the future of online learning. What accounts for this vast, alarming discrepancy between intellectual values and institutional practices? Economics play a big part, no doubt, but I think there are also conceptual deficits to acknowledge. While we like the idea of having an active audience, we have been slow to theorize new roles and practices that would cast traditional spectators into more actionable roles. The failure to fully dismantle sender-​receiver hierarchies in higher education is not unique,

91

Creating Actionable Media  • 

199

of course, as a multitude of communication conventions in other sectors still retain similar dichotomies: politician/​citizen, broadcaster/​viewer, programmer/​user, etc. Adding comment boxes to the bottom of documents or asking for questions at the end of a public address does little to reconfigure the systemic hierarchies that Freire identified. Recasting the audience—​be they students in a classroom or smartphone users in the streets—​entails corresponding transformations to all sides of the rhetorical triangle. In the wake of Freire’s critique, Boal’s notion of “the spect-​actor” affirms a promising way to proceed. His remarks about audiences facilitate transdisciplinary exchanges between theatrical performance and deliberative rhetoric, which are, as I  will show, quite illuminating for actionable media producers aiming to create civic apps or public interactives designed to foster participatory democracy in the age of ubicomp. The thread connecting Boal’s theater with the exigencies of actionable media begins with Aristotle and his unraveling. Boal’s boisterous history of dramatic theory reads like an indictment, and his most severe accusations target Aristotle’s Poetics. The rise of Greek tragedy, analyzed and formalized by Aristotle, marks for Boal the invention of perhaps the most effective, quasi-​coercive vehicle for maintaining political dominance in otherwise democratic societies. Theatrical performance held a much more fluid place in the everyday lives of ancient citizens during the centuries before Greek tragedies took center stage. There were no “spectators,” no formal stages or distinct protagonists—​“theater was the dithyrambic song: free people singing in the open air” (Boal 119, my emphasis). Oppression begins, Boal suggests, with the introduction of walls: built walls separating the theater from other urban spaces in the polis, invisible walls bracketing the protagonist from the chorus, and the fourth wall dividing actors on stage from the crowd in the seats. The citizen qua spectator is reduced to a voyeur, driven to empathize and identify with the protagonist (assuming the playwright has followed Aristotle’s guidelines). At the play’s end, however, it is revealed that the character traits which elicited empathy for the hero are the very qualities that led to his/​her tragic fate. By undergoing the tragedy vicariously, spectators are taught to purge themselves of such undesirable qualities—​namely, of critical or revolutionary impulses “capable of changing society,” claims Boal (155). Granted, some alternative modes of engagement exist in conventional staged theater. Boal commends and builds off of Bertolt Brecht’s “alienation effect,” for example, which deploys meta-​theatrical techniques to disrupt the spectator’s empathetic gaze. But ultimately, in order to restore to spectators their capacity for both critical thought and action, Boal stakes his vision

02

2 0 0   • Act i o n a b l e   M e d i a

of theater on a resolute proposition. “The walls,” both visible and invisible, “must be torn down” (119). The theatrical wall that kept all spectators bracketed from the scene of dramatic action closely parallels those borders demarcating the terms that shape the Neo-​Aristotelian rhetorical triangle (i.e., audience, speaker, message). Such iterations of the triangle (Figure 6.3), mapped over Aristotle’s persuasive appeals, have been widely featured in rhetoric textbooks such as Writing Arguments. The speaker produces the message, the audience receives the message, and the message, in turn, is the content that has been transmitted from the speaker to the audience. It is a basic sender-​receiver model of communication. Understood thus, sender-​receiver interactions may inform rhetorical thinking at the level of anticipation and feedback, but the means and processes of message production remain within the sender/​speaker’s purview. This model is ill-​suited, of course, for describing the production and circulation of actionable media, particularly in the case of public interactives and other sensor-​actuator networks wherein rhetoricity accrues at the interstice of bodies in motion and embedded computation. Learning from Boal and the new forms of spect-​actor theater he invented, how might we recast these basic rhetorical roles to better account for and invoke the capacities of an actionable audience? Noah Wardrip-​Fruin practically dares us to take up that question: “It remains to be seen if Boal’s non-​ Aristotelian poetics can be compellingly applied to new media design . . . the possibility is certainly intriguing” (New Media Reader 340). First a thesis, then its explication: I submit that actionable media producers would do well to adopt “the spect-​actor” as their ideal, invoked audience when ETHOS

(Speaker)

LOGOS

(message)

PATHOS

(audience)

Figure 6.3  A diagram of the rhetorical triangle. Image by the author.

210

Creating Actionable Media  • 

201

conceiving and analyzing rhetorical situations associated with ATLAS interfaces. The spect-​actor is not (only) a viewer, reader, writer, user, or prosumer. One becomes a spect-​actor only in the moments s/​he simultaneously receives and acts upon cues in the lived scene s/​he presently inhabits. Conventionally, rhetoricians reference the concept of an “invoked audience” as a means to explicate how writers (and speakers) craft “the resources of language to provide cues to the reader,” in order to nudge audiences to assume the role(s) conducive the writer’s message (Ede and Lunsford 83). The audiences invoked by intellectual discourse are, by and large, cast into whatever pensive postures are available to them (to echo Madrigal’s point above) while “sitting in their office,” “curled up on the couch,” or in a movie theater or lecture hall. The scope of action appropriate to pensive audiences is limited to facial expressions, monosyllabic reflexes, note-​taking, laughter, and gentle gasps. Responses more conspicuous than these are coded as rude, unprofessional, or distracting. Boal’s spect-​actor is, on the other hand, disruptive by definition. To invoke spect-​actors is to provoke pensive spectators to act into the scene that surrounds them, as well as to prompt actants toward a more critical awareness of their own present actions. Such are the available prerogative of actionable media production, and individual projects may aim to achieve either of these rhetorical goals. Boal had an arsenal of techniques for achieving both. Boal’s myriad forms of spect-​actor theater (e.g., “image theater,” “forum theater,” “legislative theater”), all revolve around a few common premises and practices. Chiefly, they amass dramatic tension over the course of two distinct scenes, often performed in succession: the “finished” scene that appears to represent the social routines and power relations that currently underlie everyday life in a given locale, and then the “rehearsal” of that scene now amenable to spect-​actor interventions. During this latter rehearsal-​scene, Boal instructs the audience (now spect-​actors) to interrupt the dramatic action in the way a director might. Spect-​actors jump on stage when they see fit, intermittingly coaching the actors to take different courses of action, attempting alternative solutions to the dilemmas the characters face, and so on. Spect-​ actors can even replace an actor on stage—​when a stage is involved—​and improvise the role themselves, according to their own dramatic tactics. Quite notably, Boal’s “invisible theater” practice has actors (who never reveal themselves as “actors”) perform quasi-​scripted scenarios in public settings, wherein bystanders (e.g., waiters and fellow restaurant patrons) variously (re)act into the scene without knowing that it is a theatrical performance. In every case, Boal’s spect-​ actor practices dramatize a conflict that accentuates inequalities currently plaguing the city or village in which the

20

2 0 2   • Act i o n a b l e   M e d i a

performance is taking place. While in the fishing port of Chimbote, Peru; for example, Boal staged a scene for fish factory workers that emblematized their exploitative labor conditions (140). After the finished-​scene, the factory workers (impoverished and largely illiterate) proposed numerous solutions, which they rehearsed with the actors on stage. The rehearsal-​scenes enabled the workers, as a collective, to see how their prospective solutions might play out in action: it is a political debate conducted exclusively in theatrical language. The laborers, qua spect-​actors, rehearsed bombing the factory only to realize this would eliminate the source of their wages. They tried out a labor strike, which also proved undesirable because, as the actors playing the bosses made clear, the factory owners could easily replace the strikers in Chimbote, a city full of unemployed people who were “hungry enough and [had] little enough political consciousness” (141). Finally, the spect-​actors performed another rehearsal-​scene that they all deemed to be the most desirable solution: they decided to work faster, loading fish into the machine at a rate that would cause it to malfunction. Their expediency would please their boss, and when the machine eventually broke down, the laborers would get an opportunity to rest, which their boss had previously denied to them by enforcing twelve-​hour shifts of nonstop labor. Spect-​actor theater, here and elsewhere (it is practiced worldwide today), is a medium to pursue grassroots rapid prototyping for communal action. As Boal insists, “It is not the place of theater to show the correct path, but only to offer the means by which all possible paths may be examined . . . a rehearsal of revolution” (141). Theatrical performance and deliberative rhetoric coalesce. Spect-​actors reflect on and intervene in the spectacles of their everyday lives. Actionable media projects can attune citizens to the available means of critical intervention intrinsic to particular scenes of local action, and do so in the moments citizens actually inhabit those settings. A public interactive embedded inside a train station or city park, for instance, need not reproduce a “finished scene” put on by actors. These places host readymade dramas, whereby diverse groups of people perform daily routines wrought with ideological commitments, value hierarchies, and varying degrees of conflict. To actionable media producers, the readymade dramas of public life do the dramatic work of Boal’s finished-​scene, sufficing in themselves to initiate rehearsal-​scenes that may channel the spect-​actors’ capacities for critical intervention and/​or crowdsourced rapid prototyping. The rhetorical challenge is to design a public interactive (or a civic app) which frames the scene of local action as a spectacle that calls for alterations people are capable of making or proposing in the here and now. The actionable media spect-​actor is

230

Creating Actionable Media  • 

203

she who regards the scene in question (which is not actually staged) as both a dramatic performance and a rhetorical platform. Akin to Boal’s theater, invoking spect-​actors of this sort involves deconstructing walls and recasting roles that conventionally organize political deliberation and communicative action. Actionable media projects envelop the spectacle as such, but they are not themselves the spectacle. They are the frame instead of the framed. Their rhetorical function as “message” is to graft digital networks onto a public setting in a manner that turns bystanders into audience members and, at the same time, enables/​provokes them to become informed speakers or actors. That is to say, at least in the more activist iterations, actionable media projects are not messages in the usual, sender-​receiver sense. They operate more like stage directions, which are read to be performed, received only to be sent anew. And in some cases, actionable media content gets projected and displayed only after the receiver’s activity has triggered the sender’s message into being. Consider this dynamic with reference to the embedded sensor-​actuator networks examined earlier in this book (e.g., David’s Way, Digital Water Pavilion, Trash|Track). Each operates as a public interactive that translates local activity into (a basis for arranging and delivering) networked multimedia that either attunes people to proximate events or produces an indexical trace of moving bodies. The MIT SENSEable City Lab’s Trash|Track project, which you will recall from c­ hapter 5, illustrated how we might learn to craft persuasive data visualizations by tracking the real-​time performance of complex systems that underwrite institutional initiatives for social change (North America’s infrastructure for recycling, in the case of Trash|Track). If we regard spect-​actors as the ideal audience for actionable media, however, then Trash|Track bears one glaring flaw: it is displayed to spectators in white cubes (museum and library visitors) rather than spect-​actors in situ. The real-​time information events that the research team captures are subsequently delivered as objects for contemplation walled away from the very situations that the data visualization’s rhetorical appeals mean to affect. The project positions viewers to apprehend recycling as a spectacle, yet it elicits this viewership at a space and time in which no relevant course of action is available to the spectators. And so they remain spectators; action may only come later or not at all. This is a missed opportunity, as one can imagine ways of circulating these visualization more aligned with the moments of decisions and scenes of action that mediate individual post-​consumption habits and collective patterns of waste management, which are above all what Trash|Track hopes to alter.

024

2 0 4   • Act i o n a b l e   M e d i a

In order to invoke spect-​actors, the data streams engendered by sensor-​ actuator networks, public interactives, and civic apps should be channeled back into the local scene, toward the stakeholders responsible for it. Where Trash|Track fails to implement this kind of actionable circulation, another MIT project developed at the Center for Civic Media enacts a more instructive model. Action Path (2014), a civic app by MIT researcher Erhardt Graeff, delivers informal referendums to citizen-​subscribers, which solicit their opinions about a proposed course of action (e.g., construction projects, public policies, institutional plans) pertaining to the place they are currently inhabiting or passing through. Think about what this actionable rhetorical gesture—​the grafting of pop-​up digital forums onto local, lived scenes—​appears to initiate from the standpoint of urban politics: first, Graeff ’s app cultivates a process of selection that privileges the proximate pedestrians who occupy, traverse, and reside near the area in question. The process is extremely convenient for them and less so for everyone else, which presumably allows researchers, urban planners, or policymakers to craft a more coherent signal-​to-​noise ratio.6 Furthermore, Action Path addresses these proximate pedestrians during the moments they inhabit the locale in question. Citizens are thus positioned to see the proposed action in context, and so the contingent spectacles that constitute the everyday rhythms of the lived scene become inextricable from public deliberations regarding the referendum. When Graeff derides “existing platforms for civic engagement” for being “disconnected from the source of issues they are meant to address,” he is fundamentally critiquing the representational modes of politics that modern democracies typically rely upon (“Action Path Summary”). Rather than represent a political problem-​to-​be-​ solved (via select testimonials uttered within the walls of city hall), actionable forums like Action Path could eventually do for city governance what Boal did for theatrical performance. The app stands to empower pedestrians as spect-​actors in the sense that it relocates and redistributes the practice of deliberative rhetoric, making spacetimes for voices in the streets that, though uttered outside the confines of institutional buildings, still get heard by the city’s planners and policymakers. Of course, responding to referendums constitutes only a minimal degree of civic engagement, and this action alone hardly amounts to political change. Nonetheless, Action Path creates actionable data streams that may help citizens—​hopefully to much greater degrees in the future—​enrich public decision-​making about urban infrastructure and civic policy through local, in situ knowledge ​building. While critics rightly bemoan the fact that companies such as Google and Facebook harvest our digital data in the service

205

Creating Actionable Media  • 

205

of targeted marketing campaigns, actionable media producers should regard projects like Action Path as a call to reimagine how data gathering technologies might be crafted for the sake of promoting modes of public deliberation, which aim to meet citizens when and where the issues actually occur—​as opposed to more common forums for democracy, which are representational all the way down. To invoke spect-​actors, one must eschew representation in favor of figuration. This is where Boal’s performances resonate with Barthes’s texts. The spect-​actor is a byproduct of figuration, while the spectator’s immobile gaze reflects the subject position invoked for him by works of representation. As stipulated above: figuration produces desire within the reader on the basis of his or her present situation; representation references events that are spatially and/​or temporally distant from the speaker’s podium, the context, or virtual window. Representation effectively brackets events from the audience’s presence, all the while rendering their surroundings into a spectacle for viewing but not for acting into. Look but don’t touch; listen but don’t speak. As a principle of actionable media, the spect-​actor ideal encourages digital rhetors to elicit and orient local actions by framing and foregrounding certain aspects of a scene as people (may otherwise) live it. Ultimately, public deliberation qua actionable media is about remapping political forums across the spacetimes of everyday life in vibrant urban settings.

Writing Events: Janet Cardiff’s Incidental Choreography “How can an event be written? What can it mean to say ‘Writing the Event’?” asked Barthes, in his 1968 essay bearing that title (Rustle of Language 149). Writing events is, after all, what actionable media producers aim to do. In addition to selecting kaleidoscope scenes and invoking spect-​actors, one must stage meaningful interplays between designed compositions and circumstantial happenings. This task seems paradoxical, at least from the perspective of representation, whereby discourse is thought to manifest in an event’s wake, preserving a record of what happened after it has already occurred. To write the event is to break away from après coup testimony. The writer aspires instead to strike a more synchronic, proximate relationship with the object of her writing. Barthes leaves us with a clue and a case study that both smack of figuration. “The present event,” Barthes advises, “can only furnish marginal fragments of writing” (149). Marginal fragments—​anything more overwhelms the metonymical link that positions writing (or any media) to be experienced as a part of the whole event. A media artifact that cannot be apprehended in

026

2 0 6   • Act i o n a b l e   M e d i a

conjunction with the present scene becomes a scene unto itself, an annex in which everything must be re-​presented. Writing events is the art of arranging marginal fragments of multimedia amid a scene, so as to establish rhetorical-​ aesthetic interstices between composition and circumstance. To illustrate “the writing of an event” in action, Barthes offers a historic example during which such interstices came about, we should note, more through happenstance than by any design. He recounts the peripheral, actionable role played by “radiophonic speech” (i.e., live radio coverage) during the events associated with the student protests and worker strikes of May ‘68. Contrary to printed newspapers, for example, radiophonic speech “clung to the event, as it was occurring” while protestors (“their ears glued to transistor radios”) listened in the streets to the live broadcasters whose “informative word was so closely involved with the event, with the very opacity of its present, as to become its immediate and consubstantial meaning, its way of acceding to an instantaneous intelligibility” (Barthes, Rustle of Language 149). The difference between an actionable broadcast and a deferred report cannot be overstated in this case. Hearing news of a nearby fight as it breaks out is, to a protester in the streets, categorically distinct from reading about that fight a day later. Between the broadcaster’s mobile news crew and the protesters’ transistor radios, networked messages and local actions carried on a dialogic exchange, their reverberations perpetually folding into the present event. “The age old distance between act and discourse, event and testimony, was reduced” (Rustle of Language 150). In the 1990s, when transistor radios became supplemented by portable CD and MP3 players, Canadian artist Janet Cardiff invented a new art form crafted around similar points of convergence between recorded audio and urban life. It has been dubbed “the audio walk.” Unlike May ‘68, however, the actionable interstices that underlie Cardiff ’s audio walks are the result of painstaking choreography. Cardiff researches and charts particular routes through various cities, structuring each walk with a symphonic blend of oral storytelling, direct address, and sound effects. Her work garners extensive acclaim from contemporary art critics and new media artists; Lev Manovich insists that Cardiff ’s oeuvre features “some of the most amazing art of our time” (“Poetics of Augmented Space” 236). Here, in the final section of this chapter, I want to examine the eccentricities of Cardiff ’s research-​creation process—​what I will call her “incidental choreography”—​in order to derive an actionable approach for performing rhetorical analysis and critical theory in situ. My ultimate question is this: how might Cardiff ’s artsy audio walks lead us to conjure “rhetoric walks,” “theory walks,” and, more generally,

027

Creating Actionable Media  • 

207

principles for inventing, arranging, and delivering humanities discourse to the inhabitants of networked locales? Consider first the artistic aims that motivate Cardiff ’s on-​site research forays. Of all the cultural precedents and stylistic models Manovich profiles in “The Poetics of Augmented Space,” none do more than Cardiff to “demonstrate the aesthetic potential of layering new information [e.g., digital sound] over a physical space” (226). Unlike, say, a conventional docent tour guide who provides museum visitors with relevant art historical information as they view paintings within a white cube gallery, most of Cardiff ’s audio walks take spect-​actors outside of museums (e.g., Words Drawn in Water) and libraries (e.g., The Missing Voice (Case Study B)), guiding them through highly variable public settings such as The National Mall, Central Park, or London’s East End. Her objective is not to contextualize immobile objects, but to trigger aesthetic experiences amidst everyday hustle and bustle. For each project, she amasses a cacophonous database of percepts, sensations, materials, and patterns of proximate activity that are likely to constitute the varying conditions in which people experience her work. Furthermore, these ecologies of local action in potentia serve as a primary heuristic and generative source for the recorded content, narrative voiceovers, and structuration of Cardiff ’s compositions. About her field research, she says, “The first part of the process [is] finding a route that interest[s]‌me . . . After establishing the route the next part [is] to do a lot of walking, listening and looking. This is how I find themes that echo the location” (Cardiff qtd. in “Hirshorn Interview”). The present scene is Cardiff ’s canvas, and it is far from blank. An impetus of her preliminary observations is to anticipate probable, recurring events. All of the audio gets composed and eventually heard on the basis this experiential database. The audio is a marginal fragment furnished by the scene’s visual, tactile, or sonic properties. The metaphorical importance of the fountains foregrounded along Cardiff ’s route in the 2005 audio walk Words Drawn in Water (recalling to mind Monet’s use of fog as a painterly keynote) is emblematic of her tendency to induce motifs from the materiality particular to a given walk. Another characteristic technique occurs during The Missing Voice (Case Study B), for instance, when Cardiff ’s voice leads you past a café-​lined street just before the track gives way to a conversation recorded at one of those cafés. “I record right on the site, following the exact route that you eventually listen to it” (Cardiff qtd. in Gordon). This simple recording tactic causes profound aesthetic effects. While experiencing Cardiff ’s projects, one cannot avoid a certain degree of perceptual confusion. You have trouble identifying which sounds are coming

208

2 0 8   • Act i o n a b l e   M e d i a

from the headphones and which are emanating from the people and things around you. It is probably the most uncanny characteristic shared by all her audio walks, making it the hardest aspect of her oeuvre to describe for people who have not undertaken her walks. As you follow the sound of Cardiff ’s footsteps, for example, you might also hear the footsteps of people walking right behind you. Music fades in and out of the track, mimicking the feel of a passing car’s stereo or a faint tune coming from a restaurant across the street. Though these sonic conflations may not be especially meaningful in themselves, their persistence encourages the spect-​actor to willfully suspend her concern to differentiate between composed audio and environmental noise. At the very least, one realizes it’s futile to try, because so many of the sound effects and recorded conversations playing in the headphone could have sprung from the surroundings. If you are uncertain whether a sound came from your headphones or the person behind you, then you have little choice but to treat everything as an aesthetic phenomenon, as a possible component of the artwork. As such, all borders conventionally bracketing art from happenstance, text from context, become porous. Accidents of the sensible become essential to each audio walk’s reception. The circumstantial, material makeup of settings along the walk’s route constantly factors into Cardiff ’s soundtrack, both figuratively and sensorily. Inevitably, as critics writing about the walks so commonly attest, spect-​actors encounter moments of spontaneous correspondence between Cardiff ’s “writing” and the events unfolding nearby. Less noted, however, are the ways in which such correspondences also serve as a basis for aesthetic dissonance and narrative digression. Indeed, at the rhythmic core of Cardiff ’s sonic arrangements is a programmatic dance between correspondence and dissonance—​between “the seen and the imaginary” (Scott 14). It is an incidental choreography performed around two kinds of enigmas: (1) singular accidents that Cardiff cannot totally foresee (e.g., random environmental noises which listeners may mistake for composed audio); and (2) Cardiff ’s chronic narrative digressions that conflate the present scene with imaginary events unforeseeable to the spect-​actor. The persistence of the latter confers the artistic, literary status of her walks. One moment, Cardiff plainly references a church or statue in your field of vision; then her voice purports to describe, in the same nonchalant tone, things that no listener is likely to actually see: “you can see the church steeple . . . barbed wire broken windows, men with guns and face masks, fires all around” (The Missing Voice (Case Study B), my emphasis). These imaginary events, cognitively reinforced by corresponding sound effects, are projected onto the present scene only after Cardiff establishes a sense of common ground with the spect-​actor. The bulk

029

Creating Actionable Media  • 

209

of The Missing Voice (Case Study B) proceeds like a pendulum. Cardiff ’s narration sways from referential directives into audiovisual mirages and back again. Neither mode continues for longer than ten or twenty seconds, as if Cardiff set a constraint for herself. This back and forth, fort-​da pattern is fundamental to the walk’s aesthetic. Affirming a connection to the setting—​confirmed by the spect-​actor’s own observations—​allows Cardiff to embed her authorial fragments into “the singularity of your perception and experience” (Scott 4). She adds a voiceover and soundtrack to the scenes you are living through. That which is not actually occurring emanates directly from what is there. The narrative digressions, therefore, are taken in stride: the spect-​actor may regard them either as a past event that has conditioned the present, or as an impending event that will have been conditioned by what is here and now. Manovich has such moments in mind when he assesses the efficacy of Cardiff ’s walks, “the virtual [i.e., the imaginary] becomes a powerful force that reshapes the physical” (227). In other words, Cardiff ’s digressions into the imaginary thus prompt spect-​actors to do more than simply “tune in” to their surroundings. She also positions us to inhabit an/​other stream of consciousness—​to perceive present events in light of the memories, sensations, and phantasms she has cognitively mapped onto our proximity. This mode of interfacing with local action conjoins recorded audio and primary retention according to a grammar that is much more complex than that of a conventional tour guide. The latter speaks to the scene in the manner of an additive conjunction (e.g., and this happened, as well as this). The scene is treated as a stable reference point, a geospatial bookmark ordinating the delivery of historical re-​presentation. Park rangers at a monument and museum docents in a gallery rarely stray from the additive register: “While we are standing here, let me tell you about past events that also happened here . . . As you view this painting, consider some biographical information about the artist.” This is the syntax of edutainment. No problems posed, few questions raised. Cardiff ’s audio walks, on the other hand, engage the scene with the contentious force of adversative conjunctions: but, yet, however, etc. Like all conjunctions, adversative conjunctions link different clauses together in the same unit of discourse (e.g., a sentence), but the fusion they render also manifests a critical tension. They are connectors that at once signal discord. Cardiff ’s narrative digressions effectively insert adversative conjunctions into the flow of her multimodal passages, each of which is composed partially by the verbal clauses she utters and partially by the material clauses at play in one’s surroundings. When the relationship between her audio layer and the present event shifts, from correspondence to dissonance, Cardiff syntactically compels

120

2 1 0   • Act i o n a b l e   M e d i a

the spect-​actor to attend to enigmas, contrasts, contradictions, and paradoxes. Likewise, if actionable media producers aim to invoke critical or deliberative positions for spect-​actors to inhabit, then their work must forge adversative bonds between networked multimedia and local action, not just additive ones. Intellectual discourse thrives on adversative conjunctions. A scholarly audio walk must be punctuated with the critical connectivity that “but” and “however” perform in the discursive economy of a printed page. Cardiff ’s narrative digressions provide a stylistic model better suited to humanities work than the more familiar art/​history audio tour. Whereas such tours congeal around additive background information, Cardiff ’s audio walks incite moments of speculation, analysis, subversion, and perplexity. How else, then, might the incidental choreography through which Cardiff “writes events” inform actionable media production across the humanities? Most broadly, her audio walks bespeak a dynamic genre that may become to wearable computing what the essay has been for journal publishing. Compared to public interactives or AR layers, the audio walk seems readily amendable to the traditional mainstays of essayistic expression: authorial voice, structural cohesion, linear progression, etc. In turn, audio walks prompt reconsiderations of the rhetorical conventions that have guided essayistic expression, in print and on the Web. Instead of being bound within a collection of documents, the audio walk binds authorial voice to ecologies of local action. Recorded utterances get delivered in situ, and a work’s formal coherence becomes subjected to new measures. Namely, rhetors must communicate, theorize, analyze, and make arguments with/​in the proximate scene. Discourse that does not cohere to (and brush up against) the present event can become incoherent to pedestrian spect-​ actors moving through the built environment. Familiar “topic outlines” for arranging paragraphs do not translate very well to these emerging rhetorical situations. Expound on one abstraction for too long and you may lose an urban pedestrian’s attention, since she (unlike the reader at a desk) cannot afford to be fully immersed in your words to the neglect of her surroundings. Recall Barthes’s adage: “The present event can only furnish marginal fragments of writing.” As we have seen, one technique for achieving structural coherence in such cases is to weave a linear progression of fragments that intermittently ground one’s voice in the scene (correspondence) and then project complications onto it (dissonance). Public intellectuals may appropriate Cardiff ’s fort-​da rhythm, mobilizing this key element of her aesthetic in the service of research questions and found objects associated with one’s expertise.

21

Creating Actionable Media  • 

211

The idea is to do criticism or theory en plein air—​not only gathering data or examining sources through site-​specific fieldwork, but actually performing your inquiry and publishing your argument on-​site, too. In situ theory building, nascent as it may seem, is actually as old as theory itself, or “theoria” as the pre-​Socratics called it. Deeming the practitioners of theoria “the first theorists,” sociologist E. V. Walter describes their method as “a complex but organic mode of active observation—​a perceptual system that included asking questions, listening to stories and local myths, and feeling as well as seeing and hearing” (18). Ancient theoria groups closely attended to places in order to, as Gregory Ulmer notes, investigate problems of communal interest and to discover causes, gain insights, or identify available courses of action.7 When their observations were complete, they returned to their communities and gave a report in the agora. Modern academic research tends to replicate this process: scholars examine sources (be they sites, texts, or human subjects), then articulate their inquiries or findings qua differed archives that are spatiotemporally removed from those sources. Remaking intellectual discourse in the forms of actionable media beckons a return to theoria that radically extends its experiential orientation to the entire lifecycle of research, particularly to processes of composition, circulation, delivery, and reception. Experiential genres being pioneered by scholars in place-​rooted fields such as cultural geography offer promising glimpses into actionable modes of knowledge work lying in wait at the nexus of ancient theoria and sound art practices like Cardiff ’s. For example, urbanist Anja Bieri (who rightly calls herself an “artist-​scholar”) has developed “a pedagogy and research program  .  .  .  centered around the creative process of conceptualizing, producing, and analyzing audio walks” (2). Bieri’s critical audio walks, as well as those composed by her students, draw upon technologies and techniques similar to those Cardiff employs. The key differentiator is their rhetorical intent. Rather than elicit aesthetic experience through narrative digressions, Bieri’s audio walks aim to introduce the spect-​actor to concrete, street-​level case studies that depict the social affordances and de facto consequences of urban planning/​policy decisions in particular cities. She establishes correspondence through in situ analyses of each example, then creates dissonance by bringing disciplinary theories to bear on (or inducing concepts from) the proximate scene in hopes of formulating new foci for community-​oriented development initiatives, as well as new questions for academic research (Bieri 5). Producing a “sensory ethnography,” the researcher or student who follows Bieri’s approach “attempts to gather the multiple layers of geography, history and culture that structure the urban fabric, in order to understand how we

21

2 1 2   • Act i o n a b l e   M e d i a

created this and how it creates us” (2). If one’s raison d’être is to alter, challenge, or enrich how people understand and perceive situated social practices, then making critical audio walks in public settings holds obvious advantages that may compliment the work of publishing articles in specialized journals. Moreover, the ongoing “crisis in the humanities” seems to necessitate that a greater number of scholars diversify their research output (i.e., reallocating at least some time and energy toward genres that better address local publics and nonacademic stakeholders). It would buttress the public perception of the humanities if more urban historians, in addition to writing scholarly monographs and trade books, occasionally collaborated on critical audio walks that wove their insights about a city or landmark into the very fabric of these well-​ trodden places. Notwithstanding institutional pressures, we have the means to infill urban space with some of the breadth and depth of knowledge that our libraries contain. Following Bieri’s lead, scholar-​teachers and students across the humanities may begin by asking themselves which of their objects/​ topics of study best lend themselves to actionable modes of analysis. Few disciplines are as tailor made to in situ inquiry as cultural geography or urban history. Nevertheless, experiments may be forged from current intellectual trends now pushing many disciplines to grapple extensively with issues of materiality, embodiment, public discourse, and spatiality. Rhetoric and communication studies abound with such bourgeoning approaches: new materialism, distributed cognition, public memory, posthumanism, object-​oriented ontology, actor-​network theory, ecocomposition, and civically engaged pedagogies (to name just a few). Each of these field-​changing frameworks introduces theories, methods, and concerns that appear well suited to actionable cultural criticism. Consider, for instance, Jenny Rice’s brilliant analyses of the rhetorical ecologies underlying public discourse in Austin, Texas. In “Unframing Models of Public Distribution” and more recently in her book Distant Publics, Rice tracks the collective affects expressed, adopted, and remixed by various iterations of the slogan “Keep Austin Weird,” as well as the formation of public sentiments about development cultivated by local news reports and city blogs (e.g., the evolution of local enthymemes related to the gentrification of Austin’s East Side neighborhood). In addition to examining public rhetorics pertaining to urban sprawl, Rice champions the notion that rhetorical analyses of community discourses can spur local action: My approach, which I call a “publics approach,” understands publics and their discourse as the best site for making interventions into material

213

Creating Actionable Media  • 

213

space. In other words, rhetorical theory and rhetorical pedagogy can make a difference to the current development crisis not by interrogating “place” but by helping to shape different kinds of subjects who can undertake different kinds of work. (Distant Publics 7–​8, my emphasis) The bulk of Distant Publics shows Rice carefully discerning appeals and feelings recurring in Austinites’ arguments about their rapidly growing city. Driven by rhetoric’s “powerful commitment to repairing and strengthening public discourse,” Rice’s case studies highlight a disconcerting political subjectivity, which she terms “exceptional public subjects” (68). This figure arises from “a space of exclusion that allows subjects to remain ‘publicly oriented’ while also being distanced from the kind of rhetorical gestures necessary for sustainable interventions” (62). Dwelling on this definition, I cannot help but lapse into a thought experiment: what if Rice’s rhetorical analyses, in addition to being expressed as a scholarly monograph, were also adapted and remade to circulate amid Austin’s public spaces qua actionable media? This would not be a simple cut-​and-​paste endeavor, of course. The process, akin to film adaptations of literature, would involve feats of fragmentation and concision, reconfiguring passages around their relations with action occurring at a given site. The words would no longer need to represent dynamic, audiovisual phenomena. Rhetorical analyses of public discourse about Austin’s East Side, which could assume the form of on-​site audio walks through that very neighborhood, would position Rice’s remarks in an altogether different rhetorical ecology. Coupled with other forms of actionable media, Rice’s insights could be tagged to and layered over her objects of study. Her remarks about the “Keep Austin Weird” campaign would literally circulate with all the T-​shirts, billboards, and bumper stickers bearing the logo. Optimized for ATLAS interfaces, both technically and rhetorically, her intellectual contributions could bleed more pervasively into everyday life—​namely, the local actant-​networks in which much of her prose aims to intervene. Imagining the possibilities and challenges potential to actionable criticism, such as rhetorical analyses of public argumentation performed in situ, begs fruitful questions about the future of intellectual discourse (which I hope will not be misconstrued as a call to abandon or devalue traditional genres). While the academic monograph will remain essential for making conceptual contributions to every humanities field, how effective is it as an outlet for “making interventions into material space” and “shap[ing] different kinds of subjects” actually implicated in situations like Austin’s development

214

2 1 4   • Act i o n a b l e   M e d i a

crisis? Do our disciplinary rituals and institutional compulsions around differed archives also distance us “from the kinds of rhetorical gestures necessary for [making] sustainable interventions” into the local communities and social problems that so often motivate our research? Scholarship absolutely requires venues where advanced researchers can address each other without having to make their work enjoyable for generalists. Insularity of this sort promotes complex interchange. Problems ensue, however, when we lack and/​or devalue complementary modes for communicating scholarly insights to broader public audiences. The gulf widens. We perhaps begin to doubt the social importance of our own expertise—​or, at least, many people outside the humanities do. This is both unfortunate and absurd. No other set of disciplinary traditions is more central to the human experience. Diversifying the genres that shape humanities research output is paramount to cultivating a more consequential presence in vibrant public settings, which the forms of actionable media may uniquely enable. The recent push toward open access Web-​based publication is helpful, but it may prove insufficient if apps continue to eclipse websites. Differed archives, no matter how accessible they might be, are not designed to engage readers who read while doing something else. As networked populations around the world continue their post-​desktop migrations—​as we increasingly read and write while doing—​humanities scholars who value public engagement will do well to consider creating actionable media designed to circulate across ATLAS interfaces. Monet, Boal, and Cardiff each model key considerations that may inform rhetorical thinking and digital humanities work amid these emerging exigencies. Many more stylistic models, relevant to actionable media, remain to be analyzed, appropriated, and learned from. The short list I have explored here is ultimately a call to fellow humanists concerned with the plight of critical expression in the twenty-​first century. Rhetoricians, designers, and media theorists will continue, I hope, to identify precedents and exemplars from various domains of cultural production in a collective effort to amass rhetorical-​ aesthetic principles that help scholars and students take to the streets—​to create intellectual discourse with/​in the scenes of everyday life.

Notes 1. Stephen Ramsay, a leading digital humanities scholar, has gone so far as to claim that building or making critical media artifacts (software, visualizations, databases,

251

Creating Actionable Media  • 

215

etc.) is the quintessential, field-​defining activity of digital humanities research. At MLA in 2011, Ramsay’s remarks cast a bold (and highly disputed) vision of the field: “Personally, I think Digital Humanities is about building things. I’m willing to entertain highly expansive definitions of what it means to build something. I also think the discipline includes and should include people who theorize about building, people who design so that others might build, and those who supervise building” (“Who’s In and Who’s Out”). 2. Media aesthetics, as a mode of inquiry, “can focus on how we perceive the world in and through new technologies and new forms of media” (Bolter et al. 39). Considering issues of media aesthetics as a matter of course in the process of producing scholarship may lead many (future) humanists to choose different media on a project-​by-​project basis, selecting a given technology or form in an effort to craft an intentional platform for reading/​viewing experiences, or to enact a certain kind of cognitive/​cultural intervention. Conventionally, scholars have largely bypassed the media aesthetics of their own scholarly work, quite understandably, since most universities and departments still prize journal articles and print books above all other genres. 3. For a comprehensive introduction to Greenspan’s research-​ creation work at Carleton’s Hyperlab, see his essay “Songlines in the Streets:  Story Mapping with Itinerant Hypernarrative.” 4. When asked in an interview to explain her concept of “intra-​action,” Karen Barad offered the following definition: “The usual notion of interaction assumes that there are individual independently existing entities or agents that preexist their acting upon one another. By contrast, the notion of ‘intra-​action’ queers the familiar sense of causality (where one or more causal agents precede and produce an effect), and more generally unsettles the metaphysics of individualism (the belief that there are individually constituted agents or entities, as well as times and places). According to my agential realist ontology . . . ‘individuals’ do not preexist as such but rather materialize in intra-​action. That is, intra-​action goes to the question of the making of differences, of ‘individuals,’ rather than assuming their independent or prior existence” (Barad qtd. in Kleinman 77). 5. See Walter Ong’s Ramus, Method, and the Decay of Dialogue for a thorough assessment of the egregiously reductive orientation of Peter Ramus’s influential diagram-​ driven pedagogy. 6. Of course, any suggestions that a mobile app like Action Path marks a promising mode of democratic deliberation are likely to provoke valid objections referencing the digital divide. At the same time, we must remember that, while more conventional civic practices (e.g., city council hearings) may carry less technological barriers to public involvement, they nonetheless entail daunting access hurdles by requiring interested citizens to attend meetings at a set day and time. 7. Watch The Ulmer Tapes for Gregory Ulmer’s more extensive discussion of theoria groups in ancient Greece and their value as a model for contemporary humanistic inquiry.

126

EPILOGUE

Kairotic Intellectuals

Throughout Actionable Media, I have argued for a certain way of understanding the current technocultural paradigm shift and its relevance to intellectual work in media studies across the humanities. Ubicomp is not merely the latest digital revolution. The interfaces it has spawned differ profoundly from conventions of archivization and communication that have generally held sway since the invention of writing, well before the rise of personal computers. As a rhetorical framework, actionable media apprehends post-​desktop milieus on the basis of long-​marginalized archival modalities, emerging metonymical cultural forms, and stylistic principles that eschew the studio for the streets. Technologists, designers, artists, and storytellers have been the overt protagonists of my case studies, while humanities intellectuals think things over in the wings, lending a theory here and a history there. But what is happening to intellectuals amid this play of emerging technologies and practices? How will intellectual discourse have changed—​whether it takes technology as its subject matter or not—​when ubicomp reaches the climax of its global adoption? The concepts, examples, and analyses assembled here provide reference points for speculation and invention, but they do not portend a firm answer. Still, we may consider the question in earnest. As a parting thought, I want to revisit a proposition wagered more than once in the proceeding chapters: that intellectual discourse may, here and there, be productively reconfigured around the affordances of actionable media—​and furthermore, that ATLAS-​enabled genres of critical expression hold some potential to more visibly and consequentially situate humanities knowledge among the scenes, exigencies, and dilemmas of everyday life in public settings.

217

Epilogue  • 

217

For actionable media production to gain currency as a modality of cultural criticism, it needs to be properly characterized as an intellectual endeavor. It must be understood in light of a tradition and set of values that run complimentary to (and distinct from) more vaunted outlets. The present relationship between academic scholarship and public intellectualism is a relevant case in point. Texts authored in the latter vein rarely “count” as scholarly research, yet a review essay published in The Atlantic carries great capacity to bring disciplinary insights to bear on issues that matter to general audiences. Only the most insular of academics will dismiss the social value of such work. The “public intellectual” has become a well-​defined, accepted role within which scholars are encouraged to do some of their writing. Writing as a public intellectual entails writing differently. Cultivating a “different relationship to the lay public” requires authors, as Russell Jacoby insisted, to compose “in a public prose” (7). Departing from academic conventions, in this sense, can merit another kind of intellectual payoff, underwritten by an alternative ideal. If, as I  have argued, actionable media constitute significantly different rhetorical ecologies than the deferred achieves that host so much of our intellectual work, then we are going to need another ideal, another concept of authorship that corresponds to the notion of delivering one’s insights, criticism, and commentary amid the contingencies of a particular site of local action—​be it a museum, park, plaza, or wherever. The public intellectual concept is relevant, but it doesn’t quite fit. Typically, public intellectuals become defined as such whenever their writing reaches a broad audience, well beyond the confines of a specialized field. The label signifies a high degree of circulation and influence, which public intellectuals earn to the extent that they make their prose accessible and engaging, and to the extent that they address topics that are of national or global interest. While actionable media projects are situated in public settings, they do not follow this model of circulation. They do not go viral. They are often bound to a single locale. To foreground these distinctions, I  propose the term “kairotic intellectual” as a conception of authorship proper to actionable media production. The kairotic intellectual signals an emerging agenda, whereby students and scholars may employ ubicomp technologies for the in situ generation and dissemination of humanities knowledge. Ultimately, actionable media projects appear to be opening up new channels through which intellectuals may communicate with general readers and external audiences on their own turf. It’s possible to imagine a future in which any type of object in our surroundings may be digitally annotated by cultural critics. Urban spaces could play host to embedded rhetorical analyses as much as they currently harbor commercial

218

2 1 8   • Ep i l o g u e

and municipal signage. That is, in addition to writing essays, books, and blog posts about the objects we study, we might also refashion our work as attachments to those objects. So, for instance, if I’m writing an institutional critique, I could link actionable bits of my criticism to that institution’s building or logo, such that wherever people encounter the logo they might also opt to see or hear the critique. Criticism, crafted qua actionable media, may reverberate with lived scenes akin to the way Walter Benjamin’s red neon sign reflected against the wet asphalt (85–​86). Our discourse may be unbound from document-​to-​document bindings, and become bound instead to built environments and the networked objects, surfaces, and flows that comprise them. Kairotic intellectuals pursue humanistic inquiries that not only address a general audience. Their work performs in the streets; it interfaces with local action amid highly public settings. This is not to say that public intellectuals are unconcerned with kairos. Articulating timely responses to timely issues is what public intellectuals are all about. They are thoroughly kairotic, if we take kairos to be a purely temporal matter. More robust theories of kairos are quick to stipulate, however, that the concept means to call rhetors’ attention to the contingencies of time and place.1 The contemporary emphasis on kairos as timeliness, which dominates rhetoric textbooks and popular definitions of the term, seems to be a function of our publishing habits. Whenever we publish something in print or on the Web, readers may access it from almost any point in space. I can take a book or ebook with me to the bathroom or the beach. Likewise, most material on the Web is accessible on a more or less global basis. When publishing in these venues, there’s no way for a writer to limit or specify the places in which readers will encounter her work. The contingencies of place are outside the bounds of rhetorical decision-​making. Ubicomp technologies foreground relationships between texts and their proximities, bringing the contexts of reception back into the rhetor’s purview. Actionable media operates like speech in this regard: utterances are in both cases resolutely bound to a single proximity. You have to be within earshot of the speaker to hear the speech. (Speech that is recorded and broadcast is no longer speech in a full sense—​recorded speech circulates more in the manner of writing.) To experience Janet Cardiff ’s Missing Voice (Case Study B), you have to walk the streets on London’s East End; to engage with my students’ augmented reality projects on the cultural history of animals, you have to be in a particular wing of the Denver Museum of Nature and Science. From the perspective of writing and the massive scale of circulation it enables, this fidelity to proximity is seen as a shortcoming, a limit that we don’t need to

219

Epilogue  • 

219

abide by. On the other hand, from the standpoint of spatial kairos, texts that circulate without condition bear a different shortcoming, which can be just as limiting: they are written without any concern for how they may or may not resonate with the readers’ proximity. Traditionally, questions of spatial proximity—​of how one’s text might inform or be informed by the reader’s actual surroundings—​have been neglected because they cannot be answered. It is impossible to predict the settings in which a reader, let alone multiple readers, will read one’s book or website. Creating actionable media, by contrast, often involves the demarcation of a reading space that is beholden exclusively to a particular setting. Thus it becomes possible, even paramount, for a rhetor to account for some of the local action at play in that place. In such instances, the singular site in which the text is set will always shape readers’ experiences of that text. The contingencies of local action can no longer be regarded as mere contingencies—​they are not extraneous to the work; they are absolutely indivisible from it, in the same way a timely moment may condition the invention and affect the interpretation of spatially unbounded texts. These contrasting valuations of spatial proximity and its rhetorical importance are quite related, I think, to some distinctions at the core of Hannah Arendt’s philosophy of action. While Arendt wrote at the summit of academic rigor and in the most public outlets of her era, she also gestured toward a third path for intellectuals. In theorizing this latter dimension, Arendt unearths a countertradition, a set of values, and objectives that are helpful for characterizing actionable media production as a kairotic intellectual endeavor. It begins with Arendt’s contentious excavation of vita activa and vita contemplativa, a conceptual dichotomy she chronicles over the course of a grand narrative in The Human Condition. Arendt regales the rise of vita activa in the early Greek polis, only to lament its perennial decay in societies that followed. Her ambition is to mitigate this long-​standing divide between action and contemplation, and to convince her intellectual peers to assume more active roles in public affairs. (Before I delineate the terms of the opposition, here’s a quick stipulation about the concept of authorship I’m proposing: the kairotic intellectual pursues inquiry in the manner of vita activa, in addition to the contemplative traditions.) Championing vita activa—​life of action—​over a life of pure contemplation, Arendt advances a critique of Western philosophy and Christianity. From Plato to St. Augustine, philosophers and theologians valorized the contemplative life (Arendt 14). Under this perspective, participating in public affairs was seen as a chore, like mowing the lawn once a week. You just had to do enough of it to maintain the kind of social order

20

2 2 0   • Ep i l o g u e

that would allow for uninterrupted study, meditation, or prayer (Arendt 16). Serious thought and critical reflection become synonymous with places and postures of withdrawal, within which the thinker might better contemplate transcendental truths (e.g., the forms, God, spirit) without being distracted by the mere shadow plays of the material, social world. The momentum of vita contemplativa swept philosophy out of the city streets—​where Socrates and Diogenes had once performed it—​and into the Academy, just outside the walls of Athens, where Plato and Aristotle would professionalize it. The same ethos of contemplation is celebrated in Thoreau’s spirited retreat to the Concord woods, especially in his Walden chapter “Reading.” For Thoreau, close reading (or “reading, in the high sense”) is defined in opposition to “the trivialness of the street,” “the fleeting spoken language,” and “the event and the crowd which inspire the orator” (353–​355). The orator is too immediately situated, too much in the midst of things, such that he unwittingly “yields to the inspiration of a transient occasion, and speaks to the mob before him,” rather than to “the intellect and heart of mankind, to all [true readers] in any age who can understand him” (Thoreau 355). True reading entails sacrificing our most vital energies to the bound page: “it is worth the expense of youthful days and costly hours . . . It requires . . . the steady intention almost of the whole life to this object. Books must be read as deliberately and reservedly as they were written” (Thoreau 353). Thoreau suggests that private communions with classical texts yield much greater insight than the conversations he has with fellow townsmen, who often become the butt of his jokes. Throughout the chapter, the temporal lineage of contemplation is privileged over the local space of action: “I aspire to be acquainted with wiser men than this our Concord soil has produced . . . questions that disturb and puzzle and confound us have in their turn occurred to all the wise men; not one has been omitted; and each has answered them” (359–​360). Words preserved across time are thought to transcend the contingencies of place, context, and locale. It is thought best to read and write in a place removed from one’s contemporaries—​at the serene edges of Walden Pond, or under the shade of trees outside the walls of Athens.2 Arendt believes that the historical privileging of contemplation over action led to a major shift in civic values and communication practices. From the contemplative standpoint, freedom meant being “free from entanglement in worldly affairs” (14).3 What had been the raison d’etre for earlier Greek citizens—​civic involvement was once regarded as the highest expression of human freedom in the early polis—​eventually became something to be avoided at all costs. In fact, Arendt notes that the kairotic foundation of

21

Epilogue  • 

221

action was already starting to unravel when the Greek polis was at its apex, at the dawn of alphabetic literacy. To Arendt’s mind, the literate revolution in Ancient Greece exacerbated the division between action and speech, even among citizens who did not write: In the experience of the polis  .  .  .  and even in the political philosophy which sprang from it, action and speech separated and became more and more independent activities. The emphasis shifted from action to speech, and to speech as a means of persuasion rather than the specifically human way of answering, talking back, and measuring up to whatever happened or was done (26, my emphasis). Speech (especially democratic deliberations) tended toward codified venues like the forum and the agora; oratory becomes enunciated and framed within a discrete, relatively closed communication space rather than a circumstantial, on-​the-​spot mode of “answering, talking back, and measuring up to whatever happened.” Collective contemplation is progressively bracketed off from local action. The public spheres of vita contemplativa—​including the French salons and British coffeehouses discussed in Jürgen Habermas’ The Structural Transformation of the Public Sphere—​are largely enclosed. In each case, speech becomes more like writing; utterances happen in a distinct space that is structurally walled in from whatever might be happening around it. This tendency amplified over the course of the twentieth century, as mass broadcasting and the Web both forged even greater gulfs between action and speech.4 Recorded speech, in the form of radio and Web-​based audiovisuals, turn speech into a written sign that is iterable and citational: recorded speech can be repeated as is across context, ad infinitum, always retaining its essential orthographic character. For Arendt, the grammatization of speech effectively constrains and erodes our capacity to take action in the public realm. As a result, Arendt suggests that, today, we don’t really know what action is and, furthermore, that action is barely even possible in modern societies. So, what does “action” mean for Arendt and her notion of a public realm? And why does she regard action as an endangered species of human behavior? What I  find crucial is the way Arendt defines action by relating it to speech. Throughout The Human Condition, Arendt claims that action and speech are absolutely interdependent. She describes them in tandem. For instance, action and speech are “indissolubly tied to the living flux” and they can “only take place in the presence of others” (187–​188). And, most strikingly, she writes that action is a matter of “finding the right words at the right

2

2 2 2   • Ep i l o g u e

moment, quite apart part from the information or communication they may convey” (26). Arendt’s theory of action is essentially kairotic. Action is measured as a function of kairos. And, given the fact that words almost always manifested through speech during the Hellenic period (and even into what McLuhan called “the manuscript era”), this idea of kairos as the opportune moment also carries a necessary spatial dimension. Speech is indissolubly contextual: it’s bound to the body of the speaker, limited by the reach of the voice. If, as Arendt insists, action is a matter of “finding the right words at the right moment,” then we should stipulate something else about action: it is also—​ implicitly and necessarily—​a matter of delivering those words in the right place (26). Kairos is also a matter of opportune proximity. To count as action, words and deeds must be uttered and performed freely amid shared locales of the public realm. Actionable media projects, when crafted by kairotic intellectuals, exhibit digital texts in a manner reminiscent of speech. That is to say, the entire multimodal spectrum of networked texts, audio, images, and video circulate like spoken words, as they become invented, arranged, and delivered in response to—​in opportune proximity amidst—​live, local action. What makes ubicomp unique, as an emerging cultural archive, is its capacity to sync digital media with local activity—​to remain “tied to the living flux.” Existing deferred archives may also be made actionable; new archives can be made actionable from the outset. All archives, the intellectual performances they shelve, are now primed to spill out from library stacks and server farms, and flood (back) into the streets. In turn, a significant political affordance of actionable media is that it enables students, scholars, artists, and writers to position texts and audiovisuals in their most kairotic environments. Intellectuals may publish and circulate their work at whatever proximity seems most opportune. Historically, this capability has been restricted to commercial advertising and municipal messaging. Via ATLAS forms, digital content can be overlaid, geotagged, or embedded at the scene of local action—​without physically altering the material composition of real estate. Whereas print media cultures have blanketed the built environment almost exclusively in corporate-​or state-​sponsored signage, ubicomp-​saturated “smart cities” may play host to a variety of content created by a gamut of contributors. At the very least, there will be fewer technological constraints in place to limit the right to author the public realm. Proximate, polyphonic expression is, according to Arendt, the originary condition of a public realm: our shared infrastructure only becomes a public realm to the extent that multiple perspectives articulate themselves in opportune proximity to a common object of perception. Arendt warns, “The end of

23

Epilogue  • 

223

the common world has come when it is seen only under one aspect and is permitted to present itself in only one perspective” (58). By Arendt’s measure, most of our built environments do not make the cut. The sphere of authorship is too homogenous or too restricted to qualify as a public realm. If the only embedded texts we gather around in public are stop signs and product packaging, then we are not inhabiting a public realm. The public realm strictly excludes “everything [that is] merely necessary or useful,” the set of practices which Arendt collects under the categories of labor and work (25). Democracy happens through action or not at all. It’s not contemplated, nor fabricated. It is only performed, within polyphonic locales, where anyone may deliver “right words [or any media] at the right time” in the right place. This is where the technology and the terminology may be pushed, in small ways by many people, to converge in the service of a civic agenda. As an authorial ideal, the notion of the kairotic intellectual may buttress diverse experiments with actionable media across the humanities, and it may be taken up by educators who wish to democratize the rhetorical power to deliver right words at the right time in the right place. From a kairotic intellectual standpoint, the purpose and value of actionable media production is not about infusing physical spaces with digital media, as if extending the digital frontier were a progressive end in itself. Rather, ubicomp interfaces and ATLAS forms are promising insofar as we harness their capacity to support a plurality of expression amid the public realm. It is precisely at the threshold of this historic, political undertaking that we may stake the potential merits of kairotic intellectual work. For far too long, our built environments have presented themselves in only one or two perspectives.

Notes 1. Most notably, the rhetorician James Kinneavy has led the field toward more robust understandings of kairos and its concern with the contingencies of place (in addition to timing). In his last interview, Kinneavy makes it a point to remind us of the spatial aspect of kairos: “By the way, a second meaning of kairos was ‘the right place’ in addition to the right time” (Kinneavy qtd. in Thompson 83). 2. For all Thoreau’s praise of the contemplative ideal in “Reading,” his following chapter “Sounds” speaks to the limits of vita contemplativa. “Much is published, but little printed,” he writes, advising us to “read” the world around us with as much vigor as we bring to books: “What is a course of history or philosophy, or poetry, no matter how well selected . . . compared with the discipline of looking always at what is to be seen?” (363). The fidelity to local action and materiality, which Thoreau espouses here, carries over from his reflections on reading to his remarks about writing, as he

24

2 2 4   • Ep i l o g u e

observes the cargo of a passing train: “This carload of torn sails is more legible and interesting now than if they should be wrought into paper and printed books. Who can write so graphically the history of the storms they have weathered as these rents have done?” (371) Why write upon paper when the objects, from which paper would be fabricated, already harbor eloquent significations (in this case, the indexical trace of storms weathered)? Why not write upon, over, or next to those actual objects as opposed to blank pages? Thoreau, of course, didn’t have an app for that. In “Sounds,” he does the next best thing—​he recounts his critical mediations about the scenes occurring in his proximity. The passing train and its whistle trigger assertions about the effect of railroad travel on the temporality of nineteenth c​ entury cities. Amid the sensory presence of the train before him, he exclaims, “The startings and arrivals of the [railroad] cars are now the epochs in the village day. They go and come with such regularity and precision . . . that the farmers set their clocks by them, and thus one well-​conducted institution regulates a whole country” (369). Thoreau repeats this practice of actionable analysis/​speculation with the following sounds and sights he encounters in the woods around him: church bells, cows, frogs, and several species of birds. In each case, his will to contemplate clings to particular sources of local action (or actant-​networks) in his proximity. In “Sounds,” Thoreau is a kairotic intellectual, or at least, he is wanting for the means to become so. 3. For a meticulous analysis of the contemplative ideal in Plato’s thought, see Catherine McKeen’s essay “ ‘Standing Apart in the Shelter of the City Wall’: The Contemplative Ideal and the Politically Engaged Philosopher in Plato’s Political Philosophy.” 4. In “Further Reflections on the Public Sphere,” Habermas draws a prescient analogy to describe what could be regarded as the Web’s structural transformation of the public sphere: “Many of the features of our ‘Information Age’ make us resemble the most primitive of social and political forms: the hunting and gathering society. As nomadic peoples, hunters and gatherers have no loyal relationship to territory. They, too, have little ‘sense of place’; specific activities are not totally fixed to a specific physical settings. The lack of boundaries both in hunting and gathering and in electronic societies leads to many striking parallels” (456).

25

BIBLIOGRAPHY

Andersen, Christian Ulrik and Søren Pold. “The Scripted Spaces of Urban Ubiquitous Computing:  The Experience, Poetics, and Politics of Public Scripted Space.” Fibreculture Journal 19, 2011, http://​nineteen.fibreculturejournal.org/​fcj-​133-​ the-​scripted-​spaces-​of-​urban-​ubiquitous-​computing-​the-​experience-​poetics-​and-​ politics-​of-​public-​scripted-​space/​, Web. Accessed 22 Jan. 2012. Anderson, Chris, and Michael Wolff. “The Web is Dead. Long Live the Internet.” Wired.com, 17 Aug. 2010, https://​www.wired.com/​2010/​08/​ff_​webrip/​. Web. Accessed 3 Feb. 2011. Anderson, Monica. “Six Facts about Americans and their Smartphones.” Pewresearch. org., 1 Apr. 2015, http://​www.pewresearch.org/​fact-​tank/​2015/​0 4/​01/​6-​facts-​ about-​americans-​and-​their-​smartphones/​ Web. Accessed 4 May 2015. “Annual Estimate: 2010.” Nsc.org. Distracted Driving Resources. Accessed 2 Nov. 2012. “ARCs:  Augmented Reality Criticisms.” Ufl.edu. Trace Innovation Initiative at the University of Florida. https://​trace.english.ufl.edu/​projects/​arcs/​ Web. Accessed 2 Mar. 2016. Arendt, Hannah. The Human Condition. University of Chicago Press, 1958. Print. Aresty, Abby. “Notes.” Abbyaresty.com, Paths II:  The Music of Trees, 25 Nov. 2012, https://​abbyaresty.com/​projects/​pathsii-​themusicoftrees/​ Web. Accessed 11 Jan. 2013. Auslander, Philip. Liveness:  Performance in a Mediatized Culture. Routledge, 2008. Print. Bardzell, Shaowen. “Feminist HCI: Taking Stock and Outlining an Agenda for Design.” Proceedings of the 2010 SIGCHI Conference on Human Factors in Computing Systems, edited by Elizabeth Mynatt, ACM, 2010, pp. 1301–​1310. Print. Baron, Dennis. A Better Pencil: Readers, Writers, and the Digital Revolution. Oxford University Press, 2009. Print. Barthes, Roland. The Pleasure of the Text. Translated by Richard Miller, Hill and Wang, 1975. Print.

26

2 2 6   • 

Bibliography

Barthes, Roland. S/​Z :  An Essay. Translated by Richard Miller, Hill and Wang, 1975. Print. Barthes, Roland. Image-​Music-​Text. Translated by Stephen Heath, Hill and Wang, 1978. Print. Barthes, Roland. Camera Lucida:  Reflections on Photography. Translated by Richard Howard, Hill and Wang, 1981. Print. Barthes, Roland. The Rustle of Language. Translated by Richard Howard, Oxford: Basil Blackwell, 1986. Print. Baxandall, Michael. Giotto and the Orators: Humanist Observers of Painting in Italy and the Discovery of Pictorial Composition. Clarendon, 1986. Print. Bell, Genevieve, and Paul Dourish. “Yesterday’s Tomorrows:  Notes on Ubiquitious Computing’s Dominant Vision.” Personal and Ubiquitous Computing vol. 11, no. 2, 2007, pp. 133–​143. Print. Bennett, Jane. Vibrant Matter:  A  Political Ecology of Things. Duke University Press, 2010. Print. Bennington, Geoffrey. “Emergencies.” Oxford Literary Review vol. 18, nos. 1–​2, 1996, pp. 175–​216. Print. Bieri, Anja. “Audio-​Walks: Documentary Art for Planning Education and Research.” Anjahaelgbieri.files.wordpress.com, 2 Nov. 2012. Web. Accessed 14 May 2014. Biermann, B.C and Jordan Seller. “The AR|AD Takeover:  Augmented Reality and the Reappropriation of Public Space.” Academia.edu, 27 Oct. 2011, https://​www. academia.edu/​756642/​The_​AR_​AD_​Takeover_​Augmented_​Reality_​and_​the_​ Reappropriation_​of_​Public_​Space Web. Accessed 4 December 2012. Boal, Augusto. The Theatre of the Oppressed. Translated by Charles A.  McBride and Maria-​Odilia Leal McBride. Theatre Communications Group, 1985. Print. Bogost, Ian. “The Rhetoric of Video Games.” The Ecology of Games: Connecting Youth, Games, and Learning, edited by Katie Salen, MIT Press, 2008, pp. 117–​140. Print. Bogost, Ian. Persuasive Games:  The Expressive Power of Videogames. MIT Press, 2010. Print. Bolter, Jay David. Writing Space: Computers, Hypertext, and the Remediation of Print. 2nd Ed. Routledge, 2001. Print. Bolter, Jay David, and Richard Grusin. Remediation: Understanding New Media. MIT Press, 2000. Print. Bolter, Jay David, Maria Engberg, and Blair MacIntyre. “Media Studies, Mobile Augmented Reality, and Interaction Design.” Interactions vol. 20, no.  1, 2013, pp. 36–​45. Print. Bottéro, Jean. Mesopotamia: Writing, Reasoning, and the Gods. Translated by Zainab Bahrani and Marc Van De Mieroop. University of Chicago Press, 1995. Print. Bredin, Hugh. “Metonymy.” Poetics Today vol. 5, no. 1, 1984, pp. 45–​58. Print. Brin, Sergey. “Why Google Glass?” Ted.com, 28 Feb 2013, http://​www.ted.com/​talks/​sergey_​brin_​why_​google_​glass/​transcript?language=en., Web. Accessed 12 May 2013.

27

Bibliography  • 

227

Brooke, Collin. Lingua Fracta:  Towards a Rhetoric of New Media. Hampton, 2009. Print. Brown, J.J., Jr. “Essjay’s Ethos: Rethinking Textual Origins and Intellectual Property.” College Composition and Communication vo. 61, no. 1, 2009, pp. 238–​258. Print. Burke, Kenneth. “Four Master Tropes.” The Kenyon Review vol. 3, no.  4, 1941, pp. 421–​438. Print. Campbell, Scott W. “Mobile Media and Communication: A New Field, or Just a New Journal?” Mobile Media and Communication vol. 1, no. 1, 2013, pp. 8–​13. Print. Campbell, Scott W., Joseph Bayer, and Rich Ling. “The Case of the Disappearing Phone: Implications of Google Glass for the Embedding of Mobile Communication.” Living Inside Mobile Social Information, edited by James Katz. Greyden, 2014. pp. 23–​41. Print. Cardiff, Janet. The Missing Voice: Case Study B. Artangel, 1999. Print. Cardiff, Janet, and George Bures Miller. The Killing Machine and Other Stories, 1995–​ 2007, edited by Ralf Beil and Bartomeu Mari. Hatje Cantz, 2007. Print. Carr, Nicholas. The Shallows: What the Internet is Doing to Our Brains. W. W. Norton & Company, 2010. Print. Case, Amber. Calm Technology:  Principles and Patterns for Non-​Intrusive Design. O’Reilly Media, 2016. Print. Castagnary, Jules. “L’Exposition du Boulevard des Capucines—​Les Impressionnistes.” Artchive.com, 29 April 1874, http://​www.artchive.com/​galleries/​1874/​74critic. htm, Web. Accessed 23 July 2012. Castells, Manuel. The Rise of the Network Society. Blackwell, 2000. Print. Chesneau, Ernest. “Le Plein Air, Exposition du Boulevard des Capucines.” Artchive. com, 7 May 1874, http://​www.artchive.com/​galleries/​1874/​74critic.htm, Web. Accessed 23 July 2012. Christiansen, Nancy. Figuring Style: The Legacy of Renaissance Rhetoric. University of South Carolina Press, 2013. Print. Cixous, Hélène. “The Last Painting or the Portrait of God.” The Continental Aesthetics Reader, edited by Clive Cazeaux. Routledge, 2000, pp. 583–​597. Print. “Communication, Rhetoric, and Digital Media.” North Carolina State University. Crdm.chass.ncsu.edu., n.d., Web. Accessed 10 Aug 2013. Crary, Jonathan. Techniques of the Observer: On Vision and Modernity in the Nineteenth Century. MIT Press, 1990. Print. Crawford, Alice, and Gerard Goggin. “Geomobile Web:  Locative Technologies and Mobile Media.” Australian Journal of Communication vol. 36, no.  1, 2009, pp. 97–​109. Print. “Crowdsourcing Seahorses.” News.ubc.ca., 8 Oct. 2013, https://​news.ubc.ca/​2013/​10/​ 08/​crowdsourcing-​seahorses/​ Web. Accessed 10 Mar. 2014. Deleuze, Gilles, and Félix Guattari. Anti-​Oedipus: Capitalism and Schizophrenia. Translated by R. Hurley, M. Seem, and H.R. Lane. Continuum, 2004. Print.

28

2 2 8   • 

Bibliography

De Man, Paul. Allegories of Reading: Figural Language in Rousseau, Nietzsche, RiIke, and Proust. Yale University Press, 1979. Print. Depew, David J.  “Revisiting Richard McKeon’s Architectonic Rhetoric:  A  Response to ‘The Uses of Rhetoric in a Technological Age: Architectonic Productive Arts.’ ” Reengaging The Prospects of Rhetoric:  Current Conversations and Contemporary Challenges, edited by M. Porrovecchio. Routledge, 2010. pp. 37–​56. Print. Derrida, Jacques. Writing and Difference. Translated by Alan Bass. University of Chicago Press, 1978. Print. Derrida, Jacques. Limited Inc. Translated by Jeffrey Mehlam and Samuel Weber. Northwestern University Press, 1988. Print. Derrida, Jacques. Archive Fever: A Freudian Impression. Translated by Eric Prenowitz. University of Chicago Press, 1996. Print. Derrida, Jacques. Of Grammatology. Trans. Gayatri Chakravorty Spivak. Johns Hopkins University Press, 1997. Print. Derrida, Jacques and Bernard Stiegler. Echographies of Television. Translated by Jennifer Bajorek. Polity, 2002. Print. de Souza e Silva, Adriana. “From Cyber to Hybrid: Mobile Technologies as Interfaces of Hybrid Spaces.” Space and Culture vol. 9, no. 3, 2006, pp. 261–​278. Print. de Souza e Silva, Adriana and Daniel Sutko. “Theorizing Locative Technologies through Philosophies of the Virtual.” Communication Theory vol. 21, no. 1, 2011, pp. 23–​42. Print. Dourish, Paul. “Seeking a Foundation for Context-​Aware Computing.” Human-​ Computer Interaction vol. 16, nos. 2–​4, 2001, pp. 229–​241. Print. Dourish, Paul. Where the Action Is:  The Foundations of Embodied Interaction. MIT Press, 2001. Print. Dourish, Paul, and Genevieve Bell. Divining a Digital Future: Mess and Mythology in Ubiquitous Computing. MIT Press, 2011. Print. Dourish, Paul and Scott Mainwaring. “Ubicomp’s Colonial Impulse.” Proceedings of the 2012 ACM Conference on Ubiquitous Computing, compiled by Anind Dey, et al. ACM, 2012, pp. 133–​142. Print. Dow, Steven, Jaemin Lee, Christopher Oezbek, Blair MacIntyre, Jay David Bolter, and Maribeth Gandy. “Exploring Spatial Narratives and Mixed Reality Experiences in Oakland Cemetery.” Proceedings of the 2005 ACM SIGCHI International Conference. ACM, 2005, pp. 51–​60. Print. Drucker, Johanna. “Humanities Approaches to Interface Theory.” Culture Machine, vol. 12, 2011, 13 Dec. 2011, https://​www.culturemachine.net/​index.php/​cm/​article/​viewArticle/​434 Web. Accessed 22 Jul. 2012. Eco, Umberto. The Open Work. Translated by Anna Cancogni. Harvard University Press, 1989. Print. Ede, Lisa, and Andrea Lunsford. “Audience Addressed/​Audience Invoked: The Role of Audience in Composition Theory and Pedagogy.” College Composition and Communication vol. 35, no. 2, May 1984, pp. 155–​171. Print.

29

Bibliography  • 

229

Eisenstein, Elizabeth. The Printing Press as an Agent of Change. Cambridge University Press, 1980. Print. Ekman, Ulrik. “Interaction Designs for Ubicomp Cultures.” Fibreculture 19, 2011, http://​nineteen.fibreculturejournal.org/​fcj-​129-​interaction-​designs-​for-​ubicomp-​ cultures/​, Web. Accessed 2 Jun. 2012. Ekman, Ulrik. “Introduction.” Throughout: Art and Culture Emerging with Ubiquitous Computing, edited by Ulrik Ekman. MIT Press, 2013. pp. 1–​60. Print. Emerson, Lori. Reading Writing Interfaces from the Digital to the Bookbound. University of Minnesota Press, 2014. Print. Eyman, Douglas. Digital Rhetoric:  Theory, Method, Practice. University of Michigan Press, 2015. Print. Farman, Jason. Mobile Interface Theory: Embodied Space and Locative Media. Routledge, 2012. Print. The Gates. Directed by Antonio Ferrara and Albert Maysles. Maysles Productions, 2007. “First Electric Traffic Signal.” History.com, This Day in History, n.d, http://​www.history.com/​this-​day-​in-​history/​first-​electric-​traffic-​signal-​installed, Web. Accessed 1 Nov. 2012. Flusser, Vilém. Does Writing Have a Future? Translated by Nancy Ann Roth. Minnesota University Press, 2011. Print. Frabetti, Federica. “Rethinking the Digital Humanities in the Context of Originary Technicity.” Culture Machine, vol. 12 (2011), 13 Dec. 2011, https://​www.culturemachine.net/​index.php/​cm/​article/​viewArticle/​431 Web. Accessed 20 Jul. 2012. Freeman, John Craig, Mark Skwarek, and Lily and Honglei. “ZERO1: From Lewisberg, PA to Silicon Valley.” Accessed 12 Sept. 2012, https://​johncraigfreeman.wordpress. com/​zero1-​from-​lewisburg-​pa-​to-​silicon-​valley/​ Web. 10 December 2012. Freire, Paulo. Pedagogy of the Oppressed. 30th Anniversary Edition. Bloomsbury, 2000. Print. Freud, Sigmund. A General Introduction to Psychoanalysis. Translated by G.  Stanley Hall. Boni and Liveright, 1920. Print. Friedberg, Anne. The Virtual Window:  From Alberti to Microsoft. MIT Press, 2006. Print. “Georgia Tech Cycling App to Assist City of Atlanta.” Gatech.edu. Georgia Tech Institute for People and Technology, n.d., http://​ipat.gatech.edu/​georgia-​tech-​ cycling-​app-​assist-​city-​atlanta, Web. Accessed 10 Aug. 2016. Glazer, Nathan. “Christo in Central Park and in Harlem.” The Public Interest vol. 68, 1982, pp. 70–​77. Print. Goetz, Adriane. “The Invisible Artist Layer Guides You through London’s Art Museums.” Layar, 15 Feb. 2011, https://​www.layar.com/​news/​blog/​2011/​02/​ 15/​the-​invisible-​artist-​layer-​g uides-​you-​through-​londons-​art-​museums/​., Web. Accessed 15 December 2012. Goggin, Gerard. “Ubiquitous Apps: Politics of Openness in Global Mobile Cultures.” Digital Creativity vol. 22, no. 3, 2011, pp. 148–​59. Print.

320

2 3 0   • 

Bibliography

Goggin, Gerard. “Driving the Internet: Mobile Internets, Cars, and the Social.” Future Internet 4.  20 Mar. 2012, http://​www.mdpi.com/​1999-​5903/​4/​1/​306, Web. Accessed 7 May 2014. Goody, Jack. The Domestication of the Savage Mind. Cambridge University Press, 1977. Print. Gordon, Kelly. “Hirshhorn Interview with Janet Cardiff.” Hirshhorn Museum. Hirshhorn.si.edu, July 2005, http://​www.cardiffmiller.com/​press/​texts/​kg_​jc-​ interview.pdf., Web. Accessed 1 Aug. 2014. Graeff, Erhardt. “Action Path Summary.” MIT Center for Civic Media. Actionpath.org., n.d., http://​actionpath.org/​media/​actionpath-​onepager.pdf, Web. Accessed 28 Oct. 2014. Green, Chris. “Wearable Technology:  Latest Devices Allow Employers to Track Behavior of their Workers.” Independent.co.uk., 13 Aug. 2015, http://​www.independent.co.uk/​life-​style/​gadgets-​and-​tech/​news/​wearable-​technology-​latest-​devices-​ allow-​employers-​to-​track-​behaviour-​of-​their-​workers-​10454342.html, Web. Accessed 30 Jul. 2016. Greene, Jacob. “From Augmentation to Articulation: (Hyper)linking the Locations of Public Writing.” Enculturation, vol. 24, 2017, 4 Apr. 2017, http://​enculturation.net/​ from_​augmentation_​to_​articulation, Web. Accessed 10 Apr. 2017. Greenfield, Adam. Everyware:  The Dawning Age of Ubiquitous Computing. Berkley, CA: New Riders, 2006. Print. Greenfield, Adam. “Worth a Thousand Words.” Speedbird.wordpress.com, 19 Aug. 2008, https://​speedbird.wordpress.com/​2008/​08/​19/​worth-​a-​thousand-​words-​etc/​, Web. Accessed 25 Jan. 2015. Greenspan, Brian. “Hyperlab Projects.” Thehyperlab.ca, n.d., http://​thehyperlab.ca/​ projects/​, Web. Accessed 10 Jan. 2017. Greenspan, Brian. “The New Place of Reading:  Locative Media and the Future of Narrative” Digital Humanities Quarterly vol. 5, no. 3, 2011, http://​www.digitalhumanities.org/​dhq/​vol/​5/​3/​000103/​000103.html. Web. Accessed 7 Jan. 2014. Greenspan, Brian. “Songlines in the Streets:  Story Mapping with Itinerant Hypernarrative.” New Narratives:  Stories and Storytelling in the Digital Age, edited by Ruth Page and Bronwen Thomas. University of Nebraska Press, 2011, pp. 153–​169. Print. Gries, Laurie. “Agential Matters:  Tumbleweed, Women-​Pens, Citizens-​Hope, and Rhetorical Actancy.” Ecology, Writing Theory, and New Media:  Writing Ecology, edited by Sidney Dobrin. Routledge, 2012, pp. 67–​91. Print. Gries, Laurie. Still Life with Rhetoric: A New Materialist Approach for Visual Rhetorics. Utah State University Press, 2015. Print. Grudin, Jonathan. “A Moving Target:  The Evolution of HCI.” Human-​ Computer Interaction Handbook, edited by Julie Jacko. 3rd Ed. CRC Press, 2012, pp. xxvii–​Ivi. Print.

213

Bibliography  • 

231

Habermas, Jürgen. The Structural Transformation of the Public Sphere: An Inquiry into a Category of Bourgeois Society. Translated by Thomas Burger. MIT Press, 1989. Print. Habermas, Jürgen. “Further Reflections on the Public Sphere.” Habermas and the Public Sphere, edited by Craig Calhoun. MIT Press, 1992, pp. 421–​4 61. Print. Halsey, Ashley. “Teen Drivers are Texting, Just like their Parents.” Washingtonpost.com, 13 May 2012. https://​www.washingtonpost.com/​local/​trafficandcommuting/​ teen-​drivers-​are-​texting-​just-​like-​their-​parents/​2012/​05/​13/​gIQA8raQNU_​story. html?utm_​term=.ecef79d10681, Web. Accessed 7 Oct 2012. Hansen, Mark. “ ‘Realtime Synthesis’ and the Différance of the Body: Technocultural Studies in the Wake of Deconstruction.” Culture Machine, vol. 6, 20 Oct. 2004, https://​www.culturemachine.net/​index.php/​cm/​article/​view/​9/​8, Web. Accessed 16 Jul. 2012. Hariman, Robert, and John Louis Lucaites. No Caption Needed:  Iconic Photographs, Public Culture, and Liberal Democracy. University of Chicago Press, 2007. Print. Havelock, Eric. Preface to Plato. Harvard University Press, 1963. Print. Hawk, Byron. A Counter-​History of Composition: Toward Methodologies of Complexity. University of Pittsburgh Press, 2007. Print. Hayles, N. Katherine. “RFID: Human Agency and Meaning in Information-​Intensive Environments.” Theory, Culture & Society vol. 26, nos. 2–​3, March/​May 2009, pp. 47–​72. Print. Hayles, N. Katherine. How We Think: Digital Media and Contemporary Technogenesis. University of Chicago Press, 2012. Print. Heidegger, Martin. Being and Time. Translated by John Macquarrie and Edward Robinson. HarperPerennial, 2008. Print. Henkin, David M.  City Reading:  Written Words and Public Spaces in Antebellum New York. Columbia University Press, 1998. Print. Holmes, Ryan. “From Inside Walled Gardens.” Fastcompany.com, 9 Aug. 2013, https://​ www.fastcompany.com/​3015418/​from-​inside-​walled-​gardens-​social-​networks-​are-​ suffocating-​the-​internet-​as-​we-​know-​it, Web. Accessed 20 Feb. 2014. Holmevik, Jan. Inter/​vention: Free Play in the Age of Electracy. MIT Press, 2012. Print. House, John. Monet: Nature into Art. Yale University Press, 1986. Print. Humphreys, Lee. “Mobile Social Media: Future Challenges and Opportunities.” Mobile Media and Communication vol. 1, no. 1, January 2013, pp. 20–​25. Print. Ihde, Don. Postphenomonology and Technoscience. State University of New York Press, 2009. Print. “Introduction to Trash|Track.” Mit.edu. SENSEable City Lab, n.d., http://​senseable. mit.edu/​trashtrack/​index.php?id=1, Web. Accessed 10 Aug. 2012. Janney, Christopher. “David’s Way.” Janneysound.com, 21 Jun. 2005, http://​www.janneysound.com/​project/​davids-​way-​2/​, Web. Accessed 16 Feb. 2012. Jenkins, Henry, Sam Ford, and Joshua Green. Spreadable Media: Creating Value and Meaning in a Networked Culture. New York University Press, 2013. Print.

32

2 3 2   • 

Bibliography

Johns, Adrian. “Foreword.” Ramus, Method, and the Decay of Dialogue: From the Art of Discourse to the Art of Reason, by Walter J. Ong. University of Chicago Press, 2004, pp. v–​xiii. Print. Johnson, Steven. Interface Culture. Basic Books, 1997. Print. Jurgenson, Nathan. “When Atoms Meet Bits:  Social Media, the Mobile Web and Augmented Revolution.” Future Internet vol. 4, no. 1, 2012, pp. 83–​91. Print. Kassabian, Anahid. Ubiquitous Listening: Affect, Attention, and Distributed Subjectivity. University of California Press, 2013. Print. Kirschenbaum, Matthew G.  Mechanisms:  New Media and the Forensic Imagination. MIT Press, 2008. Print. Kleinman, Adam. “Intra-​ actions:  Interview of Karen Barad.” Mousse 34, 2012, pp. 76–​81. Print. Kozel, Susan. “Sinews of Ubiquity: A Corporeal Ethics for Ubiquitous Computing,” Throughout: Art and Culture Emerging with Ubiquitous Computing, edited by Ulrik Ekman. MIT Press, 2013, pp. 337–​350. Print. Lanham, Richard. The Economics of Attention:  Style and Substance in the Age of Information. University of Chicago Press, 2006. Print. Latour, Bruno. “Technology is Society Made Durable.” A Sociology of Monsters? Essays on Power, Technology and Domination, edited by John Law. Routledge, 1991. pp. 103–​131. Print. LeFevre, Karen Burke. Invention as a Social Act. Southern Illinois University Press, 1986. Print. Leroy, Louis. “Exposition des Impressionnistes.” Artchive.com., 25 April 1874., http://​ www.artchive.com/​galleries/​1874/​74critic.htm, Web. Accessed 23 July 2012. Levy, David M.  Scrolling Forward:  Making Sense of Documents in the Digital Age. Arcade, 2001. Print. Lindgren, Tim and Derek Owens. “From Site to Screen, From Screen to Site: Merging Place-​based Pedagogy with Web-​based Technology.” The Locations of Composition, edited by Christopher J.  Keller and Christian R.  Weisser. State University of New York Press, 2007, pp. 195–​212. Print. Liu, Lydia. The Freudian Robot:  Digital Media and the Future of the Unconscious. University of Chicago Press, 2010. Print. Madrigal, Alexis. “The World is Not Enough: Google and the Future of Augmented Reality.” The Atlantic, 25 2012 Oct., https://​www.theatlantic.com/​technology/​ archive/​2012/​10/​the-​world-​is-​not-​enough-​google-​and-​the-​future-​of-​augmented-​ reality/​264059, Web. Accessed 18 2012 Nov. Manovich, Lev. The Language of New Media. MIT Press, 2002. Print. Manovich, Lev. “The Poetics of Augmented Space.” Visual Communication vol. 5, no. 2, 2006, pp. 219–​240. Print. Manovich, Lev. “Inside Photoshop.” Computational Culture, 2011 Nov., http://​computationalculture.net/​article/​inside-​photoshop, Web. Accessed 10 February 2013.

23

Bibliography  • 

233

Marino, Mark. “Critical Code Studies.” Electronic Book Review, 4 Dec. 2006, http://​ www.electronicbookreview.com/​thread/​electropoetics/​codology, Web. Accessed 10 May 2011. McCorkle, Ben. Rhetorical Delivery as Technological Discourse: A Cross-​Historical Study. Southern Illinois University Press, 2012. Print. McCarthy, Anna. Ambient Television: Visual Culture and Public Space. Duke University Press, 2001. Print. McCullough, Malcolm. Ambient Commons:  Attention in the Age of Embodied Information. MIT Press, 2013. Print. McKeen, Catherine. “ ‘Standing Apart in the Shelter of the City Wall’:  The Contemplative Ideal and the Politically Engaged Philosopher in Plato’s Political Philosophy.” Southern Journal of Philosophy vol. 48, no. 2, 2010, pp. 197–​216. Print. McKeon, Richard. Rhetoric:  Essays in Invention and Discovery, edited by Mark Backman. Ox Bow, 1987. Print. McLuhan, Marshall. The Gutenberg Galaxy: The Making of Typographic Man. The New American Library, 1969. Print. McLuhan, Marshall. “The Playboy Interview:  Marshall McLuhan.” Nextnature.com., March 1969, Web. Accessed 15 October 2012. McLuhan, Marshall. “At the Moment of Sputnik the Planet became a Global Theater in Which There Are No Spectators but Only Actors.” Journal of Communication vol. 24, no. 1, 1974, pp. 48–​58. Print. “MIT Researchers Map the Flow of Urban Trash.” Mit.edu., Sensible Cities Lab, n.d., http://​senseable.mit.edu/​trashtrack/​downloads/​trash-​track-​nsf.pdf., Web. Accessed 11 Aug. 2012. Mitchell, Amy and Tom Rosenstiel, Laura Santhanam, and Leah Christian. “Future of Mobile News.” Journalism.org, Pew Research Center, 1 Oct. 2012, http://​www. journalism.org/​2012/​10/​01/​future-​mobile-​news/​, Web. Accessed 7 Oct. 2012. Mitchell, W.J.T., and Mark Hansen. “Time and Space.” Critical Terms for Media Studies, edited by W.J.T. Mitchell and Mark Hansen. University of Chicago Press, 2010, pp. 36–​48. Print. Morey, Sean. Rhetorical Delivery and Digital Technologies: Networks, Affect, Electracy. Routledge, 2016. Print. Morozov, Evgeny. “The Rise of Data and the Death of Politics.” Theguardian.com., 20 July 2014, https://​www.theguardian.com/​technology/​2014/​jul/​20/​rise-​of-​data-​ death-​of-​politics-​evgeny-​morozov-​algorithmic-​regulation., Web. Accessed 12 Aug. 2014. Murray, Janet H. Inventing the Medium: Principles of Interaction Design as a Cultural Practice. The MIT Press, 2011. Print. National Highway Traffic Safety Administration. “Blueprint for Ending Distracted Driving.” Washington, DC: U.S. Department of Transportation, National Highway Traffic Safety Administration, 2012. Print.

324

2 3 4   • 

Bibliography

Nolan, James. “A Conversation with Sherry Turkle.” The Hedgehog Review, Spring 2012, pp. 53–​6 4. Print. Ong, Walter J. Ramus, Method, and the Decay of Dialogue: From the Art of Discourse to the Art of Reason. University of Chicago Press, 1958. Print. O’Toole, James. “Mobile Apps Overtake PC Internet Usage in U.S.” Cnn.com, 28 Feb. 2014, http://​money.cnn.com/​2014/​02/​28/​technology/​mobile/​mobile-​apps-​ internet/​, Web. Accessed 14 Aug. 2014. Paul, Christiane. “Context as Moving Targets:  Locative Media Art and the Shifting Ground of Context Awareness.” Throughout:  Art and Culture Emerging with Ubicomp, edited by Ulrik Ekman. MIT Press, 2012, pp. 399–​418. Print. Paul, Jesse. “Woman who Defaced National Parks, Including 2 in Colorado, Pleads Guilty.” Denverpost.com., 14 June 2016, http://​www.denverpost.com/​2016/​06/​14/​ guilty-​defacing-​national-​parks-​rocks/​, Web. Accessed 23 Sept. 2016. Peters, John Durham. “Two Cheers for Technological Determinism.” Conference on Media Histories: Epistemology, Materiality, Temporality, Vimeo.com., 25 June 2011, https://​vimeo.com/​25591045, Web. Accessed 2 Mar. 2014. Pfanner, Eric. “Developing Countries Surge in Mobile Broadband, U.N. Finds.” NYTimes.com., 26 Sept. 2013, https://​bits.blogs.nytimes.com/​2013/​09/​26/​ developing-​countries-​surge-​in-​mobile-​broadband-​u-​n-​finds/​?_​r=0, Web. Accessed 30 Sept. 2013. “Ph.D. in Film & Digital Media.” UC Santa Cruz. Film.ucsc.edu., n.d., http://​film.ucsc. edu/​phd_​program, Web. Accessed 15 Mar. 2013. Pollack, Barbara. “Janet Cardiff, Her Long Black Hair.” Time Out New York, 29 July–​5 Aug. 2004, p. 58. Print. Price, R. Darren. “Rare Pokémon Sparks Stampede in Central Park.” NBC 4 New York, 18 Jul. 2016, http://​www.nbcnewyork.com/​news/​local/​Pokemon-​Go-​Players-​ Stampede-​New-​York-​Central-​Park-​387303572.html, Web. Accessed by 20 Jul. 2016. Propen, Amy. Locating Visual-​Material Rhetorics:  The Map, the Mill, and the GPS. Parlor, 2012. Print. Quigley, Aaron. “From GUI to UUI:  Interfaces for Ubiquitous Computing.” Ubiquitous Computiong Fundamentals, edited by John Krumm. Chapman and Hall, 2010, pp. 237–​284. Print. Quintilian. Institutio Oratoria. Rhetoric.eserver.org., n.d., http://​rhetoric.eserver.org/​ quintilian/​, Web. Accessed 12 2014 Jan. Raley, Rita. “Walk this Way:  Mobile Narrative as Composed Experience.” Beyond the Screen:  Transformations of Literary Structures, Interfaces and Genres, edited by Jörgen Schäfer and Peter Gendolla. Transcript Verlag/​ Transaction, 2010, pp. 299–​316. Print. Ramage, John D., and John C. Bean. Writing Arguments: A Rhetoric with Readings. 3rd Ed. Allyn & Bacon, 1995. Print.

235

Bibliography  • 

235

Ratti, Carlo. “The Adaptable Bus Stop.” MIT SENSEable City Lab, 2006, http://​senseable.mit.edu/​bus_​stop/​, Web. Accessed 14 December 2014. Ratti, Carlo. “Architecture that Senses and Responds.” Ted.com., 3 May 2011, https://​ www.ted.com/​talks/​carlo_​ratti_​architecture_​that_​senses_​and_​responds, Web. Accessed 12 Jun. 2012. Rice, Jeff. The Rhetoric of Cool: Composition Studies and New Media. Southern Illinois University Press, 2007. Print. Rice, Jeff. Digital Detroit: Rhetoric and Space in the Age of the Network. Southern Illinois University Press, 2012. Print. Rice, Jenny. “Unframing Models of Public Distribution:  From Rhetorical Situation to Rhetorical Ecologies.” Rhetoric Society Quarterly vol. 35, no. 4, 2005, pp. 5–​25. Print. Rice, Jenny. Distant Publics: Development Rhetoric and the Subject of Crisis. University of Pittsburgh Press, 2012. Print. Rickert, Thomas. Ambient Rhetoric: The Attunements of Rhetorical Being. University of Pittsburgh Press, 2013. Print. Ridolfo, Jim, and Dànielle Nicole DeVoss. “Composing for Recomposition: Rhetorical Velocity and Delivery.” Kairos vol. 13, no.  2, 2009, http://​kairos.technorhetoric. net/​13.2/​topoi/​ridolfo_​devoss/​, Web. Nov 10 2014. Rieder, David. “Making Wayves.” Enculturation.net, vol. 23, 2016, http://​enculturation.net/​making-​wayves, Web. Accessed 4 Dec. 2016. Rivera, Janessa and Rob van der Meulen. “Gartner Says 4.9 Billion Connected ‘Things’ Will Be in Use in 2015.” Gartner.com, 11 Nov. 2014, http://​www.gartner.com/​newsroom/​id/​2905717. Web. Accessed 30 Apr. 2015. Roberts, Ben. “Stiegler Reading Derrida:  The Prosthesis of Deconstruction in Technics.” Postmodern Culture vol. 16, no. 1, 2006, https://​muse.jhu.edu/​article/​ 192267 Web. Accessed 7 May 2012. Rod, Jan. “Post Human-​Centered Design Approaches for Ubiquity.” Escholarship.org, Proceedings of the Digital Arts and Culture Conference, 12 Dec. 2009, http://​ escholarship.org/​uc/​item/​7nx6199f, Web. Accessed 18 Oct. 2012. Rogers, Yvonne. “Moving on from Weiser’s Vision of Calm Computing:  Engaging Ubicomp Experiences.” Ubicomp 2006: Ubiquitous Computing: 8th International Conference:  Proceedings, edited by Paul Dourish and Adrian Friday. Springer, 2006, pp. 404–​421. Print. Rose, David. Enchanted Objects:  Design, Human Desire, and the Internet of Things. Scribner, 2014. Print. Rosenzweig, Roy, and Elizabeth Blackmar. The Park and the People: A History of Central Park. Cornell University Press, 1992. Print. Rueb, Teri. “Shifting Subjects in Locative Media.” Digital Tools in Composition Studies, edited by Ollie O.  Oviedo, Joyce R.  Walker, and Byron Hawk. Hampton, 2010, pp. 167–​178. Print.

326

2 3 6   • 

Bibliography

Rueb, Teri. “Itinerant.” Turbulence.org, n.d., https://​vimeo.com/​128827447, Web. Accessed 28 June, 2012. Ruskin, John. The Elements of Drawing. New York: John Low, 1804. Print. Sánchez, Raul. The Function of Theory in Composition Studies. SUNY University Press, 2005. Print. Scharf, Aaron. Art and Photography. Penguin, 1974. Print. Schapiro, Meyer. Impressionism:  Reflections and Perceptions. George Braziller, 1997. Print. Scott, Kitty. “I Want You to Walk with Me.” The Missing Voice (case study b), by Janet Cardiff. London: Artangel, 1999. Print. Simondon, Gilbert. On the Mode of Existence of Technical Objects. Translated by Ninian Mellamphy. University of Western Ontario, 1980, http://​dephasage.ocular-​ witness.com/​pdf/​SimondonGilbert.OnTheModeOfExistence.pdf, Web. Accessed 25 Apr. 2014. Sikora, Dave. “How Mobile Apps are Reviving Branded Walled Gardens.” Mobilemarketer.com, 23 Sept. 2013, http://​www.mobilemarketer.com/​ex/​mobilemarketer/​cms/​opinion/​columns/​16208.html, Web. Accessed 12 May 2014. “Statistics and Facts on Mobile Internet Usage.” Statista.com, n.d., https://​www.statista. com/​topics/​779/​mobile-​internet/​, Web. Accessed 26 Aug 2014. Stein, Bob. “The Future of the App.” Futureofthebook.org, 2 Aug. 2010, http://​futureofthebook.org/​blog/​2010/​08/​02/​the_​future_​of_​the_​app/​, Web. Accessed 12 May 2012. Stein, Bob. “SocialBook.” Digital Platforms and the Future of Books Symposium. Youtube.com, 20 Mar. 2012, https://​www.youtube.com/​watch?v=tzJiJQB6M6I, Web. Accessed 3 Aug. 2012. Stiegler, Bernard. Technics and Time, vol. 1, The Fault of Epimetheus. Translated by Richard Beardsworth and George Collins. Stanford University Press, 1998. Print. Stiegler, Bernard. “Our Ailing Educational Institutions.” Translated by Stefan Herbrechter. Culture Machine, vol. 5, 29 Jun. 2003, https://​www.culturemachine. net/​index.php/​cm/​article/​view/​258/​243, Web. Accessed 25 Mar. 2012. Stiegler, Bernard. Technics and Time, vol. 2, Disorientation. Translated by Stephen Barker. Stanford University Press, 2009. Print. Stiegler, Bernard. “Memory.” Translated by Mark Hansen. Critical Terms for Media Studies, edited by W.J.T. Mitchell and Mark Hansen. University of Chicago Press, 2010, pp. 64–​87. Print. Stiegler, Bernard. Technics and Time, vol. 3, Cinematic Time and the Question of Malaise. Translated by Stephen Barker. Stanford University Press, 2011. Print. Stiegler, Bernard. The Decadence of Industrial Democracies. Translated by Daniel Ross. Polity, 2011. Print. Stiegler, Bernard. “The Carnival of the New Screen: From Hegemony to Isonomy.” The YouTube Reader, edited by Pelle Snickars and Patrick Vonderau. National Library of Sweden, 2010, pp. 40–​59. Print.

237

Bibliography  • 

237

Stiegler, Bernard. “Teleologics of the Snail.” Throughout:  Art and Culture Emerging with Ubicomp, edited by Ulrik Ekman. MIT Press, 2012, pp. 479–​492. Print. Stiegler, Bernard, W.J.T. Mitchell, and Mark Hansen. “On Friedrich Kittler.” Cultural Technologies Podcast. vol. 7, 29 Oct. 2010, http://​bernardg.com/​podcast/​bernard-​ stiegler-​mark-​hansen-​and-​wjt-​mitchell-​friedrich-​kittler-​episode-​7, Web. Accessed 12 Mar. 2012. Telhan, Orkan. “Urban Botanical Garden.” Oobject.com, Oct. 2006, http://​www.oobject.com/​wp-​content/​uploads/​2008/​04/​main.html, Web. Accessed 3 May 2012. “The Fort Vancouver Mobile Project.” Neh.gov, n.d., https://​www.neh.gov/​films/​the-​ fort-​vancouver-​mobile-​project, Web. Accessed 3 Dec. 2013. Thompson, Roger. “Kairos Revisited:  An Interview with James Kinneavy.” Rhetoric Review vol. 19, nos. 1–​2, 2000, pp. 73–​88. Print. Thoreau, Henry David. The Portable Thoreau, edited by Carl Bode. New York: Viking, 1947. Print. “Toronto Traffic FAQ.” Toronto.ca. Transportation, n.d. http://​www1.toronto.ca/​wps/​ portal/​contentonly?vgnextoid=80c7a84c9f6e1410VgnVCM10000071d60f89R CRD, Web. Accessed 1 Nov. 2012. Trotman, Andrew. “The Future of Facebook is Telepathy, Reveals Mark Zuckerberg.” Telegraph.co.uk, 1 July 2015, Web. Accessed 26 Aug. 2015. Ulmer, Gregory L.  Applied Grammatology:  Post(e)-​Pedagogy from Jacques Derrida to Joseph Beuys. Johns Hopkins University Press, 1984. Print. Ulmer, Gregory L. Heuretics: The Logic of Invention. Johns Hopkins University Press, 1994. Print. Ulmer, Gregory L. Avatar Emergency. Parlor, 2012. Print. Ulmer, Gregory L., and John Craig Freeman. “The Ulmer Tapes.” Youtube.com, 19 Jun. 2001, https://​www.youtube.com/​watch?v=PyncBy4SLyU, Web. Accessed 12 Jan. 2012. UNESCO. “The State of Broadband 2013:  Universalizing Broadband.” Broadbandcommission.org, Sept. 2013, http://​unesdoc.unesco.org/​images/​0022/​ 002236/​223620E.pdf., Web. Accessed 2 Nov. 2013. Ungar, Steven. Roland Barthes: The Professor of Desire. University of Nebraska Press, 1983. Print. “U.S. Smartphone Use in 2015.” Pew Research Center, 1 Apr. 2015, http://​www.pewinternet.org/​2015/​04/​01/​us-​smartphone-​use-​in-​2015/​, Web. Accessed 3 May 2015. van Dam, Andries. “Post-​WIMP User Interfaces.” Communications of the ACM vol. 40, no. 2, 1997, pp. 63–​67. Print. Vee, Annette. “Understanding Computer Programming as a Literacy.” Literacy in Composition Studies vol. 1, no. 2, 2013, 42–​6 4. Print. Venn, Couze, Roy Boyne, John Phillips, and Ryan Bishop. “Technics, Media, Teleology: Interview with Bernard Stiegler.” Theory, Culture & Society vol. 24, nos. 7–​8, 2007, pp. 334–​341. Print.

328

2 3 8   • 

Bibliography

Venturi, Lionello. “The Aesthetic Idea of Impressionism.” The Journal of Aesthetics and Art Criticism vol. 1, no. 1, 1941, pp. 34–​45. Print. Wajcman, Judy. Pressed for Time:  The Acceleration of Life in Digital Capitalism. University of Chicago Press, 2015. Print. Walter, E.V. Placeways:  A  Theory of the Human Environment. University of North Carolina Press, 1988. Print. Want, Roy. “An Introduction to Ubiquitous Computing.” Ubiquitous Computing Fundamentals, edited by John Krumm. CRC, 2010, pp. 1–​36. Print. Wardrip-​Fruin, Noah. “From Theatre of the Oppressed.” The New Media Reader, edited by Noah Wardrip-​Fruin and Nick Montfort. MIT Press, 2003. pp. 339–​340. Print. Weiser, Mark. “The Computer for the 21st C ​ entury.” Scientific American vol. 265, no. 3, 1991, pp. 94–​104. Print. Weiser, Mark. “The World is Not a Desktop.” Interactions vol. 1, no.  1, 1994, pp. 7–​8. Print. Weiser, Mark. “Building Invisible Interfaces.” Ubiq.com., 2 Nov. 1994, http://​www. ubiq.com/​hypertext/​weiser/​UIST94_​4up.ps, Web. Accessed 14 Jan. 2012. Weiser, Mark. “The Technologist’s Responsibility and Social Change.” Computer-​ Mediated Communication Magazine vol. 2, no. 4, 1995, 17. Print. Weiser, Mark and John Seely Brown. “The Coming Age of Calm Technology.” Beyond Calculation, edited by Peter J. Denning and Robert M. Metcalfe. Copernicus, 1997, pp. 75–​85. Print. White, Hayden. Metahistory: The Historical Imagination in Nineteenth ​Century Europe. Johns Hopkins University Press, 1973. Print. Wohlsen, Marcus. “PC Sales Slump as Kids Say No to Computers.” Wired.com, 11 Oct. 2012, https://​www.wired.com/​2012/​10/​pc-​sales-​slump/​, Web. Accessed 22 Jan. 2013. Wolf, Maryanne, and Mirit Barzillai. “The Importance of Deep Reading.” Educational Leadership vol. 66, no. 6, 2009, pp. 32–​37. Print.

239

INDEX

Note: Tables and figures are indicated by an italic t and f following the page number. Actant-​networks agency in, 133 attunement to, 173 data generation from, 58 as networked gramme, 134 spontaneous actions of, 145 tracking of, 157–​158, 159 transcoding, 142, 160 Actionable archives built environment as, 127 contextual considerations for, 112–​113 figuration and, 184 grammatological shift toward, 20 local action and, 109 orthographic continuum and, 110, 119 perceptual conditions of, 123 real-​time information events of, 122 rhetorical-​aesthetic considerations for, 118–​119 techno-​geographic arrangement of, 121, 131 of ubicomp, 84n6, 109, 126, 128 underdevelopment of, 108, 125 Actionable criticism, 195, 196–​197, 213–​214 Actionable intelligence, 17

Actionable media ATLAS interface and, 20–​21, 138–​139, 214 barriers to production of, 79 community-​driven advocacy and, 54, 150–​152 context-​awareness in, 68–​69 cultural development of, 85 deep attention fostered by, 82–​83 defining, 14–​19 in driverless vehicles, 77–​78 emergence with ubicomp, 25, 58 GPS navigation as, 74–​76 humanistic production of, 181–​182, 183–​184 innovation in, 135 intellectual discourse impacted by, 21 by kairotic intellectuals, 182, 217–​219, 222, 223 location-​awareness features of, 67 meaning-​making in, 14, 69 modeling, 70–​78 moral dilemmas resulting from, 51 motivations for theory development, 4, 15 objectives of, 14, 17

420

2 4 0   • 

Index

Actionable media (contd.) open networks of, 80 opportunities and challenges presented by, 86 polyrhetorical roles in, 134 precedents for, 17 spect-​actors in, 199, 200–​204, 205 stylistic principles for, 194–​195 techno-​geographic operations of, 133, 134 Actionable narratives, 127–​128, 134–​135 Action Path (app), 204, 205, 215n6 Actuators, 169–​176. See also ATLAS interface in AR|AD Takeover project, 170–​171, 171f democratization of, 175 embedded, 169, 172–​173, 175, 203 functions of, 142, 143t, 169 rhetoricity of, 169–​170, 174 in smart environments, 169 in traffic lights, 175–​176 The Adaptable Bus Stop (Ratti), 62–​63, 63f Adobe Photoshop, 144–​145 Adversative conjunctions, 209–​210 Algorithmic regulation, 53–​54 Alienation effect, 199 Alone Together (Turkle), 6 Alphabetic writing contextual considerations in, 111–​113 grammatization and, 95–​96, 102, 103, 111 history of, 33, 39, 102, 110 technics of, 76 Ambience, defined, 33 Ambient interfaces, 49 Ambient Rhetoric (Rickert), 140, 156 Ambient Television (McCarthy), 89–​90 Anderson, Chris, 146, 147 Anthropocentrism, 75–​76, 132, 140–​141, 157

Apple products, 36, 49–​50, 167 Applied Grammatology (Ulmer), 92 Apps, 146–​153. See also ATLAS interface actionable media growth through, 153 augmented reality, 69–​70 breakthrough, 31–​32 citizen science, 148, 149–​150 civic, 148, 150–​152, 199, 204 as cultural platform, 148 data extraction from, 9 facial recognition, 131–​132, 132f functions of, 142, 143t Internet access through, 6–​7, 146 locative media and, 66, 67, 68–​70, 83n1 polyrhetorical role of human geography in, 131 reactionary nature of, 147 as silos of digital content, 146, 147 streamlining of, 144 AR. See Augmented reality AR|AD Takeover project, 170–​171, 171f, 172 Arche-​writing, 92, 101–​102, 103, 105, 106n4, 112 Archives actionable (see Actionable archives) deferred (see Deferred archives) Derrida on, 81, 108, 109 grammatization and, 20 layers and, 164–​165 post-​desktop interfaces and, 19, 81 ARCs (augmented reality criticisms), 182 Arendt, Hannah, 21, 219–​223 Aresty, Abby, 163–​164 Aristotle, 3, 15, 193, 199–​200, 220 AR Occupy Wall Street project, 196–​197, 196f Art and Photography (Scharf ), 188 Artificial intelligence, 169 “As We May Think” (Bush), 30 ATLAS interface, 142–​176

214

Index  • 

actionable media and, 20–​21, 138–​139, 214 actuators in (see Actuators) apps in (see Apps) criticisms of, 146 defined, 20 design philosophy of, 142 elemental features of, 142–​143, 143t kairotic intellectuals and, 182, 222 layers in (see Layers) metonymical nature of, 144–​145 sensors in (see Sensors) tags in (see Tags) Audio walks in Central Park, xi, xvi–​xix, xviiif, 14, 68–​69 critical, 211–​212 incidental choreography in, 206–​207, 208, 210 rhetoric in, xviii, 210 spect-​actors in, 208–​210 Augmented reality (AR) advancements in, 176 apps in, 69–​70 artworks in, 68 content availability in, 10–​11, 11f figuration process in, 190 Google Glass and, 16 layers in, 160–​163, 164–​165, 170 local action in, 160 misconceptions regarding, 9–​10 in museum exhibits, 12–​13, 12f, 22n2 techno-​geographic orientation of applications in, 129, 161 wearable technologies and, 195 Augmented reality criticisms (ARCs), 182 Augmented space, 7, 8–​9 Auslander, Philip, 73 Authorship Barthes on, 183 in built environment, xii, xvi, xvii, xix

241

challenges to, 19 digital, 65 by kairotic intellectuals, 21, 182, 217–​219, 222, 223 multimedia, xi, xviii public, xv, 59, 79–​80, 83, 85, 115 by public intellectuals, 21, 116, 176, 182, 210, 217 restrictions on, xiii–​xiv for sensor-​actuator networks, 176 of urban signage, 126 on Web 2.0 platforms, xix, 6, 135 Automobiles, 70–​78 distracted driving in, 70, 71, 72–​74 driverless, 70, 71, 77–​78, 84n4 GPS navigation in, 70, 71, 74–​77, 125, 160 hands-​free devices for, 72 networked multimedia in, 71 Banking concept of education, 198 Barad, Karen, 192, 215n4 Bardzell, Shaowen, 56n4 Baron, Dennis, 24 Barthes, Roland, 20–​21, 40, 180, 183–​ 186, 192, 205–​206, 210 Bayer, Joseph, 72 “Being alone together” phenomenon, 5 Bell, Genevieve, 55, 55n1 Benjamin, Walter, 78, 218 Bennett, Jane, 133 Bennington, Geoffrey, 107n6 A Better Pencil (Baron), 24 Bianchi, Melissa, 182 Bieri, Anja, 211–​212 Bitzer, Lloyd, 15 Blippar (app), 80, 88, 131, 132, 132f Boal, Augusto, 21, 182, 185, 198–​205 Bogost, Ian, 87, 88 Bolter, Jay David, 57, 95, 177n2, 181 Boston Common, actionable narratives in, 127–​128, 134–​135

42

2 4 2   • 

Index

Bottero, Jean, 110, 111, 112, 119–​120 Boulevard des Capucines (Monet), 191 Breakthrough apps, 31–​32 Brecht, Bertolt, 199 Bredin, Hugh, 144 Brin, Sergey, 15–​16, 38 Brooke, Collin, 24 Brown, John Seely, 24, 47, 71 Built environment as actionable archive, 127 artificial intelligence in, 169 ATLAS interface in, 173, 186 augmentation of, 12, 18 authorship in, xii, xvi, xvii, xix circulation of networked multimedia in, 89 intellectual discourse in, 195, 210, 218 as public realm, 222–​223 technological unconscious in, 175 writing in, 40 Burke, Kenneth, 23n3 Bush, Vannevar, 30 Calendarity, 122 Calm Technology (Case), 38 Campbell, Scott, 65, 72 Campbell’s Soup Cans (Warhol), 112–​113 Capitalism, computational, 81 Cardiff, Janet audio walks by, xi, xvi–​xix, xviiif, 14, 68–​69, 206–​210 creative practices and works of, 21, 172, 182, 185 fort-​da rhythm of, 210 incidental choreography of, 206–​207, 208, 210 research process utilized by, 207 Cardinality, 104, 122 “The Carnival of the New Screen” (Stiegler), 103 Carr, Nicholas, 4–​5, 6, 30–​31, 41 Cars. See Automobiles

Case, Amber, 38 Castagnary, Jules, 191 CATTt framework, 33–​34, 35, 51 Cell phones. See Smartphones Censorship, 41 Central Park (New York City) art exhibits in, xiv–​xvi audio walks in, xi, xvi–​xix, xviiif, 14, 68–​69 pedestrian sign network in, xi, xiii Pokémon Go in, 69, 70f Chesneau, Ernest, 191 Christiansen, Nancy, 19 Christo (artist), xiv–​xv, xvi, xvii Cicero, 2–​3 Cinematic computing, 44–​45, 88–​90 Citizen science apps, 148, 149–​150 Civic apps, 148, 150–​152, 199, 204 Cixous, Hélène, 186–​187, 193 Command-​line interfaces, 36, 48, 96 Computational capitalism, 81 “The Computer for the 21st Century” (Weiser), 35, 38, 51, 52, 56n4 Computer Lib/​Dream Machine (Nelson), 30 Computers as Theater (Laurel), 30 Computer vision, 80, 88, 170, 171 Context-​awareness in actionable media, 68–​69 in good tool ideal, 44 in human-​computer interaction, 67 promotion of, 64 in ubicomp, 29, 71, 78 in writing systems, 111–​113 Contextual noise, 162–​163 Crary, Jonathan, 187–​188 Crawford, Alice, 66 Critical Code Studies, 87 Critical-​creative categories, 20 Critical invention, 81, 82 Crowdsourcing, 62, 70, 148, 149, 196 Cubism, 177n1

243

Index  • 

Cuneiform, 5, 94, 102, 103, 110–​111, 119 CycleAtlanta (app), 54, 68 Database imagination, 88 David’s Way Plaza project, 157, 172–​173 Davis, Gordon, xvi Deep attention, 59, 81, 82–​83 Deep reading, 5, 40, 41 Deferred archives categorical impulses of, 115 contextual considerations for, 112, 113 conversion to digital formats, 181 desktops and, 84n6, 109, 117 grammatological shift from, 20 interface forms for, 113–​114 libraries as, 112, 115, 126 lists as used in, 115 orthographic continuum and, 108, 110 representation and, 184 Deleuze, Gilles, 156, 192 Deliberative rhetoric, 3, 21, 199, 202, 204 Delimited virtuality, 117 Depew, David, 2 Derrida, Jacques on archives, 81, 108, 109 grammatization models of, 99–​100 grammatological perspective of, 40, 94, 97, 99, 101–​102 Stiegler’s critique of, 101–​102, 105–​106, 107n6, 112 on writing systems, 91–​92, 105, 106–​107n4, 119 Desktop metonymy ATLAS interface and, 143, 144, 146 grammatization processes in, 153 imperatives of, 120–​121 in techno-​geographic milieus, 131 Desktops. See also Personal computers data extraction from, 9 deferred archives and, 84n6, 109, 117 demise of, xi

243

Graphical User Interfaces for, 36, 37, 44, 47–​48, 96 housing of media in codified spaces, 20 intellectual ethic for, 31 Internet access on, 26 invention of, 28 networked multimedia access on, 3 predetermined programs on, 118 technological constraints of, xix, 6 virtual window of, 117, 120, 121 Weiser on, 36, 46, 47 WIMP interface for, 20, 47–​48, 138, 139–​140 De Souza e Silva, Adriana, 68 Developing countries, Internet access in, 26 DeVoss, Dànielle Nicole, 22n1 Digital communication divided attention due to, 72–​73, 74 educational programs in, 179–​180 flow of information in, 47–​51 grammatological approach to, 88 implications of advancements in, xii, xix intellectual ethic for, 31 rhetoric and, 2, 22n1 spreadability of, 15 theorizing, 83 Digital distraction, 4–​7, 13, 15 Digital divide, 26 Digital-​physical convergence, 11, 13, 29, 50–​51, 190 Digital Water Pavilion (Ratti), 61–​62, 62f Digital writing, xviii, 42 Digitization of territories, 130, 134 Diogenes, 193, 220 Distant Publics (Rice), 212–​213 Distracted driving, 70, 71, 72–​74 Dobrin, Sidney, 182 Does Writing Have a Future? (Flusser), 86–​87

42

2 4 4   • 

Index

Dourish, Paul, 46, 47, 55, 55n1, 56n4 Dream censors, 174–​175 Driver, Godfrey, 91 Driverless vehicles, 70, 71, 77–​78, 84n4 Drucker, Johanna, 142 Echo chambers, 171–​172 Echographies of Television (Stiegler & Derrida), 101 Education banking concept of, 198 digital communication programs, 179–​180 informal, 181 rhetorical, 3 smartphone dependence and level of, 26 Eisenstein, Sergei, 88 Ekman, Ulrik, 44, 137 Electronic textuality, 57, 82, 88, 93 Embedded technologies actuators, 169, 172–​173, 175, 203 prototypes of, 26 sensors, 37, 49, 169, 172–​173, 175, 203 tags, 170, 173 Emerson, Lori, 43 Enchanted Objects (Rose), 16 Engberg, Maria, 163 En plein air painting methods, 21, 184, 185–​189, 194, 195 Existential phenomenology, 22, 33, 46–​47 Facebook, 66, 153–​154, 204–​205 Facial recognition apps, 131–​132, 132f Farman, Jason, 8, 60–​61, 73–​74, 83n1, 95, 164 Figuration in augmented reality, 190 enactment through montage, 166–​167 principles of, 183–​185 spect-​actors as byproduct of, 205 theory of, 20–​21, 186

Figurative imagination, 19, 20, 137, 139 Figurative tradition of rhetoric, 19, 20, 23n3 First Impressionist Exhibition (1874), 190–​191 Fish Communication project, 133 Fitbit wristbands, 46 Fitzpatrick, Kathleen, 180 Flores, Fernando, 45 Flusser, Vilém, 86–​87, 88, 168 Forensic rhetoric, 3 Fort-​da rhythm, 210 Fort Vancouver Mobile App (Oppegaard), 181 Frabetti, Federica, 107n6 Frankenstein (Shelley), 127 Freeman, John Craig, 166–​167 Freire, Paulo, 198, 199 Freud, Sigmund, 34, 174–​175 Friedberg, Anne, 117 From Lewisburg, PA to Silicon Valley (Freeman et al.), 166–​168, 167f Gartner, Inc., 97 The Gates, Central Park, New York, 1979–​ 2005 (Christo & Jeanne-​Claude), xiv–​xvi A General Introduction to Psychoanalysis (Freud), 174 Geoinformation, 74, 128, 130–​131, 132 Geotags, 18, 80, 154, 170 Gestures of critical thinking, 17 extraction of data from, 9 interplay with gramme, 97–​98, 99, 118 in post-​WIMP interfaces, 141 transcription into software moves, 49, 88 Glazer, Nathan, xiv–​xvi Globalization, 31 Godard, Jean-​Luc, 88 Goggin, Gerard, 66, 148

245

Index  • 

Goody, Jack, 95, 114, 115 Google Earth, 70 Google Field Trip (app), 195 Google Glass, 15–​16, 72 Google Maps Navigation, 80 Goto, John, 197 GPS navigation, 9, 70, 71, 74–​77, 125, 160 Graeff, Erhardt, 204 Grammatization. See also Writing and writing systems alphabetic writing and, 95–​96, 102, 103, 111 archives and, 20 as comparative method, 104–​106 definitions of, 93, 94 Derridean, 99–​100 desktop metonymy and, 153 of gestures, 97 industrialization as stage of, 97–​98 overview, 85–​86 of phonemes, 103 processes of, 94–​98 of speech, 221 in techno-​geographic milieus, 128 ubicomp and, 91, 95, 98, 100, 120 Grammatology. See also Writing and writing systems defined, 19–​20, 90 Derridean, 40, 94, 97, 99, 101–​102 of digital interfaces and multimedia artifacts, 88 phases of development, 91–​92 as transdisciplinary research agenda, 91, 100 ubicomp and, 40 Gramme (written marks) defined, 90, 94 Derrida on, 102 of desktops, 121 digital interfaces and multimedia artifacts as, 88, 96

245

evolution of, 92, 105, 106 formalization of, 95 interplay with gestures, 97–​98, 99, 118 metonymical invention of, 95, 99–​100, 103 networked, 132, 134 in post-​desktop interfaces, 113 technocultural development of, 104 Graphical User Interfaces (GUIs), 36, 37, 44, 47–​48, 96 Green, Jacob, 182 Greenaway, Peter, 88 Greenfield, Adam, 67, 68 Greenspan, Brian, 81, 181–​182 Gries, Laurie, 32, 33, 133 Griffith, D. W., 89 Groupon, 67 Grudin, Jonathan, 176 Grusin, Richard, 95 Guattari, Félix, 192 Guimbal turbine, 129, 134 GUIs. See Graphical User Interfaces Habermas, Jürgen, 221, 224n4 Hands-​free devices, 72 Hansen, Mark, 106n3, 107n6 Havelock, Eric, 114 Hayles, N. Katherine, 157, 173–​174, 177n3 HCI. See Human-​computer interaction Headworn/​headmounted displays. See also Wearable technologies advancements in, 176 Google Glass, 15–​16 intellectual ethic for, 31 layers in, 168 media consumption on, xviii, 4 retention and, 124 for texting, 72 Heckscher, August, xiv Heidegger, Martin, 22, 45–​4 6, 129, 130 Henkin, David, xiii, 125, 126

426

2 4 6   • 

Index

Her Long Black Hair (Cardiff ), xvi–​xix, xviiif, 14, 68–​69 Heuretics, 33–​34, 56n3 Hieroglyphics, 91–​92, 102, 103 Highlight (app), 131 Historical consciousness, 76, 87, 125, 168 Historical grammatology, 91, 92 Holmes, Ryan, 147 House, John, 193–​194 HTML tags, 145, 153 Human-​computer interaction (HCI) command-​line interfaces and, 36 context-​awareness in, 67 diversity of, 57 in GPS navigation, 75 limitations on, 35, 46 relocation to periphery of extracomputational activity, 120 subject/​object dualism in, 140–​141 tangible modalities of, 48 ubicomp and, 35, 46 in videogames, 87 The Human Condition (Arendt), 219–​220, 221–​222 Humanism actionable media production in, 181–​182, 183–​184 posthumanism, 33, 75, 132, 134, 140 in production and analysis of texts and audiovisual works, 18 technology studies and, 93 on telepathic technologies, 43 ubicomp and, 19, 29 Husserl, Edmund, 46, 120, 123, 124 Hybridity theorists, 7–​8 Hybrid space, 7, 8 Icons. See WIMP interface If-​then propositions, 87–​88 Ihde, Don, 100–​101 Impression, Sunrise (Monet), 191

Impressionism, 163, 185–​192. See also Monet, Claude Incidental choreography, 206–​207, 208, 210 Indexical recording, 89, 161, 162 Information landscapes, 7, 61, 83n1 Institutio Oratoria (Quintilian), 177n1 Intellectual discourse actionable media, impact on, 21 audiences in, 201 built environment as outlet for, 195, 210, 218 critical-​creative categories for, 20 emerging genres for, 185 experimentation with, 180 housing in differed spaces, 18 public, 78 venues for intersection with everyday life, 12 Intellectual ethic, 30–​32, 33, 64 Interfaces ambient, 49 ATLAS (see ATLAS interface) command-​line, 36, 48, 96 cultural production of, 2, 16 for deferred archives, 113–​114 for fighter jets, 17 GUIs (see Graphical User Interfaces) location-​awareness of, 67 mobile, 7, 8, 28 post-​desktop (see Post-​desktop interfaces) for smartphones, 59 tangible, 48–​49 in ubicomp, 44, 48, 120–​121 user-​friendly, 43, 117, 141 WIMP (see WIMP interface) Internet. See also Networked multimedia in developing countries, 26 digital distraction and, 4–​7 in-​vehicle, 71

247

Index  • 

modes of access, 6–​7, 26, 146 Internet of Things, 38, 97, 98, 154, 155, 176 Intra-​actions, 192, 194–​195, 215n4 The Invisible Artist (Goto & Leach), 196, 197–​198, 197f Invoked audiences, 200–​201 iSeahorse (app), 149–​150 Itinerant (Rueb), 113, 122, 127–​128, 134–​135 Jacoby, Russell, 217 Jakobson, Roman, 23n3 Janney, Christopher, 172 Jeanne-​Claude (artist), xiv–​xv, xvii Jenkins, Henry, 14, 154 Johnson, Steven, 37, 177n2 Judicial rhetoric, 3 Kairos action as function of, 222 defined, 10 in digital humanities projects, 181 restrictions on positioning text and multimedia in, 126 in social networking, 73 spatial, 219, 223n1 timeliness in, 218 Kairotic intellectuals, 21, 182, 217–​219, 222, 223 Kaleidoscopic composition, 163, 186–​187, 189–​190, 193 Kassabian, Anahid, 89–​90 “Keep Austin Weird” campaign, 212, 213 Kimme Hea, Amy, 43, 64 Kinneavy, James, 15, 223n1 Koolhaus, Rem, 172 The Language of New Media (Manovich), 88, 106n1 Lanham, Richard, 19, 90, 106n1

247

Lansdowne Revived (Greenspan), 181–​182 Laptops, xix, 3, 6, 26 Latour, Bruno, 133 Laurel, Brenda, 30 Layered time, 168 Layers, 160–​168. See also ATLAS interface archives and, 164–​165 in augmented reality, 160–​163, 164–​165, 170 cultural production of, 168 in David’s Way Plaza project, 157, 172–​173 functions of, 142, 143t historical, 164–​165, 166 in From Lewisburg, PA to Silicon Valley, 166–​168, 167f metonymical evolution of, 144–​145 of multimedia, 12, 121 in Paths II installation, 163–​164 rhetoricity of, 163 in Streetmuseum project, 164–​166, 165f in wearable technologies, 168 Leach, Matthew, 197 Le Dantec, Christopher, 54 Lefebvre, Henri, 66 LeFevre, Karen Burke, 32 Leroy, Louis, 191 Libraries as deferred archives, 112, 115, 126 street-​libraries, 10, 13, 127, 135 Licklider, J. C. R., 118 Lily and Honglei (artists), 166–​167 Linear time, 168 Ling, Rich, 72 Lingua Fracta (Brooke), 24 Lists, 113, 114–​115 Literacy, 39–​42, 87, 161 Liu, Lydia, 90

248

2 4 8   • 

Index

Local action actionable archives and, 109 in augmented reality, 160 contingencies of, 219 correspondence of digital media with, 98 defined, 3 GPS navigation and, 74, 75, 76 incorporation into texts and audiovisual works, 17, 18 Monet and, 193, 194 networked multimedia and, 3, 4, 58, 142 perceptual awareness of, 73, 119, 121 publics approach to, 212–​213 real-​time information events and, 125 withdrawal of attention from, 117 Locative media apps and, 66, 67, 68–​70, 83n1 commercialization of, 66 in digital arts and urban informatics, 83n3 emergence with ubicomp, 58 humanistic studies of, 29 interpretations of ubicomp paradigm using, 66–​67 rhetorical relationships in, 67–​68 shift to actionable media from, 67–​69 user-​generated content, 66 “Made Not Only in Words” (Yancy), 24 Madrigal, Alexis, 7, 179, 195 Mainwaring, Scott, 56n4 Manovich, Lev on Adobe Photoshop, 144–​145 on augmented space, 8–​9 on Cardiff, 206, 209 on cinematic computing, 44–​45, 90 on mobile media societies, 63–​6 4 on tele-​action, 3 on writing, 86, 88–​89, 106n1 MapMyRide (app), 54

Marginal fragments of writing, 205 Marí, Bartomeu, 116 Marino, Mark, 87 McCarthy, Anna, 89–​90 McCullough, Malcolm, 1, 80, 141, 173 McKeon, Richard, 1–​3 McLuhan, Marshall, 99, 135 Meaning-​making, 14, 69, 111 Media aesthetics, 137, 180, 215n2 Media theory, 19, 24, 86, 89, 99, 106 Meme campaigns, 14–​15 Menus. See WIMP interface Merleau-​Ponty, Maurice, 120 Metahistory (White), 23n3 Metaphor, defined, 120, 144 Metonymical invention, 95, 99–​100, 103 Metonymy, 119–​121, 137, 144–​145 Military technologies, 17 Miller, Carolyn, 15 The Missing Voice (Case Study B) (Cardiff ), 207, 209, 218 Mnemonic devices, 111 Mnemotechnics, 97, 98, 99, 100 Mobile devices anytime-​anywhere connectivity expectations and, 72, 73–​74 data extraction from, 9 design philosophies for, 37 Internet access on, 6–​7, 26, 146 Mobile interfaces, 7, 8, 28 Mobile Interface Theory (Farman), 60 Mobile media. See also Mobile devices definitions of, 65, 66 divided attention in, 82 emergence with ubicomp, 58 humanistic studies of, 29 interpretations of ubicomp paradigm using, 59–​61 research agenda for, 60, 65–​66 sociocultural impact of, 59, 63–​6 4 Moleskine “Smart Writing Set,” 48

249

Index  • 

Monet, Claude, 185–​195 composition techniques of, 191–​194 creative practices and works of, 21, 182 en plein air painting methods used by, 21, 185–​189, 194, 195 kaleidoscopic composition of, 186–​187, 189–​190, 193 photography as influence on, 188, 189 Montfort, Nick, 30 Moral dilemmas, 51 Morozov, Evgeny, 53–​54 Multimedia. See Networked multimedia Museums augmented reality projects at, 12–​13, 12f, 22n2 white cube in, 115–​117 Mystory genre, 92 Nanotechnology, 78, 97 Navigation apps, 9 Nelson, Ted, 30 Networked multimedia in automobiles, 71 circulation in built environments, 89 in citizen science apps, 149 defined, 3 embedding of, 80 local action and, 3, 4, 58, 142 mobility of, 138 New materialism, 32, 33 New media cinematic influences on, 89–​90 concerns regarding, 5 interdisciplinary study of, 21, 22n1 marginalization of writing in, 90 scholarship on, 2, 42, 86, 88, 139 New Media Reader (Wardrip-​Fruin & Montfort), 30 Nuclear traits of writing, 105, 107n4, 112 O’Doherty, Brian, 116 Offenhuber, Dietmar, 159

249

Of Grammatology (Derrida), 92, 101, 177n4 Olmsted, Fredrick Law, xi, xii, xvi, xvii On the Mode of Existence of Technical Objects (Simondon), 128–​129 Oppegaard, Brett, 181 Optical sensors, 170 Orthographic continuums actionable archives and, 110, 119 deferred archives and, 108, 110 real-​time information events on, 122–​123 techno-​geographic milieus on, 128 tracing, 91, 99–​104, 105 Orthographic writing, 39, 124 Paths II: The Music of Trees (Aresty), 163–​164 Paul, Christine, 162, 163 Pedagogy of the Oppressed (Freire), 198 Pedestrian sign network, in Central Park, xi, xiii Personal computers (PCs). See also Desktops globalization and, 31 laptops, xix, 3, 6, 26 limitations of, 35–​36, 48 ubicomp vs., 35–​38 Weiser on, 35–​36, 38, 46, 47 Personal ownership principle, 60, 61 Persuasive Games (Bogost), 87 Pew Research Center, 26 Phenomenology existential, 22, 33, 46–​47 of memory and perception, 123 postphenomenology, 100–​101 Phonemes, 102, 103 Phones. See Smartphones Phonocentrism, 92, 101, 105, 119 Photography, 124, 187–​189, 190, 192 Physical computing, 48 Pictograms, 110, 119, 121

520

2 5 0   • 

Index

Picto-​ideo-​phonographic writing, 97, 103 Planned Obsolescence (Fitzpatrick), 180 Plato, 193, 220 The Pleasure of the Text (Barthes), 184 Poetics (Aristotle), 199 “The Poetics of Augmented Space” (Manovich), 172, 207 Pointers. See WIMP interface Pokémon Go (app), 31, 69–​70, 70f Political rhetoric, 3 Pollack, Barbara, xvii Polyaesthetic design space, 163 Polyfunctional elements, 129 Polyrhetorical roles, 131, 132, 134 Post-​desktop interfaces. See also Actionable media archival modalities associated with, 19, 81 authoring of built environments in, 25 content creation for, 83 gramme in, 113 mobile media scholarship on, 59, 60 opportunities presented by, xix, 12 in urban infrastructure, 79 Post-​desktop practices automobile computing and, 71 connection to environment in, 65 criteria for delineation of, 58 critical interpretations of, 19, 57, 74 shortcomings of, 81 ubicomp and, 66 Posthumanism, 33, 75, 132, 134, 140 Postphenomenology, 100–​101 Post-​WIMP discourse, 139–​141 Predetermined programs, 118 Primary retention, 123, 124–​125 Print culture, evolution of, 5 Programming vs. writing, 87 Proletarianization, 97, 98 Propen, Amy, 75

Public intellectuals, 21, 116, 176, 182, 210, 217 Publics approach to local action, 212–​213 Push notifications, 67, 148 QR coding, 154 Quigley, Aaron, 48 Quintilian, 139, 177n1 Radio-​frequency identification tags. See RFID tags Radiophonic speech, 206 Raley, Rita, 66 Ramsay, Stephen, 214–​215n1 Ramus, Peter, 23n3, 193 Ratti, Carlo, 4, 61, 62 Readerly text, 40 Readiness-​to-​hand theory, 45–​4 6 Reality. See Augmented reality Reality-​effect of photography, 124, 192 Real-​time information events, 76, 121, 122–​128 Remediation theory, 95 Renaissance painting, 177n1 Re+Public AR|AD Takeover project, 170–​171, 171f, 172 Retention categories, 123, 124–​125 RFID (radio-​frequency identification) tags, 9, 80, 154, 156, 157–​159, 177n3 Rhetoric actancy and, 133 of actuators and sensors, 169–​170, 174 as architectonic productive art, 1–​2, 3 in audio walks, xviii, 210 augmented reality and, 10 of community discourse, 212–​213 digital communication and, 2, 22n1 figurative imagination of, 19, 20, 137, 139 genres of, 3

215

Index  • 

in GPS navigation processes, 75 invention and, 32–​33 layers and, 163 locative media and, 67–​68 situational character of, 15 tags and, 155–​157 uses of, 1–​2, 3 Rhetorical triangle, 199, 200, 200f Rice, Jenny, 22n1, 212–​213 Rickert, Thomas, 33, 140, 155–​157 Ridolfo, Jim, 22n1 Roberts, Ben, 101 Rod, Jan, 140–​141 Rodowick, N. D., 44–​45 Rogers, Yvonne, 44 Rose, David, 16 Rueb, Teri, 113, 122, 127–​128, 134–​135 Ruskin, John, 189 Sal anecdote (Weiser), 52–​53, 54 Schapiro, Meyer, 191 Scharf, Aaron, 188, 189 Screen-​cameras, 161, 162–​163, 164, 166, 169 Sculpture in Environment (exhibit), xiv Secondary retention, 123 SeeClickFix (app), 151–​152 SeeWorld (Bianchi et al.), 182 Sender-​receiver models of communication, 22n1, 40, 200 SENSEable Cities Lab, 61, 156, 158, 203 Sensors, 169–​176. See also ATLAS interface in AR|AD Takeover project, 170–​171, 171f democratization of, 175 embedded, 37, 49, 169, 172–​173, 175, 203 functions of, 142, 143t, 169 optical, 170 rhetoricity of, 169–​170, 174

251

in smart environments, 169 in traffic lights, 175–​176 Sensory ethnographies, 211 The Shallows (Carr), 4–​5, 6 Sheer portability principle, 60, 64, 65, 67, 71 Shelley, Mary, 127 Shirky, Clay, 50 Signification, 10, 113, 119–​120, 121, 185, 194 Sikora, Dave, 147 Simondon, Gilbert, 93, 128–​130 Situated software, 50 Smart cities, xix, 54, 79, 222 Smart environments accessibility to, 60 actionable feedback loops in, 37 actuators and sensors in, 169 The Adaptable Bus Stop, 62–​63, 63f Digital Water Pavilion, 61–​62, 62f emergence of, 29, 31, 35, 54 gramme and gesture networks in, 98 opportunities presented by, 44 Smart homes, 61 Smartphones commercialization of ubicomp through, 57 digital distraction and, 6 distracted driving and, 72 interfaces for, 59 Internet access on, 26, 146 networked multimedia access on, 3 retention and, 124 versatility of, 58, 63 Social networking, 5, 73, 74, 131–​132, 147, 150 Socrates, 4, 220 Souza-​e-​Silva, Adrianna, 8 Spatial kairos, 219, 223n1 Spect-​actors, 21, 199, 200–​205, 208–​210

25

2 5 2   • 

Index

Speech action and, 221–​222 alphabetization of, 96 figures of, 23, 138, 142, 177n1 grammatization of, 221 metaphysical opposition of writing and, 91 patterns of, 19, 95 radiophonic, 206 Speech-​to-​text software, 72 Spreadable media, 14–​15 Stein, Bob, 147 Stiegler, Bernard on computational capitalism, 81 on critical invention, 81, 82 Derrida as critiqued by, 101–​102, 105–​ 106, 107n6, 112 on formalization of new gramme, 95 grammatization as defined by, 94 grammatological perspective of, 40, 92, 97 on industrialization, 97–​98 intellectual contributions of, 90, 91, 106n3 on interplay of gestures and gramme, 97, 99 on literate experience of historical consciousness, 76 on mnemotechnical systems, 99 postphenomenology of, 100 on real-​time information events, 76, 121, 122–​124 on retentions, 123 on techno-​geographic milieus, 121, 128, 129–​130, 131, 132, 160 on techno-​human relations, 93–​94, 98, 99 on writing systems, 85–​86, 111, 112 YouTube analysis by, 103, 104 Strava (app), 54 Street-​libraries, 10, 13, 127, 135 Streetmuseum project, 164–​166, 165f

The Structural Transformation of the Public Sphere (Habermas), 221 “Style and its Image” (Barthes), 183 Subject/​object dualism, 140–​141 Surveillance, 44, 75, 79, 155, 157, 160 Sutko, Daniel, 68 Tablets Internet access on, 26, 146 networked multimedia access on, 3 retention and, 124 versatility of, 58, 63 Taco Bell, use of digital coupons by, 67 Tags, 153–​160. See also ATLAS interface embedded, 170, 173 evolution of, 144, 145, 153–​154 functions of, 142, 143t geotags, 18, 80, 154, 170 HTML, 145, 153 political implications of, 154–​155 QR coding, 154 RFID, 9, 80, 154, 156, 157–​159, 177n3 rhetorical analysis of, 155–​157 in Trash|Track project, 157, 158–​160, 158f Tangible interfaces, 48–​49 Technics and Time (Stiegler), 91, 99, 101, 130 Techno-​geographic milieus, 121, 128–​135, 160, 161 Techno-​human relations, 93–​94, 98, 99, 129 Technological determinism, 93 Technological unconscious, 173, 174, 175 Tele-​action, 3–​4, 14, 140, 142 Telepathic technologies, 43 Territories, digitization of, 130, 134 Tertiary retention, 123, 124–​125 Texting while driving, 72, 73 Theatre of the Oppressed (Boal), 198 Theoretical grammatology, 91–​92

253

Index  • 

Theoria (intellectual seeing), 77, 84n5, 211 Thoreau, Henry David, 53, 220, 223–​224n2 Thrift, Nigel, 173–​174 Time, linear vs. layered, 168 Topoi, 193–​194, 197 Townsend, Anthony, 50 TRACE projects, 182 Tracking devices, 54, 155. See also Locative media Traffic lights, 18, 175–​176 Transcoding, 142, 160, 189–​190 Trash|Track project, 157, 158–​160, 158f, 203 Trope of hybridity, 8, 9, 29 Turkle, Sherry, 5–​6 Twitter, 154 Uber, 50, 69 Ubiquitous computing (ubicomp), 24–​56 actionable archives of, 84n6, 109, 126, 128 commercialization of, 57 context-​awareness in, 29, 71, 78 democratization of, 79–​80 dissemination of, 55, 89 emergence of media with, 25, 58 flow of digital information in, 47–​51 grammatization and, 91, 95, 98, 100, 120 grammatology and, 40 humanism and, 19, 29 industrial development of, 80–​81 intellectual ethic for, 30–​32 interface design in, 44, 48, 120–​121 interpretations of, 59–​61, 64–​65, 66–​67 invention of, 32–​35, 89 personal computers vs., 35–​38 phase1 capabilities of, 26–​27, 51

253

philosophical foundations of, 46–​47 political implications of, 154–​155 readiness-​to-​hand theory and, 45–​4 6 real-​time information events of, 124–​125 remediation theory on, 95 research agenda for, 64 research and development of, 28 rise of, xi, 4, 18, 44 Sal anecdote of, 52–​53, 54 software for, 20 tabs, pads, and boards in, 39, 51–​52 as technocultural paradigm, 24, 29, 90, 108 technological challenges associated with, 57–​58 tracking devices in, 54, 155 Weiser as father of, 21–​22, 31 writing and literacy analogies in relation to, 28, 38–​42 Ubiquitous Listening (Kassabian), 89–​90 Ulmer, Gregory, 33, 34, 51, 56n3, 91, 92, 97, 211 Understanding Computers and Cognition (Winograd & Flores), 45 Ungar, Steven, 186 Urban Botanical Garden (bus stop interface), 49 Urban signage, 21, 102, 125–​126 US Department of Transportation, 72 User-​friendly interfaces, 43, 117, 141 User-​generated content, 66, 153–​154 US National Safety Council, 72 Van Dam, Andries, 139 Vehicles. See Automobiles Venturi, Lionello, 189 Vertov, Dziga, 88 Videogames, 87 Virtual reality, 8, 37, 45, 89 Virtual windows, 117, 120, 121 Voices of Oakland (Bolter), 181

524

2 5 4   • 

Index

Wajcman, Judy, 53 Walden (Thoreau), 53, 220 Wall-​e (film), 77 Walled gardens, 147 Walter, E. V., 211 Wardrip-​Fruin, Noah, 30, 200 Warhol, Andy, 112–​113 Water Lilies series (Monet), 192 Watkins, Kari, 54 Wearable technologies. See also Headworn displays actuators and sensors in, 169 ambience of, 38 augmented reality and, 195 data extraction from, 9 Google Glass, 15–​16 gramme and gesture networks emerging with, 98 hands-​free, 72 layers in, 168 prototypes of, 26, 29 in techno-​geographic studies, 133 versatility of, 58, 63 in workplace, 79 Web 2.0 platforms apps and, 149, 150 authoring opportunities presented by, xix, 6, 135 location-​based features of, 66 tags in, 144, 154 Web 3.0 platforms, 144, 145, 154–​157, 160 “The Web is Dead. Long Live the Internet” (Anderson & Wolff ), 146 Weiser, Mark CATTt framework as applied to, 33–​34, 35 on desktops and personal computers, 35–​36, 38, 46, 47 devices prototyped by, 26, 27, 29 as father of ubicomp, 21–​22, 31 good tool ideal of, 42–​43, 44, 46

Heidegger’s influence on, 45, 46 intellectual ethic in work of, 31, 32, 33, 64 on negative impact of technology, 24, 47, 71 posthumous influence of, 55n1 Sal anecdote by, 52–​53, 54 tabs, pads, and boards outlined by, 39, 51–​52, 138 ubicomp initiatives of, 28, 47 on virtual reality, 37 on writing and literacy analogies in relation to ubicomp, 28, 39–​4 0, 41, 161 White, Hayden, 23n3, 120 White cubes, 115–​117, 203 Wikipedia, 66, 68, 135 WIMP interface for desktops, 20, 47–​48, 138, 139–​140 historicization of figures from, 177n2 layers in, 144, 145 limitations of, 117–​118, 140 for mobile media, 118 post-​WIMP discourse, 139–​141 Windows. See Virtual windows; WIMP interface Winograd, Terry, 45 Wolff, Michael, 146, 147 WordPress, 135, 154 Words Drawn in Water (Cardiff ), 207 Writerly text, 40 Writing and writing systems. See also Grammatization; Grammatology alphabetic (see Alphabetic writing) analogies in relation to ubicomp, 28, 38–​42 arche-​writing, 92, 101–​102, 103, 105, 106n4, 112 attributes of, 87 in built environment, 40

25

Index  • 

context-​awareness in, 111–​113 cuneiform, 5, 94, 102, 103, 110–​111, 119 demise of, 86–​87 digital, xviii, 42 in electronic textuality, 88 of events, 205–​206, 210 evolution of, 39, 86, 91 hieroglyphics, 91–​92, 102, 103 line development in, 177–​178n4 lists, 113, 114–​115 marginal fragments of, 205 marginalization in new media, 90 metaphysical opposition of speech and, 91 nuclear traits of, 105, 107n4, 112 orthographic, 39, 124 pictograms, 110, 119, 121 picto-​ideo-​phonographic, 97, 103 political constraints on, 41

255

programming vs., 87 representation function of, 95–​96 signification in, 119–​120, 121 Socratic critiques of, 4–​5 technics of, 93–​99, 100 Wyoming Migration Initiative, 133 Xerox PARC commercial motivations of experimentation at, 25 desktop inventions at, 28 devices inventions and prototypes, 20, 26, 27, 138 ubicomp developments at, 26, 38, 51 Yancy, Kathleen Blake, 24 Yelp, 10, 11f, 66 YouTube, 103, 104, 135, 154 Zuckerberg, Mark, 43

526

257

528