Empirical Comics Research: Digital, Multimodal, and Cognitive Methods 2018014754, 9781138737440, 9781315185354

Citation preview

Empirical Comics Research

This edited volume brings together work in the field of empirical comics research. Drawing on computer and cognitive science, psychology and art history, linguistics and literary studies, each chapter presents innovative methods and establishes the practical and theoretical motivations for the quantitative study of comics, manga, and graphic novels. Individual chapters focus on corpus studies, the potential of crowdsourcing for comics research, annotation and narrative analysis, cognitive processing and reception studies. This volume opens up new perspectives for the study of visual narrative, making it a key reference for anyone interested in the scientific study of art and literature, as well as the digital humanities. Alexander Dunst is Assistant Professor of American Studies at the University of Paderborn, Germany Jochen Laubrock is Senior Lecturer in Cognitive Psychology at the University of Potsdam, Germany Janina Wildfeuer is Researcher in Multimodal Linguistics at Bremen University, Germany

Routledge Advances in Comics Studies

Edited by Randy Duncan, Henderson State University Matthew J. Smith, Radford University

Reading Art Spiegelman Philip Smith The Modern Superhero in Film and Television Popular Genre and American Culture Jeffrey A. Brown The Narratology of Comic Art Kai Mikkonen Comics Studies Here and Now Edited by Frederick Luis Aldama Superman and Comic Book Brand Continuity Phillip Bevin Empirical Comics Research Digital, Multimodal, and Cognitive Methods Edited by Alexander Dunst, Jochen Laubrock, and Janina Wildfeuer

Empirical Comics Research Digital, Multimodal, and Cognitive Methods

Edited by Alexander Dunst, Jochen Laubrock, and Janina Wildfeuer

First published 2018 by Routledge 711 Third Avenue, New York, NY 10017 and by Routledge 2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN Routledge is an imprint of the Taylor & Francis Group, an informa business © 2018 Taylor & Francis The right of editors Alexander Dunst, Jochen Laubrock, and Janina Wildfeuer to be identified as the authors of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Names: Dunst, Alexander, 1980– editor. | Laubrock, Jochen, editor. | Wildfeuer, Janina, 1984– editor. Title: Empirical comics research: digital, multimodal, and cognitive methods / edited by Alexander Dunst, Jochen Laubrock, and Janina Wildfeuer. Description: New York: Routledge, 2019. | Series: Routledge advances in comics studies; 6 | Includes bibliographical references and index. Identifiers: LCCN 2018014754 | Subjects: LCSH: Comic books, strips, etc.—History and criticism. | Graphic novels—History and criticism. Classification: LCC PN6714 .E48 2019 | DDC 741.5/9—dc23 LC record available at https://lccn.loc.gov/2018014754 ISBN: 978-1-138-73744-0 (hbk) ISBN: 978-1-315-18535-4 (ebk) Typeset in Sabon by codeMantra

Contents

List of Figures and Tables Acknowledgements 1 Comics and Empirical Research: An Introduction

ix xv 1

A lexander D unst, J ochen L aubroc k , and J anina W ildfeuer

Part I

Digital Approaches to Comics Research

25

2 Two Per Cent of What? Constructing a Corpus of Typical American Comic Books

27

B art B eaty, N ic k S ousanis , and B enjamin Woo

3 The Quantitative Analysis of Comics: Towards a Visual Stylometry of Graphic Narrative

43

A lexander D unst and R ita H artel

4 “The Spider’s Web”: An Analysis of Fan Mail from Amazing Spider-Man, 1963–1995

62

J ohn A . Walsh , S hawn M artin , and J ennifer  S t. G ermain

5 Crowdsourcing Comics Annotations

85

M ihnea T ufis and J ean - G abriel G anascia

6 Computer Vision Applied to Comic Book Images C hristophe R igaud and J ean - C hristophe  B urie

104

vi Contents Part II

Linguistics and Multimodal Analysis

125

7 From Empirical Studies to Visual Narrative Organization: Exploring Page Composition

127

J ohn A B ateman , A nni k a B ec k mann , and  RO C Í O I N É S VA R E L A

8 Character Developments in Comics and Graphic Novels: A Systematic Analytical Scheme

154

C hiao - I T seng , J ochen L aubroc k , and  J ana  P flaeging

9 How Informative are Information Comics in Science Communication? Empirical Results from an Eye-Tracking Study and Knowledge Testing

176

H ans - J ü rgen B ucher and B ettina B oy

10 The Interpretation of an Evolving Line Drawing

197

Pascal L ef è vre and G ert M eesters

Part III

Cognitive Processing and Comprehension

215

11 Viewing Static Visual Narratives through the Lens of the Scene Perception and Event Comprehension Theory (SPECT)

217

L ester C . L osch k y, J ohn P. H utson ,  M averic k  E . S mith , T im J . S mith ,  and J oseph P. M agliano

12 Attention to Comics: Cognitive Processing During the Reading of Graphic Literature

239

J ochen L aubroc k , S ven H ohenstein , and  M atthias K ü mmerer

13 Reading Words and Images: Factors Influencing Eye Movements in Comic Reading C lare Kirtley, C hristopher M urray, P hillip B .  Vaughan , and B enjamin W. Tatler

264

Contents  vii 14 Detecting Differences between Adapted Narratives: Implication of Order of Modality on Exposure

284

J oseph P. M agliano , J ames A . C linton , E dward   J .   O ’ B rien , and David N . R app

15 Visual Language Theory and the Scientific Study of Comics

305

N eil C ohn

Glossary List of Contributors Index

329 337 347

List of Figures and Tables

Figures 2.1 Number of comic books in the sampling frame per year 35 3.1 Illustration of Shape definition. The image is an adapted excerpt from https://commons.wikimedia.org/ wiki/File:BD-propagande_colour_ en.jpg (Licensed under CC BY) 50 3.2 Mean Brightness across genres: Graphic Memoir Graphic Novel (p < 0.016), Graphic Fantasy Graphic Novel (p < 0.000) 52 3.3 Year vs. Mean Brightness by Genre 53 3.4 Year vs. Standard Deviation from Mean Brightness by Genre 54 3.5 PCA Author Style 55 3.6 Standard Deviation from Shapes per Page 1 Author - Author + Illustrator (p < 0.0114) 56 3.7 Standard Deviation of Mean Brightness vs. Standard Deviation of Number of Shapes by Book Titles 57 3.8 Mean Brightness by Publication Format: Book Publication - Limited Series (p < 0.001), Book Publication - Unlimited Series (p < 0.004), Book Series - Unlimited Series (p < 0.02) 58 4.1 TEI XML encoding of a letter-reply group 65 4.2 Number of letters per year 68 4.3 Yearly average of words per letter, per reply 68 4.4 Representation of multiple topics related to making and makers throughout the corpus 73 4.5 Two topics associated with major characters: Topic 8 (Character: Gwen Stacy) and Topic 13 (Character: Peter Parker) 75 5.1 Marking the elements on a comic page in Comics++ 90 5.2 Transcribing marked characters in Comics++ 91 5.3 The page distribution of marked-up elements for each of the four stories: (a) “Dash 1Dillon,” (b) “Nightro,” (c) “Truant Toy,” (d) “Shrinking Horror” 95

x  List of Figures and Tables 6.1 The interaction loop between low and high level features 114 6.2 Example of initial (top) and final (bottom) representation of the knowledge for an image (I) containing two panels (P). In the initial stage, the system makes hypotheses (dashed squares) about the unrelated positions of panels (P), balloons (B, with tail Q), and text regions (T). After processing, the validated information (solid squares) is structured into two distinct panels containing balloons (B) and speech balloons (SB), and also text and speech text (ST). The link between speech balloons and some speaking characters SC has also been established, according to the tail (Q) direction detection 115 6.3 Evolution of the amount of discovered information for panels P, balloons B, text lines T, comic characters C, STSB and SBSC extractions (solid line) using the F-measure. The first and second rows correspond to the hypothesis and evaluation steps of the first and second iterations of the process respectively. Values correspond to F-measures for P, B, T, C and accuracies for STSB and SBSC. The dashed line represents the best score using our model on the data extracted from the ground truth (optimal condition). The solid line is the performance of the framework using all automatic extractions and element associations 117 7.1 Classification options under ‘multicell’ configurations (i.e., page designs giving rise to collections of ‘cells’ such as panels) with example layout realizations for the main features shown on the right. Square brackets 132 indicate strict alternatives, curly brackets conjunction 7.2 Schematic illustration of a filled gap composition. In the page on the left-hand side, the target panel has no borders and runs out ‘behind’ the other panels on the page; in the page on the right-hand side, the target panel has the usual panel borders of the (in this case: 135 row-based vario-table) layout as a whole 7.3 The gridding stimuli page organizations. Panels are numbered for purposes of analysis and showing results below; this does not necessarily correspond to 138 reading order 7.4 Distributions of visiting times for the gridding dataset showing both overall correlations (R 2) between the predefined abstract panel numbering and all visiting times, and correlations specifically with the medians of the visiting times per panel (R m2) 140

List of Figures and Tables  xi 7.5 Schematic depictions of the test materials for the gapping experiment, showing the original and modified pairs. The panels affected by the 143 manipulation are indicated by arrows 8.1 Pages 2 and 3 of the graphic novel City of Glass. The red circles indicate visual or verbal references to the protagonist 157 8.2 Cohesive chains of the dominant narrative elements from pages 2 and 3 of Karasik and Mazzuchelli (2004). Capital letters are text and descriptions in 158 square brackets refer to visual depictions 8.3 Action chains in the first seven pages of City of Glass. The numbers refer to the panels and pages in Figures 8.2, 8.4, and 8.5.  The actions in square brackets are depicted as images; caption texts in the graphic novel are type-set in capital letters 162 8.4 Schematic representation of action pattern and actoractivity relationships during the initial seven pages in City of Glass 163 8.5 First eight pages of Thomas Ott’s Dead End (2002) 164 8.6 A generalized event pattern abstracted from the first eight pages of Dead End 165 8.7 Examples of the three manipulation types as used in 167 stimulus creation 9.1 Diegetic and non-diegetic strategies of combining narration and information 180 9.2 Percentage distribution of total dwell time on different types of AOIs 184 9.3 Scan path of a test person with focus on speech bubbles (left) and scan path of a test person with focus on pictorial elements (right) 189 9.4 Navigation path and logic of the story with scan path of test person 2 (diameter of circles represents fixation duration) 192 10.1 The fifteen static pictures of the evolving drawing 200 (used in the second and third condition) 10.2 The percentages of the total responses that the most 203 popular concepts represented for each phase 10.3 The most popular concepts, totaling more than 10% of the responses, detailed in percentages per phase and 205 per condition 10.4 The normalized number of concepts suggested for 208 every phase 10.5 The percentages of blank responses for every phase 209 11.1 Box model of the scene perception and event comprehension theory (SPECT) 219

xii  List of Figures and Tables 11.2 Experimental manipulation conditions for bridging inference. A complete 3-image target episode from Figure 11.2 is shown, including beginning state, bridging event, and end-state images. The missing bridging-event condition requires the viewer to infer the bridging event when they see the end-state image 220 11.3 Illustration of event segmentation task in a BDF story (adapted from Mayer’s “One Frog Too Many”). After participants read the entire story, they saw thumbnails of all the images in sequential order. Their task was to click on each picture when they thought a change in the story situation occurred. Normative event segmentation for this story is indicated in this illustration by dashed lines around the most frequently segmented images 222 12.1 Proportion of fixations falling on different regions of interest (ROIs: character, caption, balloon, remainder of panel) is separate for different classes of fixations. Namely, first and later fixations (point shape) in first pass or higher passes (line color). The dotted black line indicates the baseline area taken up on average by the ROI classes. The inset shows the selectivity-offixations measure that was used in the analysis. See text below for details 249 12.2 (a) Page 78 from the German edition of Daniel Clowes’ graphic novel Ghost World showing empirical fixation locations (gray dots) and Deep Gaze II predictions (contour lines). [Original ­material used with kind permission by Reprodukt] (b) Distribution of information gain explained of empirical fixations by Deep Gaze II over all pages of PoCoCo-1 252 13.1 Example data of a reader showing A: Landing position within a panel; B: A regressive saccade and fixation; C: A first pass skip of a panel; D: A total skip of a panel. Drawing by lead author 270 13.2 Influence of the presence of text in the current and subsequent panel on (left) first pass skipping and (right) full skipping 275 15.1 Two pages analyzed for their (a) narrative structure, and (b) external compositional structure 311 15.2 Sequences created through ‘paraphrasing’ the narrative of the pages in, by (a) dropping out all but Peak panels, and (b) dropping out all by Initial panels 312 15.3 Scatterplot of different comics’ frequency using the basic narrative progression and

List of Figures and Tables  xiii environmental-conjunction. ‘Manga-­influenced’ works are those imitative of manga, but created by English speakers outside Japan 314 15.4 Basic features of the external compositional structure of a page layout 317

Tables 5.1 Overview of the different types of elements identified with Comics++ 94 7.1 Interaction effects for the generalized linear model relating modification status and pages, with participants and panels as random effects (top) 147 8.1 Absolute frequencies per response category, chi-square values, degrees of freedom, and p-values for each of the six single-choice questions 171 9.1 The corpus of the study and its features (6 double pages, 3 single pages) 181 9.2 Average points for correct answers in the recall test (maximum: 9 points, n = 25) and total dwell time (from lowest do highest recall score) 185 9.3 Average number of revisits and dwell time on selected pages: diegetic vs. non-diegetic comics (from least to most revisits) 186 12.1 Fixed effects coefficients for the Poisson generalized linear mixed effects regression model predicting the number of first fixations as function of ROI (panel background, character, caption, balloon), fixation type (first or later fixation in ROI), and pass (first or higher pass). Random effects were estimated for participant (sd = 0.113) and book (sd = 0.358) 250 12.2 Coefficients of the linear mixed effects regression model predicting the log of fixation durations as a function of a large number of predictors. Random effects for this model were as follows: participant, sd = 0.112; book/title, sd = 0.038; residual, sd = 0.466. Numbers in the “Estimate” and “Std. Error” columns are multiplied by 1000 for better readability 254 13.1 Predictors influencing the average total dwell time within whole panels and text or image regions 271 13.2 Factors influencing the landing position of the first fixation to a panel 273 13.3 Factors influencing first pass and full skips of panels, and text and image regions within panels 274 13.4 Factors influencing regressions from and to a panel 276

xiv  List of Figures and Tables 14.1 Example protocols reflecting the different types of discrepancies and their relationships with the prior exposure of BMY1 294 14.2 Differences in the types of discrepancies noticed as a function of the modality of the second exposure 296 14.3 Differences in the relationship between versions as a function of the modality of the second exposure 297

Acknowledgements

A number of chapters collected in this volume were first presented at a conference titled ‘The Empirical Study of Comics,’ which took place at the University of Bremen in February 2017. The editors would like to thank the Institutional Strategy of the University of Bremen, funded by the German Excellence Initiative, and the German Federal Ministry of Education and Research (BMBF) for their generous support of this event.

1 Comics and Empirical Research An Introduction Alexander Dunst, Jochen Laubrock, and Janina Wildfeuer Indeed if I had one single ambition in literary studies it would be to rejoin them with experimental science. (Williams 341) We have only begun to discover the benefits of seeing science and art as one. (Kuhn, “Comment” 405) It is a lesson that is imparted to us from every part of the history of the natural sciences that the progress of any science is closely connected to the progress made regarding its methods. (Wundt xi, translation ours)

1.  Empirical Cultural Research Compared to dominant hermeneutic paradigms, empirical research that engages with culture and the arts plays a minor, sometimes even neglected, role. Several factors are currently challenging this status quo. The combination of large-scale digitization efforts, computational tools that enable sophisticated analysis on personal computers, and the online availability of research software are reconfiguring the humanities in ways that are imperfectly captured by references to a digital turn. To date, commentators have paid most attention to quantification—both in terms of the sheer volume of humanities data that is now becoming accessible and efforts to ­ nderwood). adjust the scales of inquiry accordingly (Schöch; English and U At least equally important are arguably more basic consequences of ­digitization: The transformation into binary information makes cultural data amenable to computational analysis via the machine-readable formalization of concepts and their integration into software. Unlike a hermeneutical process, whose outlines may be documented in writing but remain bound to subjectivity, computational calculation can be archived directly, as well as studied, repeated, and adapted by others. This externalization of the research process generalizes an element that has long characterized the sciences but played only a minor role in

2  Alexander Dunst et al. the humanities—empirical testing. Testing in turn introduces a novel explicitness and provides the basis for mediating between different arguments and positions. As Andrew Piper argues, the humanities by and large develop in an agonistic fashion. Too often in the humanities, what counts as a research outcome takes the form of disagreeing with an existing opinion. The sheer number of methodological frameworks that fragment into incompatible schools, the recourse to individual sets of texts that form the objects of study, and the ultimate opacity of hermeneutics all mitigate against agreement. In contrast, formalization and computation act as constraints that make each step of the research process legible. As a consequence, Piper writes, “the tools and information for mediating disagreement are made more mutually available” ­(“Numbers”). In this sense, empirical cultural research presents an ­opportunity to readjust the scales, not only by way of quantification, but also by making the humanities more consensus-driven. Where tools and corpora, algorithms and data can be shared digitally, we might see the emergence of a qualitatively different research culture in the humanities: One in which existing knowledge provides the building blocks of future research, where the daily practice of scholarship becomes more collective and less captive to the mystique of individual insight. It is this spirit that animates the present volume. Most of the chapters collected here were presented at a conference titled “The ­Empirical Study of Comics” and held at the University of Bremen in February 2017. This meeting, the first to focus on the topic, brought together an unusually broad mix of disciplines in an atmosphere of genuine scientific discovery. In similar ways, this volume combines input from linguistics, literary and media studies, as well as cognitive and computer ­science. Comics, once looked down upon as a lowly form of mass c­ ulture, have interested researchers for several decades now. Their study has become increasingly institutionalized, with new journals and associations accompanying a constant stream of publications. In keeping with the humanities at large, however, empirical comics research remains at an early stage of development. One motivation in editing this book consists in providing a single reference point that may form the basis of future research. More fundamentally, it is our aim to sketch the contours of a new research program. In keeping with this ambition, we define empirical comics research as a set of methods capable of supporting or falsifying its hypotheses about the medium of comics, often constructed by theoretically and methodologically fine-grained analyses, with the help of empirical testing and quantitative corpus studies. We strongly believe that empirical approaches have the potential to transform research on comics, and narrative media more generally. Success in this undertaking ultimately will depend on the development of a distinct research culture that frees itself from the (often implicit) expectations of parent disciplines to produce results that

Comics and Empirical Research  3 appeal to scholars from a variety of backgrounds. Given the number of disciplines that may potentially contribute to empirical comics research and the strength of established paradigms, this ambition remains an acute challenge. Nonetheless, a shared set of concerns can be seen to emerge from this volume. These include: 1 A clear distinction between the told and the telling, or, in other words, between a multimodal document and its reception by empirical readers. The consequences of this distinction can be felt across several contributions, most importantly in the inclusion of a section on cognitive research. In a sign of the methodological synthesis that will form a necessary part of the research culture envisioned above, eye-tracking studies and reader questionnaires contribute to chapters written by linguists and psychologists on topics such as page layout, narrative cohesion, mental model construction, and bridging inferences (see Bateman et al., Tseng et al., Magliano et al., Loschky et al., Kirtley et al., Laubrock et al., this volume). 2 An emphasis on representative corpora that challenges the dominance of impressionistic case studies in comics research. While much work remains to be done before reference corpora are fully established and can be shared among researchers, this volume includes several chapters that document their construction and base their analysis on them (see Beaty et al., Dunst and Hartel, Walsh et al., this volume). 3 The integration of case studies, corpus analysis, and reception ­research into a triangulated framework. As several contributors note, one of the most promising areas of future growth lies in ­mobilizing elements of the research process that too often remain isolated. ­Creating feedback loops between them will allow for the f­ ormulation of new, and the testing of established, theories, the development of experimental methods, and the integration of disciplinary frameworks into a more fine-grained understanding of comics (see Cohn, Kirtley et al., Laubrock et al., this volume). 4 Finally, the emergence of a distinct vocabulary of empirical comics research is apparent throughout this volume in the repeated use of terms such as saliency, page composition, and low-level features. Because this vocabulary draws on the different disciplines that contribute to empirical approaches, their meaning may not be readily apparent, even to other researchers in the field of comics research. The glossary we have included in this book intends to address this issue. As editors, our belief in the transformative potential of empirical research is strengthened by a number of intersecting developments in contemporary academia. We have already mentioned the explosive growth

4  Alexander Dunst et al. of what has come to be called the digital humanities. The present volume contributes to this emerging field with a section that decisively expands computational work on comics: marking the first application of a method known as topic modeling to comics text, documenting the construction of the first representative corpus of US-American comic books, applying convolutional neural networks to comics, and, for the first time, proposing a visual stylometry for graphic novels. However, the emphasis on reception studies across these pages also moves this volume beyond a purely quantitative perspective. In recent years, concepts drawn from cognitive psychology have had a considerable impact on literary studies and linguistics. Within the mainstream of literary studies, including narratology, most of this influence has been limited to the importation of theoretical concepts. At times, this limitation has also characterized comics research (see, for instance, Kukkonen) but the wide readership enjoyed by the work of Neil Cohn has ensured an openness to experimental science (Visual Language). Further afield, the discipline of empirical aesthetics has enjoyed somewhat of a renaissance, albeit without taking much interest in comics so far (e.g., Shimamura and Palmer; Palmer et al.). Linguistics, on the contrary, has successfully expanded into media analysis, leaving behind an earlier emphasis on verbal data for a focus on multimodal resources. The area of multimodal linguistics, or multimodal semiotics in particular, offers frameworks for the study of comics that focus on the patterns of meaning-making, the interplay of text and images, and the specific affordances of the medium (e.g., Bateman et al., Multimodality). ­Working closely with cognitive science, significant advances have been made in the computational modeling and processing of natural languages. Once we add the analysis of digital data and corpora, linguistics provides an important foundation for interdisciplinary empirical research. Of particular value here are the ways in which linguistics offers levels of abstraction that enable researchers to go beyond theoretical and methodological discussions. These intersections bring Raymond Williams’ aim to unite the humanities and experimental science, quoted at the start of this introduction, a step closer to fruition. First expressed during the height of poststructuralism’s influence on the humanities, the developments we have sketched here suggest that Williams’ ambition is much more in step with today’s academic landscape than it was forty years ago. The digital humanities, the growing awareness of the cognitive dimensions of story­ irection telling, and the shift towards media specificity all point in this d and enable more fluid exchange between disciplines. A literary historian and ­theorist by trade, Williams imagined such a project as an “active collaboration” that would investigate “the overlap of the ­biological and the social in the acquisition and deployment of ‘ways of seeing’” (Williams 341; Prendergast 43). Coming of age at a time of large-scale

Comics and Empirical Research  5 interdisciplinary efforts during World War II, Williams and the physicist (and later historian of science) Thomas Kuhn were acutely aware of the potential benefits of these border crossings and the risks of disciplinary detachment. The seemingly straightforward distinction between the sciences and the humanities, of course, masks a more complex reality. The distance between these two terms in Anglophone countries is less apparent in languages such as Russian, German, and Italian, which all extend the moniker science to the study of language, literature, and culture—­ variously speaking of Literaturwissenschaft or scienza della letteratura (Bod et al. 4). And while the division between the human and the natural sciences goes back to Giambattista Vico’s Scienza Nuova, published as early as 1726, it was canonized in the late nineteenth-century philosophy of Wilhelm Dilthey. In fact, the basis for Dilthey’s dichotomy proves informative precisely because recent developments so thoroughly call it into question. For Dilthey, the sciences and the humanities could be distinguished according to their methods and their objects of study. Where the former explain the world as countable and measurable regularities, the latter seek to understand the expressions of the human mind (Bod 3; Dilthey). The digital turn has severely undermined this distinction, and it might, in time, vanish altogether. Very soon, most expressions of the mind will likely become countable, their regularities measurable, and scholars will become capable of explanation and interpretation. Kuhn’s anticipation of the unity of science and the arts regains new urgency in this context, as does Williams’ call for active collaboration—a convergence that will prove all the more fruitful if its participants can construct an equitable engagement. As several commentators have noted, the digital humanities have mostly explored the practical uses of computer science to date (e.g., Hall 782). In practice, this relationship often assumes the character of a direct importation of tools and methods that delegates computer scientists to the status of mere technicians. In turn, the sciences are frequently content to see the humanities as providers of complex data and a smattering of cultural expertise. These academic one-way streets, mirror images of one another, impoverish all participants. Hall’s argument that we should not conflate the computational turn in the humanities with the current state of scholarship may inspire us to imagine a different or, as he calls it, “postdigital humanities” (782). According to Hall, these postdigital humanities would explore the “irresolvable yet productive tension” between the human and natural sciences and prove “capable of generating new findings, insights, and realizations in the other—to the point where both of their identities are brought into question” (802). Only time will tell whether such a convergence can be forged. However, “the benefits of seeing science and art as one,” to cite Kuhn’s words once again, are already becoming visible (405).

6  Alexander Dunst et al. Somewhat counterintuitively, the most interesting path may not lie in asking what benefits the sciences bring to the humanities. After all, this direction is being taken already by the current upsurge in quantitative and empirical humanities research—with the visible benefit of a set of increasingly unified methodologies, described below. Inspired by this pragmatic advantage, the humanities are undergoing a period of intense self-examination. As James English and Ted Underwood write from their own standpoint in English ­Studies, “We are experiencing a much more concerted effort than occurred in the late twentieth century actually to recalibrate the entire analytic apparatus of literary study” (282). Their diagnosis may be extended to other humanities disciplines: Difficult questions are being asked about the basic assumptions of humanistic study, and will continue to be asked over the next few years. What are the scales of analysis at which cultural research may best be conducted, and how can they be connected? What does the digitization of objects of study mean for the theoretical frameworks of the humanities, both in their philosophical ontologies and in their research epistemologies? How can the dependence on digitality be squared with the necessary return to analog sources? How can new standards of evidence be constructed that simultaneously satisfy humanistic self-reflexivity and a scientific empiricism? Or, in other words, how do we combine explanation with interpretation? To reverse the direction, to ask what new findings, insights, and realizations the humanities can provoke in the sciences means to go against the grain of the digital humanities as they are practiced today. This reversal would return to the sciences the questions that digital humanities asks itself, and challenge the basic tenets of their research culture. In The ­S tructure of Scientific Revolutions, Kuhn describes the every-day business of “normal science” as one that purposefully eschews the “deep debates over legitimate methods, problems, and standards of solution” that characterize the paradigm shifts of the book’s title (48). One of the primary benefits that the humanities may bring to a converging ­research program consists in a closer entwining of method and reflection, of e­ xplanation and critique. Williams had written of another entanglement, between the biological and the social, as the site of this “active collaboration” (341). The ways of seeing that arise from this ­i ntersection lie at the center of the empirical study of comics, a medium that c­ ombines reading and viewing in ways we are ­ umanistic reflection can thus correct a only beginning to understand. H tendency to simplify the meeting point of biology, society, and culture. This constant triangulation of biology, society, and culture seems fundamental to empirical research in the humanities. As much as works of art emerge from their historical and political context, cognitive processes do not operate in isolation, but are always situated. To give an

Comics and Empirical Research  7 example that applies specifically to comics research: There is a clear need for analytical guidelines that enable individual case studies to contribute to systematic knowledge and subsequently encourage empirical testing and computing. A broader understanding of visual artifacts, their internal structures and contexts, can guide further experimentation and testing on their reception and interpretation. As a consequence, here as elsewhere, empirical research demands constant self-reflection from its practitioners, who must remain aware of their own ideological preconceptions to move forward in their research program. However, such reflection often remains excluded from the everyday activity of empirical disciplines. To combine these two might provide a starting point in the effort to transcend the division instituted by Dilthey and to approach the vision expressed by Kuhn, of science and art as one. In its focus on ­comics—a popular form that lies at the intersection of art history, literary studies, and empirical aesthetics but escapes their disciplinary boundaries—this volume hopes to mark one beginning of this convergence. The present collection aims to do so by providing insight into three emerging areas of study, which categorize the contributions in this book. We will elaborate on this categorization in the remainder of the introduction.

2.  Digital Approaches to Comics Research Computational methods represent a relatively recent addition to comics research, and it’s fair to say that their potential is not yet recognized by the majority of scholars. Nonetheless, digital approaches to comics already demonstrate a remarkable range across disciplines. At a basic level, any form of empirical research today depends on computation—for the manipulation of experimental material, for implementing and documenting surveys or experiments, and, most importantly, for subsequent statistical analysis. Fields such as linguistics, sociology, and cognitive science have already gone through their respective ‘digital turns’ and, as a consequence, may offer significant expertise to humanists interested in empirical work. For most humanities scholars, in contrast, computation first enters their professional lives in the form of digitization. Given that comics were historically disseminated in print, their retroactive digitization constitutes an essential undertaking that enables further research but also raises important questions in its own right, including documentation standards and copyright law. These will continue to prove important considerations for empirical comics researchers for the foreseeable future. However, digitization constitutes only one starting point if we want to apply computation to comics. Scholars such as John Walsh, Christophe Rigaud, and John Bateman have laid important foundations in recent years by formally describing comics in ways that make them amenable

8  Alexander Dunst et al. ­ achineto computational analysis. Walsh, for instance, has adapted the m and human-readable markup language XML for many of the structural elements found in comics, such as panels and speech balloons, and called it “Comic Book Markup Language,” or CBML for short. Aided by its conformity with the widely used TEI standard for text documents, CBML now forms an important basis of further research on comics, including work by Tufis and Ganascia, as well as Dunst and Hartel, in this volume. Another approach to automatic page segmentation is the ­eBDthèque corpus of comic book layouts (see Guérin et al.). More recently, Bateman and colleagues have furnished a sophisticated proposal for the annotation of comics layout (see their contribution to the second section of this volume). These authors build on their existing work in this book, moving to integrate new approaches and broadening the scope of digital scholarship. Annotation, which adds information to existing documents in a structured format such as XML, represents an essential method for describing the structure and content of comics. At the same time, manual annotation remains a time-consuming and frequently error-prone task that can also incur significant financial costs. Two main solutions present themselves: The first involves accessing outside support for annotation, either on a fee basis or by convincing members of the public to participate in this process. The motivations for participating can be diverse and include turning annotation into a ludic experience, a practice known as gamification, or accessing and learning about comics (Tufis and ­Ganascia, this volume). The alternative lies in the partial or full automation of manual processes. As a consequence, efforts are underway to automatically recognize the constituent elements of comics. In the case of panels, balloons, and captions, computer vision already achieves excellent results. The recognition of complex objects such as human figures or visual scenes still presents considerable challenges, as do quasi-handwritten fonts, which differ significantly between comics, or even within single titles (Rigaud and Burie, this volume, provide an overview). In all these cases, the increasing adoption of artificial neural networks—algorithmic structures that do not depend on prior instruction for each task but are capable of learning autonomously—holds enormous promise. However, neural networks also bring their own problems. On the one hand, they ­function best when fed large amounts of data. Quantity notwithstanding, ­opportunistic data collection frequently begs the question of how representative these comics are of the medium as a whole (Dunst et al.). On the other hand, the automated decision-making processes in neural ­ networks, which take place largely beyond our control, may introduce a different set of biases than human judgment. To an extent, neural networks thus raise the same ethical and political issues when applied to comics as they do in other areas of contemporary society.

Comics and Empirical Research  9 Digitization, modeling, annotation, and automatic recognition constitute four points of intersection between comics and computation. In “Two Per Cent of What? Constructing a Corpus of Typical American Comic Books,” Bart Beaty, Nick Sousanis, and Benjamin Woo engage at length with the aspects of digitization and modeling. ­Introducing the section on digital approaches, they also provide the first extensive account of their research project “What Were Comics?”. Perhaps even more than the study of other cultural forms, comics scholarship has usually relied on impressionistic samples that run the danger of seriously skewing research results. Beaty et al.’s work presents a convincing critique of such methods, which may be traced to the dominance of close reading in the assessment of the medium. More importantly, their project also provides a practical foundation for changing this status quo: Based on a sampling frame of all comic books published in the United States from 1934 to 2014, they are assembling a randomly selected corpus consisting of two per cent of these titles, or 3,563 books. Signaling a shift that is typical of DH research— from a focus on exceptional to typical works of art—they envisage a “genealogy of comics storytelling” that breaks with pre-established notions and rewrites its history through the use of data-driven scholarship. Corpus analysis promises to transform our understanding of comics. Alexander Dunst and Rita Hartel’s contribution, titled “The ­Quantitative Analysis of Comics: Towards a Visual Stylometry of Graphic ­Narrative,” reinforces Beaty’s central argument. Where Beaty details the construction of a corpus of US comic books but excludes what is commonly referred to as graphic novels, Dunst and Hartel analyze 209 book-length titles of this genre. Combining three low-level visual measurements, they automatically distinguish different subgenres, publication formats, and individual authors based on their stylistic features. Similar work has moved to the forefront of digital literary studies in recent years, but here it is performed for the first time with comics. The results shed a light on the characteristics of graphic narrative. As the authors show, graphic memoirs, which have been canonized with the help of critical successes such as Art Spiegelman’s Maus and Alison Bechdel’s Fun Home, can be distinguished from other subgenres by their relatively unvarying artistic style: on the whole, they are significantly brighter and internally more consistent. Their use of monochrome and visual regularity may thus be understood as approaching a literary aesthetic that distances itself from the popular tradition of earlier comic book narratives. In “The Spider’s Web: An Analysis of Fan Mail from Amazing Spider-­M an, 1963–1995,” John Walsh, Shawn Martin, and Jennifer St. ­Germain explicitly return to this popular tradition to illuminate one of its more underresearched aspects. Their careful study reveals comic book readers’ highly sophisticated engagement with the medium. Letter columns not only helped establish relationships between readers, creators, and publishers.

10  Alexander Dunst et al. Applying a method known as topic modeling to a corpus of fan letters, the authors argue that the most consistent theme running through a 33-year period concerns the “creation and craft of comic books”— an interest that readers demonstrated with almost scholarly attention to detail by citing evidence and advancing careful interpretations of their own. Throughout their chapter, Walsh et al. move between giving quantitative evidence and providing insight into individual letters. This mixed method combines hermeneutic approaches with the use of computational tools. “The Spider’s Web” thus forms a practical case study for combining applied computer science with humanistic scholarship and provides a counterargument to the simplistic opposition between close and distant reading. Communities of fans continue to be central to comics, whether on the letter pages of comic books or in the digital age. Walsh’s emphasis on the sophistication of fandom finds further support in Mihnea Tufis and Jean-Gabriel Ganascia’s exploration of crowdsourcing as a resource for comics scholarship, titled “Crowdsourcing Comics Annotations.” The chapter sets out by discussing the disadvantages of both manual annotation and automatic recognition. For the first, these include potential errors, the time-consuming nature of annotation, and the financial costs of paying trained annotators. In the meantime, machine-learning approaches that promise automatic recognition also lead to high error rates and ultimately demand retrospective annotation. Building on their practical experiences with the crowdsourcing platform Comics++, the authors explore how annotation may be turned into a citizen science. Where web services such as Amazon’s Mechanical Turk provide financial incentives, non-commercial projects such as Comics++ depend on the enthusiasm of fans and allow them to acquire new knowledge, or simply provide them with new reading material. Tufis and Ganascia’s chapter not only provides a practical guide for other scholars. Perhaps as importantly, it shows how empirical comics research can open itself to an interested public beyond the confines of the university. The final chapter in the section on digital approaches returns to the issue of automatic object recognition, offering a complementary vision to Tufis and Ganascia’s emphasis on annotation. Over the last few years, Christophe Rigaud and Jean-Christophe Burie have pioneered the application of computer vision to comics. In their contribution, they begin by offering an up-to-date survey of research in this area, including attempts to recognize panels, balloons, and text. Their chapter also tackles the recognition of human characters—one of the most challenging and potentially most rewarding endeavors for computer vision, which often struggles with the simplified and often non-perspectival drawings found in comics. The authors’ iterative scheme, which builds on cases of successful recognition and continually adds more contextual information, drives their holistic approach to computer vision. Their close attention to

Comics and Empirical Research  11 comics also emphasizes the benefits of combining humanistic knowledge and computational expertise toward constructing a better understanding of the medium for researchers across disciplines.

3.  Linguistic and Multimodal Approaches to Comics Research Linguistic interest in comics developed in the 1960s and ‘70s in ­order to examine combinations of verbal and pictorial material as components of meaning-making. However, multimodal analysis has only recently been recognized as relevant to the field of comics research. ­Multimodality is generally seen as a “way of characterizing communicative situations (considered very broadly) which rely upon combinations of different ‘forms’ of communication to be effective” (Bateman et al., ­Multimodality 7). These forms of communication can be described as semiotic resources, modes, or modalities, which contribute to the overall meaning of the m ­ edium (see overviews in Bateman et al., Multimodality; Jewitt). In these contexts, comics are defined as multimodal artifacts that contain closely interwoven combinations of pictorial materials and written text. These materials construct modules and entities (often on a two-­dimensional page) that have specific communicative tasks and fulfill narrative or educational functions, among others. A specific color, for instance, might have the function of identifying a character in one comic and constructing a particular time frame in another. In turn, a typical 4x4 table grid might create a contrast between settings or, on the contrary, show various views of one setting. The use of certain resources or modalities, as well as page composition and layout, therefore plays an important role in the narrative arc of a comic, and often brings about specific effects (such as in the gutter, the gap between panels) that may give rise to interpretation (see, e.g., Barnes; Goggin and Hassler-Forest; Postema; Cohn, Visual Language; Wildfeuer and Bateman). In order to examine the patterns and regularities formed by these functions and the meaning they construct in comics, it is crucial to inquire into the specific multimodal affordances of comics. In other words, we need to investigate the particular combination of modalities in comics. The general communicative properties of comics thus play a dominant role for the linguistic and multimodal study of the medium. Linguists usually describe these properties on various levels. For instance, as “very small units, i.e., individual identifiable visual elements contributing to a visual depiction, as well as larger units, such as regular geometric patterns of panels or other identifiable visual configurations spread over an entire page or spread” (Bateman and Wildfeuer 376–377). The focus in formulating precise descriptive methods often lies on the role of visual elements in contrast to verbal units, and therefore concerns the visual style (McCloud; Lefèvre) or specific visual sequentiality of

12  Alexander Dunst et al. the narrative (Eisner; McCloud). Even smaller details, such as the size of a panel, the design of pictorial runes (Forceville) or other stylistic units (Forceville et al.), as well as their function for the narrative, are also of interest (see Horstkotte; Lefèvre). For multimodality, the functions of elements in the meaning-making process and their patterns have become a central research question. How can they be analyzed, especially with regard to the interplay of all semiotic elements in a comic or graphic novel? This question marks a point of connection not only with literary, media, and cultural studies, but also and in particular with the digital humanities and their aim to make material under consideration available as research data. An important prerequisite for different kinds of data analysis is the systematic access to it in terms of levels of abstraction, which allows researchers to answer questions like the one mentioned above. Digitized verbal text, for instance, can only be analyzed if the abstract linguistic units of the text (such as words, letters, p ­ unctuation, etc.) are made available for computational processing. In the case of language-­ based data, corpus linguistics has developed a variety of methods to ­ rogressively describe digitized documents. Multimodal linguistics p ­follows in these footsteps in order to describe media artifacts of all kinds. ­However, there is still a considerable lack of systematic access to material. Much of the work on comics in multimodality so far has chosen a theoretical and methodological perspective on its units and structural qualities. Thus, scholars have inquired into the relations between visual and verbal elements (e.g., Cohn, “Beyond”; Wartenberg) or reflected upon the narrative or discursive structure of comics pages and their arrangement of panels (e.g., Bateman and Wildfeuer; Veloso and Bateman; Cohn, “Navigating”). These approaches often use small example analyses and, despite their comprehensive claims and potential for corpus-based research, do not always offer further empirical verification—Cohn’s work being a rare exception. Generally speaking, current efforts in linguistic and multimodal research still focus on the theoretical clarification of terminology, or the appropriate way of collecting and categorizing comics as data. Other work addresses the fine-grained analysis of text and discourse properties in order to capture the communicative function of comics. From this follows a need for more systematic research that extends beyond theoretical foundations and continues into precise empirical investigation and verification. These latter two areas of research are currently gaining increased ­attention, particularly with regard to the cognitive comprehension of comics and in combination with psychological approaches. Advances in this area focus on corpus-based description and the processing of visual and verbal elements as units on a page, including panels, speech bubbles, or layout (see Cohn, Visual Language and “Navigating;” Bateman et al.

Comics and Empirical Research  13 “Multilevel Classification”). The aim of these studies lies in showing how the structural elements of comics combine to support and guide interpretation. In doing so, this line of research synthesizes the description of basic elements and higher-level interpretation. Here, linguistic and multimodal analysis offers hypotheses that need to be tested empirically with the help of eye tracking, fMRI, or other cognitive methods. Yet, overall, interdisciplinary approaches that integrate theoretical ­discussion and empirical falsification remain rare. In contrast, the contributions to this section of our volume fashion a platform for approaching linguistic and multimodal research on comics empirically and connect grounded theory to exploratory experiments. John Bateman, Annika Beckmann, and Rocío Inés Varela link categorizing page design in comics and their role in narrative construction. In the first part of their chapter, they offer a method that systematically analyzes page composition in reference to its visual description. They then use this systematic categorization as a basis for eye-tracking studies to learn more about the behavioral properties that these compositions evoke during reception. In particular, the authors study the effect of layout manipulation in the case of what they term gridding and gapping. As they show, more ambiguous panel layouts lead to higher variability in readers’ fixation choices. A further hypothesis is that differences in compositional strategies might be reflected in recipients’ gaze behavior. However, the results of their study do not fully verify this hypothesis. With their chapter, the authors demonstrate how detailed analytical frameworks, in this case a multi-level annotation scheme, may work at the interface between the perceptual properties of an artifact and more abstract levels of description, such as narrative construction, in order to learn about their interrelationships. Chiao-I Tseng, Jochen Laubrock, and Jana Pflaeging provide e­ mpirical testing of the structural specificity of comics by focusing on character development in graphic novels. Building on the linguistic theory of cohesion, the authors describe the cohesive chains built via the multimodal representation of characters in two example comics. This systematic approach is then tested empirically with the help of an eye-tracking study to find out more about readers’ comprehension of characters. In manipulating several comics pages, the authors replace elements of the cohesive chain relating to the main character of David Mazzuchelli’s City of Glass with alternative background objects or structures. Their results show longer fixations on the original version, suggesting that readers selectively ­attend to elements that prove central for constructing and maintaining cohesive chains. In a further study, the authors employ the same linguistic method to compare cohesive event structures in the silent graphic novel Dead End. Results indicate that the developments of characters’ action types may substantially differ from conventional graphic novels when verbal cues are missing.

14  Alexander Dunst et al. Hans-Jürgen Bucher and Bettina Boy present a further eye-tracking study from a multimodal and discourse analytical perspective. In particular, they focus on the genre of information comics and their reception. The authors ask how information is conveyed via the specific features of comics, and whether these features follow a diegetic or non-diegetic strategy of knowledge transfer. The study combines results from eye tracking with two different forms of knowledge tests in order to examine the reception of two comics that address ecological issues. The authors compare the different strategies deployed by these comics with respect to their educational potential in promoting the public understanding of science and discuss how these texts present different comprehension challenges to readers. Their results also enable the authors to suggest strategies for the successful construction and design of information types in comics, for example in the form of inserts, whose use requires particular measures of coherence. Pascal Lefèvre and Gert Meesters are equally interested in the formal features of visual media, and how they guide interpretation. H ­ owever, they focus specifically on the dynamic process of an evolving line ­drawing. With the help of exploratory experiments, the authors study the perception and interpretation of several representations of the line drawing (in an animated video or various orders of stills taken from the video) and compare them to the reception of verbal elements. Building on the concepts of hysteresis and adaptation, they explain that an understanding of the dynamically emerging drawing is both based on prior knowledge about it and the new displays becoming visible to the recipients. Lefèvre and Meesters provide insight into visual aspects of comics that have not been addressed in any detail by researchers thus far: the artistic stages that take place before page composition, i.e. the process of drawing itself. The authors underscore that these details bring with them specific challenges for reception and interpretation that should be taken into consideration in future comics scholarship. In their exemplary analyses, the contributions to this section set the ­ ultimodality. stage for further empirical research in linguistics and m Most of the work presented in these chapters has an exploratory character and invites more detailed falsification of their results, as well as ­testing of their methodological approaches with larger data sets. Therefore, this section calls for additional interdisciplinary work that challenges the theoretical frameworks presented in these chapters and provides additional experiments.

4.  Cognitive and Psychological Approaches to Comics Research Like many humanities disciplines, cognitive and experimental psychology studies the human mind. However, the emphasis lies on the inner workings of the mind and its cognitive processes, rather than on cultural

Comics and Empirical Research  15 artifacts and the narratives they tell. Unlike much of the humanities, cognitive and experimental psychology has been firmly rooted in an empiricist tradition from its very beginning. At this point, we will briefly sketch the development of this field, which might have the potential to serve as a template for methodological unification in the (digital) humanities. Following in the footsteps of Hermann von Helmholtz’s physiology and Gustav Theodor Fechner’s psychophysics, the father of modern experimental psychology, Wilhelm Wundt, reasoned that little progress could be expected if the study of the mind remained in the realm of metaphysics. Observing that advances in the sciences had been due to the use of a common method, he had the insight that the scientific method could be applied to the study of the human mind. Wundt thus established the first experimental psychology laboratory. The scientific method requires that all hypotheses and theories must be tested against empirical observations rather than resting solely on a priori belief, argument, or intuition. The most successful method of inquiry in the sciences is the experiment, in which careful control over independent and extraneous variables and objective measurement are, ideally, used to establish a causal relationship between independent variables and measurements. However, experimental control is not always possible. For example, when studying aging cross-sectionally, the variable of interest cannot be experimentally manipulated, and resorting to quasi-experimental techniques is required, with age groups that are matched on a number of potentially confounding variables. Similar concerns arise when comparing linguistic material. Following Wundt’s tradition, cognitive psychologists have based their reasoning on data collected through systematic observation or experiments that are, in principle, open to replication and falsification by peers. Despite the recent discussion about a replicability crisis (Button et al.), most core findings on which theories in cognitive psychology rest have been replicated many times. The success of this research program is fundamentally tied to the development of a set of methods, which can be considered a methodological canon. As Lee Cronbach puts it, “our methods of inquiry have become increasingly stable, and it is these methods which qualify us as scientists rather than philosophers or artists” (“Two Disciplines” 671). In an essay titled “The Two Disciplines of Scientific Psychology,” Cronbach noted that a schism similar to the one between the sciences and the humanities (if on a smaller scale) appeared in the field of psychology during the mid-twentieth century: One stream is experimental psychology; the other, correlational psychology. […] Psychology continues to this day to be limited by the dedication of its investigators to one or the other method of inquiry rather than to scientific psychology as a whole. […] The well-known virtue of the experimental method is that it brings situational

16  Alexander Dunst et al. variables under tight control. It thus permits rigorous tests of hypotheses and confident statements about causation. The correlational method, for its part, can study what man has not learned to control or can never hope to control. (671) Even though both Neil Cohn and Lester Loschky et al. (this volume) provide examples of tight experimental control in the study of comics, such control proves difficult with much of the material that the humanities are concerned with, which makes the addition of correlational methods unavoidable. Cronbach called for a reunification of the two camps, apparently with some success. Revisiting the topic in 1975, he diagnosed that a “hybrid discipline is now flourishing,” though the debate over the epistemological value of correlational techniques continues (Kliegl et al.; Rayner et al.; Kliegl). Be that as it may, there is now at least some communication between the two approaches, and the acknowledgment that statistical control of extraneous variables should be attempted where experimental control is not feasible. Statistical control proves particularly crucial in the study of complex stimuli such as visual scenes, linguistic material, or works of art. The ideal of experimental control has led some researchers to shy away from such stimuli, which, from the viewpoint of the humanities, must feel like throwing out the baby with the bathwater. Isolated control of a single feature may not be possible in studies such as Joe Magliano’s contribution to this volume, which focuses on adapted narratives and compares the reception of comics and film. When switching presentation format, a number of covariates by necessity also change. Even less experimental control is available when different sets of materials are included with the goal to generalize, as in corpus studies. Such corpus research therefore relies on the statistical control of potentially confounding variables. Regression modeling techniques are the dominant method in these cases, and the present volume includes a number of chapters in which they are applied (Laubrock et al.; Kirtley et al.; Tseng et al.; Bateman et al.). What is the value of this exposition to the humanities? One hope is that the progress brought about in experimental psychology by the relatively early agreement on a common method could serve as a model for those areas of the humanities that deal with empirical data. A field like the digital humanities, where immense data sets are acquired for empirical study, may then be able to help to unify the fragmented methodology apparent in other humanities disciplines. Openness to empirical study implies openness to replicability and falsification. Even though such a change might come close to a Kuhnian scientific revolution, in the long run it will require a willingness to add statistics to the methodological canon, given that data only really speak with the use of inference statistics. Without them, any signal or pattern observed in the data may

Comics and Empirical Research  17 just be noise. Methods within the general linear model framework, such as analysis of variance, regression, and particularly its modern variants like (generalized) linear mixed models, could prove well-suited analytical frameworks for the humanities. Whereas the previous sections mainly dealt with aspects of the material, the present section focuses more on the cognitive processes of the comics reader. Two chapters include broad theoretical frameworks: the first originates from the area of attention, perception, and cognition ­(Loschky et al.), the second from cognition and psycholinguistics (Cohn). Loschky et al. distinguish between lower-level front-end and higher-level back-end processes. The remaining chapters are arranged in terms of this taxonomy. Two studies describe corpora of eye movement recordings but with a different unit of analysis, leading to a focus on front-end processes (Laubrock et al.) or mid-level processes (Kirtley et al.). The experimental study by Magliano et al. deals with the higher-level processes of mental model construction. Lester Loschky, John Hutson, Maverick Smith, Tim J. Smith, and ­Joseph P. Magliano lay out the Scene Perception & Event Comprehension Theory (SPECT). SPECT is an integrative framework that synthesizes a number of theories from the areas of scene perception, event perception, and narrative comprehension. SPECT distinguishes between front-end mechanisms that involve information extraction and attentional selection during single-eye fixations, and back-end mechanisms that involve creating the event models (i.e., one’s current understanding) across multiple fixations in working memory and storing them in long-term memory. The chief back-end mechanisms are: laying the foundation for the event model, mapping incoming information to it, and shifting to create a new event model. These back-end mechanisms were originally proposed for text comprehension. Focusing on the visual content of graphic narratives, Loschky et al. test whether the mechanisms will generalize to ­visual narratives. Using empirical data collected with the ‘Boy, Dog, Frog’ (BDF) wordless picture stories (e.g., Mayer) they find evidence from event segmentation and bridging inference generation data for such ­generalizability of the back-end mechanisms. SPECT is also used to generate novel hypotheses about the bidirectional interactions between front-end and back-end processes. These were tested using eye movement recordings. Observations include changes to eye movements due to ­ ridging (1) laying the foundation for the event model, and (2) generating b inferences while mapping incoming information to the event model. In their chapter “Attention to Comics,” Jochen Laubrock, Sven Hohenstein, and Matthias Kümmerer provide the foundation for a description ­ otsdam of attentional processing during comics reading. Introducing the P Comics Corpus-1, a corpus of eye movement recordings consisting of almost one million fixations intended as a research tool for cognitive science and the humanities, they offer a complementary readers’ perspective to the

18  Alexander Dunst et al. material-oriented corpus research presented in earlier chapters. Laubrock et al. analyze these data using linear mixed modeling and convolutional neural networks. Focusing on front-end processes in the terms laid out by SPECT, they find evidence for a large number of well-orchestrated cognitive processes that operate in parallel. The panel of a comic page, for instance, acts as a visual scene for the reader: Quick orientation in this scene is followed by later inspection of interesting detail. Text processing dominates, with most refixations devoted to text, and a word-frequency effect showing that difficult passages slow the reader down. Preview of upcoming fixation locations is used both between and within panels. Between panels, peripheral and parafoveal vision select informative elements such as characters or speech bubbles as saccade targets. The panel background remained relatively unattended by readers during the first pass, suggesting that they are initially content just to understand the gist of the visual scene. Within the panel, preview was concentrated on text and characters and was reduced when difficult passages were encountered or when the context was switched. The distribution of fixation locations is well approximated by Deep Gaze II, a computational model of saliency based on a deep convolutional neural network. Taken together, reading comics involves a complex interplay of several cognitive processes. Many of these have been identified in the isolated study of reading or scene perception, but the entwining of text and images in comics adds an exciting new twist. Similar to Laubrock et al., Clare Kirtley, Christopher Murray, ­Phillip B. Vaughan, and Benjamin W. Tatler inspect eye movements during comics reading using a linear mixed modeling framework. However, by using aggregate measures such as average dwell time and regression probability, they move up a step towards back-end processes in terms of Loschky and colleagues’ SPECT framework. Kirtley et al. found verbal text to be the most consistently influential element of comics on reading behavior. Presence and quantity of text affected whether readers fixated a panel, whether they read it in order, and how much time readers spent on the panel. Regression goal probability was increased for panels containing words, suggesting that revisiting text areas helped resolve comprehension problems. Parafoveal preview of the upcoming panel affected several measures: First, fixations in a panel were mostly on regions containing words, suggesting that they can be identified from the preceding panel. Skipping of panels was strongly reduced when the panel contained words. A focus of the panel on a character also reduced skipping, suggesting that both text and characters can be identified in parafoveal vision while fixating the preceding panel. Readers may be able to obtain the gist of the scene in parafoveal vision and decide to skip detailed processing of the less informative panels without text or a character focus, at least in first-pass viewing. Again, these results nicely show the interplay of text reading and image processing during comics reading and the selectivity of attention to informative regions.

Comics and Empirical Research  19 Attention, as well as memory encoding and retrieval, are prerequisites for the higher-level processes of mental model construction studied by Joseph Magliano, James Clinton, Edward O’Brien, and David Rapp. Several recent films have been adapted from comics and graphic novels, leading to the question of how different adaptation formats affect reception. Magliano et al. studied the cognitive processing of adapted narratives in the theoretical context of how memory retrieval supports mental model construction. Their participants both read Frank Miller and David Mazzuchelli’s graphic novel Batman: Year One and watched the animated film by Liu et al., writing down any differences they noticed during the second presentation. The order of presentation was then counterbalanced. Whether a change was noticed differed between types of events, with narrative events and verbal content being more prone to change detection than setting or character discrepancies. Viewers were more likely to notice changes to narrative events when they watched the film second, and vice versa for changes to verbal content. The former finding is probably related to guidance of attention and the film director’s narrative understanding, and the latter suggests that retrieval cues for verbal content are stronger when reading than when watching. ­Additions were generally detected better than omissions, and particularly so when reading came second, compatible with results from the episodic memory literature. This exploratory work shows that adapted narratives have great potential in the study of cognitive processes related to memory encoding and retrieval, as well as mental model construction. Neil Cohn’s work has been influential for a number of years. His ­Visual Language Theory (VLT) approaches the visual language of comics as a cognitive system. In his review chapter, he introduces VLT as an approach for studying the structure of sequential images, analogous to the structure of verbal language. By taking a broader perspective, Cohn not only includes different national comics traditions but also considers sequential images ranging from cave paintings to instruction manuals. Cohn argues that visual languages are structured and processed in similar ways to other linguistic forms. The defining features of a language are meaning, modality, and grammar. Meaning is mapped to sound in a verbal language, and to graphic structure in a visual language, which in both cases creates a lexicon. Grammar organizes images into sequences. This argument implies that the rich toolbox developed by linguistics can be applied to the study of comics. It also implies that comics are an interesting subject for cognitive studies because they tell us something about a specific form of cognitive processing. Cohn’s chapter gives an overview of the vocabulary, and other basic structures, of visual language. Cohn covers the nature of visual lexical items, the narrative grammar of sequential images, and the compositional structure of page layouts, summarizing the theoretical and analytical modeling opportunities and insights that VLT provides. With examples drawn from corpus and

20  Alexander Dunst et al. experimental literature, and using both behavioral and neurocognitive measures, as well as their discussion from a cross-cultural perspective, Cohn argues for a scientific study of comics grounded in theoretical predictions of how graphic information communicates.

5.  A Novel Approach to the Study of Comics As our summaries of the contributions show, this volume brings together theoretical and methodological innovations for empirical accounts of comics that are sensitive to its specificities. Therefore, this collection aims to function as an introductory field guide for researchers working in the field of comics studies and, more generally, for empirical researchers in the humanities, as well as for researchers in cognitive studies interested in topics from the humanities. Aided by the toolsets and frameworks that our authors provide, we hope that readers will continue to move towards the intersection of cultural studies, linguistics, the digital humanities, and cognitive psychology. The workshop that formed the foundation for this volume reflected the excitement of truly interdisciplinary exchange in a field that still remains relatively unexplored. However, during the editing process it became clear that it is not always easy to integrate approaches that originate from different academic cultures. Two examples are the citation format and the use of statistics. A slightly adapted version of the MLA citation format is used here as a courtesy to readers from the humanities, where it is often seen as a standard. Scientists who are used to communicating empirical research will regard some of its particularities as impractical for their own concerns. These readers shall rest assured that all citations map to a unique entry in the list of references, even though this mapping might not be as easy to remember as in the beloved author-year format. The second difference lies in the use of statistics. Readers from psychology and linguistics have grown accustomed to reading numbers interspersed with text due to the reporting format required by most journals, as formalized in guidelines (such as the APA Publication Manual) that are published by learned societies. However, even these readers may sometimes feel that results sections in academic journals can be hard to read, and this is exactly the feedback that readers from the humanities gave us in response to earlier drafts. We therefore decided to delegate the reporting of statistics to tables or figures where possible. Nonetheless, the statistical analysis contained in some chapters may prove challenging for humanists. In contrast, cognitive scientists may feel that some of the arguments made in other chapters lack a bit of statistical support. As we continue to build new bridges between humanistic and empirical disciplines, these paths will begin to feel shorter and the benefits of seeing art and science together hopefully will stand out more clearly with every new attempt.

Comics and Empirical Research  21

Works Cited American Psychological Association. Publication Manual of the American ­Psychological Association. 6th ed., American Psychological Association, 2010. Barnes, David. “Time in the Gutter: Temporal Structures in Watchmen.” KronoScope, vol. 9, 2009, pp. 51–60. Bateman, John A., Francisco O. D. Veloso, Janina Wildfeuer, Felix Cheung, and Nancy S. Guo. “An Open Multilevel Classification Scheme for the Visual Layout of Comics and Graphic Novels: Motivation and Design.” Journal of Digital Scholarship in the Humanities, vol. 32, no. 3, 2017, pp. 476–510. Oxford Academic, doi:10.1093/llc/fqw024 Bateman, John A., and Janina Wildfeuer. “A Multimodal Discourse Theory of Visual Narrative.” Journal of Pragmatics, vol. 74, 2014, pp. 180–218. Bateman, John A., Janina Wildfeuer, and Tuomo Hiippala. Multimodality. Foundations, Research and Analysis. A Problem-Oriented Introduction. De Gruyter, 2017. Bod, Rens Julia Krusell, Japp Maat, and Thijs Weststeijn. “A New Field: ­H istory of Humanities.” History of Humanities vol. 1, no. 1, 2016, pp. 1–8. Button, Katherine S., John P. A. Ioannidis, Claire Mokrysz, Brian A. Nosek, ­Jonathan Flint, Emma S. J. Robinson, and Marcus R. Munafò. “Power ­Failure: Why Small Sample Size Undermines the Reliability of ­Neuroscience.” Nature Reviews Neuroscience, vol. 14, 2013, pp. 365–376. Nature, doi:10.1038/ nrn3475. Cohn, Neil. “Beyond Speech Balloons and Thought Bubbles: The Integration of Text and Image.” Semiotica, vol. 197, 2013, pp. 35–63. ———. “Navigating Comics: An Empirical and Theoretical Approach to ­Strategies of Reading Comic Page Layouts.” Frontiers in Psychology, vol. 4, no. 186, 2013, pp. 1–15. ———. The Visual Language of Comics: Introduction to the Structure and Cognition of Sequential Images. Bloomsbury, 2013. Cronbach, Lee J. “Beyond the Two Disciplines of Scientific Psychology.” ­American Psychologist, vol. 30, no. 2, 1975, pp. 116–127. ———. “The Two Disciplines of Scientific Psychology.” American Psychologist, vol. 12, no. 11, 1957, pp. 671–684. Dilthey, Wilhelm. Introduction to the Human Sciences. Edited by Rudolf A. Makkreel, and Frithjof Rodi, Princeton UP, 1989. ­ arrative Dunst, Alexander, Rita Hartel, and Jochen Laubrock. “The Graphic N Corpus: Design, Annotation, and Analysis for the Digital Humanities.” ­Proceedings of the 14th IAPR International Conference on Document ­Analysis and Recognition (ICDAR 2017), 13–15 Nov., Kyoto, CPS, 2017, pp. 15–20. Eisner, Will. Comics and Sequential Art. Kitchen Sink P, 1992. English, James F., and Ted Underwood. “Shifting Scales: Between Literature and Social Science.” Modern Language Quarterly, vol. 77, no. 3, 2016, pp. 277–295. Forceville, Charles. “Pictorial Runes in ‘Tintin and the Picaros’.” Journal of Pragmatics, vol. 43, no. 3, 2011, pp. 875–890. Forceville, Charles, Elisabeth El Rafaie, and Gert Meesters. “Stylistics and Comics.” The Routledge Handbook of Stylistics, edited by Michael Burke, Routledge, 2014, pp. 485–499.

22  Alexander Dunst et al. Goggin, Joyce, and Dan Hassler-Forest. “Out of the Gutter, Reading Comics and Graphic Novels.” The Rise and Reason of Comics and Graphic Literature, edited by Joyce Goggin, and Dan Hassler-Forest, McFarland, 2010, pp. 5–24. Guérin, Clement, Christophe Rigaud, Antoine Mercier, Farid Ammar-­Boudjelal, Karell Bertet, Alain Bouju,… and Arnaud Revel. “eBDthèque: A ­Representative Database of Comics.” Proceedings. 12th International C ­ onference on Document Analysis and Recognition ICDAR 2013, 25–28 Aug., ­Washington, 2013, pp. 1177–1181. Hall, Gary. “Toward a Postdigital Humanities: Cultural Analytics and the Computational Turn to Data-Driven Scholarship.” American Literature, vol. 85, no. 4, 2013, pp. 781–809. Horstkotte, Silke. “Zooming In and Out: Panels, Frames, Sequences and the Building of Graphic Storyworlds.” From Comic Strips to Graphic Novels: Contributions to the Theory and History of Graphic Narrative, edited by Daniel Stein and Jan-Noël Thon, de Gruyter, 2013, pp. 27–48. Jewitt, Carey, editor. The Routledge Handbook of Multimodal Analysis. 2nd ed., Routledge, 2014. Kliegl, Reinhold. “Toward a Perceptual-Span Theory of Distributed P ­ rocessing in Reading: A Reply to Rayner, Pollatsek, Drieghe, Slattery, and Reichle (2007).” Journal of Experimental Psychology: General, vol. 136, no. 3, 2007, pp. 530–537. PsycNet, doi:10.1037/0096-3445.136.3.530. Kliegl, Reinhold, Antje Nuthmann, and Ralf Engbert. “Tracking the Mind During Reading: The Influence of Past, Present, and Future Words on ­Fixation Durations.” Journal of Experimental Psychology: General, vol. 135, 2006, pp. 12–35. Kuhn, Thomas S. “Comment.” Comparative Studies in Society and History, vol. 11, no. 4, 1969, pp. 403–412. Kuhn, Thomas S. The Structure of Scientific Revolutions. 2nd ed., U of Chicago P, 1970. Kukkonen, Karin. Contemporary Comics Storytelling. U of Nebraska P, 2013. Lefèvre, Pascal. “No Content without Form: Graphic Styles as the Primary Entrance to a Story.” The Visual Narrative Reader, edited by Neil Cohn, Bloomsbury, 2016, pp. 67–88. Li, Luyuan, Yongtao Wang, Zhi Tang, and Liangcai Gao. “Automatic Comic Page Segmentation based on Polygon Detection.” Multimedia Tools and ­Applications, vol. 69, no. 1, 2014, pp. 171–197. Liu, Sam, and Lauren Montgomery. Batman: Year One. Warner Bros, 2011. Mayer, Mercer. A Boy, a Dog, and a Frog. Dial Books, 1967. McCloud, Scott. Understanding Comics. The Invisible Art. Harper Perennial, 1994. Miller, Frank. Batman: Year One. Illustrated by David Mazzuchelli, DC ­Comics, 2007. Palmer, Stephen E., Karen B. Schloss, and Jonathan Sammartino. “Visual ­Aesthetics and Human Preference.” Annual Review of Psychology, vol. 64, no. 1, pp. 77–107. Pastierovic, Robert. “Advanced Comic Book Format.” Canonical, 2004–18, launchpad.net/acbf. Accessed 17 Jan. 2018. Piper, Andrew. “There Will Be Numbers.” Journal of Cultural Analytics, 23 May 2016, culturalanalytics.org/2016/05/there-will-be-numbers. Accessed 24 Oct. 2016.

Comics and Empirical Research  23 Postema, Barbara. Narrative Structures of Comics: Making Sense of Fragments. RIT P, 2013. Prendergast, Christopher. “Evolution and Literary History: A Response to Franco Moretti.” New Left Review, vol. 34, 2005, pp. 40–62. Rayner, Keith, Alexander Pollatsek, Denis Drieghe, Tim J. Slattery, and Erik D. Reichle. “Tracking the Mind during Reading Via Eye Movements: C ­ omments on Kliegl, Nuthmann, and Engbert (2006).” Journal of E ­ xperimental Psychology: General, vol. 136, no. 3, 2007, pp. 520–529. PsycNet, doi: 10.1037/0096-3445.136.3.520. Schöch, Christoph. “Big? Smart? Clean? Messy? Data in the Humanities.” ­Journal of Digital Humanities, vol. 2, no. 3, 2013, pp. 2–13. Shinamura, Arthur P., and Stephen E. Palmer, editors. Aesthetic Science: ­C onnecting Minds, Brains, and Experience. Oxford UP, 2012. Stommel, Martin, Lena I. Merhej, and Marion G. Müller. “Segmentation-Free Detection of Comic Panels.” Computer Vision and Graphics: International Conference ICCVG 2012, edited by Leonard Bolc et al., 24–26 Sep., ­Warsaw, Springer, 2012, pp. 633–640. Veloso, Francisco O. D., and John A. Bateman. “The Multimodal Construction of Acceptability: Marvel’s Civil War Comic Books and the PATRIOT Act.” Critical Discourse Studies, vol. 10, no. 4, 2013, pp. 427–443. Walsh, John A. “Comic Book Markup Language: An Introduction and R ­ ationale.” Digital Humanities Quarterly, vol. 6, no. 1, 2012, www.­digitalhumanities. org/dhq/vol/6/1/000117/000117.html. Accessed 13 Apr. 2018. Wartenberg, Thomas E. “Wordy Pictures: Theorizing the Relationship between Image and Text in Comics.” The Art of Comics: A Philosophical Approach, edited by Aaron Meskin, and Roy T. Cook, Blackwell Publishing, 2012, pp. 87–104. Wildfeuer, Janina, and John A. Bateman. “Zwischen Gutter und Closure. Zur Interpretation der Leerstelle im Comic durch Inferenzen und dynamische ­Diskursinterpretation.” Closure: Kieler e-Journal für Comicforschung, vol. 1, 2014, pp. 3–24. Williams, Raymond, Politics and Letters: Interviews with New Left Review, NLB, 1979. Wundt, Wilhelm. Beiträge zur Theorie der Sinneswahrnehmung. Winter, 1862.

Part I

Digital Approaches to Comics Research

2 Two Per Cent of What? Constructing a Corpus of Typical American Comic Books Bart Beaty, Nick Sousanis, and Benjamin Woo 1.  Introduction Writing in The Field of Cultural Production, French sociologist Pierre Bourdieu argues, “one of the most significant effects of the transformations undergone by the different genres is the transformation of their transformation-time” (Bourdieu 52). For Bourdieu, the model of ‘permanent revolution,’ by which he means the constant struggle, often generational, by actors within creative fields to define and redefine the principles by which the field operates has increasingly come to define the arts generally (“Field of Cultural Production” 52). Here, Bourdieu suggests that the velocity of innovation within cultural fields is variable. Although it is asserted more strongly than it is demonstrated in his writing, Bourdieu’s notion of a ‘transformation of transformation-time’ fruitfully points to an understanding of cultural change that seems both commonsensical and highly elusive. In the field of comic books, it is almost intuitively logical to suggest that there are stylistic, narrative, and generic conventions that are more closely tied to historical periodization than to the particularities of individual creators, titles, or publishers. That is to say, certain comic books have the appearance of products from the 1940s or 1990s, even though we are not able to state with any precision what the normative characteristics of those eras might be. We can perhaps “feel” the fact of diachronic change better than we can precisely define it. Moreover, we may believe that rates of change within comics have accelerated over time without being able to define the parameters of that change. The What Were Comics? project is a large-scale data-driven study of comic books in the United States. The project seeks to come to terms with processes of change within the American comic book industry. In this chapter, we outline the foundational first step in our larger program of work that seeks to reorient the study of comics (comic books, comic strips, and graphic novels) through the use of data-driven research. What Were Comics? has three specific, interrelated goals: First, we have produced a sampling frame of comic books published in the United States between 1934 and 2014, as well as a randomly selected corpus

28  Bart Beaty et al. of 2% of that total, stratified by year; second, we have developed a coding protocol for translating those works into a series of sixty-one variables and created an open-access software tool for the study of comic books; third, we will draw upon the data produced by this tool to examine the history of the American comic book through a new lens. By dramatically expanding our frame of reference from what has been normative in humanities-­influenced studies of comic books, we have shifted the study of comics away from the study of exceptional works and towards a focus on works that typified cultural production over time. This perspectival shift in method will produce new theories of the comic book as we facilitate a move from asking the abstract question of ‘what are comics?’ to the empirically grounded question of ‘what were comics?’ throughout the entire history of the popular publishing format. Today, the comic book industry faces a period of profound institutional, technical, and aesthetic transformation. Both the ‘popular’ and ‘literary’ poles of the field are benefiting from increased public attention and esteem, attracting new audiences with tastes and expectations that differ from those of long-established fan subcultures. Meanwhile, digital distribution of comics to computers and mobile devices promises to disrupt existing print-centric business models, even as cartoonists explore the affordances of new technologies (e.g., animation or scrolling along an ‘infinite canvas’, crowd-funding) to challenge many of our assumptions about what comics could be. It is our contention that the ability to properly understand and evaluate these rapid changes is hampered by the distorted picture of the American comic book that has been constructed over generations. In order to think about the future of comics, it is necessary that we first ask what comics were. For example, it is widely argued that the history of the American comic book can be chronicled as a series of ‘ages’ of comics production, that the introduction of the Comics Code (a regime of industrial self-censorship developed in the mid-1950s) resulted in a ‘dumbing down’ of the aesthetically and politically progressive potential apparent in comics of the 1940s, and that contemporary comic books are radically different than their forebears in stylistic terms (for instance, offering fewer panels per page and less text per panel than they once did). It is important to note that each of these (and other) claims is merely impressionistic. To date, scholars have had no way of testing and validating them empirically. The rapid development of comics studies over the past two decades has taken place overwhelmingly in languages and literature departments. As with other scholarly subfields that emerged within humanistic research traditions, comics studies has consistently struggled with the problem of the typical. Key studies in comic book aesthetics, narratology, history, and culture depend primarily or exclusively on the study of a canon of ‘exceptional’ texts that are regularly enlisted to stand in for comics per se.

Two Per Cent of What?  29 To take but one obvious example, the Bonner Online-­Bibliographie zur Comicforschung lists 243 scholarly contributions about Art ­Spiegelman’s autobiographical comics masterpiece Maus, but only two on the more than seventy years of comic books published by Archie Comics, a company that was at one time the top-selling comic book publisher in the United States (Beaty and Woo). While, of course, much can be learned from close readings of the ‘best’ comics, these atypical works provide an unreliable basis for generalizing about the form in any meaningful way. We have addressed this problem by constructing a randomly selected corpus of comic books published in the United States over a period of eight decades. By re-inserting what Margaret Cohen has termed “the great unread”—the forgotten works that make up the vast bulk of cultural output—into the study of comic books, we will be able to more accurately mark the shifts in genre, composition, and materiality that have characterized this art form over time (Sentimental Education 23). As Woo has conclusively demonstrated, the historiographical system that comics studies has uncritically adopted from fan traditions is fundamentally flawed. What we propose is a genealogy of comics storytelling that will, for the first time, be augmented by substantial empirical data on the field of comics production as a whole. Taking a cue from French sociologist Pierre Bourdieu, What Were Comics? investigates the transformation of transformation-time in American comic books.

2.  Underlying Assumptions and Rationale One of the central thrusts of the first decades of research on comics was the definition of properties that would distinguish comics from literature, cinema, and the visual arts. The earliest book to address this topic ­(Sheridan) defined comics exclusively as newspaper strips, but the rapid expansion of the American comic book industry over the course of the 1940s and into the early-1950s, avant-garde ‘underground comix’ in the 1960s and 1970s, the development of the graphic novel format as trade standard in the 1990s, and webcomics in the 2000s have forced repeated considerations of the form. Proposed definitions of comics, which rely upon some combination of image sequentiality, speech balloons, recurring characters, or mass reproduction (Waugh; Kunzle; Eisner; Inge; McCloud; Blackbeard and Crain), are more normative than analytically descriptive. They are motivated rather than objective definitions, and they are based on highly selective corpora of examples. Indeed, efforts to define the essence of comics often rely on the formal operations of limit cases, such as wordless c­ omics (Beronä; Postema) and abstract comics (Groensteen ­“Monstrator”; ­Molotiu). The What Were Comics? project is not interested in ­understanding what comics could possibly be (can comics be sculpted in three dimensions? can they include aspects of live performance?), but rather focused on the concrete analysis of what comics have historically been.

30  Bart Beaty et al. At its base level, our research questions have been inspired by ­ ordwell, Thompson, and Staiger’s groundbreaking film studies work, B The ­Classical Hollywood Cinema, which traced the development of film style through shot-by-shot analyses of one hundred randomly selected American feature films produced between 1917 and 1960. Bordwell et al. demonstrated the persistence of certain narrative forms, the prevalence of continuity editing, and the development of a unified conception of space and time in Hollywood productions. That is, they were able to describe the typical or ordinary Hollywood film in terms of its formal and stylistic characteristics. The drive to demonstrate the continuity of traditional formal patterns in cinema located its antecedents in the work of Gombrich, whose aphorism ‘making precedes matching’ highlights the highly-schematized nature of artistic conventions, and ­Mukař ovský’s conception of the way that aesthetic, technical, and practical norms undergird socio-political norms. Following Osherson and Smith, we understand ‘typicality’ as a function of classification that refers to the extent to which objects are ‘good examples’ of concepts (Smith and Medin; Hampton; Rosch; Kamp and Partee). To arrive at a more complete understanding of typicality—and, specifically, of a diachronic sense of typicality within the American comic book industry—we have followed the example most famously advanced by Moretti of extensive, rather than intensive, reading (Graphs). Moretti has helped to popularize new methods in literary studies by turning his attention to extremely large textual corpora; the title of one of his essays, “Style, Inc.: Reflections on 7,000 Titles,” is suggestive of this approach. On the cutting edge of trends in the humanities (digital humanities) and the social sciences (‘big data’), this data-driven approach to the study of literature has flourished in areas, such as Victorian literature (Liddle; Mussell; Nicholson), that have been extensively digitized thanks to both lapsed copyrights and regularized printing formats. Today, comic studies is only beginning to benefit from developments in digital humanities and big data, as the visual nature of comics, like Medieval manuscripts, makes them more difficult to mark up (Bateman et al.; Pederson and Cohn). Our project extends the insights developed by Moretti and his students to the field of comic books by creating a sample frame and corpus, and by mining that corpus for insights into the historical evolution of the comic book as a publishing format. To this end, we are not interested in revealing an eternal essence of comics that characterized so much of the modernist project (Bazin; Greenberg; Lessing), and, by extension, comics studies (Groensteen, “Système de la Bande Dessinée”; McCloud). Rather, we will examine the uneven historical emergence of a new media form in the twentieth century, one that was put to different uses in different times for different audiences. The media genealogies undertaken by Gaudreault and Marion (“Cinema as a Model”; “Medium is Always Born Twice”)

Two Per Cent of What?  31 offer an example of the way that the formal operations of comics can be analyzed. The detailed explication of the origins and development of specific tropes within the field of cultural production forms the basis of this project. The What Were Comics? project seeks to challenge the hegemony of a certain kind of literary method of analysis—namely, the close reading of the exemplary text—at a moment when comics studies is at a crossroads between expanding into a thoroughly trans-disciplinary field of study and remaining a marginal specialism within literature departments. Since our study depends upon a corpus that represents a randomly selected sample of 2% of the comic books published in the United States between 1934 and 2014, several conceptual issues immediately arose: Namely, how many comic books have been published in this time frame, and, as a corollary, what exactly is a comic book, anyway? To create their study of classical Hollywood cinema, Bordwell et al. largely outsourced their selection of films by adopting the list of films released in the United States found in Chester B. Bahn’s The 1961 Film Daily Yearbook of Motion Pictures. Unfortunately, within the comic book industry, no such definitive account yet exists, and we were compelled to create our own sampling frame rather than borrowing an existing one. One of the earliest decisions that we faced in constructing our sampling frame was the definition of the term ‘comic book’. Somewhat surprisingly, our review of the relevant literature turned up very little in the way of analytic tools for the defining the limits of the ‘population’. ­Suggested definitions based on popular usage also presented certain problems. Each of these is worth considering in turn: Publishing Format. While the format of the American ‘comic book’ is popularly understood, there are a large number of works that might logically be included in the sampling frame but that do not adhere to the standardized industry formats which, after all, vary over time. Notably, the most common publishing formats for comic books have shifted substantially over the history of the form, dropping from sixty-four pages to thirty-two pages, but with examples ranging into hundreds of pages. Moreover, a focus on normative formats would rule out magazine-sized comics from publishers as diverse as Warren and Fantagraphics, as well as digest-­sized comics from publishers like Archie. It is clear at a glance that comic books have been published in the United States in an extremely wide variety of formats. For this reason, we ruled out this factor as defining. b Seriality. The presumption that comic books are serial is countered by the history of the industry, where publishing strategies have shifted over time within and between differing publishers. Comic books have frequently been published not only on a monthly schedule, but a

32  Bart Beaty et al. are also commonly bi-monthly, quarterly, and, more rarely, weekly or bi-weekly. Of course, many comic books are not serialized at all, but are instead stand-alone works. There is nothing specific about seriality that seems relevant to the definition of the comic book, and, consequently, we also eliminated this from our deliberations. c Genre and Content. One of the explicit goals of this project is to determine what has been the content of comic books over time, so any attempt to define this a priori could not work. Additionally, as we note in Section 5, our corpus fundamentally undermines the common equation of comic books with superheroes. Finally, definitions of genre are as vexed, or even more so, than those of comic books. A reliance on this category seemed to be trading one set of problems for a different one. d Distribution Network. The possibility of defining comic books through their distribution network at first appears to have some merit, but immediately runs into a number of historical issues. First, predominantly magazine distributors released comic books from the 1930s through the 1980s, necessitating some way of distinguishing between magazines and comic books. This returns us to the already dismissed issue of format. Second, since the 1980s comic books have been largely, but not exclusively, distributed to the direct market of comic book stores, but focusing on this retail channel would exclude certain works, including many comic books circulated to book stores. Finally, at the practical level, it seemed that absent publicly accessible records it was unlikely that we could generate lists of comic books circulating through newsstand distribution networks, even if we wished to do so. e Publisher. Finally, we briefly considered a definition of comic books as those artifacts published by comic book publishers. That is to say, that works released by Marvel Comics, DC Comics, Archie Comics, and so on would be considered comic books. This became an overly complicated and analytically circular process. Again, we would simply be trading one definitional problem for another, namely: What is a comic book publisher? Ultimately, we recognized that, in definitional terms, comic books are best understood through institutional theories of art. Drawing upon the example provided by Arthur Danto and George Dickie, Beaty’s conception of the ‘comics world’ (Comics versus Art) suggests that ‘comic books’ are artifacts regarded as ‘comic books’ by participants in the comics world, however circular that definition might be. To this end, and in contradistinction to Bordwell and his colleagues who were able to draw upon a definitive list of American films (Classical Hollywood Cinema), we opted to develop our sampling frame by triangulating data

Two Per Cent of What?  33 sources representing three distinct subsections of comics fandom: fan historians and archivists, comic book collectors, and comic book retailers. Thus, we define a ‘comic book’ as any artifact listed in each of the Grand Comics Database, The Overstreet Comic Book Price Guide, and the database of the retailer MyComicShop.com. The logic undergirding our selection is: The Grand Comics Database (GCD). The GCD is simply the largest and most comprehensive public database attempting to catalogue comics production from around the globe. While the GCD has inclusion criteria that seemed to be at odds with some of our understandings, we were confident that its highly inclusive nature would give us the largest possible starting framework and that the other sources would serve to curtail the expansive impulse demonstrated by their volunteers. b The Overstreet Comic Book Price Guide. This has been the standard reference work for comic book collectors for well over forty years and is arguably the most venerable and authoritative institution within comic book fandom. Considerably more selective than the GCD, the influence of Overstreet on the shaping of the historiography of the American comic book is difficult to overstate. c MyComicShop.com. It seemed important to cross-reference the other sources with a large retail site on the assumption that comic books might be those items that are sold by comic book stores. ­MyComicShop.com has one of the most extensive databases that we encountered. It was clear that their database was not derived from the GCD and, therefore, would not replicate any errors or biases that might be found in that site. a

Finally, we recognized that our study relied on a fairly unrefined understanding of what was meant by the geographical locator in ‘­American comic books’. Notably, for instance, comic books published by the ­Montreal-based publisher Drawn and Quarterly circulate in comic book networks within the United States, but these are not ‘American’ comic books in any meaningful sense. Including them would necessitate the inclusion of all foreign-produced comic books that enter American circulation, even to the point of including regional distribution of ­Mexican comic books in border towns, or the circulation of Franco-Belgian bandes dessinées in French-language bookstores. We have restricted our understanding of ‘American comic books’ to those that are edited, ­published, and circulated within the United States. It is our hope that future comparative studies of Canadian, Mexican, Franco-Belgian, or other national tendencies, comic books can be undertaken in the future using our research tools and methodology.

34  Bart Beaty et al.

3.  Methodology The processes involved in creating the What Were Comics? sampling frame have been labor-intensive, due to the vagaries of our sources. The first step in our process was to download the MySql file from the Grand Comics Database, current as of 29 April 2015. Data was extracted for series published in the United States with queries for series_name, number, volume, publication_date, key_date, and publisher_name. The resulting data was compiled in an Excel database and comprised 293,282 entries beginning in 1867. After eliminating works that fell outside of our historical range (1934–2014), we noticed several issues in the database that needed immediate attention, including the duplication of certain records and, more pressingly, incomplete entries. The most significant issue with the data, from the point of view of our project, was the fact that 78,073 entries lacked a publication date, a key date, or both. This fact added a substantial and unexpected roadblock to the project. Having obtained the GCD data and discovering that almost 27% of the entries lacked key information, we set out to rectify the situation. ­Entries that did have associated dates were sorted into spreadsheets by year, and research assistants were tasked with the job of associating dates with the undated entries by comparing the entries (title, number, and publisher) with our other sources. By hand-coding the data in this way, the research assistants were able to confirm dates for more than 98% of the undated entries, and these were added to the spreadsheets in the appropriate year. The next step in hand-curating our data was to eliminate obvious duplicates. The GCD counts comic books with different covers (variant editions) as separate publications but, for our purposes, these are considered the same comic books. Thus, to take an example at random, the 2014 publication of Star Trek/The Planet of the Apes: The Primate Directive #1 by IDW appeared in our initial spreadsheet eight times because it has eight different covers, but seven of these were deleted prior to the sampling. During the peak years of the speculator boom (1990–1994, and the 2010s) we found that duplication due to variant covers amounted to as much as 70% of individual entries in a given year. Having sorted the GCD data set to individual years and eliminated duplicate issues, research assistants were then employed to compare our data to our other two sources. Once again, this job was performed by hand and proved time-consuming. Comics were first cross-referenced against MyComicShop.com and marked as present or absent from that data set. They were then re-checked against the 46th edition of The Overstreet Comic Book Price Guide (2016–2017), which exists only in print. Comic books that did not appear in all three of these sources were marked for deletion from the sampling frame and review by the principal investigator and then removed. All of the resulting spreadsheets were read by the principal investigator and reviewed for accuracy.

Two Per Cent of What?  35 With the sampling frame completed, we turned our attention to identifying our corpus. For each year, we calculated 2% of the sampling frame (rounded up) and used the Random Integer Set Generator on random.org to produce our results. The Random Integer Set Generator allowed us to produce, for each year, one set of integers arranged sequentially in ascending order. This list of integers was then cross-referenced against our spreadsheet and the corresponding row defined the comic book that became part of our corpus. While not germane to this chapter, our next step will be to acquire copies of every comic book in the corpus for coding purposes. We intend to house this collection in Special Collections at the University of Calgary when the What Were Comics? project has been completed.

4.  Results The results generated by this project to date are both very s­ traightforward: First, we have generated a completed sampling frame; second, we have randomly selected 2% of the comic books per year from the sampling frame (rounded up) to create our corpus of comic books for future study. Again, our sampling frame consists of comic books ­published in the United States from 1934 to 2014 that are indexed in The Grand Comics Database, The Overstreet Comic Book Price Guide, and on ­MyComicShop.com. Works that are excluded from any one of these three indices are not included in our sampling frame (the implications of these exclusions are discussed in the next section). Significantly, our sampling frame contains 176,275 comic books published during the eighty-one year period with the range within years stretching from 57 (1935) to 4,297 (2014). Figure 2.1 demonstrates the number of comic books in the sampling frame per year.

Figure 2.1  N  umber of comic books in the sampling frame per year.

36  Bart Beaty et al. The resulting corpus of works tracks the sampling frame almost exactly. After rounding upward (the 116 comics in 1934 produces three works in the corpus rather than 2.3) the total number of works in our corpus is 3,563. Both the sampling frame and the corpus are housed in the “Tools” section of the What Were Comics? project webpage at: www.whatwerecomics.com/tools/, and are freely available for use by other scholars. The corpus links to online versions of the comic books where they are in the public domain for reference, and we are working to compile a collection of every comic book in our corpus to be housed at the University of Calgary.

5.  Discussion When we claim that 176,275 comic books have been published in the United States between 1934 and 2014, what we are averring is that this is the number of comic books that meet our definition of ‘American comic books’ because they can be found in all three of our indices. In other words, this is the smallest possible number of comic books that could be included in our sampling frame. Had we required that a work only appear in any two of the three data sets, our number would have increased substantially. Indeed, our initial download of the data included 293,282 comic books, although some of these were published before 1934 and were thus quickly excluded. Nonetheless, what is clear is that a sampling frame with a different working definition of ‘American comic book’ will produce a result of a different size. 176,275 comic books is the number of works produced for our purposes, but a change in emphasis will produce a different result. It is clear that all three of our data sources have different biases. The GCD, for instance, is by far the most expansive of the three. The GCD defines success in its project as containing “data for every comic book ever published in every country around the planet” (GCD FAQ). F ­ urther, they define comic books as works containing at least 50% comics content, although they permit works with less comics content if they are flagged as such. One consequence of this vision is that the dataset includes works that only very few people might define as ‘comic books,’ including, for example, Playboy Magazine (tagged as less than 50% ­comics). On the other hand, The Overstreet Comic Book Price Guide is the most restrictive of the three datasets. There is a practical reason for this. As a print guide, Overstreet is theoretically limited in size (the 2016–2017 edition is 1,216 pages long). Yet the Overstreet Guide also makes ideological decisions that they foreground. In the 1989 edition, for example, the Guide indicated, “Just because someone puts out a black and white comic out of their basement does not mean we should acknowledge its existence in this book” (Overstreet, “Price Guide 1989–1990” A9). ­Similarly, ­MyComicShop.com is limited by choices of what is appropriate to sell

Two Per Cent of What?  37 made by the company’s owners. Significantly, the GCD data was reduced by more than 110,000 comic books during the course of assembling the sampling frame. Among the most important consequences of that process we eliminated: Multiple and Variant Covers. It has been commonplace since the 1970s for certain publisher to publish comic books with minor distinctions on their covers. These include split-run pricing (for Canada and for the United Kingdom) and differing distribution systems (a direct market edition and a newsstand edition). For our purposes, these are not different comic books, although the GCD counts them twice (or sometimes more). Beginning in the 1990s, and continuing to this day, the practice of creating variant covers (often limited-­edition) has been a widespread practice among comic book ­publishers. Again, for our purposes these are the same comic book, and so comics that the GCD may list dozens of times were reduced to one instance. Duplication was (by a very wide margin) the largest category of comic books eliminated from the sampling frame. b Give-away Comics. Comic books produced to be given away by corporations, civic organizations, or industry groups were not ­always indexed by Overstreet, despite the fact that the Guide has a s­ ubstantial section for ‘Promotional Comics.’ Among the comic books found in only one of the datasets were comics promoting shoe companies, gas stations, and breakfast cereals. This resulted in an extremely small number of exclusions. c Underground Comix. One of the most contentious omissions from the sampling frame are so-called ‘underground comix’ produced in the 1960s and 1970s. Despite their collectability, Overstreet has deliberately excluded these works since its origins in 1970 and even the best-known works in the genre (such as Zap Comix) do not appear in its pages. At the beginning of this project we considered using an additional data source (either Jay Kennedy’s The ­Official ­Underground and Newave Comix Price Guide or Dan Fogel’s ­Fogel’s Underground Price and Grading Guide 2015–2016), but ultimately opted not to complicate our initial assumptions. We do not see this as a limitation of our sampling frame. Rather, we acknowledge that the sampling frame represents a certain conception of ‘mainstream’ American comic books as it has evolved over time and which regards the undergrounds, with their separate modes of distribution, as a distinct publishing phenomenon. It is highly likely that a future project will create a new sampling frame for underground comix that will allow us to conduct a comparative study. Notably, Fogel’s guide includes more than 9,600 listings (counting multiple printings), which means that a substantial comparative analysis would likely generate significant results. a

38  Bart Beaty et al. d Erotic Comics. It should be noted that, to the extent that they are considered related to underground comix, Overstreet does not generally index erotic comic books of the type published by Eros ­Comics and similar publishers and, also, many of them are not retailed by MyComicShop.com. This is another interesting avenue for future research, although there is no standard reference work comparable to Fogel’s guide. e Manga. While our project does include Japanese comics as they appear in English-language translation, it does so only selectively. Overstreet is highly selective about the inclusion of manga. For instance, the serialization of Katsuhiro Otomo’s Akira in thirty-eight issues by Marvel Comics (1988–1995) is found in Overstreet (and MyComicShop.com) but versions published by Epic (1990–1993), Dark Horse (2000–2002), and Kodansha Edition (2009–2011) do not. This means that the manga that do appear in the sampling frame are most frequently those that were published in the prototypical American comic book format rather than as graphic novels or approximations of Japanese tankōbon. Again, the correction for this omission would produce an interesting case study. f Graphic Novels. In the 1991 edition of the Overstreet guide it is noted that long-form comic books like Art Spiegelman’s Maus (1986) are “a real dilemma for the collector in our hobby” because they “generally do not appreciate in value” (Overstreet, “Price Guide 1991–1992” A26). What, it is implied, is the use of cataloguing works that don’t change their prices in a price guide? To this end, while Overstreet does, in fact, include Maus (in its many editions), it does not include many other well-known graphic novels (such as Alison Bechdel’s Fun Home (2006)). This significantly skews our sampling frame, once again placing an emphasis on a kind of normatively understood ‘comic book.’ The vast majority of exclusions from the sampling frame fall into one (or more) of the categories above, and these are the necessary caveats for understanding its limitations. Centrally, our project is concerned to determine what American comic books have been throughout their history. As a result, it was impossible to assume a definition of the comic book a priori without fundamentally predetermining our findings. To this end, we ­outsourced the definition of the ‘comic book’ to institutions within the comics world that are concerned with data collection, collecting, and retailing. The resulting sampling frame was compiled without any bias on our part, and we believe that the corpus represents the fairest account of the ‘typical’ comic book compiled to date. The limitations that arise from the biases introduced by our source datasets do not, for us, represent errors in the sample so much as they present opportunities for future research. We look forward to collaborating with scholars who would seek to fill these gaps.

Two Per Cent of What?  39

6.  Outlook As we outlined in the first section, the What Were Comics? project has three goals. The now completed construction of the sampling frame and corpus is only the first of these, and it is, arguably, the least generative. At the time of writing, we are employing research assistants to markup the works in our corpus so that we might trace the transformation of the American comic book through a survey of its typical output. While we are coding for sixty-one different variables, a few will suffice to provide a sense of what we hope to discover. Generally, we are collecting data in four primary areas: a

The Comic Book. In addition to the identifying data (publisher, date of publication, title, issues number, price(s)) we are concerned to discover the periodicity of the comic books in our corpus, their seriality, and their type (for instance, single or multi-story). We are particularly interested in determining the shape of the comic book publishing industry over time. For example, there are more than 11,000 separate comic book series in the sampling frame and more than 7,000 comic books that are not part of any series. This means that the average comic book series contains slightly more than fifteen issues. The rate at which new comic book series were created and cancelled, and, importantly, the way that this rate has shifted over time, will be an important finding. b The Comic Story. Some comic books are comprised of multiple stories, some by a single story, and some are part of a longer serialized story. We are concerned not simply with determining the length of typical comic book stories, but are also interested in shifting conceptions of the production process over time. To this end, we have charged the research assistants not only with recording the length of comic book stories, but the presence or absence of certain type of production credits and editorial cues. While it has frequently been suggested that comic book credits were uncommon in comic books before the 1950s, our initial data suggests that this is not exactly the case. c The Comic Page. Our markup of the comic pages in our corpus has been greatly influenced by Bateman et al. and also by the work of Kaitlin Pederson and Neil Cohn. Comics pages are being studied for layout type to understand the evolution of composition over time. d The Comic Panel. Our base unit of analysis will be the comic panel, which will be marked up according to graphic and linguistic elements. To take but one example, we are curious to determine the amount of text present in panels over time and to determine not only if the impressionistic understanding that comic books have become less ‘wordy’ is accurate, but also to pinpoint key shifts over time.

40  Bart Beaty et al. It is our hypothesis that the study of our corpus will generate new insights into the history of the American comic book that will enable us to move beyond a simple theory of comic book ages by providing empirical evidence of how comic books have been constructed at various points in their eight-decade history.

Acknowledgements The authors would like to acknowledge the financial support of the ­Social Sciences and Humanities Research Council of Canada, which is the major sponsor of the What Were Comics? Project through the Insight Grant program. Additional funding for this research has been provided by The University of Calgary, Carleton University, and the ­Ontario Work Study Program. We are grateful to the research assistants who have contributed to this work: Maria Ahmad, Braydon Beaulieu, Sidney Cunningham, S­ abreen El-Awad, Mehedi Hasan, Zach High-Leggett, Nicole Maillete, Tom Miller, Sarah Pink, Jacob Pitre, Tom Sewel, Tiffany Sostar, and V ­ eronika Wilinska. A special thanks is due to Rodrigo Baeza for his work with MySql files from the Grand Comics Database. We are ­thankful to the Grand Comics Database for sharing their dataset with us and encourage interested researchers to visit Comics.org.

Works Cited Bahn, Chester B. The 1961 Film Daily Yearbook of Motion Pictures. Film Daily, 1961. Bateman, John, Francisco O. D. Veloso, Janina Wildfeuer, Felix HiuLaam Cheung, and Nancy Songdan Guol. “An Open Multilevel Classification Scheme for the Visual Layout of Comics and Graphic Novels: Motivation and Design.” Digital Scholarship in the Humanities, vol. 32, no. 3, 2017, pp. 476–510. Bazin, André. Qu’est-ce que le cinéma? Éditions du CERF, 2003. Beaty, Bart. Comics Versus Art. U of Toronto P, 2012. Beaty, Bart, and Benjamin Woo. The Greatest Comic Book of All Time: ­Symbolic Capital and the Field of American Comic Books. Palgrave, 2016. Beronä, David. Wordless Books: The Original Graphic Novels. Abrams, 2008. Blackbeard, Bill, and Dale Crain. The Comic Strip Century: Celebrating 100 Years of an American Art Form. Kitchen Sink P, 1995. Bordwell, David, Janet Steiger, and Kristin Thompson. The Classical ­Hollywood Cinema: Film Style and Mode of Production to 1960. Columbia UP, 1985. Bourdieu, Pierre. “The Field of Cultural Production, or: The Economic World Reversed.” The Field of Cultural Production, edited and introduced by ­Randal Johnson, Columbia UP, 1993, pp. 29–73. Cohen, Margaret. The Sentimental Education of the Novel. Princeton UP, 1999. Danto, Arthur. “The Artworld.” Journal of Philosophy, vol. 61, no. 19, 1964, pp. 571–584.

Two Per Cent of What?  41 Dickie, George. Art and the Aesthetic: An Institutional Analysis. Cornell UP, 1974. Eisner, Will. Comics & Sequential Art. Poorhouse P, 1985. Gaudreault, André, and Philippe Marion. “The Cinema as a Model for the ­G enealogy of Media.” Convergence, vol. 8, no. 4, 2002, pp. 12–18. ———. “A Medium Is Always Born Twice.” Early Popular Visual Culture, vol. 3, no. 1, 2005, pp. 3–15. Taylor & Francis, doi:10.1080/17460650500056964. “General FAQ.” Comics.org, 23 Oct. 2016. Grand Comics Database, docs.comics. org/wiki/General_FAQ. Accessed 16 Nov. 2017. Gombrich, Ernst H. Art and Illusion. Phaidon P, 1960. Greenberg, Clement. “Modernist Painting.” The New Art: A Critical A ­ nthology, edited by Gregory Battock, Dutton, 1966, pp. 68–69. Groensteen, Thierry. Système de la bande dessinée. Presses Universitaires de France, 1999. ———. “The Monstrator, the Recitant, and the Shadow of the Narrator.” E ­ uropean Comic Art, vol. 3, no. 1, 2010, pp. 1–22. Hampton, James A. “Prototype Models of Concept Representation.” C ­ ategories and Concepts: Theoretical Views and Inductive Data Analysis, Academic P, 1993, pp. 67–95. Inge, M. Thomas. Comics as Culture. UP of Mississippi, 1990. Kamp, Hans, and Barbara Partee. “Prototype Theory and C ­ ompositionality.” Cognition, vol. 57, no. 2, 1995, pp. 129–191. ScienceDirect, doi:10.1016/00100277(94)00659-9. Kunzle, David. The Early Comic Strip: Narrative Strips and Picture Stories in The European Broadsheet from c. 1450 to 1825, U of California P, 1973. Lessing, Gotthold Ephraim. Laocoon: An Essay on the Limits of Painting and Poetry. Translated by Edward Allen McCormick. Bobbs-Merrill, 1962. Liddle, Dallas. The Dynamics of Genre: Journalism and the Practice of L ­ iterature in Mid- Victorian Britain. U of Virginia P, 2009. McCloud, Scott. Understanding Comics: The Invisible Art. Tundra, 1993. Molotiu, Andrei. Abstract Comics. Fantagraphics Books, 2009. Moretti, Franco. Distant Reading. Verso, 2013. ———. Graphs, Maps, Trees: Abstract Models for a Literary History. Verso, 2005. ———. “Style, Inc.: Reflections on Seven Thousand Titles (British Novels, 1740–1850).” Critical Inquiry, vol. 36, no. 1, 2009, pp. 134–158. Mukařovský, Jan. Structure Sign and Function: Selected Essays. Yale UP, 1978. Mussell, James. The Nineteenth-Century Press in the Digital Age. Palgrave, 2012. Nicholson, Bob. “‘You Kick the Bucket; We Do the Rest!’: Jokes and the Culture of Reprinting in the Transatlantic Press.” Journal of Victorian Culture, vol. 17, no. 3, 2011, pp. 273–286. Taylor & Francis, doi:10.1080/13555502.201 2.702664. Osherson, Daniel, and Edward E. Smith. “On Typicality and Vagueness.” Cognition, vol. 64, no. 2, 1997, pp. 189–206. ScienceDirect, doi:10.1016/S0010-0277 (97)00025-5. Overstreet, Robert. “Preface.” The Overstreet Comic Book Price Guide, 1989– 1990. Gemstone Publishing, 1989, A8–A9. ———. “The 1990 Market Report.” The Overstreet Comic Book Price Guide, 1991–1992. Gemstone Publishing, 1991, A19–A27.

42  Bart Beaty et al. Pederson, Kaitlin, and Neil Cohn. “The Changing Pages of Comics: Page ­Layouts across Eight Decades of American Superhero Comics.” Studies in Comics, vol. 7, no. 1, 2016, pp. 7–28. Ingenta, doi:10.1386/stic.7.1.7_1. Postema, Barbara. “Draw a Thousand Words: Signification and Narration in Comics Images.” International Journal of Comic Art, vol. 9, no. 1, 2007, pp. 487–501. Rosch, Eleanor. “Principles of Categorization.” Cognition and Categorization, edited by Eleanor Rosch and Barbara B. Lloyd, Erlbaum, 1978, pp. 27–48. Sheridan, Martin. Comics and Their Creators: Life Stories of American C ­ artoonists. Hyperion P, 1942. Smith, Edward E., and Douglas Medin. Categories and Concepts. Harvard UP, 1981. Waugh, Coulton. The Comics. Macmillan, 1947. Woo, Benjamin. “An Age-Old Problem: Problematics of Comic Book Historiography.” International Journal of Comic Art, vol. 10, no. 1, 2008, pp. 268–279.

3 The Quantitative Analysis of Comics Towards a Visual Stylometry of Graphic Narrative Alexander Dunst and Rita Hartel 1.  Introduction: The Increasing Diversity of Comics Studies For most of its relatively brief history, academic comics research was dominated by methods drawn from literary studies. A distinct form of close reading characterized this growing field of inquiry: Focusing on a small, emerging canon, scholars offered reflections on selected comics series, graphic novels, and manga that sought to historicize the development of the medium or put forward versions of ideology critique inspired by cultural studies. As Bart Beaty has argued, this methodological focus tended to neglect the visual aspects of comics insofar as it read them as a form of literature rather than art (Comics). More recently, a wider range of empirical and formalist approaches has supplemented literary ­methods—a consequence in part of the rising cultural prestige of comics. Multimodal research, the first forays by cognitive scientists, an emerging transmedia narratology, computer science, and the digital humanities (DH) now all contribute to our understanding of comics. This chapter adds a decidedly quantitative approach to the study of comics by introducing a stylistic analysis of a large corpus of booklength comics, or graphic narratives. As the next section explains in more detail, even empirical research in the field has yet to move beyond the analysis of a limited canon. To date, humanities scholars interested in comics have lacked the means to study large numbers of images given the combined challenges of digitization, annotation, and computation. In the few cases where computer scientists have assembled or analyzed large collections of comics, their work has focused on the automatic recognition of formal features such as panels, speech balloons, or characters, rather than researching stylistic or narrative features (Fujimoto et al., Iyer et al., Guerin et al.). In this essay, we begin to remedy this situation and introduce methods that distinguish between different genres, detail the historical evolution of graphic novels and memoirs, and shed light on central aspects of authorship and publication formats. Our visual analysis is based on the first representative corpus of graphic narrative, which we understand to be book-length comics that are aimed at

44  Alexander Dunst and Rita Hartel an adult readership and tell continuous, or closely related, stories. Thus, we follow scholars such as Hillary Chute, Daniel Stein, and Jan-Noël Thon in distinguishing between graphic narrative as long-form comics that tell both fictional and non-fictional stories, and individual genres such as the graphic novel and graphic memoir (Chute 453 and Stein & Thon 4–7). While many publishers, literary critics, and even comics artists casually refer to the graphic novel in this earlier sense as an umbrella term, this usage is highly misleading. Neither Art Spiegelman’s Maus nor Joe Sacco’s Palestine are graphic novels in any meaningful sense of the word but constitute examples of graphic non-fiction—a memoir in the first and journalistic reportage in the second instance. These theoretical distinctions take on practical importance in empirical research and will form the conceptual foundation for the automated genre distinction we present here. In this chapter, an introduction to our research question and hypotheses are followed by sections on method, corpus design, and data analysis. For readers unaccustomed to technical discussions, these subchapters may present a challenge. However, comprehension of the chapter does not require the understanding of equations or statistics. Two final sections analyze results and offer an outlook on future work.

2.  Rationale and Hypotheses: Moving from Qualitative to Quantitative Research A number of roadblocks have impeded quantitative research on comics. Until recently, studying comics was the domain of individual fans or scholars who often lacked access to technical infrastructure and expertise. The large-scale digitization of comics by publishers, researchers, or on crowdsourced online databases represents a recent phenomenon that faces severe copyright restrictions. Crowdsourced materials add further challenges, as their quality may differ drastically from one scanned image to another. Skewed or yellowed pages, blurred captions or other scanning artifacts will impede text and image recognition. In turn, copyright restrictions may not only restrict digitization but also prevent the sharing of corpora that already exist. Like other researchers in DH, comics scholars also face methodological challenges and increasingly need expertise in statistics and practical programming skills in addition to disciplinary knowledge. Where these hurdles are overcome, annotation adds another, if frequently necessary, bottleneck. Even semi-­ automated mark-up remains a cost-intensive and time-consuming task, proving prohibitive for studying large data sets. Overcoming these challenges becomes an even more urgent task given the necessary limitations of qualitative research. Experimental studies of cognitive processes add an invaluable perspective to our understanding of comics narratives. For the first time, methods such as eye tracking

The Quantitative Analysis of Comics  45 and EEG allow for the direct study of reception processes and brain functions. However, experimental set-ups remain restricted to studying excerpts, shorter texts such as comic strips and brief stories, or smaller numbers of long-form comics. Similarly, narratology and multimodal approaches currently lack the powers of automation that would enable their application to large corpora of visual narratives. This restriction to excerpts and small samples risks obscuring both the historical evolution and aesthetic properties of the medium. Individual case studies, however pertinent the insights they may produce, must be checked against quantitative analyses of comics as a cultural system. Ultimately, cognitive and literary case studies may form one element within a tripartite research program. Ideally, corpus research will eventually complement experimental studies and close readings (Beaty, Sousanis & Woo; Cohn, this volume). Both, in turn, may then be embedded in a cultural sociology that mediates between different levels of analysis and provides a media-specific theory of production, circulation, and consumption ­(Underwood & English). Automated stylistic measures mark an important step in making this ambitious research program a reality. In what follows, we introduce several measurements that describe the visual style of graphic narrative. This focus on visual properties means that text enters into our analysis only as image, not linguistic, data. We maintain this focus for pragmatic as well as methodological reasons. The automatic detection of text on comics pages remains a work in progress, and optical character recognition (OCR) still struggles with handwritten or quasi-handwritten comics fonts. As was mentioned earlier, manual mark-up remains extremely time-intensive. This means that we cannot easily annotate the text contained in hundreds of book-length comics. At the same time, there exist a wide range of methods for analyzing literary texts that may be applied to comics once we successfully adapt OCR to the latter’s idiosyncrasies. In contrast, our research for the first time describes and differentiates between some of the central concepts that underpin the study of comics on the basis of visual style. The work we present in this chapter draws its inspiration from ­several developments in DH. Issues of genre and authorship have been at the ­center of digital literary studies recently, drawing on stylometry, topic modeling, and social network analysis (Moretti, Jockers, and ­Underwood). These methods will likely gain importance for comics research in the years ahead. However, comics are also, and primarily, a visual medium whose verbal components are presented as written text, rather than given auditorily. Any systematic study of comics must ­therefore engage with their image content and the issue of artistic style. One of the first DH scholars to address this question was Lev Manovich, whose research presented exploratory visualizations of manga and ­modernist painting and led to the development of a software tool for

46  Alexander Dunst and Rita Hartel large image sets (“Style Space”). Manovich’s methods were highly suggestive. However, they proved mostly illustrative when applied to specific media and offered little insight into aesthetic form. James Cutting’s quantitative studies of Hollywood film, which attend to aspects such as shot length, movement, and color to trace cinema’s formal development, proved another inspiration. Cutting’s work, based in psychology, shares our own interest in popular culture and visual style and presents opportunities for comparing two related media. In formulating our main research questions and hypotheses, we drew on the approaches elaborated above: Stylometry and literary theory, digital art history, and empirical film studies. We were also guided by the experience of assembling and studying a growing corpus of graphic narratives. Thus, it seemed to us that graphic memoirs were often brighter than other examples of long-form comics. In one of his papers, ­Cutting described the development of Hollywood cinema with the terms “quicker, faster, darker” (“Quicker” 569). Quicker and faster—a decrease in the average shot length and an increase in motion and movement—seemed specific to film. Brightness, however constitutes an important aspect of comics as well. If Hollywood cinema was becoming darker (noticeable in the action films that generate a large share of industry profits), would we find a similar change in graphic narratives? Another intuition concerned the visual regularity of graphic novels and memoirs. Prominent examples of the two genres—as different as ­Alison Bechdel’s Fun Home and Chris Ware’s Jimmy Corrigan—­eschewed the visual spectacle of contemporary Marvel and DC comics to tell their stories with the help of regular panel grids and restrained color palettes. This observation gave rise to our second research question: Were graphic narratives becoming less animated in their visual style? If so, did this development affect all genres within that broad category? Earlier studies, in which we analyzed over a hundred cover images, suggested a different hypothesis (Dunst et al. “Corpus Analysis”). The increasing stylistic variety of covers seemed to suggest a process of internal diversification. Artists and publishers, we speculated, might be trying to distinguish themselves in a crowded market by designing increasingly varied covers. Would we be able to discern a similar process at work within graphic narratives? Thinking about genre and the dynamics of the still-developing format of graphic narrative led us to the nexus of authorship and ­publishing. ­Historically, most comics were published serially, often anonymously, and by teams of writers, illustrators, letterers, and inkers. S­ erial ­publication remains relatively common for graphic narratives, a c­ haracteristic they share with the literary novel at an earlier stage of its development in the eighteenth and nineteenth centuries. Most typically, however, graphic narratives are a one-shot form, published as novel-length books. S­ imilarly, a ­ owever, the preteam of creatives collaborating on a book is no rarity. H ponderance of single authors means that a version of auteur theory has

The Quantitative Analysis of Comics  47 grown around research into graphic novels. Did these configurations of authorship and publication leave artistic traces that we could detect with the help of automatic measures? In other words, and with the proviso that we were just beginning this kind of work: Are concepts such as the single author and the book format meaningful for the study of graphic narratives?

3.  Method: Automatic Measurements and Artistic Style in Graphic Narrative This section describes the motivations for building a corpus of graphic narrative, presents a brief overview of the principles underlying the corpus design, and details the basic measurements and procedures on which our analysis of comics images is based. 3.a  Material: The Graphic Narrative Corpus In a recent interview, Art Spiegelman described the graphic novel as the “dominant form” of contemporary comics, defining it simply as the “single book that tells a story” (“Public Conversation” 35). Our interest in graphic narrative—what Spiegelman termed, following popular convention, comics novels—stems from a similar observation: The rapid rise of long-form comics in the United States and internationally, and its transformative effect on the medium as a whole. Studying the evolution of graphic narratives thus offers the opportunity to do cultural history in situ: to explore the dynamics of a popular form during a period when it continues to morph into high, or at least middlebrow, culture, in a process we might term aesthetic gentrification.1 The fact that graphic narratives are highly labor-intensive and constitute a niche product in market terms, often devised by a single author over a period of several years, means that they are published in far smaller numbers than literary novels or feature films. Graphic narrative’s brief history and comparatively small numbers enable even a mid-sized corpus to provide insight into an entire cultural form. Their appropriation of aspects of the novel and cinema also make graphic narratives an ideal test case for inter- and transmedial research. Motivated by these research interests, the graphic narrative corpus (GNC) collects fictional and non-fictional texts, including graphic novels and memoirs, as well as graphic journalism (Dunst et al., “Graphic ­Narrative Corpus”). An additional element is what we refer to as graphic fantasy, which includes examples of the fantasy, fairy tale, horror, mystery, superhero, and science fiction genres. In our definition, the term graphic narrative refers to book-length comics that exceed 64 pages in length, tell one continuous or closely-related stories, are aimed primarily at an adult readership, and form one single volume or a limited series (such as a trilogy, etc.). Historically, our corpus stretches from the mid-1970s, when the graphic novel came into its own, to the present. At  the time of

48  Alexander Dunst and Rita Hartel writing, we include around 240 titles. This collection of texts has been conceived as a stratified monitor corpus. This means that we keep adding new titles to increase representativeness and aim to balance aspects like genres or the gender of our authors within the overall corpus. Due to their pop-cultural status and the recent advent of systematic research on the topic, it is impossible to know exactly how many graphic narratives exist. Most libraries, if they collect graphic narratives at all, have only recently begun to do so and lack systematic focus in their acquisitions policy. Established online databases such as the Grand C ­ omics Database (GCD) theoretically provide a more complete overview but do not always distinguish graphic narratives from serial comics. ­Therefore, we have drawn on a wide range of sources in constructing our corpus. These are: international comics prizes (Eisner, Ignatz, H ­ arvey, and the  British Comics Award), academic databases (JSTOR, MLA, Bonn ­Online Bibliography of Comics Research), Amazon.com bestseller lists, online bibliographies (Grand Comics Database, Comicvine), library collections (Library of Congress, Billy Ireland Cartoon Library at Ohio State ­University), literary histories, a survey of international comics experts, and media articles about graphic narratives (The Guardian, Time, etc.). By casting our net widely, we aim to balance pop-cultural ­narratives drawn from genres such as superhero and horror comics with more prestigious forms such as graphic memoirs and offset the biases of individual sources. 3.b  Design and Procedure: Basic Measurements This section provides a brief overview and explanation of the stylistic measurements we used in our research for this chapter. At the time of analysis in November 2017, 209 full-length books in the corpus running to nearly 50,000 pages had been scanned by a commercial provider at 600dpi and saved in PNG format. Scans were checked manually for quality, and pages containing scanning artifacts were replaced with new scans. For each page of each graphic narrative in our corpus, we collected the following three basic measurements: Brightness In order to measure the mean brightness of a page, we transformed the former into a grayscale image by computing the Luma of each pixel, i.e., the weighted sums of the gamma-compressed RGB values of the image, which can be viewed as a linear approximation of the pixel’s luminance. The resulting matrix contained one brightness value for each pixel of the image. The mean brightness of a page can then be calculated as the mean value of all brightness values of all pixels of the page. In addition, we calculated the standard deviation of all brightness values to receive a measurement of the page’s diversity in brightness.

The Quantitative Analysis of Comics  49 Entropy In information theory, entropy (also known as Shannon entropy) may be defined as the expected value of the information contained in a message. The entropy H(X) of a message X = (x1,…,xn) of length n is defined to be H (X) := − ∑ n i =1 P ( xi ) ∗ log 2 P ( xi ) .

(

)

To calculate the entropy of an image, the message X of the entropy is the list of the brightness values of each pixel, with the xi range between 0 and 255. In addition, n is the total number of pixels of the image. As P(x i) denotes the probability or relative frequency of item xi, we can compute P(xi) for a given xi by P(xi):= (Number of pixels having value xi)/(n = total number of pixels). The lowest entropy H(X) = 0 will be measured for an image of a single color, as this image does not contain any unpredictable information. A black-and-white image that contains only two colors will have entropy between zero and one, whereas it reaches a value close to zero if one color clearly dominates the other. A value approximating 1 will be measured if both colors occur equally within the image. The highest possible entropy will be achieved in cases where each level of brightness occurs in a color image and all levels of brightness are equally distributed. As soon as one brightness value dominates the others, the entropy becomes smaller (as the image becomes more predictable). Number of Shapes Similar to entropy in that it measures the unpredictability of an image, the number of shapes contained in an image describes its agitation. In order to yield normalized values, and thus values that are comparable within a graphic narrative as well as between them, we scaled the image to a height of 250 pixels. We then split grayscale images into five binary sub-images of different equal-sized brightness intervals, where each sub-image contains a 1-bit for a pixel p at coordinate (x,y) if the pixel of the original grayscale image belongs to the brightness interval assigned to this sub-image. Next, we filled small holes of 0-bits in the image up to a diameter of four pixels. For each sub-image we then counted the number of connected components, i.e., we collect areas of 1-bits that are next to each other. The number of shapes of an image thus amounts to the sum of connected components of all its sub-images. In a final step, we discounted components that came to less than 10 pixels in size. We did so in order to discount individual letters of the comics text as separate shapes. ­Figure 3.1 shows this process in simplified fashion. First, the picture is split into five sub-­images. In this case, the darkest part contains the outlines of a woman and a book, the second incorporates the woman’s shirt and hair, as well as the book cover, and the third

50  Alexander Dunst and Rita Hartel

Figure 3.1  Illustration of Shape definition. The image is an adapted excerpt from https://commons.wikimedia.org/wiki/File:BD-propagande_ colour_en.jpg (Licensed under CC BY).

just contains visual noise. The fourth sub-image contains the woman’s face, neck, arm, and pants plus some noise, and the brightest part encompasses the background. Staying with our example, we would count four connected components (face, neck, arm, trousers) for the fourth image but ignore the noise, as these components are too small. For the third sub-image we would not count any shapes at all, as they are all too small. Thus, in total, we would compute that this image contains 13 + 7 + 0 + 4 + 3 = 27 shapes. 3.c  Data Processing Following the calculation of our images’ raw data (brightness, entropy, and number of shapes) as described in the previous subsection, we processed these measurements for different purposes. Median Values for Brightness, Entropy, and Shapes First, we aggregated the values per page of a single measure to one single value per graphic narrative by calculating the median of this measure of all pages.

The Quantitative Analysis of Comics  51 Stylistic Diversity within a Graphic Narrative In order to measure the stylistic diversity within a text, we calculated the standard deviation of the three basic measures: brightness, entropy, and number of shapes for all pages of a graphic narrative. 3.d  Data Analysis We used our three basic measurements, as well as standard deviations from these measures of internal diversity, for the analysis of the following concepts: genre, authorship, and publication format. We performed Anova and Tukey’s HSD tests to distinguish individual categories, with p < 0.05 for statistical significance. Genre All titles in the corpus were assigned one or several of a total of 23 subgenre categories. As these categories proved too fine-grained to produce reliable results, subgenres were further grouped into four genres: graphic novel, graphic memoir, graphic fantasy, graphic non-fiction. 2 A small number of titles that did not fit these larger categories were collected under the term miscellaneous. A research assistant assigned subgenre terms on the basis of information provided by publishers, booksellers, or, if necessary, a book’s content. Genre categories were checked for consistency by the authors before our measurements were calculated. Authorship Our data set included several authors who were represented with more than one graphic narrative. For scatter plots that aimed to distinguish between individual artistic styles, we chose authors with three or more titles in our sample. Another series of plots distinguished between the following authorship categories: single author, collaboration between one author and one illustrator, and multiple authors. Publication Format As we mentioned earlier, serial publication remains relatively widespread for graphic narratives. Therefore, we distinguished between four different formats: single-issue, or so-called one-shot, book publications; graphic narratives that were published as part of a book series, such as a trilogy or more than three novel-length installments; graphic narratives that originally appeared as a limited series; and titles that assemble parts of an unlimited series in one volume.

52  Alexander Dunst and Rita Hartel

4.  Results and Discussion Results show that all three of our analytic categories—genre, authorship, and publication format—are meaningful distinctions for the quantitative study of graphic narratives. This confirms our initial research question about the applicability of these concepts to the study of graphic narratives. 4.a  Genre The genres graphic fantasy, graphic memoir, and graphic novel demonstrate significant distinctions in visual style, both synchronically and diachronically. Graphic fantasy, in particular, is significantly darker than other genre categories (Figure 3.2). Titles in this group are also distinctly less regular, with graphic fantasy showing higher median entropy levels than all other genres, in particular graphic memoirs. Examples in our data set include canonical texts such as Watchmen and V for Vendetta, the superhero arcs Batman: Year One and Batman: Dark Victory, or the fantasy narrative Fables: 1001 Nights of Snowfall. This result supports our anecdotal observation that the core genres of graphic narrative, the graphic novel and the graphic memoir, are characterized by a relatively unvarying artistic style. Titles with the lowest entropy and lowest deviation from mean brightness are overwhelmingly graphic memoirs and graphic novels.

Figure 3.2  M  ean Brightness across genres: Graphic Memoir - Graphic Novel (p < 0.016), Graphic Fantasy - Graphic Novel (p < 0.000).

The Quantitative Analysis of Comics  53 Results did not support our broad analogy between graphic narrative and Hollywood cinema. Instead, our measurements indicate a more precise and fascinating parallel between film and graphic fantasy. Cutting had demonstrated a historical evolution towards ever-darker images. As we just saw, mean brightness proved a useful category when it came to distinguishing graphic memoirs from other genres. These distinctions are also visible in a diachronic view. Yet, Figure 3.3 does not show a development akin to Hollywood film in graphic narrative as a whole. If we follow the curve of all graphic narratives contained in our sample, we see that the form seems to have been at its darkest from the early to mid-1980s to the late 1990s. This extreme was followed by a steep upswing, and mean brightness has remained more or less stable since the 2000s. Cutting attributes the decrease of luminescence in ­Hollywood cinema to a number of factors, such as the greater visual range of modern film stock and then digital video, as well as cognitive benefits. As he writes: “a darker film in a dark theatre allows for greater dynamic contrast, which in turn allows for better control over viewers’ attention” (574). While this development has no equivalent in graphic narrative as a whole, it is worth noting that graphic fantasy evinces greater brightness c­ ontrasts and darker images than other genres. Our comparatively shorter time period shows only ­ onetheless, graphic a slight increase on these scores since the 1980s. N fantasy demonstrates clear parallels with contemporary ­Hollywood ­cinema—a form with which it shares an emphasis on visual spectacle, and for which it serves as frequent source material. Whether these similarities are motivated by cognitive benefits, or whether we should attribute the darker images of graphic fantasy to its depiction of personal conflict and dystopian worlds, remains an open question.

Figure 3.3  Y  ear vs. Mean Brightness by Genre.

54  Alexander Dunst and Rita Hartel

Figure 3.4  Year vs. Standard Deviation from Mean Brightness by Genre.

The diachronic view also fleshes out our characterization of the graphic memoir as comparatively uniform in style. Figure 3.4 details how this development has become markedly more pronounced since the mid-2000s. These years saw the publication of well-known examples of the genre such Bechdel’s Fun Home, which is included in our sample. The book’s visual regularity and monochrome color scheme contributes to this development, but also can be seen to influence the further evolution of the graphic memoir. 4.b  Authorship As with genre, our automated measurements were able to distinguish between individual authors on the basis of visual style. Figure 3.5 visualizes the artistic signature of authors who are present with three or more titles in our sample. Authors such as Jason Lutes, Ozamu Tezuka, and Chester Brown, who evince a comparatively consistent style across several titles, take up a smaller space within the overall matrix. Other authors cover a far larger swathe of “style space” (Manovich). These include Frank Miller, Dave McKean, and David Mazzuchelli. To anyone who has read books by these authors, the results will not come as a surprise. At one end of the scale, Brown favors a highly distinctive drawing style characterized by simple lines and the autobiographical description of everyday life. At the other extreme, Mazzuchelli’s graphic novel Asterious Polyp marks a conscious attempt to combine radically different styles within one over-arching narrative. As an

The Quantitative Analysis of Comics  55

Figure 3.5  P  CA Author Style.

aesthetic experiment, it is worlds apart from his other titles contained in our sample: City of Glass, an adaptation of Paul Auster’s novel that successfully translates a literary aesthetic into the comics format, and his work in the superhero genre: Daredevil: Born Again and Batman: Year One. Tezuka’s mingling among the crowd highlights the limits of this approach. Clearly, these stylistic measurements do not help us differentiate between individual manga titles and Western graphic narratives. Whether they’ll be able to do so once we’ve constructed a comparison corpus that includes a sufficient number of manga remains to be seen. Alan Moore and Harvey Pekar present different challenges. These two are featured on the book covers as the authors of their works. Unlike the other authors included in this scatter plot, however, they work with professional artists to draw their comics. And yet, their titles show remarkable consistency, especially in Pekar’s case. It is tempting to attribute these results to Moore and Pekar’s authorial influence over these graphic narratives. Moore, for one, has been known to imagine his stories in meticulous detail that includes many of their visual features. In turn, most of Pekar’s comics are characterized by a relatively narrow set of locales and themes—the depiction of everyday life in Cleveland. Thus, it may be that our stylistic measurements, despite their small number and relative simplicity at this point, capture something of the underlying content described by their authors and realized visually by different artists. A less optimistic take may question whether these measurements remain too broad at this point to ground such an interpretation.

56  Alexander Dunst and Rita Hartel

Figure 3.6  Standard Deviation from Shapes per Page 1 Author - Author + I­ llustrator (p < 0.0114).

That the division of labor between writer and visual artist does lead to measurably different results becomes clear in Figure 3.6. Titles that are co-authored tend to be more varied stylistically, showing a higher standard deviation from the number of shapes. We can interpret this as a surplus of artistic creativity—but this surplus may also make a graphic novel less coherent, and therefore more difficult to read. Figure 3.7 uses the same two measures but places individual book titles in a scatter plot. Two contrasting perspectives command attention here. The first focuses on extremes: Titles that stand out as particularly diverse or consistent. Thus, Brown’s memoir I Never Liked You shows up as the most internally consistent title in our sample. Given its black and white pages and what we noted earlier about Brown’s drawing style, this may come as little surprise. The same goes for Marjane Sartrapi’s Persepolis— another graphic memoir drawn in black and white and a pared-down visual style. Frank Miller’s installment Sin City: The Hard Goodbye may be a more surprising find at this end of the scale. Black-and-white pages play their part. More decisively, perhaps, Frank Miller’s jagged edges and reduced ­settings are uniformly so, evoking a world that is as bleak as it is repetitively brutal. At the other end, featuring extremely diverse graphic narratives, Neil Gaiman and Dave McKean’s Signal to Noise lives up to its name. Another perspective may prove equally if not more interesting. ­Exceptions are one thing—and the humanities have long focused on what they deem to be extraordinary cultural artifacts. Yet our plot shows that some of the most successful authors in the business, those who have managed to publish and sell several graphic narratives, congregate in a

The Quantitative Analysis of Comics  57

Figure 3.7  Standard Deviation of Mean Brightness vs. Standard Deviation of Number of Shapes by Book Titles.

relatively tight cluster. Examples include Will Eisner, Alan Moore, and Craig Thompson. Thus, it might be at the center of these two scales, at a distance from the extremes of internal variety or uniformity, that graphic narratives function most successfully: varied enough to retain the interest of comics readers, sufficiently consistent not to disrupt narrative flow. Thus, the process of diversification that we discerned in book covers seems not to extend to graphic storytelling. Where covers must arouse interest, the narratives that follow need to retain reader’s attention, apparently at the cost of too much variety. 4.c  Publication Format Finally, our measurements show significant differences between publication formats. The measures entropy and number of shapes did not give significant results. However, graphic narratives that were originally issued as book publications, either as single-issue volumes or together with other installments, are much brighter than titles that first appeared as limited or unlimited series (Figure 3.8). One reason for this result lies in the impact of genre: Graphic memoirs and graphic novels are groups

58  Alexander Dunst and Rita Hartel

Figure 3.8  Mean Brightness by Publication Format: Book Publication - Limited Series (p < 0.001), Book Publication - Unlimited Series (p < 0.004), Book Series - Unlimited Series (p < 0.02).

of texts that are generally characterized by higher brightness values. At the same time, prominent exceptions featured in our corpus are not difficult to find: Spiegelman’s Maus, Daniel Clowes’s Ghost World, and Charles Burns’s Black Hole all appeared in serial form first.

5.  Conclusion & Outlook In a famous essay originally published in 1958, the French historian ­Fernand Braudel foresaw “the advent of a quantitative history” ­(“History”). According to Braudel, this new historiography would “get past superficial observation in order to reach the zone of unconscious or barely conscious elements, and then to reduce that reality to tiny elements, minute identical sections, whose relations can be precisely analyzed” (44). Sixty years later, such a quantitative history is appearing on the horizon for the humanities. As a medium, comics have only recently garnered attention from cultural historians and are only starting to interest digital humanists. Still, with every step forward additional possibilities become imaginable. Here, we have presented an initial study, which functions as proof of concept for a stylometric approach that analyzes graphic narratives based on their image content alone. Our corpus will soon grow from around 200 to 250 full-length graphic narratives and supply reference corpora of German graphic novels, Franco-­Belgian bande dessinée, and Japanese manga. These numbers may seem small in

The Quantitative Analysis of Comics  59 comparison to some literary corpora. Conceivably, a single person may read all of them over a couple of months. Yet even 250 books stretch the human capacity for synthesis. And no human eye or brain matches a household computer when it comes to data retention, arithmetic precision, or even pattern recognition. As we add texts, we will also introduce new measurements to supplement our stylistic analysis, starting with colorfulness, and extending to algorithms that distinguish between different kinds of drawn edges and lines. Most importantly, we need to improve automatic text location and OCR capacities to open the treasure trove of textual stylometry. Only combinatory methods of visual and textual analysis will allow us to study a large number of comics in their full complexity and understand the minute interaction between those levels. Distinguishing genre and author signals constitutes another area for improvement. This essay has demonstrated that texts grouped together under the mantle of authorship or genre affiliation share certain stylistic traits. With a sufficiently large data set, we may be able to train the computer to identify other members of these groups, or to establish where and how they overlap. Yet given that artists repeatedly publish in one genre, it is very likely that author signals are interfering with what we take to be the characteristics of genre. Digital literary studies offers examples of how these categories can be disentangled. We need to adopt its methods with care, and adapt and invent wherever necessary. Given his focus on socioeconomic data, Braudel’s essay does not engage with the potential of experimental science. Nonetheless, ­reception data adds a decisive element to quantitative analysis. DH methods tend to examine the formal characteristics of cultural objects in ­isolation, continuing a long tradition that reduces reception to the intuitions of the expert reader. The successful combination of quantitative and cognitive methods will depend on an additional, third, element. ­Quantitative, as much as qualitative, methods need a strong ­g rounding in theory—a cultural theory that lays the groundwork for operationalizing its concepts and connects different media and aesthetic structures. Only then will we be able to shape a truly novel method of analyzing comics.

Acknowledgements This work was funded by the German Federal Ministry of ­Education and Research (BMBF) as part of an early-career research group on digital and cognitive approaches to graphic narrative. For a project description and other information see: graphic-literature.upb.de. We are also grateful to our research assistants Volker Deppe and Svitlana Zarytska for data processing and database entry in preparation of this essay.

60  Alexander Dunst and Rita Hartel

Notes 1 Beaty similarly speaks of the graphic novel as a “gentrifying term” (“Introduction” 108). 2 Subgenres were grouped as follows: graphic novel (action/adventure, coming of age, crime, fiction, historical fiction, romance, comedy, satire); graphic memoir (autobiography, memoir); graphic fantasy (fairy tale, fantasy, ­horror, mystery, science fiction, superhero); graphic non-fiction (biography, educational, historical non-fiction, graphic journalism, travel writing).

Works Cited Beaty, Bart. “Introduction.” Cinema Journal, vol. 50, no. 3, 2011, pp. 106–10. ———. Comics versus Art. U of Toronto P, 2012. Braudel, Fernand. “History and the Social Sciences: The Long Durée.” On ­History, translated by Sarah Matthews, U of Chicago P, 1980, pp. 25–54. Chute, Hillary. “Comics as Literature? Reading Graphic Narrative.” PMLA, vol. 123, no. 2, 2008, pp. 452–465. Cutting, James E., Caitlin L. Brunick, Jordan E. DeLong, Catalina Iricinschi, and Ayse Candan. “Quicker, Faster, Darker: Changes in Hollywood Film over 75 Years.” i-Perception, vol. 2, 2011, pp. 569–576. Dunst, Alexander, Rita Hartel, and Jochen Laubrock. “The Graphic N ­ arrative Corpus: Design, Annotation, and Analysis for the Digital Humanities.” ­Proceedings of the 14th IAPR International Conference on Document ­Analysis and Recognition (ICDAR 2017), 13–15 Nov., 2018, Kyoto, pp. 15–20. Dunst, Alexander, Rita Hartel, Sven Hohenstein, and Jochen Laubrock. “The Corpus Analysis of Multimodal Narrative: The Example of Graphic Novels.” Digital Humanities 2016, 11–16 July 2016, Cracow. Conference Paper. English, James F., and Ted Underwood. “Shifting Scales: Between Literature and Social Science.” Modern Language Quarterly, vol. 77, no. 3, 2016, pp. 277–295. Fujimoto, Azuma, Toru Ogawa, Kazuyoshi Yamamoto, Yusuke Matsui, ­Toshihiko Yamasaki, and Kiyoharu Aizawa. “Manga109 Dataset and ­Creation of Metadata.” MANPU 2016: Proceedings of the 1st International Workshop on Comics Analysis, Processing and Understanding, 4 Dec 2016, Cancun, ACM, 2016, pp. 1–5. Guérin, Clément, Christophe Rigaud, Antoine Mercier, Farid Ammar-­Boudjelal, Karell Bertet, Alain Bouju…, and Arnaud Revel. “EBDtheque: A Representative Database of Comics.” ICDAR 2013: Proceedings.12th ­International Conference on Document Analysis and Recognition, 25–28 Aug. 2013, ­Washington DC, HAL, 2013, pp. 1145–1149. Hal.archives-­ouvertes, hal-00914860. Iyyer, Mohit, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan BoydGraber, Hal Daumé III, and Larry Davis. “The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives.” Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition, 26 June–1 July 2016, Las Vegas, CPS, 2016, pp. 1–10. Jockers, Matthew L. Macroanalysis: Digital Methods and Literary History. U of Illinois P, 2013. Manovich, Lev. “Style Space: How to Compare Image Sets and Follow their Evolution.” Manovich, 2011, manovich.net/index.php/projects/style-space. Accessed 25 June 2017.

The Quantitative Analysis of Comics  61 Moretti, Franco. Graphs, Maps, Trees: Abstract Models for a Literary Theory. Verso, 2005. Spiegelman, Art, and W. J. T. Mitchell. “Public Conversation: What the %$#! Happened to Comics.” Critical Inquiry, vol. 40, no. 3, 2014, pp. 20–35. Stein, Daniel, and Jan-Noël Thon. “Introduction: From Comic Strips to Graphic Novels.” From Comic Strips to Graphic Novels: Contributions to the Theory and History of Graphic Narrative, edited by Daniel Stein and Jan-Noël Thon, DeGruyter, 2013, pp. 1–23. Underwood, Ted. “The Life Cycle of Genres.” The Journal of Cultural A ­ nalytics, 2016, culturalanalytics.org/2016/05/the-life-cycles-of-genres. Accessed 25 Oct 2016.

4 “The Spider’s Web” An Analysis of Fan Mail from Amazing Spider-Man, 1963–1995 John A. Walsh, Shawn Martin, and Jennifer St. Germain 1. Introduction The following study provides an analysis of 33 years of fan mail published in Marvel Comics’ Amazing Spider-Man (ASM) comic book, from the beginning of the series in 1963 through 1995. Like advertisements for Sea-Monkeys and Charles Atlas bodybuilding products, the fan mail feature is a common and familiar paratextual element in the American comic book, particularly from the 1950s through the end of the twentieth century, when communication among fans, publishers, and creators increasingly moved to online forums and social media. Letters columns were a regular feature of science fiction magazines from the early 1930s. Comics scholar Matthew Pustz discusses how these columns contributed to the development of science fiction (and comic book) fandom: In the late 1920s and 1930s, pulp magazines such as Amazing Stories began to publish letters from readers that included their authors’ names and addresses, thereby allowing fans of the genre to find and meet others with similar interests... Organizations devoted to science fiction quickly developed, and these organizations spawned amateur magazines—“fanzines”—and conventions. People interested in science fiction began to discuss comic books in these fanzines. (“2. EC Fan-Addicts and Marvel Zombies”) Later, the letters columns common in science fiction magazines would find their way into comics. Target Comics #6 from Novelty Press is frequently cited as the first American comic book to include a fan letter page (“Target Comics #6 Mile High pedigree”). Superman #124 (September 1958) marks the beginning of the first letters column appearing as a regular feature in a mainstream superhero title (Irvine 91). Letters columns helped publishers establish relationships with their readers. Publisher EC comics was particularly adept at using letters columns to build a rapport with their fans: Other companies, such as EC Comics, were consolidating their younger audience by creating a sense of cooperation between readers

“The Spider’s Web”  63 and creators. Letters pages in EC titles such as Vault of Horror and Weird Science allowed fans to interact with each other and, in the company’s horror comics, with the titles’ [fictional] hosts. (Pustz “2. EC Fan-Addicts and Marvel Zombies”) Stan Lee and Marvel Comics also adopted the letters columns as a vehicle for establishing relationships with fans. In Fantastic Four #10 (January 1963), Stan Lee opens the “Fantastic 4 Fan Page” with a bold statement advocating a more informal and personal relationship between creators and readers: Hi, fans and friends! Look—enough of that “Dear Editor” jazz from now on! Jack Kirby and Stan Lee (that’s us!) read every letter personally, and we like to feel that we know you and that you know us! So we changed the salutations in the following letters to show you how much friendlier they sound our way! After this proclamation, the salutations in fan mail to Marvel comics were changed from “Dear Editor” to something more personal and informal like “Dear Stan and Jack.” Into the twenty-first century, fan mail features remained a staple in mainstream comics from Marvel, DC, and other major publishers and were also adopted by independent, alternative, and underground comics. However, with the rise of the internet and social media, communication between fans and creators migrated to online discussion forums and platforms such as Facebook and Twitter. In recent years, a retro-nostalgia has motivated a resurgence in comic book fan mail pages, with comics such as Vaughn and Staples’ Saga and Fraction and Zdarsky’s Sex Criminals including many pages of fan mail per issue. Sex Criminals #8 (October 2014) includes nine full pages of fan mail, far more than the half-page, one-page, or two-page letters features one would find in a typical ­t wentieth-century comic book.

2. Methodology Our study employs mixed methods, including text encoding and analysis, topic modeling, and close reading. It is worth noting that the digital methods described below were exceptionally helpful, but not strictly necessary for a corpus of relatively small size like our own. At about 300,000 words, our corpus is roughly the same size as a single Victorian novel like George Eliot’s Middlemarch, and significantly smaller than The Fellowship of the Ring, the first volume of Tolkien’s trilogy The Lord of the Rings. Our fan mail corpus is a minute fraction of data sizes typically associated with Big Data or digital humanities ‘distant reading’ methods (Moretti). Nonetheless, as we will illustrate below, these

64  John A. Walsh et al. methods provide conveniences and efficiencies and support data manipulation and information visualizations that aid in the exploration and analysis of even a relatively small data set. 2.a  Data Collection & TEI Encoding We obtained our data from the DVD title The Amazing Spider-Man: The Complete Collection, which includes PDFs of every issue of ASM from the series beginning in 1963 through 2006. Fortunately for our project, the PDFs contain the complete publication, including fan mail, advertisements, and other paratextual elements that are generally omitted from more recent print and digital reprints. The fan mail pages were extracted from the PDFs and automatically transcribed with FineReader optical character recognition (OCR) software. The results were typically very accurate. For example, we manually checked a random 1000-character sample from the OCR output from ASM #176 (January 1978) against the original PDF and found only three errors—an accuracy rate of 99.7%. A check of a random 1000-character sample from an earlier issue, #8 (January 1964), revealed twelve errors—an accuracy rate of 98.8%. Despite the very high accuracy of the OCR process, obvious errors were corrected during the encoding process, discussed below. Transcribed texts were encoded following the Text Encoding Initiative’s Guidelines for Electronic Text Encoding and Interchange (TEI Consortium). This Text Encoding Initiative (TEI) encoding supports the semantic tagging of the text, with XML elements and attributes, for reuse and analysis. TEI provides tags for encoding structural elements, such as headings, divisions, and paragraphs, and for encoding content of interest, such as titles, personal names, place names, etc. For example, the TEI element represents a generic division of the text, such as a volume, chapter, or section. The @type attribute on the element, provides more specificity, e.g., or . For the ASM data, we created a separate TEI data file for each issue with relevant metadata, such as: issue number, month, year, and the number of pages containing fan mail per issue (0, 1, or 2). We did not distinguish between a partial and a full page. letter”>), The fan mail content was divided into letters (|z|) 0.0000 *** 0.9978 0.8063 0.7721 0.5851 0.0951 . 0.4287 0.8896 0.0110 * 0.0955 . 0.4610 0.9789 0.0151 * 0.0063 ** 0.5025 0.1894 0.0045 ** 0.3043 0.0007 *** 0.0000 *** 0.2413 0.0496 * 0.1584 0.0107 * 0.0185 *

***p < 0.001, **p < 0.01, *p < 0.05, . p < 0.1

Table 7.1  Interaction effects for the generalized linear model relating modification status and pages, with participants and panels as random effects (top). The labels in the left-hand column indicate differences either with respect to the contrasting value of the feature (e.g., to ‘modified’ and ‘nontarget’ in the cases of ‘mod_status’ and ‘Panel_Status’ respectively) and, in the case of the pages, differences with respect to interaction with ‘page 1’ which was selected as reference value.

The results from the durations, pupil diameters, and visits are in many respects comparable, but they do not yet necessarily tell us about any potential narrative consequences of the page composition decisions. Moreover, these measures may also correlate amongst themselves and so may not necessarily be independent indicators either. The variation among pages was also uniformly high: For certain pages, it appears to be

148  John A Bateman et al. the case that the change of status of their gapped panel to a non-gapped panel had significant effects; for some other pages not. Even the directions of effects were not always constant. Further points of triangulation are needed in order to see whether we are indeed on the track of any narratively relevant decisions. To assess this, we drew on the results of the independent ‘pilot’ study as well, in which an independently selected set of participants were asked to evaluate narrative, temporal, causal, and foregrounding differences directly. These results generally confirmed the anecdotal responses reported at the beginning of this chapter: There was an overwhelming tendency to state that the manipulation made some difference. The pairs of pages also varied considerably, however, concerning just how strong that difference was felt to be. As a broad indicator, the factors addressed in the questionnaire concerning judged narrative difference, temporality, and causality were added together to give an informal ‘narrative ranking’ for the 12 pages used in the current study (mean score = 338.8, SD = 36.25). The least narrative difference was given for page 1 (score = 266), which then also served as the reference page in our calculated models above. Interestingly, the five least narratively different page-pairs (3,5,2,8,1), all with below-average scores, correspond to the pages for which no significant effect was found in the final panel visiting model described above. Moreover, the six pages exhibiting statistical significance for their interaction with modification status for panel visits (cf. Table 7.1) were all among the top-ranked pages in the pilot study. This gives a first point of triangulation relevant for our research question. In the eye-tracking experiments, there was at no time any explicit reference to questions of narrative; readers were taken to be reading comics ‘as usual,’ which would normally involve them attempting to ‘follow the story.’ Interactions between modification status (gapped vs. ungapped) and pages appeared to be more likely in just those cases where the manipulation was judged to have stronger narrative effects. However, the interpretation of the coefficients shown in Table 7.1 is by no means unproblematic. First, there is no correlation between the size of the significant coefficients, i.e., the effects of the identified interactions, and the narrative scores of the respective pages. Significance arises due to the relatively narrow confidence intervals calculated for these coefficients rather than their values. Moreover, the direction of the effects is not immediately intuitive. Since all of the coefficients of the interactions with individual pages are negative, they operate to offset the increase indicated by the interaction with modification status. That is, the pages with higher narrative scores reliably gave rise to fewer revisits. This would speak against any hypothesis of the form that narrative might lead to more ‘integrative’ saccades suggesting that readers are bringing various sources of information together.

From Empirical Studies  149 To probe this result more closely, two further models were calculated with the same fixed and random effects shown in Table 7.1, but taking different pages as ‘reference’ value for comparison. The first page taken was page 11, which received a narrative score close to the mean score for all pages. With this page as reference value, only interactions between modification status and pages 1 and 7 appeared significant; modification status alone was not significant. Moreover, the directions of the effects for pages 1 and 7 were opposite to one another: Page 1 increased the number of visits and page 7 reduced the number of visits with respect to the baseline set by page 11. Again, page 1 received a narrative ranking well below average in the pilot study, while page 7 was above average. Moreover, in this case, the interaction coefficients overall ordered the pages analogously, but by no means exactly, to the narrative scores. The second page taken as reference was page 12, one of the highest narrative scorers. The results here were a mirror image of those shown in Table 7.1. Modification status was still (marginally) significant and six pages showed significant interactions with modification status; five of these six were rated below average on their narrative scores. As would now be expected, all of the page interaction coefficients in this latter case were positive while the modification status effect was negative, again indicating that larger narratively relevant differences appeared to correlate with a reduction in visits in the gapped versions. The correlation between the narrative rankings given and the interaction effects involving the gapping manipulation is then quite suggestive, although by no means clear in its implications. Further studies are necessary to take this further. Such studies will need to pay particularly close attention to precisely which pages are selected for manipulation. Identifying cases where readers make different rankings of narrative effects would then offer a further way of achieving more homogeneous sets of stimuli. Our broad hypothesis that the difference in compositional strategy might be reflected in gaze behavior thus remains an open question. Certain pages appear to support such a reading, and others less so. Assumed or suggested differences in narrative effect may also be the indirect result of other compositional differences, such as the presumed tendency in weakly framing compositions to encourage movements between adjacent panels. It will be important for future work, therefore, to characterize data in terms that are more sensitive to such dimensions of potential variation and to collect more substantial and appropriately classified test corpora; a combination of the gridding study above with varied strengths of framing would be one way of addressing this specific question as well. The issue is also naturally raised as to whether we can find further differences across the adopted pages that might help reduce and focus the variation observed. Finding properties of formally gapped

150  John A Bateman et al. pages that mitigate narrative differences would be of considerable use to restrict the kinds of materials sensibly contrasted. Conversely, any differences found in response to gapping/non-gapping pairs might then also be used to suggest cases where readers are making narratively relevant distinctions, and where they are not.

7.  Conclusions and Outlook The difficulties in relating issues of creative design and empirical investigations of reception effects continue to hinder communication between these research traditions. This situation weakens both camps: On the one hand, empirical analyses are deprived of more sophisticated research hypotheses concerning what might be sensibly probed using empirical methods while, on the other hand, more abstract, hermeneutic analyses are deprived of potential support from broader systematic engagement with actual production and reception processes. In this chapter we have approached this challenge by suggesting that the level of abstraction of empirical studies can be progressively raised by employing multi-level annotation schemes to serve as a bridge between ‘surface-near’ perceptual properties of an artifact that may be directly probed and more abstract potentially ‘narratively relevant’ levels of description. Even though we have not reached final answers on these questions, our discussion has nevertheless suggested one way in which this goal can be pursued. With respect to two selected dimensions of variation in page composition, we related these both down to behavioral properties explored using eye-tracking techniques and upwards to considerations of narrative import beyond questions of basic event structure. The need for empirical research of this kind is particularly evident when addressing aspects of a medium that are not well understood—as is the case with the form and function of mise-en-page in comics and graphic novels. Methods to move beyond interpretations of single cases are essential for progress. Our application of the annotation scheme for page composition was therefore intended as an experiment precisely in this direction and, despite the potential confounding influences present due to the wide variety evident in the pages examined, several tendencies appeared that are at least suggestive of narrative strategies or of design effects that could lend themselves well to narrative application. We take this as a positive indication that further refinement of such abstract sources of potential influence, as well as working with larger samples instantiating the abstract classes of compositions investigated, would be beneficial. We also see positive consequences of such empirical studies for annotation design. In addition to the usual questions that must be raised of

From Empirical Studies  151 any annotation scheme concerning its reliability and usability for characterizing data, we can now consider whether differences in annotation can be shown to correlate with measurable reception differences. We might then consider annotation distinctions that correspond to, or correlate with, differences in reading behavior to be more motivated than annotation distinctions for which such evidence is lacking. This does not mean that annotation features for which behavioral effects cannot be found are automatically inappropriate. If such schemes can be reliably applied, then they may still yield valuable information for other purposes, such as indexing data or classifying styles. Nevertheless, we would expect that annotation distinctions that are attempting to make more ready contact with issues of interpretation, narrative, and style should give rise to perceptual differences as well. Finally, Baetens and Frey argue that previous page-based classifications need to be complemented with classifications of panel content (The Graphic Novel, 131). We also see this as a logical and necessary step to take with respect to our own study, since it may well be the case that there are interactions between types of content and the decisions made for traversing the page compositions as a whole. We suspect here that one essential component of any such scheme for content would need to address the internal dynamicity of panel depictions, including accounts of ‘visual vectors’ of various kinds (cf. Kress and van Leeuwen; Boeriis and van Leeuwen). That such vectors are influential in reception generally is well established in design and so it is to be expected that they are at work in comics and graphic novels as well, strengthening or weakening the gaze directions suggested by any whole-page composition strategies. This speaks further to the need to maintain multi-leveled annotation schemes, as argued in Bateman et al., so that further empirical studies can select stimuli for empirical investigation appropriately. Making collections of data available that are already annotated according to such schemes will significantly aid this process.

Acknowledgements We gratefully acknowledge the support of Paulina Burczynska of SensoMotoric Instruments (SMI) concerning use of all aspects of the eyetracker and associated software as well as SMI for agreeing to loan us the eyetracker and software for the duration of the experiments and data gathering. We also thank Jana Pflaeging (Bremen/Salzburg) for helping to perform the experiments and for discussions throughout the design and subsequent evaluation process, and Jochen Laubrock (Potsdam) for invaluable pointers concerning the statistical techniques to employ and potential problems with our analyses and interpretations. Any errors and misunderstandings remaining are our own.

152  John A Bateman et al.

Notes 1 www.thevisuallinguist.com/2014/04/the-visual-language-fluency-index. html. 2 Pages 2 (estimate = 0.311, t = 2.43, CI: [0.0599, 0.5626]), 6 (estimate = 0.379, t = 2.70, CI: [0.1038, 0.6547]), 7 (estimate = 0.277, t = 2.16, CI: [0.0253, 0.5279]), and 10 (estimate = 0.319, t = 2.27, CI: [0.0436, 0.5943]) were significant predictors for increases in the fixation times at the 95% level, the remaining pages were not. The intercept value was 5.91.

Works Cited Baayen, R. Harald. Analyzing Linguistic Data. A Practical Introduction to Statistics Using R. Cambridge UP, 2008. Baayen, R. Harald, Douglas J. Davidson, and Douglas M. Bates. “Mixed-Effect Modeling with Cross Random Effects for Subjects and Items.” Journal of Memory and Language, vol. 59, no. 4, 2008, pp. 390–412. Baetens, Jan, and Hugo Frey. The Graphic Novel: An Introduction. Cambridge UP, 2015. Bateman, John A., Francisco O.D. Veloso, Janina Wildfeuer, Felix HiuLaam Cheung, and Nancy Songdan Guo. “An Open Multilevel Classification Scheme for the Visual Layout of Comics and Graphic Novels: Motivation and Design.” Journal of Digital Scholarship in the Humanities, vol. 32, no. 3, 2017, pp. 476–510. Bateman, John A., and Janina Wildfeuer. “A Multimodal Discourse Theory of Visual Narrative.” Journal of Pragmatics, vol. 74, 2014, pp. 180–218. ScienceDirect, doi:10.1016/j.pragma.2014.10.001. Bates, Douglas, Martin Maechler, Ben Bolker, and Steve Walker. “Fitting Linear Mixed-Effects Models Using lme4.” Journal of Statistical Software, vol. 67, no. 1, 2015, pp. 1–48. doi:10.18637/jss.v067.i01. Beatty, Jackson. “Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources.” Psychological Bulletin, vol. 91, no. 2, 1982, pp. 276–292. Boeriis, Morten, and Theo van Leeuwen. “Vectors.” New Studies in Multimodality: Conceptual and Methodological Elaborations, edited by Ognyan Seizov and Janina Wildfeuer, Bloomsbury, 2017, pp. 15–36. Chavanne, Renaud. “The Composition of Comics.” European Comic Art, vol. 8, no. 1, 2015, translated by Anne Miller, pp. 111–144. Cohn, Neil. “Navigating Comics: An Empirical and Theoretical Approach to Strategies of Reading Comic Page Layouts.” Frontiers in Psychology, vol. 4, no. 186, 2013, pp. 1–15. doi:10.3389/fpsyg.2013.00186. ———. The Visual Language of Comics: Introduction to the Structure and Cognition of Sequential Images. Bloomsbury, 2013. ———. “Visual Language Theory and the Scientific Study of Comics.” Empirical Comics Research: Digital, Multimodal, and Cognitive Methods, edited by Alexander Dunst, Jochen Laubrock, and Janina Wildfeuer, Routledge, 2018, pp. 305–328. Cohn, Neil and Hannah Campbell. “Navigating Comics II: Constraints on the Reading Order of Comic Page Layouts.” Applied Cognitive Psychology, vol. 29, no. 2, 2015, pp. 193–199. Wiley, doi10.1002/acp.3086.

From Empirical Studies  153 Cohn, Neil, Martin Paczynski, Ray Jackendoff, Phillip J. Holcomb, and Gina R. Kuperberg. “(Pea)nuts and Bolts of Visual Narrative: Structure and Meaning in Sequential Image Comprehension.” Cognitive Psychology, vol. 65, no. 1, 2012, pp. 1–38. Fresnault-Deruelle, Pierre. “Du Linéaire au Tabulaire.” Communications, vol. 25, 1976, pp. 7–23. Groensteen. Thierry. The System of Comics. Translated by Bart Beaty and Nick Nguyen. UP of Mississippi, 2007. Hatfield, Charles. Alternative Comics: An Emerging Literature. UP of Mississippi, 2006. Kress, Gunther, and Theo van Leeuwen. Reading Images: The Grammar of Visual Design. 2nd ed., Routledge, 2006. Magliano, Joseph P. and Jeffrey M. Zacks. “The Impact of Continuity Editing in Narrative Film on Event Segmentation.” Cognitive Science, vol. 35, no. 8, 2011, pp. 1489–1517. Magliano, Joseph P., and Lester C. Loschky. “Viewing Static Visual Narratives through the Lens of the Scene Perception and Event Comprehension Theory (SPECT).” Empirical Comics Research: Digital, Multimodal, and Cognitive Methods, edited by Alexander Dunst, Jochen Laubrock, and Janina Wildfeuer, Routledge, 2018, pp. 284–304. McCloud, Scott. “Blood in the Gutter.” Understanding Comics: The Invisible Art, Harper Perennial, 1994, pp. 60–93. Pederson, Kaitlin, and Neil Cohn. “The Changing Pages of Comics: Page Layouts across Eight Decades of American Superhero Comics.” Studies in Comics, vol. 7, no. 1, 2016, pp. 7–28. Peeters Benoît, “Four Conceptions of the Page.” 1998. ImageTexT: Interdisciplinary Comics Studies, vol. 3, no. 3, 2007, pp.  41–60. english.ufl.edu/ imagetext/archives/v3_3/peeters/. Sperber-McQueen, C.M., and L. Burnard, eds. TEI P4: Guidelines for Electronic Text Encoding and Interchange. 4th ed., Oxford, 2002. Tseng, Chiao-I, Jochen Laubrock, and Jana Pflaeging. “Character Developments in Comics and Graphic Novels: A Systematic Analytical Scheme.” Empirical Comics Research: Digital, Multimodal, and Cognitive Methods, edited by Alexander Dunst, Jochen Laubrock, and Janina Wildfeuer, Routledge, 2018, pp. 154–175. Walsh, John A. “Comic Book Markup Language: An Introduction and Rationale.” Digital Humanities Quarterly, vol. 6, no. 1, 2012. Dhq, digitalhumanities.org/dhq/vol/6/1/000117/000117.html. Witek, Joseph. “The Arrow and the Grid.” A Comics Studies Reader, edited by Jeet Heer, and Kent Worcester, Mississippi UP, 2009, pp. 149–156.

8 Character Developments in Comics and Graphic Novels A Systematic Analytical Scheme Chiao-I Tseng, Jochen Laubrock, and Jana Pflaeging 1. Introduction This chapter presents a scheme for analyzing how character developments in comics and graphic novels are signaled to readers. In particular, we focus on how the analytical scheme can be used effectively to address empirical questions in broader cultural contexts, and how the structures (re)constructed on the basis of this scheme can be employed further to empirically test readers’ narrative comprehension process. A considerable body of studies has either focused on narrativity in comics in general (Lefèvre, Mikkonen, Pratt), or has given empirical evidence of comprehension in comics (cf. Cohn, The Visual Language, “The Architecture”; Cohn and Bender). However, the question of how to systematically analyze comic characters and narrative events beyond the analytical unit of panel, and how to address higher-level issues regarding differences between modes and genres of comics have received little attention. This chapter will present an analytical method, which has the potential to bridge the gap between the empirical investigation of a reader’s narrative interpretation process and the systematic comparison of different comic genres and styles. Moreover, this chapter also presents an empirical pilot study and combines the analytical scheme with exploratory eye-tracking experiments and the manipulation of cohesion mechanisms. Two interrelated aspects are addressed in this chapter. First, we deal with the question of how to empirically approach the conventions of comic aesthetics and the features of genre. In particular, we will show how the analytical scheme, which we have developed on the basis of functional linguistics, has the potential to bridge the gap between h ­ igher-level interpretations of cultural conventions and lower-level ­empirical investigation, such as the reader’s comprehension process. S­ econd, we are concerned with the affordances and constraints of different media and materialities. One main focus of this chapter is the comparison between character development in visual-only media, for instance, a silent graphic novel, and conventional graphic novels constructed via visual-verbal

Character Developments in Comics and Graphic Novels  155 coherence. The narrative patterns constructed on the basis of the analytical scheme will reflect the different affordances and constraints of these media. Our approach is anchored in the conceptualization of multiple ­inter-linked levels of meaning construction (cf. Bateman, Elleström). This approach encompasses at least three basic levels, on which Tseng has substantially based her previous research (cf. Tseng, “Analyzing Characters’ Actions”, Cohesion in Film, 2016, “Revisiting Dynamic Space,” ­“Beyond the Media Boundaries”): (1) the pre-semiotic dimensions of materiality at the lower-level, i.e., different modes, whose uses are constrained by the contexts provided by media, (2) the multiple mid-level discourse (semiotic) dimensions that are formulated within each medium and draw on the affordances and constraints of modes and materialities—namely, on the basis of the configuration of modes and materialities determined by media, and (3) higher-level cultural conventions. It is precisely this multi-levelled conceptualization which underlies the formulation of the analytical scheme in this chapter and enables the analytical bridge across media affordances, narrative comprehension, and cultural conventions. The main aim of this chapter is to unravel the narrative mechanisms at work at the mediating middle level.

2.  Tracking Character and Event Developments This section delineates the analytical scheme, which systematically tracks recurring narrative elements, such as main characters, objects and places, and establishes generalized event structures on the basis of the inter-relations between these narrative elements. Our analytical scheme targets event progression beyond the boundaries of the panel. Our assumption is that, although panels often act as a general indicator of how events are segmented spatially and temporally, narrative configurations of people and objects within panels also determine the perception of event developments. The analytical focus of our scheme lies precisely in tracking people and objects (including characters and things) within and across the panel frames. The well-tested empirical approach to event segmentation (Zacks; Zacks and Tversky) relates event chunks to what Zacks and Tversky term “behavior episodes” (“Event Structure”, 6). These episodes can be broken down into actions and movements of people or things. Empirical evidence shows that people’s comprehension of the boundaries between behavior episodes often lies in the dynamic changes of actions. On the basis of a similar analytical focus, our scheme also analyses how the actions, behavior, and movement of prominent people and objects develop throughout comic narratives. It should be also noted here that, although the analytical scheme we present is based on similar cognitive principles of event segmentation, it

156  Chiao-I Tseng et al. differs from the event segmentation approach (Zacks) in that our analytical scheme does not examine the moment-by-moment unfolding of events. Rather, our method focuses on the most narratively significant event structures and the most dominant narrative elements. On this ­basis, we examine the transitions and development of these episodes, or event patterns, which are triggered by changes in the configuration of actions, objects, characters, etc. As we will see in the later sections, establishing generalized event patterns is particularly useful when we aim to conduct a comparative analysis to address higher-level issues. Such analysis requires a certain degree of abstraction for systematic bottom-up comparison. In this pursuit, the implementation of this method encompasses two bottom-up stages: (1) determining the most dominant characters, objects and settings, and (2) investigating how these elements are related to each other. We will exemplify this method by using examples from the graphic novel City of Glass (2004), Karasik and ­Mazzuchelli’s adaptation of Paul Auster’s (1985) eponymous novella. 2.a  Cohesion: Tracking Recurring Objects To systematically track recurring objects, such as people, things and places, we analyze the linguistics-based model of cohesion in comics and graphic novels. This method applies the functional linguistic theory of cohesion developed by Halliday and Hasan (Cohesion in English) and then extended by Tseng (“Analyzing Characters’ Actions”) and Tseng and Bateman (forthcoming) to audiovisual media, including comics. We will illustrate our method in this subsection and combine it with action analysis in the next. In brief, cohesion allows for a systematic empirical investigation of how readers comprehend the presentation and tracking of characters, objects, and places across panel frames. Although there have been other attempts at applying notions of cohesion to comics, these have tended to restrict their accounts to specific components of cohesion and have not explicitly extended cohesion to analyze events or to address cultural issues. The proposal of Stainbrook (“A Little Cohesion”), for instance, focuses almost exclusively on sequential relations between panels in the style suggested by McCloud (Understanding Comics). In contrast, the account of cohesion employed here examines a broader selection of the formal devices by which reoccurrence, repetition, and modification are signaled, in order to obtain maximum leverage on the task of further constructing generalized event structures. Tseng and Bateman (forthcoming) have provided an introduction to the concept and strategies of cohesion, demonstrating its potential for articulating finely-grained theoretical accounts of narrative construction. Their work shows how descriptions of cohesion that anchor narrative construction in the operation of concrete discourse mechanisms can help unpack the narrative complexity of texts that involve more

Character Developments in Comics and Graphic Novels  157 cognitive load for their readers’ narrative interpretation processes. The theory of cohesion deals with the presentation of characters, objects, and settings in coherent narratives and how they may be tracked throughout a graphic novel. Coherence here lies in the progression of cohesive strategies across visual narrative. The cohesive strategies adopted for one element are collected together in order to build cohesive chains. These cohesive chains bind together information about salient characters, objects, and settings across images and texts. Cohesive ties between each (re)appearance of an object provide important cues that guide the viewer along intended paths of interpretation. We provide an example of cohesive structures by discussing the construction of cohesive chains in the first two pages of City of Glass, displayed in Figure 8.1. Paul Karasik and David Mazzuchelli’s (2004) graphic adaptation of Paul Auster’s novel City of Glass (1985) employs a variety of metaphorical images to delineate issues of identity, time, social and family relationships. Therefore, one might expect the text to be cognitively challenging for readers who track recurring characters, objects and places, as well as their interrelations. Our specific goal for this chapter is then to show that, despite this narrative complexity, cohesion

Figure 8.1  Pages 2 and 3 of the graphic novel City of Glass. The red circles indicate visual or verbal references to the protagonist. Source: Reproduced with permission from the Carol Mann Agency.

158  Chiao-I Tseng et al. still offers a rather straightforward analysis of character tracking across visual and verbal elements. As Figure 8.1 shows, the story begins with the description of the protagonist Quinn and portrays his lonely life, his family, and his profession. The cohesive chains established on these two pages are mapped out in Figure 8.2. They track the most dominant elements: Quinn, phone, written work, and apartment. We will start to describe our method for constructing cohesive chains by detailing the strategies used for Quinn. On page 2, in panel 3, Quinn’s identity is presented for the first time as “HE” in the caption text (before his full identity is revealed as “Quinn” in the first panel on page 3). The same identity is presumed/tracked again in panel 4, which explicitly repeats the pronoun “He” in the caption. The cohesive chain starts to build across these two elements. Figure 8.2

Figure 8.2  C  ohesive chains of the dominant narrative elements from pages 2 and 3 of Karasik and Mazzuchelli (2004). Capital letters are text and descriptions in square brackets refer to visual depictions.

Character Developments in Comics and Graphic Novels  159 shows the maintenance of the identity chain with the help of arrows that link successive elements back to previous elements of the same chain. In panels 8 and 9 at the bottom of page 1, Quinn appears as a visual figure, with his foot referring back to the earlier use of “HE.” Cross-modal references constitute an important property of the cohesive framework of visual narratives that distinguishes it from that of the language system. In visual narratives, a character, object, or specific setting may be presented simultaneously or successively in different modes. It is worth noting that our method does not distinguish the two modes at the outset before we investigate their relation. On the contrary, our analytical perspective treats comics as multimodal texts and examines how certain discourse elements (here, the identities of narrative elements) are realized across different modes. The cohesive chain shows the ties built for Quinn: Cohesive strategies such as explicit repetition of images and text are at work to track his identity across the page. In panel 10, Quinn’s name is mentioned for the first time and appears together with the cross-modal reference to his foot and the possessive pronoun “HIS.” The verbal elements “HE,” “HIM,” and “HIS” follow in panels 11, 12, 13, 16, 17, and 18. In addition to presenting and tracking the protagonist throughout the two pages, the cohesive chain also emphasizes how certain panels (10, 14, and 15, for example) use more than three cross-modal elements to portray Quinn’s identity. In Section 4, we will conduct an exploratory study to analyze the extent to which the cross-modality of the Quinn impacts on readers’ narrative navigation processes. Apart from Quinn, who has dominant narrative status, as well as the richest blend of cross-modal realizations of identity, Figure 8.2 shows that three other narrative elements can be identified prominently based on their participation in cohesive structures: A telephone, written works, and the setting of Quinn’s apartment. Each of these elements participates in a cohesive chain made up of a sequence of relations, just as we have seen for Quinn’s cohesive chain. The object chain of the telephone is first introduced cross-modally in panel 1 with the text “TELEPHONE” against a close-up of Quinn’s telephone. The visual presentation of the telephone employs the dynamism of a gradual zoom out portrayed across four panels. In panel 4, the telephone is also cross-modally realized with its sound effect. The chain pattern shows a transition from one object chain to another between panels 6 and 8: The ending of the telephone chain is followed by the opening of a written works chain. City of Glass establishes this second cohesive chain with the help of cross-modal objects—POETRY, ESSAYS, PLAYS, and an image of books sitting on a shelf—which all fall broadly into the same object category of written works. In other words, the relation that holds these elements together is not strictly co-reference but another type of cohesive mechanism, comprising a semantic relation that realizes ties of

160  Chiao-I Tseng et al. similarity. Drawing on this linguistic notion, a semantic relation is established through two types of similarity ties: meronymy, a part-whole relation between two elements, and hyponymy, which refers to elements under a common broader classification. In the present case, POETRY, ESSAYS, and PLAYS are co-hyponyms, and can be considered as falling under the same classification of written work, while the relation between STORY and the images of books denotes a part-whole relation. Finally, panel 7 introduces the third chain, the setting of Quinn’s apartment— through partial, gradual revelation since the floor (and perhaps the bed) initially suggest such a setting without revealing it to be Quinn’s. Its specific identity will be fully revealed in panels 8 to 10, when Quinn gets up from the bed and walks barefoot towards the living room, contextualized by the books, shelf, and wall paper. We have described the elements identified so far as prominent narrative elements. It’s crucial to note that we do not make this selection on the basis of a prior interpretation of the text and its narrative but solely depending on the inclusion (or not) of elements within cohesive chains. Many more elements could have been mentioned but these do not participate in further chains and so may be formally removed from consideration. Thus, other narrative elements that might have been relevant by virtue of their presence in a panel are considered to be parts of the background since they do not participate in cohesive chains. In this way, the focus on participation in cohesive chains serves as a ‘self-­ selection’ device. Any elements that do not reappear in chains are not presented as contributing to the narrative’s development (Tseng, “Cohesive Harmony”). More generally, the chain pattern emphasizes how the comic constructs different kinds of transitions. In the present example, the explicit transition between two object chains signals the thematic change of the story content. The center of attention shifts from Quinn’s ringing phone to Quinn’s writing, which the story portrays in relation to Quinn’s apartment. It is precisely this potential for reflecting how the thematic changes are related to the main character that makes cohesive chains a perfect basis for examining the systematic progression of character development and this can be achieved by examining how the co-patterning of these cohesive chains are realized. 2.b  Structures of the Protagonist’s Behavior Episodes The method for tracking the character’s behavior episodes lies in a combination of cohesive chains and the analysis of action and behavior. The latter focuses on prominently presented static relations and dynamic interactions which interrelate with the cohesive chains of characters, objects, and places unraveled in the previous section. Prominent actions are captured by establishing action chains. The creation of action chains

Character Developments in Comics and Graphic Novels  161 draws centrally on the notion of process types developed within functional linguistics (Halliday and Matthiessen). These process types characterize the very general kinds of activities that are constructed by the grammar of a language: This refers to how any particular language construes activities and events in the world, embedding them within particular configurations of categories deemed to be culturally significant and relevant. They describe, therefore, ‘what is being done.’ Kress and van Leeuwen (Multimodal Discourse) subsequently applied this linguistic notion to visual analysis. Tseng (Cohesion in Film, “Beyond the Media Boundaries”) further extends this concept for the analysis of audio-­visual media and intermedial comparison. The analysis of process types distinguishes perceptible dynamic actions, movements, and interactions. Thus, in Figure 8.1, for instance, William Wilson holds a gun. The character or object who initiates actions may be called the actor, for example, William Wilson and Quinn. These processes have both an agent or action initiator (Wilson and Quinn) and an object at which they direct the action. In contrast to such transactional processes, non-transactional processes only have an agent but lack an object. Thus, we see Quinn walking in panels 8, 9, and 10 in Figure 8.1. In addition to processes with explicit actions, there are also verbal processes when characters are seen speaking (speech bubbles in comics), mental or sensory processes when thoughts or feelings are involved, and a process of looking when characters are seeing and observing something. In the present example, the most prominent action types in the first seven pages are grouped together in Figure 8.3. Drawing on the functional semantic categories of action and interaction types, e.g., types of dynamic, dialogical, gazing actions and interactions between characters and characters’ mental and sensory representations (Tseng, “Analyzing Characters’ Actions”, Cohesion in Film), there are six basic categories of actions which link the characters and objects across these pages. Type (a) refers to verbal processes, capturing action and behavior while characters speak. Type (b) groups together the action of writing. Type (c) is about sensory processes, which depict what Quinn feels, likes, and thinks. Type (d) describes Quinn’s non-transactional action, which are actions initiated by Quinn without connection to other people and things, such as walking, leaving, and going. Type (e) groups together transactional action found in text and images when Quinn holds his son or the telephone. Type (f) describes the ringing of telephone. The purpose of establishing cohesive chains of characters, objects, places, and actions lies in constructing generalized and prominent event patterns based on the interaction of these elements. Figure 8.4 brings together the cohesive identity chains established on pages 2–7 and action chains visualized in Figure 8.3 to describe their development. Taken together, the boxes arranged in the diagram constitute the cohesive chains of these initial pages. The chain begins with phone. This

162  Chiao-I Tseng et al.

Figure 8.3  Action chains in the first seven pages of City of Glass. The numbers refer to the panels and pages in Figures 8.2, 8.4, and 8.5.  The actions in square brackets are depicted as images; caption texts in the graphic novel are type-set in capital letters.

dominant element in the cohesive chain seen in Figure 8.2 interacts with the action type (f): The telephone’s ring sound shown in Figure 8.3. The pattern also highlights the additional objects depicted on pages 4–7: NY/­labyrinth (the labyrinthine quality of New York), boy (Quinn’s son), phone, voice (a stranger’s). The overall event pattern indicates that there are explicit thematic changes in these pages as well. For instance, we see Quinn switching between performing actions of his own, walking, feeling/liking and interactions with other people and objects, such as writing stories, speaking with an unknown man on the phone, and holding his son. Figure 8.4 thus displays a generalized event pattern but also presents a more abstract account of subject matters on these pages. These include Quinn’s feelings as they relate to his interactions with others, including his son and the stranger he talks to on the phone. In sum, this style of analysis shows how particular cues present within comic images can be abstracted to produce generic schemes of more prominent actions, roles within actions, and relations between actions. We could hypothesize that the patterns in Figure 8.4 are typical of the introductory sequences of strongly character-centered/subjective types of visual narratives, such as have been found, for instance, in our previous work on film noir (Tseng, “Analyzing Characters’ Actions”, 85–107). To examine whether this hypothesis can be supported or refuted for comics,

Character Developments in Comics and Graphic Novels  163

Figure 8.4  S chematic representation of action pattern and actor-activity relationships during the initial seven pages in City of Glass.

it will become necessary to apply the method to a larger sample in the future. Nevertheless, to further demonstrate the empirical potential of the method for a broader corpus study, the next section provides a comparative, qualitative analysis.

3.  Application of the Scheme: Aesthetic Comparisons In this section, we will apply our method to the early pages of the silent graphic novel, Dead End (2002) by Thomas Ott. The difference to our earlier example is that the comic is not only wordless but also a different genre, a thriller. Thus, we will see to what degree our event analysis reflects the mediatic, generic, and thematic differences from our analysis in the previous sections. Dead End’s first eight pages are displayed in Figure 8.5. The plot is rather straightforward—it begins with a man driving a car and crashing into a field. While he is dying, two other men discover him and run to the crash site. They discover a suitcase full of cash lying next to the dying man. Instead of rescuing the man, they let

164  Chiao-I Tseng et al.

Figure 8.5  F  irst eight pages of Thomas Ott’s Dead End (2002). Source: Figure details: reproduced/Ott (2002) Dead End, Fantagraphics Books, with permission of Edition Moderne.

him die, take the suitcase, and drive away. While one man tries to bury the suitcase, the other man kills him with a shovel. Drawing on the analytical scheme delineated above, Figure 8.6 maps out a generalized event pattern of the eight pages by interlinking cohesive identity and actions chains. From this event pattern, we can see that there are six highly salient people and objects in these pages: Man 1 (M1), his car, Man 2 (M2), Man 3 (M3), the suitcase full of cash and the shovel used by M2 and M3. There is no location element because no particular setting is specified. In contrast to the event pattern in City of Glass in Figure 8.4, the event structure in this thriller is substantially action-driven. These pages do not feature a clear protagonist with a narrative status comparable to Quinn in City of Glass, who is the only character evolving across the book’s dramaturgical development and performing different kinds of action types. In contrast, there are at least three unnamed men in Dead End, who are all involved in different action types. Among them, we can see how Man 2 (M2) is engaged in most types of actions compared to Man 1 and Man 3. He is the one who looks at his friend while driving and later kills him with a shovel. More generally, we can say that the action element of looking is prominently linked to four out of six object/character elements. This feature could arguably support the main

Character Developments in Comics and Graphic Novels  165

Figure 8.6  A generalized event pattern abstracted from the first eight pages of Dead End.

emotional representations of a silent thriller that consequently lacks verbal and audio cues. Using our analytical scheme to systematically compare how characters are involved in interactions or actions supports higher-level narrative interpretation, such as power differences between characters. Tseng (“Analyzing Characters”, 126–143) analyzed power inequality in Alfred Hitchcock’s North by Northwest (1959) by examining the event patterns of the two main characters. Summarizing, we can say that the comparative analysis of the introductory sequences has shown how our analytical scheme reflects components of genre, aesthetic features, and media affordances, by drawing on the systematically generated bottom-up event patterns. The pattern in City of Glass shows a prominent character element initiating a variety of action types, ranging from dynamic engagement, behavior, interactions with other people and objects, as well as expressing feelings. By comparison, the generalized pattern of the silent graphic novel appears action-oriented, lacking prominent character development. Given the mediatic lack of verbal cues, the silent medium employs a focus on characters’ eyes and where and how they look to communicate narrative interpretation specific to the genre of thriller. The remainder of this chapter presents an exploratory empirical study of Quinn’s cross-modal cohesive chain in City of Glass.

166  Chiao-I Tseng et al.

4.  Further Empirical Investigations: Eye-Tracking Experiments and Questionnaires 4.a  Previous Studies, Rationale and Hypotheses Previous empirical work on the perception of comics and graphic novels, especially in the fields of linguistics and cognitive psychology, has provided valuable insight into recipients’ reading behavior. A relatively large number of studies has focused on attention (e.g., Omori et al.), and specifically participants’ navigation strategies when presented with various types of comic page layouts (see, e.g., Cohn “Beyond Speech Balloons”; Cohn and Campbell). In addition, attempts have been made to chart the complexities of producing and comprehending comics in analogy to the structural constituents and functional relations characteristic of “the organization of a linguistic system” (Cohn, The Visual Language, 1; see also Cohn, “Navigating Comics”). Higher-level phenomena along the lines of visual narrative comprehension have not yet received much scholarly attention, but progress has recently been made to shift such questions into focus (e.g., Bateman and Wildfeuer; Foulsham et al.). With the following empirical explorations, we aim to contribute to this emerging discussion. Given our particular interest in recipients’ character tracking as part of the narrative comprehension process, we manipulated a set of character-based cohesion chains in City of Glass by removing any traces of visual representation, while keeping ­language-based cohesive ties intact. This was done to test for the effect that the presence of visual character cues has on narrative processing behavior and, hence, comprehension. We hypothesize that manipulating the cohesion chains in said way has an effect in this respect. 4.b  Materials & Questionnaire To test recipients’ perception and comprehension of the character development in the graphic novel, we established two conditions by creating two distinct sets of stimuli: The first set, henceforth CoG orig, comprised an original version of the first eight double-pages of City of Glass. To establish the second set, henceforth CoGmanip, we conducted manipulations on all cohesion chains of the graphic novel’s protagonist Quinn; all manipulations were done using the masking, eraser and stamp tools of Adobe Photoshop CS 6. The cohesion structures of the characters Virginia Stillman and Peter Stillman were left intact. The manipulations performed on Quinn’s cohesion chains followed three main strategies that all pursued the same goal: to remove any pictorial traces of Quinn from the panels in question: As for manipulation type 1, any pictorial element representing Quinn was erased from the panels. This was possible in cases in which the

Character Developments in Comics and Graphic Novels  167

Figure 8.7  E  xamples of the three manipulation types as used in stimulus creation. Source: Figure details: City of Glass with permission from Carol Mann Literary.

panel background did not establish any particular setting or objects but merely was either black or white. Figure 8.7 includes an example. As for manipulation type 2, any pictorial traces of Quinn were removed by extending any elements of the setting established. In the example given in Figure 8.7, Quinn’s foot was removed from the image by clone-­stamping parts of the sheets, bed, and floor. As for manipulation type 3, any pictorial element that depicted Quinn was replaced with a contextually plausible object (but not character). In the example in Figure 8.7, Quinn was replaced by a telephone. All manipulations were done with the intention of removing visual traces of the main character, while avoiding any changes that would attract recipients’ attention. The decision for one or the other manipulation type was based on the particular panel design. If a panel showed a character in isolation, without featuring any depiction of background, then the third strategy was implemented (cf. Figure 8.7). In cases like this, the tails of speech bubbles were retained, whereas they were removed in all other cases following one of the three strategies. In addition to eye-tracking experiments, we used questionnaires to empirically test participants’ character tracking as part of their narrative comprehension. Six multiple choice questions were formulated and grouped into two sets: The first question set (Q 1a–c) relates to Quinn and his visual presentations in panels. The second question set (Q 2a–c) refers to ­Virginia Stillman and Peter Stillman. Q 1a Q 1b Q 1c Q 2a Q 2b Q 2c

What kind of hair does Quinn, the main character, have? Is Quinn, the main character, shown naked? Is Quinn, the main character, shown holding a telephone? What kind of hair does Virginia Stillman, the female character, have? Is Virginia Stillman, the female character, shown naked? Is Peter Stillman shown holding a telephone?

168  Chiao-I Tseng et al. These six questions were implemented by a single-choice test with three answer choices each, namely ‘Yes,’ ‘No,’ and ‘I don’t know.’ We expected a high discrepancy in response correctness between readers of the original as opposed to the manipulated version for Q 1a–c, and equal responses to Q 2a–c in the manipulated compared to the original version due to the lack of visual information required to answer the questions about Quinn. Such response patterns could then be regarded as being indicative of an active cross-modal processing of character-based cohesion chains. 4.c  Equipment, Participants, and Procedure The experiment was set up using an SMI Red250Mobile eye-tracker, which was mounted onto the bottom part of a Lenovo ThinkVision L2251p 22″ Wide Flat Panel LCD monitor. The monitor was placed on a desk, positioned at eye-level, and operated using its default resolution of 1680 × 1050. Participants were also provided with a keyboard to start the experiment, navigate between stimuli, and respond to the questionnaire. All components were connected to a laptop. Stimuli were presented using SMI Experiment Center. We conducted subsequent statistical analyses using R within R Studio. The experiments were run in May 2017 at Bremen University. ­Sixtytwo participants took part in the study: Sixty-one undergraduate students enrolled in introductory linguistics classes and one member of staff. Participants ranged in age between eighteen and forty-seven years, with an average age of 21.7 years. Before performing the experiments, all participants completed a questionnaire that collected data on age, gender, education, and potential problems of vision and reading. Moreover, their familiarity with comics and graphic novels based on the “visual language fluency index” was calculated (Cohn et al.; and see Bateman et al., this volume for further information on VFLI scores in the context of the experiments run in May 2017 at Bremen University). Given our hypothesis and the manipulations introduced above, we evenly and randomly distributed participants across two conditions: Thirty-one participants read the original version (CoGorig), while another thirty-one participants read the manipulated version (CoGmanip). Before reading the pages, we introduced all participants to the general procedures of the experiment session and asked them to sit about 50–70 cm from the screen. After calibrating the tracker, participants took part in several short (randomly sequenced) experiments. Participants were informed that they would be shown “double pages from a graphic novel,” and asked to “look at them as you normally would.” They were also told that they could, by pressing , “move to the next page” at their own pace (but after a maximum of 120 seconds) and would be asked a few short questions on what they had seen. In both test conditions,

Character Developments in Comics and Graphic Novels  169 the stimuli were sequenced as in the original. The questionnaire constituted the final part of each session: Both groups of participants were presented with the complete set of six three-answer single-choice questions (cf. Section 5.2), which were displayed in randomized order. 4.d  First Results and Objectives for Future Research Eye-Tracking Data For the target panels indicated above, we analyzed total viewing times and number of fixations on pre-defined areas of interest (AOIs). These areas corresponded to the manipulated aspects or their corresponding area in the original version. We used linear mixed models (as implemented in the lmer and glmer functions in R package lme4, Bates et al.) to compare the effect of the manipulation (original vs. manipulated). Linear mixed models allow for simultaneous statistical control for individual differences between participants and between items, which are considered normally distributed random effects. We also adjusted for visual fluency individually by including a random slope of VLFI. As additional fixed effects predictors, we included the manipulation type (‘simple erase’ vs. ‘background extension’ vs. ‘replacement by plausible object’), the interaction of manipulation and manipulation type, the order of the manipulation in the narrative chain, page number, panel number, and VLFI. Starting from this model, non-significant predictors were subsequently dropped until the Bayesian Information Criterion (BIC) no longer decreased. Only the final model for which a decrease in BIC was observed will be reported here. Since linear mixed models are sensitive to assumptions about the distribution of errors, we log-transformed viewing times and used a Poisson generalized mixed model for the count data in number of fixations. Significance tests are based on the Sattertwaithe correction to the degrees of freedom as implemented in the lmerTest package. All effects were tested at the conventional 0.05 significance level. For log total viewing time on an AOI, the final mixed effects model included fixed effects for condition, manipulation type, and VLFI. We also included random effects of stimulus and participant, and random slopes for VLFI for each participant. Estimates for the random effects suggest that there is more variance due to material (sd = 0.304) than due to participants (sd = 0.212, with a random slope for VLFI of sd = 0.015 and a residual sd = 0.806). The intercept was estimated at b = 6.07, corresponding to an average fixation duration of 432 ms in the manipulated condition with a simple erase-type of manipulation. Most importantly, the fixed effect of the manipulation was significant, b = 0.276, t = 2.21, p = 0.028. Viewing times in the original condition were, on average, 138 ms longer than in the edited condition. This suggests that readers invested time to keep track of visual character cues in the panels.

170  Chiao-I Tseng et al. Another significant effect was observed for the type of manipulation: AOIs in which we had replaced Quinn by a plausible object were fixated longer than the other AOIs in which changes were made either by a simple erase of the character’s visual identity or extending the background. However, this tendency was independent of the panel’s manipulation; That is to say, it also occurred for the original version. It is likely that this is just a selection effect; somehow the panels that we chose for replacement by plausible objects might have been slightly more interesting than the panels chosen for the other manipulations. In the Poisson generalized linear mixed model of the number of fixations, a very sparse model achieved the smallest BIC. Only the manipulation condition had a marginally significant effect, b = 0.131, z = 1.95, p = 0.052. There was a tendency for more fixations on the original AOIs than on the modified AOIs. For completeness, we also report the intercept (b = 0.437) and the random effects. There was more variance in the number of fixations between participants (b = 0.167) than between AOIs (b = 0.123). The results show that our rather subtle manipulations influenced participants’ fixation behavior. We obtained similar results from analyzing questionnaire data. Questionnaire Data At the end of each experiment session, both CoGorig and CoGmanip, we asked participants to respond to a questionnaire comprising six three-answer single-choice questions. Table 8.1 shows the absolute frequencies for each answer option in relation to each question, as well as the p-values using Pearson’s chi-square test (and Fisher’s exact test since we expected low values in some cells) to evaluate the statistical significance of the difference between sets of answers given by CoGorig and CoGmanip participants. The differences between both conditions were all significant at p