Why Digital Displays Cannot Replace Paper: The Cognitive Science of Media for Reading and Writing [1 ed.] 9789811594755, 9811594759

Citation preview

Hirohito Shibata Kengo Omura

Why Digital Displays Cannot Replace Paper The Cognitive Science of Media for Reading and Writing

Why Digital Displays Cannot Replace Paper

Hirohito Shibata • Kengo Omura

Why Digital Displays Cannot Replace Paper The Cognitive Science of Media for Reading and Writing

Hirohito Shibata Fuculty of Social Information Gunma University Maebashi, Gunma, Japan

Kengo Omura Fuji Xerox Co., Ltd. Yokohama, Kanagawa, Japan

ISBN 978-981-15-9475-5 ISBN 978-981-15-9476-2 https://doi.org/10.1007/978-981-15-9476-2

(eBook)

© Springer Nature Singapore Pte Ltd. 2020 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Foreword

Will screens ever replace paper in the workplace, in schools and institutions of higher learning, or for leisure reading? More than twenty years ago, the first purposebuilt e-book readers were introduced to a skeptical marketplace. The early devices were heavy, hard to hold, had limited battery life and poor screen resolution, and were subject to unwelcome new phenomena like glare and inflexible digital rights management. Publishers offered a limited range of material to readers, and libraries struggled with the practical problems of lending e-books to their patrons. In short, e-books were clunky and cumbersome: in spite of developers’ promises, e-books did not give print books and paper documents serious competition. As the years passed, more and more core technologies fell into place. Although Microsoft had introduced a tablet computer five years earlier (a device that met with little fanfare), in 2007, Amazon sold its first Kindle, and in 2010, Apple its first iPad. I got my hands on an iPad right after it became available for sale. I was impressed by the iPad’s screen, but taken aback by its weight. The Kindle was far lighter, but was it light enough to become a third electronic device (an extra burden on top of a laptop and a mobile phone) for consumers to tote with them when they traveled? Yet both devices caught on quickly. The displays—electronic paper in one case, LCD in the other—were finally good enough; batteries lasted; the industrial design was slick and appealing. People finally started reading novels, comics, and magazines on tablets. Was a revolution about to begin? More than just being “paper emulators,” researchers thought e-books would transcend print and open whole new realms for readers. What, though, did readers want to do that they could not when they were reading on paper? Did they want to follow links or look up unfamiliar words or find supplementary material? It turned out that the biggest advantage of e-books was likely to be accessibility: readers could obtain (and indeed carry) an entire library’s worth of books around with them. It became increasingly easy to find bestsellers to read during a vacation or to carry around one’s personal library on a portable device. At the same time, it was difficult to ignore that words presented on a screen still were not the same as words printed on paper. Early research to create a paper-like v

vi

Foreword

experience was successful enough to present at conferences, but effecting a widespread change in reading habits proved to be a more elusive goal. In offices, the paper began to take on a transitional role: it was no longer the recommended way to store a document, but it remained the preferred way to read one. The truth was active reading practices like highlighting and annotation—measures of true engagement with material—lost their unselfconscious quality when readers were faced with these new devices. It was not the same to riffle through a book using a slider nor to go back and forth between pages as it was with a well-placed thumb. As good researchers, we had to ask ourselves why. Study participants have never failed to remind me that some aspects of reading on paper are not so readily replaced by words rendered on a high-resolution screen. Some brought up Proustian qualities: the smell of a book or the places they would like to read as children. Others explained with pragmatic logic that they could not review a journal article as carefully without a pen in hand to scribble in the margins. Still others revealed the shortfalls of e-books only through their actions, as they held a partially turned magazine page between their fingers to make the fluid transition from one page to the next. As Dr. Shibata and Mr. Omura have emphasized, reading always involves more than just the comprehension of dematerialized words passing before our eyes and into our brains: reading, writing, annotating, and drawing all have kinesthetic elements. Dr. Shibata and Mr. Omura’s book makes a detailed and quantitative examination of reading as an embodied, physical act. Through their own lab’s experiments and by consulting the ample literature on reading, they have explored such diverse topics and cognitive mysteries as the differences in understanding subjects’ experience reading on paper and on the screen, how readers (and writers) handle multi-page assignments, the effects of page-turning (a favorite of mine), as well as physiological factors such as eye strain. The results have implications for both researchers and educators: for example, we discover why students appear to learn more when they read on paper than when they use e-textbooks. The aspect of this work that is particularly important is Dr. Shibata and Mr. Omura’s detailed investigation of the affordances of paper, work whose genesis can be traced back to the early 1990s when Xerox PARC researchers—ethnographers, psychologists, sociologists, and computer scientists—began to ask themselves how people used paper documents at work, from post-its to sketchpads to technical manuals. The research presented in this book builds substantively on this body of work, emphasizing new quantitative results that show how the physical aspects of paper contribute to its readability. As the authors conclude in Chapter 5, “the readability of paper is largely due to the ease of handling rather than the ease of viewing.”

Foreword

vii

I am particularly fond of the personal reflections sprinkled throughout the chapters. They leaven the experimental results and literature review with wonderful anecdotes and make reading the book a pleasure. I hope you will find Dr. Shibata and Mr. Omura’s book as thought-provoking and thorough as I did. Texas A&M University, College Station, TX, USA July 25, 2020

Catherine C. Marshall

Preface

“If you are just looking at documents, a PC or tablet is sufficient. However, if you want to read or write documents seriously, you will probably want to work with paper.” We guess that many people feel this way. The aim of this book is to clarify “the greatness of reading and writing with paper.” There have been many books professing admiration for literary reading and writing on paper from a humanistic point of view. However, there are few books that have unraveled the wonder of reading and writing on paper through scientific evidence and numerical data. We wanted to write and publish such a book. Contrary to our initial plan, this has become a book praising paper. We would first like to explain the reasons for this. The authors of this book started work at Fuji Xerox believing in the possibilities of digital media. Our former parent company, Xerox Corporation in the USA, grew by developing photocopying and printing services, but at the same time it also had a vision for the future of digital documents and digital communication. Xerox created many advanced technologies that supported the foundation of today’s information and communication technology (ICT), including the concepts of dynabook and ubiquitous computing. At Fuji Xerox, the authors engaged in research to explore new forms of media with the hope of creating such new technologies. Still young at that time, we believed in the future realization of paperless work or the paperless office. We also had the idea that paper would eventually disappear naturally or else that it would be something we must destroy. We were not fascinated by paper from the beginning. Rather, it was the opposite. In 2007, we published a Japanese translation of the book “The Myth of the Paperless Office” written by Abigail Sellen and Richard Harper. Using an ethnographic approach, Sellen and Harper carefully observed how paper was used in offices and discovered that paper documents played an important role in organizations that could not be fulfilled by digital tools. We also began to do research on reading and writing using paper. We adopted the methodology of using experiments to compare reading and writing performance (e.g., reading speed or accuracy) when using paper and digital media. Comparisons ix

x

Preface

of performance can objectively and quantitatively illustrate the relative merits of paper and digital tools. Moreover, we conducted surveys to investigate how people were using paper and digital media and how they felt about them. The biggest appeal of this book is that it clarifies that the ease of reading from paper is due to the manipulability of paper as a physical object. While reading, people use their bodies, both hands, and all 10 fingers to perform various actions such as arranging documents, turning pages, and pointing to text. The ease of performing these actions greatly supports the ease of reading from paper. Our summaries of these experiments are the main topics of this book (see Chap. 5). Based on these experiments, this book attempts to caution against an excessive or overly hasty transition to digitalization. Human cognitive functions adapt to digital environments. How will this change human reading and writing abilities? Will humans become more intellectual and creative? This is also a problem closely tied to the education and future of our children. Many recent studies indicate negative changes in human cognitive functions caused by the abuse of digital tools. In the future, we expect the emergence of digital media that will support reading and writing better than paper. We believe that the ideal future technologies to support reading and writing should be considered based on an understanding of the reading and writing currently done on paper. We present some suggestions for future digital media for reading and writing. With this in mind, we especially hope that the following types of people read this book. The first type are those who want to gain a scientific understanding of the wonders of reading and writing on paper. The second type are those who want to work more efficiently by selectively using paper and digital tools as needed. The third type are those who are engaged in or interested in the development of new media or services for reading and writing. We hope that by reading this book, these people will be able to reconsider the power of paper in preparation for the wonderful future of reading and writing. This book introduces the results of various experiments. It may be categorized as a specialized book for researchers. However, we made efforts to describe the experiments in plain terms to ensure that anyone, even those without expert knowledge of cognitive science or ergonomics can read it easily. For this reason, there are some parts where we have purposely omitted details. If you would like to learn more, please refer to the references. Maebashi, Gunma, Japan Yokohama, Kanagawa, Japan September 2020

Hirohito Shibata Kengo Omura

Trademarks

• • • • • •

Windows is a trademark of Microsoft Corporation. Adobe Reader is a trademark of Adobe Systems Incorporated. iPad and iPad Pro are trademarks of Apple Inc. Kindle and Kindle Paperwhite are trademarks of Amazon. LIBRIe, Sony Reader, and Digital Paper are trademarks of Sony Corporation. Other company names and product names listed in the book are registered trademarks or trademarks of their respective owners.

xi

Acknowledgments

Much of the research in this book was conducted at Fuji Xerox Co., Ltd. We especially express our appreciation to the following colleagues. The research was launched with the help of late Mr. Kiyoshi Saito, a former managing executive officer, and Mr. Hirotsugu Kashimura, a former director of the Incubation Center. Next, Mr. Yasuhiro Onishi, a former managing executive officer, and Mr. Kun’ichi Yamashita, a former director of the Research Promotion Group, helped us to continue our research. We also thank Mr. Takeshi Yoshioka, a former director of the Key Technology Laboratory, and Mr. Masakazu Okuda, the laboratory’s current director, for their support in publishing this book. Dr. Kentaro Takano, Mr. Kiyoshi Hosoi, and Mr. Naoki Hiji also helped with our research. The inspiration for our research originated from the book “The Myth of the Paperless Office” written by Dr. Abigail Sellen and Dr. Richard Harper of Microsoft Research Cambridge. They helped us to translate their book into Japanese and gave us advice for our research. We also thank Dr. Pernilla Qvarfordt, a former senior scientist at Fuji Xerox Palo Alto Laboratory, Prof. Makoto Omodani of Tokyo Denki University, and Prof. Shun’ichi Tano of the University of Electro-Communications. Ms. Alexa Cowing, a former English specialist at Fuji Xerox, thoroughly checked our English and gave valuable advice. Ms. Shino Watanabe and Ms. Akiko Ishige, English specialists at Fuji Xerox, also kindly checked our English. Ms. Mio Sugino of Springer Japan helped with the publishing of this book as an editor. Finally, we express our special thanks to our families.

xiii

Contents

Reading and Writing in the Digital Age . . . . . . . . . . . . . . . . . . . . . . 1.1 The Current State: A Rapid Change from Paper to Digital Media . . . 1.2 The Convenience of Digital Media in Reading and Writing . . . . . . 1.2.1 The Pleasures of the Kindle: The Convenience of E-readers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 Writing Using Word Processors . . . . . . . . . . . . . . . . . . . . 1.3 Overdependence on the Web Has Changed Human Cognition . . . . 1.4 The Positioning of This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 5 7 10 12

2

Various Media and Their Characteristics . . . . . . . . . . . . . . . . . . . . 2.1 Paper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Definition of Paper . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 Roles and Types of Paper . . . . . . . . . . . . . . . . . . . . . . . . 2.1.3 Physical Characteristics of Paper . . . . . . . . . . . . . . . . . . . 2.2 Digital Media . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Display Panels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Digital Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Limitations on the Use of Digital Media . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

15 15 16 17 18 21 21 23 24 25

3

The Ease of Reading from Paper and the Difficulty of Reading from Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Paper Is the Preferred Medium for Reading . . . . . . . . . . . . . . . . 3.2 Factors Contributing to the Difficulty of Reading from a Display . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

27 27 31 33

Effects of Presentation Quality in Reading . . . . . . . . . . . . . . . . . . . 4.1 Influence on Reading Speed and Understanding Level . . . . . . . . 4.2 Impact on Eye Strain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

35 36 38 40 41

1

4

1 2 3

xv

xvi

5

6

Contents

Effects of Operability on Reading . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Various Types of Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 The Cognitive Load of Operations . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 The Measurement of Cognitive Load . . . . . . . . . . . . . . . 5.2.2 The Cognitive Load of Page Turning . . . . . . . . . . . . . . . 5.3 Overview of Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 Operations Performed While Reading . . . . . . . . . . . . . . . 5.3.2 An Overall Picture of All Experiments . . . . . . . . . . . . . . 5.4 Document Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Cross-Reference Reading of Multiple Documents (Experiment 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.2 Ease of Document Movement and Alignment (Experiment 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.3 Analyzing Window Operation Cost (Experiment 3) . . . . . 5.5 Page Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.1 Reading While Moving Back and Forth Between Pages (Experiment 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.2 Reading to Find Answers Within Text Documents (Experiment 5) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.3 Searching for Photos Within a Photo Book (Experiment 6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6 Content Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.1 Finding Contextual Errors (Experiment 7) . . . . . . . . . . . . 5.6.2 Reading from Paper with Restricted Operations (Experiment 8) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.3 The Position and Orientation of Documents When Reading and Writing (Experiment 9) . . . . . . . . . . . . . . . . 5.7 Mixed Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7.1 Reading for Discussion Purposes (Experiment 10) . . . . . . 5.8 Summary and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.8.1 Paper as an Operation Medium . . . . . . . . . . . . . . . . . . . . 5.8.2 Why Is Paper Easy to Handle? . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ease of Concentration on Reading . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Inability to Concentrate on Reading in a Digital Environment . . . 6.2 Digital Environments Inhibiting Concentration on Reading . . . . . 6.2.1 Distracting Stimuli Unrelated to Reading . . . . . . . . . . . . . 6.2.2 Links with Hypertext That Disperse Thinking . . . . . . . . . 6.2.3 Multifunctional Digital Devices Encouraging Multitasking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.4 The Inconvenience of Performing Operations While Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

43 43 46 46 48 50 50 51 52

.

52

. . .

57 61 64

.

64

.

71

. . .

76 82 83

.

87

. . . . . . .

90 97 97 102 103 105 107

. . . . .

111 112 112 113 114

. 115 . 119

Contents

xvii

6.3

People Pay Attention to Concrete Things in Digital Environments and Abstract Things on Paper . . . . . . . . . . . . . . . . . 120 6.4 Summary and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 7

8

9

Effects of Writing and Drawing by Hand . . . . . . . . . . . . . . . . . . . . 7.1 The Convenience of Writing by Hand and Digital Writing . . . . . . 7.2 Do Word Processors Improve the Quality of Your Text? . . . . . . . 7.3 The Advantages of Taking Notes Manually . . . . . . . . . . . . . . . . 7.3.1 Students Who Turn into Typing Machines in the Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.2 Handwriting Enables Thinking While Inputting Text . . . . 7.4 Hand-Drawing Sketches Is Superior to Using Digital Drawing Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Drawing Tools Require You to Make Decisions Before Drawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.2 Drawing Tools Guide Our Thought in the Wrong Direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Why Are Handwritten Letters Preferred? . . . . . . . . . . . . . . . . . . 7.5.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.2 Evaluation 1: The Personality of Senders . . . . . . . . . . . . . 7.5.3 Evaluation 2: Evaluation of Letter Content . . . . . . . . . . . 7.5.4 Evaluation 3: Feelings of Gratitude . . . . . . . . . . . . . . . . . 7.5.5 General Discussion of the Three Evaluations . . . . . . . . . . 7.6 Summary and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

125 125 129 132

. 133 . 136 . 140 . 141 . . . . . . . . .

142 144 145 146 148 148 150 151 152

Discussion and Proposals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 A Brief Review of the Experiments . . . . . . . . . . . . . . . . . . . . . . . 8.2 Why Digital Displays Cannot Replace Paper . . . . . . . . . . . . . . . . 8.3 Future Media in the Office . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.1 Using Both Paper and Digital Media Wisely . . . . . . . . . . . 8.3.2 Coordination of Paper and Digital Media . . . . . . . . . . . . . . 8.3.3 The Ideal Future Office: The Stockless Office . . . . . . . . . . 8.4 Toward Digital Active Reading . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.1 Personal Experiences of Digital Reading . . . . . . . . . . . . . . 8.4.2 Suggestions for Supporting Digital Reading . . . . . . . . . . . 8.5 Media in Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.1 Concerns About the Digitalization of Education . . . . . . . . . 8.5.2 The Digitalization of Children’s Textbooks Should Be Considered Carefully . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

155 155 156 158 158 161 161 163 164 168 172 172 175 176

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

About the Authors

Hirohito Shibata is a Professor at the Faculty of Social Information, Gunma University (Japan). He received his Master of Science from Osaka University in 1994 and Doctor of Engineering from the University of Tokyo in 2003. He worked for Fuji Xerox Co., Ltd. as a researcher from 1994 to 2000 and from 2003 to 2020. He was also a part-time lecturer at Tokyo University of Technology and Otsuma Women’s University. He was a board member of the Japanese Society for Artificial Intelligence (JSAI). He is a Fellow of the Imaging Society of Japan (ISJ). He is also a vice president at the Electronic Paper Consortium, Japanese Business Machine and Information Systems Industries Association (JBMIA). His specialties are user interface design and cognitive science. He is also interested in supporting human intellectual activities using computers. The research on media introduced in this book has been conducted since 2007. At present, he is working on research to design a new digital medium for reading and writing. Kengo Omura completed his doctoral degree from Department of Psychology, Graduate School of Sociology, Keio University. After joining Fuji Xerox Co., Ltd. (Japan), he worked on studies of cognitive science, document design, and development of document creation support. He is also a part-time lecturer at Kamakura Women’s University. His specialty is cognitive psychology. Books he has written include “Cognitive Psychology Course 2: Memory and Attention” (University of Tokyo Press, 1985) and “Cognitive Psychology Important Study 2: Memory Cognition” (Seishin Shobo, 1996).

xix

Chapter 1

Reading and Writing in the Digital Age

Abstract We first describe the current situation, in which digital media is rapidly becoming widespread in the field of reading and writing and paper media is being pushed aside. Next, we summarize the convenience of digital media and consider why digital technologies are used widely. We discuss e-book readers and word processors as typical examples of digital reading and writing technologies. After that, we introduce the opinions of experts who have pointed out the dangers posed by the fact that digital media dominate the intellectual activity of reading and writing. Finally, we demonstrate our awareness of the current problems, reconfirm the impact of media on reading and writing, and clarify the aim of this book.

The central theme of this book is a comparative study of paper media and digital media in reading and writing. By comparing the characteristics of these two types of media and the behavior of people when they read and write using them, we want to show the advantages and disadvantages of each type of media. In addition, we would like to think about how they can be used selectively, as well as the possibility of their coexistence or integration. We also think that learning from reading and writing on paper will help us build better digital environments for reading and writing. Before entering the main content of this book, we must define some important words. In this book, we consistently use the word media to refer to environments used to present information or “containers” in which to put information in order to display it. Paper is a medium, and a digital device used to present digital information is also a medium (the latter is an example of digital media or electronic media). The term “media” is often used in various other contexts such as “mass media” (news broadcasting), “social media” on the Internet, and “media” in the sense of the format of digital content, e.g., audio or video. To distinguish the word “medium/media” in this book from the other uses, paper and digital media used to present information may be collectively referred to as presentation media.

© Springer Nature Singapore Pte Ltd. 2020 H. Shibata, K. Omura, Why Digital Displays Cannot Replace Paper, https://doi.org/10.1007/978-981-15-9476-2_1

1

2

1.1

1 Reading and Writing in the Digital Age

The Current State: A Rapid Change from Paper to Digital Media

The media used to support intellectual activities such as reading and writing are now rapidly shifting from paper to digital media. This is due to the widespread use of digital devices and mobile information devices that enable reading and writing. Today, almost everyone in Japan owns his or her own smartphone. In the past, many people read newspapers, magazines, and books on commuter trains. Today, most people use smartphones and tablets to read news, watch videos, exchange information with other people via email or social media, and play games. At present, the shift from paper to digital media has progressed in various industries. Let us start with the publishing industry. According to a report by the Research Institute for Publishing in Japan, since its peak in 1996, the market size of the publishing industry had decreased to 52%. Book sales have currently been decreasing every year for 11 consecutive years. Magazine sales have declined year-on-year for 20 consecutive years. A true slump is occurring in the publishing industry. The number of bookstores found in towns has also dropped significantly. According to an article by Asahi Shimbun Digital published on August 24, 2017, the number of bookstores in Japan has decreased to 40% of the number in 2000. The number of local municipalities without any bookstores is also increasing, and now 20% of municipalities in Japan do not have any bookstores within their city limits. Newspapers also show a similar downward trend. According to a report by the Japan Newspaper Publishers and Editors Association, a total of 37.8 million copies of newspapers were issued in 2019 in Japan. Compared to the peak of 53.7 million copies in 1997, this is an approximate 30% decrease over a 20-year period. On the other hand, although the market size for digital publishing, including electronic books and magazines, is still smaller than that of paper publishing (total sales of e-books in 2018 accounted for approximately 16% of the entire publication market), the market has been steadily growing. In the USA, a university library containing no books at all was created in 2013 and is said to be popular with students.1 Reduction of paper is also progressing in offices. Although not as remarkable as in the publishing industry, the consumption of information paper (so-called copy paper) in the office was nearly 20% lower in fiscal year 2018 than it was at its peak in fiscal year 2008. As a move to accelerate this, there have been initiatives to create the paperless office. At IT industry trade shows or events related to environmental conservation, most companies recommend thorough digitalization. Catchphrases promoting the idea of going paperless, such as “streamlined business through paperless work,” “go paperless and change to an eco-friendly smart work style,” and “reduce costs with paperless meetings,” are widespread. 1 Nation’s first bookless library on university campus is thriving at UTSA (http://www.utsa.edu/ today/2013/03/aetlibrary.html).

1.2 The Convenience of Digital Media in Reading and Writing

3

In fact, if you visit an office in which the transition to paperless work has progressed, you can see that the cabinets previously used to store documents have disappeared, the piles of documents on desks are also gone, and the one or two printers installed in the office do not appear to be used much. There is clearly less paper in the office than before, and even if it is not a completely paperless office, it is apparent that the transition is progressing. Recently, robotic process automation (RPA) is also being introduced as a key to work style reforms and business improvement. The fact that the style of work using a pen and paper is increasingly being eclipsed is evidenced by publicized reports that simple paperwork and business processes will soon become automated and will no longer require any human involvement. In the field of education, experiments attempting to introduce digital devices and digital teaching materials are being conducted at elementary schools in preparation for the formal introduction of digital textbooks (De Bonis and De Bonis 2011; Meishar-Tal and Shonfeld 2019). In the field of education, which has a direct impact on the future of our children, there have been some prudent arguments for avoiding rapid computerization (Arai 2012; Wolf and Barzillai 2009). However, there have also been statements that the “children are happy to use digital tools” and that “the law and teachers are the ones posing obstacles to the introduction of digital textbooks.”

1.2

The Convenience of Digital Media in Reading and Writing

The transition from paper to digital media is progressing in various industries and fields. This is due to the convenience provided by digital media. In this section, we concretely describe the convenience of digital reading (using an e-book as an example) and of digital writing (using a word processor as an example). Although some are already well known, we summarize their strengths in preparation for later discussions.

1.2.1

The Pleasures of the Kindle: The Convenience of E-readers

Many people are familiar with the e-reader sold by Amazon known as Kindle. The first-generation Kindle was released in 2007, and there are currently four models on the market offering different functions. Although its weight differs depending on the model, the Kindle is about as light as a paperback book (between 131 and 217 grams), making it easy to hold with the one hand. All Kindle models feature

4

1 Reading and Writing in the Digital Age

an eye-friendly electronic paper display (described in detail later in this book), and you can turn the pages by touching the screen. Along with the release of the Kindle, Amazon also launched the Kindle Store, an online bookstore selling downloadable electronic books (so-called e-books). Currently, over one million books are available for download (either free or paid) from the Kindle Store. Kindle provides users with various benefits that are impossible to gain through paper books, including the following: 1. You can buy the desired books anytime and anywhere. To download an e-book you want to read, you simply need to connect to the Kindle Store from your device, perform a search for the book, and purchase it. After downloading the e-book, you can begin reading it immediately. There is no need to go to a bookstore or library or to worry about their business hours. It greatly reduces the time and effort required to obtain books. 2. You can store and carry many books either on your device or in the cloud. Thousands of e-books can be stored in a Kindle device. Because you can carry many books in a small device, you do not have to worry about which book to take with you when you leave the house or go on a trip. In addition, there is also a free service that stores your downloaded books in the cloud when they do not fit on your Kindle device. Since you no longer have a need to store physical books, you can conserve space in your home or office. 3. Searching is easy. Using a Kindle, you can perform a search to find out where a specific word or expression is used in a book. This is useful when you want to re-read a particular section of a book or check how a certain word was used. 4. A dictionary function is provided. If you select a word whose meaning you do not know, the Kindle refers to a built-in dictionary and displays a popup window showing the definition of the word. It is also possible to jump to Wikipedia, an online encyclopedia. If you use the function called Word Wise, the Kindle automatically searches for all words of a certain level of difficulty and displays their meanings (using simpler terms) in small text above each word. This eliminates the need to select a word and wait for the popup window to be displayed each time. 5. You can change the text size, typeface, and layout of a book. You can adjust the text size according to your own preferences. We have heard that this feature is useful for elderly people who have grown farsighted and it allows them to enjoy reading once again. In addition to switching between standard typefaces, you can also import your favorite font and read books in it. 6. You can read even in the dark. By adjusting the brightness with the Kindle’s front light function, you can read books even in a dark room. Furthermore, since it is unnecessary to use both hands to hold the pages open, it is more comfortable for reading in bed before going to sleep. (However, I learned a lesson about the safety of paperback books when I was reading in bed and accidentally dropped the Kindle on my nose.) You can also use hyperlinks to jump from the table of contents to desired pages, view a list of highlighted text, make the Kindle read text aloud, ask it to recommend

1.2 The Convenience of Digital Media in Reading and Writing

5

other books that you might be interested in, share your reading experiences with others, perform so-called social reading, and so on. The Kindle provides various functions that cannot be realized with paper books. Moreover, e-books are usually sold for a lower price than their paper counterparts. Some of these benefits are unique to the Kindle, but many are common to other e-book readers and services. Merkoski (2013), who participated in the development of the first-generation Kindle, wrote the book Burning the page: The eBook revolution and the future of reading. He pointed out that the Kindle was a device made to promote a shift from the era of reading paper books starting from their beginning (Reading 1.0) to an era of enjoying e-books while making use of their rich digital features (Reading 2.0). In his book, Merkoski predicted several possible future directions of e-books that would take advantage of digital features. He also wrote that, rather than the pursuit of convenience or economization, e-books would bring about a different kind of experience from the act of “reading a paper book.” The current Kindle is designed to be close to a paper book in look and feel. In a sense, its design is conservative although it incorporates the merits unique to e-books described above. However, e-books that move away from the style of reading traditional paper books, as Merkoski predicted, are already on the market. For example, in addition to static information such as printed words, photographs, and illustrations, these books dynamically integrate multimedia content such as video, audio, and sound. Moreover, some of these books allow their design or content to be customized in accordance with the reader’s preferences and interests. This is accomplished by changing feedback content, the types and amount of content, and the design based on the reader’s reactions or responses. Furthermore, there are even novels in which the story, or even the ending, may change each time the user jumps via a hyperlink. These go beyond the scope of books and will lead to new experiences that are different from reading paper books.

1.2.2

Writing Using Word Processors

A word processor is a piece of software used for creating, editing, and printing documents. One of the most famous examples is Microsoft Word. Why have word processors become so widely used? Although you are probably familiar with the features of word processors, let us look at the benefits offered by their basic functions. 1. Word processors make it easier to edit text. With a word processor, you can easily edit, delete, correct, insert, move, and copy text. Therefore, it is not necessary to write perfect sentences from the beginning; rather, you can begin writing from the parts that are easy for you to write and then edit the sentences later to make the writing more complete. Noguchi (2002), a former professor at the University of Tokyo, pointed out that thanks to word processors, writers can

6

1 Reading and Writing in the Digital Age

start writing from anywhere they like without worrying about the order of the sentences, which has lowered the psychological barriers to beginning writing. 2. You can create multimedia documents. You can create documents containing not only text but also images, charts, and graphs. Using these visual representations, you can create easy-to-understand, persuasive documents. 3. Word processors make it possible to change the layout and design of documents. You can create well-formatted, attractive documents using pre-designed document templates. Anybody can be an editor or publisher using word processors. 4. Functions that support document revision are available. Advanced word processors can check text for spelling errors or simple grammatical errors and correct them. Features that track the changes made to a document are also useful for collaborative document editing. In addition to these features, for the input of Japanese characters, we need to mention another important feature of Japanese word processor. To input a Kanji character (Japanese characters of Chinese origin), we must input its pronunciation (using characters called Kana) using an alphabetical keyboard. Then, the word processor displays a popup showing a list of candidate Kanji characters with the same pronunciation (this software is called Kana-Kanji conversion software) and we must select the right one from the list. In other words, even if we cannot remember how to write a certain Kanji correctly, we can input it as long as we can recognize the Kanji. Because Kanji usually have many strokes and are difficult to remember, the Kana-Kanji conversion software saves people the trouble of having to remember and write difficult Kanji correctly. We often hear complaints that many Japanese people today have forgotten how to write Kanji due to the prevalence of this function. However, for many people, it seems that the convenience of being able to easily input the correct Kanji outweighs the inconvenience of not being able to write the Kanji by hand. We often hear people say, “I cannot write text at all without a word processor.” In fact, the authors of this book feel this way as well. Word processors have become a familiar and indispensable tool for our writing. New techniques for inputting text have also become popular. Voice input is a function that allows a computer to fulfill the role of transcribing a dictation. The software recognizes human speech and converts it into text. Using voice input, you can avoid causing fatigue to your hands. Voice input is also useful for people who are not skilled at typing as well as for the visually impaired. Additionally, digital handwriting is also becoming prevalent. This is a tool for writing and drawing characters or figures on a display panel with a digital pen or stylus, just like writing in a paper notebook with an analog pen. You can not only write letters (or draw pictures) by hand but also add handwritten comments to a digital document. Some note-taking applications are able to recognize handwritten characters and convert them into typed text. At present, there are still problems with the ease of writing and the accuracy of handwriting recognition, but these are likely to improve over time.

1.3 Overdependence on the Web Has Changed Human Cognition

7

With the development of artificial intelligence (AI), we may be able to use more advanced technologies that can automatically produce easy-to-read text and create stories based on themes or partial materials provided by humans. However, these technologies are currently still in the research stage.

1.3

Overdependence on the Web Has Changed Human Cognition

As mentioned in the previous section, digital environments for reading and writing provide a great deal of convenience that cannot be realized with pen and paper. However, is something being lost in exchange for this convenience? If the convenience has caused changes in human cognitive function, this becomes an irrevocable problem. McLuhan (1964), a philosopher of media theory, argued that new forms of media change people’s ways of thinking. Human cognitive functions adapt to digital environments (Greenfield 2015). How will this change people’s reading and writing skills, that is, human intellectual abilities? Will humans become more intellectual and creative? Many researchers have pointed out that the advent of the Web has significantly affected the ways in which modern people read (Liu 2005; Jamali et al. 2009). Liu (2005) of San Jose State University performed a study surveying 113 people in the USA, including workers between 30 and 45 years of age as well as graduate students. The participants were asked how their experience of reading had changed over the past 10 years. Figure 1.1 shows the results. Participants in the study stated that their percentage of time spent browsing (glancing over only the interesting parts of text without a definite purpose), scanning (glancing over the text in order to find something), keyword spotting (used to pick up important information), one-time reading (reading text only once without re-reading), reading selectively (reading only specific parts of the text), and non-linear reading (jumping between pages via links) had greatly increased compared to 10 years before. On the other hand, their percentage of time spent on sustained attention (reading while maintaining their concentration for a long time), intensive reading, and concentrated reading had decreased compared to 10 years before. Overall, the frequency of fragmentary shallow reading had increased, while the frequency of deep reading to gain a deep understanding of the content had decreased. This change is not very surprising. The results seem to reflect the fact that, at the time of the 2003 survey, people were presented with many more opportunities to browse information using the Web than before, and conversely, their opportunities to read paper books had decreased. The shifts in the ways people read directly reflected the differences between the media characteristics of the Web and paper books.

8

1 Reading and Writing in the Digital Age

Fig. 1.1 Changes in reading due to the spread of digital media: Comparison with 10 years before (created from data of Liu 2005)

Browsing Web pages and reading paper books are done for different purposes, and naturally they involve different methods of reading (that is, methods of processing text). Using the Web is mainly done in order to search for and obtain information that gives the answers to questions. An in-depth reading is not necessary for the task of finding the answer to a question. What is the problem with this? Let us introduce some opinions of researchers and journalists. In a paper titled “From thinkers to clickers,” Thirunarayanan (2003) pointed out the dangers of overdependence on the Web. He wrote, “As interactions with the Web increase, the clicking and wandering behavior gets more deeply entrenched among human beings. Such aimless cyber wandering eventually becomes a substitute for meaningful thinking.” He says that reading books and browsing the Web are fundamentally different actions. The Web takes the action of thinking away from humans, he concludes, and continues as follows. The simple printed book is much more conducive to promoting thinking than the sophisticated Web, he explains. If a book does not provide all the information that you need, some of the missing information must be deduced by yourself, and some of it must be imagined. In other words, your thinking supplements the book’s deficiencies. When you cannot get an answer to a question by reading one book, you must read a second or third book to find the answer. The book is a slow medium, in some ways. By the time you find a book that gives the answer to your question, you have the time to think about it more thoroughly by yourself and perhaps refine the question even further. On the Web, it is entirely different. Clicking is prioritized over thinking. Imagine a situation in which you read a report and have a question related to what you just read. The answer to that question can be obtained just by clicking. The nature of the

1.3 Overdependence on the Web Has Changed Human Cognition

9

Web, in which you can obtain answers immediately, does not serve as a tool to promote thinking. Why would people take the time to think and try to come up with answers to their questions by themselves when they can get answers instantaneously with a single click (Thirunarayanan 2003)? There is a study showing that relying too strongly on the Web has negative effects on human memory. Sparrow et al. (2011) of Harvard University have demonstrated through experiments that people tend to reflexively rely on the Web instead of thinking when they are faced with difficult problems, and that when they find information on the Web, they tend to memorize how to search for that information again later instead of attempting to memorizing the information itself, which the researchers labeled the Google effect. If people know that information is saved somewhere outside their brain, they tend to memorize its location (that is, how to search for it using a computer) without trying to memorize its content. This is like saying, “I don’t know the answer, but I know someone who knows.” We cannot blame this. However, if your knowledge is all pointers to other knowledge, the information processing inside the brain becomes superficial. People can refer to the information anytime they want. However, it becomes difficult for them to resolve problems or to create something using the pointers. There are other problems. As shown by Liu’s study, excessive browsing of the Web has dramatically reduced people’s opportunities to read paper books in-depth. If, as a result of this, people become unable to read in-depth even when they need to, it becomes a problem too big to ignore. The problem would become more serious if children were to grow up without the experience of reading in-depth. Carr (2010), a journalist and the author of The shallows: What the Internet is doing to our brains, confesses that he feels this happening in himself: I’m not thinking the way I used to think. I feel it most strongly when I’m reading. I used to find it easy to immerse myself in a book or a lengthy article. My mind would get caught up in the twists of the narrative or the turns of the argument, and I’d spend hours strolling through long stretches of prose. That’s rarely the case anymore. Now my concentration starts to drift after an age or two. I get fidgety, lose the thread, and begin looking for something else to do. I feel like I'm always dragging my wayward brain back to the text. The deep reading that used to come naturally has become a struggle. (Carr 2010)

As described above, the main reason for such problems seems to be that people have become accustomed to a digital environment in which they can easily search for answers on the Web, and as a result, their skills of thinking and deep reading have declined. It is also reported that people who frequently use text messages or social media were less likely to engage in reflective thought (Annisette and Lafreniere 2017). However, are there any factors in digital media itself which promote shallow reading? This is a problem related to the design of media. If the current digital media have factors that impede reading, no matter how deeply people try to read, that goal will never be achieved. In fact, there are many claims that digital media interfere with deep reading or make the reading experience shallow. In an essay published in Time magazine in

10

1 Reading and Writing in the Digital Age

2012, Szalavitz (2012) recounted her experience of reading mystery novels with an e-book reader. She said, “. . . I soon found that I had difficulty recalling the names of characters from chapter to chapter. At first, I attributed the lapses to a scary reality of getting older — but then I discovered that I didn’t have this problem when I read paperbacks.” She also noted that Larry Page, the founder of Google, said that reading things on a screen is slower than reading on paper. Sakai (2011), a linguistic brain scientist at the University of Tokyo, points out in his book entitled “Reading to Create a Brain” that readers’ impressions of a book are completely different when they read it as a paper book and as an e-book. He said in his book, “Paper books are rich with clues to remind me of my past experiences, which helps me to vividly remember my impressions of the books I have read—for both books I liked and books I disliked. No matter how faded the book cover is, I can clearly remember the original color and design of it in my mind. If it were an e-book, I don’t think I would have such a feeling at all.” These views are all based on personal experiences, and we have some research questions. Can we explain these phenomena scientifically? Under what circumstances can such problems of digital media be observed, and why? In order to be able to select the appropriate form of media depending on the situation and make improvements to digital media for reading and writing, it is necessary for us to find the answers to these questions.

1.4

The Positioning of This Book

As I mentioned at the beginning, the transition from paper to digital media is currently a long-term trend. The benefits of computerization and digitalization seem to be obvious to anyone at first glance and promise a bright future. However, at the same time, it seems that people are oversimplifying the story. This is to say that, because of the great benefits of digitalization, its problems are being overlooked (and conversely, the role of paper is being neglected). Many people think that it is preferable to replace all paper with digital tools and digital data, and that a “smart” work style is one that does not use paper. However, we think that the following two problems exist regarding the trend towards indiscriminate digitalization. The first problem is that, whenever paper and digital media are compared, only the functions that were not provided by paper are emphasized, and the effects of media on reading and writing performance and their impacts on work style are not considered. When paper resources are replaced with digital media, the methods and processes of reading and writing usually change accordingly. Furthermore, this shift also affects people’s reading and writing performance, such as the speed and accuracy of their reading and writing, in addition to their memory retention. This is obvious, because the point of contact between humans and book content has changed. However, we think that this point has hardly been taken into consideration in the movement toward digitalization in society. In other words, many people, especially those who promote digitalization, ignore or underestimate

1.4 The Positioning of This Book

11

the risks and costs that replacing paper with digital media may have on human cognitive abilities, work styles, and work environments. The second problem is that optimistic social ideas hoping for digitalization (and conversely, paper advocates who warn of excessive digitalization) concentrate on rough and qualitative arguments, and a scientific point of view is often lacking. As the scientific point of view, we think the following three points are important. First, a micro-level analysis is necessary. We need to narrow down the purpose and circumstances of reading or writing. Reading and writing are performed for various purposes, and they occur in various situations. It is meaningless to lump all types of reading and writing together when discussing whether paper or digital media are better. If we do not clarify the point in question, the disjointed debates will last endlessly. Next, once the intended purpose and situation of reading and writing are defined, objective data acquisition is required. It is necessary to create controlled experimental settings and conditions based on the intended purpose and situation. Finally, quantitative analysis is preferable. If we create a controlled experiment to determine which is better, paper or digital media, it is desirable to clarify the differences quantitatively using numerical data. There is no doubt that personal experiences and preferences can also be important factors in decision-making, but it is preferable that serious decisions concerning the futures of organizations and countries be determined based on scientific methods. We just described how the science should be, but these are all essential requirements for comparing multiple things, and this is basic knowledge that every scientist knows. Despite this, in the case of paper, scientific analysis from the user’s point of view had rarely been discussed until recently. We think this is because paper maintained its superiority as a medium for reading and writing that no rivals could emulate for a long time (more than 2000 years). In other words, because paper reigned as the absolute king of reading and writing media for so long, objective or quantitative comparisons were not necessary. But now, with the advent of digital media, the superiority of paper has been threatened. We can say that it is now time to analyze the value and utility of paper and digital media scientifically. As we mentioned at the beginning, the aim of this book is to clarify “the greatness of reading and writing on paper.” To achieve this, this book mainly uses the methodology of experimental psychology. We introduce comparative studies of paper and digital media, focusing on the research and experiments we have conducted and of course referring to other excellent research results. Based on these findings, we describe the merits and demerits of paper and digital media for reading and writing as fairly and cool-headedly as possible. Although digital media have various advantages, there are already many good books that explain the value of digital media, and such value has already gained general recognition. Thus, this book focuses on the appeal of paper that has often been overlooked in favor of the benefits of digital media. Understanding the nature of reading and writing on paper brings two important messages. As the first message, we want to warn against going paperless indiscriminately. The overdependence on the Web has changed our cognitive functions. As

12

1 Reading and Writing in the Digital Age

you will learn later in this book, in some aspects, the use of digital media can deteriorate our reading performance (see Chap. 5), interfere with our concentration (see Chap. 6), or make it difficult to think while inputting information (see Chap. 7). We are concerned about negative influences resulting from indiscriminate digitalization. We intend to propose the ideal future work style for the use of media. We hope that we can give readers an opportunity to re-evaluate the appeal of paper and reconfirm the importance of paper’s role as a medium for reading and writing. As the second message, we want to present suggestions for designing digital environments for reading and writing. So far in this chapter, we have emphasized the trend of shifting from paper to digital media. However, paper has not been completely surpassed by digital media. Above all, paper is overwhelmingly preferred in some situations such as reading documents deeply or creating ideas while drawing (Kazanci 2015; Baron et al. 2017). In these situations, digital media is struggling to replace paper. The power of paper (and pen), which has supported human intellectual activities for many years, does not seem to be so weak. However, the advantages of computers are still very attractive. They include fast computing power, the ability to change expressions dynamically, the delivery and sharing of information via networks, and searching and learning from large amounts of information. It is enough to expect that computers have the potential to support human intellectual activities beyond pen and paper (Shibata 2020a, b). We hope this book gives new directions for research in pursuit of future digital media for reading and writing.

References Annisette, L. E., & Lafreniere, K. D. (2017). Social media, texting, and personality: A test of the shallowing hypothesis. Personality and Individual Differences, 15(1), 154–158. Elsevier. Arai, N. (2012). Honto ni iino? Dejitaru kyoukasho. Iwanami Shoten, Publisher. [in Japanese]. Baron, N. S., Calixte, R. M., & Havewala, M. (2017). The persistence of print among university students: An exploratory study. Telematics and Informatics, 34(5), 590–604. Elsevier. Carr, N. G. (2010). The shallows: What the Internet is doing to our brains. W.W. Norton & Co. De Bonis, S., & De Bonis, N. (2011). Going green: Managing a paperless classroom. US-China Education Review, A (1), 83–87. Greenfield, S. (2015). Mind change: How digital technologies are leaving their mark on our brains. Random House. Jamali, H. R., Nicholas, D., & Rowlands, I. (2009). Scholarly e-books: The views of 16,000 academics: Results from the JISC National E-Book Observatory. Aslib Proceedings, 61(1), 33–47. Kazanci, Z. (2015). University students’ preferences of reading from a printed paper or a digital screen: A longitudinal study. International Journal of Culture and History, 1(1), 50–53. Liu, Z. (2005). Reading behavior in the digital environment: Changes in reading behavior over the past ten years. Journal of Documentation, 61(6), 700–712. McLuhan, M. (1964). Understanding media: The extensions of man. McGraw-Hill. Meishar-Tal, H., & Shonfeld, M. (2019). Students’ writing and reading preferences in paperless classroom. Interactive Learning Environments, 27(7), 908–918. Taylor & Francis.

References

13

Merkoski, J. (2013). Burning the page: The eBook revolution and the future of reading. Sourcebook. Noguchi, Y. (2002). Cho-bunshou-hou. Chuokoron-Shinsha, Inc. [in Japanese]. Sakai, K. (2011). Noh wo tsukuru dokusho. Jitsugyo no Nihon Sha, Ltd. [in Japanese]. Shibata, H. (2020a). Toward the special issue on “Beyond-Paper Interaction”. The Journal of the Imaging Science of Japan, 59(2), 202–203. [in Japanese]. Shibata, H. (2020b). Cognitive science of media and reading-and-writing. The Journal of the Imaging Science of Japan, 59(2), 204–211. [in Japanese]. Sparrow, B., Liu, J., & Wegner, D. M. (2011). Google effects on memory: Cognitive consequences of having information at our fingertips. Science, 333(6043), 776–778. Szalavitz, M. (2012). Do e-books make it harder to remember what you just read? Time, March 14, 2012. Thirunarayanan, M. O. (2003). From thinkers to clickers: The World Wide Web and the transformation of the essence of being human. Ubiquity, 2003(5), ACM. Wolf, M., & Barzillai, M. (2009). The importance of deep reading. Educational Leadership, 66(6), 32–37.

Chapter 2

Various Media and Their Characteristics

Abstract Paper has been used as a medium for reading and writing for more than 2000 years. Contrasted with paper are digital media such as computer displays, tablets, smartphones, and e-book readers, which emerged in the latter half of the twentieth century. This chapter gives an overview of various media and explains their physical and functional characteristics in preparation for later discussions. First, we list up various functions of paper and clarify our target, paper for reading and writing, which consists of newspaper, printing paper, and information paper. Next, for digital media, we differentiate between display panels and display devices. Regarding display panels, we cover LCDs, organic EL displays, and electronic paper panels. Regarding display devices, we cover desktop PCs, laptop PCs, tablet PCs, and smartphones.

2.1

Paper

First, we would like to start with paper. Since reading and writing on paper is an important topic of this book, let us describe the role and nature of paper in some detail. It is said that the oldest paper in the world was made in China in 170 BC.1 Since then, paper has spread as a medium that has been indispensable to our lives over the years. In our daily lives and work, we are surrounded by many paper products such as books, magazines, newspapers, postcards, flyers, tissue paper, toilet paper, wrapping paper, milk cartons, cardboard boxes, and so on.

1 Paper is considered to be one of the four great inventions in China, along with printing, the compass, and gunpowder. There are several theories on the origin of paper. In the year 105, Cai Lun of China made paper from raw materials including wood, hemp, hemp rags, and fishnets, and gave it to the emperor. This paper is known as Cai Lun Paper. However, it has been revealed that paper was made in China before this, and the oldest example is thought to have been made in 170 BC (de Biasi 2006).

© Springer Nature Singapore Pte Ltd. 2020 H. Shibata, K. Omura, Why Digital Displays Cannot Replace Paper, https://doi.org/10.1007/978-981-15-9476-2_2

15

16

2 Various Media and Their Characteristics

Although we feel a strong sense of familiarity with paper through our daily lives, we do not know as much about paper as we do about the benefits of paper. What is paper? Let us start with our discussion from the definition of paper. Next, we discuss the types and roles of paper, the materials from which paper is made, and the physical characteristics of paper.

2.1.1

Definition of Paper

The origin of the word “paper” is papyrus, which was used in ancient Egypt and ancient Greece from around 3000 BC. Papyrus is a tall grass that grows in Egypt. In order to make a sheet for writing, its stems are opened, pounded, and stretched, and then several strips are pressed and fused together vertically and horizontally to form a paper-like sheet. On these sheets made of papyrus, people were able to write various pictures and symbols. It allowed people to record and “store” information, made it easy to transmit and exchange information, and contributed to the development of civilization. The German word for paper is “Papier” (pronounced /pa’piːe/). In French, the word for paper is also “papier” (pronounced /papje/). The origin of the word in both languages is the word papyrus. We can understand the importance of the role played by papyrus in the storage and transmission of information. However, as we will see later, papyrus is not paper. According to the Japanese Industrial Standards (JIS), the definition of “paper” is “that which is made by adhering plant fibers or other fibers. In a broad sense, it also includes synthetic paper manufactured using synthetic polymer substances as materials, as well as that made from fibrous inorganic materials” (JIS P 001 4004). To make paper, people must extract sticky fibers (known as pulp) from plants,2 either by crushing them or by using chemicals, and then tangle them together to create a thin sheet. According to this definition, papyrus, which is used in the form of sheets made by opening the stems of plants without breaking the plants down into fibers, is not true paper. Rather, in a broad sense, any sheet made from any material other than plants can be called a sheet of paper as long as intertwined with fibrous materials to form a sheet. In fact, there also exists paper made from materials such as stainless steel, glass, polyester, or limestone.

2 The main ingredient of paper is wood pulp. Generally speaking, conifer trees are used to make tough paper because their fibers are long. Typical examples of trees used as raw materials for paper are cedar and cypress. In contrast, deciduous trees have finer fibers. Therefore, they are used to make paper with a fine surface. Typical examples of such trees are eucalyptus and acacia.

2.1 Paper

17 Japanese-style paper

Newsprint paper

Printing paper (Magazines, gravure paper, postcards, etc.)

Paper

Paper

Information paper (Copy paper, form paper, etc.)

Sanitary paper (Tissues, toilet paper etc.)

Western-style paper

Packaging paper Paperboard

Fig. 2.1 Various paper types

2.1.2

Roles and Types of Paper

The functional roles of paper can be divided into three types: writing, wiping, and wrapping, or the “three W’s.” “Writing” is the function of paper to record, store, and transmit information. Since writing is also done for the purpose of being viewed later, this also includes the functions of viewing and browsing. Typical examples of paper with this function are newspapers, magazines, books, letters, flyers, etc. “Wrapping” is the function to wrap objects. Typical examples of paper with this function are giftwrap, product packaging, paper bags, cardboard boxes, etc. “Wiping” is the function of paper to wipe off dirt and moisture. Typical examples of paper with this function are tissues, toilet paper, disposable diapers, etc. Among these three roles, this book deals with the “writing” function of paper. Next, let us classify paper from the product point of view to understand the position of paper used for “writing” within the entire spectrum of paper products. First, paper can be broadly divided into Japanese-style paper and western-style paper. Today, Japanese-style (washi) paper is used for traditional Japanese sliding paper panel doors called shoji, calligraphy sheets, arts and crafts, etc., and is considered to be something of a luxury item. Most paper produced in Japan today is western-style paper, whereas Japanese-style paper accounts for only 0.3% of all paper produced in Japan. With this in mind, Japanese-style paper is not our concern in this book. As shown in Fig. 2.1, western-style paper can be further classified into the categories of paper and paperboard. In general terms, the thin, light varieties of western-style paper fall into the category of “paper,” and thick, heavyweight varieties are categorized as “paperboard.” More specifically, paperboard is differentiated from other types of paper based on its weight per square meter (i.e., basis weight) and its purpose of use. Examples of paperboard are cardboard boxes, paper packages, and wallpaper. According to statistics provided by Japan’s Ministry of Economy, Trade and Industry, in 2019, 55.0% of all western-style paper produced in Japan was “paper,” and 45.0% was paperboard. If we simply say “paper,” it refers to western-style paper other than paperboard. In this book, the term “paper” is used to refer to western-style paper other than paperboard.

18

2 Various Media and Their Characteristics

Fig. 2.2 Percentage of each type of paper produced in 2019 (based on annual production statistics data provided by Japan’s Ministry of Economy, Trade and Industry)

Furthermore, paper (i.e., other than paperboard) can be divided into newsprint paper, printing paper such as magazine paper, gravure paper, and postcards, information paper such as the copy paper we use for copying and printing and the form paper used for commercial printing, wrapping paper used to wrap gifts or other objects, and sanitary paper such as tissue paper and toilet paper. Figure 2.2 shows the percentage of each type of paper produced in Japan in 2019. Paper for “writing,” which is the focus of this book, includes newsprint, printing paper, and information paper. This accounted for 73.7% of all paper produced that year. For reference, paper used for “wrapping” accounted for 6.7%, and paper used for “wiping” (or “sanitary paper”) accounted for 13.4%. At present, most paper is used for “writing” (which also includes the purposes of viewing, reading, and saving information).

2.1.3

Physical Characteristics of Paper

Paper is a sheet made from fibers. This fact leads to various properties that are convenient for reading and writing. In this section, we discuss the nature of paper brought about by its physical properties. In particular, we look at the nature of paper related to its role as a medium for reading and writing, which is the scope of this book. First, paper is made from many fibers that are intricately intertwined, which gives it a rough surface. Thus, when we write on paper with a pencil, it produces adequate friction, creating a comfortable writing feel. When we slide a pencil across paper, the rough surface of the paper scrapes the pencil lead. The pencil’s lead then becomes lodged between the fibers, and thus information is recorded on the paper. Conversely, if we rub the surface of the paper with an eraser, the dust of the eraser removes the pencil lead left between the fibers, and as a result the written information is erased.

2.1 Paper

19

Specular reflection on a flat surface

Diffuse reflection on an uneven surface

Fig. 2.3 Diffuse reflection of light on the surface of paper

The direction of pulp flow

The direction of fibers

Fig. 2.4 The flow of pulp in a paper machine and the longitudinal direction of paper

On the other hand, it is based on a different principle that people can write on paper with a ballpoint pen, a fountain pen, or a brush. In a sheet of paper, the fibers are overlapped and bonded together, forming layers. The ink is fixed to the paper based on capillary action in which water is absorbed into the minute gaps between the fibers. Printing with a printing press basically follows the same principle. Moreover, a small amount of air is able to pass between the paper through the interstices of the fibers. As a result, the moisture of the ink escapes from the paper and the ink is easily fixed to the paper. The feel of writing, such as how comfortably or clearly people are able to write and how well ink is fixed to the paper, is all determined by the physical characteristics of the paper, pencil, and pen. Since the surface of paper is rough and not uniform as shown in Fig. 2.3, light hitting the surface of the paper is diffused, not uniformly reflected. This results in a glare-proof surface that does not reflect too much light. It is not suitable for displaying bright and glossy information, because it does not emit light itself like digital displays. However, when reading text carefully, it is easy to read from paper because the reflection of light is not too bright. On the other hand, paper used for items such as photographs or calendars has a glossy, smooth surface. This is because a coating is applied to the surface of the paper to remove its surface unevenness and smooth it, which is a called gravure paper or coated paper. Although such paper can provide a bright and shiny look, it is too dazzling and difficult to read from unless the angle of light is well adjusted, because the light hitting its surface is uniformly reflected. In most cases, there is a certain bias in the direction of paper fiber alignment. In the process of making paper with a paper machine, the pulp dispersed in water is sprayed onto wires being moved at a high speed and is dried immediately (see Fig. 2.4). Therefore, the fibers in paper tend to be aligned in the direction of flow of

20

2 Various Media and Their Characteristics

Fig. 2.5 Vertical orientation fibers in paper books

the machine. This flow direction of fibers (i.e., the direction of the manufacturing process in the machine) is the longitudinal direction of the paper. Since its fibers are aligned in the longitudinal direction, paper is stiffer in that direction. In other words, paper cannot be bent as easily in the longitudinal direction as in the lateral direction. In the case of books, most are made so that the fibers of the paper are vertically oriented (see Fig. 2.5). This is because a sheet of paper is more flexible in the lateral direction, which makes it easier to turn the pages. In addition, paper absorbs water into the interstices of its fibers, which is called water absorption. This is what allows the ink from a pen to enter the interstices of the fibers and leave a mark on the paper. In addition, when people touch paper with their fingers, the moisture from their fingertips is absorbed slightly by the paper, which creates a slightly moist feeling, as if the finger is sticking to the paper. This creates a sense of unity between the finger and the paper, resulting in the ease of handling the paper and the ease of turning pages when people manipulate sheets of paper with their hands. The feel of paper comes from the irregularity of the paper’s surface and the moistureretaining properties of the paper. For reference, tissues and toilet paper are products that have been developed to take full advantage of paper’s ability to absorb water. Another important property of paper is breathability. A small amount of air passes between the papers through the interstices of the paper’s fibers. As a result, moisture within the paper escapes, which is what allows ink to be easily fixed to the paper. Moreover, this breathability allows paper to retain information for a long time. Japanese shoji (sliding doors made of paper panels), which are traditionally used as room partitions, have long been used in Japan because they function as breathable walls. Sheets made of glass or stainless steel in fiber form are also paper in a broad sense, and all these products have been developed with the aim of utilizing the breathable properties of paper. In summary, paper is easy to write on, easy to read, easy to flip through when sheets are bundled into a book, and an excellent medium for storing information (although its durability is not excellent, because it is easily damaged by fire and water). These properties were established through technological progress accumulated over a long history. An example of this is the improvement of paper’s absorbability of ink. The physical characteristics of today’s paper products ensure that ink does not blur, dries quickly, and does not show through when viewed from the other side of the paper. This is not realized through the properties of the paper alone, but rather as a result of co-evolution with ink. That is, the development of paper has promoted the development of new ink, and the development of ink and the advent of new types of ink have in turn promoted the development of paper. Today’s

2.2 Digital Media

21

paper exists as the result of a series of long-term developments and improvements over time in order to make reading and writing more comfortable. Moreover, its developments and improvements have been achieved together with those of other writing implements.

2.2

Digital Media

Digital devices, such as computer displays, tablet PCs, smartphones, and e-book readers, are also often referred to as “digital media.” The expression “digital media” is a name that emphasizes the functional aspects of these devices as information presentation media. On the other hand, the term “digital devices” (or “electronic devices”) is an expression that emphasizes their physical aspects. Furthermore, for the purposes of future discussions, it is convenient if we distinguish the display surface of a device from its external form (including the cover). For example, even if the external shape of a device is the same as a smartphone, its appearance and feel of operation differ significantly depending on whether it uses a liquid crystal panel or electronic paper for its display surface. Moreover, even for digital devices that use the same liquid crystal panel, their operation methods are completely different between one operated by a touchpad, like a notebook PC, and the one operated by a touch panel, like a tablet PC. Therefore, when discussing digital media, we should distinguish a device’s display surface from the device as a whole, including its outer shape and input devices. We refer to as a display panel for the former and a digital device for the latter. The remainder of this section introduces major examples of display panels and digital devices.

2.2.1

Display Panels

Displays were big and heavy until the early 1990s. The major player was the cathode ray tube (CRT) display. The basic principle is the same as that of a CRT television. An electron beam is emitted from an electron gun located at the back of a large vacuum tube and is made to collide with a fluorescent screen on the display surface to emit light. Therefore, these displays needed to be deep, and they were big. Next, with the spread of notebook PCs, the need for thinner displays increased, and liquid crystal displays (LCD) became widely used. In an LCD panel, liquid crystal, which is an intermediate state between solid and liquid, produces an image using the property in which the alignment of its molecules is changed by voltage. Since liquid crystal substances themselves do not emit light to create images, LCD panels incorporate a light source such as a fluorescent lamp or LED behind them to

22

2 Various Media and Their Characteristics

create a backlight system that projects an image by either blocking light or letting it pass through. In recent years, organic light-emitting diode (OLED) displays are attracting attention as panels capable of presenting beautiful images with higher brightness and contrast compared to LCDs. Since an OLED display uses a light-emitting body called organic electro-luminescence (organic EL) that emits light by itself, it does not need a backlight and allows for high-speed rendering. Furthermore, because there is no need for a light source behind the panel, an OLED panel can be thinner and more energy-efficient than an LCD panel. OLED is expected to become a nextgeneration display panel to replace LCDs. It is still expensive, but smartphones and large-screen TVs that use organic EL displays are starting to appear on the market. While the digital displays introduced so far have all been made with the concept of “high-quality television,” electronic paper (EP) was developed with the concept of “rewritable paper” (Omodani 2004). It pursues and imitates the eye-friendliness and readability of paper. Rewritable paper is not a light-emitting display that presents information by backlighting or self-lighting, but rather a reflective display that enables information to be displayed using reflected light.3 Therefore, it can be read outdoors under natural light. In addition, it is eye-friendly. The panel is power saving because it can continue to present information even without power supply. It is also thin, light, and flexible. It has been widely adopted as a display panel in e-book readers, which are described in detail later. Buxton (2008), a principal researcher at Microsoft Research, chose electronic paper technology as one of future important technologies that would change the world in the next 10 years. Table 2.1 compares the characteristics of LCDs, organic EL displays, electronic paper, and paper. In the case of paper, since the number of pages increases in proportion to the amount of information recorded, it becomes heavier and thicker as the amount of information increases. In addition, since information recorded on paper cannot be rewritten, it is impossible to update information in real time or to display moving pictures. Since electronic paper is not a light-emitting panel, its visibility is high in both indoor and outdoor environments, and its properties are similar to those of paper. Its advantages over paper are that its weight and thickness do not change even when the amount of recorded information (pages) increases and that it allows information to be rewritten. Neither LCDs nor organic EL displays are suitable for displaying information in outdoor environments because they are lightemitting displays. However, their rewriting speed is high, and they can display moving pictures.

3 There are several technologies used to realize electronic paper, each with slightly different characteristics. As a typical example of such technologies, in this book we assume the use of an electronic paper panel with microcapsule electrophoretic technology (Comiskey et al. 1998) which has been adopted in various e-book readers on the market.

2.2 Digital Media

23

Table 2.1 Comparison of display panels Display

Color Contrast Rewriting Video Color Indoors Outdoors Small number of pages Large number of pages Small number of pages Large number of pages

Visibility Weight Thinness Flexibility Cost

2.2.2

LCD + + + + + + –     – 

Organic EL + ++ + ++ + + –     – –

Electronic paper   + –  + ++ + + + + + +

Paper +  – – + + ++ ++  ++  ++ ++

Digital Devices

Next, we will compare the characteristics of desktop PCs, notebook PCs, tablet PCs, smartphones, and e-book readers as digital devices to be compared with paper. Although all of these devices are widely known, we will introduce a brief description of each device as a reference. The first device that we introduce as a digital device for presenting documents is the desktop PC. In the past, CRT displays were mainstream, but most of displays today are LCDs. In addition, the surfaces of computer displays are growing larger. Until about 10 years ago, displays measuring less than 20 inches were mainstream, but now it is hard to find desktop PCs with less than 20-inch displays. Notebook PCs were developed for carrying around. They are integrated devices with a keyboard and a display. Their sizes range from portable sizes of less than 11 inches to A4 size of more than 14 inches. The spread of smartphones began with the launch of the iPhone in 2007. As an operation method for small screens, touch-based operation was introduced. It has features such as turning pages using a swiping gesture, zooming in and out using a pinch gesture, software text input, and voice input. To make the text readable on a small screen, vector-based fonts were also developed. The tablet was pioneered with the success of the iPad, which was launched by Apple after the iPhone. Unlike a desktop PC or a notebook PC, a tablet can be operated by touch without the use of a mouse and keyboard. The screen of a tablet is larger than that of a smartphone, with enough screen space to read books and magazines. It was also a device that aimed to enter the e-book market. The e-book reader is a device specially designed for reading e-books. Since 2000, Sony released several e-book readers such as the LIBRIe and the Sony Reader. They use reflective, eye-friendly, outdoor-readable electronic paper as a display panel. It was the Kindle by Amazon that was successful as a business and became the model

24

2 Various Media and Their Characteristics

Table 2.2 Characteristics of display digital devices

Example of typical product Panel Panel size Weight Operation method

Desktop PC Various

Tablet PC iPad

Smartphone iPhone

E-book reader Kindle

9–17 inches

6–13 inches

LCD panel/ Organic EL panel 4–6.5 inches

Electronic paper 6–9 inches

1.2–3.0 kg Keyboard, Touchpad

320–770 kg Touch

110–230 kg

130–220 kg

Laptop PC

LCD panel 17 inches or more Keyboard, Mouse

for the current e-book reader. Amazon also provided rich e-book content via the Web and gave the Kindle a communication function to access them. The Kindle uses electronic paper as a display panel and is characterized by its light weight and thin, simple design that allows the user to focus on reading. Table 2.2 shows a comparison of the functional characteristics of these digital devices. We organized the representative products, panel types, panel sizes, weights, and operation methods for each category.

2.3

Limitations on the Use of Digital Media

Finally, it will also be necessary to discuss the limitations of the use of digital media. In the comparative theory of paper and digital media, what we describe here is often pointed out as weaknesses of digital media. We organize not only the advantages of digital media, but also its weaknesses. First, digital media requires a source of power in order to display information or to switch the information being displayed. They cannot be used at all when no power is supplied to a device or when a device’s battery is exhausted. Recently, when one of the authors took his Kindle on an overseas trip, he forgot to bring the Kindle power cord. As a result, he was unable to use the Kindle at all during the trip and was forced to carry around a useless device for a week. Second, in an environment or situation where communication functions cannot be used, it is impossible to access any databases on networks or the Internet. The cloud is convenient, but it is meaningless if you cannot connect to a network. In addition, even if it is possible to connect to a network, there are many organizations that restrict access to external cloud environments for security reasons. Third, in order to use these devices comfortably and safely, it is necessary to keep their OSs or applications up to date. In addition, it is necessary to periodically update security software. These update procedures can be time consuming, and devices are often not available while an update is in progress.

References

25

Fourth, you must be careful about information leakage. Of course, there is also a risk that information on paper will be leaked, but in the case of digital media, a large amount of information can easily be leaked at once. Moreover, once such information has been leaked, it can be disseminated. Therefore, the risk of information leakage is high in digital environments. The costs for security that organizations are investing to avoid these problems is by no means small. Fifth, in general, digital devices are vulnerable to impact, vibration, water and the like, and can easily break. Therefore, for important events that cannot fail, you must take other precautions such as carrying another device, printing on paper, etc. In addition, you must regularly back data up so that you can recover from a failure. Finally, it is difficult to buy accessories for digital devices, and they are often expensive. One of the authors carries around an electronic paper device, which he uses as a digital notebook. One day, he forgot to take the pen of the device on a business trip. Then, he was unable to use the digital notebook throughout the business trip. On the other hand, analog (i.e., ink) pens are consumables. You can buy a new pen at a convenience store when you lose one. If it is for temporary use, you can borrow another person’s pen. This is not possible in the case of digital pens. If you lose one, you usually must wait a few days to buy another one online, and the prices can be high, more than fifty dollars. In the case of the example given earlier in which one of the authors forgot to bring a power cord while traveling overseas, if you could buy a new power cord right away at your destination, it would not be such a big problem. Digital devices and their accessories are not commoditized enough to buy anywhere and anytime, and they are not cheap enough to lend to other people easily. On the other hand, paper does not have such severe usage limitations. This is because paper and the analog writing implements have technologically matured and penetrated into society as an information presentation infrastructure. Of course, these limitations of digital media will be gradually alleviated, but it still needs time to reach the level of the ease of analog writing materials.

References Buxton, B. (2008). Big data: The next Google. Nature, 455, 8–9. Comiskey, B., Albert, J. D., Yoshizawa, H., & Jacobson, J. (1998). An electrophoretic ink for all-printed reflective electronic displays. Nature, 394, 253–255. de Biasi, M. C. (2006). La Papier, Une aventure au quotidian. Gallimard. Omodani, M. (2004). Trend of developments on electronic paper and its prospect for applications. Journal of Information Processing and Management, 47(10), 688–697. [in Japanese].

Chapter 3

The Ease of Reading from Paper and the Difficulty of Reading from Displays

Abstract Paper is strongly preferred as a medium for reading (and conversely, a digital medium is not preferred). To clarify the reasons for this, we conducted factor analysis and obtained three factors. The first factor was display characteristics that are hard on the eyes. The second one was operational and physical constraints. The third one was disturbance of concentration. This chapter plays the role of a guide to explore why it is easy to read from paper, or why it is difficult to read from digital displays. In the following three chapters, we will discuss the effect of the three factors in more detail.

3.1

Paper Is the Preferred Medium for Reading

Whenever I (one of the authors) write a scientific paper, I type it on a PC using a word processor and submit it in digital form. However, while I am proofreading and revising my work, I always print it out on paper. I feel that reading from paper allows me to look over the whole text, and I can better concentrate on reading. Also, by putting myself in an environment that is different from when I am writing the paper while sitting in front of the PC, I feel that I can read it more objectively, as if I am reading someone else’s work. As a result, I find that I can make bolder revisions and find more mistakes. I often make corrections with a red pen until the entire sheet of printing paper is full of red marks, and I sometimes use different colored pens for different purposes. While I am working on this task, I always carry my printed manuscript with me and check it repeatedly whenever I find a little bit of free time on a train or in a café. I may end up printing out several dozen iterations of my manuscript before I am finished writing the paper. Recently, I have been intentionally using an A4-size electronic paper device (Digital Paper by Sony) to evaluate the usability of the current technology. Its panel is large and eye-friendly. The device is thin and lightweight. I also love its stylish design. However, when I am reading a document from this device, I find it difficult to scan the entire document at once because of the difficulty of moving back and forth between pages (Shibata et al. 2016). Additionally, the feel of writing with the stylus is inferior to the comfort of writing on paper. I still feel the need for paper. © Springer Nature Singapore Pte Ltd. 2020 H. Shibata, K. Omura, Why Digital Displays Cannot Replace Paper, https://doi.org/10.1007/978-981-15-9476-2_3

27

28

3 The Ease of Reading from Paper and the Difficulty of Reading from Displays

This is just my personal experience, but this feeling that people want to use paper when reading text critically and revising it to improve its quality may ring true for many people to some degree. Today, now that office work is established on the basis of digital technologies, many people usually work electronically, using a PC display to read emails or view information on the Web. However, when people want to read something carefully or read while thinking deeply (as is often the case when reading or proofreading technical documents or foreign-language documents), they are likely to print out the document and read it on paper. In fact, paper is overwhelmingly preferred as the medium for reading (BuzzettoMore et al. 2007; Kretzschmar et al. 2013; Franze et al. 2014; Kazanci 2015; Baron et al. 2017). Franze et al. (2014) conducted a questionnaire survey in their laboratory and found that even computer science researchers who are able to use rich computer environments and are accustomed to using computer tools prefer to read from paper when reading academic papers. Tees (2010) surveyed reports on the use of e-book readers for academic activities and concluded that e-book readers were sufficient for leisure reading, but insufficient for academic reading as a replacement for textbooks. We also conducted an online questionnaire survey of 826 office workers in September 2008 (Omura and Shibata 2010). Figure 3.1 is a comparison of the readability of a paper document and the readability of a document presented on a computer display. The bars of the graph represent the average scores of participants’ subjective evaluations on a 7-point scale in which
3 is the worst (very bad); 0 is neutral (neither bad nor good); and 3 is the best (very good). This data was obtained in 2008, but we believe that these results are still true, because the quality of today’s computer displays when reading text does not differ significantly from that of displays at that time. Looking at the results, we can see that ease of concentration, ease of understanding, and ease of remembering content were evaluated as being higher for paper than for computer displays. In addition, the results suggest that paper flexibly supports various types of reading such as careful and detailed reading, skipping around while reading, cross-referencing multiple documents, reading while explaining, and understanding outlines. For these types of reading, paper was rated higher than computer displays. Many participants responded that paper gave them more freedom of posture when reading, made it easier to switch between pages while reading, and took less time when opening a document to the first page. In addition, many people responded that they could detect typographical errors or badly flowing text more easily when reading paper documents, that they could read faster with paper, and that they felt less eye strain with paper. It was only for the task of reading in the dark that displays were rated higher than paper. We can see that paper was strongly preferred as a medium for reading compared to computer displays. This preference for paper was true regardless of gender, age, and occupation (e.g., salesperson, administrative professional, or engineer). Even so, reading from a computer display was still rated higher than the midpoint (i.e., neutral point) in most (17 out of 18) of the evaluation items although it scored lower than paper for each evaluation item. In other words, reading from a display

3.1 Paper Is the Preferred Medium for Reading

29

Fig. 3.1 Comparison of the readability of the paper and the computer display (2008, N ¼ 826)

was also evaluated relatively positively. Although it did not reach the level of paper, people considered it to be “not bad.” The subjects of this survey were office workers. However, there are many different types of office workers, and some readers might think that many of these subjects could be technophobes who avoid digital devices and thus tend to prefer to use paper. It would be natural to imagine that the measured preference for paper would be different if the survey was targeted only at active users of digital devices.

30

3 The Ease of Reading from Paper and the Difficulty of Reading from Displays Paper/Paper books Desktop PCs Laptop PCs

Tablet PCs with LCD panels E-book reader with electronic paper panels Mobile phones/smartphones Projectors 0%

20%

40%

60%

80%

100%

Picture books Technical books Textbooks or learning drills Academic journals Photo albums Weekly magazines Novels Leaflets or advertisements Guidebooks or books teaching skills Comics Newspapers Travel guides or tour books

Business documents Brochures or catalogs Business or technical reports Foreign-language articles and papers Daily (or weekly) reports Product instruction manuals Dictionaries Schedules or to-do lists Cooking recipes Road maps Photos taken by yourself

Maps Phone numbers or addresses Emails Twitter or Facebook Videos

Fig. 3.2 Most preferred media for reading: results of a questionnaire survey for people who regularly read books using digital media (2012, N ¼ 554)

Therefore, this time we will share the results of another survey targeting those who actively use digital devices. Figure 3.2 shows the results of a March 2012 survey of 554 people who regularly read books with digital devices (such as e-readers, tablets, and smartphones). The Kindle 2 and the iPad were released in

3.2 Factors Contributing to the Difficulty of Reading from a Display

31

2010. Just two years after this, in 2012, the subjects of this survey were already reading books using digital devices on a daily basis. Considering these facts, we can say that the participants of this survey were early adopters who like using new devices. We asked them which medium was the most readable for different types of documents. As you can see from the results, paper was considered suitable for reading a wide range of document types, from picture books and novels to more academic books such as technical books, textbooks, and journals, as well as leaflets and travel guides. Even in the case of people who made full use of digital devices and were regularly reading books on these devices, it turned out that there were many situations where they considered paper to be the best medium for reading. It seems that these people recognized the value of paper as a medium for reading but at the same time used digital media in pursuit of the conveniences of digital data that are not provided by paper (e.g., portability and searchability).

3.2

Factors Contributing to the Difficulty of Reading from a Display

Why it is easy to read from paper? This can be understood through an examination of the readability of displays compared to paper. We conducted a survey of 1053 office workers in 2008. In this survey, we listed factors that seem to be related to the difficulty of reading from displays and asked the participants to evaluate how much they thought so. We performed a factor analysis for the obtained responses and extracted three factors1 that contribute to the difficulty of reading from the displays, as shown in Fig. 3.3. The first of these three factors includes items such as bright screen light, flickering, overly high contrast, and light reflection. We named this factor display characteristics that are not eye-friendly. The second factor included operational problems for documents such as the fact that it was difficult to display the entire page in an appropriate size, bring a document to the front, scroll, change pages, and make handwritten annotations, as well as problems of physical constraints such as the fact that the same posture was required to look at the screen while reading. We named this factor operational and physical constraints. The third factor included problems related to visual stimuli that disrupted the user’s ability to concentrate on the act of reading. For example, readers may be distracted by the blinking of a cursor, the background screen, icons, gadgets, menus, toolbars, etc. We named this factor disturbance of concentration.

1

Factor analysis is a multivariate analysis of humans’ responses to multiple variables that extracts the hidden variables (called factors or latent variables) behind these variables. In this analysis, we extracted three factors by aggregating 20 variables.

32

3 The Ease of Reading from Paper and the Difficulty of Reading from Displays

Fig. 3.3 Factors contributing to the difficulty of reading from a display: Factor pattern after Promax rotation

The next three chapters will examine what kind of results are observed in psychological experiments related to each of the three factors presented here as reasons why displays were difficult to read. Chapter 4 deals with the display quality of media (corresponding to Factor 1), Chap. 5 deals with the operability of media (corresponding to Factor 2), and Chap. 6 deals with stimuli that disrupt concentration (corresponding to Factor 3). In each chapter, we will introduce studies on how these factors affect users’ reading performance.

References

33

References Baron, N. S., Calixte, R. M., & Havewala, M. (2017). The persistence of print among university students: An exploratory study. Telematics and Informatics, 34(5), 590–604. Buzzetto-More, N., Sweat-Guy, R., & Elobaid, M. (2007). Reading in a digital age: E-books: Are students ready for this learning object? Interdisciplinary Journal of Knowledge and Learning Objects, 3(1), 239–250. Franze, J., Marriott, K., & Wybrow, M. (2014). What academics want when reading digitally. In Proceedings of the ACM Symposium on Document Engineering (DocEng ‘14) (pp. 199–202). ACM. Kazanci, Z. (2015). University students’ preferences of reading from a printed paper or a digital screen: A longitudinal study. International Journal of Culture and History, 1(1), 50–53. Kretzschmar, F., Pleimling, D., Hosemann, J., Füssel, S., Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2013). Subjective impressions do not mirror online reading effort: Concurrent EEG-eyetracking evidence from the reading of books and digital media. PLoS One, 8(2). Omura, H., & Shibata, H. (2010). Survey research on reading and writing for paper and electronic media. Shigyo Times, 62(10), 18–30. [in Japanese]. Shibata, H., Fukase, Y., Hashimoto, K., Kinoshita, Y., Kobayashi, H., Nebashi, S., et al. (2016). A proposal of future electronic paper in the office: Electronic paper as a special-purpose device cooperating with other devices. ITE Transactions on Media Technology and Applications, 4(4), 308–315. Tees, T. (2010). Ereaders in academic libraries: A literature review. The Australian Library Journal, 59(4), 180–186.

Chapter 4

Effects of Presentation Quality in Reading

Abstract This chapter deals with the effects of presentation quality of media on reading. In classical experiments conducted in the 1980s, reading from computer displays was determined to be markedly inferior to reading from paper. However, many of the more recent studies using modern high-quality computer displays did not find a significant difference between reading from paper and reading from digital media. Regarding the impact of eye strain during reading, people often subjectively evaluate reading from paper as less fatiguing than reading from digital media. However, it is difficult to show clear evidence that objectively proves this. Considering these studies, we think that the effect of display characteristics on the performance of reading and writing is not significant if we use current digital displays with high image quality.

In the previous chapter, we demonstrated that paper was overwhelmingly preferred as a medium for reading. However, this is the result of users’ subjective evaluation, and it may be due to bias or preconception. What kind of difference is measured between reading documents on paper and reading them on a display when we use objective indicators such as reading speed, comprehension, and rate of error detection in proofreading? The following three chapters examine the readability of paper, and conversely, the difficulty of reading on displays, based on objective indicators of reading. The first thing to discuss is how the display characteristics of presentation media affect reading. Display characteristics of media are the most important factor explaining the difficulty of reading on digital displays (and conversely, the ease of reading on paper). When asked “why is it easy to read on paper,” many people tend to emphasize the visible differences, answering “because paper is easy to look at” or “because paper is eye-friendly.” However, people’s subjective judgments do not always coincide with objective evaluations. It is not necessarily true that you know yourself best. We want you to realize this in this chapter. Note that in this book, objective reading indicators such as reading speed, comprehension, memory, and rate of error detection in proofreading are collectively referred to as reading performance. © Springer Nature Singapore Pte Ltd. 2020 H. Shibata, K. Omura, Why Digital Displays Cannot Replace Paper, https://doi.org/10.1007/978-981-15-9476-2_4

35

36

4.1

4 Effects of Presentation Quality in Reading

Influence on Reading Speed and Understanding Level

From the 1980s to the early 1990s, when computers began to be introduced to offices and schools, many studies were conducted comparing the reading performance and eye strain caused by reading from paper and from computer displays, with the aim of assessing the impact of displays on reading experience (Mills and Weldon 1987; Gould et al. 1987a, b; Dillon 1992; Noyes and Garland 2008). These early experiments mainly examined how differences in the presentation quality of documents, such as screen size and resolution, affect reading. In addition, they also examined aspects of document format (such as character size and row length), the user’s positional relationship with the screen (such as the distance between the eye and the screen), and the orientation of the screen’s surface (vertical or horizontal). In other words, they experimentally examined how the appearance of documents affects the reading experience. The presentation quality and appearance of documents have been improved with the technological innovation of digital displays. Therefore, we do not intend to compare paper documents with documents on digital displays based on the classical studies conducted in the 1980s and early 1990s. However, to show the typical experimental methods used to compare the readability of paper documents with that of documents shown on digital displays, we introduce the outline of the experiment conducted by Gould et al. (1987a, b) of IBM. This experiment compared participants’ reading speed (the number of words read per minute) and error detection rate (the rate of detection of spelling errors) when proofreading paper documents and when proofreading documents on digital displays (Gould et al. 1987a). In the case of paper, participants were required to circle misspelled words with a pen. In the case of computer displays, they were required to point at misspelled words with a pen. Text documents were presented with the same font size and layout on both paper and digital displays. Twelve participants performed the tasks on paper placed flat on a desk (horizontal paper condition), paper put on a stand so that its surface was vertical (vertical paper condition), and a CRT display with a vertical surface (CRT display condition). The order of these three conditions was random for each participant. The CRT display used in the experiment was the IBM 3277, which was currently on the market at the time. Its resolution was 480  640, and its refresh rate was 50 Hz. Before describing the experiment results, we would like to describe the concept of statistical tests for differences in mean values among different experimental conditions. In experiments with human subjects, the same task conducted under the same conditions does not always yield the same results each time. This is because differences in the participants’ psychological state and thought process each time cause variations in the experiment results. Therefore, there is no use in comparing experiment results based on slight differences in the graphs or numerical values. It is necessary to examine whether the observed difference is within the normal range of fluctuation. When the difference is large enough that it is statistically improbable to be within the range of

4.1 Influence on Reading Speed and Understanding Level

37

Fig. 4.1 Results of Gould et al.’s experiments: proofreading speed (a) and error detection rate (b) (based on the data of Gould et al. 1987a)

fluctuation, it is called a statistically significant difference or simply a significant difference. As a stochastic difference that is improbable, its value is usually set to 5%. That is, if the probability of such a difference occurring by chance is less than 5%, we regard it as a difference that is unlikely to be accidental, and we call this a significant difference. Also, if the probability is greater than 5% and less than 10%, we call this difference a significant tendency. The results of the experiment conducted by Gould et al. are shown in Fig. 4.1. There is no statistically significant difference in proofreading speed (the number of words read per minute) between the horizontal paper condition and the vertical paper condition, while reading was significantly slower in the case of the display condition compared to the two paper conditions (Fig. 4.1a). That is, the difference between the three conditions was large enough to be considered unlikely to be within the range of fluctuation. For the error detection rate (the detection rate of spelling errors), there was no significant difference between these three conditions (Fig. 4.1b). In the case of the two paper conditions, the same rate of error detection as the display condition was achieved in a shorter amount of time. In other words, it turned out that the paper achieved higher reading performance than the CRT display. Moreover, because the proofreading speed did not vary between the horizontal paper condition and the vertical paper condition, it also turned out that it was not a big issue whether the media surface was oriented horizontally or vertically. It turns out that the reason why reading performance was higher in the case of reading from paper originated in the presentation quality of documents. However, CRT displays at that time were surprisingly inferior in presentation quality compared to today’s displays. When one of the authors was working on a computer in the early 1990s, the dots on the display were large, as if I could pick them up with tweezers. Moreover, the image of the display was flickering. I remember that I often experienced “display sickness” after working in front of the

38

4 Effects of Presentation Quality in Reading

computer display for a long period of time. Under such conditions, it is natural that reading from a display would be inferior to reading from paper. Therefore, Gould et al., in another experiment, used a more high-performance CRT display (with a resolution of 1024  1024 and a refresh rate of 60 Hz), which was especially built for the experiments, and compared proofreading from paper and from the display (Gould et al. 1987b). As a result, no statistically significant difference in either proofreading speed or error detection rate was found between paper and the CRT display. That is, as early as the late 1980s, when using highperformance displays (albeit of the time), it turns out that there was no clear difference in reading performance between reading from paper and from a digital display for the task of proofreading to detect spelling errors. In fact, liquid crystal displays that are commercially available today can present much more sophisticated, high-definition document images compared to the highperformance displays used in the experiment conducted by Gould et al. (1987b). Additionally, the technologies used to display fonts today are more sophisticated than those at the time. A document viewed on a current commercial digital display must be much easier to read than the same document viewed on the display used by Gould et al. Therefore, we expect that the difference in reading performance between paper and digital displays is no longer seen now, or it must be very slight if any. In fact, many recent results supporting this statement (that is, that there is no great difference in performance results in reading with paper and digital media) have been reported (Kou and Shiina 2006b; Kretzschmar et al. 2013; Margolin et al. 2013; Hermena et al. 2017). Although consistent results are not gained across various experiments, at least for reading novels where reading is performed sequentially, it seems that reading from digital displays has caught up with the level of reading from paper. Moreover, the presentation quality of digital displays has been improved year by year. We believe that in the present day, the effect of presentation media (i.e., paper or digital displays) on reading performance is not great, especially in the case of reading novels for leisure, although, of course, the effect is not zero.

4.2

Impact on Eye Strain

Another concern is how the difference in presentation quality between paper and digital displays causes eye strain. Subjectively speaking, paper has been evaluated as causing less eye strain than displays (Kou and Shiina 2006a, b; Benedetto et al. 2013). Let us look at the effect of media on objective indicators that are closely tied to eye strain. Isono et al. (2005) carried out an experiment on 13 university students. The experimenters had the students read novels both from paperback books and from the Sony LIBRIe, an e-book reader with an electronic paper panel, for 90 min. In order to investigate the decline in the adjustment function of the subjects’ eyes, which is used as an alternative index of eye strain, they measured three indicators: near-point

4.2 Impact on Eye Strain

39

accommodation distance, near-point accommodation time, and far-point accommodation time. The near-point accommodation distance is the shortest distance from the eyes to an object that allows the eyes to clearly see the object. When eyes become tired, this value grows larger, which means that the eyes become unable to focus on objects that are closer to them. The near-point accommodation time is the time it takes for the eyes to change focus from a distant target to a near target. The far-point accommodation time is the time it takes for eyes to change focus from a near target to a distant target. It is said that both of these times increase when eyes become tired. The result of this experiment was that no statistically significant difference was found in the measured values of the above three indicators between paperbacks and the e-book reader. Based on this, Isono et al. concluded that the level of eye strain when reading from an e-book reader with an electronic paper panel continuously for 90 min was low, and it was about the same as the level of eye strain when reading from a paperback. Kou and Shiina (2006a) conducted an experiment on 14 graduate students. The experimenters had students read novels from four different media: a CRT display, an LCD display, a Sony LIBRIe (an e-book reader with an electronic paper panel), and paper bound into the shape of a book. The media surfaces were all oriented vertically, facing the students (the LIBRIe and bound paper were placed on music stands). The experimenters measured contrast sensitivity, flicker value, and visual acuity as objective indicators, in addition to subjective assessment of eye strain. Contrast sensitivity is an indicator of one’s ability to distinguish figures from the background and to recognize subtle differences in light and dark. It is said that when one’s eyes are tired, their spatial resolution decreases, and one’s contrast sensitivity also decreases. The flicker value is an indicator that reflects one’s level of consciousness. When a light blinks, if the frequency of blinking per unit of time is high (that is, if the light is blinking at a faster rate), the eye no longer perceives it as blinking, and it appears as a steady light. The flicker value is the threshold frequency at which a flickering light is perceived as a steady light, and it is said that this value decreases as the eyes become more tired. As a result of this experiment, participants reported in the subjective evaluation that they experienced less fatigue when reading from paper and the LIBRIe than when reading from the CRT display and the LCD display. However, no statistically significant differences in contrast sensitivity, flicker value, or the objective index of visual acuity were observed between any of the media conditions. Benedetto et al. (2013) asked 12 participants in their twenties to read text from three different media: paper, an electronic paper device (the Kindle Paperwhite), and an LCD tablet device (the Kindle Fire). Then, before and after reading from each media, the experimenters measured the subjective evaluation of fatigue using a method known as the visual fatigue scale and examined the flicker value. The visual fatigue scale is a subjective evaluation of visual fatigue in which six items (difficulties seeing, discomfort around the eyes, tired eyes, numbness of the senses, headaches, and dizziness) are rated on a 10-point scale. Furthermore, Benedetto et al.

40

4 Effects of Presentation Quality in Reading

examined the frequency of eye blinks during reading. It is said that when the blink frequency decreases, it causes dry eye, which leads to eye fatigue. After participants read for 73 min, no statistically significant difference in flicker value was observed between the three media, as was the case with Kou and Shiina’s studies (2006a). However, the subjective evaluation of visual fatigue after reading was the lowest for paper, followed by the electronic paper device and then the liquid crystal tablet. In addition, the frequency of eye blinks was lower for liquid crystal tablets than for paper and electronic paper devices, whereas no significant difference was found between blink frequency for the latter two media. In the case of the LCD tablet, which presents an image by emitting light from behind the display panel (using what is known as a backlight system), the frequency of blinks was lower than that of paper, and the LCD tablet was also evaluated to be inferior to both paper and the electronic paper device on the subjective visual fatigue scale. On the other hand, in the case of the electronic paper device, which is a digital device using a reflective electronic paper panel, although its evaluation on the visual fatigue scale was inferior to that of paper, the blink frequency was comparable to that of paper. Siegenthaler et al. (2011, 2012) compared reading behavior by analyzing the eye movements of participants when reading from various media: paper, different e-book readers with electronic paper (e-ink) panels, and a tablet. The experimenters analyzed the number and duration of fixations occurring in participants’ eye movements. The results suggested that the participants’ reading behavior when reading from e-readers was indeed very similar to that when reading from paper print. As we discussed here, digital devices, including those with electronic paper panels as well as those with liquid crystal panels, are often subjectively evaluated as being more tiring on the eyes than paper. However, in the case of electronic paper devices, there was no clear evidence indicating eye strain, at least when looking at objective physiological indicators. On the other hand, even in the case of LCD panel devices, the values measured for contrast sensitivity, flicker value, visual acuity, and other indicators did not differ from those of paper. Therefore, we can say that temporary use of such devices would not cause a major eye strain problem. Nevertheless, a previous study showed that the LCD tablet led to fewer blinks (Benedetto et al. 2013), indicating that prolonged reading leads to dry eye, which may cause eye fatigue. Further studies are desirable to evaluate situations in which devices with LCD panels are used for long periods of time.

4.3

Summary

In the classical studies up to the early 1990s, the superiority of reading from paper to reading from computer displays was clearly shown as a difference in reading performance. However, this difference was only caused by the poor image quality of computer displays at the time. In comparative experiments using modern

References

41

high-quality displays, there was no clear difference in reading performance between paper and displays. However, this does not mean that reading from paper and reading from digital media are at the same level in terms of their influence on reading performance. To be precise, there might be a difference between them, but we should say that the difference is minor enough that it cannot be easily found with current evaluation methods. Nevertheless, we can say that current digital media are able to show highquality images for reading text, to the extent that no clear difference in reading performance can be observed easily. In terms of reading fatigue, digital media are often subjectively rated as being more tiring on the eyes than paper. However, in devices using an electronic paper panel, which has the same reflective properties as paper, clear evidence for eye strain is not observed in terms of objective indicators such as contrast sensitivity, flicker value, and blink frequency. On the other hand, in the case of devices with LCD panels, no significant differences were found in contrast sensitivity, flicker value, and visual acuity compared to when reading from paper, but the frequency of blinks decreased when reading from an LCD, indicating that dry eye is more likely to occur.

References Benedetto, S., Drai-Zerbib, V., Pedrotti, M., Tissier, G., & Baccino, T. (2013). E-readers and visual fatigue. PLoS ONE, 8(12). Dillon, A. (1992). Reading from paper versus screens: A critical review of the empirical literature. Ergonomics, 35(10), 1297–1326. Gould, J. D., Alfaro, L., Barnes, V., Finn, R., Grischkowsky, N., & Minuto, A. (1987a). Reading is slower from CRT displays than from paper: Attempts to isolate a single-variable explanation. Human Factor, 29(3), 269–299. Gould, J. D., Alfaro, L., Finn, R., Haupt, B., & Minuto, A. (1987b). Reading from CRT displays can be as fast as reading from paper. Human Factors, 29(5), 497–517. Hermena, E. W., Sheen, M., AlJassmi, M., AlFalasi, K., AlMatroushi, M., & Jordan, T. R. (2017). Reading rate and comprehension for text presented on tablet and paper: Evidence from Arabic. Frontier in Psychology, 8. Isono, H., Takahashi, S., Takiguchi, Y., & Yamada, C. (2005). Measurement of visual fatigue from reading on electronic paper. The Journal of Image Information and Television Engineers, 59(3), 403–406. [in Japanese]. Kou, B., & Shiina, K. (2006a). A comparative study for reading novel over various media under controlled condition: Focusing on the character of various media for novel reading. The Bulletin of the School of Library, Information and Media Studies, University of Tsukuba, 4(1), 29–44. [in Japanese]. Kou, B., & Shiina, K. (2006b). A comparative study for reading novel over various media: Influences of media handling styles on comfortable reading. The Bulletin of the School of Library, Information and Media Studies, University of Tsukuba, 4(2), 1–18. [in Japanese]. Kretzschmar, F., Pleimling, D., Hosemann, J., Füssel, S., Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2013). Subjective impressions do not mirror online reading effort: Concurrent EEG-eyetracking evidence from the reading of books and digital media. PLoS One, 8(2).

42

4 Effects of Presentation Quality in Reading

Margolin, S. J., Toland, M. J., Driscoll, C., & Kegler, J. L. (2013). E-readers, computer screens, or paper: Does reading comprehension change across media platform? British Journal of Developmental Psychology, 27(4), 512–519. Mills, C. B., & Weldon, L. J. (1987). Reading text from computer screens. ACM Computing Surveys, 19(4), 329–357. Noyes, J. M., & Garland, K. J. (2008). Computer- vs. paper-based tasks: Are they equivalent? Ergonomics, 51(9), 1352–1375. Siegenthaler, E., Schmid, L., Wyss, M., & Wurtz, P. (2012). LCD vs. e-ink: An analysis of the reading behavior. Journal of Eye Movement Research, 5(3), 1–7. Siegenthaler, E., Wurtz, P., Bergamin, P., & Groner, R. (2011). Comparing reading processes on e-ink displays and print. Displays, 32(5), 268–273.

Chapter 5

Effects of Operability on Reading

Abstract In the previous chapter, we saw that the display characteristics of different media, which was the first factor affecting the difficulty of reading from computer displays, did not significantly affect reading performance, except for some indicators related to eye strain. In this chapter, we focus on the operability of documents, which is the second factor affecting the difficulty of reading from computer displays. First, we confirm that there are various types of reading, and various actions are performed on documents during the process of reading. Next, we introduce the concept of the cognitive load of operations, which is important in examining the influence of document operability on reading. We introduce 10 experiments on various types of reading observed in workrelated or academic situations. Based on the results of these experiments, the superiority of paper seemed to be clearly observed in types of reading that involve frequent interaction with documents. Readers perform various actions on documents while reading, such as moving or arranging documents, turning the page, flipping through pages, moving back and forth between pages, or tracing text. Paper is an excellent tool for performing such actions, and the ease of doing such actions with paper (and conversely, the difficulty of doing such actions with digital media) seems to affect reading performance.

5.1

Various Types of Reading

In general, as symbolized by the painting “A Young Girl Reading” (Fig. 5.1) by the eighteenth-century French painter Jean Fragonard, when we think about “reading a document,” we tend to imagine a style of reading in which people turn pages one by one from the beginning to the end with sitting on a chair. However, this understanding is completely different from reality, at least in business or learning situations. We sometimes skim through a document to get a general idea of it, read in a zig-zag pattern by moving back and forth through the document, read multiple documents at the same time, read across different media (e.g., paper, PCs, smartphones, etc.), or make annotations while reading (Sellen and Harper 2001). In addition, we sometimes read a document with many people or read © Springer Nature Singapore Pte Ltd. 2020 H. Shibata, K. Omura, Why Digital Displays Cannot Replace Paper, https://doi.org/10.1007/978-981-15-9476-2_5

43

44

5 Effects of Operability on Reading

Fig. 5.1 “A Young Girl Reading” by Jean Fragonard. Courtesy National Gallery of Art, Washington

it while standing or lying down. Reading is performed for various purposes, in various ways, locations, and postures. In order to discuss such a diverse notion as reading, we must first organize the terms we use related to reading. In terms of the reader’s attitude toward reading, reading can be divided into two types: receptive reading and active reading. In receptive reading, the reader receives a continuous piece of text in a manner similar to listening (O’Hara 1996). A typical example of this is reading a novel in a pastime. On the other hand, reading sometimes involves critical thinking and learning to evaluate or dispute something, as well as understanding text. This method of reading with an active attitude is known as active reading (Adler and Van Doren 1972; Schilit et al. 1998). Active reading is also called responsive reading. In terms of the situation of reading, reading can be divided into two types: leisure reading and work-related reading. Leisure reading is performed in a leisure context, and work-related reading is performed in a work context. In this case, work is not restricted to office work. Reading in an academic setting also falls into the category of work-related reading. Our main concerns are active reading and work-related reading. Adler et al. (1998) analyzed work-related reading in detail. They acquired five-day activity records of 15 knowledge workers in various industries. According to their results, paper or digital documents were used during 82% of the subjects’ work hours, and

5.1 Various Types of Reading

45

Fig. 5.2 Classifications of work-related reading (based on data of Adler et al. 1998)

about 70% of these document-related activities were accompanied by some type of reading. Adler et al. categorized reading activities into 10 types according to their purposes. Figure 5.2 shows the categorization of reading activities and their frequency. Reading for cross-referencing is a form of reading that involves comparing or verifying descriptions of different parts of documents with each other. This includes tasks such as checking numbers between multiple documents and checking the definitions of terms. Reading to search for answers is a type of reading done for the purpose of finding out details or answers to questions with the assumption that the information you want to know is written in the document. This includes the task of consulting a dictionary or searching for information in a manual. Reading to support discussions is the task of reading or referring to relevant documents during a discussion. Skimming is to glance through the keywords in a document to get an overview of the document. Reading to edit or review text is reading in order to correct errors in a document or re-edit it. Reading for self-development is reading in order to acquire general knowledge for purposes of self-growth. As we can infer from these various types of reading, the physical actions performed on documents during reading are also diverse. For example, you may adjust the positions of multiple documents, change the order of overlapping documents, flip through pages to search for answers, go back and forth between different parts of a document, or point at content in a document during a discussion in order to show it to other people. In the rest of this chapter, we look at the ease of various actions performed during the above types of reading, that is, how the operability of media affects reading in business or learning situations.

46

5.2

5 Effects of Operability on Reading

The Cognitive Load of Operations

Before introducing a series of experiments examining how media operability affects reading performance, we explain the concept of cognitive load and introduce how to measure it.

5.2.1

The Measurement of Cognitive Load

Human memory consists of short-term memory, which temporarily stores information for a few seconds to several minutes, and long-term memory, which stores information for a longer period of time. Short-term memory is also referred to as working memory because information is processed in short-term memory. The relationship between short-term and long-term memory is easy to understand if you think of the relationship between a computer’s memory and its hard disk. A computer’s random-access memory (RAM), used to quickly perform calculations, corresponds to our short-term memory (working memory), and the large-capacity, relatively low-speed hard disk corresponds to our long-term memory. Cognitive load is the amount of mental resources (that is, the amount of working memory) required to perform a task (Sweller 1988; Schnotz and Kürschner 2007). There is a limit to the capacity of human working memory. The capacity differs depending on the person, but the number of meaningful pieces (chunks) of information that can be learned in the short term was first said to be 7  2 by Miller’s study (1956). Later, this value was modified to 4  11 by Cowan (2001). Rather than the actual number, we want to emphasize the small size of our working memory. It is much smaller than the memory capacity of a computer and cannot be expanded freely. Therefore, for human information processing, it is important to effectively utilize this small memory space. In general, it is impossible to directly measure what portion of a person’s working memory is used to perform a certain task. Nevertheless, it is possible to judge which task uses greater amount of mental resources among several tasks, that is, which task requires a higher cognitive load, by using an experimental method called the dualtask method. In the dual-task method, in addition to a main task such as reading text or writing notes, participants perform a sub-task such as responding to visual or audio stimuli 1 Considering when you temporarily hold a phone number in your head when you call someone, this means that the number of digits you can memorize is usually about four: three in the worst case and five in the best case. However, it is possible for us to remember more digits than this by memorizing a set of several digits as a single unit. The number four is not simply the length of a numeric string, but rather the number of semantic groupings known as “chunks” that a human is able to hold in memory. The number of bytes in a computer’s memory is different to this in nature, which means it cannot simply be compared one-to-one with the memory capacity of a human. Still, you can imagine that the memory capacity of a human is much smaller than that of a computer.

5.2 The Cognitive Load of Operations Fig. 5.3 The dual-task method

47 㼃㼛㼞㼗㼕㼚㼓㻌㼙㼑㼙㼛㼞㼥 Overflow

㻿㼡㼎㻙㼠㼍㼟㼗

㻹㼍㼕㼚㻌㼠㼍㼟㼗 Low cognive load

High cognive load

or memorizing information simultaneously. Then, based on the performance results of the sub-task, we are able to examine the magnitude of the cognitive load of the main task. As shown in Fig. 5.3, when the cognitive load of the main task is low, people can perform both the main task and the sub-task without any difficulty. However, when the cognitive load of the main task is high, the mental resources allocated to perform the sub-task decrease, and the performance results of the sub-task decline. In the dual-task method, we estimate the cognitive load of the main task based on the performance of the sub-task. That is, we consider the cognitive load of the main task to be low when the performance results of the sub-task are good, and we judge the cognitive load of the main task to be high when the performance results of the sub-task are poor. To understand this experimental paradigm, let’s imagine driving a car. If a ball rolls into the street in front of the car, a novice driver is not immediately able to step on the brake pedal. A novice driver must pay attention to many things, such as the operation of the steering wheel and accelerator as well as safety confirmation. The driver’s mind (mental resources) is occupied by these things. This causes a delay in the driver’s judgment of the situation, which delays the driver’s shift to the next action in response to the rolling ball. On the other hand, a skilled driver does not need to use a great deal of mental resources for the act of driving itself and can therefore respond to the rolling ball immediately. Considering these facts, we can estimate the cognitive load of driving on a subject from the speed of the subject’s response to the rolling ball. If we apply this framework to the dual-task method, driving corresponds to the main task and responding to the rolling ball corresponds to the sub-task. This experimental framework is used to measure the cognitive load of various tasks such as reading (Mayes et al. 2001; Wästlund et al. 2008), note-taking (Piolat et al. 2005), and writing (Kellogg and Mueller 1993).

48

5.2.2

5 Effects of Operability on Reading

The Cognitive Load of Page Turning

We have conducted an experiment which shows the difference in cognitive load between page turning using paper and page turning using digital methods. We compared the cognitive load of reading using the dual-task method (Takano et al. 2012). In this experiment, 24 participants in their twenties and thirties read novels while wearing headphones, and we asked them to step on a foot pedal as soon as they heard a beep sound from the headphones. The aim of this experiment was to measure the delay time from the beep to the participants’ stepping on the pedal and to examine the magnitude of the cognitive load of reading for different media types. In the experiment, we asked the participants to read in three types of ways: reading from a paper book, reading from a tablet device while turning pages using a swipe (a gesture of sliding the finger left and right), and reading with a tablet device while turning pages using a tap (a gesture of lightly touching the left or right side of the screen). We presented a novel with the same font size and format on the paper book and the tablet device. We used the iPad as a tablet device. The results are shown in Fig. 5.4. When the beep sounded while the participants were reading the middle lines of a page, that is, those other than the last line (i.e., the line immediately before turning the page) and the first line (i.e., the line immediately after turning the page) of a page, there was no significant difference between the time required to respond to the beep in the three conditions. This supports the previous research results that there is no difference in reading performance whether you read

Fig. 5.4 Comparison of the time taken to respond to beeps in the dual-task method

5.2 The Cognitive Load of Operations

49

on paper or on a tablet device. The results seem to show that the presentation quality of media has a small influence on reading. On the other hand, when the beep sounded while the participants were reading the lines before or after turning the page, the time required to respond was the shortest when the participants turned the page on paper, next when they turned the page with a swipe, and longest when turned the page with a tap. There were significant differences in time between the three conditions. It turned out that there was a significant difference in the cognitive load of operation between page-turning with a paper book and with a tablet, and that page-turning with a paper book had a smaller cognitive load than page-turning with a tablet. When turning the page with a tap, it is necessary to tap the edge of the screen while visually confirming whether the position of the tap is appropriate. In addition, you may fail to turn the page even though you intended to tap the panel correctly. Therefore, after tapping, you must visually confirm that the page turned successfully. In the case of page-turning by swiping, the page can be turned by a rough movement of the hand, so it is not necessary to pay attention to the position of swiping when operating. However, the operation feeling of turning a page, such as how much you need to slide your finger to turn the page, is different among applications. Therefore, it is still necessary to visually confirm whether the page has successfully changed. On the other hand, page-turning of a paper book can be performed with the touch of a hand without looking at the page, and feedback on whether the page has been turned correctly can be obtained by touch. It seems that such operational differences were measured as differences in the cognitive load of operations. Although the cognitive load of scrolling with a mouse was not measured, it is necessary to visually confirm the cursor position in order to scroll, so we can easily imagine that its cognitive load will be higher than the case of swiping or tapping gestures. Regarding these results, you may think “If I can turn the page, that’s good enough,” “I don’t care about the difference in cognitive load,” or “page-turning is something that occurs only occasionally.” Indeed, in cases of reading a novel for leisure (i.e., receptive reading), there is little turning back and re-reading, so pageturning is less frequent. Even if the special experimental method detects a slight difference in the cognitive load of page-turning, which is performed only once every few minutes, it seems that its effect on the reading experience is not significant, and is rather too small to make a difference. However, when working intensively, people dislike being interrupted, even if the interruption is very slight. For example, when you are doing complicated calculations, just having someone talking to you can make it impossible for you to perform the calculations. What matters is not the length of time of the interruption, but rather the fact itself that an interruption in thinking occurs even for a short time. If you are doing work that requires a high degree of concentration, you honestly do not want to use any mental resources for other things.

50

5 Effects of Operability on Reading

The active reading observed in business or learning situations is very different from receptive reading. As we described earlier, active reading involves performing frequent operations on documents, such as moving documents and turning pages. Typical examples of this are arranging documents to compare information in them when reading multiple documents, and referring to references at the end and then returning to the original position when reading an academic paper. These tasks include moving documents, slightly shifting the positions of overlapping documents, bundling multiple documents together, switching the order of overlapping documents, flipping pages, and moving back and forth between different pages. Such actions occur frequently during reading. If an additional cognitive load is added and the reader’s thinking is interrupted during each of these actions, the efficiency and quality of reading as a whole may deteriorate. Thayer et al. (2011) investigated how students use the Kindle DX in their academic activities through interviews and diary studies. They concluded that the Kindle DX would be acceptable for receptive reading, whose process is typically sequential, but it was insufficient for responsive reading, which typically involves critical thinking and is also known as active reading. In actuality, the superiority of paper to digital media tends to be observed in such deep reading. In a series of studies by Ackerman and Goldsmith (2011), Ackerman and Lauterman (2012), comprehension scores of on-paper learning were significantly higher than those of on-screen learning in a time-regulated setting, which is more cognitively demanding than a non-time-regulated setting. Lauterman and Ackerman (2014) revealed that in a test examining in-depth reading in which students summarized texts, students achieved higher test scores after on-paper learning than after on-screen learning. In a study by Mangen et al. (2019), students who read from paper performed better in a task involving reconstructing the plot of a story than those who read on a Kindle. In cognitively demanding deep reading (or responsive reading, or active reading), readers frequently interact with documents using both their hands. The superiority of reading from paper (or inferiority of reading from digital media) might be attributable to the cognitive load of these interactions with documents. In the remainder of this chapter, after showing the variety of actions to be taken while reading, we show experiments that examine the influence of the operability of each action on paper and digital media on reading performance. In these experiments, we deal with reading in which these actions are frequently taken.

5.3 5.3.1

Overview of Experiments Operations Performed While Reading

Before analyzing the influence of operability on reading, let’s first grasp the whole picture of what kind of operations are performed on documents during reading. As shown in Table 5.1, operations during reading can be divided into three types:

5.3 Overview of Experiments

51

Table 5.1 Types of operations performed during reading Operation type Document operations

Description Operations on documents

Page operations

Operations on pages

Content operations

Annotation Content touch

Annotating document content Touching document content

Examples • Holding documents • Moving documents • Bundling documents • Changing the order of overlapping documents • Turning pages one by one • Flipping through pages • Turning multiple pages simultaneously • Moving back and forth between different pages • Inserting a finger or a bookmark between pages • Putting a sticky note on a page • Folding the corner of a page • Adding underlines, symbols, sentences, figures, etc. • Pointing at text • Tracing text with a finger

document operations, page operations, and content operations (Shibata and Takano 2019). Document operations are operations performed on documents consisting of a single page or multiple pages. They include the actions of holding and moving documents, stacking documents, and changing the order of overlapping documents. Page operations are operations performed on pages of a document or sheets of a paper book. They include turning pages one by one, flipping through pages, moving back and forth between different pages, and inserting a finger or a bookmark between pages. Content operations are operations performed on the document content in each page. These can be divided into annotation and content touch. Annotation is the act of adding writing to the document. It includes adding underlines, symbols, sentences, or figures. Content touch is the action of touching without changing the content. It includes pointing at text and tracing text with a finger or pen.

5.3.2

An Overall Picture of All Experiments

In this chapter, we introduce experiments that examine the influence of the ease of operations on reading performance. Table 5.2 shows an overview of the experiments that we introduce in this chapter. In addition to the tasks performed in each experiment, the different media used in the experiment are shown in the right columns of the table. Annotations are discussed in detail in Chap. 7, but they are briefly touched on here.

52

5 Effects of Operability on Reading

Table 5.2 Overview of the experiments Operation type Document operations

Page operations

Content operations

Mixed operations

5.4

Purpose of experiments Experiment 1. Comparing paper and PC in cross-reference reading of multiple documents Experiment 2. Evaluating the ease of document movement and alignment Experiment 3. Comparing largescreen and dual-screen PC environments Experiment 4. Evaluating the ease of page navigation Experiment 5. Evaluating the ease of searching for text information Experiment 6. Evaluating the ease of page flipping and page navigation Experiment 7. Comparison of paper and tablets in highcognitive-load reading Experiment 8. Examining the effects of content touch Experiment 9. Examining the relationship between reading with content operation and document orientation Experiment 10. The impact of media type on reading for discussion purposes

Experimental task Find errors by referring to multiple documents

Paper ✓

PC ✓

Tablet

Arrange multiple documents in a specified layout Perform daily work using PC

✓

✓

Read documents with explanatory notes Find answers within text documents

✓

✓

✓

✓

✓

Search for photos within photo books

✓

✓

✓

Detect contextual errors

✓

✓

✓

✓ Read documents while tracing text, drawing lines, or writing characters Read for discussion purposes

✓

✓

✓

✓

Document Operations

The first experiment deals with document operations. We compare the performance of reading on paper and on a desktop PC in cross-reference reading of multiple documents, where many actions such as moving and aligning documents are performed. Then, we discuss what causes the difference in reading performance.

5.4.1

Cross-Reference Reading of Multiple Documents (Experiment 1)

Unlike when reading for leisure, when reading documents in business or research, we often check and compare descriptions and figures in multiple documents.

5.4 Document Operations

53

According to a study by Adler et al. (1998) which observed reading in the real world, in more than half of work-related reading, people were viewing multiple documents simultaneously. O’Hara and Sellen (1997) dealt with a task of reading multiple scientific articles and creating summaries. They compared the process of working with paper documents and with window documents on a PC. The results showed that paper documents effectively supported this task. When using paper documents, people were able to easily arrange and organize the documents spatially and adjust the positions of superimposed documents. They were also able to efficiently extract and compare information from multiple sources. The experimenters also observed similar merits of paper documents during an actual academic writing process (O’Hara et al. 2002). Buchanan and Loizides (2007) observed the process of categorizing documents when working with paper documents and with digital documents (PDF files) on a PC. They found that the ease of handling documents affected the participants’ way of reading. In O’Hara and Sellen’s experiment, the functionality and operability of the tools were discussed qualitatively through observation and interviews about the work process, but we wanted to quantitatively evaluate the influence of the work environment on users’ reading performance. For more information about experimental settings and analyses, refer to another article (Shibata and Omura 2010; Shibata et al. 2013).

5.4.1.1

Method

Our first experiment dealt with the task of cross-referencing four documents and detecting errors, and we compared participants’ reading performance when using paper and a PC. As an experimental task, we required the participants to view four documents as shown in Fig. 5.5 at the same time and to detect errors. One of the four documents was a text document, and the other three were graphs. The text document described the three graphs. Under the premise that there were no mistakes in the graphs, we asked the participants to detect errors, or contradictions between the information obtained from the graphs and the claims in the text document. Each set of four documents included single-document errors, which are errors that can be detected if a single corresponding graph is referred to, and multipledocument errors, which are errors that cannot be detected unless multiple graphs are referred to. We embedded four of each of these types of error (eight in total). The participants were asked to detect the errors as quickly and accurately as possible. The participants were 24 people in their twenties and thirties, all of whom had three or more years of experience of using PCs. Their corrected visual acuity was average (i.e., 14/20 vision) or better. Three kinds of working environments were provided: the paper condition, the PC condition, and the 27-inch desk condition. In the paper condition, the participants performed the task using four sheets of B5 size office paper on which documents were printed in black and white on one side. The task was performed on an office

Fig. 5.5 Example of four documents used in the experiment

54 5 Effects of Operability on Reading

5.4 Document Operations

55

Fig. 5.6 Performing a task under the 27-inch desk condition

desk with a large working space. For the PC condition, we used a large 27-inch display to ensure that the working space constraint did not pose a big problem compared to the paper condition. However, although we used a large display for the PC condition, there was still a limit to the working space. Therefore, for the 27-inch desk condition, we made a desk with a frame that had the same physical area as the 27-inch display used in the PC condition (the inner dimensions of the frame measured 58.2  36.3 cm) and asked the participants to perform the task with the paper documents within the frame as shown in Fig. 5.6. The text size of the documents in the PC condition was set to be physically the same as the text size in the paper condition. In the PC condition, the OS was Windows XP, and the window size was the same as the B5 size paper used in the paper condition. The digital documents were displayed with Adobe Reader 9. Because the purpose of this experiment was to analyze the influence that the operability of a document, not its appearance, has on reading, we prohibited the expansion and contraction (zooming in and out) of documents in the PC condition. Also, since the purpose of the experiment was not to examine the effect that ease of writing has on reading, we prohibited writing on the documents. At the beginning of the experiment, we had the participants adjust the display position and display settings (brightness and contrast) to their preferences and then instructed them to find mistakes in a set of four documents twice under each condition (i.e., six trials in total). The order of the conditions was counterbalanced over all participants to avoid an imbalance in the trial order of each condition.

56

5.4.1.2

5 Effects of Operability on Reading

Results and Discussion

We first show the comparison results of proofreading speed (the number of characters processed per minute). As shown in Fig. 5.7, there was no statistically significant difference between the paper condition and the 27-inch desk condition. Compared to these two conditions, proofreading was significantly slow under the PC condition. The participants finished the task 25.5% faster in the paper condition than in the PC condition. Figure 5.8 compares the error detection rate (the proportion of detected errors to all errors). Again, there was no statistically significant difference between the paper

Fig. 5.7 Comparison of the proofreading speed for multiple documents

Fig. 5.8 Comparison of error detection rate in proofreading of multiple documents

5.4 Document Operations

57

condition and the 27-inch desk condition, and the error detection rate was significantly lower in the PC condition compared to the other two conditions. The error detection rate was 10.7% higher in the paper condition than in the PC condition. In addition, the multiple-document errors, which is required referring to multiple graphs to detect, tended to have a lower detection rate than the single-document errors. In this experiment of proofreading multiple documents, participants performed the tasks more efficiently and with a higher level of quality when using paper than when using a PC. We wondered about the reason for this result, but it was not due to the difference in working space. This is clear from the following two reasons. First, the performance of the 27-inch desk condition was the same level as that of the paper condition, although the size of the working space was different. Second, the performance of the 27-inch desk condition was higher than that of the PC condition, although the size of the working space was the same. To carry out the proofreading task in the experiment, participants had to frequently refer to the graph and correlate the information from the graph with the description in the text document in order to confirm the accuracy of the content of the text. It is possible that the ease of document operations such as moving documents, aligning their layouts, and changing their overlapping order in the process of reading may have affected the participants’ reading performance. In fact, this is supported by the fact that the detection rate for multiple-document errors, which require many comparison operations in order to be detected, was lower than the detection rate for single-document errors, which can be detected through fewer comparisons. In post-experiment interviews, many (15 of 24) participants reported having difficulty moving the windows on the PC. Typical examples of their comments were “it’s easy to switch between windows on the PC, but it’s difficult to place them.” and “to move the document a little bit, I needed to move the cursor to the title bar.” The operations performed on windows on a PC create more load for the user than operations performed on paper, and it is speculated that this may have led to a decrease in the efficiency and quality of proofreading for multiple documents.

5.4.2

Ease of Document Movement and Alignment (Experiment 2)

In the previous study (Experiment 1), we confirmed that in cross-reference reading of multiple documents, the use of paper was more efficient and allowed for a higher quality of proofreading than the use of a PC. Moreover, we inferred that the decline in participants’ performance was caused by the difficulty of performing operations on documents. To verify this, we conducted another experiment comparing the efficiency of document operations using paper and a PC (Shibata and Omura 2010). The task in this experiment was to place documents in specified positions.

58

5.4.2.1

5 Effects of Operability on Reading

Method

In this experiment, we drew a rectangle on the desk with the same area as that of the 27-inch display. Inside this rectangle, we drew a grid four rectangles high and four rectangles wide, as shown in Fig. 5.9. The task was to place sheets of office paper in specified positions (specifically, to align the top left corner of the document with the corresponding coordinates on the grid). Figure 5.9 is an example of a four-document layout. We changed the conditions when participants performed the task and examined which conditions allowed participants to place the documents faster. In the paper environment, we drew a 4  4 grid on the desk with the same area as that of the 27-inch display (58.2  36.3 cm) and instructed participants to arrange sheets of B5 size office paper in the specified positions. In the PC environment, we drew a 4  4 grid on the desktop of the 27-inch display and instructed them to arrange windows of the same size as the B5 paper in the specified positions. Furthermore, we imposed various constraints on the way the participants had to place the sheets of paper, such as performing the task with only one hand or performing the task while touching a limited area of the sheet of paper. In the case of the latter restriction, we used sheets of paper printed with a blue band on the top, and we required the participants to touch the blue area when they manipulated the paper. This method of manipulating the paper was the same as manipulating a window by dragging the title bar. Under this condition, in order for participants to confirm that their fingers are touching the blue band, they must look at the area when manipulating the paper. Combining these restrictions, we used the following five conditions as task conditions in the experiment.

Fig. 5.9 Example of placement of documents in Experiment 2

5.4 Document Operations

59

• Free: Participants worked with paper as usual (with no restrictions on the manipulation of paper). • Single-hand (Single for short): Participants worked with paper, with the restriction of using only one hand to manipulate the paper. • Area-restricted (Area for short): Participants worked with paper, with the restriction of touching the blue area of the paper when manipulating it. • Single-hand & Area-restricted (Single & Area for short): Participants worked with paper, with the restrictions of using only one hand and touching the blue area of the paper when manipulating it. • PC: Participants worked with a PC, arranging windows by mouse operation. We also varied the number of documents in three levels: one, two, and four. The participants were 24 people in their twenties and thirties. They all had three or more years’ experience using PCs. Their corrected visual acuity was average or better.

5.4.2.2

Results and Discussion

Figure 5.10 compares the time taken to complete the task under each condition. A statistical analysis identified the conditions under which participants took a significantly shorter time to complete the task. In Fig. 5.10, horizontal lines are used to connect the conditions for which the results were statistically at the same level. For each quantity of documents placed, significant differences were as follows (“” means “no significant difference,” while “ emails typed in gothic font”. In the case of the Display condition (Fig. 7.14b), the feeling of gratitude was highly evaluated in the order of “handwritten postcards > postcards typed in handwriting-style font > postcards typed in Kaisho font > emails typed in gothic font.” In both media, handwritten postcards were evaluated the highest. Handwriting requires effort, and handwritten letters are transient and cannot be reused. Gratitude may have been strongly recognized in return for the amount of effort spent on the recipient. On the other hand, the email message may give the impression of being “simplified” because emails are easier to send, and as a result it may be judged that the level of gratitude was low. In addition, handwriting functions as a means to express yourself (i.e., showing that “I am a person who writes such characters”) and to express emotion through your writing. It evokes a sense in the recipient of wanting to give something back. As a result, a feeling of gratitude may be strongly conveyed.

7.5.5

General Discussion of the Three Evaluations

First, let us consider how to select a presentation medium and document style according to the situation and purpose. In this evaluation, the sender’s personality (especially in terms of familiarity and friendliness) was evaluated more highly in the case of letters on paper then with letters displayed digitally. Therefore, sending a letter on paper seems to be more desirable when expressing one’s gratitude. In addition, we found that handwritten letters gave the impression of being humane, warm, unique, loving, and fun. Moreover, as a means of conveying feelings of gratitude, handwritten texts were evaluated more highly both on paper and on displays. Considering these facts, to convey feelings of gratitude, it seems that a letter handwritten and delivered on paper is the most desirable.

7.6 Summary and Discussion

151

However, this is only a view of the average opinions of the whole, and it is not true for all people. It should also be noted that handwriting on paper is not always optimal and may, under certain circumstances, reduce the sender’s appeal to the recipient. For example, handwriting lowered the sender’s ratings of “reliable” and “someone I can make a request to easily.” In addition, if you want to emphasize readability or if you want to give a beautiful, well-organized, formal, and businesslike impression, the use of an email typed in gothic font or a postcard typed in Kaisho font would be desirable. If you want to give a courteous impression, the use of old-fashioned handwriting-style font will enhance that evaluation. Thus, when you send a letter, it is desirable to select the proper medium, font, and layout depending on the situation. It is important to have an appropriate methodology for using them selectively, that is, understanding how each font or layout influences the recipient’s impression. These findings may be useful for conveying your thoughts and controlling the impression you make on the recipient when sending things such as personal letters or direct mail advertisements. The impression one receives from letters of various different presentation media and document styles will vary depending on the various attributes of the recipient such as age, sex, and occupation. In addition, although this study examines the perceived degree of feelings of gratitude, there are also other purposes for sending letters, such as apologies or requests. In the future, it will be necessary to conduct further studies with various participants, letters, and purposes.

7.6

Summary and Discussion

This chapter discussed how media affect the act of writing text or drawing figures. While word processors are widely used to produce formal documents, classical research has shown that the use of word processors did not necessarily improve the quality of the text. On the contrary, some studies pointed out their negative aspects. When using word processors, writers tended to write longer text, focus on more superficial typographical corrections, and spend less time on overall modifications. However, it is true that there are many advantages to digital writing using editing or word-processing software. With the current widespread use of digital writing, the advantages of digital tools have helped them gain overwhelming support from users despite their negative effects on writing process, since they allow the creation of good-looking documents and make it easy to read, edit, reuse, share, search, and distribute documents. Because of this, we hope that those involved in the development of word processors recognize these problems and work to improve their functions and design. On the other hand, when taking notes during lectures or meetings, handwriting on paper is still relatively common. Since writing by hand exerts a low cognitive load and there is no need to check whether or not your input is correct, it has the great advantage of enabling you to take notes while listening, looking, speaking, and

152

7 Effects of Writing and Drawing by Hand

thinking. On the other hand, typing on a PC is more cognitively demanding, meaning you cannot afford to use your mental resources for other tasks while typing. Thus, when taking notes on a PC during a lecture, you may become an “input machine” that directly inputs whatever the lecturer is saying. If your goal is to understand a lecture and make the knowledge your own rather than taking beautiful notes to show to others or if you want to understand the flow of a meeting and participate in its discussion rather than write easy-to-read meeting minutes for colleagues, at the present, it would be preferable to take notes by hand, based on the studies presented here. However, this does not mean the superiority of handwriting on paper will definitely last forever. The message here is that handwriting has a lower cognitive load than typing. We believe that digital handwriting may be widely used in the future. Many tablet devices have recently introduced functions for handwriting using a digital pen or stylus. Although the comfort of writing with a digital pen still does not live up to the feel of writing with an analog pen on paper, the research and development for improving it is steadily progressing, and it will be further improved in the future. Handwriting and typing are completely different methods of text input. Therefore, when you shift from handwriting to typing, it causes changes in your thought process, which may have either positive or negative effects. On the other hand, when switching from handwriting on paper to handwriting on digital media, only the input medium is different, and there is no change in the user’s style of input. Thus, if an excellent digital medium featuring a digital pen is developed in the future, there is a possibility that many people would accept it without hesitation.

References Bilda, Z., & Demirkan, H. (2003). An insight on designers’ sketching activities in traditional versus digital media. Design Studies, 24(1), 27–50. Dzulkhiflee, M., Tano, S., Iwata, M., & Hashiyama, T. (2006). Effectiveness of annotating by hand for non-alphabetical languages. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘06) (pp. 841–850). ACM. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: The MIT Press. Haas, C. (1989a). Does the medium make a difference? Two studies of writing with pen and paper and with computers. Human-Computer Interaction, 4(2), 149–169. Haas, C. (1989b). How the writing medium shapes the writing process: Effects of word processing on planning. Research in the Teaching of English, 23(2), 181–207. Hawisher, G. E. (1988). Research update: Writing and word processing. Computers and Composition, 5(2), 7–27. Hayes, J. R., & Nash, J. G. (1996). On the nature of planning in writing. In C. M. Levy & S. Ransdell (Eds.), The science of writing: Theories, methods, individual differences, and applications (pp. 29–55). Lawrence Erlbaum Associates, Inc. Hunter, W. J., Jardine, G., Rilstone, P., & Weisgerber, R. (1991). The effects of using word processors: A hard look at the research. The Writing Notebook, 8(1), 42–46. Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus & Giroux.

References

153

King, R. N., & Koehler, D. J. (2000). Illusory correlations in graphological interference. Journal of Experimental Psychology, 6(4), 336–348. Landay, J. A., & Myers, B. A. (1995). Interactive sketching for the early stages of user interface design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘95) (pp. 43–50). ACM. Lawson, B. (1997). How designers think: The design process demystified (3rd ed.). Architectural Press. Lin, J., Newman, M. W., Hong, J. I., & Landay, J. A. (2000). DENIM: Finding a tighter fit between tools and practice for Web site design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘00) (pp. 510–517). ACM. Lutz, J. A. (1987). A study of professional and experienced writers revising and editing at the computer with pen and paper. Research in the Teaching of English, 21(4), 398–421. Matsuno, T. (2012). Actual and inferred relationships between impressions about handwriting and the writer’s personality (Vol. 14, pp. 31–40). Annual Bulletin of Institute of Psychological Studies, Showa Women’s University. [in Japanese]. Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological Science, 25(6), 1159–1168. Norman, D. A. (1988). The psychology of everyday things. Basic Books. Russo, J. E., Johnson, E. J., & Stephens, D. L. (1989). The validity of verbal protocols. Memory & Cognition, 17(6), 759–769. Schön, D. A. (1983). The reflective practitioner: How professionals think in action. Basic Books. Sharples, M. (1999). How we write: Writing as creative design. Routledge. Shibata, H., & Hori, K. (2008). Cognitive support for the organization of writing. New Generation Computing, 26(2), 97–124. Shibata, H., & Omura, K. (2017). Effects of presentation media and document styles for the evaluation for contents of letters and the personality of the senders. Journal of Printing Science and Technology, 54(1), 49–57. [in Japanese]. Shibata, H., & Omura, K. (2018). Reconsideration of the effects of handwriting: Comparing cognitive load of handwriting and typing. ITE Transactions on Media Technology and Applications, 6(4), 255–261. Suwa, M., Gero, J., & Purcell, T. (2000). Unexpected discoveries and S-invention of design requirements: Important vehicles for a design process. Design Studies, 21(4), 539–567. Tano, S., Yamamoto, S., Dzulkhiflee, M., Ichino, J., Hashiyama, T., & Iwata, M. (2012). Three design principles learned through developing a series of 3D sketch systems: Memory capacity, cognitive mode, and life-size and operability. In Proceedings of IEEE International Conference on System, Man, and Cybernetics (pp. 880–887). IEEE. Vine, I. (1974). Stereotypes in the judgement of personality from handwriting. British Journal of Social and Clinical Psychology, 13(1), 61–64. Willinsky, J. (1989). When university students word process their assignments. Computers in the Schools, 6, 83–96.

Chapter 8

Discussion and Proposals

Abstract In this final chapter, we consider the future of paper and digital media based on the results of the experiments and analysis described so far. We present our idea why digital displays cannot replace paper in the scene of reading and writing. Paper still has strong values in this area and the values are supported by the fact that paper is a physical object. Therefore, the values are difficult to replicate in digital technologies. In order to support reading and writing digitally, we emphasize the operability of media. We also discuss our future visions in more detail in the fields of office work, active reading, and education. In the future office, we think that knowledge workers should selectively use various types of media depending on the situation. We propose the concept of the “stockless office” as one of the ideal images of the future office. Various media should be coordinated with each other in this concept. Next, we present suggestions for ways to support active reading, which accompanies critical thinking or learning. After presenting our personal experiences reading academic books digitally, we present new directions for supporting reading in work or academic situations. We insist on providing tangible interaction with documents, low-cognitive-load operations for documents, flexible page navigation, and the promotion of content touch. Finally, we examine the field of education, presenting our concerns regarding excessive digitalization in education. The use of digital media in children’s reading and writing may degrade children’s reading and writing performance, impede their concentration, and change their attitude to a passive one.

8.1

A Brief Review of the Experiments

Before presenting our proposals and suggestions, we would like to summarize and reflect on the research results introduced in this book. As we saw in Chap. 3, paper was the preferred medium for reading. To investigate the reason for this, Chap. 4 introduced studies examining the influence that the display characteristics of media have on reading. We found that the current digital

© Springer Nature Singapore Pte Ltd. 2020 H. Shibata, K. Omura, Why Digital Displays Cannot Replace Paper, https://doi.org/10.1007/978-981-15-9476-2_8

155

156

8 Discussion and Proposals

media could display high-quality images in terms of reading text, and that differences in the image quality of media had little influence on reading performance. Chapter 5 examined the influence of the operability of media on reading. The results of our analysis showed that reading from paper was more efficient than reading from a computer display or tablet in work-related or academic situations such as reading across multiple documents, reading while moving back and forth between pages, and reading while flipping through pages. Using paper also improved the quality of reading, such as the degree of comprehension of a document or the error detection rate when proofreading. Paper was also excellent for operability of various actions performed during reading (conversely, the operability of digital media was inferior), and this contributed to the superiority of paper in reading and writing. As we saw in Chap. 6, the fact that paper is a simple, non-multifunctional medium means that it has fewer factors to interfere with readers’ concentration. Conversely, the graphical user interface (GUI) components of computers or tablets, such as their menus, icons, blinking cursors, and notification pop-ups, acted as factors that hindered readers’ concentration. Moreover, because digital media are multifunctional and capable of various things, readers’ attention is likely to be pulled in other directions. Chapter 7 dealt with writing and showed that writing on paper was preferred in the context of taking notes and drawing ideas. Writing by hand on paper had the advantage of allowing people to write down information while listening, talking, or thinking. Compared to students who took notes on a laptop PC during a lecture, students who took handwritten notes on paper tended to take notes in their own words and achieved higher test scores after the lecture. In addition, when sending a letter to others, the presentation medium (post cards or emails) and document style (handwritten characters or digital fonts) has a strong effect on the recipient’s impression of the sender. A letter handwritten on paper was found to be more effective from the perspective of personal recognition than a digital message. As we have mentioned repeatedly in this book, paper still provides great value in reading and writing activities. Therefore, it is unlikely that paper will vanish soon. Still, we would like to pursue a more in-depth discussion in this chapter.

8.2

Why Digital Displays Cannot Replace Paper

According to the experiments introduced in this book, we can give rough suggestions regarding how to use presentation media. If you just want to look at information, there is no problem with viewing it on a digital display. Rather, from the perspective of displaying information, using digital display has great advantages, such as the ability to show moving images and update information in real time, compared with presenting fixed information on paper. However, if you want to deeply read a document—more specifically, if you want

8.2 Why Digital Displays Cannot Replace Paper

157

to think while reading a document—it is currently better to read it on paper instead of a digital display as we describe next. When reading a document, people frequently perform various actions on the document such as adjusting its position so that it is easy to read, turning pages, moving back and forth between pages, and tracing text with a finger or pen. It is easy to perform such actions with paper and each action does not hinder reading (see Chap. 5) because paper provides the interaction with information on it through the physical object. In addition, since paper has a simple appearance and does not encourage multitasking, reading on paper promotes concentration (see Chap. 6). If you just want to write down what the lecturer is saying during a lecture, there is no problem with using a PC. Rather, it is easier to read notes back later if you save them on a PC. You can also easily share them with friends via network. However, if you want to understand what the lecturer says and take notes with your own interpretations, it is better to write it down by hand based on the following reason. Handwriting has a low cognitive load (Shibata and Omura 2018), and therefore you can listen to the lecturer’s speech and examine the lecture content while simultaneously writing (Mueller and Oppenheimer 2014). On the other hand, if you take notes on a PC, there is the risk of becoming an “input machine” that writes what the teacher says verbatim (see Sect. 7.3). Moreover, this is a problem that arises regardless of your typing skills. When drawing something, if you have already decided what to draw, and if you just want to externalize what you are picturing in your head, using CAD or drawing tools will not be a big problem. Rather, drawing with a digital tool makes it easy to edit, such as by changing the colors or shapes of the drawn figures, and it can allow drawn figures to be reused later. However, if what you want to draw is not determined from the beginning and you want to think while drawing or after drawing, it is preferable to select hand-drawing on paper at present. In CAD and drawing tools, you must decide what to draw before drawing something by selecting a command (see Sect. 7.4). This burden makes it difficult to think while drawing. Furthermore, with digital drawing tools, the figures drawn are geometrically arranged, which causes the designer’s thoughts to be guided in the direction of arranging objects neatly. Their thinking may be shifted from the reflective thinking of design to the minor modifications of painting (Tano et al. 2012). Now let us think about the question, why digital displays cannot replace paper, which is the title of this book. Our answer is simply because paper has values in reading and writing. The values have been already described above. Paper is easy to handle. Paper promotes concentration on reading because it does not provide any graphical components and commands that disperse intention. Moreover, the cognitive load of handwriting is low and paper provides the best platform to write on in current technological level. Next, we consider why paper can provide such values. These values of paper are strongly due to the affordance of paper as a physical object. We are accustomed to handling a physical object. It is an essential skill for people to survive. Also, the comfortableness of writing on paper is due to the physical compatibility between paper and pens, which have developed in the relation of coevolution for a long time.

158

8 Discussion and Proposals

On the other hand, it is difficult to provide the affordance supported by a physical object in digital technologies. Therefore, it is difficult to realize these values of paper using digital media. Of course, technologies are progressing and the operability of digital media will improve year by year, but we think it takes long time that it can achieve the same level as that of paper. We think this is the main reason why digital displays cannot replace paper. If you want to develop digital media learning from the role of paper on reading and writing, we think that you should improve the operability of documents rather than improving the image quality of the media. We do not intend to disregard the visual features of media, such as their resolution, contrast, and legibility. However, for the purpose of reading and understanding text, the presentation quality of currently available digital media is already good enough. The area in which the current digital media do not reach the level of paper is in their operability. Regarding the operability of documents, it is not an issue that it is acceptable if a system can allow the operations by providing functions. Moreover, it is not an issue that it is acceptable if users can do them quickly. Rather, it is necessary for users to perform operations with a low cognitive load (Shibata 2020). In providing information services or systems, it is often important to “be able to do what was not possible before.” However, in reading and writing work that requires a high cognitive load, not only “being able to do” but also its method and process are important. More precisely, it is required to be able to do with a low cognitive load. This is because there is a risk that adding a small amount of cognitive load may make the original work catastrophic while performing cognitively demanding intellectual work. In the remaining of this chapter, we will discuss future visions in the areas of the office, active reading, and education. In the discussion of supporting active reading, we consider how to realize low-cognitive-load operations in reading.

8.3

Future Media in the Office

This section discusses how media should be used in the future office. We consider how the office ought to be, not how it will be, pursuing the ideal use of media in the office. However, once we understand the ideal, we can consider the direction of future technologies.

8.3.1

Using Both Paper and Digital Media Wisely

So far, we have explained that there are values in using paper during both the reading and writing phases. In a certain situation, the use of paper improves the efficiency and quality of work, and conversely, digital media sometimes reduces it. Therefore, we believe it is wise to try to proactively utilize the advantages of paper in such

8.3 Future Media in the Office

159

situations. The convenience of a given medium depends on the situation at hand: there is no single medium that is the best in every situation. This is something that should be obvious, but we tend to forget it. In the debate comparing paper and digital media, we often hear extreme statements that we must select one or the other and use it in all situations. As a simple example, when the authors give presentations or speeches introducing our comparative study of paper and digital media, we are often asked by audiences, “So, in the end, is your position pro-paper or pro-digital?” We often hear polarized statements: those who prefer paper emphasize its benefits and recommend using it as much as possible, and those who prefer digital media emphasize the benefits of digital media and recommend complete digitalization. These arguments between two choices seem to be oversimplifying the issue. If you want to improve the efficiency and quality of your work, you should categorize your tasks in detail and judge one by one which media type is desirable for each task. Paper may be desirable for a certain type of task, but a PC may be more desirable for another type of task. Or, for some tasks, a tablet may be preferable to a PC. It is sometimes said that Inuit languages have more than 50 words to express the concept of snow. For those who live with snow as a part of their daily lives and something that is often a matter of life or death, it is impossible to express these diverse notions with just a single word. As we saw in Sect. 5.1, there are various types of reading. Sometimes you skim through a document quickly, and sometimes you read it carefully. Reading is also diverse in terms of the place or time in which it is performed. You may read multiple documents in parallel, not just a single document or several people may read a document together instead of one person (Adler et al. 1998). There are also diverse types of writing. We may write formal documents, jot down short memos or sentences, write while speaking, or write while listening. In each of these situations, the optimal medium is different. The fact that only two words, “read” and “write,” are used to describe such a diverse range of reading and writing activities may be a factor to simplify the issue. As an example of a “future office” or “advanced office,” we often see conceptual images showing a future vision of people working in an open space using only one digital device (usually a laptop PC or a tablet device). It seems that a “smart work style” is considered to be one with fewer physical possessions, allowing people to work anywhere with mobility. However, we feel skeptical about the idea of trying to do everything on a single digital device. Instead of restricting your thinking space to a small screen, you should be able to spread information across a larger space, allowing yourself to become physically immersed in your work. Moreover, we think that in order to improve the efficiency and quality of your work, it is important to select and use the most suitable medium for each situation from various tools, including paper. We do not think that confining your thinking to the cramped space of a laptop PC screen or tablet screen and working using your fingertip will bring about a more efficient and creative way of working. The future vision that workers will do everything with a single digital device seems to put too much emphasis on flexible mobile work. This seems to be a

160

8 Discussion and Proposals

work style that neglects the act of thinking, which is an essential activity for human beings. In our view, a laptop PC or a tablet device is like a Swiss Army knife. It comes with various functions: scissors, a screwdriver, a bottle opener, etc. However, since it is designed to be small at the expense of usability, it is not the best tool for performing any task. It would be more efficient and effective to use a dedicated pair of scissors rather than those contained in the Swiss Army knife. The same goes for the screwdriver and the bottle opener. An army knife is good as an emergency portable tool, but it is not a tool used by professionals such as carpenters or cooks. This same way of thinking can be applied to laptop PCs and tablet devices. They are general-purpose tools with a variety of applications. Therefore, they are useful, but they are not the best tool for doing anything. Rather, in many cases, they are not optimal. If you want to maximize the efficiency and quality of your work as a professional worker, we think you should pursue a work style in which various media and devices are used properly, rather than trying to do everything with a single digital device. There is a Japanese proverb that says, “Kohboh does not choose brushes.” Kohboh was a famous Japanese calligrapher. He was able to write beautiful calligraphic works with any brush and did not blame the quality of the brush for the quality of his work. The meaning of the proverb is that “a person who is called a master in a certain field can work well with any tool without complaining about them.” In other words, “A master does not select their tools.” However, our view is a little different. Professionals are obsessed with tools. Professionals never compromise on tools, especially when the value of their work is being measured or when their professional competence is in question. They want to perform the best work with the best tools. There is no reason why only IT workers should be forced to compromise with laptop PCs and tablet devices that are not the best tools for doing anything. Our ideal vision is one in which we choose the most appropriate tools or even use multiple tools simultaneously depending on the purpose and situation of the work (Shibata 2015). If we recommend using tools selectively, some people say that carrying multiple tools is troublesome. However, is there any carpenter who builds a house with a Swiss Army knife because it is too hard to carry many tools? In addition, some people think that it is difficult to select the proper tools in each situation. However, the competence to select tools properly depending on the situation is one of the essential skills as a professional. Of course, it is not always necessary to carry all the tools with us. It is not always necessary to use the optimal tools to do something. It depends on how particular you want to be about the output of your work or if you need to be particular about it. In work where compromises are not allowed, even Kohboh must choose a bush.

8.3 Future Media in the Office

8.3.2

161

Coordination of Paper and Digital Media

In order to realize a method of selecting the optimal medium when using documents, it is important to be able to mutually convert between paper and digital media simply and quickly. In this section, we discuss the importance of media conversion technologies. Even if you know that you can perform a document-related task slightly faster (for example, about 10–30 s faster) when using paper than you can when using a PC screen, we think that you would rarely actually print the document out in order to work with paper. It takes time to print it out and start working. For example, when working on a PC, it is necessary to suspend the work, specify print settings in a dialog menu, and then walk to the printer to pick up the paper output. In offices with shared printer, it is often necessary to go to one located far away from your desk. Therefore, even if we know the efficiency of using paper, it is rare for us to print out documents. At present, the use of printing is usually limited to cases of reading a long document or working on paper for long periods of time, such as when proofreading. Unless the benefit of using paper (i.e., the improvement in work efficiency when compared to working with digital media) largely outweighs the cost of printing (i.e., the time taken to specify print settings, go to the printer, and return to your desk), you would not feel like it is worth it to try to change media and work on paper. However, what if the printer is located on your desk, you can print easily without viewing a dialog menu, and the printing is fast—in other words, what if the total cost of converting from digital media to paper is close to zero? There is the possibility that paper would be used to achieve even slight benefits, such as cases where working with paper would reduce work time by tens of seconds compared with working on a PC. In that case, paper would be used and then recycled for only a few minutes of work. Ultimately, it would be desirable to be able to switch between digital and paper media in the same way as we switch between windows or applications. Similarly, the conversion from paper back to digital data would also need to be performed smoothly. If you are unable to re-capture a printed paper document, including the results of your writing on the paper, as digital data, you might decide to only work with digital data without converting it to paper, even if you are aware of the benefits of using paper.

8.3.3

The Ideal Future Office: The Stockless Office

Many IT companies believe that digitalization of work styles can dramatically improve the efficiency and convenience of work. Many people think that digitalization of documents can greatly reduce both the time and cost of work in terms of

162

8 Discussion and Proposals

storage, transfer, information sharing, retrieval, remote access, and disposal of documents. We basically agree with this idea, but we think the storage form of documents is the most important among these aspects. Paper is superior in terms of ease of handling, and its superiority over digital media becomes prominent when it is operated by hand. If paper documents are filed in cabinets where they go a long time before being held in users’ hands, we cannot say that this is using the advantage of paper as a physical object. On the other hand, if documents are stored digitally, they can be easily transferred, shared, retrieved, remotely accessed, and disposed of. Moreover, they can be converted to appropriate formats and presented on appropriate media at their time of use. Therefore, it is more convenient to store documents digitally rather than in paper form. At present, some organizations are trying to thoroughly eliminate paper from the office. However, considering the above points, our ideal image of the future office would be “an office which does not store paper” (the Stockless Office) rather than “an office which does not use paper” (the Paperless Office) (Shibata 2017). In this future vision, documents are stored digitally as a general rule, except for in the special cases described later. When using documents, paper documents are used when it is desirable to use paper in terms of work efficiency and quality. After using these documents, in most cases, the paper should be discarded and recycled without being stored.1 When people need to use the same document again, it may be output to various types of media again. At that time, the situation of its use (the location or purpose of viewing the document) may be different from before, requiring it to be output to another type of media. The lifespan of paper documents in such a work environment would be relatively short, and it may be possible to print out documents just for several minutes of use. Not storing paper documents for a long time has desirable effects from a security perspective. At present, nearly half of all information leakage incidents have come from paper media, but we can expect this problem to be alleviated within the framework of the stockless office. Moreover, if the lifespan of information on paper becomes short, we hope to expect the advent of new paper and ink with which the written information disappears after a few hours. It is desirable from a security point of view as well if information can disappear after use. Furthermore, if the information can be erased and the paper can be recycled, it is also desirable from an ecological point of view. Many current security systems prevent the leakage of information using software technology. On the other hand, if it is possible to develop ink that lets information

1 Paper is recyclable. Contrary to popular belief, as long as it is recycled after use, paper is environmentally friendly. Moreover, in cases of reading document for long periods of time or sharing documents in a small meeting, the carbon emissions caused by using paper tend to be lower than those from using a PC (Shibata and Omura 2011).

8.4 Toward Digital Active Reading

163

disappear within a few hours, it will become possible to construct an extremely secure system that guarantees information security at the physical level. You may think that the concept of “an office that does not store paper” underestimates the benefits of physical documents, such as displaying books on a bookshelf or desk, physically organizing books, making unread books stand out, and so on. In fact, the books arranged on a bookshelf have many advantages. For example, looking at books or holding them can help us to recall their content, even if we do not open the pages. The cover design, weight, and paper quality of the books works as an external memory aid. Moreover, the physical position of the books in the bookshelf can work as positional memory, and we can access them immediately using our hands when necessary. In addition, living life surrounded by the books you have read brings some sense of joy, satisfaction, and fulfillment. We do not deny these opinions. However, that would be the case in your own home or a private office. In a typical office, such luxury usage of space would be unacceptable. Moreover, the effects of arranging paper books on a bookshelf can be realized digitally to some extent. The realization of a virtual study space in which books and materials that have been read so far can be viewed side by side in a personal bookcase is also a useful concept for “an office that does not store paper.” We should also add that this concept is an ideal of the office pursuing maximum efficiency and convenience, and it oversimplifies the issue somewhat. First, at present, some types of documents, such as contracts and other legal documents, are legally required to be stored in paper form. Even if laws change to allow the storage of such documents in digital form, it will take time for society to change. Additionally, if you intend to preserve documents for very long periods of time, on the order of hundreds to thousands of years, storing documents on paper is actually one of the valid options. To store digital documents for a long time, it is necessary to maintain not only the digital data itself but also the hardware and software applications needed to read it over a long period of time, which is quite difficult. Since the history of the computer itself is less than 100 years, digital technologies have never been used to keep documents for over 100 years. In contrast, paper, as seen with historical documents, has an impressive track record of retaining information in a readable form for periods of over 1000 years. Still, it is our view that situations of using paper for preservation purposes will become less common in the future.

8.4

Toward Digital Active Reading

As mentioned in Chap. 4, when reading a document sequentially from the beginning without backtracking, which is known as receptive reading, visual differences in media (i.e., different display characteristics) do not significantly affect reading performance. For the support of reading for leisure, current digital devices for reading have had a success in a certain level. A typical example is Amazon’s Kindle.

164

8 Discussion and Proposals

In addition, the style of reading books on smartphones or tablet devices instead of dedicated devices for e-books has also been established. However, so-called active reading, which is frequently observed in work or academic situations, is different in nature from the reading for leisure (Schilit et al. 1998). In active reading, readers frequently interact with documents using their hands. In this situation, reading performance was higher with paper than with digital media, as we have shown in this book (see Chap. 5). This is because the operability of the interaction with documents affected the reading performance. Therefore, it is difficult to perform active reading with current e-book readers designed for reading novels or magazines. In fact, many devices and software have been proposed for supporting active reading (Schilit et al. 1998; Golovchinsky 2008; Tashman and Edwards 2011; Golovchinsky et al. 2011; Chen et al. 2012; Hinckley et al. 2012; Zhao et al. 2013; Shibata and Takano 2019), but they have not gained wide spread use. In fact, even among computer science researchers who are able to use rich digital environments, many prefer paper as a medium for reading academic papers (Franze et al. 2014). In this section, we will consider the requirements for a digital environment that supports active reading. Although this book has focused on explaining the value of media quantitatively based on controlled experiments, it is insufficient for capturing some aspects of digital reading such as long-term reading of many books and reading experiences embedded within the user’s daily activities. Therefore, we introduce our personal experiences with digital active reading. First, we summarize critical problems of current digital devices in the reading of our actual work situations. Next, based on them, we present some suggestions for designing digital reading applications and devices.

8.4.1

Personal Experiences of Digital Reading

One of the authors of this book currently reads almost all books digitally. Since I am engaged in the research of media, I think that it is required for me to thoroughly make use of digital devices for reading within my actual work. For reading novels, I have been using e-book readers (the Kindle DX and the Kindle Paperwhite by Amazon) for years. For viewing business documents, I have used tablet PCs (the iPad and the iPad Pro by Apple). However, for reading academic textbooks or papers during my research, I had been using paper books and paper print-outs. Two years ago, I made a big decision. I thought a new A4-size electronic paper device (the Digital Paper by Sony) would be suitable for reading large academic textbooks and A4-size academic papers. I cut out the pages of almost all my academic books (more than 250 books), scanned them, and imported them to the Digital Paper. Since then, I have cut out the pages of all paper books that I newly buy and imported them to the device. Additionally, I attempted to use this device to read

8.4 Toward Digital Active Reading

165

all academic papers (as I explain later, I later gave up on reading academic papers with the Digital Paper). Although I encounter a lot of trouble reading academic books digitally, I continue with this reading style as an experiment, because I believe that making thorough use of digital devices would give me valuable insight into digital active reading. In this section, I point out some critical issues with digital active reading obtained through my personal experience. We do not focus on the particular issues of the specific device (i.e., the Kindle and Digital Paper). We consider it to be a good typical example of the currently available digital reading devices, and so we looked at problems that seem to be common to reading devices in general.

8.4.1.1

It is Hard to Understand Where I Am

When I read a book digitally, I feel that it is difficult for me to intuitively grasp how long the book is and where I am currently reading in the book. Of course, the Digital Paper shows the current page number as well as the page count of the entire book. However, there is a big difference between knowing the length of the book and the current position in the book as numerical values and understanding them as amounts through visual thickness, tactile feedback, and weight. For reference, when using the Kindle, it is more difficult to perceive such information, since the page layout is not fixed. When I started reading e-books, I began to realize the importance of this kind of information, which is something we take for granted in paper books, for the first time. While reading an e-book, I find myself frequently glancing at the page number displayed on the Digital Paper. This may be evidence that I am checking my position to compensate for my lack of a sense of where I am in the book. In addition, whenever I finish reading an e-book, I feel as if I have reached the end of the book suddenly. I think this may be because I lack the feeling that “I will finish soon” even when I am reading the later parts of the book. As a result, I cannot feel a sufficient sense of achievement when I finish reading an e-book. In my opinion, an e-book is like a digital watch, and a paper book is like an analog watch. Both watches give us the same information. However, human perception is quite different. Let us think about a situation in which you look at a watch to know your remaining time during an exam. With a digital watch, you can find out the exact current time, but you must calculate how much time you have remaining. An analog watch is not the best way to tell the exact current time, but you can intuitively perceive the remaining time as an amount, not a numerical value, without calculating. The same information represented differently evokes different perception and different thinking (Zhang 1997).

166

8.4.1.2

8 Discussion and Proposals

I Did not Remember the Author’s Name or the Title

One day, when I was reading a final part of a book, I was shocked to realize that I could not remember the name of the author and even the exact title of the book. I have never encountered such an experience when reading a paper book. I consider there to be two possible reasons for this. First, I never looked at the front cover of the book except for when I first started reading it. In the e-book reader, when you turn the power on, it presents the page you were previously reading. It is convenient. You can resume reading quickly, and you have no need to use a bookmark. However, this convenience diminishes our opportunities to look at the front cover of the book, which vividly conveys the book’s basic information. By using an e-book, my interaction with the book, such as looking at its front cover and opening it to the page I was reading, decreased. I thought that this might have weakened my impression of the book, and I realized the importance of such interaction with a book. Next, I did not refer to the author’s information. When I read a paper book, I often refer to the author bio written at the end of the book repeatedly to check the background of the author and gain an understanding of the context of the book. In so doing, I must jump to the final page of the book and return to my previous page again. However, as we mentioned in Sect. 5.5, it is difficult to go back and forth between pages with current e-book readers and tablets. It seemed that I unconsciously avoided referring to the author’s information and continued reading with only a vague understanding of the context and background of the book.

8.4.1.3

I Did not Remember Where My Knowledge Came From

When I was invited to give a speech at a Japanese newspaper company, I read more than ten books on the newspaper industry and journalism in preparation for my speech. One day, I wanted to refer to a statement by an author I had read. I knew I had read it in one of the books, but I could not remember where it was. I knew I had obtained the knowledge from that particular device and not another device, but I could not remember in which book I had found it. It was an impressive statement, but I could not remember the author or even the context of the statement. Therefore, it took me a great deal of time to find the description. I encountered such an experience many times while I was surveying information for those books. I felt that the frequency of such an experience was greater than when I had read paper books in the past. I gained much knowledge from the e-books, but I failed to organize that knowledge. I realized that a book was a structured set of knowledge, and that I had to read a book with an awareness of this structure. Reading a book is not an accumulation of the understanding of fragmentary text. Obtaining information from text and reading a book as a set of structured knowledge are two different things. For me, the e-book reader made the former possible, but it did not support the latter well. It seems to me

8.4 Toward Digital Active Reading

167

that reading with current e-book readers is more akin to reading fragmentary text on the Web rather than gaining structured knowledge of the book. We can think of two possible reasons for this. First, I did not interact with the e-book frequently, and I had a weak impression of the book. For paper books, our interaction with the book generally increases, and interaction with the book means interaction with the structured knowledge of the book. For example, when we arrange books on a bookshelf, it means arranging the knowledge. When we glance over at the books on our bookshelf, it means glancing over at the knowledge. When we look at the front cover while reading, it helps remind us of the context of the book. Second, as Kaufman and Flanagan (2016) pointed out, in a digital environment, people become more conscious of concrete and detailed information (see Sect. 6.3). It is possible that I myself lapsed into this situation, that is, I unintentionally focused on obtaining specific, concrete details without grasping the overall meaning of the book.

8.4.1.4

It is Hard to Deviate from the Main Text

With the Digital Paper, I felt that I was processing the text of books single-mindedly from the beginning to the end. The direction of reading was almost always linear, and I rarely deviated from the main text. This caused various problems for my deep reading. While reading with the device, because I did not go back to the table of contents, I felt I could not grasp the whole structure of the book. Additionally, academic and business books usually contain many figures and tables, and the sentences referring to them are often located on different pages including the figures and tables themselves. Even when reading such books, I often did not refer to the figures and the tables, and this seemed to give me a shallow impression of the figures or tables or even of the content of the book. Moreover, when I read paper books, I frequently referred to the references listed at the end of the book and searched on the Web to get more detailed information; that is, I deviated from the book in a good sense. However, I rarely did so when reading e-books. I have the feeling that not accessing any related information blocked my ability to consider or think critically about the book. The reason why I rarely diverged from the main text when reading e-books was obvious for me: it was due to the difficulty in page navigation with the device. For example, in order to refer to a figure on another page, you must move to the new page to look at the figure and then return to the previous page again. In such cases, as we saw in Sect. 5.5, with a paper book, you can simply insert a finger between the pages to mark where you were previously reading before jumping to the new page (Shibata et al. 2014). This makes it possible to easily return to your previous spot after referring to the figure. The inserted finger works as a temporary bookmark for page navigation. However, such inter-page navigation cannot be performed smoothly with the Digital Paper, and this is the same with other digital environments as well.

168

8 Discussion and Proposals

Marshall and Bly (2005) have demonstrated that while reading paper magazines at home, people frequently look ahead or backtrack quickly, turn pages in half, and flip through pages. They referred to this as light-weight navigation and pointed out that it was an unconscious act that the readers themselves were hardly aware of. Although light-weight navigation is possible with e-books, it has not reached the level of operability in which we can do it unconsciously. In reading that requires deep understanding, such operations may interfere with your reading if they cannot be performed unconsciously while reading. It seems that light-weight navigation is performed more frequently when reading academic books than when reading magazines. My experiences with digital reading made me realize that I frequently performed light-weight navigation more than I thought, and that the lack of this navigation may be blocking me from comprehending the book. For reference, I also attempted to read academic papers with the Digital Paper in the beginning, but I have given up on this. My main reason for giving up was due to the difficulty in page navigation (Shibata et al. 2016b). I felt a strong sense of frustration in not being able to smoothly and flexibly navigate through pages while reading papers digitally. The support for light-weight navigation seems to be an essential requirement for digital environments for reading academic papers.

8.4.2

Suggestions for Supporting Digital Reading

In the previous section, we described many problems I have experienced during my digital active reading. However, I must say that I also felt many advantages of digital reading. In fact, I continue to read digitally even now (except for the case of reading academic papers). By using digital devices, I can bring many books with me and start reading them quickly, anytime, anywhere. The function to enlarge the text is also useful for someone like me who is growing farsighted. These are just some of the advantages of digital reading. However, there are much to be improved in current digital reading devices toward the use of them in work or academic situations. In this section, we present suggestions for digital environments to support active reading.

8.4.2.1

Strengthening the User’s Impression of the Book

If a system provides a command for displaying the book’s information, users can easily refer to the author bio and the book title. Kindle provides such a function for specially-designed e-books. However, to strengthen the user’s impression of the book, it is not sufficient. Users will not refer to the book information frequently beyond necessity. That is because selecting a command is a conscious act, and lightweight navigation is an unconscious act.

8.4 Toward Digital Active Reading

169

It would be desirable for a digital environment to provide a framework in which users can learn the book title, the author’s name, the length of the book, the position where they are currently reading, and other important information related to the book, naturally or at least easily, through the action of handling the book. In other words, when they are reading a book, the environment should make them naturally aware of the background of the book, the context of the reading, and the differences between that current book and other ones. To achieve this, we think that it would be desirable to provide a tangible environment in which users could operate books with tactile feedback using physical objects. Mangen et al. (2013, 2019) found that reading an e-book did not promote reconstructing the plot of a story. They believed that a coherent temporal organization of the story would be supported by the kinesthetic feedback of page turning and that the operation of physical paper sheets with the reader’s hand contributed to their organization of temporal information of the story. We also think that the kinesthetic feedback of handling physical books would promote users’ awareness of each book and their organization of structured knowledge.

8.4.2.2

Low-Cognitive-Load Operations for Documents

In order to support reading and writing activities, we have emphasized the importance of low-cognitive-load operations for documents. To achieve this, it may be effective to provide a tangible environment in which each document can be manipulated physically. People are good at manipulating physical objects. In addition, by manipulating each document as a physical object, we would be able to effectively utilize positional memory and the movement of the body by arranging documents spatially or organizing them in piles. If people can handle information unconsciously, they become able to handle externalized information as the extension of the brain. Being able to handle information unconsciously is an essential requirement in order for people to use that information to create ideas or make important decisions. In order to improve the operability of the device, it is significant to make the device lightweight and thin so that it is easy to hold and move documents. Furthermore, in order to easily hold the device placed on the desk, it is necessary to take measures such as bending the end of the device to make it easy to put in a finger under the device or making it flexible like paper. It is also necessary to use a material that does not scratch the desk or device even if you slide the device on the desk and that does not make a loud noise when the device is placed on the desk. In short, it is necessary to design a device that can be easily handled like an analog stationery rather than an electronic device as a precision machine (Shibata 2020). In knowledge work or academic scenes, information from multiple document sources is often compared (O’Hara et al. 2002), so it is important to be able to easily arrange and move documents. In active reading, people often trace text of a document and write down marks or text in the document. The more people interact with a document, the more they adjust the position and angle of the document so that it can

170

8 Discussion and Proposals

be easily operated by hand (Shibata et al. 2018). Therefore, even when reading a single document, it would be important to be able to unconsciously adjust the position and angle of the document. As a catch copy to sell, information devices using electronic paper panels (Amazon Kindle, Sony Digital Paper, etc.) often appeal that a reflective panel is eye-friendly. However, since an electronic paper panel does not require glass for surface protection, a lightweight and thin device can be constructed using it. This point is a great advantage in realizing a device that supports reading. In fact, when we analyzed people’s behavior with different media, their behavior toward an electronic paper device (a white-colored device made from thin, flexible, lightweight plastic material) was close to the behavior toward a sheet of paper in comparison with the behavior toward a LCD tablet device (Shibata and Omodani 2019). Next, we should use the interaction of handwriting effectively as a method of inputting information with low cognitive load. In Sect. 7.3.2, we described how the cognitive load of handwriting was lower than that of typing with a keyboard even for people who are good at typing (Shibata and Omura 2018). While writing by hand, people can think about other things, listen to other people’s speech, and talk with other people. Moreover, people can also draw rough sketches or figures in addition to text, with no mode switching necessary to switch between both these modes of expression. Finally, for the pursuit of low-cognitive-load operations, it would be effective to provide users with various ways of achieving goals. If there are multiple ways to achieve a goal, users can select the best way depending on the situation, allowing them to select the best way to avoid disrupting their reading or thinking.

8.4.2.3

Supporting Flexible Page Navigation

To support active reading, which proceeds in a zig-zag manner, providing support for flexible page access is essential. Many studies have pointed out the difficulty of flexible page access in current e-book readers when used in academic or work situations (Princeton University 2009; Thayer et al. 2011; Shibata et al. 2016b). Some people insist that scrolling is sufficient for digital reading since digital documents do not have the notion of pages. However, the scrolling is effective only in the case of linear reading. It is hard to use scrolling for zig-zag reading that involves frequently going back and forth, because we must visually check our scrolling position, which interrupts our reading. Additionally, in such environments of endless scrolling, our location memory of where information is located on the page also becomes meaningless, because the page layout is not fixed. To support active reading, it is desirable to introduce the notion of pages so that our location memory can be used effectively. With a paper book, we flip the pages using our fingers’ sense of touch, which means we do not need to direct our line of sight to the operation position while flipping pages. Since successful page turning can also be confirmed by our fingers’

8.4 Toward Digital Active Reading

171

sense of touch, there is no need for us to visually confirm whether the page has been turned correctly after each page is turned. Furthermore, by using the finger as a temporary bookmark, it is easy to move back and forth between pages. This is an essential requirement when referring to other pages during reading, when comparing information on different pages or when flipping through pages to find the necessary information. With a paper book, you can unconsciously set up anchors in multiple places in the book using all ten fingers of both hands. This enables the flexible page access of paper books. The importance of supporting page access for active reading has been recognized for a long time, and various systems have been proposed in research.2 However, no single system is widely accepted as a practical solution. For approaches using software solutions, the ability to provide haptic feedback becomes the bottleneck. For approaches using special input devices, the cost and the size of devices seems to be the bottleneck. If there is no general solution available at the current time, it will be necessary to find solutions individually according to the situation and purpose. We must provide different support methods for each reading situation. It is desirable that systems provide various customizable page access methods so that users can select the right methods depending on the situation.

8.4.2.4

Not Prohibiting Content Touch

As we saw in Sect. 5.6, we frequently point at words or trace sentences using a finger or a pen (i.e., content touch) during active reading. If these actions are restricted in active reading, reading performance may suffer (see Sect. 5.6.2; Shibata et al. 2015, 2016a). Additionally, content touch is often performed unconsciously. Therefore, it is necessary not to prohibit content touch during active reading. When using tablets or smartphones, we can easily turn pages using touch operations. However, a tap gesture used to turn a page is the same as pointing to a word, and a swipe gesture is the same as tracing a sentence. Therefore, the reader’s unconscious acts of content touch are recognized by the device as taps or swipes, causing unintentional page turning and interfering with the reader’s concentration. To support active reading on touch-sensitive devices, we must carefully design touch

2 There have been many trials of providing tactile feedback for page turning. To turn pages, these systems may use a dedicated page-turning button (Schilit et al. 1998), bendable input devices (Schwesig et al. 2004; Watanabe et al. 2008), or flexible slate devices (Gallant et al. 2008; Wightman et al. 2011). To support flexible page access, some systems provide an overview of pages (Cockburn et al. 2006; Li et al. 2013) or a visualization of the user’s page navigation history (Alexander et al. 2009). Additionally, in order to support moving back and forth between pages, some systems allow temporary bookmarks to be added by touch operation (Yoon et al. 2011), or the use of a paper book itself as a page-turning input device (Masunaga et al. 2017).

172

8 Discussion and Proposals

operation gestures so as to not compete with other gestures naturally performed during reading.3 Additionally, in our experiments, some participants hesitated to touch the glassy panel of a tablet, not wanting to leave their fingerprints on the screen used for reading. Moreover, the shiny surface of the glass makes it difficult to read when light is reflected on it, and readability is dependent on the reading angle. It may be desirable to avoid using a shiny glassy material for the surface used for reading.

8.5

Media in Education

This book has discussed the impact of media on reading and writing performance. In most experiments in this book, the participants were either office workers or college students, that is, adults over the age of 18. However, it seems that the impact of media on reading and writing performance will be more severe in children than in adults. Although we are not experts on pedagogy, we would like to describe our views on the digitalization of education based on the results of various studies comparing paper and digital media.

8.5.1

Concerns About the Digitalization of Education

It is obvious that digital tools bring value to education. Digital tools have many advantages that cannot be provided by conventional paper media, such as playing audio pronunciations of words in foreign languages, showing the procedure of a scientific experiment in a video, checking the progress of students’ homework, sharing information with parents through social media, etc. However, in situations where students are expected to read documents carefully or solve math problems, we need to be cautious about using digital tools. In consideration of the comparative studies of media, we feel skeptical about the digitalization of education based on the following aspects.

8.5.1.1

Deterioration of Reading Performance

First, we are concerned that the digitalization of education may lead to a decline in children’s academic abilities. We have repeatedly described that, unlike reading novels, active reading is accompanied by frequent interaction with documents, such

We proposed a touch operation framework supporting flexible page navigation without restricting content touch (Shibata and Takano 2019). We think this is one of the possible solutions for supporting active reading on touch-sensitive devices. 3

8.5 Media in Education

173

as holding or moving documents, turning pages, and pointing at text. In such reading, the superiority of paper has been significantly demonstrated (see Chap. 5). When children read textbooks and reference books, they are likely to interact with the documents frequently. The authors have observed the learning process for fill-in-the-blank tests on social studies in an experiment (Takano et al. 2014). In this process, the participants frequently underlined or circled important points, drew arrows, wrote words in the margins to memorize them, tapped words, hid important words with a hand or pen to check whether they memorized the words. As we saw in Chap. 5, the use of digital media has been demonstrated to reduce the efficiency and quality of work in the case of reading with frequent document interaction, and reading in education includes frequent interactions with documents. Therefore, if digital media are used in an educational setting, this may degrade the efficiency and quality of learning. Additionally, since children’s reading and writing abilities have not yet been developed, it seems that they are prone to the effects caused by differences in media. Although there have been some studies showing no significant difference between reading from paper and reading from digital media (Higgins et al. 2005; Dundar and Akcayir 2012), many studies observed that reading from paper was significantly superior. For example, for students under 12 years old, Grimshaw et al. (2006) revealed that children took more time to read from a computer screen than they did to read from paper. Jeong (2012) found that students who read a paper book showed better reading comprehension compared to students who read an e-book. For students under 15 years old, Mangen et al. (2013) found that they could reconstruct the plot of stories better when reading from a paper book than they could when reading in a computer environment. Porion et al. (2016) found that students scored higher on tests of inference comprehension when reading text from paper than from a computer screen. For students under 18 years old, Kang et al. (2009) found that students read faster from paper than from an e-book reader. Kim and Kim (2013) found that students read faster and scored higher on comprehension tests when reading from paper than when reading from a computer screen. These experiments all dealt with reading novels or explanatory articles, in which interactions with documents are relatively few compared with the work-related reading examined in Chap. 5. However, they all found significant differences in reading performance between reading from paper and reading from digital media. The negative effects of the use of inappropriate media on reading performance seem to be more serious for children than for adults.

8.5.1.2

Impeding Concentration

Our second concern is that the current digital tools come with many impediments to concentration. As we saw in Chap. 6, the graphical user interface (GUI) components of digital tools, such as menus, toolbars, blinking cursors, and pop-ups, become visual disturbances that hinder the reader’s concentration. Moreover, access to

174

8 Discussion and Proposals

games and the Internet via digital tools becomes a temptation to escape from one’s duties. Arai (2012) of the National Institute of Informatics points out that the detailed or related information provided by hypertext rarely aids readers’ understanding of sentences. Rather, when readers feel difficulty in comprehending a sentence, they may click on the links in the text to escape from the stress of that difficulty. Then, they may refer to the linked information, escape from concentration, and obtain an unnecessary sense of satisfaction by checking a lot of information that does not lead to their comprehension of the sentences. Hypertext may be useful for those who have acquired specialized reading skills in a certain area. However, Arai points out that people with poor reading skills are less likely to be encouraged to read with the help of hyperlinks, and that hyperlinks may rather become a disincentive to concentration. In the face of difficulties in reading comprehension or calculation, if a factor interfering with concentration arises, people may become easily distracted by the interference and give up trying to comprehend the text carefully. If this is a temporary problem that only occurs when using digital tools, the problem may not be so serious. However, if children do not have the experience of thinking carefully while reading during their childhood, and if they grow up without learning to do so, it may be problem they are unable to remedy for the rest of their lives.

8.5.1.3

Affecting Children’s Attitudes Toward Learning

Our third concern is the possibility that the rich information and interactive applications provided by digital tools may make children’s attitudes toward learning more passive. A comparative experiment of proofreading on paper and proofreading on tablets (Sect. 5.6) found that tablets did not facilitate content touch, and the error detection rate on tablets was lower than on paper (Shibata et al. 2015, 2016a). In this experiment, a participant reported that “by reading without touching the paper document, I felt as if I was viewing it on a bulletin board from a distance.” We can expect that manipulating a document with your own hands increases your sense of ownership of the document, making you feel as if the document is the extension of your body. By using a tablet that does not promote content touch, children may feel as if they are reading documents from a distance, and this may prevent them from having an attitude of learning actively. In fact, when observing my daughter, I sometimes feel that there is a big difference in her attitude toward learning when using a paper textbook and when using a tablet application. She enjoys learning on the tablet. However, whenever she encounters a problem that she cannot solve, she immediately tries to refer to the answer without trying to think it over carefully. In addition, she seems to be satisfied with the fact that such a reaction can be obtained from the device, and I sometimes feel as if her goal might be to obtain the reaction itself. It seems that she thinks that a digital device is not a place for thinking but rather a place to refer to answers or a place to enjoy interactive communication with the system.

8.5 Media in Education

8.5.1.4

175

Current Textbooks Have Been Made with Paper in Mind

Our last concern is that the current existing educational content and teaching methods have been made based on the premise of using paper, and therefore, they have been made to be optimal for being shown on paper. For example, in the case of the process of going paperless, existing work styles in offices have been established with the assumption of the existence of paper as a presentation medium, so attempts to replace paper resulted in not just a change in medium but also the problem of changing work styles (Shibata 2017). As a result, there have been many examples of failed attempts at going paperless in the office (Sellen and Harper 2001), and the progress toward realizing the paperless office is currently moving at a slow pace. The same problem is also relevant to the computerization of education. The present teaching materials have likely been designed and optimized for paper in the process of their coevolution with paper. Therefore, we can expect that various problems in learning and teaching methods will arise if paper materials are computerized as-is. It is thus desirable for the teaching materials and teaching methods themselves to be restructured for digital use.

8.5.2

The Digitalization of Children’s Textbooks Should Be Considered Carefully

What happens to children if their textbooks are computerized and they grow up without having the experience of thinking carefully? According to developmental psychologist Wolf (2008), the circuit of the brain (the structure of synapse connections) used for reading text is formed after birth through learning during childhood. Wolf points out that people will have problems reading written language throughout their lifetimes if they do not have the necessary learning to build their brain’s hardware for reading. For adults who have already achieved a certain level of reading and writing ability, it may not be a big problem even if they select the wrong medium for reading and writing and reduce their performance temporarily, or if they are guided in the wrong learning direction by selecting the wrong tools. However, if the subjects are children, spending time learning with the wrong medium during their learning years may cause irrevocable problems in the sense that they cannot develop the necessary attitude for reading and thinking. As mentioned earlier, the use of digital media as teaching materials does have significant advantages over the use of paper materials. It may be possible for digital media to implement methodologies or learning programs for more efficient learning and improved memorization and to encourage children to actively engage in learning. On the other hand, the use of paper teaching materials is also beneficial.

176

8 Discussion and Proposals

Therefore, as mentioned repeatedly, it is necessary to selectively use paper and digital media properly according to the intended purpose and the situation. However, neither students nor teachers currently have enough knowledge to judge when paper teaching materials or digital devices should be used. Therefore, it will become an urgent task to develop guidelines on the selective use of media based on an empirical knowledge of education. The most dangerous approach would be to give tablet devices to children, indiscriminately digitalize all teaching materials, and pack them into the devices. As a matter of fact, the effects and risks of using digital tools in education are not clear. There is even a chance that the negative effects caused may be so small that people who do not use digital tools may be negatively affected instead, in the sense that they do not benefit from the great advantageous effects of digital media. However, we think that there is no harm in waiting to use digital tools to develop basic skills in reading and writing until we can correctly estimate the value and risks associated with their use. Rather, we think we should be concerned about the risk of careless computerization and causing irreparable damage.

References Adler, A., Gujar, A., Harrison, B. L., O’Hara, K., & Sellen, A. J. (1998). A diary study of workrelated reading: Design implications for digital reading devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘98) (pp. 241–248). ACM. Alexander, J., Cockburn, A., Fitchett, S., Gutwin, C., & Greenberg, S. (2009). Revisiting read wear: Analysis, design, and evaluation of a footprints scrollbar. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘09) (pp. 1665–1674). ACM. Arai, N. (2012). Honto ni iino? Dejitaru kyoukasho. Iwanami Shoten, Publisher. [in Japanese]. Chen, N., Guimbretière, F., & Sellen, A. (2012). Designing a multi-slate reading environment to support active reading activities. ACM Transactions on Computer-Human Interaction, 19(3), 1–35. Cockburn, A., Gutwin, C., & Alexander, J. (2006). Faster document navigation with space-filling thumbnails. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘06) (pp. 1–10). ACM. Dundar, H., & Akcayir, M. (2012). Tablet vs. paper: The effect on learners’ reading performance. International Journal of Elementary Education, 4(3), 441–450. Franze, J., Marriott, K., & Wybrow, M. (2014). What academics want when reading digitally. In Proceedings of the ACM Symposium on Document Engineering (DocEng ‘14) (pp. 199–202). ACM. Gallant, D. T., Seniuk, A. G., & Vertegaal, R. (2008). Towards more paper-like input: Flexible input devices for foldable interaction styles. In Proceedings of the Symposium on User Interface Software and Technology (UIST ‘08) (pp. 283–286). ACM. Golovchinsky, G. (2008). Reading in the office. In Proceedings of 2008 ACM workshop on Research advances in Large Digital Book Repositories (pp. 21–24). Golovchinsky, G., Carter, S., & Dunnigan, A. (2011). ARA: The active reading application. In Proceedings of the ACM Multimedia (MM ‘11) (pp. 799–800). ACM. Grimshaw, S., Dungworth, N., McKnight, C., & Morris, A. (2006). Electronic books: Children’s reading and comprehension. British Journal of Educational Technology, 38(4), 583–599.

References

177

Higgins, J., Russell, M., & Hoffmann, T. (2005). Examining the effect of computer-based passage presentation of reading test performance. The Journal of Technology, Learning and Assessment, 3, 4. Hinckley, K., Bi, X., Pahud, M., & Buxton, B. (2012). Informal information gathering techniques for active reading. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘12) (pp. 1893–1896). ACM. Jeong, H. (2012). A comparison of the influence of electronic books and paper books on reading comprehension, eye fatigue, and perception. The Electronic Library, 30(3), 390–408. Kang, Y.-Y., Wang, M.-J., & Lin, R. (2009). Usability evaluation of e-books. Displays, 30(2), 49–52. Kaufman, G., & Flanagan, M. (2016). High-low split: Divergent cognitive construal levels triggered by digital and non-digital platforms. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘16) (pp. 2773–2777). ACM. Kim, H. J., & Kim, J. (2013). Reading from an LCD monitor versus paper: Teenagers’ reading performance. International Journal of Research Studies in Educational Technologies, 2(1), 15–24. Li, L.-Y., Chen, G.-D., & Yang, S.-J. (2013). Construction of cognitive maps to improve e-book reading and navigation. Computers and Education, 60(1), 32–39. Mangen, A., Olivier, G., & Velay, J.-L. (2019). Comparing comprehension of a long text read in print book and on Kindle: Where in the text and when in the story? Frontiers in Psychology. Mangen, A., Walgermo, B. R., & Brønnick, K. (2013). Reading linear texts on paper versus computer screen: Effects on reading comprehension. International Journal of Educational Research, 58, 61–68. Marshall, C., & Bly, S. (2005). Turning the page on navigation. In Proceedings of ACM/IEEE Joint Conference on Digital Libraries (JCDL ‘05) (pp. 225–234). ACM. Masunaga, S., Xu, X., Terabe, T., Shibuta, K., & Shibata, H. (2017). A paper book type Input device for page navigation in digital documents date of evaluation. IEICE Transactions on Electronics, E100-C(11), 984–991. Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological Science, 25(6), 1159–1168. O’Hara, K. P., Taylor, A., Newman, W., & Sellen, A. J. (2002). Understanding the materiality of writing from multiple sources. International Journal of Human-Computer Studies, 56(4), 269–305. Porion, A., Aparicio, X., Megalakaki, O., Robert, A., & Baccino, T. (2016). The impact of paperbased versus computerized presentation on text comprehension and memorization. Computers in Human Behavior, 54, 569–576. Princeton University. (2009). The e-reader pilot at Princeton. Schilit, B. N., Golovchinsky, G., & Price, M. N. (1998). Beyond paper: Supporting active reading with free form digital ink annotations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘98) (pp. 249–256). ACM. Schwesig, C., Poupyrev, I., & Mori, E. (2004). Gummi: A bendable computer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘04) (pp. 263–270). ACM. Sellen, A. J., & Harper, R. H. (2001). The myth of the paperless office. The MIT Press. Shibata, H. (2015). Future of the paper in the office. Shigyo Times, 67(18). [in Japanese]. Shibata, H. (2017). Why does not paperless office come? Where is paper used in the future. Journal of the Imaging Society of Japan, 56(5), 538–533. [in Japanese]. Shibata, H. (2020). Cognitive science of media and reading-and-writing. Journal of the Imaging Science of Japan, 59(2), 204–211. [in Japanese]. Shibata, H., Fukase, Y., Hashimoto, K., Kinoshita, Y., Kobayashi, H., Nebashi, S., et al. (2016b). A proposal of future electronic paper in the office: Electronic paper as a special-purpose device cooperating with other devices. ITE Transactions on Media Technology and Applications, 4(4), 308–315.

178

8 Discussion and Proposals

Shibata, H., & Omodani, M. (2019). Evaluating digital slate devices from users’ behavior: Electronic paper devices as stationery. Journal of the Imaging Society of Japan, 58(4), 370–377. Shibata, H., & Omura, K. (2011). Paper and electronic media as presentation media: A comparison from the environmental point of view. Fuji Xerox Technical Report, 20, 85–95. [in Japanese]. Shibata, H., & Omura, K. (2018). Reconsideration of the effects of handwriting: Comparing cognitive load of handwriting and typing. ITE Transactions on Media Technology and Applications, 6(4), 255–261. Shibata, H., Omura, K., & Qvarfordt, P. (2018). Optimal orientation of text documents for reading and writing. Human-Computer Interaction, 35(1), 70–102. Shibata, H., & Takano, K. (2019). Touch operations supporting flexible page access and text touching during active reading. Journal of the Information Processing Society of Japan, 60 (10), 1929–1940. [in Japanese]. Shibata, H., Takano, K., & Omura, K. (2014). Comparison of paper and computer displays in reading including frequent movement between pages. In Proceedings of the Australian Computer-Human Interaction Conference (OZCHI ‘14) (pp. 549–558). ACM. Shibata, H., Takano, K., & Tano, S. (2015). Text touching effects in active reading: The impact of the use of a touch-based tablet device. In Proceedings of the IFIP International Conference on Human-Computer Interaction (INTERACT ‘15), Lecture Notes in Computer Science (Vol. 9296). Springer. Shibata, H., Takano, K., & Tano, S. (2016a). Text touching effects: The impact of the use of a touch-based tablet device in active reading. Journal of the Information Processing Society of Japan, 57(9), 2131–2141. [in Japanese]. Takano, K., Shibata, H., Ichino, J., Hashiyama, T., & Tano, S. (2014). Microscopic analysis of document handling while reading paper documents to improve digital reading device. In Proceedings of the Australian Computer-Human Interaction Conference on Designing Futures (OZCHI ‘14) (pp. 559–567). ACM. Tano, S., Yamamoto, S., Dzulkhiflee, M., Ichino, J., Hashiyama, T., & Iwata, M. (2012). Three design principles learned through developing a series of 3D sketch systems: Memory capacity, cognitive mode, and life-size and operability. In Proceedings of IEEE International Conference on System, Man, and Cybernetics (pp. 880–887). Tashman, C., & Edwards, W. K. (2011). LiquidText: A flexible, multitouch environment to support active reading. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘11) (pp. 3285–3294). ACM. Thayer, A., Lee, C. P., Hwang, L. H., Sales, H., Sen, P., & Dalal, N. (2011). The imposition and superimposition of digital reading technology: The academic potential of e-readers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘11) (pp. 2917–2926). ACM. Watanabe, J., Mochizuki, A., & Horry, Y. (2008). Bendable device for browsing content using the metaphor of leafing through the pages. In Proceedings of the International Conference on Ubiquitous Computing (UbiComp ‘08) (pp. 360–369). ACM. Wightman, D., Ginn, T., & Vertegaal, R. (2011). Bendflip: Examining input techniques for electronic book readers with flexible form factors. In Proceedings of IFIP Conference on Human-Computer Interaction (INTERACT ‘11) (pp. 117–133). Springer. Wolf, M. (2008). Proust and the squid: The story and science of the reading brain. Harper Perennial. Yoon, D., Cho, Y., Yeom, K., & Park, J.-H. (2011). Touch-Bookmark: A lightweight navigation and bookmarking technique for e-books. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘11) Extended Abstracts (pp. 1189–1194). ACM. Zhang, J. (1997). The nature of external representations in problem solving. Cognitive Science, 21 (2), 179–217. Zhao, Y., Qin, Y., Liu, Y., Liu, S., & Shi, Y. (2013). QOOK: A new physical-virtual coupling experience for active reading. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST ‘13). New York: ACM.

Chapter 9

Conclusion

Abstract As a conclusion of this book, we briefly summarize important statements of this book and describe our future vision. Paper has values as a medium for reading and writing. We learned why digital displays cannot replace paper. We want to make use of the findings to design a digital environment for reading and writing.

Paper, computers, tablets, and smartphones are all familiar things that we use in our daily lives. Therefore, many people have deployed their own arguments comparing paper and digital media. However, many of these arguments are qualitative and based on individual experiences and preferences. They also include many contradictory claims, since their assumptions and conditions are not clear. In this book, based on psychological experiment methods, we clarified work tasks as experimental conditions and tried to examine the effects of paper and digital media as quantitatively as possible. In other words, we tried to clarify each medium’s relative strengths and weaknesses scientifically, based on numerical values and data. Many experiments have been introduced in this book, but of course, they do not cover all of the wide variety of reading and writing situations. Particularly, the following research directions are strongly expected to be studied in the future: analysis of the influence of media type on learning and memory in children and the elderly, analysis of the cognitive effects of body movement on information access and memorization, and neuroscientific analysis of the influence of media type on reading and writing. As mentioned in this book, the influence of media type on children’s attitudes toward learning as well as education processes is also an important issue and we want to ask for arguments based on objective data beyond empirical theory. Although there is still much to do, we have summarized our past achievements in the form of a book because we wanted a correct understanding of the influence of media on reading and writing to be spread within society. In particular, this book focused on introducing the benefits of paper that are often overlooked due to the overwhelming popularity and strong expectations of digital media. People say that paper is easy to read, but the readability of paper is due to its ease of handling rather than its ease of viewing. If you are just looking at a document, it is difficult to see a big difference in reading performance whether you look at it in the © Springer Nature Singapore Pte Ltd. 2020 H. Shibata, K. Omura, Why Digital Displays Cannot Replace Paper, https://doi.org/10.1007/978-981-15-9476-2_9

179

180

9 Conclusion

form of paper or digital media. Rather, in cases when performing deep reading is not necessary, there are many advantages in digital media, such as the ability to display moving images and information that changes in real time. However, if, in addition to viewing, you want to deeply understand the content of a document, you will need to frequently move and arrange the document, flip through its pages, go back and forth between different pages, point at text, and trace text. In such a situation, the superiority of paper over digital media is clearly demonstrated. Documents are, of course, read with one’s eyes, but sometimes they are also read by one’s hands. Paper as a tangible physical object exerts a great power over the ease of manipulating documents using the hands. Performing operations on paper does not exert a high cognitive load, and it does not interfere with the process of thinking while reading. The value of paper becomes prominent in situations where you are not only looking at documents but also using your hands to handle the documents. If we want to emphasize the strengths of paper, we could say that paper is a form of operation media rather than presentation media. In addition, paper does not interfere with the reader’s concentration. It looks simple, is not as multifunctional as digital media, and has no visual movement, which is why you can concentrate on reading. You can also write on it. When simply inputting text, there are many people who can type faster than they can write by hand. However, a big advantage of handwriting is that you can simultaneously talk, listen, or think while writing. We emphasized paper’s utility, but we have no intentions of denying the use of digital media. As we have described many times throughout the book, digital media have great value. In fact, the two authors frequently use and take advantage of digital media in our workplace, a computer science laboratory in a company, where we are surrounded by advanced technologies. However, as with any media, there are also negative effects behind the benefits. It is the claim of this book that we should properly recognize the harmful effects of digitalization and make selective use of each medium based on their characteristics or use both of them together depending on the situation, rather than computerizing everything indiscriminately. We learned about the nature of reading and writing on paper and examined how the use of different media impacts the process or performance of reading and writing. We believe that this knowledge can be used to help build a better digital environment for reading and writing. For deep reading and note-taking, digital media have struggled to replace paper. The requirements of digital displays for viewing and digital displays for reading are quite different. In developing digital media for reading, it is necessary to focus on supporting various actions that occur during reading rather than simply improving the image quality. It is not sufficient to merely provide functions that support these actions. It is important to reduce the cognitive load of the actions and to enable the user to think simultaneously while performing the operations. Here lies the difficulty in developing digital displays for reading. Regarding digital media used for writing, we should proactively use handwriting, with its low cognitive load, when taking notes or generating ideas. To support the

References

181

initial stages of design, we should also take care not to encourage the wrong cognitive mode by showing geometrically arranged figures. In recent years, one of the authors has been working on developing a digital environment for reading and writing. The aim is to develop a “next-generation pen and paper” using computers as a medium beyond paper (Shibata 2020a, b). If I am able to successfully achieve this aim, it will be thanks to my learning the nature of reading and writing on paper and the drawbacks of the currently available digital media. If, however, I am unable to achieve this aim, it will be proof that paper is not so weak. In any case, I am sure I will appreciate the benefits of having learned deeply about media for reading and writing.

References Shibata, H. (2020a). Toward the special issue on “Beyond-Paper Interaction”. Journal of the Imaging Science of Japan, 59(2), 202–203. [in Japanese]. Shibata, H. (2020b). Cognitive science of media and reading-and-writing. Journal of the Imaging Science of Japan, 59(2), 204–211. [in Japanese].