Advances in Design and Digital Communication IV: Proceedings of the 7th International Conference on Design and Digital Communication, Digicom 2023, ... Series in Design and Innovation, 35) [1st ed. 2024] 3031472802, 9783031472800

Citation preview

Springer Series in Design and Innovation 35

Nuno Martins Daniel Brandão   Editors

Advances in Design and Digital Communication IV Proceedings of the 7th International Conference on Design and Digital Communication, Digicom 2023, November 9–11, 2023, Barcelos, Portugal

Springer Series in Design and Innovation Editor-in-Chief Francesca Tosi, University of Florence, Florence, Italy

Series Editors Claudio Germak, Politecnico di Torino, Turin, Italy Francesco Zurlo , Politecnico di Milano, Milan, Italy Zhi Jinyi, Southwest Jiaotong University, Chengdu, China Marilaine Pozzatti Amadori, Universidade Federal de Santa Maria, Santa Maria, Rio Grande do Sul, Brazil Maurizio Caon , University of Applied Sciences and Arts, Fribourg, Switzerland

35

Springer Series in Design and Innovation (SSDI) publishes books on innovation and the latest developments in the fields of Product Design, Interior Design and Communication Design, with particular emphasis on technological and formal innovation, and on the application of digital technologies and new materials. The series explores all aspects of design, e.g. Human-Centered Design/User Experience, Service Design, and Design Thinking, which provide transversal and innovative approaches oriented on the involvement of people throughout the design development process. In addition, it covers emerging areas of research that may represent essential opportunities for economic and social development. In fields ranging from the humanities to engineering and architecture, design is increasingly being recognized as a key means of bringing ideas to the market by transforming them into user-friendly and appealing products or services. Moreover, it provides a variety of methodologies, tools and techniques that can be used at different stages of the innovation process to enhance the value of new products and services. The series’ scope includes monographs, professional books, advanced textbooks, selected contributions from specialized conferences and workshops, and outstanding Ph.D. theses. The volumes of the series are single-blind peer-reviewed. Keywords: Product and System Innovation; Product design; Interior design; Communication Design; Human-Centered Design/User Experience; Service Design; Design Thinking; Digital Innovation; Innovation of Materials. How to submit proposals Proposals must include: title, keywords, presentation (max 10,000 characters), table of contents, chapter abstracts, editors’/authors’ CV. In case of proceedings, chairmen/editors are requested to submit the link to conference website (incl. relevant information such as committee members, topics, key dates, keynote speakers, information about the reviewing process, etc.), and approx. number of papers. Proposals must be sent to: series editor Prof. Francesca Tosi ([email protected]) and/or publishing editor Mr. Pierpaolo Riva ([email protected]).

Nuno Martins · Daniel Brandão Editors

Advances in Design and Digital Communication IV Proceedings of the 7th International Conference on Design and Digital Communication, Digicom 2023, November 9–11, 2023, Barcelos, Portugal

Editors Nuno Martins Research Institute for Design, Media and Culture, School of Design Polytechnic Institute of Cavado and Ave Barcelos, Portugal

Daniel Brandão Communication and Society Research Centre, Institute of Social Sciences University of Minho Braga, Portugal

ISSN 2661-8184 ISSN 2661-8192 (electronic) Springer Series in Design and Innovation ISBN 978-3-031-47280-0 ISBN 978-3-031-47281-7 (eBook) https://doi.org/10.1007/978-3-031-47281-7 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Paper in this product is recyclable.

Preface

This book presents the proceedings of Digicom 2023 – the 7th International Conference on Digital Design and Communication. The 77 papers included in this book were selected out of 114 submissions (68% acceptance rate), upon a rigorous double-blind peer review process. The Digicom conference was held on November 9–11, 2023 in a hybrid form, at Teatro Gil Vicente, Barcelos, Portugal and also online, in a very prolific environment for sharing and scientific debate. Digicom is an annual event organized by the Design School of the Polytechnic of Cávado and Ave, and by ID+, Research Institute for Design Media and Culture, in cooperation with CECS-UMinho, which has been bringing together researchers, academics and designers from around the world. “Digital” is becoming increasingly ubiquitous and prevalent in our networked and global society. Digicom has aimed to be a space for reflection and analysis on the constant challenges digital communication poses to society, institutions and brands. Despite its strong focus on the area of Communication Design, the objective of the conference— which is very much reflected in the present book—is the promotion of an open, broad and plural discussion, aggregating different areas of knowledge, namely Arts, Technology, Communication Sciences, Education Sciences, and Branding, among others. The conference thus seeks to stimulate interdisciplinary relationships that contribute to a solid development of scientific activity. Digicom 2023 received three keynote speakers: Ann Bessemans is a legibility expert and award-winning graphic and type designer. She founded the READSEARCH legibility research group at the PXL-MAD School of Arts and Hasselt University where she teaches typography and type design. Ann is the program director of the international Master program ‘Reading Type & Typography’. Ann received her PhD from Leiden University and Hasselt University under the supervision of Prof. Dr. Gerard Unger. She is a member of the Data Science Institute UHasselt, a member of the working group Art, Science and Technology at the Royal Flemish Academy of Belgium for Science and the Arts and lecturer at the Plantin Institute of Typography. Studies performed at READSEARCH are an important contribution to legibility research and international type design. Emília Duarte, designer with a PhD in Human Kinetics and specialization in Ergonomics, is full professor at IADE, Universidade Europeia, Lisbon, Portugal and the coordinator of UNIDCOM/IADE - Research Unit in Design and Communication. She also coordinates the PhD in Design course at IADE. From 2014–2016 she was appointed vice-rector for research of IADE-U. Her research interests are related with subjects as cognitive ergonomics, user experience, interaction design, emotional design, design for behaviour change and design for health and wellbeing. Simon Dixon has run creative agencies for over three decades. His conviction that great thinking and beautifully crafted design make the world a better place remains undimmed.

vi

Preface

In 2001, Dixon founded DixonBaxi with his business partner Aporva Baxi. Together they have a global reputation as leading creatives in branding and design, and are sought out by some of the world’s most forward-thinking companies to help them connect with and inspire their audiences. DixonBaxi believes there has never been a greater need to future-proof brands in a world of rapid, constant change. They’re trusted by Netflix, IMAX, Audible, AC Milan, AT&T, Headspace, The Premier League and British Land. Companies that seek to be bold and challenge convention. In addition to the guest speakers, we also had the opportunity to attend a set of 77 communications, strictly selected by Digicom’s Scientific Committee, from different international researchers and designers. The three days event resulted in extensive debate sessions, where the participants’ personal and professional perspectives and experience encouraged the discussion, showing the importance of this kind of events. The promotion of a panoramic vision of Digital Design and Communication is a trademark that Digicom has been affirming and reinforcing through its editions, and that organisers and participants will continue to build and consolidate in their future works and the upcoming conference editions. With this book, we would like to transmit this message to a broader audience and to attract contributors who share our vision and/or are willing to join our discussion in future conference editions. We would also like to acknowledge all those who have believed in Digicom so far, and contributed to its success in many ways. Again, a big thanks to all participants of Digicom 2023, and to all the readers of this book, hoping that you’ll enjoy and find it useful for your future research. Looking forward to seeing you next year! Nuno Martins Daniel Brandão Chairs of Digicom

Organization

Chairs Nuno Martins

Daniel Brandão

Research Institute for Design, Media and Culture, School of Design, Polytechnic Institute of Cavado and Ave, Barcelos, Portugal [email protected] Communication and Society Research Centre, Institute of Social Sciences, University of Minho, Braga, Portugal [email protected]

Team Ana Catarina Silva Cátia Vidinhas Isabel Xavier José Raimundo Leonardo Pereira Maria Helena Carvalho Vânia Vieira

Scientific Committee Adérito Fernandes-Marcos Albert Inyoung Choi Alberto Sá Álvaro Sousa Ana Carla Amaro Ana Catarina Silva Ana Clara Roberti Ana Filomena Curralo Ana Margarida Ferreira Ana Paula Faria André Rangel Andreia Sousa

Media Arts at CIAC/University of Saint Joseph, Macao College of Design - Hanyang University, Korea Universidade do Minho, Portugal Universidade de Aveiro, Portugal Universidade de Aveiro, Portugal Escola Superior de Design IPCA, Portugal CITCEM, Portugal IPVC, Portugal UBI & LabCom, Portugal DEUS: human(ity)-centered AI, Portugal FBAUP CITAR, Portugal Universidade Lusófona do Porto, Portugal

viii

Organization

António Maneira Bruno Giesteira Camila Soares Catarina Lelis Catarina Moura Cátia Rijo Cláudia Lima Cristina Sylla Daniel Brandão Daniel Raposo Dario Lanza Demétrio Matos Dina Riccò Doris Kosminsky Elga Ferreira Eliana Penedos Elisabete Rolo Emília Duarte Fabrício Fava Fernando Moreira da Silva Fernando Olivares Delgado Fernando Suarez Carballo Filipa Martins de Abreu Filipe Lopes Frederico Braida Gerry Leonidas Hande Ayanoglu Heitor Alvelos João Abreu João Alves de Sousa João Brandão João Neves João Tiago Santos Jorge Pereira Jorge dos Reis José Gago Silva

UNIDCOM - IADE Universidade Europeia, Portugal FBAUP, Portugal FBAUP, Portugal University of West London, UK Universidade da Beira Interior, Portugal Escola Superior de Educação IPL, Portugal Universidade Lusófona do Porto, Portugal Universidade do Minho, Portugal Universidade do Minho, Portugal Politécnico Castelo Branco, Portugal Universidad Complutense de Madrid, Spain Escola Superior de Design IPCA, Portugal Politecnico di Milano, Italy Universidade Federal do Rio de Janeiro, Brazil Politécnico de Leiria, Portugal Universidade do Porto, Portugal Faculdade de Arquitectura - Universidade de Lisboa, Portugal IADE, Portugal Faculdade de Belas da Universidade do Porto, Portugal Faculdade de Arquitetura da Universidade de Lisboa, Portugal Universidad de Alicante, Spain Universidad Pontificia de Salamanca, Spain University of Saint Joseph Macao Escola Superior de Media Artes e Design IPP, Portugal Universidade Federal de Juiz de Fora, Brazil University of Reading, UK IADE, Portugal Universidade do Porto, Portugal Politécnico de Lisboa, Portugal Universidade Lusófona do Porto, Portugal Faculdade de Arquitectura, Universidade de Lisboa, Portugal Politécnico Castelo Branco, Portugal Escola Superior de Artes e Design, Portugal Escola Superior de Design IPCA, Portugal Faculdade de Bela Artes U. Lisboa, Portugal Universidade de Évora, Portugal

Organization

José Raimundo Juan Ra Martin Karel van der Waarde Krasimira Borisova Drumeva Leonardo Pereira Lev Manovich Luís Leite Luís Santos Manuel Montes Vozmediano Maria João Félix Mario Buono Marta Fernandes Miguel Carvalhais Miguel Terroso Nelson Zagalo Nuno Coelho Nuno Martins Patricia Ranzo Pau Garcia Paulo Maldonado Pedro Amado Pedro Beça Pedro Cardoso Pedro Portela Priscila Farias Ricardo Melo Rita Coelho Rita Espanha Rita Santos Rodrigo Carvalho Rodrigo Hernández-Ramírez Rui Rodrigues Sara Balonas Sara Goldchmit Sara Pereira Sara Santos Sara Velez Susana Barreto Suzana Dias Tiago Navarro Marques

ix

Escola Superior de Design IPCA, Portugal Universidad Pontificia de Salamanca, Spain Swinburne University of Technology, Melbourne, Australia ST Cyril and St. Methodius, University of Veliko Tarnovo, Faculty of Fine Arts (BG) Universidade de Aveiro, Portugal City University of New York, USA ESMAD-IPP, Portugal Universidade do Minho, Portugal Universidad Rey Juan Carlos, Spain Escola Superior de Design IPCA, Portugal University of Campania, Italy Politécnico do Porto, Portugal INESC TEC & FBAUP, Portugal Escola Superior de Design IPCA, Portugal Universidade de Aveiro, Portugal Universidade de Coimbra, Portugal Escola Superior de Design IPCA, Portugal Università degli Studi della Campania. Italy Domestic Data Streamers/Elisava, Spain Universidade de Évora, Portugal Universidade do Porto, Portugal Universidade de Aveiro, Portugal Universidade de Aveiro, Portugal Universidade do Minho, Portugal Universidade de São Paulo, Brazil Fraunhofer Portugal AICOS, Portugal ESMAD, IPP, Portugal ISCTE, Portugal Universidade de Aveiro, Portugal ESMAD-IPP, Portugal UNIDCOM/IADE, Portugal ESMAD-IPP, Portugal Universidade do Minho, Portugal Universidade de São Paulo, Brazil Universidade do Minho, Portugal Universidade de Aveiro, Portugal UBI & LabCom, Portugal Universidade do Porto, Portugal Escola Superior de Design IPCA, Portugal Universidade de Évora, Portugal

x

Organization

Vanda Sousa Verónica Metello Virgínia Tiradentes Souto Vítor Quelhas

Politécnico de Lisboa, Portugal CEIS 20, Universidade de Coimbra, Portugal Universidade de Brasília, Brazil ESMAD-IPP, Portugal

Contents

Digital and Interaction Design Using Augmented Reality for Training Computer Workstation Ergonomic Issues: An Exploratory Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rui Raposo, Mário Vairinhos, Pedro Reisinho, Grzegorz Zwolinski, Anna Laska-Lesniewicz, and Dorota Kaminska Design Methodology for a Shared Mobility App . . . . . . . . . . . . . . . . . . . . . . . . . . . Ana Filomena Curralo, Sara Paiva, Catarina Fernandes, and Luís Barreto Brain Computer Interface: Behavioral Differences in the Calmness Level and Reading Time, Considering Screen or Paper, Serif or Sans Serif Typefaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ana Rita Teixeira, Sónia Brito-Costa, Sílvia Espada, and Maria Fernanda Antunes Anticipation as a Tool for Designing the Future . . . . . . . . . . . . . . . . . . . . . . . . . . . . Joana Cerejo and Miguel Carvalhais

3

15

26

37

Designing Audio-Based Multimodal Interfaces for English Teaching: A Conceptual Model Based on an Integrative Literature Review . . . . . . . . . . . . . . Jéssica Rodrigues Esteves and Berenice Santos Gonçalves

53

From the Algorithm to the New Art Collector. Design, Development and Launch of an Innovative NFT Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dario Lanza

67

From Print to Pixels: Prototyping a Virtual Exhibition for the Faro Museum Poster Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alexandre Martins, Jorge Carrega, Rui d’Orey, and Bruno Mendes da Silva Identifying Design Guidelines of Interactive Whiteboard Interfaces Through Usability Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Raiyan Jahangir, Zinia Sultana, Nasif Shahriar Mohim, Md Tariquel Islam, Mahedi Kamal, Iyolita Islam, and Muhammad Nazrul Islam

79

94

xii

Contents

Peer-Produced Archives, Peer-Designed Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Pedro Sá Couto, Miguel Carvalhais, and Pedro Cardoso Usability Analysis of Augmented Reality-Based Learning Applications for Kids: Insights from SUS and Heuristic Evaluation . . . . . . . . . . . . . . . . . . . . . . . 122 Ramisha Baki, Afnan Mumu, Riyadil Zannat, Mouneeta Rahman, Hasan Mahmud, and Muhammad Nazrul Islam Investigating Usability Indicators for the Adoption of AI Models in Heuristic Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 Demerval Gomes S. Júnior, Rodrigo Hernández-Ramírez, and Jacinto Estima Critical Digital: A Taxonomy to Classify Digital Integration in the Museum Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Fernando Sousa and Francisco Providência Exploring Hazard Perception and Compliance: A Pilot Study on the Influence of Safety Signs in a Non-conspicuous Hazard Situation . . . . . . . 165 Lara Reis, Emília Duarte, and Francisco Rebelo The Impact of Tangibility in the Input of the Secondary Car Controls: Touchscreens vs. Physical Buttons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 Manuel Masseno, Inês Lopes, Rita Marques, Francisco Rebelo, Elisângela Vilar, and Paulo Noriega Dep-Project: Comparative Analysis of Online Solutions Aimed at Personalization and Purchasing Prostheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Sara Peixoto, Nuno Martins, Demétrio Matos, Vítor Carvalho, and Daniel Miranda Digitization Project of Museum Collections as a Way of Preserving Memory and Cultural Heritage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Clara Pache de Faria and Rafaela Norogrando A Social IoT Hybrid Scenario to Promote the Physical, Psychological, and Social Well-Being of Elderly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 Imran khan and Lidia Oliveira The Expanded Book - Various Perspectives in Augmented Reality . . . . . . . . . . . . 224 Andreia Fernandes and Luís Leite

Contents

xiii

Society, Communication and Design Design and Development of a Pill Dispenser: Prototype Assembly, Electronic and Mechanical Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 Óscar T. Viana, Octávio Lima, Miguel Terroso, and João L.Vilaça Design and Craft Come Together - A New Perspective to Synergistic Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Krasimira Drumeva Guidelines for Sustainable Online Behaviors. Green Computing Practices to Reduce the Digital Carbon Footprint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 Sónia Rafael and Matilde Reis Emotional Design to Create Furniture for Children’s Well-Being . . . . . . . . . . . . . 276 Sara Reigoto and Liliana Soares Analysis of Cyberaggression in Social Networks Involving Students and University Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 Eliza Oliveira, Lídia Oliveira, and Vania Baldi Child Safety on Bicycle Seats. A Study Evaluating the Need for Additional Markers to Increase the Visibility of Children When Transported by Bicycle . . . 301 Rita Martelo Digital Citizenship: The Cyber-Plagiarism Dilemma in Compulsory School . . . . 311 Fernanda Vicente and Manuel Meirinhos No More Cookbooks? No, More Cookbooks. Reflections on a Print Book in the Age of Digital Media . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Suzana Parreira Designing a Creative Process Between Music, Performing and Visual Arts and the Business World . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 Liliana Soares, Ermanno Aparo, and Rita Almendra The Strange and Not So Strange About Disney’s Strange World: A Visceral and Social Semiotic Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 Edgar Nunes Quintero, Catarina Lelis, and Bernardo Meza Guzman A Systematic Literature Review of Inclusion and Representativity of Women in Videogame Live Streaming Between 2017–2023 . . . . . . . . . . . . . . . 357 Maria Júlia Vieira and Pedro Cardoso

xiv

Contents

A Survey of the Digitisation of Collections in Italian Typographic Museums . . . 371 Emanuela F. Bonini Lessing Contemporary Tattoo and Social Media: A Special Relationship . . . . . . . . . . . . . . 381 Bruno Cordeiro and Suzana Parreira Design Contributions to Raise Awareness of Dementia and Tackle Stigma . . . . . 390 Inês Colaço, Cláudia Lima, Rui Mendonça, and Lígia Lopes Creation of Podcasts to Promote Students’ Engagement in Technical and Vocational Schools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402 Elton Rubens Vieira da Silva and Pedro Beça Usability Requirements for Visualization Technologies in Industrial Field . . . . . 413 Giovanna Giugliano, Mario Buono, Sonia Capece, and Francesco Caputo Image Dysmorphia on Digital Platforms: an Analysis of the Policies and Terms of Use on Instagram and Spark AR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 Jenifer Daiane Grieger, Rodrigo Eduardo Botelho-Francisco, and Lídia Oliveira Protecting Personal Data in Surveillance Society: A Designerly Approach to Privacy in Service Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438 Davide M. Parrilli and Rodrigo Hernández-Ramírez Improving Software Design Teaching, a Comparative Study . . . . . . . . . . . . . . . . . 449 Verónica Silva, Rodrigo Hernández-Ramírez, and Rodrigo Morais Oficina de Tipo: Comparative Study of Wood Type Production Methods (CNC and Pantograph) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459 Ana Manique and Aprígio Morgado When in Memeland, Speak in Memes: Contributions of Design Towards the Betterment of Online Behavior Regarding Public Health . . . . . . . . . . . . . . . . . 470 Heitor Alvelos, Jorge Brandão Pereira, Abhishek Chatterjee, Susana Barreto, Pedro Alves da Veiga, Cláudia Raquel Lima, and Eliana Penedos-Santiago Understanding the Methodological Value of Mapping Alternative Fiction Scenarios in the Context of Domestic Violence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487 Mariana Alves Carvalho, Rodrigo Hernández-Ramírez, and Hande Ayanoglu

Contents

xv

“Nothing in This House Is by Chance”: Design Ethnography for the Well-Being of Children with Autism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498 Emanuele Dutton, Paula Trigueiros, and Alison Burrows Educating the Future Digital Maturity Enablers. Learning from the Experience of the DC4DM LLabs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508 Valentina Vezzani, Luís Ferreira, Marita Canina, and Eva Monestier Effects of Customer Service Quality in Online Retail Stores . . . . . . . . . . . . . . . . . 520 Pedro Espírito Santo and Patrícia Azinheira Cardoso Design for Well-Being, Empowerment and Experience of Care: Minimum Design Requirements of a Children Prosthetics for Initial Approach to Sport Activity and Aquatic Physiotherapy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 530 Piera Losciale, Alessandra Scarcelli, Annalisa Di Roma, and Giovanni Pappalettera Cognitive Load Theory Analysis of Medical Signage System for Seniors Based on General Hospitals in Shanghai . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542 Jieqiong Fan and Albert Young Choi Neuroergonomic Models and Tools Compared to Evaluate and Improve Human-Machine Interaction in Manufacturing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 Ilaria Lombardi, Mario Buono, Víctor Fernando Muñoz Martínez, Vincenzo Paolo Senese, and Sonia Capece Design Thinking Meets Academic Research: Science Communication for Design Communities Using Analogies and Illustrations . . . . . . . . . . . . . . . . . . 569 Violeta Clemente, Miriam Reis, Caroline Ezingmüller, Katja Tschimmel, and Fátima Pombo Is Gender(Less) Becoming a Counterculture in Fashion? . . . . . . . . . . . . . . . . . . . . 584 Eduarda Loureiro New Retrospective Approach for Voting Advice Applications’ Design Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594 Artur Porto de Almeida Magalhães and Pedro Duarte de Almeida Inclusiveness of Knowledge and Knowledge for Inclusiveness. Collaborative Digital Technology for Made in Italy . . . . . . . . . . . . . . . . . . . . . . . . . 608 Patrizia Ranzo and Salvatore Carleo Digital Design and Data Visualization for Society Health and Wellbeing . . . . . . . 615 Roberta Angari and Gabriele Pontillo

xvi

Contents

Mother Angel or Swiss Knife? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629 José Gago, Tiago Marques, Ema Rolo, Rita Almendra, and Noemy Berbel Information Design and Semiology: A Visual Study on Deconstructing Musical Notation for Improving First-Grade Children’s Learning . . . . . . . . . . . . . 639 Ines P. Antunes, Carlos Rosa, and Flávio Almeida Amiais@SL: A Metaverse Simulator to Support Playful Engagement in Learning About Cultural Heritage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 651 Ana Carla Amaro and Lídia Oliveira The Importance of Regional Newspapers in Portugal . . . . . . . . . . . . . . . . . . . . . . . 665 Maria João Nunes and Elisabete Rolo Designing IoT-Based Products for Indoor Air Quality Management: A Design Thinking Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 677 Ana Filomena Curralo, António Curado, Paulo Barros, Inês Freitas, and Sérgio Ivan Lopes The Importance of Chat GPT Training for Higher Education: Case Study . . . . . . 695 Maria Potes Barbas, Andreia Teles Vieira, and Paulo Duarte Branco Experiencing Design Fieldwork: Walking Interviews on Madeira Island . . . . . . . 706 Vanessa De Luca and Elisa Bertolotti Wayfinding Signs in Cultural Tourism Area Based on Analyzing Songdu Huangcheng Cultural Tourism Resort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 719 Yang Ting and Albert Young Choi Chandra-Padma: Identifying Motivations for Children to Respond and React to a Book with a Non-linear Narrative Structure . . . . . . . . . . . . . . . . . . . 732 Sajini Lankadari and Buddhika Rodrigo Graphic Design and Branding Empirical Analysis of Packaging Design and Consumer Preferences in the New York Kimchi Market . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 753 An Zhao, Wenbo Wei, and Albert Young Choi Creating a Proxy Advertisement for Cognitive Response Evaluation in Visual Metaphor Studies Under Advertisement Research . . . . . . . . . . . . . . . . . . 765 Partha Das, Gurdeep Singh, Gobinda Banik, and Manoj Majhi A Contribution of a Graphic Design Intern at Sika Portugal . . . . . . . . . . . . . . . . . . 777 Jorge Fonseca and Rita Coelho

Contents

xvii

The Expressivity of the Typographic Composition . . . . . . . . . . . . . . . . . . . . . . . . . . 787 Elisabete Rolo and João Gomes A Study of the Relationship Between Image Schema and Visual Identity Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 797 Xin Wang and Albert Young Choi Implications of New Media in Brands’ Visual Identities: A Systematic Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 809 Sofia Almeida and Catarina Lelis Intervention of Product Design in E-Commerce: Contribution to a Sustainable Proposal in Supermarket Delivery Services . . . . . . . . . . . . . . . . . 826 João Tinoco, Cláudia Lima, and Rui Mendonça Adapting Advanced Typographic Taxonomy Principles for Modular Stencil System Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 840 João Francisco Rodrigues Gomes and João Aranda Brandão Embracing a Collaborative Practice with a Stakeholder: A Challenge Extended from the Academy to the Labour Market . . . . . . . . . . . . . . . . . . . . . . . . . 856 Carla Cadete Defining the Emotional Design Stages of Package Designs by Analyzing the Melatonin Gummy Candy Packaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 873 Xie Zeyuan and Albert Young Choi Audiovisual Design and Communication Alcântaras – O Som é Um Sistema: Expanded Poetics of Production/Reception of Audio and Visual Scintillations of Sound as a System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 889 Ramusyo Brasil and Cesário Alves The Production of Effects of Presence in Live Audio-Visual Performances . . . . . 901 Frederico Dinis Perceptions About the Impact of Audiovisual Storytelling Campaigns on Young Consumers: An Exploratory Study with Higher Education Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 911 Teresa Gouveia, Ana Oliveira, Nídia Salomé Morais, and Rui Raposo

xviii

Contents

Optimizing Accessibility Features for Deaf Community in Bilingual Videos: A User-Centered Design Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 922 Jéssica Rodrigues Esteves, Luis Rocha, Berenice Gonçalves, Giselle Merino, Eugenio Merino, and Júlio Monteiro Teixeira Surfing the Landscape of Physical Computing Platforms to Develop Electroacoustic Musical Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 936 Filipe Lopes and Sérgio Coutinho Co-streaming: From a Community-Driven Audiovisual Format to a New Consumption Channel in Esports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 950 Diego Martín Muñoz Directional Acoustic Vehicle Alerting Systems: A Review of the Literature and Simulation Model for a Shared Zone Closed Car Park . . . . . . . . . . . . . . . . . . . 963 Frederico Pereira and Emanuel Sousa Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 977

Digital and Interaction Design

Using Augmented Reality for Training Computer Workstation Ergonomic Issues: An Exploratory Project Rui Raposo1(B) , Mário Vairinhos1 , Pedro Reisinho1 , Grzegorz Zwolinski2 Anna Laska-Lesniewicz2 , and Dorota Kaminska2 1 DigiMedia – Universidade de Aveiro, Campus de Santiago, 3810-193 Aveiro, Portugal

{raposo,mariov,pedro.reisinho}@ua.pt

2 Institute of Mechatronics and Information Systems, Lodz University of Technology, Lodz,

Poland {grzegorz.zwolinski,anna.laska,dorota.kaminska}@p.lodz.pl

Abstract. The number of hours dedicated to working at a desk and in front of a computer is gradually taking its toll on millions of people. It has drastically increased the number of health issues associated with this study and work habits. The ATOMIC project, presented in this paper, explores the use of Virtual Reality and Augmented Reality solutions for training and teaching activities in a diverse array of contexts, such as creating awareness of computer workstation inherent ergonomic issues. The prototype presented is closely concerned with training its users to recognize their poor decisions regarding their desk workstation organization and helping to improve their decisions and behaviour in this context. It includes an augmented reality solution projected onto the desktop, capturing information with a standard webcam, a computer vision solution using OpenCV and Python, or a QR code solution through Unity and Vuforia. The prototype was evaluated during a summer school with the participation of 20 higher education students during a 4-stage test protocol (profiling, pre-test, AR experience, post-test) during which data regarding the user experience and interface usability was collected through 3 questionnaires all answered through an app on the users’ smartphone. The results attained were, in general, positive and showed that the users comprehended the purpose of the application and regarded it as useful for further understanding the benefits of small adjustments in their workstation layouts. Although they did not feel immersed in the experience, the users did see using the application as a better method for learning about ergonomic issues as opposed to more traditional penand-paper approaches. Overall, the tests conducted provided a detailed insight into the pertinence of the prototype and its context, as well as a list of suggestions for further improvements regarding its UX strengths and weaknesses. Keywords: Augmented Reality · Workstation Ergonomics · ICT in education · Computer vision · Unity

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 3–14, 2024. https://doi.org/10.1007/978-3-031-47281-7_1

,

4

R. Raposo et al.

1 Introduction Teaching and learning activities are frequently focused on teachers communicating information in the form of lectures to a group of students and anticipating that this information will become helpful knowledge in students’ minds, behaviour, and decision-making processes. Students are sometimes asked to understand complicated concepts and recall a large amount of information without being allowed to learn by doing and have this knowledge used in real-world scenarios. The classroom has become a haven where the information, assistance, and support are frequently far from what will be helpful in a real-world practical situation. Traditional educational techniques are becoming increasingly digitised and driven by technological advancements. Classical educational methods need to help adjust to the digital environment. The school paradigm of a professor opening a course book and reading a lecture that students must learn, pass the exam, and then forget is gradually becoming inadequate [1–5]. As a result, mastery of technology is critical for building the competencies necessary for a future in which children are ready to face the continual challenges of scientific and technological advancement. As a result, developing digital competencies is critical to learning 21st-century abilities and should be regarded as a universal educational goal [6], particularly in the first years of school. Of course, there is an ongoing issue with teacher training and preparedness [7, 8] regarding the extensive use of a sometimes overabundant and volatile set of technology in education. Various efforts and national and international agendas have addressed these issues from multiple perspectives (i.e., training programs, workshops, seminars, and projects organised and funded by national agencies dedicated to this issue). They have even received a boost from the COVID-19 pandemic and its closures. Positive indicators, such as the number of teachers, students, and staff who have had their digital competencies verified, cannot conceal the reality that thousands of instructors and students are distressed and unsure of how to proceed with their teaching and learning activities [9]. Keeping students and instructors engaged during these troubling times was a complex undertaking, and the need to design and execute engagement techniques has become much more critical than before. Involving and engaging students, piquing their interest in the material, and helping them grasp its relevance [10] have been the key goals of every classroom engagement. Incorporating Information and Communication Technologies (ICT) into the curriculum and classroom has been widely acknowledged as a clear step in the right direction [10], and keeping up with ICT trends while challenging due to the diversity of the field. Unpredictability may eventually contribute to students’ motivation and engagement. Educators can immediately enhance learning results using augmented reality by improving engagement and involvement. AR solutions, using hands-on, demanding, and engaging experiences, may help students to imitate tasks and retain information. Much of the effort and outcomes addressed in the study were driven by learning by doing, as opposed to passive learning based on reading a text, listening to a lecture, or watching a video. In essence, this paper presents an AR experience, one of the four outcomes of the ATOMIC Erasmus + project, which was developed by a multidisciplinary and multinational team from five European Higher Education Institutions and focused on training students on computer workstation inherent ergonomic issues. The research’s theoretical corpus and methodological design are provided and summarised

Using Augmented Reality for Training Computer Workstation Ergonomic Issues

5

by the project’s significant concepts and aims. The collected data are also analysed to provide an overview of how the AR experience was evaluated and what was learned throughout the project’s testing stage. This paper concludes with some findings and outlines future work.

2 Theoretical Background The following section provides an overview of several basic concepts, prototypes, software and hardware technological setups, and user experience (UX) design concerns that are important for understanding the project conclusion described in this article. The presented output is part of the previously announced ATOMIC Erasmus + project [11], in which an AR-based solution is utilised to familiarise users with ergonomic norms that are useful for organising their workspace when sitting at a desk to use a laptop or desktop computer. The development team, comprising higher education professors and researchers, combined ergonomics, engineering, informatics, and interaction design expertise to create a scenario where users could comprehend, review, and improve their workplace layout decisions. Regarding the basic principles, AR, Workplace Ergonomics, and UX stood out as the three pillars supporting the final prototype. Augmented Reality refers to a set of ICT technologies that allow for the real-time blending of computer-generated information and live footage. Applications can create the illusion that these synthesised pieces are part of the natural environment, perceived by overlaying virtual elements in the user’s perceptual space. Unlike Virtual Reality (VR), AR augments reality by combining it with the real-world environment around the user, increasing or affecting their perception of the actual scene and delivering extra contextaware information [12–14]. Although AR was discussed in this study, it is considered independent of the critical idea of Extended Reality (XR). Extended Reality encompasses the entire range of settings between real and virtual. AR, Mixed Reality (MR), and Virtual Reality (VR) are part of it. The XR concept, which includes AR, is based on the Milgram virtuality continuum [15], and it encompasses existing concepts such as VR, AR, and MR. It is broad enough to include current relevant discussions in the field and any additional concepts that may emerge in the future [16]. In a broad sense, workstation ergonomics refers to appropriately adjusting desks, seats, computer spaces, and other office furniture to suit one’s physical stature while performing assigned job activities. It should be seen as a necessary proactive technique and an iterative process capable of decreasing repeated injuries at work [17]. The prototype design, on the other hand, widened the context to include non-work-related activities. A user may spend hours in front of a computer doing things other than work, such as playing, procrastinating on social networks, and editing user-generated materials. These activities may cause the same repeated injuries as job-related circumstances [17]. When attempting to comprehend the notion of user experience (UX), one may refer to the International Organization for Standardisation (ISO) definition, which defines UX as “Perceptions and responses arising from the usage and expected use of a product, system, or service” [18]. However, studies suggest that user, product, social and cultural characteristics and use context are all essential considerations when designing a product or service’s UX [19]. In the case of AR experience, user experience design, as its

6

R. Raposo et al.

principles, may span a wide range of technical setups and contexts of use and combine specific work done on variables such as interaction with the content, visual and audio cues, the text and colours used, lighting, and field depth [19, 20]. 2.1 The Workstation Ergonomics AR Experience (WEARE) The primary goal of the prototype is to provide users with an interactive AR experience that delivers a context-aware set of ergonomic guidelines to assist them in organising their workplace. The Workstation Ergonomics AR Experience (WEARE), created as part of the ATOMIC project, is a customised exercise that introduces users to a set of tiny, simple ergonomic norms that can help them organise their workspaces. It comprises an activity designed to assist users in setting up their workstations by following safety standards and professional medical advice. As illustrated in Fig. 1, a selected and tailored short list of frequent workplace items was filtered from a broad list of available objects. These included monitors, keyboards, tiny plants, books, mugs, and smartphones; the user was requested to put them according to their habits.

Fig. 1. Example of objects placed on a desk while the user tests the WEARE prototype.

Once everything has been set, WEARE moves on to the second stage, guiding the user to rearrange the objects while keeping ergonomic safety requirements and health expert advice in mind. Visual feedback is supplied to the tables in the form of arrows pointing the way and grades, such as “well positioned”, “moderately positioned”, and “badly positioned”, which illustrate where each object was initially placed and where it should go, as seen in Fig. 2. The complete work area was intended for a desk with a minimum size of 120 cm × 80 cm, and its surface was sensed by two cameras employed as application input devices. Each object was identified based on its location on a desk. The software also

Using Augmented Reality for Training Computer Workstation Ergonomic Issues

7

Fig. 2. WEARE prototype feedback on object positioning.

Fig. 3. WEARE prototype user-detection setup.

determines whether the user is left or right-handed and the length of their forearm to develop a deeper relationship with the user’s traits, as illustrated in Fig. 3. Figure 4 depicts the prototype software configuration that employs two solutions: computer vision and image recognition. The first solution was based on Vuforia [21], an augmented reality engine utilised in conjunction with the Unity 3D engine [22]. Vuforia gathers photos in the camera’s field of vision and searches for similarities with images stored in a previously constructed project database when VuMark [23], a visual indicator,

8

R. Raposo et al.

is placed within the camera’s field of view. The second option employs OpenCV [24], a computer vision toolkit that enables real-time object detection and localisation combined with Python. OpenCV and Python use a previously trained custom model to recognise and save objects in the camera’s field of vision. After identifying the items, the script transfers the stored data to a mobile application. Our team combined both techniques to account for potential data loss and false positives. Consider how an object, such as a bottle or cup, is not identified because it appears circular from the camera’s position. In this situation, the user can attach VuMark (image target) to the object, allowing Vuforia to recognise it. Figure 4 depicts the hardware configuration of the WEARE prototype, which comprises two affordable commercially available webcams, a projector, and a laptop or desktop computer. The projector should be lightweight and have a small focal length. To provide a higher identification rate for VuMarks and objects, the two cameras, one for Vuforia and one for OpenCV, should have a high resolution of 1080p or above. Given the graphics resources and performance required to operate OpenCV, the object recognition system used in the prototype You Only Look Once (YOLO) [25, 26], and Vuforia and Unity, a laptop or desktop computer requires a powerful NVIDIA graphics card.

Fig. 4. WEARE prototype hardware setup.

In addition to the team-based testing activities of the project, groups of users tested the WEARE prototype. The subsequent sections of this paper discuss the tests performed, their results, and their conclusions.

3 Methodology The prototype’s usability was evaluated during a three-day summer school program dedicated to sharing and testing ATOMIC project outcomes, including WEARE. The program was tested by its final users, all of whom were higher education students, during the event, and different opinions were provided during informal social moments. Therefore, the project team believes these events provide excellent conditions and contexts

Using Augmented Reality for Training Computer Workstation Ergonomic Issues

9

for application testing. During casual social times, open talks with users about their experiences (e.g., coffee breaks, group city tours, dinners, and sunset gatherings). They supplemented the data gathered during the formal testing procedures. The protocols and findings are presented in the following sections. 3.1 Data Collection and Participants An international group of 20 students from the Lodz University of Technology (Poland), the University of Tartu (Estonia), the University of Aveiro (Portugal), and the Polytechnic of Šibenik (Croatia) tested the WEARE prototype. The testing occurred during a summer school organised by the Trokut Incubator in Šibenik, Croatia, from 13 September to 16 2021. Participants were recruited from each Higher Education Institution through local marketing and advertising campaigns conducted via email, promotional brochures, and posters distributed by each of the four partner universities. Twenty volunteers (five from each partner university) participated in the testing activities and completed each activity. Participants began the exercise by completing a questionnaire about demographic variables, such as age and gender. The participants were 23 years old on average (median = 22.5, std = 3.54, range = 18–30), with 11 men (mean = 23.09, median = 23, std = 3.87, range = 18–30) and 9 women (mean = 22.88, median = 22, std = 3.07, range = 20–30). The information was collected using a unique mobile application developed specifically for these testing activities. Participants’ feedback was gathered according to the procedure illustrated in Fig. 5.

Fig. 5. Information collection procedure. 1: profile form; 2: pre-test; 3: AR experience; 4: posttest.

Each participant filled in the socio-economic information section at the beginning of the testing session for each project outcome, also known and illustrated in Fig. 1 as the profile form, with basic demographic information and some information about their familiarity with VR/AR technologies. Afterwards, each testing session started with the completion of a pre-test in which testers evaluated, on a 5-point scale, their current mood,

10

R. Raposo et al.

Q1 (from a negative to a positive one) and how they felt: Q2–being tensed or relaxed, Q3– apathetic-energetic, Q4 – sedated–stimulated, Q5–bored-interested, and Q6–distractedattentive. Shortly after completing the AR experience, the user was asked to complete a post-test. The post-test was divided into three parts: (1) a mood questionnaire with the same questions as in the pre-test (Q1-Q6); (2) some questions about the user’s opinion of what was tested in the application (Q7-Q19); and (3) a set of questions about the usability of the UI (Q20-Q28). All questions concerning application usability and AR experience testing are presented in Table 1. Table 1. Questions asked during the application usability testing. Label Questions

Answers

Scenarios testing Q7

What was the level of task difficulty?

Three options

Q8

Time to perform the task

Three options

Q9

How would you describe the task performed

Three options

Q10

To what extent does AR/VR replace exercise in natural conditions?

scale 1–10

Q11

To what extent tasks in AR/VR are better than paper-pencil exercises? scale 1–10

Q12

Select one of the answers below that best fits your opinion

Three options

1) In AR/VR conditions, my effectiveness of learning is worse than in the case of theoretical lecture 2) There is no difference in my learning outcomes between AR/VR and theoretical lecture 3) AR/VR improves my effectiveness of learning in relation to the theoretical lecture Q13

The tasks performed increased the degree of my “immersion.“ Into AR/VR

scale 1–5

Q14

I think this technology is perfectly designed

scale 1–5

Technology testing How would you describe your experience with this technology? Q15

Exciting

Q16

Challenging

Q17

Pleasant

Q18

Tiring

Q19

Informative

scale 1–5

UI testing Q20

Overall, I am pleased with how easy it is to use this app

scale 1–5 (continued)

Using Augmented Reality for Training Computer Workstation Ergonomic Issues

11

Table 1. (continued) Label Questions

Answers

Q21

With this app, I was able to complete tasks and scenarios quickly

scale 1–5

Q22

I think the app is unnecessarily complicated

scale 1–5

Q23

I think the support of a technical person is indispensable to being able scale 1–5 to use this app

Q24

I think the various functions of this system are well-integrated

scale 1–5

Q25

I believe there are too many inconsistencies in the application

scale 1–5

Q26

The interface of this application is pleasant

scale 1–5

Q27

The messages conveyed by the application interface are clear

scale 1–5

Q28

Every time I make a mistake in the system, I can fix it easily and quickly

scale 1–5

The results and some of the main ideas and conclusions derived from their analysis are presented in the following section.

4 Results and Discussion Figure 6 presents the results obtained as the gain of the testers’ current mood on a 5-point scale (negative to positive).

Fig. 6. Results regarding the variation of the testers concerning their current mood on a 5-point scale (negative to positive). The black line is the average value and the solid rectangle is the 95% confidence level.

Figure 7 presents the average values of the answers to the subsequent questions in Table 1. All results were normalised (linear transformation) to a range of 0–100% to standardise the comparison. Questions Q7–Q14 focused on the tasks that the users were asked to complete. It is noticeable that the respondents’ opinion regarding the difficulty of the tasks (Q7) was approximately 10–20%. The time to complete the test (Q8) was found to be below what

12

R. Raposo et al.

Fig. 7. Usability test results. The black line is the average value, and the solid rectangle is the 95% confidence level.

can be considered positive (approximately 45–50%), and the tasks were evaluated as very interesting (Q9)(approximately 60–73%). In terms of task replacement in real-life situations with alternative AR or VR experiences (Q10), it was found that the users’ evaluation of this possibility as a group ranged from a slightly negative to a favourable opinion (45–60%). The participants strongly believed that AR technology-based solutions like the one presented are better than traditional methods (paper-pencil exercises) (Q11) for teaching and learning things, as those dealt with in the AR experience (65– 75%). The respondents also claimed that their experience improved their knowledge of the matter and that the AR experience may be used as an additional tool for theoretical courses (Q12) (>90%). When dealing with immersion in the experience, the users felt they needed to be more immersed, with results barely reaching a positive impact (Q12) (38–50%). Users rated this issue positively when asked if they thought the technology was perfectly designed (Q14) (55–70%). The second part of the questionnaire (Q15-Q19) focused on the technology used in the experience and how the users described the experience (Q15– Q19). The most well-rated options were Exciting, Pleasant, and Informative (50–60%). Users did not consider the experience challenging or tiring (20–30%). The following part of the questionnaire (Q20-Q28) mainly evaluated usability-related issues. The AR experience generally obtained positive opinions for all questions, with results averaging between 50% and 70%. When questioned about the application’s unnecessary complexity (Q22) and the number of inconsistencies (Q25), users considered these issues scarce (20–35%). The information provided by the application interface was considered explicit (Q27) (50–60%), and fixing any mistakes made was considered an easy and quick task (Q28) (60–70%).

5 Conclusions This paper presents an Augmented Reality experience focused on training computer workstation ergonomic issues and was developed as part of ATOMIC, an Erasmus+funded project. In a constantly growing universe of people with desk jobs requiring endless hours in front of a computer, repetitive injuries are an unfortunate health problem that tends to become more common and frequent. The experience developed undertook the challenge of helping users understand their current practices and how they can be

Using Augmented Reality for Training Computer Workstation Ergonomic Issues

13

improved to avoid or delay the appearance of these disabling health issues. The prototype, designed and implemented based on a projected AR setup, involved a multinational and multidisciplinary team of higher education teachers and researchers from areas of expertise ranging from informatics to health and well-being. This broad insight allowed the project to proactively consider the current trends and relevant issues in the field. In several instances, the work developed took on a transdisciplinary nature due to the collaborative and willing-to-share-and-learn attitude assumed by all team members. The working prototype, with room for improvements already outlined in the tests, may be exciting proof of how AR may be a valid technology for active and interactive teaching and training activities in the field of ergonomics. The results obtained in the tests conducted during the project’s summer school are generally satisfactory and promising but provide room for additional thought and discussion as to what can be done to bring the results up to the last quarter of the evaluation. What was learned and discussed during usability testing activities may be crucial for this boost, linked to design assumptions, issues with user experience flow, and the overall level of understanding of the applied interfaces in the thought-out context of use. Work confirms that usability testing is a significant part of the project life cycle and that direct contact with target users is a valuable activity for learning and improving. Observation of users’ behaviours and live feedback provide a rich source of knowledge on how to tweak the results. We also noticed that organising brainstorming sessions in narrow user groups directly after usability testing brings about many spontaneous ideas that can provide valuable clues for paths worth exploring. The participation of target users in the design process is highly beneficial, as it promotes participation, affordance, and an immediate sense of belonging to the project. Users feel relevant because they can influence and tailor the final system shape. These lessons have already made their way into other projects, such as MrUD [27]. They have also influenced many teaching and research approaches and practices adopted by members of the ATOMIC team.

References 1. Kami´nska D et al (2021) Virtual reality-based training: case study in mechatronics. Technol Knowl Learn 26:1043–1059 2. Kami´nska D et al (2019) Virtual reality and its applications in education: survey. Information 10(10):318 3. Cvetkovski G et al (2019) ViMeLa Project: an innovative concept for teaching mechatronics using virtual reality. Przeglad Elektrotechniczny 95(5):18–21 4. Kami´nska D, Sapi´nski T, Aitken N, Rocca AD, Bara´nska M, Wietsma R (2017) Virtual reality as a new trend in mechanical and electrical engineering education. Open Phys 15(1):936–941 5. Tikk T et al (2020) ViMeLa. In: Daniela Linda (ed) New perspectives on virtual and augmented reality: finding new ways to teach in a transformed learning environment. Routledge, pp 276–286. https://doi.org/10.4324/9781003001874-18 6. Kloos CD, Muñoz-Merino PJ, Alario-Hoyos C, Estévez-Ayres I, Ibanez MB, Crespo-García RM (2018) The hybridisation factor of technology in education. In: 2018 IEEE global engineering education conference (EDUCON). IEEE, pp 1883–1889 7. Guerra C, Moreira A, Vieira RM (2018) A design framework for science teachers’ technological pedagogical content knowledge development. In: Costa AP, Reis LP, de Souza FN, Moreira A (eds) Computer supported qualitative research. Springer International Publishing, Cham, pp 193–203. https://doi.org/10.1007/978-3-319-61121-1_17

14

R. Raposo et al.

8. Guerra CV, Moreira F, Loureiro MJ, Cabrita I (2020) Programação tangível para a inclusão e promoção das STEM. APEduC Revista-Investigação e Práticas em Educação em Ciências, Matemática e Tecnologia 1(1):100–114 9. Roque V, Raposo R, de Morais NSN, Roque F (2021) Enseñanza y aprendizaje en línea durante la pandemia de Covid-19: El caso de la educación superior pública portuguesa. Revista Prisma Social 35:221–248 10. Weller M (2018) Twenty years of EdTech. Educause Review Online 53(4):34–48 11. ATOMIC (2021) https://www.atomic.p.lodz.pl/. Last accessed 22 Sep 2022 12. Azuma RT (1997) A survey of augmented reality. Presence: Teleoperators Virtual Env 6(4):355–385. https://doi.org/10.1162/pres.1997.6.4.355 13. Van Krevelen DWF, Poelman R (2010) A survey of augmented reality technologies, applications and limitations. Int J Virtual Reality 9(2):1–20 14. Mekni M, Lemieux A (2014) Augmented reality: applications, challenges and future trends. Appl Comput Sci 20:205–214 15. Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst 77(12):1321–1329 16. Rauschnabel PA, Felix R, Hinsch C, Shahab H, Alt F (2022) What is XR? Towards a framework for augmented and virtual reality. Comput Hum Behav 133:107289 17. Dubey N, Dubey G, Tripathi H, Naqvi ZA (2019) Ergonomics for desk job workers-an overview. Int J Health Sci Res 9(7):257–266 18. ISO 9241–210, J. (2010) Ergonomics of human-system interaction–Part 210: human-centred design for interactive systems. Isotopenpraxis, pp 1–19 19. Irshad S, Rambli DRA, Sulaiman S (2020) Design and implementation of user experience model for augmented reality systems. In: Proceedings of the 18th International Conference on Advances in Mobile Computing & Multimedia, pp 48–57 20. Billinghurst M, Kato H, Myojin S (2009) Advanced interaction techniques for augmented reality applications. In: Shumaker R (ed) VMR 2009, vol 5622. LNCS. Springer, Heidelberg, pp 13–22. https://doi.org/10.1007/978-3-642-02771-0_2 21. Vuforia Engine Development Portal, https://developer.vuforia.com/. Last accessed 30 Mar 2022 22. Unity Real-Time Development Platform. https://unity.com/. Last accessed 30 Mar 2022 23. Vuforia Developer Library. https://library.vuforia.com/features/objects/vumark.html. Last accessed 30 Mar 2022 24. Introduction to OpenCV. https://docs.opencv.org/4.5.3/index.html. Last accessed 8 Apr 2022 25. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788 26. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv preprint arXiv: 1804.02767 27. MrUD – Mixed Reality on Universal Design’s Secret Service. http://www.mrud.p.lodz. pl/%23. Last accessed 2 Sep 2022

Design Methodology for a Shared Mobility App Ana Filomena Curralo1,2(B) , Sara Paiva1,3 , Catarina Fernandes1 , and Luís Barreto1,3 1 Instituto Politécnico de Viana do Castelo, Rua da Escola Industrial e Comercial de

Nun’Alvares, 4900-347 Viana do Castelo, Portugal [email protected] 2 ID+ Instituto de Investigação em Design, Media e Cultura, 3810-193 Aveiro, Portugal 3 ADiT-LAB, Instituto Politécnico de Viana Do Castelo, Rua Escola Industrial e Comercial de Nun’Alvares, 4900-347 Viana Do Castelo, Portugal

Abstract. Nowadays, mobile applications (app) have become allies and indispensable in people’s everyday life. Problems that previously required some time to be solved are now solved with a few taps on, i.e., a smartphone screen. The work presented in this paper is part of a master’s project that consists of developing a user-centered procedure for a mobile application. Thus, this article focuses on studying a methodology for the user-centered interface design of an app, with the aim of making the process based on interaction design principles, considering usability aspects in order to favor the user’s work. Based on the perceived needs of the Polytechnic Institute of Viana do Castelo, and with the aim to develop a broader Academic Mobility as a Service (AMaaS), this project consists of studying a work methodology for the development of a prototype of an app with the aim of encouraging sustainable and healthy mobility within the academic community and campuses. Keywords: Mobile Interface · User-Centered Development · Mobility as a Service

1 Introduction Considering providing the convenience and ease, allowed by current technologies, the Polytechnic Institute of Viana do Castelo decided to challenge the academy to promote and foster a more active and more sustainable mobility patterns specially regarding commuting trips to the campuses, thus creating the new concept of an Academic Mobility as a Service (AMaaS). For this purpose, a mobile application (app) called On.Move started to be developed, with the main goal to contribute to the promotion of shared mobility in the academic community and, more specifically, through ride- sharing, bike-sharing, cycling and walking, with the aim of facilitating travel of the user, in order to contribute to sustainable mobility and reduce the number of transports. Road transport is the transportation subsector that most contributes to global warming, through the emission of pollutants and greenhouse gases. About 25% of total carbon dioxide (CO2) emissions result from the use of fossil fuels [1]. This project was not only related to the development of an app for shared mobility, but also to incorporate a health module mechanism, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 15–25, 2024. https://doi.org/10.1007/978-3-031-47281-7_2

16

A. F. Curralo et al.

with the main objective to promote a more active lifestyle and mobility. In this regard, the application aims to collect data that allows: classifying cardiovascular risk in view of the initial health survey; identifying the type of activity, time and distance covered daily and with the health information obtained from the survey produce a report with the correspondent MET (Metabolic Equivalents) of the user’s activity, contributing to the prevention of cardiovascular diseases, type 2 diabetes, cancer, dementia, depression, and reduced life-expectancy [2]. In conclusion, the On.Move app can provide several ride sharing services, and recommend the use of active mobility systems, registering the users daily activity regarding the health, promoting healthier and more sustainable commuting trips. The aim is thus to increase a systematized methodology as a digital product development process, supported by usability analysis and interaction design principles, making it possible to design a digital interface based on the user’s needs. This will in turn result in a higher number of calories burned by users, with its consequent impact in the health, but also an economic impact as such users will decrease their expenses in health-related services, allowing the public health system to reduce costs and improve its services [2]. The rest of the paper is structured as follows. Section 2 introduces the definitions of interaction design and user experience. Section 3 presents the methodology adopted in the On.Move application. Section 4 presents conclusions and future work.

2 Interaction Design and User Experience (UX) The development of a digital product is becoming a demanding process for a designer, as it involves understanding human beings and their relationship with the product itself. Its aim is to provide means of interaction with the object that support and help the human behavior. Interaction design (iD) is defined by the design of interactive products that help users in their daily lives, creating experiences that help in human communication and interaction [3]. This area opposes its role with the role of software engineering, which aims to create solutions for software as opposed to interaction design that focuses on the user much beyond the software. In interaction design, multidisciplinary integration is necessary to successfully develop a product, since all the factors that are addressed by the surrounding areas require a sharing of knowledge, such as behavioral factors, for example, sometimes obtained by psychologists and sociologists [3]. Thus, the interaction design process involves the steps shown in the Fig. 1. Interaction design is based on defining the user’s motivational needs to design a system that allows helping the user. Interactive products cause reactions to the user, thus being opportune to develop ways to help the user and the software behavior, through the study and creation of avatars, as a way of understanding the objectives and motivations of the user [4]. Next, the product must guarantee its usability, highlighting the importance of the evaluation during the creation process, thus incorporating usability tests to understand and discover the strengths and weaknesses of carrying out all the necessary tasks, as well as their reactions and behaviors, visualizing and analyzing its interaction with the newly developed product [3]. The interaction is learned through the use or experience with other products, and those that people use intuitively are those that have similar

Design Methodology for a Shared Mobility App

17

Fig. 1. Steps of the interaction design process. Source: Authors

characteristics to those previously developed/experienced. The past developments are also relevant in the experience, transferring between products and also probably between contexts, so that the performance is affected by the level of familiarity one may have about the technologies [5]. As for the study of user experience (UX), we must refer names such as Donald Norman (2004) [6], Marc Hassenzahl (2006) [7], Arrow Walter (2011) [8]. These authors are important in appreciative the concepts such as user experience, reaction, esthetics and, at the same time, in considerate how the user identifies the product. UX is the experience that the user gets while using a product. In most product development projects, the focus of the designer is the practical factor, demonstrating what the product does, and making the product fulfill the purpose for which it was developed, but many times, they forget the user factor, and how he will use the product, and if he will be able to use it. UX is based on the relationship between the user and the product/service and on the experience that their interaction will provide. This factor proves to be extremely important for the success and failure of the product and/or service [9].

3 The Methodology Used for Developing the On.Move App UX is the experience that the user gets while using a product. In most product development projects, the main focus of the designer is the practical factor, demonstrating what the product does, and making the product fulfill the purpose for which it was developed, but many times, they forget the factor of user, how he will use the product, and whether he will be able to use it. UX is based on the relationship between the user and the product/service and on the experience that their interaction will provide. This factor proves to be important for the success and failure of the product and/or service [9]. Within the scope of the several methodologies developed in the field of UX design, we intend to use the methodology proposed by Jesse James Garrett (2011) [9], a specialist in the field of new technologies and the web. His work is the underlying basis for many other authors in the area of user experience and proposes a breakdown of the project into five phases that succeed from the abstract to the concrete: strategy, scope/intent, structure, skeleton and surface, as shown in the Fig. 2. The choice of this methodology was because each stage must be restricted by the decisions made in the previous ones, however, it allows reviewing the decisions taken in the different stages of the project, as well as changing its direction and improve and correct previously found errors, thus leading to the importance of evaluating the process during its own execution [10]. This model is considered valuable due to its

18

A. F. Curralo et al.

Fig. 2. Jesse James Garrett model. Source: Garrett (2011)

comprehensive framework, user-centered approach, adaptability, alignment with agile practices, and its role in facilitating effective communication and collaboration among design teams and stakeholders. It has played a significant role in shaping the field of user experience design and remains relevant in contemporary design practices. Also, this methodology has been embraced by professionals in the UX field for years. Its widespread adoption speaks to its practicality and effectiveness in helping designers plan and execute successful user-centered design projects. 3.1 The Strategy Plane In this sense, it is described each step that must be followed to make this project viable: The Strategy plane is based on two factors, this is dominated by the definition of the product’s objectives and knowledge regarding the user’s needs, which automatically becomes one of its objectives. The responses to the user’s needs will be based on the use of personas (avatars), which will help in the perception of the user’s needs and consequently in the decision making of the designer and computer engineers to provide the best solution, meaning a good user experience. The persona is a character, or a profile, created to represent the characteristics and real needs of the user. Two personas were developed, a student and a teacher, who demonstrate to be available for the use of the functionality of sharing means of transport. Persona 1 is a student who must make the trip daily, so it would be more convenient to travel by car, for greater comfort and not have the need to use various means of transport to reach your destination. Persona 2 is a professor at the Polytechnic Institute of Viana do Castelo. He lives in the distance and commutes daily to the Institution, so he intends to use the car sharing service, where he offers the ride to other users, but has some fears regarding adherence to the service. Both personas cover the transport sharing feature because it involves more questions, in relation to how users establish communication, what data they must provide to offer this service, how users will have access to information and how this service is safe. The personas are created based on the knowledge obtained by the research; however certain details of some personas are created to generate life in these characters [9].

Design Methodology for a Shared Mobility App

19

The main objective of the application is to help solve problems related to health, environmental pollution and social inclusion, through the incorporation of functionalities that contribute to: promote active mobility, through recording physical activity and suggesting routes accessible on foot or bicycle, helping the user to perform the recommended exercise; promote sustainable mobility, through the use of the route system, together with vehicle sharing, public transportation and also active mobility, in order to contribute to the reduction of greenhouse gas emissions and reduce the number of vehicles in a city, thus reducing traffic and congestion; promote social inclusion, developing the application so that everyone can use it and have access to it. Given the number of functionalities that the application has, it is essential to cover the object of study in the area of user experience, in order to: provide a product accessible to all types of users; design an intuitive and easy to navigate application; reach a larger number of users within the campus. These are extremely important factors for the foresee success of the application, as the experience provided to the user, while using the app, is what allows contributing to its possible use in the future, as well as enabling its dissemination if the user recommends its use to other users. It is intended to make it easier for the user, in pursuit of a positive reaction in relation to the products and the services it provides, thus encouraging and propelling its use, as well as its promotion by the user of the services and its dissemination to other potential users. 3.2 The Scope Plane The Scope plane is the second phase of the process and consists of an objective analysis of the information obtained in the first phase, resulting in a written document that must present the content requirements and/or the functional specifications that the app must contain [11]. This step allows to analyze the situation, to clarify possible doubts and uncertainties and to establish goals for all the stages of the project [9]. In the scope plan, it is presented a table of requirements, where the requirements that the application must present are demonstrated (see Table 1). For using the application, it is essential to have an account, which will lead to access to the vehicle sharing service functionality, to the route advisory system and to the health module. In terms of health, and concerning its daily activity registration, the user will have to activate, in the app settings, the activity recognition mode and module. To effectively use the health module the user must fill out the health survey (this is an optional feature), thus enabling the app to efficiently register the physical activity and allowing the convenient health activity notifications (associated with the correspondent MET). 3.3 The Structure Plane The Structure Plane aims to establish the order in which the content will be placed. After the phases of research and analysis of the collected information, the information architecture will be considered, that is, the structures and functions of the app, which should be represented through flowcharts and organizational charts. When defining the structure, it is specifically determined how the content will be integrated, thus influencing the user experience and the final product. In the On.Move app,

20

A. F. Curralo et al. Table 1. Table of features and requirements

Application Features

Content Requirements

Create the user account

To access to the personal data

Personal car sharing service

To have an account in the application

Route advisory system

To have an account in the application

Physical activity log

To enable the activity recognition mode

Statistics (activity and sustainability) report To complete the health survey Health survey

To read and allow the declaration of consent and access to personal data

Personal vehicle sharing creation

To register the vehicle

the data obtained is used to create a profile and also for the activity registration feature. After logging on into the account, the user will be able to access the functionality of the route advisory system, and access to the mobility sharing functionality, on the main page. Through the menu it is possible to access other screens such as the user profile, the daily activity statistics, the settings and, if not filled out during the registration process, to the health survey, which will be the initial basis to define the goals for the user’s daily activity. The filling of the health survey is a mandatory requirement if the user intends to fully use the daily activity functionality. 3.4 The Skeleton Plane The Skeleton Plane allows perfecting the structure developed in the previous phase, defining essential points for the interface, navigation, and information architecture. This step allows to define how the features will be presented. In terms of structure, app navigation is part of the proposal for information-oriented projects and refers to the design of screen elements that allow the user to move through the information architecture [11]. By structuring the content and defining the navigation of the application, it becomes possible to create the skeleton plan, where the layout of the screens is presented, together with all the interface elements. At this stage, it is possible to visualize how the application will be presented to users, and it is also possible to simulate how the user will interact with the product and how the user will perform the necessary tasks. The structure plan will be presented through wireframes, as shown in Fig. 3. 3.5 The Surface Plane The Surface Plane is the final phase of the process. In this phase it is defined how the product will be presented to the users. The Surface phase focuses on the visual treatment of the app, as well as the elements of visual communication. Visual elements have the ability to be manipulated throughout visual communication techniques, corresponding to the purpose of a unique message [12]. These elements should interfere not only with the aesthetics but also with the functions of the program [11].

Design Methodology for a Shared Mobility App

21

Fig. 3. First wireframe sketch for the On.Move app

At this stage, research was carried out concerning the colors, the contrasts, the typographies, the icons, and the dimensions that best suited the vast majority of users. In this sense, one of the first visual elements to be developed was the app icon. The icon is a small graphic illustration, that is, it is a visual sign, that when combined with other signs, is used for the purpose of communication, and provide the successful flow of human machine interaction [13]. The icon is part of the composition of an interface, as well as other visual elements, and has become essential for everyone who uses a digital device. The appearance of icons in digital interfaces has transformed computational language into a common language for everybody, even if they do not have any technological knowledge. Since then, it has spread to various electronic devices that required the synthesis and display of some information on screens, as it is in the case with mobile phones. Figure 4 shows the app’s main icon, which seeks to graphically represent everything the app offers. For this purpose, this visual element is composed with the largest number of aggregated signs, namely: the heart and the tag that represent a location. This tag is

22

A. F. Curralo et al.

Fig. 4. Icon proposals for the On.Move app. Source: Authors

being used worldwide as a map marker in the digital environment, it also represents the idea of path/direction. Color is a visual element that influences communication and the dissemination of information to the user, being used as a form of communication, since the human being captures information more quickly through it. In addition, colors affect people’s emotions and propel different sensations [9]. With that, color performs several functions that allow the designer to use it as a tool in his favor, such as highlighting, drawing the user’s attention to a more important element, organizing, and prioritizing information, directing the eye to a certain element, and creating environment. The use of contrast is one of the ways to attract the user’s attention, by highlighting the essential elements of the interface that require greater attention from the user, whether in highlighting information or helping in navigating throughout the product. The On.Move app presents a color palette based on shades of blue and green because the color blue conveys calm and confidence, and the color green is linked to sustainability and the environment (Fig. 5).

Fig. 5. First wireframe sketch for the On.Move app

After defining the color palette, tests were carried out to ensure good legibility in the application. For this, the Adobe Color website was used through its accessibility tools, thus being possible to check the contrast between tones, and to check if they are suitable for colorblind people. In the contrast check, background and text tones are entered and

Design Methodology for a Shared Mobility App

23

then analyzed based on full-size text, large-size text, and by basic components. With this it is possible to visualize the proportions of contrast and verify if the tones are approved. The use of typography on the screen as a communication factor requires one important aspect to be highlighted: legibility. The legibility is the ability of typography to be easily read, especially under normal reading conditions [13]. The legibility contributed not only as an element of distinction, by allowing easy recognition of a letter, but also by providing easy reading in the blocks of text. This factor contributes to a greater clarity of the message, allowing for greater ease of reading. A good legibility of the text displayed on the screen, is related to other factors than only selecting the font, namely: size, color, contrast, and its position [13]. The typography used must be based on uncomplicated fonts, so as not to impair the readability of the text, as the more ornate and serif fonts (with serif), in digital support, cause visual and cognitive fatigue [9]. In the development of this project, it was selected the Open Sans font, as it is a font without serif, available in different languages and in different styles. Being downloadable for free, it is characterized by having good legibility due to the letters’ design, which include morphological characteristics, such as the body of the letter, the height, the width, and the thickness of its strokes. The layout is the structuring of visual components in different zones in order to help the visualization and navigation of an interface. The development of a layout is divided into sections, where the application contents and the region of the navigation elements are found. On one hand, it establishes a few limitations such as the minimum and the maximum dimensions for the screen borders, for the different sections. On the other hand, also establishes size limitations for the elements to be used. The established dimensions depend on the screen size and on the different devices used, thus being necessary to create a layout that allows adapting to the different types of interfaces. In addition, to establishing dimensions, there are some tools used in the development of a layout, such as the use of grids and the division of the screen by columns for the alignment of the elements, images and text, that results in the development of a consistent layout across all different screens. Alignment is an essential tool for visual design, as well as for the user experience, by presenting an organized and consistent interface, where the visual elements are displayed in line with each other, thus allowing to create a visual balance in order to facilitate the use of the app [12]. As an example, Fig. 6 shows two layouts of the On.Move app. The first corresponds to accessing to the functionality of the route advisory service and the notification of vehicle sharing trips. It is also possible to view the main menu icon in the upper left corner. The second screen corresponds to notifications, where the user has access to detailed information about the trip and can confirm whether he/she is interested in sharing the trip, or if not, it can reject it.

24

A. F. Curralo et al.

Fig. 6. First sketches of the layouts for the On.Move app. Source: Authors

4 Conclusions and Future Work Garrett’s methodology [9] is of great importance for the development of the app, as it is a well-structured process, which requires a great depth in each phase and mainly in relation to the user’s needs. This methodology places a strong emphasis on understanding and addressing user needs and goals. By focusing on the user throughout each stage of the design process, encourages designers to create products that are intuitive, usable, and valuable to the intended audience. The user experience is determined by the different stages and participants involved in the process. The methodology presents a holistic framework that covers various aspects of user experience design, from high-level strategy to the detailed design of user interfaces. It provides a clear structure for designers to follow, ensuring that they consider all necessary elements during the design process. Good usability is the starting point for triggering user satisfaction. In terms of the interface, it must be easy to use, and it must be pleasant for the user to use the product. A usability test with the target audience was also added to the author’s original process, as this should be the central focus for creating a good product based on UX design. Thus, through this study we contributed to the definition of good methodological practices, with the aim of creating a good user experience service and system. In the future, and using the methodology here presented, it will be developed an operational and high-definition prototype app and it will be performed the respective usability tests. Usability tests are useful to validate the interaction tools present in the interface, being effective in delimiting the actual selection of functionalities and features; the manner in which the information is organized; how efficiently the users are able to complete tasks and how users are able to perceive failures, that is where they occur and how often.

Design Methodology for a Shared Mobility App

25

As future work, utilizing the approach demonstrated here, it is intended the development and the implementation of the On.Move app, making it accessible in all the IPVC’s campus. Funding:. This work is a result of the project TECH - Technology, Environment, Creativity and Health, Norte-01-0145-FEDER-000043, supported by Norte Portugal Regional Operational Program (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, through the European Regional Development Fund (ERDF).

References 1. Tánczos K, Török Á (2008) Impact of transportation on environment. Per. Pol. Transp. Eng. 36(1–2):105. https://doi.org/10.3311/pp.tr.2008-1-2.19 2. Barreto L, Amaral A, Pereira T, Paiva S (2023) A review of use cases of gamification in Mobility Systems and services. In: Nathanail EG, Gavanas N, Adamos G (eds) Smart energy for smart transport: proceedings of the 6th conference on sustainable urban mobility, CSUM2022, August 31-September 2, 2022, Skiathos Island, Greece. Springer Nature Switzerland, Cham, pp 335–347. https://doi.org/10.1007/978-3-031-23721-8_29 3. Sharp H, Rogers Y, Preece J (2023) Interaction design: beyond human-computer interaction. John Wiley & Sons, Inc., Hoboken 4. Cooper A, Reimann R, Cronin D (2007) About face 3: the essentials of interaction design. Wiley, Indianapolis, IN 5. Lupanda IS, Janse van Rensburg, JT (2021) Design guidelines for mobile applications. In: Proceedings of the 15th international conference on interfaces and human computer interaction 2021 and 14th international conference on game and entertainment technologies 2021, pp 92–99 6. Norman D (2005) Emotional design: why we love (or hate) everyday things. Basic Books, New York 7. Hassenzahl M, Law ELC, Hvannberg ET (2006) User Experience-Towards a unified vie. Ux Ws Nordichi. 6:1–3 8. Walter A (2020) Designing for emotion. Jeffrey Zeldman, New York, NY 9. Garrett JJ (2011) The elements of user experience: user-centered design for the web and beyond. Always Learning, Berkeley 10. Curralo A, Faria P, Curado A, Azeredo P, Lopes S: Designing a UX mobile app for hydration and sustainability tracking in academia. In: Ahram, T, Falcão, C (eds) Usability and user experience. ahfe (2022) international conference. AHFE Open Access, vol 39. AHFE International, USA. https://doi.org/10.54941/ahfe1001692 11. Carvalhido A, Novo R, Faria PM, Curralo A (2022) A user experience design process in mobile applications prototypes: a case study. In: Martins N, Brandão D (eds) Advances in design and digital communication II: proceedings of the 5th international conference on design and digital communication, Digicom 2021, 4–6 Nov 2021, Barcelos, Portugal. Springer International Publishing, Cham, pp 262–273. https://doi.org/10.1007/978-3-030-89735-2_22 12. Dondis D (2003) La sintaxis de la imagem. Introducción al alfabeto visual. Gustavo Gilli, Barcelona 13. Shneiderman B (1987) Designing the user interface: strategies for effective human computer interaction. Addison-Wesley, Reading

Brain Computer Interface: Behavioral Differences in the Calmness Level and Reading Time, Considering Screen or Paper, Serif or Sans Serif Typefaces Ana Rita Teixeira1,2(B) , Sónia Brito-Costa1 , Sílvia Espada1 , and Maria Fernanda Antunes1 1 Polytechnic of Coimbra, Higher School of Education, Coimbra, Portugal

{anateixeira,silviaespada,fernandantunes}@esec.pt 2 Aveiro University, IEETA, Aveiro, Portugal

Abstract. The main aim of this study was to understand the brain interaction and behavioral differences of the reading time and level of calmness related to the reading process (paper or screen; Sans Serif typefaces such as Agency, Calibri, or Arial; Serif typefaces such as Times, Bodoni, and Cambria; and levels of text complexity such as scientific or children’s text). Two experiments were conducted with a sample of 29 (20 + 9) participants, to understand which typeface (Serif or Sans Serif) presents a shorter reading time and higher levels of calmness, considering two different complexity texts (children and scientific) in two different forms of interaction (paper reading and screen reading). Signal acquisition was performed using MUSE EEG headphones during the tests we used various sensors, such as the Brain Computer Interface (BCI), to measure brain activity (active, neutral, and calm). The results of the two experiments demonstrate that Sans Serif typeface exhibits shorter reading times and higher levels of calmness than Serif typeface, especially when the text is read on a screen. In sum, different typefaces (Sans Serif and Serif) influenced the participants brain activity. Keywords: Typography · Reading · Eye movement · Serif and sans Serif Typeface · Brain computer interface

1 Introduction The human being has access in general to information and knowledge through the screens of our computers, tablets, smartphones or in books, magazines, and newspapers. In this sense, it is important to work the typefaces according to a study of readability, accessibility, and usability, because typography is the visual presentation of the text. Typefaces are important in the weight of information and how individuals understand the content they want to convey. Since the time there has been a correlation between typography and writing, however typography emerges as the interceded discipline between the content of the message and the readers of the message [1]. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 26–36, 2024. https://doi.org/10.1007/978-3-031-47281-7_3

Brain Computer Interface

27

Good typography makes people want to read more and makes it easier to understand what they’re reading. The arrangement and order of type has a significant impact on one’s capacity for quick reading and efficient time management [2]. Designers highlight the value of choosing the use of Sans Serif and Serif letters for different works and supports. Sans Serif typefaces are becoming increasingly popular on both paper and screen, and studies have been conducted to determine how they influence reading. It has been found that Sans Serif typefaces are easier to read on screens due to the lower resolution of digital displays and they also tend to be more legible and have less of a blur effect when viewed from a distance [3]. Eye-tracking studies have compared the reading process of a text, with Serif and with Sans Serif typefaces, on screen and found that participants read from Sans Serif typefaces faster and more accurately than a Serif typeface, and no difference between participants fixation duration in Serif and Sans Serif typeface while proofreading task [3, 4]. The findings of this study suggest that using Sans Serif typefaces rather than Serif typefaces in screen text can help reduce the amount of time needed to read the text and improve accuracy when proofreading. This can be beneficial for readers in a wide range of contexts, such as educational settings, news websites, and e-books. On paper, however, Sans Serif typefaces are found to be less readable than Serif [5, 6]. This is due to the lack of Serifs, which help the eye to distinguish between letters and form words more easily and additionally, Sans Serif typefaces can be more difficult to read when printed in small point sizes [5, 7]. The designers highlight the importance of Serif in relation to continuous reading on paper, pointing out a greater distinction of the characters and simultaneously in the consecution of rhythm in textual composition [8]. Reading on paper or screen can have a significant impact on cognitive and comprehension abilities [9, 10] since the reading on paper requires more cognitive effort, as the reader needs to physically turn pages and focus on the text, while reading on a screen can be more distracting due to the presence of other elements such as advertisements, images, and videos [6, 10, 11]. Therefore, reading on paper may lead to better comprehension and retention of the material due to the increased focus and effort required. On the other hand, reading on a screen can be beneficial for faster comprehension, as it allows for easier navigation and the ability to search for specific information [12, 13]. Additionally, reading on a screen can be beneficial for those who have difficulty reading on paper, as it offers the ability to adjust font size and contrast, as well as the option to use text-to-speech [14, 15]. Reading on paper has been linked to higher levels of comprehension and cognitive functioning than when reading on a screen. Previous studies have found that when reading on paper, readers are more likely to recall information better and comprehend more deeply [9, 16]. This is because paper provides tactile feedback, making it easier for readers to visualize information, which helps them to better focus on and remember content. Additionally, paper allows for more flexibility in terms of layout and formatting, which can be beneficial for readers who prefer certain ways of organizing text [17]. Reading on a screen, on the other hand, has been found to be more conducive to scanning and skimming, as well as taking notes [18]. This can be helpful for readers who are looking for quick insights or need to quickly reference materials. Additionally, screens provide access to a variety of digital tools and resources, such as highlighting,

28

A. R. Teixeira et al.

bookmarking, and search functions, that can enhance the reading experience. However, it is important to note that these advantages come at the cost of a more distracting environment, which can lead to decreased levels of comprehension and cognitive functioning [19]. In this sense, reading on paper and screen can have different effects on the reader. Reading on paper has been linked to improved cognitive performance due to the ability to access more information than would be available when reading on a screen [20, 21]. On the other hand, readers can look back over and review information with ease, allowing them to better comprehend and retain the material. Additionally, reading on paper has been found to be more calming than reading on a screen and this is since paper reading is less stimulating and can help the reader to focus and relax [22, 23]. Readability is frequently confused with legibility. However, readability is what makes some texts easier to read than others, whereas legibility concerns typeface and layout [24]. The selection of two distinct texts—one for children and the other with scientific content—is based on the importance of text complexity, and the various reading skills necessary for acquiring differing kinds of information. The acquisition of mechanical reading skills, such as left-to-right perception progression, precise eye-return sweeps from the end of one line to the beginning of the next, a small vocabulary, word recognition, and the use of accompanying punctuation, are all intended to support comprehension and interpretation of meanings [2]. Thus, our study sought to understand the various readings and the factors related to them (paper or screen; Sans Serif typefaces like Agency, Calibri, or Arial; Serif typefaces like Times, Bodoni, and Cambria; levels of text complexity like scientific or children’s text), as well as the preferences of the participants regarding their characteristics rather than the efficiency of the participants reading itself. This paper is organized into 4 sections as follows: Sect. 1, the introduction of the problem; Sect. 2, the material and methods (participants, procedures and devices), Sect. 3, the results and discussion, and in the last section, conclusions and further work is presented.

2 Material and methods 2.1 Procedures and Participants In this study two experiments were carried out to understand which typeface (Serif or Sans Serif) presents shorter reading time and higher levels of calmness considering two different texts (children’s and scientific) in two distinct forms of interaction (paper reading and screen reading). The participants were instructed to read texts while their brain activity was being monitored. We examined the effects of visuo-typographic factors considering reading time and brain activity (Calmness level). Children and adults were both recruited. To identify possible modulations of the effects of visuo-typographic factors as a function of type of text as well as reading medium, both age groups included typical readers on paper and on screens. A questionnaire was conducted before the tests, to obtained information whether participants read regularly and wear glasses.

Brain Computer Interface

29

• Experiment I – Sans Serif Typefaces (Agency, Calibri and Arial) 20 participants, 12 are female and 8 are male. The ages range from 11 to 27 (mean = 23). Regarding reading habits, it was found that 60% (12) of the participants read regularly and 40% (8) of them do not. Regarding the use of glasses, 70% (14) of the participants have vision difficulties and 30% (6) do not. • Experiment II – Serif Typefaces (Times, Bodoni and Cambria). 9 participants, 7 are female and 2 are male. The age group volunteers range from 14 to 33 years of age (mean = 19). Regarding reading habits, it was found that 66.7% (6) of the participants read regularly and 33.3% (3) of them do not. Regarding the use of glasses, 22.2% (2) of the participants have vision difficulties and 77.8% (6) do not. All participants provided written informed consent prior to the experimental session. The research has been conducted adhering to the ethical guidelines and the legal requirements of the country in which the study took place (Portugal). 2.2 Signal Acquisition Signal acquisition was performed using MUSE EEG headset during the tests. Muse is a Brain Sensing Headband [25], which allowed to collect data on the active, calm, and neutral values by measuring the brain wave signals of the participants, while doing the two experiments. The headband works by using electroencephalography (EEG) to detect electrical activity within the brain. EEG uses electrodes placed on the scalp to measure brainwave activity [20]. The headband uses seven electrodes to measure the electrical activity of the brain. Two of the electrodes are placed on the forehead, two behind the ears, and three reference electrodes which measure the electricity of the brain. The electrodes capture the brainwaves which are then sent to a processing unit where the data is analyzed. The data is then used to measure various aspects of the brain activity, such as the active, calm, and neutral levels. The gathered variables were then subjected to statistical tests, such as Mann-Whitney U test and Kruskal Wallis test, to determine if there were any significant relationships between them. The results of the statistical tests were then interpreted to evaluate the strength of the relationships between the variables and to determine if any of them had a meaningful influence on the outcomes. The results of the statistical tests were also used to draw conclusions about the data and make predictions about future trends and outcomes.

3 Results The results presented in this section will try to answer which are the better typeface considering different texts and two interaction forms of reading. For the two experiments defined in the Sect. 2.1, the results presented are based in three analyses: 1. Comparison of time reading and calmness level between all Sans Serif and all Serif typefaces considering all variations (screen/ paper in scientific and children’s text)

30

A. R. Teixeira et al.

2. Comparison of the different forms of interaction, screen/ paper in scientific and children’s text considering the results of reading time and level of calmness for Sans Serif and Serif typefaces 3. Comparison between Sans Serif and Serif typefaces Figures 1 and 2 provides a comparison of different Sans Serif typefaces (Agency, Calibri and Arial) considering reading time and calmness level, respectively. Applying the Kruskal Wallis test between each group (Agency, Calibri and Arial) we verified that for the option Paper SC in case of reading (Fig. 1) the data are statistically different with (p = 0.0374). Using the U Mann Whitney test to compare the means in this circumstance amongst the three typefaces, we discover that Agency and Arial are statistically different at 95% (p = 0.02025). It was also noted here by visual analysis that Arial in the case of Scientific/Paper presents lower reading time as well as Calibri in the case of Scientific/Screen (Fig. 1). Screen SC

Screen CH

100 80 80

70 60

60

50 Agency

Calibri

Arial

Agency

Paper SC

Calibri

Arial

Paper CH 90

100

80

80

70 60

60 50 40 Agency

Calibri

Arial

Agency

Calibri

Arial

Fig. 1. Comparison of different fonts considering reading time

As for the level of calmness, applying the comparison of the different samples, they do not reveal differences, however it is verified that in the case of the scientific screen the Calibri typeface is the one that leads to higher levels of calmness, corroborating with the previous data on reading time (shorter reading time) (Fig. 2). Figures 3 and 4 present the comparison of the different forms of interaction, screen/ paper in scientific and children’s text considering the results of reading time and level of calmness in the task: Agency, Calibri and Arial, respectively. Using the Kruskal only the Agency presents differences for the time (p = 0.029) and for the calmness (p = 0.0033). Considering the U de Mann-Whitney test we have the following statistical differences in the average of values for the time of reading: Scientific/Screen and Scientific/Paper for the agency (p = 0.0708); Screen/Children and Paper/Children in the case of the Arial

Brain Computer Interface Screen SC

31

Screen CH 60

40 40 20

20

0

0 Agency

Calibri

Arial

Agency

Paper SC

Calibri

Arial

Paper CH

60 60 40

40

20

20

0

0 Agency

Calibri

Arial

Agency

Calibri

Arial

Fig. 2. Comparison of the different typefaces considering the level of calmness.

in the level of calmness (p = 0.085) and Screen/Children and Paper/ Children for the Agency in the level of calm (p = 0.0036). 110 110

100

100 100

90

90 90

80

80

80

70

70

60

60

60

50

50

50

40

40

70

Screen SC

Screen CH

Paper SC

Paper CH

Screen SC

Screen CH

Paper SC

Paper CH

40 Screen SC

Screen CH

Paper SC

Paper CH

Fig. 3. Level of calmness in the task: Agency, Calibri and Arial

Arial Calm Level

Calibri Calm Level

Agency Calm Level

70

70 60

60

60 50

50

50 40

40

40 30

30

30 20

20

20 10

10

10 0

0

0 Screen SC

Screen CH

Paper SC

Paper CH

Screen SC

Screen CH

Paper SC

Paper CH

Screen SC

Screen CH

Paper SC

Paper CH

Fig. 4. Level of calmness in the task: Agency, Calibri and Arial

Figures 5 and 6 show the comparison of different Serif typefaces (Times, Bodoni and Cambria) considering reading time and calmness level, respectively.

32

A. R. Teixeira et al.

Applying Kruskal’s test between each group (Times, Bodoni and Cambria) we found no differences in time reading (Fig. 1). The fact that the number of participants was small should be highlighted. By comparing the graphs in Fig. 1 with this one, we can see that Serifed text leads to shorter reading times (described in the Table 1) (Fig. 5). On the other hand, applying Kruskal’s test considering the level of calmness there are differences in the case of the Scientific/Screen (p = 0.0851). It is noteworthy that Bodoni has lower levels of calmness. In addition, the Bodoni has a very variable range of values in the case of the Children’s /Screen (Fig. 6).

Fig. 5. Comparison of different typefaces considering reading time between each group (Times, Bodoni and Cambria)

Figures 7 and 8 present the comparison of the different forms of interaction, screen/paper in scientific and children’s text considering the results of reading time and level of calmness in the task: Times, Bodoni and Cambria, respectively. Applying Kruskal Wallis to for the levels of calm, the Bodoni type shows a different distribution p = 0.00345 (Fig. 7). It is pointed out that the results of U Mann Whitney test for Bodoni Paper/Scientific is statistically different from all the others as is visible from the graph. However, given that the number of individuals is very small in this experiment, it will be necessary to increase the number of participants (Fig. 8). Tables 1 and 2 resume the average values of reading time and calmness level considering Screen vs Paper, Serif vs Sans Serif and Children vs Scientific text. As we see, Serif typefaces present the faster reading time, namely the Cambria typefaces. The same conclusion is obtained by calmness level. Serif typefaces present higher level of calmness comparing with Sans Serif.

Brain Computer Interface Screen SC

33

Screen CH

40

40

30

30

20

20

10 Times

Bodoni

Cambria

Times

Bodoni

Cambria

Paper CH

Paper SC 50 50 40

40

30

30

20

20

10

10 Times

Bodoni

Cambria

Times

Bodoni

Cambria

Fig. 6. Comparison of the different typefaces considering the level of calmness (Times, Bodoni and Cambria) Times

Bodoni

Cambria

70 65

60

65

60

60

55

55

50

50

55

50

45

45

45

40

40

40

35 35

35 30

30 Screen SC

Screen CH

Paper SC

Paper CH

Screen SC

Screen CH

Paper SC

Paper CH

30 Screen SC

Screen CH

Paper SC

Paper CH

Fig. 7. Level of time in the task: Times --- Bodoni ---- Cambria Bodoni Calm Level

Times Calm Level 45 40

Cambria Calm Level

50

50

45

45

40

40

35

35

30

30

25

25

20

20

35 30 25 20 15

15

15

10

10

10

5

5

5

0

0 Screen SC

Screen CH

Paper SC

Paper CH

0

Screen SC

Screen CH

Paper SC

Paper CH

Screen SC

Screen CH

Paper SC

Fig. 8. Level of calmness on the task: Times --- Bodoni ---- Cambria

Paper CH

34

A. R. Teixeira et al. Table 1. Average reading time: Screen vs Paper, Serif vs Sans Serif, Children vs Scientific Serif

Average

Screen Paper

Sans Serif

Times

Bodoni

Cambria

Agency

Calibri

Arial

Children

51,55

49

47,33

66,4

60,6

58,5

Scientific

51,77

48,44

46,66

62,85

58,75

58,8

Children

48,66

47,55

47,11

57

56,25

56,2

Scientific

45,33

44,88

44,66

57,15

56,95

56,1

Table 2. Average level of calmness: Screen vs Paper, Serif vs Sans Serif, Children vs Scientific Serif Average

Sans Serif

Times

Bodoni

Cambria

Agency

Calibri

Arial

14,88

22,66

20,33

35,55

34,75

31,25

Screen

Children Scientific

25,22

28,11

20

38,9

31,95

33

Paper

Children

23,33

23

21,44

24,1

28,35

27,2

Scientific

18,66

24,66

26,33

23,1

26,95

26,35

4 Discussion and Conclusion In this study, two experiments were carried out to understand which typeface (Serif or Sans Serif) presents shorter reading time and higher levels of calmness considering two different complexity texts (children’s and scientific) in two distinct forms of interaction (reading on paper and reading on screen). Overall, the results of our two experiments demonstrate that the Sans Serif typeface presents shorter times of reading and higher levels of calmness than the Serif typeface, especially when the text is read on a screen, which corroborates the results found in previous studies [3–8], and may help to improve comprehension and to be beneficial for readers in a wide range of contexts, such as educational settings, news websites, and e-books. The cognitive effort needed to read on paper can have a positive impact on comprehension, as it allows the reader to better focus on the text and process it more accurately and deeply [10]. Moreover, paper-based reading can also be beneficial for memory retention, as it enables the reader to better visualize the text, thus facilitating the process of remembering [9, 11, 26]. On the other hand, reading on a screen can be more distracting due to the presence of other elements [6, 11]. This can lead to reduced comprehension and difficulty in concentrating on the text. Furthermore, reading on a screen can also cause physical and mental fatigue due to the strain of staring at a digital device for an extended period [6, 11]. Overall, reading on paper can be beneficial for comprehension and memory retention, while reading on a screen can lead to distractions and fatigue.

Brain Computer Interface

35

Thus, it is important to consider the type of reading material and the environment in which it is being read in order to maximize comprehension and learning. In addition, both paper and screen formats have their advantages and disadvantages when it comes to reading comprehension and retention. While reading on paper may lead to better comprehension and retention due to the increased focus and effort required, reading on a screen can be beneficial for faster comprehension, as it allows for easier navigation and the ability to search for specific information [12, 13], since individuals with reading difficulties may also find reading on a screen easier than paper due to the availability of adjustable font size and contrast, as well as the option to use text-tospeech [14, 15], however, on screen readers in the pressured condition comprehended significantly less than the others [10]. In sum, the choice between paper and screen formats should be based on the individual’s needs and preferences. The study of BCI technology in the context of typography involves several issues, such as typographic element selection, text type, and EEG data processing. In this study, we used several typographic elements, including Sans Serif and Serif typefaces, to examine how brain activity varies when exposed to these elements. To assess the readability of the text, we also examined changes in brain activity when exposed to paper and screen. The main weakness of this study is the small sample size employed. As a result, we intend to continue this work by employing the same technique in a bigger sample, in which we will test additional typefaces and texts of greater complexity, and we also intend to employ. Different typefaces (Sans Serif and Serif) had an effect on the participants’ brain activity, according to our findings. The EEG results showed that the fonts influenced the participant’s brain activity. Furthermore, the results revealed that the combination of typographic elements with interaction mode (screen and paper) for children and scientific literature influenced the participants’ brain activity.

References 1. Jury D (2007) Que Es La Tipografia?/What’s Typography? (Spanish Edition) 2. Jury D (2002) About face: reviving the rules of typography 3. Dogusoy B, Cicek F, Cagiltay K (2016) How serif and sans serif typefaces influence reading on screen: 4 eye tracking study. In: Marcus A (ed) Design, user experience, and usability: novel user experiences: 5th international conference, DUXU 2016, held as Part of HCI international 2016, Toronto, Canada, July 17–22, 2016, Proceedings, Part II. Springer International Publishing, Cham, pp 578–586. https://doi.org/10.1007/978-3-319-40355-7_55 4. Josephson S (2008) Keeping your readers’ eyes on the screen: an eye-tracking study comparing sans serif and serif typefaces. Visual Commun. Q. 15(1–2):67–79. https://doi.org/10.1080/ 15551390801914595 5. Franken G, Pangerc M, Možina K (2020) Impact of typeface and colour combinations on lcd display legibility. Emerg Sci J 4(6):436–442. https://doi.org/10.28991/esj-2020-01243 6. Godfroid A, Boers F, Housen A (2013) An eye for words. Stud Second Lang Acquis 35:483– 517. https://doi.org/10.1017/S0272263113000119 7. Kitroeff N (2015) The Best and Worst Fonts to Use on Your Résumé 8. Vignon V (2012) Reading letters. Designing for legibility Sofie Beier, Amsterdam, Bis Publishers, 2012. Commu. Langages 2012(174):148–149

36

A. R. Teixeira et al.

9. Tajuddin ES, Mohamad FS (2019) Paper versus screen: impact on reading comprehension and speed. Indonesian J Educ Methods Dev. https://doi.org/10.21070/ijemd.v3i2.20 10. Delgado P, Salmerón L (2021) The inattentive on-screen reading: Reading medium affects attention and reading comprehension under time pressure. Learn Instruct 71:101396. https:// doi.org/10.1016/j.learninstruc.2020.101396 11. Hou J, Wu Y, Harrell E (2017) Reading on paper and screen among senior adults: cognitive map and technophobia. Front Psychol 8:2225. https://doi.org/10.3389/fpsyg.2017.02225 12. Kol S, Schcolnik M (2013) Enhancing screen reading strategies. CALICO J 18(1):67–80. https://doi.org/10.1558/cj.v18i1.67-80 13. Duchnicky RL, Kolers PA (1983) Readability of text scrolled on visual display terminals as a function of window size. Hum Factors: J Human Factors Ergon Soc 25(6):683–692. https:// doi.org/10.1177/001872088302500605 14. Furenes MI, Kucirkova N, Bus AG (2021) A comparison of children’s reading on paper versus screen: a meta-analysis. Rev Educ Res 91(4):483–517. https://doi.org/10.3102/003 4654321998074 15. Jabr F (2013) The Reading Brain in the Digital Age: The Science of Paper versus Screens. Sci Am. 16. Niccoli A (2015) Paper or Tablet? Reading Recall and Comprehension. EDUCAUSE Review 17. Börner K, Bueckle A, Ginda M (2019) Data visualization literacy: definitions, conceptual frameworks, exercises, and assessments. Proc Natl Acad Sci USA 116(6):1857–1864. https:// doi.org/10.1073/pnas.1807180116 18. Alrizq M et al (2021) Analysis of skim reading on desktop versus mobile screen. Appl Sci 11(16):7398. https://doi.org/10.3390/app11167398 19. Weeks JC, Hasher L (2014) The disruptive – and beneficial – effects of distraction on older adults’ cognitive performance. Front Psychol 5:133. https://doi.org/10.3389/fpsyg.2014. 00133 20. Shih JJ, Krusienski DJ, Wolpaw JR (2012) Brain-computer interfaces in medicine. Mayo Clin Proc 87:268–279. https://doi.org/10.1016/j.mayocp.2011.12.008 21. Shi Z, Tang T, Yin L (2020) Construction of cognitive maps to improve reading performance by text signaling: reading text on paper compared to on screen. Front Psychol 11:571957. https://doi.org/10.3389/fpsyg.2020.571957 22. Kazazo˘glu S (2020) Is printed-text the best choice? A mixed-method case study on reading comprehension. J Lang Linguist Stud 16(1):458–473. https://doi.org/10.17263/jlls.712879 23. Kazanci Z (2015) University students’ preferences of reading from a printed paper or a digital screen — a longitudinal study. Int J Culture History (EJournal) 1(1):50–53. https://doi.org/ 10.18178/ijch.2015.1.1.009 24. DuBay WH (2004) The principles of readability: a brief introduction to readability research. Impact Inform 25. Krigolson OE, Williams CC, Norton A, Hassall CD, Colino F (2017) Choosing MUSE: validation of a low-cost, portable EEG system for ERP research. Front Neurosci 11:109. https://doi.org/10.3389/fnins.2017.00109 26. Vogel D, Meyer M, Harendza S (2018) Verbal and non-verbal communication skills including empathy during history taking of undergraduate medical students. BMC Med Educ 18(1):157. https://doi.org/10.1186/s12909-018-1260-9

Anticipation as a Tool for Designing the Future Joana Cerejo1(B) and Miguel Carvalhais2 1 Faculty of Engineering, University of Porto, R. Dr. Roberto Freitas, 4200-465 Porto, Portugal

[email protected]

2 i2ADS/Faculty of Fine Arts, University of Porto, Av. Rodrigues de Freitas 265, 4049-021

Porto, Portugal [email protected]

Abstract. Anticipation is a concept that allows us to predict and plan for future events. As a tool, it enables us to forecast peoples’ desires or intentions and influences their behaviors and learning regarding digital experiences. This paper delves into the concept of anticipation and its relevance in designing for the future, particularly in the fields of User Experience (UX) and Human-Computer Interaction (HCI). It explores and defines anticipation concepts, definitions, and manifestations, and explores the various types and applications in various fields and their relationship with intelligent systems. This paper provides an extensive overview of how anticipation can shape the design field. This translates to leveraging Artificial Intelligence (AI) and Machine Learning (ML) to construct digital experiences that not only resonate with users but also proactively address their evolving needs. By examining the connection between anticipation and designing for the future, this paper offers a valuable resource for scholars and professionals interested in exploring the concept of anticipation and its potential to shape future outcomes. Keywords: Anticipation · Anticipatory Experiences · Intelligent System · Artificial Intelligence · Machine Learning · Future Studies · Foresight · Forecast

1 Introduction At the intersection of web and multimedia design and the future, lies foresight, the powerful concept of anticipation, a proactive approach that reaches into the future to shape present actions. In the realm of designing digital solutions, the role of anticipation is crucial. It informs how we craft user experiences, optimize systems, and innovate for the unmet needs of tomorrow. This paper delves into the nuances of anticipation, especially its three primary facets in design: optimization, contingency, and novelty. Anticipation isn’t merely about predicting the future—it’s about using insights from what might come to influence and shape the present. Originating from the Latin verb Anticipalis, which means “to get ahead of,” anticipation has historically been a subject of fascination, with early inklings found in science fiction literature [35]. But its application in modern design, especially digital solutions, is far from fictional—it’s an increasing possibility. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 37–52, 2024. https://doi.org/10.1007/978-3-031-47281-7_4

38

J. Cerejo and M. Carvalhais

Anticipation provides ideas and tools to shape what humans perceive as expected futures. The critical characteristic of anticipation is the attentiveness to future states of the world as an orientation for reacting, planning actions, and making futures [36]. Due to the future-oriented view, anticipation is a very appealing and complicated idea to pursue. “Etymologically, anticipation has its roots in ante, which means before by referring to being aware of something coming at a future time. The words used as synonyms of anticipation are expectation, prospect, forestalling, foreseeing, foresight, and so on. The synonyms of anticipation as noun mean expecting and looking forward” [16]. 1.1 The Essence of Anticipation “Anticipation is defined as the process of predicting future actions or behaviors based on any kind of information from the current or past” [16]. There are a short and a long answer to what is anticipation. The straightforward answer is that “Anticipation is future-based information acting in the present.” [26] A more detailed answer is that anticipation has two aspects: the relevant system includes a model of its future development and uses the information related to that model to make its decisions in the present moment [26]. In the design field, anticipatory experiences involve using the willingness to foresight the future to facilitate decision-making, set future-oriented goals, and plan strategies for future design outcomes. This translates to leveraging Artificial Intelligence (AI) and Machine Learning (ML) to construct digital experiences that not only resonate with users but also proactively address their evolving needs. By leveraging these technologies, designers can create intelligent systems that can learn from user interactions and adapt to their needs. The term anticipatory appears in many field studies, and all the fields discuss anticipation as a notion of predicting future actions, as essential, and as an inherent component of systems. Anticipation manifests uniquely across various fields of study. In Physics, Fantappiè introduced the synthropic process in the 1940s, illustrating anticipation as a phenomenon that defies the standard flow of time [9]. In Biology, anticipation serves as the core of relational studies, viewing organisms as prime examples of anticipatory systems [26, 30]. Psychologists interpret behavior as being more influenced by future goals than past stimuli, positioning anticipation as a function of prospection and the representation of potential future states [31]. Anthropology, in its analysis of societies, regards anticipation as a means to craft the future as a cultural fact, emphasizing imagination, anticipation, and aspiration [3, 14]. Sociologists, amidst the uncertainties of modern society, perceive anticipation as a bridge between the distant future and the present [2]. Economists juxtapose their anticipatory approach with that of sociologists: while the former backtrack from the future to explain decisions by expected future rewards [1, 4], the latter see the present as an outcome emerging from the past. Philosophers dive deep into the interplay of anticipation, ethics, metaphysics, and biology, articulating that influential pasts and futures reside within the present [18]. Lastly, in Design, anticipation is harnessed to simplify system interactions, preemptively addressing user needs and automating tasks for enhanced efficiency [32]. Although anticipation, anticipatory behavior and anticipatory systems are often used indiscriminately, they all have distinctive characteristics that differentiate them from predictive systems:

Anticipation as a Tool for Designing the Future

39

• Anticipation is an overall capacity, not exclusive to humans, to act in response to preparation for a potential future reality [36]. • Anticipatory behavior is a process, or behavior, that does not only depend on past and present but also on predictions, expectations, or beliefs about the future [5]. • Anticipatory System is a system whose current state is determined by a future state. ‘The cause lies in the future’ [30]. 1.1.1 Categorizing Anticipation We can categorize anticipation into biological, psychological, and social dimensions: each has its own distinguished properties and inherent patterns. Due to the cooperation of behavior and learning capabilities of these three dimensions, we cannot study anticipation without an interdisciplinary approach [7, 19, 26, 30]. Anticipation can be a qualitative methodology supported by quantitative inputs [16]. Anticipatory systems benefit from both qualitative and quantitative techniques to glean insights, as they stem from abduction, deduction, and induction reasoning. On the qualitative side, methods like presenting behavioral information, conducting interviews with key participants, developing scenarios, hosting roundtable debates, utilizing focus groups, employing the Delphi method, and comprehensive case studies, especially in specific areas, are common [36]. Conversely, quantitative techniques for anticipation encompass surveying participants or groups, implementing formal, national-level, model-based projections, and examining time series. In the presence of time series, complex analyses are often required to discern patterns predictive of future trajectories, with these methods being prevalent in financial and business sectors. They are generally more adept at short-term predictions than at forecasting long-term shifts. In situations where time-series data is unavailable, extrapolation of historical trends is preferred [36]. Anticipation is a holistic approach to forecasting new futures, but non-systemic methods may be inconsistent and subjective. Design, as a future-oriented approach, is commonly used to solve uncertain, complex, and unknown problems, also known as wicked problems. The future-oriented approach is what bonds anticipation and design to construct future realities [36]. Therefore, the authors state that design is part of anticipatory studies, an interactive dialogue between present and future. 1.1.2 Anticipatory Behavior Anticipation guides behavior and can provide insight into the future for better decisionmaking. An everyday example of anticipatory behavior is checking the weather forecast and bringing an umbrella in anticipation of rain. In other words, anticipatory behavior is based on predictions, expectations, or beliefs about a future outcome. We are interested in uncovering when anticipatory behavior is useful in highly intelligent systems, which user data properties enable effective anticipatory behavior, what type of anticipatory behavior can be distinguished when designing anticipatory experiences, and what are the distinct behavioral impacts of anticipatory behavior processing (see Fig. 1). We understand prediction as a forecasting property and expectations as a backcasting property. Predictions influence learning and expectations behavior. Intentions initiate behavior execution [24, 26, 30]. Designers need to understand how the flux of data can play an essential role in how we design anticipatory experiences for successful behavioral

40

J. Cerejo and M. Carvalhais

Fig. 1. Our comprehension of anticipatory behavior.

change. For example, understand which kind of contextual data I will need to properly support an intelligent cognitive system that triggers anticipatory behavior. The most challenging claim of anticipative behavioral control is the idea that motor commands, or actions, are based on anticipations of their sensory feedback. This means that the usual causal relationship between actions and their sensory effects is reversed, with actions being caused by anticipations of their sensory effects [5]. 1.1.3 Anticipatory Systems How today we see anticipation was heavily influenced by Rosen’s work [30], who defines an anticipatory system as having a predictive model of itself and/or its environment, enabling it to adapt to changing states in line with predictions. Some authors describe anticipatory systems as using predictive capabilities to optimize behavior and learning [24]. This idea is not widely accepted, some authors argue that anticipation can be intrinsic to a system without an explicitly programmed model [36]. In synthesizing these perspectives—anticipation as a multi-dimensional concept, anticipatory behavior guiding actions through future insights, and anticipatory systems dynamically adapting with foresight—we discern a layered understanding that underlines the profound impact of anticipation in shaping design’s frontier. Within the realm of interaction design, this intricate dance of anticipation paves the way for more intuitive, proactive, and user-centric digital solutions, amplifying the potential for user outcomes and facilitating greater achievements.

2 Literature Review The narrative arc of anticipation offers insights into how human beings perceive, act upon, and construct their futures. From the philosophical reflections of ancient times to the rigorous empirical studies of today, anticipation has remained a focal point of wonder, holding clues to the profound relationship between the present and the desired future. Anticipation is also a mental activity that humans and other living organisms are capable of and is frequently practiced via utilization of the knowledge of future states to

Anticipation as a Tool for Designing the Future

41

guide the current behaviors [16]. In decision-making, anticipation can help individuals identify potential risks and benefits of different options, which can help them make better decisions [25]. Anticipation allows individuals to consider the consequences of their decisions and weigh the potential outcomes against each other. Anticipation studies aim to develop, categorize, and spread definitions of the process and systems inherent to anticipation. Anticipation as a field of study pursues: • • • •

A shared awareness of how to use the future to define the present. The convergence of novelty ideas influences the developments of the present. Tools and methods to support the expansion and relevance of anticipation. The expansion of human capabilities to perceive and make sense of novelty.

Anticipation involves predicting future actions or behaviors based on information from the past or present. By anticipating potential outcomes, individuals can plan and prepare for the future, which can increase the likelihood of achieving their desired outcomes. Additionally, anticipation can help individuals cope with uncertainty and change by preparing for different possibilities. By anticipating potential future scenarios, individuals can develop contingency plans and adapt their decision-making strategies as needed. 2.1 Anticipation and the Philosophical Roots of Causality Historically, the notion of anticipation has been intertwined with the philosophical understanding of time, causality, and the nature of existence. Ancient philosophers grappled with the intricate relationship between cause and effect, trying to discern the nature of events and their consequences. This profound exploration into the realm of causality paved the way for modern scientific inquiries and is reminiscent of contemporary endeavors in the field of ML. This technology, traditionally, relies on the identification of patterns from vast datasets without necessarily understanding the causal relationships underlying these patterns. However, with the advent of causality theory, there’s a growing emphasis on understanding not just correlations but the causal dynamics that drive these correlations. In other words, instead of merely predicting an outcome based on previous observations, causality-driven models strive to understand why a particular outcome might occur [22, 32]. Building on the foundations of anticipation and causality, the marriage of these concepts becomes critically relevant in the realm of user experience (UX). As the field of UX design evolved, it increasingly recognized the importance of not just understanding users’ immediate needs but also anticipating their future desires and behaviors [21]. In a sense, effective UX design operates in a space of future thinking, where designers aim to predict and respond to user needs before, they even arise. Causality theory, when applied to UX, provides a structured approach to understanding the “why” behind user behaviors [23]. Just as in machine learning where models strive to understand the underlying causal relationships instead of mere correlations, UX designers can benefit from understanding the causal factors that drive user decisions. This knowledge, rooted in anticipation, enables designers to craft experiences that not only react to but also shape user behavior in desired ways [15]. For instance, if a designer

42

J. Cerejo and M. Carvalhais

observes that users frequently abandon a checkout process at a specific step, the conventional approach might involve optimizing that step based on patterns observed. But a causality-informed approach would dig deeper, trying to understand the underlying reasons (or causes) for this behavior. Is it a lack of trust in the payment gateway? Is it the sudden appearance of unexpected fees? Or perhaps it’s the absence of a particular payment option? By anticipating users’ needs and concerns grounded in causal understanding, with the support of ML, designers can proactively address these issues, leading to more seamless, enjoyable, and efficient user experiences. The result is a UX that doesn’t merely react to users’ expressed preferences but understands and acts upon the deeper motivations and needs, leading to enhanced satisfaction and engagement [10]. At its core, the synergy between anticipation and causality theory provides a transformative lens for UX professionals. By adeptly applying these principles, they can unravel the intricate network of user behaviors, preemptively discern desires and expectations, and consequently craft innovative solutions that streamline user interactions, minimizing cognitive strain in their encounters with technology. 2.2 Future Studies, Anticipation, and its Capabilities Future studies, as an interdisciplinary field, delves deep into exploring, imagining, and creating future scenarios. Such endeavors harness a myriad of tools, methods, and frameworks drawn from various knowledge spheres, all in a bid to decode potential future paths. Central to this intricate dance of foresight is the notion of anticipation. In the evolving landscape of our interconnected world, anticipation becomes not just a tool, but an imperative. It empowers individuals and societies to shape preferred futures, charting roadmaps infused with intent and purpose [26, 34]. As the complexities of our globalized society swell, anticipation’s role in future studies magnifies, serving as a beacon guiding humanity towards sustainable and coveted outcomes [27]. Anticipation, in this context, becomes a crucial tool, enabling individuals and societies to envision preferred futures and craft strategies to achieve them [26, 34]. Compared with the growing complexities of our globalized world, the role of anticipation in future studies becomes even more critical, guiding humanity towards sustainable and desired outcomes [8, 26]. From its early philosophical undertones to its current empirical explorations, anticipation remains a compelling subject of study. The intricate interplay of past experiences, present actions, and future projections underscores the need for a deeper understanding of anticipation, especially in designing interactive systems that aspire for significant user achievements. Harnessing the insights from historical and modern discourses on anticipation promises to pave the way for more profound user-centric designs and innovations. Yet, the essence of anticipation isn’t monolithic. At its core, anticipation battles into two components: explicit and implicit [25, 26]. Both are a way of generating the necessity of imaginary futures based on probabilistic or non-probabilistic thinking to understand and act in the present. Implicit anticipations exploit what most humans acquire without effort or reflection; they are based on our aptitude for understanding future events that help us anticipate the future trajectory of objects or situations, like avoiding being hit

Anticipation as a Tool for Designing the Future

43

by a car or engaging in errors that may turn out to be irrevocable. An example of implicit human anticipation is when an “individual behavior is guided by anticipatory mechanisms that are used for behavioral control, perceptual processing, goal-directed behavior, and learning” [24]. Implicit anticipations tap into the essential, those automatic precognitions most humans wield without deliberate thought. This subconscious foresight guides our everyday actions, from escaping imminent dangers like oncoming cars to precluding irreversible mistakes. Such essential anticipations manifest when human behavior synchronizes with anticipatory mechanisms, steering perceptual processes, behavioral controls, goal orientations, and learning paradigms [24]. Within the realms of both explicit and implicit anticipatory experiences, design plays an influential role. The art of anticipatory design, rooted in these anticipatory mechanisms, refines user decision-making at three cardinal junctures [17]: • Simplifying decisions • Choice editing • Eliminating decisions Within explicit or implicit anticipatory experiences, anticipatory design can streamline the decision-making process on three levels: 2.2.1 Simplifying Selection With ML, algorithms can establish personalized pre-selection defaults and remember their user’s preferences that therefore serve them back. The Spotify feature “Your Daily Drive” is a good example. When Spotify suggests a playlist, it is not shooting in the dark but making an educated guess based on users’ past listens and preferences. It is anticipating what kind of songs users like. Because their suggestions are incredibly accurate, they do not have to go through the burden of creating their playlist. Therefore, features like this significantly reduce cognitive overload with convenience and relevance. 2.2.2 Choice Editing Curation can be essential in choice editing. Curation can reduce decision fatigue by removing options and streamlining choices for users. Easysize is an AI-powered clothing company that uses big data and AI to curate information for users, resulting in improved sizing recommendations and reduced returns. Their use of contextual data exemplifies the importance of such data for anticipatory experiences. The high percentage of returns due to incorrect sizing represents a significant cost to the fashion industry and the environment. According to the company, nearly 30 to 40% of all online fashion is returned due to the wrong size and fit. This translates into worldwide spending of $32 billion (about $98 per person in the US) on handling size-related returns annually. This represents a substantial logistic effort and cost for brands and even for the sustainability of our planet, given that the items must move around from one place to another. Their secret is a lot of contextual data and most of today’s anticipatory experiences out there rely heavily on contextual data. This will result in service optimization for the most common use case.

44

J. Cerejo and M. Carvalhais

2.2.3 Eliminating Decisions Anticipatory design allows designers to create services that surprise and delight users by reducing the choices they need to make. However, designers must be cautious about how much decision-making they eliminate. The level of eliminating decisions can be said to be the hardest to achieve successfully, but a successful example comes from the Digit app. Anticipatory design is vital for designing highly autonomous agents, and designers need to develop an anticipatory thinking skill set to navigate complex and complicated systems successfully to analyze system states, anticipate outcomes, and forecast future events [12]. This requires a human-centered AI (HCAI) mindset to create successful interactions between the user, the environment, and autonomous systems. Anticipatory design brings future-oriented goals to life and helps imagine possible alternative futures.

3 Methodology: Analyzing the Role of Anticipatory Design In an era marked by user-centric business models and accelerated technological advancements, anticipatory design emerges as a powerful tool, harnessing both explicit and implicit anticipatory mechanisms to craft systems tailored to user needs. As the future continually waves with possibilities and uncertainties, the belief that our actions today can directly shape tomorrow becomes ever more paramount. Rooted in this notion, anticipatory design seeks not just to respond, but to early address challenges and opportunities. The world of design is no stranger to the process of anticipation; from creating products and services that cater to emerging user needs to developing systems that can adapt and evolve with changing circumstances [7]. However, the concept of anticipatory design transcends mere foresight. It is the embodiment of the belief that by studying patterns, both manifest and latent, designers can preemptively influence future outcomes. Rooted in the idea that the future can be directly influenced, our research seeks to understand how anticipatory design functions within the broader framework of future studies and strategic foresight. Positioning anticipatory design within the extensive landscape of future studies and strategic foresight presents an intriguing challenge. There is very few literature or case studies that explore or exploit these fields synergies. These broader disciplines, while offering structured frameworks for future envisioning and strategizing, struggle with the complexities of deciphering and influencing possible tomorrows. While these disciplines provide frameworks for envisioning and strategizing for the future, how does anticipatory design fit into this mixture? To answer that, we support our research on ground theory, supported by an exhaustive literature review and rigorous data analysis, to navigate this uncharted terrain. Grounded theory is a systematic methodology in the social sciences involving the construction of theories through methodical gathering and analysis of data [13]. Our study employs the grounded theory methodology to unravel the intricacies of anticipatory experiences. Employing this methodology, we meticulously dissect anticipatory experiences, striving to unveil the intricate dance between anticipatory design, future studies, and strategic foresight. From understanding explicit mechanisms like optimization, contingency, and novelty to unpacking the subtleties of implicit human anticipations, our study aspires to present an all-encompassing view of the anticipatory domain.

Anticipation as a Tool for Designing the Future

45

Our journey’s lies on the committed objective of distill actionable insights that empower designers to synergize with emerging technologies, especially the expanding horizons of AI and ML [6, 7]. In doing so, we aim to optimize user experiences and catalyze genuine innovation. This paper elucidates our approach, ensuring a nuanced, thorough, and disciplined exploration of anticipatory design’s role within the vast expanse of future studies and strategic foresight. We undertake a comprehensive review of academic literature, industry reports, and academic papers concerning anticipation and the concept of anticipation in design, future studies, and strategic foresight. This provided the theoretical background and context for our grounded theory to bridge the gap of lack of studies under anticipatory experiences subject. Through a methodical dissection of anticipatory experiences, employed by the rigorous grounded theory approach, we bridge the gap to provide insights into the confluence of anticipatory design, future studies, and strategic foresight. By elucidating the nuances of explicit and implicit anticipation, from optimization, contingency, and novelty to innate human behaviors, our study aims to offer a comprehensive exploration of the anticipatory experiences in the realm of AI and ML-powered systems. We remain anchored to our core objective: to assemble actionable insights that enable designers to harmonize with future advancements, particularly the growing realm of AI and ML, optimizing user experiences, and paving the way for true innovation. This paper paves the way to ensure clarity, depth, and rigor in our pursuit of understanding anticipatory design within the expansive framework of future studies and strategic foresight.

4 Unraveling Anticipatory Experiences: Insights from Grounded Theory Analysis Designers can use explicit and implicit anticipation in different ways to explore new systems and needs. Explicit anticipation has three main use cases: optimization, contingency, and novelty, which are interdependent when imagining the future [34]. Differentiating between these categories, designers, and non-designers can shape the future through specific methods and approaches, creating sustainable and trustworthy anticipatory experiences. These three categories assist in the idea that “future goals and problems are perceptible and can be directly influenced” [34]. It’s important to distinguish these three ways of ‘using the future’ to provide a practical analytical framework for thinking about the future. The goal is to analyze how people are ‘using the future’ to make it easier to match tools to tasks. This will help establish an anticipatory system in four ways: • • • •

Accelerating the pace of change in a high level of uncertainty Engaging and facing wicked problems and existential threats Assuming shared responsibility for positive outcomes Externalized assumptions, more creative thinking, and informed decision-making

Optimization. It actively attempts to impose past patterns on the future. Means developing or improving existing systems or practices. It has the privileges of causal-predictive

46

J. Cerejo and M. Carvalhais

methods, often implemented through formal (usually algorithmic) models running historical data [34]. Optimization futures assume systemic continuity and are used to ‘colonize’ the future by assuming that specific assumptions will be estimated by extrapolation. In other words, we assume that existing trends will continue or that a current method will remain applicable. A designer may use optimization to actively attempt to improve patterns from the past to the future, creating predictive services that run on user historical data. Contingency. It is the capacity to recognize and forecast possible surprises in the future. Means readying the present for something that might happen. The end goal here is not to optimize but instead be prepared for a possible obstacle. Using novel futures to discover new ways of making sense of the emergent present provides one way of taking advantage of the unknowable as it becomes knowable, enhancing the capacity to discover the present [34]. Contingent futures can be imagined as the outcome of external forces, and they are used to prepare and foresee anticipated surprises, with the proviso that they cannot take unknowable novelty into account. A designer may use contingent futures to actively plan how the service will be prepared to recognize possible surprises (especially the negative ones). E.g., design and evaluate the reward function. Novelty. Novel futures include objects and processes, but they are unknowable in advance and present the greatest challenge for designers, requiring a capacity to explore openness. They can be used to discover new ways of making sense of the present. As far as explicit anticipation designed systems can only react in positive directions or negative evaluations. If positive it maintains its behavior pattern, but if negative behavior it must adapt to prevent anticipated negative results [26]. In sum, there is a general agreement that these three ways of using the future under explicit anticipation do not live in isolation; all are usually employed in different proportions [19, 26].

4.1 Importance of Anticipation in Shaping the Future The human cognitive mind has evolved through time and has the power to deal with the future, which opens the possibility of an entirely new set of capabilities and opportunities, and it can realize proactive and goal-oriented behavior [24]. Human cognition copes with the future through the ‘hypothesis of the apparent truth’, which represents the ability to detect or discriminate between foreseeable and unforeseeable events [28]. It guides the propensity of a human to make predictions with different degrees of confidence, ranging from complete uncertainty to firm certainty. Our anticipatory capabilities allow us to [24]: • Formulate novel goals and plans given future needs. • Formulate expectations at an increasingly high level of abstraction and use them to regulate our future actions. • Control our mental representations before or instead of acting in practice. • Heavily modify and adapt the environment to ourselves, not only vice-versa. We often act in the world to make it fit our representations of what we want, that is, our goals. • To imagine ourselves in the future and reason about possible futures.

Anticipation as a Tool for Designing the Future

47

Anticipation is a distinct human cognitive trait that has been difficult for technology to replicate. However, with advancements in AI technology, there is now an opportunity to develop anticipatory systems that mimic human cognition mechanisms. Anticipatory design can also aid in the evolution of AI by generating flexible predictions based on human needs. The capability of long-term forward model predictions and goal-oriented behavior needs to reach a symbolic level to accomplish the flexibility and determination observable in humans [24]. 4.2 Anticipation in Application to Future Studies and Strategic Foresight Future studies systematically explore possible and desirable futures to improve decisions, while foresight is the application of those methods and techniques to be prepared for the future. Foresight emerged in the 1950s as an approach to military planning and entered the business world in the 60s to deal with complexity and uncertainty. Foresight investigates, not predicts, the future, and can be used to design anticipatory experiences. Systems thinking is needed to understand the world, and future studies can anticipate an infinite number of possible futures. “The premise of foresight is that the future is still in the making and can be actively influenced or even created” [11]. The research field of studying the future and how to deal with it has evolved over time to include an increasing number of aspects (see Fig. 2).

Fig. 2. Scientific classification of research on future studies according to [29].

Strategic foresight is used for strategic planning and decision-making. It involves identifying emerging trends to create anticipatory strategies based on historical data patterns. AI and ML assist in anticipation by analyzing structured, semi-structured, and unstructured data to create foresight solutions. Forecast and foresight are related concepts but have different meanings. Forecast refers to a prediction or estimate of future events or conditions based on data and analysis of past and present trends. Forecasts are usually quantitative and aim to provide a specific prediction or projection of future outcomes. Foresight, on the other hand, refers to a process of thinking about and anticipating the future in a more strategic and holistic way. Foresight involves considering multiple potential scenarios and developing strategies to prepare for and respond to them. Foresight is usually qualitative and aims to explore possible future outcomes and prepare for uncertainty. Although similar, forecasts are more focused

48

J. Cerejo and M. Carvalhais

on specific predictions based on data, while foresight is a more strategic and holistic approach to anticipating and preparing for possible future scenarios (see Fig. 3).

Fig. 3. Difference between forecasting and foresight [33].

Foresight involves three-time dimensions: past, present, and future. The past is used to look forward to the future, while the present is a platform to identify opportunities and areas of competitiveness. Scenario planning, an essential tool for strategic foresight, involves developing systemic and plausible alternatives for the future to prepare for different outcomes. Aligning design thinking with foresight allows for the development of strategies to plan and create innovative and future-enhancing solutions from a human perspective. The problem with this method is that anticipation is left out of the future studies equation (see Fig. 4). To overcome this, Poli has proposed a third level to the future studies field – Anticipation [25–27].

Fig. 4. Visualization of the divisions of future studies without anticipation.

Anticipation as a Tool for Designing the Future

49

4.3 Three levels of Future Studies Poli [25–27] felt that future studies were missing anticipation properties because “anticipation (either explicit or implicit) is a way of generating the of necessity imaginary futures on the basis of probabilistic or non-probabilistic thinking in order to understand and act in the present” [27]. Therefore, the authors beliefs that anticipation within future studies is a process that not only depends on predictions, expectations, or beliefs about the future but is also highly dependent on the past and the present (see Fig. 5).

Fig. 5. Our view of the three levels of future studies according to Poli [25–27] theory.

Scientific analysis suggests that the future cannot be predicted with accuracy, considering the enormous number, complexity and dynamic interdependencies of parameters and variables that comprise the reality. To the extent that the future is unpredictable, only some aspects of the present reality or future alternatives can be envisioned. We are left with three actions: do nothing, prepare for the future, or shape the future actively. Forecast, foresight, and anticipation have unique properties when connected to future studies (Table 1) [20]. Table 1. The three levels of Future Studies [20]. Type

Nature

Theory

- Forecast

Previsive

Good

- Foresight

Non Previsive Poor

Complexity Orientation No

Past-oriented

Poor

Future-oriented

- Anticipation Non Previsive Presently under development High

Present-oriented

In summary, foresight applies future studies methods to prepare for the future, and designers need anticipatory thinking to align with design thinking. Futures studies are

50

J. Cerejo and M. Carvalhais

necessary for responsible decision-making, and by idealizing futures, designers can leverage AI anticipatory capabilities for better user experiences. The goal is to broaden understanding and avoid designing services that reinforce biases or fixed views of the world, with the aim of changing society toward positive behaviors.

5 Closing Insights and Prospects for Future Exploration Anticipatory experiences are a premise that explores the potential of AI technology to learn human behavior to predict choices and suggest actions or inputs on their behalf. Designing for anticipation enables the capability to predict or forecast peoples’ desires or intentions and influence their behaviors and learnings. We can predict future events, actions, and others to realize desirable future states or avoid dangerous futures [7]. We saw that anticipation works on many levels and sublevels that can manifest itself through explicit and implicit capabilities. Anticipation can be used as a tool for designing the future by exploring and imagining potential future scenarios, needs, and systems that may arise. Designers can use anticipatory thinking to identify potential challenges and opportunities and develop strategies to address them. By using explicit anticipation, designers can optimize existing systems, prepare for contingencies, and create new and novel experiences. They can also use implicit anticipation to tap into people’s needs and desires that they may not even be aware of yet. To use anticipation as a tool for designing the future, designers and non-designers can start by conducting research and analysis of trends, patterns, and emerging technologies like AI and ML. They can use this information to create different future scenarios and test how their designs might perform under different conditions. Designers can also use anticipatory thinking to create flexible and adaptable systems that can respond to changing conditions and user needs. They can use predictive models and ML algorithms to anticipate user behavior and deliver personalized experiences. Overall, anticipation can be a powerful tool for designers to create innovative and sustainable solutions that meet the needs of people and address the challenges of an ever-changing future. AI can play a significant role in shaping anticipatory experiences and designing for the future. AI can process vast amounts of data and identify patterns that humans may not be able to detect on their own. This can help designers and decisionmakers anticipate potential scenarios and make informed decisions based on data-driven insights. For example, in the field of product design, AI can be used to analyze user behavior and preferences, which can help designers anticipate and create products that better meet user needs and expectations. In the field of urban planning, AI can be used to analyze traffic patterns and anticipate potential bottlenecks, allowing planners to design more efficient transportation systems. However, it’s important to note that AI is only as good as the data it is trained on, and biases in the data can lead to biased outcomes. Therefore, it’s crucial for designers and decision-makers to be aware of these potential biases and work to mitigate them. Additionally, ethical considerations should always be considered when designing anticipatory experiences with the help of AI.

Anticipation as a Tool for Designing the Future

51

References 1. Abbott A (2005) Process and temporality in sociology: the idea of outcome in U.S. sociology. In: Steinmetz G (ed), Duke University Press 2. Adam B (1995) Timewatch: the social analysis of time, 1st edn. Polity Press 3. Appadurai A (2013) The future as cultural fact. Verso 4. Beckert, J. (2016) Imagined futures, fictional expectations and capitalist dynamics. Harvard University Press 5. Butz MV, Sigaud O, Gérard P (eds) (2003) Anticipatory behavior in adaptive learning systems: foundations, theories, and systems. Springer Berlin Heidelberg, Berlin, Heidelberg 6. Cerejo J, Carvalhais M (2019) Design after the rise of AI-driven services: learning from literature review. In: Digicom 2019 – 3Rd International Conference on Design and Digital Communication, November, pp 367–377 7. Cerejo J, Carvalhais M (2020) The lens of anticipatory design under AI-driven services. In: Digicom 2020 – 4th International Conference on Design and Digital Communication, pp 345–357 8. Engeler B (2017) Towards prospective design. Des J 20:S4591–S4599 9. Fantappiè L (1993) Principi di una teoria unitaria del mondo fisico e biologico 10. Garrett JJ (2011) Elements of user experience, the: user-centered design for the web and beyond. New Riders 11. GCPSE – Global Centre for Public Service Excellence (2018) Foresight Manual- Empowered Futures for the 2030 Agenda 12. Geden M et al (2019) Construction and validation of an anticipatory thinking assessment. Front Psychol 10(December):1–10 13. Glaser BG, Strauss AL (1967) The discovery of grounded theory: strategies for qualitative research. Routledge 14. Guyer J (2007) Prophecy and the near future: thoughts on macroeconomic, evangelical, and punctuated time. Am Ethnol 34(3):409–421 15. Hassenzahl M (2010) Experience design: technology for all the right reasons. Synth Lect Hum-Centered Inform 3(1):1–95 16. Kaya C, Öner A (2019) Anticipation: Meaning and Usage. Yeditepe Üniversitesi Yayınevi 17. Kleber S (2016) How to get anticipatory design right. https://www.hugeinc.com/articles/howto-get-anticipatory-design-right 18. Mead GH (1932) The philosophy of the present. Open Court 19. Miller R (2012) Anticipation: the discipline of uncertainty. Futures of Futures, pp 39–43 20. Miller R, Poli R, Rossel P (2013). The Discipline of Anticipation: Exploring Key Issues 21. Norman D (1986) The design of everyday things. Basic books 22. Pearl J (2009) Causality: models, reasoning, and inference. Econo Theory 19:675–685. https:// doi.org/10.1017/S0266466603004109 23. Pearl J, Mackenzie D. (2018) The book of why: the new science of cause and effect. Basic Books 24. Pezzulo G, Butz MV (2008) The challenge of anticipation: a unifying framework for the analysis and design of artificial cognitive systems. In: Pezzulo G, Butz M, Castelfranchi C, Falcone R. (eds.) Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics). Springer. https://doi.org/10.1108/ k.2010.06739eae.001 25. Poli R (2010) The many aspects of anticipation. Foresight 12(3):7–17 26. Poli R (2017) Introduction to Anticipation Studies. Spring International Publishing 27. Poli R (2019) Handbook of anticipation, theoretical and applied aspects of the use of future in decision making. Springer

52

J. Cerejo and M. Carvalhais

28. Riegler A (2001) The role of anticipation in cognition. Comput Anticipatory Syst. https://doi. org/10.1063/1.1388719 29. Rohrbeck R (2008) Towards a best-practice framework for strategic foresight: building theory from in multinational companies. In: LAMOT 2008, creating and managing a knowledge economy 30. Rosen R (1985) Anticipatory systems: philosophical, mathematical, and methodological foundations. In: IFSR international series on systems science and engineering ((2012) 2nd). Pergamon Press, Oxford 31. Seligman M, Railton P, Baumeister R, Sripada C (2013) Navigating into the future or driven by the past. Perspect Psychol Sci 8(2):119–141 32. Shapiro A (2015) The Next Big Thing in Design? Less Choice. https://www.fastcompany. com/3045039/the-next-big-thing-in-design-fewer-choices 33. Stafie D (n.d.). Future Station: Spot the early signs of future change. https://futurestation.ro/ future-preparedness-through-foresight/. Retrieved 21 Apr 2023 34. UNESCO (2018) Transforming the future: anticipation in the 21st century. In: Miller R, Gugerli D (eds.) UNESCO Co-publication 35. Walorska AM (2015) Turning data into experiences. Pro-active experiences and their significance for customers and business. Procedia Manuf 3:3406–3411 36. Zamenopoulos T, Alexiou K (2020) Collective design anticipation. Futures 120:102563. https://doi.org/10.1016/j.futures.2020.102563

Designing Audio-Based Multimodal Interfaces for English Teaching: A Conceptual Model Based on an Integrative Literature Review Jéssica Rodrigues Esteves(B)

and Berenice Santos Gonçalves

Design Department, Federal University of Santa Catarina, Florianópolis, SC, Brazil [email protected], [email protected]

Abstract. The rapid evolution of technology has led to significant advancements in the digital interfaces area. In particular, multimodal interfaces have gained popularity as they provide opportunities for incorporating multiple senses in interaction design, enhancing the user experience beyond mere visual engagement. Considering human sensory modalities, auditory mode is processed more quickly than the visual mode because humans have evolved to be particularly sensitive to sound. Extensive research in the English language learning field has established that the development of auditory and oral skills is pivotal to attaining fluency in a foreign language. It has led to the creation of new digital resources and interfaces. In this scenario, audio has emerged as a valuable resource for designing effective multimodal interfaces for English language teaching. The present study aims to identify categories for audio-based multimodal interfaces for English teaching. In order to achieve this objective, we conducted an integrative review of the literature, employing the analysis and synthesis of theoretical findings from traditional and systematic literature reviews. The study identifies two primary aspects, namely multimodal interfaces and language teaching, which are presented in a conceptual model, organized into five categories: (I) multimedia learning, (II) audio typologies, (III) emerging technologies, (IV) teaching strategies, (V) listening strategies. The model results provides guidance for developing effective multimodal interfaces for English learning. Keywords: Multimodal interfaces · Audio · English learning

1 Introduction Technological development has led to the emergence of different interface typologies that enable the use of immersive techniques, such as augmented and virtual reality, tangible elements, wearable computing, etc. (Preece, Rogers & Sharp, 2019). The expansion of interaction modes has allowed the design of digital interfaces to enhance the user experience. As multimodal interfaces become more focused on human modalities, challenges arise for their development, such as the difficulty of using interaction modes in different usage contexts (Murthy, 2020). Among the human modalities, the auditory mode is processed more quickly by the brain compared to the visual mode (Saffer, 2013), © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 53–66, 2024. https://doi.org/10.1007/978-3-031-47281-7_5

54

J. R. Esteves and B. S. Gonçalves

due to humans’ advanced ability to perceive sound patterns (Kortum, 2008). Multimodal interfaces use sound as a channel to inform system actions, such as error and success messages, notifications, and voice commands for GPS (Preece, Rogers & Sharp, 2019; Quaresma, Motta & Araújo, 2019), making audio a valuable resource for designing multimodal interfaces for teaching. In the educational context, language teaching involves stimulating four different language skills that together enable students to acquire fluency: listening, speaking, reading, and writing. Brown and Lee (2015) note that language skills are inseparable and mutually related: listening implies speaking, and speaking implies listening; just as writing and reading are related to listening and speaking. However, writing was the predominant skill in language learning until the end of the 19th century (Brown & Lee, 2015; Renukadevi, 2014). Nevertheless, listening accounts for 45% of a student’s language ability, speaking for 30%, reading for 15%, and writing for 10% (Renukadevi, 2014). Therefore, hearing is crucial for achieving fluency in a new language. Brown & Lee (2015) state that the ear receives sound waves and transmits nerve impulses to the brain for listening comprehension, which is a psychomotor process. Magal-Royo & Laborda (2020) suggest that students need to recognize sounds and consider context to reinterpret what is happening in their minds and understand the speaker’s message. Thus, the listening comprehension process can be challenging and complex. Listening is a crucial aspect of the multimodal process, in which all senses participate and can combine with various modalities. Teachers often utilize visual and auditory modalities to stimulate their students’ four communicative skills in language teaching. Therefore, audio is a fundamental resource for students to develop a comprehensive understanding of a foreign language. Recent studies address the potential of audio for English language teaching (Kao, Liao, & Lan, 2017; Magal-Royo, Laborda, & Price, 2017; Chang et al., 2017; Fan, Jin, & Antle, 2018; Murad et al., 2018; Li, Chang, & Wu, 2020; Magal-Royo & Laborda, 2020; Divekar et al., 2021), contributing to the discussion about the relationship between multimodal interfaces and English language education. However, no studies have been identified that present models, guidelines, or frameworks for the creation of multimodal interfaces based on the specificity of audio for language teaching. In this context, designers can enhance language teaching experiences by creating multimodal interfaces focusing on audio. These interfaces can incorporate different sensory modes. Therefore, the present article aims to identify categories for audio-based multimodal interfaces for English teaching. To achieve this, an integrative review of the literature was conducted, including systematic and traditional bibliographic reviews.

2 Design and Method We carried out an integrative literature review to identify categories for audio usage in multimodal interfaces for teaching English. The review analyzed relevant research to synthesize the current state of knowledge on this topic (Lopes & Fracolli, 2008). The research was divided into two parts: a systematic literature review was conducted first, and then the results were combined with a bibliographic review, resulting in an integrative review. The findings of the research were organized into a conceptual model of audio-based multimodal interfaces for English teaching.

Designing Audio-Based Multimodal Interfaces for English Teaching

55

We followed six steps (Lopes & Fracolli, 2008) to conduct the systematic literature review: (I) define the research question; (II) establish criteria for article selection; (III) identify and select articles; (IV) analyze and interpret information; and (V) present the review. In step I, we defined the research problem and questions. In step II, we established inclusion and exclusion criteria for articles and conducted searches based on these criteria. In step III, we selected articles based on the titles and abstracts of publications that met the research criteria. In step IV, we analyzed the full publications for information. Finally, in step V, we presented the results of the integrative review.

3 Work Field and Data Analysis We conducted a systematic literature review with an exploratory approach to identify and analyze current research on multimodal interfaces focused on audio in English teaching. Our study was based on three primary keywords: multimodal interfaces, audio, and English teaching. The final search string was composed of: Table 1. Keywords and Search String. Keywords

Search string

Multimodal interfaces

“Multimodal Interfac*” OR “Multimodal Interact*” OR “Multimodalit*” OR “Tangible Interact*” OR “Interaction Design” OR “User Interface Design” OR “Computer-Assisted Instruct*” OR “Computer Assisted Instruct*” OR “Computerized Self-Instruction Program” OR “Computerized Programmed Instruct*” OR “ “User Computer Interfac*” OR “User Computer Interfac*”

Audio

“Audio” OR “Sound” OR “Speech” OR “Voice” OR “Speaking” OR “Listening”

English teaching

“English teaching” OR “Language learning” OR “Second Languag*” OR “Foreign Languag*”

Searches were conducted on February 8, 2022, in the Scopus, The ACM Guide to Computing Literature, and Web of Science databases due to their relevance to the subject of this research, which spans the areas of Design, Technology, and Education. All databases were accessed through the Periodicals Portal subscribed by the Federal University of Santa Catarina to ensure comprehensive coverage of relevant literature. Our goal was to identify studies that take an interdisciplinary approach to the subject. The inclusion criteria adopted was: (I) studies published in the last 5 years; (II) quantitative and qualitative primary studies; (III) studies focusing on multimodal interfaces or recurrent terms (multimodal interaction, tangible interaction, interaction design); (IV) studies carried out focusing on audio or sound. We searched for studies published from 2017 to 2022 in the Scopus, The ACM Guide to Computing Literature, and Web of Science databases, applying the search string to title, abstract, and keywords. To facilitate the selection process, we exported

56

J. R. Esteves and B. S. Gonçalves

the references found in the databases in.BibTEX and.nBib formats and imported them into the Mendeley reference manager. We followed the steps outlined in Table 2 to select the works that contributed to answering the research question: “What research has already been or is being carried out on multimodal interfaces with a focus on audio for English teaching?”. Based on their contribution level, we finally selected eight articles. Table 2. Studies selection criteria. Filters

Results

Search results using the string

207

Exclusion of doubly results

177

Filter #1: Reading of the title, abstract and keywords

36

Filter #2: Reading of the full papers

8

After reviewing the titles and abstracts of thirty-six articles, we selected eight papers for further analysis to determine their relevance to the scope of this work. In Table 3, we have emphasized articles pertinent to the research topic by presenting them in bold. Although the others papers are related to the theme, they did not provide any categories, guidelines, or recommendations for audio usage in multimodal interfaces for English teaching. Table 3. Bibliographic portfolio of the systematic review. Authors

Title

Year

1

Kao, Liao & Lan

Listening strategy applications by learners 2017 under the context of multimodality

2

Magal-Royo, Laborda & Price

A New m-Learning Scenario for a Listening Comprehension Assessment Test in Second Language Acquisition [SLA]

2017

3

Chang et. al

Electronic feedback on second language writing: A retrospective and prospective essay on multimodality

2018

4

Fan, Jin & Antle

Designing Colours and Materials in Tangible 2018 Reading Products for Foreign Language Learners of English (continued)

Designing Audio-Based Multimodal Interfaces for English Teaching

57

Table 3. (continued) Authors

Title

Year

5

Murad et. al

SLIONS: A Karaoke Application to Enhance Foreign Language Learning

2018

6

Li, Chang & Wu

Developing a Task-Based Dialogue System for English Language Learning

2020

7

Magal-Royo & Laborda

Binaural sound to enhance listening comprehension in language tests

2020

8

Divekar et. al

Conversational Agents in Language Education: Where They Fit and Their Research Challenges

2021

Due to the low number of articles identified in the systematic literature review, it was necessary to perform an integrative literature review in order to gain a broad and comprehensive understanding of the theme.

4 Results In this section, we present an integrative review that combines the results from the systematic and the bibliographic review. 4.1 Multimodal Interfaces and Auditive Mode Multimodal interfaces process two or more user input modalities, such as speech, touch, gesture, and gaze, coordinated with the output of the multimedia system (Oviatt et al., 2017). Multimodal interfaces represent a paradigm shift, differentiating them from conventional graphical user interfaces (Furht, 2008). By using multiple modalities, such as touch, vision, sound, and speech, multimodal interfaces enrich the user experience and provide more flexible, efficient, and varied ways of human-computer interaction (Oviatt et al., 2017). In digital interfaces, the use of audio can help minimize visual information overload. Laufer and Schwabe (2009) reported that the excessive use of visual elements in interfaces could overload the visual modality, limiting the amount of information that the user could absorb. They argue that integrating different sensory modes in multimodal interfaces is necessary to address this issue. Serafin et al. (2022) presented a brief taxonomy of auditory display techniques, as shown in Table 3. The authors emphasize that these techniques overlap, as sonification can be performed through sound and auditory icons and implemented through multimodal interfaces.

58

J. R. Esteves and B. S. Gonçalves Table 4. Brief taxonomy of auditory display techniques. Technique

Description

I

Sonification

Sound used to convey information

II

Sound icons

Musical sounds used in interfaces

III

Auditory icons

Everyday sounds used in interfaces

IV

Sound interaction design

Sound used in an interactive context

V

Multimodal interfaces

Interfaces that have more than one sensory modality involved

VI

Spatial hearing monitors

Spatial hearing monitors sounds transmitted in a 3D space

Note. Based on Serafin et al. (2022)

The term sonification was coined by Buxton (1989 apud Serafin et al. 2022) and refers to the use of sound to represent data, making it the auditory counterpart of data visualization. Sound icons are non-verbal audio messages used in interfaces to inform the user about a virtual object (Blattner, Sumikawa & Greenberg, 1989). Auditory icons are everyday sounds used to convey information about events in graphical interfaces (Serafin et al, 2022). Serafin et al. (2022) also highlight sound interaction design, which explores sound as a primary channel for transmitting information, meaning, and aesthetic and emotional qualities in interactive contexts. Advancements in technology have made it possible to design systems with multimodal properties similar to those of the physical world (Franinovic & Serafin, 2013). Serafin et al. (2022) note that we live in a multisensory world and often rely on sensory substitution when dealing with a lack of sensory modalities. Sound can be used to augment, replace or complement other sensory modalities. The last technique presented by Serafin et al. (2022) concerns spatial auditory displays, where sound is emitted in 360 degrees, making it essential to understand how humans perceive sounds coming from different directions and whether they can be simulated in a human-computer interface. Given the importance of audio in improving user interaction, which is often neglected in digital interface design, Magal-Royo & Laborda (2020) describe how binaural audio can improve perception in a foreign language listening comprehension test. They present a visual model of a binaural listening comprehension test scenario and conclude that binaural sound usage is valuable for enhancing the multimodal experience during foreign language learning and testing. Chang et al. (2017) found that audio feedback can serve as an effective form of immediate correction for students. The study also revealed the potential of audio and video feedback for written language corrections, with audiovisual feedback allowing for closer communication between participants through the use of intonation and facial expressions. In a separate study, Murad et al. (2018) outlined design considerations for building a karaoke application, including multimodal instruction, voice recognition, scoring, and feedback presentation. Their results indicate that integrating computerassisted language learning (CALL) and automatic speech recognition (ASR) with singing can help improve pronunciation. Dialogue agents are also being explored for language learning in multimodal interfaces.

Designing Audio-Based Multimodal Interfaces for English Teaching

59

Divekar et al. (2021) offer intersectional perspectives on the use of audio in English teaching, including language teaching, learning, and assessment, virtual environments with embedded agents, natural collaborative dialogue, sustainable engagement, and accessibility. The authors note that while each perspective has been studied independently, there is a lack of research that integrates all of these factors. 4.2 Principles for Multimedia Learning Recent advancements in computer graphics have inspired innovative approaches to using multimedia instruction for promoting human learning. According to Mayer (2020), a multimedia resource is not just a tool to convey information, but rather a material that combines spoken or written text with visual information, such as graphics, photos, animations, and maps. Mayer emphasizes the significance of developing multimedia resources that can enhance the content, thereby advocating for the production of multimedia resources that are student-centered and adaptable to the learners’ needs for effective learning. According to Mayer’s cognitive theory of multimedia learning (2020), the learner engages in three critical cognitive processes: selection, organization, and integration. In the selection process, the learner attends to relevant words and images from a multimedia resource to create a base of words and images. In organization, the learner builds internal connections between associated words to create a visual and pictorial model. Finally, integration occurs when the learner establishes external connections between verbal and pictorial models to integrate the various components of the multimedia resource. To optimize multimedia instruction, Mayer (2021) suggests differentiating between three types of cognitive processing: irrelevant, essential, and generative. The first type, irrelevant processing, does not support the instructional objective and can lead to confusion, such as the use of unsynchronized captions in a video. The second type, essential processing, is aimed at mentally representing the visual material and requires careful management to avoid cognitive overload. In contrast, generative processing involves deep cognitive processing to integrate and organize material in new ways, facilitating the application of knowledge. Mayer’s (2021) twelve principles for multimedia learning are organized into three categories: (I) principles for reducing extraneous processing, (II) principles for managing essential processing, and (III) principles for fostering generative processing, as shown in Table 4. These principles provide guidance for designing multimedia resources that optimize the use of working memory and facilitate effective learning. Mayer (2021) emphasizes the importance of creating interactive simulations and virtual environments to enhance the instructional effectiveness of virtual learning experiences, including virtual reality. Thus, Mayer’s principles (2020) are relevant for designing multimedia learning environments that cater to individual learning needs. In the context of language teaching, multimedia resources can enhance learning, particularly for practicing oral communication skills (Paiva, 2018; Beviláqua, VetromilleCastro & Leffa, 2021; Gomes Junior, Silva & Paiva, 2022). The use of audio in the multimodal interface can stimulate these skills (Magal-Royo, Laborda & Price, 2017; Murad et al., 2018; Magal-Royo & Laborda, 2020). In the following section, we will discuss this topic and its implications for English language learning skills.

60

J. R. Esteves and B. S. Gonçalves Table 5. Principles for multimedia learning. Category

Principles

1.

Principles for reducing extraneous processing

Coherence, signaling, redundancy, spatial contiguity, temporal contiguity

2.

Principles for managing essential processing

Segmenting, Pre-training, Modality

3.

Principles for fostering generative processing

Multimedia, personalization, voice, image

Note. Adapted from Mayer (2021)

4.3 English Teaching Skills The learning of the English language involves four linguistic skills, namely: listening, speaking, reading, and writing. These skills are closely related to each other: listening often involves speaking, and in turn, speaking almost always involves listening; similarly, writing and reading are interconnected with listening and speaking (Brown & Lee, 2015). For this reason, many foreign language classes have the following integration steps: (I) initial discussion on a topic, followed by a reading or writing task; (II) listening to a lecture or monologue, taking notes, and then having a discussion; and finally, (III) writing a text (Brown & Lee, 2015). Hinkel (2006) emphasizes that in the current globalized context, the pragmatic objects of language learning must increasingly value integrated and dynamic instructional models of multiple skills focused on meaningful communication, as well as on the development of students’ communicative competence. The significance of listening skills in language learning and teaching has been elevated in scholarly through the research of Asher (1981) on the Total Physical Response theory. In this theory, the role of comprehension is highlighted, as students receive large amounts of sounds to listen to before being encouraged to respond orally. Nevertheless, listening skills were previously overlooked by researchers in the field due to their passive nature. Renukadevi (2014) contends that writing skills were the dominant focus of language learning until the close of the 19th century. Nonetheless, the author argues that a learner derives 45% of their linguistic competence from auditory skills, 30% from speaking, 15% from reading, and 10% from writing (Renukadevi, 2014). Thus, auditory skills are crucial for attaining fluency in a novel language. Brown and Lee (2015) emphasize that the importance of practicing listening skills in language learning should not be underestimated, as it is through listening that one can receive and internalize linguistic information and structures. According to the authors, the first step in auditory comprehension is the psychomotor process of receiving sound waves through the ear and transmitting nerve impulses to the brain. In the acquisition of a foreign language, listening skills are an important component in the learning process. According to Magal-Royo & Laborda (2020), listening is a difficult mental process as it requires students to recognize sounds, associating them with prior knowledge through the understanding of vocabulary and grammatical patterns. It is also necessary to consider the context so that they can reinterpret what is in the situation in

Designing Audio-Based Multimodal Interfaces for English Teaching

61

their minds and, thus, understand the message the speaker intends to convey. Therefore, the process of auditory comprehension can be considered difficult and complex. For teaching listening comprehension, Brown & Lee (2015) report that, in addition to English conversation, other forms of spoken language are also important for language learning. Nunan (1991) presents some categories, such as monologues and dialogues. In language courses, listening and speaking skills are interconnected, known as oral communication skills (Brown & Lee, 2015). Conversation, for example, is one of the performance forms in which listening and speaking are closely linked, being the most popular discourse category in language teaching. Other discourse categories, such as speeches and lectures, are also preceded or followed by various forms of oral production. The objectives and techniques for teaching conversation vary, depending on the student, teacher, and class context. Historically, conversation classes range from communicative exercises to free discussions (Brown & Lee, 2015). According to Brown & Lee (2015), the greatest difficulty encountered by learners in their attempts to speak another language is not the multiplicity of sounds, words, phrases, and discursive modes that characterize any language, but rather the interactive nature of most modes of communication. Conversation is a collaborative mode in which participants engage in a process of negotiating meaning; therefore, the learner has difficulties regarding how to say things, when to speak, and other discourse restrictions. To stimulate oral skills, Brown & Lee (2015) suggest the use of different activities to practice speaking, presented in Table 5. Table 6. Types of classroom speaking performance. Types

Goal

1.

Imitation exercises

Practice intonation

2.

Intensive speaking

Practice grammatical and phonological aspects of language

3.

Responsive speaking

Practice quick and short responses, questions or comments

4.

Transactional dialogue

Practice transmitting or exchanging specific information

5.

Interpersonal dialogue

Practice colloquial conversations

6.

Extensive monologue

Individual practice by advanced level students that can take the form of oral reports, summaries or perhaps speeches

Note. Adapted of Brown e Lee (2015)

Regarding teaching, Brown & Lee (2015) address fundamental principles for teaching oral skills. In general, it is identified that the use of these principles implies, first and foremost, in defining the learning context, and from there it is possible to determine the best techniques to make the student exercise oral skills. Kao, Liao, & Lan (2017) analyzed the most commonly listening strategies used by students in two different environment: a virtual and a physical. In this way, they

62

J. R. Esteves and B. S. Gonçalves

investigated if there were significant differences in listening strategies between students who used the game Second Life and those who used printed images as support for listening. The research results showed that, regardless of the auditory support offered, participants used more metacognitive strategies. Finally, Kao, Liao & Lan (2017) suggest that researchers and educators extend their research to the use of 3D environments for language learning, especially for auditory skills. 4.4 Studies Summary Through analyzing the studies in the integrative review, we were able to synthesize key concepts and identify categories for the audio usage in multimodal interfaces for teaching English. Our findings and their contributions are presented in Table 6, where the origin column specifies the type of review the reference was found, using the acronyms “BR” for bibliographic review and “SR” for systematic review. Also, in the “aspect” column, we identify the main aspect and the sub-aspect identified in the literature review (Table 7). Based on Table 6, we identified categories and organized them into a conceptual model. According to Frascara (2018), a conceptual model helps users to recognize system problems by grouping categories and relationships of importance among factors. Conceptual models provide a concept’s structure and interrelationships (Preece, Rogers & Sharp, 2019). In this way, our conceptual model aims to help designers and other professionals involved in the interface design field to obtain recommendations on how to design multimodal interfaces with audio integration for English teaching. We created the conceptual model based on two main aspects: multimodal interfaces and English teaching (Fig. 1). We also organized all recommendations identified in the integrative review into five categories. These categories were grouped by similarity, gathering different authors, as identified in Table 6. The first aspect, multimodal interfaces, encompasses topics that address issues related to the interaction ways in digital interfaces, focusing on the auditory mode. In this category, we included these aspects: multimedia learning (Mayer, 2021; 2021), audio typologies (Serafin et al., 2022), and emerging technologies (Magal-Royo & Laborda, 2020; Chang et al., 2017; Murad et al., 2018; Divekar et al., 2021; Li, Chang & Wu, 2020). In the second aspect, English teaching, we included topics identified in the integrative literature that had recommendations to help students develop their oral communicative skills. The categories that fall under this axis are strategies for teaching oral communication (Nunan, 1991; Brown & Lee, 2015) and listening strategies (Kao, Lian & Lan, 2017). Finally, we highlighted that the conceptual model aims to present a categorization; therefore, it is not a prescriptive model, as each interface project has its particularities. The model results aims to be a support guide for designers, researchers and multidisciplinary teams to design effective audio-based multimodal interfaces for English teaching.

Designing Audio-Based Multimodal Interfaces for English Teaching

63

Table 7. Studies summary of integrative review. Authors

Source

Theoretical contributions

Aspects

Serafin et al. (2022)

BR

We identified four audio typologies relevant to this study: (I) sound icons, (II) auditory icons, (IV) sound interaction design, and (V) multimodal interfaces

Main aspect: Multimodal interfaces Sub-aspect: Audio typologies

Nunan (1991) Brown & Lee (2015)

BR

For teaching listening comprehension, other forms of spoken language are also important for language learning, such as monologues and dialogues

Main aspect: English teaching Sub-aspect: Listening strategies

Brown & Lee (2015)

BR

To encourage oral proficiency, authors suggest different activities to exercise speaking skills, such as imitation exercises, intensive speaking, responsive speaking, transactional dialogue, interpersonal dialogue, and extensive monologue

Main aspect: English teaching Sub-aspect: Listening strategies

Mayer (2020; 2021)

BR

We have identified principles of multimedia learning that emphasize audio: redundancy, segmentation, modality, personalization, and voice

Main aspect: Multimodal interfaces Sub-aspect: Multimedia learning

Kao, Lian & Lan (2017)

SR

Authors classified listening strategies into metacognitive, cognitive, and socio-affective

Main aspect: English teaching Sub-aspect: Listening strategies

Magal-Royo & Laborda (2020)

SR

Recent research highlights multimodal audio, also known as 3D or binaural audio, as a promising way to enhance students’ experience on platforms focused on listening comprehension tests

Main aspect: Multimodal interfaces Sub-aspect: Audio typologies

(continued)

64

J. R. Esteves and B. S. Gonçalves Table 7. (continued)

Authors

Source

Theoretical contributions

Aspects

Chang et al. (2017); Murad et al. (2018)

SR

Both studies reported that students’ learning experience improves when using audio as feedback

Main aspect: Multimodal interfaces Sub-aspect: Emerging technologies

Divekar et al. (2021); Li, Chang & Wu (2020)

SR

These papers use machine learning to create virtual agents that assist students in their activities

Main aspect: Multimodal interfaces Sub-aspect: Emerging technologies

Fig. 1. A conceptual model for audio-based multimodal interfaces for English teaching.

5 Conclusion Digital technologies have the potential to bring about significant social and educational change. In the realm of education, multimodal interfaces and audio can be used to enhance language teaching by empowering learners to construct their knowledge instead of merely receiving predetermined knowledge. The purpose of this study is to address the theoretical gap in the design of multimodal interfaces that focus on audio typologies. To achieve this goal, we conducted an integrative literature review and proposed a conceptual model for designing effective audio-based multimodal interfaces for English teaching. The model comprises two main aspects: multimodal interfaces and English teaching, each consisting of several categories. The multimodal interfaces aspect focuses on interaction modes in digital interfaces, with an emphasis on the auditory mode. The

Designing Audio-Based Multimodal Interfaces for English Teaching

65

English teaching aspect provides recommendations for enhancing students’ oral communication skills. In this way, the research main contributions lie in its conceptual framework that integrates multimodal interface design with language teaching, its practical recommendations for enhancing language learning through technology, and the potential for future research and technological advancements in education. The conceptual model aims to assist designers and researchers in comprehending all of the categories involved in an audio-based multimodal interface for English learning. To test the model’s feasibility, we will utilize two research methods: interviews and focus groups. First, we will interview professionals and researchers in multimodal interfaces to comprehend the complexities involved in designing audio-based multimodal interfaces. Based on the outcomes, we will refine the model. Next, we will conduct focus groups with expert designers to assess the model’s writing, visual elements, clarity, coherence, consistency, and feasibility. Following this, we will restructure the model based on the findings. Acknowledgments. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 001.

References Asher JJ (1981) The total physical response: Theory and practice. Ann New York Acad Sci 379(1):324–331 Beviláqua AF, Vetromille-Castro R, Leffa VJ (2021) Eu não sou trapaceiro (a): a produção de REA para Letramentos Críticos e Competência Simbólica. Rev Bras de Linguíst Apl 21:929–954 Blattner MM, Sumikawa DA, Greenberg RM (1989) Earcons and icons: their structure and common design principles. Human-Comput Interact 4(1):11–44 Brown HD, Lee H (2015) Teaching principles. P. Ed Australia Chang C, Kelly JC, Satar HM, Strobl C (2017) Electronic feedback on second language writing: a retrospective and prospective essay on multimodality. Writ. Pedagogy 9(3):405–428 Divekar RR, Lepp H, Chopade P, Albin A, Brenner D, Ramanarayanan V (2021) Conversational agents in language education: where they fit and their research challenges. In: Stephanidis C, Antona M, Ntoa S (eds) HCII 2021, vol 1499. CCIS. Springer, Cham, pp 272–279. https://doi. org/10.1007/978-3-030-90179-0_35 Franinovi´c K, Serafin S (eds) (2013) Sonic interaction design. The MIT Press Frascara J (2018). Enseñando diseño: usuarios, contextos, objetivos y métodos de investigación. Ediciones Infinito Furht (ed) (2008) Encyclopedia of multimedia. Springer US, Boston, MA. https://doi.org/10.1007/ 978-0-387-78414-4 Gomes Junior RC, Silva LDO, Paiva VLMDO (2022) Tecnologias digitais para aprender e ensinar inglês no Brasil. Texto livre 15:e38008 Hinkel E (2006) Current perspectives on teaching the four skills. TESOL Q 40(1):109–131 Kao CL, Liao CY, Lan YJ (2017) Listening strategy applications by learners under the context of multimodality. In: 2017 IEEE 17th international conference on advanced learning technologies (ICALT). IEEE, pp 354–356 Kortum P (2008) HCI beyond the GUI: design for haptic, speech, olfactory, and other nontraditional interfaces. Elsevier

66

J. R. Esteves and B. S. Gonçalves

Laufer C, Schwabe D (2009) Toc-Toc, Tic-Tac, Triiiimm! Utilização de Som em Interfaces Multimodais. In: XXXVI Seminário Integrado de Software e Hardware Li KC, Chang M, Wu KH (2020) Developing a task-based dialogue system for English language learning. Educ Sci 10(11):306 Lopes ALM, Fracolli LA (2008) Revisão sistemática de literatura e metassíntese qualitativa: considerações sobre sua aplicação na pesquisa em enfermagem. Texto & Contexto-Enfermagem 17:771–778 Magal-Royo T, García Laborda J (2020) Binaural sound to enhance listening comprehension in language tests. Int J Interact Mob Technol 14(1):4–14 Magal-Royo T, Garcia Laborda J, Price S (2017) A new m-learning scenario for a listening comprehension assessment test in second language acquisition [SLA]. JUCS-J Univ Comput Sci 23(12):1200–1214 Mayer RE (2020) Multimedia learning. In: Psychology of learning and motivation, vol 41. Academic Press, pp 85–139 Mayer RE (2021) E-learning: new opportunities and a view of the future. In: The Oxford handbook of lifelong learning. Oxford University Press, p 263 Murad D, Wang R, Turnbull D, Wang Y (2018) SLIONS: a karaoke application to enhance foreign language learning. In: Proceedings of the 26th ACM international conference on Multimedia, pp 1679–1687 Murthy LRD (2020) Multimodal interaction for real and virtual environments. In: International conference on intelligent user interfaces, proceedings IUI. Association for Computing Machinery, pp 29–30 Nunan D (1991) Language teaching methodology: a textbook for teachers. Prentice-Hall, New York Oviatt S, Schuller B, Cohen P, Sonntag D, Potamianos G (2017). The handbook of multimodalmultisensor interfaces, volume 1: foundations, user modeling, and common modality combinations. Morgan & Claypool de Oliveira Paiva VLM (2018) Tecnologias digitais para o desenvolvimento de habilidades orais em inglês. DELTA: Documentação de Estudos em Lingüística Teórica e Aplicada 34(4):1319–1351 Preece J, Rogers Y, Sharp H (2019) Interaction design. Beyond human computer interaction. Wiley and Sons Quaresma M, Motta I, Araújo M (2019) Levantamento de Diretrizes de Interação Sonora para Aplicativos de Smartphones. In: Anais do 13º Congresso Pesquisa e Desenvolvimento em Design. Blucher, pp 5100–5112 Renukadevi D (2014) The role of listening in language acquisition; the challenges & strategies in teaching listening. Int J Educ Inform Stud 4(1):59–63 Saffer D (2013. Microinteractions: designing with details. O’Reilly Media, Inc. Serafin S, Buxton B, Gaver B, Bly S (2022) Auditory Interfaces. Focal Press, London

From the Algorithm to the New Art Collector. Design, Development and Launch of an Innovative NFT Collection Dario Lanza(B) Complutense University of Madrid, Urban, Spain [email protected]

Abstract. Blockchain technology is opening up new and interesting spaces for creative sectors, redefining the relationship between the artist and the public, enabling the collecting of digital artworks and the incorporation of digital artists into the art market. New artistic formats such as PFPs, 1:1 Art and Generative Art make up what we generically term NFT Art, a wide range of new offerings that is undergoing increasing interest, both for creatives and for collectors. This paper aims to aid understanding of these new paradigms, beginning with a description of the concepts in question and then developing a practical applied generative art project which will enable us to offer a first-person look at all the stages, from conception to public launch. In this way we can offer here a guide we hope will be useful to future artists and designers who wish to approach these new and fascinating art spaces. Keywords: Digital Art · Generative Art · Art Market · Non-Fungible Token · NFT Art · Algorithmic Art · Ethereum Blockchain

1 Introduction We are currently witnessing the development and consolidation of new technological and social paradigms and concepts, such as the notions of the metaverse, blockchain, NFTs or non-fungible tokens, which are profoundly changing the way we interpret many aspects of contemporary culture. Creative activities in general, especially art, are not exempt from these new paradigms, and today they are seeing the unprecedented rise of new and exciting forms of art. With a view to offering an in-depth understanding of said phenomena, we would like to propose an exploration that is as exhaustive as possible and goes from a description of the concepts that govern these new blockchain environments, to a first-person look at the new market for NFT art. To do this, we need to begin our itinerary by asking: what is blockchain? If we look at Wu & Li (2021)’s definition of the term, blockchain is “a distributed and decentralized data structure that allows users to conduct secure and verifiable transactions without the need for intermediaries” [1]: in other words, a decentralized network of interconnected computers that share and manage information between themselves. These description may initially seem identical to one of the Internet itself, but the core difference © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 67–78, 2024. https://doi.org/10.1007/978-3-031-47281-7_6

68

D. Lanza

with this new paradigm resides in the word “decentralized”. The difference between blockchain and the computer networks that make up the Internet is that the latter are centralized around servers which host the information, the “intermediaries” to which Wu & Li refer, constituting structures much more vulnerable to manipulation or even the potential for disconnection of said servers, which would lead to the total loss of the information contained in them. On the contrary, when the information is replicated on thousands of computers across the world which are constantly collating it and checking its veracity, a process normally called data mining, the information is better protected from external manipulation or accidental loss. Looking at the design of these structures and their technological capacities, we find different types of blockchains, being the Bitcoin blockchain the first and best-known of them. The Bitcoin blockchain is usually defined as an open, public, and permissionless network of computers that maintains a decentralized ledger of transactions, whose activity is focused on the computers calculating countable adjustments related to their own token, which is the bitcoin. Such is the trustworthiness, transparency and security of this network that over the past decade bitcoin has acquired solid monetary value as many users find it more reliable than traditional currencies associated with countries and managed by banking entities, which are also susceptible to fluctuations. It is possible that in the near future we will see a significant portion of the economy move from traditional financial structures to decentralized structures like this. The success, solidity and possibilities offered by the first blockchains such as Bitcoin has motivated the development of a second blockchain generation which is more technically sophisticated. The main blockchain of this second generation and the one with which this study is concerned is Ethereum, which in addition to processing its own currency, the eth, allows applications and smart contracts [2] to be executed. These two features have made this blockchain the perfect network for the creative sector by allowing the storage of audiovisual content like illustrations, art, design, photography or music, enabling algorithms to be executed as per the requirements of generative art, and letting purchase contracts for artworks be managed automatically and with transparency along with the management of royalties between creators and collectors. In the sense of token processing, these new networks can mine and manage both fungible and non-fungible tokens, the latter being of most interest to the creative industries. But what are fungible and non-fungible tokens? A fungible token is an asset all of whose units have the same value, following the “equivalence principle” [3], as defined by Casey & Wong (2018), being completely interchangeable with each other. A one-dollar coin is a fungible asset as it has the same value as any other one-dollar coin and can be interchanged with it without affecting the value of a transaction. A photo in JPG format is also fungible since it is identical to replicas produced of it. By contrast, a non-fungible token, currently abbreviated to NFT, is any type of asset whose nature is unique and not interchangeable with any other [4]. An oil painting on canvas by an artist is a nonfungible token, as it has qualities that make it unique and it is not interchangeable with anything else. For decades, digital creators and artists had very limited commercial possibilities for their work due precisely to the fungible nature of digital files by which any digital creation or design can be copied into identical replicas with no notion of which is the

From the Algorithm to the New Art Collector

69

“original”. This has disincentivized any buying and selling and has inhibited the existence of a healthy collectors’ market, something which has always occurred naturally in every other sphere of artistic creation. Now, thanks to the capabilities of this blockchain, a digital file can be “uploaded” to the Ethereum network and registered as a non-fungible element in a process known as tokenization [5] or minting, in such a way that an individual and unique registration is generated in the blockchain for that artwork in question. The ownership of this can change hands just like for any other property, which enables transactions to take place. The JPG file for the artwork is not in itself the asset being bought and sold, as it is still a fungible item that can be downloaded and replicated between computers, but what is commercialized is the countable blockchain line, the registration of ownership, which does have a non-fungible nature. This new panorama has allowed for the development of what is generically known as NFT Art: a vibrant scene of new artistic proposals that use the blockchain as a space to exhibit and commercialize artworks. Nowadays, the most commonly-found artistic categories in the blockchain are those known as PFPs, 1:1 Art and Generative Art, and of course we can expect new formats to keep developing in the near future. PFPs, or profile picture projects, are collections of images created to be used as avatars or visual representations of users’ online identities. They are normally generated by recombining individual elements based on, for example, a collection of eyes, noses, mouths, bodies and the like, in sufficient quantities to guarantee that every resulting illustration is unique. The rarest combinations are those most highly valued by collectors. The most famous examples of PFPs are CryptoPunks and the Bored Apes Yacht Club series, projects that have quickly grown popular due to the perceptions of exclusivity, value and status associated with the possession of these avatars [6]. Generative Art includes projects in which the artist creates an algorithm which automatically generates the artwork [7]. We will describe this category in greater depth in the next section, as it constitutes the central axis of this study. Finally, 1:1 Art is a wide and varied category encompassing any form of art that does not belong to the above categories and which has a more individual conception for the pieces. It has a position closer to traditional art and includes digital photography, designs, collages, 3D renders, GIFs, animations, but also physical paintings done on canvas or drawings on paper which are digitalized in order to be incorporated into the blockchain. Usually the creator of a piece produces a single token for it, hence why it is known as 1:1, but we can also find artists who sell a small numbered series of NFTs from one piece, such as what traditionally occurs with limited editions of etchings and engravings. The best-known example of 1:1 Art is the piece Everydays: the First 5000 Days by the artist Beeple, a JPG made famous for being auctioned at Christie’s for sixty-nine million dollars in 2021.

70

D. Lanza

2 State of the Art in Generative Art 2.1 Generative Art and the New Role of the Collector This paper will focus on this category, which Galanter (2017) defines as “any art practice where the artist creates a process, such as a set of natural language rules, a computer program, a machine, or other procedural invention, which is then set into motion with some degree of autonomy contributing to or resulting in a completed work of art” [8], in other words, forms of artistic expression where the artist, rather than producing a piece, designs and constructs a system that produces the pieces autonomously, introducing into the creative process elements that are not directly controlled by the artist. This incorporates an element of randomness into the creation and the artist relinquishes some control in order to allow external phenomena, generally random, unpredictable or at least not completely controllable, to participate in completing the piece. In line with this definition we see that this idea is not necessarily linked to computer use, since algorithmic behaviors are also present in non-digital works. We need only call to mind artists such as Cai Guo-Qian, Antony Gormley, Olafur Eliasson or Neri Oxman to observe notable examples incorporating physical, chemical or biological processes as active ingredients in the artistic production. Moving into the digital context, the first expressions of generative art using computers were found in the sixties with the experiences of pioneers like Georg Nees, Frieder Nake and Michael Noll in 1965, consolidated from the seventies onwards by artists such as Manfred Mohr, Vera Molnár and Herbert Franke. In digital generative art, what the artist produces is an algorithm, instructions written in programming language, which when it is executed produces an artwork, normally visual –a graphic–, although not only that. Carter & Levin (2020) even talk about “collaboration between humans and machines, where the artist sets the rules and the algorithm implements them to produce art” [9]. If the algorithm is designed with no random function and no parameter that varies every time it is run, the same output will always be produced. If, on the other hand, the artist uses variables that take different random values every time, the algorithm can produce different outputs. Digital generative art evolved gradually and silently from the sixties onwards, staying somewhat on the margins of the usual circuits of the art market and exhibition spaces. It is due to the recent development of blockchain technology, specifically networks like Ethereum with its previously described ability to run applications, that generative art seems to have found its perfect setting, piquing interest and achieving unprecedented visibility. By giving works of generative art a unique and authenticatable character, NFT technology has managed to materialize the value of these creations and incorporate them into the art market. Part of the success that generative art is currently experiencing is due to an original paradigm of generation that turns the buyer of a piece into a participant to a certain extent in the creation process. To understand this, let us go back to the idea of a generative algorithm that uses random numbers to construct pieces to make the results varied and unpredictable. The novel thing about this approach, which is making these pieces so appealing to collectors, is that now the algorithm can be configured to make the random numbers derive from the identifier of the buyer, which is called their personal hash,

From the Algorithm to the New Art Collector

71

and from data like the exact date and time that the purchase is made. This means the construction of the piece is influenced and marked by these variables, which the algorithm can use as a seed for its random numbers, so the very mechanics of the purchase makes each piece unique and different from what would be produced if the purchase was made a second later or by someone else. However, as the algorithm does not run until the collector makes the purchase, they only know the style of the collection they are interested in from some sample outputs, acquiring the piece before it is generated, before it exists. We could say that the primary buyer, in their role as co-generator, acts somewhat blindly, but this is generally compensated for by the fact they are made a participant in the creation process. Here we mean that their participation is, obviously, more mechanical than intellectual or conceptual, but it is enough to establish a new and powerful link between the collector and the piece they are acquiring. We see the artist as the designer of the system and indisputable author of the pieces, but we also see how they concede some creative space to entities they have no control over, some to unpredictable random numbers and some to the actions of an external buyer. Immediately after the purchase is made, the algorithm is run and once the piece is finished it is automatically registered in the blockchain through the process of minting described above, keeping a record of the piece in question, the authorship of the artist and the collector who owns it. This model, whose novelty lies in incorporating information of the collector to construct the piece, was originally formulated and developed by the platform for launch and generative artist promotion Art Blocks, which has been recently joined by other alternative platforms like GM Studio and Fxhash, with somewhat similar approaches. 2.2 Primary Collectors and Secondary Collectors After the primary collector generates and mints the artwork, they can freely sell ownership of it to third parties. This transaction will lead to the registration in the blockchain of this change in ownership and such information is public and transparent at all times. It is important to highlight that what is bought and sold in these markets is ownership of the piece, the countable note registered in the blockchain which certifies the legitimate owner, but not the image file, in any digital format such as JPG or GIF, which due to its fungible nature is infinitely cloneable and so cannot be transacted. Unlike the primary market, where the collector “blindly” acquires a piece which has not been generated yet, secondary buyers do know the piece they are going to acquire, a piece which was previously generated during the primary purchase, so they have the option of being selective and choosing the piece that interests them most among those on offer on the secondary market. In exchange they renounce the sensation of coauthorship of a piece that was generated and minted by the primary buyer. This ultimately establishes a primary and secondary market for digital artworks, markets which so far were inaccessible to digital artists due to the fungible nature of digital files. Another novelty that blockchain technology is incorporating into the art market context is that it is now possible to establish, in the smart contract linked to the algorithm, the percentage of royalties that the artist will receive both through the primary sale and through all successive secondary sales. This payment is made automatically, transparently and reliably in perpetuity every time a piece changes hands and it is not possible

72

D. Lanza

to elude this payment. This protects the artist, offering a flow of income that is fairer and more continuous than traditional models, where the creator was usually left on the margins. These new approaches, which interweave the artist’s creative contribution, the somewhat mechanical participation of the primary collector, the unpredictable action of random numbers and the movements of secondary collectors, are producing an interesting and vibrant art market with attractive dynamics that are not present in traditional art markets. Although some traditional art galleries and auction houses are already beginning to exhibit and auction generative work, like Christie’s and Sotheby’s, the main market on the NFT scene is in what is known as marketplaces, online platforms specifically dedicated to exhibiting and trading digital art. Portals like OpenSea, Rarible, SuperRare and Nifty Gateway open the doors to new ways of exhibiting and experiencing art, allowing for more democratization in the production and consumption of art [10]. Here we find showcases where new creations can be exhibited, with a great sense of inclusivity and openness to all kinds of art formats and varied dynamics of buying and selling including fixed-price sales to auctions by ascending and descending bids. By offering a digital platform accessible for the promotion and sale of artistic creations, these marketplaces have enabled the participation of emerging and unconventional artists in the art market, breaking down existing hierarchies and structures [11]. This democratization is also extended to collectors, allowing a wider audience to access and acquire artwork and challenging the exclusivity and elitism traditionally associated with the art market.

3 Design, Development and Launch of an Innovative NFT Collection 3.1 Concept for a Practical Project Once the keys to this new and emerging scene have been understood and assimilated, we wanted to explore this novel context in a practical way, in first person, developing an applied generative art project that we could use to document every stage, from concept and design to public launch, a large-scale project, sufficiently elaborate and complex to produce a collection of 999 different pieces to enable us to experience all the difficulties in the process and record them to help future creators. The starting point for our proposal consisted of the challenge of programming an algorithm that could generate digital art that was not perceived as geometric or mathematical. And we talk about programming because, as we’ve been describing, what we aim for with this project is not to paint physical artworks to later photograph them and upload them to the Blockchain, but to program an algorithm that digitally draws these pieces through code at the very moment of their purchase. As geometry is a relatively immediate resource for digital construction, due to the mathematical, polygonal and geometric nature of the entities that make up all programming languages, here we looked for the challenge of distancing ourselves as much as possible from that context, trying to explore the limits of programming. The starting reference, chosen because it is a diametrically opposed aesthetic to the mathematical and geometric, was the expressivity,

From the Algorithm to the New Art Collector

73

passion and energy of abstract expressionism, specifically of artists like Jackson Pollock and Franz Kline. The name of the collection, Catharsis, would evoke that passion and energy associated with expressionist creative delirium. Reaching this organic nature through geometric entities like the straight line, circle or polygon, was the first difficulty we had to overcome. It took a lot of time and effort in research and development, breaking down the main functions and working on randomness at a pixel level to manage to get away from the harmony, stability and predictability inherent to geometry. The lines of work included a realistic representation of the interweaving of canvas threads, simulating the fluidity and viscosity of paint, or its impact on the surface or transparency of color when diluted (see Fig. 1).

Fig 1. Energy, fluidity and organic sensation from geometric entities in Catharsis #204 - Nuages (detail)

Once the organic goal had been achieved, the project faced a new challenge: throwing the paint in a completely random way against the surface produced a distribution of color masses that was excessively arbitrary, homogenous, lacking in meaning and too similar from piece to piece. To avoid this problem, we had to orchestrate the throwing of the paint in such a way that, while maintaining randomness, it had a rhythm and suggested movement within the piece, evoking the arm gestures of a hypothetical painter. Designing this internal rhythm, which orchestrates the appearance of paint on the piece, meant a delicate balance between control and chaos where too much prefixing of the behavior of the paint would have resulted in pieces that were excessively repetitive, “prefabricated” and dead, while giving too much space to randomness would produce a homogenous result that was equally lacking in life. We see here the usual debate relating to the space given to randomness and external factors in generative art. It is necessary to design

74

D. Lanza

within the piece both the elements we want to control and those that we allow to remain outside our control. Technically, the creation of this internal rhythm required each piece to develop certain trajectories and attraction/accumulation spots where the paint would prefer to land, which would imitate the organic arm movements of a painter and the paint that is thrown due to those movements, while offering enough randomness around these fixed structures (see Fig. 2).

Fig 2. Internal rhythm orchestrating the movement of the painting in Catharsis #781 - Beautiful Love (detail)

3.2 Continuity and Evolution Once the rhythm of the paint on each canvas was developed, the idea came up of making that movement, the trajectory that orchestrates on each canvas the disposition of the paint, extend from one piece to the next and for there to be continuity so that the rhythm extended beyond the limits of the canvas. This idea was algorithmically complex and required each piece to know its position in the series and to be able to use information taken from the previous piece, develop itself based on that information and prepare the following piece which had yet to be generated. Once we had achieved this sophisticated development, Catharsis presented an interesting innovation in the context of generative art: the potential for a piece to continue in the next which would enable the creation of diptychs and triptychs with continuity between them. This movement, which runs along each canvas and spills over the edges of a piece extending to the next, turns the collection into an immense polyptych of enormous dimensions where there is physical and visual continuity from piece 1 to piece 999.

From the Algorithm to the New Art Collector

75

To increase the possibilities of pairing, the algorithm was refined until each piece could present continuity, not just with the pieces immediately next to it but also in multiples of 50, both to the left and to the right. So, for example, piece 394 would have continuity with pieces 393 to the left and 395 to the right, but also could be paired with 445, 495, 545, 595, 895, etc., giving the collector a wide space to configure their own diptychs and triptychs (see Fig. 3). This offered new and interesting dynamics in the generative art space, where so far each piece has always been developed on its own without an awareness of those around it, but also replants the concepts of the diptychs and triptychs of traditional art. Historically, the creation of diptychs, triptychs and polyptychs has responded to material matters, usually motivated by the impossibility of having large enough canvases or slabs, demands made by the exhibition space available, like in cycles designed for decorating retablos or predellas, or aesthetic preferences of the author, like in the triptychs of Francis Bacon, but they were always predesigned by the artist to operate as a unit in a predetermined configuration. In Catharsis, the diptychs, triptychs and polyptychs can be freely configured and recombined by collectors, creating a space for interaction between them and the artworks. Here, continuity offers an interesting creative space for the collector, who is normally kept on the margins in a unidirectional communication but who can now participate in the artistic experience by configuring sets according to their own conceptual or aesthetic criteria. This turns the collection into an immense game board where each piece can form part of a great number of different diptychs and triptychs, a new space which invites active participation where the collector’s creative contribution is essential to complete the artistic experience.

Fig 3. Painting that spreads from one work to the next. Diptych composed by Catharsis #394 After I Say I’m Sorry (left) and Catharsis #445 - Make Believe (right)

Another innovation that was incorporated into the project was the possibility of conceiving the collection, the linear series of 999 pieces, as an evolutionary process that

76

D. Lanza

would present some kind of transformation every time it was run. To achieve this goal it was necessary to incorporate into the algorithm the possibility of evolving throughout the series, presenting artworks with different characteristics as the collection progresses. With this objective, the algorithm was configured to offer pieces with a more timid and tranquil appearance at the beginning of the series, and the passion and energy represented in the painting would increase every time it was run to reach an intense and explosive finish. This made the whole collection into a linear episode of increasing intensity, a cathartic process of creative liberation. Here not only does each individual piece evoke a catharsis, but the whole collection becomes a creative catharsis, a process that begins timidly and which gradually grows in intensity and density as the paint is launched in an increasingly passionate and expressive manner (see Fig. 4).

Fig 4. Evolution in increasing intensity throughout the series Catharsis #74 - Topsy (left) and Catharsis #864 - Hymn to Freedom (right)

This conceptual axis constitutes another interesting innovation for generative art, where as we have mentioned, each execution of the algorithm had always been independent of all the others and each piece developed in an aesthetic space that was totally distant from its context within the collection. In Catharsis, the continuity and evolution of the algorithm create new and fertile relationships between the collector and the piece, and between the piece and the rest of the collection, enabling new conceptual and creative space that we believe will inspire future generative artists to continue exploring. 3.3 Launch of the Collection and Conclusions Catharsis was presented to the launch platform for generative artists GM Studio for evaluation. GM Studio boasts a team of curators comprised of artists, critics, and art

From the Algorithm to the New Art Collector

77

historians, who analyze and evaluate each project submitted through a rigorously anonymous process. This curation team accept or reject each project based on the majority’s decision, aiming to select those they deem of greater interest, innovation, and relevance while avoiding any bias. Under these circumstances, Catharsis was accepted in May 2022 and the launch of the collection took place on September 10th 2022 (see Fig. 5), with a huge success from the first moment. The 999 pieces sold out on the first day and, from there, the collection has continued to move through the secondary market where the pieces have kept increasing in value and generated over 1 million euros in the first six months, proving the interest that this project and its innovations have awakened in this particular art market.

Fig 5. Screenshot from the promotional video of the launch campaign

All of this indicates to us the great interest awoken by generative art among a new generation of collectors for whom art lives more naturally in the blockchain than in auction houses. Generative art highlights the potential and tensions that arise from the interaction between art, technology and the observer, suggesting new limits and possibilities for artistic creation and consumption. But it also speaks to the interest that the innovations incorporated into this project, like continuity and evolution, bring to generative art specifically and to the artistic context in general. The collection conceived as a linear series that evolves by itself, the collector who can now have a creative space in the artistic experience, the physicality and tangible sensation obtained out of cold geometric and mathematical entities, offer new and interesting perspectives to explore that we hope will serve as inspiration for future artists. The project, conceived with the objective of enabling us to write this paper, has offered us the chance to get close to this new and vibrant reality, beginning by understanding the key principles of the blockchain and the context of non-fungible tokens, to the design and development of a practical applied project experienced at every stage.

78

D. Lanza

New horizons for analysis emerge from this point, such as the study of the current and future evolution of generative art, the new dynamics of collecting and curating in this new art market, or the development of new long-structure projects in the future. However, at the very least, we have been able to provide a complete experience that we trust will serve to bring closer this world that is still unknown to many artists and designers, and which we can confirm offers interesting possibilities, both creative and economic, and interesting challenges for the future that are worth exploring.

References 1. Wu, H., Li, S.: Blockchain technology: a review and future research directions. Int. J. Inf. Manage. 61, (2021) 2. Swan, M.: Blockchain: Blueprint for a New Economy. O’Reilly Media (2015) 3. Casey, M. J., & Wong, P.: Global Cryptocurrency Benchmarking Study. Cambridge Centre for Alternative Finance 4. Ferrari A, Pensa RG (2020) The tokenization of assets: an analysis of the legal issues. Eur. Bus Organ Law Rev. 21(3):455–477 5. Werbach K (2018) The Blockchain and the New Architecture of Trust. The MIT Press. https:// doi.org/10.7551/mitpress/11449.001.0001 6. Schwartz, J.: The Rise of NFT Profile Pictures and Generative Art. Medium. Retrieved from https://jschwartz390.medium.com/the-rise-of-nft-profile-pictures-and-gen erative-art-414c5f410ccc (2021) 7. Smith J, Chen L (2021) Exploring the aesthetic possibilities of generative art. Int. J. Creative Technol 4(2):123–138 8. Galanter P (2017) Defining generative art: contexts, processes, and outcomes. Art Complex. 8(1):40–54 9. Carter R, Levin G (2020) Human-machine collaboration in generative art. J. Comput Cult. 1(2):64–79 10. Capurro, M., Lorusso, A., & Ragnedda, M.: NFTs, blockchain, and the future of art: A critical analysis of the art market. Information, Communication & Society,pp. 1–18 (2022) 11. Bruns C (2022) Digital art and NFTs: The rise of the virtual gallery. Art and Design Review 10(1):1–12

From Print to Pixels: Prototyping a Virtual Exhibition for the Faro Museum Poster Collection Alexandre Martins(B) , Jorge Carrega , Rui d’Orey , and Bruno Mendes da Silva Centro de Investigação em Artes e Comunicação, Universidade do Algarve, Faro, Portugal {acmartins,jmcarrega,rldorey,bsilva}@ualg.pt

Abstract. In 2022, CIAC - Research Center in Arts and Communication and the Faro Municipal Museum began a cooperation to study, enrich and disseminate a unique poster collection that belonged to the Portuguese scenographer Joaquim António Viegas. It comprises 330 illustrated posters from the cinema, circus and advertising industries. This partnership entails the coproduction of multiple exhibitions about the collection, but also future research of the materials that form this important heritage. One of the objectives/aims of this work is to develop a website which will collect and present the information about the poster collection, while also serving as a platform to show virtual exhibitions of the same collection. In this paper we present the initial stages of the development of a 3D real time exhibition which is intended to open simultaneously with a physical exhibit of the posters dedicated to early Italian cinema - scheduled to be inaugurated in March of 2024. We will describe some of the first steps we have taken so far which covers the exploration of tools devoted to building 3D environments, the tasks we already achieved and lay down a couple ideas for the future of this project. We hope the prototype we present here might function as a model and a proof of concept for digital representations and reinterpretations of future exhibits of this singular legacy. Keywords: Virtual exhibition · Movie posters · 3D Rendering · Virtual room prototype · Unity engine

1

Introduction

Joaquim António Viegas (1874–1946) was a painter and a scenographer who developed his professional career in some of the most important entertainment venues in Portugal including several of the first cinemas in the country, such as the Chiado Terrasse in Lisbon, or the High-Life Hall in Oporto. During the first two decades of the 20th century, he assembled a collection of 330 posters, consisting of three thematic groups: Cinema; Circus/Variety; and Advertising1 . 1 Even though all posters can be considered advertisements, the great majority consists of

show industry examples. The other part comprises ads for canned food, beverages, tourism destinations, an insurance company and even a Harley Davidson dealership. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 79–93, 2024. https://doi.org/10.1007/978-3-031-47281-7_7

80

A. Martins et al.

The collection, donated to the Faro Municipal Museum in 1990 by Viegas’ descendents, includes extremely rare specimens in the world and unique ones in Portugal. The cinema set features 141 posters, dating from 1904 to 1916, including 68 from France, 43 from Italy, and around 30 from Germany, Scandinavia, and North America [6]. Signed by the most notable illustrators of the early 20th century, such as Cândido Aragonez de Faria, Leopoldo Metlicovitz, and Maurice Lalau, these posters represent emblematic works of European silent cinema, such as Les Victimes de l’alcool (G. Bourgeois, 1908), QVO VADIS? (E. Guazzoni, 1913), and Fantômas (L. Feuillade, 1914). They constitute a representative sample of movie poster art in the early 20th century and a valuable testimony to the role they played in the development of cinema as a cultural industry. For several years, CIAC - Research Center in Arts and Communication (University of Algarve) and the Faro Museum have been developing a fruitful collaboration in the field of exhibition curation. Thus, in July 2021, the museum inaugurated the exhibition “1907–1914: a primeira era de ouro do cinema francês na coleção de cartazes de cinema do Museu de Faro” (see Fig. 1), curated by CIAC, which presented a collection of 33 rare French cinema posters from the first two decades of the 20 century [1, 4].

Fig. 1. Advertisement for the French poster exhibit “1907–1914: a primeira era de ouro do cinema francês na coleção de cartazes de cinema do Museu de Faro”, held in 2021 in the Faro Municipal Museum (Source: Faro Municipal Museum)

In February 2022, this same research center signed a protocol to collaborate with the museum. The purpose of this new cooperation is to study, enrich and disseminate the collection through multiple national and international exhibitions, as well as the production of conference papers, articles in journals, book chapters, and future catalogs2 - all created from the research of these cultural artifacts. We say future because the Faro Museum has published in 2021 a catalog entitled Cinema em Cartaz [6] about the collection of cinema posters which included, among others, texts by Jean-Louis Capitaine, Adelaide Ginga and Marta Mestre. Even so, the 2 The research is currently being produced by the CIAC’s Film Studies research group, and other

academic national institutions which are currently in talks to formally join this venture.

From Print to Pixels: Prototyping a Virtual Exhibition

81

collection still offers multiple possibilities for unexplored approaches, i.e., the study of film production companies, of illustrators and studios, of circus numbers and variety artists, or of the advertisement culture and industry in the early 20th century. Also due to the fact that the Faro Municipal Museum currently does not own an official website where it can display and promote its different objects, and the only information about the poster collection comes from the Faro City Hall which only dedicates a single paragraph of information about this important cultural heritage and four low resolution illustrations [7], another goal for this project is to create an open-access digital platform, where these posters will be made available in a digitized version. Furthermore, it will serve as a space to access all the scientific knowledge that emerges from the research of these objects, i.e., papers, book chapters, catalogs, etc. Finally, this online platform will also give us the possibility to host recreations and reinterpretations of the poster exhibitions held at the Faro Museum - and possibly other national and international museums - in a digital real-time 3D environment. Between the different small-scale initiatives already promoted by CIAC, which include various scientific communications about the collection [9, 10, 25, 29], for the next stage of this project, CIAC and the Faro Museum will organize its next poster exhibition entitled “A ascensão do cinema italiano através da coleção de cartazes do Museu de Faro”. Scheduled to open in March 2024, this exhibit, which brings together twelve large Italian posters (see Fig. 2), is intended to be accompanied by a virtual exhibition presented simultaneously with the opening of its physical counterpart. This might allow us to show posters that cannot be presented physically due to its state of preservation: three new ones to be more precise, which are incomplete (missing half of its entire page) or in a poor condition. We could also add new information and other forms of media which are not possible to integrate in the former exhibit. Besides that, this second concurring event will enable other types of public to learn about an important period in cinema history and discover posters whose artistic and cultural value deserves to be recognized, without the need to travel to Faro during the limited timeframe of the physical exhibition - estimated to have a duration of five to six months.

2 Museums in the Twenty-First Century The landscape of museums is evolving. They are now acknowledged as having a significance that extends beyond the presentation of objects and artworks, or their preservation and research. Because of this, museums in the twenty-first century are prioritizing their communication strategies and attractiveness, enhancing the visitor experience and enabling a deeper understanding and engagement with our cultural heritage. They aim to create an enjoyable visit while also serving as social and cultural hubs, drawing both tourists and residents alike. For individuals to revisit a museum, these institutions usually have to offer new and engaging exhibits that might provide flexible contents. According to Pescarin [23] studies have associated engagement or attractiveness of museums with exhibits that have narratives which could be rendered through information and communication technologies (ICTs) and through virtual museums.

82

A. Martins et al.

Fig. 2. Three Italian posters from the museum’s collection. From left to right: Amore Pacifico (1915), illustration by Atelier Butteri; Gli ultimi giorni di Pompei (1913?), illustration [unidentified]; and Scienza Fatale (1913), illustration by Leopoldo Metlicovitz (Source: Faro Municipal Museum)

While curators’ responsibilities encompass the preservation of collections, their exhibition and communication, or the acquisition of new objects, nowadays their roles are expanded to also include the insurance of scientific accuracy and quality of digital content, maintenance of clarity in communicating concepts to specific groups, selection of appropriate communication channels and media, and integration of digital elements into museum operations. They must also consider the innovativeness of technology in terms of visitor engagement and comprehension - interactivity, visualization, immersion and user-friendliness [13, 23]. We find ourselves amidst an era where ICTs have a profound influence on cultural heritage, an impact that remains to be fully perceived and understood. The potentials inherent to these technologies are opening fresh perspectives for experiencing cultural heritage, which encompasses aspects like visualization, representation, interaction, exploitation, enhancement and dissemination [2]. When effectively integrated into museum itineraries, digital programs take up an important position. Interactive virtual exhibitions or museums hold great potential that should be thoughtfully developed, to meet visitors’ expectations while remaining attuned to the requirements and needs of curators. Since the beginning of the 21st century, the concept of virtual museum has evolved to incorporate ideas like simulation or an interactive reading of the different potential realities of cultural information, becoming complex communication environments with interaction and immersion in 3D reconstructed settings. They can be classified based on different criteria, such as their content, interaction type, duration, communication style, level of immersion, distribution method, scope and sustainability level [3, 16]. These digital hypermedia objects are interconnected by a shared objective, an interdisciplinary theme, a concept, an idea, an individual, etc.; are presented in both 2D and 3D formats; at times, are stored across distributed networks; and can be accessed through modern

From Print to Pixels: Prototyping a Virtual Exhibition

83

technologies and a system architecture specifically designed to provide users with novel experiences. Some advantages associated with these types of presentations are the promotion of the cultural legacy safeguarded by institutions; its role as a pedagogical tool; access to valuable artworks or documents without significant risks; availability of documents and papers that may otherwise be inaccessible; long-term accessibility, as they are not limited to a specific event; availability for users worldwide, enabling visits regardless of geographical constraints; or the potential to serve as an online repository for information related to physical exhibitions [14, 17]. And while it is challenging to replicate the sensory and emotional significance of its physical counterpart, web-based virtual experiences have additional benefits like the direct access to culturally rich content from personal computers or mobile devices; freedom to explore content based on individual preferences; possibility to save, store, and reuse content for personal use; or flexibility to establish connections with other virtual exhibitions or external sites through external links [14, 20]. Overall virtual exhibitions provide flexibility by adding new components - or modifying existing ones - to the museum or gallery’s exhibit. With a virtual exhibition there is also the possibility to create virtual duplicates of valuable and delicate physical objects, minimizing the risk of damage that may occur during their handling [11]. The development of systems designed to present often rare and important artifacts through virtual exhibitions is something that museums could include in their communication plan, offering visitors the chance to consume cultural heritage in new and different forms.

3 Creating a Virtual Exhibition 3.1 Setting a Structure for a Prototype Since the signing of the protocol between CIAC and the museum, there has been a steady effort from this research center to assemble a team of experts from different research fields, departments and institutions with the objective of studying the collection. This is precisely one of the primary goals of this partnership which will enable us to gather new findings about the items included in the cinema nucleus, but also explore the other two groups - Variety/Circus and Advertising - that have not yet been studied. All this new data, which is going to include a digital record of all posters, will result in the production of future exhibitions, conferences, and catalogs - all with the full support of the Faro Museum -, an endeavor that will permit us to keep promoting this legacy. This can be designated as the main, and for now, only concrete and formal method of work: study and digitalization of the collection → dissemination of the collection through exhibitions and scientific texts. Despite this, a smaller group of CIAC researchers has joined to discuss ideas related to the eventual possibility of creating an online platform that will gather all the knowledge gained from the scientific work. From these meetings we arrived at a proposition to create a virtual exhibition - a prototype at least -, that enables us to start experimenting with different 3D building software, while presenting us a proof of concept for future virtual exhibitions once the website is running.

84

A. Martins et al.

Because we already have a scheduled exhibit for the first trimester of 2024 dedicated to a series of Italian posters, we explored the feasibility of simultaneously opening a second one, but this time in a virtual three-dimensional space that simulates - until a certain degree - the first one. The decision was to make a simple prototype with the following elements: a) create a virtual 3D environment that simulates the room where the physical exhibit will take place; b) include high resolution images of the twelve posters presented in the main event and three more that cannot because they are either incomplete or in a bad state of preservation3 ; c) include interactive elements that connect to textual information regarding each poster - museum identification number, title, year of movie premiere, movie director, poster illustrator, and small description of the movie - and another that contextualizes de exhibition4 ; d) include interactive elements connecting to online spaces that host some of the movies which are currently publicly available and in the public domain, e.g., Quo Vadis? [15]5 . If these points are not met by March 2024 and we assess that this prototype does not have aesthetical value or doesn’t function properly, we will not present it to the public, but instead will keep improving the model and use the knowledge gathered for future works. On the contrary, if we see that the project has the potential to be shown, the prospect is to display a QR code in the museum’s room and another in the program that connects to an online platform with the 3D simulation of the exhibit. 3.2 Searching for the Ideal Workspace Kickstarting from these four objectives we began a process of exploration for different programs that delved into virtual tours and more specifically virtual exhibitions. From this research we found two distinct online examples that showed some potential and gave us insights into what could be done in this context. The first one was OnCyber [21], a program described by their website as “a metaverse platform that makes it easy for anyone to create their own 3D, immersive experience that can be accessed from the browser” [22]. Although they call their platform a place to create 3D experiences, it is greatly used to create gallery type spaces to exhibit and sell digital art through Non-Fungible Tokens (NFT’s) (see Fig. 3). And when we say create, this does not mean to construct the spaces themselves, because those are mostly pre-built, but to customize them with different assets (e.g., images, gifs, videos, etc.). 3 Even regarding the twelve that will be in the physical presentation, the final selection will only

be confirmed after an evaluation from the museum about the preservation status of each one. This will not be true in the virtual one where we already know for sure that all of them can be displayed. 4 Due to space limitations of the available exhibition room, some of this information will also not be demonstrated in the museum. 5 This is another feature unique to the virtual mode.

From Print to Pixels: Prototyping a Virtual Exhibition

85

Although there is also a feature that lets you upload your own customized space, the process itself and philosophy of the platform didn’t seem quite in line with the purpose of the poster exhibition. Despite this, OnCyber showed us some promising ideas to later use, like freely controlling the movement of an “avatar” through the x and z axis - and the camera in all directions - in a real time environment, much like a first-person video game. Also, some ways that you could interact with the artworks were kept in mind. For example, if your avatar gets to a certain distance from a video, which has its images constantly playing throughout the exhibit, an event triggers the sound to play. The proximity system also shows another interesting aspect. When you are close to a particular object, the screen highlights that artwork while omitting the others, and a visual cue appears saying that you can interact with the object by clicking “E” on the keyboard. This will lead you to different information about the piece, like data about its author and his social networks. By interacting with the work, you can also place a bid for that piece.

Fig. 3. First person view of an exhibit for digital art in OnCyber (Source: OnCyber)

The other online platform we studied is called kulla, self-described as the “Ultimate Virtual Tour Software for any business”. This program enables the user to create 360º/3D tours for real estate, architecture, art galleries, etc. [19]. Kulla is a virtual tour editor that uses multiple panoramic photos to create 360º environments, by arranging and connecting the different pictures uploaded to the system. When connected you can customize the spaces with hotspots icons - meaning the different points of view where you can control the camera freely in all axes - and add labels, images, audio and sounds. In the context of the poster exhibition, this platform would allow us to capture panoramic photographs of the exhibit’s venue and then add images of the Italian posters, textual information about them, create QR codes to external platforms, or directly link icons in the virtual exhibit to the poster’s respective movies on YouTube. These are all possibilities that we attested in one of the examples studied, a virtual exhibition for digital and (audio)visual works integrated in the 2022 edition of ARTeFACTo’s conference, which took place entirely online due to COVID restrictions in the country’s venue (Macau). This exhibition was produced using kulla, and from what we could observe, besides the elements listed above, there was also the opportunity to add

86

A. Martins et al.

3D models of the artistic artifacts, although we weren’t able to find out the method used to create them - we didn’t see it advertised on kulla’s website. In one of the artworks presented in ARTeFACTo2022, entitled In MarImmersive Waves [12], which uses a marimba as an integral part of the piece, we saw the chance to add a 3D model of this instrument, hear sounds produced by it, have access to a video on YouTube with the work’s performance, and to some textual information, which was complemented by a QR code that connected to more documentation about the work in a different website (see Fig. 4 and 5).

Fig. 4. Overview of In MarImmersive Waves, one of the digital artifacts present in ARTeFACTo2022 virtual exhibition, with 3D models (Source: kulla)

Fig. 5. Connection to In MarImmersive Waves’ performance on YouTube (left); and panel with textual information about In MarImmersive Waves and a QR code to external documentation about the work (right) (Source: kulla).

kulla’s platform and this example showed us the prospect to create a virtual exhibition capable of simulating physical and existing environments. That was one of the main points we delineated for this project. But despite this, it seems that this program was missing one or two key elements to be the ideal place for us to produce this first exhibit for the museum’s poster collection. One thing that we immediately saw as constraint (literally) was the fact that the visit’s walkthrough is designed to occur from icon to icon, from one fixed point of view

From Print to Pixels: Prototyping a Virtual Exhibition

87

to another. This scenario was more limiting than the one provided by OnCyber. Because we had previously tried this second platform, it was strange to then be deprived of the freedom of movement we had experienced with OnCyber’s examples. That was a feature we intended to keep. The other component that was important, and which in this case was not possible to be achieved with both designs, was the freedom to experiment with other elements besides those which were already programmed in each system. It is important to note that besides the creation of an interesting virtual experience that concurs with the Italian poster presentation, we want to use this digital visit as a jumping point for other exhibitions. It is then important to utilize this object as a model for experimentation and to gather as much information for future iterations of the artifact. Because of this crucial goal, we realized that the most indicated path was to operate a more generic system dedicated to the creation of 3D scenes that would allow us to achieve the four points previously listed, but also discover new elements that could be then either applied to the current exhibit or reserved for future works. This is an idea that could be tested in the fields of user interface and interactivity, or on the aesthetic choices for the room itself. After some discussion between the current CIAC members in charge of this task, we decided to work with Unity for this first exhibition and possibly future ones. An engine first designed to develop games for the MacOS, Unity is now a cross platform and multimedia software used to create movies, like the short animations co-produced with Disney Studios [26], in product design, architecture, interactive and live performances, augmented and virtual reality experiences, etc. The technologies provided by the Unity engine grant us the opportunity to recreate, to some degree, all the features from the other two platforms, while also enabling us to add new ones. Overall, this engine is useful to produce real time 3D environments with many different tools at our disposal that establish the perfect setting to experiment with different designs and concepts. 3.3 Prototyping the virtual room. After agreeing on the workspace for the first digital exhibition, we started immediately experimenting concepts we saw in the other two online platforms. The first was the character and camera control. For this we appropriated a simple 3D rendering of a building from the Unity Learn [27] and created/trained the movement control in a firstperson view mode. For this alpha version we also included some images of the poster collection (see Fig. 6). The final design was later published in the Unity Play website to see how the program would run on their platform [28]. Subsequently, we decided that for the next stage of this project we should gather images of the room where the exhibit will take place in 20246 (see Fig. 7). Although the museum has a virtual 360º tour of its space available in the City Hall’s website [8], it only covers the rooms with permanent exhibitions and not the temporary ones. For this reason, we headed to the museum to capture some images of the Italian poster exhibit room. And since experimentation is a key component for the work, we decided to do 6 This stage happened after briefly trying and not getting quite what we wanted from a 2D to 3D

conversion of the floor plans of the room (using the extrude function in AutoCAd).

88

A. Martins et al.

Fig. 6. Creating and training movement in first person mode using Unity (Source: authors)

some trials with 3D scanning using the Polycam mobile app [24]. With help of this program and a camera equipped with LiDAR sensors, we were able to capture the first 3D images of the three sections that form the room in less than 10 min and without having to acquire expensive and specific tools for these tasks (see Fig. 8).

Fig. 7. View of one of the sections that form the museum’s room that will be used for the 2024 Italian film posters exhibition (Source: authors)

Although still imperfect7 and with a lot of space to improve, at the time of this writing, this is the rendering we are working with. We already went back to get another scan of the same room, but even though some parts of the design improved, there is still room to progress. We are planning to get back to the museum, this time equipped with cameras more adequate to our needs (Insta360 X3), other apps and technologies - e.g. experiment with Neural Radiance Field (NeRF) models [18] -, and with better strategies 7 This first reading has a lot of digital artifacts in the walls: some parts protrude forward while

others are missing, giving the room a feel of amorphousness. These “errors” were possibly caused by an incomplete or imperfect scanning of the room.

From Print to Pixels: Prototyping a Virtual Exhibition

89

to capture the images - e.g., improve the space’s lighting, scan movements, exposition time, etc.

Fig. 8. Different perspectives of the 3D rendering of the museum’s room (Source: authors)

For the time being the plan is to work with the scans provided from the first and second readings. This is precisely the current state of the project. After uploading the GLB files from the PolyCam scans into the Unity software we were able to start implementing some basic components for the new and current model of the exhibition (see Fig. 9). Right now, most of the work is focused on: a) implementing and testing character and camera controls; b) creating physical barriers around and beneath the scans; c) testing the space’s lighting; d) positioning the posters and editing their size and colors; e) writing the texts for each poster and for the exhibition; f) finding the original movies in online platforms that are in the public domain; g) integrating and testing interactive elements for the posters, texts and videos; h) adding extra videos and sounds. Regarding the elements listed above, presently some are in a more advanced state than others. After uploading the scans, we immediately started by creating the physical environment that surrounds the room and soon after we implemented and tested the first-person controls. All posters have been positioned on the walls, although their final place has not yet been finalized - there have also been some tests with the hue, saturation and brightness of the illustrations. Concerning the texts only a portion have been written, although this is not important for now because these are small and will take little time to produce - to test the user’s interaction with textual items we will use a sample and replace it later with the true examples. About the movies, we have done a little exploration and found four already - Quo Vadis (1913), In Hoc Signo Vinces (1913), Gli ultimi giorni di Pompei (1913) and L’Emigrante (1915) -, but there is still a lot of work to be done in this regard: we have not only to search for the other movies - even though we understand some don’t

90

A. Martins et al.

Fig. 9. From upper-left corner to lower right corner: first section of the room (north view); middle section (north view); middle section (south view); and third section (north view) (Source: authors)

have any copy available - but also to find the versions with the best image resolution possible. Finally, there’s already been some work done with interactive elements: in this field we already tested a simple method to connect a poster to the respective movie by creating a button, changing the button’s image to a 2D sprite of that film’s poster, and finally using a basic script in C# to connect to an online link of the movie. We also embedded that same movie - Quo Vadis (1913) - in an object and displayed it on one of the walls of the room, playing in a loop and without audio (see last frame from Fig. 9).

4 Future Work In general, this describes in simple terms what the current stage of the prototype looks like. We can say that this work is still in its infancy and has a lot of space and time to evolve. For now, the main goal for us is to deliver an interesting 3D real time experience that can offer materials which, due to space limitations and preservations issues, cannot be presented in the 2024 museum’s exhibit, be it the incomplete posters, extra textual information, or the possibility to watch easily and quickly one of the movies from early Italian cinema that the respective posters illustrate. And regarding the two incomplete posters shown in the virtual exhibition - for the movies Quo Vadis (1913) and Gli ultimi giorni di Pompei (1913?) -, very recently we discussed the possibility of showing not the original digital images which we have in our possession, but to try to produce a “restoration” of both posters. Since both are missing the half bottom page, we have started to experiment with A.I. image synthesizers like Stable Diffusion or Adobe Firefly to fill the missing parts. The objective is not to recreate the full poster as they were originally, but to use creative and artistic liberties to imagine different possibilities for those sections (see Fig. 10).

From Print to Pixels: Prototyping a Virtual Exhibition

91

Fig. 10. First trials to fill the half-bottom page of the poster Gli ultimi giorni di Pompei, using the inpaint(ing) mode(l) in Stable Diffusion (left) and the Generative fill function in Adobe Firefly (right) (Source: authors)

Besides this, even though we already concluded that the model of the virtual room should be somewhat faithful to the original venue for the first exhibition, for future works we would like to play with different designs for this place and other museum rooms. This means experiment with the scan captures by exploring more abstract environments. We already have begun toying with this idea by making 3D renderings of this same room with different meshes, for example one filled with dots (see Fig. 11)8 . On a final - and more formal - note, after the presentation of the virtual and physical exhibitions in March 2024, we will continue to work with the collection and begin structuring a project for the online platform. The bigger project is aimed at obtaining national funding through FCT’s grant system. This will permit us to get financial aid and more human resources which, in turn, will encourage us to develop more ambitious multimedia objects. We will also keep working on straightening the ties with the Faro Municipal Museum and see if they can support this team on future virtual exhibitions, perhaps even about other subjects besides the poster collection which are adequate for this type of representation. Overall, until the end of the first semester of 2024, CIAC will lay some of the groundwork - which also includes the research about the posters and the various scientific communications to disseminate the findings about the collection - for this bigger project which entails the creation of an online platform that will gather much of the knowledge and creative production about this important and singular heritage.

8 To test different designs of the room we have been using a different software, namely cables.gl

[4]. We are also doing some experimentation with this program because it also allows for fast prototyping and adjustments of interactive content. It also enables different members of the team to be working simultaneously in the same design. Cables might also present a possible path to pursue in future works.

92

A. Martins et al.

Fig. 11. 3D rendering of the Italian poster exhibit room with a mesh of dots. Program made with cables.gl (Source: authors)

Acknowledgments. This work is supported by national funds through FCT – Fundação para a Ciência e a Tecnologia, I.P., in the framework of project UIDB/04019/2020.

References 1. Agência Lusa: Museu Municipal de Faro mostra 33 cartazes raros de cinema francês. Observador (2021) 2. https://observador.pt/2021/07/30/museu-municipal-de-faro-mostra-33-cartazes-raros-de-cin ema-frances/ 3. Aiello D, Fai S, Santagati C (2019) Virtual museums as a means for promotion and enhancement of cultural heritage. Int. Arch. Photogrammetry, Remote Sens. Spat. Inf. Sci XLII-2/W15:33–40. https://doi.org/10.5194/isprs-archives-XLII-2-W15-33-2019 4. Barceló, J., Forte, M. & Sanders, D.: Virtual reality in archaeology. british archaeological reports, england (2000) 5. Bento, R: “Foram-se” os filmes, ficaram os cartazes. «Peças de arte» vão estar expostas no Museu de Faro. Sul Informação (2021) 6. https://www.sulinformacao.pt/2021/07/foram-se-os-filmes-ficaram-os-cartazes-pecas-dearte-vao-estar-expostas-no-museu-de-faro/ 7. cables Homepage: https://cables.gl/home , Accessed 20 May 2023 8. Câmara Municipal de Faro: Cinema em Cartaz. Caleidoscópio, Lisboa, Portugal (2021). ISBN: 9789896586539. https://caleidoscopio.pt/products/cinema-em-cartaz 9. Câmara Municipal de Faro: Coleção Cartazes de Cinema e Circo, https://www.cm-faro.pt/pt/ 1067/colecao-cartazes-de-cinema-e-circo.aspx, Accessed 20 May 2023 10. Câmara Municipal de Faro: Museu Municipal de Faro, https://vr.unit360.pt/mmfaro/index. htm , Accessed 20 May 2023

From Print to Pixels: Prototyping a Virtual Exhibition

93

11. CIAC: Seminários De Ciências Da Comunicação, https://ciac.pt/noticias/seminarios-de-cie ncias-da-comunicacao-18/19353, Accessed 20 May 2023 12. CIAC: XVI Jornadas CIAC, https://esec.ualg.pt/sites/default/files/inline-files/Programa_ XVI%20Jornadas%20do%20CIAC.pdf, Accessed 20 May 2023 13. Ciurea C, Zamfiroiu A, Grosu A (2014) Implementing mobile virtual exhibition to increase cultural heritage visibility. Informatica Economic˘a 18(2):24–31 14. DMAD: Vasco Ramalho – [IN]MarImmersive Waves, https://dmad.ciac.pt/vasco-ramalho/, Accessed20 May 2023 15. Desvallées, A. & Mairesse, F.: Key Concepts of Museology. Armand Colin, France (2010) 16. Dumitrescu, G., Filip, F. G. , Ioni¸ta˘ , A. & Lep˘adatu, C.: Open Source Eminescu’s Manuscripts: A Digitization Experiment. Studies in Informatics and Control, 19(1), 79-84 (2010). 17. En Blanco y Negro - Clásicos del cine: QUO VADIS - 1913, https://www.youtube.com/watch? v=KEv1LWzXXOY, Accessed 20 May 2023 18. Ferdani, D., Pagano, A. & Farouk, M.: Terminology, definitions and types for virtual museums. V-Must: Virtual Museum Transnational Network (2014) 19. Filip FG (1996) Information technologies in cultural institutions. Stud. Inform. Control 6(14):385–400 20. Gao, K. Y, Gao, Y., He, H., Lu, D., Xu, L. & Li, J.: NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review (version 4). Arxiv, computer science - computer vision and Pattern Recognition (2023). https://arxiv.org/abs/2210.00379 21. kulla: About, https://kuula.co/about , Accessed 20 May 2023 22. Natale, M. T. , Fernandez, S. & Lopez, M. (eds.). Handbook on Virtual Exhibitions and Virtual Performances, version 1.0. Indicate, Italy (2012) 23. OnCyber Homepage: https://www.cyber.xyz/, Accessed 20 May 2023 24. OnCyber: 1) What, https://docs.oncyber.io/overview/1-what, Accessed 20 May 2023 25. Pescarin S (2014) Museums and virtual museums in Europe: reaching expectations. Ricerca Scientifica e Tecnologie dell’Informazione 4(4):131–140 26. Polycam Homepage: https://poly.cam/, Accessed 20 May 2023 27. Tavares, M., Silva, B. M., Carrega, J., Soares, A. S., Dourado, P., Costa, S. & Valente, A. C.: Cinema e Média Arte no CIAC. In Valente, A. C. (coord.). Avanca | Cinema 2022, pp. 867–873, Edições Cine-Clube de Avanca, Avanca (2022) 28. Unity: Behind the Scenes on “Baymax Dreams” (Made with Unity for Disney Television Animation), https://www.youtube.com/watch?v=DpuUnNLZf5k&t=3s, Accessed 20 May 2023 29. Unity Learn: Create a Personal 3D Gallery Project with Unity, https://learn.unity.com/pro ject/create-a-personal-3d-gallery-project-with-unity, Accessed 20 May 2023 30. Unity Play: Faro Museum Poster Collection: Virtual Exhibition (Alpha Version), https://play. unity.com/mg/other/exposicao-virtual-cartazes-de-faro-v-0-1, Accessed 20 May 2023 31. Universidade do Porto: Aula Aberta “A coleção de cartazes do Museu Municipal de Faro. Um património ímpar em Portugal”, https://sigarra.up.pt/flup/en/noticias_geral.ver_noticia? P_NR=139866, Accessed 20 May 2023

Identifying Design Guidelines of Interactive Whiteboard Interfaces Through Usability Evaluation Raiyan Jahangir(B) , Zinia Sultana , Nasif Shahriar Mohim , Md Tariquel Islam , Mahedi Kamal , Iyolita Islam , and Muhammad Nazrul Islam Department of Computer Science and Engineering, Military Institue of Science and Technology, Mirpur Cantonment, Dhaka, Bangladesh [email protected], [email protected]

Abstract. Interactive whiteboards are used to make classroom teaching and discussion easier and are used in all levels of educational classes, corporate conference rooms, coaching studios, television studios, and the like. Interactive whiteboards need to be usable and user-friendly for the end-users having diverse backgrounds (e.g., education, familiarity with such systems, etc.). Thus, the objectives of this study are to evaluate the interfaces of the whiteboards to reveal their usability problems and to provide possible design guidelines to improve the usability of such embedded systems. To attain these objectives, two interactive whiteboards were evaluated through a heuristic evaluation (HE) and a user study. The HE was carried out with four experts, followed by a user study replicated with seven participants. As outcomes, the studies revealed several usability problems grouped into seven categories; which include, for example, inconsistent use of icons, deep navigation, lack of accelerators, and absence of flexibility. Design guidelines along with the prototypical design were then proposed to solve the revealed problems and to improve the usability of interactive whiteboards. Keywords: Interactive Whiteboard · Usability Evaluation Evaluation · User Study · Design Guidelines

1

· Heuristic

Introduction

An interactive whiteboard is a smartboard that may respond to human input directly or via other peripheral devices. Conventional whiteboards have been used frequently for idea generation, group brainstorming, information presentation, and communication. Two different types of interactive whiteboard technologies include large-size panels with a standalone system on a chip and combinations of video projectors that employ a tablet or other device with drivers for user input. Operation methods frequently include magnetic or ultrasonic pens, c The Author(s), under exclusive license to Springer Nature Switzerland AG 2024  N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 94–107, 2024. https://doi.org/10.1007/978-3-031-47281-7_8

Interactive Whiteboard Usability

95

infrared or resistive touch, or both. Keyboards may be redundant because character recognition and writing are typically done digitally. Interactive whiteboard software frequently includes easy-to-use charts, polls, graphs, and virtual replicas of standard classroom instruments like rulers, compasses, and protractors. They can play various media and afford teachers to provide interactive lessons to their pupils. The production of interactive whiteboards, which are used for conference meetings, coaching, engineering, and strategic project planning, is a billion-dollar industry. According to the Becta Harnessing Technology Schools Survey, 98% of British secondary schools and 100% of British primary schools used interactive whiteboards. Interactive whiteboards are now prevalent in both the corporate and educational sectors. Different interactive applications, tools, and approaches are utilized to simplify the presentation and summarization of challenging ideas, imitate teamwork, and improve teaching techniques [14]. In meetings between corporate workers and business clients, for instance, it is usual practice to sketch business processes using traditional modeling notation. Interactive whiteboards are very effective in teaching mathematics and other fields of science [9]. Students from pre-school to university level agreed that they understand a lecture better if the teacher uses an interactive whiteboard instead of a traditional one [18]. A comparison between a conventional and an interactive whiteboard showed that interactive whiteboards have powerful visual tools to display, making it easier for students and teachers to interact [27]. On the other hand, traditional whiteboards can only be used for writing manually with markers and as a background for slide projection using a projector, but can’t be interacted with hands. Interactive whiteboards enhance learning experiences, promote collaboration and interaction, and improve student engagement [4]. A considerable amount of research has been conducted to examine the effectiveness of interactive whiteboards and other interactive teaching tools on student learning outcomes [8,16]. However, despite the widespread adoption of interactive whiteboards in various settings, research has yet to focus on designing a usable and intuitive interface for these devices [20]. Usability is a quality attribute for any digital system and has become a crucial factor in the design and development of interactive whiteboards [3,21]. Several other studies have emphasized the importance of usability on interactive system design like AR/VR based system [10,25], Automated Teller Machine (ATM) [12], IoT based gaming system [13]. Research has shown that the usability of an interactive whiteboard can significantly affect its adoption and integration into different educational and corporate environments [24]. As such, the usability of interactive whiteboards has become a concern for people of different ages, cultures, backgrounds, lifestyles, and disabilities. Therefore, it is essential to consider usability issues in the design and development of interactive whiteboards to ensure their effectiveness and ease of use in various contexts. Therefore, the objectives of this research are to evaluate the usability of an interactive whiteboard and to propose a set of design considerations for designing an intuitive interactive whiteboard with enhanced usability. To attain these objectives, two usability evaluation studies involving eleven participants (four

96

R. Jahangir et al.

experts and seven target users) were conducted, and based on these studies, some usability issues like inconsistent use of icons, deep navigation, lack of accelerators, absence of flexibility and so on have been identified, and design guidelines have been recommended for enhancing the usability and ease of use of the interactive whiteboard systems. The remaining portions of the paper are organized as follows. The related works are discussed in Sect. 2. Section 3 covers the overall methodology in detail. Section 4 describes the research findings. Section 5 proposes design guidelines. Section 6 brings the paper to a close.

2

Literature Review

There has been a lot of research on creating interactive whiteboard surfaces in recent years. The development of touch-enabled surfaces has been a popular research area in recent years, with several innovative methods being introduced to enable human-computer interaction without the need for physical input devices. Hu et al. [11] introduced a unique method for detecting touch events on projected surfaces based on button distortion, enabling finger-touch interaction. Similarly, using merely a projector and a camera, Dai et al. [6] created a system that enables any tabletop surface to be converted to a touch-enabled surface by projecting contents and applying computer vision algorithms. Mistry et al. [17] introduced a wearable gesture interface [15]. It enables the projection of content onto any surface for user interaction. To interact with the board, fingers are wrapped in a marker, such as color tape, and a portable projector-camera system detects the color marker gesture [1]. The advancement of interactive tabletop systems has led to the development of new techniques for human-computer interaction that go beyond traditional input devices. In recent years, researchers have introduced several innovative approaches to enable interaction with projected material on tabletops. Chen et al. [5] proposed an IR pen-based tabletop system that uses the Wii remote to follow the IR pen movement and interact with the projected material. They created a revolutionary technique for this system that allows to identification and tracking of numerous IR pens concurrently. Wilson [26] developed a portable interactive tabletop projection system where a camera-projector pair is used. In addition to the newly introduced optical flow technique for interacting with the information on the screen, touch detection algorithms based on the observation of shadows were used. With the widespread adoption of large displays, there is a growing need to understand how they affect productivity and the overall user experience. Robertson et al. [22] examined the effects on productivity and methods for improving the large display user experience as huge monitors. According to their studies, users can create and manage many more windows on larger displays and participate in more sophisticated multitasking behaviors. In another study, Shannon and Cunningham [23] made children with Autism Spectrum Disorder interact

Interactive Whiteboard Usability

97

with an interactive whiteboard and checked how they adapted and got accustomed to using it. Based on their observations, they also proposed a design for the interface. Most of the above research has worked to make the interactive whiteboard better as a technology, increasing its functions, increasing its impact on various fields, improving its accessibility, etc. Very few researchers have worked on designing a suitable interactive whiteboard for ease of use and making it usable and useful to different sorts of users.

3

Methodology

Two interactive whiteboards were selected for evaluation which were available in the authors’ institution. The whiteboards had slightly different interfaces. However, both interactive whiteboards had options on the left and right sides of their interfaces that carry out different operations. The options were mainly for a software application called “Drawview” which is used to operate on interactive whiteboards. The software itself had many options which carry out different operations. Some of the options on the interactive whiteboards and the software carry out the same operations. The options include selecting pen options for marking on the board, selecting an eraser, selecting geometric shapes, selecting mathematical tools like a compass, or protractor, selecting some games to play, and so on. The software had to be installed on a computer and then projected on the interactive whiteboard by a projector to be used properly. The experiment was conducted in two phases. In the first phase, four evaluators evaluated the usability of the interfaces of the two interactive whiteboards based on heuristic evaluation. They examined the whiteboard interface and its compliance using Neilsen and Molich’s ten usability principles [19] (numbered H1-H10) as a way to diagnose usability problems. Each of them found and listed a number of usability problems. Then all the usability problems were aggregated and categorized according to their severity. The severity ratings were considered from 1 to 4, where 1 is “Cosmetic Problem”, 2 represents “Minor usability problem”, 3 is “Major usability problem” and 4 refers to “Catastrophic problem”. Then in the second phase, seven users were asked to try out different features of the system in the form of tasks. Problems were identified based on their activity, mood, and feedback. Finally, all the problems from the two phases were listed. Details of the two evaluations are mentioned as follows: 3.1

Heuristic Evaluation

Four evaluators conducted heuristic evaluations on the two interactive whiteboard interfaces highlighting any usability problems. Each of the evaluators was given a week to conduct the heuristic evaluation. Their age ranged from 30 to 42 years old. They all are graduates of computer science and have come across a set of human-computer interaction-related courses. The first evaluator was a human-computer interaction researcher with over ten years of experience in the

98

R. Jahangir et al.

field of user-centered design. The second evaluator was a usability specialist with a background in industrial design and user experience. The third evaluator was a software engineer with extensive experience in developing web-based applications, while the fourth evaluator was a user experience designer with experience in conducting usability studies and designing intuitive interfaces. All four evaluators were experts in usability, with a strong understanding of user-centered design principles and heuristics. The reason for choosing heuristic evaluation is that it is a relatively quick and cost-effective method for evaluating the usability of an application. It does not require participants or users to test the application, which can be timeconsuming and expensive. During the evaluation, the evaluators strived to find out the heuristics that were violated and examined the severity of each heuristic violation. Each evaluator listed a number of usability problems with the interface of both interactive whiteboards and gave a severity rating. The authors checked out all the problems one by one from the lists given by the evaluators and made a common list. The problems that were common to any two or more evaluators were counted as one single problem. The other problems remained as they were. In this way, a total of 23 problems were identified by analyzing the two interactive whiteboards. 12 belonged to the first interactive whiteboard, and the rest belonged to the second. The listing of each problem with the heuristic violations, their severity rating, and recommendations to solve the problem is given in Table 1. The problems of the first interactive whiteboard are numbered W1-P1, W1-P2, and so on, while those of the second interactive whiteboard are numbered W2-P1, W2-P2, and so on. The final severity rating of each problem was decided after a discussion with the experts. Two mentionable examples from Table 1 are given below: – In W1-P10, a “Monitor” icon is found to be used for two functions at two different places. One function is used to unlock the screen (the lower icon in Fig. 1a), and another one opens the desktop screen (the upper icon in Fig. 1a). This problem violates H4 (Consistency and standards) because the same icon is being used for different functions, which violates the icon’s consistency. The severity rating is given 3 because this problem was identified by all four evaluators, and this is likely to confuse users while they are using it. – In W1-P5, no hover-over is seen on the whiteboard icons when they are clicked (Fig. 1b). This violates H1 (Visibility of System Status) because the users may get confused as to whether the icons have been selected or not. The severity rating of this problem is given as 1 because only one of the evaluators has mentioned it as a problem. The other evaluators did not find it as a major problem.

3.2

User Study

Participants’ Profile: The user study was conducted by 7 participants, and they were made to carry out a number of tasks. Among the 7 participants, 4 were

Interactive Whiteboard Usability

99

Table 1. Heuristic evaluation of the two interactive whiteboards Problem No.

Problem Statement

Heuristic Violation

Recommendations

Severity Rating

W1-P1

The options on the sides of the interactive whiteboard have only icons. No labels are given on them and so it is hard to understand the operations carried out by these options in the first instant. The users may have to guess the operation of the options or try them out. Operations of some options are confusing and users might find it difficult to understand their purposes. There are a variety of options to choose the type of “pen” to write. However, due to the colored icons, the different pen options are hard to distinguish and it seems more like it changes the color of the pen instead of the type. When the software is opened, the color of the pen is red. The whiteboard supports only two color-changing options, black and blue. When either of the colors is selected, there’s no way to go back to the red color which can be a restriction to the user’s freedom. No hover-over the whiteboard icons which can make the users confused as to whether the icons have been selected or not. Option of both the software and the whiteboard can be used to carry out operations on the interface. There are some operations like “Pen” and “Eraser” which are available in both. However, when the options of the software are selected, the options on the whiteboard can’t be selected until the options on the software navbar have been deselected. That is, the software options are given more priority than the whiteboard options. However, this is not the case for all options of the software. So, this results in ambiguity, confusion, and error. For carrying out mathematical operations, a separate interface is used. However, there is no option to minimize it and work somewhere else. Each time, the interface has to be closed before going to another interface. The “Keyboard” interface that can be used on the board is very large and it takes a lot of space from the screen. It can’t be adjusted. This results in restriction of using the interface. Even though the keyboard interface can be faded, when the options underneath are pressed, the keyboard interface pops back up. The keyboard also seems to have a lot of keys that are not very necessary. There’s a “New Folder” option on the screen which when pressed creates a new folder but there’s no feedback shown as to whether this function has been carried out or not. A “Monitor” icon is found to be used for two functions at two different places. One function is used to unlock the screen and another one takes to the desktop screen. The “Laser Pen” option is not working as a typical laser pen. Rather, it works like a pen with constantly-changing ink colors. There’s no option for Help in case users not understanding how to carry out a function. Same statement as W1-P5

H6

Give labels that will describe the icon’s functions to avoid confusion.

2

H2 H4

Give icons familiar to the intended audience.

2

H8

Change the color background of the icons which can solve this confusion.

3

H3

Give options for undoing. Allow the user to select a variety of colors.

3

H1

Create a system of hover-over feedback for the users.

1

H5 H7

Allow the user to select any options without such hassles.

3

H3

There should be a “Minimize” option on every interface.

2

H5 H8

The keyboard size should be small, adjustable, and with only the necessary keys.

2

H1

Show some sort of dialog box or a progress bar that will indicate that a new folder has been created.

1

H4

Use different icons for different functions.

3

H4

Use proper icons for functions.

1

H10

There should be a help or documentation option for every operation.

2

H1

Recommendation of W1-P5 will also solve this problem.

W1-P2

W1-P3

W1-P4

W1-P5

W1-P6

W1-P7

W1-P8

W1-P9

W1-P10

W1-P11

W1-P12

W2-P1

1

continued

100

R. Jahangir et al. Table 1. continued

Problem No.

Problem Statement

Heuristic Violation

Recommendations

Severity Rating

W2-P2

Option of both the software and the whiteboard can be used to carry out operations on the interface. There are some operations like “Pen” and “Eraser” which are available in both. However, when the options of the software are selected, the option is colored differently to indicate the selected button. At this time, if an option from the whiteboard is selected, it is selected and works but the previously-selected option from the software isn’t shown to have been deselected which can create confusion for the user. Same statement as W1-P7 Same statement as W1-P8

H5 H1

Show the system status properly.

3

Recommendation of W1-P7 will also solve this problem. Recommendation of W1-P8 will also solve this problem.

2 2

Recommendation of W1-P9 will also solve this problem. Recommendation of W1-P10 will also solve this problem. Recommendation of W1-P11 will also solve this problem. Create some other ways to play the game. Make the interface respond quickly to touch.

1 3 1 2

Create familiar icons for the intended audience.

1

There should be a help or documentation option for every operation.

1

Recommendation of W1-P12 will also solve this problem.

2

W2-P3 W2-P4 W2-P5 W2-P6 W2-P7 W2-P8

W2-P9

W2-P10

W2-P11

H3 H5 H8 Same statement as W1-P9 H1 Same statement as W1-P10 H4 Same statement as W1-P11 H4 In “Number Bubble Games”, the target is to touch and destroy H1 floating H4 bubbles. However, the touch takes some time to effect but that is enough to lose the game. The slow response to touch makes the game difficult to play. Confusing buttons in the “Fractional” Game. The buttons are H2 unclear as to what they do. Not understanding how to carry out operations (wiring the H10 components, deleting the wires, etc.) in the “Circuit” Game. Same statement as W1-P12 H10

Fig. 1. Two usability problems found by heuristic evaluation

male and 3 were female. Their age ranged from 25 to 35 years old. 5 of them were faculty members, and the other 2 were lab assistants of the authors’ institution. All the participants are engaged with academic work and are supposed to work

Interactive Whiteboard Usability

101

with whiteboards. All of them have used traditional whiteboards and have some prior knowledge about how interactive whiteboards might work. However, none of them have actually used an interactive whiteboard to teach in a class. They also have no familiarity with using the two interactive whiteboards that were used in the user study. Study Procedure: The participants were brought to the human-computer interaction lab at the authors’ institution. The participants were briefed about the purpose of bringing them for user testing as part of usability evaluation and reminded that it is volunteer participation. The authors also took written consent from them regarding using them as participants in this user study and took their biographical data. Then a demonstration of the usage of the interactive whiteboards was shown to the participants. Finally, they were given to do a set of tasks. A total of 13 tasks were selected for carrying out the user study. The tasks are shown in Table 2. The user study was carried out one-onone. The participants were observed by 2 observers while they were carrying out the required tasks. The observers were there to take field notes of the observed process. Each user evaluation took approximately 30 min. At this time, many objective data, as well as subjective data, were collected. This is done because the aim of this research is to assess the user test with effectiveness, efficiency, and satisfaction. According to ISO-9241-11, usability is defined in terms of effectiveness, efficiency, and satisfaction in a specific usage context [2]. The purpose is to highlight how usability results from interaction rather than product quality. Effectiveness is measured by how precisely and completely users accomplish their intended objectives. It can be determined by measuring the completion rate. Efficiency measures the effort needed by users to accomplish their objectives. It is defined by task time. Specifically, how long (in seconds or minutes) it takes a person to perform a task correctly. Satisfaction defines the comfort and acceptability of use [7]. Finally, after the completion of the user study, questionnaires were provided to the participants to get their feedback about the usability of the interactive whiteboards. Their feedback was noted down. User satisfaction was determined from the feedback. Result Analysis of User Study: The information regarding each participant’s task completion was aggregated, and then the average of each of the data collected, for example, the time taken to complete each task, the number of times asked for help, and other necessary data, were calculated. The success rate for each participant was determined by the number of attempts taken by them to complete a task. The overall success rate of a task was determined by averaging the success rate of each participant. The data are presented in Table 3. From this table, a number of results can be concluded. Some of them are: – Task 11, 12, and 13 (Rows marked red in Table 3) took the most time to complete. This is because more than 4 clicks are needed to complete the task, and because of the confusing interfaces, the number of wrong navigations, attempts, and asking for help is high.

102

R. Jahangir et al. Table 2. The list of tasks for user study Task No. Task Description 1

Select the pen option, draw something, and then erase

2

Select an object drawn and drag it

3

Lock the screen and undo it

4

Find the options to undo and redo

5

Go to the menu and select a type of pen

6

Change the color of a pen

7

Select a shape from the menu and draw it on the screen

8

Find out the compass and draw a circle

9

Find out the protractor and draw an arc

10

Find out an image resource and drag it on the screen

11

Play the “Fractional” Game

12

Play the “Number Bubbles” Game

13

Carry out a mathematical operation in “Math” Function

– Task 11 (Play the “Fractional” Game) is taking the longest time to complete (Row marked dark red in Table 3). The number of clicks to complete the task took almost thrice the number of optimal clicks. Most of the users failed to complete it on their first attempt, and wrong navigation was the most in this task. This task also has the lowest success rate. The reason for this low success rate is that to complete the task, deep navigation and clicks are required. Also, the interface is not friendly, and users face difficulty understanding how to do the task because of the lack of any help manuals. – Task 5 (Go to the menu and select a type of pen) and 6 (Change the color of a pen) has a success rate of 100% with 0 wrong navigations and 0 requests for help (Rows marked light green in Table 3). However, more clicks were required than needed to complete the task because the users were finding it hard to understand the icons of the buttons needed to complete the tasks. Moreover, the information obtained from the feedback of the participants tells that 55.7% of the participants are satisfied, 42.85% will use it in the future, 57.14% will recommend others in the future, and 41.24% said the whiteboards are easy to use. The conclusion to this user study is that the interfaces of the interactive whiteboards have several places for improvement.

4

Research Findings

Based on the heuristic evaluation and user study of the interfaces of the two interactive whiteboards, a considerable number of problems have been identified. The authors tried to identify the potential reasons for each of the usability problems. In the case of heuristic evaluation, the authors took the suggestions

Interactive Whiteboard Usability

103

Table 3. Aggregated result of user study Average time Task

taken to complete the task (second)

Average number of clicks

Total number of optimal clicks

Average number of attempts

Average

Average

number of

number of

Success

wrong

times asked

Rate

navigation

for help

Select the pen option, draw something and then

21.2s

6

4

1.14

0.14

0.42

100%

69.4%

erase Select an object drawn and

29.4s

10

4

2

0.85

1.57

14.14s

6

2

1

0.14

0.28

100%

25.2s

7

3

1.42

0.71

0.85

85.71%

21.5s

9

4

1

0

0

100%

drag it Lock the screen and undo it Find the options to undo and redo Go to the menu and select a . type of pen Change the color of a pen Select a shape from the menu

9.71s

4

2

1

0

0

100%

10.14s

6

3

1.28

0.14

0.14

100%

61.14s

22

4

1.85

1.28

1.71

67.85%

21,42s

6

3

1

0

0.71

100%

and draw it on the screen Find out the compass and draw a circle Find out the protractor and draw an arc Find out an image resource

15.85s

7

3

1.14

0.71

0.71

92.8%

82.14s

37

11

1.28

2

3.57

42.85%

43.29s

19

5

1.14

0.29

1.14

92.86%

42.43s

22

4

1.28

0.43

2.14

89.29%

and drag it on the screen Play the “Fractional” Game Play the “Number Bubbles” . Game Carry out a mathematical operation in “Math” Function

of the heuristic evaluators as to how they identified the problems, why they might consider these as problems, and what the reasons might be causing the problems. Their suggestions were noted down. In the case of the user study, noticeable information was analyzed. For example, why some tasks were taking longer, why participants were asking for help in some tasks, and why they were not able to accomplish certain tasks with ease. Feedback from the participants was also taken to hear their opinions. After studying each of the problems, some common reasons have been identified from the results of both evaluations. The reasons have been listed and described below: – Lack of Instructions: A major reason for the users’ and the evaluators’ hassle in understanding the interactive whiteboard functions and operations is due to the lack of instruction options for all the features. This is a major design issue and has to be fixed. – Inconsistency in Icons: Usage of inconsistent and unfamiliar icons for buttons has resulted in users’ and evaluators’ misunderstanding of the button functions, and this leads to errors. It is proposed to use familiar icons for buttons. – Absence of Fexibility and Ease of Use: Navigating and switching among different interfaces of the application takes a lot of clicks, and most of them look unnecessary. This problem could be removed by introducing flexible interface switching and reducing the number of clicks.

104

R. Jahangir et al.

– Lack of Feedback: Some operations, when carried out, do not show any sort of feedback, which results in users’ unawareness of the completion of the operations. Appropriate feedback should be provided. – Unnecessary Deep Navigation: Some features, like the games, are provided under sections of some other features. So, navigating to those features is tough, and many users miss these features on their first try. These features should be available as separate independent buttons visible to the users. – Lack of Accelerators: The lack of accelerators has resulted in many users becoming annoyed over the use of these whiteboards. Some traditional accelerators should be added. – Lack of System Status: In some operations, there’s a lack of system status that leads to errors.

5

Proposed Design Guidelines

The authors proposed a set of design guidelines using which the usability of the interactive whiteboards could be improved. They are as follows: – Labels should be added to every icon, and the icon should be clear and familiar (Fig. 2a). – For choosing the color of the pen, distinguish the icons using background color (Fig. 2b). e.g.: Red background for the red pen, Green background for the green pen, and Blue background for the blue pen. – Icons should have a hover-over effect to indicate whether it is selected or not (Fig. 2c and 2d). – There should be a minimize and cross button for a separate interface (Fig. 2e). – The same icon should serve the same purpose in the software and hardware portion. The icon should be consistent and standard (Fig. 2f). – Separate interfaces should be resizable and moveable so they don’t overlap (Fig. 2g and 2h). – There should be a help and documentation option for options whose operations are confusing (question-marked option in Fig. 2i). – If any uncommon operations or icons exist in any game or other feature, there should be a textual description under it (Fig. 2j). – Allow quick and easy navigation by introducing buttons for features on the main page. Avoid deep navigation.

6

Discussions and Conclusions

This research aimed to evaluate the usability and intuitiveness of interactive whiteboards and propose design guidelines to improve their usability based on the results. The studies were conducted on two interactive whiteboards. At first, heuristic evaluation was carried out. Then the whiteboards were tested by users.

Interactive Whiteboard Usability

105

Seven participants who had some ideas about interactive whiteboard functionality acted as users. The problems from the heuristic evaluation and the user study were then aggregated, and the set of final problems was listed. Analyzing the problems, it is concluded that the second interactive whiteboard is better than the first one based on the number of problems identified as well as the severity of each of these problems. However, both interactive whiteboards have huge room for improvement in their designs. Design guidelines were proposed to solve these problems and improve the usability of interactive whiteboards. These guidelines could be used by interactive whiteboard designers to design improved and more user-friendly interactive whiteboards. The previous studies focused on exploring the comparison between traditional whiteboards and interactive whiteboards [18] and why interactive whiteboards are better than traditional whiteboards [27]. Then, the impact of interactive whiteboards on teaching various subjects like mathematics and science has been discussed [9]. After that, the exploration of finding the acceptance of interactive whiteboards by students, teachers, and people of other concerned professions has been carried out [24]. Finally, the usability of interactive whiteboards was explored in another study [8]. From the above discussion, it is noticed that none of the research focused on finding out usability problems of the interface of interactive whiteboards and recommending design guidelines. Therefore, in

Fig. 2. Prototypical designs based on the proposed recommendations

106

R. Jahangir et al.

contrast to the previous research, the focus and contribution of this research were to find out the usability problems of interactive whiteboard interfaces and recommend design guidelines. However, this study also has limitations. The study was conducted with a few evaluators and users. The study was also conducted with time constraints. The heuristic evaluation itself is a limitation as it may not reflect the real users. Furthermore, the evaluation study was conducted in a laboratory environment. Future works should involve a large number of participants from different professional groups to participate with more experiment time. In this study, only design guidelines were given. In the future, an intuitive and usable interface for an interactive whiteboard will be designed.

References 1. Argyros AA, Lourakis MIA (2006) Vision-based interpretation of hand gestures for remote control of a computer mouse. In: Huang TS, Sebe N, Lew MS, Pavlović V, Kölsch M, Galata A, Kisačanin B (eds) ECCV 2006, vol 3979. LNCS. Springer, Heidelberg, pp 40–51. https://doi.org/10.1007/11754336_5 2. Bevan N, Carter J, Harker S (2015) ISO 9241-11 revised: what have we learnt about usability since 1998? In: Kurosu M (ed) HCI 2015, vol 9169. LNCS. Springer, Cham, pp 143–151. https://doi.org/10.1007/978-3-319-20901-2_13 3. Bhowmik M, et al (2023) Evaluating usability of mobile financial applications used in Bangladesh. In: Martins N, Brandão D (eds) Advances in design and digital communication III: proceedings of the 6th international conference on design and digital communication (Digicom 2022), 3–5 November 2022, Barcelos. Springer, Cham, pp 161–176. https://doi.org/10.1007/978-3-031-20364-0_15 4. Campbell M, Detres M, Lucio R (2019) Can a digital whiteboard foster student engagement? Soc Work Educ 38(6):735–752 5. Chen R, Chen PJ, Feng R, Liu Y, Wu A, Mazalek A (2014) Scisketch: a tabletop collaborative sketching system. In: Proceedings of the 8th international conference on tangible, embedded and embodied interaction, pp 247–250 6. Dai J, Chung CKR (2013) Touchscreen everywhere: on transferring a normal planar surface to a touch-sensitive display. IEEE Trans Cybernet 44(8):1383–1396 7. Dix A, Finlay J, Abowd GD, Beale R (2003) Human-computer interaction. Pearson Education 8. Glover D, Miller D, Averis D, Door V (2005) The interactive whiteboard: a literature survey. Technol Pedag Educ 14(2):155–170 9. Greiffenhagen C (2000) Interactive whiteboards in mathematics education: possibilities and dangers 10. Hasan I, Khan MM, Rahman KT, Mayesha AS, Sultana Z, Islam MN (2022) VR glove: a virtual input system for controlling VR with enhanced usability and high accuracy. In: 2022 25th international conference on computer and information technology (ICCIT). IEEE, pp 137–142 11. Hu J, Li G, Xie X, Lv Z, Wang Z (2013) Bare-fingers touch detection by the button’s distortion in a projector-camera system. IEEE Trans Circuit Syst Video Technol 24(4):566–575 12. Islam A, Sharmee SS, Islam MN, Hasan F, Aziz A (2022) Evaluating the humancomputer interaction problems with ATM interfaces. In: 2022 25th International Conference on Computer and Information Technology (ICCIT). IEEE, pp 506–511

Interactive Whiteboard Usability

107

13. Islam MN, Hasan U, Islam F, Anuva ST, Zaki T, Islam AN (2022) Iot-based serious gaming platform for improving cognitive skills of children with special needs. J Educ Comput Res 60(6):1588–1611 14. Kudale AE, Wanjale K (2015) Human computer interaction model based virtual whiteboard: a review. Int J Comput Appl 975:8887 15. Lech M, Kostek B (2010) Gesture-based computer control system applied to the interactive whiteboard. In: 2010 2nd international conference on information technology (2010 ICIT). IEEE, pp 75–78 16. Linn SE (1997) The effectiveness of interactive maps in the classroom: a selected example in studying Africa. J Geograph 96(3):164–170 17. Mistry P, Maes P, Chang L (2009) WUW-wear UR world: a wearable gestural interface. In: CHI 2009 extended abstracts on Human factors in computing systems, pp 4111–4116 18. Mun SH, Abdullah AH (2016) A review of the use of smart boards in education. In: 2016 IEEE 8th international conference on engineering education (ICEED). IEEE, pp 120–125 19. Nielsen J, Molich R (1990) Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp. 249–256 20. Oliveira E, Vasconcelos E, Sobral E, Bittencourt S, Ramos T, Soares MM (2015) Ergonomic and usability analysis of interactive witeboards in the academic environment. In: Marcus A (ed) DUXU 2015, vol 9188. LNCS. Springer, Cham, pp 208–217. https://doi.org/10.1007/978-3-319-20889-3_20 21. Razzak MA, Islam MN, Broti T, Kamal ES, Zahan S (2023) Exploring usability problems of mhealth applications developed for cervical cancer: an empirical study. SAGE Open Med 11:20503121231180412 22. Robertson G, et al (2005) The large-display user experience. IEEE Comput Graph Appl 25(4):44–51 23. Shannon G, Cunningham SJ (2009) Impact of classroom design on interactive whiteboard use in a special needs classroom. In: Proceedings of the 10th international conference NZ chapter of the ACM’s special interest group on humancomputer interaction. pp 1–4 24. Shen C, Chuang H (2010) Exploring users’ attitudes and intentions toward the interactive whiteboard technology environment. Int Rev Comput Softw 5(2):200– 208 25. Tasfia S, Islam MN, Nusrat SA, Jahan N (2023) Evaluating usability of AR-based learning applications for children using SUS and heuristic evaluation. In: Kaiser MS, Waheed S, Bandyopadhyay A, Mahmud M, Ray K (eds) Proceedings of the fourth international conference on trends in computational and cognitive engineering: TCCE 2022. Springer, Singapore, pp 87–98. https://doi.org/10.1007/978-98119-9483-8_8 26. Wilson AD (2005) Playanywhere: a compact interactive tabletop projection-vision system. In: Proceedings of the 18th annual ACM symposium on User interface software and technology. pp 83–92 27. Yang KT, Wang TH, Kao YC (2012) How an interactive whiteboard impacts a traditional classroom. Educ Change 16(2):313–332

Peer-Produced Archives, Peer-Designed Solutions Pedro Sá Couto1(B)

, Miguel Carvalhais2

, and Pedro Cardoso3

1 Faculty of Engineering, University of Porto, i2ADS – Research Institute in Art, Design and

Society, Porto, Portugal [email protected] 2 Faculty of Fine Arts, University of Porto, i2ADS – Research Institute in Art, Design and Society, Porto, Portugal [email protected] 3 University of Aveiro/DigiMedia, Aveiro, Portugal [email protected]

Abstract. Shadow Libraries are digital media repositories whose primary goal is to disseminate content inaccessible to users otherwise. As a case study we will delve into Portuguese-Student Shadow Libraries which focus on disseminating study materials and producing new resources that serve as primary aid for peers to study and to be prepared for tests, exams and overall assessment moments. This study aims to contribute to designing and developing a framework to preserve and distribute the sensitive collections assembled by/for informal communities where access is enabled by demanded peer production. We intend to work closely with our target audience to achieve this goal, involving community members across various design processes and decision-making moments. We actively question what characteristics make Portuguese-Student Shadow Library relevant to their public and what speculative features present users envision. This paper summarises the findings from a thematic analysis of semi-structured interviews. We sought to present a representative overview of participants’ perspectives, demonstrating that unique disciplines, students, and contexts have different requirements. Presented results collect theoretical approaches to improve Portuguese-Student Shadow Libraries, making them more valuable to their users and reimagining innovative processes to publish and access disseminated study resources. Integrating diverse community members from the preliminary design stage resulted in meaningful contributions and provided knowledgeable insights that will inform the design of our framework. Keywords: Peer-Produced Archives · Academic Communities · Community Design · Knowledge Production · Media Design

1 Introduction Shadow Libraries are digital media repositories whose primary goal is to disseminate content that would otherwise be inaccessible to users. They are a transient response to limited access to published knowledge, defying high prices, paywall limitations imposed by academic publishing companies, location-based restrictions, scarcity of informal publishing spaces, and others. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 108–121, 2024. https://doi.org/10.1007/978-3-031-47281-7_9

Peer-Produced Archives, Peer-Designed Solutions

109

Dissemination of resources across Shadow Libraries does not fit conventional publishing standards. Shadow Libraries rely on informal communities to produce new resources, convert physical documents into digital files, and design, implement, and host archives that allow audiences dependent on digital and physical spaces to access them. Shadow Library collections hold specific conditions and demand distinct ways to interface with their audiences. As a case study, we will focus on Portuguese-Student Shadow Libraries (PSSL). PSSL exist in various environments, from student-led drives to independently run print shops. PSSL gather unofficial vital communities of knowledge production and dissemination. These ecosystems integrate multiple people within the academy, drawing together students, professors and other independent workers. These spaces disseminate study resources among university students, which serve as primary aid for peers to study and to be prepared for tests, exams and overall assessment moments. These initiatives share content adapted to individual curricular needs across several students and disciplines, and rely upon the academic community to produce updated resources, thereby ensuring their relevance to users. PSSL’s main characteristic is that peers create these libraries to satisfy peers’ needs. We identified it as critical to introduce a collective design methodology that gathers needs and feedback and closely works with our target audience. Our research aims to design and develop a framework to preserve and distribute the sensitive collections assembled by/for informal communities where access is enabled by demanded peer production. The present qualitative analysis seeks to comprehend the contributions of the academic community, including professors, students, and others, in such streams of knowledge dissemination and production. Each interview session aimed to understand better the usage of informal archives created to sustain participants’ academic progress. It is vital to clarify the broader perspective on comprehending the term production. The production of resources, such as sebentas—notebooks that compile exams and test questions—and the transcription of recorded classes into text files, are easily identified as material contributions to student drives. We provide a new perspective to such means of production where actions such as commenting, reviewing, updating, digitising or transforming several study materials’ formats are acknowledged participation methods and impact these spaces in meaningful approaches. Access to study materials diverges across several platforms. Student-led drives are digital and more practical to access remotely. Students and Student Groups typically use Google Drive or other centralised platforms to distribute the study material. Hosts organise these folders rigorously, structuring them by course units and curricular years, collecting all the material students need over their academic journey. Student-led drives compile various study materials from sebentas, solved exercises, tests, exams, question compendiums, books, software, etc. These drives may vary in visibility, where access may be limited to a few students, depending on who owns these archives. Print shops exist across physical formats. Often they are independently owned spaces where all university students can print sebentas, question compendiums, class transcripts and buy and photocopy books from universities’ syllabi. Resources are archived within two dimensions in these spaces; some print shops have transitioned to digital archives,

110

P. Sá Couto et al.

where students can search through computer folders for the files they need; others still rely on physical files, which are photocopied on demand to fulfil students’ requests. There are no restrictions, methodologies to learn from, or even right or wrong procedures when hosting a Shadow Library. PSSL are independently maintained, often following the strict rules imposed by centralised platform design. Design components facilitated by centralised platforms and peers’ knowledge to develop new features limits hosts of PSSL when rethinking and developing new features capable of mediating new digital interactions. Students that host such digital spaces make the most of the platform’s affordances, fostering new initiatives to motivate participation and encourage broader academic involvement.

2 Methodology We have adopted a qualitative research methodology that enabled us to integrate participants closely into our study, allowing participants to share their perspectives and experiences (Pathak et al., 2013). This methodology allowed us to collect data through procedures not focused on quantification measures. Even though qualitative research is not focused on achieving statistical representativeness, it enabled us to investigate participants’ individual experiences providing a better understanding of their views (Pathak et al., 2013). We chose semi-structured interviews as the core method for conducting our qualitative research. Interviews allow the interviewer and interviewee to connect through a conversation, facilitating the discovery of new phenomena or clarifying discussions related to the studied subject (Kvale, 1996). Interviews are particularly relevant to our study as they allow us to explore and deepen our understanding of the current publishing panorama. During interview moments, we learned more about the participants while deviating from the interview guide, taking advantage of the interactive nature of this methodology (Edwards & Holland, 2013). Due to the nature of our sample being varied regarding interviewees’ backgrounds and disciplines, we chose this interview kind to transform the interview moments into more intimate, adjusted and meaningful interactions. Even though semi-structured interviews allow improvisation and flexibility, they require highly demanding preparation and training, resulting in many possibilities to gather valuable personal insights (Wengraf, 2006). All the interviews are anonymous to protect interview participants, as interviewed communities may feel worried about sharing specific methods to obtain and circulate study material. 2.1 Sampling Probabilistic sampling is not considered effective in qualitative studies because qualitative research prioritises comprehending human-centred issues rather than obtaining conclusions that may be applied generally (Marshall, 1996). For our study, we followed a purposeful sampling qualitative technique. This technique involves selecting participants who are knowledgeable about the studied phenomenon and capable of contributing to the research with expressive, in-depth data (Gill, 2020).

Peer-Produced Archives, Peer-Designed Solutions

111

All participants were invited to participate without incentives, to avoid bias, maintain ethical standards and respect the participant’s autonomy (Robinson, 2014). Interviews were conducted between January and March of 2023. For our study, we successfully recruited 22 participants. The criteria to select eligible interview participants was divided into two moments: This first step helped us screen a sample that directly answers our research goals. Firstly, we interviewed a diverse sample representative of institutional, geographical and disciplinary contexts. 1. Disciplinary: We have interviewed participants from different faculties and understand how students, alums, professors, and other actors can impact the researched ecosystems, deploy study material using distinct strategies and reconsider digital libraries among them. 2. Institutional: We have interviewed a sample dispersed across multiple institutional settings, from Public Universities, Private Universities and Public Institutes. We have also interviewed students from different institutions that facilitate analogous courses. We intend to compare the differences in distribution and access across these diverse backgrounds. 3. Geographical: We have interviewed students from Portuguese Universities that are not located in city centres. Our goal was to cover how institutional settings more distant from urban areas can promote foreign involvement between the production and study of research material. The second step was instrumental in deepening our research hypothesis with grounded background research. We interviewed a representative portion of the ones involved in Portuguese Students Shadow Libraries streams of access. After analysing the stakeholders within these streams of access and production, we split these interviews into four different groups: 1. Representatives from Student Groups: Students often organise into groups dedicated to compiling, uploading and reviewing the study materials. 2. Students producing: The second group we intended to investigate is students who compile, produce and make available the study material. 3. Students consuming: The third group of actors are students and alums who mainly consume the study material published. 4. Teachers: The last group we will interview are teachers who produce and indicate resources present on PSSL. Our sample brought together students, alums and professors from varied backgrounds, disciplines and institutions. We aimed to combine a diversified sample through external and internal diversification. We ensured the presence of diverse participants inside the research context and integrated the presence of multiple actors inside specific research contexts. We have interviewed participants from all faculties of the University of Porto and single cases from the University of Lisboa, Algarve, Coimbra, Aveiro, Trásos-Montes e Alto Douro and from a European university. We have interviewed students from different courses inside single faculties to document how access to study materials may vary inside homogenous contexts. Our sample was also capable of reporting

112

P. Sá Couto et al.

how teaching methods may differ through interviewing professors from universities and institutes. 2.2 Thematic Analysis After transcribing the interviews, we re-listened to the audio files and cross-checked the transcribed text. This method allowed us to verify their accuracy and ensure no errors in the output text, thus confirming the validity of the interviews. The resulting transcriptions emerged as valuable instruments for future interpretation and comprehension of the analysed interviewee perspectives (McLellan et al., 2003). We have used Thematic Analysis as the primary method to analyse the gathered results. This methodology is a widely employed qualitative research method used to document patterns within the collected data, organising and describing them through a methodical approach (Braun & Clarke, 2006). Tuckett (2005) reports that there is no consensus regarding the thematic analysis’s precise definition and methodological process. The systematic nature of this methodology enables researchers to infer consistent interpretations from the collected data (Alhojailan & Ibrahim, 2012). Thematic Analysis is a flexible methodology that can be employed in a wide range of qualitative data. We chose this methodology because it can generate reliable research hypotheses (Boyatzis, 1998). After preparing all the interviews, we started coding, the process of classifying and segmenting data into meaningful structures that make evident patterns, themes and concepts across the gathered data (Elliott, 2018). We focused on identifying patterns, creating categories and dividing codes into thematic and conceptual groups. We refined the codes, made connections and established several degrees of relation amongst them. This step was instrumental for preparing the data analysis and creating a meaningful structure from which we created the following analysis narrative. During this paper, we will present the output analysis, which established a comprehensive outline of future possibilities and fundamental necessities identified by the studied participants. This component addresses new ideas and concepts that can be implemented to improve the digital platforms already available to students. To ensure we respected all advice and ideas presented by participants, rather than only focusing on explicit recommendations identified, we aimed to identify suggestions, missing features, missing resources and unique concepts that were not common among all participants. While some findings might not represent the whole population, this step is conducive to understanding how particular initiatives can be re-designed and applied to a broader population, serving more students within a shared academic context.

3 Findings Findings collect theoretical approaches to improve Portuguese Students’ Shadow Libraries’, making them more valuable to their users and reimagining innovative processes to publish and access disseminated study resources. We actively question what

Peer-Produced Archives, Peer-Designed Solutions

113

characteristics make Portuguese-Student Shadow Library relevant to their public and what speculative features present users envision. We have broken down the upcoming analysis into five themes (Table 1): 1. Users as Hosts, Users vs Hosts: We examine how the roles of users and hosts are interchangeable, where actors bear responsibility for the material they upload and make available for their peers while deriving benefits from these spaces. 2. Editing, Curating and Reviewing: We illustrate that Editing, Curating and Reviewing are crucial steps to creating a pertinent digital space and ensuring published materials are relevant. 3. Enhancing Quality and Validity through collaboration: We report initiatives and future mechanisms for sustained and validated knowledge production. 4. Enhancing Communication: We report the need to promote communication among students, professors, alums and other academic entities, proposing a new perspective where all actors can coexist and be protected. 5. Reconsidering: browsing, archiving, preserving and distributing: We document participants’ exploration of opportunities for archives that can go beyond the traditional parameters.

Table 1. Identified Needs Themes

Codes

Features

Users as Host, Users vs Host

Power Structure

Reconsidering participants’ hierarchy when Manipulation, Deleting and Enforcing several resource distribution structures

Systematic Approval

Dissolving the need for Hierarchical Roles

Commenting

Implementing feedback methodologies and Incite Peer Discussions

Rating

Implementing a set of comprehensive statistics; Quantifying characteristics such as author’s reputation and others

Transparent Updating

Displaying Provenance; Reporting updates’ history

Mentoring

Implementing formal Advice Streams

Expiring

Flagging Old Files; Self-archiving

Professors assisting Students

Implementing mechanisms that ensure Accuracy and Validity

Editing, Curating And Reviewing

Enhancing Quality and Validity through collaboration

(continued)

114

P. Sá Couto et al. Table 1. (continued)

Themes

Enhancing Communication

Reconsidering: browsing, archiving, preserving and distributing

Codes

Features

Students assisting Professors

Benefiting from Students’ Practical Knowledge

Integrating Students as Professors

Professors Prompting Production; Students facilitating workshops on practical skills

Implementing new channels Implementing class forums and real-time collaborative platforms that promote student participation (re)Integrating Alums

Rebuilding the community; Reduce peer’s dependence on centralised platforms

Anonymity as a Feature

Integrating alternatives for distinct degrees of anonymity

Sharing the web

Present the capacity to archive the perishable web through mechanisms as web-to-print

Archiving Centralised Platforms

Archive content disseminated through centralised platforms like Whatsapp and Google Drive

Sharing documentation

Enabling the dissemination of: Cookbooks; code; comments

Publishing Self-Initiated Projects

Disseminating peers’ self-directed research

Making Public

Publishing peer’s curricular work and communicating it

Displaying licences

Enabling to choose from several publishing licences; Enriching peers’ knowledge about copyright regulations

Federated Access

Distributing access across distinct Organic Units

Browsing Library Catalogues

Implement strategies to reference Library Catalogues; Share Library Resources URLs

3.1 Users as Hosts, Users vs Hosts Archives that disseminate study materials rely on the academic community to produce and submit new resources. These initiatives enable updating and maintaining existing digital spaces, keeping them relevant to students dependent on them. Responsible students or student groups commonly perform tasks such as adding new resources, controlling existing ones, and moderating user interactions; we will name them hosts.

Peer-Produced Archives, Peer-Designed Solutions

115

Interview participants reported that enacting the role of hosts is a demanding task. At the same time, hosts must maintain a level of criteria that ensures a healthy flow of knowledge being uploaded, preserved and made attainable across shared digital spaces; they are also required to control the dissemination of relevant materials, standing reliable for the resources circulated. Who is entitled to enact such roles and how they will exercise their power is controversial. If these initiatives are frequently informal, it is also true that we have documented cases where there is a democratic process for choosing representatives that will guide peers’ interaction in digital community spaces. Participants identified the need to organise themselves, forming hierarchical structures that are often well-defined. These structures connect individuals whose hierarchical positions are fluid, continually varying from hosts to users, as they also depend on such ecosystems. Participants expressed caution regarding how hosts may exercise their power to manage contents that should be accessible to fellow users. They reported that manipulating or deleting the files would be elementary, making the content unavailable if anyone could edit the files. We have identified the demand to revise such power structures where there is a dire requirement to manage streams that disseminate and produce knowledge. Still, it is imperative to ensure that such governing processes are performed, as they are required to preserve archives’ capacity to answer student needs. Participants noted it as paramount to reconsider in what terms new contributions can be added to their archives and made publicly available. While it seems positive to make it easier to upload newly produced knowledge because it would increase the amount of content available, it is equally essential to ensure that these new materials are relevant to users that access them. Participants suggested the implementation of an intermediate submission step. This step would exist between the submission of new resources and their publication. This solution introduces a methodology where a collective of individuals would verify the relevance of a specific document, filtering materials unsuitable for public access, preventing the distribution of irrelevant or repeated content. This procedure would increase the community’s capacity to regulate submissions, creating a stream that is less dependent on individual hosts and more on collective action and regulation. 3.2 Editing, Curating and Reviewing Students find themselves overwhelmed with publications that only result in published noise, making it harder to distinguish between valuable study materials and those they should avoid. This happens across several media and disciplines, from books present on library shelves, deprecated files stored in drives that are comparable to dusty library shelves, to online searches when students are in search of visual references to support their artistic research. Dissolving the borders between curating and editing, making both activities leading players in collecting knowledge, is an important exercise when creating a relevant digital space. While there is a need to collect newly produced material, there is also the demand for editing, reviewing and archiving content that is being served online but currently does not serve anyone. Students identify that documents must be up-to-date, but creating, collecting and validating new materials is demanding. These challenges mandate students

116

P. Sá Couto et al.

to perform critical roles in the knowledge distribution and production ecosystem, where users, hosts, and curators may not be formally designated. Participants noted that if the methods to comment on published Drives’ resources were more accessible, it would result in more students seeking additional intervention rather than only consuming the material made available to them. Participants added that rather than leaving a classification on a material, other users could benefit from more meaningful reviews. Suggestions indicate that using statistics within a digital sharing platform could also verify the quality of shared materials, which happens similarly to online film ratings. Rating of documents could be calculated by taking into account their year of publication, downloads, number of comments, among other alternatives that have to be redrawn, ranking them on a linear scale. Participants recognised that it is not as important to check how many times a document was downloaded; rather, it would be more insightful to document and clarify how a resource has changed during a set time. Participants in the study reported that uncategorised study materials resulted in wasted time and resources. Furthermore, students would often print several unnecessary materials only to select the ones they would find relevant. Printing several sebentas resulted in more money being spent by students and was particularly damaging, having negative environmental implications. Students have developed strategies to facilitate the selection of study resources. One method that is earning popularity involves resorting to student mentors. Student Mentors help their mentees study, guiding them amongst which materials they should use and sharing informal insights based on their experience. Even though such a strategy is effective, it can only reach a limited scope of students enrolled in mentorship programs. It would be valuable to share such experiences with a broader audience to leverage such strategies, transforming these informal transmissions of advice into formal streams. Another alternative introduced would encompass archiving content from the main drive. In addition to this, older content could be flagged with a warning when it passes a specific expiration timeframe. This method would make it straightforward for students that a resource is not recent and should be used taking a critical view. 3.3 Enhancing Quality and Validity through Collaboration During our interviews, participants reported that the main concern with disseminating knowledge produced by students is assuring that the materials circulating amongst peers are scientifically validated. Students and Professors are interested in ensuring that the materials shared amongst them are accurate so that they contribute to the quality of curricular units and the overall course. The quality of documents shared online can vary as different students produce these contributions. Creating mechanisms that aid knowledge production in a sustained and scientifically validated approach is essential. A few efforts already exist that aim to cover such assumptions. Still, it is vital to understand how to use validated examples and ensure communities frequently employ them. One intervention stood out from others we had previously encountered. Professors are producing textual materials alongside students, more specifically, transforming class presentations and slides into curricular unit text supports. On a particular curricular unit,

Peer-Produced Archives, Peer-Designed Solutions

117

professors opened a call for students from previous academic years to partake in this collaboration. The implementation of these strategies provides valuable study materials while at the same time enhancing the quality of the curricular unit, making sure that study materials are more exhaustive. Such a strategy offers future students better quality material, conferring a higher validity status to such production. Although there is no monetary compensation, the participation of students in these streams of knowledge production is valued due to its formative experience. Current collaborative forms involve creating textual materials that support the materials professors present during classes. It consists of creating textual materials that address class topics previously identified by students as deficient. They are also responsible for replying to students in the class forum, always with the support of professors. Participants reported that studying by sebentas had a negative connotation. These were regarded as shortcuts used by students who did not want to have the hard work of studying through published books. Initiatives like the one we presented revoke such ideas and motivate students to initiate such types of production. To improve the production of textual studying resources, professors recognised the importance of collaborating in developing such resources, ensuring accuracy and consistency, and endowing studentproduced material as valuable contributions. Before the interviews, it was not expected that professors prompt the production of study material. This example indicates the contrary, Participant 11, a professor, revealed that as a teacher, it was challenging to efficiently and equitably evaluate students for the practical aspect of its curricular unit. With such a problem, a group of professors presented an equally innovative solution. Professors proposed students create study material to help future peers study. Participant 21 reported another initiative where the students association would organise workshops inviting technically proficient students to teach their peers these practical skills. Afterwards, they would archive these informal classes and make them available for future students within a Google Drive. Such initiatives hold extreme relevance because they are not bound to a specific moment and remain relevant for future access. Rather than creating a moment with immediate benefits, they created a resource that could be revisited by students who were present and shared amongst students who would benefit in the future. 3.4 Enhancing Communication Archives that distribute study material are not only represented by current students; they represent students, professors, alums and other entities that contribute through formal and informal activities. Participants reported that efforts that promote collaboration and exchanging information, ideas and resources are necessary and currently lacking. We must reconsider in what ways digital spaces can facilitate the growth and development of current and past community members that are both active and inactive, but that have contributed to existing streams of knowledge production and sharing and that can continue to be valuable actors in such ecosystems. Multiple participants indicated the development of a forum feature as a tool they would highly benefit from. Participant 01 suggested this forum could be an engaging

118

P. Sá Couto et al.

space where students can discuss bibliographic references and raise questions about specific subjects. In the Portuguese context, integrating a forum to support curricular unit teaching is not a new endeavour. We have documented successful initiatives where students could post questions on a space provided by Moodle and where the teaching staff could swiftly make replies visible to the whole class, as well as failed attempts where students of the class were working professionals studying after regular work hours and therefore were more interested in asking their questions during classes, prioritising student-to-professor interaction. Participant 01 reported another strategy that is not common in the Portuguese context. While doing their masters at a European university, students used Etherpad, a real-time open-source collaborative editor (Etherpad [2011] 2023). During the classes, all students would connect to a collaborative working page and would participate by sharing links and writing annotations during classes, an initiative that could be easily adapted to the Portuguese context. Despite the decline of Facebook’s usage, a theme discussed exhaustively across students and alums; it is compelling to see persistent freelancer requests and calls for possible collaborative initiatives among peers in this space. Participants recognised the need for a specific online feature or forum that encourages informal alums’ communication while preserving a non-institutional feel. Returning to Facebook, a platform from which users fled, demonstrates alums’ need to connect with former peers. We identified the need to archive knowledge gathered informally and formally in a shared space. Not only do we identify the interactions between professors and students as meaningful, but we feel that it is crucial to promote the exchanges between studentsstudents, students-alumni, and alumni-alumni. A new perspective where all of these actors cohabit might be an endeavour that could impact in meaningful ways how the academic community functions as a whole and how students continue to be an active part of the academy after their graduation. 3.5 Reconsidering Browsing, Archiving, Preserving and Distributing What archives become possible when rethinking ways of navigating knowledge consumed and distributed through unstable communication channels? Does it make sense to think of an archive that can only serve published resources validated and produced by students? Can we use these spaces to share new types of resources, from web references to self-initiated projects that do not fit into a category expected to hold and disseminate outcomes of student production rather than their process? During this section, we will cover a few innovative possibilities, presenting participants’ thoughts and suggestions to rethink fluid approaches to building archives, disseminating content, displaying authorship of produced materials and others. The level of anonymity granted across dissemination platforms was not a consensual matter across participants. Although repositories are usually shared among trusted peers, participants indicated concerns concerning the demand for anonymity of authors who also pursue recognition for their work. Contrary to the trend towards defending anonymous uploads, students often take pride in their work and desire to be credited for their produced material. There is a clear divergence in the expectations of distinct participants.

Peer-Produced Archives, Peer-Designed Solutions

119

Participants identified that study materials continue to be actively produced and shared, however, the lack of archiving and documentation of these materials results in a potential loss of potential resources for future students. Students are experimenting with new collaborative ways to create study material, often through informal communication streams such as WhatsApp groups. These collective efforts strive to improve the quality of study material, even though shared knowledge usually ends up forgotten because there is no stream established where students or student nuclei organise and archive such documents for access by future students. Archival platforms such as print-shop archives, Google Drives, and others often prioritise making resources available for students while they are studying and preparing for formal evaluative assessments. However, participants reported the need to expand knowledge archived beyond resources that summon sporadic visits, proposing the creation of strategies that facilitate use and augment meaningful interactions through continuous usage. While student production is often not regarded as copyrighted material and can be used without restraint, the same thing is not true when students share books and other closed-access resources. By publicly displaying licences, students would be encouraged to reconsider the resources they are sharing online, as well as demonstrate which resources students can use without problems when they start their working life and may revisit these archives. This strategy would also promote awareness among students regarding copyright regulations, enabling them to publish knowledge online in informed terms they have agreed upon. At the University of Porto, it is not uncommon for students to have the chance to attend optative curricular units offered by faculties outside the one they attend. These initiatives allow students to get to know other organic units inside the university and get to know different peers, professors and alternative teaching styles. It would be highly beneficial if students could access archives from other faculties within the university, as academic paths are often intertwined, and restricting students’ access limits opportunities for interdisciplinary learning. Contributions were not limited to the presented topics. Other suggestions, such as: helping peers navigate library collections by referencing physical study materials through URLs; sharing practical assignments with peers to inform their practice; and overall making study resources more circulated, are some of the endeavours identified as relevant across interviews.

4 Conclusion This thematic analysis dissected the collection of theoretical approaches to promote existing libraries’ capacity to integrate a broader comprehension of what it means to participate in the production of study resources. Furthermore, it prompted an innovative reimagination of knowledge production, publishing practices and access to communityproduced resources. This study presents the ability to investigate further individual functionalities, which can be experimented with as unique experiences, to develop current and potential components that students can adopt when designing digital platforms for disseminating study

120

P. Sá Couto et al.

materials. Moreover, we aim to explore promoting strategies for community engagement and ascertaining ways to incorporate students in conceptualising and producing a framework that can be referenced. While collaborative methodologies can support the design and ideation processes, we identify the need to build tools and alternatives that students can manage independently. By providing autonomy in further platform development, we assure students can take ownership of these spaces, exploring their creative potential and enabling independent decisions that fit their needs. Throughout the presentation of the findings, our intention was not to encompass all the possible lines of action identified. We sought to present a representative overview of participants’ perspectives, demonstrating that unique disciplines, students, and contexts have different needs. Such diversity should always be considered when creating a space oriented towards the whole academic community rather than focusing on specific individuals. Integrating diverse community members from the preliminary design stage resulted in insightful contributions that helped us illustrate and validate our assumptions and provided comprehensive insights for designing our framework. Adopting a qualitative research methodology proved highly effective when applied to the design process. It allowed us to collect and document crucial insights that support the development of a framework where community members are the end-users and leading contributors. This work documents the first phase of the design and development of a framework to preserve and distribute the collections collected across PSSL. These interviews allowed us to create a public who is interested in continuing to advise our research and that has shown their willingness to work closely with us in the future. Acknowledgments. Pedro Miguel Sá Couto was supported by a PhD scholarship granted by Fundação para a Ciência e a Tecnologia, I.P. (FCT), Portugal, grant number: 2020.06817.BD.

References Alhojailan MI, Ibrahim M (2012) Thematic analysis: a critical review of its process and evaluation. West East J Soc Sci 1(1):39–47 Boyatzis RE (1998) Transforming qualitative information: thematic analysis and code development. SAGE Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3(2):77–101. https://doi.org/10.1191/1478088706qp063oa Edwards R, Holland J. (2013) What is qualitative interviewing? Bloomsbury Publishing. http://www.doabooks.org/doab?func=browse&uiLanguage=en&queryField=What+ Is+Qualitative+Interviewing?&x=0&y=0 Elliott V (2018) Thinking about the coding process in qualitative data analysis. The Qual Rep 23(11):2850–2861 Etherpad (2023) Etherpad: A modern really-real-time collaborative document editor. [JavaScript]. The Etherpad Foundation. https://github.com/ether/etherpad-lite (Original work published 2011) Gill SL (2020) Qualitative sampling methods. J Hum Lact 36(4):579–581

Peer-Produced Archives, Peer-Designed Solutions

121

Kvale S (1996) InterViews: an introduction to qualitative research interviewing, First edn. SAGE Publications, Inc. Marshall MN (1996) Sampling for qualitative research. Fam Pract 13(6):522–526. https://doi.org/ 10.1093/fampra/13.6.522 McLellan E, MacQueen KM, Neidig JL (2003) Beyond the qualitative interview: data preparation and transcription. Field Methods 15(1):63–84. https://doi.org/10.1177/1525822X02239573 Pathak V, Jena B, Kalra S (2013) Qualitative research. Perspect Clin Res 4(3):192. https://doi.org/ 10.4103/2229-3485.115389 Robinson OC (2014) Sampling in interview-based qualitative research: a theoretical and practical guide. Qual Res Psychol 11(1):25–41. https://doi.org/10.1080/14780887.2013.801543 Tuckett AG (2005) Applying thematic analysis theory to practice: a researcher’s experience. Contemp Nurse 19(1–2):75–87. https://doi.org/10.5172/conu.19.1-2.75 Wengraf T (2006) Qualitative research interviewing. Biographic narrative and semi-structured methods (Reprint). Sage Publications

Usability Analysis of Augmented Reality-Based Learning Applications for Kids: Insights from SUS and Heuristic Evaluation Ramisha Baki1(B) , Afnan Mumu1 , Riyadil Zannat1 , Mouneeta Rahman1 Hasan Mahmud2 , and Muhammad Nazrul Islam1

,

1 Department of Computer Science and Engineering, Military Institute of Science and

Technology, Mirpur Cantonment, Dhaka-1216, Bangladesh {ramisha,nazrul}@cse.mist.ac.bd 2 Department of Computer Science and Engineering, Islamic University of Technology (IUT), Gazipur, Bangladesh

Abstract. In today’s fast-paced world, augmented reality (AR) has emerged as a widely accepted and profitable approach for engaging children in interactive learning experiences. However, the rapid development of this technology has left many individuals struggling with even basic interactions due to a lack of prior expertise and usability. While various studies have explored the use of AR functionalities in learning applications, few have addressed the associated usability problems. To integrate AR-based applications into children’s learning platforms effectively, ex- tensive research on user adaptability is needed. The primary goal of this study is to provide a set of heuristics for evaluating the usability of AR-based learning applications for kids and to compare heuristic evaluation and System Usability Scale (SUS) evaluation results to assess their applicability. Seventeen heuristics, including Nielsen’s 10 Usability Heuristics, were employed to evaluate four AR applications designed for children’s learning. The findings revealed significant heuristic problems in all applications, with SUS scores falling below the acceptance thresh- old. Moreover, the study demonstrated that both heuristic evaluation and SUS evaluation methods yield comparable results, confirming the efficacy of the proposed heuristics in assessing the usability of AR-based learning applications for children. Keywords: Augmented Reality · Usability Evaluation · Heuristic Evaluation · System Usability Scale

1 Introduction Augmented Reality (AR) is a promising approach for engaging children in learning experiences, but usability issues persist [1, 2]. The use of AR in iOS and Android devices may vary concerning the interface and the user’s experience of using those applications. Keeping all these factors in mind, it is necessary to evaluate the design perspectives of the interface for the Augmented Learning environment application to enhance children’s mental and cognitive abilities. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 122–135, 2024. https://doi.org/10.1007/978-3-031-47281-7_10

Usability Analysis of Augmented Reality-Based Learning Applications for Kids

123

Although AR has been studied for many years, it began to be formally evaluated recently [3–5]. One of the reasons why it took so long to have user evaluations may be a lack of knowledge on how to properly evaluate AR experiences and design experiments [4]. Given the key role played by usability as a quality at- tribute in software development [6], it has become extremely relevant to evaluate and improve the usability of mobile applications in an AR learning environment. One way to standardize design and enhance the usability of a product is through the use of heuristic evaluations [7–9]. General heuristics, such as Nielsen’s 10 [10] or Schneiderman’s 8 [11] usability heuristics have been used to evaluate these technologies. These heuristics could be useful for evaluating the AR based ap- plications because they bring attention to many crucial aspects of the usability of a digital solution. However, additional aspects that could alter the users’ experience may not be assessed due to the uniqueness of AR [12]. Therefore, the objectives of this research are firstly to identify a set of heuristics to evaluate the usability of AR-based learning applications for children, secondly to evaluate the usability of multiple AR-based kids learning apps based on proposed heuristics and through SUS questionnaires and then to analyze and compare the findings of Heuristic evaluation and SUS evaluation to assess the applicability and effectiveness of the proposed heuristics. Next, the related studies, methodology and evaluation studies with findings are discussed.

2 Literature Review This section briefly presents the studies focused on the design, development, and evaluation of AR-based learning applications for kids, with a specific emphasis on heuristic evaluation-associated techniques and heuristics, as well as the System Usability Scale (SUS) [13] used for evaluation. Several studies focused on the interaction of children with AR-based learning applications, for example, Yadav et al. [14] developed an augmented reality smartphone app and conducted a study to investigate the interaction of children aged two to eight years with the app. Similarly, Moorhouse et al. [15] identified three new themes related to schoolchildren’s experiential learning experiences with augmented reality in museums. Additionally, Cheng and Tsai [16] conducted a study focusing on child-parent shared reading with an augmented reality picture book. Tuli and Mantri [17] in their study investigated the usability features of augmented reality (AR) apps, primarily focusing on portable AR applications for younger children. While their study assessed usability concerns, incorporating heuristic evaluation methods would have provided more in-depth insights into design considerations for AR-based learning environments for children. Tasfia et al. [18], in their study, proposed a novel set of heuristics to evaluate AR based learning application, which are designed for assessing traditional interfaces. In conclusion, there is a need for heuristic evaluation of AR-based systems for children’s interactive experiences, emphasizing the lack of specific heuristics for such applications. However, only a limited number of studies have focused on usability heuristics for AR-based learning apps among the broader research on AR applications for children.

124

R. Baki et al.

Thus, this study differs from others as it conducts a comprehensive heuristic evaluation, providing insights into violated heuristics, their severity ratings, and recommendations. Additionally, a comparative user study for SUS scores examines variations among different age and gender groups. Furthermore, the study presents a categorized summary of the identified problems in each application.

3 Methodology The methodological overview of the research is presented in Fig. 1. A total of four AR-based learning applications were selected based on the number of downloads, user ratings, number of reviews, and app size. These selected applications were then evaluated using heuristic evaluation, employing the selected seventeen heuristics (see Table 1) proposed by Nielsen and Molich [10] and Derby et al. [12]. The choice of using two sets of heuristics was motivated by the fact that while Nielsen and Molich’s heuristics [10] can evaluate the usability of any application, Derby et al.’s heuristics [12] are specifically designed for assessing augmented reality applications. Derby et al. [12] proposed a heuristic checklist that can be used to assess and inform design changes that influence the user experience of an AR application and/or device. These additional heuristics of Derby et al. [12] with the traditional Nielsen’s 10 Heuristics principles can help to enable the identification of specific AR-related challenges, aiding the development of better design guidelines for AR-based applications. Table 1. Heuristics Principles selected for the evaluation. Nielsen and Molich [10]

Derby et al. [12]

Visibility of system status (H1)

Collaboration (H11)

Match between system and the real world (H2)

Comfort (H12)

User control and freedom (H3)

Device Maintainability (H13)

Consistency and standards (H4)

User Interaction (H14)

Error prevention (H5)

Feedback (H15)

Recognition rather than recall (H6)

Unboxing and Set-Up (H16)

Flexibility and efficiency of use (H7)

Cognitive Overload (H17)

Aesthetic and minimalist design (H8) Help users recognize, diagnose, and recover from errors (H9) Help and documentation (H10)

Heuristics evaluation was carried out by four evaluators having a good level of expertise in Human-Computer-Interaction and heuristic evaluation. Every evaluators at first conducted the evaluations individually and then aggregated their findings altogether. They engaged in discussions to address any challenges encountered along the way. During the heuristic evaluation process, the evaluators utilized Nielsen’s Severity Ranking

Usability Analysis of Augmented Reality-Based Learning Applications for Kids

125

Scale (SRS) [20], which consists of a five-point scale ranging from 0 to 4. The violated heuristics were analyzed, and their associated severity ratings were assessed accordingly in this study. After the heuristic evaluation, the selected applications were evaluated using the System Usability Scale (SUS) questionnaire. The study involved 40 children (13 female and 17 male) between the ages of 4 and 7. Before the commencement of the study, the participants’ proficiency in using mobile-based applications was assessed by consulting their teachers and parents. Those who demonstrated a higher level of interaction and familiarity with such smart applications were chosen as participants for the study. They were informed that the purpose of the study was to evaluate the mobile applications and not to assess their individual performance. The participants were then asked to explore the studied applications, and their feedback was collected through the System Usability Scale (SUS) questionnaire. Each participant was asked to rate each application on a scale of 1 to 5, based on the 10 SUS questions provided. A total of 40 responses (10 for each application) were recorded, and the SUS scores were calculated based on these responses.

Fig. 1. Methodological overview

In the SUS evaluation, certain questionnaires contained technical terms that proved challenging for the children to comprehend. To address this issue, the questions were translated into more child-friendly language. For instance, “I found the various functions in this system were well-integrated” was rephrased as “I found that everything in this system worked really well together” to ensure better understanding. Similarly, the question “I thought there was too much inconsistency in this system” was modified to “I felt that this system didn’t stay the same in some parts, and it was confusing” with the same objective in mind. Finally, the evaluation results were analyzed, and a comparative analysis was conducted to identify the strengths and weaknesses of each application.

126

R. Baki et al.

4 Usability Evaluation 4.1 Heuristics Evaluation Moving forward, the applications will be referred to by abbreviations. Arvid Augmented Reality will be known as A1, Color Quest AR as A2, WWF Free Rivers as A3, and Leo AR Education as A4. Similarly, the problems identified in the 1st application (A1) will be numbered as A1-P1, A1-P2, and so on. The evaluators strived to identify the violated heuristics and assess the severity of each violation during the evaluation process. Through the analysis of the four mobile applications, a total of 18 problems were detected. The tables provided in the study (see Table 2, 3, 5, and 4) outline the problems, along with their associated heuristic violations, severity ratings, and recommended solutions for each application. Table 2. Heuristic Evaluation on ARvid Augmented Reality Problem No Problem Statement Heuristic Violation Recommendations

Severity Rating

A1-P1

The homepage of the application is not self-explanatory and offers no feedback

H15 H16

On the landing 2 page, further information about what the app does and how to use it should be available

A1-P2

There is no way to reduce the size of the AR object, and it does not fit on the screen

H12 H14

The ability to reduce or increase the size of AR objects should be offered

A1-P3

The screen H9 H13 H15 frequently freezes, and if the wrong button is pushed, it might be challenging to correct the error

There should be a 3 Self-Explanatory Back button in case an error occurs

A1-P4

The Icon which adds an AR element is not self-explanatory

The + icon should 2 be replaced with a labeled icon saying AR objects

H2

3

Four examples from each of the applications are discussed below: In A1-P2, the size of the AR object exceeds that of the screen, and there is no option to adjust its size or fit it within the screen (Fig. 2 (a)). This issue violates H12 (Comfort) as it obstructs the proper visibility of other features and H14 (User Interaction) as the object occupies most of the screen, limiting the user’s ability to interact with the system.

Usability Analysis of Augmented Reality-Based Learning Applications for Kids

127

Table 3. Heuristic Evaluation on Color Quest AR Problem No Problem Statement

Heuristic Violation Recommendations

Severity Rating

A2-P1

No Play Now Button to help the user start the game in home page

H2 H3 H12

Add a Play Now Button to help the user start the game in the home page

2

A2-P2

The functionality of the button in the coloring page at the below center is not clear

H2 H10 H12 H14

A quick explanation 1 of each button’s functionality in the application could be provided

A2-P3

Screen freezes H3 frequently and no H9 Exit or Close button H13 H15

The server issue 4 must be resolved, and the app must provide the necessary buttons to get out of such situations

A2-P4

There is no pause or H1 stop recording H12 option H15

Pause button can be 1 included with the recorder with proper notifications

A2-P5

No feedback when color goes out of border in the coloring page

H9 H10 H14 H15

Feedback should be 3 provided whenever a user is violating the color rules like coloring outside of borders

A2-P6

Account details in the profile page are missing

H6 H8 H11 H17

Details should be 2 provided so that the user does not need to remember the details

The severity rating for this issue is 3, as it was identified by all four evaluators and is likely to cause confusion among users during system use. In A2-P5, the lack of feedback provided to the user when coloring goes out of the border or overlaps with other areas (as depicted in Fig. 2 (b)) violates several heuristics, including H9 (Helps users recognize, diagnose, and recover from errors), H10 (Help and documentation), H14 (User Interaction), and H15 (Feedback). As the primary objective of the application is to teach children how to color various objects correctly, the absence of error messages when the children color incorrectly could be assigned a severity rating of 3.

128

R. Baki et al. Table 4. Heuristic Evaluation on WWF free rivers

Problem No Problem Statement

Heuristic Violation Recommendations

Severity Rating

A3-P1

The starting page H15 does not provide H16 clear instructions regarding how the game is supposed to be played

Instead of 2 voice-over, the landing page could have a standard Start Now button in order for the users to understand the game properly

A3-P2

The is no Return button from any page

A Back button should be incorporated in case someone accidentally clicks elsewhere

A3-P3

The software is H12 uncomfortable to H13 use because the H15 interface is partially displayed and occasionally the program freezes

There have to be 2 options for shrinking the size of the AR characters so that it fully fits in the screen

A3-P4

No Help and Documentation button visible for users

The Help and documentation section should be incorporate

H7 H9

H10

3

4

In A3-P3, the app usage is not comfortable as the environment is shown partially and sometimes the app freezes (as depicted in Fig. 2 (d)) violate several heuristics, including H12 (Comfort), H13 (Device Maintainability), and H15 (Feedback). As the primary objective of the application is to teach children about the lifecycle of rivers, the app should have had a full and clear view. In A4-P3, the issue of not having a Play Now button is mentioned (as depicted in Fig. 2 (c)) violates several heuristics, including H1 (Visibility of the system status), H2 (Match between the virtual objects and real-world) and H6 (Recognition rather than recall). Should provide a Start Now or Play Now button. So, the user knows when the application function has started. Out of the 18 identified problems, 17% of them are categorized as cosmetic issues, which may be considered less critical and can be overlooked. Additionally, 44% of the problems are classified as minor issues, indicating that they can be addressed with lower priority. Another 27% of the problems are categorized as major issues, emphasizing their importance in terms of resolution. Lastly, 11% of the problems are labeled as catastrophe problems, highlighting their critical nature and the need for immediate attention.

Usability Analysis of Augmented Reality-Based Learning Applications for Kids

129

Table 5. Heuristic Evaluation on Leo AR Education Problem No Problem Statement

Heuristic Violation Recommendations

A4-P1

No tutorials for the new users

H3 H10 H15

A4-P2

When an object is H1 uploaded its name is not mentioned

Name and some 1 knowledge about the object should be provided

A4-P3

No play now button

A Start Now button 2 to notify the user is aware when an application function has begun

A4-P4

Too many features H7 H8 H12

H1 H2 H6

Severity Rating

Some tutorial videos 2 can be provided for the children

The features icons should be more organized for easier use

3

Fig. 2. Problems with UIs from each of the applications

Considering the severity of the issues, problems A2-P3 and A3-P4 require immediate attention and urgent resolution. These problems are labeled as catastrophe problems, as

130

R. Baki et al.

indicated in Tables 3 and 5. It is crucial to address these issues promptly to enhance the overall usability of the system. The total heuristics violated by the apps are shown in Table 6 and the number of times each of the problems of the applications violated the heuristics is shown as a graph in Fig. 3. Table 6. Number of times Heuristics were violated by each application H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 H17 Total Heuristics Violated Per App A1

1

A2 1

2

1 2

1

A3

1 1

A4 2

1

1

1

1

1

1

1

1

2

3

1

2

3

1

1

1

1

1

2

2

1

1

Total Number of Times Heuristics were violated for all the four applications

2 1

1

8 1

1

22 8 10 48

Fig. 3. Number of times each of the problems of the applications violated the heuristics

Considering Table 6, which depicts the number of times the AR-based applications violated the heuristics, it is evident that a significant number of violations occurred across multiple applications. Notably, Heuristic 15 (H15) was the most frequently violated, with a total of 8 instances. Following closely behind, Heuristic 12 (H12) was violated approximately 6 times. Heuristics 2 (H2), 9 (H9), and 10 (H10) were the next most frequently violated, each occurring around 4 times. Specifically, in the case of the A1 application, Heuristic 15 was the most commonly violated. For the A2 application, both Heuristic 15 and Heuristic 12 were violated most frequently. Similarly, in the A3 application, Heuristic 15 was the most commonly violated, while in the A4 application,

Usability Analysis of Augmented Reality-Based Learning Applications for Kids

131

Heuristic 1 had the highest frequency of violations. In total, the four applications violated the heuristics a total of 48 times. These findings emphasize the importance of addressing these specific heuristics to improve the overall usability and user experience of the applications. Additionally, focusing on Fig. 3, it can be concluded that A2-P2, A2-P3, A2-P5, and A2-P6 violated the highest number of heuristics, indicating a significant lack of suitability for user experience. In contrast, all other applications, except A2, had problems that violated a maximum of three heuristics. After the analysis of Table 6, it was found that fewer violations occurred in Nielsen’s heuristics compared to Derby’s heuristics. This is because Derby’s heuristics were explicitly designed for augmented reality applications, but the AR-based applications themselves were not developed according to the recommended guidelines for creating such applications. 4.2 User Study The SUS score based on gender and age group is illustrated in Table 7. The SUS score was calculated based on the guidelines established by Brooke et al. [13]. Based on the analysis of Table 7, it was found that the application A1 was only accepted by the Male group when grouped by gender, with a SUS score above 68%. However, when the groups were based on age, no group accepted the application, with scores below 68%. Similarly, for the A2 application, it was only accepted by female participants in the gender-based group but rejected by both age groups. The A3 and A4 applications were rejected by all groups, regardless of gender or age. Although the A3 application obtained the highest SUS score of about 63.82% among all users, according to Table 7, it was still rejected by all group partici- pants. Similarly, the A4 application, which had the second highest score of approximately 62.72% among the four applications, was also rejected by all users. Despite being accepted by one participant in their respective groups, both the A1 and A2 applications had the lowest scores among all the applications, with scores of about 57.80% and 57.70%, respectively. Table 7. Summary results of the SUS evaluation Based On Gender Age All

SUS Score A1

A2

A3

A4

Male

68.12

55.55

62.12

59.23

Female

50.83

69.47

64.23

62.88

Below 5 Years

56.83

55.23

67.86

61.06

Above 5 Years

55.40

50.54

61.06

61.06

57.795

57.6975

63.8175

62.715

132

R. Baki et al.

5 Comparative Analysis and Findings The study result shows that all four applications Leo AR, WWF Free Rivers, Color Quest AR, and ARvid Augmented Reality had a lot of heuristic problems and lower SUS scores below the threshold of acceptance. The Arvid AR application was found to be too complex and difficult to use, which could result in a poor user experience. The application lacked clear navigation, and users struggled to find relevant information or complete tasks efficiently. This highlights the importance of designing user interfaces that are intuitive and easy to use. The WWF free rivers application was rejected due to its limited scope and lack of engaging features, which could result in low user engagement. The application lacked clear feedback, which made it difficult for users to understand the impact of their actions. The application could benefit from incorporating more interactive elements and providing users with meaningful feedback. The Color Quest application was rejected due to its limited educational value and repetitive gameplay, which could lead to boredom among users. The application could benefit from incorporating more educational content and providing more challenging gameplay to enhance user engagement. The Leo AR education application was rejected due to its limited content and lack of interactive features, which could hinder the learning experience. The application could benefit from incorporating more interactive elements, such as quizzes and games, to enhance the user experience and promote learning. However, based on the Heuristic Evaluation and User study of the selected applications, the following problems have been identified: Lack of Instructions: The majority of the evaluated applications lacked essential information regarding how to use the system. Given that the applications are designed to target kindergarten-aged children, a step-by-step tutorial of the system is crucial, yet it was found to be missing in most of the applications. Lack of Flexibility and Ease of Use: Both user study and heuristic evaluation identified usability issues in all evaluated apps, including missing controls and screen freezes. The absence of play, pause, or back buttons, as well as instances where the screen froze for an indefinite amount of time. Lack of Feedback: Insufficient or no feedback was observed during various operations, resulting in users being unaware of the current status of the application. No Error Messages: The absence of error messages in most of the applications was observed, which is problematic as these applications are designed for children’s learning. Basic instructions should be provided when a user performs a task in correctly. Lack of System Status: In some operations, there’s a lack of system status that leads to errors. For example, users may not know if an operation is still in progress or if it has been completed. This can lead to confusion and frustration, especially if the operation is time-sensitive or critical. Providing clear and timely system status feedback can help prevent such errors and improve the overall user experience.

Usability Analysis of Augmented Reality-Based Learning Applications for Kids

133

Possibility of Software Crashing: The integration of augmented reality into the applications has significantly reduced their response time, leading to freezing of the screen for indefinite periods and occasional program crashes.

6 Conclusion The findings of this study highlight several usability issues and shortcomings in the ARbased learning applications. Both the heuristic analysis and SUS questionnaire revealed significant heuristic violations and lower SUS scores, indicating poor user satisfaction. The applications were ranked based on user preferences, and the violations of heuristics were identified, providing valuable insights for designers to improve the applications’ usability. The individual evaluations of each application pointed out specific areas of improvement. For Arvid AR, the focus should be on simplifying the user interface and enhancing navigation. WWF Free Rivers can benefit from adding interactive features and providing clearer feedback to enhance engagement. For Color Quest, adding more educational content and challenging gameplay is necessary to prevent user boredom. Finally, Leo AR education should incorporate more interactive elements to improve the learning experience. The research also identified common usability issues across the applications, such as lack of instructions, flexibility, feedback, error messages, system status, and occasional software crashes. Addressing these issues will significantly improve the overall usability of AR-based learning applications for children. To summarize, this study contributes valuable insights into the usability challenges of AR-based learning applications and provides guidance for designers to create more user-friendly and engaging applications, ultimately enhancing the learning experience for children. The limitation of the study was the limited number of participants to evaluate only four applications. Moreover, since the evaluation is done by only two approaches a concrete analysis of the usability study of the applications could not be made. Evaluation methods such as laboratory and questionnaire-based testing were not taken into consideration. Another limitation of the research was the exclusive focus on assessing the usability criteria of these applications. However, the study did not address the evaluation of the learning ability of the mentioned applications. Therefore, a logical progression for future research would involve expanding the scope by increasing the number of applications, participants, and evaluation approaches. This comprehensive approach will enable the assessment of the learnability of all the applications, thereby enhancing the overall insights gained from the study. Future research in this area could focus on developing guidelines for designing AR applications that are more effective in promoting user engagement and enhancing the user experience. This could involve a more in-depth analysis of user behavior, preferences, and feedback, as well as the use of advanced technologies such as Artificial Intelligence (AI) and Machine Learning (ML) to improve the design and functionality of AR applications.

134

R. Baki et al.

References 1. Afrin S, Zaman SR, Sadekeen D, Islam Z, Tabassum N, Islam MN (2022) How usability and user experience vary among the basic m-commerce, AR and VR based user interfaces of mobile application for online shopping. In: Martins N, Brandão D (eds) Advances in design and digital communication II. DIGICOM 2021. Springer series in design and innovation, vol 19. Springer, Cham. https://doi.org/10.1007/978-3-030-89735-2_4 2. Sharif A, Anzum F, Zavin A, Suha SA, Ibnat A, Islam MN (2018) Exploring the opportunities and challenges of adopting augmented reality in education in a developing country. In 2018 IEEE 18th international conference on advanced learning technologies (ICALT). IEEE, pp 364–366 3. Dünser A, Grasset R, Seichter H, Billinghurst M (2007) Applying HCI principles to AR systems design 4. Dünser A, Grasset R, Billinghurst M (2008) A survey of evaluation techniques used in augmented reality studies. Human Interface Technology Laboratory New Zealand, Christchurch, New Zealand pp. 5–1 5. Edward Swan J, Gabbard, JL (2005) Survey of user-based experimentation in augmented reality. In Proceedings of 1st international conference on virtual reality, vol 22. Citeseer, pp 1–9 6. ISO ISO and IEC FCD (2004) 25000, software engineering-software product quality requirements and evaluation (square)-guide to square. International Organization for Standardization, Geneva 7. Suha SA, Islam MN, Akter S, Bhowmick MC, Halder R (2022) Assessing usability of mobile applications developed for autistic users through heuristic and semiotic evaluation. In: Uddin MS, Jamwal PK, Bansal JC (eds) Proceedings of international joint conference on advances in computational intelligence: IJCACI 2021. Springer Nature, Singapore, pp 25–39. https:// doi.org/10.1007/978-981-19-0332-8_3 8. Muaz MH, Islam KA, Islam MN (2021) Assessing the usability of truck hiring mobile applications in bangladesh using heuristic and semiotic evaluation. In: advances in design and digital communication: proceedings of the 4th international conference on design and digital communication, Digicom 2020. Springer International Publishing, Barcelos, Portugal, 5–7 Nov 2020, pp 90–101. https://doi.org/10.1007/978-3-030-61671-7_9 9. Kundu S, Kabir A, Islam MN (2020) Evaluating usability of pregnancy tracker applications in Bangladesh: a heuristic and semiotic evaluation. In: 2020 IEEE 8th R10 humanitarian technology conference (R10-HTC). IEEE, pp 1–6 10. Nielsen J, Molich R (1990) Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 249–256 11. Aottiwerch N, Kokaew U. (2017) Design computer-assisted learning in an online Augmented Reality environment based on Shneiderman’s eight Golden Rules. In: 2017 14th international joint conference on computer science and software engineering (JCSSE). IEEE, pp 1–5 12. Derby JL, Chaparro BS (2022) The development and validation of an augmented and mixed reality usability heuristic checklist. In: Chen JYC, Fragomeni G (eds) Virtual, augmented and mixed reality: design and development: 14th international conference, VAMR 2022, held as part of the 24th HCI international conference, HCII 2022, virtual event, 26 June – 1 July 2022, Proceedings, Part I. Springer International Publishing, Cham, pp 165–182. https://doi. org/10.1007/978-3-031-05939-1_11 13. Brook J (1996). SUS: a “quick and dirty” usability scale. Usability evaluation in industry 14. Yadav S, Chakraborty P, Kochar G, Ansari D (2020) Interaction of children with an augmented reality smartphone app. Int J Inf Technol 12:711–716

Usability Analysis of Augmented Reality-Based Learning Applications for Kids

135

15. Moorhouse N, Tom Dieck MC, Jung T (2019) An experiential view to children learning in museums with augmented reality. Museum Manag Curatorship 34(4):402–418 16. Cheng KH, Tsai CC (2016) The interaction of child–parent shared reading with an augmented reality (AR) picture book and parents’ conceptions of AR learning. Br J Edu Technol 47(1):203–222 17. Tuli N, Mantri A (2021) Evaluating usability of mobile-based augmented reality learning environments for early childhood. Int J Human-Comput Interact 37(9):815–827 18. Tasfia S, Islam MN, Nusrat SA, Jahan N (2023) Evaluating usability of AR-based learning applications for children using SUS and heuristic evaluation. In: Shamim Kaiser M, Waheed S, Bandyopadhyay A, Mahmud M, Ray K (eds) Proceedings of the fourth international conference on trends in computational and cognitive engineering: TCCE 2022. Springer Nature, Singapore, pp 87–98. https://doi.org/10.1007/978-981-19-9483-8_8 19. Nielsen J (1994) Usability inspection methods. Chapter heuristic evaluation, pp 25–62

Investigating Usability Indicators for the Adoption of AI Models in Heuristic Evaluation Demerval Gomes S. Júnior1(B) , Rodrigo Hernández-Ramírez1 and Jacinto Estima2

,

1 UNIDCOM/IADE – Unidade de Investigação em Design e Comunicação, Av. D. Carlos I, 4,

1200-649 Lisbon, Portugal [email protected], [email protected] 2 Department of Informatics Engineering, CISUC/University of Coimbra, R. Silvio Lima, 3030-790 Coimbra, Portugal [email protected]

Abstract. Online Learning Management Systems (LMS) have become widely used solutions over the last few years by educational institutions worldwide. Interest in evaluating the quality of these systems has been increasing, and new research to investigate the usability and user experience (UX) of these platforms has increased over the last decade. One of the common evaluation approaches is the heuristic evaluation of the interface based on selected criteria or indicators that describe well-known usability problems. However, this process remains laborious and challenging, requiring considerable effort from evaluators. Adopting automated methods is still uncommon, and approaches based on Artificial Intelligence (AI), for example, are rare. This article presents a study that investigates the potential adoption of usability indicators (Ui) for using artificial intelligence methods as supportive tools for heuristic evaluation of LMS interfaces. In our study, we developed a methodology to investigate some requirements to identify and select a set of Ui to create datasets for AI models to contribute to LMS interface inspection. The methodology allowed us to highlight a set of Ui to be potentially adopted with Machine Learning (ML) to evaluate LMS interfaces. We highlight a set of necessary assumptions to build datasets that can be used with AI models for heuristic evaluation. The methodological approach we propose can be repurposed to study new usability indicators to analyze other complex software contexts. Keywords: Heuristic Evaluation · Usability and UX · Usability Indicators · Learning Management System · Artificial Intelligence

1 Introduction Distance Learning (DL) is commonly supported by a type of system known as an Online Learning Environment (OLE), primarily represented by Learning Management Systems (LMS). These platforms are developed to contribute to the teaching and learning process through effective and efficient interaction between educators and students [1]. LMS © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 136–151, 2024. https://doi.org/10.1007/978-3-031-47281-7_11

Investigating Usability Indicators

137

encompass functionalities of content management and control, management of pedagogical activities, and communications within the educational context. The adoption of LMS can be either full or partial. E-learning encompasses two primary modalities: fully online courses or a blended learning model. In the first format, students and educators are not physically present in the same location, whereas in the latter case, a portion of the activities takes place in person, while another part occurs in an online environment [2]. Generally, LMS support synchronous and asynchronous learning processes. In the synchronous format, students and educators are online simultaneously, engaging in interactive communication within the same timeframe. In the asynchronous format, interaction occurs intermittently and, therefore, not necessarily at the same time. Communication between educators and students in LMS takes place through discussion groups mediated by the use of tools such as forums, email, chats, and other integrated solutions within the system. Nowadays, the most widely used LMS belong to two main categories: open-source and closed-source. Open-source solutions are based on free access to source code under certain conditions. Some examples include Moodle, Sakai, and dotLRN [1, 3]. Closed-source options are proprietary and do not provide access to the source code for consultation or reuse. Some examples are Blackboard, Ping Pong, Canvas LMS, McGraw-Hill Education, D2L, and Blackbaud [1, 3]. In recent years, LMS have attained a significant role in educational institutions worldwide, particularly after the COVID-19 Pandemic. Furthermore, new technological possibilities have contributed to the adoption of these systems. However, it is not uncommon to see objections and dissatisfaction from students and educators regarding the quality of these platforms, particularly in terms of usability and User Experience (UX) [1, 3–5, 5, 6]. In this context, ensuring continuous improvement of LMS through frequent evaluation of their interfaces becomes imperative. However, these evaluations are usually conducted from the perspective of usability and UX [1, 3, 6]. Regarding usability, the practical aspects of the daily activities of educators and students are evaluated [1, 3, 6]. Regarding UX, subjective matters are examined, particularly concerning hedonistic aspects related to the emotional responses that the platforms elicit [7]. When an LMS is introduced, educators and students will need time to adapt to the system and become proficient in performing typical tasks satisfactorily. This adaptation process can be more challenging if the interface presents usability issues, as users will need to overcome the inevitable frictions. Although evaluating LMS has been the subject of numerous studies in recent decades [1, 3, 8, 9], considerable challenges remain open [1]. The study developed by [4] sought to establish the relationship between ergonomics and usability in the context of e-learning. An Systematic Mapping (SM) was developed by [5] to identify publications related to the usability of e-learning systems in the mobile context, specifically on mobile devices. In another SM, [10] suggests an update to the work developed by [5]. The authors observed an evolution in the approaches and evaluation techniques of m-learning LMS. However, they noticed the absence of a comprehensive framework or methodological approach to address UX, usability, and the pedagogical aspects of mobile educational software. In response to this, they proposed a framework to evaluate mobile LMS. Subsequently, [3] developed an SM to identify publications that evaluate both desktop and mobile LMS applications from a usability and UX perspective.

138

D. G. S. Júnior et al.

Methodological approaches used to evaluate LMS often rely on interface evaluation [11] and approaches focusing on the pedagogical domain [12–14]. Using more sophisticated evaluation strategies, such as AI, to investigate the usability and UX is still uncommon [1, 3, 8, 9]. Heuristic evaluation based on predefined criteria is by far the most common method. These criteria are supported by guidelines such as the ISO usability [15, 16] and W3C [17] standards and Nielsen’s “10 Usability Heuristics for User Interface Design” [11, 18]. Research suggests that many usability evaluations conducted on LMS over the past decade are based on those heuristics (10 HU-JN) [6, 19]. One of the main challenges for adopting AI-based approaches for heuristic evaluation of interfaces is defining the set of indicators that can be converted into datasets for training AI models. Considering this challenge, we have designed a methodological approach to investigate a predefined set of 800 Ui based on Nielsen’s heuristics (10 HU-JN) to be used in the development of datasets for machine learning (ML) models. This approach supported the subsequent phase, where several AI models were tested to contribute to the process of evaluating the usability of LMS interfaces. The remainder of the article is organized as follows: Sect. 2 presents the theory used to sustain this study; Sect. 3 details the methodology adopted in our study; Sect. 4 discusses the results obtained; and finally, Sect. 5 presents the conclusion.

2 Background and Related Work Heuristic evaluations (HE) reveal usability issues early in the design process. Combined with other testing methods, they can be more cost-effective than testing involving real users [20–22]. One challenging aspect of HE is its intrinsic relationship with the evaluator’s experience and knowledge [22]. The inspection process relies on assumptions made by the expert regarding what constitutes “good” or “poor” usability. Despite the extensive research conducted over many years to establish the ten heuristics (10 HU-JN), there are inherent subjective elements to consider in the evaluation process [20, 23]. HE involves adopting a set of usability criteria or indicators that identify potential interface issues, often rooted in the system’s underlying structures, development, and design [24]. The challenge of implementing AI-based approaches lies in finding a set of Uis that can reduce the impact of subjectivity inherent to the HE process. Ui groups have been used since the 1960s. Initially, Uis were associated with studies about a system’s biomechanical and psychophysiological responses [25]. In the 1970s, concerns regarding usability criteria emerged, leading to the creation of ISO standard 9241–11 [26]. Works by Jordan [27] and Nielsen on Usability Inspection Methods [11] and Usability Engineering [28], as well as Dominique and Scapin’s research on ergonomic quality criteria for interactive systems [29], were landmarks of this period. In addition to the ISO standard, other standardizations for web content development would later emerge, such as the W3C/WAI Web Content Accessibility Guidelines [17]. The use of criteria to evaluate system interfaces has been the subject of research in seminal works over the past decades [11, 27, 29–39]. Since then, an increasing number of evaluation criteria have been proposed, resulting in a considerable set of Uis that, in some cases, may overlap [25, 40–44]. In the field of LMS studies, research point to the use of diverse sets of Uis, techniques, and different methodological approaches adopted to

Investigating Usability Indicators

139

evaluate the usability of these systems [1, 3–5, 5, 6]. Despite the proposals of different frameworks in recent years for assessing LMS usability, researchers and professionals still lack consensus regarding a unified evaluation model, and initiatives that employ AI-based approaches are still relatively scarce [3, 6]. The definition of an AI-based approach for HE implies considering problem definition and characterization, AI algorithms and models, data types, input variables, datasets, training and testing data, features, labels, among others [45]. AI encompasses two major general approaches: knowledge-based and statistical data-based. The former is referred to as GOFAI—“Good Old-Fashioned Artificial Intelligence” aiming to encode expert knowledge into software. Although in this case, the machine is not capable of learning on its own [46]. In statistical-based Machine Learning, methods are developed to enable the machine to learn through mathematical techniques. [46]. To adopt an AI-based solution, it is necessary to define the problem to be solved, which will be converted into AI tasks [47]. This process involves identifying algorithms to construct mathematicalstatistical models that can perform those tasks [48]. An algorithm is a well-defined set of instructions designed to accomplish a specific goal using input data. It operates by executing a series of finite steps. In traditional computer programming, a known and predefined algorithm performs a particular task (Input Data + Model → Output) [48]. In the context of AI, the algorithm serves as a recipe or set of instructions for applying an AI technique to obtain an artificial intelligence model [45]. The algorithm utilizes available data to construct a data-driven model. It’s important to note that different data inputs can lead to different models, even when using the same algorithm [48]. In the realm of AI, the specific model is not predetermined but is learned and generated by the algorithm based on the provided data [45]. This process allows the algorithm to adapt and derive insights from the data, ultimately producing a model tailored to the specific problem or task at hand [45]. Data can be categorized as either discrete or continuous. Discrete data corresponds to categorical values. Examples of discrete data include binary choices like “yes” or “no” and categorical options like “A,” “B,” or “C.“ On the other hand, continuous data represents values within a range or continuum. For instance, it can involve measuring the number of events occurring within a specific unit of time [45]. In programming, a variable is a storage location that holds a value and can be modified during the execution of an algorithm [49]. Two types of dependency relationships between variables are crucial: independent and dependent variables [48]. The independent variable exists independently and serves as the input data used to train the AI model. It is typically manipulated or controlled by the algorithm to observe its impact on the dependent variable [45]. The dependent variable, on the other hand, is the one that needs to be predicted or estimated, and its value is dependent on another variable. The algorithm aims to learn the relationship between the independent and dependent variables through the training process, allowing it to make predictions or draw conclusions based on new input data [45, 47]. A dataset is a vital resource utilized for training, testing, and validating AI models. Within the field of machine learning, a dataset comprises input features alongside their corresponding output labels or target values [47]. These datasets can be sourced from a wide range of origins, including experimental studies, real-world observations, or artificially generated simulations. The quality and efficacy of an AI model are often

140

D. G. S. Júnior et al.

strengthened by the presence of datasets that not only exhibit diversity but also serve as representative samples of the underlying domain [45]. The inclusion of comprehensive and representative data empowers the model to faithfully capture the intricacies and complexities of real-world scenarios, leading to enhanced performance and broader applicability [50]. In general, datasets are partitioned into training and test data, enabling the evaluation and comparison of different ML models [50]. The data is typically split into an 80/20 ratio, with 80% used for training the model and 20% reserved for testing purposes [50]. Each individual value within a dataset is commonly referred to as a data point, analogous to a row in a table that presents various patient data such as blood pressure, red and white blood cell counts, and other relevant attributes [46, 48]. Within a dataset, features correspond to the input variables that describe each available example or data point, while labels pertain to the output variable or target value. When a training dataset is employed to refine a model, it undergoes a training process often referred to as an epoch [45]. During this iterative process, the model learns from the training dataset, adjusting its internal parameters to optimize its ability to make accurate predictions or classifications based on the input features and their associated labels [46, 48]. Lastly, the concept of informational entropy holds significance in the realm of AI models as it quantifies the informational organization within a given system or dataset. Introduced by Shannon, entropy represents the average amount of information contained within a message or signal, typically measured in bits. It is calculated based on the probability distribution of each potential outcome within the system or dataset [51]. Machine Learning is fundamentally rooted in the utilization of existing data, making it data-driven at its core [52]. Unlike traditional rule-based systems, ML techniques do not rely on predefined rules but instead learn from examples derived from available data and make predictions or decisions based on incoming data. ML encompasses three primary types: supervised learning (SL), unsupervised learning (UL), and reinforcement learning (RL) [46, 48]. Supervised learning is further classified into two main categories: classification and regression. Classification involves using labeled categorical data to produce categorical results, while regression involves utilizing unlabeled data with continuous values. On the other hand, unsupervised learning deals with datasets that consist of numerous unlabeled data points [48]. In the context of this research, supervised learning was selected as the starting point due to its ability to create a dataset with appropriate labels. This choice is particularly significant in a relatively new research domain where large, labeled datasets specifically related to LMS are not yet readily available.

3 Methodology An investigation was conducted to explore a set of Uis for the heuristic evaluation of LMS using ML models. This phase was based on a collection of 800 Uis obtained through a literature review of the theoretical baseline developed by Jakob Nielsen [19]. The collection of Uis can be found at: https://jc7.co/p2dg231. The investigation of Uis for AI was carried out using a novel systematic process that aimed to identify the parameters

Investigating Usability Indicators

141

necessary to utilize Uis as a foundation for constructing datasets in AI models. The systematization process involved the following stages, which will be discussed in the following sessions. Working Document: A spreadsheet based on the set of 800 Uis previously defined, available at: https://jc7.co/p2dg232. Subjectivity Factor (SF) Identification: Establishment of an arbitrary classification based on the proximity of the usability indicators to aspects that are difficult to quantify or measure, as determined by the interface analyst’s evaluation. The SF aimed to determine if there were chances of different evaluators interpreting a criterion differently. The primary assumption adopted was to consider SF equal to 1 (positive subjectivity) whenever an adjective was encountered. SF Filter: A filter was employed to acquire Ui whose SF equaled zero. The objective was to identify indicators that could be studied through simplified, automated evaluation without the need to adopt aspects inherent to the concepts of “good” or “bad” (in this case, with negative subjectivity – equal to 0). Graphics and data are available here: https://jc7.co/p2dg233 on page 01. Definition of Data Types Associated with Uis: A suggestion for data types related to the analysis context of Uis by AI based on the following classification – Interval Data – available in the form of any intervals; Binary Data – related to the simple binary identification of the type Yes/No; On/Off; Available/Unavailable, etc.; and N/A – when it was not possible to define a data classification through immediate analysis of the context in which the Ui would be inspected. Graphics and data are available here: https://jc7.co/ p2dg233 on pages 11 and 12. Examples Related to the Identified Data: Include situations such as “conditional display occurs/conditional display does not occur/”; “status indicated/status not indicated”; “feedback occurs/feedback does not occur”; “notification occurs/notification does not occur,” among others. A restrictive classification was not defined for the examples since the purpose was to identify how the system “communicated” through the data to inform that a certain Ui had been satisfied. Graphics and data are available here: https://jc7.co/ p2dg233 on pages 13 and 14. Types of Data Related to AI Models: Based on discrete data – can assume only several countable values; and continuous data – can assume all possible values within a certain range. The objective was to find indications of potential relationships between the types of data and AI tasks suggested in the following phases. Graphics and data are available here: https://jc7.co/p2dg233 on pages 15 and 16. Proposal for an Algorithmic Evaluation of Ui by AI: We adopted a work protocol inspired by the concept of “Designer’s Proxy” from Semiotic Engineering (SE), according to which the interface of a system acts as a communicational artifact on behalf of the designer in the user interaction process [53]. In the context of this study, we considered that “AI would help the evaluator inspect the system interface”; therefore, it would act as a kind of “Evaluator’s Proxy”. The meta-communicational approach of SE advocates

142

D. G. S. Júnior et al.

that certain assumptions must be met for everything to go well. Following this principle of thought, we considered it necessary to verify what assumptions would be required for AI to contribute to making the interface evaluation process easier for the evaluator. The approach was named the EU-Machine Protocol, and it consisted of the simple exercise of asking oneself aloud: “What do I – as a machine, alone – need to do to evaluate this usability indicator without asking for help from a human being?”. During the procedure, numerous actions required to inspect the indicator on the interface were enunciated formally. These vernacular expressions were then transcribed into algorithmic form. The “Portugol” language (known as structured Portuguese in Pascal programming) was utilized as a logical reference to discern the necessary steps for evaluating the Ui on the interface. This information was subsequently converted into AI-driven analysis statements. Some indicators proved unfeasible to analyze or excessively intricate, primarily due to factors about their limited level of detail in the description, subjectivity, or complexity. These indicators were flagged and served as a foundation for identifying areas where indicator decomposition and/or system adaptation were deemed necessary. In total, 256 indicators could be examined from the AI perspective (32% of the originally proposed 800 indicators) based on our analysis. For instance, the algorithm applied to Ui-40 (1 – System Status Visibility (Heuristic) COMMUNICATION (Level 1) ALERTS (Level 2) – executed action (Level 3)) corresponded to the following statement: “Define the action to be monitored (step 1). Define an identifier for the action’s state (step 2). Define a flag to capture the action’s state (step 3). Define an element to display alerts based on the flag’s state (step 4). Define a flag for the element’s state displaying the alert (step 5). Compare the action’s flag state with the alert’s flag state (step 6). Validate if the states relate as expected.” The same procedure was applied to all usability indicators. Proposed algorithms can be found here: https:// jc7.co/p2dg232. Number of Process Steps (NPS): A variable indicating the count of steps necessary to implement the suggested algorithm minus one. Analyzing the statement from the previous example, we have NEPS = 6 (7 steps – 1). Graphics and data are available here: https://jc7.co/p2dg233 on pages 17 and 18. Decomposition or Disambiguation Requirement (DDR): The aim was to investigate whether the Ui was defined clearly, descriptively, and objectively without needing further technical specifications, elaboration, or disambiguation of the analysis proposal expressed in its statement. The variable DDR could take on two possible values. DDR = 0 indicates that the Ui did not require decomposition or disambiguation, and DDR = 1 indicates that the Ui required decomposition or disambiguation. For example, Ui15 was defined with DDR = 1. The statement “conditional display of GUI elements” was deemed insufficiently descriptive and objective, as the term “conditional” does not explicitly define under which conditions the display should occur, and the term “GUI elements” does not objectively indicate which interface elements should be conditionally displayed. On the other hand, Ui-122 was marked with DDR = 0. The statement “LIMITS/ALERTS/character quantity” was considered sufficiently descriptive and objective, as the terms “ALERTS” and “LIMITS,” combined with the term “character quantity,” indicate the need to confirm the presence of that Ui, in the form of an alert,

Investigating Usability Indicators

143

that demonstrates restricted character insertion in a specific field of a form. Graphics and data are available here: https://jc7.co/p2dg233 on pages 6 and 7. System Adaptation and/or Update (SAU): Based on the Ui statement and the suggested algorithm, the need for system adaptation to be inspected from an AI approach was determined. AI tasks involve creating a dataset to train and test models. In some cases, data collection must be done manually, either through visual inspection or, ideally, through automated techniques such as reading logs, code inspection, available browser cookies, and checking CSS and HTML properties, among other tasks. However, there are cases where automated data collection is not possible, requiring initial system adaptation to provide specific functionalities. For example, recording data in files, and storing information in databases, among others. The variable could take on two possible values. SAU = 0 – to express that potential AI inspection of the Ui did not require system adaptation and/or update, and SAU = 1 – to express the need for system adaptation and/or update. For example, Ui-424 was considered SAU = 1. Its AI analysis statement (Algorithm) is “validate the existence of a placeholder in the form field through source code analysis”. The justification relies on the idea that based on a trivial source code inspection, it is possible to confirm the existence of a “placeholder” tag in the HTML library (https://www.w3schools.com/tags/att_input_placeholder.asp). Therefore, there is no need to adapt the system for the information to be available, although methods can be considered to automate data collection to reduce or eliminate manual work. Ui-352 has an SAU = 0 and has the following AI analysis statement (Algorithm): “Define the current state of the system based on storage via a database, files, cookies, or another strategy. Define the current page. Validate if changes in data have already been stored. If they are not stored, save them in a temporary virtual environment. Define a virtual space to store the data. Validate the filling of the virtual space with the stored data. Associate a flag to identify if the action was satisfactory”. The justification is that the algorithm statement sought to address the usability indicator’s specified requirement, considering strategies that required altering the system’s structure by adding functionalities such as storing data in databases, and files, among others, and defining and associating a flag to identify the monitored action. Graphics and data are available here: https://jc7. co/p2dg233 on pages 23 and 24. Complexity Factor (CF): An arbitrary scale indicating the degree of effort required to apply the proposed algorithm. The scale considered the number of steps required for AI evaluation, the need for Ui disambiguation, and the system adaptation requirement. The established mathematical relationship adopted weights and examples can be found in supplementary material at the end of this section. Graphics and data are available here: https://jc7.co/p2dg233 on pages 9 and 10. Machine Learning Type (MLT): The aim was to identify the type of ML, whether it falls under the categories of supervised (SL), unsupervised (UL), or reinforcement learning (RL), based on the Ui statement and the proposed approach for inspecting it from the AI perspective, as outlined in the designed algorithm. The majority of MLT was classified as SL.

144

D. G. S. Júnior et al.

AI Task: The objective was to identify tasks such as classification and linear regression, typically adopted as fundamental approaches within ML. Graphics and data are available here: https://jc7.co/p2dg233 on pages 19 and 20. Decision Purpose: The purpose was to identify the decision objective of the suggested AI task for inspecting the Ui, based on the prediction and diagnosis categories. This classification followed the analysis suggestion proposed by the “AI Map” document developed by the Japanese Society for Artificial Intelligence [54]. Graphics and data are available here: https://jc7.co/p2dg233 on pages 21 and 22. Data Processing: The data and graphs were processed in the Power BI platform and are presented below.

4 Discussion This section presents the results obtained for the Ui classification stage, later used in another study to develop a methodology to adopt AI as part of the interface inspection process based on heuristic evaluation. Only Uis whose FS was determined to be equal to zero were analyzed. The following sections discuss the results found. Types of Associated Data: Approximately 82.95% (292) of the evaluated Uis could be analyzed through the collection of binary data (YES/NO) distributed across all heuristics. 17% (60) of the Uis were associated with interval data types, which were more prevalent in heuristics 5 – Error Prevention, 4 – Consistency and Standards, and 8 – Static and Minimalistic Design. Binary data can be associated to diagnose the interface, where scrutiny is performed to validate the existence of a specific element of interest through simple inspection of its presence in the expected location. Examples Related to the Data: The most recurring example across the heuristics was “Indicated Status/Not Indicated Status” (274 occurrences – 78%), except for heuristics 5 – Error Prevention and 4 – Consistency and Standards, where the most recurring example was “Records variation in item quantity” (38 occurrences – 10%). It is possible to establish a relationship between the observed pattern in these two heuristics and the associated data type, which was more strongly linked to interval data for both. Other relevant examples include “Records variation in item quantity” and “Warning occurs/Warning does not occur”. The complete list of examples is available at: https:// jc7.co/p2dg233 on page 13. Types of Input Data: Approximately 85.22% (300) of the evaluated Uis were characterized as being related to discrete values, while approximately 14.77% (52) of the Uis were associated with continuous values. Discrete data may be associated with classification tasks in AI. These values are predominant across all heuristics, except for heuristics 5 – Error Prevention and 4 – Consistency and Standards, which have Uis where the input data for the AI model is of continuous type in more than 50% of the cases, possibly because their indicators were related to interval data in several situations. Evaluation Algorithm: It was utilized to elucidate the requirements for using AI models to assist in evaluating Uis in interfaces. The algorithm contributed to identifying the

Investigating Usability Indicators

145

necessary inputs to perform the procedure and possible expected outputs at the end of the inspection. The statements from this phase were essential in highlighting the variables NPS, SAU, and CF. Number of Process Steps: The most frequent number of steps corresponds to 2, 3, 4, 6, and 5. The number of steps is directly related to the proposed algorithm. The maximum number found was 13, and the minimum was two within a range established between 1 and 15. It is expected that new rounds of algorithmic analysis will make the process more precise and detailed, increasing the number of steps and expanding the technical requirements for implementing the proposals. The task of proposing algorithms to evaluate Uis requires the involvement of professionals from design, web development, and artificial intelligence. The presented results reflect only one round of algorithm analysis to investigate the indicators, and further rounds would likely impact the number of steps, increasing them to further clarify the proposed use of AI approaches. Another point is the practical implementation of the algorithms. The execution of the proposed tasks in an experimental phase, based on initial prototypes, may demonstrate the need to modify the algorithm and, consequently, the number of steps, as well as establish the need to relate and evaluate indicators together in an inseparable manner, to see if different results are obtained. Decomposition/Disambiguation of usability indicators: The objective was to identify if the available information was sufficient to directly inspect the interface or if more details would be necessary. Approximately 86% (277) of the evaluated Uis required some form of a declarative specification to make them less ambiguous, more descriptive, and direct. Approximately 13% (45) of the Uis had statements considered direct and objective. The elimination criterion adopted was an FS equal to 0. Among the 352 Uis with an FS determined to be equal to zero, 96 were subsequently classified as ambiguous or requiring significant decomposition, which could potentially impact the suggestion of other AI-related aspects. During the study, it was observed that to evaluate interfaces with AI-based approaches, adaptations need to be made to the platforms to capture data that will be used to build datasets for the AI models. Therefore, it is necessary to prepare the systems to mark points of interest and track user actions considering privacy and security issues, among other modifications. For example, in several indicators, the use of flags (markers) in the source code was suggested in the algorithm proposals. However, most systems are not yet geared toward data collection in this manner. Some studies identified in the exploratory phase of this research demonstrate that the evaluation of LMS interfaces, for example, was done through log analysis, database analysis, or even directly by manually collecting data at specific “moments” when researchers examined the system [1]. Therefore, automated data collection to feed evaluation systems should be an objective to be considered by organizations that wish to evaluate their platforms in an automated manner. Complexity Factor: The implementation of AI-based approaches to inspect interfaces based on the Uis from the obtained corpus was considered high for approximately 44% (157) of the indicators, medium for approximately 49% (173) of the indicators, and low for 6.25% (22) of the studied Uis. Overall, the implementation of AI approaches to evaluate Uis in a heuristic evaluation of interfaces is considered high or medium for

146

D. G. S. Júnior et al.

over 70% of the studied Uis for each of the heuristics. This is mainly due to the need for system adaptation for data collection, in addition to the inherent complexity of algorithm implementation. AI Task: Approximately 84% (296) of the studied Uis were associated with classification tasks, while 16% (56) were associated with regression tasks. The relevance of the classification task may be related to the types of statements found in the Uis, where the analysis aims to confirm the presence of certain objects of interest in the interface. In other words, it is about determining whether an analyzed element is or is not where it should be, expressed as it should be, or possesses the expected characteristics, among other situations that allow for relatively easy grouping to classify the indicators. For example, in the case of Ui-40 (1 – System Status Visibility (Heuristic) – COMMUNICATION (Level 1) – ALERTS (Level 2) – action executed (Level 3)), it aims to verify if a specific alert was displayed in the interface when a certain action was performed. In this context, either the alert was displayed, and the usability indicator evaluation allows for satisfactory classification, or vice versa; if it does not occur, it is impossible to confirm the alert’s presence, indicating an unsatisfactory situation. Thus, it is possible to establish two sets and classify the indicators based on the result using a classification process. While identifying AI tasks to investigate the Uis, it was observed that, in most cases, it is necessary to combine indicators for a heuristic evaluation of the interface with AI support to become more meaningful. This is because there are complex relationships between the Ui established by the evaluator during the inspection, making the inspection richer but also challenging and complex. This presents an exciting opportunity for adopting AI-supported practices that can automate, simplify, and reduce the cognitive effort of the evaluator in establishing relationships between different analyzed features simultaneously and demonstrates how the pathway to have a complete AI technique along the process is quite challenging now. Opportunities for applying simple and multiple linear regression models and classification tests were identified in the subsequent phase of the research. Linear regression is an interesting task to establish relationships between different features in a dataset, as it provides a simple, easily interpretable, flexible, and fast learning method that involves a simple and linear relationship between input and output variables. Methods like linear regression offer a good opportunity for initial and exploratory studies where there is an interest in understanding the essential relationships between variables. Multiple linear regression, on the other hand, can identify more elaborate relationships precisely because it incorporates multiple variables as features in the learning process. Purpose of the Technique: Most Uis were associated with the diagnosis. This can be explained by the propensity of heuristic evaluation to diagnose the interface, validating attention to certain requirements established in the Ui statements. The adoption of AI approaches to support the process of interface inspection should consider complex actions and events occurring in the interaction environment, some of which are difficult to track, capture, and manage through automated data collection processes. In several analyzed situations, there are features related to the processing and storage of data in databases or browser caches to record the various events that occur in the system based on user actions or platform events. These actions occur on the system’s front end but require logs to be stored locally on the user’s computer via

Investigating Usability Indicators

147

cookies or other formats and then transferred to a database for analysis. At the same time, measures need to be taken to ensure that the infrastructure of the environment can have checkpoints for the indicators, using flags or markers that can identify the data of interest. In various situations, these and other contexts require changes and adaptations in system development so that AI can be adopted to contribute to inspecting and evaluating interfaces. During the classification process, it was observed that AI could be a strategy to address specific interface analysis problems. However, several Uis were still open-ended and lacked clear explanations about what should be assessed in the interface. Our perception regarding the use of AI techniques for Ui evaluation, aiming to inspect interfaces, is that the effort to implement such approaches is significantly greater than making the necessary adaptations in systems to ensure that interfaces meet the minimum requirements for automated analysis, especially in terms of providing the necessary data for building AI models. Therefore, we advocate for the construction and/or adaptation of systems oriented toward AI-based evaluation from the ground up. Initial analyses of this exploration demonstrated that a significant portion of Uis can be considered from a binary perspective. This is plausible, considering that they would already simplify the usability concepts that are expected to be evaluated. It is worthwhile to reconsider Uis or even new heuristics that look into the internal structures of systems, considering their architecture, source code, and backend, in areas that are not typically inspected but have an impact on system usability. When considering an interface evaluation process in the traditional approach, we are “looking” at the interface and identifying issues that resulted from a development process that supposedly did not meet the expected requirements. However, by considering this internal perspective at a deeper level, it may be possible to address problems before they impact the surface (interface). In some cases, adopting AI approaches will not require modifying systems to obtain the necessary data for creating models, but it requires further studies considering this approach. During the collection and classification of Ui’s, in several situations, it was observed, for example, that it was necessary to validate the presence of a marker (a type of flag) in the source code without requiring adaptations/updates to the software structure, only requiring finding references that indicate the presence of a specific element expressed in the source code. Experimental tests supported by the adoption of a methodology oriented towards the evaluation of Uis with the help of AI were conducted in the following stages of the research. However, it is worth noting that this analysis, in many cases, is still performed either fully or partially manually. Therefore, providing external or internal resources to the analyzed platforms to enable data collection can be a favorable factor. We also found that to adopt an AI approach to reduce the evaluator’s effort in repetitive activities it is necessary to design more restrictive statements to reduce the chances of different interpretations; otherwise, evaluators will constantly need to be involved in interpreting the statements. In the future, it will be necessary to adapt systems to emit certain signals about the user experience, allowing data to be collected while respecting privacy rules.

148

D. G. S. Júnior et al.

5 Conclusion During the classification process, it was observed that AI can be a strategy to be adopted to address specific problems in interface analysis. However, several Uis were still openended and lacked clear explanations about what should be assessed in the interface. Our perception regarding the use of AI techniques for Ui evaluation, to inspect interfaces is that the effort to implement such approaches is significantly greater than making the necessary adaptations in systems to ensure that interfaces meet the minimum requirements for automated analysis, especially in terms of providing the necessary data for building AI models. Therefore, we advocate for the construction and/or adaptation of systems that are oriented towards AI-based evaluation from the ground up. Initial analyses of this exploration demonstrated that a significant portion of Uis can be considered from a binary perspective. This is plausible, considering that they would already simplify the usability concepts that are expected to be evaluated. It is worthwhile to reconsider Uis or even new heuristics that look into the internal structures of systems, considering their architecture, source code, and backend, in areas that are not typically inspected but have an impact on system usability. When considering an interface evaluation process in the traditional approach, we are “looking” at the interface and identifying issues that resulted from a development process that supposedly did not meet the expected requirements. However, by considering this internal perspective at a deeper level, it may be possible to address problems before they impact the surface (interface). In some cases, adopting AI approaches does not require modifying systems to obtain the necessary data for creating models. In several situations, it was observed, for example, that it was necessary to validate the presence of a marker (a type of flag) in the source code without requiring adaptations/updates to the software structure, only requiring finding references that indicate the presence of a specific element expressed in the source code. However, it is worth noting that this analysis, in many cases, is still performed either fully or partially manually. Therefore, providing external or internal resources to the analyzed platforms to enable data collection can be a favorable factor. We also found that to adopt an AI approach to reduce the evaluator’s effort in repetitive activities, it is necessary to design more restrictive statements to reduce the chances of different interpretations; otherwise, there will be a constant need to involve evaluators to interpret the statements. In the future, it will be necessary to adapt systems to emit certain signals about the user experience, allowing data to be collected while respecting privacy rules. Acknowledgments. This study was supported by UNIDCOM under a Grant by the Fundação para a Ciência e Tecnologia (FCT) No. UIDB/DES/00711/2020 attributed to UNIDCOM/IADE – Unidade de Investigação em Design e Comunicação, Lisbon, Portugal.

References 1. Júnior DGS (2019) Usability and User Experience Evaluation of Learning Management Systems. In: DCC. Unidcom, IADE, Lisbon 2. Zaharias P, Koutsabasis P (2011) Heuristic evaluation of e-learning courses: a comparative analysis of two e-learning heuristic sets. Campus-Wide Inform. Syst. 29(1):45–60

Investigating Usability Indicators

149

3. Takashi Nakamura W, Harada Teixeira de Oliveira E, Conte T (2017) Usability and user experience evaluation of learning management systems – a systematic mapping study. In: Proceedings of the 19th international conference on enterprise information systems [Internet]. SCITEPRESS – Science and Technology Publications, Porto, Portugal; [cited 2020 Jun 20], pp 97–108. https://doi.org/10.5220/0006363100970108 4. Freire LL, Arezes PM, Campos JC (2012) A literature review about usability evaluation methods for e-learning platforms. Work 41:1038–1044 5. Cota CXN, Díaz AIM, Duque MÁR (2014) Evaluation framework for m-learning systems: current situation and proposal. In: Proceedings of the XV international conference on human computer interaction – Interacción ’14 [Internet]. Puerto de la Cruz, Tenerife, ACM Press, Spain [cited 20 Jun 2020], pp 1–3. http://dl.acm.org/citation.cfm?doid=2662253.2662265 6. Júnior DGS, Hernández-Ramírez R, Estima J (2022) Systematic mapping of methods used to evaluate the usability and UX of learning management systems. In: Martins N, Brandão D (eds) Advances in design and digital communication II: Proceedings of the 5th international conference on design and digital communication, Digicom 2021, 4–6 Nov 2021, Barcelos, Portugal. Springer International Publishing, Cham, pp 122–133. https://doi.org/10.1007/9783-030-89735-2_11 7. Hassenzahl M, Law E, Ebba H (2006) User Experience – Towards a unified view. UX WS NordiCHI’06: COST294-MAUSE 161 8. Cantabella M, López B, Caballero A, Muñoz A (2018) Analysis and evaluation of lecturers’ activity in learning management systems: subjective and objective perceptions. Interact Learn Environ 26(7):911–923 9. Mtebe JS, Kissaka MM (2015) Heuristics for evaluating usability of Learning Management Systems in Africa. In: 2015 IST-Africa conference [Internet]. Lilongwe, IEEE, Malawi [cited 20 Jun 2020], pp 1–13. http://ieeexplore.ieee.org/document/7190521/ 10. Navarro CX, Molina AI, Redondo MA, Juarez-Ramirez R (2016) Framework to evaluate m-learning systems: a technological and pedagogical approach. IEEE R Iberoamericana Tecnologias Aprendizaje. 11(1):33–40 11. Nielsen J, Mack RL (eds.) Usability inspection methods. Wiley, New York (1994), 413 p. 12. Hosie P, Schibeci R, Backhaus A (2005) A framework and checklists for evaluating online learning in higher education. Assess Eval High Educ 30(5):539–553 13. Mtebe JS, Kissaka MM (2015) Heuristics for evdaluating usability of Learning Management Systems in Africa. In: 2015 IST-Africa Conference [Internet]. IEEE, Lilongwe, Malawi [cited 2019 Jun 4], pp 1–13. http://ieeexplore.ieee.org/document/7190521/ 14. Oztekin A, Kong ZJ, Uysal O (2010) UseLearn: a novel checklist and usability evaluation method for eLearning systems by criticality metric analysis. Int J Ind Ergon 40(4):455–469 15. Part 11: Usability: Definitions and concepts (ISO 9241-11:2018). BSI Standards Publication (2018) 16. Standard I. ISO 9241-210 (2010) 17. W3C Web Accessibility Initiative (WAI) (2018) Web Content Accessibility Guidelines (WCAG) Overview [Internet]. Web Accessibility Initiative (WAI). [cited 7 Jan 2019]. https:// www.w3.org/WAI/standards-guidelines/wcag/ 18. Nielsen J (1994) Enhancing the Explanatory Power of Usability Heuristics 19. Junior DGS, Hernández-Ramírez R, Estima J (2023) Systematic mapping of criteria used to evaluate the usability and UX of learning management systems. In: Perspectives on design and digital communication: research, innovations and best practices. Springer, Portugal 20. Barnum CM (2021) Usability testing essentials: ready, set...test! 2nd edn. Morgan Kaufmann, Amsterdam 448 p 21. Nielsen J (1994) Heuristic Evaluation: How-To: Article by Jakob Nielsen [Internet]. Nielsen Norman Group. [cited 27 Sep 2022]. https://www.nngroup.com/articles/how-to-conduct-aheuristic-evaluation/

150

D. G. S. Júnior et al.

22. Wong E (2022) Heuristic Evaluation: How to Conduct a Heuristic Evaluation [Internet]. The Interaction Design Foundation. [cited 25 Sep 2022]. https://www.interaction-design.org/lit erature/article/heuristic-evaluation-how-to-conduct-a-heuristic-evaluation 23. Heuristic evaluations vs. usability testing [Internet] (2001) Human Factors International. [cited 27 Sep 2022]. http://www.humanfactors.com 24. Carroll JM (1997) Human-computer interaction: psychology as a science of design. Annu. Rev. Psychol. 48:61–83 25. Brangier E, Desmarais M, Alexandra N, Tep SP (2015) Évolution de l’inspection heuristique: vers une intégration des critères d’accessibilité, de praticité, d’émotion et de persuasion dans l’évaluation ergonomique. Journal d’Interaction Personne-Système 4(1):16 26. Brangier E, Desmarais MC (2014) Heuristic inspection to assess persuasiveness: a case study of a mathematics e-learning program. In: Marcus A (ed) DUXU 2014, vol 8517. LNCS. Springer, Cham, pp 425–436. https://doi.org/10.1007/978-3-319-07668-3_41 27. Jordan PW (1998) An introduction to usability. Taylor & Francis, London, Bristol, Pa 120 p 28. Nielsen J (2010) Usability engineering. Nachdr. Kaufmann, Amsterdam 362 p 29. Scapin DL, Bastien JMC (1997) Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behav Inform Technol 16(4–5):220–231 30. Bennett JL (ed.) (1984) Visual display terminals: usability issues and health concerns. Prentice-Hall, Englewood Cliffs, N.J. 297 p 31. Scapin DL (1990) Organizing human factors knowledge for the evaluation and design of interfaces. Int J Human-Comput Interact. 2(3):203–229 32. Shneiderman B, Plaisant C (2004) Designing the user interface: strategies for effective humancomputer interaction, 4th edn. Pearson/Addison Wesley, Boston, p 652 33. Shackel B (2009) Usability – Context, framework, definition, design and evaluation. Interact Comput 21(5–6):339–346 34. (2016) Advances in ergonomics in design. Springer Berlin Heidelberg, New York, NY 35. Marcus A, Wang W (eds) (2017) Design, user experience, and usability: designing pleasurable experiences: 6th international conference, DUXU 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9-14, 2017, Proceedings, Part II. Springer International Publishing, Cham 36. Marcus A, Wang W (eds) (2017) DUXU 2017, vol 10288. LNCS. Springer, Cham. https:// doi.org/10.1007/978-3-319-58634-2 37. Marcus A, Wang W (eds) (2017) DUXU 2017, vol 10290. LNCS. Springer, Cham. https:// doi.org/10.1007/978-3-319-58640-3 38. (2017) Advances in usability and user experience. Springer Berlin Heidelberg, New York, NY 39. Ahram TZ, Falcão C (eds) (2019) Advances in usability, user experience and assistive technology. In Proceedings of the AHFE 2018 international conferences on usability & user experience and human factors and assistive technology, held on 21–25 July 2018, in Loews Sapphire Falls Resort at Universal Studios, Orlando, Florida, USA [Internet]. Springer International Publishing, Cham [cited 7 Jan 2019]. (Advances in Intelligent Systems and Computing; vol 794). https://doi.org/10.1007/978-3-319-94947-5 40. Michel G, Brangier E, Brun M (2014) Ergonomic principles to improve the use of cognitive stimulation systems for the elderly: a comparative study of two software tools. In: Stephanidis C, Antona M (eds) Universal access in human-computer interaction. aging and assistive environments: 8th international conference, UAHCI 2014, held as part of HCI international 2014, Heraklion, Crete, Greece, 22–27 June 2014, Proceedings, Part III. Springer International Publishing, Cham, pp 147–154. https://doi.org/10.1007/978-3-319-07446-7_15 41. Némery A, Brangier E (2014) Set of guidelines for persuasive interfaces: organization and validation of the criteria. J Usability Stud 9(3):24

Investigating Usability Indicators

151

42. Duczman M, Brangier E, Thévenin A (2016) Criteria based approach to assess the user experience of driving information proactive system: integration of guidelines, heuristic mapping and case study. In: Rebelo F, Soares M (eds) Advances in ergonomics in design: proceedings of the AHFE 2016 international conference on ergonomics in design, 27–31 July 2016, Walt Disney World®, Florida, USA. Springer International Publishing, Cham, pp 79–90. https:// doi.org/10.1007/978-3-319-41983-1_8 43. Urrutia JIG, Brangier E, Cessat L (2017) Is a holistic criteria-based approach possible in user experience? In: Marcus A, Wang W (eds) DUXU 2017, vol 10288. LNCS. Springer, Cham, pp 395–409. https://doi.org/10.1007/978-3-319-58634-2_29 44. Urrutia JIG, Brangier E, Senderowicz V, Cessat L (2018) Beyond “usability and user experience”, towards an integrative heuristic inspection: from accessibility to persuasiveness in the UX evaluation: a case study on an insurance prospecting tablet application. In: Ahram T, Falcão C (eds) Advances in usability and user experience: proceedings of the AHFE 2017 international conference on usability and user experience, 17–21 July 2017, The Westin Bonaventure Hotel, Los Angeles, California, USA. Springer International Publishing, Cham, pp 460–470. https://doi.org/10.1007/978-3-319-60492-3_44 45. Hurbans R (2020) Grokking artificial intelligence algorithms. Manning Publications, New York, p 362 46. Frankish K, Ramsey WM (eds) (2014) The Cambridge handbook of artificial intelligence. Cambridge University Press. https://doi.org/10.1017/CBO9781139046855 47. Domingos P (2015) The master algorithm: how the quest for the ultimate learning machine will remake our world. Basic Books, a member of the Perseus Books Group, New York 329 p. 48. Russell SJ, Norvig P, Davis (2010) Artificial intelligence: a modern approach, 3rd edn. Prentice Hall, Upper Saddle River (Prentice Hall series in artificial intelligence), 1132 p 49. Liang YD (2015) Introduction to Java programming, 10th edn. Pearson, Boston, p 1320 50. Géron A (2022) Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow. O’Reilly Media, Inc., 1351 p 51. Shannon CE (1948) A mathematical theory of communication. The Bell Syst Tech J 27:379– 423, 623–656 52. Mitchell TM (1997) Machine learning. McGraw-Hill, New York (McGraw-Hill series in computer science), 414 p 53. Júnior DGS (2017) A Interação no Gerenciador de Conteúdo WordPress sob a perspectiva da Semiótica. Universidade FUMEC. http://www.fumec.br/revistas/sigc/article/view/5240 54. Japanese Society for Artificial Intelligence. Al Map Beta 2.0. Al Map Task force (2021) [cited 18 Jul 2022]. https://www.ai-gakkai.or.jp/pdf/aimap/AIMap_EN_20210901.pdf

Critical Digital: A Taxonomy to Classify Digital Integration in the Museum Domain Fernando Sousa(B)

and Francisco Providência

Department of Communication and Art, University of Aveiro, Aveiro, Portugal {fernando.sousa,fprovidencia}@ua.pt

Abstract. The evolution of the information and communication ecosystem has led to significant changes in the museum visitor’s behavior and expectations, as they now manifest a desire for participation, collaboration, and sharing – suggesting the corresponding need to readjust the institution’s discourse and communication approaches through digital integration. Despite the inherent qualities of digital media, they can be presented in various platforms, interfaces, and devices, contributing to the lack of consensus and consistency in the terminology used in academia and among designers, technologists, programmers, and developers, resulting in numerous classification proposals. This condition tends to hinder a practical and, most importantly, insightful and critical appropriation of digital technology among museum professionals. Therefore, a three-dimensional taxonomy is proposed. The first dimension – virtualization – is created based on the reality-virtuality continuum and allows assessing possible variations and compositions of real and virtual objects. The second dimension – observation distance – is focused on the formal design contours of video displays, such as the dimension and positioning related to the user. The third dimension – interactivity – classifies, based on types of interaction, the extent to which users actively modify the content in a mediated environment in real-time. Each interdependent dimension is divided into six discrete classes. This tripartite classification system streamlines the main qualitative characteristics within different digital mediation systems, serving as a theoretical framework for digital integration centered on sustainability, accessibility, inclusivity, critical thinking, and individual subjectivity during the museum experience. Keywords: Design · Digital Media · Museography · Taxonomy

1 Introduction The 20th century is marked by the museum’s departure from its traditional framework of “showcase museum” – a term introduced by Derrick de Kerckhove [1, p. 178] – configured as an institution centered on the heritage conservation of collections of objects identified as having significant cultural value [2]. In the 1970s, sociological scrutiny prompted a reevaluation of the museum and its practices. This examination, which viewed the museum as a societal constructor, focused on monitoring its ideological, political, and educational contours and declinations. As a © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 152–164, 2024. https://doi.org/10.1007/978-3-031-47281-7_12

Critical Digital: A Taxonomy to Classify Digital

153

result, the traditional notions of custodianship, preservation, and interpretation gradually lost prominence as the institution focused on its visitors’ educational engagement and the curator’s performance, whose responsibility shifted towards crafting a social and political narrative [3]. Authors including John Falk, Lynn Dierking, and Nina Simon evidence the growing need to reestablish a strong connection with the public and expand their relationship with museum artifacts on display: to create meaningful and participatory museum experiences aligned with evolving behaviors, interests, and personal identification needs and desires [4, 5]. In this context – particularly within an institution increasingly devoted to society – digital media emerges as an opportunity to integrate each visitor and enrich their experience regarding narrative and museographic content, and the museum endeavors to stay abreast of the latest technological mediation trends [6]. We consider the optimal museum experience to be the one offering “an opportunity to question reality, the world, and the present time” while preserving “critical thinking, plurality, awareness, perspective-taking, and the visitor’s subjectivity as an individual destined for freedom and self-determination”, guaranteeing “liberty and access to various layers of knowledge and meaning in an unrestricted way,” distinguishing the moment of a visit from other moments of everyday life [7, pp. 502–504]. Thus, a pivotal question emerges: How can digital media fulfill this purpose, considering these are not inherent qualities of digital media itself? This review contributes to the literature by examining the formal contours of realityvirtuality technologies and their application, role, and meaning for the museum experience. For this purpose, we review previous interpretations and clarify some terminological misconceptions to establish a clear boundary and normalize the terms describing and categorizing the array of digital mediation scenarios. To that end, a three-dimensional taxonomy is proposed. The first dimension – virtualization – is created based on Milgram and Kishino’s reality-virtuality continuum [8] and allows for assessing possible variations and compositions of real and virtual objects. The second dimension – observation distance – is determined based on the categorization of displays proposed by Bimber and Raskar [9] for Augmented Reality displays and focuses on formal design contours, such as the dimension and their positioning and distance related to the user. The third dimension – interactivity – classifies the extent to which users actively participate in modifying the content present in a mediated environment in real time and is defined based on the types of interaction proposed by Sharp and colleagues [10]. This tripartite classification system is designed to enhance comprehension regarding the influence of these technologies on the museum experience and aid designers, technologists, and curators in effectively choosing the fitting terminology for each specific reality. Furthermore, its primary objective is to foster a deliberate and insightful integration of digital technology within the realm of museum professionals. The paper concludes by outlining its implications and suggesting directions for future research endeavors.

154

F. Sosua and F. Providência

2 Methodology To categorize the array of digital systems that support communication and technologically mediated experiences in a museographic context, we review, through document analysis, different taxonomies and classification proposals for these systems, which can be presented in various formats and devices, but possess specific characteristics allowing their identification and classification. This analysis aims to clarify terminological misconceptions, define boundaries, and standardize terminology to describe the different realities. According to Milgram and Kishino, the purpose of a taxonomy is to provide a structured framework that facilitates debate and reflection, in which development can be evaluated, research conducted, and data compared in a practical, scientific, and objective way [8]. We propose a three-dimensional taxonomy as a theoretical framework that allows framing descriptions of the designed digital experiences across different categories and, grounded on the resultant classification, infer specific qualitative characteristics and their implications for the museum experience. To fulfill the objective described above, this research is divided into the following tasks (T): (TA) Analysis of Milgram and Kishino’s reality-virtuality continuum, along with subsequent interpretations and adaptations; (TB) Conceptualization of an observation distance continuum based on scientific literature; (TC) Conceptualization of an interactivity continuum based on scientific literature; (TD) Design of a three-dimensional taxonomy (as a theoretical framework).

3 Visualization Modes and Virtualization To effectively categorize and classify digital media systems, it’s crucial to comprehend and clarify the fundamental characteristic that predominantly defines technologically mediated experiences – virtualization. Despite the inherent qualities of digital media, the boundaries between the different realities present in the scope of the so-called Extended Reality (XR) – which includes Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) – are not expressly defined in the literature, denoting an evident lack of consensus and consistency in the terminology used within academia and among designers, technologists, programmers, and hardware and software developers, culminating in numerous and different classification proposals [8, 11–14]. Given this conjuncture, we consider analyzing some of these proposals (TA) relevant and necessary to revisit the core attributes of these systems and clarify their terminology, thus establishing clear boundaries between the different realities. Among all the revised taxonomies, the “Reality-Virtuality Continuum” (Fig. 1), proposed by Milgram and Kishino [8], has served as the foundational reference for researchers to categorize the array of realities.

Critical Digital: A Taxonomy to Classify Digital

155

Fig. 1. Simplified “reality-virtuality continuum” representation based on [8].

This spectrum ranges from real to virtual environments at the extremes of the continuum. Real Environments (RE) encompass reality itself and include direct or indirect views (through a non-immersive video display) of a natural scene [8]. Virtual Environments (VE) are entirely computer-generated environments constituted exclusively by virtual objects with which the user interacts in real-time through a technological interface. In this class, immersive VR stands out. By definition, it immerses the user within a fully computer-generated environment, monopolizing their attention and disconnecting them from the real physical world. The generated environment constitutes a panoramic image that envelops the observer and may include multisensory stimuli, allowing exploration of that same VE. The authors also include nonimmersive continuous virtual environments (e.g., virtual worlds in digital games) in this category [8]. The remaining spectrum presupposes real and virtual objects overlapping and is termed Mixed Reality (MR) [8, 15, 16]. Within MR, AR consists of superimposing and integrating virtual elements (images, videos, sounds, virtual objects) in the user’s perceptive space, constituting a complement to reality [17, 18]. According to Azuma [19], an AR system should contain three fundamental characteristics: a combination of the real and the virtual, real-time interaction, spatial alignment, and temporal synchronization in a three-dimensional space (registration). The last class in MR is Augmented Virtuality (AV), which is significantly less explored and consists of superimposing real elements in virtual environments [16]. Upon analyzing this continuum, we conclude that its structure gradually evolves towards synthesis – virtualization – becoming increasingly prominent the overlapping between virtual and real elements, until it reaches complete synthesis. Milgram and Kishino additionally propose that this condition translates into a progressive increase in stimuli – synesthetic sensations – generated by the machine [8]. Numerous researchers have sought to deepen Milgram and Kishino’s taxonomy, suggesting the existence of new realities following the advent of more sophisticated digital systems. Mann [13] introduces the concept of mediation to the continuum to clarify the symbiosis between the real world and the virtual world beyond mere overlap, proposing an organization based on four distinct realities: Augmented Reality, Augmented Virtuality, Mediated Reality, and Mediated Virtuality. Jeon and Choi [12] present a taxonomy that discreetly considers visual and haptic stimuli, proposing a construction based on nine dimensions ranging from the real world

156

F. Sosua and F. Providência

(visual and haptic reality) to interactive virtual simulators (visual and haptic virtuality). This organization reinforces the importance of considering multiple stimuli. However, in this proposal, visual and haptic sensations are associated with using several different devices, as in the case of complex simulators. Flavián and colleagues [11] propose a new dimension to Milgram and Kishino’s continuum, called Pure Mixed Reality (PMR), to clarify the segment of the spectrum occupied by MR. According to the authors, PMR diverges from the mere superimposition of virtual elements onto the real physical environment (as seen in AR) and involves generating and integrating virtual elements, rendering them “indistinguishable” from the surrounding environment, privileging visual coherence. PMR is associated with holographic technologies supported by devices such as Microsoft HoloLens 2 [20] or Magic Leap 2 [21]. We propose the following continuum, divided into six discrete classes, to classify the degree of virtualization of digital media systems (Fig. 2):

Fig. 2. Classification of the degree of virtualization based on the reality-virtuality continuum.

The first class corresponds to the real world, stripped of technological mediation, in which the subject interacts only with real physical objects. Direct or indirect visualization of reality through a non-immersive fixed visual display is excluded. The opposite end of the spectrum encompasses fully synthesized immersive environments consisting exclusively of virtual objects (e.g., immersive virtual reality) in which a panoramic image integrates the observer and may be accompanied by multisensory stimuli that allow exploring that same VE; and partially immersive environments in which the real world and physical objects in the user’s environment play a role (or interfere with) the computer-generated scene (e.g., large-scale projection; CAVE). The remaining spectrum corresponds to the juxtaposition of the real with the virtual and includes: Digital Image (DI), static or dynamic, constitutes a window-on-the-world; an electronically or digitally generated two-dimensional technical image, typically presented on a non-immersive fixed display. DI also encompasses non-immersive, continuous, fully synthesized environments viewed through a non-immersive fixed display; Augmented Reality (AR) is characterized by the superimposition of virtual content in the user’s perceptual field; Pure Mixed Reality (PMR) is characterized by the merging of virtuality and reality content in the user’s perceptual field as a more integrated, visually cohesive, and immersive form of AR;

Critical Digital: A Taxonomy to Classify Digital

157

Augmented Virtuality (AV) retains the definition proposed by Milgram and Kishino [8] and Tamura and colleagues [16].

4 Observation Distance and Technological Embodiment After establishing a clear understanding of the varying degrees of virtualization, it is essential to emphasize the formal design contours of these screens – such as the size and their positioning concerning the observer − which significantly contribute to creating distinct (museographic) experiences (TB). Following the categorization of screens proposed by Bimber and Raskar [9] and Schmalstieg and Höllerer [18] applied to AR, the arrangement of these systems (as depicted in Fig. 3) hinges on the relative distance between the screen and the observer’s eyes, ordered by increasing levels of that same distance.

Fig. 3. Classification of observation distance based on [9] and [18].

The first class corresponds to visual displays (or projections) associated with devices directly mounted in the observer’s eyes and is termed Eye-Mounted Displays (EMDs), representing the lowest possible distance. The second class regarding the observation distance classifies screens (or projections) associated with devices such as AR or VR helmets. These devices are commonly referred to as Head-Mounted Displays (HMDs). Milgram and Kishino [8] highlight specific characteristics of these screens that allow them to adapt and enable different levels of virtualization despite maintaining the same relative observation distance. For instance, HMDs with optical-see-through capability (to see through these displays) – as is the case of Microsoft HoloLens 2 [20] – allow seamless integration of virtual elements in the perceptive (visual) field of the user – in the form of AR (or PMR). However, suppose this condition is not verified. In that case, these displays/devices may be considered non-see-through or covering-view – implying a disconnection of the observer from the real physical world and its content, as is the case of immersive VR experiences [22]. Despite the same relative distance between the screen and the user – which is reflected in an individual, first-person experience – these two scenarios represent different modes of visualization and, as a result, different museographic experiences – of greater or lesser disconnection from the real world – corresponding, based on this condition, to distinct

158

F. Sosua and F. Providência

degrees of virtualization. For this reason, we considered it appropriate to disassociate observation distance from any specific visualization mode or degree of virtualization, concentrating solely on the relative length of observation. Mobile devices, such as tablets or smartphones, fall into the third class – Hand-Held Displays (HHDs). These devices are designed for convenient use and handling close to the user’s body space, typically held in the hands. The remaining classes correspond to interfaces, projections, or fixed displays in the physical environment, in which their size plays a significant role in defining the overall experience. Observation distance can also influence the nature of the experience, whether individually or collaboratively, in a proportional relationship with the size of these screens, drawing in and engaging larger audiences, especially in the case of LargeScale Projection setups. 4.1 Technological Embodiment Don Ihde [23] introduces the concept of technological embodiment to describe the symbiotic relationship in which technology, during mediated experiences, becomes an extension of the human body, affecting interpretation, perception, and interaction with the immediate environment. Ihde [24] posits technological embodiment – technological mediation through instruments or tools – to be intrinsically tied to perception and significantly influenced by cultural context. Complementarily, Frank Biocca [25] suggests that this relationship is essential for designing immersive experiences. This significance arises from its capacity to replicate synesthetic sensations from the real physical world and enable the exploration of multimodal perceptual senses. At the highest degree of technological embodiment, characterized by complete symbiosis, individuals reach a state where they are unable, from a phenomenological perspective, to discern or acknowledge technology as an extension of their own body [23]. In this context, observation distance, the second dimension proposed for classifying the array of digital mediation systems in the museum, manifests an inverse relationship with technological embodiment. In simpler terms, the smaller the observation distance, the greater the technological embodiment and, consequently, the stronger the perception of technology as an extension of the self, as a mere instrument or tool. In this context, we also envisage inferences related to eliciting the feeling of telepresence (hereafter presence), considering this symbiotic relationship – immediate and transparent – may reinforce the perceptual illusion of nonmediation or symbolic representation, resulting in the observer’s inability to accurately acknowledge, partially or totally, the actual role played by digital technology in the experience [26].

5 Interactivity In Human-Computer Interaction (HCI), Hoffman and Novak [27] define interactivity as interaction in a mediated environment that responds in accordance with user actions. Steuer [28, p. 14] defines it “as the extent to which users can participate in modifying the form and content of a mediated environment in real-time.”

Critical Digital: A Taxonomy to Classify Digital

159

Sharp and colleagues [10] identify five types of interaction within the scope of HCI: instructing, conversing, responding, manipulating, and exploring. Instructing involves performing specific tasks within systems that operate based on discrete instructions. This type of interaction includes actions like issuing written commands, voice instructions, selecting options on a touch-sensitive screen, using gestures, or utilizing combinations of keys, buttons, or codes. Conversing revolves around creating a dialogue between the user and the mediation system, designed and programmed to respond as if it were a person, entity, or social actor. Unlike instructing, this type of interaction is a two-way communication process. Conversing encompasses systems with simple or complex voice recognition capabilities and interfaces structured around a menu in a question-answer format. Responding implies initiative on the part of the system, often considered proactive. It involves the system recommending, notifying, describing, or displaying content deemed relevant and of particular interest to the user based on the prevailing circumstances and context. For instance, such initiatives might be triggered by detecting the user’s presence, recognizing their geographic location, or specific user actions. Manipulating entails interacting with objects within a real or virtual environment, drawing upon the user’s prior knowledge of how things operate in the physical world. This interaction can encompass selecting, moving, rotating, and even scaling virtual objects. Schneiderman [29] suggests that interaction with virtual objects should be conceptually similar to interacting with physical objects in the real world, introducing the concept of direct manipulation. Direct manipulation interfaces aim to provide users with a sense of control and direct contact with virtual objects, thus perpetuating the transparency of the system presenting them. Exploring involves navigating through a real physical, mixed, or virtual environment. The real physical environment may include technology based on sensors, in which the presence and bodily movements of the visitor are responsible for triggering analog or digital events, becoming intelligent spaces. Virtual environments, as envisioned by Milgram and Kishino [8], include non-immersive, continuous, fully synthesized worlds visualized on a two-dimensional plane (window-on-the-world); hybrid worlds, characterized by the overlapping of digital content in the user’s perceptual field visualized in AR; and immersive fully synthesized worlds visualized in VR. These virtual environments allow users, like in the physical world, to navigate and explore spaces and interact with virtual objects (e.g., walk, grab, or throw virtual objects). It remains to propose a way to classify scenarios where there is no means of interacting with the content presented. In the case of a unidirectional communication system, the user (receiver) is not provided with the means to elicit any intervention regarding the content provided by the mediation system (sender). We suggest employing the following continuum (Fig. 4), divided into six discrete classes, to classify the degree of interactivity in digital media systems (TC):

160

F. Sosua and F. Providência

Fig. 4. Classification of the degree of interactivity based on [10].

6 Taxonomy Application To clarify the results, purpose, and application of the virtualization-embodimentinteractivity taxonomy (designed in TD) and to deduce its significance for the museum experience, we proceed to classify below a paradigmatic example of a traditional immersive VR system (Fig. 5):

Fig. 5. Immersive Virtual Reality Experience categorized based on virtualization, observation distance, and interactivity.

Immersive VR, the highest degree of the virtualization continuum, places the observer in a completely synthesized, computational-generated, and often carefully crafted virtual environment, in real time, constituting an interpretation of the world. The technological goal is to imprint on the observer the impression of being at the location depicted by the technical images – the subjective feeling of presence – requiring a meticulous alignment of the immersion strategies with the physiological disposition of the human senses. For increased immersion, VR developers commonly strive to maximize realism and reproduce synesthetic sensations experienced in the real physical environment [8, 26]. This absorbing process is characterized by diminishing critical distance and awareness while privileging an emotional involvement in the experience [30]. Traditional VR systems are also characterized for monopolizing the observer’s attention and disconnecting them from the real physical world (as they are viewed on nonsee-through HMDs) – falling into the lowest (practical) observation distance class and, inversely proportional, the highest degree of technological embodiment – translating into an individual, first-person experience. This condition further enhances the perceptual illusion of nonmediation and, as it is used within the body space, emphasizes the idea of instrumentality: of technology as a mere human extension. Furthermore, this virtual environment may be accompanied by multisensory stimuli that make it possible to explore its surroundings. This scenario represents the highest degree of interactivity, as the virtual environment is perceived and experienced as a lived, responsive space that responds aligned with user actions.

Critical Digital: A Taxonomy to Classify Digital

161

As previously mentioned, the optimal museum experience offers “an opportunity to question reality” while preserving “critical thinking, plurality, awareness, and perspective-taking.” Keeping this aspiration in mind and striving for an optimal museum experience, one might consider designing a VR experience with less interactivity, reducing immersion and the subsequent subjective feeling of presence, thus, in theory, preserving a greater critical distance between the observer and the content presented. Moreover, a reduced degree of interactivity may broaden the experience to a wider audience, as instructing or conversing can typically be executed using streamlined controls, making it more accessible to a diverse range of visitors. The VR experience designed for the Dr. Louzã Henriques Ethnographic Museum’s permanent exhibit, Talasnal House VR [31] (Fig. 6), allows visitors to explore a typical 20th-century schist house from Serra da Lousã. Specific artifacts are integrated into the house, reinforcing the richness and intricate nature of numerous quotidian human activities.

Fig. 6. Talasnal House VR [31] categorized based on the virtualization, observation distance, and interactivity continuum.

Considering VR’s inherent degree of virtualization, the interaction with these artifacts and navigation in the virtual environment deliberately relies on gaze-based instructions (a lower degree of interactivity), and the VR controllers were excluded to reduce technological embodiment. The VR experience is also intentionally situated at the entrance of the third floor, so visitors interact with the artifacts within the virtual house and receive initial descriptions of these objects. After the VR tour, visitors encounter these objects in the physical exhibition space: being confronted with physicality, the presence, and direct perception of these cultural artifacts, now displayed in a “showcase museum” fashion.

162

F. Sosua and F. Providência

This might seem redundant at first glance, considering digital media is often used to display content unavailable or inaccessible in the real physical exhibition. Still, including cultural artifacts present in the VE in the physical exhibit space – a media strategy referred to as hypermediacy [32] – serves to stimulate perspective-taking regarding the things previously seen in the virtual environment: visitors can not only compare the real objects with their digital counterparts but, more significantly, inquire about the placement of the different artifacts within the museum’s physical exhibit based on the descriptions gathered in the VR tour, ensuring unhindered “access to various layers of meaning,” encouraging “critical thinking,” and offering “an opportunity to question reality, the world, and the present time” [7]. These design decisions are only made after acknowledging the high degree of virtualization-embodiment-interactivity associated with traditional immersive VR systems. Considering VR’s high degree of virtualization, one might also counteract the tendency to maximize realism by integrating unrealistic visual and audio cues, reducing immersion and the subsequent subjective feeling of presence by reminding the viewer of the medium.

7 Conclusion and Future Work This paper aimed to categorize digital systems that support communication and technologically mediated experiences in a museographic context. For this purpose, we propose a three-dimensional taxonomy and classification system. This endeavor clarifies some terminological misconceptions, establishes limits, and standardizes the terms describing reality-virtuality technologies while providing an overview of digital media’s qualitative characteristics – virtualization, embodiment, and interactivity – which affect and shape the museum experience. The main goal is to stimulate and promote the conscious and critical appropriation of digital technology among designers, technologists, curators, and other museum professionals. Although this comprehensive and concise classification might cover all digital systems integrated into the museum space, future longitudinal studies should test this threedimensional proposal and consider the impact of the design changes this proposal might instigate. The following research questions should be considered: Is visitor motivation and identity [33] (e.g., explorers, facilitators, seekers, hobbyists, rechargers) related to particular virtualization, embodiment, and interactivity arrangements? Does, in practice, tweaking the virtualization, embodiment, and interactivity sliders when redesigning a museum experience affect critical thinking, perspective-taking, and individual subjectivity? Notwithstanding, the contributions expressed in this paper provide a structured framework to enhance comprehension regarding the influence of these technologies on the museum experience, aiming to aid museum professionals in effectively choosing the fitting terminology for each specific reality (with visual support for communicating their decisions) and promote insightful digital integration in the museum focused

Critical Digital: A Taxonomy to Classify Digital

163

on sustainability, accessibility, inclusivity, critical thinking, and individual subjectivity during the museum experience. Acknowledgments. ID+ Research Institute for Design, Media, and Culture supported the study reported in this publication.

References 1. de Kerckhove D (1998) Inteligência Conectiva: a emergência da cibersociedade (tradução de Ana Viseu), 1st edn. Fundação para a divulgação das Tecnologias de Informação, Lisboa 2. Hein H (1994) Institutional blessing: the museum as canon-maker. In: Gould CC, Cohen RS (eds) Artifacts, representations and social practice. Springer Netherlands, Dordrecht, pp 1–19. https://doi.org/10.1007/978-94-011-0902-4_1 3. Brown K, Mairesse F (2018) The definition of the museum through its social role. Curator: The Museum J 61(4):525–539. https://doi.org/10.1111/cura.12276 4. Simon N (2010) The participatory museum, 1st edn. Museum 2.0, Santa Cruz, California 5. Falk JH, Dierking LD (2016) The museum experience. Routledge. https://doi.org/10.4324/ 9781315417899 6. Parry R, Dziekan V (2021) Critical digital: museums and their postdigital circumstance. In: Barranha H, Simões Henriques, J (eds) Art, Museums & digital cultures: rethinking change. Universidade NOVA de Lisboa, Lisboa, pp. 16–26 7. Sousa F, Providência F (2023) Critical digital: digital integration in the museum. In Martins N, Brandão D (eds) Advances in Design and Digital Communication III. Springer Nature Switzerland, Cham, pp 495–506. https://doi.org/10.1007/978-3-031-20364-0_43 8. Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inform Syst. https://doi.org/10.1007/978-3-642-87512-0_1 9. Bimber O, Raskar R (2005) Spatial augmented reality: merging real and virtual worlds. CRC Press, New York 10. Sharp H, Rogers Y, Preece J (2019) Interaction Design: Beyond Human-Computer Interaction, 5th edn. Wiley 11. Flavián C, Ibáñez-Sánchez S, Orús C (2019) The impact of virtual, augmented and mixed reality technologies on the customer experience. J Bus Res 100:547–560. https://doi.org/10. 1016/j.jbusres.2018.10.050 12. Jeon S, Choi S (2009) Haptic augmented reality: Taxonomy and an example of stiffness modulation. Presence: Teleoperators Virtual Env 18(5):387–408. https://doi.org/10.1162/pres.18. 5.387 13. Mann S (2002) Mediated Reality with implementations for everyday life. Presence, Connect. http://wearcam.org/presence_connect/ 14. Yung R, Khoo-Lattimore C (2017) New realities: a systematic literature review on virtual reality and augmented reality in tourism research. Curr Issue Tour 22:1–26. https://doi.org/ 10.1080/13683500.2017.1417359 15. Pan Z, Cheok AD, Yang H, Zhu J, Shi J (2006) Virtual reality and mixed reality for virtual learning environments. Comput Graph (Pergamon) 30(1):20–28. https://doi.org/10.1016/j. cag.2005.10.004 16. Tamura H, Yamamoto H, Katayama A (2001) Mixed reality: Future dreams seen at the border between real and virtual worlds. IEEE Comput Graph Appl 21(6):64–70. https://doi.org/10. 1109/38.963462

164

F. Sosua and F. Providência

17. Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B (2001) Recent advances in augmented reality. IEEE Comput Graph Appl 21(6):34–47. https://doi.org/10.1109/38. 963459 18. Schmalstieg D, Höllerer T (2016) Augmented Reality: Principles and Pratice, 1st edn. Addison-Wesley 19. Azuma RT (1997) A survey of augmented reality. Presence: Teleoperators Virtual Env 6(4):355–385. https://doi.org/10.1162/pres.1997.6.4.355 20. Microsoft (2023) Microsoft HoloLens 2. www.microsoft.com/hololens Accessed 04 Sep 2023 21. Inc. Magic Leap (2023) Magic Leap 2. www.magicleap.com. Accessed 04 Sep 2023 22. Hainich RR, Bimbe O (2017) Displays: fundamentals and applications, 2nd edn. CRC Press 23. Ihde D (1990) Technology and the lifeworld: from garden to earth, 1st edn. Indiana University Press, Bloomington 24. Ihde D (2001) Bodies in Technology, 1st edn. University of Minnesota Press 25. Biocca F (1997) The cyborg’s dilemma: progressive embodiment in virtual environments. J Comput-Mediat Commun 3(2):JCMC324. https://doi.org/10.1111/j.1083-6101.1997.tb0 0070.x 26. Lombard M, Jones MT (2015) Defining presence. In: Lombard M, Biocca F, Freeman J, Ijsselsteijn W, Schaevitz RJ (eds) Immersed in media. Springer International Publishing, Cham, pp 13–34. https://doi.org/10.1007/978-3-319-10190-3_2 27. Hoffman DL, Novak TP (1996) Marketing in hypermedia computer-mediated environments: conceptual foundations. J Mark 60(3):50–68. https://doi.org/10.2307/1251841 28. Steuer J (1992) Defining Virtual Reality: Dimensions Determining Telepresence. J Commun 42(4):73–93. https://doi.org/10.1111/J.1460-2466.1992.TB00812.X 29. Shneiderman B (1983) Direct manipulation: a step beyond programming languages. Comput (Long Beach Calif) 16(8):57–69. https://doi.org/10.1109/MC.1983.1654471 30. Grau O (2003) Virtual art: from illusion to immersion (revised and expanded edition). The MIT Press, Cambridge, Massachusetts 31. Sousa F (2022) Talasnal House VR. FSDesign. https://urlis.net/bg188j27. Accessed 28 Aug 2023 32. Bolter JD, Grusin R (1999) Remediation: understanding new media, 1st edn. The MIT Press 33. Falk JH (2016) Identity and the museum visitor experience. Routledge. https://doi.org/10. 4324/9781315427058

Exploring Hazard Perception and Compliance: A Pilot Study on the Influence of Safety Signs in a Non-conspicuous Hazard Situation Lara Reis1(B) , Emília Duarte1 , and Francisco Rebelo2 1 UNIDCOM/IADE, Av. D. Carlos I, 4, 1200-649 Lisbon, Portugal

[email protected] 2 CIAUD, Faculdade de Arquitetura, Universidade de Lisboa, Rua Sá Nogueira, Pólo

Universitário, Alto da Ajuda, 1349-063 Lisboa, Portugal

Abstract. This pilot study aimed to evaluate the impact of conventional ISO type safety signs on enhancing hazard-risk behaviours and prompting behavioural compliance, in a non-conspicuous hazardous situation (i.e., a Conveyor belt hazard, void of any visual and/or auditory cues). Using a work-related (i.e., a stereotypical, yet realistic factory setting) virtual reality-based experimental set-up, and a sample of twelve workers (20–65 yrs), the study´s objectives were to assess to what extent the printed static safety signs were effective in prompting compliant behaviours in the given situation, as well as to collect the participants’ hazard-risk perceptions regarding such a context. The study, therefore, addresses topics under the Virtual-reality domain such as: simulator sickness; level of presence; overall virtual environment design; as well as hazard-risk and safety sign perception. Results reveal that in this non-conspicuous hazardous situation, the participants adequately perceived the hazard and adopted safe behaviours, despite being unaware of the presence of such safety signs in the environment. Keywords: Virtual environments · Workplace Safety Sign Research · Ageing · Augmented Reality · Behavioral compliance · Hazard Saliency · Hazard-risk perception · Interaction · User Experience

1 Introduction One approach to enhancing workplace safety includes the development and strategic use of environmental safety signs as a means of communicating safety-related information to workers. Such mediums, usually in the format of static plates, play a vital role in alerting/informing workers to potential hazards in the environment [1]. However, despite their importance, research findings suggest that the conventional ISO type safety signs are not always effective [2], as they often fail to attract attention, provide knowledge and prompt compliant (safe) behaviours [3]. The latter being, according to most cognitive models [4, 5], the most significant variable when determining the success of such signs. Regrettably, given the high number of debilitating injuries and fatalities (approximately 9.7 deaths per 100 000 working persons) reported by work databases [6], such facts © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 165–173, 2024. https://doi.org/10.1007/978-3-031-47281-7_13

166

L. Reis et al.

highlight the critical need for innovation in workplace safety signs, especially with the rise of older working adults. With the worldwide gradual increase of life longevity, and consequently, the extension of the retirement age to 65 + years [7], the workforce population has been hit by a global “silver wave/tsunami” which has inevitably increased, over the years, the number of working older adults (55–65 years old). According to the European Labour Force survey (EU-28), the number of workers aged 55 + increased from 12.1% to 19.7% between 2003 and 2018, accounting for almost one fifth of the entire workforce (90.5 million persons from a total of 230.4 million) [6]. In the US, similar data can be found with such a population representing 22.4% (35.7 million) of the labour force [8]. Consequently, this demographic shift has raised several concerns regarding workplace safety, because with old age, job performance can be adversely affected by the natural deterioration of the physical and mental capacities which, subsequently, can make older workers more susceptible to accidents. Over the years, studies on the effectiveness of safety signs claim that with old age, the visual, auditory and cognitive capacities decline, which thereby place older workers at a disadvantage in hazardous situations [9, 10]. Fortunately, research has suggested that Augmented Reality-based safety signs (as an interactive computer generated medium which displays digital imagery/graphics over a view of the actual physical world in real-time) may be more effective in communicating safety-related information to older workers, thanks to their dynamic and conspicuous/multimodal properties [11]. In the transportation domain (where Advanced Driver Assistance Systems and Enhanced Vision Systems are used), the current body of research has emphasized that AR contributes to enhancing safety. Such studies report that such technology not only has the ability to detect (in real-time) the presence of a hazard and punctually alert the person [12, 13], but may also provide users with appropriate cognitive support, thereby enhance their sensory perception [14]. In a previous study [15], where a complex work-related virtual environment (that consisted of two different types of hazardous situations, with distinct levels of saliency) was used to assess the effectiveness of an AR safety system that was specifically designed for such a purpose, researchers concluded that the system was successful in prompting more cautious (safe and compliant) behaviours. However, in this specific study, data was unclear about whether such safety signs enhanced the workers’ sensory/hazard perception of their surroundings, especially regarding a particular situation in which the danger/hazard was non-conspicuous/salient (i.e., void of any visual and auditory cues). Researchers appoint to the fact that there was a significant decrease in the participants’ hazard-risk awareness levels. It is within this context that this paper seeks to discuss the preliminary findings of a pilot study that was designed to assess whether, in the same situation, such a phenomenon would also occur regarding traditional workplace ISO-type safety signs. In other words, the present study sought to understand if the type of safety sign (in this case the conventional ISO static printed placards) to which the participants are exposed to influences/impacts their hazard-risk awareness levels in a non-conspicuous hazardous situation. Such a study is part of an ongoing research project that is focused on improving workplace safety for an ageing workforce through the use and design of more effective safety signs. Such a project is driven by the premise (hypothesis) that, due

Exploring Hazard Perception and Compliance: A Pilot Study on the Influence

167

to their saliency/conspicuity, AR safety signs are more effective than the conventional ISO-type counterparts in promoting safety among workers. In light of this larger research project, the present pilot study used the same Virtual Reality-based (VR) experimental set-up and overall interaction framework to that of a previous study [16], in order to circumvent the methodological, economical and ethical constraints that limit this type of research, namely: the inability to intentionally/ethically expose research subjects to real hazards; the rarity/unpredictability of hazardous events; and the challenge/difficulty behind producing experimental settings which mimic reallife settings (i.e., hazards which have distinct levels of saliency) [17]. Therefore, by using the same workplace context, the study comprised of three key moments: (1) to examine whether the ISO-type warnings were effective in enhancing the participants’ behavioural compliance when confronted with the given non-conspicuous hazard (i.e., stationary and soundless); (2) to understand whether the hazard’s level of salience would have an effect on the participants’ behaviours; and (3) to analyze the participants’ hazard-risk perceptions regarding this particular situation. In order to undergo such an evaluation, this study’s results will be compared to those of a previous study [16], in which data was collected under the same safety sign conditions regarding a conspicuous hazard (i.e., with visual and auditory cues). Consequently, this paper addresses the following topics under the VR domain: simulator sickness, level of presence, overall virtual environment design, as well as hazard-risk and safety sign perception.

2 Method 2.1 Participants A sample of 12 adult workers aged between 20 to 65 years old (Mean Age = 42.6, SD = 15.2), with varying professional experiences, volunteered to participate in this study. Of these, six were men (Mean age = 42.0, SD = 13.4) and six were women (Mean age = 43.2, SD = 17.0). Before beginning the experiment, participants were required to sign a consent form and complete a demographic questionnaire; as well as were screened for color deficiencies (using the Ishihara Color Vision Test [18]) and cognitive impairment (by applying the Mini-Mental State Examination [19, 20]). In sum, participants reported to be absent of any have visual/physical/mental conditions that could impact/influence their performance during the experiment. Moreover, they stated to have no prior experience with any VR-based system set-up and/or simulation. 2.2 System Set-Up An immersive VR system set-up (based on [16]) was used to conduct the study, as well as automatically collect data regarding the participants’ interaction. The following devices were used: (1) the Oculus Rift Development Kit2 Head-Mounted Display (HMD), which was used to visualize the VE; (2) the Xbox 360 wireless gamepad, i.e., a control device used to interact with the VE; (3) wireless Sony headphones, model MDR-RF800RK, which transmitted the VE’s sounds; and (4) a Dell Alienware M18x laptop - with an Intel Core i7-3610QM processor, 16GB of memory, and a Dual 2GB GDDR5 Nvidia

168

L. Reis et al.

GTX 675M SLI graphics card - which was used to run the simulation. In order to collect quantitative data, an event log system (based on scripts and triggers that were specifically programmed for this study) was adapted and used to automatically record the participants’ interaction in real-time. The VE’s 3D models (scenes and assets) were designed using the Sketchup Pro software, and then exported to the Unity3D game engine (version 4.6.3f1) in order to fine tweak its ambiance (lights, textures and sounds) and define the simulation’s mechanics (animations and gameplay). To prevent possible simulator sickness, the following features were considered/applied: (1) the participants’ viewpoint was set at eye-height (1.53m above the ground), and their Field-of-View was set using the software’s standard default settings; (2) the velocity at which the participants could rotate their viewpoints was reduced to match real-life head movements; and (3) their travel/navigational speed (which gradually increased from 1.26m/s to a maximum of 1.35m/s) was controlled in order to simulate a more natural/life-like walking pace; and (4) the 3D model was optimized to maintain an average image frame rate above 75Hz per second throughout the simulation, in order to avoid any lag effects that could impact the participants´ overall experience. 2.3 Virtual Environment (VE) 3D Model. For this pilot study, the same work-related environment used in the previous study [15] was applied: a section of a factory comprising of two different types of hazardous situations (an Overhead hazard and Conveyor hazard), each one with a distinct level of saliency (conspicuous vs. non-conspicuous). Therefore, the 3D model consisted of two large (30m x 17m) rectangular-shaped modules, linked together via smaller (17m x 15m) open spaces, which represented the factory’s entrance/exit points. Environmental Cues. The same situational characteristics (visual and auditory cues), for each type of hazardous situation, was maintained: (1) in the conspicuous Overhead hazard, in Module 1, the crane/container was accompanied by sound (an alarm) and motion (the crane began to move); (2) whereas in the non-conspicuous Conveyor hazard, in Module 2, the conveyor belt remained stationary and soundless. Safety Signs. All modules contained a series of conventional/static safety signs and markings throughout the various sections and architectural elements (namely façades, walls, columns and floor). However, for this particular pilot study, behavioural compliance was assessed regarding the ISO-type safety signs in Module 2 (i.e., the Conveyor belt situation), which were purposely placed before the hazard, on the module´s main entrance columns (to the left of the VE - see Fig. 1). Similarly to a previous prototype [16], such safety signs were 60cm x 42cm in size (in accordance with ISO 3864–1 [21]) and mounted between 1.2m and 1.8m above the ground (as defined by ISO 16069 [22]).

2.4 Strategy to Assess Behavioural Compliance As in the previous studies, in order to perform the experiment’s task, participants had to search for and detect the dog (as it ran across the factory’s modules), as well as retrieve

Exploring Hazard Perception and Compliance: A Pilot Study on the Influence

169

Fig. 1. Screenshot of the entrance to Module 2 - Conveyor Belt Hazard.

it. As the participants followed the dog’s into the module, they were not only confronted with a set of ISO-type warnings (as referred to above), but also the conveyor belt hazard itself. In this precise moment, participants were forced to decide between two pathways: (1) to follow the dog’s path, crossing over the conveyor belt; or (2) to take the safety path, on the opposite side of the module (on the right), as indicated by the safety signs and floor markings. If participants chose not to comply with the safety signs and instead decided to follow the dog’s path, the study inferred that such signs were not effective. 2.5 Measures To prove the study’s hypothesis, two usability test-beds were conducted in order to collect both objective and subjective measures. The first test-bed sought to assess to what extent the ISO-type safety signs prompted compliant behaviours in the non conspicuous hazardous situation (Conveyor hazard); this data was collected by observing the participants’ actions and path trajectories, namely if they followed the dog across the dangerous route (i.e., over the conveyor belt) or they took the safety path alongside the module, as indicated by the safety signs and markings. The study’s hypotheses regarding this particular assessment was two-folded: firstly, the ISO-type safety signs would be ineffective (due to their lack of salience) in promoting the compliant behaviour of circulating on the safety path; secondly, participants would fail to adopt safe behaviours due to the hazard’s inconspicuous characteristics. The second test-bed aimed to collect data regarding the participants’ overall user experience, mainly their subjective hazard-risk perceptions regarding their interaction when confronted by the hazard. Due to the hazard’s passive state (motionless and soundless), it was hypothesized that the participants’ hazard perception levels would decrease, similarly to the results attained in a previous study [15]. In other words, they would fail to recognize and perceive the hazard’s severity. In order to collect such data, a number of post-hoc questionnaires (using 4 to 9-point scales, adapted from [15]) were applied to assess issues related to the participants’ hazard-risk awareness levels and their perception of the safety signs, as well as matters concerning their interaction and experience with the VE such as simulator sickness, level of presence/engagement and overall design.

170

L. Reis et al.

2.6 Procedure For this pilot study, the same methodological procedure was applied, thereby dividing the experiment into five main stages: (1) introduction to the study; (2) pre experimental training session which was divided into two different moments for interaction and perceptual assessments; (3) experimental session; and (4) follow-up questionnaires. The whole procedure lasted approximately 60 min in total. For security reasons, participants sat a desk throughout the duration of the experiment, as well as were accompanied by the researcher’s presence in order to observe and monitor the participants’ interaction inside the VE.

3 Results 3.1 Behavioural Data Data from the 1st test-bed reveals that only 33.3% of the participants followed the dog’s path across the Conveyor belt (i.e., the hazardous route), meaning that 58.3% of the participants opted to take the safety path. When comparing these results with a previous study’s findings [16], in which the same measures were addressed regarding the static/conventional ISO-type safety signs’ effectiveness in the Overheard hazardous situation (see Table 1 below), data discloses a 41.6%p.p. increase in compliance between both situations. Table 1. Descriptive statistics (Percentage values) for behavioural compliance measures. Participants’ trajectories

Module 1: Overhead hazard Module 2: Conveyor hazard

Followed the safety path

16.7%

58.3%

Followed the dog across the hazard 83.4%

33.3%

3.2 Subjective Data Simulator sickness Questionnaire (SSQ): data obtained from the SSQs, which were applied twice during throughout the procedure (namely at the end of the pre experimental training session, as well as the experimental session), exposes the incident of slight simulator sickness (25%). The most recurring general body symptoms accounted for were: sweating (42%); nausea (33%); and blurred vision (42%). With these results, one can infer that after a short (approximately 10 min in total) VR exposure, the participants’ overall well-being was affected, and thereby may have impacted their experience and interaction inside the VE. Presence Questionnaire (PQ): In what concerns the participants’ level of presence inside the VE, results reveal: that the participants felt highly immersed (Mean = 5.5, SD = 1.2) and the VE to be sensorially engaging and realistic (Mean = 5.6, SD =

Exploring Hazard Perception and Compliance: A Pilot Study on the Influence

171

1.2); and that they were able to interact with the system set-up with ease and that the control devices didn’t interfere with their interaction (Mean = 5.1, SD = 1.4). The only factor that may have impacted their experience was the visual display’s resolution, as participants reported to have had some difficulty in clearly seeing the VE’s details (Mean = 4.6, SD = 1.8). VE Design Questionnaire (VDQ): Regarding the VDQ, the attained data reveals that the participants found the experience to have been well-designed and entertaining (Mean 5.7, SD = 1.3), as well as the visual and auditory stimuli were very coherent with the given context and setting, therefore engaging and realistic (Mean = 5.8, SD = 1.2). Hazard-risk Perception Questionnaire (HPQ): Data obtained for the HPQ indicate that in this particular situation, in which the hazard was inconspicuous (stationary and soundless), participants were very much aware of its presence in the module (Mean = 7.0, S.D = 1.2) and considered it to be very dangerous (Mean = 6.0, SD = 1.8). When comparing such findings with those attained in a previous study [16], regarding the participants’ overall impressions on the Overheard hazard (characterized as the conspicuous hazard, due to its visual and auditory cues), data reveals a significant increase across all hazard-risk variables (see Table 2). However, it is to be noted that in this pilot study’s non-conspicuous hazardous situation (i.e., Conveyor belt hazard), participants found the severity of the injury only to be fairly high (Mean = 5.3, SD = 2.0), meaning therefore, that there was a 2.2 decrease in value when compared to the conspicuous hazard counterpart.

Table 2. Descriptive statistics (Mean and Standard Deviation values) regarding the Hazard-risk Perception Questionnaire (HPQ). Module 1: Overhead

Module 2: Conveyor

Measures

Mean

SD

Mean

SD

Hazard awareness

5.0

2.5

7.0

1.2

Hazard-risk level

4.9

2.8

6.2

1.8

Likelihood of injury

4.8

2.0

5.5

2.1

Severity of injury

7.5

1.1

5.3

2.0

Cautious intent

4.5

1.8

6.2

2.1

Familiarity

5.3

2.2

4.2

2.4

Control

4.7

2.6

6.6

2.1

Hazard saliency

5.8

1.8

5.6

1.8

Hazard stimulus influence

5.8

2.1

6.6

1.4

Safety Signs Perception Questionnaire (SPQ): In what concerns the participants’ impressions about the study’s ISO-type safety signs, similar results to the previous study were also found, namely: that the participants were fairly unaware of such signs (Mean

172

L. Reis et al.

= 3.4; SD = 3.3); that such signs did not totally capture their attention (Mean = 3.4, SD = 3.1); and consequently, they did not read their warning (Mean = 0.9, SD = 2.2).

4 Conclusion This paper presents and discusses the preliminary findings of a pilot study that was carried out to assess the effectiveness of ISO-type safety signs in non-conspicuous situations, namely, a Conveyor belt hazard void of any visual and/or auditory cues. It’s main goal was to compare such findings with a previous study [16], in order to understand and analyze two variables: (1) the signs’ ability to invoke behavioural compliance in the given situation; and (2) the participants’ ability to adequately perceive the hazard. Using the same VR-based experimental set-up as the previous studies (a stereotypical, yet realistic factory), two test-beds were applied to obtain both behavioural and subjective data respectfully. By analyzing the first test-bed, whose main objective was to observe the participants’ path choices, results indicate that more than 58% of the participants followed the safety path as indicated by the safety signs and floor markings. Unfortunately, for the study’s first hypothesis, such data does not disclose whether or not the signs contributed to such actions, as the SFQ reveals that participants were unaware, to a certain extent, of such signs’ existence. However, when comparing such data with the previous study (i.e., Overhead hazard, conspicuous hazard), one can infer that in this particular situation, participants were more able to identify, as well as understand what type of hazard the Conveyor belt consisted of. Subsequently, the study’s second hypothesis regarding the effect of the hazard’s level of salience, on the participants’ behavioural compliance, was refuted. As for the study’s third hypothesis regarding the participants’ hazard-risk behaviours, one can infer that, when confronted with the hazard, the Conveyor belt’s lack of salience (absence of sound and movement) had a significant influence on the participants’ behaviours, thereby disconfirming such a premise. However, although participants adequately perceived the hazard, data from the HPQ reveals that participants only classified the situation as fairly dangerous, which raises questions as to the reasons behind such an increase in behavioural compliance. Subsequently, since such findings are inconclusive due to the lack significant data across the experimental conditions. In light of the larger research project, which aims to design more effective safety signs for an ageing workforce, further testing with a larger sample, will have to be carried out in order to verify whether the type of safety sign impacts the participants’ hazard-risk perceptions, in both conspicuous and non conspicuous hazard situations. Acknowledgments. A Ph.D. scholarship (SFRH/BD/79622/2011) granted to Lara Reis, from FCT: Fundação para a Ciência e Tecnologia (the Portuguese Science and Technology Foundation), supported this study.

Exploring Hazard Perception and Compliance: A Pilot Study on the Influence

173

References 1. Mayhorn CB, Wogalter MS, Laughery KR (2015) Analysis and Design of Warnings in the Workplace. In: Wilson JR, Sharples S (eds) Evaluation of Human Work. Taylor & Francis, London, pp 331–358 2. Conzola VC, Wogalter MS (2001) A communication – human information processing ( C – HIP ) approach to warning effectiveness in the workplace. J Risk Res 4:309–322. https://doi. org/10.1080/1366987011006271 3. Rogers WA, Lamson N, Rousseau GK (2000) Warning research: an integrative perspective. Hum Factors 42:102–139. https://doi.org/10.1518/001872000779656624 4. Laughery KR, Wogalter MS (2014) A three-stage model summarizes product warning and environmental sign research. Saf Sci 61:3. https://doi.org/10.1016/j.ssci.2011.02.012 5. Wogalter MS, Laughery KR, Mayhorn CB (2012) Warnings and Hazard Communications. In: Salvendy G (ed) Handbook of Human Factors and Ergonomics. John Wiley & Sons Inc, Hoboken, New Jersey, pp 868–893 6. Eurostat: Ageing Europe - statistics on working and moving into retirement, https://ec.eur opa.eu/eurostat/statistics-explained/index.php?title=Ageing_Europe_-_statistics_on_work ing_and_moving_into_retirement#Accidents_at_work_among_older_people, last accessed 2020/10/03 7. Pensions at a Glance 2019. OECD (2019). https://doi.org/10.1787/b6d3dcfcen 8. Collins SM, Fischer D, Kelley K (2017) America´s Aging Workforce: Opportunities and Challenges (2017) 9. Laughery KR (2006) Safety communications: warnings. Appl Ergon 37:467–478. https://doi. org/10.1016/j.apergo.2006.04.020 10. Adams AS (2006) Warning Design. In: Karwowski W (ed) International Encyclopedia of Ergonomics and Human Factors, vol II. CRC Press, Boca Raton, Florida, pp 1517–1520 11. Wogalter MS, Conzola VC (2002) Using technology to facilitate the design and delivery of warnings. Int J Syst Sci 33:461–466. https://doi.org/10.1080/00207720210133651 12. Rusch ML et al (2013) Directing driver attention with augmented reality cues. Transp Res Part F Traffic Psychol Behav 16:127–137. https://doi.org/10.1016/j.trf.2012.08.007 13. Rusch ML, Schall MC, Lee JD, Dawson JD, Rizzo M (2014) Augmented reality cues to assist older drivers with gap estimation for left-turns. Accid Anal Prev 71:210–221. https://doi.org/ 10.1016/j.aap.2014.05.020 14. Wogalter MS, Mayhorn CB (2005) Providing cognitive support with technology based warning systems. Ergonomics 48:522–533. https://doi.org/10.1080/00140130400029258 15. de Amaral LR, Duarte E, Rebelo F (2018) Evaluation of a Virtual Environment Prototype for Studies on the Effectiveness of Technology-Based Safety Signs. Presented at the. https://doi. org/10.1007/978-3-319-60582-1_11 16. Reis L, Duarte E, Rebelo F (2015) Research on workplace safety sign compliance: validation of a virtual environment prototype. Procedia Manuf 3:6599–6606. https://doi.org/10.1016/j. promfg.2015.07.722 17. Duarte E, Rebelo F, Wogalter MS (2010) Virtual reality and its potential for evaluating warning compliance. Hum Fact Ergon Manuf Ser Ind 20:526–537. https://doi.org/10.1002/hfm.20242 18. Ishihara S (1988) Test for Colour-Blindness. Kanehara & Co., Ltd, Tokyo 19. Folstein MF, Folstein SE, McHugh PR (1975) Mini-Mental State: A practice method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12:189–198 20. Guerreiro MM, et al (1994) Adaptação à População Portuguesa na Tradução do “Mini Mental State Examination” (MMSE). Revista Portuguesa de Neurologia, 9 21. International Organization for Standardization: Graphical symbols - Safety Colours and Safety Signs (ISO 3864–1,2,3,4) 22. International Organization for Standardization: Graphical symbols - Safety Signs - Safety Way Guidance Systems (SWGS) (ISO 16069:2004), (2004)

The Impact of Tangibility in the Input of the Secondary Car Controls: Touchscreens vs. Physical Buttons Manuel Masseno1 , Inês Lopes2 , Rita Marques1 , Francisco Rebelo1 Elisângela Vilar1 , and Paulo Noriega1(B)

,

1 CIAUD, Research Centre for Architecture, Urbanism and Design, Lisbon School of

Architecture, Universidade de Lisboa, Lisboa, Portugal [email protected], {ebpvilar,pnoriega}@edu.ulisboa.pt 2 ITI/LARSyS, Faculdade de Arquitetura, Universidade de Lisboa, Rua Sá Nogueira, Pólo Universitário, Alto da Ajuda, 1349-063 Lisboa, Portugal

Abstract. The way car controls are displayed has been changing over time, as physical controls are being replaced by touchscreens and touch interfaces. This allows the creation of a more clean and aesthetically pleasing interface and reduces costs, but also creates the potential to increase the drivers’ distraction and error as controls may be harder to find and use. This paper presents an evaluation of driver performance in the context of a laboratory simulation of an in-car, on road driving study, that compares driver reaction time and number of errors while using physical buttons versus a touchscreen interface. The study was conducted with 20 participants, performing the same tasks in both simulations but with different interfaces. The results concluded that the type of interface impacts reaction time and greatly impacts the number of errors made by the driver in it, as well as, in the road. The data analysis showed that the reaction time to the touch screen was significantly higher and 80% of the participants commited errors in it, compared to only 20% on the physical buttons. The driving performance of the participants was also substantially impacted by the touchscreen, when compared to the response pad. These results raise important questions about the fact that touchscreens are not the best solution in terms of safety. It is true that in terms of possible configurations, touchscreens give great freedom to car manufacturers, but at the expense of road safety. Keywords: Touch input · Touchscreens · Multitasking · Mental workload · Driving performance · Road Safety

1 Introduction In recent years, in-car controls have been changing from traditional buttons in the central panel of the car to touchscreens and touch-sensing interfaces. The changing of these interfaces allows designs to be “more aesthetically pleasing, flexible and dynamic” [1] and requires a smaller space in the car panel. On the other hand it creates the potential to increase the driver’s distraction and errors while driving [1]. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 174–183, 2024. https://doi.org/10.1007/978-3-031-47281-7_14

The Impact of Tangibility in the Input of the Secondary Car Controls

175

Lee and collaborators [2] defined driver’s distraction as “diversion of attention away from activities critical for safe driving towards a competing activity”. Also, existing literature has shown that “both cognitive and visual distraction can impair the driver’s reaction time behavior” [3]. Driving is a complex dynamic performance in which the driver is required to accomplish several tasks at once, so that drivers must be able to handle both speed and heading while planning the route to their destination and trying to understand and predict the trajectories of other cars and the intentions of their drivers, this resulting in a substantial mental workload while driving [4]. Mental workload can be defined as the proportion of information processing capability used to perform a task [5]. Considering the difficulty of the driving task and the quantity of information that drivers have to process and manage inside and outside of the car, there are multiple factors that contribute to the triggering of the driver’s mental workload. The introduction of the touchscreen monitors in the central panel of the car contributes to an increase of information inside of the vehicle, resulting in increased levels of complexity in the driving task, as the driver has to increase its distribution levels on the visual and auditory resources [6], leading to paying less attention to the actual task of driving. Given the different types of people that drive cars nowadays, mental workload cannot be seen as something stable and equal to everybody, as different drivers have different approaches to driving problems and it also depends on their performance ability, the workload may differ from person to person, but the truth is that more stimuli result in a higher mental workload [7], resulting in less attention to the road. Mental workload while driving is directly linked to performance of driving and task demand [7] so the more complex the task while driving, higher is the mental workload required to continue driving, and the lesser is the performance ability to drive. So the complexity of the task to be performed in the secondary panel of the car has a direct impact in the driving attention of the driver and the secondary tasks of the car must be handled in a way that they have the smallest impact possible in the driver’s driving performance. This study aims to test if a physical interface results in a lighter mental workload for the driver, allowing them to focus more on the primary task of the car—driving; leading to safer driving. This is based on the fact that touchscreen panels allow the user to perform a lot more tasks, increasing task demand and posing a threat to the driver’s attention to the road, given that more tasks result in a higher mental workload that as referred ahead is directly linked to driver’s performance of driving. To further defend this hypothesis it has to be taken in consideration that touchscreens not only allow the user to change the objective of the task on the screen and allows them to distract with other less important tasks but also always maintain their position and format even if their task changes, creating a difficult environment for the driver to memorize the tasks available in the panel or sensing the place where the driver has to perform the task. On the other hand physical panels have a permanent and tangible 3D format that can be sensed by the drivers and memorized by them thought their haptic senses [8], therefore creating a haptic memory of them [9] and making the task more mechanical and automatic [4] in a way that it decreases the mental workload of the task [7] and

176

M. Masseno et al.

also allowing drivers to maintain the visual sense focused on the road and other more important tasks.

2 Methodology This study evaluates the difference in driving performance while a driver presses physical buttons versus buttons on a touchscreen, as a secondary task. The main stimuli is the driving of the vehicle, which was tested using a racing simulator, and requires the person’s visual and motor skills. The secondary task, which occurs at specific locations in the track, also requires the same level of attention, as the driver must take his right hand off the wheel, and in some cases look at the dashboard in order to press the correct button (physical or on a screen). The shared stimuli can contribute to a triggering of the driver’s mental workload, and therefore cause performance issues, which could lead to a road accident. How different interfaces can affect the driving performance and which one is safer to use were evaluated by measuring the reaction time to press the button as well as the mistakes made in it and the driving performance. 2.1 Experimental Conditions As an independent variable, the type of interface were defined as the “secondary controls” was switched between physical buttons and a touchscreen, as a stand-in for what would typically be found in a car’s dashboard, such as A/C and radio controls. The dependent variables are the driving performance of each participant as well as the number of errors committed in the interface and the reaction time to it. In the first testing phase, each participant drove through the circuit until they felt comfortable with the racing simulator and its main inputs—the steering wheel and foot pedals. The virtual car was in automatic mode, meaning there was no need for changing gears and the participant only had to accelerate, brake and turn the steering wheel. This phase would normally take one lap or more, depending on the person, and it was not for evaluating purposes. Its purpose was to assure the participants had a basic understanding of how to drive the car without making mistakes, in order to establish a common baseline to every participant. When the participant felt confident enough and the experimenters determined that the participant met the requirements, the evaluation began by introducing the secondary task, pressing specific buttons on an input device while driving at the same time. There were 35 events in which the participant had to press a button, and in order to know which button to press he had to hear a sound file playing the name of the color of the button, with five available options: blue, red, green, yellow and white. The sequence in which these sound files were played was initially created in a random generator (random.org) and then played in the exact same order for every participant as the events in which the participant was required to press the buttons were exactly the same as well. The lap was completed twice, once with a physical input, and again with a tactile input. The order of interfaces presentation was counterbalanced.

The Impact of Tangibility in the Input of the Secondary Car Controls

177

2.2 Participants A convenience sample was selected, mainly composed of members of the Faculty of Architecture of Lisbon University academic community. In total, there were twenty participants (eighteen students and two professors), ranging from 20 years to 63, with an average age of 27 years old. Fourteen identified as female, five as male and one as non-binary. We also inquired about whether they had a driver’s license, whether they drove regularly and what kind of input was used for secondary controls in the car they usually drive in. Concerning driver licence, 14 participants had it, and from these 8 drove regularly, and 4 of the late had some experience with a touchscreen in their car. 2.3 Tools In order to perform the tests, a racing simulator was used, specifically the Assetto Corsa videogame, which, due to its advanced physics engine, provides a “a very realistic driving experience” according to the game’s Steam store page. This simulator was run at max settings at a resolution of 1920x1080 and at a frame rate of 60 frames per second, on a desktop with Windows 10 (i7-6700 CPU @ 3.4 GHz, 16 GB and a GPU Nvidia GeForce GTX 1080 16GB). To control the car in the simulator, a Logitech G29 steering wheel and pedals served as input. This steering wheel provides a high degree of sensitivity, which allows for precise control of the car, as well as force feedback to more accurately represent real driving. To test the difference between the tactile (touchscreen) and physical (buttons) input method for the secondary controls, a Lenovo tablet computer running on Microsoft Windows 8 was used for the former while a Cedrus RB-530 response pad (Fig. 1) connected to a MacBook Air 2020 was used for the latter. Dimensions and color of the buttons on the Lenovo Tablet was adjusted to be similar to those one of the physical response pad. Secondary task stimuli was controlled with SuperLab.6 and SuperLab.5, for the response pad and tablet respectively, which allowed us to collect data on which button was pressed for each event in which the participant had to react, as well as their response time. This program was also used to provide the sound stimuli that informed the participant on which button to press.

Fig. 1. Cedrus RB-530 Response Pad

178

M. Masseno et al.

2.4 Procedures This experiment took place in the ergoUX Lab of the Faculty of Architecture of Lisbon University, in December 2022. Preparations began by setting up the simulator and inputs. The steering wheel was fixed to the table, and its position was marked with tape, to guarantee a secure position. The foot pedals were right below the desk the steering wheel was attached to, but these were not in a fixed position, so the participant could adjust them to their preferences, in order to make the experience as comfortable as possible. The computer monitor was also at a constant position marked with tape, as was the stand that supported the response pad and the tablet. This stand served to position these input devices at a comfortable angle, and to facilitate the exchange of one type of input with the other. Afterwards, several students and faculty members were personally invited. Once they arrived at the lab, participants were asked to sit comfortably in a designated chair, and adjusted the distance between the pedals and the seat. The participant was instructed to have their arms slightly bent, as well as their knees and their back and head straight. After having the setup ready, the participant was informed on the details of the overall experiment, how it would require around half an hour to be concluded, and how it would be organized in three different phases, the first one for training purposes and the latter two for performance evaluation moments where the screen would be recorded. Then, the participant was asked if they had any driving experience, and if not, the pedals’ functions were explained in more detail, in order to provide some assurance. Afterwards, the participant was informed about the rules they had to follow in the first phase, which were to drive on the right side of the road without going over the line, and to maintain a speed below 100 km/h, reinforcing the idea that the objective is not to be the fastest but to drive at a speed in which the participant feels comfortable. The participant would start to drive while the two test experimenters prepared for the next phase and observed their behavior. If the participant finished the first lap and proved they could drive according to the rules, they could pass to the next phase. If not, they were asked to drive until these requirements were met. The experimenters would then position themselves behind the participant, and the test would ensue. One of the experimenters had the task of comparing the current position of the participant on the track with the 35 defined events where the sound file must be triggered, and the second one had the task of triggering the actions, and of creating and saving the files in SuperLab and recording the screen. Whenever an activation point was reached, the first experimenter gave a sign to the second experimenter, imperceptible to the participant. The second experimenter would then trigger the event in SuperLab. The software then recorded the reaction time and the participant correct or incorrect response. Each test took, on average, around 30 min to complete. At the end of the experiment, before leaving, the participant filled out a small form with the information mentioned previously. The participant’s name was not recorded and participant consent was required. Figures 2 shows the experimental setup with a participant testenig with the touchscreen.

The Impact of Tangibility in the Input of the Secondary Car Controls

179

Fig. 2. Participant testing with the touch screen in the experimental Setup.

2.5 Data Processing Everything directly related with the secondary controls inputs, the touchscreen and response pad, was recorded using SuperLab. This means that it was only necessary to copy these results to a spreadsheet.

Fig. 3. Reference frame overlaid in the simulator

Fig. 4. Reference frame guide

Dealing with driving performance wasn’t as straightforward. To check if participants were going outside the lane or speeding, this had to be done manually by reviewing a video recording of the simulator and marking how many times these errors were committed for each participant. There were 5 levels of driving outside the lane, which were identified by overlaying an image on top of the video recording with markings for each kind of error (Figs. 3 and 4), based on tire path and third-person view examinations.

3 Results Results were treated considering three types of data: secondary controls reaction time (Fig. 5), secondary controls errors (Table 1 and Fig. 6) and driving performance (Fig. 7). To define driving performance, a Driving Error Score was calculated using a formula based on the 5 levels of driving outside the lane, and speeding: Number of times going

180

M. Masseno et al.

outside the lane (level 1) + (Number of times going outside the lane (level 2) * 2) + (Number of times going outside the lane (level 3) * 3) + (Number of times going outside the lane (level 4) * 4) + (Number of times going outside the lane (level 5) * 5) + Number of times speeding (Speed > 100 km/h). This formula weighs errors that are more severe more heavily. Going completely outside the lane increases the score much more than only going slightly outside the lane.

Fig. 5. Reaction time for the tactile interface and the physical interface with buttons.

In order to understand the difference between driving performance for the tactile and physical interface, the averages of the reaction time and errors were calculated for all 35 events for the total of 20 participants in our study. Table 1. Percentage of errors for the physical and tactile interface. Number of error

Response Pad (Physical interface)

Touchscreen

0

80%

20%

1

10%

30%

2

10%

20%

3

0%

20%

4

0%

10%

From the average of all 20 participants a comparison was made between the distribution of time and errors for the two conditions. Since the distribution of time and errors was not normal, the non-parametric Wilcoxon test for dependent samples was

The Impact of Tangibility in the Input of the Secondary Car Controls

181

chosed. The Wilcoxon test revealed the existence of a statistically significant difference, for reaction times (z = -2.277; p < 0.05), for errors (z = -3.247; p < 0.001) and for the driving error score (z = -3.283; p < 0.001). Figure 5 illustrates the distribution of times and Fig. 6 illustrates the driving error scores. Table 1 illustrate the number of interface errors committed in the form of a percentage of participants. Figure 7, displays the calculated driving error score for each participant, also for both the response pad and touchscreen.

Fig. 6. Driving error score for the tactile interface and the physical interface with buttons.

4 Conclusions Overall, the type of interface impacted reaction times and greatly impacted the number of errors in the interface. On average, participants’ reaction time was 96 ms longer with the touchscreen than with the response pad, without removing incorrect presses. As can be observed in Table 1, 80% of participants never pressed the incorrect button when interacting with the response pad, while inversely only 20% made no mistakes with the touchscreen. These results are, on their own, significant when it comes to road safety. Not only does the extended reaction time increase the amount of time in which the driver’s attention is divided between tasks, but also the large amount of errors can result in longer periods of divided attention, since the driver would need to rectify the mistake. It also puts in question the effectiveness of a touchscreen interface as a viable replacement for physical interfaces, since it results in more errors. When it comes to actual driving performance, there was a substantially larger amount of errors made while using the tactile interface for the secondary controls (Figs. 6 and 7), which means that greater reaction times translate into poorer driving performance as

182

M. Masseno et al.

hypothesized. It can also be inferred that tactile interfaces require more attention from the driver.

Fig. 7. Displays the calculated driving error score for each participant, also for both the tactile interface (Touchscren – RED) and the physical interface (Button - BLUE).

Results show that the use of a tangible interface results in safer driving, less prone to errors in the secondary controls as well as on the road. However, there are many more ways in which data that was extracted in this study could be analyzed. In the future, it could also be interesting to check how removing incorrect presses would impact the data (e.g. a participant could have a faster response time with the touchscreen but press the wrong button, which, in the way the data has been processed so far, would result in more positive results for the touchscreen), or how the difficulty of driving in each event could impact reaction times and errors. At a time when the industry in general, and the automotive industry in particular, are investing more and more in touchscreens, these results raise important questions about the fact that they are not the best solution in terms of safety. It is true that in terms of possible configurations, touchscreens give great freedom to car manufacturers, but at the expense of road safety. To properly frame the results and conclusions of this study, some limitations to be resolved in future studies must be highlighted. The sample should be increased both in its dimension and in its diversity, seeking to heterogenize the sample also in terms of more diverse age groups. The use of the simulator, based on a game computer platform, also constitutes a limit of this study. The 35 events (secondary stimuli) were activated by two experimenters. One who kept his attention on the route and at the right time for each event gave a touch to the other experimenter who triggered the secondary task. In a driving simulator dedicated to the study of car driving, this management is done automatically, always in the same place and moment. Naturally, when we pass this management on to the experimenters, we can introduce small spatiotemporal variations at the beginning of the secondary stimulus, which despite not being significant for the results of the study, deserve to be mentioned as a limit of the study.

The Impact of Tangibility in the Input of the Secondary Car Controls

183

Naturally, a driving simulator also gives automatic outputs of driving performance. In the case of this study, the performance evaluation required an analysis of all recordings of the experiments in order to properly evaluate the driving performance, which was a very time-consuming process and is not free from the possibility of error. Acknowledgments. This work is financed by national funds through FCT - Fundação para a Ciência e a Tecnologia, I.P., under the Strategic Project with the references UIDB/04008/2020 and UIDP/04008/2020, and Interative Technologies Institute -LARSyS-FCT Pluriannual funding 2020–2023 (UIDB/50009/2020).

References 1. Ng A, Brewster SA, Beruscha F, Krautter W (2017) An evaluation of input controls for in-car interactions. CHI Conf. Hum. Factors Comput. Syst. - Proc. 2017-May:2845–2851 2. Lee JD, Young KL, Regan MA (2008) Defining driver distraction. In: Driver Distraction: Theory, Effects, and Mitigation 31–40. CRC Press. https://doi.org/10.1201/9781420007497 3. Bellinger DB, Budde BM, Machida M, Richardson GB, Berg WP (2009) The effect of cellular telephone conversation and music listening on response time in braking. Transp Res Part F Traffic Psychol Behav 12:441–451 4. Aasman J, Michon JA (1992) Multitasking in Driving, pp. 169–198. https://doi.org/10.1007/ 978-94-011-2426-3_6 5. Brookhuis KA, Waard D de (2000) Assessment of Drivers’ Workload: Performance and Subjective and Physiological Indexes. Stress. Workload. Fatigue 321–333. https://doi.org/10.1201/ B12791-2.5 6. Makishita H, Matsunaga K (2008) Differences of drivers’ reaction times according to age and mental workload. Accid Anal Prev 40:567–575 7. da Silva FP (2014) Mental workload, task demand and driving performance: what relation? Procedia - Soc. Behav. Sci. 162:310–319 8. Gentaz E, Baud-Bovy G, Luyat M (2008) The haptic perception of spatial orientations. Exp Brain Res 187:331–348 9. Brumaghim SH, Brown DR (1968) Perceptual equivalence between visual and tactual pattern perception: An anchoring study. Percept Psychophys 4:175–179

Dep-Project: Comparative Analysis of Online Solutions Aimed at Personalization and Purchasing Prostheses Sara Peixoto1(B) , Nuno Martins2 , Demétrio Matos2 , Vítor Carvalho3 , and Daniel Miranda3 1 School of Design, Polytechnic Institute of Cavado and Ave, 4750-810 Barcelos, Portugal

[email protected]

2 ID+ Research Institute in Design, Media and Culture, School of Design,

Polytechnic Institute of Cávado and Ave, 4750-810 Barcelos, Portugal {nmartins,dmatos}@ipca.pt 3 2Ai, School of Technology, Polytechnic Institute of Cávado and Ave, 4750-810 Barcelos, Portugal {vcarvalho,damiranda}@ipca.pt

Abstract. This study is part of the “Dep-Project” research project, which aims to provide upper limb amputees with access to low-cost prostheses. The prostheses are 3D printed and are wearable and personalizable. In this sense, the need arose for the present study, to develop an online platform, with an intuitive interface design, which allows this type of disabled user to have easy and pleasant access to information about the prostheses and personalized them, according to their measures and personal taste. This article presents the first phase of the study, which consisted of a literature review on universal design and inclusive design; and a UX benchmarking assessment on the current main references available in the market of online platforms aimed at the personalization and purchasing of prostheses. The heuristic evaluation by Jakob Nielsen (2020) was the base methodology adopted for this study. As a result, there were identified the main requirements that an online platform must have to guarantee the inclusion of users with this type of disability. Keywords: Inclusive Design · Universal Design · User Experience · User Interface · Upper Limb Prostheses

1 Introduction This study is part of the “Dep-Project” research project, whose focus is on the development of 3D-printed personalized wearable prosthetics for upper limbs. With this, the project intends to create an intuitive, accessible and appealing interface, with the intention of helping amputees in all the process of personalizing their wearable prosthesis and, consequently, its acquisition. Through this study, it is also intended to contribute to inclusion and to the reduction of prejudice towards those who suffer from some type of disability. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 184–199, 2024. https://doi.org/10.1007/978-3-031-47281-7_15

Dep-Project: Comparative Analysis of Online Solutions Aimed

185

As a part of the “Dep-Project”, this paper focuses on the analysis of existing digital platforms, aimed at the personalization and acquisition of prostheses. It begins with a literature review on universal design and inclusive design. This section addresses authors such as Story et al. (1998) and Swan et al. (2017), who developed the principles of universal design and inclusive design, respectively. In the third section, a small introduction is made to the adopted evaluation methodologies. In the fourth section, a UX Benchmarking Analysis presented by Kshirsagar (2018) is elaborated, where the objectives of this analysis, the competitors, and the respective comparative analysis of the competitors are defined. In the following section, the Heuristic Evaluation by Jakob Nielsen (2020) is presented, characterized by the search for usability problems in the user interface design. Lastly, the main conclusions and identified problems are presented, in order to prevent them in the future.

2 State of the Art 2.1 Universal Design & Inclusive Design This project consists of developing an intuitive online platform that allows the inclusion of users with some type of disability. In this section we present a review of the literature about universal design and inclusive design, such as their focuses and principles. Universal design and inclusive design are approaches that aim to create products, environments and experiences that are accessible to a wide range of individuals, including those with disabilities. While there is some similarity between the two concepts, they have different focuses and principles. According to Ronald Mace, universal design is the usability and accessibility of all people to the design of products and environments, without the need for adaptation or specialized design (as cited in Persson et al., 2015). In other words, universal design is an approach that seeks to design products, environments, and systems that are accessible and usable by as many people as possible, not just focusing on a minority of the society (Centre for Excellence in Universal Design [CEUD], 2020). From 1994 to 1997, the Centre for Excellence in Universal Design and a group of U.S. experts conducted a research and demonstration project titled “Studies to Further the Development of Universal Design”, funded by the United States Department of Education’s National Institute on Disability and Rehabilitation Research (NIDRR). In this project, one of the activities was to develop a set of universal design guidelines. The resulting universal design principles are as follows (Story et al., 1998): • Equitable use - the design should be usable by people with diverse abilities and disabilities, so that no single user group is stigmatized or disadvantaged; • Flexibility in use - allow users to customize or adapt the design to suit their specific needs; • Simple and intuitive use - the design must be simple and intuitive, facilitating the understanding and interaction of users with the product or service, regardless of their background or prior knowledge; • Perceptible information - regardless of the user’s sensory capabilities, the design must be able to communicate information effectively and in multiple ways to the user;

186

S. Peixoto et al.

• Tolerance for error - the design must minimize the consequences of errors that the user may commit through accidental or unintentional actions, and allow for easy recovery if this occurs; • Low physical effort - taking into account people with limited strength or mobility, the design must be efficient and comfortable for the user to use with minimal physical effort. • Size and space for approach and use - provide sufficient size and space to accommodate different user capabilities and assistive devices, regardless of user’s body size, posture, or mobility. These principles are useful for “evaluating existing designs, guiding the design process, and educating designers and consumers about the characteristics of more usable products and environments” (CEUD, n.d.). Inclusive design, in turn, considers the needs and experiences of specific groups of people, such as oppressed groups that experience exclusion and face specific barriers and challenges (Querini, 2023). Therefore, it is necessary to adapt or develop design solutions that respond to the needs of these specific groups (Martins et al., 2023). Swan et al. (2017) developed a set of principles to take into account for creating a product that is inclusive for all. Therefore, these principles are as follows: • Provide comparable experience - the interface has to allow the user to carry out tasks with efficiency and quality, in a way that meets his needs, without compromising the quality of the content. • Consider situation - regardless of the user’s circumstances, the interface must be able to meet challenges and provide a valuable user experience. • Be consistent - inconsistent designs can confuse and frustrate people, so it’s important that familiar conventions are applied consistently. • Give control - a good design makes sure that people are in control, giving them the resources to access and interact with the content in the way they prefer. • Offer choice - the design has to be an accessible resource for all users, providing different ways to complete tasks, especially the complex ones. • Prioritize content - prioritizing content and layouts can help users not only focus and complete key tasks, but also find resources and information effortlessly. • Add value - designers must consider the value of device capabilities and use these to enhance the experience for multiple users, by providing efficient and diverse ways to find and interact with content. In short, universal design aims at the universality of products, creating accessible solutions for as many people as possible. Inclusive design, on the other hand, focuses on the inclusion of society by meeting the specific needs of marginalized groups, customizing user-friendly products according to their experiences and challenges. As such, both approaches are important for promoting inclusion and ensuring that everyone can fully participate in society. In this context, the “Dep project” will adopt an inclusive design approach throughout the study, as it focuses on the inclusion of marginalized groups and the creation of inclusive products that respond to their needs and experiences.

Dep-Project: Comparative Analysis of Online Solutions Aimed

187

2.2 Usability Heuristics Jakob Nielsen (2012) states that “usability” is responsible for defining the simplicity faced by the user, when interacting with an interface. Thus, the author considers the following five attributes that must be taken into account when measuring the usability of an interface: • Learnability - If the user completes basic tasks the first time they use the interface, that means that the interface is easy to learn; • Efficiency - After the user knows the interface, the time that he executes a task is what determines the efficiency of the interface • Memorability - If the user easily remembers how to interact with the interface, it means that the system is simple to memorize; • Errors - The system must have a low probability of errors and, if they exist, how serious they are and how they can be easily recovered; • Satisfaction - How pleasant is the user interaction with the interface design. In this context, Nielsen proposed ten usability principles known to be part of the Heuristic Evaluation, who are generally applicable to any type of device. The goal of this evaluation is to simplify the learning process by looking for usability problems in the user interface design and, consequently, improve interactions and promote an efficient user experience of a product or service (Schroeter, 2023). Thus, the set of Ten Usability Heuristics are as follows (Nielsen, 2020): • Visibility of system status - through feedback, the user must be informed about the system operations, which are taking place in a certain period of time; • Match between system and the real world - the designer must facilitate the user experience, considering concepts familiar used in a real experience; • User control and freedom - the designer must provide the user with the ability to undo and redo actions; • Consistency and standards - maintain a cohesive language throughout the interface, using elements and sequences that are consistent with other platforms used by the user; • Error prevention - the interface has to be able to prevent any type of carelessness on the part of the user, where these are avoided through warning messages; • Recognition rather than recall - due to humans’ limited short-term memory, it’s important to minimize the user’s memory load by making elements, actions, and options always visible and easily retrievable to the user. • Flexibility and efficiency of use - possibility for the user to customize its interface, according to the needs it has; • Aesthetic and minimalist design - present an interface with only the relevant information, so that the user is not distracted by information of little importance; • Help users recognize, diagnose and recover from errors - in order for the user to be aware of what is happening, the system must transmit error messages in a simple and clear language, indicating the problem and suggesting a solution; • Help and documentation - the user may need to refer to the documentation to clarify their doubts, so it’s important that the information is accessible and presented in a direct and clear way.

188

S. Peixoto et al.

The application of these heuristics proves to be a fundamental procedure in the development of more adaptable and intuitive digital solutions, which meet the needs of users and provide a more positive browsing experience. In this context, the usability heuristics were relevant for the analysis of the platforms’ interfaces, where a comparative evaluation between them was carried out (Jeddi et al., 2020; Khajouei et al., 2018; Ruiz et al., 2021; Silva et al., 2023).

3 Methodologies For the development of this study, evaluation methods focused on platform analysis were adopted, more specifically their functionalities and respective interfaces. In the first method, a survey and comparative analysis of digital tools aimed at customizing and acquiring prostheses was conducted. This research aimed to develop a UX benchmarking analysis to understand what has already been developed in this area: on the one hand, to identify the strengths that can help to improve this project; and, on the other hand, identify the gaps, avoiding making the same mistakes. In the second method adopted, the Heuristic Evaluation, the interfaces of the platforms previously identified in the UX benchmarking were compared based on the ten usability heuristics. In this way, it was possible to identify the mistakes that shouldn’t be committed in the development of an intuitive and adaptable solution. The tools analyzed were: ALLELES, UNYQ, Unlimited Tomorrow, Open Bionics and Esper Bionics. These tools can be used by patients who want to purchase prostheses and clinics that want to become partners of the company.

4 Development of UX Benchmarking Analysis Nielsen states that 61% of designers use the competitive analysis method for the development of digital projects (Kshirsagar, 2018). Thus, in this project, a benchmarking analysis was carried out using the competitive analysis method (Kshirsagar, 2018), where it intends to analyze the performance of digital tools directed directly or indirectly in the personalization and acquisition of prostheses. According to Schade (2013), it is important that this method is done taking into account the user’s perspective or an expert’s perspective, and not through intuition or personal perspective. On the other hand, they have benefits such as: “risk reduction” where it’s important to get early insight into features that don’t work; “adding value” by discovering relevant features that can benefit users; helping to understand our product and to solve usability problems; knowing the strengths and weaknesses of the competition; and learning more about the target audience. The competitive analysis, according to Susan (2017), is part of a cycle called the service design cycle, which contains four phases: discover, explore, test and listen. Each of these phases has multiple research methods, with competitive analysis being part of the exploration phase. This evaluation method was implemented in the initial phase of the project, with the aim of discovering the strengths and gaps of competing platforms. In this way, during the development of the project, it is known which mistakes cannot be made.

Dep-Project: Comparative Analysis of Online Solutions Aimed

189

Overall, it is important that designers know their competition and gather as much data as possible. These help create a solid, user-friendly product that users really want to use. Therefore, a benchmarking analysis was carried out according to the competition analysis method, as presented in the next section (Kshirsagar, 2018). 4.1 Define and Understand the Goals At this stage, the specific objectives and respective goals that we intend to achieve in developing this analysis have been defined (Table 1). Table 1. Define and understand the goals. Objectives

Goals

What are the platforms linked to the acquisition of prostheses? And those that allow to customize the product?

• Discover the right platforms;

How is the product personalization presented visually?

• Identify the various ways the product is being presented;

How can the interface provide a good user experience, given the user’s physical limitations?

• Identify the criteria to be taken into account for the creation of an intuitive and usable interface; • Reduce the effort in interpreting the platform; • Give control to the user to be able to change the interface according to his needs;

How can the user have greater control in handling the platform?

• Find out how other platforms do;

How does the competition present its features?

• Identify the functionalities of the platforms and how they were implemented;

What is the user interface and the visual structure of these platforms?

• Check the visual appearance of the platforms (interaction and UI);

How can the user get important information • Make a clear and objective platform; about the product without too much text load? Which platforms have also developed a complementary mobile application?

• Discover those platforms;

In what way is there psychological compensation on the part of the product?

• Identify the factors necessary for psychological compensation to be well applied;

4.2 Definition of Competitors According to Jaime Levy (2015), the competitors are divided in two types: direct and indirect. The direct competitors offer the same or similar set of resources for the same problem, that is, they are solving the same problems that we intend to solve. The indirect

190

S. Peixoto et al.

competitors offer a similar set of features to a different customer segment, or have the same customer segment as we are targeting but don’t offer the exact same feature set. Thus, direct and indirect competitors were defined as: • Direct competitors - ALLELES, UNYQ, Unlimited Tomorrow. • Indirect competitors - Open Bionics, Esper Bionics. 4.3 Competitive Analysis Matrix A matrix was prepared with direct and indirect competitors, where the platforms were properly separated according to their category. On the other hand, these were analyzed according to a set of characteristics related to the objectives and goals that were defined in the previous step (Table 2). Table 2. Competitive analysis matrix. Direct Competitors Functionalities Target: amputees Target: clinicians User adaptation Personal profile / registration Online personalization of the prosthesis 360º visualization of the prosthesis Covers and colors of the prosthesis Training videos Amputees stories, feedback,... Photographic images of product use Resources News / Blog Multiple languages Complementary mobile application Chatbot

ALLELES

UNYQ

Indirect Competitors Unlimited Tomorrow

Open Bionics

Esper Bionics

Dep-Project: Comparative Analysis of Online Solutions Aimed

191

4.4 Analysis of Competing Online Platforms Alleles ALLELES (Fig. 1) is an online platform for the purchase of lower limb prostheses, whose brand aims to change the perception of the appearance of disability. One of the characteristics that distinguishes this platform from the others is the combination of online personalization of the product, where it offers a huge variety of options, and the 360º visualization of the product, where it is possible to visualize even the smallest detail. Still about the product, this platform presents very complete resources with important information for the amputee, such as a prosthesis measurement guide and ways to modify its cover. This platform allows user registration, thus facilitating the storage of information. And, on the other hand, it has a blog with real images of an amputee using his prosthesis (Fig. 2), and it also has a chatbot incorporated that allows the user to clarify any doubts that he may have.

Fig. 1. ALLELES homepage screen. Source: http://alleles.ca/, n.d.

Fig. 2. Amputee using his ALLELES CHROME prosthesis. Source: https://alleles.ca/alleles/, n.d.

192

S. Peixoto et al.

UNYQ UNYQ (Fig. 3) is an online platform aimed at purchasing lower limb prostheses, which has a complementary mobile application. One of the features that this platform distinguishes from others is the variety of languages offered. This platform, whether in desktop or mobile, has online personalization of the prostheses, however, it has little variety of options with preconceived images. On the other hand, it’s only possible to share the photo of the personalization. If the user wants to purchase a prosthesis, he has to fill out a questionnaire so that the team can get in touch. As for the mobile application, the entire personalization process has more options, such as changing the color and specific measurements necessary for the construction of the prosthesis. On the other hand, on the platform it’s not possible to register, but in the mobile application this is conceivable. This platform also has a lookbook, which features images of amputees with their respective prothesis (Fig. 4).

Fig. 3. UNYQ homepage screen. Source: https://unyq.com/, 2022.

Fig. 4. Amputee using his UNYQ prosthesis. Source: https://unyq.com/lookbook/, 2022.

Unlimited Tomorrow Unlimited Tomorrow (Fig. 5) is an online platform aimed at purchasing upper limb

Dep-Project: Comparative Analysis of Online Solutions Aimed

193

prostheses, whose brand’s mission is to make high-quality, low-cost prostheses through the use of 3D printing. The characteristic that distinguishes this platform from others is the possibility for the user to change its interface, according to his needs. It has several options such as increasing the text size, changing the contrast, supporting people with dyslexia, among others. This is a differentiating element as it helps provide a good user experience. On the other hand, this platform incorporates a chatbot that allows the user to clarify their doubts, and also has real stories of amputees told through videos and images with them using the prosthesis (Fig. 6). Regarding prostheses, although the platform has online personalization and 360º visualization of these, the variety of personalization options is very scarce. The only possible change in the prosthesis is the skin color.

Fig. 5. Unlimited Tomorrow homepage screen. Source: https://www.unlimitedtomorrow.com/tru elimb/, n.d.

Fig. 6. Amputee using his TrueLimb Prosthetic Arm. Source: https://www.unlimitedtomorrow. com/stories/, n.d.

Open Bionics Open Bionics (Fig. 7) is an online platform for purchasing upper limb prostheses, which has a complementary mobile application. The characteristic that distinguishes this platform from the others is the fact that it has training videos and an occupational therapy training guide. This platform only provides

194

S. Peixoto et al.

thematic prostheses, and the online personalization of the prostheses is not possible. The personalization of the prosthesis can only be carried out with the Open Bionics team, and the user must fill out an online questionnaire so that this team can get in touch. This platform allows user registration, thus facilitating the storage of information. On the other hand, as previously mentioned, this brand has an application that connects the prosthesis to the smartphone via Bluetooth, and where the amputee has interactive challenges to test the prosthesis, being able to change the brightness of the LED, the power of the vibration and check the status of the prosthesis battery. This platform also has the life stories of amputees, namely the cause of their amputation, and these stories are complemented with images of them using their prostheses (Fig. 8).

Fig. 7. Open Bionics homepage screen. Source: https://openbionics.com/en/, 2023.

Fig. 8. Amputee using his prosthetic Hero Arm. Source: https://openbionics.com/bionic-zak/, 2023.

Esper Bionics Esper Bionics (Fig. 9) is an online platform for the purchase of upper limb prostheses. This platform is the most incomplete of all, since it only makes known the features of the prosthesis, tells a little about the brand’s history and has the possibility for investors to invest in the brand, among others. Despite being the most incomplete, this one has some images of amputees using prosthetics (Fig. 10), like the other platforms.

Dep-Project: Comparative Analysis of Online Solutions Aimed

195

Fig. 9. Esper Bionics homepage screen. Source: https://esperbionics.com/, 2023.

Fig. 10. Amputee using her Esper Bionics prosthetic arm. Source: https://esperbionics.com/com munity/, 2023.

5 Heuristic Evaluation As mentioned earlier, the heuristic evaluation aims to simplify the learning process, where usability problems in user interface design are looked for. Thus, this was the second evaluation method adopted, where a comparative analysis between platforms was performed based on the usability heuristics (Table 3). Regarding the platforms analyzed, the common points between all of them are the presentation of familiar concepts to the user, having a visible menu in order to facilitate user navigation and presenting an aesthetically appealing interface. The light backgrounds are the predominant factor on all platforms - except ALLELES - highlighting the use of colors such as oranges and blues, simultaneously showing concern in the perception of information and clarity in reading. However, despite ALLELES being the only platform with a dark background, the clarity in reading is not conditioned, since the typography presents a good contrast in relation to the background. On the other hand, although all platforms have an aesthetically appealing interface, Esper Bionics stands out for the minimalism of its interface, for the whole approach adopted in the presentation of its product and for the satisfactory interaction with the interface.

196

S. Peixoto et al. Table 3. Comparative analysis based on the Heuristic Evaluation. Direct Competitors ALLELES

UNYQ

Indirect Competitors Unlimited Tomorrow

Open Bionics

Esper Bionics

Match between system and the real world User control and freedom Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help and documentation

Of all the analyzed platforms, only two - ALLELES and Unlimited Tomorrow allow the user to go back if he makes a mistake. On the other hand, only the Unlimited Tomorrow platform gives the user the possibility to change its interface according to his needs. This is an important element since we are talking about people who have disabilities. With regard to help and documentation, all platforms - except Esper Bionics - try to help the user understand how to carry out their tasks, such as, for example, how to purchase a prosthesis or even after acquiring it, what training they can do to stimulate it. In this analysis, there were also identified gaps that should be taken into account when carrying out the solution, so that the same mistakes are not made. UNYQ presented the highest number of gaps, namely: • The use of little leading between lines of text and its incorrect alignment, making it difficult for the user to read. (Fig. 11); • Little spacing between elements, thus making it difficult to recognize them (Fig. 11); • Little contrast on some occasions. In this case (Fig. 12), it’s the button of the CCollection covers; • Bad distribution of information, which consequently will cause confusion in the user.

Dep-Project: Comparative Analysis of Online Solutions Aimed

197

Fig. 11. UNYQ “Our covers” screen. Source: https://unyq.com/process-for-users/, 2022.

Fig. 12. UNYQ homepage screen. Source: https://unyq.com/, 2022.

6 Conclusions This article intended to analyze existing online platforms, aimed at personalization and purchasing prostheses. This analysis showed that some of the project’s goals, such as including excluded groups and creating an intuitive interface, were achieved by some platforms, such as Unlimited Tomorrow and Esper Bionics, respectively. At the same time, characteristics were also identified that could be an added value to be incorporated in the development of the online platform, namely, the simpler the interface, the faster it is the interpretation, as it was possible to verify in the Esper Bionics interface. This interface has a strong visual coherence, making it aesthetically very appealing. Another important feature for the development of the online platform is the approach to personalize the prosthesis, in particular, personalization with several options and a well-conceived visualization of the product, as it was possible to verify at ALELLES. This offers a variety of personalization options and a 360º view of the product, allowing you to see the prosthesis down to the smallest detail. The use of videos and illustrations was also an element that stood out on some platforms, notably ALLELES and Open Bionics. Both resorted to these view for the transmission of information, such as training, how to adjust the prosthesis cover, how to measure the leg, among others. With this, these features help the user to perform certain tasks more efficiently. Finishing the sequence of valuable features, we have the personalization of the interface. Unlimited Tomorrow has implemented this functionality, giving the user the possibility

198

S. Peixoto et al.

to personalize their interface, depending on their needs. Some of these are related to typography, images, color and reading. In this study, are highlighted the following characteristics that should be taken into account when the online platform is developed, such as the creation of an intuitive interface, user assistance and visualization of the prosthesis. Regarding the creation of an intuitive interface, Esper Bionics stands out for the minimalism of its interface and a good distribution of information. With regard to user assistance, Unlimited Tomorrow stands out for implementing the customization of the interface, thus meeting the user’s needs. And, on the other hand, it has implemented a chatbot that allows the user to clarify doubts more easily. Regarding the customization of the prosthesis, ALLELES stands out due to the variety of options and the 360º visualization of the product. Keeping the user-centered design process, the next phase of the study will consist of the development of information architecture, wireframe development and prototyping, based on the data collected in this analysis (Martins et al., 2020). Throughout the entire platform development process, some of the principles presented in this article will be applied, such as the inclusive design principles of Swan et al. (2017) and Nielsen’s principles (2012, 2020). The application of these becomes crucial in the development of an inclusive and intuitive interface, since inclusive design focuses on the development of inclusive products that respond to the needs of marginalized groups, and Nielsen’s principles in the usability of an interface and the development of more intuitive products, which respond to users’ needs. The non-application of these principles would have a negative impact on the interface, resulting in an unsuccessful digital platform, since it wouldn’t fulfill the established objectives such as the development of an inclusive interface. In the last phase of the study, based on a functional prototype, usability tests will be carried out with the aim of verifying the effectiveness of the options taken and, in parallel, collecting information on possible improvements. In this way, it will be possible to create a solution that meets the needs of different users. Acknowledgements. The project is funded by national funds through FCT – Fundação para a Ciência e a Tecnologia, I.P., in the framework of the project «Ref: PeX_2022.09053.PTDC».

References ALLELES (n.d.) http://alleles.ca/ Center for Excellence in Universal Design (2020) What is Universal Design. Retrieved from: https://universaldesign.ie/what-is-universal-design/ Esper Bionics (2023). https://esperbionics.com/ Jeddi FR, Nabovati E, Bigham R, Farrahi R (2020) Usability evaluation of a comprehensive national health information system: a heuristic evaluation. Info Medi Unlock 19:100332. https://doi.org/10.1016/j.imu.2020.100332 Khajouei R, Gohari SH, Mirzaee M (2018) Comparison of two heuristic evaluation methods for evaluating the usability of health information systems. J Biomed Inform 80:37–42. https://doi. org/10.1016/j.jbi.2018.02.016 Kshirsagar M (2018). A brief guide about competitive analysis. Smashing Magazine. Retrieved from: https://www.smashingmagazine.com/2018/08/guide-competitive-analysis/

Dep-Project: Comparative Analysis of Online Solutions Aimed

199

Levy J (2015) UX Strategy: How to Devise Innovative Digital Products that People Want. O’Reilly Media, Inc. Retrieved from: https://booktree.ng/ux-strategy-how-to-devise-innovative-digitalproducts-that-people-want-by-jaime-levy-pdf/ Martins N, Campos J, Simoes R (2020) Activerest: Design of a graphical interface for the remote use of continuous and holistic care providers. Adva Sci Technol Eng Sys J 5(2):635–645. https://doi.org/10.25046/aj050279 Martins N, Brandão D, Guimarães L, Penedos-Santiago E, Brandão E (2023) The Importance of Communication Design in the Process of Disseminating Community Practices in Social Neighbourhoods: The Balteiro. Comunicação e Sociedade 43:e023010. https://doi.org/10.17231/com soc.43 Nielsen J (2012) Usability 101: Introduction to Usability. Nielsen Norman Group. Retrieved from: https://www.nngroup.com/articles/usability-101-introduction-to-usability/ Nielsen J (2020) 10 Usability Heuristics for User Interface Design. Nielsen Norman Group. Retrieved from: https://www.nngroup.com/articles/ten-usability-heuristics/ Open Bionics (2023). https://openbionics.com/en/ Persson H, Åhman H, Yngling AA, Gulliksen J (2015) Universal design, inclusive design, accessible design, design for all: Different concepts—one goal? On the concept of accessibility— historical, methodological and philosophical aspects. Univ Access Inf Soc 14:505–526. https:// doi.org/10.1007/s10209-014-0358-z Querini V (2023) What is Inclusive Design? A Beginner’s Guide. The CareerFoundry Blog. Retrieved from: https://careerfoundry.com/en/blog/ux-design/beginners-guide-inclusive-des ign/ Ruiz J, Serral E, Snoeck M (2021) Unifying functional user interface design principles. Int J Hum-Comp Int 37(1):47–67. https://doi.org/10.1080/10447318.2020.1805876 Schade A (2013) Competitive usability evaluations: Definition. Nielsen Norman Group. Retrieved from: https://www.nngroup.com/articles/competitive-usability-evaluations/ Schroeter E (2023) 11 Usability Heuristics Every Designer Should Know. The CareerFoundry Blog. Retrieved from: https://careerfoundry.com/en/blog/ux-design/usability-heuristics/ Silva R, Lima AC, Andrade E, Martins AI, Rocha NP (2023) Heuristic evaluation of the usability of a mechanical ventilator interface through a simulator. Procedia Computer Science 219:1232– 1239. https://doi.org/10.1016/j.procs.2023.01.406 Story MF, Mueller JL, Mace RL (1998) The Universal Design File: Designing for People of All Ages and Abilities. Revised Edition. Retrieved from: https://www.certec.lth.se/fileadmin/cer tec/Kirre/102-154-1-PB.pdf Susan F (2017) UX Research Cheat Sheet. Nielsen Norman Group. Retrieved from: https://www. nngroup.com/articles/ux-research-cheat-sheet/ Swan H, Pouncey I, Piclkering H, Watson L (2017) Inclusive Design Principles. Retrieved from: https://inclusivedesignprinciples.org/ Unlimited Tomorrow (n.d.). https://www.unlimitedtomorrow.com/truelimb/ UNYQ (2022). https://unyq.com/

Digitization Project of Museum Collections as a Way of Preserving Memory and Cultural Heritage Clara Pache de Faria1(B)

and Rafaela Norogrando2

1 FAL - Departament of Arts, LabCom | IA*, University of Beira Interior, Covilhã, Portugal

[email protected]

2 FAL - Departament of Arts, ID+ Research Institute for Design, Media and Culture and

LabCom | IA*, University of Beira Interior, Covilhã, Portugal [email protected]

Abstract. This article was developed through descriptive exploratory methodology, on the digitalization to preserve historical costumes in 3D. Without minimizing the work of the conservators who provide physical longevity to artifacts, it considers the difficulties present in the daily life of museums and textile conservation, and the human and virtual interaction practices. Through a bibliographic review on the techniques of capture and reproduction of objects and museum collections, it presents the conections of the technological, museological and conservation areas, and reflects on the possibility of preserving the material and non-material culture, through the identity and symbology present in this type of artifacts, even through their dematerialization into a digital object. This study aims to contribute to the research and reproduction of heritage collections, as well as to collaborate with conservators and increase the possibilities of accessibility and understanding of contents for the public. In this article, it is also proposed that, in the future, a design project will be created, in which the digitization of a historical costume will be developed by two possible methods. It is intended to find a simpler way to digitize clothing and integrate interactive displays through Augmented Reality. Keywords: Cultural Heritage · Textile Conservation · 3D Historical Costume · Interactive exhibitions · Augmented Reality · Museums exhibitions

1 Introduction The preservation of the appearance of heritage objects through digitization, proposed in this article, aims to dialogue about the possibility of contributing to the presentation of more fragile costumes, avoiding their displacement and handling, providing greater interactivity for museum exhibitions, and to complement investigations of textile collections. Based on the research developed, it was decided to address the forms of conservation and their difficulties, such as the fragility of the textile material itself, as well as issues © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 200–210, 2024. https://doi.org/10.1007/978-3-031-47281-7_16

Digitization Project of Museum Collections

201

related to the need for specialized professionals, and problems related to the resources of museums in general. It presents a brief contextualization on the use of technology and virtualizations present in museums, which currently, with the use of more interactive systems, attract audiences of different age groups. The terms preservation and conservation are used in this study to demonstrate the necessity to safeguard physical cultural heritage, but when used in the context of the digitization of artifacts, they present different characteristics. The appearance information will be preserved in a digital object with their marks and particularities, which is different from the work carried out in conservation. The latter aims to intervene minimally to prolong the life of the piece. Using integration of technology and conservation, this study intends to be a possibility of complementary material for content on historical costumes. The scanning methods addressed in this study were developed from a bibliographic review and visit to exhibitions, punctuating some of the types of capture, 3D modeling and partial or completely immersive presentation, already developed for garments of cultural heritage. This project, derived from MA thesis research, proposes the reproduction of more fragile and ancient historical costumes in 3D replicas by using Augmented Reality (AR) technology to create interactive content for exhibitions, and as a support for researchers and general audiences who have an interest in clothing and fashion history.

2 Textile Conservation and Its Difficulties The International Council of Museums (ICOM-CC) defines conservation as “all measures and actions that aim to safeguard the cultural heritage (…) to future generations” (ICOM-CC, 2008b, cited by Cohn, 2020, p. 28) [4]. The preservation of clothing and textile materials as cultural heritage comes with a need to endure the memory, characteristics from different eras, and the history of civilizations for future generations, as referred above. This enables the diffusion of not only material cultural knowledge, with clothing and objects, but also non-material culture, through expressions, meanings, know-how and traditions of peoples, factors that can be perceived and transmitted using the impressions present in these types of artifacts [7, 18]. Historical textiles have more fragility than other museum objects as their composition is mostly organic. In the case of costumes, for example, in addition to the composition, they are also originally items of utilitarian character, which results in a greater deterioration caused by usage and time. These are factors that lead to difficulties for the full conservation of historical costumes, which can be found incomplete or with more deteriorated parts and pieces depending on the period. Usually, the costumes of the nobility are better preserved, due to the little use of the pieces and greater versatility of wardrobes, unlike the poorer or working-class people, who wore the same clothes daily, leading to a greater wear and tear [20]. Garment related research is often based on paintings, illustrations, books, wills, and inventories from each period [20]. That is because the older the civilization, the more difficult it will be to find a complete costume.

202

C. Pache de Faria and R. Norogrando

For this reason, textile conservation and restoration are very important works, as they enable the durability of costumes and textile fragments, providing a tangible “proof” of the history of civilizations. However, for proper conservation to be made, specialized professionals are required. These professionals will know how to properly identify the materials, sanitize, catalog, and accommodate pieces or fragments, preferably without directly interfering in the structure and form of the museum object. To better define conservation professionals and their broad functions, ranging from structure control, archiving, analysis, documentation, preventive protection, to restoration, ICOM-CC defines the areas of activity as: “preventive conservation”, which aims to minimize the risks of deterioration of objects, in order to increase the durability of an artifact or a set of them; “remedial conservation”, which acts in the stop of processes of deterioration of objects, or even reinforces their structures; and “restauration”, which performs procedures on each object in order to restore the piece to its original form as much as possible, trying to preserve its aesthetic, historical and physical integrity [6, 17]. Textiles and historical costumes conservation is easier to maintain with the ideal conditions in collections and reserves than in exhibitions. This is because pieces in exhibitions suffer more stress and risk of deterioration, for example, due to light, gravity, insects, and pollution [14, 19]. Already in reserves, the control is greater thus enabling the pieces longevity. Depending on the material needs, the possibilities for storage of textile collections carried out by conservators are as follows: vertically, flat, suspended, rolling, padded hangers, on conservation or exhibition supports, or even in furniture with drawers. Always with covers or in boxes of acid-free materials, to avoid contamination, and accommodated without creating folds or creases. It is also important to emphasize that flat is the packaging method used for more fragile or older pieces [14, 19]. To conclude, it is important to mention that the fragility of the textile material increases its susceptibility to deterioration due to light, pollution, humidity, gravity, and handling, reducing, and hindering the exposure of the pieces for a prolonged time. Therefore, large spaces are required for the storage of the collections, and qualified professionals are also needed, to avoid risk of damage to the collections. Another important topic that can be unfavorable to the preservation of such items is the budgetary difficulty, experienced by many museums in the world that need an organization of business plan and budgets to manage the expenses of structure and collaborators each period, which can be annual or every five years [1]. Often because they do not have sufficient profit from the ticket office, museums resort to other means of raising more funds, such as internal gift shops, events, temporary exhibitions, itinerant exhibitions and even providing services of conservators. There are also donations and investment from companies, in addition to those from governments [1].

3 Digitization of Historical Collections and Costumes With the increasing use of technology, and software evolution, one of the methods of preserving objects of material cultural heritage is digitization, which preserves the object appearance by transforming it into a binary form, allowing its information to be

Digitization Project of Museum Collections

203

reproduced, stored and made available in a simple and inexpensive way, as mentioned by Montesiewicz (2021), who also states that in this way the “object” becomes practically indestructible [11]. The online presence of fashion and costume museums around the world with digitized collections is currently an established practice, which happens through the sharing of their contents in a virtual and complementary way to the exhibitions, available for consultation on their websites or blogs. For instance, it is possible to cite: the Costume Institute of The Met (NY); the Museum of the Fashion Institute of Technology (FIT NY); the National Museum of American History, Clothing and Accessories (Washington); the Victoria and Albert Museum (London); and the Kyoto Costume Institute Digital Archive (KCI) [10]; and the Google Arts and Culture platform, which has digitized numerous collections around the world. Some examples of the use of digitization through the capture of high-definition images are: the case of the “Kongedragter”, a virtual exhibition at Rosenborg Castle in Denmark, which allows an interaction of royal costumes via rotation and zoom, where several images in different angles of the same piece were captured and transformed into a.tiff file [8]; as well as the Drexel Digital Museum Project, which has similar functions of rotating and zooming some costumes; and the case of the Virtual Museum of Fashion of Catalunya, which offers an online catalog with more than 600 pieces, photographed with high quality and organized by chronology and type of clothing [10]. As examples of the use of three-dimensional pieces, through photogrammetry process, which will be developed in the following section, it is possible to mention the Museum of Art and History of Santa Cruz, in California, where this technique was used to digitize part of the collection [20]. Other 3D proposals for historical collections developed through 3D modeling are the virtual historical reconstruction project, as immersion in a kind of “metamuseum” with replicas of Spanish costumes of the fourteenth century, developed with the Clo3D software [10], which is also used by Kuzmichev et al. (2018) to recreate a nineteenthcentury costume [9]. This type of software provides a greater realistic capacity, an important factor for the veracity of the pieces through representation of forms, volumes, and fabric texture. This was achieved with the evolution of software developed for digital clothing creation of animations, video games, and for the Apparel Industry, as a way of making the fashion chain process faster and more sustainable. The use of such tools for digitization enabled a creation of more realistic reconstruction of historical costumes, which has been investigated and developed [9, 10, 20].

4 Technologies for 3D Garment Representation 4.1 3D Capture and Modeling Some of the technological solutions that have been used to capture information for scanning material culture artifacts are Touch scanning, Laser scanning, Structured light scanning, Computerized tomography scanning, Photogrammetry and CAD modeling [21].

204

C. Pache de Faria and R. Norogrando

Some of these can automatically reproduce the properties of a 3D model, such as Touch scanning, which can measure the object accurately and then reproduce it. But it turns out not to be an appropriate method to costumes, due to the fragility of the material that would be in contact with the device. Unlike laser scanning, that uses light to read volume and dimensions of the object, such as Structured light scanning [21]. The table below exemplifies the types of capture delimited for this project. The effectiveness of the technique, the ability to capture color, and the possibility of causing damage to the suit are analyzed (Table 1). Table 1. Table of 3D capture accuracy and devices.

˙ Although Zyła et al. (2021) classifies photogrammetry separately from scans [21], this has been one of the most widely used techniques for capturing dimensions and shapes of object surfaces, as it allows highly detailed images, and zoom in and zoom out capabilities, in addition to the rotation function [20]. Even though it is a method that has emerged to determine the geographical topography of an area, it is already widely used for cultural heritage objects [15]. This technique only requires the capture of photographs from various angles of the piece. It can be combined with the use of scanner, such as Lidar, for a better precision of the virtual object through a computer with a specific software. But it is already possible to use iPhones and iPads with Lidar technology in combination with some photogrammetry app, such as Polycam or Reality Scan, for the creation of a 3D object. Although not included in the table, a no less important technique for capturing information is the Computerized tomography scanning, which works as a way of visualizing the interior of historical pieces, used to understand their construction without causing damage to it. For 3D modeling, CAD modeling is a widely used tool for the reproduction of costumes, but in this case, it is necessary to have the measurements and visual information, which may have been captured through some of the scanners mentioned above, except for photogrammetry, which does not capture measurements [21], or manually. As can be seen in the study by the project “Virtual Reconstruction of Historical Men’s Suit” [9], which recreates a costume of Prince Albert of Saxe-Coburg and Gotha, from 1840, developed in Clo3D, through studies of modeling manuals and measurements, as well as the analysis of paintings and illustrations of the time. For this type of reconstruction,

Digitization Project of Museum Collections

205

an experienced professional and more time dedicated to the garment are necessary, so that there is a greater proximity to the real costume [21]. There are many 3D modeling software, but some of the most currently used for construction, improvement and texturing of realistic clothing are Clo3D, Marvelous Design, Blender, Maya and 3DS Max. 4.2 Devices and Types of Presentation At the beginning of the twenty-first century, the technology used by museums was restricted to touch screen, videos, and audio guides, but currently new technologies are adopted, for instance, historical information, exhibition guides, and online catalogs are also offered [20]. Bekele et al. (2018) cites that the use of multimedia is already an established trend around the world and that it provides an improvement of exhibitions, exploration, reconstruction of heritage, and virtual museums [2]. To evaluate the presentation formats, it is important to define the types of realities and concepts that provide different forms of interaction. Augmented Reality (AR) and Virtual Reality (VR) are types of realities used for entertainment, cultural, and museum presentations. These technologies can be presented in various devices, which in this study will be divided into: AR and VR headset, projection, screens, and mobile devices. To present the concepts of realities, it can be said that Augmented Reality and Virtual Reality are experienced through a device that, in the case of AR, will superimpose digital elements on the real environment. On the other hand, VR includes a more complete immersion experience, capable of changing the perception of the environment where the user is. (Qiao et al., 2019; Bekele et al., 2018). The possibility of overlapping virtual content in the real world, achievable with AR, allows the user to have a sensory experience with the purpose of improving the perception and understanding of the real world [2, 13]. This type of reality is widely used through the camera of smartphones and in AR headsets. According to Qiao et al. (2019), VR includes a more immersive experience, “which can alter the perception of our presence in the world” (p.652) [13]. Thus, assuming VR creates a complete immersion environment where the user does not have any perception of the real world, screen devices and projections are not evaluated for this type of reality in the table below. This is because they are considered AR, as they merely add virtual elements to reality, as observed in projections of exhibitions such as “Impressive Monet”, presented in Porto and Lisbon in 2022/23, where even though the audience has a sense of immersion, the paintings, lines, and colors are in fact projected onto real structures (Table 2). In the table, it is possible to observe the types of existing devices defined in this study for the presentation of cultural goods, entertainment, and heritage objects. The types of reality supported by each of them are also evaluated. In the case of VR headsets, only the Valve Index is solely for virtual interaction within the full VR immersion, as it was created with a focus on video games, although it has the possibility to enable the camera to physically interact with the real environment. For this reason, it was not classified as AR. In the case of Meta Quest, in addition to VR immersion, it is possible to use its resources in AR due to the pass-through function, which allows the interaction of VR content also with the visualization of the real world.

206

C. Pache de Faria and R. Norogrando Table 2. Table of realities and device types.

5 Museum and Virtualization Museums have the social role of collecting, preserving, investigating, disseminating knowledge, and dialoguing with society through exhibitions, information strategies, and educational actions. But beyond the social function, museological institutions are a multiple system of methodological procedures, infrastructure, policies, procedures, and research; it is worth mentioning that all of these are important to promote the construction of the museological cultural heritage [12, 16]. Once known as passive and contemplative places, art museums, for example, were seen as collectors of works of art, but today they renew themselves, informing and leading the public to question, criticize, debate, and experiment. Technologies are already incorporated to make the exhibitions more interactive, to instigate the imaginary and the senses of the visitor [16], therefore attracting various types of spectators. In the case of museums that are not entirely dedicated to clothing and fashion, Ferreira (2021) affirms that there is a strong influence and use of clothing as means of supporting the narrative, as observed in museums of history [7]. But in the case of specific fashion museums, clothing becomes the central point, where broader aspects will be approached, such as materials, their uses, history, symbology, techniques, memories, and society. According to Costa (2019), there is an imperative need to build communication in an affective and social way, which can be provided through technologies and virtuality transmitting memory and knowledge produced over the centuries [5]. Museums and exhibitions increasingly have interactivity devices, for instance, in QR code, with information about the exhibition or the artifacts, in projections, or in other interactive systems, such as the Velazquez Tech Museum in Madrid, which used the painting named “Las meninas”, by Diego Velázquez, as inspiration to create an environment full of projections and LED panels. Another example is the Money Museum where it is possible to interact by capturing coins virtually through a large LED panel using the ticket, which acts as a storer of the interactions along the route [3].

Digitization Project of Museum Collections

207

Although technologies are increasingly present in exhibitions, as well as in people’s lives, it is important to reflect on their use in a balanced way, to complement and contribute to the exhibitions in a reflective way, as mentioned by Francisco Providência [3]. Since it is important to maintain the interest of visitors of various age groups, but with the harmonious integration between the artifacts and the virtuality, so that the public can contemplate, explore, and reflect through the curatorial proposal.

6 Project Considering the difficulties inherent in textile conservation, this project proposes the digitization of pieces from heritage collections to enrich the narrative of exhibitions with the use of AR and add complementary content for researchers on the historical periods and techniques of making clothes and costumes. The proposed techniques below can be viable options for museums conserving historical costumes, although they have different levels of needs for resources, which will be discussed below. The AR has been widely used in several areas due to the ease of interaction for the user, mostly because users do not need to have deeper knowledge of new technologies [3]. This technology makes it possible for the public to live different experiences through smartphones. That is why two possible paths were defined for the digitization of interactive AR costumes, which are in the initial phase of testing to be developed in the future. The first one proposes the creation of a three-dimensional replica. This can be made with a study of pre-existing structure and pattern, developed in specific books and manuals for the study and construction of historical costumes, such as the book “Patterns of Fashion”, by Janet Arnold or “Woman’s Dress Patterns”, by V&A Publishing. The latter has studies of patterns, images of details of the internal construction, positioning of the pattern in the fabric and computed tomography of the costume. Alternatively, it can be done through the measurement study carried out in conjunction with a conservation team, where the study can later be reconstructed in a 3D modeling software by a trained professional. In the future, it is intended to develop a second part of the research of this project, which also intends to create 3D models the interiors of the pieces (Fig. 1).

Fig. 1. Design project from conservation study

208

C. Pache de Faria and R. Norogrando

In the second path, the intention is to capture the shape, dimensions, and information of appearance of the piece for the creation of the digital object. This technique will be performed with an app of scanner and processing, or a photogrammetry app through an iPhone with Lidar (Fig. 2). After the capture, the need for improvement of the object will be evaluated, which can be executed in a 3D modeling software such as Blender. And finally, the.obj file will be developed on a platform for AR presentation and can be viewed and interacted on a website. It is important to say that the portable devices currently with Lidar are the Pro models of iPhones and iPads after 2020 and Samsung S20.

Fig. 2. Project design through capture

Both methods intend to develop the presentation layers in a similar way through an AR Framework, a set of software that assist the development of AR products using the.obj file exported from the preparation layer, where the 3D modeling, texturing and necessary improvements will be prepared. Considering the simplest method and the least investment of resources, time, and handling of parts between methods, the second possibility will be the most viable, since it can be executed with the models that are on display without interfering in the costume or the work of the conservation team.

7 Conclusions Evaluating the words of Montesiewicz (2021) on the digitization of objects of material cultural heritage and realizing the fragility of textile conservation, it is possible to say that the digitization of historical costumes can be an effective method of preservation for heritage collections [11], which can facilitate research on and reproductions of costumes and characteristics of different periods and civilizations, without minimizing the importance of conservation, that allows the pieces to be preserved as physical objects, different from the digital form proposed in this study. When discussing the technical and social aspects of historical costumes, it is important to reflect on how much of the non-material culture present in such objects will be maintained with the capturing method, considering that it is through the appearance and digital realism, as well as the symbology of an historical period expressed by the virtual

Digitization Project of Museum Collections

209

object, it will be possible to understand the history, culture, traditions, and knowledge of society. Always considering that it will be possible to digitally reconstruct the clothing as “new”, as proposed in the first method, but also capture the characteristics in which the object is, to preserve the current state of the artifact. It can even be a way of evaluating the conservation practices of the collections and their adversities around the world. It is important to emphasize that the digitization of costumes can be a way to preserve the image representation and symbolism of a piece, since this process will develop a virtual object with the characteristics and appearance of the physical object, but it will not provide more durability, which will only be achieved through the work of conservation. From this research, it is possible to realize that the digitization of artifacts should no longer be considered a trend, because it is already established in several museums, regardless of the means of capture or digital reproduction chosen. The digital objects are used interactively in exhibitions and educational actions, which attract audiences of different interests and age groups. It is worth mentioning that integration of the digital with the museum allows to complement, improve, and increase the accessibility of the contents, even overcoming geographical barriers with contents available on websites and blogs of institutions, or through virtual museums, as proposed by Meier et al. (2021), who mentions the possibility of having digitized models displayed inside a museum or also online, with the differentiated material, so as not to be copies of the face-to-face exhibition [10]. Regarding the types of realities used for the sensory experiences developed in interactive exhibitions, it is possible to evaluate that the use of VR headset, by providing a type of complete immersion through the device, will be an individual and personal experience. Unlike the immersion designed in real environments, which allow a collective and social experience, such as the exhibition “Impressive Monet”. Therefore, it is possible to evaluate that AR projections are methods of greater involvement of the public with the content of the exhibitions due to the social character. The schemes made for the definition of the scanning methods proposed in this project will be developed and evaluated in the future to define simpler ways of capturing and reproducing the virtual costumes. This is intended to complement the contents of both research and interactive exhibitions. Finally, with the presence of technologies increasingly effective in the contemporary world, it is important to reflect on how fundamental or complementary their use will be to museological practices, not removing the value of real artifacts and experiences through visits to exhibitions but integrating technology with other areas. Acknowledgments. LabCom | IA* - Arts group of Communication Sciences Research Institute. This work is financed by national funds through FCT - Foundation for Science and Technology, I.P., under the Project UIDB/04057/2020.

References 1. Constantino BJC (2017) Gestão de Museus em Portugal: Impacto dos Serviços Adicionais na sua Performance [MA thesis]. Universidade de Aveiro

210

C. Pache de Faria and R. Norogrando

2. Bekele MK, Pierdicca R, Frontoni E, Malinverni ES, Gain J (2018) A survey of augmented, virtual, and mixed reality for cultural heritage. ACM J Comp Cultu Herit 11(2):1–36. https:// doi.org/10.1145/3145534 3. Boelter V, Vairinhos M, Sousa Á (2018) The augmented reality in the portuguese museological context [Conference Paper]. UD18, Portugal 4. Cohn M (2020) Conservação Preventiva de Têxteis Arqueológicos em contexto de Reserva [MA thesis]. Universidade do Porto 5. Costa HH (2019) Museus se expõem: diálogo ou contemplação? In: Arte contemporânea : ações expositivas e estratégias museais, pp. 57–71. Ed.PPGART. Santa Maria, RS 6. Djordjevi´c DS, Smelcerovic M, Urosevic S, Djordjevic S (2017) Textile protection through conservation and restoration. Zaštita Materijala 58(1):94–99. https://doi.org/10.5937/zasmat 1701094d 7. Ferreira, E (2021) A arte de salvaguardar os têxteis museológicos para o processo criativo do designer de moda [MA thesis]. Universidade da Beira Interior 8. Johansen, K (2010) “Digital Costume Display” Textile Society of America Symposium Proceedings 26 9. Kuzmichev V, Moskvin A, Moskvina M (2018) Virtual reconstruction of historical men’s suit. Autex Res J 18(3):281–294. https://doi.org/10.1515/aut-2018-0001 10. Meier C, Berriel IS, Nava FP (2021) Creation of a virtual museum for the dissemination of 3D models of historical clothing. Sustainability 13(22):12581. https://doi.org/10.3390/su1 32212581 11. Montusiewicz J, Milosz M, K˛esik J (2021) Structured-light 3D scanning of exhibited historical clothing—a first-ever methodical trial and its results. Heritage Science 9(74) (2021). https:// doi.org/10.1186/s40494-021-00544-x 12. Norogrando R (2015) EXPOSIÇÕES MUSEOLÓGICAS: A Moda por narrativas, experiências e conexões [PhD dissertation]. Universidade de Aveiro 13. Qiao X, Ren P, Dustdar S, Liu L, Ma H, Chen J (2019) Web AR: a promising future for mobile augmented reality—state of the art, challenges, and insights. Proc IEEE 107(4):651–666. https://doi.org/10.1109/jproc.2019.2895105 14. Robinson J, Pardoe T, Scottish Museums Council (2000) An Illustrated Guide to the Care of Costume and Textile Collections. Museums & Galleries Commission. London 15. Skublewska-Paszkowska M, Milosz M, Powroznik P, Lukasik E (2022) 3D technologies for intangible cultural heritage preservation—literature review for selected databases. Heritage Science 10(3). https://doi.org/10.1186/s40494-021-00633-x 16. Souza M (2019) Museus e suas novas museografias: Uma teoria museológica através da expografia e curadoria na arte contemporânea. In: Arte contemporânea : ações expositivas e estratégias museais, pp. 84–99. Ed.PPGART. Santa Maria, RS 17. Terminologia para a definição da conservação-restauro do património cultural material. Resolução aprovada pelos membros do ICOM-CC durante o 15.º Encontro Trienal, Nova Dehli, 22–26 de Setembro de 2008. Conservar Património, 6, pp. 55–56 (2008) 18. Viana F, Neira LG (2010) Princípios gerais de conservação têxtil. Universidade de São Paulo, Revista CPC 19. Vieira T (2021) Setenta Anos de História Teatral Conservação e Restauro de Figurinos Pertencentes ao Círculo de Cultura Teatral/Teatro Experimental do Porto [MA thesis]. Universidade Católica Portuguesa 20. Villarreal, N (2020) The Use of 3D Apparel Simulation Software for Digitizing Historic Costume [PhD Dissertation]. North Carolina State University ˙ 21. Zyła K, K˛esik J, Santos F, House G (2021) Scanning of historical clothes using 3D scanners: comparison of goals, tools, and methods. Appl Sci 11(12):5588. https://doi.org/10.3390/app 11125588

A Social IoT Hybrid Scenario to Promote the Physical, Psychological, and Social Well-Being of Elderly Imran khan1(B)

and Lidia Oliveira2

1 Wenzhou-Kean University, Wenzhou, China

[email protected]

2 University of Aveiro, Aveiro, Portugal

[email protected]

Abstract. The investigation aimed at an IoT-based social hybrid scenario model for the promotion of psychological and physical well-being amongst older adults. Owing to the ever-increasing elderly population, elderly adults face certain psychological, physical, and social well-being challenges that may lead to mental problems and frailty. Through this research, we seek to create a model that enables caregivers, doctors, and family members to monitor elderly adults’ physical data remotely. Additionally, the model enables environmental influence control to promote intentional communication such as online text, audio, video interactions and picture sharing. Furthermore, the model facilitates outdoor activities in the company of peers and friends. The model aims to involve an extensive array of parties that can contribute towards its development and validation, including government bodies, healthcare service providers, primary and secondary care providers, and pharmaceutical companies. This study results indicate that this model results in a paradigm shift in the healthcare delivery system for elderly adults and serves as a deducible step towards enhancing the quality of their lives by developing well-being and removing social isolation. Keywords: Older adults · IOT · elderly Well-being · elderly isolation · social model

1 Introduction The modern society’s population is ageing. Gradually, ageing is becoming a complicated situation for developed nations to ignore the rapid growth of older adults and the decrease in birth rates in societies. The rapid growth of older adults indicated that Older adult numbers are competing with youngster numbers. If the situation remains; soon, Older adult numbers will be more than young people in developed countries. More Older adults in society means reducing the workforce, and a huge number of older adults will be a burden on the state economy because, in developed countries, after retirement, the state takes the responsibility for taking care of older adults’ healthcare and economic issues. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 211–223, 2024. https://doi.org/10.1007/978-3-031-47281-7_17

212

I. khan and L. Oliveira

According to a United Nations report in 2017, a huge number of older adults’ population will increase by 2050. There is another report that reveals UN databases (United Nations 2017) predicted that older adults would double between 2017 to 2050. Thus, the growth of older adults in society causes many healthcare and social challenges, and it is impossible to deal with all these challenges at the same time. Therefore, the Older adult population has many social, psychological and physical problems (Partridge et al. 2018). There are several reasons behind physical and psychological health issues such as social isolation, reduction of social contacts, less interaction with the public, and fewer outdoor physical activities, likewise, these severe reasons can cause a high level of depression and amount of sudden death (Storrs 2016). Although the Older adult population needed health care facilities on an emergency basis. However, there are many countries’ governments, the private sector, and institutes that neglect Older adults’ physical and psychological health issues and there is no determined effort visible that highlights and resolves Older adults’ issues (Partridge et al. 2018). In Older adult health care regards, information, and communication technology (ICT) replaces the traditional healthcare system and offers innovative healthcare systems based on the Internet. ICT is improving human life quality. Therefore, many novel concepts are emerging into our daily lives, as is the case with the idea of the “Internet of things” (IoT). In a smart world, IoT is an innovative and growing concept, in which digital things relate to computers, mobiles, actuators, vehicles and everyday objects (Tamrakar et al. 2022). The IoT is a concept that represents a mixture of different networks, where physical objects and sensors can connect through electronic devices, and software applications, that allow the physical objects to gather data from various destinations, such as browsing websites and sending email, and multimedia content. Due to the importance of IoT applications in many developed countries, such as the United States of America, Japan, and Europe, IoT is a field of novelty and development, both at the academic and commercial levels. Therefore, numerous universities and industries are researching IoT technologies (Da 2022). The IoT is a unique and suitable concept to design an environment in which it is possible to promote the social and physical well-being of older adults as this study proposed a social hybrid scenario model that will help to improve older adults’ life quality. After considering extensive aspects of IoT technologies, this study will propose a design of a social hybrid scenario model, to be validated. The model of the social hybrid scenario will provide two types of communication services, namely passive communication, and intentional communication. Intentional communication includes social interaction, participation and sharing of ideas, namely through social media, and outdoor activities. At the beginning of Internet services, interactive media (social media) communications were minimal, because users had only email or message services, allowing them to communicate with a limited number of people. However, over time, people integrated social media into their lives for different purposes, such as education, communication, knowledge exchange and the development of opinions about an issue (Boyd and Ellison 2013). Due to secure access to the latest technologies, isolated communities, such as the Older adult population, are becoming active members of the digital world, through smartphones, laptops, and

A Social IoT Hybrid Scenario

213

computers (Sixsmith et al. 2022). The latest smart devices are equipped with multimedia content that persuades the Older adult population to use these devices in a different context. According to Na-Songkhla (2011), “The interactive media is a so-called white road”, in which we can design anything that interacts with people according to their desires. Many modern societies (Europe, the USA, & Australia) are using interactive content (social media) for interaction, entertainment, education, and cultural sharing. Therefore, there are many people using dozens of interactive media devices, such as a computer, smartphones, and televisions. These smart devices are disseminating interactive media content via new innovative technologies such as the internet. Innovative technologies are providing a different platform for social interaction, participation, and discussion. Likewise, the isolated Older adult population can minimize their physical isolation and loneliness by using these platforms (Facebook, Instagram, and Twitter) and interacting with other people (Kim and Jasmine 2020). The IoT can make devices and objects smart with the help of different application domains that help to improve passive communication in the context of healthcare and medical services. Due to IoT technologies, it is possible to design modern healthcare and social environments that will make it possible to provide intentional and passive communication scenarios (Punj and Kumar 2018). Passive communication is important for health care personnel and doctors because they can receive physical data of the patient and real-time monitoring, such as information about heartbeats, level of oxygen in the blood, body temperature, heart pulse or how many steps patients walk per minute, hours, or a day. Moreover, through these IoT devices, relatives, friends or loved ones, can also get information about older adults, such as information about geographical positioning and daily activities. This study will design and validate a social IoT hybrid scenario model, intended to promote the physical, psychological, and social well-being of older adults. The prototype will support both intentional and passive communication: functionalities for intentional communication will help the older adult to communicate with relatives or other community people for social interaction, participation, discussion, and outdoor activities; features for passive communication will take care of the patient’s physical health condition. We believe that the study findings will help to promote the psychological, physical, and social well-being of older adults.

2 Research Method In the present study, a quantitative approach has been adopted to collect data and analyze and interpret the perception of older adults on their social and psychological well-being. in this sense, a questionnaire has been developed and used for the online survey. it is possible to verify the flow of construction of questionnaires by online surveys based on general and specific objectives which are defined for the investigation. The flow of questions was adjusted to discourses found and adapted up to the responses of participants in this sense that it is possible to select the right option from the questionnaire and provide as much information as possible. With regards to collecting the data, a completely online data collection was planned. Therefore, the data collection and online survey were carried out between March 4th to March 29th of 2022. For the online survey, the first email was

214

I. khan and L. Oliveira

sent to senior Universities on 7th March 2022 and a reminder email was sent on 21st March 2022 to get more participants, as many as the study can. The reminder email worked perfectly, and the study got more participants which enhances its authenticity level. The study defines the following phases to execute the model validation methodology. 2.1 Target Population Commonly, the target population in qualitative research is key when the research population is significant. The study portrays the research objectives, assumptions, and backgrounds that can dictate the selected community’s material and concentration in the qualitative research (Asiamah et al. 2017). The study’s target population will be individuals, and older adults aged more than 50 years. In general, people who are retired from their jobs have physical, psychological, and social issues. Gradually, their social circle becomes smaller and smaller, impacting their psychological immune system and making them fragile within the community. Due to this reason, this study selected more than 50 years old or more as the target population. 2.2 Sample Thus, to define the sample of the proposed study, procedures related to the demographic composition of 411 senior Portuguese Universities were considered. These 398 universities are in eighteen different districts of mainland Portugal. The remaining 13 senior universities are situated on two different Portuguese islands, namely Azores and Madeira. A total of 179 participants have participated in the sample, 64.2% or 115, are female and 35.8% are 64 males. Regarding the age group who participated in the sample, 11.7% are aged between 55–59 years, 14.5% are aged between 60–64 years, 22.3% are aged between 65–69 years, 30.7% are 70–74 years, 16.8% are aged between 75–79 years, 3.4% are 80–84 years and 0.6% (one participant only) is aged between 85–89 years. Regarding the education level of participants in the sample, 19.0% have 1st cycle of basic education (4th class), 2.2% have 2nd cycle of basic education (6th year), 10.1% have a 3rd cycle of basic education (9th year), 22.3% have finished secondary education (12th year), 36. % have graduated with a bachelor’s degree, 6.7% have a master’s degree, and 2.8% have a doctorate. 2.3 Study Instruments Defensible inferences to be made based on the data, and research tools used (such as questionnaires, interview schedules and observation schedules) must be internally valid and reliable (Gray 2013). An instrument’s validity must be designed in a unique way to simplify the analysis of sample data. The primary tool used to collect data from older adults; is the IoT social hybrid scenario validation questionnaire. The study designed a questionnaire that contained 45 questions and five major parts. The first part of the questionnaire has 7 questions that require a declaration of informed consent, age, education, employment, household, and a list of smart devices that an older adult is using. The second part questionnaire has 14 questions designed to model social interaction

A Social IoT Hybrid Scenario

215

with family and friends. The third part of the questionnaire has 11 questions about the medical follow-up of the model. The fourth part contains 8 questions ‘Your opinion about the system’s advantages and disadvantages about model design. The fifth part of the questionnaire has 12 questions about ‘Your predisposition to use or not use a system of this nature model. The questionnaire was validated by three senior researchers from the University of Aveiro and five respondents from different parts of Portugal. This process led to the reformulation of some aspects, namely: changing the nomenclature of a level of the scale (from ‘No opinion formed’ to ‘Neither agree nor disagree,’ to be more in line with the lexicon of the remaining items). 2.4 Survey Procedure The presented study gets data through the survey, that participants answered online. To facilitate the understanding of the model, the participants were invited to watch an animated video before starting the answer (https://youtu.be/kLuvZJP-uck.) and prototype Figma App to report their feedback. The Animation video describes the main functions, characteristics, and scenarios of the proposed model in motion graphics format, but Figma prototype features allow a user to explore the different scenarios and interact with the proposed design. Figma prototypes are a fabulous way to preview interaction and user flows. Through, the Figma prototype people can understand, what the proposed model application looks like for example what colour, font size, and icon and what Figma application gives a response if they click on different icons and buttons. The Figma prototype also lets them know how they can interact with family and doctors and what kind of physical and social activities they can perform through the proposed model. For more information regarding the Figma prototype app please follow the link https://www.figma.com/proto/gcoX09Xuqn4AQKFrX5PwVe/Prototypingin-Figma?node-id=0%3A2&scaling=scale-down&page-id=0%3A1&starting-pointnode-id=0%3A2&show-proto-sidebar=1.

3 The Social Hybrid Scenario Model The model’s sketch contains these passive and intentional layers (Fig. 1 A layer passive, and B layer intentional) that depict the social hybrid scenario’s primary character and how combining passive and intentional layers offers an environment where older adults spend their healthy autonomous lives. If you look at Fig. 1, two layers are mentioned. On the left side is a layer, which represents the passive layer. The passive layers can predict, notify, suggest, make appointments, memories sharing, and live and real-time health data for its management and allow the patients to call their doctors and family members. The B side of the model offers an intentional communication service in which older adults can make invitations: invite people to outdoor and indoor activities, make audio/ video calls, show: multimedia comments, participate in city tours, hiking, and cultural activities: watch movies with friends and family members, play: outdoor and indoor games and log on to online classes and workshops. However, the systems offer health care and social communications platforms; systems must use Wi-Fi internet, cellular data internet, or IoT cloud service. Unlike family members and medical doctors, they

216

I. khan and L. Oliveira

need to install the system on their mobile phones and monitor older adults’ physical data. The system provides live data on older adults’ physical health, in which doctors can analyze the older adults’ physical data and suggest how to improve their physical well-being. Likewise, the older adults’ family members will take care of older adults’ daily activities, such as how many steps They walk per day and their geolocation, but in a critical situation, they will also receive the notification in the same manner and while doctors receive it in cases such as heart attack and fall detection. Intentional Layer Passive layer

Fig. 1. Diagram of Social Hybrid Scenario model

4 Results In the sample, 11,7% have 55–59 years, 14,5% have 60–64 years, 22,3% have 65– 69 years, 30,7% have 70–74 years, 16,8% have 75–79 years, 3,4% have 80–84 years and 0,6% (one element) has 85–89 years (Fig. 2).

Fig. 2. Frequencies: Gender

4.1 Part 1. Social Interaction with Family and Friends The agreement is high for all statements, higher for “8. I like the idea of using a system that allows me to contact, by various means (voice, video, messages…), my family and

A Social IoT Hybrid Scenario

217

friends” (91,3%), followed by “11. I like that the system allows the organization of groups to carry out activities (for example, inviting friends for a walk on Saturday afternoon).” (85,0%), then by “12. I like to create shared agendas to combine social activities (dinners, outings…) with my friends and family.” (76,7%), followed by “10. I am enthusiastic to participate in online group activities via videoconferencing (gymnastics, dance…), connecting the camera of my device (mobile phone, tablet, computer…)” (69,8%), then by “9. I agree with the connection of my social networks (Facebook…) to the system.” (62,7%), and lower for “13. I like the idea that, instead of watching TV and movies alone, I can do it with other people, even if each one is at home.” (51,2%) (Fig. 3).

Fig. 3. Frequencies: Part 1. Social interaction with family and friends (without missing answers)

In the sample, the mean for the agreement is higher for “8. I like the idea of using a system that allows me to contact, by various means (voice, video, messages…), my family and friends” (M = 4,47), followed by “11. I like that the system allows the organization of groups to carry out activities (for example, inviting friends for a walk on Saturday afternoon).” (M = 4,07) and “12. I like to create shared agendas to combine social activities (dinners, outings…) with my friends and family.” (M = 3,91), then by “10. I am enthusiastic to participate in online group activities via videoconferencing (gymnastics, dance…), connecting the camera of my device (mobile phone, tablet, computer…)” (M = 3,77) and “9. I agree with the connection of my social networks (Facebook…) to the system.” (M = 3,60), and finally by “13. I like the idea that, instead of watching TV and movies alone, I can do it with other people, even if each one is at home.” (M = 3,47), all items with mean value higher than the scale middle-point. 4.2 Part 2. Medical Follow-Up The agreement is very high for all statements, in the following order, “17. I find it useful to be able to consult the records of my health data (blood pressure, heart rate, oxygen level, etc.).” (90,7%), “23. I like the idea that, if I feel disoriented/lost, the system allows me to share my location with my family or friends.” (88,8%), “20. I find it useful for the system (through the wristband, mobile phone app, television, etc.) to alert me when taking medication or scheduling appointments.” (87,5%), “18. I appreciate the idea of

218

I. khan and L. Oliveira

using an alarm system bracelet for emergency situations (falls, sudden illness, etc.).” (86,4%), “22. It reassures me that the system alerts my doctor if there is a serious change in my health data (blood pressure, heart rate, oxygen level, etc.)” (85,3%), “24. I find it useful for the system to know which pharmacy is on duty in my location, allowing me to identify it on the map and call it directly.” (85,1%), “15. Would you wear a bracelet to measure your health data (blood pressure, heart rate, oxygen level, etc.)” (84,5%), “21. I like the possibility of being able to talk to my doctor and/or follow-up nursing team through the system (for example, by video call).” (83,7%), “16. Knowing that my doctor receives a report with data regarding my health is reassuring.” (83,1%) and “19. I am excited about the idea of using technology (a bracelet, a mobile phone application, television…) that encourages me to do physical activity.” (72,1%) (Fig. 4).

Fig. 4. Frequencies chart: Part 2. Medical follow-up (without missing answer)

4.3 Part 3. Your Opinion About the System’s Advantages and Disadvantages The agreement is very high for all statements, in the following order, “27. I am reassured by the fact that the system securely restricts access to my personal data.” (88,6%), “30.

A Social IoT Hybrid Scenario

219

I consider it essential to be able to disable some system features whenever you want.” (88,1%), “28. It reassures me to be able to customize privacy levels, defining who can see my data and shares.” (86,5%), “29. I like to be able to define how each friend and family member can communicate with me through the system.” (86,1%), “31. I am concerned that other people may unduly gain access to my personal data recorded in the system.” (84,7%), and then “26. I like that the system integrates a variety of features in a single application (interacting with family and friends, interacting with healthcare teams, organizing and participating in activities, etc.).” (71,8%), and “32. I am concerned that the system constantly monitors my social interactions and health data.” (71,0%) (Fig. 5).

Fig. 5. Part 3. Your opinion about the system’s advantages and disadvantages (without missing answer)

4.4 Part 4. Your Predisposition to use or not use a System of this Nature The agreement is relatively high for some statements, in the following order, “34. I would use the system because it would help improve my physical well-being.” (79,6%), “40. A more assiduous and direct interaction with my doctor generates a feeling of security and well-being and, therefore, I would use the system.” (78,9%), “35. I would use the system because it would help improve my psychological well-being.” (77,0%), “36. I would use the system because it would help improve my social well-being.” (71,9%), “38. I would use the system to have a more assiduous and direct interaction with my family and friends.” (68,3%) and “43. Being able to participate in activity groups (dance, gymnastics, cooking…) would improve my level of social interaction, promoting my well-being and, therefore, I would use the system.” (56,3%), but lower for other statements, in the following order, “39. I give privilege to face-to-face contacts and, therefore, I would not use the system.” (44,0%), “41. Through the system I could meet new people, expanding my social circle and, therefore, I would use the system.” (42,0%), “42. I already use other platforms to ensure social contacts with friends and family and I’m not willing to change, so I wouldn’t use the system.” (37,6%), “37. I do not feel comfortable centralizing data of a private nature in a digital system and, therefore, I

220

I. khan and L. Oliveira

would not use the system.” (33,9%) and “44. I’m not a skilled user of technology, so I wouldn’t use the system.” (28,3%) (Fig. 6).

Fig. 6. Part 4. Your predisposition to use or not use a system of this nature (without missing Answer)

A Social IoT Hybrid Scenario

221

5 Discussion and Conclusion The results discuss the benefit of a social hybrid model for older adults, particularly in improving their social and physical interactions and preventing psychological problems. The model provides a communication platform for older adults to connect with family members, friends, and doctors, and encourages participation in outdoor physical or social activities. The stakeholders agree that older adults have limited opportunities for social or physical activities, and there is a gap between existing knowledge and older adults’ well-being. Another discrepancy in the perspectives and needs of older adults in terms of communication, particularly passive and social communication. The social hybrid scenario model is proposed to provide a social environment that prevents social isolation and psychological health issues. The model encourages the use of digital media and faceto-face or online contact to enhance social participation with friends and family. The involvement of all stakeholders, including family members, friends, medical doctors, and older adults, is important for fostering literacy on older adults’ well-being and ensuring that passive and social communication activities are without any hurdles and distance problems. The use of popular communication platforms is recognized as pertinent in the context of social and psychological well-being (Nieman and Wentz 2019). The study offered a social hybrid model that provides opportunities for older adults to engage in social and physical activities through various means such as text messages, audio or video calls, and online gym or dance classes. Physical activity is important for maintaining physical and mental health, especially among older adults (Lee et al. 2022). The model also allows older adults to invite their friends and participate in outdoor activities such as walks or cultural, political, and religious events. Overall, the study shows that older adults enjoy participating in both online and outdoor activities for social and physical engagement. The model also includes an online health monitoring service that allows for the management of physical health data and quick response times. The study results reveal that older adults want a system in which medical practitioners can monitor their physical health data and communicate with them. The model also includes an online medical appointment and consultation feature that reduces time consumption and is costeffective. The study design provides an opportunity for older adults to make strong connections with their peers, friends, and doctors through passive and intentional communication for the sake of their well-being development. The scientific community points out that the model’s characteristics must be addressed to validate its impact level on different individuals (Strutt et al. 2022). Regarding he functionality of a model that has three different logs for different types of users (older adults, family friends, and doctors). The model includes features such as proximity, dynamizing and disseminating events, social and passive communication, privacy control options, alerts, and notifications, fall detection, real-time geolocation monitoring, and multiple passive and intentional scenarios to promote the physical and psychological well-being of older adults. The model aims to increase the participation and interaction of older adults and improve their well-being level. A study was conducted to understand the characteristics of a proposed social hybrid model for the well-being development of older adults. The data was collected through

222

I. khan and L. Oliveira

an online survey from senior universities located in Portugal. The study proposes three major changes in the initial proposed model, which include increasing font size, adding a location-sharing tool, and providing a privacy control tool for older adults to restrict access to their medical or social data without permission. These changes were made based on the feedback received from the survey participants, and the final proposed model aims to remove or decrease the isolation of older adults and increase their physical and social well-being levels (Chadwick et al. 2022).

6 Conclusion The study discusses the issue of social and physical well-being among the ageing population in Portugal and how the use of IoT-based technology can help to address this issue. The paper presents a social hybrid model based on IoT technology that can improve the physical and psychological well-being of older adults. The model includes features such as remote monitoring, real-time tracking of health conditions, and social interaction. The results of the study show that the model was well-received by older adults and has the potential to improve their social and physical participation and remove social isolation, leading to better well-being levels. The study investigates the role of IoT technology, passive, and intentional communication in promoting physical, psychological, and social well-being among elderly participants in Portuguese senior universities. The study aims to answer the research question by conducting a literature review, designing, and validating an IoT social hybrid model, and identifying scales to measure the well-being of the elderly. An IoT social hybrid scenario model that aims to improve the well-being of older adults. The model is based on passive and intentional communication and is designed to identify the physical and psychological health issues of older adults. The model’s characteristics are based on the functionalities of IoT-based digital services, and social and healthcare services must be explored in IoT-based services, as well as passive and social communication. The model provides passive and intentional communication opportunities to enhance the physical and physical well-being of older adults. The goal is to provide all the qualities that make up this model with a framework of connection and interconnectedness. The development of a hypothetical IoT social hybrid model gives clues that its characteristics can promote the well-being of older adults and it has the potential for real practical applicability because this model can integrate with new technology via Bluetooth, and it represents the real needs and demands of older adults. Future research on the scalability and applicability of the social hybrid scenario model approach to promoting well-being. It presents potential applications of the IoT social hybrid model in software engineering development and highlights the importance of developing a physical social hybrid model with working prototype software to promote the well-being of older adults. The film emphasizes the necessity of continuing to develop and validate functionalities of the IoT social hybrid model, as well as robust integration and convergence planning of systems and the dynamics of interaction and communication based on an IoT technology, passive, and intentional communication approach.

A Social IoT Hybrid Scenario

223

Acknowledgments. We would like to express our sincere gratitude to the members of WenzhouKean University, particularly Associate Dean Dr. Denise M. Horn, for their unwavering support. We are deeply thankful to the research department for providing the necessary resources and facilities to conduct this study.

References Asiamah N, Mensah HK, Oteng-Abayie EF (2017) General, target, and accessible population: demystifying the concepts for effective sampling. Qual Rep 22(6):1607–1621. https://doi.org/ 10.1002/chem.201102203 Chadwick Z, Edmondson A, McDonald S (2022) Engaging with animal-assisted interventions (AAIs): exploring the experiences of young people with ASD/ADHD diagnoses. Supp Learn 37(1):44–61. https://doi.org/10.1111/1467-9604.12394 Da Xu L (2022) Emerging enabling technologies for Industry 4.0 and beyond. Inf Syst Front (2021). https://doi.org/10.1007/s10796-021-10213-w Ellison NB, Boyd DM (2013) Sociality through social network sites. 1. https://doi.org/10.1093/ oxfordhb/9780199589074.013.0008 Gray ED (2013) Doing Research in the reak world, 3rd ed Kumar N (2018) IoT architecture and system design for healthcare systems. In: Proceedings of the 2017 international conference on smart technology for smart nation, SmartTechCon 2017, pp 1118–1123. https://doi.org/10.1109/SmartTechCon.2017.8358543 Kim K, Jasmine Q (2020) Social media communication and loneliness among OlderAdults. Clin Nutr 1–41 Na-songkhla J (2011) An effect of interactive media in a social awareness ubiquitous learning community. In: International conference on lifelong learning 2011 (ICLLL2011): transforming nations through enculturation of lifelong learning, pp 1–24. http://proxy.lib.utk.edu:90/login?. http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED536195&scope=site Nieman DC, Wentz LM (2019) The compelling link between physical activity and the body’s defense system. J Sport Health Sci 8(3):201–217. https://doi.org/10.1016/j.jshs.2018.09.009 Partridge L, Deelen J, Slagboom PE (2018) Facing up to the global challenges of ageing. Nature 561(7721):45–56. https://doi.org/10.1038/s41586-018-0457-8 Lee S, Ryu J, Heo J (2022) Leisure and social supports in relation to positive affect among OlderAdults. J Appl Gerontol 41(2):551–559. https://doi.org/10.1177/0733464821990485 Storrs C (2016) People who are and lonely are at a greater risk of the heart disease. https://edition. cnn.com/2016/04/20/health/can-loneliness-lead-to-heart-disease/index.html Sixsmith A, Horst BR, Simeonov D, Mihailidis A (2022) Older people’s use of digital technology during the COVID-19 pandemic. Bull Sci Technol Soc 42(1–2):19–24 Strutt PA et al (2022) Stress and coping in older Australians during COVID-19: health, service utilization, grandparenting, and technology use. Clin Gerontol 45:106–119. https://doi.org/10. 1080/07317115.2021.1884158 Tamrakar AK, Shukla A, Kalifullah AH, Reegu FA, Shukla K (2022) Extended review on internet of things (IoT) and its characterisation. Int J Health Sci 8490–8500. https://doi.org/10.53730/ ijhs.v6ns2.7177 World Aging Population (2017) In United Nations New York. https://doi.org/10.1049/el:20000788

The Expanded Book - Various Perspectives in Augmented Reality Andreia Fernandes1 and Luís Leite2(B) 1 Mestrado em Sistemas e Media Interativos, ESMAD/P.Porto, Vila do Conde, Portugal 2 uniMAD/ESMAD/P.Porto, Vila do Conde, Portugal

[email protected]

Abstract. This research aims to understand how Augmented Reality (AR) can enhance and enrich the traditional experience of reading an illustrated children’s book. In this context, a children’s book has been developed that explores the digital component of AR, along with the characteristics of a physical book, resulting in a synergy with the printed narrative. The narrative is presented in two layers, a printed one and a digital one, the latter reveals characters that are not seen in the page layout. Thus, after the interactive book prototype was created, tests were conducted with a sample of the target audience. These tests helped to understand that this combination resulted in a captivating experience for the user who demonstrated a positive reaction to the book and the activity. The project revealed that it can bring the user and reading closer together through the use of technology, which has proven to be a relevant and engaging factor among young audiences. These tests also revealed that participants did not have difficulty handling both dimensions, the book and the mobile device, simultaneously. Similarly, the test also focused on information retention about the narrative to assess the ability of AR to improve the understanding of illustrated story reading. Keywords: Augmented Reality · Narrative · Interactivity · Interaction Design · Reading

1 Introduction With the beginning of the digital era and its evolution over time, new forms of communication and new digital experiences for users have emerged. In this context, through the technology industry’s major companies, the component of AR emerged in the digital market, presenting the concept of visualizing digital elements on the real world. It is a phenomenon that provides a digital experience without completely abstracting the user from the real world, bringing together the physical and digital worlds in a new dimension. Books are objects that have been present in our society for thousands of years and are the most important means by which we humans tell stories and record historical events. However, in today’s digital culture, it is possible to combine these objects rooted in society with a new form of reading. The goal is to enrich these conventional printed media with new forms of reading through digital means, taking advantage of the dynamics and interactivity that these means offer. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 224–237, 2024. https://doi.org/10.1007/978-3-031-47281-7_18

The Expanded Book - Various Perspectives in Augmented Reality

225

The intention to study the expanded narrative problem through AR in books emerged, and in what ways this combination of elements can provide a stimulating and interesting experience for users in the reading of illustrated stories. Therefore, in the project component, it was necessary to explore the technologies to be used, as well as the interaction design between the physical book and AR. It was necessary to consider some questions that should be answered throughout the process, such as “How can AR provide greater interactivity in the physical book?” and “How can a mobile and three-dimensional book in conjunction with AR enhance the reading experience?” Therefore, a children’s book was developed in traditional format, with illustrated elements in the page layout and, in a second layer, as a second reading, the characters of the fictional and imaginary world will be revealed through AR. To complement this interaction, the paper sheets will be manipulated and folded in a way that provides greater interactivity between the book and the user.

2 Methodology For the conception of this project, exploratory research was carried out which allowed acquiring knowledge about the theme and developing its concept as a visual component. According to Maxwell (2011), “exploratory studies are generally useful for diagnosing situations, exploring alternatives, or discovering new ideas”. Thus, through artistic references, dissertations, case studies, and websites, it was possible to provide this project with a more consistent concept. Subsequently, it was necessary to understand the historical context of the first three-dimensional books, as well as their evolution up to the digital age. After this exploratory methodology, the narrative and its purpose were defined, as well as who the book is intended for and how the narrative is important for the application of the AR component. Therefore, it was defined that it would be directed to the children’s audience with the theme of the imaginary world, for an age range between 7 and 12 years old. This narrative will have two perspectives, one with elements printed in the book and the others referring to the imaginary world, will be revealed through AR. After defining these ideas, it was equally important to understand the objective of this research and what change it would provide in illustrated books. Thus, following the practice-based research methodology, as defined by Barab and Squire (2004), “A series of research procedures applied to the development of theories, artifacts, and pedagogical practices that are of potential application and usefulness in existing teaching-learning processes”, it was essential to understand whether this new way of exposing the narrative has considerations in information retention or not. Therefore, the study focuses on the participants’ relationship with this new paradigm that the artistic work presents. In an experimental phase, the project component was initiated with the study of interaction design in a physical book. As this was the most important and decisive point of the project, the experimental component was crucial for its evolution, mainly because it depends on the results obtained from the experiments, according to Gil (2008), “submitting study objects to the influence of certain variables, under controlled and known conditions by the researcher, to observe the results that the variable produces in the object”. In the book prototype, experiments were carried out to study how reading can be enhanced and to identify its limitations. From this procedure, the visual component of the

226

A. Fernandes and L. Leite

book was initiated, which helped in creating the paper interactions (folding mechanisms) present throughout the book. The visual aspect, together with the interactive component of the physical book, plays a fundamental role in the user experience when browsing the book. Therefore, it was necessary to dedicate time to experimentation to understand what type of interaction would be created. After this analysis, with the study of the characters and how they should act, the illustration component was initiated, which would determine how they would appeal to the user for AR. This led to the idea of presenting them in the physical book in the form of a silhouette. Next, the animation component was initiated, which has the role of surprising the user with the characters in motion, demonstrating the action told in the narrative.

3 State of the Art 3.1 Expanded Books in Augmented Reality Books are objects that are very present in our society, they are a means of passing on knowledge, educating and entertaining (Altinpulluk & Kesim, 2016). On the other hand, the growth of technology has transformed the way we live, however, the book has remained a traditional object for centuries, but now it has also been adapting to this change. Electronic books or eBooks have emerged in an attempt to simulate the traditional paper reading experience on screen, with the intention of replacing this conventional object with technology. Despite the intention, this new way of reading has not convinced readers, who continue to prefer the physical book because it influences reading through its physicality (Grasset et al., 2008). On the other hand, augmented reality makes it possible to combine the tangible and physical characteristic of the book with the digital elements, multimedia and multisensory stimuli that have enhanced this object - the book (Altinpulluk & Kesim, 2016). In children’s literature, it’s common for children to imagine themselves as part of the story, which is why a project called MagicBook1 emerged in 2001 with the intention of making this desire real. Through a hand held display, it was possible to see 3D animations on the pages of the physical book, which were seen from the user’s point of view. The user “enters” the story through an avatar that moves around the scenes through the eyes of the user in both AR and Virtual Reality (VR). VR users circulating in the virtual environment of the narrative can see AR users looking down on their respective avatars and thus collaboratively explore the environment, as shown in Fig. 1.

Fig. 1. MagicBook from the perspective of AR and VR.

The Expanded Book - Various Perspectives in Augmented Reality

227

It was in children’s books that AR had the greatest impact, since they have a very strong visual component, it was easier to adapt the stories to the animations and illustrations. It was through Disney’s major productions that the publisher Carlton Books Kids launched the first AR books for the films Toy Story (2019) Fig. 2 - Left, Ice Age (2012) and Princesses (2012), opening doors to this new experience. Also of a more educational nature, this publisher has launched books with themes about the universe, namely the planets and the solar system, as well as dinosaur exploration, as shown in Fig. 2- Right (A. Pina, 2015).

Fig. 2. (Left) “Toy Story” AR Book; (Right) “iSolar System” AR Book.

As such, AR has demonstrated its importance in the educational field as a new way of teaching. In a more dynamic and didactic way, AR has shown that it has a greater capacity to capture the user’s attention because it offers a set of characteristics that are different from other technologies (Santos et al., 2014). Such as the implementation of real-world annotations and visual and haptic content that help the user retain information better (Santos et al., 2014). This is how Blippar came about, an AR project that created an AR component in educational books, called Blippbuilder2. This tool was created during the Covid-19 pandemic, with the aim of helping teachers, students, and parents with informative content on various subjects during the lockdown period. On the pages of the book, 3D digital elements appear with visual explanations of the content, as well as the transformation of images into interactive elements, through quizzes and treasure hunts that promote the reader’s involvement in the physical world, as shown in Fig. 3.

Fig. 3. Example of a book in AR from Blippbuilder.

The application of AR in books has been expanding into various areas, with innovative ways of being applied, with increasingly daring and interesting concepts. This is the case of designer Camille Scherrer, an artist who works between art and technology, developing projects in the fields of video mapping, interaction design, motion design and AR. In this way, she developed the AR component of the 2009 book “Des quatre”3

228

A. Fernandes and L. Leite

by Mathias Vayer for the luxury brand Louis Vuitton. The book features elements from nature, such as flowers and animals, which fly over the book and often exceed the book itself, invading the real environment in which it is located. There are also other elements, such as small circles or squares that detach from the text or printed figures, which, when animated in AR, move out of the book, as can be seen in Fig. 4. Other visual elements with the presence of the brand’s logo invade the entire book, as well as the surrounding environment, creating the illusion of clouds in the sky that cast a moving shadow across the physical space. All these interactions enrich the book with visual content that makes the user want to explore and observe it as an artistic piece.

Fig. 4. AR book by Camille Scherrer for Louis Vuitton.

Through this research, it is possible to understand that the application of AR in books is vast and has the potential to be present in various areas and with different themes. We can apply it to books with stories for children, education, and the arts. 3.2 Case Studies “Mon Papi”. During the process of exploring the subject, several cases of AR being applied to physical books were found. Some with a greater degree of interactivity than others, but all with something in common: the desire to take the conventional object, the book, to another level of reading and interaction with the reader. Since this is the purpose of the project, two case studies were taken into consideration that most closely relate to this investigation. Manon Louart is a French designer and illustrator who graduated in 2017 with a degree in Graphic Design from UQAM (Montréal). Her work focuses mainly on Motion Graphics and illustration, with the theme of nature and meditation always present in her creations. For her final course project, she developed an illustrated AR book entitled “Mon Papi”1, which depicts the Alzheimer’s disease diagnosed in her grandfather. The theme of the book explores the adventures experienced by the grandfather and granddaughter, presenting scenarios such as going to the beach, hiking in the mountains, playing in the snow and nights spent watching the stars in the sky, as shown in Fig. 5. Throughout the book, the illustrations are very prominent due to their size, which takes up the entire page, and also due to the lack of masses of text. As a result, the book is visually attractive with a pastel color palette, with shades of blue and pink that make the book more aesthetically interesting and enhance the landscapes and scenery illustrated.

The Expanded Book - Various Perspectives in Augmented Reality

229

The book has two reading levels: in the first, you can see the incomplete illustrations printed in the book, which show your grandfather in different settings, alone. On a second reading, through AR, you can see the same illustrations, but more detailed, with the addition of floral elements, animals, different textures, as well as the female figure representing the author herself. With this, the author’s aim is to remind her grandfather of the moments they spent together and how together they made the experiences richer and more beautiful. In this way, as a complement and a fundamental element in understanding the narrative, AR enhances interactivity as well as the emotional side of the project. AR is used through a mobile device, which allows the user to explore more of the page and look for surprise elements in the illustration itself. However, although interactivity through AR exists, the book does not explore physical interactivity through the pages of the book, which limits the exploration of the artifact as well as the digital interactivity aspect. The lack of this complement allowed us to understand what could be implemented in the project presented in this article, in order to enrich and improve the reading experience in picture books. Despite being a case study that influenced and motivated the creation of this project, it is possible to consider that it had shortcomings in terms of interactivity with the pages of the book and therefore consider it incomplete.

Fig. 5. Mon Papi, 2017.

“Jekyll and Hyde”. In order to explore the possibilities of AR in printed media, (A. Pina, 2016) the augmented book “Jekyll and Hyde”2 was created by two Swedish students as their final course work in Arts and Visual Communication at the FHNW HGK in Basel, Switzerland. The narrative presents the horror classic “The Strange Case of Dr. Jekyll and Mr. Hyde” written in 1886 by Robert Louis Stevenson, a book known for its disturbing and mysterious plot, which tells the story of a lawyer investigating the strange relationship between his good friend Dr. Jekyll and the dubious Mr. Hyde. The lawyer tries to understand the reason for his friend’s dubious behavior and eventually discovers that Dr. Jekyll and Mr. Hyde are the same person. An experiment in the laboratory didn’t go as expected and their personalities ended up split, creating the lovable Dr. Jekyll and the fearsome Mr. Hyde (Lebeau, 2018). It therefore gained a more digital and interactive version with the application of AR, which enhanced the darkness and tension of its narrative, as can be seen in Fig. 6. When you open the book, shadows fly across the pages, elements move and confuse the reader, such as candles being lit, fog and eyes suddenly moving. As well as typographic games that appear with messages and photographs that, when put together through the

230

A. Fernandes and L. Leite

movement of the pages, result in the appearance of a video (A. Pina, 2016). These are the elements that increase immersion, suspense and tension when reading, as well as making the narrative more “alive” and convincing for those who read it. It is a book that can be read without the use of AR because it contains all the text printed in the book, but it is through AR that the atmosphere of darkness is enhanced, resulting in a more intense reading. In each photograph, illustration or even text, you can see movement and interactivity, such as folded pages that hide animated images and others that are cut out like a puzzle to form elements that interact together. In this experimental project, various unusual ways of combining a physical element, the book, and the digital component were produced. Through visual and interactive elements that enhance the user’s experience with the technology and the narrative itself (A. Pina, 2016). This is an example of exploring the two realities, with different approaches to manipulating the pages with AR.

Fig. 6. AR book “Jekyll and Hyde”.

4

Project: The book “My Monster Friends”

4.1 Narrative The stories in children’s books have evolved over the years. Initially, the narrative had a moralistic and educational character with the intention of showing children the difference between good and evil. This way, the idea was conveyed that children were punished for their disobedience or bad behavior (Rubin, 2005). Classic narratives such as “Little Red Riding Hood” presented this type of “educational” narrative, where the girl who did not obey her mother’s orders is subsequently punished with the appearance of the evil character, the Big Bad Wolf. Today, the stories in children’s books play an essential role in the development of a child’s intellect. They encourage imagination and the creation of ideas that help in their approach to life and human relationships (da Cruz, 2021). Currently, these narratives are geared towards learning and the development of children, but they also have a strong entertainment aspect, with fun and creative stories. In addition, it is increasingly common for children’s books to use other modalities such as music, illustration, and multimedia to enhance the relationship between the story and the child (da Cruz, 2021). For the interactive book project “My Monster Friends”, the narrative was a fundamental element that determined how the AR component would be applied. The description and actions of the characters would dictate which interactions to develop and how they would attract the child audience. Therefore, the story presents the character Olivia, a

The Expanded Book - Various Perspectives in Augmented Reality

231

child who interacts with characters from the imaginary world that have fun personalities, as well as peculiar habits and obsessions. The story describes Olivia’s daily life from the moment she wakes up until she reaches the kitchen for breakfast. During this journey, she encounters these characters scattered throughout the house, and in these encounters, the characters interact and talk, revealing their personalities, habits, and likes. 4.2 Visual Component Children are most curious and eager to interact with everything around them during childhood. Therefore, reading is often a motivating factor and a driving force in a person’s creative and cognitive development (Massoni, 2018). In children’s books, visual content is a tool with the ability to communicate without text, as it turns words into images while still conveying a message. Sometimes, these images best translate an idea, a feeling, or even an action, and it is because it is such an effective element that it is frequently used. When we talk about illustration for children’s books, its language must be unique and attractive enough to capture the attention of this audience who easily lose interest in what they are seeing. Thus, the illustrations developed for this project took this factor into consideration, with a colorful design for the characters, with rounded shapes and elements that made them more engaging (Fig. 7). It was important that the visual was balanced and that all illustrations were conceptually interconnected.

Fig. 7. Characters from the book “My Monster Friends”.

Another important element was the pattern created with symbols that were assigned to each of the characters. These had the function of defining the personality of the character or the place where they were, as seen in Fig. 8 (left), the character Alberto Stini, an intelligent monster who devours books, has a pattern of letters and numbers throughout the page layout. In the figure on the right, the “Sem-Lâmpada” monster stays in the child’s room, so elements such as toys, books, and photographs are used. These created patterns, besides being elements that aesthetically influence the page, also fill the page layout to highlight the character and to contextualize and situate the user. In addition to the main illustrations that presented the characters and the narrative action, other illustrations were created with the aim of incorporating the text. Due to their extensive size, text blocks would gain too much prominence in the page layout and detract from the illustrations and interactions. Therefore, it was decided that the text would have its own page layout, and to fill it with some visual content, illustrations were produced with the representation of the various rooms of the house where the narrative takes place, (Fig. 9). For this project, several interactions were created in the book with the aim of exploring the artifact by the user. To achieve this, the book as an object was explored to

232

A. Fernandes and L. Leite

Fig. 8. (Left) Alberto Stini character; (Right) Sem-Lâmpada character.

Fig. 9. Illustrations of the rooms in the house where the narrative takes place.

understand its potential, along with the interaction design conceived for the tangible component of the physical book and for the virtual component applied to AR. Through the narrative, which helped to define a path, it was possible to create a more concrete design of the interactions. The project’s basic concept was based on the idea of having silhouettes that hid information and that with AR, this information would be revealed. Therefore, this concept was applied throughout the book, but over time it was perceived that the idea of physical exploration of the book by the user was not enough. Therefore, mechanisms were created with the book’s own pages through folds and loose pieces of paper, which helped to make the book more interactive. On the first page of the book, with the introduction of the narrative and the first monster, the “Sem-Lâmpada”, a harmonica-shaped mechanism was created to represent the storybook that the monster holds. When pulled horizontally, it is possible to observe a silhouette that is not very revealing, but with the mobile device, a character walking through the city to the sound of car horns and people’s movement is revealed. On the same page, we can also see the animated “Sem-Lâmpada” monster, shaking its legs while reading the story, as seen in Fig. 10.

Fig. 10. Page layout for the character Sem-Lâmpada.

The Expanded Book - Various Perspectives in Augmented Reality

233

On the next page, the second monster is revealed, a frenzied sock collector who stands up and faces the user with the message that they cannot touch her socks. Two actions were defined for the user. The physical component suggests opening two flaps placed in the page layout, which when opened, reveal drawers with organized socks. On the other hand, in the digital component, the user by pointing the mobile phone to the silhouette, which functions as a marker, reveals the character, as shown in Fig. 11 (Left).

Fig. 11. (Left) Page layout for the character Meia Leca; (Right) Page layout for the characters Pavi and Motti.

In the interaction design for the next page, the protagonists are the characters Pavi and Motti, and the layout shows in the top right-hand corner instructions on how the user should proceed with the interaction. Two tabs have been introduced into the layout, placed over the eyes of each character, with two variations, one tab with the eyes closed and the other with the eyes open. Each of these tabs presents a different interaction, with the mobile device pointing to the closed eyes tab, we can see the characters singing, and with the open eyes we can see the characters dialoguing, as shown in Fig. 11 (Right).

Fig. 12. (Left) Page layout for the characters Alberto Stini and Papa Letras; (Right) Page layout for the character Lavinci.

As we turn the page, we are introduced to the next characters as well as new interactions. We are introduced to the character Alberto Stini hidden in silhouette on top of the illustrated books, as well as the second character, Papa Letras kept in a bag. In this bag, the user is told: “Assemble the puzzle and pass the phone”, inside we can see the character segmented into several pieces, which once assembled, reveal the Papa Letras. This character acts as a marker for AR, coming to life (movement) via the mobile device, as shown in Fig. 12 (Left). On the next page, we are introduced to the last character in the narrative, Lavinci. A painting monster who, with his magic paint, can turn anyone into an animal. Lavinci’s personality led to the definition of three different actions, with indications to help the user proceed. So, in the first interaction, we have the silhouette that reveals the Lavinci character painting the face of the main character, Olivia. In the second, a paper circle with two sides is used, one with the figure of a cat and the other with the figure of a dog,

234

A. Fernandes and L. Leite

which, when placed over Olivia’s face, can be seen transforming into the animal chosen by the user. The second interaction is due to the transformation into the dog and the third into the cat, as shown in Fig. 12 (Right).

5

Tests and Results

The primary objective of this project was to evaluate the user experience with the book and AR. However, as the project developed, other questions arose that became more pertinent, namely the complexity or difficulty of manipulating both dimensions (physical book/mobile device) simultaneously by the child. It was also important to understand whether the information provided by the narrative was retained or not, or whether the child easily became distracted and focused only on the digital interaction. Thus, an experience combining the book and AR prototype was designed, carried out by children aged 8–9, aimed at testing the degree of difficulty in handling the mobile device with the book, as well as in understanding the narrative. The experience was conducted with 20 participants from the Matosinhos Primary School, a representative sample of the target audience defined. The participants were 3rd grade students, aged between 8 and 9 with an average age of 9, of which 12 were female and 8 were male. The experience took place on June 22th and 24th, at the library in the school (controlled without disturbances). Two groups were organized, a control group and an experimental group. The experimental group had contact with both dimensions (book/AR), while the control group had access to only one dimension (book). The groups were randomly defined. The experience was carried out individually, without the other participants having contact, thus avoiding contamination. The experience started with a brief explanation of the procedures, but without explaining the narrative or how to interact. The smartphone used was the same in all sessions (Samsung Galaxy S8), as well as the book. The idea of using a tablet or iPad during the experience was excluded due to its size and not being easily adapted to a child’s hands, with the mobile device being the most suitable. Each experience lasted an average of 15 min. After the experience, the participants were subjected to a questionnaire with questions about the narrative, as well as general and personal questions. Questions such as “Do you like to read?” and “Do you have a mobile phone?” were asked to understand if the children were familiar with the project components. Then the reading of the narrative began, read aloud, with the participant following the text in the book.

Fig 13 - Some participants during the activity.

Specific questions were asked to the experimental group (book/AR) in order to understand whether the use of the mobile phone was or was not an inconvenience in the

The Expanded Book - Various Perspectives in Augmented Reality

235

reading process. Therefore, the following questions were asked, “Did you like seeing the book with the mobile phone?” and all 10 participants answered unanimously with “Yes”, Fig. 14. In the second question, “Was it difficult to see and manipulate the book while holding the mobile phone?”, 70% of the participants revealed that they did not have difficulty manipulating the book while holding the mobile phone, highlighting the 30% who defined it as “More or less”. It was observed that none answered “Yes”, Fig. 14. This leads us to conclude that for most of the participants, the use of the mobile phone and the manipulation of the interactive pages was not problematic and that both actions could be executed perfectly.

Fig. 14. Chart with participants’ responses.

Through observing the individual activity of each participant, it was possible to notice that the experimental group showed great enthusiasm in each interaction. They expressed admiration and satisfaction for the emergence of animations and the interactions of the physical book. This passion was revealed through smiles, pleasant words, and the desire to go to the next page and discover more characters. On the other hand, the control group, without the RA, showed apathy towards the activity. Three of the participants even revealed signs of disinterest, getting distracted by external elements. These behaviors were not observed in the experimental group.

6

Final Considerations

Throughout the centuries, the book has developed as an artifact used for the transmission of stories and knowledge. Despite being an ancient object that has accompanied much of human history, few changes have been made to it. New attempts to transform the book into something more interactive have emerged, such as pop-up books that surprised readers with vertically positioned pages revealing scenes or characters from the narratives. This physical interaction allowed the user to interact with the story and feel more engaged with the action being told. However, the advancement of technology has provided digital ways to share the same content, as well as improving the experience of sharing it. Throughout this document, we can see several cases of the combination of AR technology with books in various areas that enhance the transmission of knowledge, as well as the concept of a book as an interactive experience. In the development of the book “My Monster Friends,” we thought about how AR could amplify and enrich the traditional reading experience of a book. Thus, an interactive book was developed with silhouettes and other markers that, when triggered by a mobile

236

A. Fernandes and L. Leite

phone, revealed content, expanding the narrative. Through the narrative, it was possible to describe and carry out the implementation of AR through Unity3D software, which allowed for the creation of the digital layer. The exploration and understanding of the software slowly developed, which resulted in a not very significant initial development with the emergence of doubts, as well as compatibility issues with the software. The need to use a mobile device compatible with the software also created constraints in finding an Android device that met the program’s requirements. In addition to these fewer positive aspects, it was possible to understand that there are various possibilities in applying AR. In the digital layers, it is possible to apply various contents, such as visual and sound elements, 2D and 3D objects, as well as multimedia content (e.g., animations, videos, etc.). Tests were conducted with a sample of the target audience, which showed that participants did not find it difficult to manipulate the two dimensions (book/AR) simultaneously. Similarly, a test was conducted to assess the ability of AR to improve understanding and retention of information about the narrative. The results were surprising, as the understanding of the narrative was lower in the experimental group that used AR than in the control group without AR. This may be because participants become involved in the story in a different way, from a different perspective, breaking the linearity of the narrative. In other words, perhaps because they handle the mobile device and visualize moving visual content, which results in greater distraction from the narrative. On the other hand, it was possible to observe during the tests that expanded reading with AR brings the participant closer, engaging and captivating them. To reduce reader distraction and increase the levels of comprehension of the story, it may be necessary to establish a balance between the elements. For this project, it was never the intention to replace the book with technology, but rather to subject this object to a better reading and entertainment experience. By understanding that we live in an increasingly technology-driven society and therefore more distant from traditional book reading, it is possible to see that combining these two components brings users and reading closer together. As was observed in the children who participated in the tests of this project, many of whom claimed not to like reading, showed interest and enthusiasm for the activity. Reflecting on the creation of other similar projects to this one, it is possible to predict growth in interest and enjoyment in literature, especially aimed at children. Although it is not possible to generalize the results obtained due to the limited sample size, very important indicators are presented for future work with AR.

References 1. Altinpulluk H, Kesim M (2016) The classification of augmented reality books: A literature review. 4110–4118 2. Barab S, Squire K (2004) Design-based research: putting a stake in the ground. J Learn Sci 13(1):1–14. https://doi.org/10.1207/s15327809jls1301_1 3. da Cruz DM (2021) As histórias infantis e o desenvolvimento das crianças. Rev. Primeira Evolução 1(21):23–31 4. Ferreira de Oliveira M (2011) Metodologia científica: Um manual para a realização de pesquisas em Administração. UFG; Universidade Federal de Goiás, Catalão

The Expanded Book - Various Perspectives in Augmented Reality

237

5. Gil AC (2008) Métodos e técnicas de pesquisa social (6th ed.). Atlas 6. Grasset R, Dunser A, Billinghurst M (2008) The design of a mixed-reality book: Is it still a real book? 99–102 7. Lebeau V (2018) The strange case of Dr. Jekyll and Mr. Hyde. In Encyclopedia Britannica. https://www.britannica.com/topic/The-Strange-Case-of-Dr-Jekyll-and-Mr-Hyde 8. Massoni LFH (2018) Ilustrações em livros infantis: Alguns apontamentos. DAPesquisa 7(9):121–129 9. Pina A (2015) Narrativas aumentadas [Monografia (Graduação em Produção Editorial)]. Escola de Comunicação, UFRJ 10. Pina A (2016) Jekyll e Hyde, de Martin Kovacovsky e Marius Hügli. Medium. https://med ium.com/historias-aumentadas/jekyll-and-hyde-de-martin-kovacovsky-e-marius-h%C3% BCgli-2a629b1d92a 11. Rubin EGK (2005) Pop-up and Movable Books In the Context of History. The Pop-Up Lady. The Pop-up Lady. https://popuplady.com/about-pop-ups/pop-up-and-movable-books-in-thecontext-of-history 12. Santos MEC, Chen A, Taketomi T, Yamamoto G, Miyazaki J, Kato H (2014) Augmented reality learning experiences: survey of prototype design and evaluation. IEEE Trans Learn Technol 7(1):38–56. https://doi.org/10.1109/TLT.2013.37

Society, Communication and Design

Design and Development of a Pill Dispenser: Prototype Assembly, Electronic and Mechanical Hardware Óscar T. Viana1(B)

, Octávio Lima1

, Miguel Terroso2

, and João L.Vilaça1

1 2Ai – School of Technology, IPCA, Barcelos, Portugal

{osviana,olima,jvilaca}@ipca.pt

2 Media and Culture, School of Design, ID+ - Research Institute in Design, IPCA, Barcelos,

Portugal [email protected]

Abstract. The gradual ageing of the global population has introduced new challenges to healthcare, creating the opportunity to develop new products that might aid the elderly in independently managing their daily health necessities. In this research project, the proposal is the development of a device that combines an automatic pill dispenser with the smart capabilities of modern devices, such as a virtual assistant, remote management capabilities, video calls, among others. This article pertains to the third and final phase of early development of this product, focusing on the CAD modelling, 3D printing, assembly and configuration of electronic and mechanical hardware, prototype assembly, as well as preliminary tests of the pill dispensing mechanism. From this process we were able to identify some shortcomings of the design, a credit to the benefits of rapid prototyping, and the possible corrections to be implemented in future iterations. Keywords: Pill dispenser · Rapid prototyping · Electronic and mechanical hardware

1 Introduction This paper regards the third phase in a research project that aims to mitigate the difficulties of elderly people in independently maintaining strict and complex medication regimes. The proposed solution is the development of a product that combines an automatic pill dispenser and smart features that would assist the users in improving medication regime adherence and communication with caregivers and healthcare professionals, such as nurses and doctors. Previous phases in the development process concerned the acquisition of knowledge regarding the limitations the elderly population might face when attempting to manage their daily routine and personal health and how to address these difficulties. Factors such as reduced physical strength and agility, vision and hearing deterioration, and especially © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 241–250, 2024. https://doi.org/10.1007/978-3-031-47281-7_19

242

Ó. T. Viana et al.

cognitive and memory degradation were all signalled as possible issues when using the device and considered during the design process. [2, 3, 5, 6, 9, 10, 12]. Additionally, research was made into the current state of ambient assisted living and available devices addressing the same concerns, focusing on pill dispensers and virtual assistants.[8] Using that knowledge, we were able to define the user experience based on the sequential model by [1] through which it was concluded that there would be three types of user experiences: in a home environment, where it will take advantage of all the features (automatic dispensing, virtual assistance, and the home service that replaces the cartridges); in professional temporary environments such as day-care centres and hospitals; and in long-term care facilities such as nursing homes where the device would be mainly used for automatic dispensing of pills, to facilitate labour. Based on these user experiences, the business model was built, though a canvas, structured by [11], which divides business models in nine building blocks: customer segments, value propositions, channels, customer relationships, revenue streams, key resources, key activities, key partnerships, and cost structure. From this, the needs the device should respond to were extracted, grouped, and converted into requirements, which were then specified and classified in terms of cost-benefit and degree of importance.[7] Subsequently, the necessary components and architecture, so that the product could respond to the projected needs, were defined. In total, 10 components were selected, namely: a 10.1inch touch screen, a 12-V 3-amp power supply, a 6000 mAh battery with a voltage of 12.6, a BMS (Battery Management System) for power control, a 5-V buck converter, a Raspberry Pi 4 model B, a 20-W sound amplifier, an 8-megapixel camera with a 120º field of view, two 57 mm 20-W speakers, and lastly, a board with 6 long range microphones. For the final part of the project the focus is on refining the design of the product, the arrangement of internal and external components., creating a CAD model, 3D printing and assembling the prototype with all electronic components, to allow for the execution of some preliminary functionality tests (Fig. 1).

Fig. 1. Sequential phases in the project development, this article refers to the third phase.

Design and Development of a Pill Dispenser

243

2 CAD Modelling To begin developing the geometry of the device, cardboard models were built of the previously defined components which were then arranged in various configurations to identify the possible shapes and dimensions of the frame. After some preliminary sketches, a paper model of the frame was built to help define general dimensions of the product and components when assembled (Fig. 2). This stage was also essential to determine ergonomic factors such as the position and inclination of the touchscreen, and the dimensions and placement of the handle. During this stage it was also specified how the pill dispensing mechanism would function. A circular cartridge divided into segments, with each segment containing a dose of medicine, is inserted into the back of the device and at indicated times a motor turns the cartridge, and the medication is transported through a canal connecting the top segment of the cartridge to a cup located at the front of the device, from where the patient will then take the medication.

Fig. 2. Cardboard model of early product geometry and preliminary sketches.

Subsequently, based on this model, digital modelling was started (Fig. 3). For this stage SolidWorks was the selected platform, being the most familiar to the authors while also standing as one of the most widely used tools for product engineering. From inception, the product was designed with 3D printing in mind as the main production method, necessitating that from the earliest stages of design, concerns of print orientation, support requirements, print plate dimensions and material selection were considered when designing the product geometry. Modelling started with the general volume from which the main cavities such as the cartridge support and the cup space. Afterwards were added other features such as the cartridge mechanism, lids, speaker covers, the handle, and the rotating base. The

244

Ó. T. Viana et al.

shell function was then used to make the volume hollow, with a wall thickness of 2 mm, enough to ensure the stability of the product, while retaining a low material demand.

Fig. 3. 3D Render in ¾ front and rear views.

Fig. 4. 3D Render with view of electronic and mechanical hardware.

To accurately represent the electrical circuit of the device, all hardware components were modelled in full detail, displaying their relative positioning and all the electrical and data connections to be made between them, shown in Fig. 4. For enabling the fixing of these components to the plastic frame, 3mm round cavities with a raised edge were created in the model, where, after printing, a metallic threaded insert could be fused, making it possible to use screws to fix hardware components in place. These same inserts were also used in places to assemble together the various parts of the frame.

Design and Development of a Pill Dispenser

245

Meanwhile, Fig. 5 shows an exploded view of all the components in the device, displaying where the hardware arrangement is to be placed among the plastic frame parts, and how these are fixed together by screws in specified points.

Fig. 5. 3D Render of exploded view with all components represented.

3 3D Printing and Prototype Assembly 3.1 Frame 3D Printing The main challenge when preparing to print the frame of the device was the limited plate dimension of the available printers, which could not accommodate most of the frame parts. To overcome this, in SolidWorks, the parts were split evenly until of adequate dimensions, and additional fitting geometry was added to guarantee the parts could be solidly and accurately reassembled. The segmented parts were exported in STL format and individually processed in the Prusa Slicer software. Here, the model is split into layers and support material was added where necessary to retain geometry while printing. After adjusting print speed and infill settings, the printing files were generated and saved to the printer’s memory. Figure 6 displays an example of the processing to which the parts were submitted, where “1” indicates the device rear, “2” the back plane, subdivided to fit the print plate dimensions, “3” shows one of the resulting subdivided parts.“4” and “5” show the part in the Prusa Slicer software before and after processing, showing the placement of support material, characteristics and expected duration of the different fills, and the print head path.

246

Ó. T. Viana et al.

Although printing of all parts required approximately 153 h, using 3D additive manufacturing as a method of rapid prototyping still proves an efficient solution, with highly reduced costs, accessibility, and versatility.[4].

Fig. 6. Software processing for 3D printing.

3.2 Prototype Assembly When printing of all parts was completed, the support material was removed, and surfaces were sanded to achieve a smoother finish. Subdivided parts were glued together along the fixation lips, to match the combined state shown in Fig. 5. In some areas of increased stress, additional fixation geometry was added, such as in the cartridge cavity, where four subdivided parts were combined. The next step was placing the threaded inserts. These were positioned, and a soldering iron was used to heat and push them into position, softening the surrounding plastic, which when cooled encased the insert firmly in place. All hardware components were then arranged and screwed in position. When beginning work on the electrical circuitry the 3D model was used as reference to create the graphic in Fig. 7, as a more readable guide to making the electrical connections. In the left diagram is shown in blue the AUX cable, in green the USB cables powering the screen and in purple the screen’s HDMI cable. In the central diagram red and black represent the positive and negative DC power distributors, and the multi-coloured cables show the data connection from the Raspberry Pi to the motor driver. The yellow cables show the connection between the batteries and the power management circuit.

Design and Development of a Pill Dispenser

247

In the rightmost diagram are shown the flat ribbon cables that connect the Raspberry Pi to the camera and microphone array.

Fig. 7. Device wiring diagram.

3.3 Preliminary Tests of the Pill Dispensing Mechanism After all the electronic connections were made, we advanced to testing the components: the touchscreen, camera, microphone, speakers, and dispensing mechanism. After booting the Raspberry Pi connected to the touchscreen, via the two USB and one HDMI cables, the screen displayed the operating system’s desktop environment and touch inputs were registered as expected, confirming the correct functioning of the component. To test the microphone array, the software Audacity was installed and used to record a short message, which was then played through the speakers, verifying their state as functional. However, some noise was detected during playback, suggesting that a filter should be installed in future iterations. When installing the camera some compatibility issues with the Raspberry Pi were detected. However, due to time constraints of the project, a correction of this issue was not yet possible. An online guide [13] was followed to determine the correct connections between the Raspberry Pi and the step motor’s driver. From this guide was also extracted the base Python code that would control the motor. This code was then adjusted so the motor would only execute the number of rotations necessary to match the rotation of one segment in the cartridge, dispensing the pills contained in that segment. When testing the mechanism, it was noted that on some occasions the mechanical parts would not engage correctly, resulting in an incomplete rotation and failure to dispense the pills. To fix this error, revising the geometry and positioning of the mechanical parts will be necessary.

248

Ó. T. Viana et al.

In this stage it was also noted that the cup’s original geometry proved inefficient in catching the pills as they were dispensed, implying revisions were necessary. To correct this, the cup was slightly elongated, the brim was widened, creating a tapered profile. These adjustments proved effective in improving the cup’s ability to catch dispensed pills. Magnets were employed to ensure the correct placement and orientation of the cup (Fig. 8).

Fig. 8. Successful pill dispensing test.

Additional issues were also detected and will require correction before further prototyping efforts (Table 1). These issues were identified and categorized as Electrical, when referring to changes that should be made to the circuit; Structural, for issues concerning the stability and integrity of the device; and Functional, for issues that impacted correct function and operation of the device. Table 1. Issues identified and suggested corrections. Electrical Diode required at power input;

Structural Fixaon to base in 2 corners is insufficient;

Funconal Geometry of rear cartridge lid prohibits opening, edges should be rounded;

Screen power must bypass RaspberryPi;

Fing lips should be thicker to support weight from components;

Mechanical gear posioning should be adjusted to ensure constant engagement;

A noise filter should be added to the speakers;

Screen requires greater support than the current 3 fixaon points;

Model should include adequate cavies for magnets under the cup and in the cartridge lid;

Supporve roller bearings should be added below the cartridge for smoother rotaon;

The current camera is incompable and should be replaced;

Design and Development of a Pill Dispenser

249

Further development of this project would implicate finding adequate solutions to these issues, implementing them, and producing an updated version of the prototype. This iteration would then be submitted to additional testing to continue improving on the design and function of the device until satisfactory.

4 Conclusion This article presented the concluding phase of a project for the development of an automatic pill dispenser enabled with smart features. This development followed previous research to propose an effective method of aiding the elderly population, health professionals and caretakers in improving adherence to medication regimes, while considering the physical and cognitive limitations of the target demographic. The design process took an approach starting from physical models where dimension and ergonomics concerns were studied, advancing to detailed CAD modelling and 3D printing, concluding with the assembly of electrical and mechanical components into the printed frame. These stages resulted in a functioning prototype that could be submitted do testing to evaluate the viability of contextual implementation of the device. Moreover, using 3D printing as a rapid prototyping method enabled the detection of various issues to be resolved in further iterations, in a relatively fast and inexpensive way, contributing to a more efficient production and more refined results. Further development of the project would see a corrected prototype, with a fully programmed user-friendly interface that could be employed in contextual testing with the target demographics. Acknowledgements. This work was funded by the project “NORTE-01–0145-FEDER-000045”, supported by Northern Portugal Regional Operational Programme (Norte2020), under the Portugal 2020 Partnership Agreement, through the European Regional Development Fund (FEDER).

References 1. Beyer H, Holtzblatt K (1997) Contextual design: defining customer-centered systems. Morgan Kaufmann Publishers Inc. 2. Czaja SJ, Boot WR, Charness N, Rogers WA (2019). Designing for older adults. CRC Press. https://doi.org/10.1201/b22189 3. Fernihough A, Mcgovern ME (2014). Physical stature decline and the health status of the elderly population in England. https://doi.org/10.1016/j.ehb.2013.12.010 4. Gibson I, Rosen D, Stucker B (2015). Additive Manufacturing Technologies, 2nd ed. Springer, Heidelberg. https://eprints.ukh.ac.id/id/eprint/183/1/2015_Book_AdditiveManufacturingTec hnolog.pdf 5. Gomes Cancela DM (n.d.) O processo de envelhecimento 2007 Trabalho realizado no Estágio de Complemento ao Diploma de Licenciatura em Psicologia pela Universidade Lusíada do Porto. www.psicologia.com.pt. Accessed 20 Mar 2023 6. Gopinath B, Rochtchina E, Wang JJ, Schneider J, Leeder SR, Mitchell P (2009) Prevalence of age-related hearing loss in older adults: blue mountains study. Arch Intern Med 169(4):415– 416. https://doi.org/10.1001/ARCHINTERNMED.2008.597

250

Ó. T. Viana et al.

7. Lima O, Terroso M, Dias NL, Vilaça J, Matos D (2023) Development of a pill dispenser: system requirements and product architecture. In: Martins N, Brandão D (eds) Advances in design and digital communication III, vol 27. Springer series in design and innovation. Springer, Cham, pp 336–344. https://doi.org/10.1007/978-3-031-20364-0_29 8. Lima O, Terroso M, Dias N, Vilaça JL, Matos D (2022) Development of the business model and user experience for a pill dispenser: a designer perspective. In: Martins N, Brandão D (eds) Advances in design and digital communication II, vol 19. Springer series in design and innovation. Springer, Cham, pp 552–564. https://doi.org/10.1007/978-3-030-89735-2_45 9. Lord SR, Ward JA (1994) Age-associated differences in sensori-motor function and balance in community dwelling women 10. Nyberg L, Lövdén M, Riklund K, Lindenberger U, Bäckman L (2012) Memory aging and brain maintenance. Trends Cogn Sci 16(5):292–305. https://doi.org/10.1016/J.TICS.2012. 04.005 11. Osterwalder A, Pigneur Y (2010). Business model generation: a handbook for visionaries, game changers, and challengers, p 278. https://www.wiley.com/en-us/Business+Model+Gen eration%3A+A+Handbook+for+Visionaries%2C+Game+Changers%2C+and+Challengersp-9780470876411 12. Owsley C (2011) Aging and vision. Vision Res 51(13):1610–1622. https://doi.org/10.1016/ J.VISRES.2010.10.020 13. Weaver K (2018) Controlling stepper motors using Python with a Raspberry Pi. Medium. https://keithweaverca.medium.com/controlling-stepper-motors-using-pythonwith-a-raspberry-pi-b3fbd482f886

Design and Craft Come Together - A New Perspective to Synergistic Interactions Krasimira Drumeva(B) Department Graphic Design and Visual Communication, St. Cyril and St. Methodius University, 48 Nezavisimost, 5000 Veliko Tarnovo, Bulgaria [email protected]

Abstract. The development of mass industrialization slowly and steadily breaks the connection between crafts and design. Step by step, extreme functionalism has come to prevail, which leads to dehumanization of industry, mass production and market consolidation. The result of these tendencies is that the living environment is oversaturated with standard uniform and indistinctive mass products. A consumeristic attitude is encouraged, emphasizing the use of increasing amounts of cheap goods, which quickly become obsolete or discarded simply because new, attractive, but some low-quality articles have arrived. Hence, the consumption circle turns faster, stretching up to formidable proportions. But just as any other tendency, which does not settle for long, consumers and designers got tired of the extreme utilitarianism of industrial production. Design is increasingly taking a new line of limited-edition handmade articles, which demonstrate more freedom, diversity and personalization. This results in creative collaborations between craftsmen, designers and artists, which work together and share experience, techniques and innovations. This study seeks to provoke an active discussion among artistic circles on bringing back crafts into contemporary art and design projects, especially when specific local conditions offer numerous possibilities for successful collaborations. To realize these intentions, we should specify areas where crafts and design overlap and stand apart, in order to define the possibilities for integration of good practices and elimination of the synectic factors. Based on the conclusions drawn, this study will define several starting points for future research. Keywords: Craft Products · Sustainable Design · New Product Development · Design Thinking · Craft Knowledge · Cultural Identity

1 Introduction In its very emergence, design was fundamentally interwoven into traditional crafts. The first articles and prototypes of articles were designed by artisans or were based on techniques passed on from generation to generation [1]. With the development of industrial production, these connections were broken, which lead to full exclusion of the decorative effect, the handicraft and the pursuit of ultrafunctionalism. These changes are more tangibly manifested in the background of the digitalization and globalization of both © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 251–260, 2024. https://doi.org/10.1007/978-3-031-47281-7_20

252

K. Drumeva

creative and production practices. These processes are undoubtedly leading to decline and even to extinction of crafts. But just as any other tendency, which does not settle for a long time, consumers and designers got tired of the ex-treme utilitarianism of industrial production. A number of artists take their inspiration from the unparalleled subtle antiquities and devote entire collections to the search of their contemporary interpretation. As a result, creative collaborations emerge between local craftsmen, designers and artists, which work together and share experience, techniques and innovations. To achieve an efficient and commensurate cooperation between both parties, based on research method - comparative analysis, this study will pin down zones where design and crafts overlap and stand apart, in order to define the possibilities for integration of good practices and elimination of the synectic factors. The methodology is based on theoretical research by performing a traditional literature review and an empirical research method by comparing existing design projects. For the sake of broadening the view, design will be examined as part of the global consortium of industrial production, since at this stage it has become an inseparable part of the entire process.

2 Design and Craft Come Together - A New Perspective to Synergistic Interactions 2.1 Comparative Analysis of the Essential Characteristics of Crafts and Design Based on an overview of the stylistic and technological methods applied in the production of mass products, it can be deduced that they are diametrically opposite to the handmade articles. It can be noted that they are mass-scale, uniform in terms of shape and materials used, and their main purpose is low cost, quick sale and profit. As a result, the consumeristic attitude is encouraged, using plenty and cheap products that are quickly worn out or discarded. In his research, Boris Serginov selects a number of negative results, such as degradation of individuality and artistic taste. A tendency for marginality is observed, which turns into a kind of mirror that reflects modern reality. A parallel social and cultural surface is developed that reflects an inverted and grotesque image of the world [2]. This undoubtedly has a negative impact on mass culture and generates immense quantities of non-degradable waste. On the other hand, in the sector of luxurious articles, particular attention is given to overly expensive materials, complicated processing techniques and eccentricity. The products lose their functionality and become works of art. The main distinctive feature of industrial production is the division of labor [3]. On the one hand, the designer defines the vision of the product, the engineer selects the materials and the structural components, the technologist determines the method of production and so on, until the product is ready for packaging, market placement and transportation. Of primary importance for the qualities of industrial goods are market competitiveness, functionality, durability, aesthetics, optimized serial production, cost effectiveness of materials, production, transport and sale. Referring to the research of Nigel Cross [4], the following advantages can be synthesized: – Serial production of identical products. – Technological innovations.

Design and Craft Come Together

– – – – – – –

253

Simplicity and affordability for the mass user. Prototyping and testing. Planned management of the design, production, and marketing process. Machine production, which alleviates the labor of workers. Diversity of materials. Division of labor – each activity is assigned to a highly specialized worker. Globalization of the market.

Let us draw a parallel to traditional crafts, where the main characteristic is the manual production and partial use of tools. Mechanical aids are carefully selected, controlled and limited to the immediate manual work of the craftsman, and it is the main component in making the final product. Such products may be utilitarian, artistic, decorative, traditional, religious and socially symbolic [5]. High-quality natural materials are used, which require specific and sometimes long preparation. The efforts and costs devoted guarantee the durability of the articles. Crafts are passed on as cultural heritage, they preserve the traditions and cultural diversity, which have tangible and intangible dimensions. The tangible dimensions are related to creating objects and tools, use of natural resources, construction of workshops and provisions of jobs. According to the authors of the article “Representation and Preservation of Heritage Crafts”, crafts are maybe the most tangible intangible heritage. The intangible dimensions include preservation of the knowledge, skills and socio-historical content of communities and regions and passing them on to the next generations. They are an integral part of the folklore and the economic life of the regions and communities, in which they flourish [6]. Based on the aforementioned sources, we can summarize a number of advantages of the artisanal production, such as: – – – – – – – – – – –

Creation of a unique handmade product. The selected material dictates the shape in the process of production. Ecology. Sustainable use of raw materials and labor force. Development of poor and rural areas. Preservation of the traditions and the cultural identity of the regions. High quality – certified craftsmen. Deftness, artistry and resourcefulness. Aesthetics. Personalization. Closed production cycle.

2.2 Comparative Analysis of the Areas of Overlapping and Differentiation Between Crafts and Design Although we classified crafts and design into two separate categories, they overlap in a number of areas. For instance, both designers and artisans apply innovative and creative practices with an aim to improve the surrounding environment and to create products that are practical, aesthetic and economic. The main linking piece in the comparative analysis of crafts and industry is the process of creation of prototypes [7]. For instance, in industrial production, the designer participates in an interdisciplinary team, which creates

254

K. Drumeva

the final product. The prototype is used as an instrument to find and resolve problems, and to express concepts. While the participants from different spheres are working together, they strengthen the creative communication in the team. This facilitates the generation of ideas and their optimization. In this key moment, the creative process in crafts is to some extent differentiated with regard to the leading figure. The craftsman dominates the process and also defines the result, in order to create an unique product. This approach includes taking the responsibility for the artistic work itself, which is also connected with taking the risk regarding the feasibility of the production process and the quality of the result. The experience, the skill and the devotion which the author puts into his work define the final result. As seen from this definition, the craftsmanship is based on individual creative effort and work, which characterizes the process of creation of the article. While the designer may prefer to create one prototype for better understanding of the product, for the craftsman this is the actual product. This comparison may be used to summarize the specific nature of authorship in crafts and design, as well as their contribution to development of new products [8]. Based on comparative analysis, the following differences may be pointed out between design and artisanal practice: – The education of designers is multifaceted and integrated, combining the areas of science, economics, art and social culture, while artisans are trained through personal contact between craftsman and apprentice, which is focused on acquiring a specific skill. – Designers use a great variety of materials supplied from all over the world, while artisans use the traditional raw materials available in the specific region. – In the designer’s practice, the planned production is managed through division of labor in an interdisciplinary team, while crafts include independent and overall production from the very concept to the final product. – The designer creates articles, which are manufactured by using machines and automated methods, while craftsmen produce a small series of products focusing on the manual workmanship. – Designer products are standardized, in order to satisfy the public demand, while the crafted product is personalized and carries specific cultural and emotional symbols [9]. The successful creative collaboration between crafts and design is a mechanism for mutual training and sharing of experience, where both parties may exchange knowledge and improve their professional capacities. Designers have the capacity to play a catalytic role in facilitating the dissemination of shared experience in a wide range of industries [10]. Industrial production and modern technologies can provide artisans with the possibility to create uniquely crafted objects and at the same time to ensure sufficient quantities for the achievement of sustainable models. Modern production methods are adapting to the workmanship processes in an attempt to encourage new ways for development and enhancement of small series of products. Traditional crafts include practices, which are rooted in local knowledge and accumulated over time, they have become part of the cultural heritage and should be preserved and revived. From the point of view of encouraging regional competitiveness through cultural industry, the development of products based on the cultural heritage and local

Design and Craft Come Together

255

resources may possibly enhance the local economic development. Based on the research of Walter Santagata [11] may be defined a number of collaboration areas, such as: – Artisans may provide knowledge about the properties of the material. – The design approach may be beneficial to artisans by revealing new, advantageous perspectives for development of the initial concept. – Design is a vehicle of innovation and may change the artisans’ conservative approach. – Design has a critical and creative orientation towards aesthetic and technical production, which combines modern information and communication technologies. – Crafts rely on the experience and skills gathered over generations, which are an important factor in creation. – Designers make several sketches, digital drawings and models. – Artisans follow the characteristics of the material. – Planned management of the design, production and marketing process. – Preserving and sharing the traditions and the cultural identity of the regions. – Ecological and sustainable use of raw materials and labor force. – Personalization. – Closed production cycle. A number of overlapping areas were established between the industrial and artisanal production. Based on the knowledge about the specific features of both types of production, a range of good practices may be successfully applied in order to achieve a balance between local and global aspects, unique and mass production, good price and quality. The collaboration may give rise to a “new generation” of products with characteristic durability, added value and sustainability. Design can adequately play the role of a mediator between industry and crafts, because the methodology of design has a synergistic connection with handicraft in many of the stages of research, analysis and trial of the creative concepts. The production lines may be optimized in terms of sustainability by applying the supply practices of traditional workshops. This is related to the selection of raw materials, which are extracted and produced in the same region by local companies. Such initiatives are particularly efficient in poor and technologically underdeveloped communities. 2.3 What Happens in Practice? Currently, in Bulgaria and most of the European countries, crafts are dying out. Their place is taken by mass production, which floods the market with huge quantities of low-cost and easily-sold products. This forces artisans to sell their handicraft unique articles at industrial prices. And makes their work worthless, as well as the time and the materials used for production. As a result of that, less and less people are willing to continue the tradition and these small workshops are dying out. At many places, they have been turned into souvenir and custom product shops, which cannot replace the authentic atmosphere of workshops. Thanks to the market demand and the creative concepts of artists, some design studios have created their own small workshops, to explore traditional crafts and apply them in their products. This undoubtedly happens by updating the technology in line with new developments and by interpreting the tradition in the modern context. Contemporary

256

K. Drumeva

production methods, such as casting, laser cutting and digital print, have been adapted to the craft processes to encourage the development and optimization in small batches. The integration of digital technologies in the traditional processes of product creation supports artisans in their search for new approaches to the development of their crafts, in order to remain competitive, to personalize their production, to maintain flexibility by applying the innovations of industry. According to Fang-Wu Tung this wide range of production possibilities provides the artisans with innovations and opportunities to apply entrepreneurship and marketing approaches. Designers on the other hand, can stimulate local artisans to use productions techniques by means of collaboration and can thus empower them to deal with the processes and consequences of industrialization [6]. We can still not ascertain the existence of an effective synergy between industrial designers and artisan groups, which can stimulate to a sufficient extent the development and preservation of local crafts as a sustainable practice. The conceptual pursuits of the representatives of the Bauhaus visual and plastic arts school offer better understanding of the interaction between the handmade and machine produced articles. Their leading direction was the comprehensive knowledge of crafts as the original source of art and a basis of the new functional works of design. Their projects are modern and humanistic, recognizing both the handicraft and the industrial product, the boutique and the mass production [1]. The hybridization of crafts and industry is a constructive strategy for new product development and it can definitely be said to boost competitiveness. More and more often, designers use local raw materials with maximum feasibility to emphasize the specific identity of the region. Thus, for instance, the Hungarian textile designer Kata Mónus created her diploma project in 2011 as an experimental piece of

Fig. 1. Designer: Kata Monus, photo: Réka Hegyháti [12]

Design and Craft Come Together

257

storage furniture. The innovative solutions were the constructive links of knitted textile. Her Crossropes cabinet was designed in collaboration with Tamas Bozsik and received the honourable Green Furniture Award 2012 (see Fig. 1). While crafts are a disappearing phenomenon in Europe and the developed societies, in Asia and Africa they are still a main means of living for the people. For that reason, the designer Porky Hefer travelled a lot in Africa to explore the production practices of the local craftsmen. He was so much inspired by nature that he created his extraordinary collection of “human nests”. His aim was not to copy the bird nests, but to develop the structure into new functional and comfortable pieces of furniture for people. They look more like intricately woven cocoons, which create an internal microclimate and allow the user to isolate from the outside world. Just like bird nests, Hefer’s nests surround the body, alleviate the senses, giving a chance for reflection and a feeling of comfort. The author intentionally used traditional techniques, which rely on human labor and not on machines, to prove the timeless skills of many generations of craftsmen. For that

Fig. 2. Designer: Porky Hefer, Craft Partner: Cape Society for the Blind [13]

258

K. Drumeva

purpose, he collaborated with local craftsmen and gave them the opportunity to revive and spread their production. He worked with the skillful weavers of the Cape Society for the Blind to create nests adapted to human ergonomics. He intentionally selected reed as the main raw material to build his masterpieces, to ensure natural recycling. Thus, he created a sustainable design of his products, which do not become an issue when they wear out, but they just disappear in nature (see Fig. 2). Even big companies like Panasonic implement a number of projects in cooperation with local craftsmen. “Electronics Meets Crafts: Engraving Phenomena” is a joint creative project with Kyoto Kaden Lab. - a craftsmen cooperative in Kyoto. The main concept is to resonate with the five human senses by demonstrating the high mastery and aesthetic awareness, typical for the Japanese culture. The high technologies of Panasonic are combined with the traditional craft of bamboo basket weaving to create Soyo Gu - floor fan with innovative design. The interdisciplinary team is focused on rendering three key elements: “the feeling of the gentle breeze from head to toe”, “the feeling of silence” and “the feeling of grass swaying in the wind”. To make the product, materials

Fig. 3. Company: Panasonic, Japan, Team Lead: Kyohei Mori and Takahiro Yagi, Design: Enrico Bergese, Design Partner: Kyoto Kaden Lab., Japan [14]

Design and Craft Come Together

259

such as brass, cherry wood and bamboo were used, because they age up beautifully, developing patina over time, and thus turn into a truly unique masterpiece (see Fig. 3).

3 Conclusion The value of crafts lies in the creation of handicraft articles, the use of natural materials and the fine tactile stimulation upon touch. The competitive utilitarian function gives way to the precision of handicraft, environmental friendliness, and the nostalgia for tradition. Exactly these characteristics make the articles authentic and comfortable for use. The fusion between crafts and industry takes the form of a designer practice and a model, which is a prosperous strategy for future development of industry. The exploration and practice of traditional crafts by students and professional designers is an efficient approach towards realization of their individual creative pursuits in the form of sustainable practices. This research did not find any preponderance of crafts or design, but synthesized a balanced approach for cooperation, equality and freedom of expression. The settlement of a creative dialogue is a precondition to focus on the best ideas and on the implementation of innovations. In addition, this approach to design sets up constructive directions for use of everyday consumer products. Shapes up sophisticated aesthetics and a conscious attitude towards consumerism. The products acquire a new dimension, their own character and convey the invisible aura of their creator. Teaches respect to the cultural and historical heritage, and crafts in particular. Represented in such a way, the design attains a soul, cultivates better living conditions, makes us feel comfortable and inspires a positive vision for the future.

References 1. Fiell C, Fiell P (2001) Design of the 20th century (Icons). Taschen America LLC, Los Angels 2. Serginov B (2015) Dizayn: dialog i monolog. Fakti, metodi, prognozi. New Bulgarian University, p 365 3. Arora A, Gambardellak A (1994) The changing technology of technological change: general and abstract knowledge and the division of innovative labour. Res Pol 23(5):523–532 4. Cross N (2021) Engineering design methods: strategies for product design, 5th edn. Wiley, Hoboken 5. UNESCO (2009) The UNESCO Framework for Cultural Statistics (FCS). UNESCO Institute for Statistics, p. 26 6. Zabulis X, Meghini C, Partarakis N, Beisswenger C (2020) Representation and preservation of heritage crafts. 12(4):1461 7. Yang MC, Epstein DJ (2005) A study of prototypes, design activity, and design outcome. Design Stud 26(6):649–669 8. Temelta¸s H (2017) Collaboration and exchange between “Craftsman” and “Designer”: symbiosis towards product innovation. Design J 20(sup1):S3715 9. Tung F, Chen I (2013) Collaboration of designers and craft artists on innovative design praxis. In: Proceedings of the 5th international congress of international association of societies of design research (IASDR) Tokyo, JP 10. Dodgson M, Gann D (2018) Innovation: a very short introduction. Oxford University Press, Oxford

260

K. Drumeva

11. Santagata W (2002) Cultural districts, property rights and sustainable economic growth. Int J Urban Reg Res 26(1):9–23 12. Experimental hybrid storage furniture graduation project (2012). http://katamonus.blogspot. com/p/furniture.html. Accessed 21 Mar 2023 13. Porky Hefer exhibits human-sized nests made from woven plant stalks, https://www.dez een.com/2017/01/18/porky-hefer-heart-lightness-exhibition-human-sized-nests-r-companynew-york/. Accessed 21 Mar 2023 14. Panasonic. https://panasonic.net/design/. Accessed 21 Mar 2023

Guidelines for Sustainable Online Behaviors. Green Computing Practices to Reduce the Digital Carbon Footprint Sónia Rafael1(B)

and Matilde Reis2

1 ITI/LARSyS, Faculty of Fine-Arts, University of Lisbon, Largo da Academia Nacional e

Belas Artes, 1249-058 Lisbon, Portugal [email protected] 2 Faculty of Fine-Arts, University of Lisbon, Largo da Academia Nacional e Belas Artes, 1249-058 Lisbon, Portugal

Abstract. The current state of concern for the environment and awareness of the climate emergency have become prevalent issues. The environmental impact resulting from the use and expansion of digital infrastructure highlights the need for greater awareness and action on this topic. While the consumption of material resources can often be perceived through the associated financial cost, the energy consumption caused by the use of digital products, services, and systems remains invisible to users. This article proposes to focus on the area of Green Computing, as it is believed that this discipline can contribute to the reduction of the digital carbon footprint. The aim of the study was to conduct a literature review and a critical and informed analysis of Sustainable Online Behaviors. Building upon the guidelines proposed by Elgaaïed-Gambier et al., 2020, we sought to extend their scope in order to derive an expanded set of Green Computing Practices aimed at mitigating the Digital Carbon Footprint. Through the proposed guidelines, we believe that Green Computing practices can be disseminated and reach various layers of society, contributing to digital citizenship that can educate children and young people about the importance of this issue. Keywords: Green Computing · Digital Consumption · Guidelines for Sustainability · Multimedia Design

1 Introduction This article assumes as a starting point the relationship between the environmental crisis we face and the centrality that digital consumption acquires in our daily lives. We are often confronted with two paradoxical trends. On the one hand, there is currently an environmentally friendly narrative, essentially based on principles of material consumption, aimed at reducing the ecological footprint caused by our lifestyle. On the other hand, metrics on digital consumption reveal that we are spending more and more time in front of screens, either for leisure or professional reasons. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 261–275, 2024. https://doi.org/10.1007/978-3-031-47281-7_21

262

S. Rafael and M. Reis

We know that the decisions made during the interaction design stages of digital product and service projects are crucial to the resulting environmental impact of the use and growth of digital infrastructure. While the resource consumption involved in the transportation sector, for example, is often perceived through the financial cost associated with fuel, the energy consumption caused by the use of digital products, services, and systems becomes invisible to users. The emergence of video streaming services is one of the factors that most contributes to the exponential increase in digital consumption today, representing up to about 80% of total data traffic worldwide. The growing incorporation of persuasive technologies and data tracking mechanisms in digital platforms is another factor contributing to the drastic increase in data consumption and resulting carbon footprint. In this context, it is proposed that this article focuses on the areas of Green Computing, as it is considered that this discipline can contribute to adapting the digital carbon footprint to the reach of the common user and promote awareness of the issue. The relevance of this topic is essentially related to its current nature. Across various sectors, there is an increasing dependence on digital technologies. At the same time, environmental concerns and awareness of the climate emergency are emerging. The numerous summits and initiatives for environmental preservation are issues that mark the present time. Guidelines rooted in Elgaaïed-Gambier et al.’s research [17] were created, expanded through a comprehensive literature review. This led to a wide-ranging set of guidelines for easy dissemination. The study highlighted the limited focus on Green Computing awareness for everyday users and its absence in digital citizenship education. The proposal aims to raise awareness of adapting the digital carbon footprint and comprehending the environmental impact of digital expansion. The proposed guidelines aspire to promote Green Computing practices across society, fostering digital citizenship that educates youth about their significance.

2 Metrics on Digital Consumption There is limited research on the definition of digital consumption, and there is also ambiguity due to the multiple terms used, such as “traffic,” “internet consumption,” “data consumption,” and “online consumption.“ This article adopts the term “digital consumption” to refer to the current paradigm of accessing and consuming content through digital environments, with the aim of providing a clear and intuitive definition for better understanding of the topic. In its broadest sense, digital consumption refers to the sum of entertainment and information media consumed by an individual [1], including the production and reproduction of everyday life and ways of experiencing the world, with individual, social, and cultural characteristics [2]. According to a study by Internet World Stats, the percentage of the global population with internet access has almost doubled in the last decade, contributing to an unprecedented volume of data traffic [3]. A global study on data consumption conducted by Tammy Everts highlights the timeliness and emergence of awareness of this issue [4]. Since 1995, the weight of web pages has increased about 150 times. In 2012, websites

Guidelines for Sustainable Online Behaviors

263

averaged 660 MB, contrasting with more than 2 MB in 2022 [3]. The size of a web page is dramatically increased by the use of scripts and the incorporation of video content, resulting in excessive content delivery that may not have been requested by the user [5, 6]. The following figure illustrates the distribution of global data traffic on the internet in 2021 by categories, with streaming video accounting for 60.6%, followed by web pages at 13.1%, social networks at 6.1%, file sharing at 4.2%, marketplace at 2.6%, VPN at 1.6%, messaging at 1.6%, cloud at 1.4%, and audio at 0.4% [7].

Fig. 1. The Internet data traffic distribution in 2021. Adapted from Science et Vie (2021). Les clés pour comprendre la pollution numérique [7].

The percentages attributed to internet data consumption shown in the image above (Fig. 1) correspond to a carbon dioxide emission rate, a metric used to calculate the ecological footprint of the identified services. The increase in time spent online emerges as another relevant indicator for describing the global landscape of digital consumption. According to the agency We Are Social [8], 6h54 represents the average time that users between the ages of 16 and 64 spend on the internet, on any device, recorded in 2021. At the extremes in the ranking presented by the DataReportal platform [9], the Philippines and Japan are positioned (with daily averages of 10h54 and 04h25, respectively). Immediately above the average (6h54), the United States of America (7h11), Hong Kong (7h15), and Portugal (7h20) figure, represented as one of the European countries that spends the most time in front of screens, daily [9].

3 Expansion of Digital Infrastructure 3.1 Impact of Digital Technologies on Ecological Footprint Among the numerous reasons for the writing of this article, facts that reflect the challenges imposed by the environmental reality we face are highlighted. The environmental crisis scenario can be defined by a set of different factors. They are clearly spelled out in the manifesto presented by The Shift Project [10].

264

S. Rafael and M. Reis

The galloping increase in time spent online, the growing availability of smart devices (interconnected and used simultaneously), and the emergence of digital assistance systems in our daily lives [9] are factors that promise to contribute to a drastic increase in the global ecological footprint in the coming years, along with the development of Artificial Intelligence and the Internet of Things [11]. In this context, James Bridle states that “the cloud doesn’t just have a shadow; it has a footprint” [12] (p.21). Recent studies on the environmental footprint associated with Information Technologies [13] highlight the impact of cloud-based services, such as video streaming and the widespread use of smartphones [14]. In 2018, these devices accounted for 52% of global website traffic [8]. Although their operation tends to be more energy-efficient than desktops, mobile phones encourage more intensive use of digital tools [15]. Figure 2 illustrates the estimated annual average carbon emissions caused by the use of digital technologies on an individual scale.

Fig. 2. The CO2 Footprint of our Digital Lives. Estimated CO2 emissions related to the use of digital technologies per person and year (in kg). Calculations made in Germany (April 2020). Statista.

The chart shows that 849 kg represents the estimated total CO2 emissions caused by the use of digital technologies associated with a user residing in Germany in 2020. As shown, 40% (246 kg) of the carbon footprint is due only to the production and assembly of electronic devices such as television, laptop, smartphone, etc. The use of these devices’ accounts for 22% (189 kg) of CO2 emissions. The use of the internet network, such as streaming services and social media, is associated with 8% (64 kg) of the annual carbon footprint. The operation of digital infrastructures such as data centers, antennas, cables, servers, web page research and consultation, is responsible for 30% (250 kg) of carbon emissions. In 2015, the consumption of data centers worldwide, where exabytes of digital data are stored and processed, represented about 3% of the world’s electricity and accounted

Guidelines for Sustainable Online Behaviors

265

for 2% of total global emissions [16]. It is expected that this consumption will massively escalate as a result of the expansion of digital infrastructure and the expected increase in global average temperatures [16]. Although in recent years there has been an improvement in the energy efficiency associated with the operation of data centers, there is currently an increase in energy consumption associated with the increase in demand and use of digital technologies, globally [17]. This phenomenon is called the Jevons Paradox [18] and is characterized by the assumption that the improvement of the energy performance of Information and Communication Technology (ICT) infrastructures does not keep up with the increase in data traffic, given the growing digital consumption [19]. In 2040, Information and Communication Technologies may account for a significant contribution (over 14%) of greenhouse gas emissions at the global and individual level, corresponding to over half of the current carbon footprint of the transportation sector [6]. Between 2010 and 2020, Belkhir and Elmeligi found that, among the various categories of electronic devices analyzed, smartphones are the devices that contribute the most to GHG emissions, with values ranging from 4% to 11% [20]. A laptop connected for eight hours a day consumes between 150 and 300 kWh and emits between 44 and 88 kg of CO2 per year. In standby mode, a device’s energy consumption drops to about a third [21]. Sending an email emits about 4 g of CO2. If the body of your message includes attachments, the GHG emissions can reach 50 g [22]. The environmental cost of downloading a video game is higher than the production and distribution of Blu-ray discs in stores. The first study conducted to calculate the energy demand of the gaming industry in the USA accounted for the production of 24 megatons of carbon dioxide per year [23]. Technologies as promising as the Bitcoin cryptocurrency also have extremely harmful environmental consequences. This emerging resource of centralized financial systems involves energy consumption equivalent to the daily electricity consumption required by nine US homes to carry out a single transaction [24]. This type of virtual transaction is equivalent to the emission of 22 million tons of carbon dioxide, representing an annual ecological footprint equivalent to that of Jordan [25]. The human influence on global warming mentioned by the Intergovernmental Panel on Climate Change (IPCC) highlights the urgency of drastically reducing CO2 and other greenhouse gas (GHG) emissions [26]. This context of climate crisis is compounded by the technological impact caused by the COVID-19 pandemic. Studies indicate that internet use during periods of confinement increased by 40 to 100% compared to the pre-pandemic scenario, and this has contributed to a significant increase in energy consumption [27]. In the coming decades, the rise in global average temperature will exacerbate the impact caused by computational data infrastructures that are already overheated [12]. The operation of data centers and individual electronic equipment generates excessive amounts of heat, which require corresponding amounts of cooling [12]. The Eureca project, funded by the European Commission, highlights that in 2017, data centers located in EU countries consumed 25% more energy compared to 2014 [28].

266

S. Rafael and M. Reis

Contrary to what the promise of digital dematerialization may promote, the maintenance of data centers and so many other digital infrastructures involves the occupation and construction of physical spaces composed of urbanistically developed campuses on the scale of huge metropolises [29]. In the last decade, there has been a drastic increase in data storage and processing capacity. For this reason, the amount of energy required by data centers has doubled every four years and should triple in the next ten years, globally [30].

4 Constraints and Motivations for Adopting Sustainable Online Behaviors 4.1 Pro-Environmental Practices Environmental sustainability is generally defined as a present capacity for agency that does not threaten the environment in the future [33]. One of the aspects that contributes to the reluctance observed in the receptiveness to the change of online habits with the objective of reducing the digital carbon footprint by users is based on a dichotomy of motivations that compete with each other. On the one hand, there is an environmental concern arising from a planetary environmental crisis. On the other hand, users fear losing the benefits and the sense of well-being perceived in the use of the Internet [17]. One of the most recent definitions of digital well-being corresponds to a state where subjective well-being is maintained in an environment characterized by digital communication overabundance. Within a condition of digital well-being, individuals are able to channel digital media usage towards a sense of comfort, safety, satisfaction and fulfillment [34]. Another study presented an approach to the theme, based on an individual perspective that reveals the contribution of intrinsic and extrinsic motivations to the adoption of Green IT practices [35]. Sensations of satisfaction and financial savings as intrinsic and extrinsic motivations, respectively, constitute decisive aspects for the perception about the implementation or discarding of the possibility of using Green IT artifacts capable of reducing electricity consumption resulting from online activities [35]. With the purpose of analyzing the motivations for the adoption of pro-environmental online behaviors, Gnanasekaran et al. conducted an exploratory study on the inherent reasons for digital ignorance revealed by the participants, in order of relevance: a) lack of technological knowledge; b) lack of accessible public information; c) lack of social awareness in school context [36]. At the level of reasons that contribute to the adoption of pro-environmental behaviors, dichotomies were identified between personal well-being versus environmental well-being, a utilitarian approach versus a hedonistic use of digital technology, and involvement in social activities, in the sense that users recognize that the decrease in encounters with other people results in more time spent in front of screens, particularly in the use of video streaming platforms. Regarding the receptivity to the adoption of pro-environmental measures online that users are willing to implement in order to decrease the negative impact of Internet use on the environment, some participants stated that they are willing to use less harmful

Guidelines for Sustainable Online Behaviors

267

video streaming services, even if they involve an extra monthly cost. Other examples include the use of “green” search engines. They also consider that if their use does not compromise the search experience and contributes to a reduction in the digital ecological footprint, then their use should be normalized. The last aspect mentioned is the prioritization of content over low quality in video streaming contexts. Participants prefer a greater supply of low-resolution video content over high-definition streaming with a smaller content offering. Two factors were identified as greater challenges to the adoption of pro-environmental online behavior: on the one hand, participants refer to the limited impact of individual actions when the dynamics of big tech companies have a much greater environmental impact, revealing a reluctance to recognize individual responsibility; on the other hand, users’ familiarity with the applications they use in their daily lives, such as the personalized service that Spotify provides based on data collection from each individual’s music patterns and preferences, results in resistance to replacing it with other applications [36]. The combination of psychological factors evidenced by users’ behavior, such as the sense of material comfort (suggested by the notion of digital well-being), is crucial for the development of digital tools, products, and services that promote conscious behavior of their environmental impact. 4.2 The Intangibility of Pro-Environmental Online Behavior Typically, the notion of intangibility is associated with something that is considered intangible, encompassing what cannot be touched, seen, smelled, tasted or heard [37]. From the individual’s perspective, some authors [38] have analyzed the effects of intangibility on consumers’ online behavior in a context of purchasing habits. The difficulty of mentally comprehending a phenomenon by users generates uncertainty about the negative impact caused by their behavior. In the case of online behavior, this aspect is considered preponderant, since technological dematerialization contributes to the potential reduction of perceived environmental harm [29]. While, for example, resource consumption involved in the transportation sector is often perceived through the financial cost of fuel, the energy consumption of digital services and products is omitted from users through the interface [5]. When we perform a search through a search engine, the ecological impact of the data traffic involved in this action becomes less evident. Although a recent study [17] revealed that the intangibility of online behaviors did not prove to be the most decisive factor in the intentional process of acts, participants believe that the intangibility of environmental consequences resulting from digital consumption is associated with a lack of information about the problem. According to the surveyed users, the ubiquity of the internet makes it difficult to implement behavioral change.

268

S. Rafael and M. Reis

5 Green Technology and Green Computing In order to establish a relationship between technological development and environmental sustainability, a study [39] highlights the potential beneficial effects of technology use in general, and the positive impact of IT on sustainability in particular [40]. Proenvironmental behavior can be defined as all actions taken to avoid damage and/or preserve the environment [41]. Although many pro-environmental initiatives are tendentially associated with the consumption and management of material goods and resources (e.g., recycling practices) [42], other types of behaviors may also be included in this scope, such as current patterns capable of characterizing a digital consumption society [17]. Before understanding a definition of Green IT, it is important to frame this concept from a broader notion: Green Technology. “Green technology” is a term that describes a form of technology that is deemed eco-friendly due to its manufacturing process or supply chain. Green technology’s mission is to protect the earth and, in some situations, also to undo previous environmental harm [43]. In order to define objective principles for technology to be considered green, Venkadeshwaran K gathered 5 generic criteria satisfactory to determine if a technology conforms to its ecological dimension. The following image describes them [43].

Fig. 3. Strong criteria for a technology to be considered green. Adapted from Venkadeshwaran K., 2016 [43].

As shown in Fig. 3, Green Technology presupposes the transformation and utilization of resources, devices, and systems that contribute to the protection of the environment, minimizing the destructive effects of human activity. Green Technology involves environmentally conscious practices that reduce ecological costs caused by products or services provided solely for the benefit of humans. Ecological practices involve the use of renewable energy, avoiding the emission of greenhouse gases. Once the definition of green technology is considered, it is important to apply the same principles to digital technology. Given the availability of studies on the negative environmental implications associated with traditional ICTs [44], there is a growing adoption of Green IT practices, especially at the organizational level.

Guidelines for Sustainable Online Behaviors

269

Also known as Green Computing, Green IT corresponds to the “(…) study and practice of designing, manufacturing, using, and disposing of computers, servers, and associated subsystems – such as monitors, printers, storage devices, and networking and communications systems – efficiently and effectively with minimal or no impact on the environment” [45] (p.2). Most academic work done on the topic of Green Computing concerns the behavior of professionals in the field of ICT [46] or the practices adopted by companies [47].

6 Green Computing Guidelines for Reducing the Digital Carbon Footprint The limited existence of academic work focused on user receptivity to adopting Green IT behaviors and the exponential increase in the percentage of the world population with internet access (from 28.8% in 2010 to 58.8% in 2019) [3] are understood as opportunities to contribute to existing literature. In order to exemplify “green” online behaviors or those considered less harmful to the environment, Elgaaïed-Gambier et al. compiled practical recommendations for the common user, based on various sources such as environmental agencies, consultants, academic articles, and blogs [17]. The behaviors were grouped by the category of online activity (searching, emailing, data storage, etc.) and according to the nature of the behavior itself. Each practical principle includes the optimization of a habitual behavior and/or the substitution of that behavior. This distinction between activities was based on reducing data requested by distant servers and data centers, which results in increased energy consumption. Although the information found identifies online behaviors presented as being less harmful to the environment, the authors remind us that interviews conducted in the scope of the study of user receptivity to implementing Green IT practices show that such approaches are still unknown to the general public. Thus, the following figure aims to promote Green Computing recommendations that can be implemented by the common user in their day-to-day activities. It is expected that Table 1 will contribute to the integration of strategies for reducing carbon footprint into digital literacy principles, as well as to the consideration of environmentally friendly practices in users’ digital routines. Therefore, the dissemination of this chart aims to democratize more eco-friendly digital consumption patterns, in order to promote less polluting digital consumption patterns for societies and the planet.

270

S. Rafael and M. Reis

Table 1. Practical Green Computing guidelines for reducing the digital carbon footprint generated by the use of technologies. Based on Elgaaïed-Gambier et al., 2020 [17].

(continued)

Guidelines for Sustainable Online Behaviors

271

Table 1. (continued)

7 Challenges and Future Work As future work, we intend to continue studying the subject, as well as taking these guidelines to a school context, disseminating them, and applying them in specific contexts with students. Our next step will be to apply these guidelines, first to students of the bachelor’s degree in Communication Design. This is so that we have easier access to a large sample size and the added benefit of involving students in this research. Looking to the future, the next steps must involve a paradigm shift in the development of values that integrate environmental concerns through Green Computing practices. After analyzing the results obtained, we have a research project to obtain financing in Digital Citizenship.

272

S. Rafael and M. Reis

It is our intention to provide training in secondary schools on Green Computing practices with the production of concrete results where students implement these practices in specific projects.

8 Conclusion We started by investigating digital consumption metrics and realized that in the last decade, the percentage of the world’s population with internet access has doubled [4]. This fact contributes to the production of an unprecedented volume of data in internet traffic. The galloping increase in time spent online, the growing offer of interconnected smart devices, and the emergence of digital assistance systems in our daily lives [9] are factors that promise to contribute to the drastic increase in the global ecological footprint in the coming years, alongside the development of Artificial Intelligence and the Internet of Things. Digital products, services, and systems and the interactive devices that support them increasingly depend on digital infrastructure to function. Contrary to the promise of digital dematerialization, the maintenance of data centers and other physical infrastructure involves the construction of physical spaces composed of campuses on a metropolitan scale [29]. The dominant design paradigm in digital products and services described by Preist et al. [6] reinforces the belief in the continuous growth of digital infrastructure and the notion of infinite scalability of the cloud. Sometimes cloud-based services can reduce their energy and resource consumption through efficient use of servers hosted in the latest generation green data centers, particularly in corporate contexts. However, the problem persists in the end-user context. To analyze the motivations for adopting pro-environmental online behaviors, an exploratory study was conducted by Gnanasekaran et al. on the reasons for digital ignorance revealed by the participants. The following reasons were found, in order of relevance: lack of technological knowledge; lack of accessible public information; lack of social awareness in the school context [36]. A recent study [17] reveals that the intangibility of online behaviors and the resulting environmental consequences of digital consumption are associated with a lack of information about the problem. According to the surveyed users, the ubiquity of the internet makes behavior change difficult to achieve. Although many pro-environmental initiatives are tendentially associated with the consumption and management of material goods and resources, other types of behaviors may also be framed in this context, such as current patterns capable of characterizing a digital consumer society [17]. In this sense, Green Technology describes a form of technology that is deemed eco-friendly due to its manufacturing process or supply chain. Green technology’s mission is to protect the earth and, in some situations, also to undo previous environmental harm [43]. Although the information found identifies online behaviors presented as less harmful to the environment, the authors [17] recall that the interviews conducted in the study on users’ receptivity to the implementation of Green IT practices demonstrate that this type of approach is still unknown to the public.

Guidelines for Sustainable Online Behaviors

273

There are also few works oriented towards the environmental awareness of the user as a responsibility or duty of the digital citizen. In this sense, it is expected that this article will contribute to the creation of conditions that enhance the construction of a digital citizenship based on the responsibility and environmental ethics of technological use. It is hoped that these guidelines can contribute to the reduction of carbon footprint, as well as to the consideration of less harmful practices for the environment in users’ digital routines. Acknowledgments. This research is funded by ITI/LARSyS (Interactive Technologies Institute) – ITI-LX UIDP/50009/2020.X

References 1. Narang R, Tiwari S (2021) Antecedents and consequences of the growing popularity of digital consumption. In: Khosrow-Pour DBAM (ed) Encyclopedia of information science and technology. 5th edn. IGI Global, Hershey, pp 1066–1084. https://doi.org/10.4018/978-17998-3479-3.ch073 2. Mairano MV (2021) Emotions and digital delivery platforms: an approach to the configuration of sensibilities during the isolation caused by the COVID-19 pandemic. In: Korstanje M (ed) Socio-economic effects and recovery efforts for the rental industry: post-COVID-19 strategies. IGI Global, Hershey, pp 178–193 (2021). https://doi.org/10.4018/978-1-7998-7287-0.ch010 3. Internet World Stats. Internet usage statistics. https://www.internetworldstats.com/stats.html. Accessed 27 May 2022 4. Page Bloat Update. https://www.speedcurve.com/blog/ten-years-page-bloat/. Accessed 21 June 2022 5. Parikka J (2021) Working for systems that do not do the work. Joana Moll’s the hidden life of an amazon user. Interface Critique 3:81–85. https://doi.org/10.11588/ic.2021.3.81325 6. Preist C, Schien D, Shabajee P, Wood S, Hodgson C (2014) Analyzing end-to-end energy consumption for digital services. Computer 47(5):92–95. https://doi.org/10.1109/MC.201 4.110 7. Gougis L. Les clés pour comprendre la pollution numérique. Science et Vie. https://www. science-et-vie.com/article-magazine/les-cles-pour-comprendre-la-pollution-numerique. Accessed 14 Apr 2022 8. We Are Social. Percentage of all global web pages served to mobile phones from 2009 to 2018. https://wearesocial.com/uk/blog/2018/01/global-digital-report-2018/. Accessed 16 Jan 2023 9. Digital 2021: Global Overview Report. DataReportal–Global Digital Insights. https://datare portal.com/reports/digital-2021-global-overview-report. Accessed 13 Dec 2022 10. The Shift Project. https://theshiftproject.org/en/our-purpose/. Accessed 11 May 2022 11. Monserrate SG (2022) The cloud is material: on the environmental impacts of computation and data storage. MIT case studies in social and ethical responsibilities of computing. https:// doi.org/10.21428/2c646de5.031d4553 12. Bridle J (2018) New dark age: technology and the end of the future. Verso, UK 13. Suski P, Pohl J, Frick V (2020) All you can stream: investigating the role of user behavior for greenhouse gas intensity of video streaming. In: Proceedings of the 7th international conference on ICT for sustainability (ICT4S2020). Association for Computing Machinery, New York, pp 128–138. https://doi.org/10.1145/3401335.3401709

274

S. Rafael and M. Reis

14. Suckling J, Lee J (2015) Redefining scope: the true environmental impact of smartphones? Int J Life Cycle Assess 20:1181–1196. https://doi.org/10.1007/s11367-015-0909-4 15. Fehske A, Fettweis G, Malmodin J, Biczók G (2011) The global footprint of mobile communications: the ecological and economic perspective. IEEE Commun Mag 16. Bawden T. Global warming: data centres to consume three times as much energy in next decade, experts warn, independent. https://www.independent.co.uk/climate-change/news/ global-warming-data-centres-to-consume-three-times-as-much-energy-in-next-decade-exp erts-warn-a6830086.html. Accessed 28 June 2022 17. Elgaaïed-Gambier L, Bertrandias L, Bernard Y (2020) Cutting the internet’s environmental footprint: an analysis of consumers’ self-attribution of responsibility. J Interact Mark 50:120– 135. https://doi.org/10.1016/j.intmar.2020.02.001 18. Alcott B (2005) Jevons’ paradox. Ecol Econ 54(1):9–21. https://doi.org/10.1016/j.ecolecon. 2005.03.020 19. Jones N (2018) How to stop data centres from gobbling up the world’s electricity. Nature, 561:163–166. https://www.nature.com/articles/d41586-018-06610-y 20. Belkhir L, Elmeligi A (2018) Assessing ICT global emissions footprint: trends to 2040 & recommendations. J Clean Prod 177:448–463. https://doi.org/10.1016/j.jclepro.2017.12.239 21. Energuide. Consumption. https://www.energuide.be/en/consumption-and-rates/consumpti on/10/. Accessed 22 May 2022 22. Griffiths S. Why your internet habits are not as clean as you think. BBC Future. https://www.bbc.com/future/article/20200305-why-your-internet-habits-are-not-asclean-as-you-think. Accessed 11 July 2022 23. Mills E, Bourassa N, Rainer L, Mai J, Shehabi A, Mills N (2019) Toward greener gaming: estimating national energy use and energy efficiency potential. Comput Games J 8(3–4):157– 178. https://doi.org/10.1007/s40869-019-00084-2 24. Holthaus E. Bitcoin could cost us our clean-energy future, Grist. https://grist.org/article/bit coin-could-cost-us-our-clean-energy-future/. Accessed 27 July 2022 25. Stoll C, Klaaßen L, Gallersdörfer U (2019) The carbon footprint of bitcoin. Joule 3(7): 1647– 1661.https://doi.org/10.1016/j.joule.2019.05.012 26. Masson-Delmotte V et al (2021) Summary for Policymakers. Climate Change 2021: the physical science basis. contribution of working group I to the sixth assessment report of the intergovernmental panel on climate change. IPCC, Switzerland 27. Jiang P, Fan YV, Klemeš JJ (2021) Impacts of COVID-19 on energy demand and consumption: challenges, lessons and emerging opportunities. Appl Energy 285. https://doi.org/10.1016/j. apenergy.2021.116441 28. Climate change: is your netflix habit bad for the environment? BBC News. https://www.bbc. com/news/technology-45798523. Accessed 03 Mar 2023 29. Bartelmus P (2003) Dematerialization and capital maintenance: two sides of the sustainability coin. Ecol Econ 46(1):61–81 30. Institute of Energy Economics. Japan long-term energy outlook – a projection up to 2030 under environmental constraints and changing energy markets. www.neken.ieej.or.jp. Accessed 09 Feb 2023 31. Microsoft Azure. https://azure.microsoft.com/en-gb/services/cloud-services/. Accessed 12 Feb 2023 32. Williams DR, Thomond P, Mackenzie I (2014) The greenhouse gas abatement potential of enterprise cloud computing. Environ Model Softw 56:6–12 33. Foster ST, Sampson S, Wallin C, Webb S (2016) Managing supply chain and operations. Pearson, NY 34. Gui M, Fasoli M, Carradore R (2017) Digital well-being. Developing a new theoretical tool for media literacy research. Italian J Sociol Educ Padova (2017)

Guidelines for Sustainable Online Behaviors

275

35. Mayers K, Koomey J, Hall R, Bauer M, France C, Webb A (2015) The carbon footprint of games distribution. J Ind Ecol 19(3):402–415. https://doi.org/10.1111/jiec.12181 36. Gnanasekaran V et al (2021) Digital carbon footprint awareness among digital natives: an exploratory study. No. 1. NIK Norsk informatikkonferanse, Norway 37. Mittal B (1999) The advertising of services: meeting the challenge of intangibility. J Serv Res 2(1):98–116. https://doi.org/10.1177/109467059921008 38. Mazaheri E, Richard MO, Laroche M, Ueltschy LC (2014) The influence of culture, emotions, intangibility, and atmospheric cues on online behavior. J Bus Res 67(3):253–259. https://doi. org/10.1016/j.jbusres.2013.05.011 39. Ahn M, Kang J, Hustvedt G (2015) A model of sustainable household technology acceptance: sustainable household technology acceptance model. Int J Consum Stud 40(1):83–91. https:// doi.org/10.1111/ijcs.12217 40. Cerri D, Terzi S (2016) Proposal of a toolset for the improvement of industrial systems’ lifecycle sustainability through the utilization of ICT technologies. Comput Ind 81:47–54. https://doi.org/10.1016/j.compind.2015.09.003 41. Steg L, Vlek C (2009) Encouraging pro-environmental behavior: an integrative review and research agenda. J Environ Psychol 29(3):309–317. https://doi.org/10.1016/j.jenvp.2008. 10.004 42. Hadler M, Haller M (2011) Global activism and nationally driven recycling: the influence of world society and national contexts on public and private environmental behavior. Int Sociol 26(3):315–345. https://doi.org/10.1177/0268580910392258 43. Venkadeshwaran K. Green technology and its effect on the modern world. Jetir. https://www. jetir.org/papers/JETIREW06032.pdf. Accessed 13 Jan 2023 44. Røpke I, Christensen TH (2012) Energy impacts of ICT and Insights from an everyday life perspective. Telematics Inform 29(4):348–361. https://doi.org/10.1016/j.tele.2012.02.001 45. Murugesan S, Gangadharan GR (2012) Harnessing green IT: principles and practices. Wiley, UK 46. Chugh R, Wibowo S, Grandh S (2016) Environmentally sustainable information and communication technology usage: awareness and practices of Indian information and communication technology professionals. J Clean Prod 131:435–446. https://doi.org/10.1016/j.jclepro.2016. 05.004 47. Bose R, Luo X (2011) Integrative framework for assessing firms’ potential to undertake green IT initiatives via virtualization – a theoretical perspective. J Strateg Inf Syst 20(1):38–54. https://doi.org/10.1016/j.jsis.2011.01.003 48. Daly HE (1993) Sustainable development: an impossibility theorem. In: Daly HE, Townsend KN (eds) Valuing the earth. economics, ecology, ethics. The MIT Press, Cambridge

Emotional Design to Create Furniture for Children’s Well-Being Sara Reigoto1 and Liliana Soares1,2,3(B) 1 Instituto Politécnico de Viana do Castelo, Av. Atlântico. 644, 4900-348 Viana Do Castelo,

Portugal [email protected] 2 CIAUD, Research Centre for Architecture, Urbanism and Design, Universidade de Lisboa, Rua Sá Nogueira, Polo Universitário do Alto da Ajuda, 1349-063 Lisboa, Portugal 3 Lisbon School of Architecture, Universidade de Lisboa, Lisboa, Portugal

Abstract. This paper is part of a design research and aims to present furniture solutions to create furniture and spaces that support children’s well-being. Based on the values of Emotional Design, this paper analyzes five case studies of furniture for children on western context, from the 1920s to current reality. This analysis present singular and common considerations, gathering design principles that guide the creative process and validate the designer as an interpreter of his/her reality, considering time, space, and circumstances. The study is based on a methodology framed in three stages - exploratory, generative, and evaluative. With this idea in mind, the authors want to prove that the theoretical and practical aspects analyzed and linked with Portuguese productive sector, can guide this study to design and develop furniture for children’s well-being. The study highlights the limits, the values and the meanings that characterize children’s universe. Keywords: Emotional Design · Furniture for Children · New Product Oriented · Children’s well-being

1 Introduction People live experiences of sensations and knowledge in the relationship they build with material culture. Donald Norman reinforces the importance of emotions in daily decisions, since they have an impact on how we behave and how we think. Transferring this concept to everyday objects, the author presents the three levels of design: Visceral, Behavioral and Reflective, accompanied by a description of each moment of interaction between the individual and the product, emphasizing the idea that “each of the three levels (…) plays its part in shaping your experience” [1]. The article is based on the three levels of emotional design to understand its application in the design and development of furniture for children. In this sense, the text analyzes case studies from the western context from the 1920s to the present day.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 276–286, 2024. https://doi.org/10.1007/978-3-031-47281-7_22

Emotional Design to Create Furniture for Children’s Well-Being

277

2 Methodology The study is based on a methodology framed in exploratory and generative stages. The exploratory stage what exists and has been done to contextualize and achieve a study about the process of producing furniture for children. The generative stage produces new information and knowledge to define a briefing with the interested parties to develop processes that can determine the evolution of the theme and create links between all those involved.

3 Emotional Design for Children’s Well-Being This paper analyzes five case studies for children’s furniture from the western reality, from the 1920s to the present day. The aim is to understand common points to design and develop a future product for children (Table 1). Table 1. Furniture for children analyzed in this paper, from the 1920s to present day. Designer

Product

Time/Place

Marcel Breuer

Ti 3a chair’

1923/Germany

Charles and Ray Eames

Elephant Stool

1950/USA

Enzo Mari

The place of games

1960/Italy

Sena da Silva

Sena Chair

1970/Portugal

Hella Jongerius

Porcupine Desk

2007 Netherlands

3.1 The “Ti 3a” Chair by Marcel Breuer (1920s) The history of the Bauhaus school is intrinsically linked to the historical/social context of the time. A part of the school was still clinging to the past and differing from the thinking of many other individuals determined to find a new way of life [2]. The structure of the Bauhaus teaching program had “the objective of exalting creativity so that art should be born as an act of life, at the same time rescuing artistic making and placing it in a present position in everyday life, inserted in any and all context” [3]. Among the masters who studied at the Bauhaus and who would later occupy a teaching position, Marcel Breuer (1902–1981) stands out. Throughout his work he assumed a rationalist aspect, which is one of the characteristics for which the German school is remembered. However, “in Breuer’s own work, the balance between expressionist fantasy and freedom on the one hand, and the rationalistic discipline and clarity of another, remained a constant feature” [4]. While he was a student at the carpentry workshop, he started designing his first models focused on furniture design. In furniture design for children, Marcel Breuer designed furniture such as the Ti 3a chair in 1923. This equipment was born in the carpentry workshop of the Bauhaus

278

S. Reigoto and L. Soares

school in Weimar in three different dimensions, including one for adults. Like many other models that he designed until 1925, he was influenced by the De Stijl group, which, despite the question of his role in the Bauhaus being controversial on the part of some masters of the school, “As for Breuer; however, the De Stijl influence on his furniture is too strong and too specific to be denied” [5], and the designs of Gerrit Rietveld (1888–1964) can be highlighted as having an important role in his work. Regarding the visceral level, one notices the designer’s desire to connote the furniture with a palette of bright colors that come from the legacy acquired at the Bauhaus, where, under the direction of Walter Gropius, priority was given to “(…) the basic shapes (squares, triangles, and circles) and the primary colors (red, yellow, and blue)” [6]. Parallel to the influence that the De Stijl group imposed on the shape of their products, the application of color also has a relationship with what Theo van Doesburg considered to be the development of mechanical aesthetics, using only primary colors and noncolors (black, white, and grey). The emphasis that Marcel Breuer gave to color was also transversal to his architectural works, where, contrary to what was common at the time, he applied surfaces in bright red and blue to his buildings [6]. As for the behavioral level, it is important to mention that plywood was used in its construction for the seat and back, which subsequently underwent a lacquering process. This feature is pertinent since, unlike some chair models previously developed, such as the Armchair in 1922, where it combined padded materials to ensure the user’s comfort, the Ti 3a model only has rectangular plates at the points where the individual would be directly in contact. This perspective denotes that “(…) the support chairs seemed more manifests of a certain artistic sensibility than carefully thought-out chairs, designed to sit comfortably” [5]. However, certain models have some dysfunctions such as the type of joints that are not adequate, “these initial errors are largely overcome by the general clarity of the structural concepts, the absence of ornamentation and the emphasis on fine finishing (…)” [4] previously prominent in the work of Marcel Breuer. Finally, on a reflective level, it is possible to interpret this product as the beginning of a study for future projections of children’s equipment, which would later also be applied in their tubular steel models. The Ti 3a chair together with the simple side table were produced in series in the school’s carpentry workshop, where “(…) they received several commissions to equip kindergartens (Kindergartens) and houses for children (…)” [7]. It is also noteworthy that among many of his works, the models did not vary between adult and children’s designs, the only distinction being their dimensions. This choice of projection reflects what Walter Gropius intended to apply to workshop products from 1922 onwards, “(…) each design should only consist of a few simple pieces so that it could be easily adapted to industrial production (…)” and by combining them “(…) [6] these designs also intended to be adapted to variation” thus becoming characterized as standard types. 3.2 The “Elephant Stool” by Charles & Ray Eames (1940s) In the 1950s Charles Eames (1907–1978) reflected his concern to respond to the social needs of the United States of America, considering that design was influenced by the socioeconomic environment of the time. The importance of recognizing problems and subsequently presenting solutions that embraced innovative technological concepts was

Emotional Design to Create Furniture for Children’s Well-Being

279

a milestone for the beginning of what would become “(…) a pioneering aesthetic for modern life” [8]. In this scenario, Charles, and Ray Eames (1912–1988) stands out through the application of new techniques and materials such as plastic and plywood, they presented new approaches to home furnishing. The house should not have too many demands, but it should function as a support for working life [9]. Located in Los Angeles in 1941, it began the development of the plywood molding process, the result of the collaboration between Charles Eames and Eero Saarinen. Along the way, they quickly realized that “the creative use of this industrial material, the Eameses sought a strong, flexible product capable of taking on myriad shapes and forms” [10]. Among a variety of proposals, products aimed at children are also included. The prototypes in this set, worked using the molding technique, mostly feature animal morphologies such as the bear, the horse, the frog, the seal, and the elephant. Regarding the visceral level, the physiognomy they attribute to the pieces is one of the characteristics that carry the weight of creating a positive visceral moment. In the case of the Elephant Stool, created in 1945, its shape evokes the recognition and nostalgia of the animal that, “admired for their majestic size and gentle nature, elephants are part of our everyday experience as animated characters, storybook figures, and children’s toys” [11]. More than that, it also highlights the connection that the elephant has with the couple, since it is often present in Charles Eames’ photographic documentation relating to Indian culture and the world of the circus. At the behavioral level, while three of the tests were produced in metal, the Elephant Stool stands out as the only one that remained entirely in plywood. Due to its morphology composed of sharp curves, it proved to be the most difficult to execute in technical terms, consequently, the prototype did not reach series production at the time. The design of the pieces was thought to allow a variety of interactions, starting from their sitting function, and allowing children to crawl underneath or even stack several models [11]. Finally, on a reflective level, evidencing the importance that Charles and Ray Eames attribute to connections, details that, introduced correctly, bring the object to life, Ralph Caplan indicates that the value of these toys is beyond the ideal of immediate reward and values which in educational terms may be the most important principle of all, the connection with the user, since “(…) are never prescriptive play products; they are invitations to connect” [12] and the child has the opportunity to absorb the pleasures of discovery. Another feature valued in their designs is based on the thought that toys have a certain meaning in view of the culture in which they are inserted. In the short film they made in 1969 called “Tops”, they put in the foreground a toy that cuts across cultures and eras, the spinning top. Returning to the principles of Donald Norman’s Emotional Design, it is possible to analyze this film as having the ability to represent the universality of a child icon, which, accompanied with the music composed by Elmer Bernstein, reflects the diversity of visceral, behavioral, and reflective values of different cultures. Currently, the toy is available in plastic in a wide range of colors, thus contributing to a greater attraction on the part of the public for which it is intended. The use of a more durable material also allows the product to be used in new environments, namely an outdoor context, not limiting the space to play and, consequently, the child’s imagination [13].

280

S. Reigoto and L. Soares

3.3 “the Place of Games” by Enzo Mary (1960s) The 1960s were marked by the growth of the consumerist society. Opposed to this marketing domain, Enzo Mari promoted the idea of creating for people, stressing the importance of “(…) free things of their mere value as commodities (…)” [14], and worrying about what they can communicate. From this perspective, Enzo Mari presents the concept of Autoprogettazione (Self-projection), removing the passive character from the user, who now becomes responsible for the process and function of the objects he uses. In the manual he developed in 1974, he proposes to the consumer the elaboration of an individual exercise, through the construction of his own pieces using wooden boards and nails. The final product, in addition to its usability, conveys a much more important meaning, its educational value. This concern that an increasingly wide audience had access to design, with the intention that everyone freely create or use their products, was always present throughout their work [15]. For Enzo Mari, knowing how to design for children means an evolutionary action, since “(…) from two months up to one year we have one situation, from one year to two and a half years we have a completely different situation, and so on” [16]. Consequently, the products cannot be the subject of a generalization. This thought is a criticism of what the play space has become, an environment full of objects imposed on children without having any meaning or connection, and often characterized as being childish in the pejorative sense of the word. The way to combat this reality is to understand that children enjoy playing as a way of exploring the world and should be surrounded by objects that promote their development [16]. Among the products he developed for children, “The place of games” designed in the 1967 and produced by Danese, stands out. It is a representation of an object that privileges free use of the product. At the visceral level, combined with the imaginary, Enzo Mari claims that “toys to be freeing, not limiting, without putting dampers on the possibilities of play” [14], such as the sun, waves, and bricks, since they value a language with a certain level of abstractionism, making them recognizable, but at the same time stimulating the user’s imagination. Likewise, the use of primary and secondary colors that is present in various children’s products and equipment, as highlighted by Donald Norman, are intrinsic characteristics of the first level of interaction. Considering Enzo Mari knowledge about visual language, the combination of shapes and colors “(…) allow children to organize their own space. The prints and cutouts evoke recognizable elements, allowing children multiple associations and projections (…)” [14]. At the behavioral level, it is possible to verify that the object has a character closer to a place than an object itself, allowing the child to organize his play space and transport it easily. Composed of 10 cardboard panels that reach 3 m in length and 90 cm in height, it reveals the author’s care for child anthropometry, favoring the function of use. However, the function of the product is not predisposed since it is expressed as being an object of free use. Thus, each user gains freedom to modify the behavior of the panels according to their cognitive and emotional values [14]. For the reflective level, Enzo Mari based himself on the essence of the metamorphosis that classifies the play space, referring that it is pertinent to interpret it as an identity in constant transformation and not static. Taking advantage of this knowledge

Emotional Design to Create Furniture for Children’s Well-Being

281

and combining it with the relationship that children have with tales and stories, Enzo Mari asked the question, “Why not let children tell themselves their own story?” [16]. This thought gave rise to “(…) toys to be freeing, not limiting, without putting dampers on the possibilities of play” [16] and whose function and meaning varied depending on the user. On the other hand, when using the card as a basis for the project, “(…) wich challenged the idea of associating these materials with objects of immediate consumption” [17], reformulating the disposable concept that was related to lack of quality and little environmental awareness. The application of the card opened the possibility of using didactic, malleable, and sustainable methods of construction and use, which made it possible to create a canvas that could be molded to the children’s imagination, with the ability to transform into different scenarios and add different stories [17]. 3.4 “Sena” Chair by António Sena Da Silva (1970s) The transformations evident in furniture design during the 1960s and 1970s contributed to the recognition of the discipline and its professionals in Portuguese territory. Despite this evidence being a consequence of antecedents in different historical moments, it is noteworthy that it was in the 50’s that the social formation regarding the discipline of Design began. This period is marked by the concept of “total design”, which values the global projection of the environment, as well as the adoption of a new modernism in the face of the consumption demands of the time. “(…) The object needs to be understood not only for its utilitarian side, but also for its aesthetics, and the industry has to respond to that purpose” [18]. To encourage quality and industrial modernization, in 1959 the Estado Novo created the National Institute of Industrial Research, with the purpose of structuring industries based on improving the conditions of the factory space for mass production and always focusing on quality control. The favorable and pertinent contextualization of the previous decade allowed the slow start of the process of generalization of design recognition in the 1960s with the appearance of architecture studios, industry, and educational institutions. One of the greatest references in this context was Frederico George (1915–1994), who, while teaching at the António Arroio School of Decorative Arts, promoted the introduction of the concept of design in several curricular units, influenced by the legacy of the Bauhaus, on which “it formed a generation of disciples with whom he developed the concept of Design as a globalizing practice (…)” [19], recovering the modernist values of the German school. Among this group, António Sena da Silva (1926–2001) and Daciano da Costa (1930–2005) stand out, who had a predominant role in the recognition of the discipline and its connection with the industry, whose partnership culminated in several equipment projects, working both in public and private spaces. This article highlights the “School Module” design, which began in 1962 and was born from the collaboration of António Sena da Silva and Leonor Álvares de Oliveira. With the aim of presenting new solutions to the “(…) use compatible with the dynamics of group work and the optimization of postures for writing and reading” [20], António Sena da Silva recognizes the importance of designing furniture for children. On a visceral level, considering the designation ‘Sena-Paint it yourself’ that accompanied the production of the piece, it is plausible that its finish ended in the sanding process, allowing the user to later apply the desired color. This freedom to choose the

282

S. Reigoto and L. Soares

final appearance of the piece, “giving you the right to intervene in your own implementation: leaving open the alternative between maintaining the texture of the wood, enhanced or not by the use of varnish, but very right to the use of color, with everything that this allows in the domain of an affective approximation to the piece” [20], strengthens the relationship between the final product and the user. At the behavioral level, Sena da Silva sought to integrate anthropometric and ergonomic values into his design, combined with the importance of resistance in the final product. Through an evolutionary path marked by experimentation with previously developed prototypes, António Sena da Silva, and Leonor Álvares de Oliveira maintained their constant concern for ergonomics and for achieving a result pedagogically suited to the needs of children. Lumbar support being a concern in the design of the chair, the back has a concavity that promoted a correct posture. This accentuated shape was achieved by gluing wooden sheets that subsequently underwent a cold pressing process. The tola wood chair, which went into series production in 1972 by Olaio, had the particularity of being a stackable model, which allowed for the optimization of transport and storage. It is important to point out that the model presents a second version produced by FOC in 1989, which, “(…) despite being identical in appearance, the almost imperceptible changes introduced by the authors, both in terms of shape and construction, make it impossible to consider them as a single piece” [20]. Through the analysis of the target audience, it was observed that they tend to recline when seated, and that this action could compromise the stability and union between the uprights and the transverse beam. In this way, the changes in the new version point to an effort to try to eliminate this fragility, with the inclination of the uprights forward, a more pronounced concavity in the seat and backrest and the introduction of a new horizontal crosspiece. The system also presents changes. The 1989 version has a hole and tenon joint, while the previous version used dowels [20]. Finally, the reflective level senses that this project, which was born from the collaboration of a multidisciplinary team with the presence of furniture design technicians, professionals in the field of orthopedics, pedagogues and manufacturers, sought to meet the requirements of the proposal by producing equipment “(…) marked by rationality, functionality and efficiency, combined with a remarkable constructive simplicity (…)” [21], of which the Sena chair gains an iconic highlight in the history of Portuguese design. Effectively, there is a desire on the part of the project team to assert the right of children to be individuals at a crucial moment in their development, such as the school period. On the other hand, this project developed for Portuguese society with the national industry contributed to business sustainability, and once again created a link between the act of designing and the productive environment, responding to a social need at the time. 3.5 “Porcupine Desk” by Hella Jongerius (2007) Hella Jongerius’s work is characterized by the contrasts she presents in her work, which includes industrial and artisanal, contemporary, and traditional aspects, combining them with a mix of old and recent technologies. The beginning of her training as a designer is related to weaving and knitting techniques and their relationship to the industrial environment. “The new design collective – meaning “dry” in Dutch –was all about the

Emotional Design to Create Furniture for Children’s Well-Being

283

freedom of exploring different concepts and rethinking the consumerist norm” [22] is a hallmark of Droog Design, which she joined in 1993. Later, she ended up establishing her studio where, by rethinking the essence of objects, she continues to explore different concepts of materials, colors, and textures. Throughout her career she has been part of several partnerships, of which her work with Vitra stands out, for which she developed the “Library of Color and Materials” with the intention of offering a resource that helps to reconcile the visual and functional aspects of the products, offering a diverse combination of materials and colors [23]. When designing for children, she recognizes that exploration is a consequence of children’s incessant curiosity, and it is opportune to interpret “(…) this make the design elements of the furniture, which are the mediums for exploring the environment, important for both sensorial and emotional development (…)” [24] and with the ability to foster your creativity. Taking advantage of the discovery process, this study analyzes the Porcupine Desk produced in 2007 in partnership with Vitra. In this case, the visceral aspect is accentuated by the shape of the table, which seeks references in the natural world. Particularly in the fauna, removing characteristic elements of the animal and transporting them to a different environment, it values the asymmetry marked by the inclination of two of the legs and the combination between the beech wood and the painted component [25]. In addition to these structural components, illustrative elements were introduced on the top of the table using the silkscreen technique. This type of representations, which can vary between toys, characters, natural elements, among others, “have an important role in increasing the child’s interest in exploration” [25] by providing a greater connection between the individual and the object. Particularly in the design of the Porcupine Desk, it is possible to observe the integration of various illustrations of personal projects and other products from the Vitra collection. United by the representation of a bracelet, the pendants represent the Elephant Pad developed by Hella Jongerius in 2010 and inspired by the ‘Panton chair’ (1959) by Verner Panton and the ‘Eames House Bird’ (1930). Also present is the product developed in 1993, Soft Urn, recognized as being one of the examples of Dutch conceptual design, such as the Delft Bue B-Jug (2001), which is born from the previous collection called B-set, where highlight the irregularities of the pieces to avoid the ideal shape. Its functionality, which is inserted at the behavioral level, is fulfilled by providing the user with a work and leisure space enhanced by its didactic and interactive features. Since the table has several sections to store material, it also offers its own environment, where the individual can store all the equipment that corresponds to this activity. Usability, which Donald Norman [1] points out as one of the main values of the second level of interaction, is verified in the projection of holes to store pens and markers. In addition to its visceral and reflective aspect, which seeks to create assimilations with the reference animal, it is also a feature designed from a behavioral point of view, where the diameter of the holes and their depth validate the function for which they were designed. As for the reflective level, the incorporation of holes on the top of the table to place pens, aims to add value to the relationship between the shape of the desk and the Object, resembling the silhouette of the table to the characteristics of the porcupine. Although the animal’s physiognomy is not directly represented, the projection of this

284

S. Reigoto and L. Soares

element invokes its shape, encouraging children to associate a different environment with a piece of furniture. Another feature of the reflective analysis of the product is one of the illustrations used on the desktop. Among the others is a heart that, together with the description that “the decoration on the top of the table is similar to a girl’s beaded bracelet” [25], allows us to invoke the interpretation of the object as being a jewel of design, resembling the bracelets from an iconic jewelry company, Tiffany & Co (Table 2). Table 2. Emotional design in the creation of five furnitures for children’s well-being Designer

Visceral level

Behavioral level

Reflective level

Ti 3a chair

Color Specific finish, Morphology of the pieces

Ergonomics and anthropometric solutions Easy transport and storage

Product oriented to children needs, desires and imagination

Elephant Stool The place of games Sena Chair Porcupine Desk

4 Conclusions Based on the values of Emotional Design, this paper presented furniture solutions for the creation of spaces that support children’s well-being. The experimental feature is a common characteristic of the case studies analyzed. Based on an exploratory nature, they seek to present innovative solutions both in terms of shape and technique, sometimes using materials that do not fit consumerist thinking. The role that furniture plays as a means of responding to social and cultural needs is also evident, presenting new effective, comfortable, and safe proposals for the public and environment for which it was designed. The physical space and the elements that compose it have an important role in the way the person acts and feels. In particularly, in this paper, it is essential to recognize children’s needs as emotional value can be communicated by furniture. Therefore, this study was focused on the three levels of Donald Norman’s Emotional Design, in favor of the research that allows linking ideas in a later phase of the study. From the visceral level, the case studies take a transversal care with the finishing of the pieces. With Marcel Breuer the choice of color was guided by what is considered the Bauhaus visual language. For Enzo Mari, the illustrations and saturated colors reflect his knowledge of children’s preferences. For António Sena da Silva, the choice not to apply a specific finish to the chair is interpreted as a way of bringing the object closer to the individual who uses it, allowing him to determine the final appearance of his product. The morphology of the pieces is also an opportunity to create a positive visceral moment, as evidenced by the case study of the Eames couple and the secretary of Hella Jongerius, where, through the transfer of values from the animal world to the furniture sector, they manage to create associations and concepts with the public.

Emotional Design to Create Furniture for Children’s Well-Being

285

At the behavioral level, ergonomic concerns and the dimensions applied in designs highlight the importance of thinking about the object in terms of the individuals who use it and the way they behave. The ‘Ti 3a chair’, the ‘Elephant Stool’, ‘The place of games’, the ‘Sena chair’ and the ‘Porcupine Desk’ are examples that reflect these values, presenting themselves as anthropometric solutions that focus on the relationship between the human being and the object. The easy transport and storage due to the stackable form of some of the designs contributes beyond their use, as the behavior of the piece when it is not in use was considered. As for the reflective level, the will to leave free use of the product and not limit children’s imagination is a virtue in Enzo Mari’s project. With the Porcupine Desk by Hella Jongerius, and the Elephant Stool by Eames, associations are born between the physical characteristics of the object and the animals that inspired them, also allowing the user to embark on an exploratory journey. This rationalization also shows the social nature of Marcel Breuer’s Ti 3a chair and Sena chair, as they were intended to respond to the needs of their time. Finally, the authors want to prove that Emotional Design becomes a way of analyzing products, proving that they can, in fact, be thought of as a means of exploring emotions and shaping the different levels, the communication, the use and the meaning. The next step will be to apply this knowledge in the design and development of furniture for children in a public school in Portugal. Acknowledgments. The authors would like to acknowledge the Instituto Politécnico de Viana do Castelo.

References 1. Norman D (2004) Emotional design. Basic Books, New York 2. Bayer H (1938) Bauhaus, 1919–1928. The Museum of Modern Art, New York 3. Paschoarelli L et al (2018) Bauhaus: métodos de ensino em Weimar, Dessau e Berlim. Convergências - Revista de Investigação e Ensino de Artes. http://convergencias.esart.ipcb.pt/? p=article&id=192 4. Blake P (1949) Marcel Breuer architect and designer. Museum of Modern Art, New York 5. Wilk C (1981) Marcel Breuer furniture and interiors. The Museum of Modern Art, New York 6. Droste M (2007) Bauhaus 1919-1933. TASCHEN, Koln 7. Rowland A (1997) Bauhaus source book. Grange Books, Londres 8. Normandin K (2012) Charles and Ray Eames: modern living in a postwar era. Docomomo, 46:23–27 9. Koenig G (2007) Charles and Ray Eames. TASCHEN, Koln 10. LNCS Homepage. https://www.metmuseum.org/toah/hd/eame/hd_eame.htm. Accessed 03 Apr 2023 11. LNCS Homepage. https://www.eamesoffice.com/the-work/eames-elephant/. Accessed 03 Apr 2023 12. Caplan R (1976) Connection: the work of charles and ray eames. Opposite: Eames Office, Los Angeles 13. LNCS Homepage. https://www.vitra.com/en-us/product/eames-elephant. Accessed 03 Apr 2023

286

S. Reigoto and L. Soares

14. LNCS Homepage. https://www.domusweb.it/it/dall-archivio/2012/08/11/il-metodo-di-enzomari.html. Accessed 03 Apr 2023 15. LNCS Homepage. https://blog.archtrends.com/enzo-mari/. Accessed 03 Apr 2023 16. Small T (2009–2010) Mari, Enzo. Interview by Tim Small. Apartamento Magazine n.4 17. LNCS Homepage. https://www.italianarchitecture.org/architecture/enzo-mari-il-posto-deigiochi-and-cardboard-shelving-system-p8krk. Accessed 03 Apr 2023 18. Pombo F, Cruz R (2018) O impacto da “geração intercalar” no design de mobiliário de escritório em portugal. A inovação das décadas de 60 e 70. Res Mobilis - Revista internacional de investigación en mobiliario y objetos decorativos 7(8):118–138. https://doi.org/10.17811/ rm.7.8.2018.118-138 19. Tostões A, Martins JP (2000) A construção do Design em Portugal: de 1960 à Revolução de Abril de 1974. O tempo do Design. Anuário do Centro Português do Design, 21–22:64 20. Pedroso G (2017) A cadeira Sena – Na Senda do Assento Escolar Ideal. Design, Fashion and Product, Edições IPCB, Castelo Branco, 255–260 21. Santos RA (2001) Morreu Sena da Silva, uma voz do ‘design’ em português. Público, Ípsilon 22. LNCS Homepage, https://designdefender.com/designers/hella-jongerius-queen-of-colour/, last accessed 2023/04/03 23. LNCS Homepage, https://www.vitra.com/en-pt/about-vitra/designer/details/hella-jongerius, last accessed 2023/04/03 24. Soyupak, I.: Kid’s Furniture as a medium of exploration. Marmara University Fine Arts Institutes Interior Design. Ankara 25. LNCS Homepage, http://www.jongeriuslab.com/work/porcupine-desk, last accessed 2023/04/03

Analysis of Cyberaggression in Social Networks Involving Students and University Environments Eliza Oliveira1(B)

, Lídia Oliveira1

, and Vania Baldi2

1 Digital Media and Interaction Research Centre, University of Aveiro, Aveiro, Portugal

{elizaoliveira,lidia}@ua.pt

2 Iscte - University Institute of Lisbon, Lisbon, Portugal

[email protected]

Abstract. Online aggressions have been widely reported worldwide and referenced as great harm since they bring immeasurable negative repercussions to those involved. Although social media websites’ official policy prohibits hate speech, the literature shows that aggressive content is still noticeable among the comments and disseminated publications in many online social networks. Meanwhile, the aggressive posts’ contents have yet to be explored in detail, including those associated with Portuguese social media groups. This paper aims to provide information on how aggression translates and is expressed online, considering two sources. The first sample is taken from eight transcribed interviews with Portuguese university students who have already been involved in aggressive online situations and shared the exact text content of the attacks. The second source was a series of posts from 151 Portuguese university student open groups on Facebook and Instagram that presented aggressive comments. A content analysis was performed with both samples separated for comparison, seeking similarities and patterns in both aggressive text analyses. Moreover, a thematic analysis was conducted to gather knowledge regarding the most frequent subjects among the aggressive content posted. While communications reported in interviews focus on image and body, death, and gender identity, themes such as immigrants, politics, lack of university support, and academic traditions emerged in the social network. The findings enhance knowledge regarding linguistics on cyberaggression and provide parameters for future research on detecting aggression situations online. Keywords: Cyberaggression in Higher Education · Labeled Aggressive Content · Social Media · Qualitative Content Analyses · Qualitative Thematic Analyses

1 Introduction As digital technology use has exponentially increased in recent years, human tasks are being massively migrated to the online environment, transforming habits and outlining new forms of communication that manifest in different ways in cyberspace (Gonçalves & Oliveira, 2021). In this context, characteristics such as anonymity propitiate cyberaggressions on the Internet, increasing the likelihood of aggressive online episodes and © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 287–300, 2024. https://doi.org/10.1007/978-3-031-47281-7_23

288

E. Oliveira et al.

often leaving the aggressors unpunished (Young et al., 2018). While cybersecurity incidents have been widely reported worldwide, their impacts have been inexorable for those involved, causing adverse effects in all aspects of life. In this sense, the World Health Organization (WHO, 1998) has considered aggressive online behaviors a severe public health issue. As aggressive online situations occur in the world, Portugal has not been different. While it is difficult to pinpoint the cyberaggression prevalence precisely (Neves, 2016), results of the EU Kids Online study show that the number of young people involved in online aggression tripled from 2014 to 2019 (Ponte & Batista, 2019, p. 31). In addition, 61% of primary, secondary, and higher education Portuguese students (485 students) have been victimized since the confinement (Lusa, 2020). Although social media websites’ official policy prohibits hate speech, the literature shows that aggressive content is still noticeable among the comments and disseminated publications in many online social networks (Ernst et al., 2017; Young et al., 2018). Whereas research across disciplines has these rates of cyberbullying and proposed prevention strategies, there has been comparatively little focus on the discursive content of online aggression. Simultaneously, while university students are the ones who most frequently use digital technologies in Portugal (PORDATA, 2021), they are naturally more exposed to cyberaggression hazards. Hence, research on the practices of this age group may be relevant to identify how aggression manifests itself in online environments (Van Schaik et al., 2017). Regarding studies in literature, in Germany, there were a series of aggressive situations that, according to the investigations by Müller & Schwarz (2017), are correlated with posts published on social networks. In this context, an unknown person painted graffiti on a house where refugees lived. The graffiti read “Deutsch Nantional (German-National, misspelled in original), “18 (code for Adolf Hitler), and “88 (code for Heil Hitler). Ben-David & Matamoros-Fernández (2016) found words about aggression against Muslims and the Islamic religion on the Facebook pages of far-right Spanish political parties. Ernst et al. (2017) applied a qualitative content analysis on videos and samples of comments randomly selected on YouTube videos, finding words that show discrimination against Muslims and Islam. Further, Hosseinmardi et al. (2015) aimed to study incidents of cyberbullying on Instagram and conclude that, within the framework of the study, the categories “body” (e.g., face, clothing) and “sexual” (e.g., slut - slut, rapist - rapist) were the most common among cyberaggressive content. Young et al. (2018) detected aggressive content that includes “Nobody likes you,” “hated,” or “couldn’t stand” towards the victim, as well as statements about sexual behavior such as “whore” and “slut”. To our knowledge, there is no study until this data that presents a precise analysis of aggressive online communication regarding Portuguese social media content. To fulfill this gap, the main goal of this paper is to present and analyze aggressive Portuguese communication online from two different sources. The first source was the data collected from eight interviews with Portuguese university students. The second came from analyzing 151 groups in the social networks related to the university environment. This document emerges from a study that aims to identify the languages that characterize aggressive content in social media university groups in the Portuguese context, intending to acquire information on how aggression is shaped online, providing

Analysis of Cyberaggression in Social Networks Involving Students

289

parameters for future research on the detection of cyberaggression situations. The following research question arises: What languages characterize aggressive discourse in Portuguese social media groups related to the university context? The following section presents the methodology used in this study, while the Sect. 3 presents the main results gathered from the interviews and social media group analyses. The Sect. 4 intends to present the discussion, with implications for developing anti-cyber aggressive initiatives that promote effective online behavior. The Sects. 4 and 5 presents the most relevant conclusions.

2 Methodologies This study was realized in two complementary phases. The first is the interview with university students, and the second is the search for aggressive content in the open social media group. The interviews and group analysis methodologies will be presented separately in this section to clarify the data. In this sense, the procedure and dynamization of the interviews will be presented first, followed by the social group scan methodology. Subsequently, the data processing and analyses will be presented, considering thematic and content analysis as the two main strategies used in this work to identify the subjects, ways of aggression manifestation, and words related to the aggressive situation in the online Portuguese environment, respectively. 2.1 Interview Methodology The interviews aimed to understand involvement in cyberaggressions, identify the causes and impacts of online aggression, the students’ precautionary behavior, and identify the words that characterize aggressive content in social media groups in the Portuguese context. The data collection was carried out only after signing a consent form prepared within the scope of the investigation, which was presented in digital format, through a link, or by reading a QR code. The interview was conducted with eight students, four of whom were realized online through the Zoom Platform, and four were carried out in person at the facilities of the Digital Media and Interaction Research Centre at the University of Aveiro. All participants have been previously enrolled in cyberaggression situations as targets, aggressors, or/and bystanders. Students from three Portuguese universities participated (University of Coimbra, University of Aveiro, and University of Porto). All the measures related to Covid-19 were considered. It was emphasized that participants could refuse to answer questions and withdraw from participating at any time. The interviews conducted online were recorded on video and using an audio recorder. The video recording ensured the visualization of the participant’s behavior patterns, facial expressions, and other information beyond speech to express all the information that a semi-directive interview allows. Face-toface interviews were recorded using an audio recorder. The interviews lasted an average of two hours and were carried out between April and July 2022. The interviews were transcribed manually into digital documents in September 2022.

290

E. Oliveira et al.

2.2 Group Analysis Methodology A search was carried out in the open social network groups of the Academic Associations and Student Nuclei of the universities of Aveiro, Coimbra and Porto. The manual search was carried out on the social networks Facebook and Instagram, considering that it is in these social networks that the Academic Associations and Student Nuclei of the universities have their groups. In addition, after the events at the University of Aveiro, in which a professor from the Department of Physics was denounced for hate speech and discrimination against the LGBT group through social networks by students, an analysis was also made of the post related to this topic on the official page of the University of Aveiro on Facebook and Instagram. As these are open groups, approval from members is not required. The manual search for these contents occurred after the transcription of the interviews in October 2022 and lasted four weeks. In this context, the search was restricted to content published from April 1st (the start date of empirical data collection) until October 2022. This study obtained approval from the Ethics and Deontology Committee of the University of Aveiro (No. 14-CED/20221 ). Therefore, the data protection rules were obeyed since all data were treated anonymously and the aggressive publications will not be exposed. Only the content and thematic analyses of the corpus gathered from the aggressive publications will be presented. The scan was carried out in 151 open groups on the social networks Facebook and Instagram, with the most significant number being from the University of Aveiro, followed by the University of Coimbra and Porto. Altogether were 80 Nuclei and Academic Associations of the Universities. It should be noted that some Academic Centers did not have pages on Instagram and/or Facebook at the time the scan was carried out, so the number of groups observed in the two social networks does not reflect the exact amount of these representative institutions of university students. However, all Centers with pages on the previously mentioned social networks were analyzed. Furthermore, while the Universities of Aveiro and Coimbra have Academic Nuclei representing the students of the different degree courses and a General Association of Students, the University of Porto only has Academic Associations which represent each of the University colleges. The posts and comments related to the aggressive material were extracted from Facebook and Instagram using the print tool of the browser, transforming the posts and comments into a PDF format to be scrutinized in NVivo 12 software. 2.3 Content and Thematic Analysis of the Aggressive Content Thematic Analysis. The interviews were analyzed through a thematic analysis using Excel and NVivo 12 software. In this sense, when the interview guide was being constructed, the main categories of analysis were already defined, which directed the systematic coding of the analysis corpus, giving rise to the central themes that emerged from the interviews. Such categories are related to the interviews’ objectives that guided the topics for this study phase. However, through an inductive method, while the corpus was being read, other relevant themes emerged, giving rise to other important subjects 1 The approval of the Ethics Committee can be found at the following link: https://zenodo.org/

record/8199478.

Analysis of Cyberaggression in Social Networks Involving Students

291

to be analyzed. Specifically, these secondary themes are related to the relevant characterization of the aggressive language in terms of the subjects they are related to and how aggression is manifested (e.g., offenses, threats). Equally, thematic analysis proved efficient in translating the ways of manifestation and subjects linked to the aggressive behavior of university students in social networks. In this phase, throughout an inductive reference encoding process, the corpus taken from the social network groups was carefully codified in codes that identify and translate the themes, in this case, subjects and ways that aggression manifests online. The thematic analysis carried out in this work followed the Braun & Clarke (2006) method. The authors systematize the thematic analysis process into steps. In the first phase, the authors transcribe the data, read and reread them, recording the initial ideas. Subsequently, the initial codes are created, and the corpus elements are codified. The themes are identified after this step, grouping codes in the same theme. Then, the analysis is carried out, redefining the specificities of each theme and creating clear definitions for each theme (Gonçalves et al., 2021). Finally, the last phase encompasses the production and writing of the text and corrections. In the context of the present study, after first reading the transcribed interviews and the extracted posts, an initial record was made on the topics that initially stood out, such as gender identity, immigration, and aggression against women. Records on observed trends relative to groups from each university were also made. Then, the aggressive comments were labeled according to the theme in which they were inserted. Some comments were labeled by one or more overlapping themes that emerged from the analysis of the text. For instance, an aggressive comment could be inserted in both themes related to immigration and gender identity. Content Data Analysis. In order to carry out the analysis of the aggressive language reported by the interviewees, as well as the posts with aggressive content found in the open groups of Instagram and Facebook, it was defined that the content analysis strategies would be the most adequate to the objectives of this work. Bearing in mind that it is a question of identifying which the languages that characterize the aggressive discourse in Portuguese social media, the content analysis is efficient as it presents itself as “a set of communication analysis techniques” (Bardin, 2009, p. 37), which allows the calculation of the frequencies of encrypted elements, obtaining concepts or categories that describe a phenomenon based on a system or map of categories, concluded by inference and deduction (Costa & Amado, 2018; Bardin, 2009; Gonçalves et al., 2021). Furthermore, content analysis goes beyond spoken speeches and is also used to analyze the environment and context characteristics that portray how everyday life works (Bardin, 2009). In the present case, the content analysis allowed the extraction and calculation of the frequency of the words that represent aggressive discourse in Portuguese social media, identifying, in a systematized table, the most common ones. Hence, to identify the frequency of words representing aggressive speech collected through the interviews, the participants were asked which phrases were used in the aggressive situations they experienced. These sentences were transcribed and presented as a corpus of the interview to be analyzed. Afterward, the aggressive languages and communications were coded in NVivo 12, and then, the restricted part of the interviews,

292

E. Oliveira et al.

which is related to the aggressive content reproduced by the students, was used to calculate the word frequency. Concerning the aggressive content found in the social media groups, the comments (corpus of analyses) were also codified, and the word frequency was also calculated using NVivo 12. Some vernaculars were added to the list of prohibited words in the frequency query in the coded text, such as the following: it, was, for, thing, was, now, in, that, does, where, think, then, among others. In addition, the “derived words” option was activated so that similar words were not considered in the results. Thus, words like think, thought, and thought were considered derived words that were excluded from the query about frequent aggressive vernaculars in coded texts. Likewise, derived words were considered in the list of frequent hostile vernaculars (e.g., sandpaper, sanded, garbage; homophobe, homophobic, homophobia). In this way, it is possible to have a more comprehensive result of similar words that are present in the aggressive language in Portuguese university social network groups, which would not be possible to identify without using the “derived words” functionality in NVivo.

3 Results The results obtained through interviews and analysis of Portuguese university students’ open groups on social media will be presented separately in this section. Thus, firstly, a characterization of the participants will be made, and subsequently, the results of the thematic and content analyses carried out will be presented. Next, the thematic and content analysis from social media groups’ contents will be presented. 3.1 Interviews Participants. Concerning gender, five are women, and three are men, evidencing a female majority among those who agreed to participate in the interview. Furthermore, all participants are Portuguese and Caucasian. Regarding the university, five participants are from the Aveiro, one participant studies at the University of Porto, and two are from the University of Coimbra. Students varied between different areas of knowledge, and the diversification in the academic years shows that the sample contains individuals with different levels of familiarity with the academic context. Two students are 20 years old, three are 22 years old, two are 23 years old, and one is 37 years old. In this last case, the student interrupted his studies for a significant time, which he did not mention in the interview, having returned recently. The reason for the interruption was also not mentioned. All participants involved in cyberaggression. Half participated more than once in cyberaggression situations and presented multiple roles. In this context, six of the students were victims. One student declared himself an aggressor, and four people reported having observed situations of aggression online. The cyberaggressions mainly occurred via Facebook, Twitter, and Instagram, with WhatsApp, mobile phone, and TikTok being less referenced. Also, the reported cyberaggression was related to primary, secondary, and university levels across different educational levels.

Analysis of Cyberaggression in Social Networks Involving Students

293

Codes from Thematic Analysis of the Interviews. As already mentioned, the main categories of analysis resulted from a deductive process related to the specific objectives. Following the interview objectives, these primary themes are participation, causes and impacts, precautionary behavior, and aggressive online words. Further, through an inductive process, other codes were identified, translating the themes that reflect the most common subjects related to cyberaggressive content and how aggression manifests in online social networks. Therefore, the inductive process in the interview analyzes words that characterize aggressive content in the Portuguese context. An emphasis will be given to the outcomes obtained in the category “aggressive online words” to focus on identifying the most relevant subject related to aggressive content in the Portuguese online settings. In this sense, only an overview of the other categories will be presented. Participation. Concerning the Participation category, the two main themes related to the environment where the cyberaggressions took place were the transposition of aggressions from the digital to the physical world and Social media. However, other themes that emerged but did not stand out include the mobile phone and SMS. With regard to cyberaggression, cyberbullying is the most evident theme among the types of aggression presented by the students. After cyberbullying, the most mentioned and referenced themes were dissing, cyber harassment, hate speech, griefing, sexting, trolling, and identity theft. Other assaults mentioned were online stalking, fake accounts, flaming, and psychological coercion. It is imperative to emphasize that the distribution of situations in different types of aggression was a challenge since the characteristics of the aggressions are transposed. Causes and Impacts. Regarding the specific objective related to the causes and impacts, the most relevant themes presented as the cause for cyberaggressions were game dynamics, image and body (physical stereotypes), gender, a sense of impunity, personal values, and breaking academic tradition. With regard to gender, although only one female participant brings this issue as a cause for cyberaggressions, it is clear in the interviews that people who have suffered constant online bullying at any stage of life are represented, totally, by the female gender. Among the effects of cyberaggressions, those related to mental health were frequently reported by participants. Among the emerging themes in this area are, for example, low self-esteem, fear, discouragement, anxiety, depression, and humiliation, among others. The same participants commonly mention such themes, so the repercussions on mental health often occur together. The impact on friendships, the decreased participation in social networks, and the adverse effects on studies were also common effects, reinforcing that the social and educational components were affected at different levels. Precautionary Behavior. The results in this category suggest that responsibility for cyberaggressions is, in its entirety, linked to the user’s behavior to prevent attacks. Despite this, the codes were divided into general user behaviors and other categories that direct precautions into more specific reported actions. The codes that emerged from the reports are, therefore, the general conduct of the user, ignoring the aggression, abandonment of the aggressive context, self-protection reaction, and verification of technical security settings in Social Media.

294

E. Oliveira et al.

Thematic and Content Analyses of Aggressive Online Words from Interviews Aggressive Online Words. From the analysis of the interviews, four categories appear more frequently and characterize the aggressions as follows: offenses, threats, incentives to suicide, and personal insults. Regarding the most frequent subjects, three main themes emerged: gender identity, aggression against women, physical appearance, and death. Concerning the accounting of the frequency of aggressive words that reflect aggressive expressions, the table below shows the twenty most frequent elements, taking into account the sentences recorded during the interviews. It appears that the most frequent words suggest different aggressive contexts, which are related to situations of exclusion and rejection, incitement to suicide (disappear, tolerate), insults to physical appearance (fat), to family members (mother), to female gender (whore), including other generic insults (shit, garbage, pig, snake). Other words mention body parts, suggesting that aggressions are related to the physical appearance of the victim (armpits, mouth, head). Other words have a negative connotation (inquisition, destroy, evil, money, steal, cancer). This frequency of words alludes to social media groups related to university students and primary and secondary education, which build the (cyber)aggressive grid reported by participants in the interviews. Thus, Table 1 shows an overview of the aggressive words found through a frequency of words calculated using NVivo 12. Table 1. Frequency of words in aggressive speech reported in interviews. Words

Frequency

Words

Frequency

1.Disappear

5

11.Pig

2

2.Whore

3

12.Steal

2

3.Destroy

2

13.Tolerate

2

4.Money

2

14.Armpits

2

5.Fat

2

15.Mouth

1

6.Trash

2

16.Dumb

1

7.Bad

2

17.Head

1

8.Shit

2

18.Cancer

1

9.Mother

2

19.Snake

1

10.Never

2

20.Inquisition

2

3.2 Open Social Media Group Analyses This section presents the thematic and content analyses realized from the social media content. About the open groups on social networks, based on the analysis of the posts on NVivo, no essentially aggressive posts were found in the groups of the Nuclei and Academic Associations of the Universities of Aveiro, Coimbra and Porto. However, 17 posts contained aggressive content reflected in comments through texts, emojis, and

Analysis of Cyberaggression in Social Networks Involving Students

295

images. Such posts will not be presented here to preserve the users’ anonymity. In total, 526 aggressive comments were analyzed, with the University of Aveiro groups having the highest incidence. It is worth highlighting that most of the aggressive comments were taken from the posts referring to the hate speech practiced by the Professor of the Department of Physics at the Aveiro against the LGBT group. Regarding the aggressive comments taken from the social network groups of Academic Associations and Student Nuclei of the universities, subjects related to the traditional university parties that take place at the end of the school year, such as the “Queimas das Fitas” and the “Enterro”, and other relevant political events, such as Freedom Day (April 25th), and the elections of academic groups are highlighted in thematic analysis. These outcomes show that online aggression is also related to cultural traditions, specifically those related to academic contexts. Other recurring themes are immigrants, gender identity, aggression against women, and the lack of institutional support for mental health at the university. Further, considering that most of the comments were taken from the aforementioned hate speech posts, among the 526 aggressive communications, 372 are related to the LGBT group theme, which stands out in this study. The thematic analysis of the corpus from social media groups also shows that aggressive criticism of posted content, pejorative insults, and inflammatory speech is the most common manifestation of aggression. In this analysis, the aggressions are also manifested in sarcasm and irony. Other manifestations, such as accusations, ostracism, and rejection, have fewer encodings among the aggressive comments. Regarding the accounting of the frequency of aggressive words that make up aggressive expressions, Table 2 shows the twenty most frequent elements in aggressive speeches that occur in open social media groups linked to Academic Associations, Academic Nuclei, and the University of Aveiro posts cited earlier. The most frequent word is “shame,” which can be inserted in several speeches that aim to convey the loss of dignity, lack of personal value, humiliation, demotion, dishonor, or ridicule (Infopédia, 2023), which may be associated with different aggressive contexts. In addition, the most frequent aggressive communications refer to the situation of hate speech given by the Professor of physics and, therefore, to the positioning post published by the University of Aveiro (e.g., freedom, hate, speech, Professor, rectory, institution). Other words prove that a significant frequency of aggressive language is linked to contexts related to the LGBT group, consolidating the previously presented information that most of the aggressive communications observed are related to this group (e.g., homophobic and homophobia). However, the other frequent words can easily be inserted in different aggressive contexts (e.g., fear, violence, enough, garbage, blah and discrimination, crime), such as in aggressive communications related to politics, the female gender, immigrants, Queima das Fitas, Enterro week or the lack of institutional support for mental health.

296

E. Oliveira et al. Table 2. Frequency of words in aggressive speech found in the social network groups.

Words

Frequency

Words

Frequency

1.Shame

198

11.Respect

28

2.Freedom

109

12.Homofobia

26

3.Hatred

107

13.Never

25

4.Expression

97

14.Bla

25

5.Speech

95

15.Discrimination

23

6.Teacher

66

16.Fear

23

7.Crime

46

17.Rectory

22

8.Shit

42

18.Homophobic

20

9.Professor

34

19.Violence

19

10.Institution

33

20.Inquisition

19

4 Discussion Naturally, the subjects on which the attacks are addressed are reflected in the aggressive language on social networks. Thus, while aggressive communications reported in interviews are focused on image and body (e.g., fat, head, mouth, armpits), death (e.g., disappear, destroy, inquisition), and gender (e.g., whore), in the languages found in social network groups, themes such as LGBT emerged. Issues common to those taken from the interviews are linked to gender identity and aggression against women. The presence of these subjects in aggressive content manifested in different education cycles in Portugal, including basic, primary, secondary, and higher education, reflects the cultural character behind online aggression. Gender-related cultural influence is also mentioned in the works of the eMORE Project (D’Angelo et al., 2018), which warns that the aggressive communications found in social media analyses in several European countries show the cultural tendency to highlight the difference between men and women built over time. Likewise, the online attacks against the LGBT group follow a correspondence between the European countries included in the eMORE Project, highlighting a generalized discriminatory tendency towards European countries (D’Angelo et al., 2018). Issues related to traditional festivals in university environments are highlighted in this study and are also quite evident in the reports of the eMORE Project, which presents the general tendency to maintain the culture, traditions, and social practices linked to each of the places from which aggressions emerged, following a line with the history and habits of each country. For example, anti-Semitism and Islamophobia are continuous or emerging phenomena, very present in countries that still manifest historical sentiments, as observed in Germany, Italy, and Romania (D’Angelo et al., 2018; eMORE, 2017). Aggressive communications related to the preservation of local culture give rise to many forms of hate speech, as in the case of aggression related to religion, which often appears to preserve the belief of a particular people or country (Young et al., 2018). This fact is

Analysis of Cyberaggression in Social Networks Involving Students

297

verified in the following excerpt from the final report of the eMORE Project (D’Angelo et al., 2018): “They live like us, with us, repeating once again the division’ us’ and ‘them.‘ Moreover, he fears that Muslims will take over Belgian culture, needs, and traditions, imposing their traditions over the Christian ones.“ (p. 86). However, while the results of the eMORE Project also encompass results that indicate racism, these types of aggression are not among the discourses found within the scope of this study. It is noteworthy, however, that all interview participants are Caucasian, which is one of the reasons why racial discrimination is not noticed. Still, any racial discrimination was detected among the aggressive content from social networks. Some studies in the literature agree with the results of this study. For instance, the studies by Hosseinmardi et al (2015) labeled aggressive images and comments on Instagram. The categories labeled on personal pages were similar to the results in this work, in which the words related to cyberbullying were “body” (e.g., face, clothes), “sexual” (e.g., slut, rapist) and “anger,” providing evidence of appearance- and genderbased cyberbullying on social media. Other categories, such as “death” (e.g., burying, killing), are also highly associated with cyberbullying. Additionally, the results of the textual analysis performed by Ben-David & Matamoros-Fernández (2016) also show similarities with the results of this study. An open group about politics on Facebook highlighted derogatory insults and online attacks against immigrants. The results of this study also corroborate studies by Fortuna (2017) found in the literature, in which aggressive subjects and language were significantly related to the body (e.g., fat, ugly, thin, tall, short), homophobia (e.g., dyke, butch, dykes, pride, fagot, straight, gay, gays), and sexism (e.g., woman, fat - female, dumb - female, ugly - female, man). Other issues found related to behavioral issues (e.g., insecure, sensitivity); religion (e.g., Islam, Muslim, mosque, bomb), racism (e.g., racism, white, black, nigger), ideology (e.g., feminism, lefties, feminazi) and health (e.g., disabled people). However, Malmasi & Zampieri (2018) state that discriminating against hate speech online is not a simple task, which may require a deeper understanding of the text and the languages used, such as local jargon. In this regard, the studies by (Zhong et al., 2016) also warn that the labeling and characterization of what constitutes cyberaggression or cyberbullying on the Internet was a difficult challenge since many Instagram media sessions that presented a high percentage of negative words were not considered cyberbullying. Despite this, among the samples that determined cyberaggressions in their work are images about “drugs” and comments about “death and suicide” (e.g., kill yourself). Although this study does not provide results about images associated with cyberaggressions in the Portuguese context, the studies by Hosseinmardi et al. (2015) corroborate those of Zhong et al (2016), as they show that most drug images are related to aggressive online communications, while other image content, such as bicycles and food, have a low relationship with cyberaggression. These studies partially correspond to the results found in the analysis of this study, since while “death” is a subject present in aggressive communications, also in Portuguese social network groups, subjects related to “drugs” were not frequent among the contents of cyberaggression. Additionally, the causes of cyberaggression reported by the students in the interview are reflected in the corpus of analyses since gender, image and body, and cultural

298

E. Oliveira et al.

and academic traditions are found among the aggressive content taken from social network groups posts. Still, while the interview students reported that the responsibility for cyberaggressions is entirely linked to the user’s behavior, some implications exist for preventing aggression at the Portuguese educational level. Therefore, the recommendation of online civility training at all educational levels is emphasized (Oliveira & Baldi, 2022). Furthermore, prevention and combat programs against cyber-attacks in universities and schools should be mandatory (Cunningham et al., 2015). Other relevant actions reported in the literature are regarding when and how to respond to support victims, highlighting the bystanders’ importance on the cyberaggression dynamics (Young et al., 2018). Finally, considering the findings of this study, specific training in social kindness and information about the causes and impacts of cyberaggression should also be provided to students of all educational levels. Simultaneously, actions should also focus on the minorities’ vulnerability, with the educational body developing inclusive and mutual respect policies and projects to increase support for minorities and equality in universities. Based on this work’s limitations, considerations are developed for future work. The interview sample is small, which restricts the results achieved and does not allow generalizations to be made. Thus, future work should ensure that a larger sample of university students participate in the interviews for broader data collection. Most aggressive words obtained through the group analyses were taken from the University of Aveiro open group. Thus, the frequent words reflect the theme related to the post, which is related to the LGBT group, making this sample biased. Future work should conduct a new scan for possible aggressive content. The new scan encompasses, for instance, social media groups from Academic Associations and Students Nuclei from other universities in Portugal. Studies that can provide information about closed university groups on Facebook and WhatsApp are also suggested to deepen the aggressive language used in restricted environments. The findings of this work are preliminary and not generalizable to all online university Portuguese environments. However, if future scans of social media groups, such as the one suggested above, confirm similarities in cyberaggression linguistics at Portuguese university online social network groups, a more profound knowledge regarding this matter can increase the effectiveness of the automatic detection of aggression in online settings to prevent and fight the cyberaggressions in the university Portuguese environment.

5 Final Remarks The present study is one of the first to address aggressive linguistics in a Portuguese online environment, focusing on university-related open social network groups. Further information regarding cyberaggressive content in different educational levels is presented through interviews, confirming the presence of aggressive situations at all ages of development. Subjects that frequently appear among the analyzed aggressive communications are gender identity, aggression against women, cultural academic traditions, and image and body. Manifestations of aggression encompassed offenses, threats, incentives to suicide, inflammatory speech, sarcasm, irony, and rejection, among others. The number of posts with aggression on the university students’ social networks was 17, with

Analysis of Cyberaggression in Social Networks Involving Students

299

the largest number of aggressions taken from the University of Aveiro post, biasing the results. Future studies must include a higher number of participants in the interviews and the observation of closed groups on Facebook and WhatsApp. Cyberaggression linguistics reflects the causes and the subjects of aggressive communications gathered through thematic analyses. Lastly, the content and thematic analysis proved to be a feasible way to identify information about how aggression is shaped online, providing parameters for research on detection of cyberaggression situations. Acknowledgments. This work is financially supported by national funds through FCT – Foundation for Science and Technology, I.P., under the project UIDB/05460/2020, and through the FCT fellowship 2020.04575.BD.

References Bardin L (2009) Análise de Conteúdo Edições 70 (ed) Persona Ben-David A, Matamoros-Fernández A (2016) Hate speech and covert discrimination on social media: Monitoring the Facebook pages of extreme-right political parties in Spain. Int J Commun 10:1167–1193 Braun V, Clarke V (2006) Using thematic analysis in psychology. Psychiatr Q 3(2): 77–101. http:// www.ncbi.nlm.nih.gov/pubmed/11752478 Costa AP, Amado J (2018) Análise de Conteúdo suportada com software. Ludomedia Cunningham CE et al (2015) Modeling the anti-cyberbullying preferences of university students: Adaptive choice-based conjoint analysis. Aggressive Behav 41(4):369–385. https://doi.org/10. 1002/ab.21560 D’Angelo E, Ferrero F, Scioneri V (2018) eMORE webapp for monitoring and reporting online hate speech: a cross-cutting analysis of data collected using the eMORE webapp in 9 EU countries. 1–104. https://www.rissc.it/homepage/our-projects/emore-project/ eMORE (2017) An overview on hate crime and hate speech in 9 EU countries. 1–42 Ernst J, Schmitt JB, Rieger D, Beier AK, Bente G, Roth H (2017) Hate beneath the counter speech? A qualitative content analysis of user comments on YouTube related to counter speech videos. J. Deradicalization 10:1–49 Fortuna P (2017) Automatic detection of hate speech in text: an overview of the topic and dataset annotation with hierarchical classes [University of Porto]. https://repositorio-aberto.up.pt/han dle/10216/106028 Gonçalves L, Oliveira L (2021) When knowledge meets digital: a systematic literature review about digital platforms and knowledge flow. Springer Series Design Innov. 8:35–48. https:// doi.org/10.1007/978-3-030-49647-0_3 Gonçalves SP, Gonçalves JP, Marques CG (2021). Manual de Investigação Qualitativa. In Manual de Investigação Qualitativa, p. 300. PACTOR Hosseinmardi H, Mattson SA, Rafiq RI, Han R, Lv Q, Mishra S (2015) Analyzing labeled cyberbullying incidents on the instagram social network. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 9471:49–66. https://doi.org/10.1007/978-3-319-27433-1_4 Infopédia (2023) Vergonha Lusa (2020)Covid-19: Maioria dos estudantes foi vitima de bullying durante pandemia, diz estudo. 1. https://www.iscte-iul.pt/noticias/1706/maioria-estudantes-vitima-de-cyberbullyingdurante-pandemia

300

E. Oliveira et al.

Malmasi S, Zampieri M (2018) Challenges in discriminating profanity from hate speech. J Exp Theor Artif Intell 30(2):187–202. https://doi.org/10.1080/0952813X.2017.1409284 Müller K, Schwarz C (2017) Fanning the Flames of Hate: Social Media and Hate Crime. Ssrnhttps://doi.org/10.2139/ssrn.3082972 Neves F (2016) 10 a 20% dos jovens são vítimas de cyberbullying. https://www.dn.pt/sociedade/ 10-a-20-dos-jovens-sao-vitimas-de-cyberbullying-5141247.html Oliveira E, Baldi V (2022) Systematic Review on Cybersecurity Risks and Behaviours: Methodological Approaches [Paper presentation]. COMPLEXIS 2022: 7th International Conference on Complexity, Future Information Systems and Risk. https://www.scitepress.org/Publications Detail.aspx?ID=J1bkQc7t494=&t=1 Ponte C, Batista S (2019) EU Kids Online Portugal. Usos, competências, riscos e mediações da internet reportados por crianças e jovens (9–17 anos). http://fabricadesites.fcsh.unl.pt/eukids online/documentos/ PORDATA (2021) Indivíduos com 16 e mais anos que utilizam computador e Internet em % do total de indivíduos: por nível de escolaridade mais elevado completo. https://www.pordata.pt/ Portugal/Indivíduos+com+16+e+mais+anos+que+utilizam+computador+e+Internet+em+per centagem+do+total+de+indivíduos+por+nível+de+escolaridade+mais+elevado+completo1141 Van Schaik P, Jeske D, Onibokun J, Coventry L, Jansen J, Kusev P (2017) Risk perceptions of cyber-security and precautionary behaviour. Comput Human Behav 75(May 2017): 547–559. https://doi.org/10.1016/j.chb.2017.05.038 WHO (1998) WHOQOL User Manual (Issue January) Young R, Miles S, Alhabash S (2018) Attacks by anons: a content analysis of aggressive posts, victim responses, and bystander interventions on a social media site. social media and society 4(1). https://doi.org/10.1177/2056305118762444 Zhong H et al (2016) Content-driven detection of cyberbullying on the instagram social network. IJCAI International Joint Conference on Artificial Intelligence 2016-Janua: 3952–3958. https:// doi.org/10.1145/2742647.2745908

Child Safety on Bicycle Seats. A Study Evaluating the Need for Additional Markers to Increase the Visibility of Children When Transported by Bicycle Rita Martelo(B) CIAUD, Research Centre for Architecture, Urbanism and Design, Lisbon School of Architecture, Universidade de Lisboa, Polo Universitário do Alto da Ajuda, Rua Sá Nogueira, 1349-063 Lisboa, Portugal [email protected]

Abstract. The number of cyclists and bicycles in the urban environment have been increasing substantially in recent years. Bicycle seats, trailers, and cargo bikes allow parents to transport their children on their bicycles switching from the car into greener mobility options. Nevertheless, safety becomes a priority when transporting children by bicycle. Different cities experience different levels of cycling culture and development. Lisbon (Portugal) is a city with poor cycling infrastructure and low cycling modal share, and cyclists find it difficult to ride safely due to the lack of dedicated infrastructure and the constant sharing of the road with reckless drivers. There is a gap in the literature of studies looking at child visibility and safety when transported by bicycle, especially in cities where cycling is starting to grow. The methods used in this study are the survey by questionnaire and the artifact analysis. These methods allowed the understanding of the demographics, cycling characteristics and habits, how children are being transported on the bicycle, and which safety strategies are in use. In addition, the analysis of bicycle rear seats in use provided evidence of which information or visual cues are available for signaling a child on board. This mixed-method study provides a comprehensive understanding of the safety strategies used by parents and the design features of bicycle seats that promote child safety. The results highlight that, although parents feel relatively safe when transporting their children by bicycle, additional safety features to increase visibility are most welcome. Keywords: Child safety · Bicycle attachments · Visibility aids

1 Introduction Cycling is becoming more and more popular due to the growing interest in sustainable and greener modes of transportation. The bicycle has multiple advantages and benefits and is considered a real alternative in the urban environment for short distances (up to 5 km). Lisbon has recently experienced a steep increase in both the availability of cycling infrastructure and number of cyclists [2, 4, 7]. Despite the strong car dependency © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 301–310, 2024. https://doi.org/10.1007/978-3-031-47281-7_24

302

R. Martelo

(60% use the car against 0.6% of bicycle use) [3], many parents now prefer to use alternative options to carry their children on the bicycle before they can ride on their own. Transporting children on bicycles by using dedicated seats has recently become popular in the European Union [1]. From seats attached in the front or in the back, trailers, or cargo bikes, these transportation options have advantages and disadvantages as presented in the study from 2009 [9]. Child bicycle seats and trailers are typically designed to transport children from the ages of 1 up to 6 years old, depending on their weight. The most common and accessible attachment is a seat attached in the back. There are some suggested techniques to enhance children’s visibility that might prevent injuries such as wearing a helmet, bright colours, use of reflectors, attaching a flag, and using safety harness [8]. Nevertheless, little information is available about bicycle-mounted child seats despite their general use. Visibility aids have the potential to reduce child and cyclist injury, but scarce data exist relating to their use or to interventions for increasing visibility aid use among children. Campaigns encouraging the use of visibility aids and information regarding road safety should be encouraged from an early age [8]. There is a study investigating the safety and vulnerability of children when transported by bicycle during an accident scenario involving a motor vehicle. The main injuries are in the head and neck, emphasizing the use of the helmet and safety harnesses [11, 12]. Also, visibility enhancements can act as a passive safety conveyor. Being visible contributes to a safer cycling experience. The aim of this study is to evaluate the need for additional visual markers to increase the visibility of children when transported by bicycle.

2 Methodology This study follows a mixed-method approach. Quantitative methods provide numerical data that can be analyzed statistically to identify patterns and relationships, while qualitative methods provide in-depth, contextual understanding of a phenomenon. Combining a questionnaire and the artifact analysis will enable a more comprehensive understanding of the safety strategies used by parents and the design features of bicycle seats that promote child safety, as the strengths of one method can offset the limitations of the other.

3 Questionnaire The questionnaire is a mixed-method selected to better understand preferences and behaviour of cyclists. It allowed to collect data and information from cyclists and noncyclists, including their habits, preferences, behaviours, cycling characteristics and safety strategies when transporting children on the bicycle. An online survey was prepared using Google Forms with a total of twenty-seven questions from close-ended to open-ended questions including rank preferences. The first two sections survey the demographics of the respondents, including age, gender, nationality, and residence. The third section focuses on cycling habits, i.e., understanding whether respondents’ cycle or not. Section IV relates to cycling characteristics including frequency of cycling, Bicycle Sharing Systems (BSS) usage, and if BSS bicycles should include a quick fix child seat. Furthermore, it assesses the main purpose of bicycle rides,

Child Safety on Bicycle Seats

303

number of bicycles owned, environment of cycling, frequency of cycling, and safety equipment in use. Section V assesses if participants have children, how many, their age and if they are transported on the bicycle. Section VI focuses on the purpose of transporting children on the bicycle, which attachments are used, and which additional safety features are considered for children. Section VII focuses on strategies to reduce risk, including the behaviour of drivers when facing a cyclist transporting a child, the opinion of participants regarding additional visibility and safety features, and their preference over some illustrative visual prototypes. Finally, participants were asked to share some recommendations for those carrying children on the bicycle. The survey was shared over multiple social platforms (Facebook, LinkedIn, Instagram, WhatsApp) and by email across friends and family, professional contacts, as well as cycling groups on a national and international level. The diagram in Fig. 1 highlights the structure of the survey.

Fig. 1. Questionnaire structure.

4 Artifact Analysis The artifact analysis is a qualitative method to systematically examine qualities of an object, material aesthetics and interactive qualities. It can help identifying what an object has to say about people and their culture, time, and place [5]. Considering a seat attached on the back as one of the most popular and affordable options to carry children, this analysis focuses on this type of attachment only. The analysis identifies brand manufacturers and aesthetics details, including colours, safety features and visibility add-ons of seats. This method allows to visualise quantities but also the quality of seats in use. The information was collected from an online marketplace platform in Portugal [10], where people sell their used items, getting a better overview of what people really own. The keywords “cadeira para bicicleta” (rear seat for bicycle) presented more than 300 results. No additional filters (e.g., location or price) or categories were selected. Only the first one hundred results were listed in an Excel spreadsheet and organised according to the following categories: Photo, colour, brand, number of harness points, cushion colour, visibility add-ons, location, other safety details, and asking price. An example is displayed bellow (Table 1).

304

R. Martelo Table 1. Categories from artifact analysis.

5 Results 5.1 Questionnaire The questionnaire was online for 3 weeks and had a total of 181 respondents. Most respondents revealed to be males aged between 35–44, although female cyclist also had a strong participation. Most of the respondents are Portuguese (65,7%), living in the greater Lisbon area, and almost a quarter are European. Although focusing on a national level, the international participation was positive to enable a cross-country comparison. Data revealed that the majority of respondents cycles (80%), against 20% who do not cycle. The ones who cycle do it regularly, everyday (43%) or at least once a week (34%), the ones who don’t cycle admitted to drive (Fig. 2).

Fig. 2. Types of users and frequency of cycling.

Regarding the number of bicycles per household, three quarters of the respondents have 3 or more bicycles at home. The vast majority cycles in an urban environment, although nature and the countryside are also a preference. The main purpose of bicycle rides is for recreational purposes (56,6%), followed by utilitarian/commute (55,2%), shopping or work, i.e., deliveries, couriers, etc. Almost 80% of respondents have kids, which also reflects the age range of parents. Very relevant to this study is the number of parents carrying their children on a bicycle. Half of the respondents who have kids are transporting them on the bicycle, with the main purpose of taking them to school. Following previous studies, the seat on the back revealed to be the most used option in Portugal as well (Fig. 3).

Child Safety on Bicycle Seats

305

Fig. 3. Most common types of bicycle attachments in use.

Regarding safety equipment, cyclists seem to be concerned about safety, the majority use lights and helmets frequently. About a quarter admitted being wearing high-visibility clothing. Also, most children seem to be protected by a helmet and parents use additional lights when transporting them (Fig. 4). Most parents admitted feeling safe when transporting their children on the bicycle.

Fig. 4. Safety/visibility features in use.

When asked if bicycle seat manufacturers should invest in additional safety/visibility features, the majority of the respondents agreed (72%), whereas almost a quarter disagreed or gave other reasons such as: not enough information/data to reply, less traffic, or cities should invest in safety. Then, respondents were presented with some illustrative prototype options to state their preference if they were about to buy a new rear seat. Regarding informative text, the majority have selected ‘No text’ (29,7%), followed by ‘Child on Bike’ (26,2%), and ‘Atenção! Criança a bordo’(Attention! Child a board) (23,8%). Regarding visual examples, the results did not confirm a strong preference. However, 34% of respondents would like to include reflective stickers, 37% include additional lights, and 36,5% would like to include a flashing detached sign (Fig. 5 and Fig. 6). Regarding non-cyclists, 100% admitted keeping distance as a driver when facing a cyclist transporting a child aboard, 85% admitted reducing driving speed, and only 8%

306

R. Martelo

Fig. 5. Preference of visibility features.

Fig. 6. Favorite illustrative prototypes.

giving notice when approaching. The majority of non-cyclists (88%) agree that bicycle seat manufacturers should invest in additional safety/visibility features. The last question of the survey was an open question asking for recommendations for those who transport children by bicycle. The results were listed, analysed, and adjusted to the same language and terminology. To better visualise what respondents’ recommend, a Word Cloud Generator [6] was used to highlight the most frequent words. ‘Use cycle lanes’ stands out as the most frequent recommendation (Fig. 7). 5.2 Artifact Analysis From the 100 seats analysed, the results revealed a wide variety of brands and colours. The most popular brand is Polisport with 32 examples, followed by Hamax with 27, Btwin with 14, and other brands in a total of 13. There were, however, 14 results where the brand could not be identified. Regarding colour, the most popular is grey. Within its multiple shades, there are 56 seats in grey, followed by 22 in blue. The least popular colours are white, purple, lilac, and green with just one seat each. Although yellow is considered a very conspicuous colour, only 3 examples with this colour were found. Regarding the geographic distribution, most of the seats are located in the Lisbon and Porto Districts, which reflects the most densely populated areas and cycling urban environments (Fig. 8).

Child Safety on Bicycle Seats

307

Fig. 7. Recommendations for those carrying children on the bicycle (WCG By MonkeyLearn).

Fig. 8. Most common colours of seats and distribution across Portugal.

The sample reflects that most of the seats have no additional visual markers whatsoever. Conspicuous colours like yellow, green, or red were found in just 8 seats, and small reflective signs in the back were found in 14 results (Fig. 9). The use of additional visibility signs in the back is linked with the most expensive models of seats.

308

R. Martelo

Fig. 9. Visual markers on rear seats.

6 Discussion and Conclusions The number of cyclists in the urban context has been rising thanks to the investments in cycle infrastructure and the sustainable conscience for greener modes of transportation. Many parents have been looking at alternative options to carry their children by bicycle. The seat in the back is still one of the most used, accessible, and affordable attachments for children between 1 and 6 years old. In a country like Portugal with such a strong car dependency, inevitably urban cyclists share space with drivers. Therefore, safety and visibility become a priority when transporting vulnerable children by bicycle. This study is very timely and relevant for the Portuguese context since there has been no data so far regarding safety and visibility of children when transported by bicycle. The results from the questionnaire revealed that half of the respondents who admit to cycle carry their children by bicycle. This is a solid indicator in support of the relevance of this study and a sign that the mobility paradigm is shifting. Although parents admitted feeling relatively safe when transporting their children by bicycle, they also feel that seat manufacturers should invest in additional safety/visibility features. Considering their preferences and behaviours, additional lights in the back of the seats seem to be the most obvious choice to further develop and study. Most of the available seats cyclists’ own are presented in a variety of greys or blues, which are not very conspicuous colours, i.e., not very visible/noticeable in the urban environment. It would be interesting to understand the reasons behind the choice of these colours, and seats manufacturers’ design strategies. Also, to cross-reference information with models available in stores. Perhaps people just buy what it is available. Seats with conspicuous and bright colours such as yellow, green, or red, with an additional socket to fit lights in the back, could be a possible and effective solution no enhance visibility to others. Perhaps in the future, approaching seat manufacturers with this study could encourage them to develop some prototypes to be tested in a real cycling environment. The most popular brand in this study is Polisport. Founded in Portugal in 1978, Polisport group is the world leader in child bicycle seats. After a brief approach, Polisport confirmed that all their seats contain reflector devices. However, there are examples

Child Safety on Bicycle Seats

309

of seats with greater visibility according to higher price positioning. Regarding colors, Polisport exports around 98% of its production to Nordic countries, where darker colours are much more preferred (gray and blue). This seems to be the preference in Portugal as well. They also showed interest in testing innovative solutions in the future that allow them to make their products even safer as long as the price is consistent with their positioning strategy. Moreover, it will be positive to confront drivers and other road users with these prototypes and test their real effectiveness. The rear seat is a short-term attachment before children can ride on their own. Nevertheless, safety and visibility should not be overlooked.

7 Limitations In order to have a more representative sample, it would be important to extend this research to a larger cycling audience of parents across Portugal. Since the main reason for carrying children is to take them to school, perhaps a collaboration with kindergarten and schools could identify more cycling parents. Another limitation of this study was the number of seats analysed. The 100 seats being sold on the date the data was collected might not be the most representative sample. Perhaps, extending the artifact analysis to other marketplace platforms and to the market of new seats could help in crossing information and having a better overview. This study was limited to the Portuguese context. Nevertheless, it could be positive to extend this study to other countries with similar levels of cycling modal share. Acknowledgments. Many thanks to all the participants in this study, namely the respondents of the questionnaire for their valuable contribution and availability. Special thanks to Adriana Santos, Marketing Manager at Polisport, who kindly shared the safety and visibility strategies of the world leading company in child bicycle seats. Moreover, special thanks for all the comments and suggestions from Professors Colette Jeffrey, David Vale, Rui Marcelino, André Castro, João Brandão, and Gianni Montagna. This work is financed by national funds through FCT - Fundação para a Ciência e a Tecnologia, I.P., under the Strategic Project with the references UIDB/04008/2020 and UIDP/04008/2020.

References 1. Biernat E, Buchholtz S, Bartkiewicz P (2018) Motivations and barriers to bicycle commuting: Lessons from Poland. Transp Res F: Traffic Psychol Behav 55:492–502. https://doi.org/10. 1016/j.trf.2018.03.024 2. Félix R, Cambra P, Moura F (2020) Build it and give ‘em bikes, and they will come: The effects of cycling infrastructure and bike-sharing system in Lisbon. Case Stud Transp Policy 8(2):672–682. https://doi.org/10.1016/j.cstp.2020.03.002

310

R. Martelo

3. INE (2018) Mobilidade e funcionalidade do território nas Áreas Metropolitanas do Porto e de Lisboa 2017 4. Marrana JC, Serdoura F (2017) Cycling Policies and Strategies: The Case of Lisbon. https:// doi.org/10.17758/URST.U0917311 5. Martin B, Hanington B (2012) Universal Methods of Design. Rockport Publishers 6. MonkeyLearn. (n.d.). Word Cloud Generator. https://monkeylearn.com/word-cloud/. Accessed 15 Oct 2022 7. Moura F, Félix R, Reis AF (2020) Relatório: Análise dos dados das contagens de bicicletas2020 8. Mulvaney CA, Kendrick D, Watson MC, Coupland CAC (2006) Increasing child pedestrian and cyclist visibility: Cluster randomised controlled trial. J Epidemiol Community Health 60(4):311–315. https://doi.org/10.1136/jech.2005.038216 9. Murray J, Ryan-Krause P (2009) Bicycle Attachments for children: bicycle seats, trail-abikes, and trailers. J Pediatr Health Care 23(1):62–65. https://doi.org/10.1016/j.pedhc.2008. 10.003 10. OLX (n.d.). Online marketplace. www.olx.pt. Accessed 20 Oct 2022 11. Oxley J, O’Hern S, Raftery S, Woolley J (2016) How safe are children when transported by bicycle? Traffic Inj Prev 17:163–167. https://doi.org/10.1080/15389588.2016.1199866 12. Ptak M, Wilhelm J, Sawicki M, Rusi´nski E (2019) Child safety on various bicycle-mounted seats during vehicle impact. Transport 34(6):684–691. https://doi.org/10.3846/transport. 2019.9083

Digital Citizenship: The Cyber-Plagiarism Dilemma in Compulsory School Fernanda Vicente1

and Manuel Meirinhos2(B)

1 Instituto Politécnico de Bragança ESE, Campus Santa Apolónia, Bragança, Portugal 2 Centro de Investigação em Educação Básica (CIEB), IPB-ESSE, Campus Santa Apolónia,

Bragança, Portugal [email protected]

Abstract. In today’s digital society, knowledge is built through relationships with others and the environment. Schools have a vital role in promoting and developing skills in children and youth until they complete compulsory education. The use of technology to access and share information is unavoidable and has led to cyber-plagiarism to become a significant problem faced by educational institutions, particularly among primary, secondary, and high school students. Digital citizenship skills such as researching, selecting, collecting, and processing information through critical thinking are necessary to prevent cyber-plagiarism. Its use raises ethical concerns and undermines the fundamental values of a democratic society. This work reflects on the practice of cyber-plagiarism by students, defines the concepts of plagiarism and cyber-plagiarism, identifies their causes and consequences, and proposes strategies to fight against it in schools. A narrative literature review methodology, which includes national and international research sources and official documents, is used to present guidelines for educational institutions and communities to adopt common strategies to prevent cyberplagiarism. The work also suggests fundamental strategies for the initial and continuous training of teachers to develop skills that promote digital citizenship and prevent cyber-plagiarism. Keywords: Cyber-Plagiarism · Digital citizenship · Digital responsibility

1 Introduction Technological evolution has transformed societies and created moral dilemmas regarding man’s relationship with information and knowledge. The emerging digital society, where the production of knowledge is accelerated, also facilitates access to this knowledge, as well as its rapid dissemination. A new citizenship necessarily emerges, which we now call digital citizenship, requiring new knowledge, new skills, and new values for citizens of the 21st century. Mandatory schooling cannot be oblivious to the promotion of this new citizenship if it wants to empower future citizens to fully live in the digital society. It becomes inevitable to reflect on the decisive role that schools play in the development of students critical thinking skills in primary, secondary, and high school education [1], © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 311–322, 2024. https://doi.org/10.1007/978-3-031-47281-7_25

312

F. Vicente and M. Meirinhos

providing them with a knowledge solid foundation that enables them to exercise their citizenship and navigate in the information society. It is in this context that educational institutions in the country, from primary to university education, are struggling with a problem – cyber-plagiarism. However, our concern regarding this problem is mainly confined to primary, secondary, and higher education. The use of digital technologies to access, store, and share information is an undeniable reality in the digital society. The ease of accessing, processing, and disseminating information necessarily leads to the alteration of previous habits, work methods, and conceptions that are characteristic of an analog world. Research on this issue has mainly focused on higher education, where visibility and consequences are more evident. We believe that these habits may already be deeply ingrained. For this reason, our concern regarding this problem is mainly focused on primary, secondary, and high school education. The development of values regarding this practice can occur in mandatory education, which is considered the period during which all young people receive a basic or essential education, developping fundamental skills for life in society. The concern with the development of students’ digital competence, specifically digital citizenship skills, has been well evidenced in the digital competence frameworks, notably in DigCompEdu [2]. The development of these competencies by teachers aims to enhance the development of digital citizenship skills in students. Many countries seek to adjust educational programs to meet the new demands of citizenship in the digital society. In Portugal, for example, the documents Essential Learning (AE) and the Profile of Students at the End of Compulsory Education (PASEO) constitute the starting point for teachers to plan and create conditions for students to develop competencies, and for both teachers and students to pedagogically evaluate the process and quality of learning. The practice of cyber-plagiarism raises questions of ethics and honesty, values included in PASEO, but also integrated in the National Strategy for Citizenship Education, being teachers responsible for promoting ethical behavior and developing students’ awareness of their conduct so that they can assess the consequences of their actions. Therefore, schools, as an integral part of a changing world, cannot shirk their responsibilities, playing a crucial role in the education of children and young people. Within this setting, the purpose of this work is to raise awareness of a practice that is quite common in the academic work of compulsory education students. We believe that this awareness will require a greater investment from the educational community in the development of digital citizenship, avoiding dishonest practices that may continue into higher education. The originality of the research lies in seeking to bring into the educational debate a problem that is well evidenced in schools and associated with the moral development of children in the digital society, but to which it has not been given the importance it deserves. Thus, the study may assist institutions and teachers in implementing policies for teacher training and competency development in students to try to avoid dishonest practices. The methodology adopted for this research was based on narrative bibliographic review, seeking to establish the “state of the art” on this issue, fostering reflection on the practice of cyber-plagiarism by compulsory education students, framed in the context

Digital Citizenship: The Cyber-Plagiarism Dilemma

313

of digital citizenship. Within the framework of this methodology, searches were carried out in important reference databases, seeking to locate relevant research on the subject of digital citizenship and cyberplagiarism. For this purpose, we address the concept of plagiarism and cyberplagiarism, the causes of these phenomena, the consequences of these acts, the way in which copyright is or is not considered, and, finally, suggest strategies to make students more aware and to fight against cyber-plagiarism in schools.

2 Cyber-Plagiarism Concept With the emergence of digital society, the practice of plagiarism has been transposed into cyberspace. Although there is no universally accepted concept of cyber-plagiarism by the scientific community, some proposals have been shared that even present the concept of digital plagiarism pari passu with cyber-plagiarism. In 2007, the definition of academic cyber-plagiarism emerged in Comas and Surema’s [3] article: “the use of ICT (mainly the Internet and its associated resources - especially the WWW-) for the (total or partial) plagiarism of academic work by students. That is, the location, adoption and presentation of ideas, theories, hypotheses, results, texts, etc. of others as one’s own in any academic work” (p.1). Later, Patterson & McCall [4] describe it as the “act of taking and using someone else’s work without proper attribution in a digital or electronic format” (p.67). Researchers [5, 6] use the term digital plagiarism as a synonym for cyber-plagiarism, with the former specifying that “it occurs when students use electronic resources to copy, paste, and submit someone else’s work as their own” (p.456). Wang, Zhu, and Gao [7] express the same understanding, referring to it as “involving the use of digital technology to copy, download, or otherwise use the work of others without proper attribution” (p.189). Dumitrina et al. [8] define the phenomenon of cyber-plagiarism “as the appropriation of information in any format (text, images, video, etc.) from the Internet and its use as one’s own without any reference to its author” (p.113). In a more recent study, cyber-plagiarism “is a phenomenon of academic dishonesty where students copy and paste information from digital sources and submit it as their own work. It includes various forms of unethical behaviors, such as verbatim copying of text, images, or ideas from digital sources without proper citation, paraphrasing without proper attribution, self-plagiarism, and contract cheating.“ (p. 75) [9], relating the practice to the notion of dishonesty and sharing concerns within the scope of digital citizenship. In line with this view, Horkoff [10] defines cyber-plagiarism as “the act of misrepresenting someone else’s work as your own. Sometimes a writer plagiarizes work on purpose—for instance, by copying and pasting or purchasing an essay from a website and submitting it as original course work” (p.341). The widespread use of cyberspace has created the idea that everything on the Internet can be accessed and therefore copying and pasting can be done without any duties or restrictions [11]. Based on the reviewed literature, the perception of young people is associated with the judgment that it is not illegal, immoral, or criminal to, partially or completely, plagiarize someone else’s work. The thematic literature also suggests that the practice of cyber-plagiarism is widespread among students, especially those in the secondary and high school education, who look for works on the internet that may

314

F. Vicente and M. Meirinhos

correspond to the requests from their teachers and use a quick and simple process to integrate the information they collect, which is often scattered throughout cyberspace. This generation, which has easy access to information and uses simple commands like “ctrl + c and ctrl + v” for this purpose, is currently known as “copy-paste” or “copy and paste.” An approach to digital citizenship in the educational context, through various initiatives, could make elementary, secondary, and high school students more aware of their responsibility, namely their rights, but also their duties when using digital sources and the resources available therein. Digital citizenship requires the adoption of responsible and ethical behaviors in both face-to-face and online contexts. This responsibility includes creating original content and respecting intellectual property rights, as well as a conscious use of digital information.

3 Cyber-Plagiarism in Academic Environments Cyber-plagiarism is an issue deeply rooted in the academic context of primary, secondary, and high school education. In this section, we seek to analyze its causes and consequences, as well as its relationship with ethics in the digital society, therefore we cannot exclude the need to respect copyright. 3.1 Cyber-Plagiarism in Compulsory School The prevalence of cyber-plagiarism in higher education, proven by several studies, led us to investigate when this practice begins in academic life. To better understand, detect and combat it, it is necessary to analyze it in the academic context of primary, secondary, and high school education. Therefore, it seemed essential to investigate the causes and possible motivations that lead to this problem. Copying and pasting has become an almost thoughtless act, in which the absence of writing skills, combined with a lack of ability to research and treat information ethically, is evident. Thus, it is a widespread practice among students, characterized by academic dishonesty, which students give little importance to, and being overlooked by most teachers. Therefore, the authors of cyber-plagiarism reveal a gross ignorance of the procedures inherent to the research and citation process, but also to the copyright laws that they consciously or unconsciously infringe. In a case study, Meirinhos and Valadar [12] confirmed the existence of plagiarism/cyber-plagiarism in their students’ work. On the other hand, secondary school students, according to the answers given to the survey, seem to have a precise notion of what plagiarism and cyber-plagiarism are. International research [8], conducted in the last two decades, corroborates these ideas. The main concerns of this research are to quantify plagiarism activities and identify plagiarizers, as well as to identify the types of plagiarism practiced and the causes associated with its practice. Permana and Santosa [13] addressed this dimension of the “act of cut-paste source from the internet commonly known as cyber-plagiarism” (p.128). In an article of their

Digital Citizenship: The Cyber-Plagiarism Dilemma

315

own, they present percentages of 33% to 44% of secondary school students who admitted to using the cut-and-paste process without knowing the source. The lack of a school common policy to deal with issues related to this problematic, which affects the exercise of citizenship by students, is another reason cited for this fraud. According to literature, the fact that students are not taught to respect copyright from an early age and in a continuous manner contributes significantly to the dissemination of the practice of copying work without any criteria and without proper referencing [14]. Granitz and Loewy [15] present the pressure of deadlines, the difficulty in keeping up with subjects, the lack of preparation, the fact that teachers do not reference or warn students about the infraction, or the fact that students made plagiarism a recurrent practice without ever being reprimanded for such fraud, as justifications for not considering plagiarism a serious problem. In the same line of ideas, Valadar’s thesis [16] points to several causes for this practice: easiness of copying information available on the internet, disinterest of teachers in plagiarism, similar behavior of colleagues, procrastination in doing the work until the last moment, lack of instructions or guidance for doing the work, difficulty of the work, freedom to copy everything on the internet, seeking higher grades in copied work, saving effort for better results, shortage of time to write the work, tight deadlines for doing the work, lack of information about the consequences of plagiarism, ease of sharing, copying and pasting information, and easy mastery of the internet and technology. Rasco et al. [17], in a study dedicated to plagiarism in secondary education, justify the tendency to copy texts from the internet with the students’ lack of knowledge of the elaboration and referencing rules when carrying out academic work. The devaluation of cyber-plagiarism, resulting from the unconsciousness of the moral seriousness of students’ actions, justifies this practice. Sanchez and Innarelli [18] add other factors, such as personal beliefs and values, the influence of the group, pressure situations, the expectation of obtaining easy results, and confidence in the ability to circumvent controls. Some authors [13, 18] emphasize the idea that the practice of cyber-plagiarism becomes easier by resorting to “Cut-paste,” as it is enough to click on the Ctrl-C keys to copy a document and Crtl-V to paste it into another. The easiness of access to downloading from sources and the gratuity are some of the reasons pointed out by Kokkinaki et al. [19] for the dissemination of cyber-plagiarism. The possession of devices and connection to the internet are also factors to consider [13]. For other researchers [20, 21], the justification for this act is intellectual laziness. Galloway’s study [22] highlights other reasons, such as the school culture centered on grades, resulting from parental pressure to achieve high results, as well as little concern for the learning process. The author, as well as Permana and Santosa [13], relate cyber-plagiarism/plagiarism to the sacrifice of intellectual integrity. Summing up, several factors that lead to the practice of cyber-plagiarism fraud can be listed, namely those that result from the absence of a policy by educational institutions to act in the prevention of this problem, promoting actions in the field of digital citizenship that allow knowledge of the problem and how to avoid it. On the other hand, we find academic reasons that justify behaviors and attitudes in which students sacrifice their ethics. The easiness associated with the practice of fraud and the obtention of better results, with little effort and time, as well as the devaluation of learning, are other factors

316

F. Vicente and M. Meirinhos

to consider in the field of this problem. Digital citizenship values are not present in students who practice cyber-plagiarism. This is a violation of ethics, as it represents dishonesty towards teachers, colleagues, and authors, whose copyrights have not been respected. 3.2 Cyber-Plagiarism and Ethics In an academic context, it is essential to develop a set of values and principles associated with integrity and digital citizenship in the students. It is precisely to these values that one should pay attention when addressing the issue of cyber-plagiarism. These practices go against the values that shape the ethics of any responsible citizen. Dishonesty restrains a quality learning experience and provides space for various crimes of this nature to occur, e.g., plagiarism, cyber-plagiarism, copying, buying works, etc. [8, 13, 23, 24]. Bonette and Vosgerau [25] concluded that the ethical dimension underlying the appropriation of information is not dealt with by high school students, who have never tackled this topic. Here we can identify a failure of the educational institutions, which do not integrate this issue into digital citizenship education. As Granitz and Loewy [15] argue, illegal behavior in school can result in future illegal conduct in the workplace and within the community. Consequently, it is important to understand this phenomenon in order to correct it. It is imperative to promote learning about copyright and its violations as well as the respective punishments to also counteract the high likelihood of elementary and secondary school students to engage in illegal downloads of music and videos from the internet. Several studies reveal a lack of awareness of the moral gravity of plagiarism and cyber-plagiarism [14, 17]. The ethical concerns are completely absent from the practices of digital natives who access, copy, paste, and share what they find written on the internet, without morally validating these acts, as Pinto and Guerrero [26] observe: “They are navigators in oceans of information that present a relaxation towards written culture, in favor of images, sound, multimedia. They do not know the value of information and lack a qualitative framework to assess it. They do not know how to successfully delimit their information needs and face an advanced search. They also ignore the ethical/legal aspects of information use” (p.29). Cyber-plagiarism is one of the paradigmatic examples of how technological evolution deregulates behaviors and previously established values and leads to little reflection and, consequently, little conscious use of emerging technologies. This practice goes against the recognition of copyright, namely individual property rights. Therefore, any individual who uses the result of artistic or intellectual production must necessarily acknowledge the credits of that work. For this purpose, it was important to create legislation that foresees the existence of the crime of plagiarism and frames this practice, distinguishing its various typologies and their punishments. In Portugal, the Copyright and Related Rights Code was created to safeguard the interests of authors and was developed by the Artists’ Rights Management Association, dating back to 1985. These rights are established by law and published in Decreto-Lei n.º 143/2014, September 26th. It is a responsibility of the General Inspection of Cultural Activities to guarantee its observance. At the international level, it is the World

Digital Citizenship: The Cyber-Plagiarism Dilemma

317

Intellectual Property Organization (WIPO), a United Nations agency, that ensures compliance with international standards in this area. There are also several agreements that the Berne Convention is the depositary of, as well as WIPO Treaties on Copyright and Performances and Phonograms. The issue of copyright is closely associated with the Open Access movement. Only the author of an intellectual or artistic work can freely use the result of their creative act. The Open Access movement or Creative Commons licenses may allow others to use their work, by respecting their scope. The discussion around the importance of the Open Access movement in preventing plagiarism has divided researchers, as demonstrated by Ocholla and Ocholla [27]. The possibility of accessing almost unlimited knowledge and information through the internet is the same as copying phrases, sentences, or paragraphs from one document to another. Although it is very easy to plagiarize, some authors [28] argue that the Open Access movement is an ally in detecting and preventing plagiarism, precisely because it allows everyone to read and detect plagiarism, leading to greater care with texts intended for publication. The creation of Creative Commons licenses aims to adapt copyright to digital information and to prevent plagiarism by providing conditions under which the work can be used. The typology of Creative Commons licenses is quite diverse. For this reason, as part of digital citizenship education, teachers and schools should provide knowledge on the variety they encompass. The concern for copyright and its dissemination among primary, secondary and high school students justified the creation of the Segura Net website [29] which aims to inform its users about plagiarism and the ways in which it should be avoided. Similarly, the School Libraries Network (RBE) has also developed various actions, within the scope of digital literacy, to promote behaviors that are part of an ethical and responsible conduct, particularly among children and young people. On the RBE website, there is a duty section that includes the duty of not to plagiarize. Regarding PASEO, this framework advocates the values of Integrity, Responsibility, and Freedom, as well as multiple literacies, one of the measures that can be taken to prevent cyber plagiarism. Reference frameworks as DigCompEdu also incorporate actions aimed at mitigating these problems. In Portugal, the National Strategy for Citizenship Education also implicitly integrates respect for copyright when it determines the respect for Human Rights and the recognition of democratic citizenship values. Citizenship and Development, addressed as a cross-curricular area in primary education, as a disciplinary area in secondary education, and again as a non-disciplinary, cross-curricular area in high school education, has got the attention of the European Council, which has published various materials and developed some actions to fight against and prevent behaviors that violate copyright, specifically plagiarism. These concerns have spread throughout Europe, namely in Portugal. However, we believe that these actions are still diffuse, not assuming a dimension that mobilizes various fronts in this fight. In the case of Citizenship and Development, this issue is not expected to be addressed, but there is the possibility of integrating this discussion into one of the presented topics. In short, respect for copyright and intellectual property, resorting to Open Access, can prove to be a weapon against cyber-plagiarism, and the contribution that the guidelines

318

F. Vicente and M. Meirinhos

that advocate for the integral training of students as responsible and conscious digital citizens can provide in this matter is very important.

4 Measures to Combat Cyber-Plagiarism The measures to fight against cyber-plagiarism are directly related to the development of digital citizenship skills, as they require the responsible and ethical use of digital technologies. The consequences of this problem affect students, teachers, and institutions. Firstly, they affect learners, whose ability to learn throughout their educational journey will be compromised. In primary, secondary, and high school education, students do not adequately develop skills such as synthesis, research, collection, and proper treatment of information. Consequently, at more advanced levels, students are unable to write original texts based on the information they gather from various sources, revealing a great propensity for cyber-plagiarism. On the other hand, this action, performed easily and without criteria, inhibits their analytical ability and the adoption of a critical attitude towards the information they access, not even making the distinction between important and accessory information for the task they intend to perform. Therefore, it is imperative to find solutions that focus mainly on prevention and not so much on sanctions or punishments, whether they are stipulated by law or integrated into regulations of institutions or entities responsible for this matter. As cyber-plagiarism is a problem that involves several actors, solutions require concerted actions from governmental entities, educational institutions, teachers, and finally, students. It is up to governmental entities to provide training opportunities for teachers [3] so that they can promote a culture of common integrity in educational institutions at all levels. Regulation, in defining precisely the acts that constitute this practice and the sanctions to be applied transversally, should emanate from these bodies. After defining the rules and norms and their respective sanctions, the application of existing legislation will remain as a deterrent measure against the practice of the crime. Still within the sphere of action of governmental entities, we advocate the inclusion, within the Citizenship and Development area, of a topic aimed at promoting digital citizenship, which concerns the responsible use of the internet and ICT in a conscious and informed manner, respecting the norms of conduct, rights, and duties established in accordance with ethics and morality. It is essential to clearly define what is meant by cyber-plagiarism, as proposed by various authors [3, 16, 30]. Educational institutions should establish rules and regulations to follow in case of the detection of this practice, recommending penalties to be applied [3, 30]. These rules and regulations could largely correspond to general laws, foreseeing situations in line with their particularities, in the internal regulations of the concerned institutions, as proposed by Valadar [16]. It is fundamental that educational institutions promote preventive activities [3, 30], such as conducting workshops and awareness-raising actions aimed at students so that they respect copyright and recognize cyber-plagiarism, alerting them to the existence of punishments established in the legislation to be applied to perpetrators of crimes or frauds related to cyber-plagiarism (plagiarism). As such, two fundamental goals are fulfilled: informing and raising awareness.

Digital Citizenship: The Cyber-Plagiarism Dilemma

319

The dissemination of plagiarism detection software is also a combat measure that can be implemented by institutions, as Rico-Juan et al. [31] argue. School Libraries can also play a relevant role. They are responsible for developing and providing literacy guides, working aids to adopt from a transdisciplinary perspective, with clear rules for work preparation. Therefore, teachers should take advantage of the contribution of this service to promote training directed to students. Furthermore, it is the teachers’ decision to assess their students’ writing skills (synthesis, note-taking, paraphrasing) and invest in their promotion and development. This will undoubtedly contribute to a better students’ performance. On the other hand, providing information on the academic integrity policies and responsibilities adopted by the educational institution that students attend may warn them of the problem and help prevent it. It should also be noted that the attention and coherence in teachers’ practices inevitably influence students’ behavior. If they are used to memorizing and reproducing ideas, words, and phrases from others, which they memorize, it makes no sense to penalize them for this practice [32]. As proposed by Williamson & McGregor [33], it is essential to develop other teaching and evaluation practices, as the information collection processes used to classify, which mainly measure knowledge and do not assess skills, in some way encourage this practice. However, the use of outdated evaluation strategies somehow encourages cyberplagiarism and inhibits the development of critical thinking in students as a strategy to combat this dishonest practice [34]. It is also necessary to challenge students to build on new knowledge based on acquired knowledge. To do so, active methodologies are a solution that needs to be implemented, as they are much more challenging and allow students to develop skills and not just retain crystallized content over time. When proposing a task to students, the teacher must provide clear guidelines regarding its structure, as proposed by Valadar [16], which will prevent the possibility of students quickly accessing works already done and published in cyberspace that can be easily copied and submitted without proper authorial references. On the other hand, the proposal of having students orally defending their work is also presented. The following diagram visually summarizes the measures presented to fight against cyber plagiarism (See Fig. 1). In short, we believe that the best way to fight against cyber-plagiarism is to unite the efforts of all those involved in promoting awareness and consequences of this phenomenon. On the other hand, the investment in prevention, through digital citizenship education, will be crucial for the success of a mission that must be shared by all potential victims.

320

F. Vicente and M. Meirinhos

Fig. 1. Measures to fight against cyber-plagiarism

5 Conclusions The phenomenon of plagiarism is not new, but the expansion of the internet and the digital society has widened its scope and made it recurrent in primary, secondary, and high education. The impunity of this fraud has spread the wrong idea that everything available in the virtual space can be used, and students state various reasons for this practice. This article presents some useful proposals to fight against this problem, namely the creation of workshops and training for students, regarding the concept of cyberplagiarism/plagiarism, the consequences of committing this fraud, and the sanctions to be applied. Investing in an effective policy of detection, prevention, and regulation will spread a culture of integrity in schools, supported by moral, ethical, and civic values that are transversal to institutions with educational responsibilities. The action developed by School Libraries and within the scope of Citizenship and Development, whose topics propose the approach of Digital Citizenship, will contribute to knowledge of rules and regulations related to respect for copyright and the use of Open-Source resources. It will be the teachers’ decision to design activities for the development of information and media literacy, aiming to train active and committed digital citizens. This reflection, based on a literature review, aims to contribute to the implementation of practices to fight against plagiarism and cyber-plagiarism, which involves everyone and aims to combine their efforts, thus allowing for serious investment in education for digital citizenship. Cyber-plagiarism should be observed within a broader framework of digital empowerment of schools, the development of teachers’ digital competence, and students’ digital citizenship, preventing digital plagiarism and promoting the production of original works, guided by respect for copyright, according to an unquestionable ethical behavior, thus avoiding the use of punitive measures.

Digital Citizenship: The Cyber-Plagiarism Dilemma

321

References 1. Comas R, Sureda J (2010) Academic plagiarism explanatory factors from students’ perspective. J Acad. Ethics 8:217–232. https://doi.org/10.1007/s10805-010-9121-0 2. Redecker C (2017) European Framework for the Digital Competence of Educators: DigCompEdu. In: Punie Y (ed), EUR 28775 EN, Publications Office of the European Union, Luxembourg 3. Comas R, Sureda J (2007) Ciber-Plagio Académico. Una aproximación al estado de los conocimientos. Revista TEXTOS de la CiberSociedad 10 4. Patterson RW, McCall M (2011) Cyberplagiarism and its implications for modern pedagogy. Int J Inf Commun Technol Educ 7(3):67–75 5. Merz JP, Bennington C (2007) The different faces of plagiarism: from detection to correction. J Inf Syst Educ 18(4):455–466 6. Martorana CM, Thomason SB (2017) Digital plagiarism: the role of society and technology in defining and addressing misconduct. J Acad Ethics 15(4):261–276 7. Wang X, Zhu L, Gao W (2013) Cyberplagiarism: a study of students’ perception. J Educ Comput Res 49(2):187–200 8. Dumitrina NO, Casanovas M, Capdevila Y (2019) Academic writing and the internet: cyberplagiarism among university students. J New Approaches Educ Res 8(2):112–125 9. Kanwal F, Azeem M, Iqbal M (2020) Exploring students’ perception of cyberplagiarism: a qualitative study. J Inf Technol Educ 19:67–81 10. Horkoff T (2021) Writing for Success – 1st Canadian H5P Edition. BCcampus 11. Rodrigues S, Lopes C (2019) Plágio na Educação: reflexões em torno da literatura internacional e nacional. Cadernos de pesquisa, pp 89–106 12. Valadar A, Meirinhos M (2016) O ciberplágio como fenómeno emergente no ensino secundário. Revista Electrónica de Investigación y Docencia 15:7–22 13. Permana GY, Santosa H (2018) EFL students’ perception on plagiarism. in Rumimpunu CI et al (eds) Language in the Online and Offline World 6: The Fortitude, pp. 128–133. Surabaya U. Petra Press 14. Sureda J, López MM (2014) El fenómeno del plagio académico: niveles de extensión y características de éste en las diferentes etapas educativas. In: Comas R, Sureda, J (ed) El plagio académico en educación secundaria: características del fenómeno y formas de intervención, pp. 46–68. Palma EIC 15. Granitz N, Loewy D (2007) Applying ethical theories: interpreting and responding to student plagiarism. J Bus Ethics 72(3):293–306. https://doi.org/10.1007/s10551-006-9171-9 16. Valadar AI (2012) O plágio em contexto escolar: estudo de caso numa escola de ensino secundário. Escola Superior de Educação – Instituto Politécnico de Bragança 17. Rasco FA, Gil MM, Rocio RV (2014) Plagio y ciberplagio entre el alumnado de ESO de Andalucía. Primeros resultados. In: Comas R, Sureda J (ed) El plagio académico en educación secundaria: características del fenómeno y formas de intervención, pp. 69–84. Palma: EIC 18. Sanchez O, Innarelli P (2012) Desonestidade acadêmica, plágio e ética. GV-executivo 11(1):46–49 19. Kokkinaki AI, Demoliou C, Iakovidou M (2015) Students’ perceptions of plagiarism and relevant policies in Cyprus. Int J Educ Integrity 11(3):1–11. https://doi.org/10.1007/s40979015-0001-7 20. Orozco OBO (2011) El plagio, una práctica cotidiana. Revista LOGOS CIENCIA TECNOLOGÍA 2:104–111 21. Nongxa L (2020) Intellectual laziness and academic dishonesty: A threat to academic freedom?. South Afr J Sci 116: 5 Special Issue

322

F. Vicente and M. Meirinhos

22. Galloway MK (2012) Cheating in advantaged high schools: prevalence, justifications, and possibilities for change. Ethics Behav 22(5):378–399 23. Ferreira MM, Persike A (2014) O tratamento do plágio no meio acadêmico: o caso USP. Sigmótica 26(2):519–540 24. Mahabeer P, Pirtheepal T (2019) Assessment, plagiarism, and its effect on academic integrity: Experiences of academics at a university in South Africa. S Afr J Sci 115(11–12):32–39 25. Bonnette LMC, Vosgerau DSR (2010) O plágio por meio da internet: uma questão ética presente desde o ensino médio. Educação em Revista, Marília 11(2):7–22 26. Pinto M, Guerrero D (2014) La importancia de las competencias informacionales para afrontar el plagio académico: estrategias para la alfabetización informacional del alumnado de secundaria y bachillerato. In: Comas R, Sureda J (ed) El plagio académico en educación secundaria: características del fenómeno y formas de intervención, pp. 28–36. Palma: EIC 27. Ocholla D, Ocholla L (2016) Does open acess pevent plagiarism in higher education? Afr J Libr Arch Inf Sci 26(2):1–15 28. Brandt J, Gutbrod M, Wellnitz O, Wolf L (2010) Plagiarism Detection in Open Access Publications. 4th International Plagiarism Conference, p 5 29. Segura Net. http://cctic.ese.ipsantarem.pt//jogos/direitos-de-autor-3ociclo-conteudo/. Accessed 02 Apr 2023 30. Berlinck RG (2011) The academic plagiarism and its punishments - a review. Revista Brasileira de Farmacognosia, 21, pp. 365–372. São Paulo 31. Rico-Juan JR, Gallego AJ, García Avilés JM (2016) Estrategias para programar la detección de plagios en actividades basadas en texto. In: Actas de las XXII JENUI, pp.187–194 32. Pennycook A (1996) Borrowing others’ words: text, ownership memory and plagirism. TESOL Q 30(2):203–230 33. Williamson K, McGregor J (2011) Generating knowledge and avoiding plagiarism: smart information use by high school students. Sch Libr Res: Res J Am Assoc Sch Librarians 14:165–186 34. Shkvyr O, Haidamashko I, Tafintseva S (2020) developing critical thinking in younger pupils using ICT. BRAIN. Broad Research In Artificial Intelligence And Neuroscience 11(2): 230– 242

No More Cookbooks? No, More Cookbooks. Reflections on a Print Book in the Age of Digital Media Suzana Parreira(B) Faculdade de Belas-Artes, Centro de Investigação e de Estudos em Belas-Artes (CIEBA), Universidade de Lisboa, Largo da Academia Nacional de Belas-Artes, 1249-058 Lisboa, Portugal [email protected]

Abstract. Despite the fact that ebooks are increasingly more popular, some print cookbooks are still being sold in large numbers. Discussing the cookbook specific nature, this essay reflects on how digital media practices made print books sales decrease in different genres whilst print cookbooks kept thriving. Change in publishing in the digital age appears to show a pattern for other book genres that doesn’t apply to cookbooks in a similar way. Considering some iconic cookbooks published in Portugal in the last 100 years as case studies, the paper navigates both the technical-scientific and narrative nature of cookbooks, whilst also looking at the relationship between form and content, plus how that impacts the reader’s choice regarding print and digital books in food-related contexts. Throughout time the cookbook as an instrument of power, a tool for change and a source of knowledge evolved to accommodate not only the print book but also the digital, keeping its specific nature of being both a handbook with recipes and a book to be read and visually experienced as part of an interplay between different components. We argue that, as food will remain a major contender in digital media landscapes in the near future, phygital becomes the norm for cookbooks and the digital dimension of food will change consumption influenced by an invisible algorithmic logic and culture. The cookbook of the future will certainly also be shaped by AI presence, regarding authorship (recipe choice, diet trends) and cookbook typology (phygital, immersive). Keywords: Portuguese Cookbooks · Visual Narratives · Communication Design · Food Design · Phygital Experiences

1 Food, Media and Cookbooks 1.1 Anatomy of a Cookbook Although the cookbook goes back many centuries, its production increased as well as its availability in the second half of the 20th century due to a large food abundance period. In the last decades, the publishing food-related market has grown with the digital era. The appetite for cookbooks has only increased since digital and social media became © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 323–335, 2024. https://doi.org/10.1007/978-3-031-47281-7_26

324

S. Parreira

omnipresent in our everyday life, with epublishing prospering in different areas. Food related artifacts and mediums achieved its peak in the beginning of the 21st century, with mixed media playing an important role in the cooking mania installed for both old and new media consumers of newspapers, magazines, cookbooks, food blogs, cooking platforms or social networks [1]. As a cultural artifact, a cookbook is a system of different components working together in order to provide an editorial output. However, cookbooks are always more than writing and cooking: for the final result, one must also consider layout, imagery, typography and (if applicable) the printing and binding work that makes a print book—a strategy involved in expressing information and creating narratives through design. Because a cookbook is build “on linguistic, visual, typographic, and embodied or experiential modes of literacy in order to understand the text” (p.5) [2], the relationship with the reader relies, additionally, on food as a metaphor (or a culture proxy) with strong historical, social and cultural significance. Visual narratives create a bond with the reader by connecting memories with sensorial experiences, individual and collective events with personal and social identity that translates on cultural heritage [3]. Publishing practices have design at its core when ideas are transformed into editorial products, articulating text and image and creating a relationship between various signs, in a mediation process between authors and readers. With the hypermediacy that marks the digital era, content is borrowed from one medium to another without the medium itself being appropriated. In an ideal context, there would be no difference between the experience of using a cookbook or seeing the same content on the screen, however that’s not the case. Remediation attempts to restore this gap, “repurposing” the reuse with a redefinition of both a cookbook’s content and medium [4]. Even if not always recognized as the most valuable literature [1], cookbooks set representations of the world, can redefine both our cooking and eating habits as well as depicting different types of knowledge. Using some iconic cookbooks published in Portugal in the last 100 years as a starting point, we will cover the technical-scientific and narrative nature of cookbooks, and look at how form and content relate with each other, discussing the impact in the reader’s choice of cookbooks, either print or digital. 1.2 The Cookbook as an Instrument of Power Throughout time, the printed food-related book was frequently a journal (usually from royalty or upper classes), a technical recipe book written by chefs or by women (either for chefs or the domestic cook), or a history compendium/art statement set from the author’s unique voice and personal choices. The oldest Portuguese recipe’s manuscript (16th century) belonged to Infanta D. Maria de Portugal, who married an Italian nobleman and moved to Naples, taking some ancient manuscripts with her to cook favourite recipes away from home [5]. The journal is a living source for Portuguese culinary habits at the time: for example, the excessive mixture of condiments and the abundance of spices from the Portuguese empire frequently appears in daily meals. Not just a recipe book, it’s a personal document meant to be a pragmatic compilation of food-related knowledge, kitchen tools and “homemade remedies”, revealing interesting information about a lady of the 16th century Italian high nobility. Even though the manuscript certainly influenced other women in the same

No More Cookbooks? No, More Cookbooks

325

social circle, it may not have had a major impact in Portuguese cuisine at the time as it was not until the end of the 1960s that the cookbook was published in Portugal. In the following centuries, chefs writing about cooking and gastronomy were men, using their voice to set expectations and determine how recipes should be done as well as influencing eaters’ habits: “when what you ate defined who you were, when the meal functioned as a register of social reputation, one needed to be told not only what to eat, but when, where, and how to eat it” [6]. The oldest printed Portuguese cookbook known to this day is A Arte de Cozinha by Domingos Rodrigues, first published in 1680. The author was a chef in different aristocratic houses in the north of Portugal, until he was named Master of the Royal House, and his work managed to influence Portuguese cuisine until the middle of the following century. Compared to other European countries, before the 20th century, the publication of cookbooks in Portugal was scarce [7]. Until the 1950s a handful of cookbooks were published, mostly centered into national cuisine (an important question for Estado Novo, the authoritarian regime in power) or authored by women for the domestic cook (frequently, other women). Even so, cookbooks written by women for women only became popular in Portugal in the 20th century. O Livro de Pantagruel by Bertha Rosa-Limpo (1945) is one of the best known and most successful cookbooks in Portugal (currently with more than 80 editions). In a postwar period, when women were expected to take care of the house and their children, Bertha Rosa-Limpo embarked across the world in a professional career as a lyrical singer. She compiled 3,000 recipes from Portugal and the rest of the world for the first edition, with a special place in the cookbook for Italian cuisine, something that ended up boosting a national wave of interest. Today, it is a beloved book for generations of Portuguese people, being a living memory of their grandmothers’ kitchen shelves. In the digital era, journals or personal documents became part of a network of selfpublishing platforms, with blogs and social media replacing the paper written page and setting a range of information widely accessible to the public. The food blogosphere replaced personal diaries with a difference: words, photos and stories were shared with thousands (and then millions) of people, not bound by geographical borders, time zones or physical proximity. Recipes became available through platforms bringing together users and setting new search tools, by ingredient, price, type or difficulty level. Changing the public’s habits and food patterns through the recipes chosen and cooked by celebrities on tv or social media stars, becomes part of the circle of influence. A close relationship between print and digital is common when it comes to TV shows and their accompanying print cookbooks, the same happening with food bloggers turned cookbook authors who emerge as influencers. Case Study 1 – Cozinha Tradicional Portuguesa (1982). Nowadays, culinary shows are a prolific source for the publishing market, with authors creating content both for the screen and for the page, and each TV show having its own cookbook, either print or ebook, or both. Nevertheless, the connection between TV and printed materials is not new. Maria de Lourdes Modesto started her pioneering live cooking program called “Culinária” on the Portuguese national TV in 1958. For many years, compiling a book that would bring together the most authentic and complete traditional Portuguese recipes was just an abstract idea.

326

S. Parreira

Published in 1982, Cozinha Tradicional Portuguesa by Verbo Editors reflects several social and cultural dimensions, combining the heritage of different cuisines with the construction of a national culinary identity [8]. For each region there’s a small text with its characteristics, habits, table protocols and quotes from individuals of literary, political and artistic elites. With 800 recipes covering the whole country, from Entre Douro e Minho to Algarve, Madeira and Açores, it’s the most iconic cookbook in Portugal, having sold over 400 thousand copies. The author Maria de Lourdes Modesto considered the cookbook a collective endeavour, highlighting the contribution of the graphic designer Sebastião Rodrigues and the photographers Augusto Cabrita and António Homem Cardoso to the narrative structure of the book, as the design project creates a permanent dialogue between recipes, text, photography, typography, together with the binding and printing work. Cozinha Tradicional Portuguesa marks the beginning of a new era in Portuguese cuisine and changes the way people see regional cuisines, for the first time creating visibility for different recipes and the specific versions in every region. Until today, it remains the “bible” for cooks, chefs, hospitality students, and the public in general, recognizing Maria de Lourdes Modesto as the main reference for traditional Portuguese cooking. Despite its relevance, Cozinha Tradicional Portuguesa is not available in digital format and it’s still on print at the present. New generations get to know this iconic book on the shelves of schools, libraries or family kitchens, rather than the screen. After four decades and by editors’ choice, this cookbook remains only a physical artifact, on sale until now and being bought by people of all ages as part of strategy based on distinctness and the book being recognized across generations as an iconic historical cookbook (Fig. 1).

Fig. 1. Cozinha Tradicional Portuguesa (1982) front cover (left). Double spread (right), recipes for fish and seafood, Entre Douro e Minho (p. 20) and photos (p. 21), fish stall at Mercado do Bolhão (top) and Hake Poveira Style, Póvoa do Varzim (bottom).

No More Cookbooks? No, More Cookbooks

327

1.3 The Cookbook as a Tool for Change Cookbooks are also vehicles for different world views, creating visibility for certain cuisines or authors but also providing a fine way to address racial disparity in food, forgotten minorities or invisible communities: “[t]hese books are artifacts that impose the stillness of a shot, obtained through the printing process, on a sector like cookery, whose recipes, having their own precise storytelling structure and sequence, are subject, wherever there remains an oral tradition, to unceasing interpolations, changes, translations, betrayals” [3]. In 1938, edited by the Portuguese Institute of Fish Conservation, Como cozinhar sardinhas portuguesas de conserva (How to cook canned Portuguese sardines) is a cookbook directed to women who are responsible for the domestic kitchen, with recipes written by renowned cooks in Portugal in order to prepare Portuguese Canned Sardines that “can be served in absolutely original ways, without, however, losing any of the nutritional properties that made them famous” (p. 1). Mostly used as a resourceful, cheap alternative, canned fish is shaped into a cookbook, whose recipes elevate the modest ingredient to a new level, with illustrations by Swiss graphic artist Fred Kradolfer. Mutations in recipes are both natural and welcome, creating a continuous food discourse based on its symbolic nature, characterized by enormous complexity and associated cultural capital. Many people contribute to provide a unique voice and confer credibility and desirability to the book but the author assumes a prominent role. Information disseminated by food celebrities is not just teaching cooking skills or know-how about ingredients, but also translate into desirable lifestyles, although “even in an omnivorous era, the cultural products of some groups are valued more than others, reflecting hierarchies of class, race and gender” (p.22) [9]. Therefore, chefs’ cookbooks are always more than just recipes and techniques, their influence extended to what is validated (or not) concerning a certain lifestyle or status quo. Different Portuguese chefs become cookbook authors in the 21st century, capitalizing on the attention their restaurants and media presence provides. In 2011, Um Chef em Sua Casa (A chef at your home) is chef José Avillez’s first cookbook (with over 16 thousand copies sold) displays a set of techniques and advice with simple recipes, looking for a combination of quality products and basic preparations to obtain a refined final dish. As Maria de Lourdes Modesto asserts in the preface “the book is intended for people with good taste, beginners or connoisseurs, who seek in the exercise of cooking, the pleasure of doing well, seducing, dazzling and above all giving pleasure to others” (p. 9). A certain lifestyle, and the way to achieve it, is outlined in the cookbook—having the cookbook allows for the reader to become part of the group. Paradoxically, it sometimes can reinforce their sense of belonging to the very class they are trying to escape [10]. Most cookbooks in the 20th century are straightforward collections of recipes, frequently with little or no photography, which increasingly changes in the 2000s, when images become central to the cookbook, turning it as visually appealing as it has been practically useful. Consequently, food diets and eating habits have changed influenced by what is printed, disseminated and marked as acceptable and socially validated: “[t]o think critically and politically about food involves an exploration of the cultural politics of power and appropriation, of how meaning is constructed and identities constituted” [11]. In 2015, chef Henrique Sá Pessoa, writer José Luís Peixoto and photographer Nicolas

328

S. Parreira

Lemonier publish A Viagem do Salmão (The Salmon’s Journey). A travel and memory book, it’s a journey of encounters and experiences around salmon by a chef, a writer and a photographer, with recipes introducing this ingredient’s culinary and historical influences from Portugal to Norway and to Japan. The cookbook is sponsored by NSC (a public company owned by the Ministry of Trade, Industry and Fisheries of Norway), as part of a campaign to increase the value of Norwegian seafood resources. To introduce an ingredient in a country’s culinary culture is common, and it has been done before by the Norwegian in the mid-1980s with the invention of salmon sushi, a concept they sold to Japan and the world. A similar campaign happened with mackerel in Portugal in 2012, with Portuguese chefs playing an important role selling the fish to the domestic cook, increasing the sales to the fishmen, contributing to more sustainable practices and changing the population’s food habits [12]. Case Study 2 – Lisboeta: Recipes from Portugal’s City of Light (2018). A particular city that inspires a cookbook, written in celebration of a place and a sense of belonging by its author, is a common typology in culinary writing: a chef goes in search of a particular dish through history, people and places. Lisboeta: Recipes from Portugal’s City of Light is Nuno Mendes cookbook about Lisbon, a set of essays and collected memories about a chef’s ‘heart’s home’ with Portuguese contemporary recipes, and tematic inserts about Portuguese Discoveries, Coffee Culture, Taverns, Fish, Beach Life, and St. Anthony (Lisbon’s patron saint). Lisbon’s food scene, steeped in centuries-old traditions, is shaped into themes that are the cookbook backbone, mixed with Portuguese recipes. The cookbook is available in Portuguese and in English, the latter having a print and a digital version with the same content concerning photography, essays and recipes. Nuno Mendes’ work has been broadly acclaimed in the UK, his dishes widely known and used as a blueprint for understanding Portuguese cuisine. In the British mediascape, the chef’s cookbook is also seen as a guide for Lisbon, as well as a starting point for recognizing and cooking Portuguese food. Even if some recipes are the chef’s signature dishes with a contemporary approach more than a traditional take on the existing versions, for the cookbook’s readers it is seen as Portuguese cuisine nonetheless. The book’s structure with the smaller inserts and wide double spread’s photos makes for a difficult translating into epublishing, the editor having chosen to mark different content with different colours and resizing the photos. Although the content is the same, the Kindle version, for example, is a far call to the more immersive and magical experience the print book offers, the digital version being a small part of the cookbook’s total sales (Fig. 2). 1.4 The Cookbook as a Source of Knowledge To increase food literacy, recipes tell stories both about food and about the people who wrote them, as well as those they’re intended to. In that sense, cookbooks are not descriptive but in fact prescriptive, showing a glimpse of “the culinary worlds their authors would like to see” (p. 66) [13]. By writing the cookbook, the author contributes to creating a culinary world inexistent before, with consequences to the reader and the vast context in which it belongs (for example, a national cuisine, a special diet or an ingredient).

No More Cookbooks? No, More Cookbooks

329

Fig. 2. Lisboeta: Recipes from Portugal’s City of Light (2018) front cover (left). Double photo spread (right) with a smaller insert entitled “Fish”. Design Charlotte Heal.

In 1941, Cozinheira Ideal (The ideal cook), written by Alda de Azevedo and published by Livraria Civilização in Oporto, had advice to offer about nutrition, domestic economy and party planning, supplying a set of 365 menus for lunch and dinner and covering a full year of meals for a household. Some recipes are simple, while others are not so straightforward, but all were tried and tested by the author, in a total of 2550, plus culinary wisdom and a detailed index for easy finding. The cookbook’s goal is to support women in their daily ordeal of feeding a family, with ideas and advice in order to increase culinary literacy and kitchen efficiency. Because “one ‘uses’ a cookbook—one does not passively ‘read’ or ‘observe’ the text” (p. 52) [14], the cookbook is literally seen as a source of knowledge, with cookbook authors using the discourse of their time to establish themselves as authorities on domestic matters. Several cookbooks are part of an approach to memory and cultural archive, such as A Doçaria Portuguesa (The Portuguese confectionery), a project by Cristina Castro started in 2017, and ongoing. At the moment, 3 cookbooks—South, Centre and North of Portugal—are part of a collection with an encyclopedic spirit and comprehensive information about the people, the recipes, gestures and habits that make the traditional confectionery of a country. The books are organized geographically, covering the territory and registering the recipes, their authors, shops that sell those recipes and other traditions. As a design option, different colours create distinction between the various volumes and provide identification for each one, with the same layout for the whole collection. The cookbook is visually vibrant, relying on photography by Gonçalo Barriga and illustration by Ana Gil, as well as ‘Bazar’ typeface by Olinda Martins—a revivalist font created from the collection of shops’ commercial signs in Oporto. The author Cristina Castro decided not to have a digital version for the cookbooks, however a YouTube channel with complimentary videos is available with open access (https://www.youtube.com/@NoP onto_pt). Both collective archive and place of memory, A Doçaria Portuguesa is not strictly a recipe book with the main focus being on the people, rather than the products: it’s part of a broader campaign to create visibility on the country’s food culture.

330

S. Parreira

Case Study 3 – Fabrico Próprio: The Design of Portuguese Semi-Industrial Confectionery (2008). Fabrico Próprio translates to “Own Production”, a term used by most Portuguese cafés to advertise their in-the-house production of cakes. It’s an encyclopedia containing all the pastry cakes that are part of a Portuguese breakfast or snack, usually eaten in a coffee shop. Altogether there are 92 cakes and their variations, with historical notes and a glossary of pastry terms to guide the reader through the history and culture around each cake. Presently with two editions (and a third to come shortly), also noteworthy are special dedicated inserts and essays about Castella of Portugal, Bolas de Berlim for All, a Special Recipe (Snail with Fruit and Ice Cream), Proper Places (Bakeries of great tradition), Sweet Memories, Diaspora, Cakes at Dawn and Rare Species. With only one recipe, the cookbook offers a global perspective on the semi-industrial confectionery in Portugal, where professional bakers make the same recipes everyday on an industrial scale, using the same ingredients and techniques to bake the cakes Portuguese people have eaten for generations. The social, cultural and economic flows that make the cookbook show an explicit connection between memory and everyday practises, as part of a historical path, material culture and personal experience. The project is a cake encyclopedia and the authors Frederico Duarte and Pedrita “see these cakes as manifestations of a national culinary culture, which incorporates external ingredients and formats […]. By allowing for plenty of variation and innovation in both ingredient combinations and formal configurations, this disseminated production has fostered a culinary culture where creativity trumps orthodoxy” [15]. Fabrico Próprio is, more than a conventional cookbook, the result of a research approach combining food and design with the aim to contribute to the discussion about Portuguese semi-industrial confectionery. The book design is by Atelier Carvalho Bernau, with an important visual dimension: photography by Soraya Vasconcelos, Namiro Kitaura, Pedro Garcia, Hugo Teixeira and Tomás Nogueira and illustrations by Rui Tenreiro, João Fazenda, Júlio Dolbeth, UIU, Benedita Feijó and Guida Casella. Previous 1st and 2nd editions are print cookbooks, and the third edition (expected in 2024) will remain a printed artifact with some online resources in open access to reinforce its encyclopedic nature (Fig. 3).

2 Reflections on the Future of Cookbooks 2.1 What is a Cookbook in the Digital Age? Both in print and digital, the cookbook’s popularity is directly connected with the digital age, after “the 1990s food and food related elements had become a major focus of interest of all old and new media. The first two decades of the new millennium record a remarkable amount of published cookbooks” (p.19) [16]. When settling arguments for the close relation between food and design across time, Margolin identifies the cookbook as “a designed artifact, one of many that range from restaurant facades and interiors to the creation of new items for the home such as tableware, dishes, glasses, and even tablecloths” (p. 379) [16]. Nevertheless, cookbooks are special artifacts that allow fictional food narratives to be created from texts, recipes

No More Cookbooks? No, More Cookbooks

331

Fig. 3. Fabrico Próprio (2012) 2nd edition front cover (left). Double photo spread (right).

and the living memories of the reader or the cook, frequently displaying a high level of intertextuality and shared visual signs. In the age of multimedia, the interplay between different media occurs only for the reader who knows both print and digital cookbook’s versions and can compare them [4], yet the synergies go beyond the functional: the digital era created the perfect conditions for food to be regarded as a popular destination and relentlessly used as a social marker. Ubiquity and hypermediacy characterize “the relation between food and media as interactive, and extremely important, as media pervade all spheres and all chains of contemporary food-ways, from certified labels to television chefs and blogs and recipes on the internet” (p. 2) [18]. In contemporary mediascapes, food is a common language adopted by everyone and used to communicate status, lifestyle, personal and collective identity or cultural shared values and principles. The increase in food blogs, digital recipe sharing, and social media posts about food is a sign digital platforms created an ideal space for food-related content. Despite that early success, cookbooks have become popular, thrived in the digital age and kept in print which shows a complex interaction between new and old media technologies and their mutual influence. Expectations about the e-publishing were that it would democratise cookbook publishing by reducing production costs but the trend for the traditional ink-on-paper format over digital remains strong, and “what has happened is a certain bifurcation of the market into high end, expensive, glossy colour cookbooks and a new boom in small publishing and self-publishing” (p. 32) [18]. Often, photography is used together with typography and the printing and binding work to create final results that turn cookbooks into art pieces, ‘objects of desire’ or coffee table books to be admired and shown, rather than read and used. This trend reinforces the rise in popularity of image-based social media channels like Instagram or Pinterest, usually recognized for having food-related content with a strong visual presence [19]. One of the main reasons why the digital cookbook doesn’t quite convince the readers is the interface, with many of them feeling that a screen in the kitchen is not the easiest way to follow a recipe while cooking it, with extra fear of ruining the device [18]. Other issues relate to how visually appealing digital cookbooks are, with some arguing

332

S. Parreira

that ebooks “still aren’t aesthetically pleasing despite the development of interactive apps and embedded content. There may be an important physical connection between the tactile, sensory nature of cooking and the flipping of pages, scribbling in margins, even staining for posterity, that can only happen with physical cookbooks” (p. 32) [18]. The connection between the reader and the physical cookbook relies on an experience also built on making notes or further annotations on the recipes, creating a bond that prevails and that digital books haven’t yet been able to achieve, as “e-cookbooks still don’t consistently offer the tools needed for readers to interact with the content in the same way” (p. 31) [19]. Reputation may also be an issue to the food-related digital media success, as users usually “perceive chefs as trustworthier than official advice, and cookbooks as more credible than Google” (p. 145) [17]. In addition, cookbook readers feel they get a lesser experience from a digital book, adding that the price is frequently not much different than the print book. It’s still prevalent the perception by the reader that the digital cookbooks offer little more than a pdf document and are “cheaper, pared-down versions of a title” (p. 33) [19]. The 2020s ideal cookbook is mostly a print book, typically resulting from a TV show, a food blog or a social media personality deal, frequently too by a renowned food writer or a celebrity chef. 2.2 Recipes for the Future Food is a common language being “everywhere in contemporary mediascapes, as witnessed by the increase in cookbooks, food magazines, television cookery shows, online blogs, recipes, news items and social media posts about food. This mediatization of food means that the media often interplays between food consumption and everyday practices, between private and political matters and between individuals, groups, and societies” (p. i) [17]. The lifestyles, habits and cultural heritage influence collective identities, as diets also shape personal health and well-being, both gaining extra relevance with digital media, online platforms and social media. This trend shows no sign of losing strength, so we argue food will remain a major contender in digital media landscapes in the near future, with cookbooks holding a relevant role. Presently, audiobooks and podcasts about food are a successful experience, with plenty of listeners worldwide. As much as it reinforces the central role food takes in media nowadays, it also shows how print cookbooks will not be replaced by these products and are often used simultaneously, rather than being interchangeable. Phygital is the new paradigm, although it can be argued that designing for the physical as well as the digital is not new. Living in a time when cooking shows are popular and cookbooks take up a lot of shelf space, the connection between TV and printed materials is several decades old. However, phygital as a concept reinstates that the connection, more than complementary, is built on mutual reinforcement of the physical and the digital together. It may, nevertheless, rely on readers to keep on moving away from intrusive digital technologies which can be a sign of digital saturation. Considering handwritten cookbooks have persisted through centuries and survived printing technologies and the digital era, the death of the print cookbook can be seen as exaggerated: in “the case of cookbooks [it] demonstrates societal attitudes and practical considerations around culture, society and food” (p. 30) [18]. Moreover, preserving and

No More Cookbooks? No, More Cookbooks

333

transmitting culinary traditions, especially those clearly rooted in oral tradition, has been through the handwritten and the print cookbook, and it will keep up in the future. Material cookbooks are editorial artifacts, printed and binded, that use typography and imagery in order to provide information regarding food, often in the form of recipes or instructions. It can be said that “[d]igital versions of the traditional recipe take advantage of its dynamic nature, enhancing the recipe with hyperlinks, photos, videos, and user comments” (p 1) [2]. That may be an advantage for a direct use, when the user looks for a certain recipe or wants feedback regarding an ingredient or process but the experience of reading a cookbook, and interacting with it, is still easier and more prolific to the reader than the digital interaction offered by digital cookbooks. Considering the rapid obsolescence some media show and having memory and knowledge as intrinsic to the cookbook nature, “if cookbooks are both a practical guide and a repository of future knowledge, one has to wonder what happens when the devices on which one reads become obsolete […] As a medium for recording and preserving favourite meals, then, the print cookbook offers enormous advantages that ebooks can’t duplicate and perhaps even undermine” (p. 32) [18]. Despite their diminished aesthetic appeal and (still) low functionality, we believe digital cookbooks will keep on expanding and sales will be growing in the next decade. The increasingly digital dimension of food can change, literally and metaphorically, the way food is consumed without most people realizing the differences, more and more influenced by an invisible algorithmic logic and culture. Cookbooks will undergo a similar transformation, with AI playing a part, both in authorship (recipe choice, diet trends) and cookbook typology (phygital, immersive). Nevertheless, the cookbook nature will not change, remaining, at the same time, rooted on narrative dynamics and technical dimensions—“a section in itself, with its own history, structure, specific language, is that of cookbooks: veritable handbooks for the preparation of foods” [3]. Conclusions Considering print cookbooks are still thriving nowadays and continue to resist the rise in popularity of e-book versions, we discuss this apparent paradox when compared to other types of books. The reason for this continuous success is the cookbook’s specific nature, both narrative and technical, and the different dynamics that the act of cooking or consuming food literature creates. Each person builds a relationship with food, while cooking from a print cookbook or reading about food, that is not yet replicable with e-books. Either because the screen is not welcome in the messy kitchen environment or because connecting with food through photography and the author’s memories and experiences is much more rewarding on paper, the digital cookbook is yet to be seen as a first choice or strong alternative to print. The cookbook can be an index for the design field and the future of technology, with the challenges between old and new media giving way to an increasingly phygital cookbook. Moreover, instead of having ebooks completely replacing print, the market will keep on building a close relation between both versions, with one reinforcing the other. Regarding AI presence in edition and authorship, even if recipes, especially ingredients, are not subject to copyright (confection techniques are a different matter), the accelerating market of AI bots brings new concerns about authorship of texts and pictures, as well as the cookbook’s profile and structure. At the moment, bots haven’t succeeded in

334

S. Parreira

duplicating the cultural and personal history that informs what people cook but it’s just a matter of time. Although these artificial intelligence machine learning tools can be very efficient and automate grammar checking or proofreading, the cookbook of the future can be completely generated by AI with a change of scale in recipe and food memory sharing. Will the reader connect with a non-identified author or will it accept an author created by AI (and will they be able to tell the difference)? Questions like these remain unanswered as we stand at the beginning of the way in AI everyday common use, seeing it simultaneously as a powerful helper and a threatening competitor to human editors and food writers.

References 1. Notaker H (2017) A History of Cookbooks: From Kitchen to Page over Seven Centuries. University of California Press, Berkeley 2. Fleitz E (2010) Cooking codes: Cookbook discourses as women’s rhetorical practices. Present Tense 1(1):1–8 3. Ferrara C (2019) Cookbooks. Narrative spaces and user manuals. Progetto Grafico (35) 4. Bolter JD, Grusin R (1999) Remediation: Understanding New Media. The MIT Press, Cambridge 5. Arnaut SD (1967) A arte de comer em Portugal na Idade Média. In: Arnaut SD, Manuppella G (eds) O Livro de Cozinha da infanta D. Maria de Portugal. Universidade de Coimbra, pp. XXIII-CXXX 6. Presnal S (2017) Art to table: the power of aesthetics in women’s cookbooks of the Belle Époque. Hist Reflections/Réflexions Historiques 43(3):63–81 7. Saramago A (2004) Para uma história da alimentação de Lisboa e seu termo. Assírio & Alvim 8. Abreu MMSGMPD (2018) Cozinha Tradicional Portuguesa de Maria de Lourdes Modesto: Contributos para a construção das identidades e do património culinário português (Doctoral dissertation) 9. Johnston J, Rodney A, Chong P (2014) Making change in the kitchen? A study of celebrity cookbooks, culinary personas, and inequality. Poetics 47:1–22 10. LeBesco K, Naccarato P (2016) Politics and the future of food and media. In: Leer J, Povlsen KK (eds) Food and Media: Practices, Distinctions and Heterotopias, p. 197 11. Baron IZ (2019) Cookbooks, Politics, and Culture. In Science, Technology, and Art in International Relations, pp. 185–193. Routledge 12. Parreira S (2020) Sustainability on the menu: The chef’s creative process as a starting point for change in haute cuisine (and beyond). In Experiencing Food: Designing Sustainable and Social Practice, pp. 47–51. CRC Press 13. MacClancy J (2004) Food, identity, identification. Researching Food Habits: Methods and Problems, vol 5, p. 63 14. Fleitz EJ (2021) (Re) Mixing up literacy: cookbooks as rhetorical remix. Community Literacy J 15(2): 6 15. Duarte F (2021) The everyday challenges of Portuguese semi-industrial confectionery. ReSEED. https://reseed.uc.pt/index.php/2021/09/06/the-everyday-challenges-of-por tuguese-semi-industrial-confectionery/. Accessed 04 Apr 2023 16. Tomascikova S (2020) Postmillennial Media–Discourses Where Food Cultures Meet Everyday Practices. Tekct: lektponny/S. Tomascikova. http://repo.ssau.ru/bitstream/Monogr afii/Postmillennial-Media---Discourses-Where-Food-Cultures-Meet-Everyday-Practicesmonograph-Tekst-elektronnyi-84084/1/Tomascikova%20S.%20Postmillennial%20Media% 202020.pdf. Accessed 04 Apr 2023

No More Cookbooks? No, More Cookbooks

335

17. Margolin V (2013) Design studies and food studies: parallels and intersections. Des Cult 5(3):375–392 18. Leer J, Povlsen KK (2016) Food and Media: Practices, Distinctions and Heterotopias, pp. 1–18 Routledge 19. Albala K, Larson C (2015) The Evolution of Cookbooks in the Digital Age. Food and Communication: Proceedings of the Oxford Symposium on Food and Cookery, in Oxford, England, United Kingdom (2015) 20. Dunlop LA (2016) E-Cookbooks: An analysis of profitability. Simon Fraser University (2016)

Designing a Creative Process Between Music, Performing and Visual Arts and the Business World Liliana Soares(B)

, Ermanno Aparo , and Rita Almendra

CIAUD, Research Centre for Architecture, Urbanism and Design, Lisbon School of Architecture, Universidade de Lisboa, Polo Universitário do Alto da Ajuda, Rua Sá Nogueira, 1349-063 Lisboa, Portugal [email protected]

Abstract. This paper is an opportunity for the business fabric and the performing arts to change. The introduction of creative process in the business world and in the performing arts sector proved to be a response to the effective improvement of the shows produced and business competitiveness. In this study, a territorial network system was developed consisting of professors and design researchers, a lighting company, a raw materials industry, a municipality, and entities related to the entertainment sector. With this work, it was possible to develop a system carrying of new semantic pathways, communication, performance, and interaction with people. The study contributes to the constant analysis of designing products with high efficiency and low environmental impact. The paper shows that the process of cooperation between different areas is a choice that respects and values the project partners, assuming a social commitment. Keywords: Creative Process · Territorial Network System · Communication and Performance · Aesthetic · Economic and Social Sustainability

1 Introduction The primary purpose of this study to identify and analyze lighting design as a system that completes and balances all the components that make up a product’s communication and performance. On the theatrical stage, lighting is an essential component of any event and light design is an example of synergy that conceives reality. This is the hypothesis of the development of lighting products that affect the entire organism, and incorporate historical and semantic references, reclassifying previous concepts in an updated existence as design requirements [1]. The very concept of liquid reality [2] requires constant research, challenging designers to create holistic and sustainable products. Due to financial, social, and public health issues caused by the COVID-19 pandemic, some entities in the show business were forced to eliminate disciplines from the creative process. This fact contributed to show design solutions becoming stagnant and devoid of change. This process disconnected from visual culture can contribute to the public ignoring the participation of drawing as an area of knowledge [3] in the communication © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 336–344, 2024. https://doi.org/10.1007/978-3-031-47281-7_27

Designing a Creative Process Between Music, Performing and Visual Arts

337

of a message. That is, the application and dissemination of a methodology that bears the culture of drawing is an opportunity to transmit knowledge to the audience of the shows. This circumstance associated with the fact that, since the end of the sixties of the last century, the sense of material culture has undergone changes in the perception and has contributed to the public’s detachment from some shows. On the one hand, it includes the secondary function [4] of the objects and on the other hand it contains the respect for products and the encounters with the other [5]. Regarding the emotional connections that individuals produce with material culture, sociologist Jean Baudrillard explained that, due to the conjunctures, speed, complexity, and contradiction that characterized that reality, people began to create affective relationships with objects, as if the objects were their domestic animals; the ‘Animal Domestique Parfait’ [6]. Later, Andrea Branzi takes Jean Baudrillard’s definition, applying it in the definition of a domestic furniture product system entitled “Animali Domestici” [7]. Donald Norman will enunciate the principles of Emotional Design (2005), reinforcing the sensory and knowledge need that people establish with objects, be it a relationship of attraction or a relationship of repulsion. As Donald Norman explains “(…) our behavior, the material things we have, are public expressions of our identity, of ourselves.” [8]. This truth is that people encounter things – be they objects, constructions, scenarios – in search of experiences. In this sense, the relationship that a spectator establishes with culture also includes the connection with the scenic space, the props, the scenography, the lighting. Considering the enigma of the development of culture, the anthropologist Franz Boas (1939) [9]. Explained that this mystery was limited to the study of the psychological and social conditions that are for humanity, and for the effects of historical events and the natural and cultural environment. Specific. That is, when one thinks of an object as a construction for a play or a musical concert, one must consider the spirit of the place [10] to which it refers. For this reason, designing can improve the relationship between those constructions and the context, exploring the competence of semiotics, conveyed by the drawing, by the identification of specific, unique, and inimitable symbols and by the recognition of ancestral constructive systems of the reference context. In the world of spectacle, this connection can be understood as an occasion to insert the factor of representation and acting with an audience in the act of designing, in the sense that, as [11] refers today, design is always an act of redesign. An action that is committed to the past to interpret the future.

2 Theoretical Framework This paper adopts the principle of communicating the message to the receiver without the need for interpretations. For this reason, a universal theme was sought, in the sense that the same rationality is assumed, varying its application. Thus, in terms of concept and theoretical framework, the paper referred to an universal concept, namely, the concept of nymphs, etymologically associated with Greek mythology as the spirits of fairies. In Ancient Greece, for instance, the muses represented the superior ideal of Art, considered daughters of Zeus and Mnemosyne with a very important place among the deities of mythology. Greek mythology has been a

338

L. Soares et al.

constant source of inspiration for the arts, both in Ancient Greece itself and in the civilizations that followed. Stories, legends, myths, gods, and heroes have attracted the attention of artists from different genres and areas. Rich in metaphors and with a strong symbolic component, mythology has been used over the centuries as a reference in works that, in some cases, have marked the history of humanity. Many of the elements - characters or narratives - were used to create behavioral references, explain natural phenomena or to individualize values and meanings. The Nymphs were the daughters of Uranus, considered minor deities related to Nature and associated with other gods such as Apollo, Poseidon, Dionysus, among others [12]. These female natural spirits were classified according to the different natural domains with which they were associated. In this case, this paper focuses on the typology of the Dryads, considered the nymphs of the woods, and associated with trees oaks. These Dryads, generous with all who respect Nature, are known for the spells and spells produced to protect the forest and its inhabitants, thus being known as the protectors of forests and Nature - an universal theme qualified of being communicated and understood by all. Unlike the Amadriads, who lived inside the trees, the Dryads were free to leave the trunks where they lived, to dance, fall in love and join human beings. Among the Dryads, we particularly remember Euridice, daughter of Apollo and bride of Orpheus, with whom she becomes the protagonist of one of the most remarkable love stories in mythology. Orpheus, a deity mainly related to the musical arts, poetry, and other divinatory arts, was the theme of the works of authors such as Virgil and Ovid, but also of painters such as, for example, Michele Ciampanti (1463–1510), Tintoretto (1518–1594), Rubens (1577–1640). In this paper, “dryas’ concept was assimilated as an opportunity to include Nature, environment, art, and persons (…) to think of design as a prudent and responsible action” [13]. Today, reality seems to ask for less interpretation and more experimentation, determining profound changes in the economy and social behavior, but also in habits, rituals and ways of producing and consuming cultures and products. This scenario has forced companies to reinvent their dissemination strategies, using social networks as a means capable of communicating narratives and winning over the public. Like a Dryad that accompanies its tree, from birth to death, maybe design can metaphorize this idea, committing to products and entities during their life cycle. In this case, the creative process between design, music, performing and visual arts and the business world is sustained by the communication of a universal message such as mythology – specifically, the Dryads. This can be an opportunity to create sustainability and innovation, improving both the shows produced and the presentation of new products by companies, adding value and meaning for everyone.

3 Methodology The study is based on a methodology framed in exploratory and generative stages. The exploratory phase consisted of a study about the fundamental concepts, such as the notions of mythology and dryads. Moreover, this stage consists of creating links between design and music, based on musical compositions from the end of the 19th century and the first half of the 20th century. The generate phase produced new information and knowledge. This stage contributes to define a brief with the academia, the business world,

Designing a Creative Process Between Music, Performing and Visual Arts

339

and the entertainment sector to develop processes that can determine the evolution of the theme and create links between all parts. Methodologically, it is expected to apply the concept in an experimentation phase with partner companies, in a trial/error process, of generating ideas and creating project hypotheses (Fig. 1).

Music, performing and visual arts

Academy

Creative Process

Business World

Fig. 1. Designing a creative process between music, performing and visual arts and the business world. Source: the authors.

4 Design as Interpretation In this paper, design process links music with performing and visual arts. In its connection with music, the study focused on musical composers who had interpreted the themes of mythology and Nature. Thus, in cooperation with professional musicians, the design team - made up of four designers - selected four composers that communicate the themes of Dryads. This creative process resulted in the following musical works: 1) The piece ‘Revêrie’ (1890) by the French composer Claude Debussy (1862–1919), performed by a wind quintet; 2) the composition ‘F major, Op. 96, the American Quartest’ (1897) by the Czech author

340

L. Soares et al.

Antonín Dvoˇrák (1842–1904), performed by a string quartet; 3) the A-flat Op. 14 (1903) by the English composer Gustav Holst (1874–1934) played by a wind quintet and; finally, 4) the piece ‘Early Hungarian dances from the 17th century (1959) by the Hungarian composer Ferenc Farkas (1905–2000) performed by a wind quintet. On the one hand, the musical compositions would be performed by a string quartet and a wind quintet in a theater. On the other hand, the musical works would be interpreted by the design team as lighting products at the prototype level and, for this reason, with a strong experimental component (Table 1). Table 1. Music musical compositions that interpret the themes of Mythology and Nature. Piece

Composer

Time

Revêrie

Claude Debussy

1890

F major, Op. 96

Antonín Dvoˇrák

1897

A-flat Op. 14

Gustav Holst

1903

Early Hungarian dances from the 17th century

Ferenc Farkas

1959

4.1 Revêrie (1890) by Claude Debussy Claude Debussy was influenced by different currents and movements such as Impressionism, Pre-Raphaelism, Art Nouveau or Japanese prints [14]. Debussy assumes the impressionism’s female figure who walks among the trees with her parasol, like the new Dryads of the painters of the time. For design, the aim was to achieve geometric simplicity at the visual level and structural simplicity at the abstract level. Thus, as in the piece Reverie, the images are stronger than their referent, so the decision was to draw combinations and use colors in the light instead of the color itself. In creating his music, Claude Debussy takes on the theme of long walks in the Bois de Boulogne Park in Paris, as a reference to his childhood and the daily life of Parisians [15]. This park was a large forest along the Seine River, between Paris and Rouen, the Forest of Rouvray, which among other species had large oaks. This statement reinforces the choice of cork as a raw material in the design of a lighting product that should also highlight the characteristic of walks in the woods. 4.2 F Major, Op. 96 (1897) by Antonín Dvoˇrák The string quartet in F major, Op. 96 by Antonín Dvoˇrák is described by its melody and technical cleanness. The piece’s inspiration resulted from the composer’s personal experience when he spent his summer vacation in Spillville in United States of America [16]. Precisely, in the third stance, there is a reference to a local and colored bird that lived in oaks and that the musician includes in the piece. For design, this reference is interpreted as a vernacular premise connected to cork to create a lighting product that explore shaping action of light and visual elements, such as, color and texture (Fig. 2).

Designing a Creative Process Between Music, Performing and Visual Arts

341

Fig. 2. From left to right: sketches for the design phase of musical compositions “Revêrie” (1890) by Claude Debussy “F major, Op. 96” (1897) by Antonín Dvoˇrák and “A-flat Op. 14” (1903) by Gustav Holst. Source: Ermanno Aparo and Liliana Soares.

4.3 A-flat Op. 14 (1903) by Gustav Holst Gustav Holst view was dominated by astronomy, Nature, and mythology. His work was described as process orient process. Specifically, ‘A flat Op. 14’ is a peace with four movements, and links western and eastern worlds. In addition, the composition also acclaims Nature and its supernatural supremacy [17]. The dualism between the different cultures and the notions of power in Nature were interpreted to design a lighting product. The opposition of the ideas define the identity of a new organism, determining bridges between the harmony and the chaos of today’s reality. 4.4 Early Hungarian Dances from the 17th Century (1959) by Ferenc Farkas Ferenc Farkas conducted research of his own into traditional Hungarian music by collecting folk songs. In Hungarian culture dances are classified in three categories, such Verkbund, a kind of prodigious importance and which become a national symbol [18]. The word means recruitment. Historically, at the end of 17th century, recruits from the Hungarian imperial army danced in the bars and around bonfires, performing circular movements that represents the battles they faced. During this war period Hungarian army took 15 fortresses in the Carpathian Bay, the second longest European mountain full of oaks trees and colored rollers. These deliberations were interpreted with a storytelling process focus on designing light with sustainable meanings and symbolic values. The lighting product would be

342

L. Soares et al.

modular in a clear allusion to the dynamics of Hungarian dancers and using cork and brass. The first material referring to the trees present in the mountains. The brass material symbolizing the bonfires where men danced around (Fig. 3).

Fig. 3. From left to right: sketches for the design phase of musical compositions “A-flat Op. 14” (1903) by Gustav Holst and “Early Hungarian dances from the 17th century” (1959) by Ferenc Farkas. Source: Ermanno Aparo and Liliana Soares.

5 Design as Experimentation In its application, this paper presents the creation of a show in a local theater, bringing together different areas of knowledge and professionals – designers, musicians, entrepreneurs, light, sound, image, and video technicians – knowing that the relationship that a spectator establishes with the culture includes the connection with the scenic space, props, scenography, lighting. Therefore, it was designed a territorial network system with a brass firm and a cork company. Brass refers to the material used by the lighting company, while cork alludes to a raw material company also from the same region. Brass connotes the product with the brightness of the mythology theme. Cork metaphorizes the concept of dryads who live and die with their trees, in an interpretation that allows one to think of design as a prudent and responsible action. Like a nymph, who designs and develops his product, calculating the impact of the product’s life cycle, protecting all the beings involved and the environment. In other words, the concept of Driades allows us to reflect on design with a sense of innovation, contributing to the definition of a more conscious society. The choice of the universal theme of the Dryads also makes it possible to relate the

Designing a Creative Process Between Music, Performing and Visual Arts

343

nymphs to the trees, with wood and cork being interpreted as elements present in the trees and the lighting products being developed with these materials. The second stage of the experimentation phase consists of integrating the lighting objects on the theater stage, together with the musicians. Specifically, designing a pilot installation of a recorded and filmed show that connects professionals from performing and visual arts, business world, design, and music.

6 Conclusions This paper intended to prove that the creative process that combines design, music, performing and visual arts with entrepreneurial initiative contributes to the effective improvement of the produced shows, revealing itself to be an experience that transmits value and meaning. This premise is validated with the creation of a performance in a local theater, precisely, with the design of a pilot project of a recorded and filmed show that brings together professionals form different areas of knowledge. The creative process is also supported by the choice of mythology as a universal theme that becomes comprehensible to the public. For the Academy, the paper reveals that creative process between business world and performing and visual arts can be an opportunity to stimulate modern liquidity [2], connecting academics in research and development projects, with high application slope. The academy benefits from the experiences carried out to be used in future teaching and learning initiatives as a diffused education. Academia can, also, benefit from this methodological action, building teaching models based on collaborative platforms. For Society, the theatral stage becomes an experimental laboratory of current reality, interpreting creativity as a symbiosis between different actors. Moreover, it is an occasion to reflect on existing systems and assuming the impacts of factors external in the design process. In this sense, the media – as a current phenomenon – strengthens the dissemination of knowledge in symbiosis with design methodologies, building moments of discussion and critical analysis. From the stage, creative and sustainable strategies can be designed, in the sense that design interprets and experiments a reality where companies have difficulties to promote their creations. For live presentations, the paper suggests possibilities for establishing new partnerships and new financing and profitability paths. For manufacturers, it shows new ways to inscribe the identity of a new product. By stimulating the links between the local productive culture and the territory, it is possible to restructure not only offices, but also a process that adds social and economic value, capable of modernizing, promoting, and sustaining a place. The study proves that cooperation between different areas of knowledge can contribute to the improvement of the shows produced. That is, the world of entertainment and visual arts can help the business context to present its products in a more intimate way, in smaller spaces such as a theater. For live presentations, it suggests possibilities for establishing new partnerships and new financing and profitability paths. Finally, the paper proves that the cooperation between different areas of knowledge can contribute to the improvement of the shows produced. That is, the world of entertainment and visual arts can help the business context to present its products in a more intimate way, in smaller spaces such as a theater.

344

L. Soares et al.

Acknowledgments. The authors would like to acknowledge the CIAUD, Research Centre for Architecture, Urbanism and Design, Lisbon School of Architecture, Universidade de Lisboa and The Fundação para a Ciência e a Tecnologia (FCT).

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

12. 13. 14. 15. 16.

17. 18.

Gadamer H-G (1976) Philosophical Hermeneutics. University California Press, Berkeley Bauman Z (2005) Modernità liquida. Laterza, Bari Munari B (2009) Design e comunicação visual. Edições 70, Lisboa Eco U (1979) A Theory of Semiotics. Indiana University Press, Boomington Deleuze G, Guattari F (1994) Difference & Repetition. The Athlone Press Limited, London Baudrillard J (1968) Le Système des Objets. Gallimard, Paris Branzi A (1985) Le Design Italien: La casa calda. L’Equerre, Paris Norman D (2005) El Diseño Emocional: Por qué nos gustan (o no) los objetos cotidianos. Ediciones Paidós Ibérica, Barcelona Boas F (1939) The Mind of Primitive Man. Macmillan company, New York Bucci A (2003) L’impresa guidata dalle idee: Leezioni di management creativo dalla moda e dal design. Arcipelago, Milano Latour B (2009) A Cautious Prometheus: A Few Steps Toward a Philosophy of Design (with Special Attention to Peter Sloterdijk)”. In Proceedings of the 2008 Annual International Conference of the Design History Society, pp. 2–10, Universal Publishers, Falmouth Papachristos M (2015) Muses - Nymphs - Other Gods. R.E.I., Le Grau-du-Roi (Blind for review) Isacoff S (2012) A Natural History of the Piano: The Instrument, the Music, the Musicians from Mozart to Jazz and Everything in Between. Souvenir Press, London Hahn H (2009) Scenes of Parisian Modernity: Culture and Consumption in the Nineteenth Century. Palgrave Macmillan, New York Cartes R (2000) Dvoˇrák and America: a new film traces the composer’s life in the United States and his championing of black music. In: Beatty at all (eds) Humanities: a bimonthly review published by the National Endowment for the Humanities (v. 21 Nº 3. May/June. p. 24). University of California, Riverside Holst I (2012) Gustav Holst: a biography. Faber & Faber, London Tari L (2012) The Verbunkos, a music genre and musical of Hunagary. Brasov: Bulletin of the Transilvania University of Brasov. Series VIII: Art. Sport, vol 5 (54). Nº1

The Strange and Not So Strange About Disney’s Strange World: A Visceral and Social Semiotic Analysis Edgar Nunes Quintero1(B)

, Catarina Lelis1

, and Bernardo Meza Guzman2

1 University of Aveiro, Aveiro, Portugal

[email protected] 2 University Antonio Ruiz de Montoya, Lima, Peru

Abstract. Disney is well-known for producing entertainment contents that engage equally children and adults. However, some of the messages conveyed in many of Disney’s productions involve a sense-making that young children cannot attain due to a lack of pre-existing knowledge-based structures. This paper presents a semiotic analysis through a brief and synthetic deconstruction of animated film by Disney, Strange World. Social Semiotics is used to speculate on the possible strategies the creators had when having to communicate through a digital medium grounded on moving images. Using concrete examples, it is possible to observe how the content created follows a series of good practices to better communicate through an engaging and attractive narrative with the target audience, both children and adults. However, there is much information at different layers within the movie that children cannot reach, which leads to a discussion on what may prevail and what may be left out in a message that ought to be passed out to the younger generations. Keywords: Social Semiotics · Multimodality · Emotional Design · Sense-Making · Disney Animation · Audiences

1 Contextualisation One of the most important variables in developing effective digital content relates to how we communicate that content. Hence, it is primordial to know who we are talking to, which may be simple when we aim at one persona, but it becomes a challenge when more than one is part of the strategy and mission of the product or service to be delivered. American mass media multinational The Walt Disney Company is known worldwide as a large business that develops products for families, namely series, films, and many other entertainment solutions. Despite the generalised idea that Disney productions target mainly children, Walt Disney knew well who he was trying to reach with his films and, later, his theme parks. Some of his famous quotes are “Animation offers a medium of storytelling and visual entertainment which can bring pleasure and information to people of all ages everywhere in the world” and “You’re dead if you aim only for kids. Adults are only kids grown up, anyway”. Hence, Disney’s cartoons and movies have historically © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 345–356, 2024. https://doi.org/10.1007/978-3-031-47281-7_28

346

E. N. Quintero et al.

targeted not only children but also adults. According to Statista (2022) the wide range of movies, TV content, merchandise, and experiences under the Disney brand, are clear manifestations that Disney targets all age groups and genders. Moreover, as for Disney+, 45% of its users are aged 18 or under, and a further 49% are aged between 18 and 54 (DemandSage, 2023). Hence, the expansion of Disney by acquiring the rights to products such as Marvel among others, is another expression of Disney’s intentions to include within its target public, besides children, the adults who are already familiar with these brands and for many years have followed and accompanied their evolution. Consequently, Disney inevitably has been having to consider that when creating a product that is appealing, entertaining and promotes the engagement of those who watch and which, it will have to use a code that communicates with both children and adults. In many cases, one or the other is not fully aware of this. This trend of double and often hidden meanings (Booker, 2010) has been widely used for 100 years in developing Disney products. Approaches such as User-Centered Design help to easily enrich possible solutions that address the needs of the target audience (Lowdermilk, 2013) but they do not tell us how the different audiences interpret the same message. Thus, by acknowledging this, it is relevant to research to what extent this practice leads to success in the sense making processes of both children and adults, since interpretation and meanings formulated will be necessarily different.

2 Emotional Design There are many studies and theories on how our brain and mind articulate and process external stimuli. Daniel Kahneman (2011), awarded the 2002 Nobel Memorial Prize in Economic Sciences, states the existence of two types of response to these stimuli: the first (System 1) entails a subconscious response, where our brain functions in a quick, automatic, effortless way, and the other (System 2), where our thinking acts in a slow, thoughtful way, assessing consequences and possible outcomes, referred to as conscious thinking. Regarding the types of processing done by our brain to generate the respective responses, there are three levels to be considered, according to Norman (2004). A Visceral one, linked to our muscular system, where decisions are quickly made about our surroundings, almost automatically, also called the “reptilian” brain, and often referred to as a defence mechanism. The second level, the Behavioural, is where in a way we turn on the auto-pilot, and specific tasks can be performed without us being aware of the details, usually because of experience gained over a long period of time, which trains our brain to provide a specific response to a specific stimulus. These two levels of processing are subconscious responses to external stimuli, hence corresponding to the thinking fast System 1 proposed by Kahneman. The third and final level is the Reflective one, the conscious response to a stimulus which happens slowly and deeply, in some cases a few moments after the stimulus, as a way of evaluating the experience, pondering the effect it had on us and its influence in future decisions (Norman, 2004). Animated movies are particularly effective at spreading meaningful messages to both youngsters and adults. In these visually attractive means of communication, emotion

The Strange and Not So Strange About Disney’s Strange World

347

is influenced by the colour scheme, character design, imagery, narration, music, and environment design. Also, it can be observed that animated characters’ emotionally depicted behaviour resembles that of real-life individuals. According to Norman (2004), the three levels interact with one another, each modulating the others. When activity is initiated from the lowest visceral level, the process is called “bottom-up”. Animated movies resort to this process by stimulating “bottom up” reactions in the viewers, through inputs such as colour and sound, which trigger the rawest of our sensory receptors, mostly effective to engage and entertain the younger audiences. For that reason, in this paper we shall focus on the Visceral level of emotional design, since we are interested in analysing the scope of meaning that this slice of Disney’s audience can potentially reach.

3 Method: Social Semiotics and Multimodalities Whilst, according to Sless and Shrensky (2023) semiotics is the application of stand-for relations created by signs that represent something other than themselves, semiotics is known as “the study of all manifestations of semiosis” (ibidem, p. 59). Hence, semiotics is crucial for analysing that messages are correctly understood by the recipient. Ambiguities can be clarified, and context consistency provided with the use of semiotics. Recently, semiotics has been used as a method to examine and research end-user behaviour and branded communications (Signsalad, 2023). In this paper, semiotic data from the animated movie Strange World is analysed. The semiotic information and its contribution to the storytelling are demonstrated in certain scenes that were carefully chosen. Semiotic resources are supposed to enhance the creative substance of the film and, in these cases, draw the audiences in with relevant ideas. Also, the reason why we resort to a few specific scenes is based on Barthes’ analysis of frames (1978), which contends that the still has a special gift that allows it to resist the imposing rules of narrative time and best relate to cultural practices. The analysis of the animation was done from the perspective of social semiotics (Kress, 1997, 2003, 2010), which views text as the materialisation of semiotic modes and resources. Social Semiotics (SS) is an extensive and dynamic theory, and, therefore, we do not intend to scrutinise it here. Inspired by the studies of Michael Halliday, Gunther Kress and Robert Hodge publish the work Social Semiotics (1988), in which they discuss social issues related to meaning, through texts whose materialisation may include the verbal (speech and writing), but not limited to it. Thus, Social Semiotics is configured as a proposal to understand communication processes, considering the diversity of semiotic modes that may be present in texts, which, in the context of this article, and as proposed by Sless and Shrenski (2023), are all kinds of messages composed by signs (images extracted from a film). The main foundation of the SS theory is the notion of a motivated sign. Refuting the view of traditional semiotics that the sign is arbitrary, SS defends the argument that every sign is the result of the interest of the text producer/author, who makes the choices they believe to be most appropriate for their purpose. Social semiotics is, therefore, focused on how people create content and the meaning attributed to that content (Bezemer & Jewitt, 2009). Hence, to materialise signs, people use semiotic modes, which are material and socially constituted forms to produce meaning.

348

E. N. Quintero et al.

Halliday’s theoretical concept of “metafunction” is cited by Kress and van Leeuwen (2006), who propose three metafunctions in their Visual Grammar: compositional meaning, interactive meaning, and representational meaning. According to Kress and van Leeuwen (2006), the representational metafunction regulates how represented participants relate to one another to depict the world around us when applied to visual systems which leads to analysing how we make an interpretation of the world around us depending on the stimuli we are exposed to. The Representational metafunction includes two dimensions of analysis which are, usually, not coincidental: Narrative involves ongoing activities or occurrences in which individuals are represented by explicit or imaginary vectors; Conceptual images are related to the classification or analysis of participants in terms of their stable and timeless essence (Kress & van Leeuwen, 2006). For the analysis of this movie, we shall resort to Narrative since the nature of the chosen media is that of moving image. The Interpersonal metafunction regulates the relationship between the represented participants and viewers, hence the analysis focuses on the specific relationship created between the content and who is viewing it. Special interest is given to ways of Contact (between the characters and the audience), Distance (visual approaches to proxemics) and Perspective (camera position and angles, indicating structures of power). The Compositional metafunction assembles the parts of the image into a complete work from the relationship between the previous two metafunctions. This means the analysis focuses on the way the interpretations are possibly made from a given stimulus (such as framing, colour, lighting, focus, salience, set design, etc.) and how the context leads to sense making and meaning creation (Burn, 2013). By proposing an analysis based on SS, we intend to reach conclusions that refer to broader issues related to the social, such as interests, power relations, identity, stereotypes, representations, etc., but also the pictorial and narrative choices that are purposeful design creations SS helps analyse the visual elements used in design, such as colours, shapes, typography, and icons. By understanding the meaning attributed to these visual elements within a specific context (such as demographics), animation designers can ensure that their works convey the intended message clearly and effectively. Given that Disney movies target both adults and children, it is appropriate to ask: What aspects are mainly suggested? What semiotic strategies and tools are common and expectedly effective in both audiences? How are these strategies and tools put together, and what possible interpretations are generated? Although the film has been considered in its entirety, the cut made on this article includes scenes in which the characters face moments of crisis and tension, expectedly those when the audiences’ levels of arousal increase, namely where conflicting power dynamics (due to the three featured family generations of insecure men), and fights for species survival (because of a mysterious ecological crisis) are clearly depicted. In that sense, the following moments were chosen: • • • • •

Figure 2 - Presentation of Searcher’s world (0:06:41–0:06:47). Figure 3 - Disagreement between Searcher and his father Jaeger (0:05:10–0:05:53). Figure 4 - Searcher and his father Jaeger reunited (0:29:10–0:29:55). Figure 5 - Searcher and his father Jaeger as equals (0:58:50–1:01:00). Figure 6 - Searcher finds Pando’s energy source (0:03:26–0:03:48).

The Strange and Not So Strange About Disney’s Strange World

349

• Figure 7 - The expedition leader attacking a red flying creature (0:22:50–0:23:01). • Figure 8 - Plot twist, the known world is actually a giant creature (1:28:47–1:29:16).

4 Analysis of Strange World Strange World is the 61st film produced by Walt Disney Animation Studios. Directed by Don Hall in his solo directorial debut, the film had its premiere on November 24th , 2022. It has a run time of 89 min and seven seconds. It is composed of more than 40 sequences, and each of those can be from 11 s up to four minutes long (Walt Disney Animation Studios, 2023).

Fig. 1. From left to right, Jaeger Claude, Searcher Claude e Ethan Claude (screen captured by the authors).

The film’s central idea depicts a story of divergence, separation, and reunion of three generations of explorers, where the main characters are Jaeger Claude (grandfather), Searcher Claude (father) and Ethan Claude (son) (Fig. 1). The father is presented as the central figure. The narrative is mostly focused on the dynamic between the father and grandfather, which is deteriorated by differences between them and leads them to distancing from each other, ending up taking opposite paths. Then the narrative shifts focus to the father and son relationship, where Searcher faces again the bitterness of disagreement, just like with his father, now with his own son, refusing to see Ethan for who he is. As a backdrop to this family affair, the film presents us with an element that threatens the ecosystem that Searcher has helped to build, so he is called upon to find a solution that will allow him to maintain his lifestyle and prevent the end of civilisation as it is known to all. From early in the movie, we are invited to follow the journey of Searcher, as he is expected to be the centre of our attention. By following his gaze (implicit vectors), he offers his point of view of the world, and the several close-ups or shots where he is visible from the waist up make the audience feel close(r) to him (Fig. 2). Worth of note are the shots where Searcher is with his father Jaeger, which communicate the dynamic that exists between the two and how it evolves throughout the story. At

350

E. N. Quintero et al.

Fig. 2. Searcher looks at his farm of green plants (Pando) (screen captured by the authors).

first, they are on opposite sides, and this divergence of ideas eventually separates them. It is also observable how Jaeger’s figure overlaps Searcher’s, in a superior angle from top to bottom, being Searcher dominated with a constrained posture looking upwards. In such moments, close-ups help the viewer focus on the emotions of the characters, clear on their facial expressions. Their gazes towards outside the frame create an intense connection between the two (Fig. 3).

Fig. 3. On the left frame, Jaeger looks down at his son; on the right frame, Searcher watches his father go (screen captured by the authors).

A few years later, when Searcher and Jaeger reunite, the dynamic between them is quickly established as having not changed, with Searcher being portrayed in the shadow of Jaeger (Fig. 4). Towards the end of the story—when the two characters understand their similarities such as the adversities and shared experiences—that make them conclude they are in a similar situation—they are featured on an equivalent level (medium angle), where a dominating relationship between the two is no longer present (Fig. 5). The close-ups in all these emotional moments lead the viewer to feel they are part of the action, allowing a greater involvement, facilitating the engagement with the events and the feelings of the characters. Other pictorial decisions were made to create and to strengthen our connection with the narrative. The use of colour to communicate certain ideas is worthy of analysis regarding both salience and modality. At some point in the movie, Searcher finds Pando, a plant of intense, bright, saturated, and vibrant shades of green, and which is depicted

The Strange and Not So Strange About Disney’s Strange World

351

Fig. 4. Power dynamics between Searcher and Jaeger (screen captured by the authors).

Fig. 5. Two frames where both Jaeger and Searcher’s eye lines are at the same level (screen captured by the authors).

as a solution to the problems of civilisation (Fig. 6). Viewers naturally associate this element to something alive, full of chlorophyll, positive and beneficial to the characters and to the story, since green is associated with luck and growth (Clarke & Costall, 2008).

Fig. 6. Searcher finds Pando, the green plant (screen captured by the authors).

As the story unfolds, we become aware that the green plant, which is a source of energy and believed to sustain the society in which the characters live, is in danger. As we

352

E. N. Quintero et al.

ponder and reflect on the information we have received, it is only natural that we share the concern visible in the characters, feel empathy for their cause and hope that they can solve this problem to maintain the quality of life and status quo of their civilisation. We later understand that the main source of threat to Pando is a species of birds of prey that attack the plant. These birds, depicted in bright and intense red and resembling a flying reptile in all similar to Cretaceous pterosaurs, become quickly associated to the enemy and therefore, it is not strange to the viewer the efforts of the main characters to get these creatures destroyed, which are presented in wide shots and from high angles, positioning the fighting humanised characters in a situation of power (Fig. 7).

Fig. 7. The expedition leader, Calisto Mal, attacking a red flying creature (screen captured by the authors).

In Western societies, we are culturally oriented to associate colour with certain emotions or states of mind, being green usually associated with positive connotations, with something being right, permitted, and moving forward towards the future, and red with danger, threat, incorrection, and fixed states of inertia. The film uses these assumptions in high modality for a plot twist near the end of the narrative: the characters are no more than microscopic beings living inside another organism, a turtle of giant proportions (Fig. 8), whereas the birds, that menacingly attack the plant and fight the story’s heroes, are no more than defence mechanisms of this giant, comparable to white blood cells and their deterrence toward an invader in our circulatory system. It becomes then clear that Pando the green plant is the villain of the story, a parasite that is consuming and destroying the giant being in which the characters’ world exist. Hence, Pando is a source of energy that represents the oil industry in our relatable world.

5 Discussion Since humans use semiotic modes to materialise signs, Strange World, as much as any other Disney production, does that too. However, we argue that these signs (socially constituted forms to produce meaning) convey much more than what is actually being interpreted by, not only children, but also adults. Assuming that, from the perspective

The Strange and Not So Strange About Disney’s Strange World

353

Fig. 8. The salience of the organism in which the movie’s action takes place (screen captured by the authors).

of emotional design, the strategy is a bottom-up one, the visceral level will lead to some possible dead ends or bottlenecks, even for adults, with a lifetime of experience gathering knowledge, in conscious and unconscious ways, developing everlasting memories. In fact, if we focus on the crises around Pando: • Colour associations have been widely researched. According to Whitfield & Wiltshire (1990), the individual perceiving colour automatically evaluates such colour, a process that leads to interpretation and behaviour. To most of us, as mentioned before, green has positive connotations and, when used in opposition, red will assume the negative side of the sign. However, in Chinese culture, vivid red has a positive connotation: it is associated with strength, vigour and confidence. Is it the case that the Chinese audience experience the plot twist less than the Western ones? Or is it possible that far east viewers allocate some characteristics to the birds that the western ones cannot?—such as a strong character, an inclination to progress, and an intention towards establishing a better world? • Still on the birds, their shape invites some reptilian reactions as well—being drawn to resemble pterosaurs, which were carnivores and scavengers, it triggers our innate fear of reptiles, an inherited state of anxiety that most humans experience when facing reptiles. Such response leads to a normal reaction of scare, viewing them as potentially harmful or dangerous. • The main idea of finding out the truth about Pando is a clear message that human beings cannot trust appearances. This indeed means that we cannot trust our senses, and therefore we cannot trust our emotions. And this is a lesson given by someone (in this case, Disney) who is reaching audiences emotionally. Moreover, according to the US Forest Service of the Department of Agriculture (2023), Pando means “I spread out” in Latin and, beyond “Strange World”, it actually exists. It is an aspen clone that is thought to have begun at the end of the last ice age, developing thereafter from a single seed, expanding by putting up new shoots from a massive root system. It can be found in central Utah, weighs nearly 6000 tonnes, and it is the biggest and densest organism ever discovered. It has more than 40,000 distinct trees and covers 106 acres. However, as in the movie, the real Pando is showing signs of

354

E. N. Quintero et al.

decline. Hence, it can be said that the whole situation around Pando in Strange World represents our own world, struggling with survival and sustainability. It is a call to action regarding the fossil fuel industry in which we still very much depend (The Guardian, 2022). The construction of meaning around Pando certainly relies on the visceral and behavioural levels supporting each other, namely during the plot twist; but these kinds of associations are mostly reflective, since they happen at a much deeper level of reasoning, most likely after the contact with the stimulus, at a moment of pondering the effect it had on us and its influence in our future decisions.

6 Final Thoughts One of the situations that was not covered by this analysis is the one around Ethan, Searcher’s son. He refuses to follow his father’s footsteps as a farmer of Pando whilst finding support in a male friend, which leads to their disagreement. The interesting thing about Ethan is that he is one of the first LGBTQIA+ characters in a Disney film, and that is made overtly and positively clear, easy to grasp, even by the very young. In fact, much of the interpretation required in these movies is subject to the audience’s ability to draw inferences, something that adults are better equipped to do compared to children. There certainly is a trend in animated movies (Disney, Pixar and Dreamworks’s) to include hidden messages that most children will not fully grasp: the sexualised and domestic female maturity of Wendy (Peter Pan), the need of becoming something other than what a being (a fish) is naturally meant to be (Finding Nemo), the clichés around

Fig. 9. Official movie posters of both Strange World (left) and Indiana Jones and the Last Crusade (right) (screen captured by the authors).

The Strange and Not So Strange About Disney’s Strange World

355

America as the land of opportunities in sayings such as “Everyone’s special” (The Incredibles), among many others (Booker, 2010). This level of complexity is particularly interesting in Disney’s contents in the sense that children’s cognitive and emotional structures may not be complete, which means they will only understand part of what is being conveyed. Finally, the style and tone presented throughout the animation evoke the culture of the 30s to 50s of last century, where stories of exploration and adventure were very popular, meeting the theme that underpins Strange World. That is another association and level of meaning that will probably be attained by adults but hardly by children. In fact, one could argue that to some extent the movie shares the style and tone of Indiana Jones’s saga, where the main characters are explorers as well, and which graphic title (skewed, written in perspective, resorting to all capitals in a gradient from yellow to orange and a hard drop shadow) was most likely inspirational to Strange World’s designers (Fig. 9). Again, it is not probable that youngsters can establish such a relation, whereas their parents count Indiana Jones as a popular character of their VHS/Beta times. We suggest that System 2 of the younger audience of Disney’s movies is not sufficiently developed so they can fully experience these productions, which are filled with meaning. The feast of wonderful graphics of Strange World, with its beautiful colours, and myriad of delightful textures, will undoubtedly please both audiences, but those who have already begun thinking about the environment, the planet, and its sustainability may be left wishing there was more to help consolidate the frameworks of those still very young. This is where SS and design principles can significantly improve strategies in the field of animated movies targeting heterogeneous audiences, since both provide deeper understanding of how signs and meaning-making processes influence human perception and communication. And since signs are artificial designed creations, social semiotics provides the underpinning material for communication designers and script writers to better understand the cultural context to be represented, helping them recognise the environments in which their designs and stories will be situated. In fact, SS can aid them in crafting powerful narratives through design. By understanding how signs and symbols construct meaning, designers can develop compelling visual stories that engage and captivate the audience, with influence in matters such as those related with inclusivity and diversity. Consequently, SS helps in creating persuasive messages, by assisting these professionals in the creation of contents that influence the audience’s attitudes and behaviours particularly relevant for building a strong brand identity for the movie, ensuring that visual elements consistently convey the intended message and values while increasing the production’s brand recognition.

References Barthes R (1978) Image-music-text. Hill and Wang, New York Bezemer J, Jewitt C (2009) Social semiotics, pp 1255–1268. John Benjamins Publishing Company, Philadelphia. https://doi.org/10.1075/hop.13.soc5 Booker MK (2010) Disney, Pixar, and the hidden messages of children’s films. ABC-CLIO, Santa Barbara

356

E. N. Quintero et al.

Burn A (2013) The Kineikonic mode: towards a multimodal theory of the moving image. MODE working papers. NCRM/MODE/IOE, London Clarke T, Costall A (2008) The emotional connotations of color: a qualitative investigation. Color Res Appl 33(5):406–410. https://doi.org/10.1002/col.20435 DemandSage (2023) Disney Users (2023) – Global Data, Facts & Figures. Retrieved from https:// www.demandsage.com/disney-users/. Accessed 23 Mar 2023 Hodge B, Kress G (1988) Social semiotics. Polity Press in Association with Basil Blackwell, Cambridge, Oxford Kahneman D (2011) Thinking, fast and slow. Farrar, Straus and Giroux, New York Kress G, van Leeuwen T (2006) Reading images: a grammar of visual design. Routledge, London Kress G (2003) Literacy in the new media age. Routledge, London Kress G (2010) Multimodality: a social semiotic approach to contemporary communication. Routledge, London Kress G (2015) Semiotic work. applied linguistics and a social semiotic account of multimodality. Aila Rev 28:49–71 Lowdermilk T (2013) User-centered design. O’Reilly Media, Sebastopol Norman D (2004) Emotional design: why we love (or hate) everyday things. Basic Books, New York Signsalad (2023) Our Thoughts—What is Semiotics? Sign Salad. Retrieved from https://signsa lad.com/our-thoughts/what-is-semiotics/. Accessed 23 Mar 2023 Sless D, Shrensky R (2023) A new semiotics: an introductory guide for students, 1st edn. Routledge, London. https://doi.org/10.4324/9780367809393 Statista (2022) The Walt Disney Company - statistics & facts. Retrieved from https://www.sta tista.com/topics/1824/disney/#topicOverview. Accessed 23 Mar 2023 The Guardian (2022) Strange World review – Disney’s tangled alt-family eco-drama means well. Retrieved from https://www.theguardian.com/film/2022/nov/21/strange-world-reviewdisneys-tangled-alt-family-eco-drama-means-well. Accessed 31 Mar 2023 USDA Forest Service (2023) Pando—(I Spread). Retrieved from https://www.fs.usda.gov/detail/ fishlake/home/?cid=STELPRDB5393641. Accessed 30 Mar 2023 Walt Disney Animation Studios (2023) What we do: filmmaking process. Retrieved from https:// www.disneyanimation.com/process/. Accessed 27 Mar 2023 Whitfield TW, Wiltshire TJ (1990) Color psychology: a critical review. Genet Soc Gen Psychol Monogr 116(4):385–411

A Systematic Literature Review of Inclusion and Representativity of Women in Videogame Live Streaming Between 2017–2023 Maria Júlia Vieira(B)

and Pedro Cardoso

Department of Communication and Art, University of Aveiro/DigiMedia, Aveiro, Portugal {mjlvc26,pedroccardoso}@ua.pt

Abstract. Active participation of women in the world of videogames is still a matter of debate in gender studies. Women have historically been underrepresented and undervalued in the gaming industry, which has led to a lack of diversity and an unwelcoming environment for many women gamers. The practice of live streaming brought this debate to the fore, as gender discrimination and harassment is still prevalent in the gaming community. Women are often subjected to sexist comments, trolling, and other forms of online abuse. This paper presents a systematic literary review using PRISMA methodology, between the years 2017–2023, focusing on the participation and representation of women and in the contemporary phenomenon of live streaming of videogames. This article contributes to the application of the perspectives of methodological approaches with netnography and ethnography through action research Furthermore, this research addresses the current reality of the social inclusion of women in the culture of the videogame community. Keywords: Feminism · Genre equity · Inclusion · Live Streaming · Representation · Videogame

1 Introduction The lack of female representation and inclusion in the gaming community has been resulting in a significant gender gap that affects both gaming culture and the industry. Live streaming platforms like Twitch,1 YouTube Gaming,2 and Facebook Gaming,3 emerged as popular platforms for gamers to showcase their skills and interact with 1 Twitch is a live video streaming service that focuses on live video game streaming, including

broadcasts of e-sports competitions. URL: https://www.twitch.tv. Date of access: 21 March 2023. 2 YouTube Gaming is a gaming portal, developed by Google, featuring information, livestream gameplays, trailers and more. (https://www.youtube.com/gaming) date of access: 20 March 2023. 3 Facebook Gaming is Facebook’s live video streaming platform for e-sports games and championships. (https://www.facebook.com/gaming/video/) access date: 20 March 2023. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 357–370, 2024. https://doi.org/10.1007/978-3-031-47281-7_29

358

M. J. Vieira and P. Cardoso

their audiences in real-time. Nevertheless, the representation of women on these platforms has been historically low, with many facing challenges like gender discrimination and online harassment [1]. To address these issues, these platforms introduced various measures to promote inclusivity and equal gender representation. For example, Twitch created a dedicated channel called Women of Twitch,4 highlighting female streamers and providing them a platform to connect with each other and their viewers. They have also introduced policies to prevent online harassment and abuse, e.g. Twitch’s Hateful Conduct and Harassment Policy5 . Despite such efforts, female gamers still face many challenges regarding representation and inclusivity. In the study conducted by Orme [1], the interviewed women players reported that feeling isolated and marginalized led to lack of motivation to continue streaming or pursuing a career in gaming. This gender imbalance in the gaming industry also affects the type of games that are being developed, with many games still being designed with male audiences in mind. Since their emergence, videogames have maintained a predominance of great interest on by men, which laid foundations based on gender biases and stereotypes, from gameplay to narrative, to advertising and communication, driving the industry towards men [3]. Although streaming platforms are open to any player community, the social scene of gamer culture embraces gender bias and discrimination towards women regarding access, acceptance and representation. Cullen states that the norms of the culture of lives streaming,6 as well as the live transmission platforms themselves, support sexist, misogynistic and anti-feminist beliefs and behaviors inherited from videogame culture, leading to the belief that women’s space in streaming is labelled and bound to sexist stereotypes [2]. This model of under- and misrepresentation of women in the videogame industry stems largely from a gender socialization process that codifies behavior in accordance with the social definition of sports and leisure activities, divided between “female activities” and “male activities”, in which women are directed towards games focused on beauty, aesthetics, decoration, motherhood, etc., while men are directed to specific games, such as football, war games, car games, etc., and everything that escapes this is outside societal norms [4]. According to Nakamura e Wirman [5], when games are created according to gender biases, they strengthen stereotypes. They state that the choice to create ‘games for girls’ is to incorporate features that girls like in games designed primarily for men. They also state that the game may repel its original target audience if it has characters or game modes aimed at both female and male players – it is something that may influence players’ decision to play. Research and debate about inclusion of women in gamer culture is increasingly present in science and in everyday life, and in high demand today. Debates on gender issues and representation have increased significantly in recent years, as gender equality among players is still not satisfactory for the gamer community, demonstrating the importance and relevance of bringing new studies and perspectives [6]. This paper presents a 4 https://www.twitch.tv/directory/all/tags/woman – access date: 20 March 2023. 5 https://safety.twitch.tv/s/article/Community-Guidelines?language=en_US – access date: 20

March 2023. 6 Lives are lingo for online live stream game sessions.

A Systematic Literature Review of Inclusion and Representativity of Women

359

systematic review of recent literature on women representation and participation in live streaming of videogames, providing an overview on the topic and identifying potential avenues for future research, focused on the problematic: How does the discrimination of gender culture present in gamer culture influence the lack of representation of women in the execution, interaction and participation in the new game model characterized by streaming platforms? We resorted to the following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) method to: (i) investigate and understand the research already carried out in this field, especially in live broadcasts on streaming platforms with feminist perspectives; (ii) observe the analysis model made in a qualitative way regarding the participation and representation of women in streaming; (iii) analyze the difficulties and lags of the works in this field of research, thus being able to present new models of future research for this subject. To understand the type of intervention that will be analyzed and whether there will be a comparison between the interventions and the result obtained. Thus, it is essential to contextualize this study [7] through the PICO approach: • (P)opulation – Women in the gamer community and gender discrimination in live streaming • (I)ntervention – To select and to analyze other studies to identify possible resolutions and contextualization of this problem through the PRISMA method. • (C)omparison – Selection of analyzed studies with identification of the methodologies used, thus resulting in the meta-analysis of each selected study. • (O)utcome – Results obtained from the systematic literature review and prospection of relevant future studies. This paper is composed by three main sections, besides this Introduction. Section 2 is where the Systematic literature review process is presented, where the process of preparing a systematic literature review is explained and justified, as well as its eligibility subtopics, until achieving the selection model of studies for the proposed topic. Afterwards, Sect. 3 Meta-analysis, presents the synthesis of each result found in the selected studies, divided into: area of study and contextualization; research question; methodology used in the study and conclusion. In this section, one can also verify the importance of the selected studies, presenting the case studies already carried out in this context and identifying relevant factors for future research. Section 4 presents an individual analysis of each study that was carried out to compose the meta-analysis. Finally, Sect. 5 Conclusion and future works, is where a global overview of each chosen study is presented, pointing out positive points and highlighting observations for possible future work.

2 Systematic Literature Review Process (PRISMA Method) This systematic literature review follows the PRISMA guidelines [8]. The research structure comprises: (i) eligibility criteria for studies relevant to the topic; (ii) information sources and database for research; (iii) search for studies and choice of keywords; (iv) selection of studies: (v) data analysis; (vi) collection result; (vii) final discussion.

360

M. J. Vieira and P. Cardoso

Bibliographic searches were carried out in 2 different databases to try to minimize possible equality of researched works: Scopus (n = 136)7 and SBGames (n = 22)8 . The keywords selected for the search in the databases for this review were chosen based on the problem of the theme presented in the Introduction, namely: (streaming* OR LIVE*) AND (girl* OR WOMAN* OR FEMINISM*) AND (game *). Initially, the search was open to all idioms. The “title, abstract and keywords” filters were used in the Scopus database and in the SBGames annals database it was filtered by the “culture track” topics; “education track” and “industry track”, as well as analysis of the titles of the documents available on the SBGames. The bibliographic search was executed considering works published from the year 2017 onwards, since the practice of live streaming in videogames is recent. The eligibility criteria for selecting the articles for this systematic literature review were based on the relevance of the topic through keyword searches, focusing on the problematic characterized in the Introduction of this paper, with the following main inclusion criteria: 1) Research that relates female participation in live streaming, as well as the active practice of videogames; 2) Works that demonstrate discrimination against women in the gamer culture; 3) Articles and research results on female and feminist insertion in video game streaming lives; 4) Articles published from 2017 onwards in English and Portuguese. The exclusion criteria were: 1) Papers that did not mention streaming and/or female participation; 2) Articles published before 2017; 3) Articles outside the scientific article format and book chapters; 4) Papers that are not in English or Portuguese. As shown in Fig. 1, 158 articles were included in the initial search, and 67 were eliminated after applying the eligibility criteria. Thus, of the 91 remaining papers, 71 papers did not meet the relevant eligibility criteria, and thus 20 were selected by the relevance criterion, among which, after the last selection filter (presented in Fig. 1), only 7 were included in this systematic literature review.

3 Meta Analysis The meta-analysis accomplished based on three aspects: (i) the research question and relevance of the study for this research; (ii) the methodology used for the development of the research to contribute to the development of the theme; (iii) the context in which videogame live streaming and women are inserted (Table 1). We found that there was no information on statistical data and practical case studies in the context of the articles included in this review, making this systematic literature review (SLR) a qualitative 7 The Scopus database was chosen because it covers studies from all scientific areas from all over

the world. It is a database of abstracts and citations of articles for academic journals/journals that covers about 19,500 titles from more than 5,000 international publishers, including coverage of 16,500 peer-reviewed journals in the scientific fields,2 thus generating credibility regarding research and literature reviews. 8 The SBGames database was chosen for the credibility of its scientific articles focused on the study of videogames, as it is the largest academic event in Latin America in Games and Digital Entertainment carried out by the Brazilian Computing Society with the object of investigation and development product.

A Systematic Literature Review of Inclusion and Representativity of Women

361

Fig. 1. Structure of the systematic literature review in the PRISMA model.

systematic review, therefore presenting a discussion on trends and results of empirical data obtained in the studies.

4 Analysis of Individual Studies Of the 7 articles selected for this review, 5 had streaming* or live* as keywords in common, 6 feminism* or *girl or woman* and 7 game*, all in the same context of the problem proposed in the Introduction of this paper. These 7 papers, include important factors about gender bias and the insertion of women on video game platforms made through live streaming. From the meta-analysis, present in table 1 about the bias towards women in online video game live streaming are observed: In paper 1, the authors state that the formation of the gaming industry aimed at the male audience originates in the 1980s, due to the major segmentation of the development of videogames arising from electronic engineering and the market crisis of that time, thus becoming a vicious circle in relation to the production of videogames by men and resulting in a strong advertising campaign that games are made for boys, which continues to this day.

Research question

How the contribution of digital games in the reproduction and trivialization of gender violence reflects in a scenario of increasing participation of women in the games industry

Study area and Context

Paper 1 Almeida et. Al (2019) |Culture track Digital games, gender violence and inequality

Literature review -analyzing the construction of violence against women in games, with a focus on the digital games industry

Methodology

Table 1. Meta analysis of individual results of studies selected for SLR.

(continued)

1. The growth of female gamers relative to men has by no means been reflected within the gaming industry: cisgender, straight, and white men are still the predominant profile among developers 2. The main factors for the contribution of violence in games occurred very early in the industry, in the profile of the first developers, in the context of the cold war, which became an important part of the narrative, aesthetics and mechanics of games and in the segmentation of the public for the male gender, marketing technique for toys 3. Women are still a minority among game developers, resulting in games that are still strongly linked to violence and the male gender, productions that still carry sexist values and stereotypes

Conclusion

362 M. J. Vieira and P. Cardoso

Research question

How and why the presence and legitimacy of feminism in games continues to be contested and demonstrates what is at stake for feminists who desire to participate in building a future for themselves in video game live streaming?

“How Community Guidelines construct and regulate “legitimate” bodies – both the bodies of streamers and the bodies of game characters – especially the sexualized bodies of women?”

Study area and Context

Paper 2 Cullen (2022) | Arts and humanities Feminism and live streaming

Paper 3 Cullen & Ruberg (2019) [12] | Computer Sciences Live streaming, gender, and game culture

Qualitative methodologies of content analysis and critical reading to perform a socially informed critique of community guideline documents for live streaming platforms that have a strong video game presence

Qualitative analysis to several Reddit threads that address the subject of feminism in live streaming

Methodology

Table 1. (continued) Conclusion

(continued)

1. The reviewed platforms often use vague and imprecise language to protect the operations of their websites and allow for broad, subjective, and potentially discriminatory application of their rules 2. Community guidelines for live streaming platforms may present themselves as objective and gender-neutral, but in fact they disproportionately regulate certain bodies, appealing to presumed notions of appropriateness 3. The guidelines further undermine the legitimacy of women in gaming spaces such as livestreams by suggesting that their bodies must be actively regulated to make a “positive” and “safe” streaming platform for a mainstream gaming community

1. Feminism is popular and welcome in gaming and streaming, but some comments analyzed suggest that feminism supports a belief that is incompatible with gaming and streaming culture 2. Feminists are often denied legitimacy as members of the gaming community precisely because of their attempts to make games more inclusive and to confront hegemonic masculinity

A Systematic Literature Review of Inclusion and Representativity of Women 363

Research question

Critical discourse analysis and engagement with games scholarship allow me to ask how cultural narratives about popular egirls form and “how they are reflected in media coverage of streaming platforms”

Critical analysis of the ways Twitch defines sexual content through its Community Guidelines, Terms of Service, and other policy documents

Study area and Context

Paper 4 Tran (2022) [11] |Social Sciences Video games - live streaming and sexuality

Paper 5 Ruberg (2021) | Social Sciences Community guidelines, video game live streaming, and sexual content in a female context

1. It alerts to the need for more studies of live streaming among young people seeking to recover economic and creative agency 2. Hubs for self-titled “egirls” such as Twitch and TikTok (and nascent service like “E-pal”) reflect an microcelebrity ecosystem where the language of realness—or at least believability—is a valued commodity

Conclusion

(continued)

Humanistic method of close reading as an 1. Twitch definition of sexual content is entry point to a critical cultural analysis that indeed vague, subjective and reveals both the patterns of how guidelines contradictory and allows for gender define sexual content and the broader discrimination 2. Although Twitch addresses sexual implications of those definitions content and related issues in six different public-facing policy documents, no clear and concrete definition of such sexual content can be found in this network of texts 3. More concrete and carefully crafted community guidelines are needed for a platform like Twitch to become truly inclusive or “diverse”

Qualitative methodologies this transformation within two primary phenomena: (1) the growth of women game influencers who reclaimed “egirl” slurs in their self-branding and (2) the launch of “Egirl.gg,” a platform for paid gaming companions

Methodology

Table 1. (continued)

364 M. J. Vieira and P. Cardoso

Paper 7 Santos et. Al (2020) |Culture track Sexualization of women, media and games

Quantitative and qualitative research being chain of actions: investigation of the incidence of the term female representation; investigation of environments conducive to female representations; understanding the levels and orders of female representation; culture analysis

The problematic discusses the way in which women are seen in the media and the violence that surrounds such representation relative to symbolic gender violence

Paper 6 Ruberg et. Al (2019) [13] Social Sciences Live streaming of video games and female harassment

Methodology

Research question

This article offers an in-depth critique of the Qualitative analysis of comments on forum harassing discourse directed at the bodies of threads on the Twitch subreddit (r/Twitch) female video game streamers

Study area and Context

Table 1. (continued) Conclusion

1. Games, particularly those in the fighting genre, need to represent women more broadly 2.The impact caused by the media and the games industry can affect women, making them try to achieve something impossible, in addition to submitting to a series of things that were established for the desire of men 3. A complex network of meanings and signifiers inherent to the issue of female representation in games was discovered. Envisioning future developments for relevant investigations, addressing historical dimensions, other genres and diverse cultural contexts

1. Terms like “titty streamers” may seem dismissively juvenile, or simply more evidence of the misogynistic attitudes of the toxic gamer culture 2. The sexist attitudes that pervade gaming culture take many forms that have not yet been sufficiently addressed, such as the discrimination that develops at the level of the literal body of gender 3. Analysis done on the r/Twitch subreddit forum reveals that terms like “titty streamer” and their usage serve as windows into questions of legitimacy, work, merit, attention, and authenticity as it relates to video games and gender bodies

A Systematic Literature Review of Inclusion and Representativity of Women 365

366

M. J. Vieira and P. Cardoso

In addition to this study, the reflection on the female context and the inclusion of women in the streaming culture is analyzed through a case study in papers 3 and 4, where the same concept of gender bias is historically associated and presented in practice. Videogames have been evolving technologically, presenting innovative ways to experience a game, to interact with its contents, narratives and between players [10], and live streaming of gaming sessions is one such recent forms, observed in this study. In paper 2, by means of a case study, the author observes that all norms of live streaming culture, mainly characterized in streaming platforms, in particular their resources, support sexist, misogynistic and anti-feminist beliefs and social postures, a context inherited by the culture of videogames. From this context, his study reveals three disturbing patterns of behavior within live broadcasts, namely: 1) undue influence of the wrong type of feminist; 2) the harm that feminists and feminism cause to games and players; and 3) the consequences experienced by women for being perceived or recognized as feminists at the time of broadcasts [2]. The author concludes that women do not feel represented. In addition to the feminist identity model within platforms, female gamers are judged by male audiences for not taking on the “correct” role of feminism, despite all the harassment still reported on current video game streaming platforms. In this sense, the author concludes that most of the time. However, it was also analyzed that some comments made during video game streaming, while being broadcast by women. The author explains that feminists are often denied legitimacy as members of the gaming community, because they confront the hegemonic masculinity present in most live streaming videogames. In addition to this study, the reflection on the female context and the inclusion of women in the streaming culture is analyzed through a case study in paper 3, where the same concept of gender bias is historically associated and presented in practice. In the analysis, the author reflects on the need for more studies of live broadcasting regarding the legitimacy of women, as well as their inclusion and forms of representation. Furthermore, the study shows “egirls” like Twitch and TikTok (and nascent services like “E-pal”) that reflect an ecosystem of microcelebrities where the language of reality – or at least credibility – is a valuable commodity [11]. The “Obscene, pornographic or objectionable” [6], study analyzed in paper 5, takes a critical analysis regarding how the Twitch platform defines sexual content through its community guidelines, observing the terms of service and other policy documents that are available on the platform. The author concludes that Twitch’s definition of sexual content is really vague, subjective and contradictory, which allows and encourages gender discrimination in the gamer community. The research also reports that although Twitch addresses sexual content and related issues in six different public-facing policy documents, no clear and concrete definition of such sexual content can be found in this network of texts. In this way, the author explains that more concrete and carefully crafted community guidelines are necessary for a platform like Twitch to become truly inclusive or “diverse”. Papers 6 and 7 explore the non-identification of women, gender violence and female stereotypes caused by prejudiced factors inherited not only by gaming culture, but also: (i) the clothing that characterizes the streaming player as she narrates the live transmission;

A Systematic Literature Review of Inclusion and Representativity of Women

367

(ii) the type of game being broadcast; and (iii) the angle at which the camera is positioned on the player at the time of transmission. In paper 6, the authors conduct a qualitative study that shows the “good” versus “bad” representations of women’s bodies in life, characterized as non-sexual content and sexual content, respectively, shaped by social prejudices that reinforce gender inequalities. The authors used the qualitative methodology of content analysis and critical reading within the live streaming communities: Twitch, Mixer and Caffeine9 . They analyzed the cited platforms and realized that these tend to use vague and imprecise language to protect the rules of conduct, thus allowing the broad, subjective, and potentially discriminatory application of their rules. They found that community guidelines for live streaming platforms may present themselves as objective and gender-neutral, supposedly without inequality, but in fact they disproportionately regulate certain bodies, appealing to presumed notions of appropriateness. As such, this study concludes that the guidelines further undermine women’s legitimacy in gaming spaces such as livestreams, suggesting that their bodies should be actively regulated to create a “positive” and “safe” streaming platform for a community. of conventional games, which is completely out of line with equality and combating gender bias within videogame live broadcast platforms. In paper 7, the authors portray the reality of gender stereotyping suffered by women in videogame live streaming, characterized by the term “titty streamers” [13]. They claim this term can seem contemptuously juvenile or simply more evidence of the misogynistic attitudes of toxic gamer culture, which is still very present in broadcasts. These sexist attitudes that permeate gaming culture take many forms that have not been duly addressed, such as the sexualized prejudice and discrimination of the female body, which develops at the level of the literal gendered body. As analyzed in the studies selected for this systematic literature review, the stereotyping of the woman’s body as a sexual figure also resorts to the sexist history of the conception, mediatization and creation of game narratives in a sexist way [14]. Despite the changes seen over time in the representation of women in diverse media, the study by [14] presents the historical chronology of the creation of female characters based on male taste and gaze, thus creating the “stereotypical ideal model” of the perfect woman on the videogame screen. Besides characterizing, through a literature review, that the understanding of the digital games being a gender technology, that also reinforces these female stereotypes acting in the way of acting of the stigmatization and marginalization of certain identities, gender expressions and sexual orientations [14]. Women then become a minority both in terms of being identified as active players in live broadcasts and in the role of spectators of lives because they do not feel represented nor included in this culture [1].

9 Twitch is a live video streaming website that focuses on video game live streaming, including

broadcasts of esports competitions. (www.twitch.com - access date January 07, 2023).Mixer was a Seattle-based video game live streaming platform owned by Microsoft. The service was officially launched on January 5, 2016, as Beam, then renamed Mixer in May 2017, which ended on July 22, 2020.Caffeine is a social broadcasting platform (live streaming) that delivers live, interactive content at the intersection of gaming, sports, and entertainment. (www.caffei ne.tv – access date January 07, 2023).

368

M. J. Vieira and P. Cardoso

5 Conclusion and Future Work Since its inception, electronic games have maintained a predominance of interest largely by a male audience, which is unfortunately reflected in the lack of gender inequality today. The reflection of this preference is visible in its aesthetics – e.g. despite the fact that the protagonist of Tomb Raider (1996) is a woman character, its narrative and design were developed for men [3] –, which reinforced gender discrimination, female stereotypes regarding the sexualization of their body, and thus transforming the relationship of advertising communication aimed at a male audience. Therefore, despite the great technological advances witnessed in the videogame industry, the social mentality regarding gender discrimination that surrounds it is still quite backward, which is why the incessant debate on this topic makes it necessary (particularly within the scientific community) for more studies to be carried out on female inclusion and equality. The trend of live streaming of games has resulted in a large increase in financial investment by game developers, resulting in great engagement in the industry and players communities alike. Women are minority in this industry causing an imbalance in active and interactive participation in the gaming community. The stereotyping of female players and social machismo are present in the gamer community, where women are target of gender violence, prejudice, sexual violence and exclusion. The contextualization for the development of the articles analyzed leads us to believe that the gamer community needs not only to evolve socially, but also to transform the practice of gaming in an egalitarian and unprejudiced way. In this culture, women need to be seen as players as much as men and not pigeonholed by their body or behavior. Our systematic literature review evidences the scarcity of recent studies on gender discrimination on streaming platforms, of quantitative research, and few practical case studies with focus groups. This warns us of the dire need for such studies, aiming at analyzing and demonstrating practical cases of social resolutions, and targeting the conception of guidelines that alleviate prejudice in live broadcasts. It also reveals a certain level of concentration of the same authors when the topic is addressed in scientific research, reinforcing the need for new researchers and more diversified analysis methods, thus contributing to the scientific gamer community. It is also noted that the industry’s quest to reach out for women in the gamer market is based on a preconceived assumption that women prefer novels or activities related to this genre, as was the case of the so-called pink games, in the 1990s, which did not represent the women preferences in leisure activities, especially videogames [4]. Such biases affect women available choice of games to broadcast live on the platforms without being harassed by their audience. It is important to highlight the fact that most case studies found in this systematic literature review are solely focused on Asia (China) and on the USA. Since the use of streaming platforms is accessible worldwide, without any geographic limitation and that gender discrimination and prejudice against women is present all over the world, more comprehensive studies in other countries are required. But it is also worth mentioning that this research resorted to two search databases only, which may limit its final conclusions. It is important to emphasize that further research is needed to understand specific points of the suggested problem, with participative and fieldwork methodologies, as this systematic literature review is only focused on comprehensively explaining the reality

A Systematic Literature Review of Inclusion and Representativity of Women

369

of female players who broadcast their games on live streaming platforms. It is then concluded that it is necessary to generate more scientific research on representation and participation of women in videogames live streaming, exploring the causes and effects of persistent gender discrimination, not only in gamer culture but also in technological platforms that sustain these activities and communities. It is necessary to find ways to encourage women to actively participate in the gaming community and to fight gender discrimination and bias. It is necessary to present integrative solutions that break these barriers and define norms of conduct that do not exclude and empower women in lives. Acknowledgements. This work is financially supported by national funds through FCT Foundation for Science and Technology, I.P., under the project UIDB/05460/2020.

References 1. Orme S (2022) “Just watching”: a qualitative analysis of non-players’ motivations for video game spectatorship. New Media Soc 24(10):2252–2269. https://doi.org/10.1177/146144482 1989350 2. Cullen ALL (2022) Just on the right side of wrong: (de)legitimizing feminism in video game live streaming. Television New Media 23(5):542–552. https://doi.org/10.1177/152747642210 80937 3. Santos Saláfia J, Ferreira NB, Nesteriuk S (2018) Os estereótipos em jogos de luta: da indumentária à hipersexualização de personagens femininas. Anais do XVII SBGAMES, ser. SBGAMES 2014:225–232 4. Seoane A, Sanmartín M (2019) As mulheres que amavam os videojogos, 1 edn. AGAL. Associaçom Galega da Língua. IBSN: 978–84–16545–30–8 5. Nakamura R, Wirman HE (2005) Girlish counter-playing tactics. Game Stud 5(1). http://gam estudies.org/0501/nakamura_wirman/ 6. Ruberg B (2021) “Obscene, pornographic, or otherwise objectionable”: biased definitions of sexual content in video game live streaming. New Media Soc 23(6):1681–1699. https://doi. org/10.1177/1461444820920759 7. Galvão MCB, Ricarte ILM (2020) Revisão Sistemática de Literatura: Conceituação, produção e publicação. LOGEION: Filosofia da Informação, Rio de Janeiro 6(1):57–73. https://doi.org/ 10.21728/logeion.2019v6n1.p57-73 8. Moher D et al (2016) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Revista Espanola de Nutricion Humana y Dietetica 20(2):148–160. https://doi.org/10.1186/2046-4053-4-1 9. Almeida BDO, et al (2019) Violência de Gênero nos Jogos O papel da indústria dos jogos na banalização da violência contra a mulher, pp 826–834 10. de Araújo LB, Martins L (2021) “Eu faço lives”: Intersecções entre lazer e trabalho a partir da plataforma de transmissão ao vivo. In: Twitch.tv. SBC – Proceedings of SBGames 11. Tran CH (2022) “Never Battle Alone”: egirls and the gender (ed) war on video game live streaming as “Real” work. Television New Media 23(5):509–520. https://doi.org/10.1177/ 15274764221080930 12. Cullen ALL, Ruberg B (2019) Necklines and “naughty bits”: constructing and regulating bodies in live streaming community guidelines. In: ACM international conference proceeding series. https://doi.org/10.1145/3337722.3337754

370

M. J. Vieira and P. Cardoso

13. Ruberg B, Cullen ALL, Brewster K (2019) Nothing but a “titty streamer”: legitimacy, labor, and the debate over women’s breasts in video game live streaming. Crit. Stud Media Commun 36(5):466–481. https://doi.org/10.1080/15295036.2019.1658886 14. Santos KP, de Lima ES, Cunha A (2020) A Evolução da Representação da Figura Feminina nos Jogos: Um Estudo de Caso das Séries Bayonetta, Tomb Raider e The Witcher. In: SBC – Proceedings of SBGames, pp 458–466

A Survey of the Digitisation of Collections in Italian Typographic Museums Emanuela F. Bonini Lessing(B) Università Iuav di Venezia, Venice, Italy [email protected]

Abstract. The paper presents the findings of an extended research study into the status of the digitisation of items concerning pre-digital typographic techniques and technologies, to be found in Italian museums. The paper explores: (1) the cultural context and the relevance of the topic with regard to the renewed interest in long-neglected typographic forms and sources; (2) what is meant by ‘typographic items’ and their digitisation (archiving and display); (3) what the corpus of typographic museums we have taken into consideration is, namely the Italian Association of Printing and Paper Museums; (4) the methodology that was implemented to define the type of digitisation of items and the accessibility of the digitised files to museum staff and/or external users; (5) the possibility of expanding the research framework, by including other actors – other museums not belonging to the Association and the many printing workshops scattered across the country, which have saved traditional printing equipment and supplies with the intent in part to preserve their historical value, but primarily for their functions and the possibility of bringing them back into use. (6) Some final remarks, including a comparative study of the Italian case studies and others in Europe. Keywords: Typographic Heritage · Italian Printing Museums · Digitisation of Museum Collections

1 The Cultural Context and the Relevance of the Topic In Italy and abroad, a renewed - albeit niche - interest in forms of typography and publishing linked to the reactivation of pre-digital tools and techniques has been noted for more than a decade [1, 2]. Alongside the ‘mass’ production enabled by digital technologies, manufacturers, designers and users have recently shown a growing interest in bringing back manual making and in the material, aesthetic and sensorial qualities of small-scale or customised artefacts [3]. Designers and printers are interested in collecting and reviving typefaces, presses and tools for typesetting by hand, papers and binding techniques that could be used again today, making them part of contemporary graphic repertoires [4]. At the same time, design historians and theorists are turning their attention to original or long-neglected forms and sources, such as pre-digital techniques, that precede the development of the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 371–380, 2024. https://doi.org/10.1007/978-3-031-47281-7_30

372

E. F. Bonini Lessing

discipline of graphic design. Historians and designers also share similar needs - although the former for cultural purposes and the latter for commercial aims - and they express an articulated demand for access to typographic deposits [5]. A series of research studies1 carried out at the Università Iuav di Venezia sought to ascertain whether museums of typography and printing are equipped to respond to this demand. In particular, one study sought to establish how much of the collections held by museums is available in digital format, both in the museum exhibitions and online.

2 What is Meant by ‘Typographic Items’, Their Digitisation, Archiving and Display Within the research framework, the term ‘typographic items’ embraces a variety of tools and artifacts, ranging from various kinds of printing machines, sets of movable type in different sizes, forms (typefaces), materials, printing papers, inks and binding tools, as well as their production outputs such as posters, books, type-specimens, and documents detailing the profiles of entrepreneurs and/or previous commercial activities [6]. Within the framework of the research, the term digitisation is understood in a very broad sense, verifying the existence of any of the following services [7]: a) acquisition of digital photographs of the various typographic collections, b) photographic scanning or two/three-dimensional digital reproductions of items, c) organisation of the documents in the form of databases, d) possibility of direct access to the database that can be queried by museum staff or by external users (scholars, designers, amateurs).

3 The Typographic Museums in Italy The technical and functional value of printing tools has always led to their gradual replacement with improved printing technologies. In Italy, letterpress technologies were often discarded and abandoned: their value as historical artefacts was only recently recognized, after many of them had already been dismantled, heavily damaged, disassembled or dispersed2 . These are only some of the reasons why there are currently very few museums and archives devoted to pre-digital techniques that are organized and available to the public, often the result of initiatives by enthusiasts. At the same time, the existence of unregistered private collections scattered throughout the country cannot be ruled out. There is therefore no precise reference framework, no real census of museums or collections open to the public, by reason in part of the heterogeneity of 1 We refer to the following studies: the completed POR-FSE 2018-19 research study promoted by

the Regione Veneto, with the Università Ca’ Foscari – coordinator Professor Emanuela Bonini Lessing, Research Fellows Maria D’Unonno, Federico Rita, Nello A. Marotta coordinated by Emanuela Bonini Lessing and Fiorella Bulegato, Visiting Professor Priscila Lena Farias (FAUUSP); the ongoing study – since 2022 - on the digitization of materials in typographic museums in Italy led by Professor Emanuela Bonini Lessing and Research Fellow Nello A. Marotta. 2 An infamous episode is the dispersion of the corporate memory archives of Società Nebiolo — the largest Italian type foundry and printing machinery producers, active from 1878 to 1993.

A Survey of the Digitisation of Collections in Italian Typographic Museums

373

the artefacts they may conserve, which makes it more difficult to establish their specific museographic categorization (parts of them may be included in museums of science and technology, in civic museums, in libraries etc.)3 . For these reasons, the study chose to examine the museums that are members of the Associazione Italiana Musei della Stampa e della Carta – Italian Association of Printing and Paper Museums, the only institution expressly focused on the subject. This organization is made up of seventeen museums and its main objective is ‘to promote and give visibility to museums and collections concerning the historical and artistic and cultural heritage of the graphic arts and paper production’4 (Fig. 1).

Fig. 1. In blue the typographic museums that are members of the national association “Associazione Italiana dei Musei della Carta e della Stampa” (AIMSC). In red the typographic museums that are members of the European Association of Printing Museums (AEPM). (Color figure online)

4 The Research Methodology The Italian Association of Printing and Paper Museums does not provide any documents such as guidelines specifically focused on how to digitize materials, or presentations of best practices derived from their experiences in digitisation. Though every museum, 3 See the museum categories listed by the Italian Ministry of Culture (previous MIBACT) –

https://www.beniculturali.it/luoghi/cerca-luogo?regione=tutte-le-regioni&tipo=museo-gal leria-non-a-scopo-di-lucro-eo-raccolta. 4 With regard to the aims of the association, see: https://aimsc.it/atto-costitutivo/. At the time the questionnaires were administered, the membership included 17 museums: Tipoteca Italiana joined in August 2023. For a list of members see https://aimsc.it/category/musei/.

374

E. F. Bonini Lessing

institution and archive is doing its best to save original items, to preserve them and to pass them on to future generations, by means of digitisation processes among others, there is a substantial lack of shared information on this topic. Before our study, there was no quantitative or qualitative map on the current situation. The decision was made therefore to conduct a questionnaire, in order to establish an initial understanding of the level of digitisation and accessibility of the typographic collections among the associated members. A survey was conceived and conducted by administering questionnaires to museum directors or curators, and verifying the answers through the observation of data available on institutional websites. Four main questions required yes/no answers, providing quantitative data, concerning the following issues: a) b) c) d)

acquisition of digital photographs of various typographical items photographic scanning or two/three-dimensional digital reproductions of the same organisation of the digital collections in the form of databases possibility of direct access to the digitised items by museum staff or by external users (scholars, designers, amateurs).

Fourteen museums provided complete information, two were incomplete, one provided no information/data. The results of the questionnaire show that: a) 85% of the museums claims to have photographic acquisitions of the typographical hardware in their possession b) 60% has started a process of digital reproduction or is about to start it. c) 60% report that they have organized the digital items in the form of a database, of which d) 35% offer direct access to external users (scholars, designers, amateurs), 25% to museum staff only. The questionnaire also aimed to expand on the above-mentioned information by acquiring more details. A second set of questions aimed to establish what the prevailing purpose of digitisation is, how much of it spills over into information available on remotely accessible devices, or within the exhibition spaces. These were mainly qualitative questions, often involving the possibility of multiple answers (Fig. 2). Survey Questions

Answers

Approximately what percentage of the collection is photographed or digitised with respect to the number of items owned by the institution?

The answers vary widely. On the average, digitised items correspond to about 20% of the collection, but one museum claims to have digitised the entire collection, while two museums state that digital collections exceed those that are physically stored

How did you carry out these operations (e.g. by outsourcing to specialised companies or were they done in-house, etc.)?

Predominantly (60%) with external assignments

For what purposes are the photographic images or digitisations utilised (e.g. website, printed catalogues, databases, etc.)?

The digital items are mainly used for communication media (the website, online or exhibition videos, online catalogues) but also to a large extent to print catalogues and brochures (continued)

A Survey of the Digitisation of Collections in Italian Typographic Museums

375

(continued) Survey Questions

Answers

Does the museum have multimedia facilities within the exhibition spaces?

85% answered yes, often in multiple device combinations. Videos are present in 80% of museums, touchscreens 60%, audio guides and tablets 15% each, very few interactive multimedia stations

In what areas of the museum are the multimedia facilities located?

Of the museums that have them, most are distributed along the tour route, while many are only located at the beginning

What is the content of the audio-visual communication?

Mainly illustrating the use of printing presses hence technical - but also historical and biographical

What audiences is the content addressed to?

Adults, schoolchildren, generalist. It is not specified in how many and which languages the audio-visual communication is offered (only one museum specifies that it is only available in Italian)

How often are the multimedia contents updated/implemented/replaced?

About half of the respondents claim that their multimedia devices are newly installed (within two years), so they do not feel the need to update them, while the others claim that they are rarely or never updated or replaced

Approximately how many items are on The percentage varies significantly from display in the museum premises compared to museum to museum, with an average of 55% those held in storage?

5 Expanding the Research Framework The study assessed the presence of at least two important institutions within the national borders, the Museo Bodoniano and the Fondazione Italiana Tipoteca, that at the time of the survey were not members of the Italian Association of Printing and Paper Museums and were therefore not included in the survey. But they are members of the Association of European Printing Museums (AEPM) – which also includes the Italian Association of Printing and Paper Museums. A preliminary search of documents on the digitisation process of typographic collections within the grouping has revealed a lack of shared information. As Dr. Susanne Richter, director of the Museum of the Printing Arts Leipzig/ Museum für Druckkunst, Germany and member of AEPM, made clear: “To use a picture: we are all sitting in the same boat, but throughout Europe we are paddling in different directions”.

376

E. F. Bonini Lessing

Fig.2. Clockwise, from Archivio Tipoteca Italiana: Detail of dies for the casting of an Egyptian typeface by the FTC Fonderia Tipografica Cooperativa, Peschiera Borromeo (Milan), 1940s; Case of copper matrices of the Roman font (Black roman and italics, 20 pt) of the FTC Fonderia Tipografica Cooperativa, Peschiera Borromeo (Milan), early 1900s; Fluidum Nero, from the type specimen “Caratteri Legno Nebiolo”, Società Nebiolo, Torino, 1960 ca.; Serie Etna, 30 cicero, pear wood, Xilografia Meneghello & Belluzzo, Porto Legnago (Verona), 1930s.

The Museo Bodoniano is the oldest printing museum in Italy, opened in 19635 . Following a long process, the museum’s new layout – part of the Palatine Library – has been open since November 2022. It is located inside the Palazzo della Pilotta, which also hosts the Farnese Theatre, the National Gallery of Parma and the National Archaeological Museum. Preliminary research still in progress has shown that the Museo Bodoniano now has more digitised items on display than there are ‘analogue’ items. The relocation of the museum has restricted its space but has provided the impulse for a large-scale digitisation of what it originally held [8]. Tipoteca Italiana is a private, non-profit foundation established in 1995 and promoted by the Antiga brothers, owners of the Grafiche Antiga6 printing and publishing company.The Foundation’s aim is to highlight and enhance the value of the Italian type and printing heritage, and it includes a museum, archive7 , library, print shop and auditorium. 5 Its foundation coincided with the 150th anniversary of the death of Giambattista Bodoni, the

typographer from the Piemonte region who made Parma the world capital of the press starting in the second half of the eighteenth century – https://museobodoniano.com/ 6 http://www.tipoteca.it/en/tipoteca/tif-tipoteca-italiana-fondazione/. 7 The first operation on the Bauhaus archives of the ‘Hidden Treasures’ project led by type designer Erik Spiekerman for the Adobe software house, was followed by an investigation at the Fondazione Italiana Tipoteca with the aim of bringing to light certain typeface families that

A Survey of the Digitisation of Collections in Italian Typographic Museums

377

It is a centre for studies and current typographic production [9]. Initial investigations conducted on the Tipoteca show that digitised items make up a very low percentage of the actual collection. The study of the typographic heritage has also assessed that, in addition to museums, there are at least 29 active printing workshops scattered across the country8 [7, 10]. Their sometimes minimal functions of conservation and exhibition coexist with the prevalent commercial function of contemporary publishing production [11] often sharing the same organisation of physical displays and workspaces. Traditional printing equipment and supplies have been saved with the intent in part to preserve their historical - almost archetypal - value, but primarily for their functions and the possibility of bringing them back into use. The ‘toolbox’ of the workshops thus consists of sets of analogue and digital typefaces, derived more or less faithfully and officially from historical font families [12, 13]. Internet sites declare the existence of collections and samples, some of which are digitised, that can be used for current production, but often offer no better viewing conditions and direct access to them than museums do. Social networks document the varied publishing production more extensively, thus re-actualising the functions of classical typeface specimens: although the original items are hardly visible, the representation of their ‘performing’ qualities is extensive [14]. Around them, there are opportunities for insiders to meet, temporary exhibitions, festivals, workshops, independent bookshops and limited editions9 and sometimes even co-working spaces.

6 Concluding Remarks The paper has presented the findings of an extended study on the status of the digitisation of items concerning pre-digital typographic techniques and technologies, to be found in Italian printing and paper museums. It has described the local cultural context and the relevance of the topic in connection with the renewed general interest in long-neglected could be found only in the Fondazione’s physical archive. Unfortunately, the newly digitised typefaces - Filicudi, Pantelleria, Flegrei - are only available free of charge in the Typekit accessible to Adobe purchasers. Nothing remained in Tipoteca: https://fonts.adobe.com/fonts/ tipoteca-series#fonts-section. 8 Among them, Tallone Editore claims to have the most extensive collection of original typefaces from the 17th century onwards. Alberto Tallone was the pupil of Maurice Darantière – who claimed to have set by hand and letterpress-printed James Joyce’s ‘Ulysses’ for its first edition in 1922. In 1938 he bought the studio from Darantière and since then Tallone Press has been printing literature and poetry masterpieces and a selection of contemporary authors entirely typeset in movable type. The publishing house offers an insight into its archive through a vast on-line gallery, as well as regularly scheduled on-site guided tours of its collections. https:// www.talloneeditore.com/tallone/ – https://www.archiveofstyles.com/. 9 Emblematic was the 2020 operation concerning the re-edition of Alfa-Beta, a text by Aldo Novarese considered a milestone in the history of 20th century Italian design, the original editions of which are almost impossible to find. The activity took place largely thanks to the initiative and technical expertise of the Archivio Tipografico staff in Turin, who ‘gathered’ around various experts, design historians, a publishing house and a social crowdfunding campaign. The Politecnico di Torino, where the original materials essential for the reproductions were found - played a substantially marginal role.

378

E. F. Bonini Lessing

typographic forms and sources. The framework of the research study embraced a variety of artefacts and equipment under the term ‘typographic items’, and the word ‘digitisation’ included the acquisition of digital photographs and/or photographic scanning as well as two/three-dimensional digital reproductions of the typographic items. The methodology implemented to establish the type of digitisation and the accessibility of the collections to museum staff and/or external users was based on a survey administered to the museum directors or curators belonging to the Associazione Italiana Musei della Stampa e della Carta – Italian Association of Printing and Paper Museums. The principal findings reveal that 85% of these museums have photographic acquisitions of the various typographical items in their possession, while 60% of those interviewed have started a process of digital reproduction within the past three years, or are about to start one. The prevailing purpose of photographic acquisitions is to provide the museums with multimedia devices to support the visit - especially videos, which mainly pursue the objective of helping to reconstruct the general narrative, while sometimes providing more specific, technical or monographic in-depth information. Once defined, they are continually re-proposed in a unidirectional manner, from museum to visitor. Of those who have organized their digital information in a database, 35% allow direct access by external users (scholars, designers, amateurs), 25% by museum staff only. Most of the associates who have carried out the digitisation did so through external assignments, which also means the rest of them were able to complete the work relying on the skills of their internal staff. The novelty of the results of the questionnaire consists of a first quantitative and qualitative map on the status of the digitisation of collections, having ascertained a substantial absence of guidelines on how to proceed from the start, as well as a lack of discussion and exchange of views regarding the practices that were conducted. Although the study focused on the Italian Association of Printing and Paper museums, it highlighted the presence of other actors - not included in the list – who played a relevant role in the conservation of analogue typographic items and their digitisation. On the one hand, there are the many printing workshops scattered across the country, which collect these items with the purpose of reintroducing them into everyday practice, both physically and through the digitisation of the materials, and making the files available. On the other hand, there is still one important museum, the Museo Bodoniano, which is not a member of the Italian Association though it belongs to the Association of European Printing Museums. Though the mission of both the Italian and the European groupings is to organize forms of activities to publicise and ensure the promotion of traditional printing trades and their actors, and the continuity of these professions, there is no specific reference to how to order the real or digitised items, nor how to expand the knowledge and how to make it more accessible relying on digital technologies [15, 16]. Further studies should be carried out with the aim of promoting guidelines on how to digitize materials from the start, building on the best practices of the institutions that have successfully done so. Museum Plantin-Moretus in Antwerp is certainly an interesting case study, provided with a very extensive archive of material available online that could serve as a model for digitisation protocols (Fig. 02) [17]. With simple queries, on the website itself, each typographic element present in the museum is shown in a filing that describes its

A Survey of the Digitisation of Collections in Italian Typographic Museums

379

physical constituent elements, its bibliographical references, as well as suggestions for subsequent research outside the institution as well. High-definition images are available and can be downloaded directly from the site and freely used by users10 . The museum also has interesting multimedia equipment to support in-person visits, and is involved in various open-source knowledge sharing initiatives. Another interesting characteristic is that in addition to serving its function of conservation, the museum is also a reference point for contemporary typographic culture. The paper has shown how this culture is nurtured in Italy today by the many printing workshops that promote the reintroduction of typographic techniques into current publishing production, enriching and keeping it up-to-date. A dialogue between these workshops and the other places and institutions that conserve typographic materials (libraries, for example, or science and technology museums) would help create greater appreciation for the existing typographic heritage and make it more accessible, not only from a perspective of conservation but from a design perspective as well.

References 1. Sherman N. Intercut wood typeface project typeface design and printing blocks; Spring 2006– present (work in progress) (formerly known as protoTYPE). https://nicksherman.com/design/ Intercut/. Accessed 24 Apr 2023 2. Farias PL, Hanns DK, Aragão IR, Dixon C (2018) Designing the early history of typography in Brazil, starting from printing in São Paulo. In: Back to the future. The future in the past. Conference proceedings book. Universitat de Barcelona, Barcelona, pp 493–498 3. Bonini Lessing E, Bulegato F, Farias PL (2020) Italian typographic heritage. A contribution to its recognition and interpretation as part of design heritage. In: Lessons to learn? Past design experiences and contemporary design practices. Proceedings of the ICDHS 12th international conference on design history and design studies, vol 1. UPI2M Bookks, Zagreb, pp 293–305 4. Bonini LE, Bulegato F, D’Uonno M, Marotta NA, Rita F (2019) Editoria e innovazione fra analogico e digitale. Università Iuav di Venezia, Venice 5. Huppatz DJ (2015) Globalizing design history and global design history. J Des Hist 28(2):182– 202 6. Cooper A, Gridneff R, Haslam A (2014) Letterpress: looking backward to look forward. Visible Lang 47(3):52–72 7. Bonini Lessing E (2022) Patrimoni tipografici, digitalizzazione, community di utenti. Officina 38:22–27 8. De Pasquale A (2010) I progetti di digitalizzazione del Museo Bodoniano di Parma. Digitalia 2:135–143 9. Clough J et al (2018) Tipoteca. Una storia italiana, Antiga Edizioni, Crocetta del Montello, Italy 10. Scotucci E (2019) Tipofilia. Una strategia per la tutela, la comunicazione e la condivisione della cultura tipografica in Italia. [Master’s thesis, Sapienza Università di Roma], Rome 10 In the U.S.A., the Hamilton Woodtype and Printing Museum – https://woodtype.org/ – presents

an extensive gallery of digitised materials, some free of charge and some for a fee, available via the website. The Type Legacy Project puts its machinery and the skills of museum staff at the disposal of contemporary designers, in order to realize new wood typefaces in homage to the production of the past, also releasing digital versions for a fee, thanks to their collaboration with a specialised digital foundry. On this topic see [18].

380

E. F. Bonini Lessing

11. Miller R. Unique international printing presses. Boxcar Press. https://www.boxcarpress.com/ blog/unique-international-printing-presses/. Accessed 24 Apr 2023 12. Patané M (2015) Digital Nebiolo. Tipotalia 3:88–97 13. https://www.frizzifrizzi.it/2021/01/27/stadio-now-la-fonderia-zetafonts-sta-lavorando-al-rev ival-del-carattere-stadio-disegnato-da-aldo-novarese/. Accessed 24 Apr 2023 14. Griffin D (2022) Type specimen: a visual history of typesetting and printing. Bloomsbury, London 15. Parry R (2010) Museums in a digital age. Routledge, London 16. Smith Bautista S (2014) Museums in the digital age. Changing meaning of place, community, and culture. Altamira Press, Lanham 17. https://museumplantinmoretus.be/en. Accessed 24 Apr 2023 18. Shaw P (2017) Revival type: digital typefaces inspired by the past. Yale University Press, New Haven

Contemporary Tattoo and Social Media: A Special Relationship Bruno Cordeiro(B)

and Suzana Parreira

Faculdade de Belas-Artes, Centro de Investigação e de Estudos em Belas-Artes (CIEBA), Universidade de Lisboa, Largo da Academia Nacional de Belas-Artes, 1249-058 Lisboa, Portugal [email protected]

Abstract. Historically, tattoos have a negative connotation, being mainly associated with underground culture, while contemporary tattoos carry a different status, presenting themselves as culturally more open and professional. By distancing itself from the marginal phenomenon that characterized ancient tattoos and becoming a socially immersive medium, it is necessary to look at tattooing in the present from a different perspective in order to understand how the media are greatly responsible for this practice. Initially the mass media, mostly magazines and television channels such as MTV, and currently social digital platforms, both have enabled and promoted interaction between geographically distant people with common interests. Social media became popular for tattoo artists by bringing them closer to the public, making interaction and holding international conventions and meetings easier. Instagram is a network where it is easy to share the work done with a more diverse audience and has been one of the great drivers of contemporary tattoo. This paper looks for the relationship between tattoo and media and seeks to understand how Instagram has expanded the world of tattooing and allowed tattoo artists to get closer to the public by showing their work and being in contact with the audience. Keywords: Tattoo History · Social Media · Instagram Tattoo Artists · Contemporary Tattoo

1 Contemporary Tattoo, Pop Culture and the Media 1.1 From an Underground Practice to a Mainstream Expression Tattooing is an ancient practice, linked to rituals and cultures that assume different values and meanings throughout history. Currently, it is still a hermetic topic but its social perception has changed in recent decades. Despite its origins and cultural meanings, contemporary tattoo has reached a new dimension, now being more associated with aesthetics, the cult of the body, and the symbolic dimension that embodies identity and personal memories. The act of tattooing consists of inserting ink into the skin, which crosses the first layer (epidermis) and lodges the pigment in the second layer (dermis). The technique’s © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 381–389, 2024. https://doi.org/10.1007/978-3-031-47281-7_31

382

B. Cordeiro and S. Parreira

origin is not precise, as there are traces of its presence in different cultures with no apparent relationship between each other [1]. Many tattoos produced today are influenced by cultural traditions from the indigenous communities of the Pacific Islands, as part of a ritual of transition to adulthood, as a form of joining a group and spiritual power. The first instruments used for tattooing were basic, such as thorns or pieces of bone, while their designs were simple geometric patterns, inspired by elements of nature such as plants, animals, waves, mountains and stars. The oldest records refer to Ancient Egypt, between 4000 and 2000 BC, and to the indigenous tribes of Polynesia, the Philippines, Indonesia and New Zealand [2] (Fig. 1).

Fig. 1. A M¯aori chief with tattoos (moko) seen by James Cook and his crew (drawn by Sydney Parkinson, 1769), engraved for A Journal of a Voyage to the South Seas by Thomas Chambers.

In 1769, the British captain James Cook carried out a maritime expedition in the Pacific islands, in which he recorded an indigenous culture, and brought tattooing to Europe. Due to the influence of different religions and the colonization of the islands, tattoos took on a negative connotation and became mainly associated with criminality, sometimes used to mark prisoners or slaves. In 1827, in Japan, the traditional Japanese

Contemporary Tattoo and Social Media: A Special Relationship

383

tattoo (also known as Irezumi) showed bright colors and large dimensions, focused on a central figure, inspired by myths and monsters, with exaggerated details and surrounded by natural elements, such as clouds and waves. This style is mostly associated with the Yakuza gang, reinforcing the underground nature of the practice as part of a certain subculture [1]. In the United States, the traditional American tattoo was mainly used by sailors. In 1891, the American Samuel O’Reilly created the electric tattoo machine that revolutionized the way of tattooing, making this process much faster to perform [2, 3]. Tattoo artists quickly covered the bodies of sailors, turning them into circus attractions, which, due to the strangeness of their appearance, aroused interest in the public. The style grows to be known as traditional American and consists of icons with bold outlines, bright colors and black shadows. Symbols tattooed by sailors for centuries continue to be inspirational and used today [4]. With the invention of the machine, tattooing became accessible to a wider audience. On one hand, while it became popular, on the other, a wave of exaggerated exoticism was generated, with all kinds of people (men, women, couples, families) completely tattooed, exhibiting their naked bodies in circuses and fairs. The tattoo continues to be a marginal class symbol, with prisoners tattooing themselves with signs that represent the type of crime and the time in prison [4] (Figs. 2 and 3).

Fig. 2. Tattoo Design with a Naval Theme, circa 1900–1945, Metropolitan Museum of Art

Fig. 3. Mrs. M. Stevens Wagner with arms and chest covered in tattoos, 1907. The Plaza Gallery, Los Angeles, California (right).

In the 1950’s and 1960’s, there was a major transformation in the tattoo consumer public, now associated with individuals from counterculture movements: gangs, hippies and punks had a distinctive style (also) determined by their tattoos [5]. In this sense, tattoos reveal a symbolic potential of social revolt, resistance, freedom and individualization; those who have them show a clear intention of defying norms and asserting control of their body in society, namely more vulnerable groups, regarding family, school and

384

B. Cordeiro and S. Parreira

church. These movements’ followers claim a new way of life, in which the body and sexuality must be lived more freely and far from capitalist ideologies [6]. The rise of tattooing in popular culture is now recognized as mainstream, being present in the entertainment and advertising industry. The approval and admiration of tattooed public figures is an incentive for those who idolize them to follow in their footsteps [7]. In the end of the 20th century, tattoos’ presence in the mass media caused changes in social paradigms by transforming it into a consumer product and a daily practice regardless of age, class or ethnicity, distancing it from patterns and associations with specific identity groups. However, patterns related to the subcultures still exist, but the media have become more careful with their communication, thus raising the practice’s status. Furthermore, the irreverence of the tattoo is also used as a communication asset in advertising, consequently making it a mainstream consumer product and something desirable by all social classes. 1.2 The Media’s Role in Contemporary Tattoo Tattooing is one of the oldest artistic expressions in the world, whose presence has been recorded on all continents and has been reinvented countless times throughout history. In the middle of the 20th century, tattoos were a characteristic symbol of individuals in counterculture movements: gangs, hippie and punk movements’ followers distinguished themselves by style and tattoos, affirming political, ethical and aesthetic ideologies, contrary to the social norm at the time [6]. The media created a narrative regarding tattoos that changed the way people accepted the fenomen and connected with this personal expression: “If you turn on your television, open a magazine, or go see a movie, you will likely encounter a tattooed body. Actors, models, musicians, and idolized athletes proudly herald the mainstreaming of a previously marginalized and historically underground practice” [7] (p. 1035). Tattoo’s visibility increased in 1981 with the rise of MTV and the influence of rock music. Unlike mass-produced products, tattoos are individualized and personal, the result of a creative process shared between tattoo artists and tattooed, which is complex due to simultaneous events, both ritualistic, economic, consumerist and individualistic. This process stems from the search for unique, visually and socially attractive creations [7, 8]. Tattoos result from sketches, arrangements and ideas that clients describe and design together with the tattoo artist. These ideas can be pre-conceived or customized images, created from the concept described by the client. Tattooing as an artistic expression has the potential to provoke fascination or revulsion in those who observe it, and this generates different ways of interpreting and communicating [9]. Television shows like Miami Ink and later LA Ink and NY Ink brought people closer to the reality of tattooing. These and other similar programs showed a more intimate and realistic dimension about the tattooing process, the personal meaning that tattoos can convey, its proximity to art and individualism for unique creation, far from pre-established prejudices related to the underground [10, 11]. The media has influenced tattoo’s culture and has been crucial to change perception and social acceptance. Although there is no conscient intention of the media to change this perception, the growing exposure of the practice has been fundamental to normalize and raise awareness. Because society is interested in knowing more about tattoos, by

Contemporary Tattoo and Social Media: A Special Relationship

385

providing this information in a certain way, the media manipulates perception. When a new concept of tattooed people emerges, it does not mean that the associated subcultures cease to exist, it’s just that the middle-class individual simply becomes the focus [12]. Likewise, through the media and with new technologies, tattoo artists reinvent the way they relate to the market, using different means of disseminating their work, from magazines, social networks (for example, Facebook, Instagram), reality shows to national and international conventions [3]. Wymann suggests moving away from questions about what tattooing is, in terms of materials used or who is practicing it, to focus on the way it communicates, referring that tattooing is conceived as a means of communication through which different ways of communicating arise [9]. Currently, millions of people around the world use digital platforms to create and share sophisticated cultural artifacts [17]. In the last decade, social media platforms have proliferated in contemporary society and have changed the way people interact socially at an informal and at a professional level. The rapid growth of social media requires an adaptation to a new reality, where the distribution of information, news and entertainment is no longer exclusive to traditional means of communication (mass media). Social media applications are based on the ideological and technological foundations of Web 2.0 and allow the creation and exchange of content between users [14]. The Internet and social media have alleviated geographical and chronological barriers, enabling the most isolated communities to be connected with the world, allowing them to keep in touch with friends and family or associate with other users [15]. With the presence of tattoo artists on social media (for example, Facebook, Twitter, Tumblr and Instagram), it became easier to show their work and reach a larger audience. The connection between professionals from all over the world makes room for conventions and meetings on a world scale, which results in the internationalization of tattoo artists. Due to easy access to content and a bigger visibility that the digital platforms allows, many tattoo artists limit themselves to copying and reproducing works, leaving new creations aside [16]. Instagram: How Social Media Created a Special Relationship Between the Public and Tattoo Artists The emergence of Instagram boosted the world of tattoos, an example being the United States, where the number of tattooed Americans has almost doubled in recent years. Individuals want to express themselves, they want to make a difference and the tattoo represents this act of individual affirmation [11]. Founded in 2010, Instagram’s main goal is to share images and it became one of the most popular social media applications, with more than 500 million registered users in 2016. The success is explained by how easily it became possible to share photographs, with the use of filters giving users the ability to take pictures and make them visually interesting [13]. Platforms like this are constantly changing; they grow, improve and adapt to users, follow the evolution of cameras and mobile phones, offer new resources and this is reflected in the aesthetics of the images. One restriction is the fact that it is only possible to publish via smartphone, however, it allows instant sharing in other places, such as Flickr, Facebook, Twitter and Foursquare. Captions, image hashtags and identification of other people by username can be used, as the essence of the platform is simple and combines several activities in a single structure. There are different types of users for which care and concern for aesthetics differ. A significant number of people are

386

B. Cordeiro and S. Parreira

concerned with aesthetics and apply specific visual styles, in a sophisticated and subtle way, in order to define their identity and give a professional look, aimed at a specific niche [13, 17, 18]. The evolution of cameras and technology has contributed to a better visual performance on the platform. With the possibility of importing content to mobile phones, many users choose to shoot and film using professional cameras that offer better image quality. The same goes for its treatment, as the use of image editing software allows you to go beyond the filters available on Instagram. Some users have a greater aesthetic concern, which is technically reflected in terms of design and photography. Certain users develop their own style and for that a distinct but consistent appearance is expected. Although the contents are about everyday life, their projection and sharing is not done arbitrarily. Some users opt for a minimalist version, others use local contemporary aesthetics as a reference and others mix different versions in their content. In general, shared artifacts can be more or less realistic, more or less elaborated, depending on the objective. It should be noted that the available tools allow all kinds of results, but the success of a given profile boils down to the consistency with which the contents are shared [13]. Thelander & Cassinger [18] define two types of users: Instagrammers and Professionals. The first type are experienced social media users, usually with more than one account, both for private and professional use. These users favor smartphones to photograph and prioritize the filters available on social media, as their goal is not to capture technically perfect photos but to register the moment and connect with the viewers. The focus of Instagrammers is to gain followers and “likes”, so they follow some strict communication strategies and objectives, such as sharing current, daily and constant content, as well as the use of hashtags. Thus, feedback is seen as important, which leads to frequent accounts checking, in order to understand the success of their images. The “likes” and the number of followers are seen as a reward and as such, even if this type of user does not prioritize editing photographs, the photos and contents have to be thought out in order to captivate and attract more followers. This situation creates a paradox because users want to show naturalness and spontaneity, but at the same time they intend to reach more people, so they have to offer what the public expects to see. Professional users, on the other hand, circumvent the limitations of smartphones and Instagram, using professional cameras and suitable editing programs. All content published is carefully curated, they don’t value the date the photograph is taken, nor the constant need to publish—their content is planned and treated as if it were part of an exhibition. These users create their hashtags, aiming to develop their own concepts, to explore the media and their own possibilities: “likes” are a bonus and not a priority, as these users’ ambition is to explore the platform limits. Due to its rapid growth, Instagram has also become an important commercial medium, where micro and macro companies create profiles to promote their brand, products or services. Social media enabled greater proximity and interaction between people and this forced advertising to adapt. Instagram allowed companies to get closer to users, communicate with them quickly and directly, through storytelling, publications and hashtags. The use of hashtags has become essential, as it allows for content to be filtered and dedicated search to a specific interest [19]. With this proximity, organizations are able to present a more personal side of the brand and convey a more honest image, thus strengthening the bond with the public. However, this proximity requires

Contemporary Tattoo and Social Media: A Special Relationship

387

organizations to be more careful in direct communication, as a negative experience on the part of a customer spreads more easily than in traditional media [15]. In order to understand how Instagram boosted and transformed the conventional way tattoo artists exhibit their work, Force [20] developed an exploratory study that showed Instagram is a visual content platform, and tattoo artists need to pay special attention to how they share images, namely color treatment and visual consistency. Unlike other media, old and new, Instagram has become an easily accessible and free advertising medium for tattoo artists. Nevertheless, the artist’s signature and style impacts the way the public perceives the work, as more authorial tattoos tend to create a dedicated audience and attract followers that identify with the style (Fig. 4).

Fig. 4. Instagram’s tattoo artist Dave Paulo (https://www.instagram.com/davepaulo_tattooartist)

Despite maintaining conventional physical media, tattoo artists are giving more and more importance to their digital portfolio, as it allows them to reach people more quickly and also share flashes (that is, designs available to be tattooed). These designs give tattoo artists the possibility of building an audience around a particular style of their

388

B. Cordeiro and S. Parreira

preference, therefore expressing their individuality as artists. Contrary to old media, Instagram has become an easily accessible and free advertising medium, which allows all types of artists, more or less reputable, to create and promote their brand identity. Many artists and studios have store managers who liaise with the customer, however, instagram allows all artists to create and promote their identity autonomously. Nevertheless, this management requires an extra and more intensive effort on the part of tattoo artists, so that their Instagram account can be seen as a portfolio. Tattoo artists, by being present in the digital world, make their work extend outside the studio, so their success and notoriety are also conditioned by the way their image is worked on social media. The tattoo is no longer just an artifact that the customer acquires, it now becomes a vehicle for interaction and sharing in the digital environment [20]. Conclusions At the end of the 20th century, tattoos became popular with a broader public as the influence of the digital media increased. The growth of tattooing has been continuous until today with more and more people willing to get tattooed, regardless of social status or age, and more artists with diverse styles and influences are emerging, using more technology and better quality materials, with more hygienic standards. Contemporary tattoo results from a shared co-authorship between the client and the tattoo artist, as the client can grant greater or lesser creative freedom, although in most cases the choice of the tattoo artist is already made depending on the style that (s)he produces regularly [21]. Due to the media presence and easy access to information, tattoo artists can be inspired by other tattoos, drawings, illustrations or photographs, combining their technical, creative and authorial capacity. The more authorial and consistent the work of the tattoo artist, the greater their recognition and visibility especially if presented to a broader audience and communicated successfully. Tattoo artists’ digital portfolio is becoming more relevant, easily reaching people willing to be tattooed and those who are changing their minds regarding this practice. Building an audience is part of a signature style, usually followed by a large group of people which marks the artist’s individuality, transforming their work and bringing it to a new level. As a main influencer in tattoo current status, social media appearance promoted significant change in the practice’s social perception and, consequently, in contemporary popular culture at large. Thanks to a strong digital presence, tattoo artists work for a vast audience and contribute to eliminating negative stereotypes regarding tattoos.

References 1. Baptista RLP (2010) A identidade estampada na pele: o quotidiano de um estúdio de tatuagem e body-piercing em Lisboa (Doctoral dissertation, ISCTE-Instituto Universitario de Lisboa (Portugal) 2. Krakow A (1994) The total tattoo book. Time Warner, New York 3. Sad B (2016) A Tatuagem como processo 2002(1):35–40 4. Gilbert S (2000) The tattoo history source book HC. Juno books 5. Rodriguez LDS, Carreteiro TCOC (2014) Olhares sobre o corpo na atualidade: tatuagem, visibilidade e experiência tátil. Psicologia sociedade 26:746–755

Contemporary Tattoo and Social Media: A Special Relationship

389

6. Ferreira V (2008) Marcas que demarcam: tatuagem, body piercing e culturas juvenis. Imprensa de Ciências Sociais 7. Kosut M (2006) An ironic fad: the commodification and consumption of tattoos. J Popular Cult 39(6):1035–1048 8. Machado CA (2018) Cocriação de valor no consumo de tatuagens: um estudo a partir da visão de tatuadores e tatuados (Master’s thesis, Universidade Federal de Pernambuco) 9. Wymann C (2010) Tattoo: a multifaceted medium of communication. MedieKultur J Media Commun Res 26(49):41–54 10. Patterson M (2018) Tattoo: marketplace icon. Consum Mark Cult 21(6):582–589 11. Posner J, Klein E (2018) Explained: Tattoo [Documentary Television Series]. Netflix 12. Porcella A (2009) Tattoos: a marked history 13. Manovich L (2017) Instagram and contemporary image 14. Van Dijck J, Poell T (2013) Understanding Social Media Logic. 1 15. Bergström T, Bäckman L (2013) Marketing and PR in Social Media: how the utilization of Instagram builds and maintains customer relationships 16. Walzer A, Sanjurjo P (2016) Media and contemporary tattoo. Commun Soc 29(1):69–81 17. Hu Y, Manikonda L, Kambhampati S (2014) What we instagram: a first analysis of Instagram photo content and user types. In: Proceedings of the international AAAI conference on web and social media, vol 8, no 1, pp 595–598 18. Thelander Å, Cassinger C (2017) Brand new images? Implications of Instagram photography for place branding. Media Commun 5(4):6–14 19. Demezio C, Silva D, Rodrigues D, Oliveira G, Barbosa K, Melo CO (2016) O Instagram como ferramenta de aproximação entre Marca e Consumidor. In: XVIII Congresso de Ciências da Comunicação na Região Nordeste, pp 1–12 20. Force WR (2022) Tattooing in the age of instagram. Deviant Behav 43(4):415–431 21. Cordeiro B (2023) A autoria na tatuagem contemporânea: uma perspetiva de design. Doctoral thesis, Faculdade de Belas-Artes, Universidade de Lisboa

Design Contributions to Raise Awareness of Dementia and Tackle Stigma Inês Colaço1(B)

, Cláudia Lima2

, Rui Mendonça2

, and Lígia Lopes2

1 Faculty of Fine Arts, University of Porto, Av. de Rodrigues de Freitas 265, 4049–021 Porto,

Portugal [email protected] 2 ID+ – Faculty of Fine Arts, University of Porto, Av. de Rodrigues de Freitas 265, 4049–021 Porto, Portugal

Abstract. Dementia affects around 50 million people worldwide, and that number is expected to rise to 152 million by 2050. Around 40% of dementia cases arise from risk factors that could be avoided. It is crucial to bring prevention factors into public discourse, as well as the importance of timely diagnosis. Awareness-raising campaigns are a means of informing the public so that people can detect symptoms early, as well as tackle stigma and discrimination. This article is a preliminary study aimed at analyzing initiatives in dementia awareness and prevention. For the purpose of the research, documents were analyzed, and an analysis of case studies was made through web platforms and scientific articles. It was found that in Portugal, dementia awareness initiatives are limited to awareness campaigns, with a small number of projects in this area. Design as a means to raise awareness is an asset both in bringing dementia to the public discourse and in destigmatizing it. Hence, greater design investment in the area of dementia is considered essential. Keywords: Awareness campaigns · Dementia · De-stigmatization · Health literacy · Social design

1 Introduction The World Alzheimer’s Report by Alzheimer’s Disease International [1] indicates that dementia affects around 50 million people worldwide, and this number is expected to increase to 152 million by 2050. Dementia affects individuals, their families, and the economy. The prevalence of dementia is about 24.8% in people over 85 years old [2]. Although cognitive decline and/or behavioral changes are often associated with natural aging [3], there is increasing incidence in groups with a younger age range (age below 65 years), calling this factor young or early-onset dementia. Because it is associated with people in older age when symptoms appear in younger people in their 40s and 50s, they are not immediately associated with dementia hence a timely diagnosis is not made. Identifying the symptoms is key to early diagnosis, however one needs to be aware of them. Lack of knowledge and awareness about dementia by the public is a significant barrier to obtaining a diagnosis and accessing care [3]. An early diagnosis allows a person © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 390–401, 2024. https://doi.org/10.1007/978-3-031-47281-7_32

Design Contributions to Raise Awareness

391

to seek support and available pharmacological and non-pharmacological therapies that can improve cognition. It also allows to inform those closest to them about the new condition and collectively find new solutions and adapt in an organized and stable way. Although the symptoms most associated with dementia are memory loss, changes in mood, difficulty communicating and loss of fine motor skills, dementia affects each person uniquely, and general health and social situation can influence how dementia manifests itself [4]. The most common warning signs are usually forgetfulness, loss of time and failure to recognize familiar places. This article consists of a preliminary study within the scope of a research project under the master’s in Industrial and Product Design at the Faculty of Fine Arts, University of Porto, which approaches this issue through the analysis of documentary information as well as initiatives that aim to raise awareness of dementia, symptoms and forms of manifestation among the general public.

2 Problem Definition Prince et al. [5] define dementia as “a clinical syndrome caused by neurodegeneration, (…) characterized by inexorable progressive deterioration of cognitive abilities and independent living”. According to the World Alzheimer Report by Alzheimer’s Disease International [1], dementia affects around 50 million people worldwide, a number of cases that is likely to reach 152 million by 2050. Dementia affects individuals, their families, and the economy. Portugal is the fourth OECD country with the most cases of dementia. In 2018 the incidence rate of dementia was estimated at 19.9 cases per thousand inhabitants. This translates to approximately 182,000 Portuguese people living with dementia [6]. Being dementia a progressive syndrome, the symptoms have a notorious evolution, leading to dependence in daily life at a more advanced stage. Awareness of symptoms is essential, and it is important for the public to be able to recognize them [8]. Cognitive decline and/or behavioral changes are often associated with normal aging, depression or confused with other mental illnesses [3]. As aforementioned, though the symptoms commonly associated with dementia are memory loss, changes in mood, difficulty communicating and loss of fine motor skills, dementia affects each person uniquely: “no two people will have symptoms that develop in exactly the same way” since “an individual’s personality, general health and social situation are all important factors in determining the impact of dementia on them” [4]. Symptoms can be particularized into three phases: early, intermediate, and advanced phase. In the first phase the symptoms are often undervalued and associated with other diseases resulting from aging. The most common symptoms are forgetfulness, disorientation in time and failure to recognize familiar places. In the intermediate phase the symptoms become clearer and more evident, the person with dementia may forget recent events and people’s names and fail to recognize their own home. Communication difficulties become more evident and the need for help with personal care increases. Behavioral changes can also occur. In the advanced phase, the symptoms become more pronounced, leading the person living with dementia to become fully dependent. This phase is characterized by difficulties in distinguishing time and space, identifying family and friends and motor function difficulties [9].

392

I. Colaço et al.

There are several types of dementia, with different symptoms, different causes and affecting the person in distinguished ways. The types of dementia include Vascular Dementia, Parkinson’s Dementia, Lewy Body Dementia, Frontotemporal Dementia (FSD), Huntington’s Disease, Korsakoff Syndrome, Creutzfeldt-Jacob Disease and Alzheimer’s Disease which accounts for 50% to 70% of cases, being the most dementia-related type [2]. As aforementioned, the number of people with dementia is increasing. Predictions about future trends in dementia prevalence vary depending on underlying assumptions and geographic region, but generally suggest substantial increases in overall prevalence related to population aging. As stated by Nichols [11], “the overall age-standardised prevalence of dementia between 1990 and 2016 was relatively stable, but with ageing and larger populations, the number of people with dementia has doubled since 1990”. There are several factors and risks that can lead to dementia, and these can be characterized according to two groups: modifiable factors (which can be prevented or avoided) and non-modifiable factors (which cannot be avoided such as aging or genetics). Potentially modifiable risk factors can prevent or delay up to 40% of cases of dementia. According to the Lancet Commission’s 2020 report [12] Dementia prevention, intervention, and care the nine modifiable factors are: smoking, sedentary lifestyle, obesity, hypertension, diabetes, infrequent social contact, depression (which leads to a variety of possible psychological or physiological mechanisms), education (as the low level of schooling early in life affects cognitive reserve) and hearing impairment. There are also other factors noted: head injuries, air pollution (which accelerates neurodegenerative processes through cerebrovascular and cardiovascular diseases) and excessive alcohol consumption. The non-modifiable factors closely associated with genes and aging are not entirely taxing. If a person inherits a dementia risk gene, it does not mean that he/she will have dementia. The individual risk of developing dementia is a combination of genes, lifestyle, environment, and age. Dementia cases are more prevalent in women than men. Although there is no scientific explanation for this, the Lancet Commission’s 2020 report [12] states that this is “probably partly because on average older women have had less education than older men”. This report also states that “all-cause dementia incidence is lower in people born more recently, probably due to educational, socio-economic, health care, and lifestyle changes”, pointing out, however, that in countries with greater economic power “increasing obesity and diabetes and declining physical activity might reverse this trajectory”. It is also highlighted that “age-specific dementia prevalence in Japan, South Korea, Hong Kong, and Taiwan looks as if it is increasing, as is Alzheimer’s” in low-income and middle-income countries [12]. It is worth highlighting that, according to Alzheimer Portugal [13], dementias are currently the 7th leading cause of death and one of the main causes of disability and dependence worldwide — it is estimated that approximately 200,000 people are living with dementia, a number that Alzheimer Europe estimates may increase in 2050 to 350,000 [14]. Most individuals are not aware that it is possible to reduce the risk of developing dementia and how this can be achieved [15]. Given the fact that around 40% of dementia

Design Contributions to Raise Awareness

393

cases arise from preventable risk factors, it is crucial to bring prevention factors as well as the importance of timely diagnosis into public discourse. Early diagnosis is only possible with prior knowledge of the symptoms of dementia. Therefore, knowing how to identify the symptoms helps to undertake an early diagnosis. Early diagnosis brings several advantages such as seeking support and available pharmacological and non-pharmacological therapies that can improve cognition and quality of life. It also allows people living with dementia to inform their relatives of this new condition and, therefore, to find the best solutions and provide necessary adaptations in daily life for the well-being of the person living with dementia. Hence, it allows to maximize quality of life and plan for the future. Participation in support groups at an early stage and acquaintances with people in a similar situation also helps demystify more negative connotations associated with dementia and thus reduce the associated stigma. The lack of knowledge and awareness about dementia is a significant barrier for people to obtain a timely diagnosis and appropriate care [3]. In this sense, it is fundamental to invest in awareness campaigns to inform the public about the factors and risks that might result in dementia and the symptoms to achieve a timely diagnosis. According to Soong et al. [16], people with dementia and their caregivers do not receive enough information, especially about the management of non-cognitive symptoms and legal and financial issues that arise throughout the progression of the disease. Awareness campaigns are also critical in increasing health literacy among formal and informal caregivers. Lack of understanding causes fear of developing dementia and leads to stigmatization and discrimination. People with dementia are often denied their human rights both in the community and in care homes [19]. The stigma and negative stereotypes associated with dementia prevent many individuals, family members and caregivers from seeking help and the support they need [3]. Raising awareness among the public will contribute to bring dementia into the public discourse and help demystify the situation. Health literacy includes providing knowledge and developing personal skills “conducive to individual and community health” [20]. As advocated by Catarina Alvarez [18] at Alzheimer Portugal Conference, 2022, health literacy entails understanding and acceptance of the situation by both people living with dementia and those closest to them. Several health associations call on governments to run awareness and health literacy campaigns to tackle stigma and make the public more aware. For example, Alzheimer’s Disease International calls on governments to run national awareness campaigns on warning signs and timely diagnosis [3]. And the World Health Organization (WHO) advocates the organization of national and local public health campaigns to increase public awareness, acceptance and understanding about dementia [19]. The main aims of these campaigns are to improve public information about dementia; reduce stigma and dispel myths; promote timely diagnosis; and recognize the human rights and respect for the autonomy of people with dementia. The information to be provided in campaigns and/or other means of public health information and awareness raising is based on 4 themes: general information on dementia; which health services are available and how to access them; care giving to the person living with dementia; information for informal and professional caregivers. The main target audiences for these campaigns will be the community and the public; children and young people (who may have older relatives

394

I. Colaço et al.

experiencing this condition); people with memory complaints; people diagnosed with dementia; family members and caregivers. Regarding public policies for dementia, in Portugal, two dispatches have come out in Diário da República recently: Order No. 5988/2018 and Order No. 12761/2021. Order No. 5988/2018 [21] stipulates to implement measures aimed at developing public health awareness of dementia where it is stated “it is justified that the State defines a strategy that allows the development of public health policies, namely the design of the care pathway for people with dementia”. The Health Strategy in the Area of Dementias was drawn up, which defines the principles that should be followed in the care of people with dementia, the criteria to be used in preventive intervention, the measures to be adopted regarding early detection, the measures for access to medical diagnosis as well as to comprehensive diagnosis. Order no. 12761/2021 [22], on “Awareness and literacy campaigns”, envisages “a National Dementia Health Plan composed of Regional Dementia Health Plans, already drafted and now to be implemented by each of the five Regional Health Administrations, I. P.”. The Regional Plans aim to “integrate and systematize the information (…) and to list a set of recommendations”.

3 Raising Awareness of Dementia 3.1 Awareness Campaigns According to Catarina Alvarez, head of Institutional Relations at Alzheimer Portugal, in Portugal “there is still a lot of ignorance about dementias, namely regarding risk factors and prevention strategies”, making it preponderant to “increase the health literacy of the Portuguese on this subject” [18]. In this sense, awareness campaigns could be a means to fight stigma and inform the public. In 2018, the Alzheimer Portugal association organized the “Memo and Kelembra in Schools” project aiming to increase the level of literacy in dementia among children from the 1st and 2nd cycles of basic education. With a total of 3581 participants aged between 6 and 12, from 11 districts in mainland Portugal, the project was awarded the Maria José Nogueira Pinto Prize in Social Responsibility. The campaign aimed to promote awareness and the strengthening of relations of help and proximity to people living with dementia, fostering autonomy and well-being through intergenerational relationships. According to Alzheimer Portugal [23], dramatization and information sessions were held supported by the bilingual book The Little Elephant Memo, by Alzheimer Portugal. Each session started with “a dramatization moment with the participation of the students and teachers, followed by an information session” where issues related to dementia were addressed, including forms of manifestation and the importance of intergenerational relationships. In September 2021, Alzheimer Portugal with the support of Roche, as part of the World Alzheimer’s Day held the campaign “Sharing experiences for future memory”. Aiming to raise awareness of the risks of dementia, the campaign invited citizens to take part in a series of experiments, mostly online, to promote knowledge of the modifiable risk factors and ways to prevent or reduce them [18]. The campaign consists of seven experiments run by public figures, which include reading sessions, introduction to a new language and the sharing of cooking recipes, with advice for healthy eating. This

Design Contributions to Raise Awareness

395

campaign also aimed to emphasize that dementia is not part of normal aging and can be prevented or postponed by always taking care of the brain and throughout life. In the following year, Alzheimer Portugal launched another campaign aimed at raising awareness that dementia can happen in anyone’s life, silently and from one day to the next. The campaign “The unknown portraits” with the slogan “We never imagine forgetting those who for us are unforgettable” was launched in video format1 . The video shows effects of Alzheimer’s through interventions inside regular citizens’ own homes. An Alzheimer Portugal team enters their homes with the help of a friend or relative and exchanges their photos for images with unknown people. The new photographs partially keep the appearance of the original ones (background and owner of the house in photographs with several people remain). Hidden cameras are installed in the house to capture the reaction of the owners when they see these “new” images. When they return home, they are faced with family photos in which their face is the only one that has not been replaced. Doubt sets in and is accentuated when the person sees that there are other photographs in which he or she is accompanied by more strangers. Using a metaphor, this campaign is a kind of wake-up call, especially for those who have not yet experienced living with or caring for a person with dementia. According to Catarina Alvarez, “with this video and this campaign, we want people to imagine that one day they may forget those who have always been part of their lives”. This campaign allowed, on the one hand, regular citizens to experience symptoms of dementia and, on the other hand, provides viewers with first-hand accounts of these experiences. The “Ativamente - The Brain Club” campaign, organized by the Alzheimer Portugal association, took place in an interactive space at CascaiShopping (shopping center) with the support of Roche, between 16 and 21 September 2022 in order to celebrate World Alzheimer’s Day. The aim was to provide knowledge on how to reduce the risk of developing dementia. It was created “a space divided into five different areas — Cognitively, Physically, Socially, Nutritionally and Musically” corresponding to “strategies that can be implemented to reduce the risk of Dementia”. Hence, visitors were encouraged “to learn about risk factors and what they can do to prevent Dementia, in a playful and interactive way” [29]. The campaign reached about 972,133 people and was widely disseminated in online and offline media. These campaigns had a positive impact on society, raising awareness of dementia issues and mitigating stigma. Nevertheless, its time frame and population reach tend to be limited, which leads Catarina Alvarez to state, as mentioned above, that there is still a lack of knowledge about dementia, risk factors and prevention strategies. Looking across borders, it is worth highlighting a set of examples of campaigns in this field from the last five years. The Dementia Expertise Centre of Flanders launched an awareness campaign entitled “SaniMemorix”, a public health campaign carried out in Flanders, Belgium, for 7 months (September 2018 — March 2019) that includes media and an online individual risk assessment tool [24]. With a representative sample of adults aged 40–75 years, an impact evaluation was conducted with surveys held before and after the campaign. The results were positive and showed the public usefulness of health awareness campaigns. After the campaign, more individuals (10.3%) were aware that it is possible 1 https://youtu.be/hfC--if-VQE?t=86.

396

I. Colaço et al.

to prevent the risk of dementia and more individuals correctly identified 10 out of 12 modifiable protective factors and risks of dementia. Most of the remainder (89%) also considered they did not have the knowledge to make changes to improve brain health. The difference in awareness before and after the campaign was observed in both genders, in all age groups, in individuals living in rural and urban areas and in people with higher and intermediate education. The increase in awareness was not significant in individuals with low education. According to the report Denmark: Country Health Profile [25] dementia is the third most common cause of death in Denmark, which led the Danish Alzheimer’s Association to launch the campaign “Test yourself and keep your brain in trim”, consisting of a brain health test with personal advice and knowledge on what to do to keep your brain healthy and prevent dementia. The Danish Alzheimer’s Association is responsible for the development of the “Brain Health Test” in collaboration with the Danish Dementia Research Centre, the Intersectoral Prevention Laboratory, and a wide range of health experts. The “Brain Health Test”, available free of charge, gives advice on what it takes to keep the brain healthy and reduce the risk of developing dementia. According to Maria Fisker Ringgaard, project manager at the Alzheimer’s Association, “the test poses a series of questions about your information and habits, and based on your answers, it then creates a personal profile that tells you in which areas you are already doing well, and where you can take action to improve your brain’s health” [26]. A survey conducted by MegaFon for the Association showed that almost half of Danes (46%) doubted what preventive measures could be taken to reduce the risk of dementia. On 21 September 2022, Alzheimer Europe launched the initiative “What Makes You, You?”, a collaborative effort between Roche, the European Working Group of People with Dementia (EWGPWD) and Alzheimer Europe to celebrate World Alzheimer’s Day2 [17]. The aim was to combat stigma by emphasizing the importance of looking beyond the diagnosis of a person living with dementia. The initiative features a set of short films with three EWGPWD members living with Alzheimer’s disease along with some of their family members. Each one tells their own story in their own words. The second video features EWGPWD member Helen Rochford-Brennan, who emphasizes how vital it is to keep focusing on the same old things: “Working for the rights of all people is important to me to this day. Equality for all. I am also a political activist and I think I must keep doing that, to keep being the person that I am. All that has changed is that I have a short-term memory problem” [17]. In UK, the Alzheimer’s Society created the Dementia Action Week to encourage people to “take action against dementia” [27]. In 2022, in a bid to combat stigma, they launched the campaign — “It’s not called getting old, it’s called getting ill” — as the motto for Dementia Action Week, encouraging people who are worried that they or someone they know is experiencing symptoms of dementia to seek help. Over the week, there was an average 43% increase in calls to the ‘Dementia Connect’ support line. The campaign was launched through a short film telling a true story of a couple, 66-year-old Bill Wilson and 68-year-old Jo diagnosed with dementia, where initially symptoms with 2 In 2012, Alzheimer Europe set up the ‘People with Dementia Europe Working Group’ solely

made up of people with dementia who work to ensure that Alzheimer Europe’s activities, projects and meetings reflect the priorities and views of people living with dementia.

Design Contributions to Raise Awareness

397

dementia are downplayed and mistaken for ageing. Bill tells how it was about 18 months after the first signs of dementia that Jo turned to a specialist doctor. The Alzheimer’s Society conducted a separate public survey which found that lack of knowledge about dementia symptoms and fear are significant barriers to seeking help, with 19% in the North East admitting they did not recognize dementia symptoms and 21% admitting they had not undertaken diagnosis due to fear. New research has found that over 35% of people with dementia in the North East ignored symptoms for more than two years after they first appeared, and only then sought a diagnosis. The survey, of over 1,000 people with dementia and carers, marked the start of Dementia Action Week and showed that a further 25% of respondents in the area waited between one and two years. One of the main reasons for the delay, according to 41% of respondents in the North East, was that they assumed the symptoms were part of ageing. More than 35% of people with dementia in the North East ignored symptoms for more than two years after they first appeared, and only then sought a diagnosis. The survey, of over 1,000 people with dementia and carers, marked the start of Dementia Action Week and showed that a further 25% of respondents in the area waited between one and two years. One of the main reasons for the delay, according to 41% of respondents in the North East, was that they assumed the symptoms were part of aging. Another initiative launched by the Alzheimer’s Society, in collaboration with the England and Australia women’s soccer teams as part of the Fifa Women’s World Cup 2023, aimed to raise awareness through the shock of what can be some of the symptoms of Alzheimer’s disease—the loss of identity and memory. Some members of the two national teams took to the field wearing jerseys designed especially for the match, with the names of their respective players Tameka Yallop (number 13), Katrina Gorry (19) and Charlotte Grant (22) not appearing on the back of their jerseys. The players whose names were not on their jerseys represent a proportion of one in three. This figure is no coincidence as it refers to the prevalence and impact of Alzheimer’s disease: on average, one in three people will be affected by Alzheimer’s at some point in their lives. The initiative highlights that Alzheimer’s disease affects a significant number of people and dementias are not an isolated issue, but a global and social concern. Since soccer is a mass sport, the Alzheimer’s Society aims to bring dementias into the public discourse by using a platform seen by millions of people such as the Fifa Women’s World Cup for this purpose. 3.2 Contributions of Design to Make Awareness Design contributions in dementia usually focus on products and environments that help people living with dementia to improve their quality of life and well-being. It enables the person to make the most of their abilities and helps boost the development of their interpersonal relationships [28]. However, design can also play a leading role in raising awareness and preventing dementia. Donald Norman [29] suggests that there are various types of reflexive emotional connections with products, which can evoke memories and meaning in the user. This “meaning” associated with objects deals with self-image and how this is reflected around the user. In Design Activism: Beautiful Strangeness for a Sustainable World, Fuad-Luke [30] presents the idea of “symbolic capital” as being a result of the blending of nature’s

398

I. Colaço et al.

capital and humans’ (and financial) capital into ‘man-made’ capital that design has the power to convert, giving it form, incorporating meaning. This “symbolic capital” is defined by social values and projects our perception of reality. Hence, the author states that design is how these capitals are transformed into materialized and symbolic languages. The meaning associated with objects has always been part of their existence. The attribution or emphasis of meanings is often explored using stylistic resources, with the common use of metaphor. Through metaphors unknown problems are presented to public by bringing them into line with known situations, allowing a better understanding from unorthodox perspectives. Gentner et al. [31] refer to three fundamental stages in the use of metaphor in problem solving: the first stage the drawing out of unfamiliar concepts from public domains with the problem in question; the second stage mapping deep relationships between the metaphorical concept and the problem; the last stage applying the structural correspondences associated with the metaphorical source to the problem in question, which in the end usually leads to a new solution. When designing in the context of dementia, the metaphor is often an asset to understanding this syndrome and forms of manifestation. A clear example in this field aimed at raising awareness and tackling stigma is the project promoted by The Enable Foundation that resulted in a series of “Dementia Things”. The Enable Foundation is a non-profit social design collective with expertise in intergenerational co-creation and design for real-world impact research based in Hong Kong. One of the themes addressed by them is dementia and one of their goals is to demystify it. As stated, “from an everyday life perspective, one can discuss and explore the possibility of fostering a cognitively friendly and inclusive community through different objects, tools and designs, as well as questions and imaginations about future life with dementia” [32]. A central concern is how design can contribute to reduce stigmatization of dementia: “can designers advocate innovative cross-generational and cross-disciplinary co-creation models through a cultural perspective?” [32]. From this premise a set of cocreation initiatives were developed, including “Dementia Co-Creation Things” to design “Dementia Things”, a collection of objects that metaphorize situations of dementia. These objects were presented in several international events involving young and old citizens in their co-creation. A central question was raised: “What if everyday objects and even the city became demented?”3 . Through metaphor, objects and environments were designed to provide scenarios in which any citizen can experience situations common in the daily lives of people living with dementia, such as disorientation and states of confusion or difficulty carrying out simple tasks such as switching on the light or putting on a coat. Projects like this clearly demonstrate the potential of design to raise awareness and prevent dementia, as well as to reduce the associated stigma.

4 Final Considerations Currently, the number of people living with dementia are expected to grow considerably. It was found that part of the cases could have been avoided or prevented, as there are risk factors that lead to dementia such as: smoking, sedentary lifestyle, obesity, 3 A presentation of this project is shown at https://www.youtube.com/watch?v=-tzFE3jgrYY.

Design Contributions to Raise Awareness

399

hypertension, diabetes, infrequent social contact, depression, low levels of education and hearing impairment. Potentially modifiable risk factors can prevent or delay up to 40% of dementia cases, yet several studies have shown that a significant proportion of people are unaware of these risks. Hence, it is paramount public awareness of modifiable risks and the knowledge of possible ways to prevent dementia. Another aspect highlighted is the importance of early diagnosis, which requires recognizing the symptoms of dementia. Studies have also revealed a high percentage of people who were only diagnosed 1 to 2 years after the first symptoms, precisely because they did not recognize them as manifestations of dementia, but rather as signs of age. Accordingly, several studies and organizations have highlighted the importance of the recognition of dementia as a public health and social priority and therefore the need to bring this subject to debate in the public sphere. It is fundamental to bring prevention factors into public discourse, as well as the importance of timely diagnosis, which is only possible when there is prior knowledge of the symptoms of dementia. Raising awareness about dementia not only allows the population to be aware of its risks and thus adapt their behavior accordingly, but also tackles the stigma associated to dementia. As shown in this article, awareness can be raised through campaigns and social design projects that promote health literacy. Design is an advantageous tool to structure campaigns to inform the population and to tackle stigma, either through interactive projects or personal confrontation initiatives that show the reality of a person living with dementia. The projects and campaigns analyzed were essentially aimed at raising awareness, improve public information about dementia; reduce stigma and dispel myths; promote timely diagnosis; recognize human rights and respect for the autonomy of people living with dementia. In this preliminary study it was also observed that, in Portugal, there are few initiatives to raise awareness in this area and mostly limited to campaigns, despite the call for urgency by health organizations. It is therefore considered essential, on the one hand, to invest more in public awareness campaigns and initiatives and, on the other, to extend these campaigns in time, since most of them are held around the World Alzheimer’s Day celebrations. Future developments of this research will explore further contributions of design to raise awareness of dementia and tackle stigma.

References 1. Patterson C (2018) World Alzheimer Report 2018. Alzheimer’s Disease International, London 2. Santana I, Farinha F, Freitas S, Rodrigues V, Carvalho Á (2015) Epidemiology of dementia and Alzheimer’s disease in Portugal: estimates of prevalence and the financial burden of medication. Acta Médica Portuguesa 28(2):182–188. https://doi.org/10.20344/amp.6025 3. Alzheimer’s Disease International, University, M.: World Alzheimer Report 2021. Alzheimer’s Disease International, 2–314 (2021). https://www.alzint.org/resource/world-alz heimer-report-2021/ 4. Alzheimer’s Disease International: Symptoms of dementia. (2020). https://www.alzint.org/ about/symptoms-of-dementia/ 5. Prince M, Bryce R, Albanese E, Wimo A, Ribeiro W, Ferri CP (2013) The global prevalence of dementia: a systematic review and metaanalysis. Alzheimer’s Dement. 9:63–75. https:// doi.org/10.1016/j.jalz.2012.11.007 6. Wimo A, Gauthier S, Prince M (2018) Global estimates of informal care. In Alzheimer’s Disease International. https://www.alz.co.uk/adi/pdf/global-estimates-of-informal-care.pdf

400

I. Colaço et al.

7. Organization for Economic Cooperation and Development: Development Cooperation Report 2017: Data for Development, Organisation for Economic Co-Operation and Development (2017). https://doi.org/10.1787/20747721 8. Alzheimer Portugal: Alzheimer Portugal 2022 Conference (2022). https://alzheimerportugal. org/conferencia-alzheimer-portugal-2022/ 9. World Health Organization: Dementia, Key Facts. Demencia (2020). http://www.who.int/ mediacentre/factsheets/fs362/en/%0A 10. Collaborators GBDD (2019) Global, regional and national burden of Alzheimer’s disease and other dementias, 1990–2016: a systematic review for the global burden of disease study 2016. Lancet Neurol 18:88–106 11. Nichols E et al (2019) Global, regional, and national burden of Alzheimer’s disease and other dementias, 1990–2016: a systematic analysis for the global burden of disease study 2016. Lancet Neurol 18(1):88–106. https://doi.org/10.1016/S1474-4422(18)30403-4 12. Livingston G et al (2020) Dementia prevention, intervention, and care: 2020 report of the Lancet Commission. The Lancet 396(10248):413–446. https://doi.org/10.1016/S0140-673 6(20)30367-6 13. Alzheimer Portugal: World day of people with Alzheimer’s disease (2022). https://alzheimer portugal.org/dia-mundial-da-pessoa-com-doenca-de-alzheimer/ 14. OECD: Health at a Glance 2021: OECD Indicators, OECD Publishing, Paris (2021). https:// doi.org/10.1787/ae3016b9-en. Accessed 01 Mar 2022 15. Van Asbroeck S et al (2021) Increasing knowledge on dementia risk reduction in the general population: Results of a public awareness campaign. Prev Med 147:106522. https://doi.org/ 10.1016/j.ypmed.2021.106522 16. Soong A, Au ST, Kyaw BM, Theng YL, Tudor Car L (2020) Information needs and information seeking behaviour of people with dementia and their non-professional caregivers: a scoping review. BMC Geriatr 20(1):1–18. https://doi.org/10.1186/s12877-020-1454-y 17. Alzheimer Europe: “What Makes You, You?” This World Alzheimer’s Day, Alzheimer Europe is proud to collaborate on a new initiative that looks beyond an Alzheimer’s diagnosis and celebrates the person. Alzheimer Europe (2022). https://www.alzheimer-europe.org/news/whatmakes-you-you-world-alzheimers-day-alzheimer-europe-proud-collaborate-new-initiative 18. Alvarez C (2022) Alzheimer Portugal 2022 Conference. Health Literacy in Dementias. https:// alzheimerportugal.org/conferencia-alzheimer-portugal-2022/ 19. World Health Organization: Global action plan on the public health response to dementia 2017 - 2025. Geneva: World Health Organization, 27 (2017). https://apps.who.int/iris/bit stream/handle/10665/259615/?sequence=1%0Ahttp://apps.who.int/bookorders.%0Ahttp:// www.who.int/mental_health/neurology/dementia/action_plan_2017_2025/en/ 20. WHO: Health Promotion Glossary. Geneva (1998) 21. Official Journal of the Republic n.o 116/2018. (2018). Despacho n.o 5988/2018, de 19 de junho (p. páginas 17094–17101). https://dre.pt/dre/detalhe/despacho/5988-2018-115533450 22. Official Journal of the Republic n.o 251/2021: Despacho n.o 12761/2021, de 29 de dezembro (pp. 85–87) (2021). https://dre.pt/dre/detalhe/doc/12761-2021-176724079 23. Alzheimer Portugal: The Little Elephant Memo. Alzheimer Portugal (2018). https://alzhei merportugal.org/o-pequeno-elefante-memo/ 24. Expertisecentrum Dementie Vlaanderen.: The Flanders Centre of Expertise on Dementia – our mission and main projects (2022). https://www.dementie.be/the-flanders-centre-of-exp ertise-on-dementia-for-the-international-audience/ 25. Busse R, Blümel M, Scheller-Kreinsen D, Zentner A (2017) Denmark: Country Health Profile 2017. OECD and World Health Organization 22(2):1–20. https://doi.org/10.1787/888933 593475. http://www.euro.who.int/en/about-us/partners/observatory/publications/studies/tac kling-chronic-disease-in-europe-strategies,-interventions-and-challenges. https://www.oecdilibrary.org/social-issues-migration-health

Design Contributions to Raise Awareness

401

26. The Lundbeck Foundation: Test yourself and keep your brain in trim (2021). https://lundbe ckfonden.com/nyheder/test-yourself-and-keep-your-brain-in-trim 27. Alzheimer’s Society: Dementia Action Week. Alzheimer’s Society (2022). https://www.alz heimers.org.uk/get-involved/dementia-action-week 28. Zeisel, J.: Improving Person-Centered Care Through Effective Design (Issue August) (2013). https://www.researchgate.net/publication/289840853_Improving_person-centered_ care_through_effective_design 29. Norman, D.: Emotional Design: Why We Love (or Hate) Everyday Things (2004) 30. Fuad-Luke, A.: Design Activism: Beautiful Strangeness for a Sustainable World (2009) 31. Gentner, D., Bowdle, B., Wolff, P., Boronat, C.: Metaphor Is Like Analogy. Metaphor Is Like Analogy (2001).https://doi.org/10.7551/mitpress/1251.003.0010 32. Foundation, E.: Co-creating our future Dementia Hong Kong. Consultado em dezembro 5, 2022 (2020). https://www.enable.org.hk/en/enablers/1-5-more-than-500-viewers-lets-talkabout-dementialand-enable-talk-4-5-on-youtube-live

Creation of Podcasts to Promote Students’ Engagement in Technical and Vocational Schools Elton Rubens Vieira da Silva(B) and Pedro Beça University of Aveiro, 3810-193 Aveiro, Portugal {elton.vieira,pedrobeca}@ua.pt

Abstract. Engaged students have more chances to achieve personal e academic success. Schools have the challenge of promoting student engagement, which uses several pedagogical tools. A podcast is a potential tool to promote academic engagement. Listening to podcasts is an innovative way to consume didactic content, but developing podcasts transforms students into the constructor of their knowledge. This paper presents a methodological proposal for the creation of a podcast development framework, focused on pedagogical use as a classroom didact activity in technical and vocational schools, using the theories of student engagement as pedagogical structures. Design-Based Research (DBR) is the methodological approach chosen for this investigation. DBR structure shows cyclical e iterative steps to develop a solution for a pedagogical problem in the following structure: (a) Literature analysis to understand the problem context; (b) Professors and students of the technical and vocational schools were indicated as stakeholders of this investigation; (c) Initial study applied to understand how academic engagement occur in technical and vocational schools; (d) Prototypes built, tested and refined; (e) A full version of the framework was analyzed; (f) Surveys, interviews, documental research, and logbooks used as collect data tools, and; (f) the data analyzed and used for a refining step of the podcast development framework. This study offers relevant discussions on the process of developing a pedagogical intervention and, in addition, designs an investigation to understand students’ engagement in technical and vocational schools. Keywords: Podcast in education · Student engagement · design-based research

1 Introduction Digital games, virtual learning environments, online communities, and, recently, IA software are samples of digital technologies that are changing the way to think and do education. The challenge is to create a relevant pedagogical teaching and learning environment that uses innovative technologies. The podcast is one of these new digital tools that gain educational professionals’ attention around the world [1, 2]. Podcast is media of connection, that use strategies like attractive narratives and storytelling to deliver information to listeners in an accessible and convenient way [3]. That media, mostly consumed in audio format, allows the listeners to pause, replay, and © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 402–412, 2024. https://doi.org/10.1007/978-3-031-47281-7_33

Creation of Podcasts to Promote Students’ Engagement

403

carefully attend to content, thereby enabling more information comprehension [4]. As democratic media allows creators without complex technical, legal, or financial resources to create and distribute content. These characteristics have enabled podcasts to become a popular teaching practice in higher education as early as the 2000s as a means of engaging students in their didactic activities [5, 6]. Educational podcasts are excellent primary or supplementary instructional materials that can be utilized by teachers and consumed by students both inside and outside the classroom [7]. However, we aim to go beyond the perspective of using podcasts as instructional materials. Rather, the focus of this research is how to apply podcast production as a pedagogical activity. The production of podcasts can be a powerful tool to encourage student engagement with instructional content and schools [8]. The main contribution of this work is to propose methodological steps, inspired by Design-Based Research theory, this research seeks to investigate if and how podcasts can be used as didactic activities to enhance student engagement levels in professional education institutions. This article is structured in the following sections: Sect. 2 introduces the literature review in areas such as educational podcasts and student engagement. Section 3 summarizes the method we propose to use in our research. Section 4 concludes this work and gives an outlook into the future research direction.

2 Literature Review Berry [9] asserts that a podcast is an audio content created in the.mp3 format, which can be subscribed to, received, downloaded, and listened to on various devices. However, attempting to define podcast anchored solely in its technical characteristics is doomed to render such an obsolete description. Bontempo [10] defines a podcast as an audible experience that can be accessed wherever, however, and whenever the listener desires. Rime et al. [11] pointed out different reasons people listen to podcasts: entertainment, social belonging, companionship, and educational purpose. Educational podcasts can be a method for helping to build a strong relationship between theory, research, and practice in a space between formal learning and entertainment [12]. Listen is only one way to use podcasts in education. Teachers are already using the creation of podcasts as a didactic activity, capable of enhancing the engagement to certain themes [13]. Students who create podcasts undergo a process of assimilation and reflection on the topics discussed in the classroom, transitioning from followers of instructions to producers of knowledge [14]. And this active role in the learning process directly influences student engagement. Hu and Kuh [15] define engagement in the educational context as the quality of effort that students dedicate to educational activities that directly contribute to desired outcomes. It can still be understood as an amalgamation of active and collaborative learning, participation in academic challenges, interpersonal communication, engagement in academic enrichment activities, and a sense of legitimacy and support from the academic learning community [16]. Student engagement increases the chances of acquiring competencies such as general skills and critical thinking, cognitive development, self-esteem, psychosocial development, productive formation about genre and race issues, ethical and moral development, better grades, the gain of social capital [17,

404

E. R. V. da Silva and P. Beça

18]. Where students are engaged, co-participation and collaboration in the construction of the educational environment emerge [19]. Audio has shown the ability to facilitate authentic engagement with pedagogical content, allowing students to connect with the world in several ways, both as listeners and producers [20].

3 Method Our research approach is guided by the concepts of Design-Based Research (DBR). The method literature describes DBR as a connection between educational research and real-world problems [21], focused on developing a solution for a practical problem in the education field, seeking to understand how interventions can positively influence learning [22]. This paper presents the steps proposed to investigate the podcast as a didactic tool for enhancing levels of engagement. The model of DBR presented by McKenney and Reeves [23] was chosen for this investigation. The authors built your model by analyzing other models e frameworks (Fig. 1) .

Fig. 1. Design-Based Research structure by McKenney and Reeves [23].

This DBR intervention adopts a qualitative approach, yet examining quantifiable data. Interviews, focus groups, document analysis, logbooks, and surveys are the investigation tools planned to be used in this interventionist research. The extensive range of data collection tools in this investigation is justified by the acknowledged challenge, recognized in the literature, of understanding student engagement levels. Several validated tools for measuring engagement exist [24], but most of them are self-reporting in nature. Thus, employing other tools for data triangulation can provide a more comprehensive perspective on how engagement is perceived in the classroom by students, teachers, and educational institutions. The research target audience is composed of teachers and students of technical and vocational schools in Brazil and Portugal. The experience of these actors is essential to understand the characteristics of this type of professional formation.

Creation of Podcasts to Promote Students’ Engagement

405

4 Methodological Proceedings This section has presented the proposal of the methodological approach used to investigate the podcast as a didactic tool for enhancing levels of engagement. 4.1 1st Phase: Analysis/Exploration Analysis – Initial Orientation. The initial phase of the investigation is where the problem, context, and stakeholders of the research are defined (Table 1). Table 1. Initial orientation. Problem

To address the lack of student engagement, a possible solution is to use the process of creating a podcast to stimulate students to engage with academic content. However, creating a podcast presents technical, organizational, and operational challenges

Context

Technical and Vocational Schools

Stakeholders

The DBR has the collaborative participation of all those involved in the construction of a pedagogical intervention. Therefore, students, teachers, education professionals, and the research team itself will be impacted by the project

Analysis – Literature Review. The analysis process is carried out in a structured manner, to have a concrete view of the research topic. The topics selected for the literature analysis process were (Table 2): Table 2. Literature Review. Podcast

Podcast in Education

Student engagement

Constructivism Learning

Design-Based Research

Technical and vocational school

The use of podcasts in education was identified as the focus of the investigation. A Systematic Literature Review was then conducted to understand how podcasts have been used in didactic activities within classrooms. Analysis – Field-Based Investigation. As an important part of this investigation, we need to understand how the field behaves even before any intervention to promote student engagement. To understand the field of this investigation, the methodological steps defined were (Table 3): Initial Study The pilot study is a small-scale preliminary study conducted to assess feasibility, time,

406

E. R. V. da Silva and P. Beça Table 3. Field-based investigation.

Document analysis

Documents that record the level of student engagement will be analyzed, such as attendance records and grades

Interviews/surveys

Interviews and surveys will be conducted with teachers and students to understand their perspectives on student engagement

Initial Study

A pilot study will be conducted to comprehend which actions promote engagement and whether these actions are effective

and cost, and analyze any potential adverse effects [25]. The pilot study is conducted with a sample of the population, but not those who will participate in the final sample. The design of the initial study in this investigation was developed so that information on student engagement could be analyzed before the development of any intervention (Fig. 2).

Fig. 2. Initial Study design

Phase 1 of the pilot study will cover field recognition activities. To do so, theoretical studies will be carried out on student engagement and how to identify levels of engagement in the classroom. Interviews and questionnaires will be applied to students and teachers to understand how school dynamics related to engagement activities occur. Phase 2 of the pilot study proposes the development of a podcast activity by the students. Research tools will be used to understand how this activity impacted the stimulation of student engagement. This phase is also important to investigate the relationship between

Creation of Podcasts to Promote Students’ Engagement

407

students and teachers with podcast creation technologies, and how the technical dimensions of development can affect student engagement with the activity. The task proposed to be performed by students has some suggestions for execution, however, the classroom context will be considered. Phase 3 is the analysis of results and the understanding of the two main dimensions of the initial study: the perception of podcast creation and the evaluation of student engagement after the task application. Finally, in Phase 4, data collected will be interpreted to build important information about the study itself, as well as to use this information as input for the next phases of the Design-Based Research methodology. Exploration – Site Visits. The site visit aims to sensitize the actors involved in the research about the study. As defined, field visits will be carried out in technical-vocational education institutions in Brazil and Portugal. Exploration – Professional Meeting. Exchanging experiences with researchers in the field is important to understand what is being done and to learn from ongoing research. To this end, we will write articles (short and full-length) to participate in events with experts and to put its methodological procedures up for peer review. Exploration – Networking. The literature review and meeting with experts are important actions for the creation of a network. 4.2 2nd Phase: Design/Construction Design – Exploring Solutions. The information gathered in Phase 1 serves as input for the intervention development process. The beginning of Phase 2 is characterized by the generation of possible alternatives to resolve the identified problem. DBR values collaborative construction of educational intervention, meaning that all actors involved in this investigation should be part of the entire development process. Teachers and students will also be invited to generate ideas and analyze solutions. Examples of techniques for generating and analyzing ideas include (Table 4): Table 4. Exploring solutions. Brainstorming

SCAMPER

Mental maps

Lateral Thinking

Some more complex project development techniques, such as Design Thinking or Design Sprint, may be used punctually. However, such techniques will be used in this phase only in their activities of generating solutions and stimulating divergent thinking. Design – Mapping Solutions. In the previous phase, many alternatives are expected. This phase then employs convergent thinking to identify which alternatives are worth the effort to be developed. Teachers, students, and education professionals are also invited to participate in this phase. Among the main activities are (Table 5):

408

E. R. V. da Silva and P. Beça Table 5. Mapping Solutions.

Refining requirements and propositions Analyze and understand, through the alternatives, which of them have the most potential to solve the pedagogical problem. Gather information about what can be done and how the idea can be developed Skeleton Design

Indicate the scope of the project, and define it in terms of goals, people, time, and budget

Detailed design specifications

Evaluate the specific needs of the solution chosen

Construction – Create Initial Prototypes. Prototypes are important tools for understanding the functioning of the chosen alternatives. This strategy is used in important development methodologies such as Design Thinking (Brown 2005). A prototype should be analyzed directly with the target audience of the intervention. Professional podcast producers, teachers, and students will be invited to participate in the prototype evaluation process. For the development of the intervention in this research, it is planned to use a prototype with the following characteristics: Low-complexity and high-complexity prototypes. Where a low-complexity prototype has a low level of detail, often developed with pen and paper and at a low cost. And high-complexity prototypes already have a visual and functional aspect very close to the final version. Construction – Revise Prototypes. Prototypes are built to be tested. Feedback from producers, teachers, and students will be collected and analyzed. Researchers must understand if the data is sufficient to begin the development of the intervention, or if the prototype needs to be evolved for another round of testing. As it is an interactive and iterative phase, this cycle of co-construction through the prototype can be performed in several cycles. However, in the end, the goal is to have assertive information for the development of the educational intervention. 4.3 3rd Phase: Evaluation/Reflection Evaluation – Planning. The successive approximations, from a low-complexity prototype to a high-complexity prototype, present us with important analysis scenarios. However, once the intervention has been developed, it must be evaluated in a real-world scenario. Assisted by the information gathered through the initial study and prototype analysis, we seek to understand which investigation processes will be used. The objective of this phase is to establish the focus of the evaluation, define the questions, select strategies, determine methods, and plan documentation. The investigation methods considered for this phase are (Table 6): Evaluation – Field Work. After the research instruments have been defined, it is time to go to the field. This investigation takes place in technical and vocational schools. Plomp [26] adapted studies by Mafumiko [27] to propose which actors are important for testing the intervention. Therefore, it is considered to use the following participants in the research (Table 7):

Creation of Podcasts to Promote Students’ Engagement

409

Table 6. Planning. Focus Group

After teachers and students have used the developed intervention, they will be invited to participate in a group discussion about the tool. The discussion will be recorded and analyzed

Interview/Survey

Teachers and students are interviewed/answered questionnaires to gather their impressions on the developed intervention and how it influenced student engagement

Document Analysis

The documents that record the level of student engagement after using the intervention will be analyzed, such as attendance records and grades

Test

Teachers and students will use the intervention. The goal is to raise awareness about the tool and collect users’ impressions

Logbook

Students will be asked to report their experiences using the developed intervention in a logbook

Table 7. Field Work. Students from Technical and Vocational Schools in Portugal

2 Classes

Students from Technical and Vocational Schools in Brazil

2 Classes

Teachers from Technical and Vocational Schools in Portugal

2 teachers

Teachers from Technical and Vocational Schools in Brazil

2 teachers

Podcasts producers in Portugal and Brazil

5 producers

Design-Based Research (DBR) is a methodology that views the researcher as a part of the actors involved in the research, along with their personal and professional contributions. Therefore, the research team will also participate in the process of using the podcast development framework. Evaluation – Meaning Making. The fieldwork will be responsible for data collection. In this phase, the data will be analyzed to draw conclusions and answer the main research questions. The findings from the use of the intervention will be considered and weighed for information extraction from the collected data. Reflection – Organic. After the data collection process, there will be a pause in the documentation process to allow for a free reflection on the collected information. Reflection – Structured. Structured reflection is part of the knowledge construction process based on the information gathered during the evaluation activities. Among the reflection activities are (Table 8):

410

E. R. V. da Silva and P. Beça Table 8. Structured.

Preparation

Compiling the main facts/observations to be considered in the investigation

Image Forming

Perform a selection and synthesis of the most relevant facts and observations for the investigation

Conclusion Drawing Using the results of the previous two steps, build the final considerations on the research and establish future activities

4.4 Implementation and Spread McKenney and Reeves (2012) describe that there is an implicit fourth phase in DBR. The intention is to make the developed intervention capable of reaching other scenarios and be used in different contexts. Implementation. Results have been generated. The intervention is ready for use. It is time to prospect new users of the intervention through activities such as courses, the creation of explanatory material, workshops, etc. Small initial initiatives (few students, few teachers, few classes, few schools, etc.) and growing over time. And still maintaining the iterative characteristic of DBR, these activities will generate information for the evolution of the intervention itself. Spread. Propagation of designed interventions (or their underlying ideas) for use outside the initial context of field testing. Stimulating the sharing of information about interventions, e.g., through presentations at conferences, workshops, publications in journals, or other means of communication. Another way is to encourage practitioners to exchange information about the intervention, organize demonstrations, or train each other.

5 Conclusion and Future Work This work proposes to present the methodological steps for the creation of a podcast development framework, with the purpose of being used by teachers and students of technical and vocational schools. The chosen research methodology was design-based research, which is based on the construction of an intervention for a real pedagogical problem. The intervention proposed by this investigation is based on theoretical studies on student engagement. Understanding student engagement is about perceiving how students actively participate in schools. Engaged students have a greater chance of achieving personal and academic success. The creation of podcasts as a didactic tool aims to increase student engagement. Students’ engagement, in turn, is a pedagogical approach that emphasizes the active construction of knowledge by the student. That is, students are encouraged to develop problem-solving skills, critical thinking, and creativity by solving real-world problems.

Creation of Podcasts to Promote Students’ Engagement

411

The process of developing a podcast by students can be planned to reflect the concepts of the constructivist approach to teaching and learning. This project is still in the planning phase, and as such, is subject to adjustments. Understanding the strengths and limitations of the methodological plan is important to generate a real impact on technical and vocational schools.

References 1. Ahmad Shahrizal AZS et al (2022) A systematic literature review on the use of podcasts in education among university students. Asean J Teach Learn High Educ 14(1). https://doi.org/ 10.17576/ajtlhe.1401.2022.10 2. McNamara SWT, Shaw M, Wilson K, Cox A (2021) Educational podcasts in kinesiology: a scoping review. Kinesiol Rev 10(1):88–100. https://doi.org/10.1123/kr.2020-0007 3. Nicola W (2022) Enhancing student social work practice skills and critical thinking through podcast production. Int J Teach Learn High Educ 33(2):242–247 4. Drew C (2017) Educational podcasts: a genre analysis. E-Learn Digit Media 14(4):201–211. https://doi.org/10.1177/2042753017736177 5. Bolden B (2013) Learner-created podcasts. Music Educ J 100(1):75–80. https://doi.org/10. 1177/0027432113493757 6. Powell L, Robson F (2014) Learner-generated podcasts: a useful approach to assessment? Innov Educ Teach Int 51(3):326–337. https://doi.org/10.1080/14703297.2013.796710 7. Goldman T (2018) The impact of podcasts in education, Santa Clara. https://scholarcommons. scu.edu/engl_176/29/. Accessed 18 Apr 2023 8. Hill A (2021) The power of voice: using audio podcasts to teach vocal performance and digital communication. J Commun Pedagogy 4:38–50. https://doi.org/10.31446/JCP.2021.1.04 9. Berry R (2006) Will the ipod kill the radio star? profiling podcasting as radio. Convergence Int J Res into New Media Technol 12(2):143–162. https://doi.org/10.1177/1354856506066522 10. Bontempo R (2020) Podcast Descomplicado: Crie podcasts impossíveis de serem ignorados, 2nd ed., vol 1. Bicho de Goiaba 11. Rime J, Pike C, Collins T (2022) What is a podcast? considering innovations in podcasting through the six-tensions framework. Convergence Int J Res into New Media Technol 28(5):1260–1282. https://doi.org/10.1177/13548565221104444 12. Fronek P, Boddy J, Chenoweth L, Clark J (2016) A report on the use of open access podcasting in the promotion of social work. Aust Soc Work 69(1):105–114. https://doi.org/10.1080/031 2407X.2014.991338 13. Menezes C, Gamboa MJ, Brites L, Oliveira M (2021) Dez minutos de conversa podcasting como recurso de formação multidimensional. Comunicação.Pública, vol 16 14. Kendall R (2014) Podcasting and pedagogy. In: Mobile pedagogy and perspectives on teaching and learning, pp 41–57. https://doi.org/10.4018/978-1-4666-4333-8.ch003 15. Hu S, Kuh, G (2001) Computing experience and good practices in undergraduate education: does the degree of campus wiredness matter?. In: Annual meeting of the american educational research association, Seatle 16. Beer C, Ken C, Jones D (2010) Indicators of engagement. In: Ascilite 2010, Sidney: Brisbane, Qld.: University of Queensland 17. Kuh GD (2009) The national survey of student engagement: conceptual and empirical foundations. New Dir Inst Res 2009(141):5–20. https://doi.org/10.1002/ir.283 18. Trowler V (2010) Student engagement literature review. High. Educ. Acad 11 19. Trowler V (2010) Student engagement literature review. york: the higher education academy. https://www.academia.edu/download/30908577/StudentEngagementLiterat ureReview.pdf. Accessed 03 Apr 2023

412

E. R. V. da Silva and P. Beça

20. Middleton A (2009) Beyond podcasting: creative approaches to designing educational audio. Res Learn Technol 17(2). https://doi.org/10.3402/rlt.v17i2.10871 21. Jayatilleke BG, Ranawaka GR, Wijesekera C, Kumarasinha MCB (2018) Development of mobile application through design-based research. Asian Assoc Open Univ J 13(2):145–168. https://doi.org/10.1108/AAOUJ-02-2018-0013 22. van den JJH, Jan JH, Akker et al (2013) Educational design research/part a: an introduction 23. McKenney S, Reeves TC (2013) Systematic review of design-based research progress. Educ Res 42(2):97–100. https://doi.org/10.3102/0013189X12463781 24. de Martins LM, Ribeiro JLD (2017) Engajamento do estudante no ensino superior como indicador de avaliação. Avaliação: Revista da Avaliação da Educação Superior (Campinas) 22(1):223–247. https://doi.org/10.1590/s1414-40772017000100012 25. Hulley SB, Cummings SR, Browner WS, Grady, DG (2007) Newman, designing clinical research, 3rd ed., Philadelphia: Lippincott Williams & Wilkins, vol 1 26. Plomp T (2013) Educational design research: an introduction. In: Educational design research - part a: an introduction, Enschede, The Netherlands: SLO, pp 10–51 27. Mafumiko FSM (2006) Micro-scale experimentation as a catalyst for improving the chemistry curriculum in Tanzania. University of Twente, Enschede

Usability Requirements for Visualization Technologies in Industrial Field Giovanna Giugliano(B)

, Mario Buono , Sonia Capece , and Francesco Caputo

Department of Engineering, University of Campania “Luigi Vanvitelli”, Aversa, CE, Italy [email protected]

Abstract. Data communication and visualization play a significant role in contemporary society in which the amount of information available is constantly growing. The paper focuses on the importance of communicating and visualizing data clearly and effectively, with a focus on industrial manufacturing contexts. Data visualization is a crucial element in enabling users to understand complex information sets through visual representations that make them easily accessible and interpretable. It is the process by which objects, numbers and concepts are transformed into an easily understood form. Within production processes, where information flows are numerous and complex, dashboards are used for rapid monitoring of metrics and performance indicators. To ensure the effectiveness of a dashboard, it must be adapted to the specific needs of the intended user and be consistent with its original purpose. The paper proposes an iterative process for designing the interaction between the user and the dashboard, based on the analysis of theoretical models related to visual perception, visual communication, and information visualization. From the collection of heuristics in the literature, requirements for evaluating the usability of information visualization systems were identified. The set of heuristics has been and are going to be a valuable resource for defining and implementing evaluation tests useful in designing new dashboard systems that make information easily accessible. Keywords: Visualization of Information · Usability Requirements · Interactive Dashboards · Visual Perception · Evaluation Heuristics · Cognitive Workload

1 Introduction Automation and the constant development of recent technologies in industrial contexts have led to the integration of complex control systems that modify the user’s working methods, increasing the workload. For this reason, the user has to construct new forms of interaction with technology that require greater cognitive skills and abilities. According to the European Commission (2021), a prerequisite for shaping the new Industry 5.0 is that technology should serve people and thus is adapt to the needs and diversity of workers in industry. According to the general measures for the protection of workers’ health and safety (Art. 15 Italian Legislative Decree 81/08), prevention planning must be aimed at integrating the company’s technical production conditions with the influence of factors in the environment and work organization by respecting ergonomic principles. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 413–424, 2024. https://doi.org/10.1007/978-3-031-47281-7_34

414

G. Giugliano et al.

To promote the well-being of workers, it is necessary to change the design paradigm and focus on human factors and ergonomics requirements (Kadir & Broberg 2021). It therefore becomes necessary to design and configure human-machine interfaces that ensure a natural interaction capable of reducing the cognitive and physical workload induced by communication that pervades all work activities. Users use dashboard systems to manage and compare values with past performance and future goals (Pappas & Whitman 2011). In understanding dashboards as a means of communication, effective design is related to various theoretical foundations such as cognition and perception, situational awareness, and visualization technologies. (Park & Jo 2019) More specifically, the configuration of a dashboard should be based on an understanding of how the user “sees and thinks” (Few 2009). Usability related to information visualization tools focuses on the tool’s ability to display data in an understandable way and allow the user to explore and interact with the data. Due to the recent emergence and popularity of dashboards, studies on design principles for specific outputs are still insufficient. Currently, heuristic evaluation is adopted in human-computer interaction studies as a method to assess the usability of information systems. This method applies criteria/heuristics to evaluate the interface and provide feedback on critical usability issues of the system (Dowding & Merrill 2018). The paper presents the analysis of sets of heuristics proposed in the literature and the study on visual perception that led to the identification of requirements for the usability evaluation of information visualization systems. Specifically, the paper describes the reference scenario, defining the critical issues that emerge in the industrial context due to the cognitive workload caused by the diffusion of information visualization systems. The iterative process behind the communication and transmission of dashboard messages to users is defined, based on Munari’s model (1968) and on the analysis of theories and models for the visualization and understanding of information. According to the UNI CEI ISO/IEC 25010:2020 standard for the evaluation of the use quality and product quality model of systems and software and for the analysis of visualisation models, a set of heuristics was defined for the evaluation of dashboards and analysed with respect to the tasks (Jeffrey & Shneiderman 2012) that influence the interactive process under study. The results emerged from the entire critical-reconnaissance analytical in literature of existing dashboards solutions and visualization models and have formed the basis for foreshadowing future developments of research and defining new dashboards systems capable of making information accessible to the user-operator.

2 The Cognitive Workload in the Human-Centric Industry 5.0 In the European Commission’s (2021) “Industry 5.0 - Towards a sustainable, humancentric and resilient European industry”, the main axes identified by the European Union for defining Industry 5.0 emerge: human-centricity, sustainability and resilience. In detail, from the transformation of industrial spaces into safe and inclusive environments

Usability Requirements for Visualization Technologies

415

and through meeting needs, we are witnessing the changing role of the user “positioned” at the center of production processes to implement levels of psycho-physical well-being (Giugliano et al. 2023). In Industry 5.0, Romero (2016) defines Operator 4.0 as a skilled user who performs not only “cooperative work” with robots - but also work supported by machines by means of cyber-physical systems, using advanced human-machine interaction and adaptive automation technologies. With the increase of cyber-physical systems following the advent of Industry 4.0, digital technologies, technical, organizational and design changes have been introduced. Smart Products and devices introduced by Industry 4.0 are designed to provide real-time communication between machines, work resources and users, thus establishing a basis for the implementation of new production processes and semi-autonomous control of individual stages of their production (Leso et al. 2018). This has led to the definition of new forms of human-machine interaction and a new balance between physical and cognitive load. Mental and physical workload and psychological stress are common in work environments, and these concepts can influence each other (Parent et al. 2019). Due to environmental factors, task complexity or repetitiveness, and work organization, Operator 4.0 is exposed to significant cognitive load. Therefore, proper design of work activities and workstations must ensure full operator involvement, risk assessment, and interaction to improve safety (Lombardi et al. 2023). The decision-making process behind the choice of data to be visualized defines the “problem” to be communicated and implicitly highlights the values of the possible solutions (Sarikaya 2019), and for this reason, to reduce the workload triggered by the transmission and reception of information, it is necessary to design the visualization of work metrics or KPIs in such a way that, in addition to performance, they are able to communicate the priorities and “value” of the data (Sarikaya 2019).

3 Visual Elements of the Dashboard In order to configure work environments where human-machine interaction is facilitated, dashboards must be effective in transmitting information and capable of adapting to the intended user, in a consistent way with the achievement of the identified objective. Dashboards are commonly categorized according to several variables (Few, 2006), including role, data domain, update frequency, type of interactivity and visualization mechanisms. According to Sarikaya et al. (2019) it is possible to identify two design perspectives distinguishing dashboards in “visual genre”, i.e. the visual representation of data, structured according to a layout of graphs and/or numbers; or, “functional gender” interactive displays that allow real-time monitoring of dynamic data updates. The most common taxonomy of dashboards refers to the role they play, which can be strategic, analytical, and operational. This is the only classification that significantly highlights the differences in the visual elements that make up dashboards. In order to define a categorization of interaction systems applied to production contexts, the components that constitute dashboard systems were analyzed and divided

416

G. Giugliano et al.

into three macro-elements (Dhanoa et al. 2022): data model, visual components and interaction, and control elements. The users can manipulate data models indirectly through interactions with dashboards. Such indirect manipulation of the data model often results in an update of the visual components presented to users. Visual components are the dashboards’ones that are represented to the user with visualization elements such as bar and line graphs. These are included among the interaction and control elements, along with interactive components that can be controlled by the user through input functions. Interactions are closely related to the visual components: some manipulate the data model; others may directly affect them without altering the model itself. Thus, the definition of visual components involves the determination of the several types of interactions that may affect communication processes with visualization systems. According to Munari (1968), the visual communication of information between an emitter (dashboard) and a receiver (the user) may be subject to visual noise in the environment that may alter or invalidate the message. This message will be perceived by the user according to three filters (Munari 1968), i.e., the (I) sensory filter, the (II) cultural filter and finally the (III) operational filter of a psycho-physiological nature, which can contaminate each other and define the attainment of the correct information and, consequently, the user’s behavior (Fig. 1).

Fig. 1. Graphic reworking of Munari’s (1968) model of visual communication transmitterreceiver.

4 Theories and Models for Visualizing and Understanding Information From the study of visual design of 144 dashboards case studies Bach et al. (2023) come to the determination of guidelines for configuring new dashboard systems. Therefore, such systems should: not overwhelm users; avoid visual clutter and carefully choose

Usability Requirements for Visualization Technologies

417

KPIs to represent; align with existing workflows; not display too much data; have functional and visual characteristics; provide consistency, interaction possibilities and manage complexity; and organize graphs symmetrically, group and sort them by time. In understanding dashboards as a communication medium, effective design is related to the theoretical foundations of cognition and (i) visual perception and (ii) situational awareness (Park & Jo 2019). For the design of dashboard systems, Few (2013) formulates three principles on (i) visual perception. Users have a limited working memory and therefore in order to promote efficient perception and memory retention it is necessary to use graphics instead of numbers and text; in order to promote fast perception, pre-attentive attributes (color, shape, spatial position and movement) should be used correctly and elements such as proximity, closure, continuity and connection should be considered, and in line with Gestalt principles (Gibson 1950). Based on the model proposed by Munari (1968), these visual perception can be considered as components that constitute and influence the user’s sensory (I) filter. On the other hand, Endsley (2012) defines (ii) situational awareness according to three levels, in terms of the identification of information that is fundamental for the performance of a given task and/or objective: level 1_ perception of the elements of the environment; level 2_understanding of the current situation; level 3_projection of the future state. These levels of situational awareness can be divided between the components that make up the (I) sensory filter - level 1 - and the (II) cultural filter - levels 2 and 3. Downstream of the determinations of (i) visual perception and (ii) situational awareness, the implications of Few’s design principles for dashboards predict (Park & Jo 2019): – the hierarchization of information to ensure the relevance of the most important data; – the clarity of visualization to facilitate the rapid perception of information using different visualization technologies; – the immediacy of information to support user activities, goals and decision-making. For the design of dashboards, it is necessary to analyse the user learning process in order to identify critical issues related to the information transmission and the ability to use visualisation systems. To determine the effectiveness of visual perception and situational awareness, however, it is necessary to consider the learning process that characterizes the interactive phase of the user’s reception of images. In fact, the user learning process needs to be analyzed in order to identify critical issues related to the way information is transmitted and the ability to use visualization dashboard systems. Among learning analysis models, Verbert et al. (2013) introduce a model based on personal computer applications that analyses the user’s information behavior and control in four stages: – awareness of the data displayed; – reflection on data of interest and questioning of past behavior;

418

G. Giugliano et al.

– creation of meaning to answer questions and define nine insights; – impact, i.e. the user deduces the new meaning and changes the behavior. Such a model is useful for identifying how the user will receive and interpret the received message and consequently the effectiveness of the output he or she transmits. Instead, for the evaluation of learning, Kirkpatrick (2006) defines a tiered model based on four questions: 1 - reaction/satisfaction, i.e. how users perceive and react to the dashboard; 2 - learning, i.e. the extent to which users understand the statistical information on the dashboard; 3 - transfer, i.e. the extent to which users change their behavior using the dashboard; 4 - outcome, i.e., the extent to which the dashboard affects users’ learning outcomes. This model will be necessary for the evaluation of the output transmitted by the user. Figure 2 shows the schematization of the iterative process of the design of the issuerreceiver interaction applied to the Munari model (1968), defined from the principles of design and understanding of the visual components of the dashboards (Few 2013; Endsley 2012; Park & Jo 2019), from Verbert et al. learning model (2013) and from Kirkpatrick’s assessment model (2006). In fact, based on Munari’s (1968) determinations on how a receiver’s visual messages are acquired by the issuing system, the processes involved in the transmission phases have been identified.

Fig. 2. Schematic representation of the dashboard-emitter and user-receiver interaction applied to Munari’s (1968) model.

In this process, it is relevant that the dashboard interface system communicates with the user effectively and efficiently. Therefore, it is necessary that these systems correspond and comply with the requirements and standards to enhance the transmission of dashboard messages and improve the user’s perception and usability of the system.

Usability Requirements for Visualization Technologies

419

5 Approaches and Methods to Evaluate the Usability of Data Visualization Systems Visualization systems provide tools and techniques for transmitting data sets, for understanding information or amplifying user knowledge. Among the methods for evaluating visualization dashboard systems, Ma & Millet (2021) propose usability tests for assessing the effectiveness, efficiency and user satisfaction using dashboards. According to the authors, the tasks used in usability tests should include the distribution, filtering and grouping of data and the identification and comparison of displayed parameters. Usability related to information visualization tools focuses on the ability of the tool to convey data in an understandable way and to allow the user to explore and interact with that data. Heer & Shneiderman (2012) propose a taxonomy of interactive dynamics that contributes to successful data visualization. This taxonomy consists of 12 task types grouped into three categories that incorporate the critical tasks for iterative visual analysis: (a) Data and View Specification (visualize, filter, sort, and derive); (b) View Manipulation (select, navigate, coordinate, and organize); (c) Process and Provenance (record, annotate, share, and guide). For the generation and exploration of large data sets the tools of (a) Data and View Specification must allow data specification and definition of views of interest. This allows to selectively view data and filter information of interest with the aim to support determining and detecting relevant items. It also allows to sort the information displayed, show models, and provide visual analysis tools that should include functions that can be helpful to get new data. After data visualization and specific view definition tasks, users must be allowed to manipulate the view itself - (b) View Manipulation - in order to deepen the elements and details of interest. It is necessary to enable the user to select, highlight, filter and operate on elements and to provide intuitive navigation tools between data, which can coordinate multiple views and organize working spaces. Visual analysis, as a process of iterative exploration and interpretation of data, must include tools that preserve processes related to user activities - (c) Process and Provenance. To do so, it is necessary to: record actions performed to support the analysis process and the subsequent refinement of work activities; annotate patterns to enable documentation of results; share views and annotations to enhance collaboration processes and guide users through analysis activities. Among human-computer interaction studies to evaluate the usability of information systems, heuristic evaluation is the “inspection” method in which defined criteria are used to evaluate the interface and provide feedback on potential usability problems of the system. Elements such as the visual representation of information, perceptual/cognitive issues related to such representations, and data interaction mechanisms are not specifically addressed in common heuristics.

420

G. Giugliano et al.

Dowding & Merrill (2018) propose a list of ten heuristic principles and usability factors for evaluating dashboard systems, incorporating seven of Nielsen’s (1990) usability principles – (1) visibility of system status; (2) match between system and the real world; (3) user control and freedom; (4) consistency and standards; (5) recognition; (6) flexibility and efficiency of use; (7) aesthetic and minimalist design/remove the extraneous – and three specific principles for displaying information – (8) spatial organization; (9) information coding; and (10) orientation. Further principles and factors for evaluating the usability of dashboard systems can be found in the UNI CEI ISO/IEC 25010:2020 standard, which defines the in-use quality model and the product quality model of systems and software.The in-use quality model consists of five characteristics relating to the results of interaction with a system and 11 associated requirements for each characteristic and refers to the result of interaction when a product is used in a particular context of use: (11) effectiveness; (12) efficiency; (13) satisfaction; (14) absence of risk; and (15) context. The proposed normative qualitative model for measuring the quality perceived by the user when using software is composed of eight characteristics, divided into 31 factors, and refers to the static properties of the software and the dynamic properties of the computer system: (16) functional appropriateness; (17) performance; (18) compatibility; (19) usability; (20) reliability; (21) security; (22) maintainability; and (23) portability. 5.1 Heuristics for Evaluating Dashboards Starting from the design principles proposed by Few (2013) (Park & Jo 2019) for visual perception and situational awareness levels introduced by Endsley (2012) (Park & Jo 2019) of dashboards, were identified the requirements for the assessment of the quality of systems, defined by UNI CEI ISO/ IEC 25010:2020 and the checklist proposed by Dowding & Merrill (2018), and defined a set of heuristics (Fig. 3) which contains the synthesis of the four models analyzed. Specifically, a set of 12 heuristics was collected as follows: Hierarchization of Information; Clear information; visibility and clarity of the system; system-real-world coherence; user control; Conceptual model conformity; Flexibility; Minimalist aesthetics and organization; Quality perceived by the user; Usability; Performance and maintainability; and tolerance to error. The identified heuristics were subsequently compared (Fig. 4) to the actions defined in the taxonomy of Heer & Shneiderman (2012) in order to identify criteria for the evaluation of the activities of use of dashboard systems. These heuristics will represent the tools for the definition of evaluative tests to be administered to experts in order to reduce the cognitive load of the user-operator, identify the criticalities of current systems and the elements for the design-new data visualization models.

Usability Requirements for Visualization Technologies

421

Fig. 3. Collection of identified principles: Dowding & Merrill checklist (2018); UNI CEI ISO/IEC 25010:2020 requirements; Few’s principles of visual perception, (2013) and Endsley’s situational awareness levels (2012). Summary and definition of the set of 12 heuristics.

422

G. Giugliano et al.

Fig. 4. Comparison of the set of 12 identified heuristics with the activities defined by the taxonomy of Heer & Shneiderman (2012).

6 Conclusion In Industry 5.0 it will be necessary to introduce adaptive and reconfigurable systems with real-time data processing capabilities that can address ergonomic issues while ensuring productivity in human-machine collaboration. From the analysis of Munari’s (1968) theoretical foundations, it was possible to define a model of visual communication, supplemented by approaches and models in the literature for visualizing and understanding information. Furthermore, from the requirements identified in the literature, it was possible to define a set of heuristics for the definition of evaluation tests and guidelines for the configuration of new dashboard systems able to make information accessible for application in industrial environments. By means of natural human-machine communications, including empathic understanding, it will be possible to configure anthropocentric models and processes (Lu et al. 2021), converting current optimization practices oriented towards production control for the benefit of user-operator well-being. This transition towards flexible collaboration will bring new challenges and opportunities to ensure increased production and worker well-being.

Usability Requirements for Visualization Technologies

423

References Bach B et al (2023) Dashboard design patterns. IEEE Trans Visual Comput Graph 29(1):342–352. https://doi.org/10.1109/TVCG.2022.3209448 Dhanoa V, Walchshofer C, Hinterreiter A, Stitz H, Groeller E, Streit M (2022) A process model for dashboard onboarding. Comput Graph Forum 41(3):501–513 Dowding D, Merrill JA (2018) The development of heuristics for evaluation of dashboard visualizations. Appl Clin Inform 9(3):511–518 Endsley MR (2012) Designing for situation awareness: An approach to user-centered design. CRC Press, Baco Raton European Commission Industry 5.0. https://op.europa.eu/en/publication-detail/-/publication/468 a892a-5097-11eb-b59f-01aa75ed71a1/. Accessed 11 Dec 2022 Few S (2006). Information dashboard design. the effective visual communication of data. O’Reilly Few S (2009) Now you see it: simple visualization techniques for quantitative analysis. Analytics Press, Burlingame Few S (2013) Information dashboard design: Displaying data for at-a-glance monitoring, 2nd edn. Analytics Press, Burlingame Gibson JJ (1950) The perception of the visual world. The Riverside Press, Cambridge, USA Heer J, Shneiderman B (2012) Interactive dynamics for visual analysis. Commun ACM 55(4):45– 54 ISO/IEC DIS 25010(en) Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Product quality model Kadir BA, Broberg O (2021). Human-centered design of work systems in the transition to industry 4.0. Appl Ergon 92:103334 Kirkpatrick DL, Kirkpatrick JD (2006) Evaluating training programs: the four levels, 3rd edn. Berrett-Koehler, San Francisco Kumar SM, Belwal M (2017). Performance dashboard: cutting-edge business intelligence and data visualization. In: International conference on smart technologies for smart nation (SmartTechCon), pp 1201-1207 Giugliano G, Laudante E, Formati F, Buono M (2023). Approaches and technologies for designing the human-centered industry 5.0. Proyecta56 Ind Des J (3):78–90. https://doi.org/10.25267/ P56-IDJ.2023.i3.05. ISSN 2386–5414, e-ISSN 2340–8391 Legge 9 aprile 2008, n. 15 Testo unico sulla salute e sicurezza sul lavoro Leso V, Fontana L, Iavicoli I (2018). The occupational health and safety dimension of industry 4.0. La Medicina Del Lavoro, 109(5), 327 Lombardi I, Martínez V, Senese V, Capece S (2023). Design, human factors and neuroergonomics for safety in manufacturing. In: Matteo Zallio (eds) human factors in accessibility and assistive technology. AHFE (2023) international conference. AHFE open access, vol 87. AHFE International, USA. https://doi.org/10.54941/ahfe1003653 Lu Y, Adrados JS, Chand SS, Wang L (2021) Humans are not machines—anthropocetric human– machine symbiosis for ultra-flexible smart manufacturing. Engineering 7(6):734–737 Ma Q, Millet B (2021) Design guidelines for immersive dashboards. In: Proceedings of the human factors and ergonomics society annual meeting, vol 65, no 1, pp 1524–1528 Munari B (1968). Design e comunicazione visiva. Laterza, Bari Nielsen J, Molich R (1990). Heuristic evaluation of user interfaces. In: CHI ‘90 proceedings of the SIGCHI conference on human factors in computing systems Seattle, Washington. ACM, New York, pp 249–256 Pappas L, Whitman L (2011) Riding the technology wave: effective dashboard data visualization. In: Smith MJ, Salvendy G (eds) Human Interface 2011, vol 6771. LNCS. Springer, Heidelberg, pp 249–258. https://doi.org/10.1007/978-3-642-21793-7_29

424

G. Giugliano et al.

Parent M, Peysakhovich V, Mandrick K, Tremblay S, Causse M (2019) The diagnosticity of psychophysiological signatures: can we disentangle mental workload from acute stress with ECG and fNIRS? Int J Psychophysiol Offic J Int Organ Psychophysiol 146:139–147. https://doi. org/10.1016/J.IJPSYCHO.2019.09.005Park,Y.,Jo,IH.:Factorsthataffectthesuccessoflearning Park Y, Jo IH (2019) Factors that affect the success of learning analytics dashboards. Educ Tech Res Dev 67:1547–1571 Romero D, et al (2016) Towards an operator 4.0 typology: a human-centric perspective on the fourth industrial revolution technologies. In: Proceedings of the international conference on computers and industrial engineering (CIE46), Tianjin, China, pp 29–31 Sarikaya A, Correll M, Bartram L, Tory M, Fisher D (2019) What do we talk about when we talk about dashboards? IEEE Trans Visual Comput Graph 25(1):682–692. https://doi.org/10.1109/ TVCG.2018.2864903 Verbert K, Duval E, Klerkx J, Govaerts S, Santos JL (2013) Learning analytics dashboard applications. Am Behav Sci 57(10):1500–1509

Image Dysmorphia on Digital Platforms: an Analysis of the Policies and Terms of Use on Instagram and Spark AR Jenifer Daiane Grieger1(B) , Rodrigo Eduardo Botelho-Francisco2 and Lídia Oliveira3

,

1 Universidade Federal Do Paraná, Curitiba, Brasil

[email protected]

2 Universidade Federal Do Paraná, Curitiba, Brasil

[email protected]

3 Universidade de Aveiro, Aveiro, Portugal

[email protected]

Abstract. This paper aims to investigate structural and social elements aimed at settling or exposing individuals to digital risks and vulnerabilities through Instagram policies and guidelines. We focus on content management, especially image, such as filters of face editing and modification that are available on these platforms, in the Brazilian context. Therefore, we seek to understand how these elements can contribute to the wider discussion on the use of filters and their relationship with mental disorders, such as Body Dysmorphic Disorder (BDD). Such idealization rely on the relationship between the beautiful and the desirable and can be maximized by the dissemination of augmented reality (AR) resources. The research follows a methodological approach based on Bardin’s Content Analysis, consisting of three phases: pre-analysis and material exploitation, development of analysis categories, and treatment of results. Our corpus of analysis is composed of the documents shared by the platforms: terms of use, community guidelines, data/privacy policies, and policy on cookies. The main contributions of this paper include: i) Emphasizes the importance of considering content in platform studies, an often-neglected dimension; ii) Explores interoperability between platforms and the balance between centralization and decentralization; iii) Recognizes the significant role of content in attracting users and understanding consumption trends; iv) Systematically analyzes the policies and guidelines of platforms such Instagram, Spark AR, and Facebook, to understand user-platform dynamics; v) Highlights the need for further research and best practices in content practices, particularly in appearance-related features like filters, with potential impacts on body image and self-esteem. Keywords: Digital platforms · Instagram · Spark AR · Policies of use · Image dysmorphia

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 425–437, 2024. https://doi.org/10.1007/978-3-031-47281-7_35

426

J. D. Grieger et al.

1 Introduction Since 2019, Facebook has been reevaluating its policies, especially those related I’m user well-being. This research paper examines the policies and terms of use on Instagram and Spark AR in relation to image dysmorphia. Focuses on the impact of content management, specifically filters for face editing and modification. The research includes a literature review and content analysis of the platforms’ documents, including terms of use and community guidelines. It also explores the outsourcing strategies related to image editing provided by Instagram. Over the past few years, the platform came to develop ideal “embellishment” filters for self-portraits, popularly known as selfies. These filters can be added both photos and videos. Their popularity and ease of face modification have led to changes in behavior of the platform users and, consequently, their perceptions concerning beauty and aesthetics. This research paper belongs to the field of platform studies and is focused according to Van Dijck et al., when highlighting the dimension of the matter of content, which, according to the author, has been neglected in the studies of platforms. Still, this research is based on the interoperability factor between platforms that reveals an “ecosystem” of platforms that articulates in a distributed manner (Van Dijck 2013). Centralizing and, at the same time, decentralizing is only one of the “paradoxes” that mark the action of online platforms (Van Dijck et al. 2018), “what is surprising, since the content (texts, images, photos, music, game, and videos, for example) are responsible for aggregating users into a platform, in addition to indicating consumption trends and preferences” (Montardo and Prodanov 2021). “The focus now is the complex and refined articulations among business models, infrastructures, databases, algorithms, governance rules, and a diversity of uses that constitute Twitter, YouTube, Uber, and many other services (D’Andréa 2020). The original element of this research is the investigation of how image content is managed by social network platforms in their documents and how it can contribute image dysmorphia. Our challenge here includes systemically analyzing the diversity of documents available on Instagram and Spark AR, in addition to the Facebook terms of services and community standards. The perspective of platforms as aggregators focused on performances and multiplicity of roles allows us to realize how companies like Uber have drivers who are not “theirs” (Poell et al. 2019), and Instagram filters are not designed by the developers in the company. Uber uses an outsourcing strategy both for the operation and the responsibility for the services provided. We understand that the perspective of platforms is necessary based on the understanding that the image content is treated in the analyzed corpus. In this sense, we established the following research question: How is image content managed by social network platforms in their documents and how can it contribute to image dysmorphia? Next, we present the theoretical background and the analysis model proposed for this investigation, the methodological procedures, discussion and, finally, the conclusion.

Image Dysmorphia on Digital Platforms: an Analysis

427

2 Theoretical Background Thomas Poell, David Nieborg, and José van Dijck define these platforms as “(re) programmable digital infrastructures that facilitate and shape customized interactions between final and complementing users, organized through systematic collection, processing algorithm, monetization, and data circulation” (Poell et al. 2019). As for the dynamics of the platforms, the authors state that a platform is fed with data, automated, and organized through algorithms and interfaces, formalized through relations of propriety guided by business models, and ruled by user agreements (Poell et al. 2018). For this research, we resort, especially to different platforming perspectives proposed by Van Dijck, Poell, and Wall (2018) based on the understanding that such a process unfolds into the following three institutional dimensions: infrastructures of data, markets, and governance. According to Semicek et al. (2021) “such the multiplicity is fundamental for its current constitution, thus requiring the data set of policies to regulate its relationship with consumers” (Semicek et al. 2021). Thereby, the company unfolds into a digital social network, social media, and social actor. From the infrastructure perspective, factors such as APIs and SDKs can be identified by the algorithm and by social data that are collected, stored, and monetized. Commercial market relations are business models that reorganize commercial transactions in varying productive sectors, through the data-transformation of commercials (van Dijck 2013), provided by third parties in the configuration of advertising and ads shown to users. Still, in the perspective of governance, we can glimpse a convergence among terms of use, laws, guidelines, as well as contracts, permissions, and licenses that guide the relations between actors in action. Lindner’s thesis (2021) deals with the effects, enabled by augmented reality, from the concept of “Proteus Effect”, also providing empirical data that assists in understanding the phenomenon of dysmorphia, especially when related to technological mediation, such as “Zoom Dysmorphia”. According to her, the study is based on the description and origin of AR face filters, the Technology and applications of AR face filters, the use of face filters on social media platforms, also investigating the effects of AR face filter on self-perception, including phenomena such as Body Dysmorphic Disorder (Lindner & Singh 2021). Despite gaining further contours from contemporary digital technologies, image dysmorphia relies on the relationship between the beautiful and the desirable carrying the aesthetic values of each time. The concepts of the beautiful can be seen throughout the history of humanity, expressed in the narrow relationship between beauty and art, such as in the renascence paintings, as well as in the modernity with photographs, television, and advertising (Eco 2004). Beauty is the cultural, social, and universal element that concerns appearances and not the being. The experience of beauty has an immediate, sensory, and intuitive nature. It is part of the context in which we live, of our desire for harmony, adequacy, and civility, as well as our desire for expression and validation (Scruton 2018). Beauty, however, is a social convention. As standards change throughout history, it is natural to see that different beauty standards exist and coexist at the same time, in different cultures. Pursuing it as an end per se may generate disorders in the human being. In this sense, image dysmorphia features individual concepts and beauty standards that

428

J. D. Grieger et al.

are often untouchable, which, in digital environments, with content aimed at the image, transform insecurities and small “flaws” into even further dimensions. The dissatisfaction with appearance and impacts on health manifest throughout the decades on physical and psychological disorders, such as depression, anorexia, and fatphobia, crossing our imaginary, promoting beauty ideations and validation involving actors, social practices, behaviour, and attitudes both collective and individual. Beauty ideals have always been present in the collective imagination but are now combined with technologies such as face recognition, artificial intelligence, and data processing in building an idealized self. Despite not being new, technologies like face recognition operate through the system using an algorithm and software programs to map face patterns. Simply put, the human face, despite its individuality, has a basic composition that does not change and can be read by applications through shared features (eyes, nose, mouth), varying according to the system complexity. On Instagram, as well as on other digital platforms and their beauty filters, these functions aren’t only mapping but allow for face modifications and improvements. Briefly, these tools combine artificial intelligence and computer view not only to detect facial features but modify them. Despite the simple functioning, they are indeed a technical stunt enabled by neural network advances, a computer model based on the human brain, which allows for the data processing required to alter videos in real-time. Studies on Instagram focusing on physical appearance have indicated perspectives aimed at understanding the consumption behaviour in these spaces and through them from areas like psychology and medicine, focusing on the effects and impacts of social networks on individuals. Recently published, the paper of Montardo and Prodanov (2021) suggests early clues of what they conceptualize as platforming of beauty. Their research sought to learn how the embellishing filters of Instagram stories influence the beauty perception of their users. Still, as the practice of selfies has guided the youngsters towards pursuing aesthetic surgeries and procedures, Lamp et al. (2019) associated selfies on Instagram with self-objectification behaviours with symptoms of depression. Another relevant factor concerning the content posted on Instagram is the comments and likes that show engagement of other users with the content posted Tiggemann et al. (2018) and Tiggemann and Barbato (2018), these papers explore self-objectification behaviours based on photos, selfies, and comments about appearance. However, what happens when idealized beauty reaches and is mediated through technology? Currently, digital platforms like Instagram paradigmatically change sociability and identity expression. The human face becomes the set of mappable and formattable points through data processing. Socialization, as well as the forms of representation and identity, are maximized through technological mediation, allowing for constructions and ideations focused on physical appearance. As Couldry (2016) suggests, “the social theory needs to be fully reconsidered in the world of digital media and platforms of social media triggered by data processing” (Couldry and Hepp 2016). Content (texts, images, photos, music, games, and videos, for example) is responsible for aggregating users into a platform, in addition to indicating consumption trends and preferences Montardo (2019). In addition to our perspective, on these platforms, content

Image Dysmorphia on Digital Platforms: an Analysis

429

is multifaceted elements that are considered social data that seek to translate behaviours to convert them into increasingly more accurate and customized ads. 2.1 Analyse Model Proposed The guidelines of Quivy and Campenhoudt (1995) concerning the construction of the analysis model allowed us to establish the relationship between the concepts. Based on the steps of exploitation, problem definition, and formulation of the research question, it is possible to develop an analysis model with methodological rigor. According to the authors, such an analysis model aims to allow the researcher to conduct the systematic work of gathering and analysis of data from observation or experimentation that must be followed. The analysis model elaborated for this research is presented below, according to Table 1. Table 1. Analyse model Concept

Dimensions and categories

Digital platforms

Infrastructure Market Governance

Dysmorphia

Idealization (identity and beauty) Artificial Effects (filters)

Image Content

Appropriation Intellectual Propriety Aesthetics

The following section presents the methodological procedures.

3 Methodological Procedures This is a methodological paper that aggregates a review of the literature based on Bardin’s Content Analysis, structured into three phases: pre-analysis and material exploitation, including the floating reading of documents and the development of categories of analysis and encoding Bardin (2011). Finally, we introduce the treatment of results, inferences, and interpretation of findings. Our analysis is composed of the common documents between the Instagram and Spark AR platforms: terms of use, terms of service, community guidelines, data policies, and policy on cookies. We collected the documents in May 2022, on both platforms. The research operates in two steps. The first step analyses only Instagram documents of terms of Service, Data policies, information on law applications, community guidelines, and policies of cookies. The second step incorporates the and guidelines of the

430

J. D. Grieger et al.

AR software, technology used in the development and availability of filters, and Spark AR. We analysed the Spark AR policies and Facebook terms of service combined with the Instagram terms of use, and community standards. Based on the theoretical background proposed for the concepts of digital platforms, image dysmorphia, and imagery content, we established the categories of the analysis presented in the analysis model.

4 Discussion Instagram is a platform created in 2010 and is aimed at the sharing of visual content and is now called Instagram by Meta. This change was published on January 4, 2022, and referred to the other Facebook product, which is called Meta. Thus, there were updates, for example, on the terms of use, data policy, and policy of cookies, as well as other documents, to bring the new name of the organization (Meta 2022). Based on the perspective proposed by Van Dijck, Poell, and Wall (2018), we sought to analyse the Instagram and Spark AR platforms according to their infrastructure, market, and governance. Within the scope of this research, and in the infrastructure dimension of referred platforms, we identified citations that guide us concerning the technological elements that form the company’s activities, organization, and functioning, as well as access to its products and services. An element that consolidates and singularises the idea of an “online platform” is the increasing use of a computer architecture based on connectivity and data interchange (D’Andréa 2020). The idea of platforms can be understood as a dynamic and complex set of technologies and actors in interactions. According to the company itself, this set consists of the integration of technologies like APIs, SDKs, tools, plugins, codes, content, and services that allow other people, including developers of applications and operators of sites, to develop functions, recover data of the Meta and any other Meta product, or provide data to us (Meta 2020). For this analysis, focused on image content management, first we should point out what is indeed data from the Meta Company and its products, in this case, Instagram and Spark AR. In more detail, “Platform data” are considered “any information, data, or other content that the user receives from the company, through the platform or its application, either directly or indirectly, and before, after, or in the data in which they accept these terms, including anonymized data, aggregates, or derivates of these data. Platform data include tokens of applications, pages, access, and users and secrete keys of applications” (Data Policy, Meta 2022). According to the Facebook Terms of Service (2022), “We use and develop advanced technologies (such as artificial intelligence, machine learning systems, and augmented reality) for people to be able to use our products with safety”. The excerpt below also addresses the organization and integration among the Meta companies and the process of data management.

Image Dysmorphia on Digital Platforms: an Analysis

431

“To operate our global service, we need to store and transfer data between our systems around the world, including outside their country of residency. The use of this global infrastructure is necessary and essential to provide our service. This infrastructure can belong to or be operated by Meta Platforms, Inc., Meta Platforms Ireland Limited, or their affiliates (Instagram platform terms 2022)”. In addition to the organization and functioning of the Meta products, the following excerpt addresses the services provided and the experience of users during use, to the same extent involving the technologies that allow improved them. “Your experience on Facebook cannot be compared not anyone else’s. This ranges from publications, stories, events, ads, and other content that you see on the newsfeed or our video platform to Facebook pages that you follow and other resources [...] we use the data that we have (such as connections established, choices, and configurations selected, sharing, and activity inside and outside our products) to customize your experience (Facebook Terms of Service 2022)”. It is worth highlighting that the technologies used to enhance the experience are also tools for the design of advertising and ads that are increasingly more accurate and built based on the collection and analysis of social data. The following excerpt addresses the contractual dynamics and, consequently, the permissions granted by individuals in the context of the Meta companies. “As our service is financed by Meta, instead of paying for the use of Instagram, using the service established in these terms, you acknowledge that we can show you ads that companies and organizations pay us to promote inside and outside the products of Meta companies. We use your data, such as information on your activities and interests, to show you more relevant ads. […] We do not share information that identifies you directly (such as your name or e-mail address, which someone could use to get in contact with you or identify who you are) unless you give us specific permission (Terms of Meta Platform 2022). Such a relationship between platforms and consumers encompasses market dimensions that include business partners, governance, policies, and guidelines, as well as laws, contracts, and permissions that manage both the relationships and content. Focusing on the infrastructure and the integrity of the services provided, concerning the content available on the platforms, the moderating process is emphasized without further details of its operation: “We develop automated systems to improve our ability to detect and remove abusive and dangerous activities that may cause damage to our community and the integrity of our products (Facebook Terms of Service 2022). Moderation appears in the corpus of analysis, especially in the terms of use, guidelines, and community standards, as well as in the Spark AR policies, an augmented reality software used to create effects (filters), always associated with content monitoring and removal. Further on, we approach the monitoring process in further detail. As for the intellectual propriety of the content, the terms of use of Instagram and Facebook mention that even after the change of the company name, it doesn’t relate to the rights over the user’s content.

432

J. D. Grieger et al.

“We do not claim the propriety of your content that you publish in the service or through it. In addition, you are free to share your content with anyone else, wherever you want. However, we require that you grant us certain legal permissions (known as “licenses”) for us to provide the service. When you share, publish, or upload content protected by rights of intellectual propriety (such as photos or videos) into our service or in connection with it, you grant us a non-exclusive, free, downloadable, sublicensable license and valid worldwide to host, use, distribute, modify, publish, copy, exhibit, or execute publicly, translate and create work derivates of your content (Instagram Terms of use 2021)”. According to the Meta company, the license granted ends when the content is deleted from the systems of the platform, and in case the user requests the removal of either the account or the content. Concerning image content, the categories of analysis were established based on factors such as appropriate – of technologies and the content available on the platforms, as well as of the actors involved – and the perspectives of intellectual propriety, aesthetics (format), and content. “We collect content, communications, and other information that you provide when using our products, including when you register to create an account, create, or share content, send messages, or communicate with other people. This may include present information or the content that you provide (such as metadata), like the location of a photo or the data in which a file was created. This may also include what you see through the resources that we provide, such as our camera so that we can execute actions like suggesting masks and filters that you may like or give tips on the use of camera formats. Our systems process the content automatically and the communications that you and other people provide the purpose of analysing the context and the content included in these items […]” (Instagram Data Policy 2022). As for the pursuit of beauty and the dysmorphia phenomenon, we established the following categories of dysmorphia: idealization, artificialization (computerized beauty), and filters (effects). Dysmorphia, especially in the context of digital platforms, is indicated here in from standardization perspectives, as well as based on the ideation with concepts of beauty and identity that design the desired appearance. In the context of the appearance available on image content, the documents that explicitly address this issue are the Facebook Community Standards, Instagram Community Guidelines, and Adequacy for the Base of Users; the latter is contained in the Spark AR policies. The topic of adequacy for the base of users established which content created by the users (developers) can be available and which is prohibited. In this sense, they also instruct on the content standards for the final base of users. For this research, we highlight: “The content must be adequate for consume by the general base of users of Facebook, Inc. (Including people aged 13 years or more)”; “ The content must not be of adult or sexual nature (for example, nudity, representations of people in explicit or suggestive positions or activities that are sexually suggestive or provoking)”;

Image Dysmorphia on Digital Platforms: an Analysis

433

“ The content must not be discriminated, bother, provoke, or undermine people” (Spark AR Policies 2022). The filters of Instagram and Facebook are created on the Spark AR platform. The Spark AR Studio and the Frame Studio, both “platforms of effects” and belonging to the Meta company, allow the creation and/or send content to create effects and/or themes to be used by people in a variety of products and services. The Spark AR Studio is a software developed by Facebook to design effects of augmented reality for applications and devices of Facebook, including a gallery of ready-made templates with models for the user to create their effects from a database, in addition to allowing them to create from scratch. The process of effecting creation works with graphic resources combined with technologies of augmented reality that map the face and allow for the overlay of images on videos responsively. Effects, in this interpretation, are technological mechanisms that guide and allow the achievement of idealized concepts of the beautiful and of the desirable. Despite being visual resources available on the platforms, these effects end up influencing behaviours and actions outside their scope (Montardo e Prodanov 2021). Concerning effects and filters, the adequacy for the base of users, according to the Spark AR policies, instructs that: “The content must not classify individuals according to their personality or physical appearance. [...] the content must not promote the use or represent the selling of potentially dangerous cosmetic procedures, according to the Facebook Community Standards. This includes effects that represent such procedures through surgical lines (Spark AR Policies 2022)”. Even though this analysis encompasses ten documents, policies and guidelines that are available on the three platforms of the Meta company, and the content image is the focus of two of them, only the two excerpts above are explicit about good content practices focused on appearance and aesthetics. As for the infrastructure of these mechanisms, the Spark AR policies mention matters of technical order of the actors involved, the platforms (Meta, Instagram, and Spark AR), developers (external but users), and final users. For the developers, according to the policies, the effect content must meet the Instagram Community Guidelines (for users of Instagram services), community standards (for users of any other of our products), and the Spark AR and Frame Studio policies. The following excerpts provide further detail on such dynamics. 1) “You are responsible for any effect content sent to us related to effect platforms”; 2) Facebook reserves the right to monitor or audit its compliance with the terms and update them from time to time. We can change the terms without prior notice and/or with notice through an update in the data of the “last modification”. The continuous use of effect platforms implies the acceptance of these changes (Frame Studio and Spark AR Terms). “Considering that you are a user of any another of our products, the effect platforms are part of the Facebook, according to the Facebook Service Terms, your

434

J. D. Grieger et al.

use of the platforms of effects is considered part of your actions on Facebook, and the effect content (including when applied to an effect or the content created by the user) is considered your “content”, according to the Facebook Service Terms. In the case of any conflict between these terms and the Facebook Service Terms, these terms will rule solely concerning the issue herein described and solely to the extent of the conflict (Frame Studio and Spark AR Terms 2022). Concerning the intellectual propriety of these functions, in further detail, “You are the owner of your effect content. You grant Facebook companies a world, irrevocable, perpetual license (subject to the applicable disabling request defined below), royalty-free, and sublicensable (to users) to copy, reproduce, modify, edit, make derivative works of, publicly use, publicly display, distribute, communicate to the public or make use of your effect content in connection with (a) providing users with the applicable effect through our products (which, for the avoidance of doubt, may include changes for purposes of technical compatibility) and (b) providing you with the effect platforms. You understand that this license includes the right for the Facebook companies to make the effect available to users (subject to the applicable Disablement Request set forth below) to use the effect with their photos or videos (“User-Created Content”) through the resources and functionality provided with Our products, and to make use of the User-Created Content, including downloading, publishing and sharing the User-Created Content (whether publicly or privately, or in any other manner) (Frame Studio and Spark AR Terms 2022)”. “Once you provide or receive or collect any information concerning your effect content (“effect data”), you agree to the following: 1. Ensure that any effect data is shared only with someone who is acting on your behalf, such as your service provider. You are responsible for ensuring that your service providers protect any effect data by limiting their use of this effect data only as necessary for the provision of services and keeping it confidential and secure. 2. Not use the effect data for any objective in the condition of providing the effect content directly associated with the effect data, or on an aggregated and anonymous basis and only to evaluate the performance and effectiveness of its effect content. 3. Do not use the effect data, including the steering criteria for your effect content, to create, attach, edit, influence, or augment user profiles, including profiles associated with any mobile device identifiers or any unique identifiers for any user, browser, computer, or device. 4. Do not transfer any effect data (including anonymous, aggregated, or derived data) to the ad network, ad exchange, data broker, or other ad-related service or profitability” (Frame Studio and Spark AR Terms). These effects use technologies that screen the face or the iris to allow for graphic changes through virtual reality. To be published, the visual effects created on the Spark AR and available on Instagram and Facebook need to meet the terms of use and community guidelines and standards. Community standards are guidelines that describe the expected standards on the content published on Facebook and the activity on this platform or any other Meta product (Facebook Terms of Use 2022). These policies and guidelines

Image Dysmorphia on Digital Platforms: an Analysis

435

inform what can and cannot be shared on the platforms, in addition to technical elements and, to some extent, social elements and the welfare of the consumers. Section 1 of the Spark AR policies considers elements of technical and structural order that guide the creation of effects and the performance of these resources. Effects must have the expected performance and provide a stable experience. Effects must be responsive and cannot slow down or delay. Effects must have a reasonable time of loading. In addition, effects must use only the resources that are allowed on Facebook, Instagram, Messenger, or Portal (Spark AR policies 2022). For technical and performance order, Sect. 2 addresses the AR content standards. The challenge is to analyse the standards of the content on the Spark AR to create filters combined with the Facebook Community Standards, Instagram Community Guidelines, and, finally, the Adequacy for the Base of Users. According to the Spark AR policies (2022), “any content (including all effects, names, icons, videos of the demonstration, and other material loaded or sent to Spark AR) must follow the standards and policies”. In general, the dynamics of responsibility, right, and control expresses asymmetries in the relationship between individuals and platforms since anyone aged thirteen years or more can create an account and use all functions of the three platforms. By agreeing with the terms of use, the individual grants the platforms and other related third parties a license, often free of duty, to manage personal data and images as information sources. This attribute the individual, or their parents, with the responsibility to preserve the safety, the development of digital skills, and the reduction of fragilities and vulnerabilities, without, however, providing neither functions or information to fill such a gap. In addition, the policies are explicit about not being responsible for content generated by third parties, even if they are users of the platform. Considering such a scenario, we suggest that further studies focus on the development of what has herein understood as good practices for functions like filters centered on physical appearance. Good practices may be based on regulations to rule the public and practices in collaboration with entities and scholars of the sector, as well as on the improvement of internal governance such as moderating and monitoring before the effects are published.

5 Conclusion This paper focused on the analysis of digital platforms, specifically Instagram and Spark AR, and their potential contribution to image dysmorphia. The research followed a methodological approach based on Bardin’s Content Analysis, consisting of three phases: pre-analysis and material exploitation, development of analytical categories, and treatment of results. The theoretical background of the research was based on the concept of digital platforms as programmable infrastructures that facilitate customized interactions and are organized through data collection, processing algorithms, and monetization. The content on these platforms plays a crucial role in aggregating users and indicating consumption trends and beauty idealization. The research also emphasizes the need for good practices and regulations to govern functions like filters centered on physical appearance.

436

J. D. Grieger et al.

The original element of this research was the investigation of how image content is managed by social network platforms in their documents and how it can contribute to image dysmorphia. The main contributions of this study include: i) Focusing on the dimension of content on platform studies: highlights the importance of considering the content appearance in the analysis of platforms, which has often been neglected in previous studies; ii) Analyzed the interoperability factor between platforms: explore the ecosystem of platforms and their distributed nature, highlighted the paradoxical nature of centralization and decentralization on platforms actions; iii) Examined the role of content in aggregating users and indicating consumption trends: recognizes that content plays a crucial role in attracting users to platforms and providing insights into consumption preferences; iv) Investigate the policies and terms of use that regulate platform dynamics and actors: _ analysis the policies and guidelines of Instagram, Spark AR, and Facebook to understand the responsibilities, rights, and control dynamics between individuals and platforms; v) points gaps in content practices focused on appearance and aesthetics: highlights the need for further research and the development of good practices for functions like filters centered on physics appearance, considering the potential impact on body image and self-esteem. Overall, this research provides insights into the dynamics of digital platforms, their influence on self-objectification behaviours, and the importance of considering the ethical implications of these platforms. Finally, the proposed analysis model can be used in future investigations with the same focus on digital platforms. The paper concludes with reflections on the structure of these platforms and their relationship with users. Acknowledgments. This research was funded by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes), Brazil, and by national funds through Fundação para Ciência e Tecnologia (FCT), Portugal, under the UIDB/05460/2020 project.

References Bardin L (2011) Análise do Conteúdo - Edição revista e ampliada Carlos d’Andréa (2020) Pesquisando plataformas online: conceitos e métodos (EDUFBA-Ed) Couldry N, Hepp A (2016) The mediated construction of reality Eco U (2004) História Da Beleza. In: Editora Record, vol 1 Instagram (2022, 3 de maio). Terms of Service. https://help.instagram.com/581066165581870/ FACEBOOK. Data policy (2022). https://pt-br.facebook.com/privacy/explanation/. Accessed 05/22/2022 FACEBOOK. Community Standards (2022). https://transparency.fb.com/pt-br/policies/com munity-standards/?source=https%3A%2F%2Fpt-br.facebook.com%2Fcommunitystandards. Accessed 05/22/2022 FACEBOOK (2022) Facebook terms of service. https://pt-br.facebook.com/terms Accessed 05/22/2022 Feltman CE, Szymanski DM (2018) Instagram use and self-objectification: the roles of internalization, comparison, appearance commentary, and feminism. Sex Roles 78(5–6). https://doi. org/10.1007/s11199-017-0796-1 van Dijck J, Poell T, de Waal M (2018) The platform society

Image Dysmorphia on Digital Platforms: an Analysis

437

INSTAGRAM (2021) Instagram terms of use. Accessed 05/22/2022 INSTAGRAM (2021) Instagram community guidelines. https://pt-br.facebook.com/help/instag ram/477434105621119. Accessed 05/22/2022 Lamp SJ, Cugle A, Silverman AL, Thomas MT, Liss M, Erchull MJ (2019) Picture perfect: the relationship between selfie behaviors, self-objectification, and depressive symptoms. Sex Roles 81(11–12):704–712. https://doi.org/10.1007/s11199-019-01025-z Lindner A, Singh A (2021) The use of augmented reality (AR) face filters on social platforms. University College London Montardo SP (2019) Selfies no Instagram: implicações de uma plataforma na configuração de um objeto de pesquisa. Galáxia (São Paulo) 41:169–182. https://doi.org/10.1590/1982-255420192 37688 Poell T, Nieborg D, van Dijck J (2019) Platformisation. Internet Policy Rev 8(4). https://doi.org/ 10.14763/2019.4.1425 Quivy R, vanCampenhoudt L (1995) Manual de insvestigação em Ciências Sociais Montardo SP, Prodanov LS (2021) Filtros embelezadores no Instagram Stories: pistas iniciais sobre a plataformização da beleza. LOGOS: Comunicação E Universidade, 2 Scruton R (2018) Beleza by Roger Scruton Semicek PH, Botelho-francisco RE, Vecchio de Lima MRD (2021) This is your digital life: redes, mídias sociais e o impacto do caso Cambridge Analytica na Política de Dados do Facebook. Revista Fronteiras – Estudos Midiáticos 23(3):131–141. https://doi.org/10.4013/ fem.2021.233.11 SPARK AR (2022 )Spark AR and frame studio policies. https://pt-br.facebook.com/legal/camera_ effects_platform_terms. Accessed 05/22/2022 Tiggemann M, Barbato I (2018) You look great!: the effect of viewing appearance-related instagram comments on women’s body image. Body Image 27:61–66. https://doi.org/10.1016/j. bodyim.2018.08.009 Tiggemann M, Hayden S, Brown Z, Veldhuis J (2018) The effect of Instagram likes on women’s social comparison and body dissatisfaction. Body Image 26:90–97. https://doi.org/10.1016/j. bodyim.2018.07.002 van Dijck J (2013) The culture of connectivity: a critical history of social media. In: Oxford University Press vol 9780199970. https://doi.org/10.1093/acprof:oso/9780199970773.001. 0001

Protecting Personal Data in Surveillance Society: A Designerly Approach to Privacy in Service Design Davide M. Parrilli1(B)

and Rodrigo Hernández-Ramírez2

1 Faculdade de Design, Tecnologia e Comunicação, Universidade Europeia, UNIDCOM/IADE,

Lisbon, Portugal [email protected] 2 Faculdade de Design, Tecnologia e Comunicação, Universidade Europeia, UNIDCOM/IADE, Lisbon, Portugal [email protected]

Abstract. Service design emerged in the last decades as the design practice that creates the conditions of interactions in the context of services. Contemporary business models involve interactions between users and the service and between end-users, where personal information is exchanged and processed. Hence, in a world labelable as ‘surveillance society’ or ‘surveillance capitalism’, the privacy and confidentiality of users’ information, image, and identity face serious threats. Building upon the human-centered design paradigm, service design is called to take an ethical stance to design services that do not jeopardize users’ privacy. First, however, service design should build its own comprehensive designerly understanding of privacy, based on input from other disciplines and on research, through design ethnography, to meaningfully tackle privacy challenges. Then, service designers should develop a designerly, project-based approach to protect informational privacy. Through tests and the solution to speculative and thought problems, service designers develop a service design thinking for privacy to design privacy-oriented services. We argue that a culture of designerly understanding of privacy shared across the service design community leads to a better service design. Keywords: Personal Information · Privacy · Service Design

1 Introduction: Service Design in a Fluid Surveillance World Design is constantly changing and growing, influenced by—and influencing at the same time—both human progress and technological advancements. Design is traditionally concerned with the creation of tangible artifacts such as objects (industrial design), clothes (fashion design), visual content (graphic design), and spaces (interior design and architecture) (Walker 2015). However, in the last decades, design expanded into the immaterial digital world (Hernández-Ramírez 2019). Redström and Wiltse grasp the complexity of modern digital tools—i.e., the everyday artefacts that combine a physical and a digital form—and call them “fluid assemblages” (2020). © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 438–448, 2024. https://doi.org/10.1007/978-3-031-47281-7_36

Protecting Personal Data in Surveillance Society

439

For instance, a smartphone is a physical object—a product of industrial design—and a system that allows users to interact with people and information digitally. It emerges that the user, and not the designed object, is the main concern for contemporary designers. In contrast, Archer, in his 1965 definition of design and the act of designing, mentions “the formulation of a prescription or model”, “the embodiment of the design as an artifact”, and “a creative step” (Archer 1984, p. 58), with no reference whatsoever to the user of that artifact. Designers like Archer and Alexander (1963) focused more on addressing the objective needs and problems of the users rather than seeing users as individuals who interact with designed objects for their own unique and personal (and often unpredictable) reasons. For 21st-century designers, ‘designing for people’ (Dreyfuss 2003) means being interested in shaping and improving the user and consumer experience with the designed object (Brown & Katz 2019). The role of the user experience (UX) designer has emerged as that of an expert in improving how users interact with and experience a designed object. UX design “encompasses all aspects of interacting with a product” (Blythe 2004, p. 41), including the design graphics, the interface, and the interaction with the user through touchpoints (Brown & Katz 2019). Although UX design has advantages, we argue that it has limitations because it primarily focuses on the user’s interaction with the designed object (whether digital or not), without considering the broader context in which the interaction occurs. Service design emerged as design practice at the end of the 20th century to fill this gap. It builds on the assumption that design would be increasingly concerned with creating systems for specific purposes rather than consumer goods (Meroni & Sangiorgi 2011). Service design is naturally holistic, for it is the design of the area where the service and the user interact (Pacenti 1998), while the focus of UX design is on the interaction between the user and a specific product or service. Since its beginning, service design has been connected to human-centered design research and methodology (Meroni & Sangiorgi 2011; Wetter Edman 2014). Humancentered design is a design approach and philosophy with a broad vision and action that considers the people involved in the relation between the designed object and the world, their desires, needs, and abilities (Norman 2013). Regardless of its limitations (Norman 2005), human-centered design should not be (completely) abandoned because it is still a powerful tool to affirm human dignity (Buchanan 2000) and is “an antidote to the cold dominance of technology and its inherent bias to replace or devaluate the contribution of people” (Brown & Katz 2019, p. 4). Interactions nowadays often involve the digital world, regardless of whether the service or object is physical or digital. Even when the interaction between the user and the provider happens entirely offline (e.g., when a customer buys a good in a brick-andmortar shop), an invisible part of that relationship often occurs in the digital domain. The collection and processing of consumers’ personal information is ubiquitous in the interactions between users and providers. The practical impossibility of concealing our information justifies the claim that we live in a surveillance society (Capurro 2005) or under surveillance capitalism (Doctorow 2020; Foster & McChesney 2014; Véliz 2020; Zuboff 2019). The critical question for service design is how to deal with it: should service designers passively accept surveillance

440

D. M. Parrilli and R. Hernández-Ramírez

society, including its most nefarious consequences, or take a stance against it in the name of users’ and stakeholders’ interests and rights? More worryingly, surveillance society is fueled not only by corporations and governments but by end-users themselves—and serious threats to privacy may come from other users (Fukuyama 2022). 1.1 Scope and Methodology This position paper addresses the research topic of how to build a designerly approach to privacy in service design. The methodology relies primarily on a multidisciplinary literature review to fill a gap in the design research about service design and informational privacy. When relevant, a case study from the authors’ recent research is used to sustain our findings. To our knowledge, no service design scholar has addressed the topic of privacy within and for service design. This paper answers the following research questions: 1) How can service design build a designerly understanding of informational privacy? 2) What methods can service designers use to establish a service design thinking for privacy? In Sect. 2, we position service design as a practice that involves interactions with personal information. Section 3 explores why privacy should be a service design concern. Section 4 defines the idea of designerly understanding of informational privacy for service design and discusses how designers could develop such understanding and its implications. We also identify the methods to achieve a designerly understanding of informational privacy. In particular, through design ethnography and other related qualitative research methods, service designers can observe how users react to privacy challenges and develop such designerly understanding. Section 5 shows that the designerly understanding of privacy should lead to a service design thinking for privacy, intended as a methodology to integrate privacy concerns and requirements into service design based on tests, prototypes, beta versions, and speculative and thought experiments. Finally, in the last part of the paper, we discuss the further potential developments of our research.

2 Service Design for Interactions The quality and functionality of a design product can be gauged by assessing its UX However, a consistent assessment of a product should not only consider its intrinsic quality, functionality and usability, and its overall UX. In today’s highly interconnected world, interactions are fundamental for any complex designed object, whether a product, service, process, brand, or organization (Buchanan 2005). Interactions happen between the user and the device and its interface, but also between the user and other users and between the users and the service provider. Consumers’ data are collected, processed, and possibly manipulated in all these interactions. Service design emerged as a field of study in the 1990s (Meroni & Sangiorgi 2011) and focuses on the design of such interactions. Interactions, however, cannot be designed: service design can only create the conditions for interactions to happen (Penin 2017). Consequently, designers have little or no control over the outcome of the interaction. In the real world, platforms created for sharing opinions and news between users have become toxic places where too many interactions involve hate speech, cyberbullying,

Protecting Personal Data in Surveillance Society

441

and the spread of disinformation. Some critics, however, argue that these platforms have been designed expressly for such noxious user engagement and interactions to take place (Monteiro 2019). Service design is the appropriate lens through which to address the topic of privacy protection in design, given its emphasis on interactions between end-users and between users and platforms. Personal information is often involved in service design practice. When service designers observe and understand user behaviors and experiences, personal data is collected and processed (Brown & Katz 2019; Meroni & Sangiorgi 2011; Tekes 2007). Service design products where users interact with a platform and other humans inescapably gather personal information, as we will discuss further in the next section. Service design has the necessary background to deal with privacy issues because it is inevitably ethical (Penin 2017). However, we argue that the ethical approach is ineffective without a service designerly understanding of privacy, intended as the tools to comprehend the issues at stakes from different perspectives and design an action plan that leads to solutions that work. Building on Cross’ contribution (2006), a designerly understanding of privacy is problem-solving—hence it is not just an epistemological understanding but is an understanding of action—and is human-centered—it is based on research of the issues at stake and of the stakeholders’ needs, threats, worries, expectations, and desires. It is not user-focused but stakeholder-focused because it encompasses all people affected by a service design solution. It is not reactive but proactive because it should also anticipate the possible malicious uses of the service design product by some stakeholders. It is holistic and multidisciplinary since it considers the corpus of knowledge of other disciplines as part of the designerly understanding of privacy.

3 Privacy as a Service Design Concern Service design products involve interactions between users—who are also data subjects whose personal information is collected and processed by the service provider— and digital touchpoints. Digital touchpoints collect data subjects’ information through interfaces. After data collection, the user loses control over the information and its diffusion. However, depending on the nature of personal information, sharing data with service providers and (potentially uncountable) third parties may significantly impact the data subject’s personal and social life—e.g., in health and justice-related service design scenarios. It emerges that informational privacy must be a chief concern for service designers. Design ethicists (Bowles 2018; Falbe et al. 2020; Ngai 2020) justify this claim on ethical grounds, although the conversation about ethics and privacy in the service design field is still in embryo. Convincingly, Value Sensitive Design (VSD) advocates that privacy is a value for design (Friedman et al. 2003; Friedman & Hendry 2019). However, it is still unclear on which grounds privacy should be a design concern—and not only a legal compliance issue, and why privacy is a value for design. To find answers, one needs to go beyond the field of design and explore other fields, such as philosophy, anthropology, social sciences, economics, and the law. Should service designers be experts in all these domains? To fulfill the intrinsic innovative mission of design (Zurlo 2019), it emerges that curiosity about other disciplines fuels innovation and creativity, but expecting a service

442

D. M. Parrilli and R. Hernández-Ramírez

designer to be a proficient philosopher, anthropologist, social scientist, economist, and lawyer is unreasonable. Cooperation with other disciplines, rather than isolationism and self-reliance, is critical to good design (Brown & Katz 2019). The mission of design is to define preferable futures working in collaboration with experts from other fields (Dunne & Raby 2013). Design, including service design, relies on other disciplines and areas of study to thrive. Service designers do not need to explain or defend the importance of the issues they work on, like protecting personal information, because it is already widely recognized as necessary. Additionally, protecting personal data against invasive and excessive processing activities by companies and governments is regulated by legislation in several countries. However, service designers should not just do what they are told by others in charge. They must develop their designerly, practice-based understanding of privacy. The information philosopher and ethicist Floridi claims that privacy deserves protection because it refers to an ontological essence of our being: personal information. Violating that information is an act of aggression against the data subject (Floridi 2014). Data ethicists of non-Western background show how privacy has ethical relevance in Asian cultures, despite their traditional collectivist approach to life and society (Ess 2020; Mizutani et al. 2004; Nakada & Tamura 2005; Yao-Huai 2005). Some scholars conclude privacy is a universal human value (Hongladarom 2007). These contributions should nurture the awareness of privacy inside the service design community and be used as a starting point to develop a service designerly understanding of privacy.

4 First Step: Building a Service Designerly Understanding of Privacy Though, the approach to be followed by service designers differs from those adopted by philosophers and ethicists. Service designers ultimately work with and for people— this explains the relevance of human-centered design, despite its limits. Because it is a practice-based activity, contemporary human-centered design involves observing and understanding human behaviors and needs before designing effective solutions (Norman 2011, 2013). We argue that observing facts and establishing corresponding behavioral patterns are fundamental steppingstones to building an understanding of informational privacy for service design. In the service design literature, design ethnography emerged as a methodology to understand the users of a design, working from the perspective of these users (Stickdorn & Schneider 2011). Design ethnography is qualitative ethnographic research within a design context. Despite its potential value, design ethnography has not yet been employed in the context of privacy, either in research or practical applications. Because design ethnography aims to “make sure that the foundations for the designs are well understood and accepted” (Stickdorn & Schneider 2011, p. 93), it appears to be the proper methodology to build a service designerly understanding of privacy across multiple contexts. Design ethnography includes ethnographic visual observation, which can provide unbiased information to the researcher (Candy 2006) and, consequently, is a valuable

Protecting Personal Data in Surveillance Society

443

tool to achieve a general understanding of privacy for service design. Recently, we tested ethnographic visual observation in research about a prototype of a privacy-enhancing instant messaging app. We selected a small number of end-user participants, who were asked to simulate interactions involving the exchange of personal information in pairs and to adjust the privacy settings accordingly in the app interface. The research team recorded the workshop and the interactions between users to analyze how real people react when confronted with privacy challenges. Such assessment—which will be the object of a future ad-hoc publication—has been valuable to validate and improve the design features of the prototype. However, it emerged from our research that ethnographic observation has limitations. In particular, we argue that ethnographic observation—a research method that is essentially qualitative—should be supported by quantitative research to achieve a proper understanding of privacy. For instance, assessing how many users installed privacyprotecting apps on their devices or refused non-essential cookies when accessing websites is helpful to complement the findings reached through a qualitative assessment of how people behave regarding their informational privacy in a specific context. Building on these considerations, we will further test design ethnography in ongoing research about service design touchpoints, privacy, and UX to observe how users regulate the privacy settings of an interface based on the nature of the information and the relationship they have with the information recipients. Therefore, we aim to learn if and how people value their privacy in an extended set of specific contexts. We expect to definitely validate our finding that design ethnography can provide a comprehensive understanding of privacy issues, extending our research with the involvement of more end-user participants and designers. Nevertheless, service designers must consider that design ethnography may be an appropriate tool to investigate the menaces perceived by users, but not necessarily the real ones. For instance, the fact that many corporate consultants and executives—including Mark Zuckerberg—cover the camera of their laptops with a tape reveals a diffused worry about being spied on, but it does not prove the frequency of remote camera monitoring. When using design ethnography, it is vital to consider the unique characteristics, culture, and values of each place and also to look at its cultural output, such as books, magazines, newspapers, films, and interviews. Further, design ethnography—and ethnographic research in general—shall not be ethically neutral: when users share their personal information recklessly because of their age or lack of privacy awareness, service designers should not jump to the conclusion that no privacy issues exist in that context. On the contrary, observing how users misbehave regarding privacy can strengthen the service designer’s understanding of privacy and reinforce the importance of creating service design solutions that prioritize privacy and security. Users’ careless attitude toward their informational privacy takes place in the context of interactions with other consumers or with the service provider. Service designers work to make interactions possible and are never morally neutral. Service design creative process involves forecasting the possible consequences of those interactions and their acceptance. Penin claims that: “The main role of service designers is therefore being able to define what are the interactions contained in a service; what needs to happen

444

D. M. Parrilli and R. Hernández-Ramírez

in these interactions; what they enable; and how a digital platform, system, or interface behaves with a user.” (2017, pos. 3812). These choices have ethical implications. Therefore, service designers are called to act morally and to make ethical decisions based on their general designerly understanding of privacy (Bowles 2018; Falbe et al. 2020; Monteiro 2019; Papanek 2019). However, this is not necessarily a straightforward process. A design choice can be ethical because it protects the users’ informational privacy, but it can be unethical because it hinders other values or users’ rights. For instance, if a digital platform does not allow users to take screenshots of messages and media, it will make it harder for victims of online harassment to provide evidence and report the harassment to the police. Additionally, a platform that bans the private sharing of intimate photos and videos on ethical grounds would unreasonably limit the users’ personal freedom and agency.

5 Second Step: Developing a Service Design Thinking for Privacy Ethical dilemmas can only be solved by considering the specificities of every service design project. When designing a product or solution, service designers should build a project-specific designerly action plan for privacy since design is pragmatic (Dorst 2019b). In complex situations, solutions are best achieved through design thinking methods. We build on Brown’s definition of design thinking to sustain service design thinking for privacy: “The mission of design thinking is to translate observations into insights and insights into products and services that will improve lives.” (2019, p. 49) The purpose of service design thinking for privacy is to translate the understanding of privacy into services and solutions that will improve lives. Service designers must apply the knowledge and insights that form their designerly understanding into project-based action plans to create privacy-enhancing products through an effective problem-centered methodology. The solution to problems is a key concern in design thinking. Brown claims that the first step of any human-centered design process is to comprehend the range of the problem to solve (Brown & Katz 2019). According to Dorst the designer is called to find a design solution to a design problem in a design context through a design process (Dorst 2019a). This prescription works efficiently when the designer is confronted with an actual identifiable problem, but it has loopholes when a service designer creates a service with countless and unpredictable interactions but not with a single specific problem to solve. The problem-solution paradigm is troublesome also when the issue to solve is what Rittel and Webber call a “wicked problem”—informational privacy is an example of a wicked problem because solutions to privacy problems are not true-orfalse, but good-or-bad with different conclusions depending on the chosen point of view (1973). However, the fact that service designers do not necessarily have an immediate problem to solve or that the issue will not lead to a definite solution does not mean that the problem-solution approach should be abandoned. An effective methodology of service design thinking for privacy relies on the immersion into both real and imaginary design problems to understand how to improve the quality of the designed products. When researching service design solutions, designers typically use a range of tools and techniques to immerse themselves in the context of the problem. Tests, prototypes,

Protecting Personal Data in Surveillance Society

445

and beta versions help service designers to observe the potential consequences of their work in the real world (Stickdorn & Schneider 2011). In our research briefly discussed in the previous section, we used a prototype of an instant messaging app to assess how it could potentially work in reality when users are confronted with pressing privacy challenges. These tools are built—or refined and adjusted, like we did in our investigation—based on the service designer’s understanding of privacy—therefore, they are, at least at an abstract level, privacy-protecting and privacy-enhancing. They allow to refine the designer’s understanding and, more importantly, to improve the designed solution in an iterative design process (Gould & Lewis 1983). Tests, prototypes, and beta versions, commonly used to test the security of digital solutions, reveal how users behave regarding their information: are they willing to provide the requested personal data? Do they consciously or recklessly share sensitive information with the service and other users? Therefore, testing, prototyping, and making beta versions of the service design solutions should be the privileged approach to follow whenever it is possible or feasible. However, these tools are fundamental but may have loopholes. In particular, users can act consciously regarding their personal information because they know they are interacting with a prototype; or, in the case of A/B testing, the results can be biased due to the limited number of participating users or other factors. Through imagination and abstraction, service designers should solve fictional, speculative problems that users and other stakeholders may encounter when the designed solutions are implemented. Designers cannot prevent users from doing what they want with the designed solutions (Brown & Katz 2019), unless they rely on invasive surveillance tools to monitor that users adhere to the intended use of the product—which would be frankly unacceptable. However, the freedom granted to users does not exonerate service designers from considering the potential misuse of their designed solutions. Even if no problems are discovered during testing, when designing for interactions between users, designers should imagine potential problems and devise solutions to prevent or fix them. Thought experiments can help to facilitate understanding and promote critical thinking and have been successfully applied in the field of ethics to evaluate alternatives and scenarios (Ihde 1990; Kovács 2021). In our research where the privacy features of a privacy-enhancing instant messaging app have been tested, real-user participants were confronted with thought scenarios—ranging from the request to share intimate pictures with a virtual date to the need to sending sensitive information to a doctor and real estate agent—and asked to interact and adjust the privacy settings of the app for each thought case. Speculation helps to improve the quality of the service being created or to abandon the project altogether if the potential consequences are negative. For instance, service designers should wonder whether a communication platform can become a tool for harassment and persecution; or whether a personal data-driven service design solution favors social integration or exclusion. Thought experiments can be generated and discussed in brainstorming sessions (Stickdorn & Schneider 2011). Brainstorming, a traditional design thinking method, fosters imaging probable, possible, and improbable privacy threats of the service. After an expansive phase, a synthesis and mapping of risks are necessary to focus attention and energies on threats that are more acute, frequent, or difficult to mitigate. Other valuable techniques that could meaningfully applied in the

446

D. M. Parrilli and R. Hernández-Ramírez

field of privacy and service design are the future workshop method (Jungk & Müllert 1996) and design fiction (Bleecker et al. 2022). Ultimately, the application of service design thinking for privacy and its insights prompts service designers to propose privacy-oriented solutions, make better products, and improve lives.

6 Conclusions: The Role of Design Education Service design must aspire to create solutions that are much more than just usable, engaging, and aesthetically appealing. Service designers are called to design solutions that enhance the well-being and integrity of consumers and communities. Informational privacy is one of the most pressing issues of our time. In this paper, we assessed on which grounds privacy is a concern for service design, and we set the grounds to build a service designerly understanding of privacy. Although these considerations reflect common sense, perhaps surprisingly service design literature has been seldom concerned about informational privacy. Through this paper, we firmly put privacy in the service design map. We also identified a fundamental conceptual and practical tool for service designers confronted with (potential) privacy issues: the designerly understanding of privacy for service design. In particular, we commented on one specific method for building an understanding of privacy for service design—design ethnography—and we established an overview of potential methods of service design thinking for privacy. Our research reveals that design ethnography is suitable to comprehend the privacy issues at stake in a service design project and to design—and further validate and improve—privacy-oriented products and solutions. Again, our research is pioneering because, to our best knowledge, design ethnography has never been used in the field of privacy and service design. Some of the most advanced design companies know that protecting users’ information is equivalent to securing and expanding their business. Apple’s customers accept to pay a premium price to ensure that their personal data will be treated with respect. However, privacy should not be a luxury. It is a fundamental right, and all design solutions should be at least privacy-protecting. Dark patterns, tracking pixels, and similar design tricks and technologies aimed at stealing consumers’ personal data go exactly in the opposite direction. Nevertheless, being privacy-protecting is not enough. Service design solutions must be privacy-enhancing to guarantee that they go beyond users’ expectations and that they can protect consumers even when they do not feel that their data is in danger. Better products result from better designers who deeply understand privacy issues, both at the abstract and project-specific levels. This can be achieved through better design education that teaches future designers to know and interpret the world, work in close cooperation with scientists and experts from other disciplines, and understand that making good products results from a holistic and challenging mindset. The discussion of how design education can achieve these targets goes beyond this contribution’s scope: we plan to explore this topic through future research.

Protecting Personal Data in Surveillance Society

447

References Alexander C (1963) The determination of components for an Indian village. In: Conference on design methods. Pergamon Press Archer LB (1984) Systematic method for designers. In: Cross N (ed.) Developments in design methodology. John Wiley & Sons Ltd. pp 68–82 Bleecker J, Foster N, Girardin F, Nova N, Frey C, Pittmann P (2022). The manual of design fiction. near future laboratory Blythe, MA. (Ed.) (2004) Funology: from usability to enjoyment. Kluwer Academic Publishers Bowles C (2018). Future ethics. NowNext Press Brown T, Katz B (2019) Change by design: how design thinking transforms organizations and inspires innovation (Revised and updated edition). HarperBusiness, an imprint of HarperCollinsPublishers Buchanan R (2005) Design ethics. In: Mitcham C (ed.), Encyclopedia of science, technology, and ethics. Thomson Gale, vol 2, pp 504–510 Buchanan R (2000) Human dignity and human rights: toward a human-centered framework for design. Reshaping South Africa by design, cape town Candy L (2006) Practice based research: a guide. Creativity Cogn. Studios Rep 1(2):1–19 Capurro R (2005) Privacy. an intercultural perspective. Ethics Inf Technol 7:37–47. https://doi. org/10.1007/s10676-005-4407-4 Cross N (2006) Designerly ways of knowing. Springer Doctorow C (2020). How to destroy surveillance capitalism (1st edition). Medium Editions Dorst K (2019a). Design beyond design. She Ji J Des Econ Innov 5(2):117–127. https://doi.org/ 10.1016/j.sheji.2019.05.001 Dorst K (2019b) What design can’t do. She Ji J Des Econ Innov 5(4):357–359. https://doi.org/10. 1016/j.sheji.2019.11.004 Dreyfuss H (2003). Designing for people. Allworth Press Dunne A, Raby F (2013) Speculative everything: design, fiction, and social dreaming. The MIT Press Ess C (2020) Digital media ethics (Third edition). Polity Falbe T, Andersen K, Frederiksen MM (2020) Ethical design handbook. Smashing Media AG Floridi L (2014) The 4th revolution: how the infosphere is reshaping human reality (First edition). Oxford University Press Foster JB, McChesney RW (2014) Surveillance capitalism: monopoly-finance capital, the militaryindustrial complex, and the digital age. Monthly Rev 66(3). https://monthlyreview.org/2014/ 07/01/surveillance-capitalism/ Friedman B, Hendry D (2019) Value sensitive design: shaping technology with moral imagination. The MIT Press Friedman B, Kahn P, Borning A (2003) Value sensitive design: theory and methods Fukuyama F (2022) Liberalism and its discontents (First, American. Farrar, Straus and Giroux Gould JD, Lewis C (1983) Designing for usability—Key principles and what designers think. In: Proceedings of the SIGCHI conference on human factors in computing systems - CHI 1983, pp 50–53. https://doi.org/10.1145/800045.801579 Hernández-Ramírez R (2019) On the origins and basic aspects of user-centered design and user experience. In: Ayanoglu H, Duarte E (eds) Emotional Design in Human-Robot Interaction. Human–Computer Interaction Series, pp 71–92. Springer. https://doi.org/10.1007/978-3-31996722-6_5 Hongladarom S (2007) Analysis and justification of privacy from a buddhist perspective Rittel HW, Webber MM (1973) Dilemmas in a general theory of planning. Policy Sci4(2):155–169. JSTOR

448

D. M. Parrilli and R. Hernández-Ramírez

Ihde D (1990) Technology and the lifeworld: from garden to earth. Indiana University Press Jungk R, Müllert N (1996) Future workshops: how to create desirable futures. Institute for Social Inventions Kovács G (2021) Thought experiments in ethics. Pécsi Püspöki Hittudományi F˝oiskola Meroni A, Sangiorgi D (2011) Design for services. Gower Mizutani M, Dorsey J, Moor JH (2004) The internet and Japanese conception of privacy. Ethics Inf Technol 6(2):121–128. https://doi.org/10.1023/B:ETIN.0000047479.12986.42 Monteiro M (2019) Ruined by design: how designers destroyed the world, and what we can do to fix it Nakada M, Tamura T (2005) Japanese conceptions of privacy: an intercultural perspective. Ethics Inf Technol 7(1):27–36. https://doi.org/10.1007/s10676-005-0453-1 Ngai J (2020) Good design: principles, ethics and how to approach the design process Norman DA (2005) Human-centered design considered harmful. Interactions 12(4):14–19. https:// doi.org/10.1145/1070960.1070976 Norman DA (2011) Living with complexity. MIT Press Norman DA (2013) The design of everyday things (revised and expanded edition). Basic Books Pacenti E (1998) Il progetto dell’interazione nei servizi. Un contributo al tema della progettazione dei servizi [PhD]. Politecnico di Milano Papanek V (2019) Design for the real world. Thames & Hudson Penin L (2017) An introduction to service design: designing the invisible. Bloomsbury Publishing Redström J, Wiltse H (2020) Changing things: the future of objects in a digital world (Paperback edition). Bloomsbury visual arts Stickdorn M, Schneider J (2011) This is service design thinking: basics, tools, cases. Wiley Tekes (2007) Seizing the white space innovative service concepts in the United States Peer Insight [LLC. Tekes Véliz C (2020) Privacy is power: reclaiming democracy in the digital age. Bantam Press Walker JA (2015) Design history and the history of design. Pluto Press. https://doi.org/10.2307/j. ctt18mvngc Wetter Edman K (2014) Design for service: a framework for articulating designers’ contribution as interpreter of users’ experience. School of design and crafts, faculty of fine, applied and performing arts, university of Gothenburg Karlstads Universitet Yao-Huai L (2005) Privacy and data privacy issues in contemporary China. Ethics Inf Technol 7(1):7–15. https://doi.org/10.1007/s10676-005-0456-y Zuboff S (2019) The age of surveillance capitalism: the fight for a human future at the new frontier of power (First edition). PublicAffairs Zurlo F (2019) Designerly way of organizing. Il design dell’organizzazione creativa|designerly way of organizing. the design of creative organization. Agathon 5:11–20. https://doi.org/10. 19229/2464-9309/522019

Improving Software Design Teaching, a Comparative Study Verónica Silva1(B) , Rodrigo Hernández-Ramírez2

, and Rodrigo Morais2

1 IADE – Faculty of Design, Technology and Communication, Universidade Europeia, Avenida

D. Carlos I, 4, 1200-649 Lisbon, Portugal [email protected] 2 UNIDCOM/IADE, Avenida D. Carlos I, 4, 1200-649 Lisbon, Portugal {rodrigo.ramirez,rodrigo.morais}@universidadeeuropeia.pt

Abstract. With the constant development of software technologies, it has become increasingly difficult for universities to design curricula that provide students with technical skills that stay relevant by the time they join the workforce; this is particularly true for design. This paper proposes an approach for training students in design software. We aim to understand whether alternative methods for teaching Adobe Photoshop, Illustrator, and InDesign to undergraduate students could effectively replace traditional approaches. We conducted a comparative study with first-semester students: two groups underwent training following traditional methods (lectures followed by in-class exercises), whereas two groups learned using the eduScrum methodology. For the latter approach, students were given 15 assignments to be developed during the semester, with complete freedom on where and how to research the necessary knowledge to achieve the required results. During this process, the instructor acted as a facilitator rather than an instructor, as the Scrum methodology advocates. As we foresaw, some of the tools studied during the semester were surpassed by software updates Adobe introduced, thus adding to our initial argument for developing the methodology in the first place. While we firmly believe that the eduScrum methodology can become a successful approach for teaching design software, further studies are necessary to investigate whether a hybrid methodology might be a better solution to help students transition to a more independent way of learning. Keywords: Design · Design education · Design Software · Alternative Pedagogical Methods · eduScrum

1 Introduction Due to the constant upgrades to software technologies, it has become a challenge for universities to design curricula that ensure students are equipped with all the skills they need by the time they join the workforce. “Formal education needs new paradigms to fit into the new millennium requirements. Schools and especially universities face challenges” (Tomozii et al. 2014). Present-day technology has radically changed how we learn and interact with the world and will likely continue to do so, causing a natural evolution in the way Universities teach. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 449–458, 2024. https://doi.org/10.1007/978-3-031-47281-7_37

450

V. Silva et al.

It is unerring that software has gone through various updating cycles by the time students graduate. At the same time, “courses need to be examined so that students are better prepared to cope with the challenges of a rapidly changing software industry” (Morrogh 2000). Alternative teaching methods have to be implemented so students can learn to learn instead of memorising theoretical concepts and replicating practical exercises. In that sense, we believe the eduScrum methodology could be a successful means to achieve that. This paper is part of an ongoing research carried out with students from an undergraduate design course in Portugal that covers Adobe Photoshop, Illustrator, and InDesign. In the second section, we introduce the Scrum methodology, while section three briefly explains the main characteristics of eduScrum. Section four summarises how we applied the eduScrum method during class and the resulting data. Section five analyses and explains our findings, and sections six and seven contain our conclusions and future work.

2 The Emergence of the Scrum Methodology and eduSCRUM Agile methodologies have been used worldwide to develop software faster, and while the origins can be traced back to the “XP programming method” from the late 90s (Beck 1999a, Beck 2000, as cited in Abrahamsson et al. 2017), the 2001 Agile Software Development Manifesto represents the clear starting point of the so-called Agile movement. This manifesto makes four central claims: (1) Individuals and interactions are more important than processes and tools; (2) A working software is more important than extensive documentation; (3) We should aim for customer collaboration instead of contract negotiation; (4) Instead of following a set plan, the team should react to change (Beck et al. 2001, Cockburn 1998, as cited in Abrahamsson et al. 2017). Hence, an agile methodology aims to accelerate the software development process. The Scrum methodology is an agile framework where individuals can effectively tackle intricate challenges and generate valuable products with exceptional creativity and productivity; it was first presented at the 1995 Object-oriented Programming, Systems, Languages, and Applications (OOPSLA) Conference (Schwaber et al. 2017). Scrum has noticeable advantages compared to other methodologies, and it is arguably the dominant approach in the software industry, finance, and other research-oriented fields (Srivastava et al. 2017). Nonetheless, our research will not focus on Scrum but on the “eduScrum” approach. EduScrum is a byproduct of the Scrum approach that follows the same principles and framework. According to the people behind it, eduScrum is about “ownership,” that is, giving young people “the space to determine their own learning process” within “the set frameworks and learning objectives” (eduScrum team 2020). We believe eduScrum is a suitable approach to teaching design software because the methodology’s focus on student autonomy and the “work environment” it fosters in the classroom mimics the environment students will experience in design agencies after graduation.

Improving Software Design Teaching

451

3 EduSCRUM Attributes Since the eduScrum methodology derives from the Scrum methodology, it shares some basic pillars and values. For instance, they are both built on transparency, investigation, and adaptation. In that sense, it is imperative that students have access to all information regarding the learning process, that they do research and continuous internal reviews, and that both students and the teachers can modify the plan to ensure the learning process will continue without deviations. Similar to Scrum, this alternative teaching methodology encourages participants to decide how the projects will be performed, and in this case, the Professor is responsible for determining the why and the what, i.e., the learning outcomes and class program. Finally, the teams need access to a board where they can see what projects or steps need to be done, which are being done at the moment, and what work has been considered complete (eduScrum team 2020, Wijnands et al. 2018). In the next section, we further explain how the methodology was implemented in class. Evidence shows that more practice exercises and shorter periods that divide the subject to be learned are more effective than purely theoretical lectures with fewer periods throughout the semester. Software teaching, specifically, should be undertaken with an agile procedure like eduScrum or similar to promote experimentation, practice, and self-learning in all students.

4 Methodology: How We Applied EduSCRUM The comparative study described in this paper took place in a Design Software class at a University in Portugal that will remain anonymous for this phase of the submission. In this module, part of the Design undergraduate program, students learn Adobe Software, specifically Photoshop, Illustrator, and InDesign. During the 2021 Fall semester, two classes of about 30 people each underwent a traditional teaching method consisting of lectures followed by exercises accompanied by step-by-step instructions from the instructor. The following year, during the 2022 Fall semester, two classes underwent the eduScrum system to learn the same software. The students in the classes that followed the eduScrum methodology were asked to form teams of three to four members to fulfil the agile methodologies requirements and work together at the computer labs in school. The instructor made sure that each team had (1) at least one member fluid in English (given that most material for learning software online is in English, but also because the instructor’s first language is not Portuguese); (2) a member who had previous experience with the software (so they could act as tutors and team captains); and finally (3) at least one member who required more learning support. Before each lesson began, the instructor created a Notion board (Fig. 1) containing all the 15 exercises to be developed during the semester. As seen in Fig. 2, each exercise card contained all the information necessary for the students to develop the task, including the title of each task, instructions on how to carry them out, the type of files to be delivered, examples of the finished exercises, and links to online tutorials as a guide for their research. All the projects were due at the end of the semester, so students had the freedom to work at their own pace and in the order they preferred.

452

V. Silva et al.

Fig. 1. Notion plan that contains all the projects to be completed during the semester.

Fig. 2. Example of instructions for one project, with deliverables and all the information needed.

At the start of each lesson, the instructor would speak with every team and ask about the status of the exercises. If they had any issues, the goal was to eliminate any obstacles jeopardising the team’s productivity. The instructor constantly monitored the students during the lesson and provided help when needed. At the end of each class, students were asked to write an entry within the team’s Notion page describing what went well during that session, what needed improvement, and what they would do to improve things. This way, the teams would “close” the sprint before the next class.

Improving Software Design Teaching

453

By the end of the 2022 Fall semester, an online survey was sent to all the students from the four classes. We obtained 32 answers from the 2021 students and 47 from the 2022 students. The first part of the survey consisted of preliminary questions regarding demographics and knowledge of the software. The second section was designed to learn more about the experience they had during lessons. Furthermore, they were asked about the things they learned in addition to the curricular unit’s syllabus, whether they had received help from their colleagues, how motivated they were during the lessons, and the level of support from the instructor. The questions are listed in Table 1, and the results of these surveys and further analysis are summarised in section five. Table 1. Questions from the survey sent to the students.

454

V. Silva et al.

4.1 Hypothesis We presume that: • H1: eduScrum might be a feasible alternative to teach students how to solve problems by themselves instead of following step-by-step instructions from the instructor. • H2: Students might feel empowered and motivated to learn on their own terms. • H3: Anyone can replicate this experiment and use the eduScrum methodology in the classroom, as long as the instructor is capacitated and has support from the school.

5 Findings Although the groups selected to perform this comparative study were from the same year and program (second semester of the first year for the Design program), we noticed some key differences. As we can observe in Table 2 below, the students from the 2021 semester had a broader age range (19 to 58 years) than the group of 2022, who ranged from 18 to 26 years old. To be more specific, the students from the 2021 semester were part of the night classes targeted at people who already work; therefore, we had a 58-year-old student, one who was 50 and another who was 38. For that reason, the age range differed so much from one scholar year to the other. This age difference between the groups directly impacted their previous experience with the software, as we can appreciate in Table 3. Nonetheless, the average (mode) age was roughly the same, and the older students were the exception to the rule. Additionally, most students live in the Lisbon council. Consequently, the sample could be considered somewhat similar, with a few exceptions of older students with previous knowledge of Adobe software. Table 2. Basic demographics of the students.

According to the survey sent to the students from the 2021 Fall Semester, 72% felt they had support from the instructor (Table 4), and about half of them reported that they did not ask for support from colleagues, while the other half did. Various factors influenced that behaviour, including the fact that some students had previous experience; hence, they were implicitly seen as “tutors” by their less experienced colleagues. Some students also reported that they asked for extra help not because the instructor did not explain the subject but because they were distracted, did not want the instructor to repeat the class, or just wanted to reinforce what they had already learned.

Improving Software Design Teaching

455

Table 3. Previous Adobe Software Experience from the students.

For the 2022 Fall class, the eduScrum process was explained to the students during the first class; they were very curious and asked many questions. Most of the students were very excited to “be their own boss” and learn at their own pace, but some were hesitant, and they noted they were unsure they could learn anything and do the exercises without help from the instructor. Everyone was told the instructor would be there during the lesson, providing support when needed. Table 4. Level of Motivation during the semester & support received from the instructor.

During the first weeks of the semester, all the students were working quickly. However, about a quarter of the students asked the instructor how to do the exercise step by

456

V. Silva et al.

step and got upset when they were told they needed to do some research by themselves. Additionally, when the instructor would go around the classroom asking how things were going, the same students would reply apathetically, saying, “I’m not learning anything because nobody is teaching me anything,” “I don’t know how to do anything because I never used the software before.” On top of that, the same students who were reluctant to work with this methodology refused to do any research or work outside of the classroom, even though they were falling behind with the exercises. By mid-semester, about 50% of the groups continued steady progress. The other groups needed to catch up, and the instructor had to get into the Project Manager role to give them homework and try to bring them up to the pace of the rest of the groups. Another issue was that some students, for personal reasons, directly affected the group’s performance. In a standard setup, the students who miss classes harm only their progress; in this case, they also affect their teammates. During the semester and in the survey results, the students working with the eduScrum methodology gave positive and negative feedback. One student, who had zero experience with the software before coming to class, mentioned that “the instructor should change the teaching methodology and not ask students to do any kind of research.” Realistically, this request is untenable, as any learning methodology should encourage students to research, not only because it “enables students to develop independent critical thinking skills” (Petrella et al. 2008) but also because it “allows students to develop professionally and personally in ways not possible through traditional lecture and laboratory courses” (Padmaja et al. 2015). Other students protested similarly, claiming that “nobody was teaching them anything.” However, in every case, these students had zero experience with the software and felt the support from their experienced classmates was insufficient. According to the general feedback and the information we have, we may conclude that the students who felt unmotivated were the same that did not agree with the instructor requesting them to do research for the class. Nonetheless, there was considerable positive feedback regarding the methodology. During the semester, some students who first resisted “learning by themselves” got excited and encouraged when they discovered how to perform the exercises. These small wins seemed to motivate them even further to continue working. On the other hand, the students who already had experience working with the software realised that some tools they had learned before were obsolete, so learning how to research and find new tools and methods is important to be up to date. Besides, some realised they knew the software better than they thought when helping their classmates. Overall, students shared that besides the curricular unit’s contents, they also learned to work in a team, be more autonomous and creative, and plan projects more effectively. All these skills are fundamental in any workspace. As Ramsier et al. explain, “what is needed in undergraduate education is a learning experience that requires teamwork and more closely simulates what students will experience in the industry” (Ramsier et al. 2003). The above is valid for the design industry, and most work environments students will encounter when they graduate. Finally, 72% of the students from the 2021 fall semester felt like the support from the instructor was “high,” compared to 55% of the participants from the 2022 fall semester who said that the support was “normal.” We consider this decrease normal because the

Improving Software Design Teaching

457

eduScrum system requires the Professor to be a facilitator rather than a provider of learning material. It is crucial to underline that we are aware that to obtain enough data to make a solid comparison in the change of behaviour in undergraduate students, we would need to undertake this study for several years. Nonetheless, this is the emergence of the study, and continuous experimentation is being performed to this day, with plans to publish the research in the near future.

6 Implications An alternative teaching methodology for an undergraduate Design course may be needed, especially for fast-changing areas, such as software instruction. The students must learn to learn instead of memorising material and following instructions; otherwise, they will lack fundamental skills when joining the workforce. EduScrum, or a similar approach, might be a feasible solution for the ever-changing design market of these times. As explained in this paper and the work of various authors, the benefits outweigh the difficulties we might encounter. For example, Müller-Amthor et al. believe that while following the eduScrum methodology, “students figured out their own learning and team working obstacles by using the solution-focused coaching.” (Müller-Amthor et al. 2020). Similarly, Kuz states that “the extrapolation of Scrum to education pursues a purely practical and experiential learning, to achieve the full development of its autonomy, abilities, and skills.” (Kuz 2021). Additionally, most students had a positive experience when working with a non-standard learning methodology, and if applied gradually, we might obtain better results. Just as with any other teaching methodology, eduScrum requires preparation from the instructor in order to obtain better results and achieve a positive learning experience. As shared in the surveys, students were unsure that the method would work mainly because most students are used to learning with traditional methods; therefore, they might be hesitant about trying a novel one. Perhaps doing an introductory course on the method or performing a hybrid approach might be useful to prepare the students better to work with eduScrum. It is also important to communicate the benefits to them and give them enough creative freedom so they can be encouraged to do the work throughout the course. As usual, limitations and obstacles will be faced, but the instructor must find the best solution. Finally, instructors need to have enough support from the school to perform this kind of practice and trust a process that has been proven to be successful in various schools and courses.

7 Conclusions and Future Work Technology evolves rapidly nowadays, causing updates and changes in Design software. Therefore, we must find a more suitable method for teaching software at university level right before students join the workforce. As described in this paper, eduScrum might be a feasible alternative to ensure students learn how to work in teams and solve problems by themselves instead of following step-by-step instructions from the instructor. If students comply with this learning method, they might acquire skills that would help them stay updated with the latest software updates throughout their careers.

458

V. Silva et al.

Contrary to what we were expecting, the most surprising obstacle we encountered was the mentality of some students, who refused to investigate and learn by themselves, claiming the instructor’s role was to give all the pieces of information so they would solely effortlessly grab everything. Before performing the experiment, we were hopeful that students would feel challenged and motivated to work on the projects because they were free to learn however they thought it was best for them. The eduScrum methodology may be applied to any course or subject as long as the instructor understands the benefits of a different teaching approach and the challenges and potential problems encountered. As for this study, further experimentation and a hybrid methodology will be tested to get the students acclimated to the eduScrum method and obtain the best results possible.

References Abrahamsson P, Salo O, Ronkainen J, Warsta J (2017) Agile Software Development Methods: Review and Analysis. arXiv:1709.08439. https://doi.org/10.48550/arXiv.1709.08439 Beck K (1999) Embracing change with extreme programming. IEEE Comput 32(10):70–77 Beck K (2000) Extreme Programming Explained: Embrace Change. Addison-Wesley Professional Beck K, et al (2001) Manifesto for Agile Software Development 2002(22.3.2002) http://AgileM anifesto.org Cockburn A (1998). Surviving object-oriented projects: a manager’s guide, Addison Wesley Longman eduScrum team, The eduScrum Guide (2020) https://slabstatic.com/prod/uploads/oui3ndfr/posts/ attachments/7FUaIX0xUgIjc_Wyfudi0Vq9 Kuz A (2021) Scrum: a new framework applied to education. In: Eduweb vol 15, no 3, pp 10–17. Editorial Primmate. https://doi.org/10.46502/issn.1856-7576/2021.15.03 Morrogh P (2000). Is software education narrow-minded? In: Proceedings of the 22nd international conference on software engineering - ICSE 2000. The 22nd international conference. ACM Press. https://doi.org/10.1145/337180.337451 Müller-Amthor M, Hagel G, Gensheimer M, Huber F (2020) Scrum higher education – the scrum master supports as solution-focused coach In: 2020 IEEE global engineering education conference (EDUCON), Porto, Portugal, pp 948–952. https://doi.org/10.1109/EDUCON45650. 2020.9125304 Padmaja A, Laxmi Ramana VSV, Reddy PR (2015) Importance of research at undergraduate level. In: Natarajan R, (Ed.), Proceedings of the international conference on transformations in engineering education, pp 631–632. Springer India Petrella JK, Jung AP (2008) Undergraduate research: importance, benefits, and challenges. Int J Exerc Sci 1:91–95 Ramsier R, Cheung HM, Evans E, Broadway F, Qammar H. (n.d.) (2003) Focusing on teamwork vs. technical skills in the evaluation of an integrated design project. In: 2003 annual conference proceedings. annual conference. ASEE conferences. https://doi.org/10.18260/1-2--11633 Schwaber K, Sutherland J (2017) The Scrum Guide, pp 3–5. https://www.scrumguides.org/docs/ scrumguide/v2017/2017-Scrum-Guide-US.pdf Srivastava A, Bhardwaj S, Saraswat S (2017) SCRUM model for agile methodology. In: 2017 International conference on computing, communication and automation (ICCCA). IEEE. https:// doi.org/10.1109/ccaa.2017.8229928 Tomozii SE, Topal˘a I (2014) Why do we need to change the educational paradigms? Procedia Soc Behav Sci 142:586–591. https://doi.org/10.1016/j.sbspro.2014.07.670 Wijnands WJA, Stolze A (2018) Transforming education with eduscrum. agile and lean concepts for teaching and learning

Oficina de Tipo: Comparative Study of Wood Type Production Methods (CNC and Pantograph) Ana Manique(B)

and Aprígio Morgado

Research Laboratory in Design and Arts, School of Fine Arts and Design ESAD.CR, Polytechnic of Leiria, Caldas da Rainha, Portugal {ana.manique,aprigio.morgado}@ipleiria.pt

Abstract. This article documents the research work developed by the “Oficina de Tipo”, an academic project aimed at producing wood types to equip a letterpress workshop at the School of Fine Arts and Design (ESAD.CR), contributing to the development of experimental practices, new pedagogical approaches, and to establish an academic and professional network for the study of typography. The research aims to study, recover, and adapt old technologies of movable wood type production for the reproduction of display typefaces created at the academy or identified in private collections to study, preserve, and print with them. This is done through analogical (pantograph) and digital (CNC) means. Based on a literature review of industrial technologies to produce movable wood types, and on expert inquiries, were analysed and reproduced two grotesque uppercase types. The results of the pantograph reproduction showed significant differences in the use of pine, mahogany and ash wood. Mahogany offered better resistance to milling, requiring less finishing and presenting a lower risk of fracture due to its greater resistance to the pressure of the press. In handling the equipment, was observed that letters with concave angles maintain the integral design of the type, while the convex angles are rounded by the cutter. Therefore, it is necessary to adjust the drill bits to letters with convex angles of reduced amplitude. The results of the production of characters by CNC presented a uniform cut and recess. At sharper vertices, some letters show a slight curvature compared to letters with concave angles and open counters. Regarding the formal differences between types produced by CNC and pantograph, it is possible to state that the types produced by CNC present a more regular contour and, therefore, require less post-finishing. Keywords: Typography · Letterpress · Wood Type

1 Introduction The manufacture of the first wooden typefaces and subsequent printing originated in Eastern civilizations. In the late 13th century, Wang Chen, an official of the Yuan Dynasty, in the Chinese province of Anhui, was responsible for introducing and developing the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 459–469, 2024. https://doi.org/10.1007/978-3-031-47281-7_38

460

A. Manique and A. Morgado

production of wooden characters, and designing a system of composition and revolving storage, to efficiently structure the typographic practice for the printing of literary works. Before Chen’s invention, printing in China was done with woodblocks, and as described by him, this technique did not become a frequent practice due to the difficulty in making the matrices and the cost of labour [1]. Paul Pelliot led an expedition to Central Asia in the early 20th century, resulting in several significant discoveries in the region. One notable finding was the use of wooden types for Uigur writing, dating back to around 1300, found in the caves of Tun-Huang. Thomas Carter, in his work “The Invention of Printing in China and its Spread Westward”, describes the types discovered by Pelliot as the oldest existing wooden types [1]. This discovery provides evidence that the utilization of wooden letters and their application in printing had spread throughout Central Asia. Although the first wooden types originated in the 13th century, their mass production and widespread use did not occur until the 19th century. Since Gutenberg’s invention of the printing press, the paradigm of letterpress printing relied on metal type. However, the demand for larger-sized typefaces for commercial printing, such as broadsides, necessitated an alternative material as metal proved unsuitable for producing display faces. Theodore Low De Vinne’s work presented various production methods for larger-scale types in metal, including sand casting, casting on arches, and stereotyping. Nevertheless, these methods posed challenges such as uneven solidification of surfaces, fragility of characters after cooling, weight, and high production costs [2]. Wood emerged as the most suitable alternative to metal, leading to the invention of machinery and tools for producing wood type by United States manufacturers, in the 19th century [2]. The wood used in this context was preferably maple wood, cut into slabs from the end of the log with the height of the types. Curing is carried out before the wood cut, and different forms of cure have been used, such as the use of kilns and steam-dry houses [3]. To efficiently mass-produce large types, a more effective process was required to replace manual methods of cutting and deburring wood. In 1827, Darius Wells invented the routing machine, which enabled the cutting of typefaces through the rotating movement of a steel cutter on a wooden block, removing excess negative space surrounding the letter. In 1828, Wells published the first catalogue of wooden typefaces, and in 1834, William Leavenworth introduced a new process by adapting Wells’ machine to the pantograph. Despite the emergence of other production methods, the pantograph remained the most widely used technology for manufacturing wood types. In the Portuguese context, the practice of typography has roots dating back to the 15th century. However, the production of movable metal type gained momentum with the establishment of the Fábrica de Caracteres by Jean Villeneuve in 1732, associated with the foundation of the Academia Real da História Portuguesa during the reign of D. João V. This Factory, which later became part of the Imprensa Nacional under the reign of D. José I, formerly known as Imprensa Régia, had the responsibility of supplying type to all printers in the kingdom and ensuring the teaching of the craft [4]. Regarding the mass production of wood types in Portugal, there is limited or scarce information available. During the early period of the Imprensa Régia, the kingdom’s protectionist policies restricted the import of foreign types, although this imposition was not upheld in the long run. Wood types existing in Portugal were mostly imported,

Oficina de Tipo: Comparative Study of Wood Type Production Methods

461

apart from those that were handcrafted [5]. The typographic industry in Portugal heavily relied on foreign products1 , and consequently, as described in Aspectos da Tipografia em Portugal, Portuguese printers reproduced what they deemed most relevant, often based on foreign catalogues, highlighting the influence these catalogues had on the national printing market [7]. In Portugal, it can be inferred that the surviving wood types today are replicas of other collections, either imported or copied from catalogues. With the introduction of offset printing, the use of movable type gradually declined, resulting in only a few collections of wood typefaces remaining. The origins of these collections are unknown, and most of them are incomplete and worn out2 .

2 Methodological Procedures 2.1 Analysis of the Wood Types and Survey To understand the constituent parts of a wood type, two grotesque typefaces existing in the Engraving workshop at ESAD.CR, were selected for analysis. The examination involved measuring the type height, point size (body), character width, sidebearings, identifying the type of wood used, and examining the producer’s stamps. To clarify questions that arose during the study of the analysed specimens and obtain additional technical information specifically related to the production of wooden typefaces, a survey was conducted. The survey consisted of semi-open questions and was sent to a selection of experts in the field of movable type typography. Out of the 17 individuals contacted, 7 responded: Ane Thon Knutsen, Celene Aubry (Hatch Show Print), Georgianne Liesch (Hamilton Wood Type Museum), Manuel Diogo, Marko Drpi´c (tipoRenesansa), Rick Griffith, and Rubén Dias. The answers obtained in the questionnaire influenced certain factors in the production, such as the use of a solid block of wood and the use of shellac as the preferable kind of finishing. 2.2 Production The production of new wood typefaces was carried out using two different technologies: 1) pantograph (analogue method); 2) and CNC machine (digital method). For the replication of the types, it was necessary to consider three distinct stages, essential for proper reproduction: wood preparation, cutting, and finishing. 1 At the beginning of the 20th century, prior to the Great War, national production of type

represented only a quarter of the imports. According to Manuel Canhão, in 1934, wood types were imported from Germany, Spain, Italy, France, the Netherlands, England, Austria, the United States, and Switzerland, in their respective order of quantity [6]. 2 Susana Durão describes a space attached to a typography, where obsolete material, such as wood types was placed: In the shed, in coexistence with cockroaches and rats there is a huge volume of rejected material: drawers of wooden types that did not survive the innovations of the technique and that became obsolete when the poster press and the poster composition section were dismantled. (…) the old and rare wooden types that once served to compose the posters, which gave fame and recognition to the printing works, were used by the construction workers as firewood in the summer barbecues next to the referred shed. (authors´ translation) [8].

462

A. Manique and A. Morgado

It is important to note that the types used as references for replication in the technologies are two different alphabets, and their manufacturing origin is unknown. Pantograph Before using the pantograph, the pine and mahogany wood were prepared for production: the available wooden boards were prepared and then rulers with a height of 2,365 cm were cut, considering the longitudinal arrangement of the wood fibres. For the reproduction of the pantograph, a mahogany grotesque font was used as reference, with a length of 239 didot points, approximately 9 cm. (see Fig. 1).

Fig. 1. Reference font.

The pantograph used was the Deckel G1L Pantograph (Fig. 3), manufactured by the Friedrich Deckel company. This pantograph is a standing machine that consists of two working tables connected by a hinged iron arm, which can be adjusted to different reproduction scales. On the stencil table, there is a guide rod (Fig. 2) whose function is to transmit the shape of the letter to the cutter on the cutting table through the mapping of the reference letter. The cutter (Fig. 4), located on the cutting table, accurately reproduces the movement of the guide rod on the wood block, proportionally within the calibrated reproduction range of the articulated arm. To ensure the exact reproduction of the reference types, a reproduction scale of 1:1 was adopted. The cutter used and the guide rod have both a 3 mm diameter, assuring the correct correspondence between them. The measure of the high relief (beard) of each letter is 4 mm and, in order to lower this measure on the wood block, the available handle was manipulated with the function of raising and lowering the work table. The reproduction time of each letter varied according to the character design detail. The letters reproduced were O, U, and J, with an execution time of approximately 40 min each.

Oficina de Tipo: Comparative Study of Wood Type Production Methods

463

Fig. 2. Work on the stencil machine, with the guide rod.

Fig. 3. Deckel G1L Pantograph.

After using the pantograph, it was necessary to cut each letter from the wooden rules, using a circular saw. Since they have the width and height of the types reproduced, it was only necessary to cut the letters with the previously mentioned length of 239 didot points. Then proceeded the smoothing of the milled sides and the removal of the wood excesses,

464

A. Manique and A. Morgado

Fig. 4. Cutting router.

using hand sandpapers. To prevent the absorption of solvents and other cleaning liquids in the printing phase, and to avoid the humidity penetration, shellac was applied to all the types made. CNC In the first phase, a digital file (Fig. 5) was developed with the vectorial design of the reference alphabet (Fig. 6), whose manufacturing origin is also unknown. For the CNC production, an ash wood board was used. The available wood board was prepared until the type height was reached (2.365 cm) and then two usable slabs were cut. The CNC machining allows the automation of the milling tools movement, from a CAM software. To program the machining to be performed, it is necessary the existence of a Gcode that communicates all the information to the CNC robotic arm. Therefore, a vector file was developed, in DXF format, with the complete alphabet distributed in two plates, with the same measures of the ash boards used. Despite the anatomy of the reference characters, it was preferred to reduce the scale of the types to be produced, so that it would be possible to execute the complete alphabet on the available ash wood. Before machining, it was necessary to use the program Estlcam. In this program, certain characteristics essential for machining were configured: the diameter of the cutter, the speeds and rotations of the cut, the paths for machining and their properties: depths, areas for filling, recessing, cutting, and bridges between wood elements. After this configuration and application of CAM characteristics to the DXF file, the machining direction was created, to indicate to the machine where to start in the work area. Finally, the project was saved and the Gcode was generated.

Oficina de Tipo: Comparative Study of Wood Type Production Methods

465

Fig. 5. Vector drawing based on the reference types.

Fig. 6. Proof-print with the reference alphabet. Imprinted by Ricardo Castro.

The OptiGraph software (NC numerical control program) was then used to read the Gcode. At this stage, the position of the drawing was assigned according to the material, the dimensions of the plate and the marking of the zero point, to orientate the drawing in the cutting area. Special attention was also given to the precise measurement of the material thickness, inserting the correct value in the Z axis, since wood is a material susceptible to moisture absorption, the thickness may oscillate from the moment of board preparation to machining. Finally, the working time was calculated and then machining began.

466

A. Manique and A. Morgado

To cut the typefaces, 3 mm milling cutters were used to perform the recess of the letters with more accentuated vertices and convex angles (A, K, M, N, S, V, W, X, Y), and 6 mm for the remaining ones (B, C, D E, F, G, H, I, J, L, O, P, Q, R, T, U, Z). By using a 6mm cutter for most of the letters, it is possible to cut more wood and shorten the machining time. Then, with the same cutter, the types are fully cut to their depth. At the end of the machining, it was necessary to remove the bridges with a chisel in the opposite direction of the piece, to prevent the types from being damaged, and apply the shellac finishing afterward. It should be noted that all the work done in CNC was accompanied by a specialized professional, considering that it is necessary a vast incident knowledge in this manufacturing process and in the software necessary for its use.

3 Discussion of the Results The answers obtained from the survey were synthesised and analysed to obtain specific technical information on the production of wood types. In certain questions, the same replies were provided, which allowed for greater certainty and substantiation of certain technical doubts. For example, in the question “What kind of finishing do you recommend? Beeswax, shellac, or hard wax oil? Any other tip on this subject?”, most replies received referred to the use of shellac as the preferred finishing. Regarding the ideal type of wood for type cutting, the highest recommendation was the use of a hard wood source, with the smallest grain possible. The answers ranged over different types of wood, having pear, boxwood or maple being the most referred to. It was observed in several types, that the portion of the letter cut was glued to the wooden base, raising the question: “Do you recommend cutting the wood type in two parts, which would then be glued, or cutting a single solid piece?”. In the obtained answers it was observed that the laminated version is only recommended if the wood source is not high-quality in terms of hardness, or, if there is no access to wood blocks with the full height of the type. To obtain strong and durable characters, cutting type into a single solid block is the best production method. Regarding the pantograph production results, the differences between the woods are noticeable. During the letter cutting, pine presents a lower mechanical resistance to the rotations of the milling machine in comparison with mahogany, which results in a higher need for finishing. Pine has other characteristics that make it disadvantageous for milling, such as the presence of knots and the malleability of its surface. Mahogany types present superior features than pine: Mahogany is a harder wood, and it needs few finishes, due to its good milling resistance, lower fracture risk, and higher resistance to printing press pressure. In Fig. 7, it is possible to observe two letters reproduced in the pantograph: O and J. In each pair of characters, on the left there are the guide types, and on the right the types reproduced in the pantograph. It should also be noted that it is important to adapt the use of cutters to the cutting of letters with convex angles and smaller counters, pointing letters such as R, M, or W. The ash types made in CNC, present a uniform cut and recessing, due to the automated machining, in which the milling characteristics are pre-programmed. It is noteworthy that some letters show a slight curvature in convex angles, compared to other letters with

Oficina de Tipo: Comparative Study of Wood Type Production Methods

467

Fig. 7. Detail magnification - On the left, are the reference types, and on the right, are the types reproduced in the pantograph.

more round or open shapes (Fig. 8). This characteristic is due to the impossibility of executing convex angles with the 3 mm cutter for the size of the types made in CNC (265 didot points). Some solutions are suggested to resolve this issue: the use of cutters with a smaller diameter, increasing the size of the letters to be cut, or adjusting the type design, conditioned by the size of the cutter to be used. Regarding the formal differences between CNC and pantograph types, it is possible to state that the CNC types have a more regular contour, and therefore less need for postfinishing. Nevertheless, pantograph production can be optimised by using milling cutters with smaller diameters and by improving the reproduction technique, with particular attention to the time taken to handle the rod which surrounds the guide letter. This ensures that the milling cutter can recess the letter with the greatest possible perfection. Both techniques are useful for type production, although they suit different possibilities: Pantograph reproduction allows the replication of types and typefaces identified in private or public collections, with a focus on their study, preservation, and above all the possibility to print with these artefacts. With CNC machining, it is possible to create new types from digital fonts, developed in the academy. This technology allows the creation of wooden alphabets for experimental letterpress printing, using contemporary fonts. The recovery and study of old technologies to produce movable type in wood, such as the pantograph, and the adaptation of the CNC to the production of types, reveal new possibilities for an experimental pedagogical practice in typography, highlighting the importance of this project in relation to the practice of typography and moveable printing.

468

A. Manique and A. Morgado

Fig. 8. Detail magnification – The figures show an image of the ash types made on the CNC machine, digitally outlined with the original shape (red), and marked with the diameter of the cutter used (cyan blue). In the A type, the inner vertices in the letter anatomy are slightly rounded, compared to the C type on the right, where the typeface design is unchanged due to the letter’s concave angles.

4 Conclusion The present research, within the scope of the project “Oficina de Tipo” at the Research Laboratory in Design and Arts, has allowed for the acquisition of knowledge for the manufacture of wooden types. The project was initially intended to supply movable type to a letterpress workshop created at ESAD.CR in 2022. The project has proven to be opportune for other academic projects, particularly the ongoing study of letterpress workshops in the district of Leiria. The intention of this study is to conduct a rigorous survey of the existing typographic material under the scope of the Estudo Nacional de Tipografia de Caracteres Móveis3 . The study has already identified several sorts of wooden types in private collections and established contacts to acquire or loan them for study, preservation, and replication, for further use in the workshop. It is worth mentioning that the letterpress establishment at the academy does not emerge from a historicist vision of teaching art and design, but rather from the need to strengthen the connection between theory and practice, valuing repetition, experimentation, and error in the education of visual arts and design. In the academy, the composition with movable type imposes itself as a methodical exercise, where the metric, the limit of material resources, the block of the letter, the line and the text, and the rules of 3 An initiative of the researchers Jorge dos Reis (FBAUL), João Neves (ESART, IPCB), Tiago

Marques (EA, UÉ), and Vítor Quelhas (ESMAD, IPP) proposed to several teams of district researchers, distributed throughout the country.

Oficina de Tipo: Comparative Study of Wood Type Production Methods

469

orthogonality, dictate the practice of the project. A counterpoint to the immediacy and arbitrariness of the digital, promoting a critical and enlightened practice. Acknowledgment. This work is financed by national funds through FCT – Fundação para a Ciência e Tecnologia, I.P., in the scope of the Programmatic Funding allocated to the Research Laboratory in Design and Arts (LiDA) with the reference “UIDP/05468/2020”. The Project was developed with the collaboration of LiDA researchers: Ricardo Castro, Ricardo Santos, Sérgio Gonçalves, and Nuno Gomes from ESAD.CR.

References 1. Carter T (1955) The invention of printing in China and its spread westward. New York: Ronald Press Co 2. De Vinne TL (1900) The practice of typography: a treatise on the processes of type-making, the point system, the names, sizes, styles and prices of plain printing types. The Century Co, New York 3. Kelly RR (2010) American wood type: 1828–1900. Notes on the evolution of decorated and large types. Saratoga: Liber Apertus Press 4. Queiroz MI, José I, Ferreira D (2019) Indústria, arte e letras: 250 anos da Imprensa Nacional. Lisboa: Imprensa Nacional 5. Diogo M (2016) A tipografia de caracteres móveis no contexto da produção editorial contemporânea. (dissertação de mestrado) Universidade de Lisboa – Faculdade de Arquitetura, Faculdade de Belas-Artes 6. Canhão M (1941) Os caracteres de imprensa e a sua evolução histórica, artística e económica em Portugal. Lisboa: Tipografia do Anuário Comercial 7. Araújo N, Mendes AP (1914) Aspectos da Tipografia em Portugal. Lisboa: Imprensa Nacional 8. Durão S (2003) Oficinas e tipógrafos: cultura e quotidianos de trabalho. Publicações Dom Quixote, Lisboa

When in Memeland, Speak in Memes: Contributions of Design Towards the Betterment of Online Behavior Regarding Public Health Heitor Alvelos1(B) , Jorge Brandão Pereira2 , Abhishek Chatterjee3 , Susana Barreto1 , Pedro Alves da Veiga4 , Cláudia Raquel Lima5 , and Eliana Penedos-Santiago1 1 University of Porto, Portugal/ID+ Unexpected Media Lab, Porto, Portugal

[email protected]

2 Polytechnic University of Cávado and Ave, Portugal/ID+ Unexpected Media Lab, Barcelos,

Portugal 3 University of Aveiro, Portugal/ID+ Unexpected Media Lab, Aveiro, Portugal 4 Open University of Portugal/CIAC Center for Research in Arts and Communication,

Lisbon, Portugal 5 Lusófona University, Portugal/ID+ Unexpected Media Lab, Lisbon, Portugal

Abstract. This research tackles a current tendency towards scientific mistrust among segments of the population: we posit that channels traditionally employed by scientists and policy-makers may not be sufficient to reach skeptical individuals and communities, as exponentially and pervasively evident in social media circles. We thus propose to explore informal online channels and formats such as memes, short videos, and forums, where the involved demographics tend to obtain information they deem reliable and relatable: in fact, formal channels of scientific communication are often regarded by skeptics as authoritarian and driven by obscure agendas. In this research, we focus on health policy behavior as a locus of conflict in recent years due to the pandemic, with Portugal as a case study. Informed by both online field work and in-person interviews with health policy skeptics, the project has so far produced a collection of visual artifacts for social media circulation. Validated specimens aim to become templates for the adoption of health policies. Keywords: Design for Science · Health policy · Social Media Design · Pandemic memes · Science communication in Portugal

1 Introduction Recent empirical and documented evidence has been pointing towards online social media as a primary factor in the rise of scientific mistrust (Yeung, A. W. K., et al., 2021; Brownell, S. E., & Price Jr., J. V., 2021); the issue became particularly pressing during the recent global pandemic. We therefore posit that both the semantics and contexts employed by scientists and decision-makers in their pedagogical and persuasive © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 470–486, 2024. https://doi.org/10.1007/978-3-031-47281-7_39

When in Memeland, Speak in Memes: Contributions of Design

471

Fig. 1. Doge meme generated by the authors.

attempts may benefit from addressing emerging, informal online channels and formats. This hypothesis is informed by public health and psychology studies that reveal frequent cognitive biases that tend to evade fact-based persuasion (Betsch et al., 2010; Broniatowski et al., 2018; Donzelli et al., 2018; Igoe, 2019; Gerts et al., 2021), as well as the observation of a potential over-formalization of channels of health policy often regarded by population segments as authoritarian and elitist (Frosch, D. L., May, S. G., Rendle, K. A., Tietbohl, C., & Elwyn, G., 2012). We further posit that communication design may play a specialist role in this calibration of health policy behavior in social media platforms: we aim at testing the possibility of a pedagogy of science and behavioral persuasion through the adoption and configuration of memes, infographics, short videos, data comics and humorous textual content. This possibility is in line with both the aforementioned evidence of subjective components in cognitive processes, and the roles of communication design as a harmonizing agent between the rational and the expressive. Empirical evidence has been largely collected during the COVID-19 pandemic in the context of Portugal, namely in regards to confinement rules enacted by the government, daily statistics and outlooks provided by health authorities, and vaccination programs set from early 2021 onwards. The largely failed employment of a tracking mobile phone app has been observed as well, both as an indicator of a possible over-reliance on digital technology, and as evidence of a self-imposed limitation on the semantics of digital communication.

472

H. Alvelos et al.

It must be pointed out that the present research is not focused on occurrences of flatout science denial, including so-called complementary and alternative medicine practices (Coulter et al., 2004), nor conspiracy theories as collective phenomena of mobilization: both the motivations and socio-ideological dynamics of these radical expressions tend to entail a degree of cognitive and behavioral entrenchment that memes and data comics, however accessible, would inevitably struggle to address; instead, the research chooses to focus on relative degrees of hesitancy and inability to process, translate and apply complex reliable information in daily contexts (Fig. 1). The objectives of the research can thus be summarized as: – To test the viability of employing social media content to engage audiences that are skeptical or hesitant regarding the adoption of public health policies; – To acknowledge and employ subjectivity and expression as rooted components of a reliable engagement with public health; – To contribute to the promotion of civic vocation in participatory online media; – To broaden the scope for more receptive and engaged dynamic between scientific knowledge, and citizen narratives and expectations; – To broaden the scope of design studies and practice among contemporary online phenomena; – To legitimize exploratory roles design may perform in the receptivity of health policy within emerging media contexts; – To translate complex health information, knowledge and data into broadly accessible and persuasive formats. Despite the specificity and ambition of the aforementioned objectives, it must be emphasized that the project follows an overall exploratory approach; both the volatility of social media (in its nature and impact) and the unique circumstances of the recent pandemic, dictate an ability to open up to potentially unprecedented approaches to design. As a consequence, the above objectives are primarily regarded as long-term outcomes of a consortium of interested parties, with an iterative methodological vocation; within this broader scope, the tangible aim of the present, exploratory project is primarily to open up a space of scrutiny and possibility, along with the respective seeds of engagement.

2 State of the Art The recent global pandemic crisis (COVID-19, 2020–2022) has presented multiple challenges to scientific development and policy implementation: among a diverse range of responses, both by authorities and citizens, Portugal stood as an overall successful case of viral containment by social behavior and vaccination implementation: explicit dissent was rare, and episodes of public hostility even more residual. However, three core issues remained throughout: – A general absence of a tangible, accessible communication strategy on the part of authorities was evident: a press conference in a formal environment frequently employing hermetic terminology was broadcast daily on open TV channels (Serviço Nacional de Saúde, n.d.); the press conference was accompanied by fairly abstract and generic slogans, devoid of concrete information or instructions (“Protect yourself”, “We will beat the virus”, and “Take care of yourself, take care of others”) (Fig. 2).

When in Memeland, Speak in Memes: Contributions of Design

473

– The attempt to trace infections through mobile phone technology (Oliveira, R., & Mendonça, J. M., 2020) largely proved to be a failure, even the subject of ridicule (Pinto, S., Oliveira, E., & Costa e Silva, E., 2021); – A discreet segment of the population chose not to follow governmental instructions, due to a multitude of reasons including suspicion of the timing of the vaccine, privacy and surveillance concerns, superstition, and a trust in alternative tips and therapies primarily disseminated through online platforms - as ratified by Dimoff, J. D., Dao, A. N., Mitchell, J., & Olson, A. (2021).

Fig. 2. Still from daily press conference by Portuguese National Health authorities during the pandemic, broadcast live on national television. Source: Serviço Nacional de Saúde, n.d.

With regards to the first issue, successful examples of tangible instructions were identified, such as a campaign for mask wearing that makes effective use of subjective components (Fig. 3) - thanking people for wearing a mask, rather than imposing it, and having actual individuals (health workers) setting the example in order to provide a more personal semantic of engagement. However, examples of positive communication were still the exception. On the other hand, the issue of pandemic containment through digital technology can essentially be regarded as a deterministic endpoint of over-reliance on big data; despite the promise of its theoretical model, its implementation proved impossible: it would involve complex processes of registration, activation, professional involvement, and software update. When the Portuguese Prime Minister suggested the possibility of its mandatory use, the software became the subject of irreversible ridicule - this, adding to the debate on the aforementioned issues of privacy and governmental surveillance. This approach to pandemic containment, we argue, suffered from both a degree of digital dogma and a lack of awareness of subjective components, while breaking a tacit yet

474

H. Alvelos et al.

Fig. 3. Examples of positive reinforcement campaign employing testimonies and gratitude. Source: Serviço Nacional de Saúde, n.d.

pervasive contemporary expectation that the vocation of mobile technology be driven by entertainment and ease. However, the core issue under discussion in the present research is the understanding that most instances of skepticism occurred due to both a lack of accessible and concise information, as well as an over-formalisation of health policy discourse on the part of authorities - as proposed by Dillard & Shen’s “Theory of Psychological Reactance” (2005). As traditional media channels focused on intricate statistical and infographic representation of the pandemic, online media channels engaged in a diametrically opposite exercise: humorous content as a tool for overcoming anxiety (Fig. 4), slogans, expressions of solidarity, and personal testimonies. The problem, however, was that this largely benevolent and innocuous content randomly coexisted with misleading health information, factual manipulation and statements of a conspiratorial nature, and wellness philosophies aspiring to become an alternative to scientific efficacy. Furthermore, the appeal of this latter content was rarely matched by reliable sources of information and pedagogy: given the choice between a formal governmental edict and an ironic set of enjoyable memes, however questionable, a large segment of online users instinctively gravitate towards the latter. This was empirically witnessed and experienced by the authors as design and media researchers, and as citizens and online users themselves during the various stages of the pandemic. On an international level, specific entities attempted to reconcile this dichotomy by employing online languages and aesthetics in their official communication; an example is the set of memes and tags created by the U.S. National Foundation for Infectious Diseases (Figs. 5 and 6). These were made available on their website, along with the message: “Share these infectious disease memes to help spread awareness, not disease. Select an image below to view a full size version and save to your device. Share on social media with the hashtags #GetVaccinated, #VaccinesWork, and/or #FightFlu. Tag your friends and family to remind them to #GetVaccinated to help stay healthy!”.

When in Memeland, Speak in Memes: Contributions of Design

475

Fig. 4. Y U NO meme generated by the authors, based on empirical analysis of pandemic memes.

Fig. 5. Example of meme made available by the U.S. National Foundation for Infectious Diseases (n.d.).

It is this type of engagement that the present research attempts to pursue, by further weaving authoritative health information and policy into social media, and by generating specimens and prototypes covering subjective behavioral response. Subsequently, we expect to propose an equivalent harmonization of formal and informal channels of communication in the Portuguese context. A proposition on the recent pandemic could be regarded as redundant, given its retrospective nature. However, we argue for its validity for two main reasons: firstly, the design and management of communication during the pandemic contained an inevitable degree of improvisation as the crisis unfolded, with little time for the strategic analysis that may now be ensured; and secondly, the present study ultimately aims at being relevant and partially contributive to future crises scenarios.

476

H. Alvelos et al.

Fig. 6. Example of meme made available by the U.S. National Foundation for Infectious Diseases (n.d.).

3 Literature Review Numerous studies in recent years have highlighted the influence of the internet, and in particular, social media towards scientific literacy and population behavior (Kata, 2010; Mitra et al., 2016; Benoit & Mauldin, 2021). The emerging general consensus is that the casual and unregulated nature of social media has profoundly altered access to reliable information, and changed the way knowledge is approached and disseminated (IDA, 2018; Benoit & Mauldin, 2021). This new “boundless ecosystem” (Karafillakis et al., 2021) has inevitably affected conversations on public belief: studies suggest that while the immediacy and informal nature of social media may appear to provide an advantage to thought leaders towards influencing public perception (IDA, 2018), it also fosters an unmediated spread of misinformation (Chou et al., 2009; Keelan et al., 2010; Igoe, 2019). This “infodemic” of differing veracities is presently in sharp focus since social media platforms acting as vectors are actively scaffolding the spread of wrong and unfounded information (Wang et al., 2019; Puri et al., 2020); recently, it has undermined worldwide efforts towards controlling the COVID-19 pandemic (Gerts et al., 2021; Loomba et al., 2021; Demuyakor et al., 2021). The sheer bulk of varying information has also aggravated the mistrust of data linked with risk perceptions and safety of the vaccines (Kennedy et al., 2021), thereby driving hesitancy and risking further lives (Wilson & Wiysonge, 2020; Latkin et al., 2021; Loomba et al., 2021; Chadwick et al., 2021). One of the ways in which studies have sought to determine the underlying factors is by profiling anti and pro vaccination individuals and groups in terms of their communication behavior on web and social media (Davies, P., & Chapman, S. & Leask, J., 2002; Arif, et al.; 2018; Broniatowski et al., 2018; Igoe, 2019). Results correspondingly

When in Memeland, Speak in Memes: Contributions of Design

477

show that whereas pro-vaxxers tend to quote scientific literature and population-based statistical evidence when sharing views online (Betsch et al., 2010; Faasse et al., 2016; Xu et al., 2019), anti-vaxxers generate content based on personal opinions and emotional experiences which resonate with their target socio-demographic audiences (Bessi et al., 2015; Donzelli et al., 2018). A further scrutiny of thought leaders driving the anti-vaccination movements reveals that in conjunction with celebrities who use their prominence on social media to speak on topics that they are not experts in (Igoe, 2019; Herrera-Peco et al., 2021), there are social media influencers who spread scientific sounding disinformation by providing “misleading causal explanations” (Igoe, 2019). Correspondingly, there is evidence that since individuals exhibit a greater proclivity towards engaging with negative information around vaccination than positive views (Blankenship et al., 2018; Li et al., 2020; Puri, et al., 2020), a fraction of anti-vaxxers, who produce a majority of content, can cluster audiences and become “echo chambers” (Levy, 2021; Karafillakis et al., 2021; Demuyakor et al., 2021; Jiang et al.; 2021) for further amplification. Conversely, content producing pro-vaxxers have been found not engaging sufficiently with a broader community with similar concerns (Demuyakor et al, 2021). Adding to this is the plain evidence that scientific knowledge tends towards epistemological complexity and disciplinary hermeticism, whereas disinformation tends to offer a readily understandable reading of the issue at stake; consequently, “people learning about topics are vulnerable to hubris” (Ballantyne and Dunning, 2022). Furthermore, the ample evidence of this polarization indicates a pressing need for a third stance towards mutual engagement: we thus argue that a dialogue may be fostered within a shared dialectic involving scientists, policy-makers and moderate skeptics. This research, accordingly, posits that to understand, and specifically, to mitigate such occurring antagonisms within the general public towards the filtering and processing of relevant information in matters of health and safety, it has become imperative for governing bodies to examine and incentivise forms of new media that are actively demonstrating the capacity to engage and influence public opinion. Memes, as Nieubuurt (2021) indicates, hold the ability to function as digital propaganda leaflets in the same way as their analogue antecedents, bonding people under an idea. As exploratory signs with semiotic reasoning, they are a “tool for cognitive representation” (Scherz, 2022). Furthermore, as stated by Grace and Fiyinfolu (2022), “[…] satire, music and comedy are not just made to make the audience laugh; the underlying issue is for them to be used as tools for reforming the society.” The study elaborates its mapping from the aforementioned COVID-19 crisis, particularly in the Portuguese context, as a primary case study, from where it endorses to understand, and specifically, to employ it as source material towards the organization of taxonomies and communication patterns, including visual and discursive ones. This will enable the integration of findings on the allowances and affordances of designing reliable health information in public health research and communication, namely for online social media contexts.

478

H. Alvelos et al.

Fig. 7. Self-reflective meme generated by the authors.

4 Methodology and Preliminary Findings The present study is being informed by the retrospective analysis of empirical recollections and collection of online specimens and discussions during the pandemic; this empirical activity took place mostly on Facebook, from the onset of the pandemic up to its gradual replacement by subsequent global narratives. Despite a predominant nonparticipatory approach in order to ensure the neutrality of the observation, the occasional written online exchange in comments and forums largely confirmed an entrenchment of users’ beliefs, often mistrustful or derisive of official health communication channels and content.

Fig. 8. Example of imgflip meme template and user interface: Buff Doge vs. Cheems Meme. Source: imgflip.com.

When in Memeland, Speak in Memes: Contributions of Design

479

The above content and experience has therefore paved the way for both an empirical identification of characteristics to be employed by reliable sources in order to reach skeptical online segments, and an exploratory production of meme prototypes. These have mostly made use of the popular online meme generator “imgflip”, thus ensuring both a semantic recognisability upon the employment of popular meme imagery, and an intuitive interface with user-friendly design tools. An example of a pre-existing visual template and user interface is presented as Fig. 8.

Fig. 9. Metaphorical meme with pedagogical component, generated by the authors.

A set of ongoing in-person interviews with COVID skeptics complements the aforementioned participant observation, where anonymity is offered as desired by interviewees. Patterns running through these interviews include: – Skeptical subjects tend to avoid calling themselves “deniers”, while in fact they do often deny the efficacy of vaccines, or bundle their suspicion with broader geopolitical stances that include financial interests on the part of pharmaceuticals and governmental motivations; – There seems to be a prevalent suspicion of official channels of communication (television is mentioned often), accompanied by a degree of pride in resorting to alternative means of information such as personal youtube channels. The aforementioned activity has been complemented by the design of a series of creative activities, where both members of the research team and design students have been generating meme prototypes for online and offline engagement, trust and pedagogy. These activities have so far included private weekly online responses to pre-selected meme imagery, the organization of exploratory meme generation and discussion sessions with team members (Figs. 7, 9 and 11), and the curricular insertion of meme production

480

H. Alvelos et al.

workshops in Design and Digital Media Arts courses, as well as an exploration of online/offline engagement by printing out generated content, thus opening up further contexts of resonance (Fig. 10). Further iterations of these various actions will ensure gradual cycles of validation and adjustment (Fig. 13).

Fig. 10. Classroom workshop in the scope of the present research group activity. Photography by Cláudia Raquel Lima.

We regard this latter activity as particularly relevant, as it brings input by younger generations with a more intricate perspective on online dynamics, circulation and semantics, while allowing personal testimony to play a key role in engagement. Concomitantly, these various outputs are being shared within the context of events and channels of scientific dissemination - conferences, posters and articles (Fig. 12), in order to further open the debate on the legitimacy and efficacy of using meme aesthetics and semantics in the persuasion of skeptical individuals and communities. Online presence remains discreet, while pondering the resolution of viral dissemination versus centralized communication on the part of the research. Empirical findings emerging from the above activities include the following: – Humour tends to be pervasive in social media, and proves to be an effective device for engagement; – However, with the above in mind, irony-based humor tends to both antagonize dissenting views and entrench viewpoints; – Caption-based memes using pre-circulating imagery denote a knowledge of the medium, and therefore tend to legitimize those producing and sharing them (Fig. 7); other graphic elements such as specific typefaces may still ensure a level of familiarity among online users;

When in Memeland, Speak in Memes: Contributions of Design

481

Fig. 11. Humorous meme generated in project workshop context.

Fig. 12. Setting up a display of meme printouts for open feedback at an international visual methods conference. Photography by Susana Barreto.

– Nevertheless, pre-circulating imagery does not need to be the exclusive visual source: memes may be constructed from original images;

482

H. Alvelos et al.

– Tangible metaphors tend to be useful in illustrating complex information and knowledge, thus rendering them more accessible and understandable by online audiences (Fig. 9); – Personal testimonies tend to more readily engage with the readers’/viewers’ subjective components; – Content production should avoid mission statements that, in themselves, may preemptively invalidate the possibility of engagement. Providing content on a smaller, caseby-case basis may prove to be more efficient than gathering it under an encompassing viewpoint that may be regarded as patronizing or antagonizing. We argue that these empirical findings are convergent with Dillard & Shen’s “Theory of Psychological Reactance” (2005), in that they support individual freedom-restoration of freedom, and give the individual the notion of being a part of the communication/solution - rather than enduring an authoritative type of communication/solution.

Fig. 13. Project methodology summarized in meme format. Source: the authors.

Despite the gradual emergence of these empirical findings, given the aforementioned, unprecedented factors at work (as well as the intrinsic degree of complexity), full validation and measurement can only expect to be addressed as longer-term goals, namely through extrapolation and cross-disciplinary analysis; however, preliminary response from participating students and design peers has largely confirmed both the pertinence of the proposed engagement and the potential of generated specimens.

5 Outlook The present research posits that communication design and media arts may contribute to a constructive engagement with science and health policy skeptics in online environments. It acknowledges prior, psycho-cognitive evidence that knowledge and beliefs are not solely constructed by logic and factual information, but rather contain subjective

When in Memeland, Speak in Memes: Contributions of Design

483

components: it is precisely within this territory of subjectivity that we propose design may contribute, by employing online humor, aesthetics and symbolic validation as valid tools for a pedagogy of health policy behavior. The case study of Portugal in the COVID-19 pandemic is the focus of the present research: both official and unofficial communication channels are being covered and analyzed, aiming at diluting the polarized perception whereby one is dogmatic and the other is unreliable (Fig. 14). Methodologies include online ethnography, the collection of empirical evidence, retrospective access to documentation and memorabilia, the identification of underlying taxonomies, semi-structured interviews, and exploratory sessions in both research and learning contexts. We expect further iterations (Fig. 13) to validate produced materials, resulting in a set of prototypes and recommendations for science communication in online contexts that may effectively incorporate subjective components as pedagogical assets rather than impediments, as well as arguing for a core contribution of design in online behavioral changes regarding public health. Additionally, these further iterations will diversify the range of approaches beyond the premises of meme culture and Facebook, in order to include platforms, genres and formats such as TikTok, data comics, and critical readers.

Fig. 14. Meme addressing the apparent conundrum of the present research. Source: the authors.

Disclaimer Regarding Use of Third-Party Images It is recognised that some of the external images (Figs. 2 and 3) may be employed within

484

H. Alvelos et al.

the framework of the fair use doctrine, which permits their use for purposes of criticism, commentary, or education. This determination has been made after careful consideration of the nuances associated with fair use. The authors have made every effort to accurately attribute and acknowledge the ownership of the external images. Proper citations, including source information and authorship, have been diligently provided in accordance with established citation conventions, thus respecting the intellectual property rights of these images. Acknowledgements. This work is funded by national funds through the FCT – Fundação para a Ciência e a Tecnologia, I.P., under the scope of the project 2022.08322.PTDC.

References Arif N, et al (2018) Fake news or weak science? Visibility and characterization of antivaccine webpages returned by google in different languages and countries. Front. Immunol 9: art. 1215, 1–12. https://doi.org/10.3389/fimmu.2018.01215 Ballantyne N, Dunning D (2022) Skeptics say, ‘do your own research.’ it’s not that simple. The New York Times. https://www.nytimes.com/2022/01/03/opinion/dyor-do-your-own-research. html. Accessed 31 May 2023 Benoit SL, Mauldin RF (2021) The “anti-vax” movement: a quantitative report on vaccine beliefs and knowledge across social media. BMC Public Health 21(1):1–11. https://doi.org/10.1186/ s12889-021-12114-8 Bessi A, Zollo F, del Vicario M, Scala A, Caldarelli G, Quattrociocchi W (2015) Trend of narratives in the age of misinformation. Plos ONE 10(8):e0134641. https://doi.org/10.1371/journal.pone. 0134641 Betsch C, Renkewitz F, Betsch T, Ulshöfer C (2010) The influence of vaccine-critical websites on perceiving vaccination risks. J Health Psychol 15(3):446–455. https://doi.org/10.1177/135 9105309353647 Blankenship E et al (2018) Sentiment, contents, and retweets: a study of two vaccine-related twitter datasets. Permanente J 22(3):17–138. https://doi.org/10.7812/tpp/17-138 Brehm SS, Brehm JW (2013) Psychological reactance: a theory of freedom and control. Academic Press Broniatowski DA et al (2018) Weaponized health communication: twitter bots and Russian trolls amplify the vaccine debate. Am J Public Health 108(10):1378–1384. https://doi.org/10.2105/ ajph.2018.304567 Brownell SE, Price Jr JV (2021) Overcoming the discourse of science mistrust: how science education can help during times of crisis and beyond.Cult Sci Eduhttps://doi.org/10.1007/s11 422-021-10064-6 Chadwick A, et al (2021) Online social endorsement and Covid-19 vaccine hesitancy in the United Kingdom. Soc. Media + Soc. 7(2): 1–17. https://doi.org/10.1177/20563051211008817 Chou WS, Hunt YM, Beckjord EB, Moser RP, Hesse BW (2009) Social media use in the united states: implications for health communication. J Med Internet Res 11(4):e48. https://doi.org/ 10.2196/jmir.1249 Coulter ID, Willis EM (2004) The rise and rise of complementary and alternative medicine: a sociological perspective. Med J Aust 180(11):587–589. https://doi.org/10.5694/j.1326-5377. 2004.tb06099.x Davies P, Chapman S, Leask J (2002) Antivaccination activists on the world wide web. Arch Dis Child 87(1):22–25. https://doi.org/10.1136/adc.87.1.22

When in Memeland, Speak in Memes: Contributions of Design

485

Demuyakor J, Nyatuame IN, Obiri S (2021) Unmasking COVID-19 vaccine “infodemic” in the social media. Online J Commun Media Technol. 11(4): e202119. https://doi.org/10.30935/ ojcmt/11200 Dillard JP, Shen L (2005) On the nature of reactance and its role in persuasive health communication. Commun Monogr 72(2):144–168 Dimoff JD, Dao AN, Mitchell J, Olson A (2021) Live free and die: expanding the terror management health model for pandemics to account for psychological reactance. Soc Pers Psychol Compass 15(3):e12585 Donzelli G et al (2018) Misinformation on vaccination: a quantitative analysis of YouTube videos. Hum Vaccin Immunother 14(7):1654–1659. https://doi.org/10.1080/21645515.2018.1454572 Faasse K, Chatman CJ, Martin, LR (2016) A comparison of language use in pro- and antivaccination comments in response to a high profile Facebook post. Vaccine 34(47): 5808–5814. https://doi.org/10.1016/j.vaccine.2016.09.029 Frosch DL, May SG, Rendle KA, Tietbohl C, Elwyn G (2012) Authoritarian physicians and patients’ fear of being labeled ‘difficult’ among key obstacles to shared decision making. Health Aff 31(5):1030–1038 Gerts D et al (2021) “Thought I’d share first” and other conspiracy theory tweets from the COVID19 infodemic: exploratory study. JMIR Public Health Surveill 7(4):e26527. https://doi.org/10. 2196/26527 Grace AM, Fiyinfolu AI (2022) A discourse analysis of the effectiveness of online satire, music and comedy in creating awareness on Covid-19 pandemic. J Positive School Psychol 6(8): 2132–2141 (2022). ISSN: 2717–7564 Herrera-Peco I, et al (2021) Antivaccine movement and COVID-19 negationism: a content analysis of Spanish-written messages on Twitter. Vaccines 9(6): 656. https://doi.org/10.3390/vaccines9 060656 Institute for Defense Analyses. https://www.ida.org/. Accessed 31 Mar 2023 Igoe KJ (2019) Establishing the truth: vaccines, social media, and the spread of misinformation. Harvard T. H. Chan School of Public Health. https://www.hsph.harvard.edu/ecpe/vaccines-soc ial-media-spread-misinformation/ Iloh C (2021) Do it for the culture: the case for memes in qualitative research. Int J Qual Meth 20 (2021). https://doi.org/10.1177/16094069211025896 Jiang J, Ren X, Ferrara E (2021) Social Media polarization and echo chambers in the context of COVID-19: case study. JMIRx Med 2(3):e29570. https://doi.org/10.2196/29570 Karafillakis E et al (2021) Methods for social media monitoring related to vaccination: systematic scoping review. JMIR Public Health Surveill 7(2):e17149. https://doi.org/10.2196/17149 Kata A (2010) A postmodern Pandora’s box: anti-vaccination misinformation on the Internet. Vaccine 28(7):1709–1716. https://doi.org/10.1016/j.vaccine.2009.12.022 Keelan J, Pavri V, Balakrishnan R, Wilson K (2010) An analysis of the Human Papillomavirus vaccine debate on MySpace blogs. Vaccine 28(6):1535–1540. https://doi.org/10.1016/j.vac cine.2009.11.060 Kennedy EB, Daoust JF, Vikse J, Nelson V (2021) “Until i know it’s safe for me”: the role of timing in COVID-19 vaccine decision-making and vaccine hesitancy. Vaccines 9(12):1417. https://doi.org/10.3390/vaccines9121417 Latkin CA, Dayton L, Yi G, Konstantopoulos A, Boodram B (2021) Trust in a COVID-19 vaccine in the U.S.: a social-ecological perspective. Soc Sci Med 270: 113684. https://doi.org/10.1016/ j.socscimed.2021.113684 Levy N (2021) Echoes of Covid misinformation. Phil. Psychol 1–17. https://doi.org/10.1080/095 15089.2021.2009452 Li HOY, Bailey A, Huynh D, Chan J (2020) YouTube as a source of information on COVID-19: a pandemic of misinformation? BMJ Glob Health 5(5):e002604. https://doi.org/10.1136/bmjgh2020-002604

486

H. Alvelos et al.

Loomba S, de Figueiredo A, Piatek SJ, de Graaf K, Larson HJ (2021) Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nat Hum Behav 5(3):337–348. https://doi.org/10.1038/s41562-021-01056-1 McIntyre L (2022) How to talk to a science denier: conversations with flat earthers, climate deniers, and others who defy reason. MIT Press, Cambridge Meme Generator (n.d.). https://imgflip.com/memegenerator. Accessed 02 Apr 2023 Mitra T, Counts S, Pennebaker J (2016) Understanding Anti-Vaccination Attitudes in Social Media. In: Proceedings of the international AAAI conference on web and social media, vol 10(1), pp 269–278. https://ojs.aaai.org/index.php/ICWSM/article/view/14729 National Foundation for Infectious Diseases: Infectious Disease Memes (n.d.). https://www.nfid. org/immunization/infectious-disease-memes/ Nieubuurt JT (2021) Internet memes: leaflet propaganda of the digital age. Front Commun 5 (2021). https://doi.org/10.3389/fcomm.2020.547065 Oliveira R, Mendonça JM (2020) Staway Covid. contact tracing for Covid-19. INESC TEC Sci Soc 1(1): 58–61 (2020). https://science-society.inesctec.pt/index.php/inesctecesociedade/art icle/view/33 Parreira R (2022) Dois anos de COVID-19: da falta do papel higiénico à variação Omicron. Ainda se lembra dos primeiros memes? In SapoTek (2022). https://tek.sapo.pt/multimedia/ artigos/dois-anos-de-covid-19-da-falta-do-papel-higienico-a-variacao-omikron-ainda-se-lem bra-dos-primeiros-memes Pinto S, Oliveira E, Costa e Silva E (2021) A controvérsia na cobertura mediática de saúde: a aplicação stayaway Covid e as fontes de informação. Comunicação e Sociedade 40 (2021). http://journals.openedition.org/cs/6039 Puri N, Coomes EA, Haghbayan H, Gunaratne K (2020) Social media and vaccine hesitancy: new updates for the era of COVID-19 and globalized infectious diseases. Hum Vaccin Immunother 16(11):2586–2593. https://doi.org/10.1080/21645515.2020.1780846 Scherz L (2022) I’m not lonely, i have memes: the cognitive, (dis)embodied experience of depression memes. In: Arkenbout, C., Scherz, L. (Eds.) Critical Meme Reader #2: Memetic Tacticality, 140–148. Institute of Network Cultures (2022) Serviço Nacional de Saúde. Conferência de Imprensa (n.d.). https://www.sns.gov.pt/conferenciasde-imprensa-diarias/. Accessed 02 Apr 2023 Shifman L (2014) The cultural logic of photo-based meme genres. J Vis Cult 13(3):340–358. https://doi.org/10.1177/1470412914546577 Wang Y, McKee M, Torbica A, Stuckler D (2019) Systematic literature review on the spread of health-related misinformation on social media. Soc Sci Med 240:112552. https://doi.org/10. 1016/j.socscimed.2019.112552 Wilson SL, Wiysonge C (2020) Social media and vaccine hesitancy. BMJ Glob Health 5(10):e004206. https://doi.org/10.1136/bmjgh-2020-004206 Xu Z, Ellis L, Umphrey LR (2019) The easier the better? comparing the readability and engagement of online pro- and anti-vaccination articles. Health Educ Behav 46(5):790–797. https://doi.org/ 10.1177/1090198119853614 Yeung AWK et al (2022) Medical and health-related misinformation on social media: bibliometric study of the scientific literature. J Med Internet Res 24(1):e28152. https://doi.org/10.2196/ 28152

Understanding the Methodological Value of Mapping Alternative Fiction Scenarios in the Context of Domestic Violence Mariana Alves Carvalho1 , Rodrigo Hernández-Ramírez1,2 and Hande Ayanoglu1,2(B)

,

1 IADE, Faculdade de Design, Tecnologia e Comunicação, Universidade Europeia,

Av. D. Carlos I, 4, 1200-649 Lisbon, Portugal {rodrigo.ramirez,hande.ayanoglu}@universidadeeuropeia.pt 2 UNIDCOM/IADE, Unidade de Investigação em Design e Comunicação, Av. D. Carlos I, 4, 1200-649 Lisbon, Portugal

Abstract. Smart homes have revolutionized the way we live, with new technologies connecting users’ devices with everyday objects. However, these advancements have also led to the emergence of unexpected dangers relating to technologyfacilitated abuse. Domestic abusers can now maintain control beyond previous spatial boundaries, making it impossible for victims to recognize that they suffer from abuse. This paper is part of an ongoing MA project that aims to prevent smart products from being used as tools for domestic violence. The paper demonstrates the methodological value of mapping speculative scenarios during the ideation phase in the design process. The scenarios aim to encourage reflection, speculation, and debate about the potential consequences and ethical implications of emerging technologies. The paper proposes potential solutions that reimagine how applications can support healthy relationships by emphasizing transparency, consent, and trust-building. It highlights the need for awareness, empathy, and ethical design practices to ensure that smart homes promote the well-being and privacy of their occupants. Keywords: Smart Products · Domestic Violence · Privacy · Speculative Scenarios

1 Introduction According to [1], in 2021 the smart homes market was valued at USD 79,13 billion, and it is expected to reach USD 313.95 billion by 2027, registering a CAGR of 25.3% over the forecast period of 2022–2027. New technologies connect users’ devices with everyday objects, such as virtual assistants (e.g., Alexa, Google Home, Amazon Echo), smart watches, smart locks, cameras, and more. However, as [2] notes, technology has also allowed domestic abusers to “inflict old harms in new ways”. [3] similarly argue that the rapid advancement of digital communication services, smart homes, and digitalization procedures creates unexpected dangers © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 487–497, 2024. https://doi.org/10.1007/978-3-031-47281-7_40

488

M. A. Carvalho et al.

relating to technology-facilitated abuse. These technologies expand the ability of abusers to maintain control beyond previous spatial boundaries, making it impossible for victims to recognize that they are suffering from abuse [4]. As a result, victims often suffer from isolation and alienation from family members and friends, making it even more difficult to realize and prove the abuse [5]. It is impossible to deny how easier certain aspects of everyday life are thanks to smart products. However, as [3] argue, the emergence of smart systems and the entry of novel business models have also fostered harmful social interactions and unfair control dynamics. Moreover, While smart home devices offer numerous advantages, such as enhanced security, convenience, and entertainment, they have also inadvertently granted device owners significant control over their intimate partners within the same household, potentially leading to instances of tech-facilitated domestic abuse, as recent studies have indicated [6]. This paper is part of an ongoing MA project focused on the research question, “How can we prevent smart products from being used as tools for domestic violence?” To collect data, a five-part questionnaire [7] was used that covered demographic information, technological experience, personal privacy, technological abuse, and the impact of abuse. The questionnaire also included open-ended questions to gather specific data for developing speculative scenarios. This paper aims to demonstrate the methodological value of mapping speculative scenarios during the ideation phase, which is a middle stage in the design process. The data collected will be used to suggest ways to redesign smart products that protect privacy.

2 Background 2.1 Speculative Design [8] introduced a design approach that fosters reflective conversations and disagreements concerning prevailing cultural values. Originally termed “critical design,” this concept evolved into “speculative design” to better encapsulate its practice, wherein design artifacts are employed to explore potential alternative futures that are both plausible and thought-provoking [9]. The objective is to ignite discussions about the desirability of these futures by bringing to light the aspects of everyday life that often remain invisible and overlooked. According to [10], design serves as a tool for speculating about the potential trajectories of the world. While it maintains a forward-looking perspective, it refrains from making explicit predictions. Instead, its primary aim is to evoke responses and address issues related to politics, as well as the prevailing social, economic, and technological configurations. This approach contributes to the unveiling of a world that extends beyond its current representation [11], making previously hidden aspects more visible and accessible. Speculative design liberates novel ideas that can enhance the likelihood of realizing more desirable futures. By frequently utilizing the phrase “what if” in their inquiries, [10] stimulates debates and discussions regarding the kinds of futures people aspire to achieve and those they wish to avoid. They argue that as speculation becomes more pervasive, the boundaries of future realities become increasingly flexible. To illustrate

Understanding the Methodological Value of Mapping

489

this concept, they developed the model depicted in Fig. 1 to explore potential future scenarios, akin to [12] representation of concentric cones, which portrays the expanding possibilities over time.

Fig. 1. PPPP-Diagram [10]

To create provocations or “narrative elements to predict and explain possible futures for design,” speculative design uses tools and methodologies of design, scenarios, techniques, and literature review [13]. The resulting scenarios fuse realism and imagination, allowing consideration of potential outcomes, ethical considerations, and the need for reflection and critique that requires thinking outside the box. [10] argue that the emphasis has shifted from architecture, products, and the environment to laws, ethics, political systems, beliefs, values, fears, and hopes translated into tangible manifestations and serve as fragments of a different reality. The authors suggest that developing scenarios that integrate philosophical, social, and technological elements can add a further layer of credibility. Nonetheless, it can be challenging to instrumentalize speculative elaboration in a way that offers additional experiments and procedures [14]. [15] present a three-step process to consider in any speculative and critical design: 1. Establish a context for discussion: Most speculative design scenarios center on a particular subject. Typically, the topics deal with moral dilemmas brought on by problems, in this case, those involving new technology or social norms. 2. Idealize, identify problems, and construct a scenario. Using the “what if” technique, which is widely used to brainstorm and imagine solutions to issues that serve as the main idea of a scenario. 3. Materialize the scenario: This entails turning the scenario into stories, things, or a combination of both. To achieve some level of veracity, this must be elaborate and filled with details.

490

M. A. Carvalho et al.

According to [15], since the goal of speculative design is to elicit behaviors and thoughts that encourage people to reflect on and reevaluate previously accepted norms, one must keep the scenarios open-ended, ambiguous, and even complex while adding additional provocations using black humor and satire. 2.2 Domestic Violence and Privacy The use of smart products is becoming more prevalent in modern smart homes. However, user privacy concerns are also increasing. Several studies have documented privacy and security concerns related to tracking smart speakers. These studies have noted that users have been trading off privacy for the convenience provided by smart speakers, which is a concerning trend (e.g., [16]). In addition to smart speakers, numerous studies have claimed that the use of social networks, text messages, and phone calls has led to widespread surveillance (e.g., [17−22]) With the emergence of new technologies that connect to devices, such as home security systems and Internet-connected home assistants, there is an increased risk of privacy breaches and ubiquitous surveillance [23]. Particularly concerning is the fact that such technologies make it easier for intimate partners to be subjected to surveillance, which can have serious implications for privacy and safety. Therefore, it is crucial to address these privacy concerns when designing and developing new smart products. By prioritizing user privacy and safety, designers can ensure that smart homes remain a safe and secure environment for all residents.

3 Speculative Scenarios There is an opportunity for strategies that enable and facilitate understanding of the complexities of privacy and security and measuring user perceptions using speculative design about the potential moral decisions resulting from a technological intervention that smart products may experience. It is possible to pose more precise and in-depth queries about the potential function of currently in-use technologies by combining design fiction, stories, and scenarios. It is also worth mentioning that the aim of these scenarios is not necessarily to predict or propose practical solutions but to encourage reflection, speculation, and debate about the potential consequences and ethical implications of emerging technologies, policies, or societal trends. By presenting provocative and thought-provoking visions, these scenarios aim to raise awareness, broaden perspectives, and inspire critical thinking about the impact of innovation on individuals, communities, and the world at large. Moreover, [24] highlighted that victim-survivors stressed how individualized and targeted the attacks were, and they said that while some of the examples they recalled may seem unremarkable to an outsider, they may have read as unsettling or threatening to them due to their prior experiences or perceptions. Therefore, it is important to create speculative scenarios based on experiences. As previously mentioned, the scenarios were formulated using open-ended questions from a questionnaire. Participant recruitment occurred through three distinct methods: face-to-face interactions, digital platforms, and electronic means. A combination of

Understanding the Methodological Value of Mapping

491

direct invitations, dissemination through digital platforms such as Facebook, LinkedIn, and Instagram, along with email outreach, was employed. This approach primarily targeted support services for victims of domestic and sexual violence, as well as specialized communities within the study’s generalist counseling and social assistance services sample. Ultimately, 309 valid responses were gathered for subsequent analysis. The sample encompassed individuals of varying ages, ranging from 15 to 81 years (M = 36.48; SD = 14.86), comprising 233 females (M = 37.17, SD = 14.91) and 75 males (M = 34.53, SD = 14.51), all of whom possessed at least one smart device. The open-ended questions pertained to digital abuse, specifically focusing on the misuse of technology related to location monitoring, and concerns regarding smart products. 3.1 Scenario 1: Stalkerware Application Thirteen respondents identified identical situations: device cloning, unauthorized cloning, or unauthorized access to smartphones. Some quotes from the respondents are as follows: “My daughter’s boyfriend cloned her cell phone”, “an ex-boyfriend has accessed my cell phone without my authorization and read all my conversations”, “my aunt’s husband cloned her phone to have access to her conversations “, “I heard situations told by other people saying that their partners often ask for location sharing”, “after an argument, my old friend got into my phone, accessed my Instagram, changed my data and pretended to be me, offending me and posting pictures of me as a child”. Speculative Scenario. Privacy concerns have reached new heights in a near-future setting, where technological advancements have led to more sophisticated digital tools. The story follows a couple, Sarah and Mark, and delves into the consequences of their actions. Sarah, a 30-year-old woman, is plagued by insecurities and jealousy in her relationship with Mark. They recently had a bitter fight due to her unfounded suspicions, but later that evening, they decide to reconcile and spend the night at Mark’s house. However, Sarah’s doubts persist, and she is still unsatisfied with the situation, finding it hard to believe Mark’s claims. While Mark peacefully sleeps beside her, Sarah contemplates her next move. Consumed by her insecurities, she surreptitiously unlocks Mark’s iPad using his fingerprint, taking advantage of his vulnerable state. Driven by her desire for answers, she installs a Stalkerware program, cunningly disguised as a harmless calculator app. Unbeknownst to Mark, this deceptive application grants Sarah complete access to his iPad and all the programs shared with his cellphone. The Stalkerware becomes an invisible intruder, silently collecting and transmitting vast amounts of private data, including Mark’s precise location, browsing history, photos, messages, call recordings, and even camera access. Over time, Sarah becomes an unwilling observer of Mark’s digital life. She watches his every move meticulously, growing more convinced of her initial suspicions. The information she gathers through the Stalkerware feeds her insecurities, eventually leading her to confront Mark with the evidence she has amassed. As Sarah confronts him with her accusations, Mark is initially stunned and perplexed. He is unaware of how Sarah could have accessed such detailed and recent information

492

M. A. Carvalho et al.

without him noticing any password changes or security breaches. A deep sense of anxiety begins to set in, undermining his trust in the digital realm and causing doubt about his actions and privacy. This speculative scenario serves as a cautionary tale, shedding light on the ethical implications of invasive technologies and the erosion of trust within relationships. It raises questions about the boundaries of privacy, the dangers of unchecked surveillance tools, and the psychological toll they can exact on individuals and their connections. This speculative scenario proposes a potential solution that reimagines how applications can support healthy relationships by emphasizing transparency, consent, and trust-building. However, it is essential to approach the design and development of such applications responsibly, considering the potential ethical concerns and legal implications and always prioritizing user privacy and well-being.

3.2 Scenario 2: Intelligent Lights Eighteen respondents mentioned the presence of intelligent lighting in homes. Additionally, [25] provides instances of smart home-related domestic violence. This scenario consequently developed from these implications. Speculative Scenario. In a near-future setting, a three-person family—Helen, John, and their child—moves into a modern smart home, expecting the convenience and simplicity that automation promises. Life becomes more streamlined and efficient with everything controlled automatically, including the lights. Helen, who primarily works from home, finds smart home technology immensely helpful, easing her daily tasks. Through a shared application accessible to all family members, they can select colors and control the lights even when away from home. However, beneath the surface of apparent convenience, a darker dynamic unfolds within the household. One morning, Helen and John become embroiled in a heated argument over bathroom usage, leading to John’s tardiness for work. Holding onto their frustrations, an unhealthy power dynamic emerges, impacting their mental well-being. Exploiting the smart home features to assert control and inflict emotional distress, John decides to use the automated lights to annoy Helen. When Helen finds herself alone at home, the lights unexpectedly switch off. Initially, she perceives it as odd and promptly switches them back on, assuming a temporary glitch. However, the lights repeatedly go dark after a few minutes, leaving Helen bewildered and suspicious about a potential issue with the house. Seeking solace and hoping for resolution, Helen confides in John, sharing the perplexing light situation through a message. Unfortunately, instead of understanding, John responds with dismissiveness and blame, exacerbating the distress. He belittles Helen, asserting that she lacks comprehension of technology and the situation, shifting the blame onto her and exacerbating her helplessness. Realizing the deeply concerning nature of the situation, Helen begins to recognize the implications of the mental abuse she is enduring. Determined to regain control over her well-being and protect her privacy, she takes decisive action. Helen explores the smart home application’s privacy settings and security features, seeking a solution that will safeguard her autonomy and mental health.

Understanding the Methodological Value of Mapping

493

This speculative scenario delves into the deeply troubling dynamics of domestic abuse and mental well-being within a smart home context. The solution proposed centers around privacy and ethical concerns, emphasizing the importance of empowering individuals to safeguard their autonomy, establish boundaries, and seek support when faced with abusive situations. It underscores the need for awareness, empathy, and ethical design practices to ensure that smart homes promote the well-being and privacy of their occupants.

3.3 Scenario 3: AirTag The majority of the respondents (n = 48) mentioned ways of tracking. Some quotes from the participants are: “My daughter’s husband cloned her cell phone to keep an eye on her and put a chip in her car to track her and know where she is “, “the boyfriend of a close friend of mine bought a GPS tracker on the Internet to have control over where she was going”, “My husband offered me a new car in order to control me. Track all my steps through the GPS that is connected to his cell phone”. Speculative Scenario. In a contemporary setting, an ex-couple, who had ended their two-year relationship on amicable terms, finds themselves entangled in a problematic situation. Alice, a social individual who frequently ventures out to meet friends, visit coffee shops, and go shopping, experiences a growing sense of unease as her ex-partner conveniently appears in various parts of the city. Initially dismissing it as mere coincidence, the frequency of these encounters raises suspicions of stalking, leaving Alice deeply concerned for her safety. Determined to unravel the truth, Alice embarks on an online investigation, exploring possible tracking methods that could explain her ex’s ability to locate her with such precision. As she delves deeper into her research, she stumbles upon the idea that her iPhone might be compromised. However, upon careful analysis and consultation with experts, it becomes evident that her phone has not been hacked or tampered with. To her surprise, even when she purposely leaves her phone at home and relies solely on her Apple Watch, her ex continues to appear unexpectedly. After enduring weeks of tense situations, Alice confides in a long-lost friend who works at her workplace. During their conversation, they make a significant realization – these encounters consistently happen when they are driving somewhere. Intrigued by this pattern, they searched Alice’s car for suspicious devices. Their search unveils an alarming discovery – a cleverly concealed AirTag discreetly attached to the car’s license plate. This revelation takes aback both Alice and her friend. They realized that since Alice does not use an iOS device, she received no notifications or alerts regarding the tracking device following her. Armed with this newfound knowledge, fear, and apprehension consume their hearts. Alice feels utterly powerless to override her ex-partner’s actions, concerned for her safety and uncertain about future episodes. In this speculative scenario, we witness the disturbing nature of stalking and the potential misuse of tracking technology. It highlights the need for vigilance and personal security awareness, even in seemingly amicable breakups. The scenario also raises awareness about the importance of privacy measures and the ethical considerations surrounding tracking devices. It emphasizes the necessity of comprehensive security

494

M. A. Carvalho et al.

practices to protect individuals from potential harm, ensuring their well-being in an increasingly interconnected world. Given that it is as frequent as or even more frequent than individual experiences of intimate partner violence and pertains to the reporting of offline experiences, data protection intended for this smart product must anticipate technological victimization by intimate partners.

3.4 Scenario 4: iRobot Roomba Twenty respondents indicated they regularly clean their homes with the assistance of autonomous robot vacuum cleaners. In this regard, the final scenario includes an iRobot Roomba. Speculative Scenario. In a domestic setting, Ana and Jack navigate the complexities of their relationship while juggling the demands of work and raising their two young children. Having recently returned to work, Ana struggles to balance her time between her career and the responsibilities of motherhood. Despite her best efforts to prioritize herself and her work, Jack becomes increasingly demanding, constantly pressuring Ana about her absence and making her life more difficult. Feeling overwhelmed by Jack’s persistent demands, Ana occasionally leaves work earlier to address their relationship issues and spend more time with her family. On one such day, when the children are at school and will not return until late afternoon, Ana seizes the opportunity to catch up on some much-needed rest. To avoid causing alarm, she informs Jack of her decision to leave work early and take a nap, hoping it will alleviate some of the tension between them. As Ana drifts off to sleep, she is suddenly startled awake by the sound of the iRobot, an automated home device, unexpectedly running. Confused and fearful, she quickly turns it off, trying to comprehend what happened. It dawns on her that she has been experiencing instances where things are activated without her consent, and she suspects that she may have unintentionally triggered certain actions due to forgetfulness or oversight. Ana’s suspicions are confirmed when she discovers that her husband, Jack, had used the application to reactivate the iRobot just ten minutes after she turned it off. Realizing that he has succeeded in disturbing her sleep and maintaining control over her, Ana is filled with a mix of fear, confusion, and frustration. This speculative scenario delves into the power dynamics of a domestic relationship, revealing the manipulation and psychological distress experienced by Ana due to her husband’s behavior. It highlights the significance of recognizing and addressing instances of domestic violence. The scenario emphasizes the need for support systems, open communication, and resources to empower individuals like Ana, enabling them to break free from toxic cycles and create a safe environment for themselves and their children. It also raises awareness about the role of technology in exacerbating power imbalances and suggests implementing features that promote transparency, consent, and joint decisionmaking within smart home devices. By prioritizing ethical considerations, technology can contribute to fostering healthier relationships and protecting personal boundaries.

Understanding the Methodological Value of Mapping

495

4 Conclusion The paper examined the methodological value of mapping alternative fiction scenarios during the ideation phase, focusing on privacy concerns within the context of domestic violence. Four speculative scenarios were created to highlight the importance of considering unpleasant situations such as domestic violence in the design process. These scenarios demonstrate how a product can be used to harm victims, causing them to doubt themselves and experience low mental well-being. While the User-Centered Design approach typically addresses user pain points, it may not fully encompass more complex dimensions such as domestic violence. Including alternative fiction scenarios allows designers to gain a deeper understanding of the potential challenges and ethical implications associated with emerging technologies in relation to domestic violence. This understanding can inform the development of more inclusive and responsible design solutions that prioritize user safety and well-being. In conclusion, designers hold a critical responsibility in developing innovative products, systems, and services that cater to customer needs and enhance convenience. They must draw upon their understanding of user requirements and available technologies to achieve this goal. However, it is equally important for designers to consider the potential misuse or incorrect use of these technologies rather than solely focusing on their advantages. The speculative scenarios presented in this paper serve as valuable tools for highlighting the negative implications and privacy concerns associated with certain smart products within the context of domestic violence. As a future direction, conducting user testing to evaluate the potential privacy issues arising from these scenarios would provide valuable insights and contribute to a more comprehensive understanding of the challenges and ethical considerations involved. By integrating such insights into the design process, designers can foster the development of more responsible and usercentric solutions that prioritize privacy, safety, and well-being. Furthermore, it is worth noting that these scenarios consistently depict situations in which women adopt the role of victims, while men are consistently portrayed as aggressors. Given that intimate partner violence is a health concern affecting individuals of diverse genders and social backgrounds [26], it is worth considering that this conventional representation may be constraining, as the identified issue has the potential to affect relationships spanning a wide range of gender identities, generations, and familial connections, including those involving grandmothers, fathers, grandchildren, stepsons, and various other associations. Consequently, future research should broaden its scope to encompass a diverse range of potential victims. Acknowledgments. The study was supported by UNIDCOM under a grant from the Fundação para a Ciência e Tecnologia (FCT) No. UIDB/00711/2020 attributed to UNIDCOM – Unidade de Investigação em Design e Comunicação, Lisbon, Portugal.

References 1. Intelligence M (2020) Smart Homes Market Size & Share Analysis - Growth Trends & Forecasts (2023–2028). https://www.mordorintelligence.com/industry-reports/global-smarthomes-market-industry?gclid=CIjd6MXjydYCFYYDKgod4ZQFaw

496

M. A. Carvalho et al.

2. Yardley E (2021) Technology-facilitated domestic abuse in political economy: a new theoretical framework. Violence Against Women 27(10):1479–1498 3. Sovacool B, Furszyfer-Del Rio DD, Martiskainen M (2021) Can prosuming become perilous? Exploring systems of control and domestic abuse in the smart homes of the future. Front Energy Res 9:765817 4. Dragiewicz M et al (2018) Technology facilitated coercive control: domestic violence and the competing roles of digital media platforms. Fem Media Stud 18(4):609–625 5. Snook C, Safe L (2017) Tech vs Abuse: Research Findings. Comic Relief. http://media.wix. com/ugd/f86f13_366b6514c8fc4e9488fc15edf2148d52.pdf 6. Alshehri A, Salem MB, Ding L (2020) Are smart home devices abandoning IPV victims?. In: 2020 IEEE 19th international conference on trust, security and privacy in computing and communications (TrustCom), IEEE, 1368–1375 7. Alves Carvalho M (2023) Violência doméstica e tecnologia: análise de experiências e possíveis cenários em relação a smart products. [Master’s thesis, IADE, Universidade Europeia]. https://comum.rcaap.pt/handle/10400.26/44765 8. Dunne A, Raby F (2001) Design noir: the secret life of electronic objects. Springer Science & Business Media 9. Malpass M (2016) Critical design practice: theoretical perspectives and methods of engagement. Des J 19(3):473–489 10. Dunne A, Raby F (2013) Speculative everything: design, fiction, and social dreaming. MIT Press 11. Wong RY, Khovanskaya V (2018) Speculative design in HCI: from corporate imaginations to critical orientations (pp. 175–202). Springer International Publishing 12. Voros J (2001) A primer on futures studies, foresight and the use of scenarios. Prospect: Foresight Bull 6(1) 13. Tanenbaum TJ, Tanenbaum K, Wakkary R (2012) Steampunk as design fiction. In Proceedings of the SIGCHI conference on human factors in computing systems, 1583–1592 14. Bleecker J (2015) Design fiction: a short essay on design, science, fact and fiction. Accessed 16 June 2009 15. Johannessen LK, Keitsch MM, Pettersen IN (2019) Speculative and critical design—features, methods, and practices. In: Proceedings of the design society: international conference on engineering design, vol 1, no 1, pp 1623–1632. Cambridge University Press 16. Lau J, Zimmerman B, Schaub F (2018) Alexa, are you listening? Privacy perceptions, concerns and privacy-seeking behaviors with smart speakers. In: Proceedings of the ACM on humancomputer interaction, 2(CSCW), 1–31 17. Southworth C, Finn J, Dawson S, Fraser C, Tucker S (2007) Intimate partner violence, technology, and stalking. Violence Against Women 13(8):842–856 18. Finn J, Atkinson T (2009) Promoting the safe and strategic use of technology for victims of intimate partner violence: evaluation of the technology safety project. J Family Violence 24:53–59 19. Dimond JP, Fiesler C, Bruckman AS (2011) Domestic violence and information communication technologies. Interact Comput 23(5):413–421 20. Belknap J, Larson DL, Abrams ML, Garcia C, Anderson-Block K (2012) Types of intimate partner homicides committed by women: self-defense, proxy/retaliation, and sexual proprietariness. Homicide Stud 16(4):359–379 21. Zaidi T, Saxena V (2015) Modeling for minimization of loneliness of people on the social networking websites. J Psychiatry 18(257):2 22. Woodlock D (2017) The abuse of technology in domestic violence and stalking. Violence Against Women 23(5):584–602

Understanding the Methodological Value of Mapping

497

23. Messing J, Bagwell-Gray M, Brown ML, Kappas A, Durfee A (2020) Intersections of stalking and technology-based abuse: emerging definitions, conceptualization, and measurement. J Family Violence 35(7):693–704 24. Harris B, Woodlock D (2022) Spaceless violence: women’s experiences of technologyfacilitated domestic violence in regional, rural and remote areas. Trends Issues Crime Crim. Justice [Electron. Resour] 644:1–14 25. Moog EP (2019) Designing against Domestic Violence by Eva PenzeyMoog. https://www. youtube.com/watch?v=P64RHUSRtSA&t=54s 26. Scott-Storey K et al (2023) What about the men? A critical review of men’s experiences of intimate partner violence. Trauma Violence Abuse 24(2):858–872

“Nothing in This House Is by Chance”: Design Ethnography for the Well-Being of Children with Autism Emanuele Dutton1(B)

, Paula Trigueiros1,2

, and Alison Burrows2

1 School of Architecture, Art and Design, University of Minho, Guimarães, Portugal

[email protected]

2 Lab2PT, University of Minho, Guimarães, Portugal

{paula.trigueiros,alison.burrows}@eaad.uminho.pt

Abstract. The aim of this paper is to explore how design can contribute to fostering the well-being of children with Autism Spectrum Disorder (ASD) in their home environment. We report on a single case study that involved an in-depth study of the lived experiences of a child with ASD and their family at home. This qualitative and exploratory research followed the principles of design ethnography, favouring naturalistic data collection that occurred in real-world settings. The results show that technical standards, guidelines, and best practices for interior spaces are important and often appropriate for general purpose. However, ASD manifests differently for different people and this means customised solutions are required. Designers do not always have the tools and time needed to devise solutions that are not only functional, but also suited to the individual needs of a person with ASD. The research described here thus contributes to informing best practices for designing home environments that are appropriate, safe, comfortable and, above all, that respect the privacy, individuality, and preferences of children with ASD and of their families. Keywords: Interior design · ASD · Home environment · Design for inclusion · Family

1 Introduction According to the United Nations, it is believed that there are more than seventy million people with autism in the world [1]. There is a high rate of people diagnosed with Autism Spectrum Disorder (ASD) worldwide and a general lack of understanding of the condition has a great impact on individuals, their families, and communities that live with ASD. There is no known cause for the increase in diagnoses in recent years [2], but it has been speculated that it may be due to wider diagnosis criteria since 2012 (now understood as a spectrum) combined with greater knowledge and visibility of ASD within society. A diagnosis is usually made in childhood, at around two or three years of age [3], which is when the most common signs tend to appear. There is a wide range of social, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 498–507, 2024. https://doi.org/10.1007/978-3-031-47281-7_41

“Nothing in This House Is by Chance”: Design Ethnography

499

behavioural, and communication characteristics associated with autism. Additionally, there are unusual responses to sensory experiences and resistance to changes in the environment or in daily routines [4]. This means children with ASD can have more difficulty adapting to certain environments. Here, design can have an important role to play in terms of developing an appropriate sensory environment and mediating its relationship with ASD behaviour [5–7]. Designing for children with ASD benefits from trying to see things through their eyes, to think like they do, by including them in the design process [7]. We argue that design is a tool capable of creating solutions that suit each person by understanding their unique needs and desires, and so informing meaningful interventions. Homes are places for living, but they are also spaces for relationships and development. It has been said that “there is no place like home” [8] because, even though it is embodied in a physical and perhaps not unique structure, the home has been filled by its occupants with feelings and attachments that infuse it with meaning for them. Here, the notions of Place and Placelessness by Relph [9] are particularly relevant. Place has a personal identity that we have shaped – it holds meaning for us and evokes certain emotions. Placelessness refers to a lack of a unique identity, somewhere impersonal and without meaning. Designing only according to recommendations and guidelines for accessibility implies a risk of creating placelessness. We therefore identified an opportunity to explore how design can contribute to promoting the well-being of children with ASD in their home environment, by investigating a real-world case that demonstrates appropriate and meaningful solutions. The contribution of this paper lies in a more sensitive and social perspective of the home for children with ASD and their families. The following section provides a review of relevant literature on three key themes for this research: interior design, the home, and autism. We then present the methods and results, which we discuss with a view to offering recommendations for designing meaningful home environments for families with children with ASD. The conclusion identifies opportunities for future work on this topic.

2 Background: Interior Design, Home and Autism The American Society of Interior Designers (ASID) views interior design as an activity capable of impacting the human experience and influencing the way people live [10]. In fact, ASID states that, when designing interior spaces, a designer is directly contributing to enhancing the health and well-being of its occupants. Further highlighting the significant responsibility of interior designers, other authors have noted that the design of an interior space has the potential to influence human behaviour, both positively and negatively [11]. Hirano (cited by [12]) notes that design not only involves the aesthetics of tangible and interesting objects, but is characterized by intangible concepts that aim to improve society. Recent years have seen an increase in research on how physical aspects of the environment can influence the development of children with ASD. Many authors and researchers have looked to the home as an important focus for autism-friendly environments [13– 17]. Even though they have approached the issue in different ways, there is consensus

500

E. Dutton et al.

that the built environment can be designed to respond to the needs of people with ASD, specifically in terms of sensory stimuli, thus creating spaces of relaxation and stability. Interior design can be a decisive experiential factor when the focus is on the built environment, given that a space can exert both a direct and indirect influence. It is necessary to use processes that focus on obtaining rich knowledge about each individual through a personalized approach, to understand what the real needs and interests are and how to create solutions that can contribute to minimizing personal challenges. Any approach to designing autism-friendly home environments also needs to consider the comfort and well-being of all other long-term occupants [7, 16]. A greater social awareness of people with ASD means that many professionals such as interior designers are being increasingly requested to design, adapt, and build homes that best meet the needs and desires of these individuals. Designing for accessibility in architecture is not only a challenge but also a duty, because if a house does not meet the needs of its inhabitants, it can never be considered a Home. The home is more than just a place where one lives: “It is a ‘territory of meaning’, a place where pleasure, affect and aesthetics are deeply interwoven with the functional and utilitarian dimensions” [18, p. 1]. Although it is typically embodied by a physical structure, it is much more than that: “Home is a series of feelings and attachments” [8, p. 10]. There are not currently many examples of design-related projects focusing on people with ASD in the literature. Magda Mostafa is one of the world’s most prominent researchers and practitioners in the field of architecture for people with ASD, with projects that include the Advanced Special Needs Education Center [19]. Another noteworthy example of architecture for people with ASD is Sweetwater Spectrum [20], a permanent housing solution for individuals with ASD and their support staff. From a domestic perspective, design researcher and fashion and product designer Maureen Selina Laverty used sensory ethnography in what she termed wardrobe interviews with people with ASD [21].

3 Methodology This research was based on a qualitative and exploratory case study, focused on a child with ASD and their family. Pursuant to design ethnography principles [22], several methods and techniques were used, such as non-participant observation, field notes, unstructured interviews, and photographs of the home and of interactive devices used by the child with ASD. This allowed a deep and naturalistic understanding of how a home environment is experienced by all its occupants. The rationale for this methodology was informed by the notion that autism is a spectrum that encompasses a wide range of symptoms types and levels of severity, which differ from person to person [23]. This is in line with recommendations in the literature on conducting research on ASD and with people who have ASD [24]. Although the study protocol envisaged involving just one family with a child with ASD, participant recruitment was a particularly challenging aspect of this research. Many recruitment strategies were applied and the one that proved successful in the end was to approach a local Social Solidarity Cooperative (SSC) with expertise in ASD. The president of that SSC held an initial meeting with the researcher to better understand the

“Nothing in This House Is by Chance”: Design Ethnography

501

purpose of the research and to establish boundaries that would safeguard the interests of all stakeholders who may become involved as participants. Beyond the child with ASD and their family, these included professionals from the SSC who provided various forms of support. In this sense, the president of the SSC acted as Gatekeeper [25] as he mediated access to the study participants and advocated for their interests throughout. All participants gave their informed consent to take part in this research and any photographs taken were previously agreed on with the participants. The researcher took particular care to ensure that the children involved in this research were comfortable with her presence and willing to be part of the study – this was a crucial factor in the parents’ decision to participate. 3.1 Data Collection Data were collected through various methods and techniques, namely non-participant observation, unstructured interviews, a guided tour of the home, field notes, photographs of the home environment and of some of the child’s personal objects, as well as the study of the house’s architectural plans (Fig. 1). The researcher kept a fieldwork journal in which she recorded her objective as well as her personal observations.

Fig. 1. Data collection methods and techniques

Data collection took place over multiple visits to the home where the child with ASD and his family live. The non-participant field observation focused primarily on five behaviour therapy sessions carried out in the home, which followed an established structure and process. During these sessions, the researcher observed the child’s behaviour and his interaction with the home’s interior and exterior spaces, as well as interactions with people and everyday objects, through certain activities encouraged by the therapist. The sessions lasted between two and three hours, with the activities and duration of each session being previously arranged by the family and the SSC. In parallel with the observation, the researcher conducted informal interviews with the parents, the carer,

502

E. Dutton et al.

and the therapist, which allowed a deeper understanding and facilitated the mapping of this case. 3.2 Data Analysis Data analysis occurred alongside data collection, with transcription and preliminary analysis of field notes happening soon after each home visit. A fieldwork journal containing detailed descriptions of the home environment, domestic routines, interactions that took place, interpersonal relations, and the researcher’s initial impressions was written up in a digital file for rapid access. This facilitated early interpretation of the data, which in turn guided the planning and focus from one home visit to the next. Data analysis comprised two approaches: a descriptive analysis and an interpretive analysis (Fig. 2).

Fig. 2. Approaches to analysis

The descriptive analysis contained a summary of what had been objectively observed, such as: who was involved, how often they interacted, and what they did in a given space. The results from this approach are facts and there is no interpretation by the researcher. For the interpretive analysis, the digital field notes were shared amongst the authors for independent coding, which were subsequently discussed in data analysis meetings to identify common themes. This process observed guidelines for robust qualitative data analysis [26] and thus ensured the quality of the research results.

4 Results The results include a brief characterization of the main participant in this study, which are presented in Table 1. As the main focus of this research was the child with ASD, we gave him the pseudonym Francisco; all other participants were attributed the label that identifies their relationship to Francisco. As the study was carried out in a domestic environment, it was important to describe and contextualize the space as a whole. A family of four lived in the house at the time:

“Nothing in This House Is by Chance”: Design Ethnography

503

Table 1. Participant characteristics Participant

Description

Francisco

13 years old Diagnosed with non-verbal ASD

Mother

Full-time employment Actively oversees household management

Father

Used to be a stay-at-home dad to care for Francisco Follows the Mother’s lead regarding household management

Twin brother

Diagnosed with ADHD Caring and protective towards Francisco

Carer

Has worked with the family daily since 2019 Involved in all the children’s routines and activities Manages previously established household tasks

Behavioural therapist

Assists Francisco in learning to work with various materials Seeks to help Francisco develop general life skills

Gatekeeper

President of the SSC that provides ASD support Advocate for the family’s rights and well-being during the study Mediated the researcher’s access to the family and research setting

Francisco, his twin brother, and their parents. Other people who were regularly present in the home included a carer, who was there every day, and a behavioural therapist, who visited up to five times a week according to a flexible and agreed schedule. The family bought the house while it was still being built and so was able to influence certain aspects of its interior design. The house was not designed specifically for the child with ASD, although it did consider his safety as well as the well-being of all. There was an initial project and it was adapted to Francisco’s routine requirements based on the family’s experience and knowledge. Visiting this home, it was evident that it is not necessary to design a special house for this or that person. However, it is possible to make adjustments, even if minimal, in order to adapt a house to people’s individual needs, and it is very important for architects and designers to think from this perspective. Nothing in this house is by chance. Everything was thought through: what is there and what is not there. (Mother) This home had its own identity and, despite Francisco’s needs informing much of the family’s routine, they chose not to have this represented visually. Every space had a role: resting, working, learning, cooking and other typical roles of domestic spaces. Yet the same space performed different functions and was differently occupied, depending on the day and time of day. There was a clear relationship between the spaces, the routines or activities taking place there, and the household occupants involved. Figure 3 shows what spaces of the home were used for various purposes. One of the strategies used to facilitate multiple uses of a space was through the choice and arrangement of furniture. For example, the large TV cabinet in the living room stored all the children’s games, which meant the room could rapidly gain a new

504

E. Dutton et al.

Fig. 3. Spaces and purposes of the home

identity as a space for playing. A particular interesting space was the garden, traditionally a leisure space, but also used by the family as a place for learning and working. Some of Francisco’s occupational therapy and exercises were conducted in the garden because he was often rewarded for his effort by having some play time on the grass or the trampoline, both of which he enjoyed immensely. The parents were keenly aware of the importance of appropriate sensory stimulation for Francisco and invested a lot in objects of this kind. In addition, there were some electronic tools aimed at children with ASD, which can facilitate and stimulate communication and interaction skills.

5 Discussion and Recommendations This case study involved a family that had been able to influence and adapt the interior spaces of a house while it was still in construction. Moreover, their many years of lived experience of having a child with ASD had equipped them with a strong understanding of effective strategies for ASD and family living. We therefore argue that best practices in terms of domestic interior design for a child with ASD can be confidently inferred from this case study. By reflecting critically on this research project, its results, and the relevant literature, we offer some recommendations for the design of home interiors that are appropriate and desirable for children with ASD and their families. Existing legislation and guidelines serve their purpose, but interior designers need to be critical and use good design practices only as a starting point to create suitable domestic spaces. In this case study, the family followed such norms and guidelines, but they also made adaptations to their living spaces based on their own experiences and Francisco’s needs. This is echoed in the literature by several authors, who offer recommendations for ASD-friendly interior design [7, 15, 16, 27, 28]. An interior designer who is creating domestic spaces for children with ASD must work closely with the client family and other people who regularly use the home,

“Nothing in This House Is by Chance”: Design Ethnography

505

and involve them in the design process. Findings from our study support the position of Mostafa [7] regarding the importance of people in the design process. The house in this case study had been designed to be inclusive of the needs of all its occupants, so that, in addition to being appropriate for a child with ASD, it was also a home for the entire family. Clients may have just as much or even more expertise than a designer regarding their needs for a space for living and should be actively involved in solutions for them. It is important that a designer creates spaces that are dynamic and adaptable to different functions. The interior spaces of a family home have multiple purposes. It is the Home, but also a space for work and leisure for a variety of people (residents and non-residents). Although some general recommendations have been put forward [19–21], there is no single standard for designing appropriate spaces for children with ASD and their families. This means that it is essential that designers are aware of this plurality of situations and consider them in the design process. Interior designers should understand the family’s reality and family routines in order to create spaces that promote well-being and facilitate the routines of children with ASD and their family. A particularly interesting finding from our study resonates with a call for applying qualitative and interpretive research approaches to this issue [28, 29]. The ethnographic methodology used in this case study allowed the researcher to gain an in-depth understanding of the habits and needs of the child with ASD and his family in relation to their home. Such participatory and ethnographic approaches are necessary to counter the generalization of domestic interior design and create true Places [9] for living. Interior designers must create domestic spaces where environmental sensory stimuli are adequate for the child with ASD. The Advanced Special Needs Education Center [19], the Sweetwater Spectrum [20], and Sensory Nourishment [21] are three projects that demonstrate a fundamental preoccupation with matching sensory stimuli from the built environment and from products to the needs of people with ASD, based on the knowledge that poor sensory experiences can be disturbing or even disabling for them. Appropriate stimuli, on the other hand, have the power to comfort and foster well-being. The slow but steady increase in designers seeking to include the people for whom they are designing in the design process will in turn mean a better understanding of how to moderate sensory stimuli for people with ASD. There is an opportunity for design, as a discipline, to sift through the dense information collated in standards, guidelines, and scientific papers, and create tools that support families with children with ASD. As Hirano (cited by [12]) says, design has a calling to improve society. The family involved in this research was very knowledgeable and experienced in handling ASD issues. Findings from this case study can be leveraged to impress upon designers the need to develop tools to help other families who are not yet at this stage, or who still do not know what is important, thus contributing to a fairer and more inclusive society.

506

E. Dutton et al.

6 Conclusion This paper sought to explore the experiences of home of a child with ASD and his family, taking a design ethnography approach. This approach was deemed particularly suited to an in-depth and naturalistic study of a child with ASD. The researcher immersed herself in an unfamiliar domestic routine to gain a holistic view of how a child with ASD uses and relates to his home, while also acknowledging that the home is a dynamic space that is shared by multiple people who each have their own needs and desires. In this sense, design must be inclusive of all the people who regularly live in the home. By studying a family who has lived with ASD for a long time and who has a robust system in place for their son with ASD, we were able to identify some best practices that informed six recommendations for designing domestic interior spaces for similar family contexts. We believe that this paper makes a timely and necessary contribution towards encouraging more sensitive and inclusive research approaches in interior design, in particular for children with ASD. Future research could build on this work, both in terms of scale and in terms of related living contexts such as holiday dwellings and hotels. Collectively, these could serve as a basis for the development of policies to support designers in creating interior living environments that are inclusive and meaningful for children with ASD and their families. Acknowledgements. The authors are grateful to all the people who were involved in some way for their support in carrying out this research, namely the Family, the Carer, the Behavioural Therapist, and the Gatekeeper. This initiative was supported through the Multiannual Funding of the Landscape, Heritage, and Territory Laboratory (Lab2PT), Ref. UID/04509/2020, financed by national funds (PIDDAC) through the FCT/MCTES.

References 1. United Nations. https://unric.org/pt/. Accessed 31 May 2023 2. Rios C, Ortega F, Zorzanelli R, Nascimento LF (2015) Da invisibilidade à epidemia: a construção narrativa do autismo na mídia impressa brasileira. Interface (Botucatu) 19:325–336. https://doi.org/10.1590/1807-57622014.0146 3. Schmidt C (2017) Autismo, educação e transdisciplinaridade. Papirus Editora 4. Khare R, Mullick A (2013) Research tools to learn about the needs of children with autism. In: Proceedings of the human factors and ergonomics society annual meeting, pp 506–510. SAGE Publications Sage CA: Los Angeles, CA 5. Grandin T, Panek R (2015) O Cérebro Autista Pensando Através do Espectro. Editora Record, Rio de Janeiro 6. Pallasmaa J (2011) Os olhos da pele: a arquitetura e os sentidos. Bookman, Porto Alegre 7. Mostafa M (2008) An architecture for autism: concepts of design intervention for the autistic user. Archnet-IJAR : Int J Arch Res 2. https://doi.org/10.26687/archnet-ijar.v2i1.182 8. Blunt A, Dowling R (2006) Home. Routledge, New York NY 9. Relph EC (1976) Place and Placelessness. Pion 10. American Society of Interior Designers | ASID. https://www.asid.org/6422d13c8ad0c. Accessed 28 Mar 2023 11. Barbosa PG, Rezende EJC (2020) O que é o Design de Interiores? Estudos em Design 28. https://doi.org/10.35522/eed.v28i1.885

“Nothing in This House Is by Chance”: Design Ethnography

507

12. Rosa V, da Silva R, Aymone, J (2018) Design inclusivo: processo de desenvolvimento de prancha de comunicação alternativa e aumentativa para crianças com transtorno do espectro do autismo utilizando realidade aumentada. Design e Tecnologia 8, 51. https://doi.org/10. 23972/det2018iss15pp51-67 13. Ahrentzen S, Steele K (2015) At home with autism: designing housing for the spectrum. Policy Press 14. Khare R, Mullick A (2009) Incorporating the behavioral dimension in designing inclusive learning environment for autism. Archnet-IJAR : Int. J Arch Res 3. https://doi.org/10.26687/ archnet-ijar.v3i3.189 15. Mostafa M (2014) Architecture for autism: autism aspectssTM in school design. Int J Arch Res: ArchNet-IJAR 8: 143–158. https://doi.org/10.26687/archnet-ijar.v8i1.314 16. Nagib W (2014) Toward a Therapeutic and Autism-Friendly Home Environment. https://mac sphere.mcmaster.ca/handle/11375/16447 17. Sánchez PA, Vázquez FS, Serrano LA, Sánchez PA, Vázquez FS, Serrano LA (2011) Autism and the built environment. IntechOpen. https://doi.org/10.5772/20200 18. Leonardi C, et al (2009) Knocking on elders’ door: investigating the functional and emotional geography of their domestic space. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 1703–1712. ACM, New York, NY, USA. https://doi.org/10. 1145/1518701.1518963 19. Mostafa M (2023) Advance Center for Autism. https://architectureforautism.wordpress.com/ treatment-centers-for-people-with-autistic-spectrum-disorders/advance-center-for-autism/. Accessed 31 May 2023 20. LMS Architects: Sweetwater Spectrum Community. https://www.lmsarch.com/projects/swe etwater-spectrum-community?f1=all. Accessed 31 May 2023 21. Selina Laverty M (2023) Sensory Nourishment. https://www.maureenselinalaverty.com. Accessed 31 May 2023 22. Pink S (2015) Doing sensory ethnography. Sage 23. Autism spectrum disorder. https://www.apa.org/topics/autism-spectrum-disorder. Accessed 19 May 2023 24. Solomon O, Bagatell N (2010) Introduction: autism: rethinking the possibilities. Ethos 38:1–7. https://doi.org/10.1111/j.1548-1352.2009.01078.x 25. Walsh D (1998) Doing ethnography. Res Soc Cult 217–232 (1998) 26. Miles MB, Huberman AM (1994) Qualitative data analysis: an expanded sourcebook. SAGE 27. D’Astous V, Manthorpe J, Lowton K, Glaser K (2016) Retracing the historical social care context of autism: a narrative overview. Br J Soc Work 46:789–807. https://doi.org/10.1093/ bjsw/bcu131 28. Kinnaer M, Baumers S, Heylighen A (2016) Autism-friendly architecture from the outside in and the inside out: an explorative study based on autobiographies of autistic people. J Hous and the Built Environ. 31:179–195. https://doi.org/10.1007/s10901-015-9451-8 29. Dos Santos A (2018) Seleção do Método de Pesquisa: guia para pós-graduandos em Design e áreas afins. Editora Insight

Educating the Future Digital Maturity Enablers. Learning from the Experience of the DC4DM LLabs Valentina Vezzani1(B) , Luís Ferreira1 , Marita Canina2 , and Eva Monestier2 1 University of Madeira, Funchal, Portugal

{vvezzani,luis.ferreira}@staff.uma.pt 2 Politecnico di Milano, Milan, Italy {marita.canina,eva.monestier}@polimi.it

Abstract. The ever-changing digitally enabled world we live in requires preparing young creatives and entrepreneurs with more suitable abilities and set of skills to face the challenges of ongoing digital transformations. The Erasmus + funded project DC4DM aims to implement a human-centered educational model to train Digital Maturity Enablers. This new type of professionals would be able to drive small and medium organizations towards their digital maturity: owning specific creative digital skills, they would be able to extract value from what the technological landscape offers and respond to the human needs through the principles of ethics and sustainability. This paper aims to describe the DC4DM educational model and reflect on the experience of three testing sessions, called DM Learning Labs, 10-days design-led workshops involving a great diversity of university students, diverse for country of origin and study background, start-ups, and several local mentors and stakeholders to co-design compelling future tech-scenarios. The paper will present the results from a preliminary study which aims to understand how attending a DM Learning Lab has impacted on the participants. Particularly, to understand which Digital Creative Ability had developed thanks to the LLab experience and which learnings were consequently applied in new creative and co-design contexts. The paper tries to highlight how much the DC4DM methodology has the potential to change the way to educate future professionals through cross-disciplinary and collaborative learning opportunities. Keywords: Digital Creativity · Digital Creative Abilities · Design Education

1 Preparing Future Professionals for the Complexity of Digital Transformation Era The world is undergoing profound transformations in an era characterized by globalization and rapid technological advancements [1]. The pervasive influence of digital technologies has permeated every aspect of people’s lives, from how they engage in sports and consume food to the way they conduct business and interact with others. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 508–519, 2024. https://doi.org/10.1007/978-3-031-47281-7_42

Educating the Future Digital Maturity Enablers

509

These technologies have become integral to human beings’ existence, necessitating a comprehensive understanding of their impact and the need for strategic guidance amidst this ongoing transformation. It is crucial to recognize that emerging technologies have far-reaching effects, often exceeding the initial expectations. As the renowned quote by Amara [2] suggests, we tend to overestimate their short-term impact while underestimating their long-term consequences. It is precisely this recognition that drives our focus on digital maturity. The advent of new digital technologies carries the power to reshape behaviors, reinvent social relationships, and redefine the very fabric of humanity. Klaus Schwab [3] refers to it not only as the Fourth Industrial Revolution but also as a sweeping “transformation of humankind”. Amidst this dynamic and uncertain landscape, individuals must acquire new skills to navigate the ongoing digital transformation. They must embrace creativity and possess the ability to responsibly harness new technologies, contributing to the attainment of digital maturity within organizations. In this context the overarching objective of the Erasmus + funded project Digital Creativity for Developing Digital Maturity Future Skills (DC4DM), is to implement, apply, and disseminate a human-centered educational model. This model aims to empower individuals with Digital Creative Abilities (DCAs), enabling them to navigate the future emergence of technologies. By equipping individuals with the necessary skills, the project seeks to drive progress, foster responsible use of new technologies, and propel organizations towards digital maturity. The exponential growth of digital technologies is having remarkable impacts on manifold sectors and particularly disrupting the education system. As a matter of fact, the advent of groundbreaking technologies such as Artificial Intelligence, Machine Learning, Internet of Things, and Virtual Reality are opening countless possibilities to enhance learning and teaching experiences [4]. Indeed, as noted by the World Economic Forum’s report on “Schools of the Future”, traditional education systems urge for new educational models to be fully able to adapt and meet the ever-changing economic and social needs raised by globalization and rapid technological advancements [5]. Digital transformation is not only altering mindsets, behaviors, and social attitudes but also revolutionizing the processes of creation and innovation [6]. In this scenario, human capital should be provided with opportunities to enrich their essential skillset – which includes, among others, complex problem solving, strategic and creative thinking, critical thinking, emotional intelligence, communication and negotiation, and relationship and network building abilities - with digital skills necessary to guide organizations and companies in their digital maturing path [7]. To allow such a skillset integration and empowerment and to guarantee a thriving transition towards maturity, people should undertake upskilling and reskilling processes that could be offered by innovative education models aimed at equipping the next generation of professionals with a mix of “hard” skills, such as technology design and data analysis, and “humancentric” skills, including cooperation, empathy, social awareness, and global citizenship, to empower learners to shape a future characterized by inclusivity and equality [8]. To remain competitive in the rapidly evolving technological landscape, Higher Education Institutions (HEIs) – the ones offering Engineering and Design curricula – should be prone to updating and rethinking their traditional educational models acknowledging

510

V. Vezzani et al.

that digital technologies are becoming our next invisible and ubiquitous nature. Also, by observing the digitally enhanced generation, it appears clear that it exhibits distinct learning preferences, new skills, and novel social behaviors and work styles. In this unfolding scenario, the ability to think creatively proves to be the crucial skill for navigating the digital transition. Creativity empowers individuals and organizations to achieve digital maturity, enabling them to adapt to the constantly changing digital landscape, collaborate with machines, and effectively lead technological advancements to drive innovation. This human ability enables people to manage and steer opportunities presented by technological evolution and encourages the development of a strategic approach in the adoption and application of such technologies. Being extremely impactful on humans’ evolution, digital transformation is also affecting the creative abilities of the digitally enhanced generation. Considering the main goal of developing, implementing, and disseminating an educational model to upskill individuals to guide digital transformation and achieve digital maturity, in the DC4DM project framework a design-oriented definition of digital creativity has been adopted. Thus, digital creativity can be defined as the human ability to create innovative and original digital outcomes leveraging the opportunities presented by digital technologies in a strategic and responsible way. This ability is enhanced by the interconnectedness of various human factors, which are shaped and influenced by the digital age and new technologies. Digital creativity is a multifaceted phenomenon that encompasses the cognitive, emotional, physical, and social dimensions of the human experience. All these dimensions are profoundly impacted by digital technologies which are calling for the redefinition of the essential skills, knowledge, and values required to activate and foster a creative process. When properly trained and empowered, these critical digital creativity factors - cognitive, attitudinal, emotional, and social factors inherent in human beings - can influence individual digital creativity and the outcomes of the creative process. These groups of factors, essential for digital creativity, are all necessary for individuals to generate a creative outcome. Therefore, the DC4DM project is grounded on the belief that it is paramount to support individuals in the development and empowerment of their digital creativity skillset and prepare them to guide organizations in their path towards the achievement of digital maturity. This entails training individuals in both the digital creativity factors and the creative design process, equipping them with the ability to create innovative digital outcomes by strategically harnessing emerging digital technologies. 1.1 Digital Maturity Enablers and a New Educational Model To foster growth and achieve digital maturity, organisations require the expertise of a digitally wise professional who can navigate the digital landscape, comprehend social, technological, economic, environmental, and political trends, and propose strategic pathways towards innovative and sustainable solutions. Within the DC4DM project, this emerging professional role is referred to as the Digital Maturity Enabler (DME). The DME is an individual equipped with specific creative digital skills, enabling them to extract value from the offerings of the technological landscape creatively. Moreover, they respond to human needs while upholding principles of ethics and sustainability. The Digital Maturity Enabler is typically an individual

Educating the Future Digital Maturity Enablers

511

with a design, engineering, or management background. They must possess the following characteristics: (a) A conscious understanding and application of new technologies, accompanied by a thorough awareness of their potential impact from social, ethical, economic, and environmental perspectives; (b) An inclination to share ideas and specialised knowledge within cross-functional teams; (c) A strong future-oriented mindset, employing foresight methods to identify and illustrate original future scenarios. In the continuously advancing digital landscape, the Digital Maturity Enabler demonstrates the necessary competencies to assist organizations in thriving and achieving maturity. These competencies include: (a) A comprehensive understanding of technology capabilities and the ability to design digital solutions using a human-centered design approach; (b) Proficiency in collaborating seamlessly within cross-functional teams, effectively communicating with individuals from diverse disciplinary backgrounds, and fostering a shared digitally minded culture; (c) The ability to navigate complexity and uncertainty, effectively addressing and overcoming challenges arising from intricate and unpredictable situations; (d) The capability to envision potential future scenarios and formulate long-term strategies by considering both opportunities and risks that may arise from digital technologies. The Digital Maturity Enabler employs a set of essential skills to guide ongoing digital transformation. These skills, known as Digital Creative Abilities (DCAs), encompass 24 distinct abilities that are categorized into cognitive, digital, cross-functional team, and strategic vision dimensions. Collecting the essential DCAs, such dimensions represent the building blocks that support the entire process (Fig. 1). The DCAs encompass a wide range of personal skills that span cognitive, social, and emotional domains. They also involve various forms of disciplinary and procedural knowledge, in addition to specific attitudes and values that provide guidance on how to utilize knowledge and skills effectively in the face of specific challenges.

Fig. 1. The 24 Digital Creativity Abilities currently identified to prepare the next generation of DM Enablers.

The major ambition of the DC4DM project, indeed, is to provide an educational model to support people in training and gaining the necessary DCAs to become DMEs, new professional figures able to master digital technologies potential in a sustainable, ethical, and strategic way.

512

V. Vezzani et al.

The DC4DM model (Fig. 2) proposes a holistic approach where each step in the process relies on specific DCAs to foster innovation and creativity within individuals and teams. The DC4DM model features a unique Design Futures process, which integrates both Futures Thinking (FT) and Design Thinking (DT) methods. Leveraging on the essential set of DCAs, the Design Futures process enables individuals to envision future scenarios and anticipate the potential applications of digital technologies. Moreover, it encourages to anticipate the implications these technologies may have on the societal, environmental, economic, and political spheres. As shown in Fig. 2, the DC4DM model showcases a first exploration phase, called ‘Explore’ and represented by an elongated diamond overlapping the Voros’ cone [9]. ‘Explore’ is the phase that aims to create practical perspectives on potential futures and opportunities. It involves analyzing social, technological, economic, and political indicators to shape and influence the future. This phase features two steps: Horizon Scanning and Visioning, which help identify emerging trends and construct possible scenarios to anticipate new design opportunities. ‘Generate’ phase aims to transform ideas into tangible solutions that align with the context and goals of the project. This phase involves generating innovative and technological ideas relevant to the envisioned future scenario. The ‘Generate’ phase includes two steps: Ideating and Prototyping. Upon encountering a design challenge and applying the DCAs throughout the process, the subsequent iterative Post-Process focuses on the cultivation of shared knowledge structures. These structures encompass process and tools, goals, expertise and abilities, and encourage effective team interactions. By continually adding value to individuals, organizations, and the whole system, this Post-Process phase ensures ongoing growth and improvement.

Fig. 2. Training of the Digital Maturity Enabler through the DC4DM model.

2 Testing the DC4DM Educational Model Through Three LLabs The DC4DM project also incorporates Learning Labs (LLabs), which are 10-days intensive design sessions tailored to provide participants with comprehensive training and mentoring towards the development of the key digital creativity competencies. LLabs offer a unique opportunity for students, SMEs, and startups to collaborate closely on

Educating the Future Digital Maturity Enablers

513

specific design challenges and envision preferable futures. These workshops gather participants with various levels of expertise proposing challenges concerning a selected theme. Within LLabs, participants have the chance to apply relevant DC4DM methods and tools, effectively harnessing their creativity and problem-solving skills and gain a future-oriented mindset. Furthermore, LLabs serve as an invaluable platform for educators to acquire firsthand experience in applying the DC4DM model. As facilitators, they gain insights into effective teaching practices that promote digital maturity. These immersive experiences equip educators with the necessary knowledge and skills to guide their students and foster their growth in the digital landscape. The DC4DM model has been developed between 2020 and 2023 through theoretical research and testing phases. To become a practical and usable learning methodology for educators, companies, and individuals who wish to become Digital Creativity Enablers, the DC4DM model and toolkit was tested three times through three LLabs. The three LLabs had in common the following objectives: (a) to allow to learn about digital transformation; (b) to guide participants to create a working environment that enhances the value of individuals, increasing personal motivation and fostering integration with other members of the group; (c) to identify the main parts and procedures of a training format to be applied for the organisation of new LLabs. The three LLabs were organised and run differently because the goal was to understand how flexible and adaptable each part and phase of the model is; how to involve SMEs and Startups along the process to bridge their needs with the training objectives; and finally, how to engage students coming from different disciplinary backgrounds along the whole process. LLab1: Feeding Madeira hosted by University of Madeira, in Funchal, Portugal, in July 2022. The emphasis was on designing regenerative and distributive food systems and the sustainable development of the island of Madeira thanks to the use and application of new digital technologies. Of the 10-days, the first three were dedicated to building the participants awareness on the local context and challenges related to sustainable development and food chain, and pre-process activities to start training the DCAs to apply through the Design Future Process. The remaining seven days was focused on co-designing future scenarios and possible creative digital tech-responses (Fig. 3). The participation of the startups along the co-design process at the beginning and end of the Design Future Process. LLab2: Mobility hosted by Télécom Saint-Etienne, in Saint-Etienne, France, in November 2022. The emphasis was on reflecting on the mobility of the future in a broad sense. The LLab2 program was basically as the LLab1 one. The difference this time was on the effort to support better teambuilding and so allow a more efficient design process and methods application. As in the case of LLab1, startups and local organisations were called to interact only at the beginning and end of the process. LLab3: Futuring Care hosted by Politecnico di Milano, in Milan, Italy, in February 2023. The main theme was divided into macro-themes and each one was associated to invited experts and mentors who had the role to guide the teams in understanding the challenges. The selected startups and small companies were invited to participate a codesign session so that the interaction between learners and real-world entrepreneurs would inform better the Design Future process. The program presented a new idea of

514

V. Vezzani et al.

mixing the Pre-Process learning activities across Process steps so that training the DCAs was more integrated within process and teamwork (Fig. 4).

Fig. 3. The LLab1 “Feeding Madeira” program, which the LLab2 “Mobility” followed similarly.

Fig. 4. The LLab3 “Futuring Care” program.

Educating the Future Digital Maturity Enablers

515

2.1 Feedback from the LLabs Participants All LLabs participants were interviewed informally during and just after each workshop. These conversations allowed the consortium to understand what was going well and what was less clear or critical to the quality and functionality of both the DC4DM educational model and DM Learning Format. This paper demonstrates our intention in continuing the collection of feedback by presenting the results from a preliminary study which aims to understand how attending a DM Learning Lab has impacted on each learner. In particular, we want to understand which Digital Creative Ability had developed thanks to the LLab experience and which learnings have been consequently applied in new creative and co-design contexts. An online questionnaire was sent to the whole group of LLabs participants. This paper focuses on the gathered answers from 16 students, of which 11 were from Design, 4 from Engineering, and 1 from Management. The following data regards some of the topics which we have tried to explore and are relevant to the aim of this paper. We collected both qualitative and quantitative data to assess the respondents’ perceptions of the LLab, utilizing a scale ranging from 0 to 5, where 5 indicates a highly positive response. Overall, the feedback received from all the respondents was very positive. 37,5% of the respondents rated the experience as four, representing a very positive perception. 31.3% of participants described having an overwhelming experience rating it as five. Another 31,3% of the participants rated the experience with a score of three, indicating a moderately positive response. When asked why the overall experience was positive, 37,5% of the participants replied to getting to know new people. For example, participants claimed that they enjoyed “getting to know people from abroad” and to “create incredible friendships”. Moreover, 31,3% reported enjoying the programmed LLab activities related to work and morning activities such as energizers and “morning walks”. Another 18,8% reported to have enjoyed the “sharing moments” and “experiences” with the others. 12,5% of the participants enjoyed interacting with the invited startups and SMEs. Nevertheless, we also asked the participants to share what is not that positive memory from the LLab. 38% of the respondents felt stressed or exhausted during the event. 31% reported issues such as the lack of time to complete activities and tasks properly and the language barrier that occurred during teamwork. Also, 13% reported that building teams to work with was an additional stress factor. Another 13% felt confused regarding the overall learning process. We asked participants what they reckon to have learned from the attended LLab. From the answers that we gathered, we observed significant personal growth as they are of the opinion that they learned considerably about Future Design Thinking; they felt “to be open-minded about new things” and learned how to, for example, “use AI to present (…) futuristic idea(s).” In addition, participants highlighted the development of personal skills such as the art of communicating. Indeed, communication was one of the challenging skills to master during LLab. Nevertheless, many of the participants said to have learnt “to be comfortable in talking in public” and “how to express (themselves) better to other people”.

516

V. Vezzani et al.

We asked the participants to reflect on which Digital Creative Abilities (DCA) they could reckon improved thanks to the LLab. Their responses were categorized based on the DCA dimensions: Cognition: Creative combination and imagination (62,5%), analytical and critical thinking (25%), translating knowledge and storytelling (50%), adopting different perspectives (75%), humanity problem solving (43,8%), and self-confidence and self-awareness (25%). Digital Proficiency: Digital literacy (25%), information literacy (18,8%), envisioning tech opportunities (56,3%), ethical and sustainable thinking (31,3%), digital collaboration (37,5%), and healthy use of technology (18,8%). Cross-Functional Teamwork: Enabling trust (37,5%), the propensity to share knowledge (31,3%), positive mood (50%), cooperative behaviour (50%), empathy (37,5%), and relationship management (37,5%). Strategic Vision: Coping with uncertainty, ambiguity, and risk (50%), futureoriented mindset (62,5%), sustainable development (25%), driving change and innovation (31,3%), impact the strategic management (18,8%), and envisioning future scenario (68,8%). We asked: In your opinion, what else would have helped to boost your ability in future thinking? Most participants (38%) highlighted that their ability to envision different possibilities would be greatly enhanced through exposure to an example of an alternative future; participants suggested to provide “a sample of an alternative future” and to experience “anything more tangible that made (them) feel the future”. Also, 38% of the participants reported other strategies to improve their ability to envision the future. These strategies include “disconnecting a bit more from the logistics of available technologies”, fostering better communication among group members, and other suggestions. On the other hand, 19% of the participants reported that giving them more time would have boosted their ability of future thinking; although “10 days is already a lot for a project” as commented by one of the participants, there was of general opinion that more time would be ideal. Participants were asked about their ability to apply the skills acquired during the LLab in a real-world environment, following a few months after their participation. 94% of participants reported that they indeed used the skills learned in a real-world scenario, such as in academic environments (master-level thesis writing, design courses), entrepreneurship courses, artificial intelligence for ideations of novel ideas, and among other scenarios. Lastly, participants were encouraged to reflect on the DCAs once again, but identifying those which they would feel to improve further to become a Digital Maturity Enabler. Following the same categorization as above: Cognition: Creative combination and imagination (31,3%), analytical and critical thinking (43,8%), translating knowledge and storytelling (62,5%), adopting different perspectives (31,3%), humanity problem solving (25%), self-confidence and self-awareness (31,3%). Digital Proficiency: Digital literacy (50%), information literacy (31,3%), envisioning tech opportunities (56,3%), ethical and sustainable thinking (31,3%), digital collaboration (37,5%), and healthy use of technology (43,8%).

Educating the Future Digital Maturity Enablers

517

Cross-Functional Teamwork: Enabling trust (12,5%), the propensity to share knowledge (25%), positive mood (31,3%), cooperative behavior (25%), empathy (18,8%), and relationship management (25%). Strategic Vision: Coping with uncertainty, ambiguity, and risk (50%), future-oriented mindset (31,3%), sustainable development (37,5%), driving change and innovation (31,3%), impact the strategic management (25%), and envisioning future scenario (62,5%). 2.2 Reflecting on the Results of the Survey The LLabs participants who participated in the survey enjoyed the overall experience. The experience allowed students to interact not only with their peers from different backgrounds, but also with a variety of experts, local stakeholders and entrepreneurs who shared and discussed valuable insights about tech-related real-world experiences. To ensure maximum comfort for all participants, we deliberately created an open work environment that encouraged communication not only within the teams, but also across the different ones. Although the concerned participants enjoyed the LLab experience, some participants reported stressful situations related to forming teams for different projects, time constrains and language barriers. This problem occurred more frequently during the LLab1. Yet, we addressed the problem of team forming for the LLab2 and LLab3.It is common to encounter such challenges during the initial phase as different members of each group familiarize themselves with one another. Thus, we implemented team bonding exercises in the morning to foster trust and establish relationships among groups. These exercises had a very positive impact on creating an environment for collaboration which enhanced the overall team dynamic. When we asked if participants had the opportunity to apply the skills and knowledge acquired during the LLab in real-life scenarios, preliminary results indicate that most of the participants did, indeed, use the learned skills in practical situations. Such results suggest that the theoretical and practical exercises offered to the participants during the LLab have broad applicability in real-life contexts. For example, based on the participant’s feedback, these skills appear to be relevant in designing artificial intelligence related project, academic environment (thesis writing, design courses), entrepreneurship courses, and among other settings. We also asked the participants to reflect and share their thoughts on which DCAs they would like to improve to become Digital Maturity Enablers and the most prominent results of the questionnaire demonstrate a need in improving abilities such as “Translating knowledge and storytelling” (62,5%) and “Envisioning future scenarios” (62,5%). The preliminary results suggest that participants acknowledge that it is important to improve their communication abilities to effectively express complex information in a simplified manner; such improvements will allow participants to articulate ideas and narratives with their intended audience in a way that it is easily understood and relatable. Regarding the capacity to envision future scenarios, participants admitted having difficulties in visualizing, for example, alternative tech-related futures. On the one hand, this could be attributed to the fact that we currently live in the digital age where we are constantly exposed to a set of novel and exciting tech-related

518

V. Vezzani et al.

possibilities. Such saturation of novel ideas and information can make it challenging for the participants to imagine and create alternative futures. On the other hand, it is worth considering that the time constraints of the LLab may have hindered participants to envision the future. Such skill requires thoughtful reflection, research, and experimentation that necessitates a great amount of time and effort. Thus, participants may feel, as stated previously, that more time is required to mature and perfect their ability to create alternative futures. Nevertheless, by acknowledging these challenges and providing additional opportunities to assist participants to enhance their future envisioning skills, we can promote growth, proficiency, and maturity of the DCAs.

3 Conclusion Digital transformation is challenging the status quo and requiring existing systems, organizations, and people to become resilient and adaptable to proactively face the change. The deployment and uptake of emerging technologies are increasingly changing organizations and the whole society, enabling an unprecedented digital transition. New innovative educational models must provide upcoming generations with radically new skillset to enhance their creative abilities, enabling them to spot and exploit the viable potentialities of emerging technologies and unleash their creative potential. The DC4DM educational model responds to this task and promotes a learning process that empowers both individuals and teams in thinking and communicating future scenarios and design responses. As tested through the above described LLabs, and successfully presented positive results, the DC4DM methodology encourages collaborative and open-minded dynamics among cross-functional teams. The DC4DM project, which started in September 2020 and concluded in August 2023, has been the opportunity to develop a set of open-source learning materials gathered into the EDU Box virtual container [10], which can keep evolving according to ever changing digital scenarios and learning needs. Finally, the DC4DM project goes beyond the mere transfer of Digital Maturity (DM) methods and tools; it fosters collaboration among a diverse range of entities within a European network of Higher Education Institutions (HEIs), Small and Medium Enterprises (SMEs), Startups, and Business Incubators. This collaborative effort encompasses various activities aimed at sharing knowledge and cultivating digital creativity competencies. The project involves the organization of Digital Maturity and DM Sharing Days, considered as informative events. These gatherings are meant to disseminate valuable information, insights, and best practices related to Digital Maturity among the participants. Through these events, individuals and organizations gain a deeper understanding of the concepts, methods, and tools associated with Digital Maturity.

References 1. Teichert, R.: Digital transformation maturity: a systematic review of literature. In: Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis 67(6), 1673–1687 (2019)

Educating the Future Digital Maturity Enablers

519

2. Amara, R.: The futures field (Report). Menlo Park, California: Institute for the Future. OCLC 7919207 (1980) 3. Prensky, M.: Digital natives, digital immigrants. Part 1, On the Horizon, vol. 9, no. 5, pp. 1–6 (2001) 4. Schwab, K.: “The Fourth Industrial Revolution”, Portfolio Penguin. pp 7–10 World Economic Forum: The Future of Jobs Report 2020. Geneva, Switzerland, October 2020 (2016). https:// www.weforum.org/reports/the-future-of-jobs-report-2020. Accessed 31 May 2023 5. World Economic Forum: Schools of the Future. Defining New Models of Education for the Fourth Industrial Revolution (REF 09012020). Geneva, Switzerland: WEF (2020). https://www3.weforum.org/docs/WEF_Schools_of_the_Future_Report_2019.pdf. Accessed 11 Mar 2023 6. Bruno C, Canina MR (2019) Creativity 4.0. Empowering creative process for digitally enhanced people. Des J 22: 1, 2119—2131 7. Canina, M.R., Bruno, C.: Design and creativity for developing digital maturity skills. In: Proceedings of the 23rd I0nternational Conference on Engineering and Product Design Education (E&PDE 2021) (2021) 8. World Economic Forum: The Future of Jobs. Geneva, Switzerland: WEF (2020). www.wef orum.org/reports/the-future-of-jobs-report-2020. Accessed 11 Mar 2023 9. Voros, J.: A primer on Futures Studies, foresight and the use of scenarios. https://thevorosc ope.com/publications/foresight-primer/ (2001) 10. The DC4DM Project and EDU Box section. https://www.dc4dm.eu/

Effects of Customer Service Quality in Online Retail Stores Pedro Espírito Santo1(B)

and Patrícia Azinheira Cardoso2

1 Centre of Applied Research in Management and Economics (CARME), Instituto Politécnico

de Leiria, Campus 2 - Morro do Lena, Alto do Vieiro, Apt 4163, 2411-901 Leiria, Portugal [email protected] 2 Instituto Politécnico de Coimbra – ESEC, Rua Dom João III, 3030-329 Coimbra, Portugal [email protected]

Abstract. Competitiveness has intensified among online retailers, leading organizations to seek new ways to maintain competitive advantage in products and services. The quality of customer service is considered a critical aspect for organizations, due to the impacts that have on the relationship with consumers, so it is important to study the antecedents and consequences of customer service quality. Therefore, this study identified customer service customization and customer service advertising as antecedents of customer service quality. As consequences of customer service quality, this research identified satisfaction, trust and loyalty. In this sense, this research collected a sample of 176 consumers of online shops of electronic products that sought to test the research hypotheses through the methodology PLS-PM (partial least squares - path model). In this context, this study identified that the quality of customer service promotes satisfaction and confinement of individuals leading them to have loyalty to the online shop. Moreover, brands should customize the service according to customer preferences and also advertisement. Keywords: Customer Service quality · Customization · Satisfaction · Trust · Loyalty

1 Introduction Consumers are increasingly more mature, aware and demanding, when it comes to searching and giving up their personal information to organizations. Thus, it is created a greater competitiveness of online retail organizations in the dispute for consumers. This scenario is the result of organisations wanting to develop a product and service adapted to consumer needs, directing the organisation to a more individualised marketing in its offer, to retain and build loyalty [1, 2]. Thus, it is considered necessary to investigate service quality in retail companies and its impact on customer satisfaction and loyalty. It is necessary to adapt the organization in receiving, storing and disseminating information from/to consumers, to adapt the offer to consumers’ needs and change the organization’s culture and focus, turning to the relationship with the consumer [3]. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 520–529, 2024. https://doi.org/10.1007/978-3-031-47281-7_43

Effects of Customer Service Quality in Online Retail Stores

521

Thus, it is crucial for any organisation to continuously offer to its consumers a good customer service to increase sales, as well as to attract and retain consumers, ensuring their satisfaction, trust and loyalty [4, 5]. Currently, new ways to reach the consumer are necessary and organizations in recent years, have sought to collect more and more information about their customers and potential customers to characterize and better understand their profile or type of customer and nowadays, so that it is possible to offer a differentiating service [6, 7]. Anshari [8] consider that the new trends in marketing, allow a customization of communication to improve the customer service. The customization of the support service and communication becomes key in the relationship with the customer. Therefore, it is important to understand the role of service quality in the business context, in order to create a greater competitiveness of organizations in winning their consumers [9]. In this sense, this study aims to seek answers about the effects of the support service quality in the relationship between consumers and organizations and in addition, analyse the customization as a determinant of the support service quality.

2 Theoretical Framework 2.1 Customer Service Quality in customer service has become a critical service for organizations [10] and it is the consumer’s subjective evaluation of the interaction quality with a website and how well the service needs have been met [11]. A consumer not only wants a highquality product, but also high quality services. In addition, organizations are determined to provide consumers with more valuable and competitive offerings [12]. Improving the quality of a service is now possible through the evolution of collecting and analysing consumer data, such as their personal interests, experiences, and other information about their daily lives [13]. 2.2 Customization of Customer Service Advertising Messages Many organizations customize advertising messages [14] through marketing-mix actions and by collecting information about consumers [15]. The customization of advertising messages is related to the individual preferences of each consumer and [16, 17] believe that organizations can use multiple channels to convey their advertising messages, and this shows dedication to the relationship with the consumer. Barreto et al. [3] comment that advertising messages are part of a marketing process, understood between a social process, a management of the exchange of information, of stimuli between an organization and its consumers. This process aims to promote both the organization and the consumer, the consumer being the target of all the efforts of the advertising messages by the organization. In this sense, customization of advertising messages serves to create value for the consumer, leading the consumer to modify the perception about the company’s services [18]. Thus, this study will test the following research hypothesis: H1: Customization of advertising messages has a positive impact on customer service quality.

522

P. E. Santo and P. A. Cardoso

2.3 Customized Customer Service Looking at the competition in a general way and through the strategic opportunities that exist in customer service, organizations are challenged to find the most effective methodology to improve their customer service, but also in delivering value that differentiates from other providers of the same service [10]. Roy, Shekhar, Lassar and Chen [19] state that customer service customization is defined by the time and effort saved by consumers: in seeking the service, during the use of the service, and post-sale of the service (if any). Customized customer service, according to Roberts, Varki and Brodie [20], is an adaptation that tends to improve the consumer’s perceived relationship with the organization, leading them to speak well of it and recommend it to their friends and family and in this sense, Bordewich, Semple, and Steel [21] believe that it is important to assign a profile to each consumer, where each of them should be taken into account. This, we postulate the following: H2: Customized customer service has a positive impact on customer service quality. 2.4 Satisfaction Consumer satisfaction is a goal in marketing-oriented businesses, and satisfaction can arise from various perceptions and expectations [22]. Thus, any organization should aim for consumer satisfaction to survive by having their business and marketing activities directed towards it. Consumer satisfaction is defined as feelings of pleasure or dissatisfaction, which result from comparing the performance of a purchased product under the expectations initially set [23]. Satisfaction is important to business success [24]. Therefore, satisfaction is the accumulation of a series of experiences with the product or service of a particular organisation, resulting from the consumer-organisation connection over time. The organizations that deliver greater value to the consumer, more will satisfy them, as well as will be perceived as more reliable by them [25]. Customer service quality will facilitate consumer satisfaction [26]. Moreover, an organisation that in the long term improves the quality of the service provided in a continuous and incremental way [20], Thus, we will teste the following hypothesis: H3: Customer service quality has positive impacts on satisfaction 2.5 Trust Trust is about a sense of security, based on an individual consumer’s behaviour when they are motivated and guided by the optimistic and encouraging actions of the organisation [7]. Agustin and Singh [27] define trust as a belief by consumers between the perceived benefits versus the cost of maintaining the relationship with the organisation. An organization becomes reliable, when consequently it keeps its promise of value to its consumers [28], through the way the product is developed, how it is produced, how it is advertised, how it is sold, the service provided in shop and the after-sales service. In this study, we propose to study the following research hypotheses:

Effects of Customer Service Quality in Online Retail Stores

523

H4: Customer service quality has positive impacts on trust H5: Satisfaction has positive effects on trust 2.6 Loyalty The central role of marketing strategies is the development and maintenance of consumer loyalty, especially in markets with strong competition, unpredictable and diminishing product differentiation factors [29], focusing on developing long-term relationships with consumers, giving rise to a very robust organisations and brands [7]. Consumers have changed the way they interact with brands and organisations. Online consumer loyalty is influenced by a variety of factors, such as consumer satisfaction, trust and the level of engagement of the relationship with the organisation [1]. Thus, one determinant and outcome factor of loyalty is satisfaction. Additionally, trust is very important for loyalty, because [24] states that it establishes an important bond between the organization-consumer relationship (products and/or services provided and high involvement). In this sense, this study proposes to study the following research hypotheses: H6: Satisfaction has a positive impact on loyalty. H7: Trust has a positive impact on loyalty.

3 Methodology To achieve the research objectives and evaluate the proposed model, we conducted a study in the context retail online. Therefore, to test the proposed research hypotheses throughout a theoretical framework, we chose the quantitative methodology, through a cross-sectional study, given that quantitative methodologies are based on a clearer and more objective data analysis, especially when it comes to testing research models and hypotheses. Data collection was made through an online self-completion and respondents were contacted through social networks. Data collection was based on an online questionnaire designed for this purpose, in order to obtain a larger sample. Therefore, a questionnaire was designed through the google forms platform with items adapted from other studies. Consequently, for the customization of customer service advertising, we adapted 3 items from Shanahan et al. [21], for the customized support service, we used the 3 items from Bock, Mangus, and Folse [30]. In the measurement of customer service quality we used the 4 items from Shanahan et al. [21]. From Song, Wang and Han’s [7] study we used the 5 items for trust, the 3 items for satisfaction and the 3 items for loyalty. 3.1 Sample The sample consists on 176 customers of online specialized retail shops (electronic devices) from Portugal. The sample is composed of 56.3% female consumers, aged under 30 years old in most cases (in 61.4% of cases). The educational level of the respondents is mainly secondary school (48.9%) and university (49.4%).

524

P. E. Santo and P. A. Cardoso

4 Results Previously, we assessed multicollinearity through the analysis of the Variance Inflation Factor (VIF) coefficient and we verified that VIF value is less than 5 and it avoid biasing the results obtained [31]. To validate and test the model proposed in this research, the software SMARTPLS version 3 was used. Therefore, the analysis through PLS estimation is divided into two steps: the analysis to the measurement model and the analysis to the structural model. 4.1 Measurement Model The first step in the analysis through PLS is the analysis of the measurement model, which seeks to assess the ability of the items to measure the constructs, namely in terms of reliability and validity. Table 1. Measurement model Items

λ

t values

α

ρA

CR

AVE

Customized customer service

CCS1

0,846

26,884

0,77

0,78

0,87

0,69

CCS2

0,841

22,696

CCS3

0,804

16,827

Trust

TRT1

0,796

18,646

0,88

0,88

0,91

0,67

TRT2

0,852

19,255

TRT3

0,778

21,998

TRT4

0,795

18,053 0,73

0,78

0,84

0,64

0,81

0,81

0,89

0,73

0,76

0,79

0,86

0,67

0,88

0,89

0,92

0,73

Satisfaction

TRT5

0,868

32,481

SAT1

0,744

13,927

SAT2

0,789

20,717

SAT3

0,859

46,527

LOY1

0,805

22,927

LOY2

0,887

40,926

LOY3

0,863

27,263

CSA1

0,792

11,620

CSA2

0,875

20,667

CSA3

0,783

12,024

Customer CSQ1 Service Quality CSQ2

0,860

29,423

0,796

18,004

CSQ3

0,908

43,785

CSQ4

0,855

28,921

Loyalty

Customer Service Advertising

Effects of Customer Service Quality in Online Retail Stores

525

Convergent validity and reliability were analysed through the Average Variance Extracted (AVE) and Composite Reliability (CR). Thus, Table 1 shows that AVE (ranging from 0.64 to 0.73) and CR (ranging from 0.84 to 0.92) are above the threshold values (AVE > 0.5; CR > 0.7). Additionally, we analysed the reliability through the Cronbach’s α and ρA and all values are above the recommended values (α > 0.5; ρA > 0.7). In addition, we examined the standardized loadings (λ) and all of them are above the recommended value (λ > 0.7). In Table 2, we present the Fornell and Larcker [32] criterion to examine discriminant validity. We found that all AVE values (on the diagonal of the table) are higher than the squared interconstruct correlation estimates. Additionally, we analyse the HTMT ratio of correlations follow the suggestion of Henseler et al. [33] and the results are in table 3. Table 2. Discriminant Validity – Fornell and Larcker criterion TRT

CSA

Trust (TRT)

0,82

Customer Service Advertising (CSA)

0,36*

0,82

Customized Customer Service (CCS)

0,48*

0,40*

CCS

LOY

CSQ

SAT

0,83

Loyalty (LOY)

0,68*

0,29*

0,36*

0,85

Customer Service Quality (CSQ)

0,62*

0,30*

0,46*

0,62*

0,86

Satisfaction (SAT)

0,76*

0,25*

0,45*

0,64*

0,56*

0,80

CSQ

SAT

Table 3. Discriminant Validity –HTMT ratio TRT

CSA

CCS

LOY

Trust (TRT) Customer Service Advertising (CSA)

0,43

Customized customer service (CCS)

0,58

0,53

Loyalty (LOY)

0,80

0,36

0,45

Customer Service Quality (CSQ)

0,70

0,35

0,55

0,72

Satisfaction (SAT)

0,90

0,32

0,60

0,78

0,65

4.2 Structural Model The second step is the analysis of the structural model, which aims to test the hypotheses under study by analysing the path coefficients and their significance. To obtain the statistical significance of each coefficient, the bootstrapping sampling technique was used. This technique according consists of a non-parametric procedure that creates new subsamples. In this sense, for this study it was used the bootstrapping technique with

526

P. E. Santo and P. A. Cardoso

5,000 samples, as suggested by the software, in order to estimate more consistent and also more stable results [31]. In table 4 we present the hypothesis test. Table 4. Hypothesis Test Hypothesis

β

t values

95% Confidence interval 2.5%

97.5%

0,009

0,287

p values

Result

0,048

Supported

H1

CSA → CSQ

0,140

1,976

H2

CCS → CSQ

0,405

4,512

0,223

0,574

0,000

Supported

H3

CSQ → SAT

0,558

11,372

0,463

0,654

0,000

Supported

H4

CSQ → TRT

0,293

3,945

0,143

0,430

0,000

Supported

H5

SAT →TRT

0,591

11,129

0,489

0,696

0,000

Supported

H6

SAT→ LOY

0,297

2,740

0,078

0,503

0,006

Supported

H7

TRT → LOY

0,454

3,944

0,227

0,678

0,000

Supported

Notes: CSQ-Customer Service Quality; TRT = Trust; SAT = Satisfaction; LOY = Loyalty; CCS = Customized customer service; CSA = Customer Service Advertising.

5 Discussion and Conclusions The results show that customer service quality is encouraged by service customisation (βCCS→ CSQ = 0.405; p < 0.01) and also through customer service advertising (βCSA→ CSQ = 0.140; p < 0.05). In this sense, following the literature reviewed, customization and advertising create value for the consumer and modify the consumer’s perception towards the online shop. Thus, as a practical suggestion, we suggest that online shops create options so that the consumer’s profile is customized according to his/her needs. Furthermore, it is also suggested that online shops effectively communicate their value so that it is possible to improve the quality of customer service [19, 34]. Quality customer service has valuable consequences for online shops, in particular improving satisfaction (βCSQ→ SAT = 0.558; p < 0.01) and improving trust in the online shop (βCSQ→ TRT = 0.293; p < 0.01). Therefore, our study suggests that strong customer service improves satisfaction [26] and facilitates trust in the online shop [28]. Thus, we consider that it is essential for online shops to focus on the quality of service to their customers. Additionally, this research considers that satisfaction is the basis of trust (βSAT→ TRT = 0.591; p < 0.01) so online shops must focus on customer satisfaction. Loyalty appears as a determining variable in our study and has a direct influence on satisfaction (βSAT→ LOY = 0.297; p < 0.01) and trust (βTRT→ LOY = 0.454; p < 0.01). In summary, this research is based on the premise that customer service quality is central to the customer relationship for online shops to acquire loyalty. This study explores the concepts customer service advertising and personalised customer service along with customer service quality and evaluates their effects on customer loyalty and

Effects of Customer Service Quality in Online Retail Stores

527

satisfaction in online specialty retail shops. However, this study did not consider the emarketing characteristics of online shops as antecedents, nor did it examine the consumer experience in online shops. We consider that this could provide useful information in relation to customer service quality. Furthermore, this research analysed data from a sample of mostly young online shoppers. We believe that there should be further studies with more diverse samples. Acknowledgements. This work is financed by national funds through FCT - Fundação para a Ciência e a Tecnologia, I.P., under the project UIDB/04928/2020.

References 1. Santo PE, Marques AMA (2022) Determinants of the online purchase intention: hedonic motivations, prices, information and trust. Baltic J Manag 17(1):56–71. https://doi.org/10. 1108/BJM-04-2021-0140 2. Santos S, Santo PE, Ferreira S (2021) Brands should be distinct! The contribution of ad’s narrative and joy to distinctiveness. In: Martins N, Brandão D, Moreira da Silva F (eds) Perspectives on design and digital communication II. Springer series in design and innovation, vol 14. Springer, Cham, pp 319–332. https://doi.org/10.1007/978-3-030-75867-7_21 3. Barreto IF, Crescitelli E, Figueiredo JCB (2015) Resultados de marketing de relacionamento: Proposição de modelo por meio de mapeamento cognitivo. Revista Brasileira de Gestao de Negocios 17(58):1371–1389. https://doi.org/10.7819/rbgn.v17i58.2692 4. Augusto L, Santos S, Santo PME (2020) E-commerce: the main role of trust. RISTI - Revista Iberica de Sistemas e Tecnologias de Informacao 2020(E34):557– 570. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85092241718&partnerID=40& md5=f857aca41f96d013055d0516c82f7e8f 5. do Espírito Santo PM, Trigo A (2020) Influence of website design on e-trust and positive word of mouth intentions in e-commerce fashion websites. In: Themistocleous M, Papadaki M, Kamal MM (eds) EMCIS 2020. LNBIP, vol 402. Springer, Cham, pp 316–330. https:// doi.org/10.1007/978-3-030-63396-7_21 6. Ellsworth PC (2003) Confusion, concentration, and other emotions of interest: commentary on Rozin and Cohen (2003). Emotion 3:81–85. https://doi.org/10.1037/1528-3542.3.1.81 7. Song H, Wang J, Han H (2019) Effect of image, satisfaction, trust, love, and respect on loyalty formation for name-brand coffee shops. Int J Hosp Manag 79:50–59 (2019). https://doi.org/ 10.1016/j.ijhm.2018.12.011 8. Anshari M, Almunawar MN, Lim SA, Al-Mudimigh A (2019) Customer relationship management and big data enabled: personalization & customization of services. Appl Comput Inform 15(2), 94–101. https://doi.org/10.1016/j.aci.2018.05.004 9. Augusto L, Santos S, Santo PE (2020) Willingness to pay a premium price for streaming services: the role of trust in services. Smart Innov Syst Technol 167:19–28 10. Scheidt S, Chung QB (2019) Making a case for speech analytics to improve customer service quality: vision, implementation, and evaluation. Int J Inf Manag 45:223–232. https://doi.org/ 10.1016/j.ijinfomgt.2018.01.002 11. Santo PE, Cardoso P (2021) 500 brand and marketing definitions: for researchers and professionals. Independently published 12. Jung AR (2017) The influence of perceived ad relevance on social media advertising: an empirical examination of a mediating role of privacy concern. Comput Hum Behav 70:303– 309. https://doi.org/10.1016/j.chb.2017.01.008

528

P. E. Santo and P. A. Cardoso

13. Do Espírito Santo PM, Ferreira VHS, Marques AMA (2019) The impact of the employer’s brand image on knowledge sharing. In: Proceedings of the European conference on knowledge management, ECKM, 2019, vol 1, pp 312–320. https://doi.org/10.34190/ KM.19.142. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85073380403&doi=10. 34190%2fKM.19.142&partnerID=40&md5=6363c371e92c5aacc2fad83bbf2557ef 14. Bleier A, Eisenbeiss M (2015) Personalized online advertising effectiveness: the interplay of what, when, and where. Mark Sci 34(5):669–688. https://doi.org/10.1287/mksc.2015.0930 15. Arora, N, et al: Putting one-to-one marketing to work: personalization, customization, and choice. Mark Lett 19(3):305. https://doi.org/10.1007/s11002-008-9056-z 16. Do Espírito Santo, PM, Santos S (2019) Sponsorship congruity and attitude towards advertising in music festivals: the role of trust and social media engagement. In: Multi conference on computer science and information systems, MCCSIS 2019 - Proceedings of the international conferences on ICT, society and human beings 2019, connected smart cities 2019 and web based communities and social media 2019, pp 309–316. https://doi.org/10. 33965/wbc2019_201908l037. https://www.scopus.com/inward/record.uri?eid=2-s2.0-850 73146436&doi=10.33965%2fwbc2019_201908l037&partnerID=40&md5=4ec2da2ed343 bc00fa79c7c63c5f931c 17. Ferreira S, Santos S, Santo PE (2021) Social Media engagement through video advertising: informativeness and self brand connection as predictors. Adv Intell Syst Comput 1352:249– 260 18. Santo PMDE, Ferreira VHS, Marques AMA (2021) How knowledge sharing mediates the effects of internal marketing on customer orientation. In: Proceedings of the European conference on knowledge management, ECKM, 2021, pp 657–665. https://doi.org/10.34190/ EKM.21.145. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85117127956&doi= 10.34190%2fEKM.21.145&partnerID=40&md5=f3d7c0c4c6a4b4903563990ebc852333 19. Roy SK, Shekhar V, Lassar WM, Chen T (2018) Customer engagement behaviors: the role of service convenience, fairness and quality. J Retail Consum Serv 44, 293–304. https://doi. org/10.1016/j.jretconser.2018.07.018 20. Roberts K, Varki S, Brodie R (2003) Measuring the quality of relationships in consumer services: an empirical study. Eur J Mark 37(1/2):169–196. https://doi.org/10.1108/030905 60310454037 21. Shanahan T, Tran TP, Taylor EC (2019) Getting to know you: social media personalization as a means of enhancing brand loyalty and perceived quality. J Retail Consum Serv 47:57–65. https://doi.org/10.1016/j.jretconser.2018.10.007 22. Augusto L, Santos S, Santo PMDE (2020) Usability and sophistication of websites: a path to satisfaction in online retail. RISTI - Revista Iberica de Sistemas e Tecnologias de Informacao 2020(E34):542–556. https://www.scopus.com/inward/record.uri?eid=2-s2.0-850922 49923&partnerID=40&md5=95dada791e15d752621934db89ae5484 23. Kotler P, Keller KL (2009) Marketing management. Pearson Prentice Hall 24. Erci¸s A, Ünal S, Candan FB, Yıldırım H (2012) The effect of brand satisfaction, trust and brand commitment on loyalty and repurchase intentions. Procedia Soc Behav Sci 58:1395–1404. https://doi.org/10.1016/j.sbspro.2012.09.1124 25. Srihadi TF, Setiawan D (2015) The influence of different level of service characteristics and personal involvement towards consumer relational response behaviors. Procedia Soc Behav Sci 210:378–387. https://doi.org/10.1016/j.sbspro.2015.11.385 26. Seiders K, Voss GB, Grewal D, Godfrey AL (2005) Do satisfied customers buy more? Examining moderating influences in a retailing context. J Mark 69(4):26–43. https://doi.org/10. 1509/jmkg.2005.69.4.26 27. Agustin C, Singh J (2005) Curvilinear effects of consumer loyalty determinants in relational exchanges. J Mark Res 42(1):96–108. https://doi.org/10.1509/jmkr.42.1.96.56961

Effects of Customer Service Quality in Online Retail Stores

529

28. Sahin A, Zehir C, Kitapçı H (2011) The effects of brand experiences, trust and satisfaction on building brand loyalty; an empirical research on global brands. Procedia Soc Behav Sci 24:1288–1301. https://doi.org/10.1016/j.sbspro.2011.09.143 29. Mabkhot HA, Salleh SM (2017) The influence of brand image and brand personality on brand loyalty, mediating by brand trust: an empirical study. J Pengurusan 50. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85031666645&partnerID= 40&md5=eb14572b352d15d51eeb5b9477d4f6a8 30. Bock DE, Mangus SM, Folse JAG (2016) The road to customer loyalty paved with service customization. J Bus Res 69(10):3923–3932. https://doi.org/10.1016/j.jbusres.2016.06.002 31. Hair JF, Black WC, Babin BJ, Anderson RE (2018) Multivariate data analysis. Pearson Education Limited 32. Fornell C, Larcker DF (1981) Evaluating structural equation models with unobservable variables and measurement error. J Mark Res 18(1):39–50. https://doi.org/10.2307/3151312 33. Henseler J, Ringle CM, Sarstedt M (2015) A new criterion for assessing discriminant validity in variance-based structural equation modeling. J Acad Mark Sci 43(1), 115–135. https://doi. org/10.1007/s11747-014-0403-8 34. Sivapalan A, Shankar A, Das M, Konalingam K (2022) How to enhance customer loyalty to retail stores via retail service quality practices? A moderated mediated mechanism. Serv Mark Q 43(4):445–465. https://doi.org/10.1080/15332969.2021.1997504

Design for Well-Being, Empowerment and Experience of Care: Minimum Design Requirements of a Children Prosthetics for Initial Approach to Sport Activity and Aquatic Physiotherapy Piera Losciale(B)

, Alessandra Scarcelli , Annalisa Di Roma , and Giovanni Pappalettera

Politecnico di Bari, Via Edoardo Orabona, 4, 70126 Bari, BA, Italy [email protected]

Abstract. The paper is relevant to the scope of design for wellness and the medical field. Specifically, the research presents a critical analysis of existing case studies, with and without patent, and the identification of minimum requirements for sports prosthetic systems suitable for amateur children’s swimming. Traditional design responds to the “standard user” concept, which penalizes and excludes part of the population. Instead, this research uses the Design for All approach: systemic, holistic, and necessarily multidisciplinary. Methodologically, the principles of Human-Centered Design (HCD) are adopted, focusing on the real users, directly involving them in the design process. Paper’s structure provides an overview of the research area and, in line with the chosen methodological approach, proceeds with the identification of end users, their needs, and requirements through a participatory process. Subsequently, the analysis of the relevant state of art and the ensuing deductions are conducted. State of art research has identified a deficiency in experimenting with devices for aquatic sports: non-performing and expensive, but functional for taking the initial step, particularly in swimming practice. Specifically, in light of the need for children with trans-radial agenesis to undergo water-based hydrotherapy, and recognizing that a suitable device can yield more significant physiotherapy’s results and a shorter acclimatization period in water, the necessity to design new prosthetic systems has been identified. Keywords: Human-Centered Design · Swimming prothetic system · Design forwell-being

1 Introduction 1.1 Medical Design Macro-Area: Proasthetics Medical design constitutes a wide field of research, experimentation, and innovation within the context of design. This research field aims to create a system of objects, technologies, and services for the well-being and care of the end-user. The end-user © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 530–541, 2024. https://doi.org/10.1007/978-3-031-47281-7_44

Design for Well-Being, Empowerment and Experience of Care

531

becomes an integral part of the design that involves the development of innovative solutions and applications. It is essential to remember that the aforementioned centrality takes into account needs, human and cultural factors, “usability,” which, when applied in the medical field, makes clearer the identification of the user and the direct relationship between professionals and manufacturers. The broad field of medical design considers the creation of supports or replacements of anatomical segments that respond to specific patients’ needs. Among these are prostheses, external orthoses, and braces that are intended for the correction or replacement of such anatomical parts (Zoccali 2017). This research embraces the field of aesthetic prosthetics, which proves to be one of the most complex in the macro-discipline. The state of art in this area is wide and includes cutting-edge solutions that allow for the almost complete reconstruction of all the functions of a real limb. These prostheses use advanced detection and production technologies and systems: an approach that is in line with the designer’s conception expressed by Tomàs Maldonado. He believes that the designer must have extensive knowledge of existing technology to improve the user’s living conditions, as well as the ability to give form, function, and meaning to objects. “Industrial design has the task of giving shape to products,” (Maldonado 1976) considering functional, cultural, aesthetic as well as for aspects together with factors related to the product production, materializing optimal solutions in the technological and financial fields. This dichotomy is not respected today. Thanks to contemporary technologies, not only can the individual’s psycho-physical well-being be improved, but ergonomics and comfort, appropriately parameterized, break down the barriers of the traditional approach. This sudden increase in technology use in prosthetics, however, has made this field increasingly elitist; in fact the cost of using prostheses for a user is very high both for replacement and maintenance. The aspect of their versatility should not be underestimated: for example, the practice of sport requires specific characteristics of the prostheses to be used, making them even more expensive. The use of prostheses at a very young age is of fundamental importance for the users but requires frequent replacement due to the child’s rapid growth. This often forces families not to be able to offer the child secondary use prostheses such as sports ones, which are even more expensive than everyday ones. The field of sports limb prosthetics is now considered one of the most innovative and futuristic sectors. These prostheses currently allow for the practice of an increasingly wide range of disciplines thanks to the advancement of technologies, material study, new digital scanning and production methods. In fact, prothesis have been created with such high performance that they almost reflect the functionality of a natural limb. However, these aforementioned prostheses are elitist due to their costs, which are significantly higher than average, and are not available for all disciplines. The discipline of swimming is practiced at a competitive level by millions of users, and among them, those who show diversity in their limbs are quite considerable. It is interesting to notice a complete absence of device that help and level the playing field for the user in this discipline, as well as in many other fields. This certainly happens at a competitive level, although, despite rapid technological evolution, creating a prothesis that does not significantly hinder race times is very complicated. The number of amateur

532

P. Losciale et al.

users who start using a prosthesis for swimming in the sea or pool is increasingly high. Many designers have started to study and create them. The relationship with the national association Energy Family Project, which includes families with children affected by limb disabilities, has allowed us to initiate an investigation regarding the use of devices in the practice of amateur swimming. Therefore, the current research begins with the identification of users’ needs and requirements, and critically analyzes the currently available sport prostheses for those engaging in swimming activities.

2 Research Question and Objectives The research conducts a quantitative and qualitative investigation (surveys and interviews) to define users’ target, their needs and requirements. Additionally, it involves a detailed and critical analysis of the current state of art of existing prostheses and devices in the field of swimming sports, whether patented or not, in concept phase or on the market. These stages outlined in this paper are crucial for the broader research objectives. The primary goal of the research is to identify issues with current existing prostheses, in order to define the minimum requirements for designing a prosthetic system that enhances the experience and care of children affected by agenesis during amateur swimming practice. Aspects such as body buoyancy in water, hydrodynamics, user acceptance, usability, comfort, customization, replacement speed, cost, sensing/design/production technique, adaptability to children, and integration into the context are taken into account. Another important goal of the research is to work towards a social innovation (Manzini 2013), where diversity is seen as an added value. Today, design discipline is paying attention to the value systems of diversity, as they can be understood in terms of access and perception of well-being and comfort in relation to one’s physical condition, influenced by phenomena such as aging, socio-cultural diversity, disability, in favor of breaking down barriers, negative attitudes, and exclusions consciously or unconsciously enacted by society. The inclusion of users with disabilities in such as sports environments, improving their well-being and care, is thus a responsibility of design, in which this research ensures through context analysis. A future development perspective of the research is to reach the design and experimental phases of the product on specific users, using a participatory approach, typical of Human-Centered Design (HCD).

3 Methods and Tools Since 1960, scientific methodological approaches have emerged concerning product design and human-machine interfaces. During this time, a first generation focused on using a systematic, rational, and scientific methodology. Subsequently since 1970, a generation of designers was born who not only considered factors such as form, function, and material in the design phase, but also human needs. Thus, the role of the designer no longer remained the one of a unique and omnipotent figure but became part of a work team that collaborated in the development of the production process, in which

Design for Well-Being, Empowerment and Experience of Care

533

everyone is a partner and is aware of the user’s needs. According to Maldonado, design has the role of coordinating, integrating, and articulating all those factors that in some way participate in the process of constituting the form of the product. Specifically, it refers to factors related to use in individual or social product consumption (functional, symbolic, or cultural factors) and those related to its production (technical-economic, technical-constructive, technical-systemic, technical-production, technical-distribution factors). To achieve good design, according to Cross, the two generations mentioned in the 1960s and 1970s should merge. Thus, a new methodological model is proposed. It involves a “switching” between the nature of the problem and the solution in design (Cross 2019), the design thinking typical of User-Centered Design. Therefore, the methodology used is based on User-Centered Design (UCD) which emerged in the 1970s–1980s in the IT field and was adapted to the world of design. The usability of a product-service has always been the goal of this approach. Since the 1990s, UCD has undergone an evolution, defining investigative methodologies to define users’ needs. The most significant change concerned the role of the user in the design process. In fact, in 1990, ISO defined Human-Centered Design as “an iterative process that consists of users’ studies and specific design and evaluation solutions, which ultimately aim to design products and services based on the needs of end-users”. In the span of twenty years, the design world has had to broaden its boundaries: concentrating design activities on aspects such as appeal, usability, and safety is no longer sufficient. On the contrary, as Rizzo (2009, p. 11) states, design today “is called upon to act as a pragmatic approach and way of thinking, for the design of real contexts, for everyday use in people’s lives, because design is capable of acting in accordance with Human-Centered ethics, unlike other conceptual disciplines”. The evolution of approach within the Design sector mainly concerned the ways of involving end-users in the design process. Users play an increasingly active role: they are no longer seen as mere sources of information but as holders of information and experience, which can be actively and iteratively involved during every phase of the design development. This involvement focuses on the importance of emotional involvement during operation and during interactions between objects and systems. The consolidated theory that a positive or negative emotion can significantly transform the user’s experience, regardless of the product and other usability indicators, has led to an evolution of the definition of usability and the transition from User-Centered Design to HumanCentered Design. The initial goal of UCD, to assess usability, has evolved over time towards a design activity that considers the entire user experience, starting from the strategic role played by the user in every phase of the design process. The non-linear process shows how each Human-Centered design activity uses the results of the other activities: the processes are executed iteratively, and the cycle is repeated until specific usability and functionality goals are achieved. The operational mode shows how the HCD process involves users from the early stages of design (design requirements or specifications) and also allows for experimentation with early versions of the system, correcting problems based on user reactions and comments, in a process of testing and subsequent adjustments. Once the design needs for a product or service are identified, four main phases are outlined: understanding and defining the usage context, defining users’ requirements or group of users, producing design solutions, and verifying the

534

P. Losciale et al.

design solutions. Norman himself states that “the hardest part of design is discovering the right requirements,” that is, the right solution to the problem (Norman 2014, p. 231). User Experience is therefore an integral part of mentioned methodology and involves a dimension of design that revolves around the characteristics and needs of users, focusing on the universality of experiences within a specific usage context. It is believed that the users’ experience includes emotions, beliefs, preferences, physical and psychological reactions of the user, their behaviors and actions, occurring before, during and after use. The users’ experience is therefore the totality of the effects or perceived effects that a person experiences as a result of their interaction with the usage context of a system, device, or product. It includes the influence of usability, usefulness, and emotional impact during interaction, and finally, memory after interaction. “Interacting with” is a broad concept that includes seeing, touching, and thinking about the system or product, including our admiration for it and the effect of its presentation before any physical interaction. The present research uses the HCD methodology, designing and analyzing the user experience, considered of fundamental importance for the functioning of the entire system. Research identifies the real needs of end users, defines their characteristics and context of use in a shared design process with the user. The analysis tools are those typical of the methodology (observation, direct interview, questionnaire) as regards the identification of needs and target users. The objective is to employ the typical tools of the Human-Centered Design (HCD) methodology also during the product’s experimentation phase. Regarding the state-of-the-art analysis, products were analyzed and then compared according to key characteristics.

4 User Research and Needs’ Identification The user research phase plays a key role in all approaches, such as HCD, that involve user-centered design or co-design with users. In this research, an in-depth analysis was conducted to understand the characteristics, needs, and requirements of users, enabling informed and focused design. The user research was conducted using quantitative and qualitative survey instruments (questionnaires and interviews) shared with the national association Energy Family Project. The quantitative survey (sample: 101 participants) identified the users’ shared needs and issues, while the qualitative survey (interview participants: 5) subsequently rationalized the data into information and identified the project requirements, with the help of experts such as physical therapists and orthopedists. The administered questionnaire consisted of 12 questions aimed at children with limb disabilities, particularly those affected by agenesis. Consequently, the sample refers to one of the two parents responding on behalf of the child. The questionnaire was digitally sent to Energy Family Project association families, and responses were automatically recorded by the platform used for administration. Among the relevant responses, it was noted that none of the children (aged 2 to 13 years) in consideration used a swimming prosthesis, whether for the sea or pool. Equally crucial was the common practice of water-based physiotherapy (hydrokinesitherapy) by 95% of the children, all between 4 and 10 years old. Furthermore, the parents of these children (90% of responses) reported a difficulty in their children’s swimming initiation, involving longer adaptation times related to water balance and floating.

Design for Well-Being, Empowerment and Experience of Care

535

The general information gathered from the quantitative survey was further explored in the second phase of qualitative investigation. This phase delved into the psychological and social aspects of users, in addition to analyzing the survey-emerged data in more detail. Interviews were conducted, facilitated through Skype, with families willing to share their experiences. Apart from users, professionals such as physiotherapists, swimming instructors, and orthopedic technicians were also interviewed to provide a comprehensive analytical framework. Notably, significant responses confirmed the complete absence of prosthesis usage in swimming practice, the common practice of hydrokinesitherapy, and emphasized the need to integrate children into the social sports context. A trend was also observed, indicating a reluctance to use swimming prostheses, driven by the lack of suitable prostheses for specific practice, inadequate adaptability of existing non-performing devices to children in terms of size and acceptability, and the high cost of prostheses which need constant replacement due to rapid child growth. Particularly noteworthy was the comparison with professionals. The extreme importance of hydrokinesitherapy for children between 4 and 10 years old was confirmed; this physiotherapeutic therapy facilitates ideal muscle and nerve development for subsequent use of myoelectric upper limb prostheses. This therapy’s effectiveness was hindered by children’s difficulty in adapting to the water due to body imbalance, leading to prolonged therapy effects. Another critical aspect related to the detection and production techniques of prostheses, which still heavily rely on traditional plaster casting and “manual” production methods. This not only results in extended timelines and higher costs but also provides a highly negative experience for children. Therefore, this investigation deduced that the utilization of a swimming prosthesis for children would be pivotal for multiple reasons: – The identified target encompasses children with upper limb disabilities between 4 and 10 years old, corresponding to the initial adaptation phase in water and the most crucial years for hydrokinesitherapy practice; – Children need to balance their weight in water to enhance both adaptation times and the effects of physiotherapy; – The prosthesis should be cost-effective due to the constant need for replacement caused by natural user growth; – The prosthesis should enhance the user experience both during usage, being integrated into it, and throughout the detection, production, and testing phases. – The prosthesis should be custom-designed for children;

5 State of Art Having understood how swimming is one of the most practiced and stimulating motor activities for very young individuals, the aforementioned activity is considered to be part of the most comprehensive sports to engage in. In fact, many doctors and therapists recommend this discipline to very young children in order to assist them in the growth of all body segments. Taking the first steps in this activity, the most common difficulties encountered relate to initial acclimatization to water and subsequent unassisted floating. The previous investigation revealed that for children with limb disabilities, especially involving the upper body segments, the concept of balance is quite complex: the absence

536

P. Losciale et al.

of a limb or a portion thereof indeed hinders optimal balance, resulting in difficulties in floating. In the competitive sport of swimming, including at the Paralympic level, users with limb disabilities do not utilize prosthetics that enhance swimming performance. This is because a prosthetic system has not yet been designed to be sufficiently highperforming so as not to interfere with athletes’ race times. Following the investigation into the user and their needs, as well as the deductions made, a study was conducted on the state of the art regarding existing prosthetics for swimming. Available patents, market products, and concept projects found on the international web scale were considered. Below are some of the analyzed case studies, which, however, are significant for the common and relevant characteristics of the entire chosen product class. First the products were described discursively and non-critical, and then critically analyzed into expendable results. Elle (2014) is a prosthesis designed by Della Tosin for swimmers who have only one leg and an amputation below the knee. Amputees find it difficult to swim in an aerodynamic position. The human leg has a specific volume and density that competes to create our balance; the loss of one leg causes unbalanced buoyancy due to unequal upward water pressure between the left and right side of the body: first goal of the product is to balance the weight. Second objective is that of multifunctionality: there are three phases involved: launching, swimming, and turning. Each phase requires different energy release profiles to which the prosthesis wants to respond. The third goal is comfort: the leg will need to move both quickly and smoothly; it is important to ensure comfort through a stable connection between stump and prosthesis. An adjustable inner tube helps amputees maintain their aerodynamic position in the water. It is made of a nylon elastane fabric and polyurethane, a material that repels water so as to increase buoyancy. The prosthesis has at the ankle level a flexible metal material that can increase energy return: it can be changed to be compatible with the individual’s strength (Fig. 1). Neptuno (2019) is a prosthesis for lower-limb amputees below the knee, designed to facilitate physical activity in the pool; a cupped, adjustable element allows the prosthesis to be attached according to the situation-the level of amputation, the size of the limb-and to have precise control over movements in the water. In addition, a special tab adjusts the length of the fin and consequently the resistance and effort required for movement; thus, it is an exercise tool to be calibrated to one’s needs and abilities. Made of non-toxic silicone, the green parts of the fins are soft and flexible, while the more rigid blue parts are made of polypropylene, which is easily recyclable. This combination of materials pays attention to sustainability as well as being resistant to chemicals and acids, lasts a long time, and does not require excessive maintenance (Fig. 2). Oyogu (2014) is an upper limb prosthesis for a missing limb from below the elbow, designed by Irish designer Carthy. It works by making changes in its surface: the inner core has holes cut through it, while the rotating outer shell has corresponding holes that, depending on position, create very large holes. When the holes are uncovered, they catch water by acting as a solid surface. The experienced swimmer will have the holes fully exposed, allowing the least amount of resistance combined with maximum strength. While the beginner will have the hole closed with the maximum amount of resistance and the least amount of force. In the prototype, the gear had six teeth allowing 6 different speeds, but this could be changed according to the user’s preference (Fig. 3).

Design for Well-Being, Empowerment and Experience of Care

537

Fig. 1. Elle, Della Tosin;

Fig. 2. Neptuno, Richard Stark;

Freestyle Swimming Device (2019) is an amateur swimming device designed for use with a compatible docking prosthesis. The design mimics a folding wing that reduces resistance during swim recovery but opens to provide maximum resistance during the push phase. The device can be rotated to optimize various swimming styles and strokes. Comes in an adult size (Fig. 4).

Fig. 3. Oyogu, Robert McCarthy;

538

P. Losciale et al.

Fig. 4. Freestyle Swimming Device, Steeper Group;

Patent prostheses include the four cases found below: Prosthetic limb and prosthetic limb attachment for swimming; Swim fin for leg amputees; Swim fin for an amputee; Swimming prothesis. Goldstein patented a lower limb prosthesis for adults (Prosthetic limb and prosthetic limb attachment for swimming) in 2020 that can also be used during the act of swimming. This prosthesis is designed for below-the-knee amputations but can also be adapted to other types of amputations (currently being tested). It has a standard socket as an interface and is anchored to the body by elastic bands or buckles. It also has side holes that serve the purpose of adjusting hydrodynamics. Depending on the number of closed and open holes, the weight and volume of the prosthesis will be different. The use of a wetsuit is hypothesized to allow greater anchorage to the body. Such a prosthesis can be 3D printed probably in materials such as nylon or carbon fiber, but it has never been produced with this technology (Fig. 5). Lord Randall Wade & Lord Lori Elizabeth patented Swim fin for leg amputees in 2014. A swimming prosthesis for lower limb standard in size comprising a component of an end part and an interface. These two parts are held together by welding, glue, mechanical fastening or integral fabrication. The interface part is designed in softer material than the end part to provide better comfort (Fig. 6).

Fig. 5. Prosthetic limb and prosthetic limb attachment for swimming, Goldstein;

Design for Well-Being, Empowerment and Experience of Care

539

Fig. 6. Swim fin for leg amputees, Lord Randall Wade & Lord Lori Elizabeth;

Gareth E. Steele in 1992 patented Swim fin for an amputee: a swimming prosthesis specifically designed for indiscriminate use by users with both lower and upper limb amputations. It features two straps that attach the prosthesis to the affected body part, wrapping around the limb. This standardization also includes the anchoring aspect to the body, both the weight of the prosthesis and the size (Fig. 7). The last patent analyzed is Swimming prothesis, dating back to 2020 owned by Rolf Michael. The invention relates to a swimming prosthesis comprising: a connecting member adapted to be coupled to an amputation stump of an amputee; a swimming member having a concave shape and is attached to the distal end of the connecting member so that it can rotate with the proximal end. The prosthesis fits both upper and lower limb, providing standard sizes. The human-prosthesis interface is created through movable connecting elements. Hydrodynamics is addressed through adjustment of the end of the prosthesis. Depending on the style this will have different propulsion in water (Fig. 8).

Fig. 7. Swim fin for an amputee, Gareth E. Steele;

540

P. Losciale et al.

Fig. 8. Swimming prothesis, Rolf Michael;

6 Results and Definition of Design Requirements From the state of art research conducted, there was a general trend toward standardization and adaptability of devices by which is meant the presence of standard prostheses or at most with some degree of reconfigurability that allows their use in disparate situations including interchangeability between upper and lower limbs. To be even more specific, no prosthetic device designed specifically for the upper limb was found despite the relevant role of the same in the swimming gesture and an overall dynamics of the same not strictly comparable to that of the lower limb. The prior art analysis carried out on the patents and patent-free products already detailed shows devices that can be worn by all users without taking into account at the design stage the diversity of age, sex, gender, disability specificity (some are for both upper and lower limb), necessary weight, etc., in a Design for Average development perspective. This aspect represents an important gap in a context where the paradigm related to personalized medicine is becoming more and more prominent and, from a practical-operational point of view, impacts the quality of current devices, which will not be able to be sufficiently comfortable and enabling. Indirect confirmation of the above can also be found in the low use of prostheses during the act of swimming. In the analyzed devices, the human-device interface (socket), size, weight, shape, and buoyancy in water appear to be the same for all. The design process is standard and nowhere involves the CAD CAM method or 3D printing manufacturing. Further aspect recorded is the complete absence of point hydrodynamic evaluation based on pressures during the swimming act (comparison between active and residual limb) and the weight of the active limb. In some of the cases analyzed, hydrodynamics can be modified manually through a system of holes (to be opened or closed) at the user’s convenience but without a benchmark with the consequent possibility that the physiotherapeutic act may be undertraining or overtraining depending on the chosen configuration. There are also no devices specifically designed for children and their balancing in water during sports or physiotherapy practice. In all cases studied, the systems cannot be adapted to very young users.

Design for Well-Being, Empowerment and Experience of Care

541

6.1 Design Requirements Subsequently to the results obtained from the analysis provided by the research, the requirements for the prosthetic system can be outlined. It follows that the prosthesis to be designed: – will be useful for balancing in water during children’s first approach to sporting activity, ensuring shorter settling time and improving outcomes of water physiotherapy; – will be targeted to children with upper limb malformations (particularly trans-radial agenesis) with ages between 4 and 10 years old; – each prosthesis should be tailored to each user, ensuring comfort, convenience, no pain, and proper muscle stimulation on the residual limb. Customization involves making an ad hoc socket; equalizing the weight of the prosthetic device to that of the normal limb; equalizing the fluid dynamic load of the prosthetic device to that of the healthy limb; – must be integrated into the environment: possible use of the costume, customized shapes and colors and integrated into the context for greater acceptance of the prosthetic system by very young users; – will use CAD-CAM scanning and modeling methods that enable greater product economy, user welfare, design accuracy, replication process; – will need to be easily replaceable and therefore cheap;

References Bryson D, McCann J (2009) Smart clothes and wearable technology. Woodhead publishing series in textiles, New York Cross N (2019) Design thinking: understanding how designers think and work. Ava Pub Sa, Liverpool Maldonado T (1976) Disegno industriale: un riesame. Feltrinelli, Milano Manzini E (2015) Design, when everybody designs: an introduction to design for social innovation. The MIT Press, Cambridge Norman D (2014) The design of everyday things, revised and expanded edition. The MIT Press, Cambridge Rizzo F (2009) Strategie di co-design. Teorie, metodi e strumenti per progettare con gli utenti. Francoangeli, Milano Tosin D (2014) Elle prosthesis. Della Tosin Design. https://dellatos.in/ UniversalDesignStyle (2014) Oyogu | Swimming Arm Prosthetic Concept. Universal Design Style. https://www.universaldesignstyle.com/oyogu-swimming-arm-prosthetic-concept/ Wt award innovation design competition (2012) Neptune Concept. https://www.well-tech.it/WTa ward/accessibility/Acce12_NeptuneConcept_RichardStark.html Zoccali C (2017) La stampa 3D in ortopedia: indicazioni e limiti. Giornale Italiano di Ortopedia e Traumatologia 43:125–132

Cognitive Load Theory Analysis of Medical Signage System for Seniors Based on General Hospitals in Shanghai Jieqiong Fan1 and Albert Young Choi2(B) 1 Graduate School, College of Design, Hanyang University ERICA, Ansan, South Korea

[email protected]

2 Faculty of Graduate School, Communication Design, College of Design,

Hanyang University ERICA, Ansan, South Korea [email protected]

Abstract. The changing demographic structure of the population has sparked increased interest in geriatric programs and industries. Among these concerns, the pressing matter of mitigating the cognitive load among the seniors within medical environments and guaranteeing their seamless and secure access to medical services through signage systems has emerged as an exigent issue. This study focuses on three comprehensive hospitals in Shanghai and is grounded in the cognitive load theory and the cognitive characteristics of the seniors. It employs literature analysis and qualitative research methods to investigate the design issues and improvement directions of medical signage systems for the seniors. The research findings reveal that the senior’s population generally experiences a higher cognitive load when using medical signage systems. This is primarily evidenced by difficulties in comprehending signage information and swiftly recognizing crucial details, highlighting deficiencies in signage system design. Accordingly, this study presents recommendations tailored to the senior’s population to reduce their cognitive load, enhance the overall usability, and improve the accuracy of medical signage systems. Additionally, this research offers insights and discussions regarding the application of cognitive load theory in the field of design. Keywords: Cognitive Load · Medical Signage System · Seniors · Wayfinding

1 Introduction Research background and purpose: The ageing population is a powerful and transformative demographic force [1]. The World Health Organization (WHO) has advocated the importance of developing age-friendly environments by introducing the principles of proactive age-friendly healthcare [2]. Medical environments as one of the important environments essential to an aging society, should provide aging-friendly medical spaces and safe and comfortable medical environments for the seniors. As a vital component of medical environments, medical signage systems play a pivotal role in conveying medical information, guiding patients, and ensuring patient safety. However, due to the decline © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 542–554, 2024. https://doi.org/10.1007/978-3-031-47281-7_45

Cognitive Load Theory Analysis of Medical Signage System

543

in the functions of various tissues and organs of the seniors, the perceptual and cognitive abilities gradually deteriorate, which affects the perception of space, spatial cues, and signage to varying degrees. Inadequate design of medical signage systems can exacerbate the cognitive load on senior individuals, causing unnecessary distress and risks. This study aims to explore the cognitive load issues faced by the senior when using medical signage systems and proposes corresponding improvement recommendations to reduce cognitive load and enhance the usability and accuracy of these systems. Based on the Cognitive Load Theory and the cognitive characteristics of senior individuals, we conducted an evaluation and analysis of exemplary medical signage system designs. Employing literature analysis and qualitative research methods, we delved into the current state of medical signage systems in three general hospitals in Shanghai, summarizing prevalent issues, and offering improvement recommendations. The aim is to alleviate cognitive load among the senior and enhance their medical experience.

2 Theoretical Background 2.1 Cognitive Load Theory (CLT) In the late 1980s, educational psychologist John Sweller developed cognitive load theory out of a study of problem solving [3]. Cognitive load theory refers to the total amount of mental activity imposed on an individual’s cognitive system during a given operational time (Sweller 1988). The cognitive resources of an individual are limited, seniors need to consume a certain number of cognitive resources to complete the wayfinding task in the medical space. If the cognitive resources consumed in completing the task exceed the total cognitive resources that an individual has, it will cause cognitive load. John Sweller stated that the basic goal of cognitive load theory is to reduce extrinsic load. Researchers, represented by John Sweller et al. have classified cognitive load into three categories, namely intrinsic cognitive load (ICL), extraneous cognitive load (ECL), and germane cognitive load (GCL), based on the source and nature of the main factors influencing cognitive load [4]. Intrinsic Cognitive Load (ICL). Refers to the inherent level of mental effort or cognitive processing required by an individual to understand and process a specific task or information. It is associated with the inherent complexity of the task or content itself, regardless of how it is presented or organized. [5, 6]. In an unfamiliar medical space, the wayfinding task itself is difficult, and if the information of the signage system is too complex and beyond the knowledge reserve of the Seniors, there is a risk of working memory burden and higher cognitive load. Extraneous Cognitive Load (ECL). Refers to the cognitive load that is imposed on an individual due to the way information or tasks are presented or organized [5]. I posit that the environmental context surrounding the signage system, alongside the layout and color schemes employed in the wayfinding system itself, represents one of the foremost factors influencing the external cognitive load experienced by seniors. Substantiation of this hypothesis will be undertaken in the subsequent sections of this discourse. Germane Cognitive Load (GCL). A concept in cognitive load theory that refers the cognitive effort directly related to the acquisition and integration of new information or skills [8]. In the design of medical signage systems, how to reduce the germane

544

J. Fan and A. Y. Choi

cognitive load to enhance the wayfinding experience of seniors in complex healthcare environments will be discussed in the following sections. 2.2 Cognitive Characteristics of the Seniors Cognition refers to the mental processes and activities related to acquiring, processing, storing, and using information. It encompasses a wide range of mental processes, including perception, attention, memory, language, problem-solving, decision-making, and reasoning, among others. Essentially, cognition is how the brain processes and makes sense of the world around us and the information we receive through our senses [7]. Salthouse and others proposed the Theory of Age-Related Differences in Cognitive Processing Speed in 1966. The core of this theory is that as individuals age, their cognitive processing speed gradually declines [9]. Hasher, Stoltzfus, Zacks and Rypma proposed the inhibition theory of cognitive ageing, which states that as individuals age, it becomes increasingly difficult for them to focus on target information and to inhibit attention to irrelevant information [11]. Irrelevant information occupies the capacity of cognitive resources, and the diminished ability of the seniors to suppress irrelevant information increased cognitive load due to difficulty concentrating. Craik and Byrd suggested that older people experience cognitive decline because they lack an ability to ‘self-initiate processing’, as expressed in working memory, which researchers have since termed working memory theory [12]. Decreased attention and working memory make it take older people more time and effort to complete a task, and they are more likely to be distracted during the task, affecting visual search performance interfering with working memory and causing cognitive lapses [13]. The deterioration of working memory capacity and coding storage capacity prevents seniors from processing information reasonably efficiently at work and increases the working memory load. Lindenberger and Baltes (1994) found that basic audio-visual sensory functions have strong explanatory power for cognitive ageing [10]. With age, visual acuity decreases, and the cornea undergoes changes, leading to reduced sharpness of vision, affecting color discrimination, and causing a reduction in the field of view. It is evident that with advancing age, physiological changes in the human body lead to cognitive impairments, characterized by diminished sensory capabilities, slower cognitive processing speed, reduced attention and memory, and a decline in information processing abilities, among other factors. 2.3 Medical Signage System Design In general, older people have greater symbol comprehension problems than younger people (Lesch 2003; Shorr et al. 2009; Kim et al. 2009) [14–16]. A well-designed signage system together with other environmental features, appropriately designed for the target audience, can help orientate them and prevent confusion, frustration, stress, and even time losses [16]. Reduces difficulties and stress associated with the wayfinding tasks, and consequently help in health treatments and recovery [16]. To provide users with an effective wayfinding signage system, Rousek and Hallbeck, suggest that signage design must consider four factors to be effectively designed, signage recognition, signage color, signage font type, Americans with Disabilities Act guidelines [17].

Cognitive Load Theory Analysis of Medical Signage System

545

3 Reference Cases of Well-Designed Medical Signage System Based on the theoretical background, three representative medical signage system cases have been selected according to the nine effective medical signage design categories proposed by Rodrigues et al. (2019). (1. Text formatting; 2. Information hierarchy and density; 3. Language and terminology; 4. Symbols and pictograms; 5. Colors; 6. Placement, dimensions and typology; 7. Illumination, visibility and legibility; 8. Standardization; 9. Inclusivity and user characteristics), and they are subjected to an efficacy analysis [18]. 3.1 Humber River Hospital Located in Toronto. At the time of launch it was the largest hospital in Canada and the first fully digital hospital in North America. Hospital signage system introduces the design concept of airport signage, which is uniform in style, simple and sharp with strong colors. The color-coded visual system is carried out through wall and ceiling treatments, maps, directional signs, and digital signage. The surrounding environment is clean and tidy, free of irrelevant information, reducing the external load brought to the seniors by the environment, which allows seniors patients and visitors to easily identify their destination. For an analysis of the effectiveness of the signage system see Table 1. Table 1. Humber River Hospital Signage System Analysis Categories

Analyze

Text formatting

Information layout is clear, left -aligned, sans-serif font, clear font size contrast, and high readability.

Information hierarchy and density

Clear, information listed according to importance and highlighted using color and font to emphasize primary or secondary information.

Language and terminology

No long sentences, medical abbreviations, or difficult words were used to reduce the intrinsic cognitive load.

Symbols and pictograms

Public icons and pictograms are clear and easy to understand.

Colors

Using color variations for segmentation enhances information segregation and reduces extraneous cognitive load.

Placement, dimensions, and typology

Incorporating airport signage design concepts enables older individuals to intuitively grasp sign information from various directions.

Figure

Illumination, visibility, and Abundant natural lighting was utilized, without the use of reflective materials, legibility resulting in high visibility and readability. Standardization

The signage system is friendly for seniors and people with disabilities.

Inclusivity and user characteristics

A digital wayfinding strategy has been developed to facilitate access for seniors and persons with disabilities.

3.2 Home - Tachikawa General Hospital It is a comprehensive medical facility located in Tachikawa, Japan. The signage system’s design received the Japan Sign Design Association (SDA) Excellent Award in 2017. The signage system features a large, mall-like space as its backdrop, with floating white

546

J. Fan and A. Y. Choi

numbers drawing attention. As senior pass the reception desk, they can easily discern their directions and locations at a glance. The walls, ceilings, lighting, and flooring exhibit a variety of colors and materials. The signage system demonstrates remarkable effectiveness, boasts high utilization, and excels in user-friendliness (see Table 2). Table 2. Home - Tachikawa General Hospital Signage System Analysis Categories

Analyze

Text formatting

Information layout is clear, left-aligned or right-aligned, rounded sans-serif font, numbers vs. text, graphics vs. text size contrast is evident.

Information hierarchy and density

In situations where the information hierarchy is the same, there is higher information density with less emphasis on key information.

Language and terminology

No long sentences.

Symbols and pictograms

Public icons and pictograms are clear, easily understandable, and frequently used.

Colors

Use of color variations for spatial segmentation to enhance cognitive memory.

Placement, dimensions, and typology

Floating signage for clarity, People are surrounded by signage information at an appropriate height whether they are standing or sitting.

Figure

Illumination, visibility, and Part of the guide as a whole is a light box, which serves both illumination and legibility guidance. The lighting around the wall signs is soft and non-glaring. Standardization

The signage system is friendly for seniors and people with disabilities.

Inclusivity and user characteristics

The environment is clean and tidy and easy for the elderly and disabled to orient themselves, but there is no other digital signage system.

3.3 Signaletik SLK-Kliniken Heilbronn SLK-Kliniken Heilbronn GmbH operates three acute hospitals and a geriatric rehabilitation clinic. The medical signage system combines colors, letters, numbers, and icons in a harmonious and highly functional guidance system. Among other things, complex guiding information is transformed into precise elements and iconographic symbols. These principles are coded as letters for building parts and numbers for stores and lead through the hospital in the form of icons. The signage system is fully accessible for the seniors and disabled (Table 3). Through the analysis of medical signage systems in three hospitals, it was observed that effective signage systems for the elderly, aimed at reducing cognitive load during navigation, typically employ sans-serif fonts, employ left-aligned and right-aligned text layouts, incorporate a multitude of pictorial symbols, high-contrast colors, avoid lengthy sentences and abbreviations, and adhere to standardized design principles.

Cognitive Load Theory Analysis of Medical Signage System

547

Table 3. SLK-Kliniken Heilbronn GmbH Signage System Analysis Categories

Analyze

Text formatting

Information layout is clear, left-aligned or right-aligned, sans-serif font.

Information hierarchy and density

Clear, information listed according to importance and highlighted using color to emphasize primary or secondary information.

Language and terminology

No long sentences, medical abbreviations, or difficult words were used to reduce the intrinsic cognitive load.

Symbols and pictograms

Custom pictograms are clear, easily understandable, and frequently used. The overall impression is completed by wall paintings, foil plots, and foil signs.

Colors

Colorful, Use of color variations for spatial segmentation to enhance cognitive memory.

Placement, dimensions, and typology

The ceiling signage and the wall signage are installed in a reasonable position and in the right size, making it easy to be viewed by any group of people.

Illumination, visibility, and legibility

The circular target points of the illuminated tracks have been adjusted in color, size, and typography to meet the needs of severely visually impaired individuals using the Visually Impaired Simulator from ABSV. Resulting in high visibility and readability.

Standardization

Signage System fonts have been developed in accordance with accessibility standards.

Inclusivity and user characteristics

The font "Neue Frutiger 1450" was first developed in accordance with the new accessibility regulation DIN 1450 in 2013. Barrier-free orientation is supported by 60 tactile handrail signs.

Figure

4 Selected Case Studies - Three General Hospitals in Shanghai. Different countries and regions may face unique challenges and limitations in the healthcare sector. To gain further insights, we conducted an in-depth analysis of the wayfinding systems in three comprehensive hospitals in China, shedding light on potential issues. Through on-site investigations and interviews at top-tier hospitals in first-tier Chinese cities, we selected three Shanghai-based comprehensive grade 3 hospitals for the analysis, based on criteria such as citizen satisfaction, medical standing, and influence. These hospitals include Longhua Hospital affiliated to Shanghai University of Traditional Chinese Medicine (A), Zhongshan Hospital affiliated to Fudan University (B) and Huashan Hospital affiliated to Fudan University (C). The analysis of the signage systems of three hospitals was conducted based on the nine effective medical signage design categories proposed by Rodrigues et al. The analysis is as follows. 4.1 Longhua Hospital Affiliated to Shanghai University of Traditional Chinese Medicine (A) Through conducted personal visits and on-site research, it became evident that the hospital environment exhibited signage of slight aging, with noticeable modifications and patches on the signage system. While the signage boards displayed a wide variety of types, their abundance and diverse color palette, when not used judiciously, led to visual clutter. A detailed analysis is presented in Table 4.

548

J. Fan and A. Y. Choi

Table 4. Shanghai Longhua Hospital Affiliated to Shanghai University of Traditional Chinese Medicine Signage System Analysis Categories

Analyze

Text formatting

The use of sans-serif font is observed; however, the alignment is ambiguous, the font size is diminutive, the strokes are fine, spacing between characters is excessively cramped and inconsistent, and there is an absence of utilizing font stroke thickness variation to distinguish information.

Information hierarchy and density

The signage is excessively crowded and cluttered, with information hierarchy primarily differentiated by color coding; however, the organization of information layers lacks clarity.

Language and terminology

Department titles are easily understandable, with no use of abbreviations, although some lengthy technical terms are present.

Symbols and pictograms

Public symbols are used sparingly, and pictograms are not used.

Colors

Utilizing blue and orange as the unified color palette with variations in similar hues within this spectrum, the presence of multiple colors on signage has the potential to induce visual confusion.

Placement, dimensions, and typology

The signage varies in terms of size and placement, which can sometimes lead to inconsistency.

Illumination, visibility, and legibility

The signage is generally well-lit, ensuring good visibility, but legibility com-promised due to smaller text.

Standardization

There is room for improvement in standardization, as some signs deviate from the established design norms.

Inclusivity and user characteristics

The signage information is not user-friendly for senior individuals or visually impaired patients. The digital registration system in place, but it is used less frequently by seniors, with the majority of users being younger individuals.

Figure

4.2 Zhongshan Hospital Affiliated to Fudan University (B) The overall design of the signage system maintains a uniform style, demonstrating consistency in typography, text, and color. However, it exhibits a slightly outdated appearance, and there are concerns regarding the sustainability of the signage. Notably, the system lacks the capability to rectify information errors, which could have implications for its aesthetic appeal. A detailed analysis is presented in Table 5. 4.3 Huashan Hospital Affiliated to Fudan University (C) The overall design style of the signage system is consistent, featuring the hospital’s branding, which contributes to building the brand image. The signage in the Convenience Service Center incorporates a small number of graphic elements, enhancing text comprehension. The color scheme is uniform and eye-catching, ensuring high readability. A detailed analysis is presented in Table 6. An effectiveness analysis of the signage systems in three Shanghai general hospitals was conducted through on-site investigations and observational analyses, revealing several common issues. In terms of design, the signage systems of these hospitals exhibited a similar design style. Text formatting, typography, and colors were consistent, reflecting a more standardized design. However, these systems heavily relied on text-based information, with a lack of development and utilization of symbols and pictograms, posing challenges for elderly individuals in comprehending the provided information.

Cognitive Load Theory Analysis of Medical Signage System

549

Table 5. Zhongshan Hospital Affiliated to Fudan University Signage System Analysis Categories

Analyze

Text formatting

Information layout is clear, left-aligned or center-aligned, sans-serif font. Letter spacing and line spacing are wide, ensuring high readability.

Information hierarchy and density

Information hierarchy primarily differentiated by color coding; however, the organization of information layers lacks clarity.

Language and terminology

Multiple instances of highly specialized terminology and abbreviations are used.

Symbols and pictograms

Public symbols are used sparingly, and pictograms are not used. There are mural walls in the corridors that can enhance the memory of senior individuals.

Colors

Using blue as the unified color tone, variations in brightness are applied within the blue range to emphasize information.

Placement, dimensions, and typology

The ceiling signage and the wall signage are installed in a reasonable position and in the right size, making it easy to be viewed by any group of people.

Illumination, visibility, and legibility

Indoor lighting is sufficient, the wall signage materials are non-reflective, and the ceiling signage materials are self-illuminating, ensuring high visibility and readability.

Standardization

There is room for improvement in standardization, as some signage information deviate from the established design norms.

Inclusivity and user characteristics

The signage information features large text, bold strokes, and wide spacing, making it convenient for elderly or visually impaired individuals to read. There is also a voice broadcasting system in place.

Figure

Table 6. Huashan Hospital Affiliated to Fudan University Signage System Analysis Categories

Analyze

Text formatting

Information layout is clear, left-aligned, sans-serif font, high readability.

Information hierarchy and density

Information hierarchy is distinguished through color coding and enlarged fonts, but the organization of information layers within the same area lacks clarity.

Language and terminology

No long sentences, medical abbreviations, or difficult words were used to reduce the intrinsic cognitive load.

Symbols and pictograms

Pictograms are used at elevator locations and service centers, aiding recognition, and memorization.

Colors

The use of red as a unified color scheme creates both consistency and cohesion.

Placement, dimensions, and typology

The ceiling signage and the wall signage are installed in a reasonable position and in the right size, making it easy to be viewed by any group of people. However, due to the less-than-tidy surroundings, they appear somewhat cluttered.

Illumination, visibility, and legibility

Indoor lighting is sufficient, the wall signage materials are non-reflective, and the ceiling signage materials are self-illuminating, ensuring high visibility and readability.

Standardization

There is room for improvement in standardization, as some signage information deviate from the established design norms.

Inclusivity and user characteristics

There is a voice broadcast system and electronic screens available, which facilitate the usage for senior individuals as well as those with visual or hearing impairments.

Figure

Some textual information was positioned outside the normal visual range, leading to readability issues. While the color scheme was uniform, it lacked differentiation based on functional areas or department categories, necessitating additional time and cognitive effort from senior individuals to understand relevant information, thereby increasing cognitive load. The signage systems featured a variety of signage types, contributing to information overload and higher memory demands for senior users. Additionally, the

550

J. Fan and A. Y. Choi

surrounding environment planning for these systems appeared suboptimal, frequently presenting irrelevant information that could divert the attention and impair the memory of senior individuals. To comprehensively assess the practical effectiveness of the signage systems, we conducted interviews with 10 senior participants in each of the three comprehensive hospitals. These interviews aimed to gather their perspectives on the signage systems. The interview questions are outlined in Table 7. Table 7. Interview Questions Questions No Questions Q1

Your age and level of education?

Q2

Do you frequently use the signage system within the hospital?

Q3

Has the signage system been helpful to you in finding your way?

Q4

Have you ever encountered situations where the signage system was difficult to understand? Can you share some examples?

Q5

Which signage elements do you find the easiest to understand and use?

Q6

Do you have any suggestions or recommendations for improving the signage system’s friendliness towards senior citizens?

Through interviews with seniors, we have gained valuable insights into the medical signage system. The interviewees had an average age of 69 years and varied levels of education. During the interviews, it was observed that more than 50% of the seniors frequently utilize the signage system and acknowledge its functionality and effectiveness. However, some expressed concerns regarding the legibility of the floor indices, noting that the text was too small to read. They emphasized that larger fonts, simplified symbols and text, and distinct colors were immensely helpful for wayfinding. A minority of the seniors mentioned that the abundance of signage throughout the hospital could be overwhelming, leading to confusion about which signage to follow. In such cases, they relied on healthcare staff for guidance. These insights provided by seniors have been instrumental in enhancing our understanding of practical issues with the signage system.

5 Design Suggestions to Reduce Cognitive Load in the Senior Based on the definitions and characteristics of the three cognitive load types mentioned earlier, we have categorized the 9 effective medical signage design categories proposed by Rodrigues et al. into their respective cognitive load types. This categorization enables us to provide more specific and targeted improvement recommendations for each category, thereby better mitigating cognitive load for older adults. Additionally, in conjunction with the strengths identified in three successful case studies, we have established practical exemplars for the design of medical wayfinding systems. These exemplars not only serve to validate our proposed recommendations in practice but also ensure their

Cognitive Load Theory Analysis of Medical Signage System

551

feasibility and real-world effectiveness. Specific design recommendations are shown in Table 8. Table 8. Design Suggestions to Reduce Cognitive Load in the Senior Type

Categories

Design Suggestions

Intrinsic Cognitive Load (ICL)

Information hierarchy and density

1) To avoid signage overload or the absence of valuable information, careful planning of sign types and their frequency of use is necessary. Avoid signs that are overly dense, inconsistent, or cluttered 2) Information should be organized hierarchically, listing items by importance, and emphasizing primary and secondary information through font size, thickness, proportion, and color contrast 3) The arrangement order of destinations in signage should align with the actual spatial arrangement 4) Modularize the information hierarchy into groups and increase the spacing between groups

Language and terminology

1) Avoid using long sentences, abbreviations, and highly technical terms. Use short sentences that are easier to understand and memorize 2) Distinguish the terms with the help of different typeface weights, text, color Recommend combining text with graphics

Extraneous Cognitive Load (ECL) Text formatting

1) Utilize a bold sans-serif font such as Frutiger, Univers or Helvetica; emphasize information through variations in font size and thickness; Choose left-aligned or right-aligned text alignment and avoid center alignment 2) Ensure that the height of letters falls between 75% and 100% of their width. Maintain consistent spacing between Chinese characters and between Chinese characters and English characters

(continued)

552

J. Fan and A. Y. Choi Table 8. (continued)

Type

Categories

Design Suggestions

Symbols and pictograms

1) It is recommended to develop a hospital-specific symbol and pictogram system Integrate these symbols and pictograms with text where necessary, allowing symbols or pictograms to take precedence over text when needed 2) Enhancing the Use of Universal Symbols and Pictograms. Symbols or pictograms should be at least 76.20 to 203.2 mm in height to be legible [18]

Colors

1) Ensure consistency in color usage and tone uniformity throughout the signage system 2) Suggest using high-contrast color combinations to differentiate between primary and secondary information, such as red-white and blue-white 3) Recommend using variations in color to delineate different medical functional areas

Placement, dimensions, and 1) Placement should be determined typology based on pedestrian flow, viewing angles, and primary walking paths, such as main routes and turning points. Avoid placing signage directly in front of or behind intersections 2) Signage should be placed at a height that is easily visible and accessible to a wide range of individuals, including those with disabilities. In accordance with Americans with Disabilities Act guidelines, the recommended height for signage should be between 48 in. (122 cm) and 60 in. (152 cm) above the finished floor level 3) Signage types at the same hierarchy level should remain consistent, including font, size, text formatting, color, material, appearance, and installation method

(continued)

Cognitive Load Theory Analysis of Medical Signage System

553

Table 8. (continued) Type

Germane Cognitive Load (GCL)

Categories

Design Suggestions

Illumination, visibility, and legibility

1) Ensure sufficient indoor and outdoor lighting, especially in the vicinity of signage. Avoid glare and shadows to reduce visual distractions. Consider utilizing natural light or soft lighting to reduce glare and reflections, thereby enhancing visual comfort 2) Ensure signage avoids the use of materials that are reflective or prone to glare, maintaining a smooth and non-reflective surface

Standardization

1) Americans with Disabilities Act, ADA 2) International Organization for Standardization, ISO 3) Human Factors Engineering

Inclusivity and user characteristics

1) Americans with Disabilities Act, ADA 2) Use raised characters, symbols, or Braille with a size ranging from 13 to 25 mm and spacing between 7.6 to 203.2 mm, utilizing a sans-serif typeface [18]

In addition to the observations of signage usage, it was noted that seniors often encountered issues such as queuing in the wrong lines and taking incorrect routes. To enhance wayfinding efficiency, it is recommended to plan the overall layout of the hospital thoughtfully, ensuring smooth spatial connections between different departments and functional areas. Avoiding congested and complex environments, reducing branching corridors and rooms can simplify the wayfinding process for seniors.

6 Conclusions In this study, we conducted an in-depth analysis of the design and effectiveness of the medical signage system, found the factors that affect the cognitive load of the senior, and put forward design recommendations to reduce the cognitive load of the senior. It provides a reference for the design of the medical space signage system for the senior and provides reference and exploration for the cognitive load theory in design practice. However, there are still many unexplored aspects in this field. First, more extensive user experiments should be conducted in future research to verify the actual effect of our recommendations on the senior. Second, due to the continuous updating and technological development of the medical signage system, explore how technology-driven innovation affects the design and user experience of the signage system. In addition, it is

554

J. Fan and A. Y. Choi

planned to gain an in-depth understanding of the use experience of the signage system by hospital staff in order to create more comprehensive design guidelines. Finally, we will continue to pay attention to the standardization of medical signage system design internationally to ensure that our research results are instructive to the wider medical signage system design community. These future research directions will further promote the development of medical signage systems to better meet the needs of the senior and other users.

References 1. Kuo YL, Chen IJ (2019) Facilitating a change model in age-friendly hospital certification: strategies and effects. PLoS ONE 14(4):e0213496 2. World Health Organization: Age-friendly environments programme. World Health Organization, Geneva (2013) 3. Sweller J (1988) Cognitive load during problem solving: effects on learning. Cognit Sci 12(2):257–285 4. ี្ (2022) ᇶன宋䟌崆卞䏭审䙫䔳杉⛥㟮忩俨⋽宥守䟻䩝 . (䠼⣒⭍ἴ审㕮,ᒣḃ⤎⭍). https:// kns.cnki.net/KCMS/detail/detail.aspx?dbname=CMFD202301&filename=1022047424.nh 5. Sweller J (2011) Cognitive load theory. In: Psychology of learning and motivation, vol 55. Academic Press, pp 37–76 , (2008) . 6. 44(2):75–83 (2019) . Beijing Book Co. Inc. 7. 8. Debue N, Van De Leemput C (2014) What does germane load mean? An empirical contribution to the cognitive load theory. Front Psychol 5:1099 9. Salthouse TA (1996) The processing-speed theory of adult age differences in cognition. Psychol Rev 103(3):403 10. Lindenberger U, Baltes PB (1994) Sensory functioning and intelligence in old age: a strong connection. Psychol Aging 9(3):339 11. Hasher L, Stoltzfus ER, Zacks RT, Rypma B (1991) Age and inhibition. J Exp Psychol Learn Mem Cognit 17(1):163 12. Craik FI, Byrd M (1982) Aging and cognitive deficits: the role of attention-al resources. Aging Cognit Processes 191–211 13. 㯤ᕼᗞ, 㝏㲢剖, ᯘᓫᚫ , ᚰ⌮Ꮫ኱㎡඾ (2003) ୖᾏ: ୖᾏᩍ⫱ฟ∧♫ 1582 14. Lesch MF (2003) Comprehension and memory for warning symbols: age-related differences and impact of training. J Safety Res 34(5):495–505 15. Kim S, Ulfarsson GF, Anton KR (2009) Traffic sign comprehensibility in an aging society: a study of “photo-enforced traffic signal ahead” signage. Transp Res Rec 2096(1):81–88 16. Carpman JR, Grant MA (2016) Design that cares: planning health facilities for patients and visitors. Wiley, New York 17. Rousek JB, Hallbeck MS (2011) The use of simulated visual impairment to identify hospital design elements that contribute to wayfinding difficulties. Int J Ind Ergon 41(5):447–458 18. Rodrigues R, Coelho R, Tavares JMR (2019) Healthcare signage design: a review on recommendations for effective signing systems. HERD: Health Environ Res Des J 12(3):45–65

Neuroergonomic Models and Tools Compared to Evaluate and Improve Human-Machine Interaction in Manufacturing Ilaria Lombardi1(B) , Mario Buono1 , Víctor Fernando Muñoz Martínez3 Vincenzo Paolo Senese2 , and Sonia Capece1

,

1 University of Campania Luigi Vanvitelli, Aversa, Italy

[email protected]

2 University of Campania Luigi Vanvitelli, Caserta, Italy 3 University of Málaga, Málaga, Spain

Abstract. Industrialisation and the rapid progression in automation and digitalization have supported the common use of tools and machines in everyday life and workplaces, especially in industrial environments. The paper proposes a criticalanalytical investigation aimed at fostering reflections on Human Factors and Neuroergonomics concepts and approaches to improve human-machine interactions in the workplace and with the support of Neuroimaging technologies and metrics used to track the cognitive state of the user-operator. The primary objective was to identify the industrial sectors most exposed to psychosocial risks, in order to prefigure specifically adapted and highly contextualized solutions. Particular attention was given to the integration of wearable and smart devices, designed to amplify operator safety and to accurately monitor the user’s psychophysical state. By exploring and analysing the state of the art of existing wearable and intelligent devices, critical factors, functionalities, performance, innovative architectures and technologies from different industries emerged. The results of this work have allowed critical factors and opportunities to emerge that are useful in designing and developing work environments that are safe, efficient and focused on the well-being of operators. Keywords: Neuroergonomics approach · Human Factors · Industry 4.0 · Occupational safety · Mental workload

1 Introduction The reports on new and emerging risks (ESENER-3) by European Agency for Safety and Health at Work (EU-OSHA) (2019) as part of the third European survey of enterprises and recent Eurostat data (2023) show that occupational health and safety risks are being analyzed with a greater focus on psychosocial risks related to work and new technologies. On the one hand, critical points are highlighted in relation to psychosocial and ergonomic factors mainly related to human-machine interaction, and on the other hand, the role that mechanisation and digitalisation have to play in mitigating these risks is explored © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 555–568, 2024. https://doi.org/10.1007/978-3-031-47281-7_46

556

I. Lombardi et al.

(Niciejewska and Idzikowski 2022). According to the analysis of statistical data from the main European databases, such as EU-OSHA (2023), Eurostat (2023) and WHO (World Health Organization) (2023), the construction, transport and storage, manufacturing, agriculture, silviculture and fishing sectors together account for about two thirds (63.1%) of all fatal occupational accidents and more than two fifths (44.1%) of all non-fatal occupational accidents in 2020. In this year, of all fatal accidents at work in the EU, the manufacturing sector (15.2%) had the highest share. The highest share of non-fatal occupational accidents in the EU occurred on industrial sites for the manufacturing sector, totaling 78.1%, with loss of control of machines, tools or transport and handling equipment (20.8% of the total), movement of the body under or with physical stress (18.4%) and slipping or falling (17.6%) being the most common causes. Loss of control of machines, tools or transport and handling equipment was also the most common cause of fatal accidents at work, accounting for 23.2% of the total number of work-related fatalities in the EU in 2020. For the same year, the most common contact modes for non-fatal accidents in the EU included: physical or mental stress (23% of all non-fatal accidents); impact with a stationary object (in other words, the victim was in motion - 21.0%); contact with a sharp/sharp or rough/rough agent (14.7%); and being struck by a moving object/collision (11.5%) (Eurostat 2023). A detailed review of the relevant context through a critical analysis of the state of the art and national and international standards has shown that operators are often unable to develop or maintain adequate levels of awareness, due to risk factors (see Fig. 1) that assess the ergonomic, psychological and organizational factors and interaction between the “operator” and “the environment”. In particular, it’s found from statistical data and literature research, that the greatest risks are related to ergonomic (Dekker et al. 2021), transversal and physical aspects such as the high complexity of systems, lack of experience in the use of systems and inadequate training and workstation design with respect to the physical-dimensional compatibility of the operator (ESENER 2019; Nawi et al. 2022; Razali et al. 2022). Against this, current “augmented” safety devices have been researched and systematized to support the operator in machine use and maintenance activities and to assess the operator’s psychophysical state.

2 Human Error, Mental Workload and Human Reliability The concept of the working environment encompasses, in a wide sense, the manmachine-environment paradigm in constant interaction between the physical and psychological spheres. It is therefore important to consider all the factors that contribute to the occurrence of an error or accident. A correct design of work activities and workstations must consider the cognitive load and anthropometry of the operator. With the aim to address these issues, it is important to estimate workers’ awareness of their own safety (Körner et al. 2019) and relate it to their state of mental stress and fatigue in order to improve interaction with machines. It is essential to analyse the actions between the operator and the machines, emphasising the distinction between correct/incorrect behaviour and unintentional error (such as malfunctions, faults, interface incompatibility, etc.), assessing environmental factors (microclimate, noise, lighting, etc.) and the working conditions related to the design of the workstation (La Fata et al. 2023).

Neuroergonomic Models and Tools Compared to Evaluate and Improve

557

Fig. 1. Histogram re-processed by comparing and synchronising statistical data on occupational safety in industrial establishments (ESENER 2019)

One of the major causes of accidents is the physical and mental fatigue of the operator. Fatigue is defined as a decrease in mental and/or physical performance caused by cognitive overload and physical exertion. Worker fatigue has been introduced as one of the main factors that increase the error rate of workers and lead to unsafe work actions, negatively affecting their alertness, reaction time and mental acuity. Therefore, the quantification of fatigue is fundamental in relation to occupational health and safety. In addition to being a physiological response of the human body that can prevent overload, fatigue is a symptom associated with several diseases and health conditions (Kirwan 2017). Fatigue impairs cognitive and/or motor performance, reducing work efficiency, productivity and product quality, and increasing the risk of injury and death (Körner et al. 2019). Since excessive or insufficient mental workload may be associated with reduced efficiency and safety of human-machine interactions, cognitive and physical stress must be analysed and evaluated in order to design new integrated and adaptive systems that can assist the operator (Derosière et al. 2013). It is necessary to assess the impact of human factors on risk through Human Reliability Analysis (HRA), the functions of ‘identifying which errors can occur (human error identification), what is the probability of these errors (error quantification) and, finally, identifying ways in which the probability and consequences of errors can be reduced (error reduction) (Kirwan 2017). Ayaz (2012) states that mental workload reflects “how hard the brain is working to meet the demands of the task”. Therefore, the last three decades have witnessed a revolution in understanding the brain processes that regulate human performance and attention of workers (Dehais et al. 2020). Moreover, HRA applications are still scarce in the manufacturing sector, where human errors are often overlooked and, therefore, there is a need for the creation of safety tools, based on neuroimaging technologies, that are sophisticated and portable and allow for a non-invasive examination of the ‘brain at work’ in real time, monitoring the operator and providing him with all the information he needs to complete his tasks while minimising risk factors. Cognitive evaluation emphasises the role of operators as they are involved in the acquisition and processing of information through a holistic assessment of how design

558

I. Lombardi et al.

influences the acquisition of information and, conversely, how the operator’s mental models (process understanding) influence the acquisition and processing of information. This point of view is significant in the context of contemporary digital implants, where humans generally act as decision-makers and perform monitoring, diagnosing, and prognostication functions. Based on this, it is possible to develop countermeasures, such as integrated intelligent systems, that “protect” the worker from problems related to the psychophysical sphere and ensure the safety and quality of work in manufacturing environments.

3 Design, Usability and Human Factors for HMI Evaluation The interconnection between operator and industrial machine requires adequate training of the worker in the use of the machine and knowledge of the characteristics of human control and human modelling for machine design (Lu et al. 2022). It is necessary to approach the study of Human Factors from a neuroscientific perspective, looking at the new paradigms of Industry 5.0 and focusing on the operator and his capabilities (Lombardi et al. 2023). Human-machine interaction, therefore, is considered in the totality of its aspects, including those related to human reliability, fatigue and physical/cognitive stress (Perrey et al. 2010) and systems usability. There is a shift from a task-centric view of design to a human-centric view, in which the entire system is designed to ensure well-being and usability (Buono et al. 2021); tools such as sensors, body scanners, and devices for detecting biometric and cognitive parameters allow for the immediate detection of specific user characteristics. Several researchers have attempted to address consumer usage behaviour regarding smart wearable devices by extracting potential technical and psychological factors using useroriented theories and models (Park 2020). This approach allows modelling the user experience through an ergonomic and cognitive study, the analysis of interaction systems and the evaluation of usability and accessibility requirements (Buono et al. 2021), thus helping to optimise the relationship between the user, device and its environment. A total of 35 HRA methods are identified in literature, divided into first- and secondgeneration methods (Bell and Holroyd 2009), which allow risk assessors to predict and quantify the probability of human error (Kim 2001). First-generation methods focus on human actions (errors) based on skills and rules, but do not consider context, human organisational factors and cognitive aspects. These methods include the technique for human error-rate prediction (THERP), which involves performing a task analysis to identify human involvement, define the sequence of events that must be performed to ensure safety, and then quantify these sequences using a human error probability database (Derosière et al. 2013). THERP also includes the calculation of dependency, which means that the success or failure of the current action is related to the previous action or task. Second-generation HRA methods, on the other hand, introduce cognitive models to characterise human behaviour in the workplace, searching for the root causes of human errors in the application of mental processes based on perception, thinking, memory and action decision strategy (La Fata et al. 2023). These methods include personal, contextual and cognitive factors (e.g. Performance Shaping Factors - PSFs) that can influence workers’ performance. In particular, the most significant PSFs on which

Neuroergonomic Models and Tools Compared to Evaluate and Improve

559

primary actions can be taken to improve worker reliability during task performance can be identified, which are: “Training”, “Procedures”, “Interface”, “Time pressure”, “Complexity”, “Workload, stress and Stressors”, “Environment”, “Physiological parameters”, “Work process”. The fundamental aim of these approaches/analyses is to make the system safer by decreasing various hazards and reducing the possibility of human error. For example, guidelines for the design of HMI (Human Machine Interface) in a control room should be geared towards ensuring the ease of interaction of a user-operator with the computer interface by analysing various aspects such as the readability of text on an interface (e.g. font colour, size), level of detail (too much or too little), and alarm status that differs from normal system operating conditions. The main source of human error is a mismatch between human capabilities and system requirements. It is essential that system design considers user characteristics and needs in relation to anthropometric variability, usage activities, and different levels of skill, experience and knowledge under different conditions of use and for each user category. It is necessary to analyse human-machine interactions by systemising human and neuroergonomics factors such as physical-dimensional, performance (physical workload or body posture), behavioural and ability (e.g. personality, self-efficacy, personal control and motivation); cognitive (e.g. cognitive fatigue, cognitive load, decline in attention); physiological (e.g. heart rate, muscle oxygenation, body temperature, drowsiness, respiratory rate, sweating) and environmental (e.g. ultrasound, noise, microclimate, vibration, lighting) (see Fig. 2). Through the neuroergonomic approach and analysis, cognitive constructs of interest (e.g. motivation or mental load) can be assessed and analysed in highly controlled artificial environments in the laboratory. Through user-centred design, one aims to improve safety and well-being in workplaces or everyday environments with the help of neurophysiological measures that enable an understanding of the mental mechanisms of workers subjected to specific work demands (Parasuraman and Wilson 2008). Therefore, cognitive neuroergonomics investigates cognitive states and their impact on information

Fig. 2. Characterisation and synchronisation of HMI and neuroergonomic factors.

560

I. Lombardi et al.

processing in the workplace based on neurophysiological data. The continuous assessment and monitoring of mental states and/or cognitive processing can contribute to improving safety and well-being at work (Parasuraman and Wilson 2008). Measures of mental workload can be classified according to performance, in relation to the subjective self-assessment process or in response to psychophysiology or neurophysiology (Dehais et al. 2020).

4 Neuroimaging Technologies Comparison for Cognitive and Perceptual Processes’ Detection Research into the behaviour of workers, the monitoring of neuroergonomics parameters during collaborative work, and the monitoring of attention and fatigue contributed to a better understanding of the events occurring and indicate the specifics of workers’ behaviour (Savkovi´c et al. 2022). The use of neuroimaging technologies for the analysis and assessment of cognitive stress makes early and objective detection possible in the event of declining levels of attention (Derosière et al. 2013) and concentration. EEG systems, for instance, provide the possibility of a continuous and objective measurement of workers’ attention, assisting the monitoring of their performance. Advances in sensor technology have made it possible to objectively measure various aspects of human cognition (Shahab et al. 2021); measures of mental workload can be classified as performance-based, or related to the subjective self-assessment process, or associated with psychophysiology or neurophysiology. Physiological measures such as eye-tracking, electroencephalography (EEG), heart rate variability (HRV) and galvanic skin response (GSR) have proven to be useful markers in providing critical information on human cognition. Among the technologies for detecting neuroergonomic processes is Neuroimaging, or brain imaging, which uses various techniques to directly or indirectly map the structure, function or pharmacology of the nervous system. Functional Neuroimaging is used to diagnose metabolic diseases and lesions on a fine scale and is widely used in psychological, neurological and cognitive research and in the construction of braincomputer interfaces. The main approaches for assessing cognitive load include direct measurements and indirect physiological ones (Lu et al. 2022). It is important to focus on indirect measurements, which is those that estimate the mental stress or safety awareness of workers based on their performance or physiological data obtained through sensors or specialised devices. Performance is generally assessed by response time or error in completing a task, but the worker’s psycho-physical state must also be taken into account through the detection and analysis of physiological parameters such as brain response feedback (UNI EN ISO 10075-3:2005), which could be heart rate (electrocardiogram - ECG), skin conductance (electrodermal activity - EDA), muscle oxygenation (electromyography - EMG). Considering that the prefrontal cortex (PFC) is functionally connected with several regions of the brain, this region mediates the complex interactions between motor function and emotion (Doi et al. 2013; Dehais et al. 2020) and performs a control function during routine cognitive operations, such as action selection, retrieval/updating in working memory and monitoring (Kirwan 2017). Michael Posner (1980) pioneered a network approach to resource operation in the early days of

Neuroergonomic Models and Tools Compared to Evaluate and Improve

561

neuroimaging. His influential analysis describes how specific networks were dedicated to particular functions of attention regulation, e.g. alertness, orientation, focus. There is evidence that a higher probability of failure is associated with PFC deactivation, for operational performance in which failure can compromise the safety of oneself and others, a higher probability of failure can also provoke strong emotional responses that are associated with stress and cognitive interference, which can function as distractors from the task at hand (Dehais et al. 2020). Neuroimaging methods fall into two categories: those that reflect metabolic brain processes associated with neural activity, such as functional magnetic resonance imaging (fMRI) and transcranial Doppler sonography (TCD), and those that directly measure neural activity, such as electroencephalography (EEG) and event-related potentials (ERPs). The merits and disadvantages of these techniques can be considered in terms of three criteria: (a) spatial resolution in localising neural activity within the brain, (b) temporal resolution in identifying the timing of neural processing, and (c) ease of use in HF/E. Neuroimaging findings have supported the distinction between perceptual/cognitive, verbal/spatial and focal/environmental visual processing (Parasuraman and Wilson 2008). Among the most popular neuroimaging technologies, the ones mainly used for the assessment of cognitive load during work tasks by means of PFC analysis are: electroencephalography (EEG), which is a technique that detects electrical activities generated by the brain. The EEG signal is an effective signal for representing changes in the autonomic nervous system. The level of mental stress is frequently reflected by an increase or reduction in brain activity in the frequency band. The study conducted by Al-Shargie et al. (2016), used arithmetic tasks as stimuli to induce different levels of mental stress, which could then be classified according to EEG signals; functional near-infrared spectroscopy (fNIRS), a non-invasive functional neuroimaging technology widely used to detect physiological factors related to brain activity. It has a higher spatial resolution than EEG and better temporal resolution than functional magnetic resonance imaging (fMRI) (Russo et al. 2023). NIRS measures light intensity after passing through a tissue (Song et al. 2020; Perrey et al. 2010; Varandas et al. 2022). It is believed that NIRS measurement imposes considerably less physical and psychological burden than current neuroimaging techniques (Doi et al. 2013). Aghajani et al. (2017) demonstrate that a hybrid system (EEG + fNIRS) allows higher classification accuracy for mental workload than using EEG or fNIRS alone. A recent study achieved over 90% accuracy in distinguishing between stress and non-stress conditions during a mental arithmetic task using combined EEG and fNIRS (Al-Shargie et al. 2016). Since mental workload and psychological stress share many physiological markers, it can be difficult to distinguish between them. For example, both mental workload and acute stress are known to affect heart rate and heart rate variability (Parent et al. 2019), justifying the need for an integrated system of multiple sensors in order to ensure physiological feedback. Both technologies have limitations (e.g. external disturbance factors, motion artefacts, etc.), which is why the combination of EEG-fNIRS revealed better results, indicating that additional data sources may be useful for the detection of cognitive fatigue. Furthermore, the combination of different sensors such as NIRS and EMG are particularly useful for emotional studies, providing a method for examining the theoretical process of fatigue. In this case,

562

I. Lombardi et al.

a decrease in haemoglobin can lead to a lack of muscle oxygenation, which can lead to fatigue. The combination of measurements is not contaminated by technical interference as both methods are based on different working principles (Balconi and Molteni 2016).

5 Devices and Integrated Systems for “Augmented Safety” in Workplaces In manufacturing industries, which require workers to be mentally alert and to carry out repetitive tasks using specific muscle groups, cognitive and physical load levels are assessed using technologies such as electroencephalography (EEG) for cognitive tasks such as calculation or continuous performance that cause activation of the frontal region of the brain (Eyam et al. 2021), the electrocardiogram (ECG) (Parent et al. 2019), eye tracking also used to monitor driver fatigue (Balconi and Molteni 2016) and muscle electromyography (EMG) (Savkovi´c et al. 2022) for localised muscle fatigue to monitor, for example, arm-shoulder muscle load in car assembly workers (Ferguson et al. 2013). To estimate mental stress in the area of industrial environments, it is therefore necessary to study the overall function of brain activity in the workplace. Supervisors must ensure that workers are immediately aware of potential dangers to avoid accidents. Using wearable devices, workers can inform their supervisors about their location, fatigue levels, health status and surroundings (Alberto et al. 2018). This digital connectivity and data transparency allows supervisors to remotely observe workers, check safety compliance, assess potential hazards and send early warnings or requests for help. The following are examples of smart wearable devices (see Fig. 3), such as helmets, headbands, gloves, glasses, textiles, etc. that use advanced sensing technologies, such as eye-tracking, haptic feedback or neuroimaging technologies, have shown great promise in various application fields, including manufacturing. These devices are designed to provide real-time feedback and improve the user experience, making them ideal for applications where precision and accuracy are crucial. Figure 1 shows an overview containing a collection of devices currently on the market, software and designs that are used today for monitoring, tracking and providing ‘augmented’ information to assist the operator in the use and maintenance of machines. Among the devices examined, we can distinguish several categories: (1) headbands; (2) helmets; (3) gloves; (4) goggles; (5) textiles; (6) exoskeletons; (7) smartwatches; (8) and other devices and/or software. Many of the devices examined are integrated systems combining different technologies. The state of the art reveals the increasing use in industry of tools and systems to improve the safety and health of operators, including: (a) technologies for detecting physiological parameters (heart rate) and biofeedback; (b) cognitive parameter detection technologies (EEG and fNIRS); (c) eye tracking; (d) remote control of machines; (e) real-time sensing and data transfer; (f) virtual reality; (g) gyroscope and accelerometer systems; (h) tactile feedback and force translators; (i) biomechanical overload assistance. There are wearable tools and connected work platforms on the market, designed towards monitoring safety at work (safe lifting of heavy loads, lifting assistance, ergonomics, hazard identification, sleep monitoring, fatigue and stress management due to extreme temperatures); increasing worker productivity (asset monitoring, augmented

Neuroergonomic Models and Tools Compared to Evaluate and Improve

563

and virtual reality, gesture and movement control, cognitive parameter detection, work stress management); monitoring health (work-related musculoskeletal disorders, movement disorders, respiratory disorders, cardiovascular health, etc.) (Patel et al. 2022). In particular, the devices examined are designed to monitor the operator, the machine and the environment and to provide ‘augmented’ information. Protecting and improving the safety, health and productivity of workers is crucial for companies. In this regard, intelligent systems (deployable sensors and analytics) play a key role in facilitating continuous monitoring, management and forecasting of workplace risks and organisational resources. In this paper, we examined recent trends in commercial wearable technologies and connected worker solutions applied to different work environments to promote ergonomics, situational awareness, injury risk management, efficient workflow and healthy behavioural and cognitive habits. While most devices monitor human performance (e.g., biomechanical functions, physical activity and/or physiological signals), new intelligent systems are being introduced to actively monitor and manage mental health (e.g., stress, emotions, moods) using brainwave sensing, biofeedback and human-in-the-loop models. Most of the devices examined concern technologies related to the detection of cognitive parameters as, due to the growing interest in and request for these, and thanks to the rapid advances in the micro- and nanoelectronics industry, biomedical device manufacturers have been able to drastically reduce the size of EEG and fNIRS devices (Di Flumeri et al. 2019), allowing them to be used on a daily basis within the reach of all types of users. Despite this, such existing systems are often not aesthetically appealing or able to arouse positive emotions and thus influence and improve the user experience (Radüntz and Meffert 2019). Some solutions turn out to be cumbersome or painful for the operator when used over a long period of time (e.g. used during an eight-hour shift), with the risk of altering user behaviour. There are also solutions that wet and soil the hair and scalp (Di Flumeri et al. 2019) as they are made of the abrasive paste and electrolyte gel, which, although minimally invasive and not harmful, are sticky products. Another important factor is the electrode-skin impedance, which must be controlled and adjusted to obtain acceptable low values; and also the awareness of being observed and detected, and thus effectively prevented from performing the activities (Hanzal et al. 2023). For example, if we consider devices that use neuroimaging technologies such as EEG or NIRS (e.g. devices such as Emotiv Insight, Mendi or even Muse) they are discrete and (almost) invisible to others so as not to hinder the normal behaviour of the worker. This is achieved with band-based systems or the use of electrode pads, but these only cover a small area of the skull and therefore do not allow a clear separation of cortical sources or of the origins of them and are extremely sensitive to artefacts created by movements. The NirSport device, which uses Near Infrared Functional Spectroscopy technology, is lightweight and easily transportable, so that it can be used for measurements in real locations and not just in laboratory setups. However, the parameters measured by measuring the oxygen concentration in the blood vessels of the cerebral cortex being examined have an increasing and decreasing phase that results in the detection of inaccurate cerebral activation, as tissue oxygenation artefacts. Therefore, as well as devices recording the physiological parameters of the operator, there is a need to integrate different parameters, making it necessary to use adjacent systems for a complete overview of the state of health

564

I. Lombardi et al.

and to improve safety at work (e.g. the Honeywell BioHarness device).Or some devices are cumbersome, to the point of preventing the operator from moving freely and easil around the workstation or switching between workstations, tasks and activities, without removing the device. For example, the Haptx Gloves G1 remote equipment control gloves, which provide realistic tactile feedback. They feature a lightweight, wireless Airpack, worn like a backpack, that generates compressed air and precisely controls its flow; with this device, it is impossible to work on other tasks without removing the glove, and the device worn on the shoulders is bulky, heavy and needs to be powered if worn for an entire work shift (8 h), as the device’s autonomy is approximately 3 h. This can increase the possibility of cognitive overload and mental strain and often negatively affect the level of interaction and collaboration with other operators. In addition, there is a need to overcome the difficulties currently encountered in managing communication and systematising heterogeneous data collected from different devices (often

Fig. 3. Overview of intelligent ‘augmented’ security devices.

Neuroergonomic Models and Tools Compared to Evaluate and Improve

565

with incompatible SW) in relation to different parameters (physiological, cognitive, environmental).

6 Conclusion Integrated, adaptive and intelligent systems can help reduce the damaging effects of human error during the execution of complicated tasks, relying on a sensitive and accurate collection of metrics to identify the various levels and combinations of mental effort and psychological stress in real time (Parent et al. 2019). For the human factors discipline, the study of mental workload serves two main functions: (a) to quantify the transitions between operators and a set of task demands or technological systems or operational protocols, and (b) to predict the probability of performance failure during operational scenarios, which may be safety-critical (Dehais et al. 2020). A challenge the field must deal with is to delineate a consistent relationship between the measurement of mental workload and performance quality, based on complex interactions between the person and the task. Both mental workload and psychological stress are common in work environments and the two concepts can have a mutual impact on each other. In this sense, maintaining a certain level of stress may also be desirable during particular situations, such as during training, where stress may help to consolidate information in the memory. It can be argued, then, that different situations lead to different optimal levels of workload and stress may result from different situations and identifying these ‘weak points’ could maximize desirable effects such as performance and learning. Neuroergonomic approaches based on measurements of the human brain’s hemodynamic or electromagnetic activity, combined with feedback from other technologies, can provide a sensitive and reliable assessment of human mental workload in complex work environments, in order to accurately assess mental workload, which could help mitigate errors and enable early intervention by predicting the decline in performance that may result from overwork or under-stimulation. In particular, the introduction and use of digital technologies such as wearable devices and artificial intelligence represent an opportunity to support production processes and safety in the workplace, as long as the main criticalities concerning, for example, the wearability of the devices (such as the inability to complete certain functions or the incorrect positioning of the system with respect to the underlying body area or anatomical point, producing discomfort) are remedied efficiency (the inability to act correctly in synergy with the user’s movements and the wearable’s expectations); and discomfort (such as friction, restriction to movement, excessive temperature, etc.)., or possible pressure problems resulting from prolonged use of wearables).Based on the analysis conducted in this paper, a comprehensive approach is needed to address the complex issues of occupational safety and health, particularly in the manufacturing sector. The combination of different technologies, including neuroimaging, can provide valuable insights into the cognitive and physiological aspects of work and help identify potential sources of work overload and related psychosocial risks. It is therefore essential to develop an integrated system capable of analysing and evaluating multiple data sources to provide a complete picture of the operator, the machine and the working environment and ensure the well-being of workers.

566

I. Lombardi et al.

References Aghajani H, Garbey M, Omurtag A (2017) Measuring mental workload with EEG+fNIRS. Front Hum Neurosci 11:359 Alberto R, Draicchio F, Varrecchia T, Silvetti A Iavicoli S (2018) Wearable monitoring devices for biomechanical risk assessment at work: current status and future challenges-a systematic review. Int J Environ Res Public Health 15 Al-Shargie FM, Tang TB, Badruddin N, Kiguchi M (2016) Mental stress quantification using EEG signals. IFMBE Proc 56:15–19 Ayaz H, Shewokis PA, Bunce SC, Izzetoglu K, Willems B, Onaral B (2012) Optical brain monitoring for operator training and mental workload assessment. Neuroimage 59(1):36–47. https:// doi.org/10.1016/j.neuroimage.2011.06.023 Balconi M, Molteni E (2016) Past and future of near-infrared spectroscopy in studies of emotion and social neuroscience. J Cognit Psychol 28:129–146 Bell & Holroyd (2009) Review of human reliability assessment methods RR679 research report Buono M, Capece S, Giugliano G, Muñoz Martínez VF (2021) Design processes for interactive human–machine systems. In: Martins N, Brandão D, Moreira da Silva F (eds) Perspectives on design and digital communication II. Springer series in design and innovation, vol 14. Springer, Cham, pp 15–36. https://doi.org/10.1007/978-3-030-75867-7_2 Dehais F, Lafont A, Roy R, Fairclough S (2020) A neuroergonomics approach to mental workload, engagement and human performance. Front Neurosci 14:268 Dekker MC, Van Egmond R, Molenbroek JFM, Vink P (2021) Developments in work-related upper limb disorders (WRULD) amongst Dutch university students from 2004 to 2014. Work 69:379–394 Derosière G, Mandrick K, Dray G, Ward TE, Perrey S (2013) NIRS-measured prefrontal cortex activity in neuroergonomics: strengths and weaknesses. Front Hum Neurosci 7:583 Di Flumeri G, Aricò P, Borghini G, Sciaraffa N, Di Florio A, Babiloni F (2019) The dry revolution: evaluation of three different EEG dry electrode types in terms of signal spectral features, mental states classification and usability. Sensors 19(6):1365. MDPI AG. https://doi.org/10.3390/s19 061365 Doi H, Nishitani S, Shinohara K (2013) NIRS as a tool for assaying emotional function in the prefrontal cortex. Front Hum Neurosci 7. https://doi.org/10.3389/fnhum.2013.00770 Third European Survey of Enterprises on New and Emerging Risks (ESENER 3) | Safety and health at work EU-OSHA (n.d.) (2019). https://osha.europa.eu/en/publications/third-europeansurvey-enterprises-new-and-emerging-risks-esener-3 Eyam AT, Mohammed WM, Martinez Lastra JL (2021) Emotion-driven analysis and control of human-robot interactions in collaborative applications. Sensors 21 European Agency for Safety & Health at Work - Information, statistics, legislation and risk assessment tools (n.d.). https://osha.europa.eu/it, May 2023 Ferguson SA, Allread WG, Le P, Rose J, Marras WS (2013) Shoulder muscle fatigue during repetitive tasks as measured by electromyography and near-infrared spectroscopy. Hum Factors 55:1077–1087 Hanzal S, Tvrda L, Harvey M (2023) An investigation into discomfort and fatigue related to the wearing of an EEG neurofeedback headset. https://doi.org/10.1101/2023.02.16.23284115 Home - Eurostat (n.d.). Eurostat. https://ec.europa.eu/eurostat, May 2023 Home - World Health Organization (n.d.). https://www.who.int/, May 2023 Kim IS (2001) Human reliability analysis in the man-machine interface design review. Ann Nucl Energy 28:1069–1081 Kirwan B (2017) A guide to practical human reliability assessment. https://doi.org/10.1201/978 1315136349

Neuroergonomic Models and Tools Compared to Evaluate and Improve

567

Körner U, Müller-Thur K, Lunau T, Dragano N, Angerer P, Buchner A (2019) Perceived stress in human–machine interaction in modern manufacturing environments—results of a qualitative interview study. Stress Health 35(2):187–199. https://doi.org/10.1002/smi.2853 La Fata CM, Adelfio L, Micale R, La Scalia G (2023) Human error contribution to accidents in the manufacturing sector: a structured approach to evaluate the interdependence among performance shaping factors. Saf Sci 161:106067 Lombardi I, Martínez V, Senese V, Capece S (2023) Design, human factors and neuroergonomics for safety in manufacturing. In: Zallio M (eds) Human factors in accessibility and assistive technology. AHFE. International conference. AHFE Open Access, vol 87. AHFE International, USA. https://doi.org/10.54941/ahfe1003653 Lu L, Xie Z, Wang H, Li L, Xu X (2022) Mental stress and safety awareness during human-robot collaboration - review. Appl Ergon 105 Mohd Nawi MN, Zainol NA, Naim F, Mamat MN, Hamzah NA, Mohd Nawi MN (2023) Employers’ perceptions of the manufacturing industry on workplace safety culture. J Energy Saf Technol (JEST) 5(2):78–86. https://doi.org/10.11113/jest.v5n2.123 Niciejewska M, Idzikowski A (2022) The impact of psychophysical factors on the level of occupational risk in small-sized enterprises with particular emphasis on the manufacturing sector. Manag Syst Prod Eng 30:370–376 Parasuraman R, Wilson GF (2008) Putting the brain to work: neuroergonomics past, present, and future. Hum Factors 50:468–474 Parent M, Peysakhovich V, Mandrick K, Tremblay S, Causse M (2019) The diagnosticity of psychophysiological signatures: can we disentangle mental workload from acute stress with ECG and fNIRS? Int J Psychophysiol 146:139–147 Park E (2020) User acceptance of smart wearable devices: an expectation-confirmation model approach. Telematics Inform 47:101318. https://doi.org/10.1016/j.tele.2019.101318 Patel V, Chesmore A, Legner CM, Pandey S (2022) Trends in workplace wearable technologies and connected-worker solutions for next-generation occupational safety, health, and productivity. Adv Intell Syst 4:2100099 Perrey S, Thedon T, Rupp T (2010) NIRS in ergonomics: its application in industry for promotion of health and human performance at work. Int J Ind Ergon 40:185–189 Posner MI (1980) Orienting of attention. Q J Exp Psychol Radüntz T, Meffert B (2019) User experience of 7 mobile electroencephalography devices: comparative study. JMIR Mhealth Uhealth 7(9):e14474. https://doi.org/10.2196/14474 Razali AM, Zin RM, Ali Q (2022) Perceptual attitudes towards safety climate among employees of a manufacturing firm: a qualitative approach. Probl Perspect Manag 20(2):107–116. https:// doi.org/10.21511/ppm.20(2).2022.10 Russo C, Senese V (2023) Functional near-infrared spectroscopy is a useful tool for multiperspective psychobiological study of neurophysiological correlates of parenting behaviour. Eur J Neurosci 57(2):258–284 Savkovic M, Caiazzo C, Djapan M, Vukicevic AM, Pušica M, Macuzic I (2022) Development of modular and adaptive laboratory set-up for neuroergonomic and human-robot interaction research. Front Neurorobot 16. https://doi.org/10.3389/fnbot.2022.863637 Shahab MA, Srinivasan B, Srinivasan R (2021) Analysis of control room operators’ competence using cognitive engineering approaches to improve process safety. In: 2021 international conference on maintenance and intelligent asset management, ICMIAM 2021. https://doi.org/10. 1109/ICMIAM54662.2021.9715179 Song X, Chen X, Chen L, An X, Ming D (2020) Performance improvement for detecting brain function using fNIRS: a multi-distance probe configuration with PPL method. Front Hum Neurosci 14

568

I. Lombardi et al.

UNI EN ISO 10075-3:2005 - UNI Ente Italiano di Normazione (n.d.). https://store.uni.com/unien-iso-10075-3-2005 Varandas R, Lima R, Badia SBI, Silva H, Gamboa H (2022) Automatic cognitive fatigue detection using wearable fNIRS and machine learning. Sensors 22

Design Thinking Meets Academic Research: Science Communication for Design Communities Using Analogies and Illustrations Violeta Clemente1,2(B) , Miriam Reis1,2 , Caroline Ezingmüller3 Katja Tschimmel1,4,5 , and Fátima Pombo1,6

,

1 Research Institute for Design, Media and Culture (ID+), University of Aveiro, Aveiro, Portugal

[email protected]

2 School of Design Management and Production Technologies Northern Aveiro (ESAN),

University of Aveiro, Aveiro, Portugal 3 Leibniz Institute for Science and Mathematics Education (IPN), University of Kiel, Kiel,

Germany 4 mindshake.pt, Porto, Portugal 5 Engineering Faculty of Porto University, Porto, Portugal 6 Department of Communication and Art (DeCA), University of Aveiro, Aveiro, Portugal

Abstract. Science communication is recognized as an increasingly relevant field within society because it allows for the communication of research results to nonspecialized audiences, nurturing scientific literacy among citizens. In this landscape, the field of Design, especially design thinking, has been making valuable contributions by enhancing communication across various academic disciplines, making research findings more accessible to the wider public. The question is whether design can help to communicate academic design research within its own community, reaching out to design educators, students, and practitioners? Our paper explores this question through the description of an example in which two hallmarks of designerly ways of thinking - analogies and visual representation are applied to communicate a scientific argument arising from academic design research. The selected example revolves around the concept of design research categories. Consequently, a secondary objective of our paper is to investigate the use of analogies and illustrations a means to elucidate the distinctions and subtleties among various categories of research in the field of design. The process itself allowed the authors to reflect not only on the utility of such tools to develop a new way of communication, but also to assist the work of multidisciplinary teams in communicating science. Keywords: Science Communication · Analogies · Digital illustrations · Academic Design Research

1 Science Communication and Design In today’s world, science communication has never been more important, and it is crucial for it to be effective. As the science communication community becomes more professionalized, the focus has shifted from the quantity of communication to the quality of © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 569–583, 2024. https://doi.org/10.1007/978-3-031-47281-7_47

570

V. Clemente et al.

communication, with connecting with the audience in a meaningful way as a central quality indicator [1]. Design can play a crucial role in achieving this goal, and academic institutions like universities have a unique opportunity to rise to the challenge by bringing together a diverse group of stakeholders in collaborative design processes to communicate science. This includes scientists, designers, practitioners, evaluators, and other actors. Design thinking, an iterative and creative problem-solving approach, has been recognized by the science communication community as an effective framework to structure these complex processes [2–4]. Despite its potential, bringing these different mindsets together can be a challenge. Therefore, more research is needed to identify the best ways to integrate design thinking into science communication efforts and to develop effective strategies for collaboration among diverse stakeholders. In many respects, design thinking is the opposite of scientific thinking. While scientists analyze facts to identify patterns and insights, designers generate new patterns and concepts to address facts and possibilities. This fundamental difference in approach distinguishes the two disciplines. However, by combining science thinking and design thinking, a more effective source of advice can be created. Although valuable in their own right, the merging of these approaches balances skeptical inquiry with imaginative application. This balance results in a more comprehensive problem-solving method that is superior to using either approach in isolation [5]. There is a growing interest in applying design approaches, such as speculative design and design thinking tools like storytelling, to science communication and public engagement with science, as evidenced by recent research [6, 7]. However, this work takes a different perspective, focusing on how the Design discipline can aid its own communication with academic and non-academic design audiences. As design is both a practice and an academic field, how can academic research findings in design be disseminated beyond traditional academic communication formats? Given that design provides numerous tools for promoting communication and public engagement, this research explores these questions by exploring a specific example. 1.1 Science, Design, Analogies and Visual Representation Analogies have always been essential for communicating scientific ideas in a way that makes them more accessible. For example, the “plum pudding” analogy has been widely used in textbooks to explain the atomic theory proposed by Thomson in 1904. Even though the analogy may not fully capture the technical aspects of Thomson’s atomic model, it offers a vivid image accessible to the general public, making it so popular that it has been adopted by scholarly literature [8]. Analogies are now considered part of scientific language. Their purpose is to transfer knowledge from a known reality, the source, to an unknown one, the target. Analogies are an effective learning instrument because they allow learners to connect what they already know with scientific concepts or theories, making them both cognitively and affectively involved. As such, analogies are not just linguistic embellishments, but a powerful resource for thought processes and conceptual understanding. When learners seek to make sense of abstract, intangible phenomena, they can draw from embodied experiences and look to concrete entities to serve as cognitive representatives [9, 10].

Design Thinking Meets Academic Research

571

In the field of design, analogies are not only a communication resource but also a promoter of lateral thinking. They foster new inferences and offer new, insightful perspectives for problem-solving. In this context, the potential for creativity increases with the distance between the two domains being compared [11]. Combining these two lines of thought, how can design use analogies to promote the dissemination of its own academic research results amongst design communities? Art, imagery and visual language in general are also becoming favored media for conveying science to the public. By making visible what cannot be captured by explicit words, pictorial representation allows to engage the public, by reaching emotions beyond cognition, and creating a more interactive and conscious communication. Additionally, visual communication can be especially valuable during conceptual and early stages of a research project, helping to generate new ideas and thoughts [12]. In the field of design thinking, visualization plays a crucial role in facilitating communication, idea sharing, and building a shared understanding among project team members. This helps accelerate the iterative process of generating and experimenting with ideas by allowing team members to easily build on each other’s ideas and recall previous discussions. Images are processed more efficiently by the human brain than verbal or textual information, making it an effective tool for facilitating idea elicitation and synthesis, fostering involvement and engagement, and integrating different perspectives [13]. Additionally, pictorial elements are closely related to creativity through its ability to foster visual reasoning. Reasoning is defined as going beyond the provided information, and visual reasoning is a cognitive process that involves using visualization to identify patterns, draw conclusions, and generate new insights. Numerous authors start from the principle, and have demonstrated in various studies, that visual thinking and the graphic representations it encompasses play a central role in creative design thinking [14, 15]. Observing drawings and other images helps the designer to activate, perceive and process information stored in memory and to relate them to other information [16]. Analogies and imagery are frequently interconnected through visual analogies during the creative process. While they may be identified without the use of imagery, in creative exploration, analogies are often generated through the manipulation and transformation of images. Typically, a designer will consider one display and create an image of another display that is recognized as analogous to the first. The designer will then use these images to think through the problem to solve, working to bring the two images into close enough alignment to enable the mapping and transfer of ideas between the original display, or source, and the new one, the target [17]. By integrating visual elements both as a means to communicate science and as a tool within the design thinking process, the question arises: How can the utilization of visualization promote the dissemination of academic research results within design communities? In synthesis, the work presented here was motivated by the following questions: How can analogies be effectively applied to enhance the communication of academic research findings amongst design communities? How can images contribute to elucidate these analogies, adding an extra layer of instruction and making scientific information more accessible to these communities? To address these questions, we explore an example centered around concepts resulting from previous academic design research conducted

572

V. Clemente et al.

by the first, fourth, and fifth authors, as previously published [18, 19]. This work pertains to the distinction between various categories of research in design and their role in consolidating design as an academic field. To validate the use of analogies and aid in the development of digital images, we invited the third author, a science communication specialist, and the second author, a graphic designer, to join our multidisciplinary team. Our collaborative effort followed a three-step approach, which will be elaborated upon in the next section. As this description will show, analogies and imagery have also proven to be invaluable in facilitating communication within the multidisciplinary team itself.

2 Exploring an Example In this section, we delve into the realm of science communication through the lens of design thinking techniques, with a particular focus on reaching out to design communities. Our aim is to explore how academic design research can be effectively communicated within its own community. In Sect. 2.1, we present a scientific design argument derived from previous research [18, 19], adhering to the conventional academic standards of textual and visual language, just as it was originally published in academic journals. This approach allows us to lay the foundation and establish a baseline for our investigation. While the concepts presented in this section do not constitute the core of this work, they must be explained so the following sections can be understood. In Sect. 2.2, the same argument is explained by using an analogy. Embracing the ingenuity of design thinking, we uncover the remarkable potential of analogies to communicate complex ideas in a relatable and more engaging manner. Building upon this newfound insight, in Sect. 2.3, we take our exploration a step further by complementing the analogy with compelling digital illustrations. By synergizing the strength of visual representation with the richness of analogical thinking, we aim to unlock an unparalleled realm of effective science communication. 2.1 The Dual Role of Design Projects in Academic Research in Design: Communicating the Argument Although multiple authors [20–23] have made significant contributions in recent years, the methodological framework of academic design research remains unclear. The lack of clarity is not solely attributed to the plethora of terminology and conflicting definitions but also due to the ongoing debate regarding the inclusion of design projects within doctoral research and whether they should adhere to the academic standards of more established academic disciplines. Following the publication of previous academic works, the argument presented here deals with the following question: What is the relative place of the design project and academic research in design, namely in terms of doctoral research? In response to this question, based on earlier research by Frayling, etc. [20– 23], a 4-category design research model was proposed [18, 19], as shown in Fig. 1. The model includes the following categories: research ABOUT design, research THROUGH design, research FROM design and research FOR design. Research ABOUT design does not involve any kind of design project conducted by the doctoral student as design author during the investigation or in the past.

Design Thinking Meets Academic Research

573

Fig. 1. The 4-category design research model [17]

On the other hand, and outside academic realm, is research FOR design which is the same as professional practice research. This type of research produces outcomes assuming the form of a product, service or process. In this type of research, new implicit knowledge will necessarily be produced, but probably without consideration (or necessity) for academic standards adherence. Also, it is rather improbable that such implicit knowledge would be translated to a communicable and explicit form. In this model, this research category is not considered academically acceptable, meaning that a design project per se, even if more complex, is not acceptable as design doctoral research. Academic research involving design practice from the doctoral student, includes research THROUGH design and research FROM design and is the topic requiring further clarification, regarding the relative place of the design project and academic research in design, namely in terms of doctoral research. That distinction is the main scope of the ongoing discussion. The presented argument is that the difference between research THROUGH design and research FROM design lies in the time and context in which that reflection takes place. Research THROUH design explicitly refers to doctoral research incorporating a design project performed by the doctoral candidate during the doctoral program. Differently, research FROM design category refers to research that results from the diachronic study of the doctoral student own relevant and professionally validated design activity. In both categories, design projects assume a central role. However, they differ on the place where the design project is developed and, on the time, when the author’s reflection and analysis occur. While research THROUGH design involves design projects developed inside universities, whereas author’s reflection, research project and design project, all occur in parallel, at the same place and within the same period of time. In research FROM design, on the contrary, the studied design project(s) belong to the researcher’s past professional activity, developed outside the academy. The author’s

574

V. Clemente et al.

reflection and analysis are diachronic because it only happens after the output of the studied project(s) have been validated by the market. Thus, research FROM design arises from implicit knowledge previously produced by means of research FOR design, later made explicit and communicable for academic purposes. 2.2 Using an Analogy to Communicate the Design Project Duality Argument The authors’ intention was to find an appropriated analogy that could provide a helpful visual image of this intermediate perspective where a design project per se is not considered acceptable academic research, but at the same time it is certainly a fruitful source of knowledge to be explored. The authors wanted an image where both positions could reach a compromise, but not in an ambivalent way, where one cannot clearly distinguish the differences between one position and the other. To illustrate the clear understanding resulting from this investigation, the required comparison should emphasize duality. At this point, the dual nature of light behavior emerged almost immediately as a possible analogy. The most interesting feature of the wave/particle question is that it behaves like a particle or a wave depending on the experimental setup. This duality of behavior is what makes the question so intriguing and challenging to understand [24]. This is exactly what constitutes the argument presented at Sect. 2.1 about the role design projects play in design research: depending on the type of experiment conceived, meaning the research methodological approach, we can observe two very distinct and dichotomic roles of the design project. By forcing the analogy a bit we can even identify ‘corpuscular’ or ‘wave-like’ characteristics depending precisely on the ontological, epistemological and methodological assumptions considered when conceiving a doctoral research project. Modern theories about light, and the consequent debate whether light is better described as a wave or as a particle, date back to 17th century. Due to Isaac Newton’s studies about optics, the corpuscular theory of light was widely accepted until the 18th century. The scenario changed when Thomas Young carried out the pivotal doubleslit experiment around 1800. Young took light and shone it through two very narrow slits, very close together. If light was made of particles, the particles should pass straight through the slits and produce two slight stripes on the screen, approximately the same size as the slits. On the other hand, if light is a wave, then the two waves emerging from the two slits will interfere with each other and produce a pattern of many stripes, not just two. The experiment results showed the interference pattern with many stripes proving that light behaves like a wave. But the case reopened in the early years of the 20th century especially due to Albert Einstein theory of photoelectric effect, which refers to the ejection of electrons from a metal surface when light is shone on that surface at certain wavelengths. Even if ejected electrons are not visible, this effect is verified by the fact that the plate acquires a positive charge due to the loss of electrons. Einstein argued that this could only be explained if light is made up of particles, or photons. When a photon strikes the metal surface, its energy is transferred to the electron, as when two billiard balls collide. In 1905, Albert Einstein proposed the wave-particle theory of electromagnetic radiation. This theory states that electromagnetic energy is released in discrete packets of energy, now called photons, that exhibit both wave-like and particle-like behavior. With this theory, Einstein was able to bridge the gap between

Design Thinking Meets Academic Research

575

the two theories by showing that light behaves like a particle interacting with matter when it’s emitted or absorbed, and it exhibits wave-like behavior when it travels through space, air or other media. Thus, it can be stated that depending on the data that a certain researcher is looking for and the experimental set up she/he applies, they can observe light behaving like a wave or like a particle. Both theories are valid, depending on the phenomenon being examined. Once the basics of light duality are established as the analogy source, a transference to the design field, as target, can be mapped. What was proposed by the argument presented in Sect. 2.1 is that the role of design project in design research can be examined from two different perspectives, depending on the desired outcome, and the ‘experimental set up’. The proposed 4-categories model for doctoral design research included research ABOUT, THROUGH, FROM and FOR design. Since research ABOUT design does not involve any kind of design project conducted by the researcher, the role of design project is not under discussion in that case. For this reason, research ABOUT design is not targeted by the presented analogy. The other three categories always involve design projects conducted by the researcher. In research FOR design, which is conducted outside the academy, the research intends to contribute to a given design output, answering to a design brief. Inside academy walls, the remaining two categories of research involve design projects authored by the researcher, who assumes a dual role as researcher and design author. In research THROUGH design, the design project occurs during the doctoral research project timespan, but only starts after the formulation of academically worthy research questions. In this case, the design project is not the end but a means through which research questions are answered. Finally, in research FROM design, the design project which is the object of the research occurred somewhere in the past typically outside the academy following research FOR design, and the research questions are only formulated after the project being completed and validated. What is now to be argued is that the nature of that knowledge resulting from design project is dualistic, analogous to the wave-particle duality of light. Research FOR design would be analogous to the photoelectric experiment. Particle-like knowledge would be ‘clustered’ in quanta and applied in close interaction with matter, aiming to answer project briefs. Resulting knowledge is implicit, embedded into design products and or processes of the design project and is not directly usable for academic purposes. This knowledge is not explicit, declared or even noticed, exactly as the ejected electrons are not directly observable. On the contrary, what is observed is the project output, the solved problem, as in the photoelectric experiment, where what can be observed is the metal plate acquiring positive charge. Wave-like knowledge, on the other hand, would be the type of knowledge that spreads and diffuses throughout the academic community, adding contributions, confirming, expanding or even questioning previous knowledge. The key difference leading to the observation of this behavior is the fact that the experimental setup, which refers to the ontological and epistemological values framing the research and the methodological options adopted to conduct affects the outcome. Thus, for a doctoral research project, it is important to design it in a way that allows for the diffusion of knowledge, akin to the behavior of a wave.

576

V. Clemente et al.

This is analogous to the double-slit experiment, in which two slits are opened with the specific aim of observing light waves to interfering with each other. Similarly, research THROUGH design and research FROM design also require as well conceived ‘experimental setup’ aiming at producing explicit knowledge and observing how it reacts, interferes, and changes the body of existing knowledge. The two slits would represent in this case an analogy for the research question and the design question. The main goal of academic research is to answer a research question (one slit), but a design project, led by a design question (another slit), may be the mean to provide the answer. After establishing this first version of the analogy, the following step was dedicated to communicating it visually, through digital illustrations. As the following section will demonstrate, the process of drawing allowed to expand and refine the analogy. 2.3 Using Digital Illustrations to Clarify the Light Analogy As perceived in the previous section, a particular aspect of the proposed analogy is that it is based on a source that is not necessarily familiar to most design communities. While the light duality theory is commonly taught in schools, the details of the experimental setups carried out to observe each one of the behaviors may not be something that non-specialized audiences will necessarily recall from their past experience as students. For that reason, the first of this 3-step approach consisted in creating two similar and comparable images to illustrate the basic elements of experimental setups used in physics to demonstrate the corpuscular and undulatory nature of light. There are so many illustrations showing both experiments, that even in online sources, it is not easy to find images sharing similar graphic language. And that was the starting point, to uniformise the graphic language of these two experiences so that interpretation would be easier. Two images (Fig. 2 and 3) were developed. In both cases the focus was not on scientific rigor, regarding light phenomena, but instead, in capturing features from the experiments that are pivotal to the analogy establishment. Also, considering that the audience is non-specialized in physics, it seemed more appropriate to clean out any technical details unnecessary to the analogous reasoning. As an example, while it is true that photelectric effect is only observable for some light wavelengths, this was intentionally left outside in the representation for Fig. 3. Given that the experiences to be translated graphically had light as a common element, the color yellow was the obvious choice to start this process and introduce a degree of literal interpretation. We then proceeded to a complementary combination, using yellow and purple, to ensure that the composition harmonized in terms of colors and adhered to fundamental principles of communication design, especially those related to color theory. Orange, being an analogous color to yellow and located adjacent to it on the color wheel, is typically associated with ‘warmth,’ which, in this context, we connected to the concept of light waves. The next step was to adapt the previous images to the context of academic research resulting in diagrammatic illustrations. Diagrammatic illustrations, intended to communicate information, are diagrams accompanied by notes or captions that explain or clarify concepts or methods or describe objects or places, for example. They are widely

Design Thinking Meets Academic Research

577

Fig. 2. Double slit experiment representation @Authors

used in education as an instructional medium to show how things work, move or change, because “information can be ingested more readily when conveyed visually” [25]. Research FOR design is represented, by analogy, by the photoelectric effect experiment, as already explained at Sect. 3, and illustrated at Fig. 4. Representation of research THROUGH and FROM design, however, were reevaluated due to recent light theory developments. Precisely at the time the drawing process was being conducted, a new study about light undulatory nature was published. In April 2023, 200 years after Thomas Young double-slit experiment, the slits in the screen, separated in space, were replaced by “slits” in time. A team led by Riccardo Sapienza at Imperial College London, shot light through a material that changes its properties in femtoseconds (quadrillionths of a second), only allowing light to pass through at specific times in quick succession [26]. A new interference pattern was observed. Instead of showing up as bands of bright and dark, they showed up as changes in the frequency or color of the beams of light. That means light also interferes with its past self with a wave-like behavior. This new discovery presented another opportunity to reflect again on our analogy. Now, with this

578

V. Clemente et al.

Fig. 3. Photoelectric effect representation @Authors

‘diachronic’ light behavior it was possible to have a specific analogy to illustrate research THROUGH design and another one to illustrate research FROM design. That way, the classical Young experiment from 1800, where the slits are coincident in time but separated by space, would constitute an analogy to the type of research where design project and academic research occur simultaneously, at the time the doctorate is being conducted (Fig. 5). To that end, two splits must be opened at the time: the research question and the design question. They interact with each other in the sense that solving the design question will provide data to answer the research question. And they also interfere with the means where these waves travel, which is the academic milieu and the academic knowledge already existent.

Design Thinking Meets Academic Research

579

Fig. 4. Research FOR design analogy @Authors

The new double-slit experiment from 2023, where the two splits are separated in time, on the contrary, presents an even more suitable analogy for research FROM design. In that case, the design question happens before, and it is only later that the research question is formulated. But they still interfere, and they still create a visible pattern of interference between them and with the milieu. A third image was then developed to illustrate this analogy. In this case, the two slits are slightly displaced in the horizontal axis to indicate the timelapse, and the resulting pattern is illustrated by a multicolor band (Fig. 6).

580

V. Clemente et al.

Fig. 5. Research THROUGH design analogy @Authors

3 Reflecting on the Process and Envisioning Future Work Aiming to explore how design thinking can assist dissemination of academic design research findings within design communities, beyond traditional academic communication formats, two approaches were followed: the use of analogies and the combined use of analogies with digital illustrations. The team included three academic researchers in design who worked previously on the scientific argument here explored as an example, one graphic designer, and one education researcher specialized in science communication. The analogy with light duality was first proposed by one of the authors, who possesses a background in engineering, thus more familiarized with physics concepts. The analogy was primarily discussed among the first three authors and the necessity to develop images to make the analogy more accessible to non-specialized audiences was identified. Schematic illustrations were developed to communicate with the graphic designer co-author and some meetings were scheduled to verbally explain the analogy. Finally, the design communication specialist was invited to contribute on the role of consultant to check the viability of the analogy and imagery.

Design Thinking Meets Academic Research

581

Fig. 6. Research FROM design analogy @Authors

While the work was mainly conducted with the objective of developing innovative tools for science communication, the process itself confirmed how design thinking can contribute to enhance multidisciplinary teams’ communication and idea generation. Design can leverage analogies to explain complex concepts in a relatable and understandable way, making research more accessible and appealing to design educators, students, and practitioners. By using analogies creatively, design can make a significant contribution to science communication and help bridge the gap between academic researchers from different fields and with the wider public. Another feature of design thinking is the use of drawing as a way not only to communicate but also to support the thinking process. In the explored example, it was clear that during the process of thinking and discussing illustrations the argument became clearer. Additionally, the importance of joining analogies and imagery was reinforced. Through our three step example, we venture into the realm of design-driven science communication, unveiling the art of crafting captivating narratives. The next step is to validate the developed visual analogy with a group of design academics, educators,

582

V. Clemente et al.

practitioners and doctoral students, and thereby contributing to the consolidation of three categories of design research and our proposed category of Research FROM Design. Acknowledgment. This work is financed by national funds through the FCT – Fundação para a Ciência e a Tecnologia, I.P., under the scope of the project UIDB/04057/2020.

References 1. Olesk A, et al (2021) Quality indicators for science communication: results from a collaborative concept mapping exercise. J Sci Commun 20(03) 2. van der Sanden MC, de Vries MJ (2016) Innovation in science and technology education and communication through design thinking. Sci Technol Educ Commun Seeking Synerg 129–145 3. Nadkarni NM, Weber CQ, Goldman SV, Schatz DL, Allen S, Menlove R (2019) Beyond the deficit model: the ambassador approach to public engagement. Bioscience 69(4):305–313 4. Kalmár É, Stenfert H (2020) Science communication as a design challenge in transdisciplinary collaborations. J Sci Commun 19(4):C01 5. Owen C (2007) Design thinking: notes on its nature and use. Des Res Q 2(1):16–27 6. Parreira MJH (2018) Exploring interactive narrative for science communication: a design approach using interactive documentary as a proposal XploreDesign4SciComm (Doctoral dissertation) 7. Tyurina A (October 2021) Enhancing science communication through AR driven visual storytelling: suggested approach and study. In: 10th international conference on digital and interactive arts, pp 1–2 8. Hon G, Goldstein BR (2013) JJ Thomson’s plum-pudding atomic model: the making of a scientific myth. Ann Phys 525(8–9):A129–A133 9. García-Carmona A (2021) The use of analogies in science communication: effectiveness of an activity in initial primary science teacher education. Int J Sci Math Educ 19(8):1543–1561 10. Taylor C, Dewsbury BM (2018) On the problem and promise of metaphor use in science and science communication. J Microbiol Biol Educ 19(1):19–21 11. Hey J, Linsey J, Agogino AM, Wood KL (2008) Analogies and metaphors in creative design. Int J Eng Educ 24(2):283 12. Jonsson A, Grafström M (2021) Rethinking science communication: reflections on what happens when science meets comic art. J Sci Commun 20(2):Y01 13. Kernbach, S., Nabergoj AS (July 2018) Visual design thinking: understanding the role of knowledge visualization in the design thinking process. In: 2018 22nd international conference information visualisation (IV). IEEE, pp 362–367 14. Goldschmidt G (1994) On visual design thinking. Des Stud 16(2):189–209 15. Lawson B (2005) How designers think, 4th edn. Routledge, London 16. Park JA, Kim YS, Cho JY (November 2006) Visual reasoning as a critical attribute in design creativity. In: Proceedings of international design research symposium, no 2005, pp 1–11 17. Goldschmidt G (2001) Visual analogy—a strategy for design reasoning and learning. In: Design knowing and learning: cognition in design education. Elsevier Science, pp 199–219 18. Clemente V, Tschimmel K, Pombo F (2017) A future scenario for a methodological approach applied to PhD design research. Development of an analytical canvas. Des J Int J All Asp Des 20(Sup 1):S792–S802 19. Clemente V, Tschimmel K, Pombo F (2020) Mapping the territories around design research: a four-layer analysis. In: Research & education in design: people & processes & products & philosophy. CRC Press, pp 147–156

Design Thinking Meets Academic Research

583

20. Frayling C (1994) Research in art and design. Royal college of art research papers, vol 1, no 1 21. Cross N (2007) From a design science to a design discipline: understanding designerly ways of knowing and thinking. Des Res Now 41–54 22. Friedman K (2008) Research into, by and for design. J Vis Art Pract 7(2):153–160 23. Findeli A, Brouillet D, Martin S, Moineau C, Tarrago R (2008) Research through design and transdisciplinarity: a tentative contribution to the methodology of design research. In: Swiss design network symposium, p 67 24. Rab AS, Polino E, Man ZX et al (2017) Entanglement of photons in their dual wave-particle nature. Nat Commun 8:915 25. Male A (2017) Illustration: a theoretical and contextual perspective. Bloomsbury Publishing 26. Tirole R, Vezzoli S, Galiffi E, et al (2023) Double-slit time diffraction at optical frequencies. Nat Phys

Is Gender(Less) Becoming a Counterculture in Fashion? Eduarda Loureiro(B) CIAUD, Research Center for Architecture, Urbanism and Design, Lisbon School of Architecture, Universidade de Lisboa, Rua Sá Nogueira, 1349-063 Lisboa, Portugal [email protected]

Abstract. Fashion and gender are two concepts that are deeply associated in today’s western societies. The idea that one can stand without the other seems preposterous but it might be closer to the norm than we think. We can say that gender(less) is still a counterculture within fashion, but what we argue is that fashion, in its essence, should be genderless. In the sense that, everyone should be able to wear whatever they identify with. Sadly, that’s not the case. Through imposed social and cultural norms, people are made to wear garments and accessories that quickly set them apart within the male/female binary. There is this assumption that we must be one or the other, and cross-dressing can be seen as an attempt to question our sexuality, which has nothing to do with gender nor gender identity or expression. In this paper, we are deconstructing what it means to go beyond the binary and challenge the norm with fashion as the catalyst. Keywords: Counterculture · Gender Identity · Gender Expression · Fashion Design

1 Defining Fashion and Gender Fashion is a vast concept, correlated with clothing, hair, decoration and behavioral associations. It has long been present alongside several societal marks in history and we can coin certain styles and visual cues to certain periods. We could call fashion the sidekick of history because it has indeed been a way of representing cultural and societal shifts, in space and time. There’s this belief that fashion is an expression of your personality and you can tell a lot by how a person dresses. It’s becoming increasingly more common that this isn’t necessarily true (at least the first part), but instead, fashion is an expression of your identity, of who you feel you are and want to be. You can tell a lot by how a person dresses, but you don’t get to trace one’s personality based on that. This issue goes beyond personality, and what we are here to discuss is gender, or lack thereof, and how that impacts the fashion industry. Before going any further, it’s important to define This work is financed by national funds through FCT – Fundação para a Ciência e a Tecnologia, I.P., under the Strategic Project with the references UIDB/04008/2020 and UIDP/04008/2020, and a doctoral scholarship with the reference of 2020.07978.BD. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 584–593, 2024. https://doi.org/10.1007/978-3-031-47281-7_48

Is Gender(Less) Becoming a Counterculture in Fashion?

585

accurately the terms that are going to be referred to, regarding gender, gender identity and gender expression. Although Nagoshi, Nagoshi & Brzuzy [1] present academic definitions of these concepts, Laurel Wamsley [2], a NPR News Reporter, defined all three in a way we deemed most adequate for this paper: • “Gender is often defined as a social construct of norms, behaviors and roles that varies between societies and over time. Gender is often categorized as male, female, or nonbinary. • Gender identity is one’s own internal sense of self and their gender, whether that is man, woman, neither or both. Unlike gender expression, gender identity is not outwardly visible to others. For most people, gender identity aligns with the sex assigned at birth (…). For transgender people, gender identity differs in varying degrees from the sex assigned at birth. • Gender expression is how a person presents gender outwardly, through behavior, clothing, voice, or other perceived characteristics. Society identifies these cues as masculine or feminine, although what is considered masculine and feminine changes over time and varies by culture”. The history of gender is quite troubling. As a concept and lived reality, it has been changing rapidly, especially in Western cultures and societies, since the mid-twentieth century [3]. We can say it has evolved through years (and dare we say centuries) of mostly patriarchal societies, cultures and oppressed groups that are not cis-white-male dominated. Meyerowitz [4] argues that “historians should analyze the language of gender, [and] observe how perceived sex differences had appeared historically as natural and fundamental oppositions”. She also defends the idea that, in different historical contexts, masculinity represented a myriad of words like strength, protection, independence, discipline, and femininity was associated with weakness, fragility, helplessness, and emotionality. In order to justify these power relations that were socially established, people that were mostly in charge called it “natural differences between sexes” [4]. Nowadays, these concepts are all being challenged by younger generations, as it seems to be the norm with all generations. Millennials and Gen Z’ers are defying what it means to be a man, woman, masculine or feminine, and giving it a new branding. Traditional roles are being questioned and people are starting to create different dynamics than the ones most of us grew up with. Gender is a social construct [5] imposed by what we call tradition and values, and the fact is, it varies from culture to culture, meaning that, as a man-made-concept, it can be deconstructed. Universalistic claims are based on a common or shared epistemological standpoint and are understood as the articulated consciousness or shared structures of oppression [5]. All these models are flawed and exclusionary because they don’t take into consideration the many different factors that exist within a person. Since gender identity can be described as a feeling, there are no measurements or quantifiable factors to make up for it. We cannot label a feeling, so it can be difficult or confusing for most people to accept it or even understand it. The experience of being a woman in the modern world is a role accompanied by societal expectations that end up being imposed at a very young age. It’s something that is physically present in men and women’s usage of most things in society – language, clothes, makeup, traits, behaviors and even social conducts [6]. As women, men, boys and girls, we are conditioned to behave and look a certain way, but if we think about it,

586

E. Loureiro

that is only in the eyes of the beholder [7]. Western society sees and perceives children in an extremely stereotyped way, imposing on them binary ways of living. A little girl is expected to behave well, not be loud, wear dresses, frills, pink. A little boy is expected to be very active, play hard and wear shorts, caps and blue. It is indeed society that needs to know the sex of the baby so it can act accordingly [6]. Ideally, one should not feel the need to color-code babies and children, or promote certain type of activities for them to play based on their gender; they should be exposed to the exact same things and be able to choose whatever they prefer and are more inclined to – especially when it comes to clothes, colors, patterns and accessories. Gender is subject to fashion [8], implicit in the way we define, exaggerate, or choose to suppress it, but fashion has been helping gender come out of its binary bubble and create this countercultural movement, a “system of warning signals to deflect genderbased assumption” [8]. It is not yet mainstream, but we do see it coming to light more in younger generations, who widely accept the reality of fluidity in gender expression. There is this concept, the multiverse, in which a person is not only one, but can be many different things (e.g. Harry Styles, who at 29 is a British reality-competition star, a style icon, a now-solo boy-band supernova, a movie star, a pop cultural shape-shifter) and the idea is that, people don’t have to like all these versions of the same person, but only the ones that resonate with them [9]. We can say the same about the duality of gender and fashion; there is a multiverse within that allows anyone to explore any version of themselves as a fluid matter. If gender, gender identity and gender expression are in a spectrum, let fashion be the catalyst to explore all these areas, avoiding a confining definition and even perception.

2 Sex and Sexuality As all oppressed groups might say, it is a matter of having a space to make them heard and seeing them represented in the mainstream culture. The current answer, to an incredibly layered subject, might be social media. It seems like the perfect tool for nonbinary and non-conforming people to find, and even model, their unique place on the gender spectrum [10]. It’s still a sensitive subject to be discussed and target to mundane hatred, especially because there is a common misconception in associating gender with sex and sexuality, and one thing is not necessarily related to the other. In fact, they are completely independent concepts, both in theory and practice. With that, we have established the definitions of gender, identity and gender expression; let’s proceed to define sex. According to Princeton University’s Gender + Sexuality Resource Center [11], sex is associated with our physical bodies and can correlate to genitals, levels of hormones, chromosomes, and other physical aspects. When we are born, we are assigned one of two sexes, based on biological factors such as hormones, chromosomes, and genitals; and intersex if the reproductive anatomy doesn’t seem to fit the typical definitions of male and female. This seems to cause confusion because it might not be enough to describe what’s going on inside a person’s body. The reason behind using the expression “assigned [enter sex] at birth” relates to its determination by someone else, usually a doctor. Which may or may not align with what’s going on with a person’s body nor how they will feel/identify [12] in the future.

Is Gender(Less) Becoming a Counterculture in Fashion?

587

Last, but not least, the definition of sexuality is one of the many personal identifiers that best describe who you may be attracted to – sexually, emotionally, intellectually, or romantically—which, again, are very different concepts that can, or can not, be correlated. Some very common terms associated with sexuality are straight, lesbian, gay, bisexual, pansexual, queer, and other identities in the spectrum.

Fig. 1. The Genderbread Person [11].

On Fig. 1, we can analyze the Genderbread Person, a tool developed by Sam Killermman [an artist, author, speaker and educator] to explain all the concepts previously described. The definition of these terms might raise some questions as to its relation to this subject and paper, but the thing is, society commonly and mistakenly sexualizes the term ‘gender identity’ and ‘expression’, connoting it with the LGBTQIA+ community. It is, in fact, a very powerful community in bringing awareness to these oppressed groups and it help bring people together, but it has nothing to do with what a person feels they are in regards to their gender expression. Oppressed and marginalized communities are groups of people that experience social, political, and economic discrimination and exclusion because of unequal power relationships across economic, political, social, and cultural dimensions. When it comes to the latter, we can talk about the countercultural dimensions, which can be a lifestyle and set of attitudes opposed to, or at variance, with the prevailing societal norm. At this point in history, whoever seems to go against the norm, can be defined as countercultural, which in this case would be anyone not identifying with the binary standard stereotypes. These countercultural phenomena have been happening for decades, but the most recent one was in the 1960’s. It happened mostly in the UK and the United States, eventually extending for the entire western societies. This entails the Civil Rights Movement, the Women’s Movement, the Sexual Revolution, environmentalism, gay liberation, and so on [13]. These moments lead us to suggest that gender is not being undone, but rather changing in its essence, and that fashion has a key role in this change, especially when it comes to re-imagining it [3].

588

E. Loureiro

Fashion is one of the ultimate self-expression forms, allowing us to be who we identify the most with, and also a way of creating this non-verbal resistance [14, 15]. Getting dressed starts as soon as we are born, and babies do not get to pick their own clothes, which ends up being a reflection of adult tastes, preferences [7] and beliefs – culminating in the message they are trying to communicate to society (e.g. pink if you’re a girl, blue if you’re a boy). Before the twentieth century, all babies wore long white dresses; slightly older boys and girls wore dresses and skirted outfits, because pointing out the child’s sex would have been very inappropriate [7]. Nowadays, it seems like the more we know the sex, the easier it is for people to identify and adjust their behavior, attitudes, and expectations accordingly. The problem stands in the “stubborn insistence on reducing complexity to binary choices (nature or nurture, male or female, masculine or feminine)” [7]. What this invariably does, is create a stereotyped thinking and, consequently acting, on what it means to be one or the other, and the rejection of there being something else. Even within constructed categories, there are huge variations of factors that binary and stereotypical thinking simply ignore. For example, it does not make sense to try and instill social norms on babies and toddlers that have very similar body characteristics and are completely unaware of the meanings of masculinity and femininity. What this provokes is this notion that there are certain behaviors associated with being a boy or a girl and, eventually, a man or a woman. When it comes to genderless fashion, there’s this imposed impediment that says that women and men’s bodies are different and, because of that, it’s almost impossible to design clothes that fit both the same way. That is in fact true, but it doesn’t necessarily determine what one should or should not wear, and therein lies the issue. The thing with fashion, and fashion design, is that it should be genderless in its essence; meaning that everyone should be able to wear whatever they want and feel more comfortable in. This counterculture movement we have been talking about, comes from people standing up for what they identify with, and being able to wear whatever they feel like on that day. If it’s a man wanting to wear a skirt, a shirt with a pattern, or even pink, why would he not? Here is where the problem lies; we are mostly brought up in these very stereotypical environments where girls are supposed to behave and look a certain way, and the same goes for boys. If what Simone de Beauvoir argues is true, that a woman is not born a woman, but she becomes one, the same might me true for men. They are expected to “only engage in the masculine” [6]. Anything that might not be considered in a manly fashion can be seen as a threat to their sexuality and gender identity. So, it’s also true that crossing over into different gendered spaces is way easier for women than it is for men [7]. It’s considered a social norm for women to wear suits, trousers, or more masculine clothes, while it’s not for men to wear skirts, dresses or even strong patterns [7]. The fashion world of today is more informal, in the sense that there are a lot more options available, and people don’t have to be constricted to certain norms; but dress codes still determine what we wear, when we wear it and even what our clothing means [16]. Fashion can be a weapon for social change, and the long run idea is that fashion becomes something for everyone to express how they feel. In the fall of 2021, fashion weeks with collections for spring 2022 were not focused on a particular silhouette or color. A lot of designers were putting both men and women on the runways (Fig. 2) in a manner that was once coined “women’s wear” [17]. Vanessa Friedman, The New

Is Gender(Less) Becoming a Counterculture in Fashion?

589

York Times’ fashion director and chief fashion critic, wrote that “[it] wasn’t gender fluidity or gender neutrality or dual gender (…) [it] was something new. Like… gender agnosticism” [17].

Fig. 2. Miu Miu spring 2022 [18].

Coco Chanel was part of the revolution that started a century ago when she started to put women in trousers (Fig. 3). What happened in the runway in 2021 was maybe a natural end of this progression of the 1920’s [17]. It has become mainstream that women can wear pants, so maybe the same is starting to apply to men wearing skirts and dresses. But what we can argue is that, outside of the fashion bubble, most people are still freaked out by this scenario, and what comes into play are the power structures instilled in us. It may seem very disempowering for men to have access to what’s typically female territory, as it somehow weakens them – because women are seen as the weaker, more sensitive and emotional sex [19]. Several societies and cultures already understand gender as something more varied than just man or woman, and their roles within the culture; but it’s true that in most societies, a gender binary has been, and still is, the norm [10]. Fashion might be ahead of the curve on this one while still playing catch up with the broader culture [17]. And again, social media has played an immense part in the broadcasting and quicker spread to the public conversation of what it means to live beyond the binary. Wortham [10] writes that platforms like Instagram play an “unparalleled role in people’s lives” because it is a way to connect to others who can be geographically isolated, but are part of the same community. What this does is put together and expose nonbinary, queer, trans people who can advocate for those who are still in search of their own identity. Representation is key and seeing it more mainstreamed than ever before can help discuss the logistics of gender.

590

E. Loureiro

Fig. 3. Coco Chanel wearing pants [20].

ALOK Vaid-Menon is a poet, comedian, speaker, author, and the creator of #DeGenderFashion, which is an initiative to degender fashion and beauty industries [21]. ALOK goes by the pronouns they/them and is a strong advocate representing nonbinary people everywhere. Through their speeches and writing, ALOK is one of the examples setting the language that guide others and define an existence of recognition and social viability. The term ‘degender’ isn’t necessarily coined, but it very well describes this countercultural movement of de-gendering fashion. The option is not to eliminate gender from fashion but to implement this idea that anyone can wear anything. If we think about the ways we can express gender, and of fashion as its vehicle from a performative perspective, “gender can be described as a way of repeatedly styling the body” [12]. But of course, this style is regulated by cultural discourse and a set of socially accepted norms across time and space. In Japan, there is a new wave of young Japanese men who are bending gender norms (Fig. 4). They are called ‘genderless danshi’ [‘danshi’ means young men in Japanese], a group of people who defy what it means to be masculine or feminine. The idea is not necessarily to become women but not to limit their fashion choices by gender, to whom the concept isn’t necessarily real [22]. Contrary to western societies and cultures, where cross-dressing can be associated with sexuality, and the notion of defying it, in Japan it’s mostly about fashion and selfexpression. Indigenous tribes and descendants, living in Canada, believe that masculine and feminine coexist in one person and have a broader meaning, encompassing several terms used and passed down generations that see gender as a fluid and liminal term [23]. In other words, an identity not anchored on physical characteristics or a judgment in time, but a never-ending process of self-discovery.

Is Gender(Less) Becoming a Counterculture in Fashion?

591

Fig. 4. Toman Sasaki is a Japanese genderless ‘danshi’ [24].

3 Conclusions Femaleness and fashion have been historically connected [3] and widely constructed through culture and society. It’s safe to argue that garments, “as cultural artifacts”, also reinforce this gender/ing of people [3]. What gender(less) does, is play with people’s assumptions, which on its own, is a part of the counterculture. In the grand scheme of things, if we agree upon these theories, we agree that gender is inherently artificial and imposed by time and space [6], and consider it to be a learned and somewhat imposed behavior, that starts the moment we are born – and probably even before. This has led people to think about the concepts of gender, and sex, as one and the same, because that’s how we have been taught and cannot remember a time without these gendered behaviors, social views, actions, and styles. From a young age, we operate almost like scientists, experimenting and gathering information to, ultimately, make sense of our surroundings [10]. Even as adults, we tend to use all available resources to create a sense of self but are ultimately limited by our immediate environment – as are children. It’s not enough to theorize, but we have to see it, and fashion can be a vehicle for it. If we allow ourselves, and others, to dress the way we/they want, we are opening a world of possibilities. Fashion should be an experiment, because we are not the same person everyday – we can feel sad, happy, nostalgic, annoyed, feminine, masculine, and our outfits can replicate those moods. We shouldn’t shy away from buying/wearing whatever garment we want, and the reality is, a lot of people do. Whether it’s because they feel uncomfortable, because they are bullied, because they are harassed or assaulted, just for dressing up in clothes that are not suited for them. This is the meaning of counterculture, going against your reality and stepping out of your comfort zone to attain a sense of self – that can quickly feel like you don’t belong. Gender(less) is becoming more normative and has brought on a lot of conversations and even legal changes [10], but it’s still yet to become

592

E. Loureiro

mainstream. It’s true that a broader culture within the digital space is helping this process speed up and bring to light gender-explorative people. Gina Rippon, neurobiologist, and author, explains that the way a brain functions, is through connections, which is how it generates predictions of the world around us. Sort of like a ‘template’ if we will, to establish the norms about what usually happens in a normal course of events [25]. It’s now a scientific attested fact that our brains are continually changing and are in a constant adaptation of our surroundings. So, if our brain has no rules to follow, it will follow its own and go beyond what was initially thought possible [25]. “If we could follow a journey of a baby girl or a baby boy, we could see that right from the moment of birth, or even before, these brains may be set on different roads. Toys, clothes, books, parents, families, teachers, schools, universities, employers, social and cultural norms – and, of course, gender stereotypes – all can signpost different directions for different brains.” [25]. What this means is that the more exposed babies and children are to all sorts of possibilities, the more they will be willing to explore and experience. Fashion can be the perfect ground for that experiment, and by removing all barriers of what is acceptable or not to wear, we are redirecting the boundaries of the gender-binary and turning garments into the free-for-all objects they can be.

References 1. Nagoshi JL, Nagoshi CT, Brzuzy S (2014) Gender and sexual identity: transcending feminist and queer theory. Springer, New York. https://doi.org/10.1007/978-1-4614-8966-5 2. NPR. https://www.npr.org/2021/06/02/996319297/gender-identity-pronouns-expressionguide-lgbtq. Accessed 26 Apr 2023 3. Clark H, Rossi L (2020) Clothes unmake the (wo)man – ungendering fashion (2015)? In: Reilly A, Barry B (eds) Crossing gender boundaries: fashion to create, disrupt and transcend. Intellect Books Ltd., Bristol, pp 201–218 4. Meyerowitz J (2008) A history of “gender.” Am Hist Rev 113(5):1346–1356 5. Butler J (2005) Undoing gender. Routledge, New York 6. Smith J (2016) Gender as a socially constructed phenomenon. In: SEWSA 2016 – intersectionality in the new millennium: an assessment of culture, power and society. Winthrop University, Rock Hill, SC. https://digitalcommons.winthrop.edu/sewsa/. Accessed 26 Apr 2023 7. Paoletti JB (2015) Sex and unisex: fashion, feminism, and the sexual revolution. Indiana University Press, Bloomington 8. Trebay G The New York Times. https://www.nytimes.com/2020/10/07/style/mens-fashionthe-gender-reveal-that-doesnt.html?searchResultPosition=32. Accessed 26 Apr 2023 9. Harris M The New York Times. https://www.nytimes.com/2023/02/06/t-magazine/queer-bai ting-harry-styles-bad-bunny.html. Accessed 30 Apr 2023 10. Wortham J The New York Times. https://www.nytimes.com/interactive/2018/11/16/mag azine/tech-design-instagram-gender.html?searchResultPosition=8. Accessed 26 Apr 2023 11. Princeton University’s Gender + Sexuality Resource Center. https://www.gsrc.princeton.edu/ gender-sex-and-sexuality. Accessed 26 Apr 2023 12. Kaiser S (2012) Fashion and cultural studies. A&C Black, London 13. Miles B The Guardian. https://www.theguardian.com/culture/2011/jan/30/underground-arts60s-rebel-counterculture. Accessed 26 Apr 2023

Is Gender(Less) Becoming a Counterculture in Fashion?

593

14. Crane D (2000) Fashion and its social agendas: class, gender, and identity in clothing. University of Chicago Press, Chicago 15. Holland S (2004) Alternative femininities: body, age and identity. Berg, New York 16. Ford RT (2022) Dress codes: how the laws of fashion made history. Simon & Schuster, New York 17. Friedman V, Trebay G The New York Times. https://www.nytimes.com/2021/10/11/fashion/ the-end-of-gender. Accessed 30 Apr 2023 18. The New York Times. https://www.nytimes.com/2021/10/11/fashion/the-end-of-gender. html?searchResultPosition=9. Accessed 26 Apr 2023 19. Butler J (1999) Gender trouble. Routledge, New York 20. https://edition.cnn.com/style/article/coco-chanel-fashion-50-years/index.html. Accessed 26 Apr 2023 21. https://www.alokvmenon.com/about. Accessed 26 Apr 2023 22. Rich M The New York Times. https://www.nytimes.com/2017/01/05/world/asia/with-manicu res-and-makeup-japans-genderless-blur-line-between-pink-and-blue.html. Accessed 30 Apr 2023 23. Mishan L The New York Times. https://www.nytimes.com/2022/02/17/t-magazine/ queer-indigenous-artists-gender.html?action=click&module=RelatedLinks&pgtype=Art icle. Accessed 28 Apr 2023 24. https://www.nytimes.com/2017/01/05/world/asia/with-manicures-and-makeup-japans-gen derless-blur-line-between-pink-and-blue.html. Accessed 28 Apr 2023 25. Rippon, G.: Gender and Our Brains Pantheon (2019)

New Retrospective Approach for Voting Advice Applications’ Design Methodology Artur Porto de Almeida Magalhães(B) and Pedro Duarte de Almeida Universidade de Lisboa, Faculdade de Belas-Artes, Centro de Investigação E de Estudos Em Belas-Artes (CIEBA), Largo da Academia Nacional de Belas-Artes, 1249-058 Lisboa, Portugal [email protected]

Abstract. Voting Advice Applications (VAA) are tools designed to help voters with their decision in an electoral process. These applications are generally based on a questionnaire structure and user responses. As a result, they provide a ranked list of parties or candidates that are closer to the voter’s political positions. In this paper we identify the two most used design methodologies for this kind of applications and discuss their limitations. Namely, the dependence that these approaches have on the prior existence and specialized analysis of proposals and manifestos of political candidates or parties. We conclude that these methodologies, focusing on future projections made by political agents, discard their past performance and, as such, are not suitable for countries where there are inconsistencies between political practice and the ideology propagated in electoral programs and manifestos. Therefore, this paper presents a VAA, Bom Partido (“Good Party”), which follows a retrospective methodological approach which is easy and accessible to replicate. Aimed at the Brazilian party-political context, this project is based on replacing the question “Which party/candidate proposals best correspond to my political positions?” by the question “Which party/candidate acted politically in the way that best corresponds to my political positions?”. Likewise, in addition to contributing to the critical study of VAA design methodologies, this project suggests a new design approach for these applications that potentially increases the reliability of its results and whose development method can be replicated step by step. Keywords: Voting Advice Applications · VAA’s Design Methodology · Brazilian Chamber Electoral System.

1 Introduction Voting Advice Applications (VAAs) have become increasingly popular and potentially influential tools in the electoral process. They generally provide valuable information about candidates and parties and help to inform voting decisions, with the intention of bringing voters closer to their political positions (Garzia, 2014). VAAs are online applications that provide data on political parties’ positions on a range of public life issues, which, when compared to users’ positions, produce a voting © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 594–607, 2024. https://doi.org/10.1007/978-3-031-47281-7_49

New Retrospective Approach

595

advice—usually in the form of an ordered list of parties or candidates, where the party with positions closest to those of the user appears first (Cedroni & Garzia, 2010). These applications aim primarily to increase voters’ knowledge about party positions at a very low cost and with a pleasant and immediate approach. Collaterally, they can increase voters’ interest in political issues and motivate discussions about policies that affect everyone (Cedroni & Garzia, 2010). However, there is no consensus on the ideal way to design a VAA. The “Lausanne Declaration on Voting Advice Applications” (Garzia, 2014), one of the most representative documents on the state of the art of the subject, recommends parameters to be respected by any VAA creator but refrains from suggesting an “ideal form.”

2 How Are VAAs Generally Designed? When observing the universe of VAAs, two methodological approaches are identified and have been repeated, as far as it has been seen, invariably, regarding the definition of the parties (or candidates) positions on the questioned issues1 . They are, briefly: a) Parties or candidates are invited to answer the same questionnaire applied to the public. The questionnaire covers questions on relevant themes to the public life of the country/region, and the politicians’ answers reveal their positions. The votercandidate or voter-party correspondence is calculated by comparing their answers. This method depends on the candidates’ adherence to the project. That is, those who do not answer the questionnaire are excluded from the possibilities of correspondence of the application. Taking the example of the Match Eleitoral case, developed by the Folha de São Paulo newspaper and the DataFolha Institute (Brazil), it is clear how difficult it is to depend on the response of external agents for the construction of a VAA. The team contacted the candidates, and even with the credential of one of the most important newspapers and research institutes in the country, the project was launched with only 57% of the registered candidates, in the case of the dispute for the Chamber of Deputies of São Paulo in 20182 (Datafolha, 2018). Therefore, this is a formula that may work for large media outlets such as Folha de São Paulo (and still with good limitations), but it is not replicable for less renowned agents. b) The positions of parties and candidates are defined based on the analysis and classification of their communication pieces, considering positions related to previously elaborated issues. Let’s take, as an example, the materials analyzed by the 1 Some VAAs mix the two methodologies, such as Kieskompas and EUANDI (Garzia &

Marschall, 2019). 2 In the 2022 version, the platform reached 64% adherence, with 982 candidates: “To form the

database of the Match Eleitoral (a tool created to help in the choice of the candidate for federal deputy), Datafolha requested from all parties access to names and contacts of their candidates for federal deputy for São Paulo. Other sources of party information, such as lists from the Chamber of Deputies and the Electoral Justice, were also used. The application was launched with a database of 982 candidates, about 64% of the total registered in the state.“ (Folha de São Paulo/Datafolha, 2022).

596

1. 2. 3. 4. 5. 6. 7. 8.

A. P. de Almeida Magalhães and P. D. de Almeida

EUANDI2019 VAA, developed by the European University Institute in collaboration with the University of Lucerne3 , designed with the aim of “helping citizens make an informed choice in the 2019 European Parliament Elections”: “Manifesto of the national party for the 2019 European Elections; Electoral manifesto of the party; Current/previous national election manifesto; Manifesto of the European party to which the national party belongs; Other official and programmatic documents of the party; Interviews, press releases, and positions on social media (from the party leader or main candidates); Other electoral manifestos; Other sources.”

This method, therefore, depends on the participation of experts trained for such analysis, such as political scientists, journalists, researchers, economists or lawyers, not only to evaluate the positions of parties and candidates but also to define the issues addressed, to be taken as the basis of the analysis. By increasing the authors’ volume of actions, the risk of incurring interpretation biases also increases. All VAAs already analyzed by the present research—as “SmartVote” (Switzerland), “Stemwijzer” (Netherlands), “Wahl-O-Mat” (Germany), “Votematch” (UK), “Match Eleitoral” (Brazil), “VAA.fi” (Finland), “On the Issues” (USA), “Policy Pitch” (Australia), “Dating Eleitoral” (Portugal), “Observador Votometer” (Portugal), “Kieskompas” (Netherlands) and “EUANDI” (EU)—apply methodology A, B, or a combination of the two (Garzia & Marschall, 2019)4 . 2.1 What Do the Two Methodologies Have in Common? Aside from the difficulty in replicating them outside an environment with significant human and financial capital, which is difficult for independent agents to access, both repeat the same approach: they calculate the positions of parties and candidates based on the claims of the parties and candidates themselves—their own speeches and publications. Which means that, whether by responding to the questionnaire at hand (Methodology A) or having their manifestos, interviews, and statements analyzed (Methodology B), it is the parties and candidates who provide the inputs about their political positions. Both work by the logic of the question “Which party/candidate proposals best correspond to the user’s political positions?” But where are the political acts? The official decisions, parliamentary votes, legislative proposals? Such methodological approaches are focused on future visions. That is, the questions are based on the promises and projections of political actors, while real actions are completely ignored, as they reside in the past. There are no questions or 3 Among other partners, such as the Robert Schuman Centre for Advanced Studies at the European

University Institute. 4 Some authors, such as Garzia & Marschall (Oxford Research Encyclopedia of Politics, 2019),

consider that asking the parties to position themselves on the issues i) with or ii) without corrections by experts are two different methods of identifying policy positions of the parties and candidates (here simplified as Methodology A).

New Retrospective Approach

597

classifications based on recent political episodes, decisions made, votes cast; there are only elaborations about hypotheses to occur. Thus, they depend on the collaboration— and good faith—of the actors judged in the test. Important flaws are identified in this dependency. In the next section, a brief analysis of the Brazilian political system helps to demonstrate these potential flaws.

3 Trust and Loyalty in Brazilian Party Politics Let’s take as an example the Brazilian reality, for which this academic research was directed. Politics in Brazil presents specific challenges to which the common ways of building VAAs do not seem to fully meet with competence. More precisely, the historical ideological instability of many Brazilian parties, expressed in their support for agendas and governments of divergent ideologies, depletes confidence in the promises, programs, and manifestos of these parties, and therefore compromises any initiative of ideological correspondence based on them. The disposition of the Brazilian congress and political environment in general includes the presence of the so-called “physiological” parties, in opposition to the “ideological” parties (Bolognesi, 2022). Those whose voting histories are incapable to show any ideology that rules them, because the only visible pattern is the support to the current government—regardless of its political orientation. These “physiological” parties compose the “body” of the National Chamber since the last redemocratization (1989) and have formed the basis of support for presidencies of the center-right (such as Fernando Henrique’s, from the Brazilian Social Democracy Party), left-wing (such as Lula da Silva and Dilma Rousseff’s, from the Workers’ Party), right-wing (such as Michel Temer’s, from the Brazilian Democratic Movement Party) and extreme right-wing (such as Jair Bolsonaro’s, elected by the Social Liberal Party), granting their votes to ideologically positioned projects across the political spectrum. The system that maintains the votes on candidates from these ideologically pendulum-like parties cannot, logically, sustain itself by aligning with its voters’ ideas. How much do voters know about this? How much do they follow legislative activity? Can “physiological” party deputies respond to their voters’ demands while changing their ideas every four years? 3.1 Electoral Amnesia A phenomenon that helps explain the maintenance of votes for these pendular parties is the “electoral amnesia”, common in Brazil and even more frequent in votes for the legislative power. In 2018, 44% of Brazilians could not remember who they voted for federal deputy less than a month after the elections (Center for Studies and Public Opinion of Unicamp - Cesop, 2018). The scenario is worse when four years have passed: in 2022, six out of ten voters could not remember who they voted for in the Chamber and Senate in 2018 (Datafolha, 2022). That is, almost half of the electorate does not fulfill the first step of monitoring legislative activity, which is to remember who they voted for. This, coupled with the

598

A. P. de Almeida Magalhães and P. D. de Almeida

high complexity of legislative processes in the Chamber and also of the content of the matters voted on, leaves only a tiny fraction of voters who know how their candidate, if elected, actually acts. The disconnect between the voting decision and the observation of the activity of federal deputies in the Chamber’s votes is a blatant democratic anomaly. The Historical Party Distribution Record of Brazil (Fig. 1) helps to understand the party landscape of the Brazilian Chamber of Deputies. Its hyper-fragmentation causes an obvious difficulty in retaining each party’s partisan orientations, which, among other factors, induces the personalization of the voting decision. In general, the personal figure of the deputy candidate gains relevance over their party, compared to countries with a smaller number of competitive party legends. With more than 10.4 thousand valid candidacies for 513 vacancies in the federal Chamber5 , electoral amnesia is enhanced.

Fig. 1. “Party Distribution Historic” (“Party Distribution in the Deputies’ Chamber in the 6th Republic”; “[Parties]”; “Parties without present representation*”; “Extinct parties**”. Wikipedia, retrieved September 20, 2022).

In this confusing context, of parties without a clear ideology and electoral amnesia among voters6 , would it be appropriate to create a Voting Advice Application (VAA), as usual, based on the proposals and promises of candidates and parties? How can one believe in the program of a party that, despite being present in the Chamber for decades, has not shown to belong to any apprehensible ideological current? For example, a party 5 In São Paulo, the state with the largest population and candidates, there was 1,522 names for

70 vacancies in 2022. 6 Among other factors identified in the complete Master research “Good Party: Voting Advice

Application for the Brazilian Deputies’ Chamber.

New Retrospective Approach

599

that voted in favor of disarmament in one legislature and in favor of relaxing the sale of weapons in another? How can one hold candidates accountable for their promises when there is no memory of their voting record? How can one demand proposals when there is no practical tracking of legislative activity?

4 Bom Partido App Design Process The “Bom Partido” (Good Party) is a Voting Advice Application project focused on the parties of the Brazilian Chamber of Deputies, with a prototype developed using data from the 2018–2022 legislature (Jair Bolsonaro’s government). The project is part of the Master’s research “Bom Partido: Voting Advice Application for the Brazilian Deputies’ Chamber.” (Porto, 2023), developed in the Universidade de Lisboa’s Communication Design course and defended in January 2023. The application consists of a test with 16 questions about legislative proposals already voted on in the Chamber, for which the user must vote YES, NO, or abstain, just as the deputies did in the respective votes. In the end, their votes are cross-referenced with the votes of the parliamentarians to find the political party whose positions most correspond to the user’s. In other words, their ideological match. The application is not limited to advice, but also allows for a deeper understanding of legislative proposals, the history of parties, and provides other insights into the Brazilian political reality. This explanatory/investigative dimension runs parallel to the logic of the match, as it ensures a minimum contribution from the project regardless of the usefulness that the result brings to the user. By covering 16 of the most important votes that have affected the lives of Brazilians in recent years, presented in a concise and simplified manner, and then knowing how the parties positioned themselves, a significant part of the project’s intention is already fulfilled. Here are the fundamental construction parameters developed for the Bom Partido: 4.1 Selection Criteria and Sources The process of collecting and selecting the votes to be addressed in the “Bom Partido” prototype was guided by the following criteria: i). The vote was nominal, meaning there was no secrecy in the deputies’ votes; ii). The vote has a relevant impact on the lives of Brazilians; iii). The voted subject is understandable for the average voter and capable of generating a concise and intelligible question; iv). The voted subject generated repercussions in society, with media highlights and reactions from civil society; v). The result of the vote was sufficiently polarized among parties, representing differences in positions; vi). The final selection includes proposals from various areas of public life (e.g., Environment, Security, Education, Labor Rights…).

600

A. P. de Almeida Magalhães and P. D. de Almeida

Some important votes, initially selected, were excluded because: i). They result in almost unanimous consensus among parties; ii). Their complexity made it difficult to formulate a concise and objective question; iii). They consist of multiple decisions, making it difficult to formulate a single and concise question and preventing the identification of the factor that led a particular party or voter to take a position against the proposal; iv). They may be difficult for a significant portion of voters to understand, as they address issues that require highly specific prior knowledge. The preliminary collection was carried out through secondary research methods and combined inputs from different sources: i). Some of Brazil’s most recognized digital newspapers, both large and medium-sized (with a focus on political coverage), such as O Globo, Folha de São Paulo, CNN Brazil, Revista Exame, El País, Carta Capital, Nexo Jornal, Portal Metrópoles, Jota, and Congresso Em Foco; ii). Web tools that already provide selections of votes considered important, such as “O Voto dos Deputados” (The Vote of the Deputies) developed by G1, the digital politics section of O Globo, which “shows the position of each deputy and senator in the main votes of the Chamber of Deputies and the Senate” for the current and previous legislatures; “Quem Foi Quem”(Who Was Who), authored by the Department of Trade Union Advisory (Diap), which provides a selection of votes based on their importance to workers, and “Radar do Congresso” (Congress Radar) from the Congresso Em Foco portal; iii). Official portals of the National Congress and the Chamber of Deputies, which include open polls for internet users and allow filtering of the most voted and commented proposals, and host the Agência de Notícias da Câmara dos Deputados, which provides daily journalistic coverage of the voted-on matters. 4.2 Questions Formulation The density of the content of legislative proposals brought the necessity to write more extensive questions than those in the studied cases, involving a brief explanation of the most important points for understanding the subject matter. To avoid becoming tiresome, the reading of each proposal was divided into internal screens, a feature borrowed from the usability of the dating app “Tinder”. Each question occupies a maximum of four screens with brief and progressively detailed texts. The first screen provides a key phrase with the most concise wording, which allows users who already know the topic well to skip the sequential explanation and follow a shorter path to the result. Users with less familiarity are more likely to be interested in the explanations, justifying the explanatory-pedagogical dimension of the application. The fundamental criteria that guided the formulation of the questions were impartiality, conciseness and clarity. For an ideal formulation, a more in-depth investigation supported by scientific principles of research methodologies would be necessary. The task was accomplished through the analysis of the case studies.

New Retrospective Approach

601

After analyzing the formulation of the questions in a case study, it was concluded that the best way to formulate the key hypothesis is by repeating the verbal phrase with the verb “deveria” (in english, the modal verb “should”). This model accommodates, without alteration, the three possible status of the proposals addressed: i) The proposal was approved and put into practice; ii) The proposal was approved but is still in progress, and iii) The proposal was rejected. 4.3 Match Calculation Methodology The correspondence resulting from the test is calculated using a scoring system, in which a political party scores points based on how their votes align with the user’s answers. The party with the most points is the final match. i). A party scores one (1) point on a question if the qualified majority (>66.7%) of its votes on that question corresponds to the user’s answer, whether it’s YES or NO. ii). A party loses one (1) point on a question if the simple majority (>50%) of its votes on that question is contrary to the user’s answer, whether it’s YES or NO. iii). A party neither scores nor loses points if the simple majority of its votes corresponds to the user’s answer (whether it’s YES or NO) but does not reach the qualified majority (between 50% and 66.7%). iv). No party scores or loses points on a question if the user abstains. In other words, if the simple majority of parliamentarians is opposed to the voter’s position, that party is not advisable to represent the voter. And if there is a significant number of parliamentarians opposed to the majority position within their own party, that party is not safely advisable to represent the voter. If the voter doesn’t know their position on an issue, there’s no way a party can represent them. In representative democracies, it is assumed that parties must have affirmative ideological principles demonstrated to citizens, and a neutral-neutral correspondence can yield undesirable results (as seen in the case of the Observador’s Votómetro VAA). 4.4 Tiebreaker—Parties’ Internal Loyalty Index (PILI). To provide a tiebreaker criteria among parties that scored the same, the Parties’ Internal Loyalty Index (PILI) shows how much the parliamentarians adhered to their party’s orientation on the selected proposals, and ranks them from the most to the least internally loyal. The creation of the index aligns with the principle for any political party, which is to give voters confidence that its members share the same worldview and respect the common ideology expressed in the manifestos, agendas, platforms, political programs, and most importantly, in the party’s votes in the Congress. If parliamentarians from a particular party cannot agree on their stance on a legislative proposal, how can voters trust that party’s vote for future ballots, or more, how can they trust that the party fulfills its role as an ideological beacon representing them? The division of votes among parliamentarians from all parties, in all votes, was computed as follows:

602

A. P. de Almeida Magalhães and P. D. de Almeida

Sum of votes from the majority group of party X in the proposal Y ÷ Total votes of party X in the proposal Y × 100 For example, the PSDB had 14 votes in favor of the PEC 135/2019 proposal, 12 votes against, and 1 abstention, totaling 27 votes. Therefore, the internal vote correspondence percentage is 14 ÷ 27 × 100 = 52. By summing the results of the PSDB in all 16 votes, you arrive at the party’s final average percentage, its PILI: (52 + 94 + 97 + 72 + 68 + 96 + 63 + 85 + 81 + 79 + 76 + 59 + 72 + 96 + 90 + 100) ÷ 16 = 80 Therefore, the average internal loyalty value for the PSDB was 80%. The result places the party in 20th out of 25 parties in the Bom Partido’s PILI rank (Porto, 2023). In parallel with the PILI, it was noticed that some parties voted identically in the 16 selected votes, due to their strong ideological alignment. The main consequence of a tie is that, as the PILI is a fixed percentage, one of the parties would always prevail in the tiebreaker. This would be fair if the parties had the same number of federal deputies, but that is not the case. For example, REDE had only one (1) federal deputy in the National Chamber, while PT had an average of 50 deputies. REDE, therefore, guarantees 100% of PILI and would prevent PT, with 99.4% of PILI, from being the match for any user. Mobilizing 50 deputies with 99.4% efficiency seems like a greater achievement than having an inevitable 100%. For this reason, a grade on a scale from A to E was assigned to level differences of less than 10%, which adjusts the loyalty ranking: A: 90–100%; B: 80–90%; C: 70–80%; D: 60–70%; E: < 60%. 4.5 Result—the Match Display The presentation of the results to the voter can be done in various formats. The tested prototype is limited to the questionnaire stage, without calculating and revealing the obtained results. Therefore, the following combination has a propositional value: Screen #1: Match. Shows the party with the highest correspondence, prominently displayed, along with the two consecutive ones in the sequence (1st / 2nd and 3rd; with name, acronym, and visual signature). Shows how many points the top three parties scored (X/16), their PILI rating (A-E), and the average number of voting deputies on the selected proposals. Screen #2: Complete Ranking. Section with all parties ranked and their scores. Clicking on any party directs you to its individual page, which again displays its rating (A-E) and average number of deputies, PILI percentage (%), how they voted on each issue (and how the user voted), their government alignment position, visual signature, and website. Screen #3: Government Alignment Scale. If the voter was a party, where would she/he be positioned on the scale from the party that most supported the government to the party that most opposed it? Screen #4: Recap of Questions. Retrieval of the questions, presented in the same order as in the questionnaire. Shows how parties positioned themselves on each issue (and how the user voted). What was the final outcome of the votes—approved or rejected —, omitted previously, and the current status of the voted proposals—e.g. transformed into law, pending in the Senate, approved in the Senate with amendments…

New Retrospective Approach

603

4.6 Prototype The prototype can be accessed by anyone with internet access, by mobile or desktop (Fig. 2)). Prototype Test. For the first test, a small group of thirteen (13) participants, without mutual contact, was gathered. They were encouraged to try out the prototype and respond to an evaluation report without any prior explanation or guidance on the usability of the application and the content covered. All thirteen (13) invited participants answered all 16 questions in the questionnaire, even if the response was “I don’t know.” A report model was developed—based on the dimensions proposed in the Users’ Experience Honeycomb (Morville, 2004)—which was filled out by the test participants to assess the experience. 1. “In your opinion, what is the utility of the application? Did it seem useful to you? 2. Was it easy to use the application? Did you have any difficulty understanding how it works? 3. Did the length of the test seem appropriate? Was it tiring to complete? 4. Did the application pique your interest? Would you like to use such an application? 5. Did you find everything you were looking for in terms of features and content? 6. Did you encounter any difficulty reading the content? Did you have any trouble interacting with the application? 7. Does the application seem reliable to you? Do you trust that the results and data shown are true? Did you notice anything that made you suspicious? 8. How would you rate the contribution of this application to your voting decision? Positive, Negative, Neutral? 9. Comprehension of the texts—did you have any difficulty understanding any of the questions? (1–16) 10. Malfunctions? 11. Comments? Suggestions?” The results of the test were satisfactory, albeit within a limited context, and can be found in the dissertation that gives rise to this article (Porto, 2023). The next steps to evolve the prototype are: i) develop the necessary code to calculate the match, ii) design the result screens, iii) prototype a complete and functional model, and iv) test it on a larger scale with more elaborate research methods.

604

A. P. de Almeida Magalhães and P. D. de Almeida

Fig. 2. Screenshots of the app’s prototype (introduction and example of proposal). a) Cover: “(Test Version) Good Party: Your match in the Deputies Chamber”. “Start”; b) Introduction 1: “Pretend you are a federal deputy and vote in the next proposals”. “Skip intro”; c) Introduction 2: “All the presented proposals are real and were objects of vote by the Brazilian deputies in the last 4 years”. “Skip intro”; d) Introduction 3: “In the end, we will compare your votes with the deputies’ votes and calculate what party you would be part of. In other words, the one that voted most consistently with you on the proposals presented ahead. Vote with consciousness:)”. “Begin test”. e) Proposal 12, part 1: “Previous / Restart / Next”. “Prohibition of evictions”. “Evictions of properties should be suspended during the pandemic.” “Yes / I don’t know / No”. “Methodology / Privacy”. f) Proposal 12, part 2: “Previous / Restart / Next”. “Prohibition of evictions”. “The forced eviction of private or public property, resulting from the enforcement of a judicial, extrajudicial, or administrative measure, should be suspended during the COVID-19 pandemic. The proposal does not apply to occupations that occurred after March 31, 2021.” “ “Yes / I don’t know / No”. “Methodology / Privacy”.

New Retrospective Approach

605

5 Conclusion It seems naive, given the complex dynamics of politics in Brazil and, broadly speaking, in South America, to rely solely on the documents and speeches of the parties and candidates—their own promises—to define their positions, considering the historical fragility of ideological fidelity. All the VAA observed in this research operate in this sense, that is, they analyze the parties’ claims about their positions and their promises for the future. However, when the actual positions taken by parties, demonstrated mainly through their votes for legislative proposals, disregard their own promises and ideological claims defended during the electoral period and in official party documents, there is no trust in an ideological correspondence between the voter and the party that is based on these criteria. The purpose of this research was to design a tool to provide Brazilian voters with useful and reliable information to support their voting decision for federal deputies, and thus reinforce the mechanisms for maintaining democracy and political representativeness articulated by civil society. The consequential goal was to understand the dynamics of VAA construction and propose a new methodology, suitable for the Brazilian context and replicable for other democratic countries. The central guideline of this methodology is to shift the focus from the future to the past, regarding the assessment of parties’ positions and the content queried to voters. By replacing the question “What party has the proposals that best match my political opinion?” by “Which party acted politically in the way that best corresponds to my political positions?”, we seek to contribute broadly to the study of VAAs—which rarely touch on the political reality of Latin America and its peculiarities7 . 5.1 Retrospective Logic If a deputy defends position X in an interview, the same position X appears in their party manifesto and in their electoral program, but the deputy votes according to the opposing position Y when the respective topic arises for voting in the Chamber of Deputies, then position Y should be attributed to them in the VAA database. This is the main correction proposed by the developed methodology, among other aspects. The application of the retrospective logic not only provides a more reliable result regarding parties’ positions on real and current issues but also simplifies the tool’s development process. Unlike the studied cases, this approach does not depend on the collaboration of parties, parliamentarians, or candidates, nor on political experts, as it replaces a) the process of instigating political actors to respond to the questionnaire or b) the process of analyzing and classifying the manifestos, programs, and interviews of political actors with c) a simple research on recent parliamentary votes. All the necessary 7 The “Census Global de VAA 2016” (Garzia, 2016) presents a list of about 50 examples, of which

only five are from Latin America (two from Mexico, one from Brazil, Ecuador, and Venezuela; all international initiatives). As an observation, it is curious that one of the most consolidating documents in the study of VAAs, currently, was published precisely in Switzerland, one of the countries in the world that most replaces party representation with direct voting by the voter through binding referendums.

606

A. P. de Almeida Magalhães and P. D. de Almeida

inputs can be collected through the State’s official communication portals8 , in the case of Brazil and other functional democracies that have legal guarantees for data transparency. 5.2 Replicability The proposed formula, briefly presented here, allows a person or group without experience in journalism or political science, without political influence, and without expertise in analyzing party positions to appropriate the methodology to create their own VAA. In the Master thesis “BOM PARTIDO: Aplicativo de Aconselhamento de Voto para a Câmara dos Deputados do Brasil” (Good Party: Voting Advice Application for the Brazilian Deputies’ Chamber; Porto, 2023)9 , all necessary guidelines were provided to replicate the same methodology in another context (Fig. 3) .

Fig. 3. Promotional image for the testing prototype of the Bom Partido application (“Good Party”).

References Agência Brasil. (2022). Brasil tem mais de 10,4 mil candidatos a deputado federal; veja lista (B. Saniele, Ed.). Agência Brasil. https://agenciabrasil.ebc.com.br/politica/noticia/2022-08/brasiltem-mais-de-104-mil-candidatos-deputado-federal-veja-lista. Accessed 14 Oct 2022 BallotReady. https://www.ballotready.org/. Accessed 25 Mar 2023 8 The data used in the “Bom Partido” project was collected from the Brazilian Chamber of

Deputies website, www.camara.leg.br/. 9 Artur Porto’s Master Thesis at the Universidade de Lisboa, available in http://hdl.handle.net/

10451/56779 (Lisbon University Archive).

New Retrospective Approach

607

BBC News. (2010). No joke as Brazil clown tops votes for Congress. https://www.bbc.com/news/ world-latin-america-11465127. Accessed 29 April 2023 Bolognesi B, Ribeiro E, Codato A (2022). Uma nova classificação ideológica dos partidos políticos brasileiros. https://preprints.scielo.org/index.php/scielo/preprint/download/ 2552/4439/4612. Accessed 10 Aug 2023 Bundeszentrale für politische Bildung. https://www.bpb.de/politik/wahlen/wahl-o-mat/. Accessed 25 April 2023 Cedroni L, Garzia D (eds) (2010) Voting advice application in Europe: the state of art. ScriptaWeb, Napoli Chagas, P. V. (2018). O “Efeito Tiririca” e a importância da votação para deputado. Agência Brasil. https://agenciabrasil.ebc.com.br/justica/noticia/2018-10/o-efeito-tiririca-e-import ancia-da-votacao-para-deputado. Accessed 11 Aug 2022 Datafolha (2018). Match Eleitoral. Folha de São Paulo. https://matcheleitoral.folha.uol.com.br/. Accessed 20 Janeiro 2022 Esquerda.net. (2022). Observador recomenda voto no Chega aos eleitores “neutros”. Esquerda.net. www.esquerda.net/artigo/observador-recomenda-voto-no-chega-aos-eleitoresneutros/78955. Accessed 21 Jan 2022 EUANDI. https://euandi2019.eu/survey/default/EN. Accessed 20 Aug 2022 Garzia D, Marschall S (2019) Voting advice applications. In: Oxford Research Encyclopedia of Politics. Oxford University Press. https://doi.org/10.1093/acrefore/9780190228637. 013.841. https://cadmus.eui.eu/bitstream/handle/1814/69618/GARZIA_2019_2.pdf?sequen ce=1. Accessed 26 April 2023 Garzia D (2016) VAA Global Census 2016. Google Planilhas. https://docs.google.com/spreadshe ets/d/1FulfjKdoeuf9Yi6C7SfFgBjqQdKxEzujXaL6JPbSyDs/edit?usp=sharing. Accessed 22 October 2022 Garzia, Diego, e Stefan Marschall (2014) The Lausanne Declaration on Voting Advice Applications. In: Diego Garzia and Stefan Marschall (eds) Matching voters with parties and candidates. Voting advice applications in comparative perspective. Colchester: ECPR Press, S. 227–228 Kieskompas. https://www.kieskompas.nl/. Accessed 25 Mar 2023 Morville, Peter (2016). User Experience Honeycomb. Intertwingled. https://intertwingled.org/ user-experience-honeycomb/. Accessed 10 Jan 2022 Ontheissues.org. https://www.ontheissues.org/. Accessed 25 Mar 2023 Policy Pitch. https://www.policypitch.com.au/. Accessed 25 Mar 2023 Porto, Artur (2023). Bom Partido: aplicativo de aconselhamento de voto para a Câmara dos Deputados do Brasil. Faculdade de Belas Artes da Universidade de Lisboa. http://hdl.handle. net/10451/56779 Smartvote.ch. https://www.smartvote.ch/. Accessed 25 Mar 2023 Stemwijzer. https://www.stemwijzer.nl/. Accessed 25 Mar 2023 VAA.fi. https://www.vaalikone.fi/ Accessed 25 Mar 2023 Votematch.org.uk. https://votematch.org.uk/. Accessed 25 Mar 2023

Inclusiveness of Knowledge and Knowledge for Inclusiveness. Collaborative Digital Technology for Made in Italy Patrizia Ranzo

and Salvatore Carleo(B)

Università Degli Studi Della Campania “Luigi Vanvitelli”, Caserta, Italy {patrizia.ranzo,salvatore.carleo}@unicampania.it

Abstract. In line with the Next Generation EU recovery plan and with the Italian National Research Program (PNR) 21–27, the issue of inclusiveness in response to social inequalities and social exclusion is currently a topic of central importance, in addition and transversal to the issue of digital sustainability, because it is not possible to imagine a more sustainable world without asking the question of the knowledge necessary for the double sustainable and digital transition and their access. The topic of inclusiveness has roots that arise in the post-war period, as far as the West is concerned, and it is also a theme in continuous change and definition. Methodologies such as Design for All, Universal Design and Inclusive Design, that deal with the issue of inclusion, have undoubtedly provided the tools to address the issues of access to the knowledge-based society. This essay reports the experimentation, within the National PHD Course in “Design for Made in Italy: Identity, Innovation and Sustainability”, of a Cyber Physical System that connects knowledge and instruments in the key of open innovation matured and / or present in the Doctorate. In this context, the creation of a collaborative knowledge chain dedicated to Made in Italy represents a useful tool for strengthening the resilience and sustainability of the production system. Keywords: Inclusive knowledge system · Made in Italy · Collaborative research supply chain

1 Premise 1.1 Strategic Lines for the Resilience, Sustainability and Competitiveness of the Made in Italy System: Collaborative and Digital Production Chains The resilience of the Made in Italy sector, such a strategic sector for our country, must be supported through strategies capable of combining technologies, advanced innovation (both financial and technological) and humanities (social and cultural values and innovations). The latter characterize the Made in Italy in a pregnant way, which conveys the complex Italian identity condensed in the high quality of its productions throughout the world. The environmental sustainability of products and processes is now an unavoidable factor in contemporary industrial scenarios. Italy presents excellent data from this point © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 608–614, 2024. https://doi.org/10.1007/978-3-031-47281-7_50

Inclusiveness of Knowledge and Knowledge

609

of view: it is the European country with the highest percentage of recycled waste (79% of the total), it is first among the big European renewable energy sector on total energy consumption (18.3%); over 95% of waste deriving from wood processing becomes raw material for the production of semi-finished products (Source: Federlegno Arredo Research Office on UN COMTRADE data); in 2018 Italy is among the first countries in the world for the number of organic crops with 70,000 organic agricultural producers (Source: Symbola, Processed by Coldiretti on Sinab/Eurostat/FIBL-IFOAM data); our agriculture is among the most sustainable, with just 7.2% of all national emissions (Source: Symbola, Processed by Coldiretti on Eurostat 2017 data). As far as data on the agri-food and food and wine sector are concerned, Made in Italy products are world leaders in quality and safety, with the lowest amount of chemical residues beyond the legal limits (Source: Symbola, Bonifiche Ferraresi, Coldiretti on EFSA data 2019). Within the Italian fashion sector, more and more companies are sensitized in this sense, also through studies and actions by trade associations (Camera Nazionale della Moda, Italiana, 2017; Sistema Moda Italia, 2018). These results are above all the expression of small and medium-sized Italian enterprises, which transform their apparent fragility into flexibility for change. The same phenomenon also occurs in the advanced technology sector: 18.6% of companies operating in the 4.0 environment are small companies, demonstrating that the size component is not always the cause of the failure in adopting new technologies, but rather the lack of corporate strategies and visions (Bettiol M., Di Maria E., 2019). An important fact is the growth of robot installations in Italy, which between 2017 and 2018, marked +27.3%, more than in other European countries: in the last three years robots in the company have increased by +48% in the food industry, +27% in fashion; +21% in wood-furniture, +23% in engineering (Source: Symbola, IFR, World Robotics, 2019). These data demonstrate that, in order to outline solutions and strategic lines for the competitiveness of the sector, the general and systemic vision in which they are inserted is fundamental, especially with respect to the many opportunities that are already present at a European and Italian level at the moment (PNRR, PNR 2021–27) and the necessary sustainable and digital transition. 1.2 Open and Digital Innovation: Connected and Collaborative Supply Chains Connected and collaborative production chains represent a necessary transition towards an industrial system capable of responding to an increasingly changing and unstable scenario. The contemporary economic system, based on the knowledge economy, requires a close connection between research and business through the implementation of digital and smart collaborative models. Today the essential goals are: the promotion of knowledge dissemination and the systemic collaboration between research and business; the development of approaches focused on widespread skills by promoting technology transfer, inclusiveness, the reduction of economic and territorial inequalities. Similarly, in order to support small and medium-sized enterprises on the path towards sustainable production, a continuous and structural support policy is needed to implement systemic sustainability of the Made in Italy supply chains. Based on these considerations, the experience of the National PHD Course in “Design for Made in Italy – deMIT” is reported below, which documents the creation of a cyber

610

P. Ranzo and S. Carleo

physical system for sharing knowledge, laboratories, skills. This experience con figures, through a close relationship with businesses and territories, a collaborative and open knowledge chain.

2 The Experience of the National PHD Course in Design for Made in Italy. The Creation of a Cyber Physical System for Open Innovation 2.1 Design for Made in Italy: Identity, Innovation and Sustainability The PhD Course of National Interest in “Design for Made in Italyde-MIT” is part of the goals outlined by the European scenario (UN SDG’s, NGEU, Green Deal), by the PNR and the PNRR and responds to the current demand for advanced training in the scientific sectors of Made in Italy. The typical production areas of Made in Italy represent the main aspects of material and immaterial culture of the Italian system, combining creativity, manufacturing knowhow and technological innovations. These aspects constitute the expression of the Italian identity that emerges from local, natural, cultural and social specificities, generating crossover dynamics with contiguous economic sectors, such as tourism, cultural heritage, creative industries. In this context, design and creativity play a driving role in innovation, fully expressing the paradigm of open innovation. The renewed aptitude for cooperation and interaction between supply and demand, developed in a framework of reciprocity and circularity, will facilitate listening to innovation needs, promoting a substantial advancement of processes, products and services in relation to international strategic goals. The Organizational Model. The PHD Course deMIT is an inter-university program with administrative headquarters at the University of Campania “Luigi Vanvitelli” and with the participation of 14 Italian universities: University of Florence, University of Calabria, University of Genoa, University of Palermo, IUAV University of Venice, “Ca’Foscari” University, “G. d’Annunzio” University of Chieti Pescara, University of Camerino, University of Siena, University of Studies of Basilicata, University of Salento, University of Urbino “Carlo Bo”, University of Trieste, University of Milan. PHD Curricula. The PHD program is promoted throughout the country by a group of universities in associated form on the basis of a shared research program organized into five curricula: 1. Design for Inclusion It trains designers-researchers capable of producing original contributions on theoretical, instrumental and planning knowledge in the field of design for inclusion, in order to guarantee equal opportunities for participation and respect for human diversity in every Made in italy activity.

Inclusiveness of Knowledge and Knowledge

611

2. Sustainable innovation of products, services and processes for Made in Italy It trains designers-researchers and experts capable of relating to the strategic sectors of Made in Italy to promote the transfer of technology and knowledge, developing circular processes, products and services within the production ecosystem and the wider territorial and supply chain system. 3. Made in Italy and technological and digital transition It trains designers-researchers in supporting and promoting the opportunities for technological innovation, improving productivity, the use of resources, economic performance and competitiveness. 4. Made in Italy: manufacturing and new economic and business models It intends to train expert researchers capable of producing open and sustainable innovation models for Made in Italy, in relation to the need to respond to economic and social instability. 5. Made in Italy Cultural Heritage It trains researchers and experts interested in carrying out scientific research on a critical, theoretical, methodological and applicative level in the field of Made in Italy Cultural Heritage, with the aim of promoting an open and sustainable fruition and access. 2.2 The Creation of a Cyber Physical System for Open and Inclusive Innovation A Cyber-Physical System (CPS) is a system in which the physical and virtual worlds are tightly integrated. It is composed of a network of devices, sensors, actuators, computers and software that work together to control and monitor physical processes in real time. CPS technologies enable to capture, process, analyze and share data while improving productivity, efficiency and security. The PHD Course deMIT is itself a cyber physical system that involves a set of actors, companies and researchers at a national level, in a complex architecture conceived as an ecosystem of enabling technologies, experimental laboratories, knowledge and skills, aimed at open and sustainable innovation for Made in Italy. The different research curricula are focused on the integration of digital technologies with traditional production processes, promoting the technology transfer. Methodologies such as Design for All, Universal Design and Inclusive Design have undoubtedly provided the tools to address, from a design point of view, the issues of access to a knowledge based society; this access is essential to promote sustainable and inclusive processes of shared open innovation. Inclusiveness of Knowledge. A Cyber Physical System can be, in the first instance, a useful tool to improve the inclusiveness of knowledge by allowing fair and personalized access to data and resources gained in the scientific community (Fig. 1). Knowledge for Inclusiveness. A fundamental component of a Cyber Physical System is Artificial Intelligence (AI) designed to provide valuable information for the selection and organization of data. While AI can help users manage knowledge, it can present

612

P. Ranzo and S. Carleo

Fig. 1. Representation of DeMIT, National PHD Course in «Design per il Made in Italy». Is supported by 15 Italian universities and the proposing university is the University of Campania “Luigi Vanvitelli.”. The PHD Course has a shared Research Program organized into No. 5 Curricula.

problems related to implicit biases embedded in algorithms (cognitive biases). A possible solution lies in the methodologies and tools of interaction design: interaction design focuses on the user experience, starting from their needs, behaviors and expectations. This approach puts the user at the center of the AI design process, ensuring that the solutions are actually usable and acceptable. In this way, the risk of incorporating data and implicit biases that could negatively influence decisions can be reduced. Finally, interaction design promotes ethical and inclusive design, which is based on the principles of justice, equity and diversity. Access to Digital Resources. CPSs can enable broader access to digital resources such as documents, videos, e-learning, digital libraries, and more. This can improve the accessibility and inclusiveness of learning tools, enabling people with different abilities or cultural backgrounds to access educational resources fairly and without barriers.

Inclusiveness of Knowledge and Knowledge

613

Personalization of Learning. CPSs can be used to develop customized learning, study and research solutions that fit the specific needs of each researcher. Online Collaboration. CPSs can facilitate networked collaboration between researchers, allowing people from different geographic regions or different cultural back grounds to work together synchronously or asynchronously. Remote Access to Laboratory Resources. The deMIT CPS will allow remote access to the scientific laboratories of 15 universities, favoring remote experimentation and research, the development of prototypes and the sharing of research and related results. Enhancement of Skills. The deMIT CPS, through the collection of profiles of the research community, will allow the enhancement of people and the results of their research through open publications, curricula, documents. Abstraction and Architectures. For rapid design and implementation of CPSs, it is necessary to develop innovative approaches to abstraction and architectures that enable integration of control, communication and computation. For example, in communication networks, interfaces have been standardized across different layers. Once the interfaces have been established, modularity allows for specialized developments at each level. The overall design allows heterogeneous systems to be composed in a plug-and-play mode, opening opportunities for continuous innovation. Challenges and Opportunities. Progress in CPS testing and implementation can be accelerated by identifying needs, challenges and opportunities in different industrial sectors and by encouraging multi-disciplinary collaborative research between academia and industry. The goal is to develop new systems science and engineering methods to build high-security systems in which cybernetic and physical designs are compatible, synergistic and integrated at all scales. Current and past industry investment in CPS technology research has been significant, but has focused on short-term, fast-return proprietary technologies. Recently, governments and some industrial sectors are investing in longer-term pre-competitive technologies and innovative testbeds. For example, the European Union has launched a major joint technology initiative with public and private funding from European nations and industry, called “Advanced Research and Technology for Embedded Intelligence Systems” (ARTEMIS). Similarly, based on recommendations in the August 2007 report of the US “President’s Council of Advisors on Science and Technology” (PCAST), the US National Science Foundation has funded basic research and education on CPS. Similar initiatives are underway in other countries, including Japan, China, South Korea and Germany. The big challenges of CPS are articulated in many industrial sectors. The US National Academy of Engineering has listed 14 major challenges affecting the environment, health and society; these themes will clearly benefit from the progress achieved in cyber-physical systems (Fig. 2)).

614

P. Ranzo and S. Carleo

Fig. 2. Representation of the Cyber Physical Systhem DeMIT, National PHD Course in «Design per il Made in Italy». Is an open and continuously updated system.

3 Conclusions Although the experimentation of the CPS of the PHD deMIT is in the implementation phase, important repercussions are already foreseen in the context of the collaboration of researchers at a national level, of the implementation of specific knowledge and of the repercussions on Made in Italy companies as regards technology and knowledge transfer.

References Sistema Moda Italia (2018) Viaggio verso la sostenibilità della filiera. https://www.sistemamodai talia.it/it/progetti-e-attivita/sostenibilita/

Digital Design and Data Visualization for Society Health and Wellbeing Roberta Angari1(B)

and Gabriele Pontillo2

1 Department of Architecture and Industrial Design, University of Campania “Luigi Vanvitelli”,

Via San Lorenzo 31, 81031 Aversa, Italy [email protected] 2 Department of Design, Design Campus, University of Florence, Via Sandro Pertini 93, Calenzano, 50041 Florence, Italy [email protected]

Abstract. The contribution deals with the theme of the design of wearable and active devices, capable of communicating with the end user, mediators of the health condition and perceived and effective well-being of individuals. The state of the art on the subject highlights how the use of wearable devices in the field of e-health, capable of monitoring the biometric data of individuals, has grown. Nonetheless, these devices are used for limited periods of time, becoming – in fact – a sort of fad, or in any case a passing phenomenon. One therefore wonders whether it is possible to intervene on the design of this type of artefact, in such a way as to trigger a relational link between the object and the user, in order to increase the latter’s awareness of his/her health condition, with an impact relevant, with respect to the widespread quality of life, especially in cases in which one is confronted with more fragile sections of the population, in the presence of some pathological conditions, and for which monitoring of this type is therefore relevant. The approach used will make use of the analysis of the scientific literature and the state of the art of reference, useful for understanding the interest and relevance of the contribution, especially after a hybridization between knowledge and between different disciplinary areas of the design culture, namely that of digital design, communication design and data visualization, used to outline an innovative approach described in the contribution. Keywords: Digital design · data visualization · health and wellbeing · Digital Custom Design · Innovative design approach

1 Introduction The perception that individuals have not only of diseases but also of health in broader terms, can influence both negatively and positively one’s well-being, as well as the immune system, stimulating also miraculous reactions [1]. By interweaving studies ranging from the fields of medicine to information technology, sociology, and design, it emerges how the role of users has evolved, not only as patients but also for society as a whole. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 615–628, 2024. https://doi.org/10.1007/978-3-031-47281-7_51

616

R. Angari and G. Pontillo

Aspects such as self-awareness and communication with patients, trust in science and medicine, as well as in treatments and therapeutic plans, the access to medicines, the desire to feel good and/or to heal, are just a few of the themes with which design, and in particular medical and communication design, are facing, as they are capable of translating or converting the complexity of information and medical practice into something visible, concrete, and tangible. Design is capable of making enormous changes, often with solutions that, precisely thanks to disciplinary convergence, can achieve multiple objectives at the same time. For example, the area of smart and wearable devices, is of particular interest in the health design sector, because it is based on an already existing phenomenon, linked to the diffusion of technologically advanced interactive devices, which - worn on one’s body – register biometric data, such as body temperature or heart rate, also in real-time. Yet, technological performance alone is not enough to make these devices part of the object panorama of individuals, except for limited periods. Still concerning the medicine, health, and well-being sector, we observe the diffusion of digital tools and platforms, which communicate and display data, often in fascinating formats, but difficult to understand for heterogeneous user groups. On the other hand, a further risk, intrinsic in the field of data visualization [2], and in the design of medical/scientific visualization, consists in excessively simplifying the information, leading to a flattening of the multidimensionality of the data, which therefore does not represent any source of knowledge for the end user. On the other hand, the visual communication sector can reach fewer literate groups of users. As highlighted by the history of communication and information design, it is thanks to Otto Neurath and to his methods and approaches, that we can observe the firsts examples of dissemination and democratization of knowledge, also in the medical and scientific field [3]. Nevertheless, this kind of project often remain linked to traditional communicational mechanism [1], such as awareness campaigns or sector magazines, equipped with a more accessible cut to information, or dedicated to specific context, as we can observe in the hospital campaigns. The aim of this contribution is therefore to investigate the sphere of design for the health and well-being of society, through this double key of interpretation, namely that of digital design and data visualization, through the intersection of which, it is believed it is possible to lead people towards greater individual awareness, an aspect that is of particular relevance, especially when dealing with fragile user groups, for whom monitoring their health condition is crucial in their daily lives.

2 Background The spread of digital is an endemic condition of the contemporary world that has profoundly changed both daily and work actions, through tools and devices that are part of people’s everyday life [4]. This also applies to the field of health and well-being, in which digital has become a tool and a strategic means for improving the widespread quality of life: one of the most relevant reasons of this aspect, is that digital can abbreviate distances between medicine and society, also by recording biometric data in real-time thanks to portable or mobile equipment.

Digital Design and Data Visualization

617

In fact, in recent years, there has been growing attention to issues related to the quality of life, health, and well-being in society, so much to require tangible and intangible artifacts that can bring medicine closer to people, to improve the health condition of the community [5]. Themes such as misinformation on medical/scientific issues, inaccessibility of services and poor management of treatment plans and access to medicines, awareness of the growing fragility of the population to the spread of viruses and infections, as well as the aging of the population, are just some of the issues that the health sector is confronted with [6], and with which, therefore, also design, in its various facets, is called to face, because it is not possible to think that design is something separate from health [7]. This is even more true if we compare ourselves with the definition of Quality of Life, by which we mean the convergence between different domains such as psychological and physical health, independence, relationships, and social conditions, up to the sphere of spirituality, religion, and personal beliefs [8], themes that emerge also from the study on the concept of Health, reported by the Horizon EU Framework Program [9]. It is evident how the quarrel is with a plurality of emerging issues which, having have a focus on health, and require a response on the design level, which can determine not only an advance for the state of the art of reference but also have a greater impact on the population, optimistically determining an improvement in the widespread quality of life. In this regard, we recall the definition of e-health, as outlined by Eysenbach, according to which the term does not refer solely to the domain of electronics, but to all those parameters such as efficiency, enhancing quality in healthcare, evidence-based intervention, user empowerment, education, etc., which represent the challenges in the field of medical design, health and well-being. This definition is of particular interest for the research, because it is related to the possibility of expanding the boundaries of health through digital tools, that allow users to acquire information and, as well, to obtain advice or indications thanks to communicative actions that can move from the more specific and technical areas of medicine to the more general ones [10]. Finally, the theme of equity is highlighted, because, in a society that become increasingly fragmented and heterogeneous - both from the point of view of asymmetrical knowledge and a socio-cultural and ethnic point of view -, making health care equitable is an absolute duty. Therefore, also designers cannot fail to address this theme to guarantee a democratic and open diffusion of medical/scientific knowledge and services, regardless of the economic or cognitive capacity of the individual, or of the specific skills, which may affect its use. To do this, especially in the design of health and wellbeing for society, it is of fundamental importance to design by guaranteeing accessibility and social inclusion, as well as for different skills and competencies, in order to reduce as much as possible, the gap that still exists between different populations, economic contexts, age groups, gender, etc.

618

R. Angari and G. Pontillo

3 Tools and Technologies at the Service of Design for Health and Wellbeing The recovery of an ethical dimension, also in the digital-driven project, and in particular for the health and well-being sector, is believed to be an opportunity to investigate more carefully the aspects related to economic development, inclusion, and innovation in the social, individual, and collective sectors. This is because the ubiquitous diffusion of digital, allows us to rethink the entire design and production chain, finally reaching the conformation of objects augmented by digital. Since the 2000s, we have witnessed what is defined as the digital turning point, i.e. diffusion of integrated and interactive, flexible and dynamic tools, born from the combination of information technology, telecommunications technologies, media, and electronics [11], which have led to an acceleration of communication processes and a greater movement of data and information, in a process that has indiscriminately affected both the nature of objects, hardware and software, as well as the role of users, from passive consumers to active subjects. Part of this scenario are Advanced Manufacturing (AM), 3D printing (3DP), and the Internet of Things (IoT), i.e. all those processes based on open source and online communities, which are not only a testing ground in which the new generations of designers and innovators express themselves, but above all they represent the evolutionary force through which design is reinterpreting the relationships that people establish with objects and intangible assets, effectively recoding the DNA of contemporary objects [12]. In this panorama, experiences generated and managed through digital emerge, interpreting digital as a tool that accompanies the project without dominating it in the creation of new solutions in line with the advancement and evolution of society, which seek to identify contact points between advanced technologies and the world of experience. For this reason, it is possible to state that the drive towards dematerialization is counterbalanced by tangible artifacts, which mediate the relationship between people and digital information and functions. This is the reason why digital manufacturing, through 3D printing, as well as robotics, artificial intelligence, and digital co-design platforms, raise the expressive and executive possibilities of design [12], and, at the same time, establish a dialogue with the company, analyzing not only the daily needs, but also promptly intervening in the appropriate changes or customizations. These are some of the reasons why new technologies also find application in the health and well-being sectors, from whose analysis of scientific literature and the state of the art of reference, experiences emerge that use advanced technologies and tools for the creation of prototypes and/or customized products. Advanced Manufacturing, for example, has contributed to considerably speeding up the conception, creation, and finally production process of an object, thanks to 3D printing, through which it is possible to create even very complex geometries, which can be customized according to requests and body measurements of each individual. To do this it is necessary to understand that the process is not based solely on the rapid prototyping of the single artifact, but we are talking about a process that starts from the definition of the geometry, through the digital survey of the user’s body, through – for example – 3D scanning or photogrammetry [4, 13] from which it is possible to obtain models that perfectly follow the shapes of the individual’s body, adapting to it. In this

Digital Design and Data Visualization

619

regard, it should be noted that digital modeling, especially in cases where the design approach is that of generative and/or parametric design, allows for the creation - rather than the single form - of a process consisting of calculations and mathematical functions placed in relation each other, shifting the focus of the project from the realization of the form - form making - to the search for form - form finding – [14]. This type of approach is part of the digital supply chain [15] because, on the one hand, it allows the customization of the artifact through the modification of values or numerical parameters - custom design - and on the other, it is easily repeatable - open design. In this perspective, the potential of these techniques and approaches in the field of health and well-being design is evident, as it is possible to produce customized devices, with very law costs and times compared to traditional methods, also making use of a technology which minimizes the materials used and - in any case - uses sustainable materials, overall improving the patient/user experience for the results obtained [16–18]. If on the one hand, digital design allows the conformation of devices that adapt to users, another theme to reflect on is that of the Internet of Things - IoT, which would further increase both the functional and expressive capabilities of objects, making them capable of transforming anyone in a data source - people, other objects, the surrounding world - by listening, monitoring and measuring the perpetual movement of their surroundings [19]. Undoubtedly the topic of active sensors is a relevant subject, with which all areas that have an interest - not purely economic - in the recording and monitoring of data, are called to deal, also if the implications related to the protection of privacy must be taken into account consideration. Nevertheless, the potential of these technologies is undoubted, because transform objects, and more particularly wearables [20], into something that feels, and which therefore interfaces and relates with the person who wears it. By definition, wearables are devices that are worn and not carry [21]. They are therefore wearable, and always removable, tools that interface with the person wearing them thanks to the presence of biometric sensors and mechanical, visual, and sound interfaces, which allow bi-directional communication with the user, based on the receipt of recorded data of the body and the emission of feedback of various kinds - visual, sound, etc. [22]. The wearable devices currently on the market mainly belong to the sports and/or medical sector - note the vast diffusion of smartwatches -, nevertheless the potential applications of these objects are still to be investigated, especially as regards the communicative and relational interface between users and objects. It can be observed that the simultaneous technological advancement and miniaturization of electronic and sensory components have allowed the diffusion of objects, even of very small dimensions, such as smart-watches and smart-rings which instantly measure vital parameters, and the renewed attention to individual well-being and health, although they have led to a greater diffusion of these devices on the market, however do not guarantee their continuous use, for which there is still a growing abandonment rate after the first months of use of the device [23, 24], a fact that therefore highlights a real lack of need for the device on the part of users. One wonders, therefore, whether wearable devices can be useful: as Norman states, it all depends on the use made of them, whether to increase our activities or as a source of distraction [20]. In this regard, it should be noted that the role of design is decisive,

620

R. Angari and G. Pontillo

not only from the conceptual point of view but also during the research and identification of needs phase, thus basing the project not only on the market trend but on the actual needs identified by the reference context, responding to specific requests throughout the project. For this reason, it is believed that the field of visual communication and data visualization can be strategic in the outlined scenario, as through the use of communication methods and strategies, it would be possible to give shape and meaning to medical/scientific data and information, allowing as many people as possible to understand, metabolize and use these data, so that they translate into an improvement in the quality of their lives [25]. Therefore, it is up to the designer to conceive and develop artifacts for scientific communication, which can induce people to pursue healthy lifestyles and/or therapies, appropriate to their health conditions [26], communicating data and information technicians, through a non-academic, clear and complete language, useful for confirming authentic news and preventing false ones [27]. From this point of view, it is believed that the convergence between different design approaches and methods, typical of digital design, medical design, visual communication as well as data visualization, together with the new tools and technologies described in this section, can represent an important field of experimentation for the advancement of scientific literature and the state of the art on the subject, as well as in the search for new solutions designed for users, to create a new category of wearable devices, augmented by digital, which can facilitate the dissemination of medical/scientific communication that is based on individuals, and therefore is understood as a bottom-up, and not a top-down one.

4 State-of-the-Art Analysis The continuous and exponential growth of devices equipped with sensors and electronic components that become part of the object panorama of individuals is well known. It is enough to carry out a quick survey of the most widespread online shopping platforms, and it is possible to notice a decidedly vast diffusion of smart devices useful both for environmental monitoring and for the individual. All these objects generate, day after day, a multitude of information, virtually connected to places, spaces, and actions of daily life. This theme is of particular interest, especially for sociology, anthropology, etc. scholars, but how vertically do we enter the theme of health and well-being, what types of data are recorded, and how they are used by the devices that surround us? One wonders, is it really necessary to record all this information? As previously stated, there are various smart devices, such as smart-watches and smart-rings, which today can be used for monitoring biometric data they are widely used mainly in the sports sector, even amateur sports, where users need to record and visualize their data, mainly for reasons related to performance monitoring [28]. Then there is a panorama of devices that integrate a material component - hardware - and an immaterial one - software - for tracking and monitoring one’s health condition. This is the case of users who have pathologies and/or health conditions that require

Digital Design and Data Visualization

621

more in-depth monitoring of certain data, which must be made accessible to increase user awareness, but also to notify or warn both cases of normality and cases where there are anomalies or any problems. This is the case of Heylo1 , a device for people who live with a stoma, a surgically created opening in the abdomen to allow waste to escape, and for whom leaks have a significant impact on daily life, both physically and mentally. From the analysis of this need, the Heylo project was born, which offers users the possibility of tracking their ostomy on time and evaluating the integrity of their device through a special APP. Indeed, the sensor layer that is applied to the body is equipped with sensors that digitally record any leaks, and react instantly by providing feedback via the APP. Heylo is available in different sizes, this makes it comfortable for different body sizes and shapes (see Fig. 1).

Fig. 1. Hardware/software system of the device Heylo.

If the example shown above is based on a specific need, linked to an equally specific range of users, the Helios2 and OHealth3 devices, from the electronics company OPPO, are instead based on the need to ensure widespread access to information related to individual health. OHealth, the acronym of “Home Smart Health Monitors”, is a smart device for monitoring health, useful to measure heart rate, blood oxygen level, ECG, and respiratory rate, as well as heart and lung sounds, anywhere and anytime. OHealth is provided by a dedicated mobile APP, in which it is possible to create one’s electronic health record, as well as upload the data recorded in a medical database, to access remote assistance for routine diagnoses. Helios, on the other hand, is a device through which users can measure heart rate, blood oxygen level, ECG, and respiratory rate, as well as heart and lung sounds, anywhere and at any time. 1 https://www.red-dot.org/project/heylotm-58520.

https://ifdesign.com/en/winner-ranking/pro ject/heylo/348443. 2 https://www.red-dot.org/project/helios-health-device-58508. 3 https://communityin.oppo.com/thread/1222328781332021249.

622

R. Angari and G. Pontillo

Unlike Heylo, to characterize the two OPPO devices, there is also a further aspect linked to the interface between users and device, because Informations are not communicated only by a mobile ALL, instead the device, itself, have in interface which display visual stimuli and report part of data (see Fig. 2).

Fig. 2. From the two images, is visible the interface of the two devices, Helios and OHealth, understanding how the communication component is relevant in the design of object that can record health data, in order to make them accessible to different kind of users.

This aspect is considered relevant for the reference sector, since, above all to facilitate the interaction between the device and the individual, it is of fundamental importance to be able to have different levels of communication available: an immediate level, i.e. linked to visual feedback, sound, etc., which is emitted directly from the wearable device, an information level, which for example can be linked to the interaction with an APP or a digital platform, through which to understand more information on the detection recorded by the device, and finally a level of in-depth analysis, linked to the interaction with doctors and/or experts in the sector, thanks to which any problems that emerged from the survey can be investigated in more depth. Concerning the level of detail, communication between users and the doctors is a topic of particular relevance, especially in the preventive and/or management phases of certain pathological conditions. Examples of experiences conducted in this specific sector include the research conducted by Chou, Hsu, and Chou of the Research Center for Healthcare Industry Innovation, of the University of Taipei, where, in response to the theme of population aging - which is one of the topics of particular relevance for the health design sector, described in the first sections of this contribution – a kit has been developed, which allows the patient to monitor himself at home, making the recorded information always accessible to his doctor, thanks to a dedicated online platform [29]. Similarly, NuboMed Medical IoT Kit4 collects patient data, such as body temperature, heart rate, and other important vital signs, in real-time, and then transmits them wirelessly to the hospital network, helping medical staff in clinical decisions. Designed to ensure high comfort, the devices, which are worn directly on the body, are made of silicone, 4 https://en.nubomed.com. https://www.red-dot.org/project/nubomed-medical-iot-kit-58384.

Digital Design and Data Visualization

623

ensuring optimal usability, as well as low energy consumption and environmentally friendly manufacturing. Finally, we report the case of Anura5 , which unlike the examples previously reported does not make use of any physical device, other than the smartphone. Anura is an APP for smartphones, which allows the measurement of blood pressure without contact, i.e., without the need for bracelets or other wearable devices, as it is based on the measurement through a conventional video camera using the patented Transdermal Optical Imaging (TOI) technology. This APP, which also measures other physical, physiological, and psychological indicators, including heart rate, stress levels, body mass index, and cardiovascular disease risks with medical-grade accuracy (based on research and clinical studies), has, of fact, significantly modified the state of the art of reference thanks to the possibility of measuring and monitoring health and general well-being, through the sole use of one’s smartphone (see Fig. 3).

Fig. 3. The two images, show how the Anura APP works, with particular attention to the detection of the user’s face, and the translation of data recorded into diagrammatic representation available in the smartphone APP.

5 A Design Approach for Health and Well-Being Designing for the health and well-being of society, means dealing with a very complex project area. Although purely communicative system or design products can represent a possibility of advancement with respect to the reference state of the art, it is believed that the area of active and communicative wearable devices is strategic, because allow not only to record data and information but also to communicate with the end user, allowing greater self-awareness. Nonetheless, wearing an object is a choice of the individual, which can alter the balance of the body, both from a physical and perceptive point of view, an aspect that can lead to the rejection of a wearable [22], for purely ergonomic reasons such as the device is heavy or obtrusive, or tied to the ground of the object. Otherwise, a wearable 5 https://www.prnewswire.com/news-releases/award-winning-anura-convenient-and-contac

tless-health-tracking-comes-to-any-web-browser-anytime-301454313.html.

624

R. Angari and G. Pontillo

device to ne used day-by-day by the end-user, must be meaningful for a person’s life, who will wear it also if it must be recharge – i.e. To overcome the critical issues that determine the most probable crisis and abandonment factors, it is necessary to dig into the deepest and most concrete needs of the users, through which it is possible to create new design solutions which, if on the one hand constitute an advance for the state of the reference art, on the other hand, can lead to an improvement in the widespread quality of life. By exploiting cross-fertilization from different fields of investigation [30–33], it was possible to define a hybrid and open approach to people’s interactions with the design system, as well as with its outputs, real and virtual. This approach is called Digital Custom Design, the purpose of which is to define a design process augmented by digital, useful for the creation of customized and wearable devices. The approach consists of four phases, which are interoperable with each other and not necessarily sequential, namely: Digital Human and Metaverse. Dematerialization of the human body, and transposition into a digital environment through scanning and digital relief techniques. This phase allows you to acquire the geometries from which to start for the definition of a wearable device model that adapts perfectly to the geometry of the user’s body. A further potential relating to the digitization of the body consists in the possibility of generating avatars, through which to carry out virtual simulations on an ongoing basis. Digital Design and Prototyping. Conformation of the digital model of wearable devices and realization through AM technologies. In this phase it is also possible to study the integration of electronic components and IoT sensors, making them an integral part of the designed device [34]. Wearable Devices and Smart Technology. In this phase, the operating scheme of a wearable device equipped with smart technologies has been defined, which - as previously reported - allows the detection and monitoring of data, and at the same time the communication of the single device with the user and with the surrounding ecosystem, for example with other devices – i.e. smartphone – or with digital platforms and environments for storing and displaying data, thus ensuring an interface between wearable and user, responding to the different levels previously reported, i.e. level of immediate communication, level of information and, lastly, level of detail. Digital Communication and DataVis. It is the phase that deals with understanding how to convert the collected data into visual artifacts, making use of the visualization skill (Ricci, 2007), typical of visual communication designers and, more particularly, of data visualization and medical/medical visualization designers. Scientific, which starts from the analysis of the recorded data and the comparison with the cultural and scientific domain of reference, to then move on to the translation of the information into visual form, through dynamic maps, infographics, and interactive diagrams, which can be made accessible on various devices of the user, also, in this case, responding to the different levels of decoding and access to knowledge. Of particular relevance, in this context, is also the possibility of obtaining information and data which, if on the one hand, they come from a single individual, and therefore respond to a bottom-up logic, on the other hand, they can also be used to increase opportunities and the elements on which to

Digital Design and Data Visualization

625

base the project of dissemination and dissemination artifacts. Although it is necessary to obscure some information, others can be used by aligning themselves with the strategies reported in the fields of Open Science and Citizen Science, which see in the active participation of society an important opportunity to make the latter more aware of the themes and advances of scientific medical research. Furthermore, the outlined approach fits into a matrix scheme, characterized by the dichotomies: 3D Digital / 3D Physical; Products / Communications; Data monitoring / Data visualization; Wearable devices / Digital Devices. The matrix relationship between the two axes is based on the intersection of specific skills and approaches, which will be applied in the development of innovative customized solutions (see Fig. 4). Finally, although it is specified that the approach described in this section can also be scaled for other areas of the project, its connotation, centered on the user, makes it particularly useful for cases in which specific needs must be faced, and therefore, in the case of design for health and well-being, allowing in all the phases described to be based on the study and observation of both the single individual and a group of users

Fig. 4. From the diagram shown, it is possible to understand how the described approach is structured, including its matrix organization.

626

R. Angari and G. Pontillo

- think, for example, of the cases of users who fall within specific therapeutic and/or rehabilitation plans, or which must follow specific medical indications. This last aspect makes the approach responsive, at least from the point of view of the process, to the assumptions reported in the first sections of this contribution, i.e., that of being able to design wearable and communicative devices that can not only be an advance on state of the art, but also be a means to improve the widespread quality of life, even for the most fragile sections of the population, in the presence of certain pathological conditions.

6 Conclusions When dealing with the issue of health and well-being for society, the project idea tends to move towards the field of device and product design, rather than towards the visual communication sector, completely neglecting the potential that can arise from the intersection of these two areas of the project. Never as in this historical moment, the assessment of individual health is a relevant issue for individuals, who feel the need to have greater self-awareness. Yet, we remain tied to the design of wearable devices, such as smart-watches, etc., which suffer from the rejection of a wearable, except in cases where we are talking about users who have to use the device mainly for reasons related to monitoring sports performance, or for medical monitoring, in the case of patients suffering from certain pathologies. On the other hand, it is believed that increasing the use of health monitoring devices could be a strategic choice at various levels because if on the one hand, it offers the individual user the possibility of having greater awareness, on the other it also allows preventive monitoring for any health problems. At the same time, data recording is also a potential for the public utility communication sector, both in terms of dissemination and dissemination, as it allows, obviously in compliance with the legal limits of reference, to have access to a vast set of data from which to start both for medical/scientific research and for drafting reports that are of fundamental importance for society as a whole, as they represent the mirror of what is happening. In this sector, it is necessary to start from the definition of a design approach, to identify tools and technologies that can be used in the creation of active wearable devices, capable of communicating with the end user, establishing a relationship with the latter that does not stop for reasons related - for example - to recharge the battery, but the user must be encouraged to use the device. To do this, it is necessary to make use of digital technologies, such as AM, and 3DP, as well as virtual simulations, which allow the creation of customized devices, which can be equipped with smart technologies from which to start the recording of data which are then converted into information and therefore knowledge, through the best-known approaches of data visualization and information design. In this perspective, it is highlighted how data visualization can have an impact on individuals’ attitude, motivation, perception and decision-making. Interventions related to the use of data visualization have been shown to have a positive impact on cognitive change, behavior and decision-making, also leading to greater engagement and improvement in the perception of information, increasing the amount of information provided and understood by users, decreasing the cognitive and intellectual burden to interpret

Digital Design and Data Visualization

627

information for decision making [35]. It is for this reason that it is believed that research in this area must be carried out, taking care to hybridize skills and knowledge, in order to develop solutions that can benefit the health of society. Although this research activity is taking its first steps, it is believed that the subsequent topics to be treated will be linked to possible recruitments and needs study, so as to understand which data to monitor and how to make them visible, and at the same time which types of devices can satisfy the needs of users. As a conclusion, some aspects are considered to characterize the contribution described and the research activity carried out, in particular as regards the theme of project ethics, which cannot avoid both reflections on environmental sustainability and those related to the possibility of democratically accessing both devices and knowledge, thus centering the entire design activity on people and society, guaranteeing social inclusion and innovation, especially in the health and well-being sector. Acknowledgements. Although the approach and contents of the essay are shared by the authors, it should be noted that Roberta Angari is responsible for the Introduction, the sixth paragraph, and the Conclusions, while Gabriele Pontillo is responsible for the second paragraph. Both the authors wrote the third and fifth paragraph.

References 1. Maiocchi M (2008) Design e comunicazione per la Sanità. Maggioli Editore, Rimini 2. Burdick A, Drucker J, Lunenfeld P, Presner T, Schnapp J (2012) Digital humanities. The MIT Press, Cambridge 3. Angari R (2020) La democratizzazione della complessità. AIS/Des Storia e Ricerche 7(12– 13):215–230 4. Pontillo G, Angari R (2020) Acting responsibly. Des Sustain Pract Soc PAD J 20:213–233 5. Pontillo G (2022) Digital medical design: how new technologies and approaches can empower healthcare for society. In: Advances in design and digital communication iii: proceedings of the 6th international conference on design and digital communication, Digicom 2022, 3–5 November 2022, Barcelos, Portugal. Springer Nature, Cham, pp 255–269 6. Rowe A, Know M, Harvey G (2020) Re-thinking health through design: collaborations in research, education and practice. Des Health 4(3):327–344 7. Chamberlain P, Craig C (2017) Design for health: reflections from the editors. Des Health 1(1):3–7 8. WHOQOL: A cross-cultural study of spirituality, religion, and personal beliefs as components of quality of life. Social Science and Medicine 62, 1486–1497 (2006) 9. Horizon Europe General Guide. Catalyze Group (2021) 10. Eysenbach G (2001) What is e-health? J Med Internet Res 3(2):e833 11. Sciotto F, Zucchi G, Guzzetti D, Corbucci G (2003) La rivoluzione digitale. GIAC 6(2) 12. Langella C (2020) Design quotidiano al tempo della vulnerabilità diffusa. OP CIT 168:31–47 13. Zolfagharifard E (2023) When science is stylish: Designer uses 3D printing to create a range of accessories that is taking the fashion world by storm, Daily Mail. https:// bit.ly/3PsfYBx. Accessed 30 Apr 2023 14. Kolarevic B (2003) Digital production. Architecture in the digital age: design and manufacturing. Spon Press, New York

628

R. Angari and G. Pontillo

15. Proverbio P (2016) Fantascienza, tecnologia e filiera digitale. Conversazione con Denis Santachiara. In: Proverbio P, Riccini R (eds) Design e Immaginario. Oggetti, immagini e visioni fra rappresentazione e progetto, pp 225–243. Il Poligrafo, Padova 16. Tsegay F, Ghannam R, Daniel N, Butt H (2023) 3D Printing smart eyeglass frames: a review. ACS Appl Eng Mater 2(4):1142–1163 17. Thorsen R, Cugnod D, Ramella M, Converti RM, Ferrarin M (2023) From patient to makera workflow including people with cerebral palsy in co-creating assistive devices using 3D printing technologies. Disabil Rehabil Assistive Technol 1–11 18. Desselle MR, Wagels M, Chamorro-Koc, M, Caldwell GA (2023) How is point-of-care 3D printing influencing medical device innovation? A survey on an Australian public healthcare precinct. J 3D Printing Med 7(1):3DP005 19. Greengard S (2015) The Internet of Things. The MIT Press Essential Knowledge Series, Cambridge 20. Norman, D.: The paradox of wearable technologies. MIT Technology Review. https://bit.ly/ 3VYRCS9. Accessed 30 Apr 2023 21. Mann S (1997) An historical account of the WearComp and WearCam inventions de-veloped for applications in Personal Imaging. Digest of Papers. In: First international symposium on wearable computers. IEEE, Cambridge, pp 66–73 22. Zannoni M (2018) Progetto e interazione. Il design degli ecosistemi interattivi. Quodlibet, Macerata 23. Arthur C (2023) Wherables: one-third of consumers abandoning devices. The Guardian. https://bit.ly/3j3OllQ. Accessed 30 Apr 2023 24. Ledger D, McCaffrey D (2014) Inside wearables: how the science of human behavior change offers the secret to long-term engagement. Endeavour Partners 200(93):1 25. Olson GB (2000) Designing a new material world. Science 288(5468):993–998 26. Angari R, Pontillo G, Piscitelli D, Langella C (2021) TRANSITION RS: the design component for the communication of medical research. InfoDesign: Revista Brasileira de Design da Informação, 18(2) (2021) 27. Piscitelli D (2014) The Italian medical industry and medical publishing: the case of Sigma Tau. In: Bigatti G, Dardi D (eds) Business histories and design history: trials of dialogue. [S.I.]: Archetipo Books, pp 159–181 28. Kamišalic A, Fister I, Turkanovic M, Karakatic S (2018) Sensors and functionalities of noninvasive wrist-wearable devices: a review. Sensors 18:1714 29. Chou CY, Hsu DY, Chou CH (2022) A smart healthcare kit for home healthcare. J Math Tech Comput Math 1(3):232–237 30. Crippa AJ (2010) La pancia prima del mouse. Il ruolo de modelli taciti di pensiero nel processo metodologico del design. In: Maiocchi M (ed) Design e Medicina, Maggioli Editore, Rimini 31. Langella, C.: Design & Scienza. LIStLab (2019) 32. Pontillo G (2020) Il design tra medicina e natura: design parametrico bio-ispirato per lo sviluppo di dispositivi ortopedici personalizzati innovativi. In Riccini R (ed) FRID 2019: La Doppia Prospettiva della Ricerca in Design, Bembo Officina Editoriale, Venezia 33. Angari R (2020) Da John Maeda a Santiago Ortiz: La nascita del mate-grafico. In Riccini, R. (ed.), FRID 2019: La Doppia Prospettiva della Ricerca in Design, Bembo Officina Editoriale, Venezia 34. Pontillo G, Angari R, Langella C (2021) Parametric design and data visualization for Orthopedic devices. In: Perego P, TaheriNejad N, Caon M (eds) ICWH 2020, vol 376. LNICSSITE. Springer, Cham, pp 139–153. https://doi.org/10.1007/978-3-030-76066-3_11 35. Park S, Bekemeier B, Flaxman A, Schultz M (2021) Impact of data visualization on decisionmaking and its implications for public health practice: a systematic literature re-view. Inform Health Soc Care 47(2):175–193

Mother Angel or Swiss Knife? José Gago1(B)

, Tiago Marques2 , Ema Rolo3 and Noemy Berbel4

, Rita Almendra1

,

1 CIAUD Faculdade de Arquitetura, Universidade de Lisboa, Rua Sá Nogueira,

Polo Universitário, Alto da Ajuda, 1349-055 Lisboa, Portugal [email protected], [email protected] 2 CHAIA, Universidade de Évora, Largo Dos Colegiais 2, 7004-516 Évora, Portugal [email protected] 3 CLLC, Universidade de Aveiro, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal [email protected] 4 Universidad de Las Islas Baleares, Palma, Spain [email protected]

Abstract. A mobile digital app development project involves several phases. This research developed in Learning Setting promoted an exploratory phase intending to promote greater reflexivity among students, where they can iteratively explore various codes and patterns of information. This research focuses on identifying design literacy elements that can guide the future development of a design proposal. In a design course learning setting, this paper focuses on developing an exploratory phase of a project for an app for the future Central Hospital of Alentejo (Portugal) . This Study aims to answer how a strategy combining a thematic analysis and an exploratory mapping contributes to reflexivity on solutions to explore in a project. The Study follows the methodologies of descriptive case study and thematic analysis. The results allowed the researchers to verify that using different mappings in the exploratory phases allows the students to exercise Reasoning in the context of the broader find patterns that surprise them and increase their Engagement with the project. Keywords: Teaching Design · Visual Exploratory Strategies · Visual Maps · Learning Strategies

1 Introduction App development projects are born from the identification of problems. A large number of possibilities and features sometimes hampers this identification. In order to focus on the problems, it is necessary to evaluate the consumer characterization. It is necessary to develop an investigation that allows a survey of cultural characteristics and behaviors of the consumer as the territory that expands the designer’s literacy on the issue. Qualitative data can enhance understanding by bringing together context and, therefore, more information about the behaviors and attitudes of the target audience. As such, it can promote different avenues of exploration that enable researchers to understand © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 629–638, 2024. https://doi.org/10.1007/978-3-031-47281-7_52

630

J. Gago et al.

their audience better. A mixed-methods approach is also practical when a student needs to better understand who they are before starting the quantitative phase of a study. This quantitative phase of the Study can help researchers understand the frequencies of behaviors of interest and how attitudes and perceptions identified in the qualitative phase of the Study can influence specific behaviors. It is common to conduct focus groups or interviews to gain an initial understanding of the target audience and then use the findings of this phase to inform the development of a survey distributed to a larger group of participants [1]. In order to understand the consumption codes of a specific audience because we acquire a silent system of codes as we grow up within our culture [2]. These codes—the Culture Code—make the consumer American, German, or French. They invisibly shape how consumers behave personally, even when they are entirely unaware of their motives. According to Clotaire Rapaille’s interpretation, a child’s most active learning period happens in a particular country’s cultural context. Mental structures formed in that cultural identity (American, for example) environment fill his subconscious. The child, therefore, grows up an American. The influential imprints placed in their subconscious at this early age carry the influence of the culture in which the child grows up. That is why people from different cultures react differently to the same things. People’s earliest and defining experiences significantly predictably influence current and future preferences across the consumer life cycle [3]. The peers surrounding the child also have great importance and positively influence the awareness about a product and its relationship with it [4]. When we want to obtain a code of a culture or a product, the Clotaire Rapaille strategy comprises three stages and is part of the premise that “we should not believe what the consumer says.” In the first stage, the consumer answers a survey and says what he/ she likes or dislikes about the product. In the second stage, the consumer responds about which elements will change in the product. The third stage comprises the most critical information. A collection of anonymous testimonies about the first childhood memory, or the first memory about a concrete product, is conducted in a quiet, relaxed atmosphere. Textual analysis reveals the patterns sought throughout the testimonies, seeking to identify a concrete code. Clotaire Rapaille does not designate it that way, but the third phase fits the definition of a thematic analysis.

2 Research Question This article focusses the following research question: How can a strategy combining a thematic analysis and an exploratory Mapping contribute to reflexivity in exploring solutions in a project?

3 Methodology The presented research followed a descriptive case study methodology. A descriptive case study is focused and detailed, in which propositions and questions about a phenomenon are scrutinized and articulated at the outset [5].

Mother Angel or Swiss Knife?

631

Descriptive case studies describe the natural phenomena within the data in question. Descriptive case studies may be in a narrative form [6]. This work follows a research methodology composed of a cycle where the problem/idea generates research questions that define the research and finds the research outcomes that, in turn, solve the initial problem/idea [7]. Concerning the learning method accordingly, with Vygotsky (1989), Learners use cognitive tools to understand their surroundings and the world. These cognitive tools result from one” s culture. Learning occurs within a social situation, first between the teacher and the Learner, and then is internalized inside the Learner. Learners also have a ZPD; zone of proximal development (ZPD) is a concept in educational psychology. It represents the space between what a learner can do unsupported and what the Learner cannot do even with support. It is the range where the Learner can perform, but only with support from a teacher or a peer with more knowledge or expertise (a “more knowledgeable other”). Active participation, another component of this theory, allows a framework for dialogue and an exchange of ideas, which leads to advanced development [8]. In order to analyze the textual information resulting from testimonies the authors used thematic analysis. In order to analyze the textual information resulting from testimonies, the authors used thematic analysis. Thematic analysis, as visual research, allows researchers to analyze content, generate ideas, and communicate points of view, allow the analysis of textual messages, and identify dominant themes [9]. Thematic analysis is one of the most common forms of analysis within qualitative research [10]. It emphasizes identifying, analyzing, and interpreting patterns of meaning (or “themes”) within qualitative data [11]. Thematic analysis is often related to a method or technique. Thematic analysis is used in qualitative research and focuses on examining themes or patterns of meaning within data [12]. This method can emphasize organization and detailed description of the data set and theoretically informed interpretation of meaning [11]. The thematic analysis goes beyond simply counting phrases or words in a text (as in content analysis) and explores explicit and implicit meanings within the data. Teachers can use scaffolding to assist children in completing these tasks inside their zone of proximal development to promote learning and cognitive development. Alibali [13] and Belland [14] suggest that teachers can use a variety of scaffolds to accommodate students’ different levels of knowledge. Three common scaffolds are concept and mind maps, stories and diagrams such as charts and graphs, and methods of highlighting visual information. The visual strategy, the PMVC + (Table 1 includes a description of initialisms used in this article) used in this Study joins these strands. Table 1. List of Initialisms used in this article. Initialisms

Explanation

PMVC

Persona Method, through Visual Clues

PMVC +

Persona Method, through Visual Clues and Socio-Economic Scenarios Interference

632

J. Gago et al.

4 Development The project focuses on exploring a guiding code of a solution in the form of an app/ or apps dedicated to the new central hospital of Alentejo, Portugal. In 2023 The construction of the new Hospital Central del Alentejo is underway. The project was born in 2006, and the hospital will have a capacity of five hundred and three hundred and fifty beds in single rooms, although they could be increased to four hundred and eighty-seven if necessary. It would also have 1eleven operating blocks, three for conventional activity, six ambulatories, two for an emergency, five pre-operative, and forty-three recovery posts. According to the Portuguese Minister of Health, the goal of this new hospital is to increase the response capacity of the National Health Service in the region and, thus, among other things, to avoid patients from this region having to go to hospitals in Lisbon for treatment. The Minister also recalled that this is not a hospital that will serve only the district of Évora but the entire region. The hospital, ready by the end of 2023, will have clinical and technological innovations and a new form of organization that work to attract new professionals [15]. In the first phase of this research, the authors of this research began by bringing together an audience of twenty students of different ages who, under anonymous testimony, wrote about their first experience in a hospital. The authors asked for an honest narrative that described the first experience and included the emotions felt. The authors asked for an honest narrative that described the first experience and included the emotions felt. Handwritten testimonies have the potential to present a more authentic record of experiences. Although there are distinct advantages between handwriting and typewriting, handwriting allows better expression and memorization, particularly with the most extended lists of words [16]. Handwriting descriptions are an accessible and informal way to describe lived personal situations [17]. Laptop use can distract, reducing attention during tasks requiring reflection and focus [18]. When students take notes by hand, they must place the information in their own words rather than simply typing it verbatim, as with a laptop [19, 20]. The testimonies were then transcribed in a word processor, together with the descriptions, and analyzed in a single document., Fig. 1. The text document displayed the extracted codes signaled with violet, green, and blue. The color violet marked passages refer to the presence of the child’s mother in the experience (because these first experiences mostly referred to experiences in childhood), blue refers to stressful situations, and green is associated with perceptions of space. After analyzing dominant codes in the information pattern, students highlighted the positive aspect of the information associated with the Mother. Then, the group proposed the code of “Mother Angel.” as the code that should guide the user experience offered by the app. This code should relieve the cognitive load of interaction with the “product” being a facilitator to reduce friction resulting from interactions associated with stressful situations, such as wayfinding and routing situations. For standardization of visual cues, the group defined a visual element that would represent the concept throughout the design project, Fig. 2. Discussing with the students was influential in defining the limits of the concept in identifying what could be the opposite code to “Mother Angel.”

Mother Angel or Swiss Knife?

633

Fig. 1. Extraction of codes.

Fig. 2. The visual clue representing the “Mother Angel”.

The group concluded that the opposite code should be called “Swiss Army Knife.” Because the “Swiss Army Knife” offers endless possibilities, requires system knowledge and creativity in developing solutions, and requires time and focus on system discovery. After defining the support code of the application, students developed several Personal panels using the PMVC + strategy [21]. The Persona comprises the representation or life model of demographic possibilities in various consumers. The Persona comprises the representation or life model of demographic possibilities in various consumers. The persona Method, through Visual Clues and Socio-Economic Scenarios Interference (PMVC +), as a physical visual board, puts into practice the concept of the consumer. Previous studies support the PMVC +, [22]. Teams can use physical visual boards, or “information radiators,” as a central element for collaboration, coordination, and communication [23]. These physical artifacts are easy to refer to, easy to annotate, and hard to ignore [24].

634

J. Gago et al.

A physical board makes it easier to limit the amount of information. Such boards can give teams a shared mental model of the project. A visual board can be set up quickly by placing a board in a suitable location. It is essential to be visible to the development team and others interested in the team’s work. [25]. In the learning contexts related to this Study, the students developed an exploration of a map of connections between scenarios and visual boards of personas through the Persona Method Through Visual Clues (PMVC). The visual board of Persona involves the following phases: PMVC board design ↓ PMVC + montage ↓ PMVC + discovery ↓ PMVC + code identification According to PMVC, persona panels result from interactions between ecosystems, translated visually, associated with commuting between residence location and workplace. PMVC + associates the PMVC panels with socio-economic contexts of a time frame corresponding to a month from several national Portuguese and regional newspapers from Alentejo region, Fig. 3.

Fig. 3. Flow of information between PMVC and the contexts.

It was then intended in the exploratory session to feed a reflexivity session through iterative actions composed of identifying connections and defining patterns. Figure 4. From the analysis, the group inferred that there was a pollination of the scenarios in the descriptions of the Personas, which generated tensions, allowing the group to identify and discuss opportunities and problems, Fig. 5. Concerning the specific territory of Alentejo and its cultural characteristics, patterns were detected, which led to the identification of some of the following structures present in the territory, Table 2: These structures are limiting constraints and can function as narrative models for situations associated with certain characteristics more associated with the specific territory. The most important finding was the seasonal pattern, a pattern characterized by different audiences with different levels of literacy and with different languages.

Mother Angel or Swiss Knife?

635

Fig. 4. Students connecting od PMVC´s and contexts.

Fig. 5. Overview of the PMVC + assemblage.

This analysis shows that considering the code / basic metaphor, there is the possibility of developing a group of apps focused on the following problems: • • • •

visual guides for seasonal agricultural workers wayfinding guides filtering of less severe occurrences support for outpatient treatment

636

J. Gago et al. Table 2. Structures identified in the exploratory reasoning.

Normalization of behaviors - alcohol consumption Territorial mobility - agricultural workers (foreigners with low literacy levels) - property owners/ agricultural areas (foreigners with average literacy levels) Seasonality - agricultural workers - hunting season - tourism Access to nature in rural areas - regulated

5 Conclusion Regarding the research question: How can a strategy combining a thematic analysis and an exploratory Mapping contribute to reflexivity in exploring solutions in a project? The results allow us to assess that the Visual maps are visual representations ontologically independent of the phenomenon represented and the raw data that structure them. Their elaboration, analysis, and interpretation are also a matter of reflexivity [26]. Thus, they potentiate thinking against an observed tendency to devalue disciplines associated with critical reflection [27]. Designing for a specific cultural context involves Reasoning through cultural deconstruction. Cultural deconstruction activities and methods support the development of critical and analytical capacity and ‘cultural literacy’ through the deconstruction of cultural myths and preconceptions [28]. After analyzing the PMVC +, students could enhance one or more problems with a higher level of engagement, focusing on strands in a scenario of needs of a possible app initially characterized as very complex and confusing. The PMVC + allowed students to understand the imagined world within the cultural ecosystem regarding Persona’s research. Thanks to their imagination, humans invest in the world and make sense of it in a cosmos devoid of meaning. The autonomy of the imagination characterizes the human psyche, “the ability to formulate what is not there, to perceive, in just anything, what is not there” [29]. Students understood that the individual imagination is also a cabin through which one “scrutinizes” and “distorts” other “realities,” as in the series Something-Real by photographer Romain Laurent [30]. The authors constate that there is a devaluation based on the false belief that reasoning strategies often relate to theoretical instruments characterized by a dificult practical application. The present research and many of the practices applied in foresight studies related to pattern detection for developing innovative products and services demonstrate the opposite. In future research, the authors will develop a new strategy of physical visual mapping, more adapted to dynamic didactic contexts, which could allow the visualization of

Mother Angel or Swiss Knife?

637

information for extended periods and later interactions. Because it is applied horizontally on several tables, the current strategy requires taking pictures after the exploration session and the subsequent disassembly of the map. Acknowledgements. The authors would like to acknowledge the class of the course of Interface Design II and the coordination of the degree of Design, School of Arts, University of Evora, Portugal. Author Contributions. José Gago: data collection (lead), conceptualization (lead), data curation (lead). Tiago Marques: data curation (supporting). Ema Rolo: conceptualization (supporting). Rita Almendra: conceptualization (supporting). Noemy Berbel: conceptualization (supporting).

References 1. Kneebone S, Goodwin D, Borg K, et al (2021) Chapter 7 - Researching your target audience: why do people do what they do? In: The BWA MethodPublisher: BehaviourWorks Australia 2. Rapaille C (2008) The culture code: an ingenious way to understand why people around the world live and buy as they Do, 2nd ed. Crown Business 3. Braun-LaTour KA, LaTour MS, Zinkhan GM (2007) Using childhood memories to gain insight into brand meaning. J Mark 71:45–60. https://doi.org/10.1509/jmkg.71.2.045 4. Marques AMA, Pinho ARR (2018) Consumo Infantil: A Influência dos Pares na Consciência de Marca, na Relação com a Marca e na Lealdade à Marca. Rev Gestão em Análise 7:93. https://doi.org/10.12662/2359-618xregea.v7i2.p93-106.2018 5. (2010) Descriptive Case Study. In: Encyclopedia of Case Study Research. SAGE Publications, Inc., 2455 Teller Road, Thousand Oaks California 91320 United States 6. McDonough J, McDonough S (2016) Research methods for English language teachers. Taylor & Francis Group 7. Noble I, Bestley R (2005) Visual research: an introduction to research methodologies in graphic design. AVA Publishing (UK) Ltd. 8. Shabani K (2010) Vygotsky’ s Zone of proximal development: instructional implications and teachers’ professional development. Engl Lang Teach 3:237–248 9. Lupton E, Phillips JC (2011) Graphic design thinking: beyond brainstorming. Princeton Architectural Press 10. Jnanathapaswi S (2021) Thematic analysis & coding: an overview of the qualitative paradigm, pp 98–105 11. Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3:77–101. https://doi.org/10.1191/1478088706qp063oa 12. Daly J, Kellehear A, Gliksman M (1997) The Public Health Researcher: A Methodological Guide. Oxford University Press 13. Alibali (2006) Does visual scaffolding facilitate students’ mathematics learning? Evidence from early Algebra 14. Belland BR (2016) Instructional Scaffolding in STEM Education: Strategies and Efficacy Evidence. Springer https://doi.org/10.1007/978-3-319-02565-0 15. Portugal.gov.pt (2022) Ministra da Saúde visita obras do novo hospital de Évora. In: portugal.gov.pt. https://www.portugal.gov.pt/pt/gc23/comunicacao/noticia?i=ministrada-saude-visita-obras-do-novo-hospital-de-evora. Accessed 20 Mar 2023

638

J. Gago et al.

16. Bouriga S, Olive T (2021) Is typewriting more resources-demanding than handwriting in undergraduate students? Read Writ 34:2227–2255. https://doi.org/10.1007/s11145-021-101 37-6 17. Huppertz JW, Smith R (2014) The value of patients’ handwritten comments on HCAHPS surveys. J Healthc Manag 59:31–47. https://doi.org/10.1097/00115514-201401000-00007 18. Fried CB (2008) In-class laptop use and its effects on student learning. Comput Educ 50:906– 914. https://doi.org/10.1016/j.compedu.2006.09.006 19. Crumb RM, Hildebrandt R, Sutton TM (2022) The value of handwritten notes: a failure to find state-dependent effects when using a Laptop to take notes and complete a Quiz. Teach Psychol 49:7–13. https://doi.org/10.1177/0098628320979895 20. Mueller PA, Oppenheimer DM (2014) The pen is mightier than the keyboard. Psychol Sci 25:1159–1168. https://doi.org/10.1177/0956797614524581 21. Mapping persona interaction with scenarios 22. Silva J, Raposo D, Neves J (2017) Persona diagnostic, a mosaic approach - diagnostic learning module in mobile App design. In: Research and the teaching in design and music. Edition IPCB, 1st ed. IPCB, pp 157–161 23. Sharp H, Robinson H (2010) Three ‘C’s of agile practice: collaboration, co-ordination and communication. Agile software development. Springer, Heidelberg, pp 61–85 24. Sharp H, Robinson H, Segal J, Furniss D (2006) The role of story cards and the wall in XP teams: a distributed cognition perspective. In: AGILE 2006 (AGILE’06). IEEE, pp 65–75 25. Dybå T, Dingsøyr T, Moe NB (2014) Agile project management. Software project management in a changing world. Springer, Heidelberg, pp 277–300 26. Parmentier-Cajaiba A, Cajaiba-Santana G (2020) Visual maps for process research: displaying the invisible. M@n@gement. https://doi.org/10.37725/mgmt.v23i4.4501 27. Dias MA (2020) Retrocesso no ensino das Humanidades. https://www.publico.pt/ 28. Santamaria L, Escobar-Tello C, Ross T, Bohemia E (2017) Cultural context and service design: developing critical and meaning-making capacity 29. Bouchet D (2015) The specificity of human aesthetics. Knowl Cult 3:16–41 30. Laurent R (2012) Something-Real. In: romain-laurent.com. https://romain-laurent.com/Som ething-Real

Information Design and Semiology: A Visual Study on Deconstructing Musical Notation for Improving First-Grade Children’s Learning Ines P. Antunes1(B)

, Carlos Rosa1

, and Flávio Almeida2

1 IADE—Universidade Europeia—UNIDCOM/IADE, Lisbon, Portugal

[email protected] 2 UBI—Universidade Beira Interior—UNIDCOM/IADE, Covilhã, Portugal

Abstract. The Western diagrammatic system for musical notation, which represents notes and melodies through graphic forms on a five-line structure—the pentagram—is considered a universal language and is respected and followed by both formal music education and musicians around the world. It evolved from oral heritage into an intentional figurative system by which melodies could be sung and played for centuries. However, when we look at the accessibility of formal music education for children in first grade, between five (5) to seven (7) years of age—before reading age—it is easy to find gaps. We have analysed the main symbols of formal musical notation and conducted a comparative study of their structure and the relationship of musical graphic symbols to Design Principles, Sign Types and Peircean Semiotics. We also deconstructed a sheet of music to identify this graphic language’s positive and negative features in terms of Design Principles and its understanding by children up to seven years of age. When we consider children’s learning, we must additionally take into account the complexity of what is offered to the cognitive structure acquired up to that point. It can be concluded that formal music notation is quite complex and does not meet the need for simplicity that would promote children’s learning. By acknowledging children’s development stage and matching it with Design Principles, the results show that an adapted symbolic language can help the children to better understand the meaning of the musical symbols and make their translation easier and more effective. Keywords: Graphic Design · Information Systems · Figurative Languages · Musical Literacy · Children’s Development

1 Introduction Formal musical notation rised from the need to translate oral tradition towards a symbolic notation, so this cultural knowledge wouldn’t be missed throughout time. At first, music was passed trough generations by ear, and when the need to sing in choirs and churches rose, together with the need to make sure the singing/played version was the same for all © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 639–650, 2024. https://doi.org/10.1007/978-3-031-47281-7_53

640

I. P. Antunes et al.

the interpreters; from this, the first notation marks appeared as singular interpretations named ‘neumatic notation’ or ‘neume’ and it developed towards the ‘pentagram’ (five lines/staves) we know nowadays. ‹‹The form of musical notation employed in Western musical practice emerges from a long tradition of use in liturgical contexts that relied heavily on a comprehensive knowledge of local repertoire and performance context, with notation acting largely as a memory prompt›› (Stenberg and Cross 2019), having this in consideration in clear to state formal musical notation was not created having children’s learning as a concern. As any other knowledge, the learning of this visual system needs to be adapted to the abilities and stages of development of children to be learned and understood. The “universal language of music” is the formal musical notation—which is a symbolic notation which has been used for several centuries and works unquestionably when properly learned—this research is based on questioning music accessibility towards every child, no matter the background. Learning an instrument is expensive and not possible for every child. Moreover, even those who have access to an instrument often have difficulty learning to read music. There are several reasons for this: First, reading requires a mathematical understanding that develops later in children’s lives; second, the formal musical notation is highly complex due to the amount of visual information gathered to translate simple or complex sound melodies, which makes it visually confuse and not very intuitive; third, playing an instrument is an activity that requires focus, creative and motor skills in addition to logical thinking—a great effort to understand rhythm, count times and play notes while reading graphic representations. Looking at the way formal music notation is structured, we can understand the amount of information established on five lines: from many we will focus on notes, duration, rhythm, voice and instruments due to the project specifications. The complexity of its structure comes from this characteristic and, although it is accepted in a professional way (almost considered an universal language), due to a higher level of mathematical and logical thinking—and not only an immediate visual interpretation -, it is complex for beginners and children to learn it, specially when you consider children aged up to seven (7) years old. In this article, we take a step into looking to the basic symbols of music and deconstruct them by matching it with Design Principles, Sign Types and Semiotics knowledge. Together with children development criteria and previous data on children alternative notation, we will be able to guide the reader through the relevance of this research.

2 Deconstructing Formal Musical Notation Music Formal Notation is used in several cultures and it takes several years of study to be fully understood. The complexity of this notation comes from the amount of information insert in five lines baseline: ‹‹In musical notation, the notes for each instrument in a symphony or for each voice in a chorus appear on parallel staff. (…) parallel tracks (layers in time) are a crucial means for describing simultaneous events›› (Lupton and Phillips 2015). To read music, there’s a need to interpret several information, so the music can be played as stated, therefore it involves to be able to read the notes at a reading distance and

Information Design and Semiology

641

to be able to perform—to play or to sing—what is being read at the same time; therefore, it is important to stablish what are the different symbols and information gathered in a script of a music sheet, so the terms can be learned and visually identified by the readers. Also, by conducting an analyses towards the way the musical sheet is built, we can understand the amount of information gathered in a simple example, such as the C scale (Fig. 1).

Fig. 1. C Scale—Musica Escola Betulia (2014) Retrieved from: https://musicaescolabetulia.wor dpress.com/2014/11/21/music-in-english/

a. Stave The Stave is the base for all the information; in the five (5) lines all the symbols will be inserted as a way to be correctly translated. At the beginning of the stave, the matter for the rhythm in which the melody is built, can be found. From it, the notes are named and therefore can be played. b. Clef The five lines a group a first symbol (clef/clave) which is a “guidance symbol” that indicated how high or low the following symbols should be played. c. Notes In every on each five lines and in the space between then, different symbols can be applied: this symbols will represent both notes and its duration. d. Tempo Heavier marcs (full points) will mean less duration in time of that note, that empty points; which can be visually intriguing. Also, every symbol considers multiples of 8, which, again, brings up the logical and mathematical thinking needed to understand this visual system (Fig. 2).

2.1 Piercian Semiotics Charles Sanders Peirce (1839—1914), philosopher and specialized in Semiotics, studied the way objects were interpreted and accepted by human beings. The most relevant elements considered by Pierce were the sign, the object and the interpreter. As resumed by Alves (2016), the ‘sign’, which is designated as a physical pattern or aspect, represents an ‘object’ or representation of reality, which is received by the ‘interpreter’ (perceptions from the receptor). Pierce’s Semiotics Theory offers an approach to understanding signs and the role they play in human communication. ‹‹The importance of the interpretant for Peirce is that

642

I. P. Antunes et al.

Fig. 2. Baxter and Baxter (2008). The Right Way to Read Music. Constable & Robinson, Ltd., p. 14

signification is not a simple dyadic relationship between sign and object: a sign signifies only in being interpreted›› (Atkin 2023); looking at the way formal music notation is structured, we can understand the amount of information established on five lines: notes, duration, rhythm, voice and instruments, dynamics, etc.. The complexity of its structure comes from these characteristics and, although it is accepted in a professional way (almost considered an universal language), due to a higher level of mathematical and logical thinking—and not only an immediate visual interpretation -, it is complex for beginners and children to learn it, specially when you consider children aged up to seven (7) years old. ‹‹(...) Any sign, or representation as Peirce calls it at this early stage, will have a sign-vehicle, an object, and an interpretant. An important difference here though is how he thinks of the relation between signs and interpretants›› (Atkin 2023). To have the knowledge of the audience of our research is a main requirement for the results to be considered valid. Only by understanding their way of thinking and their visual application of sound we will be able to build a proper artifact [alternative iconographic musical notation, in this case] to be developed and applied in their learning. ‹‹(...) the meaning of a sign is manifest in the interpretation that it generates in sign users.›› (Atkin 2023). And this is when deconstructing a musical sheet is important to this research. Is the way formal musical notation allowing children and beginners to read it at first sight? Is it too complex for its understanding by a child up to seven (7) years of

Information Design and Semiology

643

age, considering their stage of development: motricity, literacy, mathematical thinking, logical thinking, and hability to focus? 2.1.1 Sign Types Pierce’s semiotic theory is a foundation of sign types which are used to create meaning in communication. ‹‹(...) For Pierce, not all iconic signs are visual images, and may include acoustic, tactile, olfactory forms, conceptual forms of resemblance, or may include, with the notion of pure icon, even mental quasi-forms in a state of gestation.›› (Santaella 2005. Trans.) Pierce, defined ‘sign’ in three different types: icon, index and symbol. Icons are visual images which stand directly for objects or concepts. Indexes are physically connected to the concepts they stand for, such as smoke for fire, or a pointing finger for directions. Symbols are arbitrary and generally involve words or language which stands for things. Symbols are often based on agreements within a given culture, and are not directly linked to the concepts they represent. ‹‹(...) As Morris (1964) stated, there is one characteristic that we can consider as being common to all, which is the interpreter—that is, our availability and ability to decode a message.›› (Rosa and Morais 2022). Having this in consideration, we need to understand the way musical sheets are constructed, translate on a symbolic visual notation which can be learned but is quite complex and demanding in terms of time, information and translation. Is the availability to decode the message accessible to all, even the ones whom can not expend time and resources on formal training? ‹‹The utilisation of symbols in music for representation is an unquestioned ritual of practice for teachers and, indeed for many, equates with what music is. However, the symbol is not the music, it is merely the cue to retrieve it; the music is the sound and this needs to be kept at the forefront of the composing process›› (Henderson 2007). Therefore, if we manage to create an alternative iconographic system, with icons designed based on notations originally created by children, maybe it would be able to restore the access to musical literacy by engaging the interest and learning from both children, teachers and families. In this case, the icon stands directly for the object (note) or concept (intention/melody) facilitating children exploration of music literacy and creation; our main research follows the hypothesis that a simplified iconographic system would answer to these issues. 2.2 Shape Types The sign only works together with a meaning; it is a representation of an object. The object lives independently from the sign because the sign cannot represent the object entirely. The interpretant will give sign a purpose—it doesn’t existe without it, because the interpretant will give it the meaning. This is why this triad is so relevant in communication (visual, oral, etc.), the three components only make sense if developing together.

644

I. P. Antunes et al.

‹‹While in sound language, matrix of firstness, the emphasis fell on the syntax and, consequently, on the relationship of the sign with itself, in the visual language matrix of secondness, two faces emerge, on the one hand, the form that concerns the visual sign itself itself, to its internal characters, and, on the other hand, the representation that refers to what the form is capable of representing.›› (Santaella 2005. Trans.) Santaella (2005), defined three categories of shapes, adapting it towards the reality of Pierce’s Semiotics on ‘sign’ typology, being Non-Representative Shapes (icon), Figurative Shapes (index), Representative or Symbolic Shapes (symbol): Non-Representative Shapes are characterized by suchness, they mean what they mean. As shapes they don’t need any correspondence or reference to the external because they correspond to abstract practice. They have a power of suggestion which engage the interpretant and makes them have a meaning. Kandinsky work is one example of this shape, due to his exploratory point—line—shape abstract studies. Figurative Shapes are the images that represent real objects by mimesis—they copy an object from the real world becoming a bidimensional or tridimensional replica; they are related with perception and memory. Representative or Symbolic Shapes reproduce visual references towards abstract feelings—as in Pierce’s theory, the symbol can only be interpreted as such; in this category we can mention Xenakis work as having this characteristics. Although this characteristics are stated, it is relevant to consider each sign will have its characteristics and should not be identify without considering its full existence. This classification should work as adaptable, being a guideline for shape definition but not closed on itself: ‹‹Classification is a kind of network to be used in the elucidation of visual forms. (...) The classification should not work as a static and monovalent itemization, but as focuses of intelligibility that are capable of awakening the eye and working as compasses to guide the reading of the logical principles that command the configurations of the visual language.›› (Santaella 2005. Trans.) 2.3 Design Principles The lack of white space between information on a musical sheet, the amount of symbols needed to consider for translate music notation into sound is quite complex, even for an adult beginner. The space between lines is meant to allow the reading of several information at a visual distance (50 to 70 cm away from the sheet) but is sometimes adapted to facilitate the reading for some musicians according to their instruments or conductors by adapting the scale proportion: ‹‹the notation graphic space does not have the same properties as the basic pitch versus time real space. If in the graphic space, we can represent any pitch interval size, is just a scale question dependence›› (Sluchin and Malt 2016). If we consider notes and tempo are created from the adaptability of the point shape, we can quickly think about Kandinsky work on music sheets interpretation by only considering point and line as shapes for its structure. He stated: ‹‹The point is temporally the briefest form. In theory, the point, which is I. a complex (size and form) and 2. a sharply-defined unit (...)›› (Kandinsky 1947), and as dark and bigger the point would

Information Design and Semiology

645

be, the longer the time define for that “note” to be played should be considered on his translation (Fig. 3). Even thought this example is a reference towards design, it also represents an abstract representation of music which cannot be performed without any previous knowledge or reference.

Fig. 3. Kandinsky (1947). Point and Line to Plan. The Solomon R. Guggenheim Foundation New York, p. 45

Kandinsky (1947) also stated that the characteristics of the point should change within the character intended to it. So, as we can see in his studies on music, it was a different visual dynamic that almost allowed an interpretation on intention on “how to play”. Still looking at the musical sheet and the forms we find on it, some of the symbols tend to be “heavier” on its visual form, but indicate less time in sound than “lighter” shapes. ‹‹Since, moreover, the point is a complex unit (its size plus its shape), it is easy to imagine what a storm of sounds can develop from a continuing accumulation of points on the basic plane-even when these points are identical: and how this turmoil develops and spreads out, while its further course points to the evergrowing disparity in the size and form of the points which are projected upon the plane.›› (Kandinsky 1947) On another hand, if we take a look at the work of the composer and musician Iannis Xenakis, we can visualize how an interpretation and simplification of musical notation can structurally work; from his work ‘Metastaseis’ (Fig. 4), from 1954, we can match his architecture background with his feeling and intention towards music translation. This work ‹‹(...) was based on a geometric construction, and it did not convert meticulously

646

I. P. Antunes et al.

layered, carefully placed tones into sound, but rather a complete continuum—music conceived not as a structure, but as a body in space›› (Fastner 2019). Xenakis believed music should be felt and interpreted without the need to complementary learning; the was able to ‹‹(...) to clarify the theoretical aspects of his thinking and to implement compositional procedures taking this tripartite notion of musical structure—outsite time, temporal, inside time—into account. (Harley 2017). Decades later, his structured way of thinking music was adapted into an algorithm so it could be used on a computer. Further studies have been developed afterwords using the adding up from technology and they captured the curiosity and willing from people, from all ages and backgrounds, to create music spontaneously. Xenakis work is a great example for Figurative Shapes, as ‘cipher by analogy’—analogy between notation and sound—due to the characteristics of is notation: ‹‹In this case, the shapes maintain an analogy with what they represent, but, as they are very enigmatic shapes, these analogies are not evident. Analogies appear only when, in confrontation with the objects they represent, the forms are deciphered›› (Santaella 2005. Trans.).

Fig. 4. Xenakis, I. ‘Metastaseis’. Retrieved from: https://www.elbphilharmonie.de/en/mediat heque/iannis-xenakis-architect-of-sound/287

Gestalt Laws should also be considered on this deconstruction: ‹‹A Gestalt is an integrated, coherent structure or form, a whole that is different from the sum of the parts. Gestalts emerge spontaneously from self-organisational processes in the brain.›› (Wagemans et al. 2012). The Gestalt principals were introduced by Wertheimer (1923) and were further developed by both Köhler, Koffka and Metzger, and for musical sheet we can consider some of them, such as: proximity—elements are perceived as grouped if they’re close to each other -, we can consider this principal if we look at the way the symbols are built. Staves are separate by lighter vertical lines which represent the bars, which measures the rhythm of each “time sequence”. The symbols that appear on that separation, will belong to that bar working as a group and facilitating time measuring; continuity—alignment is key to understand units as one group -, closure—if the elements become part of a close figure, the human brain tends to assume it a group; as stated before, the symbols work on top of or between every line and, if notes are

Information Design and Semiology

647

closer in time, crochets will join in with a line, creating another figure, which represents more notes over less time; and similarity—elements tend to be considered groups if there’s a similarity between them -, in musical sheets we can consider similarity as the way symbols are constructed, with similarity features between them and how they are positioned on the sheet. It is required a long time of study and contact to formal music notation to properly learn it. If we consider children, between five (5) to seven (7) years of age, we can understand that simplify this notation could potentially turn the process of learning music and the access to musical literacy somehow easier.

3 Children Development and Visual Perception—Why It Matters? ‹‹By 5–7 years: It is known that the basic functions of early sensory areas of the cortex have completed their development; nevertheless, the functional development of brain substrates for perception of complex visual scenes takes still longer›› (Farroni and Menon 2008); from 5 to 7 years old, children should also be around the first year of first grade of school. This means, they will be in a time where they’re ready to explore their thinking, question whats around them, build a though and a discourse, and is ready to explore their mark-making. Symbolic thinking and exploration are also key development characteristics relevant to engage in our research. At this age, children are ready to consolidate their communication, name their feelings and experiences, and adjectify actions, objects and concepts; this abilities allow to perform activities which promote the understanding of children’s perception and points of view. Great part of children development happens due to their sight and earing: they see objects and hear their names frequently while growing and developing their understanding of the world. They learn by repetition and mimics of the adults around them, this is why they are so easily molded to new learnings from copying. Other great characteristic from this age is children are learning to understand signs and symbols: ‹‹When children begin to write, they are constructing a code. (...) Writing develops when children begin to use symbols. Often they begin by putting letter -type-shapes into their drawings. These gradually get pushed out to the edges of the drawing, to look more like words and sentences›› (Meggit 2015). From mark-making children start creating their own symbols and signs and exploring new ways of shaping and create visual patterns of their thoughts. ‹‹Visual perception, according to Groffman, is the central process of knowledge acquisition, in which information is located and extracted from the external environment›› (Jadue and Figueroa 2017). By acknowledging this, we can understand children at this age are aware of the world around them and have the ability to create their own interpretation of a music by hearing it, defining what their hearing using their words and adjectives, and creating new signs or symbols from mark-making. Several authors enrolled in researches to understand the way children draw music before and so having some of them finding in consideration is required for a better knowledge of this matter.

648

I. P. Antunes et al.

4 How Do Children Draw Music: Previous Studies ‹‹The evolution of music during the second half of the 20th century led to the exploration of many new ways of producing sounds but also representing them›› (Karpodini and Michailidis 2022) and from this interest, several authors developed alternative systems for music learning. As examples, we can consider Colour-coded Notation, Shape Notation, Kodaly—and is hands visual system -, Suzuki Method—which involves all children social nest in her/his learning -, etc. From this, also rose the awareness towards the way children visually interpret music and what mark-making they build up from this exploration. ‹‹When children realize that marks can be used symbolically to carry meaning, in much the same way as spoken words, they begin to use marks as tools to make their thinking visible›› (Meggit and Bruce 2015); the way children draw music, the way they explore their own notation and symbolic language is allowed also due to their freedom of thinking, without boundaries and insecurities. To be able to explore this connection, between their hability to create symbolic language, matching it with music towards the development of an alternative iconographic system to improve children access towards musical literacy is the goal of our main research. ‹‹Bamberger (1991) proposed that children’s invented musical notation progresses from figural to formal as their invented drawings mature›› (Henderson 2007), giving the opportunity to also understand the way their thinking happens: when considering figurative musical examples, we should consider mark-making that represent the way the music flows, how is the rhythm and pitch patterns, and how accurate their visual representation of this features is considered on their visual assets. This mark-making tend to be more graphic and less exploratory. ‹‹Children have an immediate interest for their own compositions, as they evolve from their own experience›› (Barrett 1997), and will conduct their signs and symbols according to what their listening and focusing on the structure of it, on the intention of the melody they listen. Barrett (2006), understood children could not only create their own notation but also they would remember it and perform it latter on; so children will have a sense of recognition of their notation, despite what adults may consider at a first thought. ‹‹Advocated music practice in early childhood settings tends to focus on group music making such as ensemble performance of nursery rhymes and songs, finger-plays, action songs, alphabet and counting songs, and associated structured movement and instrumental play that emphasizes beat and rhythm›› (Barrett 2006), however, when you give children the opportunity to explore their own notations, they naturally engage in this process leading towards their creations and open up a group of possibilities that can be analysed and restructured for common good—in this case, to elaborate an alternative iconographic system that can support their future learning and musical literacy.

5 Conclusions and Next Steps Formal music learning embraces a big investment—not only temporally but also budgetary—which opens a path towards inequality in access. Invariably, the formal music learning tends to be limited to those whom can take advantage of that access. Regardless, and knowing the value of learning music to support the development of other types

Information Design and Semiology

649

of learning, adapting its “transcription” to a simplified alternative language may be the solution for more children to have access to this knowledge. In further steps, we will conduct participatory research with this aged group to understand their musical thinking and develop an alternative pictographic system, in which children’s cognitive abilities and needs are being considered, which can complement current practices to further improve children’s access towards musical literacy.

References Atkin A (2023) Peirce’s Theory of Signs. The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/spr2023/entries/peircesemiotics/. https://wp.me/pHDzN-38G. Accessed 20 May 2023 Alves LM (2016) O signo: elementos semióticos de Peirce. Ensaios e Notas. https://wp.me/pHD zN-38G. Accessed 20 May 2023 Barrett MS (1997) Invented notations: A view of young children’s musical thinking. Res Stud Music Educ 8(1):2–14. https://doi.org/10.1177/1321103X9700800102 Barrett MS (2006) Inventing songs, inventing worlds: the ‘genesis’ of creative thought and activity in young children’s lives. Int J Early Years Educ 14(3):201–220. https://doi.org/10.1080/096 69760600879920 Baxter H, Baxter M (2008) The Right Way to Read Music. Constable & Robinson Ltd., London Farroni T, Menon E (2008) Visual perception and early brain development. In Tremblay RE, Boivin M, Peters RDeV (eds) Encyclopedia on Early Childhood Development. Centre of Excellence for Early Childhood Development and Strategic Knowledge Cluster on Early Child Development, Montreal, pp1–6. http://www.child-encyclopedia.com/documents/Farron iMenonANGxp.pdf Fastner C (2019) Elbphilharmonie Magazine (2019). https://www.elbphilharmonie.de/en/mediat heque/iannis-xenakis-architect-of-sound/287. Accessed 20 May 2023 Harley J (2017) Xenakis - His Life in Music. Routledge. https://www.routledge.com/XenakisHis-Life-in-Music/Harley/p/book/9780415885386. Accessed 20 May 2023 Henderson C (2007) Music to measure: symbolic representation in children’s composition. Waikato J Educ 13:2007 Jadue T, Figueroa LF (2017) Visual perceptual skills in school children aged 7 to 12 years with refractive amblyopia. Cienc Tecnol Salud Vis Ocul 15(2):31–7 (2017). https://doi.org/ 10.19052/sv.4145 Kandinsky W (1947) Point and Line to Plan. The Solomon R. Guggenheim Foundation, New York Karpodini C, Michailidis T (2022) Making graphical scores mores accessible: a case of study. School of Compiting and Digital Technology, Birmingham Lupton E, Phillips JC (2015) Graphic Design the New Basics. Princeton Architectural Press, New York Meggitt C, Bruce T (2015) NCFE CACHE Level 3 Child Care and Education (Early Years Educator). Hodder Education Musica Escola Betulia (2014). https://musicaescolabetulia.wordpress.com/2014/11/21/music-inenglish/ Rosa C, Morais R (2022) Classes of signs in Pictograms: a case study. Int J Vis Des 16(2):19–30. https://doi.org/10.18848/2325-1581/CGP/v16i02/19-30 Santaella A (2005) Matrizes da Linguagem e Pensamento - sonora, visual, verbal. Editora Iluminuras, Ltda Sluchin B, Malt M (2016) A CAP for graphic scores: graphic notation and performance. IRCAM

650

I. P. Antunes et al.

Stenberg A, Cross I (2019) White spaces, music notation and the facilitation of sight-reading. https://doi.org/10.1038/s41598-019-41445-1 Wagemans J, Feldman J, Gepshtein S, Kimchi R, Pomerantz JR, van der Helm PA, van Leeuwen C (2012) A century of Gestalt psychology in visual perception: II. Conceptual and theoretical foundations. Psychol Bull 138(6):1218–1252. https://doi.org/10.1037/a0029334

Amiais@SL: A Metaverse Simulator to Support Playful Engagement in Learning About Cultural Heritage Ana Carla Amaro(B)

and Lídia Oliveira

DigiMedia—Digital Media and Interaction Research Centre, University of Aveiro, Campus de Santiago, 3810-193 Aveiro, Portugal {aamaro,lidia}@ua.pt

Abstract. The cultural heritage of rural communities is at risk as their populations age and their social and economic structures deteriorate. That is the case of Amiais, a small rural Portuguese village, in which only 15 inhabitants remain, mostly older adults. Therefore, the preservation of both tangible and intangible cultural heritage of these rural regions, such as Amiais, has become an urgent concern. ICT, and particularly virtual 3D worlds or metaverses, such as Second Life® (SL), have been playing an important role in this matter, offering new possibilities for enhancing and preserving culture and allowing people to learn about and experience culture in new and exciting ways. This paper describes the design and development of a simulation of Amiais village in SL (Amiais@SL), in which visitors can engage in gamified immersive cultural experiences, learn about the village and its cultural heritage, interact with virtual agents, and participate in cultural activities through different virtual tours and challenges. Keywords: Cultural Heritage · Cultural Learning · Heritage Awareness · Metaverse · Second Life · Gamification

1 Introduction Cultural heritage is one of the greatest assets of humanity. It links the past to the present, being crucial to social identity and cohesion, helping to shape the future [1, 2]. Our cultural heritage, however, particularly in rural areas, is more fragile than sustainable [3–5]. Rural areas’ depopulation and population aging, mainly caused by young adults’ migration to cities, leads to the decline of economic and social structures [6], weakening these communities and their ability to preserve their cultural heritage. As the left-behind elder population dies, valuable knowledge about oral traditions, social practices, rituals, festivities, traditional crafting, and so many other intangible aspects of our culture are lost forever. Therefore, preserving both tangible and intangible cultural heritage is of paramount importance for rural communities and future generations.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 651–664, 2024. https://doi.org/10.1007/978-3-031-47281-7_54

652

A. C. Amaro and L. Oliveira

Information and Communication Technologies (ICT) have long been used to access, understand, preserve, disseminate, and teach about cultural heritage, being found as valuable tools to create awareness and value, and to reduce the gap between people and their inheritance, namely by eliminating time and space limitations [7–10]. Recent systematic literature reviews demonstrate that 3D digital technologies, such as 3D modeling, Virtual Reality, and Augmented Reality, are the most used technologies to preserve cultural heritage, enabling not only virtual (re-)presentations of places, monuments, and ancient artifacts, but also to record and present the way of performing activities, such as dances or traditional crafting [8, 11]. The use of computer-generated immersive 3D environments with and in which users can interact, to reconstruct ancient lost sites of high historical and cultural significance, became popular during the first decade of this century [12, 13]. These virtual 3D worlds allow visitors to browse and interact with digital 3D models of heritage buildings and objects; however, their true potential lies in the possibility they offer to reenact cultural heritage [13]. In fact, in this kind of simulators that mimic real life—or metaverses, such as Second Life® (SL)—people are embodied in virtual 3D entities called avatars, through which they can perform activities and involve themselves in social and cultural processes [14, 15]. Besides, avatars can also be virtual agents, designed and programmed to look and behave in a certain way, to interact with human avatars, acting as guides or virtual inhabitants. In this way, metaverses, and particularly SL, seem to be perfect environments for creating cultural-enriched immersive experiences, offering interactive artifact-mediated resources through which people can engage in gamified scenarios and activities, making them particularly suited for education and teaching purposes [10, 15]. This paper reports on the process of designing and developing a simulation of Amiais village in SL (Amiais@SL1 ), framed within the LOCUS2 project. Amiais is a small rural village located in Aveiro district and Couto de Esteves parish, in Portugal’s central region, with around 15 inhabitants, mainly older adults. This village preserves ancient rural traditions, as is the case of the maintenance of communal threshing floors, an old gathering point for the farming communities linked to the husking of corn and the ritual of Desfolhada, which are still done today. Also, the village has well-preserved granaries, that are part of its identity. Other traditions and rituals remain alive, such as the Midnight Mass and singing the Janeiras, the slaughter of the pig, and the traditional Rojoada after the slaughter, etc. [16, 17]. By exploring Amiais@SL, visitors can engage in gamified immersive experiences based on Amiais cultural heritage scenarios and narratives, learn about the village and its cultural heritage, interact with virtual agents, and participate in cultural activities through different virtual tours and challenges. As such, the project employs an immersive playful

1 Available at https://maps.secondlife.com/secondlife/AMIAIS%201/91/200/37. 2 LOCUS—playfuL cOnneCted rUral territorieS was financially supported by FCT Foundation

for Science and Technology (Portugal), through National Funds, and by FEDER/Compete 2020/Portugal 2020 (POCI-01–0145-FEDER-029228). This work is also financially supported by national funds through FCT—Foundation for Science and Technology, I.P., under the UIDB/05460/2020 project.

Amiais@SL: A Metaverse Simulator to Support Playful Engagement

653

approach to promote cultural learning and heritage awareness. The particular focus of this paper is the conceptual and technical design and implementation of Amiais@SL.

2 Background Despite all the hype surrounding the term Metaverse, which recently has been added to the everyday technological jargon, it is definitively not a novelty, either conceptual or technologically. The term was first used in Neal Stephenson’s dystopian cyberpunk novel “Snow Crash”, in 1992, describing the convergence of physical, augmented, and virtual reality to draw a computer-generated online universe, populated by avatars and software agents [18–20]. Those kinds of platforms have been available for more than two decades, with a large number of users worldwide [18]. Massively Multiplayer Online Role-Playing Games (MMORPGs), for instance, like Roblox and Fortnite, allow users to create avatars and experience social interaction with other gamers within a virtual world [21]. Multi-user Virtual Environments (MUVEs), such as SL, also fit into the original Metaverse definition, as immersive virtual worlds parallel to the physical world, in which users interact through digital avatars [22, 23]. Unlike MMORPGs, MUVEs are not gaming environments, since there are no built-in traditional gameplay mechanisms, goals, rules, levels, missions, tasks to complete, or timers [14, 22, 24, 25]. Instead, these virtual worlds provide more open-ended environments where users can create their own content and define their own activities and goals [22]. SL was launched to the public in 2003 by Philip Rosedale and the company he founded, Linden Lab, bringing innovative features compared to other MUVEs at the time [26]. It became the largest and most used MUVE so far (Lee et al. 2021) and, according to Rosedale, “the closest thing to a metaverse that we have…” [27]. In SL, things and people exist within a virtual territory, the grid, composed of discrete regions, known as simulators, or sims. Once logged in, users become residents and represent themselves through avatars, which can resemble real life or be something completely different, since avatars are highly customizable into anthropomorphic, zoological, or fantasy shapes [14, 25, 28, 29]. Avatars travel regions by teleporting and moving through space by walking and flying or using animals or vehicles (which are sometimes ridable), exploring the environment by clicking on things around. Landscapes, buildings, objects, and everything else in the environment are created by residents. However, the learning curve is steep, and mastering the world’s building and scripting engines is difficult and takes time [14, 26, 29]. Luckily, everything can also be bought or sold within SL, as it features a resident-based economy, with its currency, the Linden Dollar [26, 29]. Aside from creating new content and buying and selling things, SL residents can do almost anything they do in real life: dancing, listening to music, performing, watching movies, going to parties, playing games, traveling, and visiting new places… [24, 25, 28]. People can also get married, have a family, perform daily-life tasks, such as washing dishes or dust cleaning, and get a job, which can range from the expected landscaper or fashion designer to the oddity of babysitting a virtual baby [29, 30]. As such, SL is a multifaceted world [28] within which all social activities—whether political, economic, religious, scientific, cultural, or educational—can be performed [30].

654

A. C. Amaro and L. Oliveira

Considering the SL’s focus on social interaction [25], communication is an important issue on many levels. Avatars can communicate in real-time through VoIP or instant messaging, in public or private chats. Text-based notes (notecards) can also be traded, and objects or surfaces can host multimedia content, such as sounds, videos, etc. [14]. Besides, avatars can undertake several non-verbal system-provided communicative actions, such as laughing, yawning, nodding, or clapping, that can be extended by purchasing or creating additional animations and gestures [14, 15, 28]. The potential of SL to preserve cultural heritage, both tangible and intangible, and to promote its learning, have long been foreseen [10, 14, 15, 31, 32]. Bogdanovych et al. [13] describes the implementation of an SL recreation of Uruk, an ancient city from around 3000 B.C., intended to enhance the educational process of history students by immersing them into the city and letting them experience how it looked and how citizens behaved in the past. The reenactment of the local culture and daily lives of the ancient Sumerians was performed by four virtual agents representing members of two fishing families from ancient Mesopotamia, programmed to move around and interact with the environment and the visitors. Similarly, the Project “City and Spectacle: A Vision of pre-earthquake Lisbon” [33] aimed to recreate the city of Lisbon before the destructive 1755 earthquake, taking into consideration its urban, architectural, social, and cultural aspects. In addition to providing the scientific community with new ways of producing knowledge about the pre-earthquake Lisbon and sharing it in an interactive and immersive virtual world in real-time, the project also intended to explore in-world ways of disseminating cultural information to the public, contributing to the preservation, promotion, and learning of Portugal’s cultural heritage. Howell and Chilcott [34] report the development of a digital interpretation of the medieval city of Newport in SL, allowing virtual access to the ruined castle in the center of the city, inaccessible to the public in real life for safety reasons. The castle houses a virtual immersive museum featuring, among other things, Newport’s medieval ships, revolutionary in terms of shipbuilding. The simulation also offers visitors the possibility to sail in one of these fifteenth-century ships and to learn about them and the development of medieval Newport. As stressed by Ibrahim and Ali [35], cultural learning immersive environments should provoke users to think, discover, and make meaningful connections with the heritage (arti)facts, instead of merely recalling them. As such, designing this kind of environment must focus on communicating cultural significance to the public, namely by including embodied interaction, supporting the contribution and sharing of heritage interpretation by users, and promoting social activities that evoke heritage awareness [36]. The highly immersive and engaging environment of SL has proved to be effective for designing these informal learning experiences, leading to historical and cultural awareness [35, 37]. In fact, by enabling higher levels of embodiment, (non-)verbal communication, and avatar realism, and by integrating playful interaction mechanisms that allow for the use of gamification, this type of virtual environment may increase engagement and motivation, benefit collaboration, and promote learning [38].

Amiais@SL: A Metaverse Simulator to Support Playful Engagement

655

3 Methods The project was structured into five main phases: Ethnographic Research; Reflection and Creativity; Development; Tests; and Envisioning: Recommendations and Models. During Ethnographic Research, researchers contacted the main project’s stakeholders (the Mayor, the President of the Parish Council, and the Presidents of two local Cultural Associations of Amiais), seeking not only partnerships but also more formal information about the socioeconomic and cultural contexts of Amiais Village. The President of the Parish Council played a key role in the process of bringing the team closer to inhabitants and earning their trust, as the main goal of this phase was to gain a deep knowledge about Amiais people and culture. The contacts with the inhabitants progressed from informal conversations in the streets during researchers’ visits to the village, to more formal and structured activities, assuming the form of workshops with defined objectives and strategies, to which people were formally invited. During these workshops, through paper-based activities and by enacting situations and playful activities, older adults taught the project team about the multiplicity of aspects of their cultural heritage. Also, during this phase, the team documented the village through photographic and video recordings. In the Reflection and Creativity phase, all the information collected during the previous ethnographic research informed the design of the Amiais’ simulation in SL (Amiais@SL), including the gamified immersive experience, the narrative and playful interaction scenarios it comprises, and the several layers of information included. The third phase involved the development of Amiais@SL, starting by delimiting the area to be recreated and setting up the virtual territory, modeling the terrain, recreating the built environment, and creating content. The playful interaction narrative and scenarios were then settled and programmed. Once a reasonably stable version of the village simulator was achieved, the project moved into the testing phase. For immersion, interaction, and learning assessment regarding the narrative, scenarios, virtual agents, and cultural heritage aspects, a heuristic-based assessment model was developed, and users were observed while exploring Amiais@SL. Additionally, the users’ perception regarding the convergence of the physical territory and the simulation, and heritage awareness and learning achieved were tested through a 21-question questionnaire. The fifth and final phase of the project aimed to design a model that will allow the replication of the LOCUS methodological approach by other projects intending to use digital platforms for preserving cultural heritage and raising cultural awareness. Additionally, a set of guidelines for designing and developing virtual immersive playful experiences aimed at promoting the cultural heritage of rural territories are under development, along with convergence theoretical models between the virtual immersive playful experiences and hypothetical experiences in the physical territory, which in a future beyond the project and unrelated to LOCUS’ objectives, could be implemented using different technologies, including IoT. This paper focuses on the work resulting from phases 2 and 3 of the project, describing the design and implementation of Amiais@SL.

656

A. C. Amaro and L. Oliveira

4 Amiais@SL 4.1 Delimiting the Area to Be Recreated and Setting up the Virtual Territory The virtual territory of SL consists of discrete simulators (sims), configured as square parcels with a 256-m side, which can have different content storing capacities depending on the monthly fee. In the case of Amiais, the virtual territory needed to encompass several key village points, namely: the village center; to the west of it, the S. Francisco chapel grounds and the Gresso river; and to the north, the village entrance, and the village gathering place. As such, Google Earth® was used to precisely locate these key points in the village, which allowed delimiting a map consisting of a square grid of 4 sims, thus resulting in a virtual territory with a 512-m side. As seen in Fig. 1, the S. Francisco chapel (circled with a blue dashed line) was slightly moved to the west, to fit within the square grid.

Fig. 1. Amiais village distributed by a 4-region grid

At this point, it was also decided how to take recourse of the different simulators’ storing capacities: Amiais SIM 1, containing the core of the village center, is a so-called augmented region, with a capacity of 22,500 objects. SIMs 2, 3, and 4, containing mainly agricultural fields and forests, are homestead regions, with a capacity of 5,000 objects each. Additionally, the 4 SIMs were edited to modify the virtual ground mesh, replicating Amiais altimetry and the level and appearance of the water, applying ground textures, and configuring the region’s ambient light, sky settings, and day cycle. The topographic information was captured from Google Earth® and loaded into the AliceVision Meshroom® app to get a first rough, dimensionless mesh of the terrain, which was later loaded into McNeel Rhinoceros® (Rhino) to be dimensioned, by superposing and scaling it to the Google Earth® map. Also, a series of level lines were extracted from Rhino and brought into Adobe Photoshop® to create a.raw3 file that was loaded into SL to configure a draft version of the terrain. The draft-configured terrain was then refined using the 3 A format defined by the SL developers for the purpose of representing 3D terrain on a 2D file.

Amiais@SL: A Metaverse Simulator to Support Playful Engagement

657

built-in world tools to smooth the terrain mesh in a realistic way, correct artifacts, and so on. 4.2 Recreating the Built Environment The recreation of the built environment began with the setting of the roads. Again, the Google Earth® measurements were used, along with an in-world tool, to set the road grid using SL’s geometric primitives (prims). After placing the roads, the prims were then assigned textures that matched the appearance of the roads. The videos and photos of the village collected in the previous phases of the project, as well as the Google Street View visualizations, were instrumental in observing the detail of the environment, such as vegetation, the presence and appearance of the roads themselves, and other elements such as dry-stone walls, road posts, and granaries (Fig. 2). It was even possible to extract the appearance of elements and transform it into a texture for the SL in-world model.

Fig. 2. Comparing a real-life photo of a granary on Amiais’ threshing floor with the equivalent 3D model in SL

4.3 Creating Content Besides the built environment—such as the village houses, the chapel, the granaries, the watermills, and elements that help define the landscape such as roads, dry stone walls, and others -, the additional content can be divided into 3 main categories: 1. Vegetation elements, such as trees, undergrowth, agricultural plants, and other landscape elements, such as the river, grapevine posts, etc. 2. Small-scale objects present around the village, that signal the human presence in the territory and their uses, such as the Alminhas4 , vehicles, agricultural implements, and others. Some of these objects are used to support the interactive component of the project. 3. People and animals. 4 Small shrines with religious motifs that are scattered across the territory.

658

A. C. Amaro and L. Oliveira

Most content in all categories was sourced from the SL Marketplace and then adapted to the project needs. SL has a vibrant community of creators, and the Marketplace is a vast resource of high-quality 3D content. Some contents, however, have unique characteristics that require them to be individually modeled to be replicas of the ones in the physical territory, as in the case of the Alminhas. For the creation of textures, which are a crucial component of content development, a photo-sourced strategy was mainly used, allowing the integration of real-life territory appearances in the virtual recreations. The placement of the vegetation elements was based on a list of the flora from the region, compiled from various documentary sources [39, 40], as well as information gathered from the inhabitants. However, it was not always possible to find models of the exact same species in the SL Marketplace, which have been replaced by models of trees of the same genus that are close in appearance. Additionally, close attention has been paid to significant vegetation elements, such as the tree shadowing the bus seat in Amiais’ favorite gathering place (Fig. 3).

Fig. 3. Bench, tree, and people at Amiais’ gathering place: Real Life vs. Second Life

The density of the forests around the village (Fig. 4) was replicated by creating layers of low-prim vegetation, that can present many copies of the same tree image, with variation in size. By placing several of these objects with different tree images close

Fig. 4. Overview of Amiais@SL

Amiais@SL: A Metaverse Simulator to Support Playful Engagement

659

together, and mixing in 3D model trees and undergrowth, a dense forest occupation was then created. Finally, several good quality mesh models of people (like the ones visible in Fig. 3) and animals were also acquired on the Marketplace, to be integrated into the environment, providing users, when clicked, with information about various aspects of Amiais’ cultural heritage, as described below. 4.4 Building the Informative Layer of Amiais@SL A layer of information about Amiais’ cultural heritage is embedded in the simulator, assuming different formats and being triggered by different visitors’ interactions. As such, and besides information panels and signage like the real-life ones, visitors can also find: • Interactive screens (flat objects surfaces, in fact) displaying image slideshows and videos of Amiais’ important cultural and religious events, like the Desfolhada festival, the Serenata ao Luar, or the festivities in honor of São Francisco de Assis and Nossa Senhora do Amparo. • Mesh models of people and animals (cats, cows, chickens…), which are placed around the village and play a walk-on role, providing information to visitors, when clicked, through online chat (Fig. 5) or by transferring a notecard with text and images. • Presence and motion-triggered information, which is displayed when an avatar arrives at a specific location, like the simulator landing point, or crosses a certain threshold.

Fig. 5. An example of a message received through online chat when an animal is clicked by a visitor

660

A. C. Amaro and L. Oliveira

4.5 Developing the Immersive Gamified Experience The immersive gamified experience is conceived as a puzzler game, in which puzzles are built into a narrative and settings in which the visitor is invited to help the inhabitants of Amiais organize the annual Desfolhada event. The narrative explores the affective, religious, and playful dimensions of Amiais’ cultural heritage, and is structured into 14 scenes spread around the village, requiring the interaction with virtual agents (non-human avatars also known as bots) and objects placed in the environment to get information and clues for complete certain tasks and solve riddles. In one of the narrative scenes, for example, the visitor encounters Emilia near the plantations (Fig. 6). She asks for help to collect the corn for the event, and the grapes and blueberries for the drinks to be sold at the party stalls. To fulfill this request, the visitor must pick ten corn ears, five grape bunches, and five blueberries. Emilia also asks the visitor to milk her cow and take it to Ana at the community threshing floor. Briosa, the cow, is in a nearby pasture ready to be milked. Once the milking is over, the visitor must click on the tin jug to deliver the milk to Ana, thus completing all tasks associated with this scene of the narrative. All the virtual agents (Fig. 7) were inspired by the inhabitants of Amiais and by characters from the traditional folklore, such as the werewolf (Fig. 8). The clues they provide encourage the visitors to explore, interact, and learn about the material and immaterial cultural heritage of Amiais village.

Fig. 6. One of the narrative scenes of the immersive gamified experience at Amiais@SL

Amiais@SL: A Metaverse Simulator to Support Playful Engagement

661

Fig. 7. Virtual agents of Amiais@SL gamified experience

Fig. 8. The appearance of the werewolf as the visitor approaches the chapel at Amiais@SL

662

A. C. Amaro and L. Oliveira

The game engine is written in Go5 and game rules are coded in Lua6 , running embedded in the game engine. The virtual agents’ state and behaviors are controlled by a Linux-running C#/.NET software—a tool developed on top of the LibreMetaverse.NET library, a free and open-source toolkit to communicate with the Second Life Grid -, with a RESTful Application Programming Interface (API) for the communication between the game engine, the virtual agents/bots, and objects inside SL. Inside SL, the interaction with and between objects and avatars is programmed using Linden Scripting Language, an event-oriented programming language used in-world. Scripts lay inside objects, detecting states and triggering events, such as communicating with the out-world applications or performing one or more behaviors. The appearance of the werewolf, for example, is triggered by a script inside an invisible object, coded to detect the presence of a visitor’ avatar. Through the RESTful API, the script communicates this presence to the out-world software controlling the state and behavior of the virtual agents, and the werewolf avatar goes online, appearing and interacting with the visitor. After fulfilling its role, the werewolf goes offline until the object detects the presence of a human avatar again.

5 Conclusions This paper reports on the design and development of Amiais@SL, a virtual 3D simulation of the Amiais village in Second Life®, aiming to promote cultural learning and heritage awareness by providing gamified immersive experiences to visitors. The methods and processes for recreating the territory, the built environment, the fauna and flora, the human presence, and all the content it comprises, are detailed. The way ethnographic research was crucial to informing the design of the narrative and playful interaction scenarios for the gamified immersive experience is also stressed, along with the description of its technical implementation. As previously stated, several authors have already acknowledged the effectiveness of the SL’s immersive and interactive environment for increasing engagement, motivation, and collaboration, benefitting learning. Results from the testing phase of the LOCUS project showed that users engaged with the narrative, interacted with the virtual agents and elements in the environment, and got curious about knowing the real territory after the virtual experience. The results also confirm that using a metaverse with playful narratives and interaction encourages cultural heritage learning and awareness7 .

References 1. UNESCO (2017) Protecting Our Heritage and Fostering Creativity. http://en.unesco.org/the mes/protecting-our-heritage-and-fostering-creativity. Accessed 15 Apr 2017 2. Turunen J (2020) Introduction: using our pasts, defining our futures – debating heritage and culture in Europe. Int J Herit Stud 26(10):975–978 5 An open-source programming language developed and supported by Google. 6 A scripting language developed and maintained by the Department of Computer Science of the

Pontifical Catholic University of Rio de Janeiro in Brazil. 7 For more in-depth information about the tests and results, see [41].

Amiais@SL: A Metaverse Simulator to Support Playful Engagement

663

3. Eppich R (ed) (2014) Cultural Heritage , Landscape & Rural Development: Good Practice, Methodology, Policy Recommendations & Guidelines for Rural Communities. INTERREG IVC, the European Union and the HISTCAPE project 4. EESC - European Economic and Social Committee (2019) Protecting and promoting our fragile and precious rural cultural heritage. https://www.eesc.europa.eu/en/news-media/news/ protecting-and-promoting-our-fragile-and-precious-rural-cultural-heritage 5. Sardaro R, La Sala P, De Pascale G, Faccilongo N (2021) The conservation of cultural heritage in rural areas: Stakeholder preferences regarding historical rural buildings in Apulia, southern Italy. Land Use Policy 109:105662 6. Li Y, Westlund H, Liu Y (2019) Why some rural areas decline while some others not: an overview of rural evolution in the world. J Rural Stud 68:135–143 7. Rueda-Esteban NR (2019) Technology as a tool to rebuild heritage sites: the second life of the Abbey of Cluny. J Herit Tour 14(2):101–116 8. Mendoza MAD, De La Hoz Franco E, Gómez JEG (2023) Technologies for the preservation of cultural heritage—a systematic review of the literature. Sustainability 15(2):1059 9. Huizinga J (1950) Homo Ludens: A Study of the Play-Element in Culture. ROUTLEDGE & KEGAN PAUL, London/Boston/Henley 10. Di Giulio R, Boeri A, Longo D, Gianfrate V, Boulanger SOM, Mariotti C (2021) ICTs for accessing, understanding and safeguarding cultural heritage: the experience of INCEPTION and ROCK H2020 projects. Int J Archit Herit 15(6):825–843 11. Skublewska-Paszkowska M, Milosz M, Powroznik P, Lukasik E (2022) 3D technologies for intangible cultural heritage preservation—literature review for selected databases. Herit Sci 10(3):1–24 12. ‘asher’ Rospigliosi P (2022) Metaverse or Simulacra? Roblox, Minecraft, Meta and the turn to virtual reality for education, socialisation and work. Interact Learn Environ 30(1): 1–3 13. Bogdanovych A, Rodriguez-Aguilar JA, Simoff S, Cohen A (2010) Authentic interactive reenactment of cultural heritage with 3D virtual worlds and artificial intelligence. Appl Artif Intell 24(6):617–647 14. Berger M, Jucker AH, Locher MA (2016) Interaction and space in the virtual world of Second Life. J Pragmat 101:83–100 15. Liang MY (2021) Pragmatic socialization through gameplay directives: multimodal conversation analysis of avatar-embodied interactions. J Pragmat 171:36–48 16. AP (2013) Aldeias de Portugal (AP). Amiais. http://www.aldeiasportugal.pt/sobre/63/. Accessed 01 Apr 2017 17. Pinto AC (2011) Pensar os territórios rurais: paisagem, planeamento e desenvolvimento em Sever do Vouga 18. Dwivedi YK et al (2022) Metaverse beyond the hype: multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. Int J Inf Manag 66(July):102542 19. Newton C (2021) Mark in the metaverse. Facebook’s CEO on why the social network is becoming ‘a metaverse company,’ The Verge 20. Novak K (2022) Introducing the metaverse, again!—enhanced reader.pdf. TechTrends 66:737–739 21. Damar M (2022) Metaverse shape of your life for future: a bibliometric snapshot. J Metaverse 1(1):1–8 22. Dionisio JDN, Burns WG, Gilbert R (2013) 3D virtual worlds and the metaverse: current status and future possibilities. ACM Comput Surv 45(3):1–38 2013 23. Lee L-H et al (2021) All one needs to know about metaverse: a complete survey on technological singularity, virtual ecosystem, and research agenda 14(8):1–66

664

A. C. Amaro and L. Oliveira

24. Quintín E, Sanz C, Zangara A (2016) The impact of role-playing games through second life on the oral practice of linguistic and discursive sub-competences in English. In Proceedings of the 2016 International Conference on Collaboration Technologies and Systems, CTS 2016, pp 148–155 25. Villar T (2022) What Is Second Life? A Brief History of the Metaverse. MUO - Make Use Of 26. DuQuette J-P (2023) Firefly Geisha & Cypris Butterflies: Lessons for Metaverses, pp 1–6 27. Gent E (2022) Lessons from a second life. IEEE Spectr 59(1):19 28. Koutra C, Papalakis S, Sanchez ED (2014) Is ‘Second Life’ taking over ‘Real Life’? Sociability and social interaction within the ‘Second Life’ environment: an electronic ethnographic study. Comput Inf Syst 18(2):2–17 29. Schneider J (2020) Taking a second look at second life. QRCA VIEWS 30. Robbins S, Bell M (2010) Second life for Dummies. Wiley Publishing, Inc., Indiana 31. Choi W, Baker D, Takahashi S, Hachimura K (2010) Implementation of Japanese intangible cultural heritage Noh play in second life. J. IIEEJ 39(1) 32. Ferguson R, Harrison R, Weinbren D (2010) Heritage and the recent and contemporary past. In: Benton T (ed) Understanding Heritage and Memory. Manchester University Press, Manchester, pp 277–315 33. Murteira H, Sequeira L, Rodrigues PS, Da Câmara AG (2015) Virtual lost cities: the case of pre-earthquake Lisbon. In: 2015 Digital Heritage International Congress, Digital Heritage 2015, pp 519–524 34. Howell R, Chilcott M (2013) A sense of place: re-purposing and impacting historical research evidence through digital heritage and interpretation practice. Int J Intang Herit 8:165–177 35. Ibrahim N, Ali NM (2018) A conceptual framework for designing virtual heritage environment for cultural learning. J Comput Cult Herit 11(2):1–27 36. Rahaman H (2018) Digital heritage interpretation: a conceptual framework. Digit Creat 29(2– 3):208–234 37. Theodoropoulos A, Antoniou A (2022) VR games in cultural heritage: a systematic review of the emerging fields of virtual reality and culture games. Appl Sci 12(17):8476 38. De Back TT, Tinga AM, Louwerse MM (2021) Learning in immersed collaborative virtual environments: design and implementation. Interact Learn Environ, pp 1–19 39. de Almeida JD (2009) Flora e vegetação das Serras Beira-Duriense. Universidade de Coimbra 40. de Botânica SP (2010) flora.on. https://flora-on.pt/sobre.html 41. Gonçalves L, Martins D, Oliveira L, Amaro AC (2022) Amiais in Second LifeTM : the use of metaverse environments to disseminate Cultural Heritage. Observatorio 16(3):101–119

The Importance of Regional Newspapers in Portugal Maria João Nunes1 and Elisabete Rolo2(B) 1 Lisbon School of Architecture, Universidade de Lisboa, Lisbon, Portugal 2 CIAUD, Research Centre for Architecture, Urbanism and Design,

Lisbon School of Architecture, Universidade de Lisboa, Lisbon, Portugal [email protected]

Abstract. We live in a time marked by significant changes brought about by the development of the Internet and digital publications, and, in this context, the newspaper is a medium that has undergone profound transformations. In a context marked by the speed of production and dissemination of news content, in which “news no longer sells newspapers”, we seek, through bibliographic research, to understand the evolution of the commercialisation of the Portuguese press – using statistical studies on the number of newspapers in circulation –, understand the differences and synergies between printed and digital newspapers, and lead a reflection on the role of the newspaper, in particular, the regional newspaper. Portugal is strongly marked by migratory dynamics from the interior to abroad and the country’s coast, which have determined the demographic depression of a large part of the territory. It is, above all, in these regions, where substantial slices of their communities depart, that regional newspapers assume a decisive role in the preservation of ties between the resident and migrant communities and in the perpetuation of their identity culture beyond the primary function of informing in an unbiased way. Keywords: Newspaper Editorial Design · Regional Newspaper · Printed Newspaper · Digital Newspaper

1 The Littoralization Process in Portugal Portugal has a long history regarding migratory dynamics. Between the 1960s and the 1980s, the desertification of rural and interior areas began (Rosado 2015). To escape hunger and misery, people migrated to more industrialised regions of Portugal, such as the peripheries of Lisbon, Porto, and Barreiro. However, much of the emigration occurred abroad, mainly to France, Germany, and Luxembourg. Portugal’s integration into the European Union momentarily changed this reality. “The economic development (…) contributed to the rise in consumption and welfare standards of families.” (Rodrigues and Moreira 2011, p. 30 apud Rodrigues and Ferreira 2014, p. 144) and caused an increase in immigration rates from a country that tended to emigrate (Quintino 2018, pp. 15 and 16), which was reinforced by the establishment © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 665–676, 2024. https://doi.org/10.1007/978-3-031-47281-7_55

666

M. J. Nunes and E. Rolo

of free movement between European Union countries in 1992 (Ramos 2012, p. 67). However, the trend reversed again with the economic crisis established in 2007 and 2008, which escalated with the request for financial rescue in 2011, also causing a decline in immigration rates (Quintino 2018, p. 16). The country’s migratory movements bring another problematic issue based on the population concentration in large urban areas, such as Lisbon and Porto. Through statistical data, in 2011, it was observed that the number of immigrants coming from another country is higher in the North region and the Lisbon Metropolitan Area. Furthermore, regarding internal migration, the highest number of immigrants from another municipality is also found in the North and Lisbon Metropolitan Area regions (Pordata 2015). Reaffirming this problem is the fact that the country’s littoralisation rates are in a growing movement, being that, in 2021, about 50% of Portugal’s population was resident in only 31 municipalities, concentrated in the central poles of the country – Lisbon and Porto (INE 2021, p. 3e). This discrepancy between the urban centres and the rest of the territory causes great contrasts in the supply of services, infrastructures, and accessibility in rural areas. It contributes to an increase in the ageing index and a decrease in population density. In the regions that are losing population, the natural balance is also negative, i.e. the number of deaths is greater than the number of births, which drives the double ageing of the population and consequently causes an average of 205 elderly for every 100 young people in 2021 (Pordata [s.d.]). Problems such as low economic activity, high unemployment rate, lack of health infrastructure, restaurants, housing, and public mobility, among others, are the cause of outflows from these rural areas, leading to a “vicious cycle” (República Portuguesa 2017b, p. 6). To address these issues, in 2017, the Government published the National Programme for Territorial Cohesion, intending to define measures to promote “an active participation and a committed involvement of local authorities, inter-municipal communities, associations, companies and people in the construction of a more cohesive, more competitive and more sustainable interior” (República Portuguesa 2017a, p. 4), stating that one of the reasons for these disparities is due to the implementation of analogous policies throughout the Portuguese territory, “treating in an equal way what is different.” (República Portuguesa 2017a, p. 5). Notwithstanding all this conjuncture, migration dynamics have revealed moments of divergence in recent years. Despite the increased desertification of the interior and the littoralization of the country, there has been an exponential rise in foreign residents between 2016 and 2021 (Pordata [s.d.]). The conjuncture of the last three years, marked by Covid-19, also brought possible changes in these dynamics since the pandemic highlighted the possibility of remote work, contributing to the exodus from cities to the countryside and the coming of digital nomads.

2 Notes on the Commercialisation of the Portuguese Press The popularity of the printed newspaper – one of the oldest and most widely used media in the world – has been declining due to the rise of new online media, such as websites, apps and social networks, and competition from traditional electronic media, such as

The Importance of Regional Newspapers in Portugal

667

radio and television, all of which operate based on immediate news, thanks to their speed of information transmission. Proving the worrying decline in interest in print media, a study conducted by the Calouste Gulbenkian Foundation in 2019 reveals that 90% of Portuguese respondents watched television on a daily basis and that 61% did not read any books in a year (Fundação Calouste Gulbenkian 2022). Still, a 2014 study conducted by ERC (Entidade Reguladora para a Comunicação Social – Media Regulatory Authority) states that there is still a majority of readers interested in traditional media, including television, radio and newspapers, reaffirming that printed newspapers remain the second most important source of news information, after television news programmes (ERC 2014, pp. 7, 28). The period of economic growth that followed Portugal’s accession to the EEC (European Economic Community) in 1986 was also reflected in the newspaper industry, resulting in an increase in the number of newspapers and sales volume between 1992 and 2004 (Figs. 1 and 2). In 2004, there were about 803 newspapers, the highest total number since 1992. However, this number has been decreasing; by 2020, there were 350 newspapers – the lowest number since 1992 (Fig. 1). This decrease in the number of newspapers on the market also reflects in their circulation, with the peak in sales occurring in 1999 with around 400 million newspapers and the peak in free newspaper distribution occurring in 2008 with a similar figure (Fig. 2). However, in 2020, these figures are expressively different, with about 86 million copies sold and about 17 million copies distributed for free (Fig. 2). 900

809

800

696 641

700 600

496

468

500 350

465

400 300 200 100 0 1992

1996

2000

2004

2008

2012

2016

2020

Fig. 1. Annual total of newspapers in Portugal between 1992 and 2020. Source Adapted from INE 2021a and Pordata 2022a

Through this analysis, we can observe that there has been an uninterrupted decline in the printed newspaper industry and in the number of readers, which has resulted in a decrease in advertising investment, one of the primary sources of income for news publications. According to data from Marktest (Obercom 2019, pp. 11 and 12), this decrease in investment in the press continued between 2008 and 2017, from e835 million to e542 million, with television remaining the medium with the largest advertising market. On the other hand, the Internet was another medium that behaved upward, having, between 2012 and 2017, registered exponential growth in advertising investments with e746 million.

668

M. J. Nunes and E. Rolo

700,000,000

624,340,827

600,000,000 463,987,383

500,000,000 353,284,030

400,000,000

356,057,405

300,000,000 200,000,000

273,076,090 210,606,098

102,524,592

157,842,933

100,000,000

85,831,060 16,693,532

81,984,858

0 1994

1998

Total

2002

2006

2010

Exemplares vendidos

2014

2018

Exemplares distribuídos gratuitamente

Fig. 2. Total annual newspaper circulation, copies sold and free copies distributed in Portugal between 1994 and 2020. Source Adapted from Pordata 2022b

Moving on to a detailed analysis of the regions of Portugal, the Algarve, Alentejo, and the archipelagos stand out as the regions of Portugal with the lowest number of newspapers (between 2014 and 2020). The Centre region had the most significant decrease in the number of newspapers (Table 1). However, the Lisbon Metropolitan Area registered the highest decrease in circulation among all regions (Table 2). The Alentejo region maintains the values of sales and circulation practically unchanged between 2014 and 2020 (despite the reduced number of newspapers), demonstrating that readers maintain their loyalty to this news object (Table 3). All this data collection and analysis is relevant to demonstrate that, despite the decline the newspaper industry has observed, the regional press has maintained balanced sales and circulation figures. In this context, the printed newspaper has a greater chance of survival. On the other hand, to cope with the transfer of advertising investments to electronic and digital media and meet public demand, traditional newspapers have tried to adapt, producing complementary digital media, such as platforms and social networks (Fig. 3). Table 1. Annual total newspapers in the regions of Portugal between 2014 and 2020. Source Adapted from INE 2021b Regions of Portugal

Total number of Newspapers 2014

2015

2016

2017

2018

2019

2020

Portugal

493

North

153

458

443

411

403

394

350

140

140

126

125

114

Center

111

175

165

153

139

142

132

124

Lisbon Metropolitan Area

89

83

85

78

75

64

60

Alentejo

33

31

30

33

29

24

25

Algarve

18

16

11

10

9

8

9

Autonomous Region of the Açores

19

19

20

19

17

16

16

Autonomous Region of Madeira

6

4

4

6

6

6

5

The Importance of Regional Newspapers in Portugal

669

Table 2. Total annual newspaper circulation in the regions of Portugal between 2014 and 2020. Source Adapted from INE 2021c Regions of Portugal

Newspaper Circulation 2014

2015

2016

2017

2018

2019

2020

Portugal

339 579 722

308 041 106

241 328 548

167 225 119

152 042 889

131 241 302

102 524 592

North

52 871 321

48 868 465

39 713 123

37 327 807

38 249 969

29 942 800

24 907 731

Center

18 467 276

17 074 254

16 640 827

16 200 053

15 276 803

14 618 692

12 280 653

Lisbon Metropolitan Area

248 890 548

224 430 513

171 026 757

99 813 539

85 704 180

75 035 432

54 205 685

Alentejo

3 302 451

3 653 653 3 005 386 3 440 501 2 597 370 2 404 987 2 262 482

Algarve

879 270

921 918

Autonomous Region of the Açores

5 673 287

5 380 761 5 361 393 5 067 294 4 950 451 4 698 589 4 349 876

Autonomous Region of Madeira

9 495 569

7 711 542 4 783 024 4 731 329 4 463 364 3 878 301 3 792 025

798 038

644 596

800 752

662 501

726 140

Table 3. Annual total number of copies of newspapers sold in the regions of Portugal between 2014 and 2020. Source Adapted from INE 2021d Regions of Portugal

Number of copies of newspapers sold 2014

2015

2016

2017

2018

2019

2020

Portugal

171 048 049

158 013 094

140 893 661

130 619 358

119 897 396

105 351 286

85 831 060

North

45 947 502

42 773 805

34 075 156

32 024 866

32 346 448

105 351 286

22 007 437

Center

13 593 440

12 538 023

12 611 594

11 886 677

11 688 210

11 056 654

9 970 120

Lisbon Metropolitan Area

93 907 119

86 519 529

81 892 774

73 860 178

63 945 735

58 201 184

43 365 652

Alentejo

2 914 158

3 291 447

2 669 589

3 083 012

2 329 493

2 110 142

2 023 752

Algarve

591 252

502 620

359 790

322 029

415 218

493 505

491 910

Autonomous Region of the Açores

5 119 591

4 929 224

4 867 911

4 852 984

4 804 836

4 573 622

4 221 689

Autonomous Region of Madeira

8 974 987

7 458 446

4 416 847

4 589 612

4 367 456

3 702 882

3 750 500

670

M. J. Nunes and E. Rolo

900 812 800

732

700 700 600

496

510

500 400 300 200

350

335 186

173

150

103

100

177

36

9 0 2004

2006

2008 Total

2010

2012 Papel

2014

2016 Digital

2018

2020

Papel e Digital

Fig. 3. Annual number of newspapers by dissemination medium in Portugal between 2004 and 2020. Source Adapted from PORDATA 2021

3 Print Versus Digital Media As described in the previous topic, the sales and circulation of printed newspapers have progressively decreased while the total number of digital newspapers has been increasing. Newspapers have become much more than printed publications; they have become a diverse set of platforms that complement each other. Nevertheless, the debate around print and digital media still exists. According to Nora (2011, p. 298), one of the reasons that contributed to the decline in print newspaper circulation was “the growth of popular newspaper markets and those for free distribution, the strengthening of community media and the development of new ways of seeking and exchanging information”. The variety of audiovisual media in online and electronic media, the real-time engagement of readers with information and the up-to-the-minute updating by journalists are some of the advantages of the online newspaper that we can see and that the printed newspaper does not offer (Jeronimo 2011, p. 475). Furthermore, due to its immediacy and the accessibility of information (Saragih and Harahap 2020, p. 546), the digital newspaper offers advantages over the physical version. Moreover, on online platforms, there is greater ease of filtering information, being one of the benefits also mentioned by Nora (2011, p. 299). However, a printed newspaper can be equally filtered through the choice of a thematic newspaper or, internally, through sections and notebooks that highlight the topics covered. Regarding the credibility of information, both print and digital media may or may not include high-quality material, so it is crucial to know how to recognise credible sources of information. However, printed newspapers are still considered a reference source by their readers (Tuzzo 2014, p. 292), unlike social media and news digital platforms, which, due to the ease of changing content, can make it dubious. “People intuitively know that when they need to really understand something complex or important, paper rules – ink on paper speaks of commitment and belief, writes Tuck (2018, p. 206) and Ludovico (2012, p. 7) reinforces: “(…) digital is built for speed, while print ensures stability (…)”.

The Importance of Regional Newspapers in Portugal

671

Other aspects mentioned as contributing to feeding the habit of reading in print are touching, leafing through and smelling a printed newspaper (Tuzzo 2014, p. 292). For assiduous readers, the digital newspaper complements the paper newspaper (Rathman 2002 apud Santos 2016, p. 10). However, this complementarity may cease to exist when both objects, print and digital, provide the same content (Kraut et al. 1998; Lee and Leung 2008, p. 146; Robinson et al. 2002, p. 247; apud Taipale 2013, p. 7). In an era strongly marked by technologies and digital media, by the logic of gratuitousness, easy access and extreme offer of news content, the printed newspaper becomes obsolete. “This multiformat and polycentric consumption, strongly sustained in the logics of gratuitousness, has disrupted the traditional hegemony of traditional formats and their unidirectional structures of dissemination and reception of news content.” (Quintanilha 2018, p. 139). Other differences we can list between digital and print newspapers focus on managing material resources. Digital newspapers have lower production costs as they do not need to use physical resources, which makes them more economical and environmentally friendly. However, they are dependent on the Internet and electricity. However, the printed newspaper has the disadvantage of being a single-use object. Even with all the advantages of online, the importance and function of the printed newspaper is a much-debated topic. According to Ludovico (2012, p. 29), the analogue object is considered a tangible repository. On the other hand, Mark Porter (Dawood 2018) and Nicholas Carr (Henriques 2012) highlight the importance of reading in a printed object and refer that digital objects influence our ability to concentrate and retain information. In this line of thought, Sónia Matos (in an interview) argues that with the supersonic speed that online media have fostered, the reader’s ability to reflect on information is compromised. Some argue that the decline in the reading of paper newspapers was due to a lack of interest in certain subjects (Chyi 2012 apud Santos 2016, p. 11) and not to the emergence of the new online media. One way to combat this problem is to carefully choose the content to be published, attracting the reader with exclusive texts, opinion sections and more in-depth reporting. In the case of online newspapers, exclusive content attracts readers to subscribe to a paid service. In both cases, quality is essential as a guarantee of sustainability since, in the case of printed newspapers, it depends on newsstand sales and advertising. In the case of online newspapers, it depends on the number of subscribers. Another perspective focuses on the definition of more extended periodicities for printed objects, as is the case of weekly and monthly newspapers, which offer the reader structured and exclusive content to face the high offer of digital content. Marco Grieco (in an interview) refers: “I think that niche newspapers, the newspapers with the widest periodicity, (…), are not going to die. I think that they are going to have to transform themselves”. Caldwell and Zapaterra (2014, p. 45) summarise this conjuncture: “News no longer sells newspapers. The internet and mobile media have made newspapers redundant as the preferred media for breaking news, and newspapers have had to reposition themselves accordingly”. In regional newspapers, digital media are not used so often because of the lack of human and economic resources. However, as Alves tells us, “(…), the Internet has

672

M. J. Nunes and E. Rolo

brought countless potentialities to proximity journalism, namely as a source of information, at the level of dissemination and distribution of contents, of the strengthening of the link with the community, especially the emigrant community, and of the conquest of new readers, mainly the younger generations.” (Alves 2020, p. 46). In this context, the evolution of the interaction between digital and paper is essential.

4 The Importance of the Press in an Inland Region At a time when we are constantly online and informed of what is happening in the country and the world, the regional press gains increasing importance through proximity to the local community: “One of the differentiating characteristics between the local and regional press and the national press is its natural vocation for the practice of ‘proximity journalism’, including in this concept the proximity to readers and sources” (ERC 2010, p. 109). With a specific target audience in a given region, it depends on customs, traditions and local events for the production of news content and, therefore, its value for the resident and migrant community increases: “The regional press plays a highly relevant role, not only in the territorial scope to which it naturally concerns most but also in the information and contribution to the maintenance of ties of authentic familiarity between the local people and the emigrant communities dispersed in the farthest parts of the world” (Estatuto da Imprensa Regional 1988, p. 1320). As Reis (2017) states, “Journalism is a public service and its role in scrutiny, in reporting, in validating facts, in confronting opinions, in encouraging debates, in the healthy habit of being countervailing power, in research, in the formation of thought is essential”. The newspaper and its news should transmit this relationship, and its dialectical and visual language should convey the place’s culture, placing the news of the region with greater prominence and priority than external news (Golding and Elliott 1979, p. 636). According to Decree-Law No. 106/88, the main functions of the regional press are: “a) To promote information concerning the various regions, as an integral part of national information, in its multiple facets; b) To contribute to the development of regional culture and identity through knowledge and understanding of the social, political and economic environment of the regions and localities, as well as the promotion of their development potential; c) To ensure easy access to information for regional and local communities; d) To contribute to the cultural and informative enrichment of the regional and local communities and to the occupation of their leisure time; e) To provide Portuguese emigrants abroad with general information on their communities of origin, strengthening the ties between them and their respective localities and regions; f) To foster a vision of regional issues, integrated with the national and international context.” (Estatuto da Imprensa Regional 1988, p. 1320). According to Alves (2020, p. 9), the regional press may recover the value of the printed newspaper, from its progressive decline in marketing, due to the proximity and loyalty of readers to the newspaper. Similarly, Mario Garcia (in an interview) argues: “If the regional newspaper does not exist, the important local news is not covered. I think that long after some national newspapers lose their status, the regional newspapers will always survive”. In turn, Tuzzo (2014, p. 298) refers that “journalists do not see the local newspaper as an alternative to boost the printed newspaper. On the contrary, they

The Importance of Regional Newspapers in Portugal

673

believe that the printed newspaper should continue to work with the National and the International, regardless of whether this news is also in the electronic media”. In 1997, Portugal had the highest number of regional newspapers per capita in the European context but the lowest marketing volume (Jerónimo 2015, apud Silva et al. 2017, p. 287). Between 2012 and 2016, the rate of change in the print circulation volume of regional newspapers compared to national newspapers was much lower, with a difference of 15.7% (Obercom 2018, pp. 17 and 18). In a study conducted by Reuters in 2017 Digital News Report, respondents answered that they preferred to consult national newspapers in an online format with a minimal margin of difference compared to the printed format, unlike the regional newspaper, in which the majority of readers preferred the printed format (78.2%) (Obercom 2018, p. 18). Reinforcing this idea that Portugal has always had a solid regional newspaper industry, the study by Bareme Regional Press, conducted by Marktest in 2009, referred that 49.7% of the Portuguese read regional newspapers (Lusa 2009). According to the study by ERC (2010, p. 22) on Regional and Local Press in Portugal, 728 regional periodical publications were identified, but with only 18 (2.5%) of daily periodicity, being mostly monthly (37.5%), then weekly (29.4%) and fortnightly or bimonthly (23.9%). The remaining are divided into bi-weekly, tri-weekly or trimonthly. The subsistence of regional newspapers depends mostly on sales and subscriptions due to low investment in advertising and the reduction of financial support, such as paid postage (ERC 2010, p. 106). To remedy this situation, some press associations have appealed to state agencies to guarantee support. It is, however, fundamental that exemption, a fundamental requirement of the news medium, is not compromised. Another very present problem in regional newspapers is the lack of human resources in the newsrooms, which overloads employees’ work, and a large part of them work voluntarily. The regional newspapers are essential for the local communities because, as Inácio states, “The identity of regions entails the need for mechanisms of symbolic production that contemplate the reinforcement of the feeling of belonging”. (Correia 1998, p. 6; apud Inácio 2015, p. 11). In this context, Sónia Matos (in an interview) refers, “(…) I think that [the regional newspapers] have immense potential, and I think that they value society”; And Marco Grieco concludes, “of the most interesting things is that the newspaper serves the community, which then gives back by consuming it.”

5 Conclusion Portugal has been marked, since the 1960s, by migratory dynamics that have led to the desertification of rural and interior areas and the concentration of population in large urban areas, such as Lisbon and Porto. In this context, the regional press is essential in maintaining ties between resident and migrant communities and their identity culture. Recent years have been marked by a great debate between print and digital media, with arguments for and against each. The popularity of printed newspapers has been decreasing, but significant transformations have been registered to adapt to the new paradigm—newspapers have become much more than printed publications. They have become diversified platforms that complement each other, such as websites, news platforms and social networks.

674

M. J. Nunes and E. Rolo

In the specific case of the regional press, sales and circulation figures have remained more stable, which predicts that it is in this context that the printed newspaper has a greater chance of survival. Furthermore, this type of press is based on proximity journalism, both to sources and the public, which is a differentiating factor in a world where generalist information is ubiquitous and circulates immediately. This research is the basis for a Communication Design Master investigation that aims to create a regional newspaper for the village of Mourão in the interior of Alentejo. It intends to respond to the need for design to assume itself as a differentiating discipline in response to society’s problems, and, in this context, it is considered that the contribution to the existence of credible and quality media is of the utmost importance. Acknowledgments. This work is financed by national funds through FCT – Fundação para a Ciência e a Tecnologia, I.P., under the Strategic Project with the references UIDB/04008/2020 and UIDP/04008/2020.

References Alves C (2020) A Imprensa de Proximidade em Portugal: Estratégia Editorial do Papel às Redes Sociais. Nova – Faculdade de Ciências Sociais e Humanas, Lisboa [Master Thesis] Caldwell C, Zapaterra Y (2014) Editorial Design – Digital and Print. Laurence King Publishing, London Dawood S: A newspaper is not a magazine – it is not an exercise in graphic design. https://www.designweek.co.uk/issues/2-8-july-2018/a-newspaper-is-not-a-magazinenewspaper-design-gestalten-mark-porter/. Accessed 20 May 2022 Estatuto da Imprensa Regional (1988) Decreto-Lei n. 106/88 (88-03-31) Sec. N. 76 – 1 Série 1320–1321 ERC – Estudo sobre a imprensa local e regional em Portugal (2010) 1st edn. ERC – Entidade Reguladora para a Comunicação Social [S.l.] ERC – Públicos e Consumos de Média (2014) O consumo de notícias e as plataformas digitais em Portugal e em mais dez países. ERC – Entidade Reguladora para a Comunicação Social [S.l.] Fundação Gulbenkian: Inquérito às práticas culturais dos portugueses. https://gulbenkian.pt/not icias/inquerito-as-praticas-culturais-dos-portugueses. Accessed 18 Sept 2022 Golding P, Elliott P (1979) News values and news production. In: Making the News, pp 632–643. Longman Henriques JG: A Internet mudou a nossa percepção do tempo. https://www.publico.pt/2012/11/28/ tecnologia/noticia/a-internet-mudou-a-nossa-percepcao-do-tempo-1573458. Accessed 10 Oct 2022 Inácio RAR (2015) Ciberjornalismo no Distrito de Santarém – A perspetiva do jornalista regional. Faculdade de Letras da Universidade do Porto, Porto [Master Thesis] INE (2021a) Jornais (No.) Anual. https://www.ine.pt/xportal/xmain?xpid=INE&xpgid=ine_ind icadores&indOcorrCod=0004133&contexto=bd&selTab=tab2. Accessed 17 Sept 2022 INE (2021b) Jornais (No.) por Localização geográfica. https://www.ine.pt/xportal/xmain? xpid=INE&xpgid=ine_indicadores&indOcorrCod=0010316&contexto=bd&selTab=tab2. Accessed 18 Sept 2022 INE (2021c) Circulação total de jornais (N.o) por Localização geográfica. https://www.ine.pt/ xportal/xmain?xpid=INE&xpgid=ine_indicadores&indOcorrCod=0010323&contexto=bd& selTab=tab2. Accessed 02 May 2022

The Importance of Regional Newspapers in Portugal

675

INE (2021d) Exemplares vendidos de jornais (N.o) por Localização geográfica. https://www.ine. pt/xportal/xmain?xpid=INE&xpgid=ine_indicadores&indOcorrCod=0010324&contexto= bd&selTab=tab2. Accessed 18 Sept 2022 INE (2021e) – Censos 2021: Divulgação dos Resultados Provisórios. https://www.ine.pt/xportal/ xmain?xpid=INE&xpgid=ine_destaques&DESTAQUESdest_boui=526271534&DESTAQ UESmodo=2&xlang=pt. Accessed 02 May 2022 Jerónimo P (2011) O uso da Internet na imprensa regional portuguesa. Universidad del País Vasco/Euskal Herriko Unibertsitatea C/Barrio Sarriena, Bilbao Ludovico A (2012) Post-Digital Print: The Mutation of Publishing since 1894. Onomatopee, Eindhoven Lusa – Quase metade dos portugueses lêem imprensa regional. https://expresso.pt/redeexpresso/ quase-metade-dos-portugueses-leem-imprensa-regional=f522165. Accessed 29 Sept 2022 Nora G: Jornalismo impresso na era digital: uma crítica à segmentação do público e à fragmentação do noticiário. Rumores (2011) Obercom – Jornais regionais: A análise de perfis de utilização. Actividade, desafios e políticas públicas para o sector. Obercom, Lisboa (2018) Obercom – O Mercado Publicitário em Portugal, Prospetivas gerais e cenários de futuro: Análise de pormenor ao sector da imprensa. Obercom, Lisboa (2019) Pordata (s.d.). Conheça o seu Município. https://www.pordata.pt/Municipios. Accessed 29 Apr 2022 Pordata (2015) População residente por migrações segundo os Censos. https://www.pordata. pt/Municipios/Popula%C3%A7%C3%A3o+residente+por+migra%C3%A7%C3%B5es+seg undo+os+Censos-48-2623. Accessed 05 July 2022 Pordata (2021) Jornais e outras publicações periódicas: total e por suporte de difusão. https:// www.pordata.pt/Portugal/Jornais+e+outras+publica%C3%A7%C3%B5es+peri%C3%B3d icas+total+e+por+suporte+de+difus%C3%A3o-2240-178812. Accessed 17 Sept 2022 Pordata (2022a) Jornais e outras publicações periódicas: títulos publicados por tipo de publicação. https://www.pordata.pt/Portugal/Jornais+e+outras+publica%C3%A7%C3%B5es+ peri%C3%B3dicas+t%C3%ADtulos+publicados+por+tipo+de+publica%C3%A7%C3% A3o-1308-10430. Accessed 17 Sept 2022 Pordata (2022b) Jornais e outras publicações periódicas: circulação total, exemplares vendidos e exemplares distribuídos gratuitamente por tipo de publicação. https://www.pordata.pt/ Portugal/Jornais+e+outras+publica%C3%A7%C3%B5es+peri%C3%B3dicas+circula%C3% A7%C3%A3o+total++exemplares+vendidos+e+exemplares+distribu%C3%ADdos+gratui tamente+por+tipo+de+publica%C3%A7%C3%A3o-2239-178791. Accessed 18 Sept 2022 Quintanilha TL (2018) 2029 – o fim dos jornais em papel em Portugal? Um estudo longitudinal sobre os principais indicadores de desempenho no sector da imprensa escrita tradicional portuguesa. Observatorio OBS* Quintino ASS (2018) Efeitos demográficos e económicos das migrações em Portugal: o caso da Segurança Social. Faculdade de Ciências da Universidade de Lisboa, Lisboa [Master Thesis] Ramos M (2012) Migrações, Desenvolvimento e Dinâmicas Locais e Regionais. In: Grandes Problemáticas do Espaço Europeu: Políticas de Ordenamento e Recomposições Territoriais nas Periferias Europeias, pp 63–102. Faculdade de Letras da Universidade do Porto, Porto Reis G: Da importância do jornalismo. https://observador.pt/opiniao/da--importancia-do-jornal ismo. Accessed 02 May 2022 República Portuguesa – Programa Nacional Para a Coesão Territorial (2017a). https://www.por tugal.gov.pt/pt/gc21/governo/programa/programa-nacional-para-a-coesao-territorial-/fichei ros-coesao-territorial.aspx. Accessed 29 Apr 2022

676

M. J. Nunes and E. Rolo

República Portuguesa – Programa Nacional Para a Coesão Territorial: Implementação, acompanhamento e monitorização (2017b). https://www.portugal.gov.pt/pt/gc21/governo/programa/ programa-nacional-para-a-coesao-territorial-/ficheiros-coesao-territorial.aspx. Accessed 29 Apr 2022 Rosado MC (2015) Diáspora. Pobreza, Emigración y Desarollo. A Viagem dos Argonautas. https://aviagemdosargonautas.net/2015/04/01/diaspora-pobreza-emigracion-y-desarrollopor-moises-cayetano-rosado-2. Accessed 29 Apr 2022 Santos, NDFSB (2016) Efeitos do consumo digital na imprensa escrita: evidência empírica para os jornais diários portugueses. Instituto Superior de Economia e Gestão, Lisboa [Master Thesis] Saragih MY, Harahap AI (2020) The challenges of print media journalism in the digital era. Budapest Int Res Crit Inst (BIRCI-J): Hum Soc Sci 540–548 Silva MT, et al (2017) Between traditional and social media: news repertoires in Portugal. Participat J Aud Reception Stud. 14(2):283–300 Taipale S (2013) The relationship between Internet use, online and printed newspaper reading in Finland: investigating the direct and moderating effects of gender. Eur J Commun 28(1):5–18 Tuck A (2018) Monocle’s Singular Vision. Newspaper Design: Editorial Design from World’s Best Newsroom. Gestalten, Berlim Tuzzo AS (2014) O impresso na era digital pela perspectiva dos portugueses. Universidade Federal de Goiás – UFG and Centro de Investigação Media e Jornalismo – CIMJ – UNL, Goiás

Designing IoT-Based Products for Indoor Air Quality Management: A Design Thinking Methodology Ana Filomena Curralo1,2(B) , António Curado1,3 , Paulo Barros1,4 Inês Freitas1 , and Sérgio Ivan Lopes1,4

,

1 Escola Superior de Tecnologia e Gestão, Instituto Politécnico de Viana do Castelo,

Rua da Escola Industrial e Comercial de Nun’Alvares, 4900-347 Viana do Castelo, Portugal [email protected] 2 ID+ Instituto de Investigação em Design, Media e Cultura, 3810-193 Aveiro, Portugal 3 ProMetheus, Instituto Politécnico de Viana do Castelo, Viana do Castelo, Portugal 4 ADiT-LAB, Instituto Politécnico de Viana do Castelo, Viana do Castelo, Portugal

Abstract. Related research reports the occurrence of several million deaths annually due to the harmful effects of air pollution, whether environmental or domestic. To reduce the impact, Indoor Air Quality (IAQ) investigation is increasingly a subject of utmost importance for mankind nowadays, since people spend most of their time indoors, where the pollution level is higher than outdoors. This work presents the design and development of an Internet-of-Things (IoT) based product for online IAQ assessment and mitigation, which included the design of three modular devices for IAQ management, where the following characteristics are studied and combined: (i) IAQ, (ii) energy efficiency, and (iii) human occupation. Based on the challenge of using IoT technologies, the design of a product case was put forward by taking advantage of the possibilities offered by these solutions as well as improving the relationship between a product function and its form. This article aims to explore a design thinking methodology to develop an organized product idea with the engineering area for an IoT device to monitor IAQ. With the interconnection of these technologies with the design, the goal is to respond to the user needs by applying a methodology able to support and sustain the design research focused on user requirements specified based upon the user perspective and highlighting the importance of multidisciplinary teams to establish a conceptual methodology. Keywords: design innovation · product design · design thinking · IoT · IAQ

1 Introduction The significance of optimal IAQ in buildings and enclosed spaces has gathered considerable attention and apprehension among the global population, primarily due to its profound implications in human health. Exposure to elevated concentrations of pollutants within such environments has been linked to a myriad of detrimental health effects. Consequently, organizations affiliated with global health have taken proactive measures to © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 677–694, 2024. https://doi.org/10.1007/978-3-031-47281-7_56

678

A. F. Curralo et al.

formulate guidelines aimed at fostering awareness, monitoring, and mitigation strategies to address this issue. In recent decades, a growing number of scientific evidence indicates that indoor air, mostly in highly industrialized urban areas, can exhibit significantly higher levels of pollution compared to outdoor air [1]. IoT is a collective term used to describe the change in the technological interaction model, in which people started to interact with objects that can communicate with each other from embedded sensors that share data over long distances using the Internet [2]. IoT is also a prolific field for commercial enterprises, as one in six companies plans to launch an IoT-based product [3]. These general implementations of digital technologies are changing the nature of things and affect how organizations develop new products and services [4, 5]. Furthermore, these factors open new opportunities in terms of how organizations expand revenue and IoT is consequently becoming a popular topic of exploration among academia and industry [6]. Design is understood as the development of products and/or services whose affinity must be, primarily, based on the user. In this way, for the design field, this means developing interactive projects, in direct relation to the experiences of its target audience, maximizing the acceptance of these new products and/or services in the market [7]. This work presents the design investigation that occurred through the development of IoT end devices to manage IAQ, to be able to correspond to the needs of a possible consumer. The ongoing research’s main goal is to tackle indoor air pollution by creating a practical, functional, and intuitive solution to empower consumers in the identification and management of radon risk exposure, as well as to promote greater attention and concern for air quality [8, 9]. As additional objectives, the project aims to highlight the need to input design in the development of IoT devices, as well as to prove its importance for the creation of environmentally friendly products. This article intends to practice design thinking with a focus on user needs, thus providing a better analysis and understanding of ergonomics, usability, and product-user interaction issues. As a final goal, it is intended to use a multidisciplinary approach between the areas of design and engineering, to enrich the exchange of ideas, information, and concepts. During the research, it was noted that IoT technology, and its devices have been calling consumer’s attention. In the end, we noticed that the unexplored points in this kind of devices are now gaining importance by professionals in different areas, especially in Design. One of the unexplored points that prove to be essential for the success of any object, especially an IoT or electronic device, is the capacity of productization. This ability refers to the practice of transforming a service or product, which has only been developed in-house, into something, which can be produced on a large scale, i.e., present a pleasing, functional design, be fully tested, packaged, and marketed [10]. Product and design development teams today, require seamless collaboration between designers and engineers who possess multidisciplinary skills [11]. Such integration leads to innovative design and engineering methodologies and new synergies, broadening the scope of service delivery. The increasing competition in consumer markets and growing recognition of the importance of design in the market have reinforced the belief that design success can only be achieved through the integration of both engineering and design skills [12]. The division between these distinct disciplines is eroding, particularly in the realm of product design and development, where designers

Designing IoT-Based Products for Indoor Air Quality Management

679

and engineers work together to expand their range of services [13]. The convergence of design and engineering brings together the human aspects and aesthetic-formal elements of design while emphasizing the technical skills of engineering and the human skills of design [14].

2 Methodology of the Process: Design Thinking Designing products has never been just about offering the best aesthetic option, but, rather, about trying to create something that can offer all the characteristics of a good product to the consumer. In this sense, product design is concerned with the functionality, usability and interaction between people and technology. Design Thinking is a philosophy with a set of tools that help companies, designers, and multidisciplinary teams to solve problems in a creative way towards innovation. This methodology does not follow an orderly and linear thought, it is rather formed by a round trip process, where each project is different from the other, resulting in different ways of approaching a project [15]. This project methodology reveals a logical process that employs multidisciplinary paths between several areas and people, intending to create innovative and effective products or services. It is based on an orientation centered on the understanding of the user and/or consumer, with innovation fueled by an in-depth understanding, through direct observation, of what people want and need in their lives and what they like and dislike about the way certain products are manufactured, packaged, marketed, sold, and supported [16]. Although this process is considered free, flexible, and characterized as a method of various back and forth, some main points help to understand and follow this philosophy. The reason for the interactive and non-linear nature of the journey is not that design thinkers are disorganized or undisciplined, but that design thinking is

Fig. 1. Phases of Design Thinking in the development of IoT devices

680

A. F. Curralo et al.

fundamentally an exploratory process [15]. The phases of this method are empathy, definition, idea generation, prototyping, and testing, as exemplified in Fig. 1. Although the project is still in its initial phase, it is planned to go through all the phases, using methods and techniques as support and as a guide for the development of a well-executed project. 2.1 Empathy Phase: Immersion Process The first phase in the process is Empathy which is probably the important distinction between academic thinking and Design Thinking [15]. Empathy is the search for understanding people’s needs, especially when we want to create a product or a service it is important to understand and observe what are the audience’s requirements: what is their world, what are their emotions and what are the things they give importance. The creators rarely design or produce for themselves, the work involves the creation and design of projects that satisfy the general population, for that it is important to interact with people and comprehend their needs. It is essential to create products and services that can improve people’s lives, inspire new ideas and recognize their inexplicable behaviors that represent different strategies for dealing with the confused, complex, and contradictory world in which we all live today [15]. During the empathy process, some activities were developed to better understand the user, including creating personas, making lists of user feedback, and photographing the environment and users, among others. However, to begin the process of empathy it is always necessary to consider these two points, observe and interview (engage). Thus, to gather empirical data on consumer perceptions and preferences regarding IoT devices and their impact on environmental aesthetics, a survey containing 16 questions was carried out with a population of 69 people, during approximately 3 weeks in March 2023, for a deeper understanding of the subjects’ preferences (see Table 1). The development of the survey involved a structured and thoughtful process, here’s a brief overview of the main steps taken for its creation: • STEP 1: Defining research objectives—Clearly define the specific information needed. The aim is to gather insights into consumer demographics, their awareness of IoT technology, factors influencing electronic device selection, and their opinions on the impact of design on environmental aesthetics. • STEP 2: Designing survey questions—Based on the research objectives, a set of well-crafted survey questions were developed, to be clear, concise, and relevant to the topics of interest. Close-ended questions were used to facilitate data analysis. • STEP 3: Pilot testing—Before launching the survey, a pilot test was conducted with a small group of individuals to identify any potential issues with question clarity, relevance, or formatting. The feedback was used to refine the survey. • STEP 4: Selecting the survey platform—Google Forms was chosen as the survey platform due to its ease of use and data management capabilities. The platform allowed for efficient distribution and collection of responses. • STEP 5: Distribution and data collection—The survey was then distributed to a target audience, and responses were collected over a specified period. It was accessible to a diverse group of participants, ensuring a broad range of perspectives.

Designing IoT-Based Products for Indoor Air Quality Management

681

• STEP 6: Data analysis—Once data collection was complete, the survey responses were meticulously analyzed. Statistical tools and techniques were applied to derive meaningful insights and trends from the data. • STEP 7: Results presentation—The survey results were presented in a clear and organized manner, as shown ahead, providing a snapshot of the findings. • STEP 8: Critical analysis—Following the presentation of results, a critical analysis was performed to interpret the implications of the findings and offer insights into the broader context of consumer preferences and attitudes in relation to IoT devices and environmental design. This structured approach to survey development allowed for the systematic gathering of information, ensuring that the resulting insights are reliable and valuable for informing decisions in product development, marketing, and technology education. Table 2 presents the survey results categorizing respondents’ awareness of IoT technology into several response options, including “Yes” and “No”, representing whether respondents were acquainted with IoT concepts or not. These categorizations provide a comprehensive view of the participants’ familiarity with IoT, offering valuable insights for technology education and IoT product development stakeholders. The survey results reveal several critical insights into consumer perceptions and preferences regarding IoT devices and environmental design. Notably, a substantial majority of respondents, comprising 77%, were not familiar with the concept of IoT, highlighting a notable lack of awareness about this technology. Furthermore, a significant proportion expressed optimism about IoT adoption, with 67% believing that most of the population would embrace it, potentially indicating a favorable market environment for IoT products. Additionally, many respondents (75%) expressed interest in acquiring devices focused on indoor air quality measurement and mitigation, underscoring the importance of addressing health and environmental concerns. However, it’s noteworthy that only a small percentage were aware of specific IAQ measurement devices (10%). The influence of design on environmental aesthetics also emerged as a significant concern, with 84% of respondents acknowledging that certain electronic devices could compromise the visual harmony of their surroundings. This suggests that careful attention to design aesthetics is crucial in IoT product development. In the end, the conclusion was that users showed interest in both the activity, the technology, and the product to be developed. In relation to the design, these showed greater interest in a simple, easy-to-use, interactive product with a reduced number of buttons (to facilitate the use) and that it is discreet and adaptable to different types of environments. The whole inquiry process allowed not only to understand the consumer’s perspective on the project as well as their perspectives, tastes, and desires. Overall, these findings provide valuable guidance for businesses and researchers seeking to align their strategies with consumer preferences in the rapidly expanding IoT market. 2.2 Definition Phase (Analysis and Synthesis) In this phase, all the information and findings of the previous phase are gathered, analyzed, and synthesized to discover a single concept. After collecting all the information

682

A. F. Curralo et al.

Table 1. Survey on consumer perceptions and preferences regarding IoT devices and environmental design Topic

Question/Statement

Demographic information

1. Please provide your age: 2. Please specify your gender:

Awareness of IoT Technology

Ownership and functionality of IoT devices

Perceptions of IoT Adoption

Indoor Air Quality Awareness

Device Selection Criteria

Options

[Textbox] a) Male b) Female 3. Are you familiar with the a) Yes concept of IoT, also known as b) No the Internet of Things? 4. Are you aware of the TECH a) Yes project at IPVC? b) No 5. Do you currently own any a) Yes, [Textbox] IoT devices? If yes, please list b) No the device(s) and describe their primary functionalities 6. Based on your knowledge of a) Yes IoT devices, do you believe b) No most of the population will embrace this technology? 7. In your opinion, do you a) Yes foresee a future where many b) No daily activities will depend on the use of one or more IoT devices? 8. Have you ever contemplated a) Yes indoor air quality concerns b) No within buildings? 9. Would you express interest in a) Yes acquiring a device primarily b) No designed for measuring indoor air quality and mitigating pollutants? 10. Are you familiar with any a) Yes, [Textbox] specific indoor air quality b) No measurement devices? If yes, please provide their names 11. Please specify the primary a) Functionality factors that influence your b) Aesthetics selection of an electronic c) Usability device d) Material composition e) Other (please specify) [Textbox] 12. In a hypothetical scenario a) Physical design (form, color, where three electronic materials, concept) devices offer identical b) Usability (ease and functionalities, what would intuitiveness) be the main factors c) Material quality influencing your choice d) Other (please specify) among them? [Textbox]

(continued)

Designing IoT-Based Products for Indoor Air Quality Management

683

Table 1. (continued) Topic

Question/Statement

Options

Environmental Aesthetics and Design

13. Do you believe that the design of some electronic devices can have a negative impact on their environmental aesthetics? If yes, please explain 14. What aspects of electronic device design do you find most problematic or potentially disruptive to environmental aesthetics?

a) Yes, [Textbox] b) No

a) Excessive controls (buttons) b) An overabundance of features c) Difficulty in discerning device purpose d) Inappropriate icons e) Other (please specify) [Textbox] 15. Which type of user-device a) Indicator lights for device interaction do you find most status appealing for implementing b) Display with basic electronic devices in information (date, temp., etc.) different environments? c) Physical buttons (e.g., on/off) d) QR Codes for accessing control apps or information e) Other (please specify) [Textbox] 16. Considering the current use a) Ease of use of electronic devices, what b) Design tailored for various improvements do you think environments are necessary to enhance the c) Design that communicates user experience? device functionality d) No improvements needed e) Other (please specify) [Textbox]

about who are the users, what are their needs, and what their problems are, it is necessary to transform all this knowledge into a logical and justifiable issue, consequently at a certain moment the team must sit down and in an intense period of synthesis, organize, interpret, and weaves data threads into a coherent story [15]. Currently, the project has limited information due to its early research stages. However, as research and methods advance, the data will progressively increase. The existing information is founded on design thinking and methodologies, encompassing project details, its environment, and user data. The analysis and synthesis of this data have been iterative, with previously discarded data sometimes becoming valuable and reintegrated into the design process. The aim of this phase is essentially to promote the development of focus and purpose for the project. Without defining a concept, it is difficult to understand design and problem-solving. Figure 2 represents a strategy to promote multi-discipline between the areas of engineering and design, to create participative synergy. Through the exchange of information, tasks, ideas and concepts between both areas, the project and its members will be able to mature, create and elaborate a product that can reflect the best that both worlds have to offer.

684

A. F. Curralo et al. Table 2. Survey Results: Consumer views on IoT devices and design

Question summary

Answers

Results

1

Age distribution

69

Youngest consumer: 16 years Oldest consumer: 71 years Most significant age group: 22 years (15%) Age range 16–23 years: 57% Age range 24–71 years: 43%

2

Gender distribution

69

Female participants: 75% (52) Male participants: 25% (17)

3

Familiarity with IoT (Internet of Things)

69

Yes: 23% No: 77%

4

Awareness of TECH Project at IPVC

69

Yes: 14% No: 86%

5

Possession of IoT devices and their functionality

69

Yes: 32% No: 68%

6

Public adoption of IoT technology

69

Majority believe in adoption: 67% Majority skeptical of adoption: 33%

7

Dependency on IoT devices in everyday life

69

Expectation of dependency: 75% Expectation of minimal dependency: 25%

8

Awareness of IAQ concerns

69

Reflective of IAQ: 58% Not reflective of IAQ: 42%

9

Interest in IAQ measurement and mitigation devices

69

Interested in acquiring devices: 75% Not interested in acquiring devices: 25%

10

Awareness of IAQ measurement devices

68

Awareness of devices: 10% No awareness of devices: 90%

11

Primary factors in device selection

67

Functionality: 48% Aesthetics: 1% Both: 50% Other: 1%

12

Factors influencing choice among identical devices

68

Form, color, materials, concept: 13% Easy and intuitive usability: 77% Material quality: 9% Other: 1%

13

Electronic device design’s impact on aesthetics

32

Influences environmental aesthetics: 84% Does not influence environmental aesthetics: 16%

14

Most problematic design aspects

68

Too many controls (buttons): 31% Excessive number of features: 25% Difficulty in understanding device purpose: 25% Inadequate icons: 18% Other: 1%

15

Preferred user-device interaction for electronic devices

68

Lights indicating device status: 27% Display with basic information: 44% Physical buttons: 19% QR code access: 10% Other: 0%

(continued)

Designing IoT-Based Products for Indoor Air Quality Management

685

Table 2. (continued) Question summary

Answers

Results

16

68

Need for improved ease of use: 48% Design adaptation for different environments: 21% Design that communicates device functionality: 21% No improvements needed: 10% Other: 0%

Improvements needed for enhanced user experience

Fig. 2. A process for designing IoT products and services

The engineering process adopted a modular approach for the prototype, which includes the design of three modules that will help to manage IAQ problems. The first module comprises a devkit with a microcontroller unit and sensors for measuring thermal comfort (temperature and relative humidity) and CO2 ; the second module is dedicated to IAQ and radon gas monitoring; and the third module includes an infrared camera to detect room occupation by people. Thus, these three modules were divided into layers, as shown in Fig. 3, which consist of several Print Circuit Boards (PCBs) capable of coupling and decoupling, according to the users’ desire or needs.

Fig. 3. Representation of the three modules and respective layers

2.3 Idea Generation Phase Idea generation is the third phase of the Design Thinking process, and usually, this process is always accompanied by a variety of methods such as sketching, prototyping, and brainstorming, among others. This phase was prepared jointly by the design and engineering areas, and with the support of some users. In terms of design, not only in this phase but also in the whole process of Design Thinking, two important thoughts

686

A. F. Curralo et al.

are commonly used: convergence and divergence. Divergent thinking essentially serves to create ideas, it explores various options to produce the maximum number of ideas possible, this also multiplies the number of created choices [15]. Convergent thinking is the opposite, in other words, this thought helps the team to focus on the essentials and discard everything that doesn’t matter [15]. Table 3 presents the comparison between the 3 main ideas, obtained through divergent thinking, in terms of strengths and weaknesses. Table 3. Comparative table of divergent thinking ideas

Complementing the model phase, two ideas were selected, and some renderings were made to be able to visualize an approximate version of reality. Figure 4 presents the two renders that were selected by the work team, made in SolidWorks and 3D Max software, for modeling and materials assignment, respectively. Rendering techniques were employed to generate visual representations that would serve as supporting images for the ultimate objects. These rendered images aimed to effectively depict the various contexts in which the products could be seamlessly integrated.

Designing IoT-Based Products for Indoor Air Quality Management

687

Fig. 4. Renders representation made through SolidWorks and 3D Max software

2.4 Prototyping Phase The Prototyping method is normally used in the final phase of the Design Thinking process, to test the users’ behavior with the prototype and thus denote new solutions to the problem or realize that some of the implemented solutions were not successful. The basis for this phase is experimentation, it is important to understand that it is not necessary to produce a very detailed and expensive prototype, but the development a prototype capable of presenting the necessary features for the user to test. According to Tim Brown [15] it is with the experimentation method that the team can understand the mistakes they are making, and which ideas and solutions are more appropriate for the final product. The project seeks the execution of a pilot that corresponds to all the objectives mentioned above. In this way, several prototypes were elaborated, with various materials, to prototype more quickly and economically, so that in the testing, users will have something with which they can interact, and the team will understand the strong and weak points to improve the project. After the draft and conceptual project reach a satisfactory conclusion, the selected prototype will be produced using additive manufacturing, so that it can be submitted to final tests and serve as proof to the whole project process elaborated so far. Following the selection of the initial concept, the team promptly initiated the process of gathering imperative prerequisites for the development of prototypes. Before commencing the printing procedure, a comprehensive 3D modelling of all constituent components was executed. This attempt encompassed meticulous attention to crucial intricacies of the internal structure planning of the housing, which accommodates the technological core shown in Fig. 5. For the union between the casing and the technological prototype, was considered the option of replacing the spacers, which join the PCB board together, with columns created in filament from the base of the modules, holding both parts. However, this option ended up bringing fragility problems due to the thinness of the columns, causing them to easily break with small movements. Therefore, to optimize the entire coupling projection, it was decided to replace the filament columns with metallic threaded rods. In

688

A. F. Curralo et al.

the 3D printed base, hexagonal cavities were inserted to fix nuts, so that when assembling all the parts, they serve to fix the prototype with threaded spacers, uniting the base, the rods, and the technical part of the modules, into a whole.

Fig. 5. Technological core (for two of the modules)

Another setback was the separation of the temperature and relative humidity sensor from its PCB, which resulted in some internal changes. The first change was the insertion of a specific compartment for this sensor, inside the body of the device, that will prevent the emission of incorrect values due to the “chimney effect” generated by the other sensors. The second change is related to the height of the casing, meaning it was necessary to increase the useful interior height to contain the main components.

Fig. 6. First and second printings

The prototypes were fabricated by using a 3D printer and procuring the necessary filament. Specifically, a Flashforge Creator 3 printer was employed, utilizing white PLA filament measuring 1.75 mm in diameter.

Designing IoT-Based Products for Indoor Air Quality Management

689

Figure 6 illustrates an example of the first and second printings. The printing process for the prototypes associated with the initial concept commenced on May 15, 2023. Each unit took approximately 75 h to complete, indicating that the printing of both concepts and their respective modules required roughly two weeks. During the initial week of the printing phase, some problems arose regarding filament expansion, thereby mandating modifications in specific dimensions related to the caps and the entrance regions of the LED, antenna, and power plug. With the conclusion of the prototyping phase, it would be natural to start the testing phase for the validation of the present study. Validation is performed through tests, where the contact between the user and the product is evaluated. Depending on the user’s reaction, the designer determines if it is necessary to resolve some aspect of the design, going back in the process until the response is satisfactory in all stipulated points. However, validation can be performed in another way, since Tim Brown (2009) states that prototyping plays an essential role in the design process, serving as validation of a subset, that is, the realization of prototypes ends up validating the whole process carried out during the investigation, which is the physical proof of its success. In short, the realization of prototypes ensured the validation of the entire conceptual process, from the use of methodologies to the generation of ideas and prototype design, thus proving its legitimacy. 2.5 Testing Phase The test phase, despite being the last phase of the process, can trigger new solutions, ideas, and even the creation of a new empathy with the users. As briefly explained in the previous phase, the testing phase is where the user feedback is given after contact with the prototype, in this phase it is normal to go back to previous phases, to add or correct errors. So, regarding the new information from consumer testing, the agile team of design thinkers will have to self-correct the product along the way [15]. The research in question intends to use the testing phase, interspersed with the prototyping phase. This means that several prototypes will be developed to ensure that they correspond to the requested objectives. Hence, particular emphasis will be placed on conducting user tests, as the interaction between the user and the prototype holds paramount importance in assessing the prototype’s quality. Usability tests will be conducted in a face-to-face setting, with a minimum of 50 participants engaging directly with the prototype and receiving explanations of its functions. Subsequently, they will assess its performance using the 19 questions outlined in Table 4. These tests are instrumental in gathering essential feedback on design, functionality, and the user-prototype interaction. This feedback will contribute significantly to enhancing project organization, validation, and the reevaluation of specific aspects for project improvement. In conclusion the testing phase will permit the creation of a communication channel with the final user to collect feedback about the product and eventually reconsider the previous phases to improve and correct mistakes.

690

A. F. Curralo et al. Table 4. Usability testing survey

Topic

Question/Statement

Options

Demographics

1

Please provide your age:

[Textbox]

2

Please specify your gender: a) Male b) Female

3

After direct interaction with a) Yes the prototypes and an b) No understanding of their operation, would you consider using this range of products?

4

Considering that module acquisition is optional, which module(s) do you prefer?

a) Indoor air quality module b) Thermal comfort module c) Human occupancy detection module

5

What specific product measurement features do you find most valuable? Please select all that apply:

a) b) c) d) e) f)

6

Have you used a similar product in the past or are you currently using one? If yes, please specify the product

a) Yes, [Textbox] b) No

7

Would you recommend this a) Yes product to your family, b) No friends, and acquaintances?

8

What are your thoughts on the location restrictions of the modules?

a) Acceptable, no issue b) Not preferable; somewhat limiting

9

During the usability test, did you encounter any difficulties in using the product? If yes, please describe the main challenges you faced

a) Yes b) No

Product Evaluation

Usability Assessment

Radon gas CO2 TVOC Temperature Relative humidity Other (please specify): [Textbox]

(continued)

Designing IoT-Based Products for Indoor Air Quality Management

691

Table 4. (continued) Topic

Question/Statement

Options

10

On a scale from 1 to 5, how [Textbox] would you rate the ease of use of the device? (1 = Very difficult, 5 = Very easy)

11

Did you experience any a) Yes difficulties in understanding b) No how to operate or use the product?

12

On a scale from 1 to 5, what [Textbox] is your overall impression of the product’s design? (1 = Very unfavorable, 5 = Very favorable)

13

On a scale from 1 to 5, how [Textbox] would you rate the interaction between the product and the user? (1 = Poor, 5 = Excellent)

14

On a scale from 1 to 5, how [Textbox] do you feel about the absence of physical buttons to activate the device, requiring only a power connection? (1 = Strong disapproval, 5 = Strong approval)

15

On a scale from 1 to 5, what [Textbox] is your opinion on the use of LED lights for real-time IAQ indication? (1 = Strongly disapprove, 5 = Strongly approve)

16

Please share what aspects of the product’s design you liked the most and what aspects you liked the least

Liked: [Textbox] Disliked: [Textbox]

(continued)

3 Final Consideration The current investigation allows concluding that Design Thinking is a powerful tool to improve the quality of the final product, by focusing on the main spotlight of the project. Therefore, design is assumed to be a methodological process, where it is necessary

692

A. F. Curralo et al. Table 4. (continued)

Topic

Question/Statement

Options

17

On a scale from 1 to 5, what [Textbox] is your opinion regarding the concept of minimalist and simple product designs for improved adaptability to indoor environments? (1 = Strongly disagree, 5 = Strongly agree)

18

On a scale from 1 to 5, how [Textbox] would you describe your overall experience with the product? (1 = Very negative, 5 = Very positive)

19

Would you like to see any a) Yes, [Textbox] additional features or b) No functionalities incorporated into the product? If yes, please specify

to go back and forth through various methods and steps, to structure, understand and substantiate the most logical and beneficial way to reach a mature product. The design methodology studies the way designers work and think, including the establishment of appropriate structures for the design process, the development and application of new methods, techniques, and procedures of design, and reflection on the nature and extent of design knowledge and its application to design problems. In this way, the use of this methodology allowed a better structuring of the process, of the research. As such, it was possible to understand what kind of techniques would be most suitable for good execution of the design project. The methodology benefit resides, essentially in their concepts, that is in addressing a design process focused on problem resolution, through the user’s needs and highlighting the importance of a study based on multidisciplinary teams. This research involves a collaborative team to solve IAQ management problems, which works to develop an IoT pilot device, capable of identifying various types of indoor pollutants and alerting users through a web application, in case of high levels. It is important to emphasize that designing problem-solving methodologies based on user needs, applies to the development of all types of products, but IoT or electronic devices require special attention, since the technological aspect is, for many people, an issue of difficult understanding and interaction. Moreover, the processes addressed in this work, contribute to the development of a better relationship with the user, understanding and meeting their needs. In this sense, another benefit of this methodology is the fact that the study can be developed in several areas at the same time. Only then it is possible to create and mature

Designing IoT-Based Products for Indoor Air Quality Management

693

an innovative project, which includes areas such as design, engineering, marketing, and health in the conceptual process. It was also possible to reach several conclusions about the project investigation. First, through the research carried out, the methods and techniques used in the design thinking methodology were better understood, increasing the overall perception of how this methodology benefits all project research. The Design Thinking methodology offers a detailed conceptual process and a deep understanding of the matters, always focused on user needs, and should be supported with excellent methods and techniques that prioritize the relationship between multidisciplinary teams. Following this process, it is perfectly understood at what stage we are at and how we can and should act to obtain good results. Ultimately, the work developed demonstrates that the designer must focus on the user’s needs, in all stages of the process. Funding. This work is a result of the project TECH—Technology, Environment, Creativity and Health, Norte-01-0145-FEDER-000043, supported by Norte Portugal Regional Operational Program (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, through the European Regional Development Fund (ERDF).

References 1. Matos J (2014) Indoor Air Quality in Hospitals: Identifying Pollutants, Sources and Strategies for Optimizing Healthcare Services. http://hdl.handle.net/10362/12321, https://run.unl.pt/bit stream/10362/12321/1/Matos_2014.pdf 2. Desbarats G (2017) Silo-Busting to create great experiences for Internet of Things. Des Manag Rev 28(1):28–36. https://doi.org/10.1111/drev.12064 3. Burkitt F (2014) A Strategist’s Guide to the Internet of Things: The digital interconnection of billions of devices is today’s most dynamic business opportunity, PWC. https://www.str ategy-business.com/article/00294 4. Yoo Y (2013) The tables have turned: how can the information systems field contribute to technology and innovation management research? J Assoc Inf Syst 14:227–236 5. Hui G (2014) How the Internet of Things Changes Business Models. Harvard Business Review. https://hbr.org/2014/07/how-the-internet-of-things-changes-business-models 6. ITU – The Internet of Things – Executive Summary (2005). http://handle.itu.int/11.1002/pub/ 800eae6f-en 7. Gasparetto DA et al (2016) Connected design: towards a world of experiences. Estudos em Design. Rio de Janeiro 24(2):112–131. ISSN 1983-196X. https://estudosemdesign.emn uvens.com.br/design/article/view/348, https://estudosemdesign.emnuvens.com.br/design/art icle/download/348/234 8. Pereira F, Lopes SI, Carvalho NB, Curado A (2020) RnProbe: a LoRa-enabled IoT edge device for integrated radon risk management. IEEE Access 8: 203488-203502. https://doi. org/10.1109/ACCESS.2020.3036980 9. Barros P, Curado A, Lopes SI (2021) Internet of Things (IoT) technologies for managing indoor radon risk exposure: applications, opportunities, and future challenges. Appl Sci 11(22):11064. https://doi.org/10.3390/app112211064 10. Rowland C, Goodman E, Charlier M, Light A, Lui A (2015) Designing Connected Products: UX for the consumer Internet of Things. O’ Reilly, Springfield. https://www.oreilly.com/lib rary/view/designing-connected-products/9781449372682/ 11. Norman D (2013) The Design of Everyday Things. Basic Books, New York

694

A. F. Curralo et al.

12. Walsh V, Roy R, Bruce M (1988) Competitive by design. J Mark Manag 4(2):201–216. https:// doi.org/10.1080/0267257X.1988.9964069 13. Rosenberg N, Nelson RR (1994) American universities and technical advance in industry. Res Policy 23(3):323–348. https://doi.org/10.1016/0048-7333(94)90042-6 14. Erlhoff M, Marshall T (2008) Design Dictionary: Perspectives on Design Terminology. Walter de Gruyter, Berlin. https://doi.org/10.1007/978-3-7643-8140-0 15. Brown T (2008) Design thinking. Harv. Bus Rev 86(6):84–92 16. Cross N (1982) Designerly ways of knowing. Des Stud 3(4):221–227. https://doi.org/10.1016/ 0142-694X(82)90040-0

The Importance of Chat GPT Training for Higher Education: Case Study Maria Potes Barbas, Andreia Teles Vieira(B) , and Paulo Duarte Branco Instituto Politécnico de Santarém - Escola Superior de Educação, santarém, Portugal [email protected]

Abstract. This paper presents a case study aimed to explore the importance of chat GPT training in higher education sector. This paper introduces the definition of what is chat GPT and the role of this tool in education, especially in higher education. A training of the chat GPT tool was conducted in Santarém Politecnis by Nathan Hunter for the academic public. The training was part of the Erasmus Plus project “Predict: jobs of the future with ai 4 vet inclusion” that intends to tackle current and upcoming challenges related to the next generation workforce, mainly those finishing VET programs or found in other similar training scenarios. After the training a survey was distributed around the participants. Despite the limitations posed by the incomplete responses, the case study offers findings that contribute to understand participants preferences. The survey methodology encountered challenges that led to its incomplete status. These challenges included limited respondent engagement and time constraints. Future research can build upon this study by conducting a more comprehensive survey to validate and expand upon the findings presented here. Keywords: Chat GPT · Education · Higher education · Digital Literacy

1 Introduction 1.1 Chat GPT Chat GPT (Generative Pre-trained Transformer) is an advanced language model based on the GPT architecture, which leverages deep learning techniques to generate coherent and contextually relevant responses in natural language conversations. GPT models [1] are trained on vast amounts of text data, allowing them to learn patterns, semantic relationships, and syntactic structures in language. It uses a transformer neural network architecture, known for its effectiveness in modeling sequential data. The model is pre-trained on a large corpus of text data using unsupervised learning, where it learns to predict the next word in a sentence given the previous context. During the pre-training phase, Chat GPT learns to understand the statistical properties of the language, including grammar, vocabulary, and contextual nuances. This process enables the model to acquire a broad understanding of diverse topics and exhibit fluency © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 695–705, 2024. https://doi.org/10.1007/978-3-031-47281-7_57

696

M. P. Barbas et al.

in generating coherent responses. However, it is important to note that Chat GPT does not possess true understanding or common-sense reasoning, as its knowledge is limited to what it has learned from the training data. After pre-training, Chat GPT can be fine-tuned on specific tasks or domains to enhance its performance in particular applications. Fine-tuning involves training the model on task-specific data or with human-generated dialogues, guiding it to generate more contextually appropriate and accurate responses in specific conversational scenarios. The deployment of Chat GPT in various applications, including education, involves leveraging its conversational capabilities to provide personalized learning experiences, tutoring, collaborative learning, and academic support: Chat GPT can facilitate and supports students in their educational journey. 1.2 The Role in Education ChatGPT serves as an advanced conversational agent capable of engaging in text-based interactions with students. Its application in education has garnered attention due to its potential to enhance personalized learning experiences, foster collaborative learning environments, provide academic support, and augment teacher roles. In terms of personalized learning [2], ChatGPT’s adaptive nature allows it to tailor its responses and explanations to meet the individual needs, learning styles, and paces of students. By addressing specific queries, providing targeted feedback, and offering customized learning resources, ChatGPT assists students in achieving optimal educational outcomes. As a virtual tutor or mentor, ChatGPT plays a significant role outside the traditional classroom setting. It acts as an intelligent conversational partner, responding to student inquiries, elucidating complex concepts, and offering additional practice exercises or supplementary learning materials. This fosters independent learning and empowers students to actively engage with the subject matter. Moreover, ChatGPT facilitates collaborative learning by simulating conversational interactions. Students can engage in dialogues with ChatGPT, stimulating critical thinking, promoting peer-to-peer discussion, and enabling exploration of diverse perspectives. This interactive nature of ChatGPT supports the development of collaborative skills and enhances the learning experience through social interaction. ChatGPT also serves as an invaluable resource for students struggling with homework assignments. It assists students by providing explanations, suggesting problem-solving approaches, and offering relevant examples. Through this assistance, ChatGPT aids in overcoming obstacles and consolidating understanding, thereby improving student performance and self-efficacy. Additionally, ChatGPT aids in research and information retrieval [3]. By suggesting relevant sources, summarizing complex topics, and assisting students in navigating vast amounts of information, it streamlines the research process and equips students with the necessary tools for effective information synthesis and analysis. While ChatGPT presents numerous advantages, it is crucial to acknowledge its limitations. It should not replace human interaction and guidance in the educational setting. Rather, it should be viewed as a complementary tool that augments the roles of teachers

The Importance of Chat GPT Training

697

and mentors. Furthermore, ethical considerations, such as data privacy and algorithmic biases, should be addressed to ensure the responsible and equitable use of ChatGPT in education.

2 Chat GPT Training The training of higher education professionals in the use of Chat GPT can bring several important benefits to the field of education [4]. Here are some reasons why it is valuable: 1. Pedagogical Enhancement: Training higher education professionals in Chat GPT equips them with a powerful tool to enhance their pedagogical practices. They can leverage Chat GPT to provide personalized feedback, offer additional learning resources, and engage in interactive conversations with students, thereby enriching the teaching and learning experience. 2. Scalable Support: Chat GPT can serve as a virtual assistant, enabling higher education professionals to provide on-demand support and guidance to many students simultaneously. Training educators in Chat GPT allows them to efficiently address common queries, offer instant feedback, and extend assistance beyond regular class hours, enhancing the scalability and accessibility of support services. 3. Efficient Assessment and Feedback: By training higher education professionals in Chat GPT, they can optimize assessment and feedback processes. The model can be leveraged to automate certain aspects of grading, such as multiple-choice assignments, freeing up educators’ time to focus on providing qualitative feedback and engaging in more in-depth discussions with students. 4. Innovative Pedagogies: Training higher education professionals in Chat GPT encourages the exploration of innovative pedagogical approaches that integrate technology and AI. This training enables educators to creatively leverage Chat GPT to engage students, foster critical thinking, and cultivate collaborative learning environments, leading to the advancement of educational practices. 5. Research and Development Contributions: Providing Chat GPT training to higher education professionals enables them to actively contribute to the ongoing research and development of AI applications in education. Educators can explore the potential of Chat GPT in addressing specific educational challenges, conduct empirical studies to evaluate its effectiveness, and contribute to the development of best practices and ethical guidelines for its use. 6. Technological Adaptability: By incorporating Chat GPT training, higher education professionals develop skills that enable them to adapt to technological advancements in AI and natural language processing. This prepares them to embrace future developments in educational technology and positions them as informed and adaptable professionals in an evolving educational landscape. However, it is important to acknowledge that while Chat GPT training offers numerous opportunities, it should be accompanied by critical AI literacy [4]. Higher education professionals must be knowledgeable about the limitations and ethical considerations associated with AI, ensuring responsible implementation, and addressing potential biases or ethical concerns that may arise.

698

M. P. Barbas et al.

Overall, the provision of Chat GPT training to higher education professionals empowers them to leverage AI capabilities effectively, enhancing teaching practices, fostering student engagement, and contributing to the advancement of the field through research and innovation. 2.1 How Can We Use It in Higher Education? Here are some examples of how Chat GPT can be used in higher education [5]: 1 Personalized Learning Support: By engaging in text-based conversations, the model can offer individualized explanations, answer questions, and provide tailored resources based on students’ unique needs and learning styles. 2. Virtual Tutoring and Mentoring: It can assist students in understanding complex concepts, guide them through problem-solving exercises, and offer feedback and suggestions to enhance their learning experience. 3. Collaborative Learning Facilitation: By simulating conversational interactions, the model can engage students in discussions, encourage peer-to-peer learning, and foster critical thinking and knowledge exchange within online learning environments. 4. Assistance with Research and Information Retrieval: Chat GPT can assist students and researchers in higher education by providing information retrieval support. 5. Exam Preparation and Study Guidance: The model can generate practice questions, provide sample answers, offer study tips and strategies, and assist in time management, helping students prepare effectively for assessments. These examples illustrate how Chat GPT can be employed in higher education to enhance teaching and learning, facilitate student support, promote collaborative learning environments, and provide valuable resources and guidance.

3 Chat GPT Training in the University Context Within the scope of the Erasmus plus project PREDICT: JOBS OF THE FUTURE WITH AI 4 VET INCLUSION the chatGPT training took place over two days during the month of May 2023 at the Polytechnic Institute of Santarem. As the project aims to provide solutions based on AI, showing that these can aid people in their skills acquisition by having tailored mechanisms which better understand their needs, progress, and curve of apprenticeship. Also wants to build new approaches and methods by defining holistic strategies that incorporate inclusiveness, lifelong learning, and other mentor-related aspects that are crucial for their career path. According to Eurostat (2021) [6], in 2020, 7% of enterprises in the EU with at least ten people employed used AI applications. If we analyze these numbers within this project, Portugal had 9% of AI used on its enterprises, followed by Greece with 7%, Estonia and Romania with 6%, and Cyprus with 3%. However, these numbers are still far from being sufficient and self-aware of how AI can benefit or contribute to higher autonomy, management, assessment, monitoring, and other aspects. Therefore, PREDICT aims to provide solutions based on AI, showing that these can aid people in their skills acquisition by having tailored mechanisms which better understand their

The Importance of Chat GPT Training

699

needs, progress, and curve of apprenticeship. PREDICT project team believes that AI can become an instrumental technology in tackling the new generation challenges in their work or future jobs. AI can also help people achieve higher performance levels within different roles or diverse teams, and this project wants to incorporate that vision in its activities and strategies. It is about turning technology into something more than innovation or disruptive methods. It is about people and Human capital and how these can benefit from technology. In this sense it was a theoretical training with about 50 participants from different scientific areas in the academic environment. Nathan Hunter, throughout the training days [7], explained how the GPT chat works and its integration in education for different target groups. 3.1 Survey At the end of the training, a survey was carried out among the participants to understand the real applicability and importance of the training days for their personal and professional context.

Fig. 1. Participants results for initial questions.

Surveys conducted at the end of training programs play a crucial role in assessing the effectiveness, impact, and overall quality of the training experience. Here are some key reasons why surveys are important at the end of training: • Evaluation of Learning Outcomes: allow trainers to evaluate the extent to which participants have achieved the intended learning outcomes. By gathering feedback on participants’ knowledge acquisition, skills development, and attitude changes, trainers can gauge the effectiveness of the training program in meeting its objectives.

700

M. P. Barbas et al.

• Feedback on Training Design and Delivery: an opportunity for participants to provide feedback on various aspects of the training, such as the content, structure, and delivery methods. This feedback helps trainers assess the strengths and weaknesses of the training program, identify areas for improvement, and make necessary adjustments for future iterations. • Participant Satisfaction and Engagement: surveys capture participants’ perceptions of their overall satisfaction with the training experience. This includes assessing the relevance and usefulness of the content, the clarity of instruction, the quality of training materials, and the level of engagement and interaction. • Identification of Knowledge Gaps: can identify areas where participants still have knowledge gaps or require further clarification. • Feedback for Continuous Improvement: by analyzing survey responses and identifying common themes or patterns, trainers can make data-informed decisions to refine training content, update instructional strategies, or implement changes that enhance the overall learning experience. 3.2 Data Collection The initial questions aimed to understand the general panorama of the participants and their knowledge about the GPT chat tool. Questions about the professional area and

Fig. 2. Participants results for second part questions.

The Importance of Chat GPT Training

701

gender were specific to the characterization of the academic target audience (Figs. 1 and 2). In a second phase of preliminary information for this survey, they were asked about the chatGPT: its definition, its use, and the future intention of use. As we can see in the graphs below. The last part of the survey corresponded to open questions about understanding the concept of chatGPT after training; advantages and disadvantages of using chatGPt in the educational context and about the future of chatgpt integration in education. 3.3

Data collection: open-ended questions.

Open-ended questions in surveys provide respondents with the opportunity to provide detailed and personalized responses in their own words. By incorporating open-ended questions in surveys, researchers can gather rich, nuanced, and contextual data that complements quantitative information, allowing for a more comprehensive understanding of the research topic. Open-ended question 1: Following Nathan Hunter’s training, how do you define the concept of ChatGPT? (Random choose completed answers from the survey participants in English).

“ChatGPT is an AI language model developed by Open AI that knows how to talk and write like a human! It's been trained on tons of different texts so it can answer all sorts of questions and have conversations just like a real person. ChatGPT is good at understanding the meaning behind words and figuring out what people are asking for. It can do all sorts of cool things like help you find information or even just chat with you about whatever you want. It's not perfect, but it's a huge step forward in making computers easier to talk to and work with”. “The GPT chat - Linguistic Capacity is an intelligence that, depending on what we search for and research, "understands" the next word in a way that facilitates the search. Compared to "SIRI" or Alexa, the difference is that the GPT chat tries to be more human in its mode of interaction, since the other platforms tend to be colder and more mechanized.” “ChatGPT is an AI-powered language model that uses natural language processing (NLP) to generate human-like responses to user input. It can be trained; it processes the user input, and it generates a response using the patterns and structures it learned during training. A concern is that is more based in the English language and that will emphasize the anglophone world we're already living in, forgetting "minor" languages.” “The presentation of Nathan Hunter showed us in practical terms the importance and mode of operation of ChatGPT. This language model was presented to us in a very interesting and challenging way. The "ChatGPT" was presented to us as having been developed primarily to facilitate conversation and communication between humans and machines. Through the given examples we learned that ChatGPT refers to a sophisticated computer program developed by Open AI. We were given to understand how ChatGPT is able to understand and respond to natural language input from users and can generate fluent and coherent text. How can we obtain information through ChatGPT, answer questions. How the conversations between this model and the users are processed. This webinar turned out to be fantastic for the way the questions were asked and presented”.

Open-ended question 2: List 3 advantages and 3 disadvantages of using ChatGPT in an educational context. (Random choose completed answers from the survey participants in English).

702

M. P. Barbas et al. “3 advantages: -Available in open access and free version. -Very intuitive and easy to use. -Allows the user to refine their searches on the same topic, while maintaining contact with the history of what they have been searching. 3 disadvantages: -Currently, the temporal scope of the ChatGPT is limited to 2021, when the databases were last updated. This limitation can affect its knowledge base, as it may not have information on more recent events, such as the War in Ukraine. -Additionally, the ChatGPT may generate incorrect information in its searches, which is a significant limitation. As Nathan Hunter pointed out in his training, the ChatGPT essentially 'guesses' the next word. -Furthermore, the ChatGPT may be less inclusive for people with special educational needs, as it is not designed to 'speak' and may not be as accessible to those with visual impairments as it is to those without such difficulties”. "Advantages: Writing aid tool and ideas for writing texts, quick answers Quick user learning and adaptation. Disadvantages: Does not recognize links, may provide non-existing links in research. Allows students to copy work instead of studying first and use it as an aid to write a paper. Has limited knowledge after 2021.” “Advantages: Personalized Learning - can be used to create personalized learning experiences for students. Increased Engagement - can help increase student engagement. Availability - can be available 24/7, allowing students to get help. Disadvantages: Lack of Emotional Intelligence - lacks the emotional intelligence that is present in human-tohuman interactions. Misunderstandings - may not always understand the nuances of language. Dependence - If ChatGPT is used as the primary means of communication between students and teachers, it could lead to students becoming overly dependent on technology, and less able to develop interpersonal skills that are necessary in the workplace and in society”. “Advantages: ChatGPT can be used to provide personalised learning experiences and to adapt to student's unique needs accordingly. With ChatGPT, students can get instant feedback on their assignments, which can be a great motivator for them to keep learning and improving. Education can be more cost-effective than hiring multiple teachers to cater to different subjects. Disadvantages: Limited context since it may not always understand the context of a question or conversation and may provide irrelevant or incorrect responses. Lack of human interaction, as we are talking about education, having personal interaction is very important, and ChatGPT is a "software". The great confidence ChatGPT shows might lead to less motivation for learning. It can discourage students from seeking out other sources of knowledge and limit their ability to learn from different perspectives.”

Open-ended question 3: How do you foresee the integration of ChatGPT in higher education? (Random choose completed answers from the survey participants in English).

The Importance of Chat GPT Training

703

“I have been using it in classes, but I don't have yet an answer. I think it can be a useful tool. I think ChatGPT has a lot of potential to revolutionize higher education in the future. One way it could be integrated is by using it as a virtual teaching assistant that students can interact with to get help with assignments, study for exams, or even just have a conversation to deepen their understanding of a topic. Another way it could be used is by incorporating it into online courses to provide personalized learning experiences for students. ChatGPT could be used to generate customized quizzes, activities, and readings based on each student's unique learning style and progress. This could be especially useful in courses where there are many students and it's difficult for the instructor to provide individual attention to everyone. Of course, there are also some potential downsides to using ChatGPT in higher education as mentioned in the disadvantages. However, I think these risks can be mitigated by using ChatGPT in combination with traditional teaching methods and ensuring that students still have plenty of opportunities to interact with their peers and instructors in person”. “I think ChatGPT isn’t a replacement for human interaction or support (for human teachers, for example), but it’s a tool that can supplement and enhance the work of human educators. It’s extremely useful to get some information fast, to have a general idea about a subject. It’s easy to use and it’s understandable that more and more students will use it and misuse it. It’s helpful to translate specially to English (as I just did).” “Despite being such an amazing tool, the lack of references of the information given can be seen as a major issue, as the users cannot garantee the realiability of the source of the given information. However, it is amazing to be used to resume papers and other documents”. “I believe that ChatGPT has significant potential for integration into higher education in various ways, particularly in online and hybrid learning environments. Here are a few possible scenarios, tutoring and Student Support: ChatGPT could be used to provide personalized tutoring and support to students. The model could analyze the student's performance and learning history and offer targeted feedback and resources”.

3.4 Data Analysis and Limitations Open-ended questions allow respondents to express their thoughts, opinions, and experiences in their own words, providing rich and detailed insights. These responses can uncover nuances, perspectives, and aspects that may not have been captured by closed-ended or pre-defined response options. With it we enable researchers to gain a deeper understanding of respondents’ attitudes, beliefs, motivations, or experiences. By allowing individuals to freely express themselves, researchers can explore complex or unique perspectives, leading to a more comprehensive understanding of the subject matter. However, and due to limitation of inquires it was not revealing unexpected or unanticipated findings. Usually, by not limiting respondents to predefined response options, researchers may discover new themes, perspectives, or insights that were not originally considered, enriching the analysis, and generating new avenues for exploration. But it was not the case. And as researchers we felt that the use of open-ended questions allows respondents to provide feedback or raise issues that may not have been anticipated by the researcher. While open-ended questions can provide valuable insights and allow participants to express their thoughts freely, they also come with some limitations. Here are a few limitations of open-ended questions on this survey: 1 Subjectivity: Open-ended questions can elicit subjective responses that vary from person to person.

704

M. P. Barbas et al.

2 Time-consuming: Open-ended questions require respondents to think and write detailed responses, which can be time-consuming. This led to respondents providing brief or incomplete answers or opting to skip the question altogether. 3 Lack of standardized data: open-ended questions result in diverse and unstructured data. Analyzing and comparing responses across participants can be difficult due to the absence of a standardized format. 4 Reduced response rate: The open-ended format intimidates some respondents, leading to lower response rates. The findings offer valuable information for higher education community, emphasizing the need to consider the chat GPT as an important tool.

4 Conclusion In conclusion, the integration of Chat GPT training in higher education holds significant potential and benefits. This advanced technology can enhance the learning experience, support student engagement, and provide personalized assistance to learners. However, it is essential to consider certain aspects to ensure its effective implementation in higher education settings. Firstly, privacy and security concerns must be addressed. As Chat GPT relies on data input and analysis, safeguarding sensitive student information is crucial. Educational institutions need to prioritize data protection measures, including encryption, secure storage, and adherence to privacy regulations, to maintain student trust and confidentiality. Secondly, the training of Chat GPT models should focus on accuracy and reliability. While the models have demonstrated impressive language capabilities, they are not infallible and may produce inaccurate or biased responses. Proper training and continuous monitoring are necessary to minimize errors and ensure that the information provided is trustworthy and aligned with educational standards. Moreover, incorporating human oversight is essential. Chat GPT should not replace human interaction but rather augment it. Faculty members and instructors play a crucial role in providing guidance, expertise, and critical thinking skills that cannot be replicated by AI. Integrating Chat GPT as a supportive tool alongside human instructors can create a balanced learning environment that leverages the strengths of both AI and human interaction. Additionally, promoting digital literacy among students becomes paramount. As Chat GPT becomes more prevalent in higher education, students need to develop the skills to critically evaluate and assess AI-generated content. This includes understanding the limitations and potential biases of AI systems, as well as the ability to validate and corroborate information from multiple sources. Lastly, ongoing research and collaboration are essential to advance Chat GPT training in higher education. Continued exploration of AI’s impact on learning outcomes, pedagogical approaches, and student experiences will help refine and optimize its implementation. Collaboration between AI researchers, educators, and policymakers can contribute to ethical guidelines, best practices, and responsible AI usage in academia.

The Importance of Chat GPT Training

705

In summary, Chat GPT training has the potential to revolutionize higher education by providing personalized, on-demand assistance to students [8]. While challenges such as privacy, accuracy, human oversight, digital literacy [9], and ongoing research need to be addressed, when implemented thoughtfully, Chat GPT can enhance learning, promote engagement, and support students in their educational journeys. By embracing AI as a valuable tool, higher education institutions can adapt to the evolving educational landscape and empower students to thrive in the digital era.

References 1. What is CHATGPT and why does it matter? here’s what you need to know (no date) ZDNET. Available at: https://www.zdnet.com/article/what-is-chatgpt-and-why-does-it-mat ter-heres-everything-you-need-to-know/ (Accessed: 23 May 2023) 2. Biswas, Som, Role of Chat GPT in Education (February 25, 2023). Available at SSRN: https:// ssrn.com/abstract 3. Halaweh M (2023) ChatGPT in education: Strategies for responsible implementation. Contemp Educ Technol 15(2):ep421. https://doi.org/10.30935/cedtech/13036 4. Gozalo-Brizuela, R., & Garrido-Merchan, E. C. (2023). ChatGPT is not all you need. A State of the Art Review of large Generative AI models. https://doi.org/10.48550/ARXIV.2301. 04655 5. OpenLibrary.org (no date) The Art of Prompt Engineering with CHATGPT (Jan 19, 2023 edition), Open Library. Available at https://openlibrary.org/books/OL47308671M/The_Art_ of_Prompt_Engineering_with_chatGPT Accessed: 23 May 2023 6. Eurostat (n.d.). https://ec.europa.eu/eurostat/databrowser/view/UNE_RT_M__custom_376 1903/bookmark/table?lang=en&bookmarkId=e8fdfad7-74a2-4074-99f1-d92cb0583cf5 Accessed 23 May 2023 7. Politécnico de Santarém promove webinars em Março, abril e maio (2023) Politécnico de Santarém. Available at: https://www.ipsantarem.pt/politecnico-de-santarem-promove-webinarsem-marco-abril-e-maio-2/ Accessed 23 May 2023 8. Thorp HH (2023) ChatGPT is fun, but not an author. Science 379(6630):313. https://doi.org/ 10.1126/science.adg7879 9. Schneider, E. F., Langenbahn, C.,S Holländer, M. Teaching ethical AI design in higher education: From Objectives to Competence Development. In European Conference on Technology Enhanced Learning, pp 363–377 (2021)

Experiencing Design Fieldwork: Walking Interviews on Madeira Island Vanessa De Luca1(B)

and Elisa Bertolotti2

1 SUPSI—University of Applied Sciences and Arts of Southern Switzerland,

6850 Mendrisio, Switzerland [email protected] 2 Department of Art & Design; ID+; ITI/Larsys (UIDB/50009/2020), UMA—University of Madeira, 9020-105 Funchal, Portugal [email protected]

Abstract. This article invites reflections on the design fieldwork practice as a process for connecting, relating and understanding complex social, natural, cultural and political systems, non-linear and interdependent visions about change at the root of many ecological issues. Through walking and talking with natural scientists in Madeira Island, we conducted a place-based inquiry that aimed to understand site-specific problems and opportunities to recount biological and cultural aspects of its peculiar forests. We used a fieldwork approach as embodied contextual analysis for amplifying the unique aspects, connecting ecological controversies, and bridging local efforts that cross both natural science and design. This contribution wants to present the place-based research project we conducted and specifically the walking interview process method we used to outline collaborative visions of development. Keywords: Design fieldwork · Walking interview · Place-based inquiry · Biocentric approach · Ecocultural design

1 Introduction One of the great contemporary challenges is to find ways to overcome the hyperspecialization of knowledge [17, 18]. Faced with the complex problems of the contemporary world and their interconnections, it is essential to overcome the gaps between areas of knowledge, to build a capacity for critical and imaginative thinking, and when designing, to take into account a posthuman horizon [3]. In this sense, the role of mediation [7] and dissemination [4] becomes fundamental to bridge design, science and society, as well as to promote biocentric approaches and more place-based and participatory processes. Wahl [27] argues that a sustainable solution is not enough to facilitate long term ecologies; “We need regenerative cultures that will emerge out of finding new ways of relating to self, community and to life as a whole”. But, how can we practically facilitate more cooperative and systemic processes? Taking in mind the potentiality of design of bridge-making between different disciplines and areas of knowledge, we developed an experimental interview framework © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 706–718, 2024. https://doi.org/10.1007/978-3-031-47281-7_58

Experiencing Design Fieldwork: Walking Interviews on Madeira Island

707

in order to deepen the learning experience of two summer schools that took place in Madeira Island in 2018 and 2019 [2]. Opposed to imposing pre-planned analysis, the interview framework was built as a data collection method for design research. Because walking is practised and experienced in innumerable contexts, generalizations are problematic [8]. However, this contribution is not aimed at defining a fixed protocol; instead, it attempts to present a set of aspects we considered in conducting our design analysis. For a comprehensive analysis linking cultural, social, natural and political systems, the research was not bounded to any existing digital tool but instead we only used a tracking App to record routes and timing and a generic voice recorder. In this article, we outlines practical challenges and understandings for a digital design research advancement.

2 Design Fieldwork Through Walking Interviews Walking has always been a way in which humankind has learned to know the world [14, 23]. The practice of walking as a way to explore territories, landscapes and learn about communities has been adopted by geographers [21], artists, architects and urban planners [5, 6, 24], sociologists and anthropologists [12]. However, communicating the experiences of walking is not easy, which is a result of the rich embodied experience of walking. Translating the experience would be a matter of keeping in mind and recounting the materiality of space and its effects on the body in relationship between the walker and the external world [8]. Dealing with the challenge of translating these interactions and complexities has taken many forms, which depend on research, educational, political and communication aims, as well as poetic questions [22]. Over the last few years, there has been a growing number of research situations where designers have walked with scientists and other contributors [2, 11]. These are intended to create opportunities for dialogue between experts from different sectors walking and talking together. The walking interview studies the subject in motion rather than taking it out of its everyday context [13] exploring the link between self and place [15]. The technique used recalls the qualitative research method within the mobility paradigm used in anthropological fieldwork. These experiences show that, through walking, designers can investigate and analyze how a space has been used and experienced; it is a way of getting to know a site and gathering qualitative data: memories, stories, impressions and interactions, taking photographs, recording maps, sounds and drawings. Among the design literature, the walking interview method have the specific purpose of connecting what participants say with where they say it, encouraging spontaneous conversation and a participant driven approach [9]. Other reflections have been addressed to the documentation of the work, thinking about a process that begins long before going into the field and potentially continues, in its elaboration, in an infinite way [19, 26]. There are some experiences in the field of design, which, while not focused strictly on walking, try to find poetic ways to tell the journey of designers who relate to nature, such as The Sea as Craftsman by Bruno Munari [20] and Objects of the Forest by Andrea Bandoni [1]. These small books suggest that there may be a way of communicating fieldwork that blends with aesthetic and poetic research. The fieldwork that particularly interested us has to do with the idea of being in the field with experts in the places where

708

V. De Luca and E. Bertolotti

they conduct research. In this case, we decided to do fieldwork while walking, and focus here on some of the interviews carried out during these walks.

3 Learning by Walking in Madeira This series of walking interviews was conceived on the occasion of a one and a half month research period in Madeira, between May and June 2021, where the authors of this essay found themselves collaborating, drawing on the experiences of the Atlantic summer schools Wonder (AW) also organized on the island in 2018 and 2019. Both summer schools were based on immersive experiences of exploring the territory through walking. These walks had the characteristic of being moments of dialogue and learning in the open air, where local natural science experts met with design historians, illustrators, designers, researchers and design students, ecologists. Two years later, the authors therefore, who had organized and participated in both summer schools, still in the midst of the pandemic, decided to take up some of the themes that emerged during AW’s travel moments, and to structure a series of interviews to be able record and share the reflections and insights with a wider audience, than just participants in the summer schools. The interviews were part of our shared study on Biocultural Design with the intention to explore methods of bringing together aspects of biology, culture and design such as new and traditional knowledge, cultural heritage and territorial resources, collaborative and social design practices around nature-related topics. We interviewed experts from different disciplines (biology, conservation ecology, environmental science, entomology) widely related to natural science. We observed how they approach the natural environment and phenomena, and we reflected together on specific topics: the fieldwork activity and its documentation; the conservation and valorization of ecosystems; the communication process; and collaboration beyond individual species. 3.1 Method The interviews created a relaxed situation for a more accurate observation of the link between the scientists and the outdoor environment. The exploratory talks were iterative and place-based, in 3 cases on the move and sedentary and semi-structured in another 3 cases. We engaged a small community of experts familiar with the endemic environment of Madeira in both cases. Some of the local natural science experts participated in AW summer schools. The other interviewees were as well local experts, known through the organization of fellow research activities. A. Researcher in Microbiology, 2021/05/9-23 walking interview at Levada dos Tornos and Caldeirão Verde B. Prof. of Entomology, 2021/05/17 videocall interview C. Researcher in Marine Biodiversity, 2021/05/27 videocall interview D. Researcher in Conservation Ecology, 2021/05/28 walking interview at Funchal Ecological Park. E. Researcher in Biology, 2021/06/01 walking interview at Ribera da Janela

Experiencing Design Fieldwork: Walking Interviews on Madeira Island

709

F. Biologist Technician, 2021/06/02 Interview at the UMa crop seed bank. During the walking interviews we produced about 4 h of stories per interview compared to 1.5 h for each sedentary interview. Informed consent and the protection of privacy and policy were established at the beginning. The walking we experienced has been essentially a hybrid of participants’ observations about their fieldwork practice with questions along the way. All interviews were recorded and transcribed. The recording methods used were audio recording, photographs, and a GPS mapping tool (Kamoot) to record the routes, places of interest that were significant to the interviewees, and the timing of the conversations [Fig. 1].

Fig. 1. Spatial transcript of the Ribera de Janela’s walk.

710

V. De Luca and E. Bertolotti

The walking interviewees were invited to choose the route they thought appropriate to introduce us to Madeira Island. Talking naturally alternated with pauses that were stimulated by the place and the walk itself. Both settings followed the same pattern of questioning: the first part was addressed to learn more about the experts’ viewpoint of the island through their research, practice and connection with it. We then asked their opinion as to what areas of Madeira need particular intervention for environmental protection. It was here that the interviewees produced their interesting stories associated with the island’s history. A second part centred on their fieldwork activity, tools and direct inquiry methods. We then explored the role of communication in their work and its dissemination: our questions tried to understand what limitations and motivations arose in these activities. The last part explored the interviewees’ visions about collaborations between disciplines and experts. What were their experiences and requirements in an effort to understand what might be the overlaps? Later, possible collaboration scenarios between design and their own disciplines were explored. This was also done through the final set of questions, which dealt with current and previous examples of good practices and ideas based on their observations about what nature can teach us. 3.2 Design Notes for a Walking Interview Intentions and goals of our research have been shared at the first contact with interviewees to engage them in the discussion on the Island’s ecologies. Given this context, the convergence of environmental and cultural aspects recurred many times in the conversations and we recast the question of how design thinking can support relations and regeneration of a new ecology of places. In the following Table 1, we propose a set of criteria we used to implement our walking interviews in Madeira Island. Table 1. Criteria to design the walking interviews. Aspects

Design Approach

Location

Chosen by the interviewee. Familiar to the interviewee

Weather

Check the forecasts and follow safety precautions

Duration

Do not rush the conversation. Take as much time as the walk takes

Rhythm

Follow the interviewee’s rhythm. Observe and track the start, end and mid points. Regard stops as observation opportunities

Structure

Describe yourself and explain the objectives and outcomes of the interview. Let the interviewees introduce the place and route. Reframe the orders of arguments based on the conversation’s flow. After the walk take notes and return them for the approval of the interviewee (continued)

Experiencing Design Fieldwork: Walking Interviews on Madeira Island

711

Table 1. (continued) Aspects

Design Approach

Ethics

Ensure participants know they are being observed and recorded. Obtain their consent. Create a confidentiality agreement. Understand sensitive information and privacy issues

Path

Look for non-human presence and open the senses to listen, smell, feel (as well as see). Notice human interventions along the way

Equipment Choose appropriate clothing (water- and wind-resistant). Tracking and recording tools for documentation. Additional batteries (if needed), torch, fully charged phone. Free up your hands as much as you can (paper or pen should be avoided) Safety

Familiarize yourself with the areas. Listen to your body and be aware of your physical preparation. Walking is not for everybody. Communicate if you have any physical impairment

Gaining access to the expert’s practice is not a given: it has to be negotiated and it takes time and patience to build trust. We practiced detachment as an objective attitude by avoiding judgement, selfishness or personal influence to the conversations giving the time needed to the interviewee to elaborate and express what they have in mind. Our attitude privileged curiosity as tool to learn from other perspectives. In the Madeira expeditions we applied the same criteria to different walks, thus avoiding arbitrary changes and refining a rigorously tried-and-tested method. Both walking and static formats were useful in reading the multiplicity of the place, but while the ethical aspects, structure and approach were common, other elements were peculiar to the walking method needs: equipment, path, duration, rhythm. Confidentiality is at the forefront and cannot always be assured in outdoor spaces. We informed the participants of the concerns about privacy and we asked for their agreement to create a safe environment. If something seemed to be of a sensitive nature, we asked permission before publishing and if any controversies emerged we opened the discussion in order to understand potential shifts and bridges. As a first attempt, we tracked our walk with the mobile phone and we took photos during the stops along the routes. This narrative representation of walks needs further development to integrate the different levels of the place-based experience as symbolic transformation of the territory [5]. The duration varied in any walk; stops, trail length, altitude, weather and rhythm all influenced the time taken. We learned not just from the conversations but also by observing the interviewees: the positions of their bodies; the tools they brought; the motivations behind their relationship with the research; and the convergences with other disciplines and networks that emerged by combining the interaction in the field and the talking itself.

712

V. De Luca and E. Bertolotti

4 Specificities of Walking Interviews During the walk, we used the same structure, always attentive to covering all the questions and ready to jump from one position to another and prioritizing the continuum of the conversation. Instead of pen and paper, we recorded time, tracks, audio and photos with portable tools and GPS, keeping hands and eyes free to focus our attention on the surroundings. Although the same topics were touched upon in both the static and walking interviews, the kind of concentration voiced in the answers was very different, with the static ones being more precise, synthetic and linear, while the walking interviews were longer, more fragmented and repetitive [7]. The walking interviews gave us many insights, but have been more difficult to use for publications than the static interviews. The walks lasted from 4 to 8 h, and although we recorded everything, the answers came at different times, in a type of discourse that was not synthetic but linked to experiencing a place by walking together, and observing together. In the walking interviews we had to balance between our plans and the unexpected: faced with an opportunity to observe or think about something that we encountered, we always decided to let the conversation take its own course. The walking interview has a number of practical and ethical considerations that must be addressed to ensure the safety of both the interviewee and the observer. In addition, walking also means having to concentrate on the act itself. Certain incidents conditioned the modalities of dialogue; for example, following a storm at the Levada dos Tornos walking was difficult in some sections. Understanding the physical preparation and the equipment needed for walking is fundamental. While planning the routes with the interviewees we checked the distances, expected times, and the degree of difficulty of the walks in order to be fully prepared and safe. Walking with others is also something to be considered carefully. Walking for the first time with other people may, in fact, hide unforeseen pitfalls. In our case, we always tried to adapt to the rhythms of the interviewees. At the same time, we were aware that adaptation to the walking habits of others could be tricky if not balanced by listening to the limits of our own body. We could see the importance of time management, which cannot be underestimated and goes hand in hand with interview management. We observed that in the case of our longer walks there were moments where we only walked. This was the case, for example, of the Caldeirão Verde walk, where in the second part of the route it was more about walking than talking because it was necessary to return before dark [Fig. 2].

5 Finding and Results Although comparisons and transcriptions were useful for a first draft of the emerging framework, our findings were triangulated with the literature to see the interrelation of stories rather than isolated topics. The examples in the following paragraph, as a pivotal part of the research, define the emerging results of the interviews into categories.

Experiencing Design Fieldwork: Walking Interviews on Madeira Island

713

Fig. 2. Caldeirão Verde.

5.1 Redefining the Place Through the Interconnections Between Land Water Systems, Crops and the Ocean Walking was a chance to integrate in the narratives of the interviewees the biological aspects with cultural and social factors. We collected the data to redraw the socialecological system of Madeira. The discourse was mainly structured geographically rather than historically, through a dialectical process of conferring meaning to the signs along the path. For instance, the stationary windward belt of clouds up in the mountains is of vital importance to the forest as well as to humans because it creates drinkable water and a humid environment, which allows this ecological system to continue to exist. In the research, the unwritten story of the levadas (water canals) occurred multiple times throughout the walks and talks. To achieve the right conditions for development, the first colonists started a fire that burned for seven years, the island was uninhabited at that time, and a number of slaves were brought onto the island to build the canals and to work in the crop fields (int. A). Those crops (sugar cane, maize, vines, bananas, beans, potatoes and others), which are now monitored as Wild Crop Relatives, would change the social, economic and environmental history of the place (int. F). Conversely, while in walking interviews there were more interactions with narratives and stories around the physical environment, static interviews were rich with insights

714

V. De Luca and E. Bertolotti

regarding social and global issues. We perceived also better the interrelations between land, ocean and local inhabitants as well as site-specific directions coming towards talks and walks. For instance, the sea is considered as an ecological system but also a very important economy source of the island. Despite the sea’s low nutrients in Madeira’s coastal area, it has attracted and sustained the population with abundant open ocean fish (int. C). To maintain this ancient practice, imported fish are being introduced into the rivers, altering the local ecosystems by threatening the survival of native species (int. D). The local incentives for cultivation have also changed the vegetation, especially at low altitudes, thus weakening the island’s resilience to climate disasters. However, they have also enriched the knowledge and experience of crop varieties and agriculture (int. F). The thread expresses the importance of a public understanding of how a ‘small’ decision can influence the entire cycle of natural and cultural resources. For the designer, it can mean the important role of a deeper understanding of a context for an integrated progression. 5.2 The Island Status In all the interviews, the status of Madeira as an island was discussed. Many of the conversations were dedicated to those endemic species at risk of extinction due, above all, to climate change and to socio-cultural changes that have occurred on Madeira (De Luca & Bertolotti 2023). The walking gave boundless opportunities to identify actual examples that made the interviewees’ descriptions evident and concrete. For example, we encountered many invasive botanical species such as the Medlars of Japan (Eriobotrya japonica Lindl.)—souvenirs of visiting walkers, or the Morning Glory (Ipomoea) that had escaped from people’s gardens and were now growing in the forest to the detriment of native species (int. E) [Fig. 3]; in the Funchal Ecological Park we found a series of non-native insects that are supplanting native insects; at Levada dos Tornos we observed a series of civil engineering works and how they are altering the landscape [Fig. 4].

Fig. 3. Funchal Ecological Park. Maravilha Butterfly—Clouded Yellow.

Experiencing Design Fieldwork: Walking Interviews on Madeira Island

715

Fig. 4. Levada dos Tornos. The levada is on the edge between the city of Funchal and the mountainous areas that have recently burned down. Here one of the streams passes, which with torrential rains in the recent past caused serious flooding in the city.

We perceived that the interviewees themselves, as inhabitants, sometimes also felt the unicity of the island’s endemism as a peculiar aspect to consider. In other walks, there was a need to create roots in the island through their activities—stewardship and educational practices addressed to the wider public, or by the recreation of laurisilva gardens in urban areas. In general, we observed the sense of co-responsibility of scientists as inhabitants of the island, who reframe—within their own paradigms—the interspecies conversation. 5.3 Communicating How Everything Is Interconnected All the interviewees highlighted how interconnections at the environmental level are not easy to study or understand. There are multiple dimensions connected to our life, especially if we study the impact of global climate change on the vulnerable situation of an island (int. B). For this reason, they stated the importance of collaboration between scientists from different disciplines, not only locally but also internationally. The ‘connections’ are part of Madeira’s historical development. The economic and political choices of colonization have had a profound influence on how the habitants live on the island, and their life is a part of the many ‘designs’ that have changed the environment and the landscape [10]. The daily reality of tourism, agriculture and development is mediated by economic discussion. In such complexity, all the interviewees stated the importance of scientific dissemination, particularly because it contributes to building social conversations on environmental issues. At the same time, they experienced the challenge of this activity, particularly when it is destined for a public of non-experts. In their eyes, this challenge has to do with the difficulty of relating individual realms

716

V. De Luca and E. Bertolotti

to global systems and of visualizing the interconnections between ecological and social aspects. The act of walking allowed participants to recall memories and stories about plants and related historical events and cultural traditions. How to document this knowledge entangled with the place is an ongoing process of investigation. By walking collaboratively with scientists, we learned to explore novel ways of linking micro and macro systems and to question the human entwining with species and systems. Furthermore, the experience stimulated practical opportunities to collaboratively answer key ecological questions.

6 Lesson Learned As mentioned before, walking allowed us the chance to explore Madeira’s communities, territories and landscapes through the encounter of physical and shared actions that became an opportunity for conversation. Faced with concrete examples, ideas became less abstract for us. Despite having to keep in mind the invisible thread of the questions to be asked during the interviewing and documenting, we acknowledged the subtle balance between planning and improvising. The walking interviews allowed us to collect the subjective stories of the interviewees in fragments starting from the places travelled together while walking. The collected stories are thus intertwined with the landscape and local landmarks. By walking collaboratively with scientists, we learned to explore novel ways of linking micro and macro systems and to question the human entwining with species and systems. Furthermore, the experience stimulated practical opportunities to collaboratively answer key ecological questions. These walking interviews therefore allowed us to go deeper into ecological issues related to design. It is a learning process situated in a territory, that informed us about the type of interconnections, making the understanding of the island of Madeira more complex. This has allowed us, above all, to collect a series of reflections and intuitions that we have been able to use in our daily practice as educators, practitioners and researchers in the field of design. This learning process has its own slowness and that has forced us to negotiate moments of meeting and exchange with many local experts: we find that it is particularly important as we are not natives of the area, precisely to avoid our too superficial gaze and to automatically propose what we have learned elsewhere here. This has allowed us, above all, to collect a series of reflections and intuitions that we have been able to use in our daily practice as educators, practitioners and researchers in the field of design. In the following years, we have in fact been able to observe how this research, as well as the previous experiences of AW, have informed many of our projects, making them more complex and imagining unprecedented collaborations at a local level. This is the case for example, of Mãos na Terra [16], the community horticultural garden built at the University of Madeira, where ecologists, agronomists and designers collaborate, and which is linked to the experience of the crop seed bank [25]. In this last case, the effort is to build a community open to the neighborhood and foster a social conversation on ecological topics, ranging from the importance of maintaining biodiversity

Experiencing Design Fieldwork: Walking Interviews on Madeira Island

717

and agro-biodiversity, supporting pollinators, imagining how to make campus activities more sustainable, for example creating compost from canteen waste. These experiences have allowed to develop relationships with other departments of the university, from economics to agronomy, and to build proposals for funded research projects. Moreover, building experimental didactic experiences with students of the bachelor in design, where we have experimented on several occasions how field work can take place for a designer, always integrating visits, invited guests from other areas of knowledge, and moments of exploration through the path of different local territories.

7 Conclusion This research suggests that there is space for alternative intersections between design and natural sciences within environmental issues. Precisely in this urgent need to foster greater collaboration between disciplines and awareness of environmental issues, we hypothesize that designers can create bridges between disciplines, perform mediation work, and encourage social conversation. Having conversations when walking in nature enabled us, as designers, to collect the complexity of insights about people, cultures, other species and their interactions, and suggested relevant indicators to frame the contextual analysis. After this series of interviews, we think it would be useful to continue with walking interviews, and to have more experience and insight of the method and process. We hope that this way of carrying out walking interviews will be extended for resulting in more literature on the subject. The walking interview offer opportunities to connect different languages by facilitating the immersion into local voices and communities. This work that we share here provide an invitation for designers and researchers in design to incorporate walking interview in their project as a way to learn in an embodied, situated way. Acknowledgements. Gratitude for the participants who offered us incredible inspiring learning opportunities through conversations and stories that are woven within this work: Graça Costa, Dora Aguin Pombo, Ricardo Araújo, Richardo Rocha, Susana Fontinha, Humberto Nóbrega. We would thank the Institute of Design (SUPSI) and the University of Madeira (UIDB/50009/2020) for their support of the authors’ project, and the ISOPlexis Lab, the researchers of the Museu de História Natural do Funchal and the Jardim Botânico da Madeira for the relevance of their work that inspired us.

References 1. Bandoni A (2012) Objetos da Floresta. Explorando a Amazônia através do olhar de designers. [Objects of the Forest. Exploring the Amazon through designers’ eyes]. Andrea Bandoni de Oliveira, São Paulo 2. Bertolotti E, Vezzani V (2021) Walk, talk and draw: fieldwork in nature to unpack the complexities of sustainability. In: Cruickshank L et al Safe Harbours for Design Research. 14th International Conference of the European Academy of Design. Blucher, Lancaster 3. Braidotti R (2019) Posthuman Knowledge. Polity Press, Cambridge 4. Bruni R (2018) Mirabilia. La Botanica nascosta nell’arte. Codice Edizioni

718

V. De Luca and E. Bertolotti

5. Careri F (2006) Walkscapes. Einaudi, Camminare come pratica estetica 6. Davila T (2002) Marcher, Créer. Déplacements, flâneries, dérives dans l’art de la fin du XX siècle. Regard, Paris 7. De Luca V, Bertolotti E (2023) Local voices: conversations with ‘insiders’ engaged with the natural science of Madeira Island. In: Bertolotti E, Gonzaga S, Vezzani V (ed) Routes and Roots on the Island of Madeira: Design Practices for Learning through Nature’. UA Editora, Aveiro 8. Edensor T (2016) Walking through ruins. In Ingold T, Vergunst JL (ed) Ways of Walking. Ethnography and Practice on Foot. Ashgate Publishing, Farnham 9. Evans J, Jones P (2011) The walking interview: methodology, mobility and place. Appl Geogr 31(2):849–858 10. Fuad-Luke A (2023) Moving beyond the violence of the Capitalocene and Plantationocene: design lessons from Madeira. In: Bertolotti E, Gonzaga S, Vezzani V (ed) Routes and Roots on the Island of Madeira: Design Practices for Learning through Nature. UA Editora, Aveiro 11. Gatto G, Cadamuro A (2021) No man’s land/everyone’s land: designing processes for enacting more-than-human worlds. The case of Bagnoli. In: 14th International Conference of the European Academy of Design. Safe Harbours for Design Research. Blucher, Lancaster 12. Ingold T, Vergunst JL (2008) Ways of Walking Ethnography and Practice on Foot. Ashgate Publishing, Farnham 13. Jones P, Bunce G, Evans J, Gibbs H, Hein JR (2008) Exploring space and place with walking interviews. J Res Pract 4(2):D2 14. Kagge E (2019) Walking: One Step at a Time. Knopf Doubleday, New York 15. Kinney P (2017) Walking interviews. Soc Res Update 67:1–4 16. Mãos na Terra. https://www.instagram.com/maosnaterrauma. Accessed 28 May 2023 17. Morin E (1999) Seven complex lessons in education for the future. UNESCO. https://une sdoc.unesco.org/ark:/48223/pf0000117740. Accessed 28 May 2023 18. Morin E (2017) Connaissance, Ignorance, Mystère. Fayard, Paris 19. Müller F (2021) Methods and aspects of field research. In: Müller F (ed) Design Ethnography. SpringerBriefs in Anthropology. Springer, Cham, pp 31–76. https://doi.org/10.1007/978-3030-60396-0_5 20. Munari B (1995) The Sea as a Craftsman. Corraini, Mantova 21. Pierce J, Lawhon M (2015) Walking as Method: Toward Methodological Forthrightness and Comparability in Urban Geographical Research. https://link.springer.com/chapter/10.1007/ 978-3-030-60396-0_5 22. Radicchi A (2017) A Pocket Guide to Soundwalking. Some introductory notes on its origin, established methods and four experimental variations. Stadtökonomie—Blickwinkel und Perspektiven Ein Gemischtwarenladen Perspectives on Urban Economics A General Merchandise Store Eine kleine Buchführung für den Ladeninhaber Dietrich Henckel a Brief Overview of the Accounts for the Shopkeeper Dietrich Henckel, pp 70–73 23. Solnit R (2001) Wanderlust. The History of Walking. Penguin, New York 24. Thompson N (2012) Living as Forms. Socially Engaged Art from 1991–2011. MIT Press, Cambridge 25. UMa crop seed bank. https://isoplexis.uma.pt. Accessed 28 May 2023 26. Van Maanen J (2011) Tales of the Field. On Writing Ethnography. The University of Chicago Press, Chicago 27. Wahl DC (2016) Designing Regenerative Cultures. Triarchy Press, Axminster

Wayfinding Signs in Cultural Tourism Area Based on Analyzing Songdu Huangcheng Cultural Tourism Resort Yang Ting1 and Albert Young Choi2(B) 1 Hanyang University ERICA, Sangnok-Gu, 15588 Ansan-Si, South Korea

[email protected]

2 Correspondence, Hanyang University ERICA, Sangnok-gu, 15588 Ansan-si, South Korea

[email protected]

Abstract. As the economy and culture continue to evolve, the fusion of culture and tourism has given rise to innovative cultural experiential destinations, offering consumers abundant cultural contact and aesthetic experiences. This study centers on the Songdu Huangcheng Cultural Tourism Resort as its focal point and seeks to delve into the concept of regional culture, propose a structural framework for regional culture, and explore the relationship between regional culture and guide. Using the methods of K-means clustering for color extraction, CIE 1931 XYZ color difference calculations, as well as observational and graphical techniques, this paper analyzes the regional culture of cultural tourism areas in different places and the color, graphics, and typeface of the wayfinding signs, and draws a conclusion that the wayfinding signs show the law of regional culture mainly through material culture and spiritual culture. Subsequently, this framework is applied to the analysis of guide signage within the Song Dynasty Imperial City Tourist Resort, considering the distinctive regional cultural characteristics of Kaifeng city, recommendations for improvements to the guide signage are presented. Through the optimization of wayfinding sign designs, regional cultural characteristics can be better conveyed, and tourists’ guiding experience and cultural awareness can be enhanced. This endeavor holds profound implications for the sustainable development of tourist resorts and serves as a valuable reference for other comparable cultural tourism destinations. Keywords: Regional Culture · Wayfinding Signs · Cultural Tourism Area

1 Introduction 1.1 Research Background and Purpose In recent years, the integrated development of culture and tourism has progressively shifted from a “wider scope” to a “deeper level” and “higher quality” [1]. In the process of urban development, cities increasingly emphasize the integration of regional cultural elements into their promotion. However, with the continuous progress of society and the growing interconnectedness between cities, the manifestation of regional cultural © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 719–731, 2024. https://doi.org/10.1007/978-3-031-47281-7_59

720

Y. Ting and A. Y. Choi

characteristics in daily life has gradually diminished. Cultural area blocks are important windows for the city to display regional culture to the outside world, and urban wayfinding systems possess historical and public characteristics, making them indispensable carriers for the inheritance of regional culture within cultural and tourism districts [2]. The research aims: Firstly, to gain an in-depth understanding of the concepts of regional culture and cultural and tourism districts and attempt to propose a structural framework for regional culture. Secondly, to use the proposed regional culture structure to systematically study and organize the regional cultural elements of Kaifeng City and the Songdu Huangcheng Cultural Tourism Resort. Thirdly, through case analysis, summarize the regional culture in the guide board performance rules. Finally, to apply these rules to the analysis of wayfinding sign designs in the Songdu Huangcheng Cultural Tourism Resort and provide improvement recommendations. It is hoped that through this research, it will provide references and insights for the construction and development of cultural tourism areas, promoting the sustainable development of urban culture. 1.2 Research Scope and Method The research focuses on the Songdu Huangcheng Cultural Tourism Resort in Kaifeng, China, with the aim of delving deep into the regional culture of this scenic area and analyzing the connection between the resort’s wayfinding sign and regional culture. This study will be based on existing literature, books, news reports, and online magazine materials. It will begin with a comprehensive review and analysis of regional culture. Investigate the regional culture and wayfinding signs of Songdu Huangcheng Cultural Tourism Resort on site and online. Online interviews with design companies responsible for the resort’s wayfinding system will be conducted to understand their design ideas. When comparing and analyzing cases of regional culture and wayfinding sign in different cities, observation and drawing methods will be employed to extract graphic elements from each case. Additionally, in conjunction with the K-means clustering method for color extraction, image pixels will be clustered, and the proportions analyzed to effectively obtain RGB values and proportions of the primary colors. Using the CIE 1931 XYZ color difference calculation method, RGB color values will be transformed into CIE 1931 XYZ color values, and color differences in the XYZ color space will be calculated to quantitatively analyze the similarity between regional cultural colors and wayfinding colors. According to the application of regional cultural elements in the design of the signage and sort out the regular conclusion. In the end, findings drawn from different cases will guide the evaluation of wayfinding sign design in the Songdu Huangcheng Cultural Tourism Resort, with the hope that these research findings will provide valuable insights and support for future wayfinding developments in scenic areas.

2 Research Study 2.1 The Connotation of Regional Culture The concept of culture has ancient origins, with its roots traced back to Cicero, an ancient Roman philosopher who initially used the Latin term “cultura animi,” signifying the “cultivation of the soul.” British anthropologist Edward Tylor was the first to provide

Wayfinding Signs in Cultural Tourism Area

721

a comprehensive and precise definition of “culture.” He stated that culture encompasses a complex whole, including knowledge, beliefs, art, law, morality, customs, and the acquired abilities and habits of a member of society [3]. Mr. Pang Pu, a Chinese cultural historian, proposed a three-tiered theory of culture in his essay “Cultural Structure and Modern China.” This theory divides culture into three levels: material, institutional, and spiritual. Material culture represents the outermost layer of culture, the spiritual aspect is the deepest layer, and the intermediate part is the amalgamation of the mind and the material world, where all the thoughts inherent in external objects reside. Regional culture, also referred to as “area culture,” typically pertains to the cultural traits that have evolved over an extended period within specific geographical regions. These traits encompass various elements, including the landscape, architectural structures, infrastructure, and more [4]. South Korean scholar Shim Hyo-jin suggests that regional culture not only includes geographic and administrative regions but also extends to social and cultural dimensions like living spaces, emotional spaces, and economic spaces [5]. Furthermore, scholars contend that regional culture is a comprehensive area of study. It investigates various manifestations of human activities intertwined with natural landscapes, the geographical characteristics specific to cultural regions, the interplay between the environment and culture, pathways and trends of cultural dissemination, as well as human behavioral systems. This multifaceted exploration encompasses aspects such as folk traditions, economic structures, religious belief systems, literary and artistic expressions, and social organizations. This includes folk traditions, economic systems, religious belief systems, literature, arts, social organizations, and more [6]. Combining the content mentioned above, it becomes apparent that the geographical environment significantly shapes the development of culture. The division of regional culture is based not only on natural geography, but also on the culture formed in the region. Different classification methods lead to different cultural types, making regional culture both distinctive and representative. Based on the understanding of “culture” and “regional culture,” regional culture has been organized and presented in Table 1. Table 1. Regional culture category. Level I

Level 2

Regional Culture Natural geography Material culture

Level 3 Climate, geomorphology, hydrology, soil, vegetation, fauna Food, clothing, architecture, transportation, tools, etc

Institutional culture Ethics, morality, law, politics, economy, etc Spiritual culture

Values, language, religion, art, customs, etc

2.2 Cultural Tourism Areas The concept of “cultural tourism” was first introduced by American scholars McIntosh and Gebert in 1977 in their book “Tourism: Principles, Practices, Philosophies”.

722

Y. Ting and A. Y. Choi

According to the definition adopted by the UNWTO General Assembly, at its 22nd session (2017), Cultural tourism entails a type of tourism activity in which the visitor’s essential motivation is to learn, discover, experience, and consume the tangible and intangible cultural attractions/products in a tourism destination. These attractions/products relate to a set of distinctive material, intellectual, spiritual, and emotional features of a society that encompasses arts and architecture, historical and cultural heritage, culinary heritage, literature, music, creative industries and the living cultures with their lifestyles, value systems, beliefs, and traditions” [7]. On April 1, 2021, the Ministry of Culture and Tourism of the People’s Republic of China issued the “Classification of Tourism and Leisure Street Blocks” report, which stipulated that tourism leisure street blocks must meet three conditions: firstly, they must have cultural symbols that showcase the historical and cultural characteristics of the city and the street block; secondly, they should organize cultural resources, explore cultural features, and incorporate them into various aspects of leisure experiences; thirdly, have intangible cultural heritage display and activities [8]. Based on the aforementioned content, it can be concluded that a cultural tourism area is a tourist destination with regional characteristics created by integrating the unique regional cultural resources (tangible culture and intangible culture).

3 Wayfinding Signs in Cultural Tourism Areas 3.1 Elemental Analysis Tourist attraction wayfinding signs are tangible elements through which tourist destinations provide visitors with implicit services. The information they provide can deepen and strengthen tourists’ overall perception of the attraction [9]. Wayfinding signs in scenic spots should not only be highly universal, but also convey the regional characteristics and cultural connotation of the block. In this chapter, the corresponding relationship between regional culture and the color, typeface and graphics of the guide plate is analyzed. Color. Color can play a role in whether signs stand out from or harmonize with the sign environment [10]. As a wayfinding tool, color can define various areas within a space or serve as a basis for understanding the organization of facilities [11]. It can also create a brand or identity, attract attention, influence emotions [12], and helps build brand awareness and status recognition [11]. In the scenic area environment, the pursuit of sign color is the effect of integration with the environment, and it is advisable to limit the number of colors used. Typeface. When selecting appropriate fonts, three factors are typically considered: whether the font is formally appropriate for the wayfinding project, the font’s lifespan is related to its style, and whether the font is clear and easy to read [10]. Fonts can be categorized into four types: Serif, Sans-serif, Handwriting, and Decorative [13]. Among these, serif fonts exude a sense of tradition and offer good legibility at close distances. However, sans-serif fonts are more eye-catching from a distance and are better suited for modern wayfinding projects. In the case of fast-moving cars in outdoor Spaces, the recognition of using sans-serif fonts is higher. In outdoor spaces with fast-moving vehicular traffic, the legibility of sans-serif fonts is higher. In pedestrian wayfinding systems, the visual recognition distance for pedestrians ranges from 5 m to 10 m. For high-priority

Wayfinding Signs in Cultural Tourism Area

723

information, fonts with higher visual recognition are used, while secondary information employs serif fonts, known for their legibility at close distances. Tertiary information uses serif fonts with smaller font sizes. For mixed pedestrian and vehicular wayfinding systems, font design should accommodate reading distances of 8 to 10 m or more [14]. Typefaces belong to spiritual culture, but when analyzing the regional cultural categories of fonts in this case, I will classify fonts according to the cultural backgrounds to which different text carriers belong. Graphics. The organization of wayfinding graphics can be divided into two main aspects: the 3D form of the wayfinding and the surface patterns of the wayfinding. Among these, surface patterns of wayfinding can be categorized into two types: symbols and other graphic elements. Symbols refer to graphic elements that convey information through images, while other graphic elements encompass auxiliary and decorative graphics that do not serve communicative purposes. Incorporating the analysis of regional culture, a process of elevation and refinement through artistic techniques is applied. This not only establishes a certain similarity or connection between graphics and tangible objects but also gives rise to unique cultural visual symbols. Generally, simpler, and sleeker sign objects, the more contemporary the appearance of the hardware system [10].

3.2 Case Study To assess the performance of cultural and tourism districts on a global scale, selected scenic spots known for their strong cultural characteristics and high visibility for analysis. The Hollywood Walk of Fame, located in Los Angeles, USA, is a cultural and historical landmark. Kuanzhai Alley in Chengdu, China, represents the first courtyard-style experiential consumption street block in the country. The Historic Centre of Macao, located within China’s borders, is the largest and most well-preserved collection of buildings with a fusion of Eastern and Western styles. Keywords related to regional culture include decorative arts, the film and television industry, celebrity culture, dream-chasing culture, and diversity. Art Deco design, which has a strong connection to Hollywood history and culture [16]. As shown in Table 2, based on the calculated color differences between the extracted regional cultural colors and the wayfinding colors, it is observed that the wayfinding colors align with the colors associated with natural geography, spiritual culture, and material culture. The sign exclusively employs English sans-serif fonts, which are suitable for modern wayfinding projects but may not convey a distinct cultural direction. The wayfinding graphics feature intricate designs, evoking a sense of refined luxury associated with decorative art, thus reflecting aspects of spiritual culture. Keywords related to regional culture include teahouse culture, leisure culture, and agrarian culture. As shown in Table 3, based on the calculated color differences between the extracted regional cultural colors and the wayfinding colors, it is observed that the wayfinding colors align with the colors associated with natural geography and spiritual culture. Chinese characters employ the more traditional Songti font, with embellishments using the highly recognizable Xingshu script, while English text uses serif fonts to enhance the historical ambiance, aligning with the overall architectural atmosphere of the street block. Given that this is primarily a pedestrian-oriented area, the use of serif

724

Y. Ting and A. Y. Choi

Table 2. Analysis of regional culture and wayfinding signs in Hollywood Walk of Fame.

Regional culture Natural Geography

Wayfinding Signs

Material Culture

Institutional Spiritual Culture Culture

Hollywood Hills fonts, Serif, Sans serif

Public Sans, Calibri, Clearview Hwy, Serif, Sans serif

Broadway, Cooper Black, American Typewriter, Geometric line font [15]

HOLLYWO OD, radial, V-shaped, zigzag

Stepped, triangular, rectangular, semicircular

Round, star

Pedestrian Directional Signage and Map

Pedestrian Directional Signage

Parking Directional Signage

Picture

Color

Typeface Graph- Curves, water ic

lines, particles, sectors

Sans serif

Table 3. Analysis of regional culture and wayfinding signs in Kuanzhai Alley, Chongqing.

Regional culture Natural Geography

Wayfinding Signs

Material Culture

Institutional Culture

Spiritual Culture

Calligraphy (Wu Yipan), Song typeface, Serif, Sans serif

FangSong, SimHei(Gothic), KaiTi, Song typeface [17], Serif, Sansserif

Bashu script [18], Cloud seal script, Official script, Seal script

sickle-shaped, hoe-shaped

Bashu graphic language, Tai Chi, flower and grass patterns, ripples [18]

Intersection Intersection Public FaciliPedestrian Pedestrian ty Signage Signage Signage

Picture

Color

Typeface

Graph- Curve, Wateric

mark, Needle

"Fish spine" shape [19], square, arch, round

Song typeface, Semi-cursive Scrip, Serif

Wayfinding Signs in Cultural Tourism Area

725

fonts does not hinder content recognition. The grid elements in the wayfinding signs are inspired by the openwork window designs commonly found in classical garden architecture, reflecting elements of material culture. Table 4. Analysis of regional culture and wayfinding signs in Historic Center of Macau.

Regional culture Natural Geography

Wayfinding Signs

Material Culture

Institutional Culture

Spiritual Culture

Calligraphy font, Regular script, Thin font, Decorative font, Sans-serif, Serif

FangSong, SimHei(Gothic), KaiTi, Song typeface [17], Sans-serif, Serif

Suzhou numerals font, FangSong, KaiTi, Cursive Script [20], Sans-serif, Serif

Wavy line, arch, square

Lotus, stripes, stars, circles

Directional Signage

Street Name Signage

Description Plate

Picture

Color

Typeface

Curve, water Graph- pattern, triangular ic prism

SimHei (Gothic), Regular script, Sans-serif

Circle, water pattern, lotus, geometric shape, flowers, and plants pattern [21]

Keywords for regional culture include Mazu culture, Portuguese culture, maritime culture, diversity and inclusion, lion dance culture, and Cantonese opera. The institutional culture of Macau’s regional autonomy has facilitated the exchange and coexistence of both Eastern and Western cultures. In the urban landscape, three major languages, namely Chinese, English, and Portuguese, are prominent [22]. As shown in Table 4, based on color difference calculations, the green color used in directional signage corresponds to the colors of the Macau flag. Chinese characters are represented using both SimHei and Regular script, while Portuguese and English are displayed in sans-serif fonts, influenced by institutional culture. The design of street name signage in Macau’s historic district preserves the characteristics of Portuguese tile art in terms of color, patterns, and production methods. Through the analysis of the three cases mentioned above, the choice of colors is associated with all four cultural types, especially since all three cases combine spiritual culture and integrate another cultural type. Regarding font selection, it is influenced by environmental characteristics and policy regulations, corresponding to material culture and institutional culture. Graphics primarily draw inspiration from both spiritual culture and material cultures. Overall, the design elements of wayfinding signs have the strongest correlation with spiritual culture within regional culture, followed by material cultures.

726

Y. Ting and A. Y. Choi

4 Analysis of Regional Culture and Wayfinding Signs in Songdu Huangcheng Cultural Tourism Resort 4.1 Regional Culture Analysis On November 28, 2008, the “Notice of the Communist Party of China Henan Provincial Committee and Henan Provincial People’s Government on Establishing Henan Provincial Cultural Reform and Development Pilot Zone” designated “Song Culture” and “Yellow River Culture” as the core positioning of Kaifeng City’s Ancient City Cultural Industry Park. The Songdu Huangcheng Cultural Tourism Resort was officially designated as one of the first batch of nighttime cultural and tourist consumption centers by the Ministry of Culture and Tourism of the People’s Republic of China on November 5, 2021. Situated within the former Imperial Palace of the Northern Song Dynasty, the resort is the core area of Song Dynasty royal culture and the focal point of cultural tourism in Kaifeng [23]. Kaifeng City, situated in the Central Plains, is part of the Central Plains culture. It is located in the middle and lower reaches of the Yellow River plain in the east, central and eastern Henan Province, is a warm temperate continental monsoon climate, four distinct seasons, the average annual temperature of 14.52 °C [24]. Numerous rivers flow through the city, belonging to two major river systems, the Yellow River, and the Huai River. The central axis of the city has never changed, and the unique landscape of cities on top of cities is rare in the history of world archaeology and capital city. During the Northern Song Dynasty, Kaifeng, known as Dongjing, represented the epitome of Chinese feudal urban society. Song culture reached its zenith, marking the pinnacle of Chinese civilization. In terms of natural geography, the presence of four major rivers, Caihe, Bianhe, Jinshui River, and Wuzhang River, flowing through the city. In terms of material culture, the architecture of the Song Dynasty is often painted, carved, and glazed tiles. The ridge and corner of the roof have the potential to warp, giving people a feeling of elegance and ease, light spirit, and soft beauty. Garden architecture displayed elegance and harmony with nature [25]. Medical science also saw significant advancements. Institutional culture-wise, in the Song Dynasty, the regional concept in politics served as a fertile ground for nurturing literary sensibilities [26]. The civil service examination system witnessed significant improvements in fairness, and the state policy of prioritizing culture over martial arts fostered cultural development. In terms of spiritual culture, the rise of Song Ci and the Sinicization and popularization of Buddhism. With the foundation of traditional Chinese Classics, a new era emerged, giving rise to a new form of Confucianism, known as new Neo-Confucianism, or the Song Dynasty Neo-Confucianism. This encouraged a deeper integration and development of Confucianism, Daoism, and Buddhism. Various artistic forms, such as calligraphy, sculpture, stone carvings, and painting reached new heights [27]. The following Table 5 is an analysis of the regional culture of the Songdu Huangcheng Cultural Tourism Resort in Kaifeng City. The Songdu Huangcheng Cultural Tourism Resort in Kaifeng City boasts a rich and diverse cultural heritage, encompassing notable aspects of geography, garden design, arts, and architecture. This cultural richness not only attracts a broad spectrum of tourists but also stimulates the local economy and tourism sector. Based on this cultural backdrop,

Wayfinding Signs in Cultural Tourism Area

727

Table 5. Analysis of regional culture in the Songdu Huangcheng Cultural Tourism Resort.

Regional culture Natural Geography

Material Culture

Institutional Culture

Spiritual Culture

Picture

Color

Song typeface, SimHei(Gothic), Calligraphy font, KaiTi

Typeface Graphic

Curve, water pattern, circle, oval

Octagon, geometric shape, water pattern, cloud pattern

FangSong, SimHei(Gothic), KaiTi, Song typeface [17], Calligraphy font, Sansserif, Serif Ladder, dragon pattern, rectangle

Song typeface, [28]

Yan Cursive Script

Pentagon, circle, geometric shape, rhombus

an analysis of the resort’s wayfinding system, including its use of colors, typefaces, and graphics, aims to further enhance the visitor experience and promote cultural heritage. 4.2 Analysis and Suggestions of the Current Situation of the Wayfinding Signs Based on on-site investigations in Kaifeng City and online interviews with wayfinding sign design companies, the signage in the Songdu Huangcheng Cultural Tourism Resort can be categorized into four primary types, which can be further classified into two categories based on their overall visual appearance. According to the design companies, the overall outline of the signage is modeled after the lantern frame window lattice. Chinese and English text are presented in serif typefaces, while Japanese and Korean text use sans-serif typefaces. The color scheme prominently features antique bronze and milky white, with golden accents along the borders. Considering the increasing popularity of nighttime cultural tourism in Kaifeng City, the central part of the guide plate is made in the form of light boxes, ensuring usability during the night, and enhancing the overall streetscape. Contrast the regional culture with the signage elements. In the color analysis, the CIE 1931 XYZ color difference calculation method is used to calculate the color differences between the signage colors and the regional cultural colors, separately. According to the calculations, the color differences between the signage system’s three colors and the colors of spiritual culture and material culture are the smallest, with color differences approximately measuring 31.32; 43.74; and 49.98, respectively. This aligns with the pattern identified in the cases: a dominance of spiritual culture, combined with other

728

Y. Ting and A. Y. Choi Table 6. Wayfinding signs analysis.

Wayfinding Signs Pedestrian Signage

Attraction Signage

Direction Signage

Road Name Signage

Picture

Color

Typeface Graphic

Source Han Serif, Times,

Pro, Apple Gothic

Rectangle, Corner lattice˄Guaizi texture˅

cultural elements, as shown in Table 6. The font ‘Source Song Serif’ used in the Chinese wayfinding sign belongs to the Song Typeface category. The Song typeface originated from the Song Dynasty and matured during the Ming Dynasty, Song typeface is a combination of Chinese calligraphy and block printing. To align with the theme of Song culture in the resort area, the Song typeface is chosen as the primary font, paying homage to the cultural heritage, representing material culture. English text, as the main foreign language of the guide, uses serifs with Chinese. This is consistent with the rule obtained in the case, but the use of serifs in street name signs may impact readability for drivers when moving quickly. From the Sui and Tang Dynasties to the Song Dynasty, window lattice designs evolved further with economic and architectural cultural development. Various patterns, including horizontal and vertical lattice, corner lattice, diamond patterns, and carved lattice, emerged [29]. The contour of the signage is inspired by the lantern frame pattern found in corner lattice designs, representing an abstraction of material culture. As a summary, it can be concluded that the signage system in the resort area is associated with both material culture and spiritual culture, material culture is reflected in the three aspects: color, typeface, and graphics. This aligns with the patterns identified in the case analyses. However, in the cases, spiritual culture predominates. In the resort area, material culture takes precedence, and only these two cultures are reflected. The spiritual values of the Song Dynasty, characterized by a pursuit of elegance in everyday items and artistic ideals, led to elegant and refined artifacts. In future signage adjustments, these keywords can be considered as the main words: in terms of color, highlighting the characteristics of cool colors such as green and green in the architectural color painting of the Song Dynasty, reflecting the leading role of the political culture of the Song Dynasty on color [30]. In terms of fonts, the top priority is ensuring legibility of signage content, followed by considerations of cultural significance. One possible approach is to use a

Wayfinding Signs in Cultural Tourism Area

729

Table 7. Element and cultural distribution statistics. Subtotal

Regional Culture Element

Case

(Element)

Natural

Material

Institutional

Spiritual

Geography

Culture

Culture

Culture

Case 1

Color

Case 2

7

Case 3 Case 1

Typeface

2

Case 2 Case 3 Case 1

Graphic

3

Case 2 Case 3

Subtotal (Culture) Color Typeface Graphic

Songdu Huangcheng

2

3

2

5 2 1 1

sans-serif fonts for primary information and Song typeface for secondary and lower-tier information, enrich cultural attributes through color decoration. Regarding graphics, in addition to extracting elements related to architecture, it may be worth considering elements from artworks such as ceramics and paintings to serve as decorative motifs (Table 7).

5 Conclusions Being a crucial medium for preserving regional culture, cultural area street signage not only provides accurate information but should also possess cultural attributes, by incorporating elements of regional culture to enhance the tourist experience. This research, by dissecting the concept of regional culture, categorizes it into four major types: natural geography, material culture, institutional culture, and spiritual culture. Based on this categorization, it analyzes and compiles the regional cultural characteristics of Songdu Huangcheng Cultural Tourism Resort. Through the analysis of various cultural tourism area signage cases, it is observed that these signs primarily utilize elements of spiritual culture and material culture to manifest regional culture. Applying this pattern to the analysis of signage in Songdu Huangcheng Cultural Tourism Resort, it was found that the resort’s signage largely aligns with the patterns identified in the case studies. However, the resort’s signage demonstrates fewer aspects of other cultural elements. Hence, recommendations for improvement in terms of color, typography, and graphics are proposed, with the hope that more cultural categories can be reflected in the signage.

730

Y. Ting and A. Y. Choi

This research approach holds certain reference value for the future development of scenic area wayfinding systems, providing ideas for the expression of regional culture in scenic spots. It is worth noting that this study’s limited number of cases does not cover all cultural categories, which presents certain limitations. Future studies are encouraged to expand the scope of case research to validate the significance of this research and to serve as a guide for the further development of cultural tourism area signage programs.

References 1. Zhou F (2023) Report on the development of china’s cultural industry and tourism: the summary of 2022 and tendency of 2023. J. Shenzhen Univ.: Humanit. Soc. Sci. 40(2):47–57 2. Chen C (2014) Strategic analysis of inheriting regional cultural characteristics in urban wayfinding systems. Art Sci Technol 10(2):6 3. Tylor EB (1958) The Origins of Culture. Peter Smith Publisher, New York 4. Kim H-M (2009) Public Design Lectures. Korean Society of Public Design, Seoul, Gain Design Lab 5. Hyojin, X .: Exploring Strategies for developing art museum education programs using regional culture: focusing on incheon area youth. Master’s thesis, korea national university of education, pp.9. (2004) 6. Zhifu Z, Pingli H (1998) Central Plains Culture, 2nd edn. Liaoning Education Press, Shenyang 7. UNWTO, https://www.unwto.org/tourism-and-culture, Accessed 3 Aug 2023 8. Classification of Tourism and Leisure Streets. LB/T 082–2021.0 9. Limei W, Xihong P (2012) An analysis of standardized design for tourist attraction wayfinding systems based on tourist experience. Packag Eng 33(12):17–20 10. Calori C, Vanden-Eynden D (2015) Signage and wayfinding design: a complete guide to creating environmental graphic design systems. John Wiley & Sons, Hoboken 11. Gibson D (2010) The Wayfinding Handbook: Information Design for Public Places. Liaoning Science and Technology Publishing House, Shengyang 12. Junyun F (2015) Research on standardized design of tourist attraction wayfinding systems based on tourist experience. Manag J 4Z:198–199 13. Nakilcio˘glu ˙IH (2013) The effects of font type choosing on visual perception and visual communication. Online J. Art Des. 1(3):35–53 14. Chen W, Xi L (2017) An analysis of font design based on readability in public wayfinding systems. Archit. Cult. 4:218–219 15. Akira K (2014) Western Fonts: Background Knowledge and Usage Methods of Fonts. Citic Press Corporation, Beijing 16. Walk of Fame Master Plan - Concept Plan (2020), https://heartofhollywood.lacity.gov/walkfame-master-plan, Accessed 18 Aug 2023 17. Layout key for official document of Party and government organs. GB/T 9704–2012 18. Xingpei, Y., Jinyu, C.: An Overview of Chinese Regional Culture, Volume: Sichuan. Zhonghua Book Company, Beijing (2014) 19. Geng C (2017) Regional cultural inheritance and development of Chengdu’s wide and narrow alleys. Art Sci Technol 30(1):153 20. Lingjie Q, Ping C (2016) East India company’s press and the publishing of Chinese classic translations in the early 19th century. J. Chongqing Jiaotong Univ: Soc. Sci. Ed. 16(6):36–39 21. Macao Cultural Heritage, https://www.culturalheritage.mo/detail/102280, last accessed 2023/8/5 22. Xueqing Z, Jielin L (2022) Study on the urban linguistic landscape of Macau. J. Shaanxi Normal Uni.: Philos. Soc. Sci. Ed. 51(6):123–136

Wayfinding Signs in Cultural Tourism Area

731

23. Culture, Radio, TV and Tourism Tourism Bureau of Kaifeng, http://wgl.kaifeng.gov.cn/index. php?id=1980, Accessed 21 Aug 2023 24. Kaifeng Diqing Network. http://www.kfdqw.com/e/action/ShowInfo.php?classid=118& id=140, Accessed 21 Aug 2023 25. Wei, G.: Characteristics and Influence of Song Dynasty Architecture. Selected Lectures on Humanities and Social Sciences in Jiangsu (2009 Edition), 60–65 (2009) 26. Weipeng, C .: Song Lyrics and Regional Culture. (Doctoral dissertation, Suzhou University). (2008) 27. Zhaoxi H (2006) My understanding of the historical status of the song dynasty. Hebei Acad. J. 26(5):101–102 28. Weisheng Y (1998) Research on the Cultural History of the Two Song Dynasties. Hangzhou University Press, Hangzhou 29. Haie M, Yalan L (2011) A brief analysis of geometric pattern decoration in traditional Chinese window frames. Beauty Time: Creative (Part I) 5:121–123 30. Wangheng C, Sijie L (2015) Discussion on the aesthetic evolution of architectural tone in song dynasty – one of the studies on the aesthetic thought of “constructing French style.” Hundred Schools in Arts 2:144–149

Chandra-Padma: Identifying Motivations for Children to Respond and React to a Book with a Non-linear Narrative Structure Sajini Lankadari(B) and Buddhika Rodrigo Department of Integrated Design, University of Moratuwa, Moratuwa, Sri Lanka [email protected], [email protected]

Abstract. Innovative primary education structures and teaching strategies are in high demand in order to develop children’s critical thinking and it is provoked by the importance of applicable and sustainable solutions. Therefore, an advanced children’s book design is an impactful tool for improving some aspects of building a harmonious and well-balanced child with high social standards. The non-linear narrative technique is increasingly popular in primary education but can be challenging for young readers to understand and engage with. While non-linear storytelling could offer children more advanced experiences, such as emotional impact, or stimulate their creativity and imagination, it is unclear how such interventions would be perceived by children. This study is to explore how children react and respond to books with non-linear narratives. Using an inductive research method, we conducted a tentative storytelling session with the experimental book ‘Chandra-Padma’ to evaluate and identify a set of themes and patterns and the impact of a child’s engagement and response to a non-linear storybook. A pilot test (n = 20) was carried out in a selected primary school with ages 8–10 children, and we collected preference data with the ‘Chandra-Padma’ book. A thematic analysis was adopted to identify significant relationships and their meanings. Results indicate that while the familiar narrative structure- linear book structure is well-rated, the non-linear narrative structure is a viable alternative. Yet, our findings of the research suggest that the use of a non-linear narrative structure can significantly improve children’s motivation to engage and react to the book in different ways. With this study, we evaluate the effectiveness of the non-linear narrative structure in ‘Chandra-Padma’ book and serve procedures for future non-linear content creators on how to understand and apply non-linear narrative strategies to children’s stories and think outside of the traditional structure of children’s books. Keywords: Non-linear narratives · Children’s literature · Reader response · Editorial design · Illustrations

1 Introduction Sri Lanka is a multicultural country with various social layers, and books are a great way for children to develop their knowledge and understand themselves and others who come from diverse cultures or different socioeconomic backgrounds (Adam and BarrattPugh 2020). Researchers in school settings have tried to determine whether story-based © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 732–750, 2024. https://doi.org/10.1007/978-3-031-47281-7_60

Chandra-Padma: Identifying Motivations for Children

733

activities and reading events enhance literacy skills by incorporating several aspects (Yazici and Bolay 2017). In addition, Research shows that children’s emotional and social skills have a significant impact on their academic performance in the early grades (Alzahrani, Alharbi, and Alodwani 2019). Children need methods such as storytelling to express their emotions and thoughts because they do not always have the words to explain their situation. Therefore, storytelling is one way that supports them in building that emotional understanding (Figueroa-Sánchez 2012). Moreover, storytelling acts as a medium to create a collaborative workspace for children by giving them to express their emotions and develop their social skills. As a result, it is necessary to develop a special approach to integrate parents, teachers, and children. With today’s increasingly complex environment, even teachers must be guided to be able to be productive as well (Nanayakkara and Kulawansa 2017). Additionally, drawing as a storytelling technique is an individual expression that can also be a visual complement to written and oral storytelling tools. Children’s drawings can improve their satisfaction and experience. (Soares de Lima et al. 2013). In the study, Web 2.0 storytelling platforms such as CBC4Kids and FaTe2 introduced picture books with animation and this platform creates stories including multimedia elements like pictures, texts, photos, sound, and music to increase child’s engagement and collaboration (Antle 2003). Children have a chance to collaboratively sketch drawings, create simple animations and share them and give comments to each other (Liu et al. 2010). The use of non-linear storytelling has made it possible to develop different kinds of entertainment applications. Digitally oriented literature has largely explored nonlinearity and multimodality in a postmodern context over the past few years. (Aliagas and Margallo 2016). Non-linear storytelling techniques combined with technology and collaborative workspaces provide a direct path to enhancing a child’s social and emotional development by enhancing their engagement with peers (Liu et al. 2010). The user can not only read a story but also select a preferred storyline. Visual narratives establish negotiations between subjective meanings, sense of self, and identity (Harpaz 2014). By involving collaborative non-linear storytelling, a child can develop social and emotional skills with peers. KidPad is a tool that allowed children to create a collaborative story by using drawing, text, and linking functionalities on 2D zoomable space (Hourcade et al. 2002). Story Mat is a tool to support children’s fantasy and storytelling while encouraging a child to be imaginative. It also gives probabilities for children to work with others within a shared fantasy world. This enhances children’s experience by supporting make connections with others and sharing fantasies in the form of open-ended collaboration work (Ryokai and Cassell 1999). However, most of the researchers are studying the effectiveness of linear storytelling structure. Even though the Reader Response framework has been used to describe reading experiences by considering that the child is an active partner in the process and contributes experiences and emotions, its use in the non-linear literature has not been widely disseminated, particularly for children. (Aliagas and Margallo 2016). It has been discovered that Sri Lankan children have improved social and emotional skills when using non-linear narrative structures (Lankadari and Rodrigo 2022). Despite the possibility that a non-linear narrative structure could provide children with more advanced experiences like emotion and social or stimulate their creativity, it is

734

S. Lankadari and B. Rodrigo

unclear how children would react and respond to such interventions. Nevertheless, some research works show that non-linearity could jeopardize since children can’t extract the original message of the book, and because of that authors only follow linear narrative structure since they consider that the major success of the book is to deliver the right message to the children (Nasir et al. 2012). Therefore, the children also should get a chance to respond and demonstrate their perspectives on their preferences for a book. Children need to select their own reading materials, within what is available to them and, it is important to ask their opinions such as, why they do or do not like a book (Saccardi 1993) or the reason to choose that book, what surprised the child, what the child learned, the child’s analysis of the writing, the elements or things that child wonders about, things that child can’t understand, what are the things that child wants to share with others, or the how child thinks other children in the class would respond to the book (Hansen 1987). According to Effy, children are responding in a verbal and non-verbal way to a story. Verbally, children tend to respond on 4 major categories; Focus on story structure, focus on meaning, focus on print, focus on illustrations. Non-verbal behaviours were; Kinesics (Body language, Acting out, Facial expressions), Silence, Drawing, Writing (Mulyasari 1993). In addition to that, Sipe has also cited that children demonstrate responses to literature in their own unique ways and a child is capable of presenting verbal, artistic, dramatic, and written responses (Sipe 2007). Human responses can also be captured and explained through written expression. For children, child’s illustrations and sketches are also considered as written expressions. Their artistic skills show how they make connections to every aspect of their lives. It allows children to understand the content of the story as well as describe it in a different way, and even control or criticize the story (Mulyasari 1993). Sipe has modelled five conceptual categories to measure children’s responses to literature; The analytical, the personal the intertextual, the transparent, and the performative, and each category can be characterised by different stances, actions, and functions (Sipe 2007). Thus, it is imperative to examine how children will respond to literature with non-linear narrative structures. In spite of the fact that non-linear storytelling may offer children more advanced experiences, such as stimulating their creativity and imagination skills, it is still uncertain how children will perceive such interventions. Hence, there is a need for research on how the non-linear narrative structure motivates children to react and respond to a book with a non-linear story and their overall enjoyment of reading by analysing the perceptions and preferences of children towards the non-linear narrative structure. In order to test this, an experimental book ‘Chandra-Padma’ was used. ‘ChandraPadma’, is a practice-based experimental book, which has a dual non-linear narration, and the stories coincide with the subjective thinking of a child. Subsequently, the book suggests a non-linear narration-based physical storybook to allow children to interact with the storylines and discover the story plots. Ultimately, this research is to evaluate the effectiveness of the Chandra-Padma project in promoting non-linear storytelling that motivates children to engage with the book as well.

Chandra-Padma: Identifying Motivations for Children

735

2 Methodology The purpose of this study is to examine how children react and respond to books with non-linear narratives. In order to answer the question, an inductive research approach was used, which allows the researcher to collect data directly from children’s experiences and observations. As a result, patterns and themes emerged naturally from the children. As part of this inductive research, thematic analysis was used, which is a qualitative data analysis technique that involves identifying, analyzing, and interpreting patterns or themes within qualitative data sets. 2.1 Participants A classroom with 20 children, (11 girls, 9 boys) age range between 8–10 was selected randomly from primary school in the Western province of Sri Lanka. We classified the children in the class as being either lower-middle or lower socioeconomic levels. From the total population of 36 students in the class, 20 of them were randomly selected for a storytelling session and activity of drawing. (Bucsuházy and Semela 2017). 2.2 Materials ‘Chandra-Padma’ book was used for this study. ‘Chandra-Padma’ is a unique non-linear narrative book and this experimental book was mainly inspired by research, ‘Storytelling for Preschool Children in Sri Lanka: Social and Emotional Development Between Linear and Non-Linear Storytelling Approaches.’ which has done by the researchers by themselves on the year of 2019. (Lankadari and Rodrigo 2022). This novel book has followed a dual-narrative structure, designed to immerse children in interactive storytelling sessions. ‘Chandra-Padma’ is designed by using two separate children living in two distinct contexts. The book combined two storylines from two different periods of time. In the design experience in ‘Chandra-Padma’, individual children can select their own story to start with. Children interact with the two contexts, the moon, and the earth where children can imagine, engage, and excite with curiosity. (See Appendix 7.2) The Storybook of ‘Chandra-Padma’ has two storylines and both two storylines start with the main character hearing a song and developing a curiosity about searching for the singer. And the book doesn’t mention who starts first, and it brings the reader to a paradox. But it has carefully avoided the complexity and confusion in the final combined story. Storylines always cross their paths through connecting events such as poems, and dialogs in this book. Time dilation theory is also used as a fundamental key to estimate the duration of a certain time interval without doing any harm to the interstress of the story plot, which has originally designed for a child. (Kanai et al. 2006). And the story’s endpoint is in the middle of the book, which combines the two storylines in a meaning full way. The narrative structure that used for designing this experimental non-linear narrative structure can be found below chart (See Fig. 1) and Appendices. • Storyline x: Story of character 1 (Chandra) • Storyline y: Story of character 2 (Padma) • Exposition x: A moon somewhere in the universe

736

S. Lankadari and B. Rodrigo

Fig. 1. Non-linear narrative structure in ‘Chandra-Padma’ book

• Exposition y: The earth • t1: Time period character 1 unitement with character 2 (t1 > > t2) • t2: Time period character 2 unitement with character 1 (t2 < < t1) 2.3 Procedure All 20 individual child participants read the ‘Chandra-Padma’ book with the trained adult research assistant. The researcher was knowledgeable about child development and data collection protocols and aware of the major hypotheses under investigation, as well as the importance of maintaining methodological integrity. All child participants were observed by the researcher before/during/after the storytelling session. (Justice, Pullen, and Pence Turnbull 2008) and the data was collected from children with the written consent of their caregivers and the school. Research assistance instructed to start the story with a brief and friendly discussion with children related to the book title in order to provide prior knowledge about the story and get comfortable with children. After that, the book was given to children to read and during that time children were motivated to respond by following three interactive behaviours throughout the storytelling session (Morrow 1988). 1. Inviting a child throughout a reading to ask questions. 2. Inviting a child to share comments related to the story. 3. Listening to a child about their personal experiences related to the story. Throughout the study, the storybook was introduced, and children were encouraged to ask questions and comment on the book or story at any time. In addition to that, non-verbal behaviours are also noted. During the storytelling session, the researcher observed the child and took field notes of non-verbal responses (Mulyasari 1993). After that, a child was invited to draw any sort of story part, but not forced to do it (Activity session). Before/during/after the activity, the researcher observed children and asked semi-structured questions, and took field notes, responses, and emotions.

Chandra-Padma: Identifying Motivations for Children

737

2.4 Data Analysis Analysis was open-ended, iteratively evolving and collecting instances of the issues we here report upon. (Duenser and Hornecker 2007). Since the various stages in analysis might be interrelated, analysis is more interactive than the hierarchical approach from the bottom to the top. (Creswell 2014). Therefore, the collected data were analysed by using a thematic analysis approach. After the storytelling sessions, thematic analysis sessions were conducted to identify each new category with research assistants. (Liebenberg, Jamal, and Ikeda 2020). Transcripts were analysed to identify new codes, and patterns, and devise themes of children’s responses. Codes were limited with the aim of providing themes or patterns that could be used easily for instructional, diagnostic, or research purposes (Nowell et al. 2017). Below steps were followed during the data analysis in this qualitative research (Creswell 2014). • Step 1: Organized and Prepared the 20 transcripts of semi-structured interviews, drawings, and field notes data of children for analysis. • Step 2: Read and looked at the data to get overall meaning and general ideas on what children were saying. • Step 3: Started coding all the data by grouping data into chunks and labelling with a child-friendly term. • Step 4: Used the coding process to generate a description of the setting or children as well as categories or themes for analysis. • Step 5: Conducted detailed discussion of several themes and conveyed descriptive information about each child in a table. (Completed with subthemes, multiple perspectives from children, specific illustrations, and quotations) • Step 6: The final step was interpreting qualitative research of the findings. Asking, “What were the lessons learned?” And meanings were derived from a comparison of the linear and non-linear story structures with information gathered from the literature and theories.

3 Results This research study aimed to explore how children are motivated to react and respond to books with a non-linear narrative structure. During the first day of the research, the mistake was identified in the storybook sample. Unintentionally, the book cover (See Appendix A) had motivated the child to read particularly one storyline. Due to this error, the book cover had to be redesigned (See Fig. 2) Therefore, the first participant’s data was excluded from the results. 3.1 Children’s Motivations to Respond or React Data were analysed by using applied thematic analysis (inductive), which is used to analyse qualitative data and drive themes with methodological rigor as described by (Sepehri et al. 2023) and (Creswell 2014). The analysis conducted for each of the major aspects in the study is attention on the one-story structure, focus on the story content, and attracted to illustrations. Data were gathered on children’s questions and their comments on the

738

S. Lankadari and B. Rodrigo

Fig. 2. Non-linear narrative structure in ‘Chandra-Padma’ book (After fixing the cover)

above aspects. While the storytelling session was taking place, non-verbal behaviours and other actions were also observed. Total comments and questions and other notes (interview transcripts and notes) were presented on the Miro board (See Fig. 2) (Fig. 3).

Fig. 3. The set-up for the thematic analysis map on the Miro board

The transcripts and notes were broken down into smaller pieces and inserted a code to each bunch of texts and a meaning was also defined to each code. Qualitative data were categorized into 3 clusters: challenges, strategies, and preferences. In the ‘Challenge’ part, some pain points and extra efforts were noticed. Positive responses and reactions are categorised under the ‘Strategies’. Children’s preferences were categorised under the last cluster ‘Preferences’. The themes which are derived from this thematic data analysis are related to the below table (Table 1).

Chandra-Padma: Identifying Motivations for Children

739

Table 1. Themes were derived from the analysis. Cluster

Theme

Categories

Number of occurrences

Challenge

Replay

I turn pages backwards and forward

7 Interviews 11 Occurrences

I repeat both stories to understand I revisit to some marked pages Confusion

I ask question to clarify which I don’t understand

12 Interviews 18 Occurrences

I ask my friends on how to read this book I feel confused when I can’t understand the story Curiosity & Prediction

I predict story end by using information from the story

6 Interviews 7 Occurrences

I explore alternative story path in this book Strategies

Empowerment

I can interact with the book I focus only for one storyline

19 Interviews 36 Occurrences

I can choose story and gives me a sense of control Comparison

I recognize environment/setting difference in 2 storylines

4 Interviews 6 Occurrences

I can analyse and compare the potential outcomes I recognize time difference in 2 storylines I recognize that 2 stories have connection Sharing

I share my knowledge with my friends

12 Interviews 22 Occurrences

I retell the story with other friends/family/ neighbours I empathise with characters (continued)

740

S. Lankadari and B. Rodrigo Table 1. (continued)

Cluster

Theme

Categories

Number of occurrences

I like to share my personal opinions and experiences related to the story Preferences

Kinesics

I nod, do facial expressions, silent 19 Interviews or hand gestures to express 26 Occurrences agreement, disagreement, surprise, or other emotional responses

Focus & Immersion

I become more deeply engaged and immersed in the story

9 Interviews 15 Occurrences

I invest time in reading and understanding the story Aesthetics

I appreciate to aesthetic beauty of the book

3 Interviews 5 Occurrences

I like the illustrations style, book cover, quality, and printing

3.2 The Children’s Interaction with Storylines X and Y Below figures show results that were noticed during the storytelling session. Based on the total number of children 19, children prefer X or Y when it comes to their first storyline. 84.2% of children couldn’t read the second story correctly. After reading it, the children had no idea how storylines X and Y ended. Following my guidance on how to read both storylines correctly, they revisit some pages. The majority of children randomly selected or preferred Storyline X as their first story. Children often pick their favourite characters from the first story they read (See Fig. 4). The discussion section will give a more enhanced and comprehensive description according to the categories for coding children’s responses during the study.

4 Discussion: ‘What were the Lessons Learned?’ The research question is to know how a book with a non-linear narrative structure motivates children to react and respond. Additionally, ‘To explore children’s overall enjoyment of reading by analysing the perception and preferences of children towards the non-linear narrative structure’. Findings show that there are different ways to motivate children to respond to a non-linear book.

Chandra-Padma: Identifying Motivations for Children Both stories were read correctly.

Only one story was read correctly.

Having an idea of what happened at the end.

Not sure what happened in the

Start with Storyline X

Start with Storyline Y

Favourite character is in Storyline X

Favourite character is in Storyline Y

741

Fig. 4. Percentages of responses and preferences made by children based on the non-linear storylines in the book.

4.1 Theme 1: Replay At first, the children seemed confused. Children struggle to choose which way to start first. The majority of children selected the storyline X. Some of the children had held pages in the important pages as they thought and revisited several times to understand the story. Some children took a long time on certain pages to read the text and understand the meaning of the illustration. Children themself wanted to draw their favourite part of the book, without any force. Many children put their best effort to do a refined illustration within a very limited time. During the activity, one girl said: “Can I have the book back, please? I need to go back to Chandra’s first story and count stars in her hair to draw in my paper.” And she counted stars and drew exactly the same number of stars as the book (Fig. 5).

Fig. 5. Revisit the page to count stars in hair.

Most children read the second story incorrectly. They read the story backward, but some of them marked the resolution with their hands. And after they read incorrectly, they realised that the story starts differently, and they spent a little time taking a quick look by snipping pages (Fig. 6).

742

S. Lankadari and B. Rodrigo

Fig. 6. This girl is holding pages, so she can come back and read it again.

4.2 Theme 2: Confusion Children became nervous when they couldn’t find the correct way to read the story. This confusion acted as a catalyst, prompting children to actively interact with the book and explore different paths and find connections. This initiates children a challenge with the non-linear narration format and drives their motivation to respond. Ultimately ‘confusion’ encourages children to explore complex narratives. Unlike regular books that have a mark resolution on the end page (last page), non-linear books make it harder for children to find the resolution (Nasir et al. 2012). Children were engaged with the first storyline well, but when comes to the second storyline, children missed the flow, due to the complexity. Hence, many children couldn’t answer the question of what happened in the end (See Fig. 4). 4.3 Theme 3: Curiosity and Prediction Although curiosity is considered a strong motivator for learning, studies suggest that curiosity is only a small contributor to learning (Wade and Kidd 2019). However, curiosity & prediction played a significant role in motivating children to respond to a non-linear book in the study. As soon as they heard about this experimental book, their natural sense of curiosity sparked, and they wanted to know more about it. Making predictions regarding how the second storyline will end drives them to uncover the second storyline. This anticipation and active engagement boosted children’s motivation to respond, as they eagerly turned pages, search for meanings, and construct their own interpretations. One boy said while reading the book: “Is this a story about a dream? They all are surely dreaming, aren’t they?”. Leaving them with a sense of surprise and satisfaction from predicting the outcomes stimulating further exploration and response to the book.

Chandra-Padma: Identifying Motivations for Children

743

4.4 Theme 4: Empowerment “Empowerment” also played as a motivating factor for children to respond to a non-linear book. A non-linear book allows children to take control of their reading experience, and they actively engage with the content as a result (Lankadari and Rodrigo 2022). It was found that most children chose a side to read first by themselves, and they immersed themselves in the first storyline they had chosen first, more than the second storyline. They considered only the one main character in the book according to their selected storyline and they empathise with that character (See Fig. 4). 4.5 Theme 5: Comparison The comparison of two storylines in the ‘Chandra-Padma’ book gave motivation for children to actively respond and engage with the content. When there is storyline X and storyline Y, children are encouraged to analyse and compare the X, Y narrative threads. By turning pages and reading both sides slapdash, some children took their time deciding what path to choose by comparing content. This theme also includes comparing time across two different story settings. One girl said: “Chandra makes me happy because she has waited so many years for her turn and has waited patiently for it. Nevertheless, others Dooli, Udhi, and Inu got permission from their granny and went to see the star overnight”. 4.6 Theme 6: Sharing Children are often filled with excitement and a sense of discovery when they engage with the book. Thus, sharing their experience with others allows them to express their thoughts, insights, and emotions related to the non-linear book. One boy said that: “I talked about this book with my neighbour brother, and I showed him what I drew!” This act of sharing not only solidifies their own understanding of the narrative, but also encourages collaboration, discussion, and social interaction. In addition to that, children found that this is a good opportunity to share their personal preferences. One boy mentioned that he learned about meteors, and he wanted to learn more about Astronomy. By listening to that, another boy joined the conversation and said: “My favourite planet is Saturn, it has a ring, here it is”. He put his finger on an illustration in the book, which looks like a Saturn. Some children also transmitted their personal experiences related to the story. One boy pointed out the illustration of a boat in the book and said, “I like it because I also have gone on a swan boat (paddling boat) recently”. Another boy poked his friend and showed a text ‘Inu’ in the book and said “Hey, your nickname is in here”. His friend’s name was also ‘Inura’, but they called him ‘Inu’.

744

S. Lankadari and B. Rodrigo

In Addition to that, the non-linear book evoked emotional responses in children, and they responded with empathy. As they navigated the complexity and two storylines within one book, they are exposed to a range of characters, experiences, and perspectives. This exposure triggered them to develop a sense of empathy towards the characters (Chandra, Bunny, Uncle Moon, Padma, Udi, Dooli, and Inu) and their journeys. Through the characters’ struggles and growth, they became emotionally attached to the story. And children persuade to draw their characters with their emotions in different scenarios during the activity time (See Fig. 7). The empathetic response motivated children to explore the nonlinear book deeper, seeking understanding and engaging with the narrative. Placing themselves in the characters’ shoes helps children gain a broader understanding, develop compassion, and become more responsive. (Anon n.d.)

Fig. 7. How two children drew the same character with different emotional facial expressions.

4.7 Other Themes: Kinesics/Focus and Immersion/Aesthetics ‘Chandra-Padma’ book has used an interactive strategy to insert two different storylines in different perspectives, where a child can navigate through turning pages. This invites children to physically engage with the book more than a regular linear book. Thus, children nodded, did facial expressions, used hand gestures, and be silent to express agreement or disagreement. In addition to that, focus and immersion played a crucial role in motivating children to respond to the book, during a non-linear narrative, children become fully immersed in the story and enter a deep state of concentration. The dual storyline structure of the book attracts their attention, and they become captivated by the challenge of making links between the two storylines X and Y. (See Fig. 8).

Chandra-Padma: Identifying Motivations for Children

745

Fig. 8. Children are reading while ignoring background distractions.

Children were also questioned about the aesthetics of the book. According to Morrow, print, story structure, and illustrations are the most effective way for a child to respond when it comes to one-to-one story reading (Morrow 1988). Hence, few children commented on the illustrations and looked for visual cues. When children were doing the activity, they imitated the same illustration style as the book. However, when it comes to the ‘Chandra-Padma’ non-linear book, children often commented on binding techniques because they thought the pages were wrongly bound. Some of the children appreciate the hard cover of the book. One girl said that: “I also have this kind of English book” (She was referring to the cover of the book). It is common that native language books are targeting the majority of Sri Lankan children, who might not be spending a higher price for a hardcover book with good printing quality. Due to this, most hardcover children’s books are imported from abroad and have limited availability since the publishers cannot profit by printing in Sri Lanka.

5 Conclusion The research study aims to explore how a book of non-linear narration structure would be perceived by children aged 8–10. In order to find the answer to the research question, on how children are stimulated to respond and react to a book with a non-linear narrative structure, a list of motivations was compiled from the thematic analysis and literature review.

746

S. Lankadari and B. Rodrigo

It appears that this was the children’s first time that read a book with a non-linear narration structure that has only two storylines. And children asked more questions and provided their comments on based non-linear narration. With all findings combined, this book provides a novel perspective on non-linear storybook design and production. It should be noted that any kind of non-linear books including fancy, attractive hard-cover books are not available for this group of children since they are coming from lower-tolower-middle socioeconomic backgrounds and are unable to afford books of that kind. Due to the fact that the book is new, it has an unfamiliar story structure, and complexity and the binding, provoking more responses during the session. While it is encouraging to see that these children were motivated to respond and react to the ‘Chandra- Padma’ book, more studies will need to be conducted with different types of nonlinear books with a greater sample size of children.

6 Appendices 6.1 Original Material: ‘Chandra-Padma’, the Non-linear Book The cover gave the wrong impression that the story begins with Chandra, because of the prominent illustration. Thus, the researcher had to change it according to Fig. 2 (Fig. 9).

Fig. 9. ‘Chandra-Padma’, the non-linear book before modification.

Chandra-Padma: Identifying Motivations for Children

6.2 Storylines X and Y (See Fig. 10).

Fig. 10. Storylines X and Y

747

748

S. Lankadari and B. Rodrigo

6.3 Book Illustrations and Descriptions (See Fig. 11).

Storyline X

One day, something magical happened! Someone is singing a song very, very far away. I'm Padma, living where lotuses bloom, love to see the bunny, hopping on the moon… Chandra wanted to find out, who the mysterious singer is.

Storyline Y

One day, something magical happened! Someone is singing a song very, very far away. I'm Chandra, living with a moon, With my bunny, but we are alone… Dooli wanted to find out, who the mysterious singer is.

Fig. 11. Interconnected dialogs between two storylines.

Acknowledgment. It is with great gratitude that we extend our sincere thanks to Sri Parakrama Maha Vidyalaya, Panagoda - Sri Lanka for graciously permitting us to conduct the research with the children. A study like this would not have been possible without the enthusiastic engagement and valuable contributions of the children.

Chandra-Padma: Identifying Motivations for Children

749

References Adam H, Barratt-Pugh C (2020) The challenge of monoculturalism: what books are educators sharing with children and what messages do they send? Aust Educ Res 47(5):815–836. https:// doi.org/10.1007/s13384-019-00375-7 Aliagas C, Margallo A (2016) Children’s responses to the interactivity of storybook apps in family shared reading events involving the IPAD: children’s responses interactivity storybook. Literacy 51(1):44–52. https://doi.org/10.1111/lit.12089 Alzahrani M, Alharbi M, Alodwani A (2019) The effect of social-emotional competence on children academic achievement and behavioral development. Int Educ Stud 12:141. https:// doi.org/10.5539/ies.v12n12p141 Anon. n.d. (2023) The psychology of emotional and cognitive empathy|lesley university. https://les ley.edu/article/the-psychology-of-emo-tional-and-cognitive-empathy. Accessed 27 May 2023 Antle A (2003) Case study: the design of CBC4Kids’ StoryBuilder, pp 59–68. https://doi.org/10. 1145/953536.953546 Bucsuházy K, Semela M (2017) Case study: reaction time of children according to age. transbaltica 2017: transportation science and technology. In: Proceedings of the 10th international scientific conference, May 4–5, 2017, Vilnius Gediminas Technical University, Vilnius, Lithuania, vol 187, pp 408–13. https://doi.org/10.1016/j.proeng.2017.04.393 Creswell JW (2014) Research design. SAGE Publications, Inc. Dünser A, Hornecker E (2007) An observational study of children interacting with an augmented story book. In: Hui K-C, Pan Z, Chung R-K, Wang CCL, Jin X, Göbel S, Li E-L (eds) Edutainment 2007, vol 4469. LNCS. Springer, Heidelberg, pp 305–315. https://doi.org/10.1007/ 978-3-540-73011-8_31 Figueroa-Sánchez M (2012) Building emotional literacy: groundwork to early learning. Child Educ 84:301–304. https://doi.org/10.1080/00094056.2008.10523030 Hansen J (1987) When writers read. Heinemann Harpaz R (2014) Narrative knowing: narrative and storytelling resources in art therapy.14 Hourcade J, Bederson B, Druin A, Taxén G (2002) KidPad: collaborative storytelling for children Justice L, Pullen P, Pence TK (2008) Influence of verbal and nonverbal references to print on preschoolers’ visual attention to print during storybook reading. Dev Psychol 44:855–866. https://doi.org/10.1037/0012-1649.44.3.855 Kanai R, Paffen C, Hogendoorn H, Verstraten F (2006) Time dilation in dynamic visual display. J Vis 6:1421–1430. https://doi.org/10.1167/6.12.8 Ankadari S, Rodrigo B (2022). Storytelling for preschool children in Sri Lanka: social and emotional development between linear and non-linear storytelling approaches. In: Martins N, Brandão D. (eds) advances in design and digital communication II. DIGICOM 2021. Springer Series in Design and Innovation , vol 19. Springer, Cham, pp 452–65. https://doi.org/10.1007/ 978-3-030-89735-2_37 Liebenberg L, Jamal A, Ikeda J (2020) Extending youth voices in a participatory thematic analysis approach. Int J Qual Methods 19:1609406920934614. https://doi.org/10.1177/160940692093 4614 Liu CC, Liu KP, Chen GD, Liu BJ (2010) Children’s collaborative storytelling with linear and nonlinear approaches. Procedia Soc Behav Sci 2:4787–4792. https://doi.org/10.1016/j.sbspro. 2010.03.771 Morrow LM (1988) Young children’s responses to one-to-one story readings in school settings. Read Res Q 23(1):89–107. https://doi.org/10.2307/747906 Mulyasari E (1993) Children responses to short stories and aloud probst Nanayakkara MS, Kulawansa K (2017) Agent based primary education supporting platform Nasir MNF et al (2012) Non-linear and interactive storytelling for children book

750

S. Lankadari and B. Rodrigo

Nowell L, Norris J, White D, Moules N (2017) Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual 16(1):1609406917733847. https://doi.org/10.1177/160940691773 3847 Ryokai K, Cassell J (1999) StoryMat: a play space for collaborative storytelling. https://doi.org/ 10.1145/632716.632883 Saccardi M (1993) Children speak: our students’ reactions to books can tell us what to teach. Read Teach 47(4):318–324 Sepehri K, Liisa H, Niasat S, Vita C, Maclea KE (2023) Beyond the bulging binder: familycentered design of an information management system for caregivers of children living with health complexity. https://doi.org/10.1145/3544548.3581459 Sipe LR (2007) Storytime: young children’s literary understanding in the classroom. Teachers College Press; Illustrated edition Soares de Lima ES, Feijó B, Barbosa SD, Furtado AL, Ciarlini AE, Pozzer CT (2013) Draw your own story: paper and pencil interactive storytelling. Entertainment Comput 5(1):33–41. https:// doi.org/10.1007/978-3-642-24500-8_1 Wade S, Kidd C (2019) The role of prior knowledge and curiosity in learning. Psychon Bull Rev 26(4):1377–1387. https://doi.org/10.3758/s13423-019-01598-6 Yazici E, Bolay H (2017) Story based activities enhance literacy skills in preschool children. Univ J Educ Res 5:815–823. https://doi.org/10.13189/ujer.2017.050528

Graphic Design and Branding

Empirical Analysis of Packaging Design and Consumer Preferences in the New York Kimchi Market An Zhao1 , Wenbo Wei1 , and Albert Young Choi2(B) 1 Graduate School, College of Design, Hanyang University ERICA, Ansan, South Korea 2 Faculty of Graduate School, Communication Design, College of Design,

Hanyang UniversityERICA, Ansan, South Korea [email protected]

Abstract. Kimchi is a food made by picking and fermenting vegetables such as cabbage and radish using a variety of seasonings, such as chili powder, green onion, garlic, ginger and fish sauce. In order to increase the sales of kimchi products in the New York market, as well as to meet the consumer preferences of New York consumers. Therefore, this study investigates consumer preferences for kimchi packaging design in the New York market and proposes a design strategy for target consumers in New York, USA. The study used both literature and empirical research methods. It investigated the package design of 10 kimchi brands and empirically analyzed package design factors such as color, font, size and layout of kimchi products sold in the New York market to 100 New York men and women aged 10–50. Finally, the study analyzed the differences in consumer preferences and suggested corresponding design strategies. Manufacturers and designers should first focus on package color and package format. Small, irregular packages of 350g and 80g, and package formats such as glass bottles and plastic tubs are key points. Low-saturation and soft beige colors are preferred, followed by product-identifying reds. Mixed language typography is best suited to the needs of New York consumers. And realistic images should be the main focus in package design. We believe that kimchi products are going global. Packaging kimchi products requires adapting to local market requirements. Designers should pay attention to the localization of kimchi products and develop design strategies accordingly. Keywords: Kimchi Packaging Design · Consumer Preference · New York Kimchi Market · Design Strategy

1 Research Background 1.1 Research Purpose COVID-19 in recent years. People are increasingly focusing on healthy eating habits. The rise of cross-cultural diets and Korean culture, convenient, healthy, and easy-to-store kimchi products have received attention from the New York market in the United States. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 753–764, 2024. https://doi.org/10.1007/978-3-031-47281-7_61

754

A. Zhao et al.

Kimchi exports have increased dramatically. As the second largest consumer market for Korean kimchi products, designers should pay attention to the packaging design of American kimchi products. The purpose of this paper is to investigate the current status and characteristics of the packaging design of the top ten pickle products sold in the New York market. Secondly, it investigates New York consumers’ preference for kimchi product packaging design. Finally, research data and models are used to derive and analyze New York consumer preferences for kimchi product packaging design, in an attempt to provide effective packaging design strategies and recommendations for kimchi manufacturers. 1.2 Research Scope and Methods The scope of the study was kimchi products sold in the New York market. The top 10 kimchi products in New York City are summarized through U.S. New York market research and website data compilation [1]. The research method is based on literature and empirical research methods. By organizing existing research theories, literature and data, the search and examination system was determined. Survey respondents: 100 New York City residents of various age groups. Conducted empirical analysis on the packaging design of Top 10 kimchi products sold in the New York market in terms of colors, fonts, sizes, packaging forms Table 1. Top 10 Kimchi products sold in the U.S. New York market

Empirical Analysis of Packaging Design and Consumer Preferences

755

and packaging design layout. Modelling was carried out using quantitative analysis methods and the SPSS program, and conclusions were drawn and analyzed. Table 2. U.S. New York. Top10 Kimchi Packaging

2 Overview of Kimchi Product Packaging in the New York 2.1 Kimchi Packaging Definitions and Standards Kimchi is a food made by picking and fermenting vegetables such as cabbage and radish using a variety of seasonings, such as chili powder, green onion, garlic, ginger and fish sauce. As with most other food packaging, the primary role of kimchi packaging is to facilitate the containment and distribution of kimchi. It protects kimchi products from the external environment and damage, slows down the deterioration of kimchi products, maintains the beneficial effects of fermentation, and prolongs shelf life [2]. The Kimchi production process for cabbage products was recognized as an international standard by the Committee on International Food Standards (CODEX) in 2001. The most commonly used kimchi packaging materials are plastic, metal, glass and paper. 2.2 Kimchi Market Environment Analysis Restrictions on kimchi production and sales due to COVID-19 embargo measures and personnel limitations resulted in a decline [3]. However, with the gradual return to normalcy globally, the growing popularity of cross-cultural diets and Korean Wave culture have contributed to an annual increase in kimchi sales in international markets [4]. The primary destinations for Korean kimchi exports are Japan and the United States, the latter being the second-largest consumer market for kimchi. In 2021, South Korea exported about $28.25 million worth of kimchi to the United States [5]. Due to the strong public interest in fermented kimchi during the COVID-19 pandemic, kimchi exports have

756

A. Zhao et al.

significantly increased over the past few years. Presently, Jongga brand kimchi alone is sold in over 40 countries worldwide, including China, Japan, Singapore, and the United States. Within five years, the company’s exports increased by 131 percent, from $29 million in 2016 to $67 million in 2021. As of the first half of 2022, sales of Jongga kimchi accounted for 60 percent of South Korea’s total kimchi exports. In 2021, exports to the U.S. reached $16.17 million, a 37.8 percent YoY increase [6]. Additionally, Korea is continuously conducting research into the trend of kimchi-related products and currently holds 408 patents in the field of developing functional kimchi for the prevention of metabolic diseases. There is a noticeable improvement in the quality of Korean kimchi products [7]. With the expansion of the demand market, competition among kimchi manufacturers in different countries has become fierce, and production and management strategies for the United States are also developing rapidly. How to better adapt to the U.S. market and come up with proactive policies to deal with it has become a key concern for manufacturers. 2.3 The Importance of Kimchi Packaging Design Research shows that the visual packaging of a product is a key factor in gaining consumers’ attention when shopping offline. As a food product that is mainly sold in offline stores, Kimchi product packaging design needs to consider how to attract consumers’ attention and stimulate their desire to buy. As the core element of brand identity, packaging design is the most effective way to present the brand and products to consumers, and plays an important role in associating the brand image and improving the brand [8]. Excellent kimchi packaging design will increase consumer user stickiness to the brand, thus promoting brand development and long-term product sales. In addition, packaging design has an iconic role that not only enhances the utility of the product, but also plays a positive role in consumer buying behavior. In today’s fierce retail competition, a brand must break through the visual clutter to capture the attention of shoppers [9]. This is also true of the competitive kimchi market in the United States. A data modeling study by S.T. Wang, E. verified that food companies should emphasize visual packaging design factors such as color, font, logo, graphics, and size to create positive consumer perceptions and brand preferences [10]. Therefore, understanding consumer preferences for kimchi product packaging design in the U.S. market has high commercial and theoretical value.

3 Empirical Analysis of Packaging Design and Consumer Preferences in the New York Kimchi Market 3.1 Research Subject Characteristics and Research Procedures This study investigated the packaging design of Top 10 kimchi products in New York, USA. Based on demographics, respondents were male and female aged 10–20, 20–30, 30–40, and 40 years and older. A total of 120 New York City residents participated in the online survey. The questionnaire was released from February 9, 2023 to February 29, 2023 and was shared by the researcher in the New York City area through an

Empirical Analysis of Packaging Design and Consumer Preferences

757

Table 3. Composition of the questionnaire Classification

Questionnaire questions

Variables

Attractive points

Kimchi product packaging design attractiveness

color/font/package form/picture/illustration

Package sizes

Packaging sizes to choose when buying kimchi products

80g/350g/500g/1.2kg/2kg

Packaging fonts

Kimchi Packaging Design Font Preferences

serif / handwriting / non-serif / multilingual font combinations

Color

Favorite colors to apply to kimchi packaging

beige/blue/red/green

Package form

Kimchi packaging form preference

glass jar / plastic bagged / plastic bucket / can / small volume shaped

Packaging Design Layout

Pickle Packaging Design Layout Preferences

text only / illustration / picture /illustration and picture / text-based with picture

online questionnaire, which was answered by study participants through self-completion. Table 3 shows the main elements of the questionnaire. A total of 100 questionnaires were obtained after excluding unreliable and unused questionnaires. There were 36 male participants and 64 female participants. The total number of male participants was 36 and the total number of female participants was 64. 14 Participants were in the 10–20 age group, 64 in the 20–30 age group, 16 in the 30–40 age group and 6 in the 40-plus age group. The final empirical findings were analyzed by the researcher using the Statistical Package for the Social Sciences (SPSS) 23.0 by cross-tabulation analysis. Specific research methods used were frequency analysis and cross-tabulation analysis (Tables 1, 2, 4, 5, 6, 7, 8, 9, 10, 11). Table 4. General characteristics of survey respondents

Gender Age

Kimchi Awareness

Variables

Number

Percentage

Male

36

36%

Female

64

64%

10–20

14

14%

20–30

64

64%

30–40

16

16%

Over40

6

6%

Knew about kimchi and bought it regularly

26

26%

Tasted, bought occasionally

46

46%

Tasted, never bought

18

18%

Heard of it

6

6%

First time

4

4%

758

A. Zhao et al.

3.2 Results of the Empirical Analysis 3.2.1 The Attractive Point of Kimchi Product Packaging Design for Consumers

Table 5. The Attractive point of kimchi product packaging design for consumers Gender Male

Female

Age

Percentage

Number

Color

Font

Package form

Picture

Illustration

10–20

66.67%

33.33%

66.67%

33.33%

0.00%

6

21–30

50%

40%

60%

30%

40%

20

31–40

75%

25%

50%

25%

25%

8

Over40

100%

100%

0.00%

0.00%

0.00%

2

10–20

50%

25%

100%

25%

0.00%

8

21–30

54.55%

31.82%

40.91%

18.18%

22.73%

44

31–40

100%

50%

75%

100%

50%

8

Over40

100%

0.00%

100%

50%

0.00%

4

Note. “Percentage” is the proportion of options for each question. “Number” refers to the number of people of different genders and age groups. The total number of participants in the questionnaire is 100

The points of attraction for consumers in kimchi packaging design were investigated. This question was set as a multiple choice question that included categories such as color, font, package form, picture, and illustration in kimchi package design (Table 3). Color was the most important focus for consumers, with a total of 62 choices. Packaging format was the next most popular choice, with a total of 56 choices. Illustration was the least of consumers’ concerns, with only 24 choices. In terms of gender variables, both male and female consumers were interested in color and package format. Female consumers gave the lowest feedback on packaging pictures and illustrations. Male consumers gave the lowest feedback on packaging illustrations. 3.2.2 Consumer Preference for Kimchi Package Sizes Consumer preference for the size of the kimchi package was investigated. This question was set as a multiple choice question and included the weight of different sizes of kimchi (Table 3). 350g of kimchi was the most preferred kimchi with a total of 56 choices and the highest percentage of choices were made by 16 males and 40 females. This was followed by the smaller 80g kimchi with 44 choices, 1.2kg with 8 choices and 2kg with 2 choices, which was the least. Looking at the different age groups, it was observed that in the 10–20 age group, the same number of males chose 80g and 350g packages, accounting for 66.67% of the total number of choices. The highest number of females chose 350g packs with 75%. 21–30 age group: the highest number of males chose 500g packages with 60%. Females chose the highest number of 350, accounting for 54.55%. 31–40 age group: males chose the highest number of 350g packages, accounting for

Empirical Analysis of Packaging Design and Consumer Preferences

759

Table 6. Consumer preference for kimchi package sizes Gender Male

Female

Age

Percentage

Number

80g

350g

500g

1.2kg

2kg

10–20

66.67%

66.67%

33.33%

0.00%

0.00%

6

21–30

30%

30%

60%

0.00%

0.00%

20

31–40

0.00%

75%

25%

50%

25%

8

Over40

100%

0.00%

0.00%

0.00%

0.00%

2

10–20

25%

75%

0.00%

25%

0.00%

8

21–30

50%

54.55%

27.27%

4.55%

0.00%

44

31–40

75%

100%

0.00%

0.00%

0.00%

8

Over40

50%

50%

0.00%

0.00%

0.00%

4

Note. “Percentage” is the proportion of options for each question. “Number” refers to the number of people of different genders and age groups. The total number of participants in the questionnaire is 100

75%. Women chose 350g the most, accounting for 100%. Over 40 years of age: Men chose 80g the most. Females chose the 80g and 350g packages the most. 3.2.3 Consumer Preference for Packaging Fonts Consumer preferences for font designs for kimchi packaging were investigated. This question was set as a single choice question and included different styles of kimchi packaging font design (Table 3). The data showed that multilingual font combinations had the highest percentage with 66 choices. This was followed by serif fonts with 14%. Handwriting and non-serif fonts had the least share, both at 10%. Observe the differences Table 7. Consumer preference for packaging fonts Gender

Age

Serif

Handwriting

Non-serif

Multilingual font combinations

Male

10–20

33.33%

33.33%

0.00%

33.33%

6

21–30

10%

10%

0.00%

80%

20

31–40

0.00%

25%

50%

25%

8

Over40

100%

0.00%

0.00%

0.00%

2

10–20

0.00%

0.00%

25%

75%

8

21–30

13.64%

9.09%

4.55%

72.73%

44

31–40

0.00%

0.00%

25%

75%

8

Over40

50%

0.00%

0.00%

50%

4

Female

Percentage

Number

Note. “Percentage” is the proportion of options for each question. “Number” refers to the number of people of different genders and age groups. The total number of participants in the questionnaire is 100

760

A. Zhao et al.

between different age group variables. 10–20 age group: males chose sans-serif fonts for the least number of packages, and females chose multilingual mixed fonts for the highest percentage. 21–30 age group: multilingual font combinations accounted for a higher percentage of packages in all, with males accounting for 80% of packages and females accounting for 72.73% of packages. 31–40 age group: males chose sans-serif fonts for the highest number of packages, with a percentage of 50%. Females chose multilingual fonts with the highest number of packages, accounting for 75%. Over 40 years of age: Serif fonts were the most popular choice for males (100%). Both serif and multilingual fonts were chosen by 50% of women. 3.2.4 Consumer Preference for Kimchi Packaging Color Consumer preference for kimchi packaging colors was investigated. This was set as a single choice question and the question contained different colors of kimchi packaging (Table 3). The data showed that beige kimchi packaging was chosen by the highest number of people with 44%, followed by red with 32%. Green kimchi packs were selected by the lowest number of people with only 10%. Looking at the different age groups, it is observed that in the age group of 10–20 years: beige was the most preferred color by both males and females. 21–30 age group: beige was the most preferred color by both males and females, followed by red. 31–40 age group: beige was the most preferred color by males, and red was the most preferred color by females. Over the age of 40: men chose blue the most. Females chose both blue and red 50% of the time.

Table 8. Consumer preference for packaging color Gender

Age

Percentage Beige

Blue

Red

Green

Male

10–20

66.67%

0.00%

33.33%

0.00%

6

21–30

50%

10%

30%

10%

20

Female

Number

31–40

50%

0.00%

50%

0.00%

8

Over40

0.00%

100%

0.00%

0.00%

2

10–20

50%

25%

25%

0.00%

8

21–30

45.45%

9.09%

27.27%

18.18%

44

31–40

25%

25%

50%

0.00%

8

Over40

0.00%

50%

50%

0.00%

4

Note. “Percentage” is the proportion of options for each question. “Number” refers to the number of people of different genders and age groups. The total number of participants in the questionnaire is 100

Empirical Analysis of Packaging Design and Consumer Preferences

761

3.2.5 Consumer Preference for Kimchi Packaging Form.

Table 9. Consumer preference for packaging form Gender

Male

Female

Age

Percentage

Number

Glass jars

Plastic bagged

Plastic bucket

Can

Small volume shaped

10–20

33.33%

0.00%

0.00%

0.00%

66.67%

6

21–30

10%

0.00%

40%

10%

40%

20

31–40

25%

0.00%

75%

0.00%

0.00%

8

Over40

100%

0.00%

0.00%

0.00%

0.00%

2

10–20

25%

25%

0.00%

0.00%

50%

8

21–30

27.27%

9.09%

22.73%

13.64%

27.27%

44

31–40

50%

0.00%

0.00%

25%

25%

8

Over40

0.00%

0.00%

50%

0.00%

50%

4

Note. “Percentage” is the proportion of options for each question. “Number” refers to the number of people of different genders and age groups. The total number of participants in the questionnaire is 100

Consumer preference for kimchi packaging forms was investigated. This was set as a single choice question and the question contained different forms of kimchi packaging (Table 3). The data showed that small volume shaped kimchi packaging was the most preferred with 32%. This was followed by glass jars and plastic buckets, both with 26%. Plastic bagged kimchi packaging was chosen by the least number of people with only 6%. Looking at the different age groups, the 10–20 age group: males and females chose small volume shaped packs the most, while the 21–30 age group: males chose plastic jars and small volume shaped packs the most, with both accounting for 40% of the total. Women chose glass jars and small volume shaped packs the most, with both accounting for 27.27% of the total. 31–40 age group: Plastic buckets were the most popular choice for men, accounting for 75% of the total, and glass jars were the most popular choice for women, accounting for 50% of the total. Over 40 years old age group: men chose glass jars with the highest percentage. Females chose plastic buckets and small-volume shaped packages the most. 3.2.6 Consumer Preference for Kimchi Packaging Typography Style Consumer preference for Kimchi package typography was investigated. This question was set as a single choice question and included different typographic styles of kimchi packaging (Table 3). The data showed that packaging typography using kimchi images had the highest number of choices at 32%. This was followed by packaging with text as the main focus, supplemented by kimchi pictures at 22%. Packaging using kimchi illustrations had the lowest number of choices, with only 6%. Looking at the different age groups, we can see that in the 10–20 age group, neither males nor females chose

762

A. Zhao et al. Table 10. Consumer preference for packaging typography style

Gender

Male

Female

Age

Percentage

Number

Text Only

Illustration

Picture

Illustration and picture

Text-based with pictures

10–20

33.33%

0.00%

0.00%

33.33%

33.33%

21–30

20%

20%

40%

10%

10%

20

31–40

25%

0.00%

50%

0.00%

25%

8

6

Over40

0.00%

0.00%

0.00%

0.00%

100%

2

10–20

25%

0.00%

25%

25%

25%

8

21–30

22.73%

4.55%

22.73%

27.27%

22.73%

44

31–40

0.00%

0.00%

75%

25%

0.00%

8

Over40

0.00%

0.00%

50%

0.00%

50%

4

Note. “Percentage” is the proportion of options for each question. “Number” refers to the number of people of different genders and age groups. The total number of participants in the questionnaire is 100

packaging with kimchi illustrations, and the other choices were more evenly split. 21–30 age group: males chose packaging with pictures the most, at 40%. Women in this age group were more evenly split between different typography styles of packaging. 31–40 age group: males chose packaging with pictures the most, at 50%, and females chose picture packaging the most, at 75%. Over 40 years old age group: men’s choice of textbased and picture-based packaging accounted for the highest percentage. Females chose picture-in and text-based picture-supplemented packaging with the highest percentage.

4 Analysis of Packaging Design Strategies for the New York Kimchi Market

Table 11. Packaging design strategy for the New York Kimchi market Kimchi packaging

Strategy

Attractive points

Gender

Male: color > form > font > picture = illustration Female: color > form > font = picture > illustration

Design strategy analysis: More than pictures and illustrations, the color, packaging form and font of the product are the key to the design strategy, to grasp the color and packaging form that consumers are most concerned about

(continued)

Empirical Analysis of Packaging Design and Consumer Preferences

763

Table 11. (continued) Kimchi packaging

Strategy

Package sizes

Gender

Male: 350g = 500g > 80g > 1.2kg > 2kg Female: 350g > 80g > 500g > 1.2kg > 2kg

Design strategy analysis: Product developers should focus on developing products in small packages of 350g and 80g size, because the package design of 500g and smaller sizes below 500g is favored by consumers Package fonts

Gender

Male: multilingual font combinations > serif = handwriting > non-serif Female: multilingual font combinations > serif > non-serif > handwriting

Design strategy analysis: multilingual mixed typeface design best meets the needs of American consumers, and should consider the design of multiple language characters, followed by the need to consider the use of serif fonts Color

Gender

Male: beige > red > blue > green Female: beige > red > blue > green

Design strategy analysis: low saturation and soft beige is the most popular color, followed by the very popular red which is very recognizable to the product Package form

Gender

Male: plastic bucket > small volume shaped > glass jars > can > plastic bagged Female: small volume shaped > glass jars > plastic bucket > can > plastic bagged

Design strategy analysis: more convenient small volume shaped packaging is the most popular form of packaging for consumers. Easy to store glass jars and plastic drums are also the packaging forms that consumers are more concerned about Package typography style

Gender

Male: picture > text-based with picture = text only > illustration > illustration and picture Female: picture > illustration and picture > text-based with picture > text only > illustration

Design strategy analysis: the display of real pictures of kimchi is very important, and the real pictures should be the main focus in the kimchi packaging design. Next, you can use illustrations or text to assist

Based on the empirical analysis of packaging design and consumer preferences in the New York kimchi market, and summarized by data model. Presented about the packaging design strategy for the New York kimchi market as shown in the figure.

5 Conclusion Due to differences in culture, politics, and eating habits between countries, the consumer base of the New York Kimchi Market is not the same as that of the local Korean Kimchi Market. However, the New York kimchi market is booming with the rise of cross-cultural diets and Korean culture, as well as the pursuit of a healthy eating life. Therefore, understanding consumer preferences for kimchi product packaging design in the New York market and empirically analyzing them are essential to developing kimchi packaging design strategies for the New York market. Through research, we have compiled the corresponding kimchi packaging design strategy: Manufacturers and designers

764

A. Zhao et al.

should first focus on package color and package format. Small, irregular packages of 350g and 80g, and package formats such as glass bottles and plastic tubs are key points. Low-saturation and soft beige colors are preferred, followed by product-identifying reds. Mixed language typography is best suited to the needs of New York consumers. And realistic images should be the main focus in package design. We believe that research related to kimchi packaging needs to be further explored, and that there are still many gaps in our investigation and analysis. However, it is hoped that this study will provide some research to promote better adaptation of packaging design for kimchi products to the situation in New York, USA.

References 1. The Best Kimchi Brands Ranked. (n.d.). Tastingtable. https://www.tastingtable.com/890170/ the-best-kimchi-brands-ranked/ 2. Jo JH, Jeon CO, Lee DS, Park JM (2007) Process stability and microbial communi-ty structure in anaerobic hydrogen-producing microflora from food waste containing kimchi. J Biotechnol 131(3):300–308 3. DATABRIDGEMARKETRESEARCH. https://www.databridgemarketresearch.com/rep orts/global-kimchi-market. Accessed 8 Apr 2023 4. KOREAHERALD. https://www.koreaherald.com/view.php?ud=20220109000123 S. Accessed 29 Mar 2023 5. Statista, https://www.statista.com/statistics/1199251/south-korea-kimchi-export-value, Accessed 29 Mar 2023 6. DAESANG, https://www.daesang.com/, Accessed 29 Mar 2023 7. Kwon, L. S., Park, H. Y., Lee, S. R., Lim, hyeon Jin, & Kwon, Y. E.: Current Status and Future Prospect of the Kimchi Industry. Journal of Wellbeing Management and Applied Psychology, Vol 4(No 2), 1–13 (2021) 8. Lee JY (2010) A study on market strategy and package design for globalization of kim-chi focused on london. Brand Des. Res. 8(2):53–62 9. Market research world. https://www.marketresearchworld.net. Accesed 8 Apr 2023 10. S.T. Wang, E.: The influence of visual packaging design on perceived food product quality, value, and brand preference. International Journal of Retail & Distribution Management, Vol. 41 No. 10, pp. 805–816(2013) 11. Kotler P (2003) Marketing Management, 9th edn. Prentice-Hall, Upper Saddle River 12. Rundh B (2009) Packaging design: creating competitive advantage with product packag-ing. Br food j 111(9):988–1002 13. Choe PI, Kim MH, Kim SE, Choe DH (2003) A study on brand and packing de-sign of agricultural products-a case study in dangjin county. J. Korean Soc. Rural Plann. 9(4):53–57 14. Gopal, R. K., & George, M.: Packaging, a visual art: An analysis on packaging for FMCG goods. Asia Pacific Journal of Research Vol: I Issue XVI (2014) 15. Agariya AK, Johari A, Sharma HK, Chandraul UN, Singh D (2012) The role of packaging in brand communication. Int J Sci Eng Res 3(2):1–13 16. Woong J, Kim NH (2019) A study on preference and expression strategy of expres-sion elements in kimchi package design. Korea Des. Forum 24(2):77–86

Creating a Proxy Advertisement for Cognitive Response Evaluation in Visual Metaphor Studies Under Advertisement Research Partha Das1(B)

, Gurdeep Singh1

, Gobinda Banik2

, and Manoj Majhi1

1 Department of Design, Indian Institute of Technology Guwahati, Guwahati, Assam, India

[email protected] 2 Department of English, Basirhat College, North 24 Parganas, West Bengal, India

Abstract. Established business houses, start-ups, government agencies, etc., in the modern commercial world rely heavily upon their advertisement content for their brand imaging and subsequent commercial success. Advertisement content helps them to connect with their potential customers. A good advertisement must convey the intended message to the user and be exciting, engaging, and immersive. Effective use of visual metaphors in advertisement creation plays a vital role in this direction. Well-thought and rigorously imbibed visual metaphors in an advertisement enhance its usability and comprehensibility. Visual Metaphor Studies (VMS) are employed to identify and assess the visual metaphors used and their associated cognitive human response that depicts the engaging power of the advertisement under observance. While conducting VMS, there lie several challenges with respect to having access to choose proper advertisement for further study purposes, viz. Copyright issues, paucity of adequate advertisement material, etc. As such, there lies a dire need to design and develop one’s own advertisement based on various visual metaphors in several unique combinations. Such proxy/mock advertisements can help the researcher to have adequate cognitive human responses from the intended user population to the stimulus used within those. Such responses help the researcher assess the advertisement’s usability and comprehensibility. In this paper, the researchers have demonstrated ‘how to design and create a mock advertisement for cognitive response evaluation for prospective VMS.’ This paper thus can act as a ready reckoner for visual creators, graphic designers, and researchers to design mock advertisements and predict their usability by conducting VMS. Keyword: Cognitive Ergonomics · Human Response Evaluation · User-centric Visual Graphics · Advertisement Management · Business Innovation · Design of Experiment

1 Introduction In the modern commercial world, marketing and brand imaging based on advertisement have witnessed a prime role worldwide. In a prevailing competitive market, advertising/advertisement managers find it utmost urgent and essential to grab consumers’ © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 765–776, 2024. https://doi.org/10.1007/978-3-031-47281-7_62

766

P. Das et al.

attention through advertisements and sales promotion, and a sizable marketing budget is spent on advertising [1–3]. Realizing the importance and power of advertisement, established business houses, start-ups, government agencies, etc., rely heavily on their advertisement content for brand imaging and subsequent commercial success. Through their rigorous research studies conducted worldwide, researchers have a common consensus that advertisement content helps them (business houses, start-ups, government agencies, etc.) connect with their potential customers [4–6]. For this, they attributed that a good advertisement must convey the intended message to the user and be exciting, engaging, and immersive [7–10]. Effective use of visual metaphors in advertisement creation plays a vital role in this direction. Well-thought and rigorously imbibed visual metaphors in an advertisement enhance its usability and comprehensibility [11, 12]. Visual Metaphor Studies (VMS) are employed to identify and assess the visual metaphors used and their associated cognitive human response that depicts the engaging power of the advertisement under observance [13–17]. In the recent past, the use of visual metaphors in marketing has witnessed keen observation and scrutiny trends from researchers and academicians [18–21]. Typically, an analogous comparison between two terms in which one term is declared to be figuratively similar to the other despite the fact that the two terms have different literal meanings is known as a visual metaphor [22]. Visual metaphors play a crucial role in advertising as they have the power to convey complex ideas, evoke emotions, and create memorable associations with a brand or product. Several researchers have proposed different typologies of visual metaphor [18, 19, 21, 23] and how these typologies work on different product categories [16, 24, 25]. Researchers have used proxy or mock advertisements (ads) to understand these typologies and their effectiveness on consumers. Proxy ads look and sound like real ones, but their primary goal is research rather than commercial promotion [16, 21, 22]. From the above discussion, it is evident that advertising researchers frequently resort to “proxy ads” to gauge the efficacy of various approaches. Proxy ads have multiple applications in ad design research. In order to ensure that the actual advertisements meet the set standards before being shown to the public, proxy advertisements can be used for testing purposes. Researchers can determine the expected success of various advertising tactics by administering tests using proxy adverts to a representative subset of the target population [17, 21, 23]. However, creating/developing a reliable proxy advertisement for subsequent VMS is challenging. While conducting VMS, there lie several challenges with respect to having access to choose proper advertisement for further study purposes, viz. Copyright issues, paucity of adequate advertisement material, etc. As such, there lies a dire need to design and develop one’s own advertisement based on various visual metaphors in several unique combinations. Such proxy/mock advertisements can help the researcher to have adequate cognitive human responses from the intended user population to the stimulus used within those. Such responses help the researcher assess the advertisement’s usability and comprehensibility. Interestingly, there lies a paucity of academic papers/guidelines/reports on this important aspect of VMS, particularly on creating/developing one’s original proxy/mock advertisement. Understanding the criticality and worth of this lacunae, the researchers in this paper have demonstrated ‘how to design and create a proxy/mock advertisement

Creating a Proxy Advertisement for Cognitive Response

767

for cognitive response evaluation for prospective VMS.’ This paper thus can act as a ready reckoner for visual creators, graphic designers, and researchers to design mock advertisements and predict their usability by conducting VMS.

2 Methodology The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) model was used for conducting the systematic literature search [26, 27]. The online databases of Scopus and Web of Science were accessed to shortlist published research work (research articles, handbooks, etc.) using relevant search terminologies related to different aspects of ‘Visual Metaphor Studies’ and ‘Advertisement Research.’ The search was constrained to literature published in the English language ranging between the years 1990 to 2023. The articles were screened to include only those that fell within the appropriate categories of VMS and advertisement research, more particularly visual metaphors and associated cognitive response evaluation. This sorted literature helped researchers understand the background and essential/fundamental aspects of the topic under consideration. It helped them further fulfill the aim and objective of the current paper. Overall, the thirty-two most relevant articles addressing the current research topic were accessed online and were used for citation purposes in different sections of the present article to cover different dimensions related to the creation and development of proxy/mock advertisement for advertisement research. In addition, after gaining critical insights about the research topic and problem/gap identified, the researchers have demonstrated, by way of creating a context-specific proxy/mock advertisement, how other researchers intending to conduct VMS can create their own proxy advertisements. The present study focused on creating a context-specific proxy advertisement for the financial institutions (banking and non-banking financial sectors). It was intended for the marketing and sales promotion of their ‘Home Loan’ product. The present approach of creating/developing a proxy advertisement comprises three broad phases: planning, creation, and validation. The details of this creation process and critical insights gained from this activity are discussed in detail in the subsequent sections and sub-sections.

3 Creating/Developing Proxy/Mock Advertisement Creating/developing a proxy advertisement encompasses three broad phases and has several sub-components. A graphical representation of these phases/sub-components is illustrated in Fig. 1. The subsequent sub-sections provide the details of these phases for easy understanding and further implementation. 3.1 Phase I - Planning The foremost Phase of creating a mock advertisement is conducting rigorous planning about the basics/fundamental aspects of need/purpose, tools/techniques available, content availability, content source, ornamental features, etc. It requires three sub-phases

768

P. Das et al.

Fig. 1. Phases and sub-phases of proxy/mock advertisement creation/development process [in the purview of current case study] (Source: Author)

to conduct efficient context-specific planning for intended proxy/mock advertisement creation. Definition Exploration Several visual metaphors exist, and researchers have defined/ described those according to their intended need/ purpose based on various typologies. For example, Forceville describes the ‘hybrid metaphor’ as a visual metaphor in which source and target domains are fused. Further, these typologies are categorized according to their 1) Mode of Representation, viz., Juxtaposition, Synthesis, 2) Visual Scenarios, viz. Realistic Symbiosis, Replacement, Artificial Symbiosis, 3) Visual structure resulting in different types of visual rhetorical figures, viz. Two dimensional, three dimensional, and 4) Visual arrangement viz. Juxtaposition, Fusion, Replacement. For an efficient selection of the tools/techniques (essential visual components and their probable arrangement/structure), it is essential for a researcher to have in-depth knowledge of the basic definition, description scheme, and structure arrangements of the basic components related to the visual metaphors. Researchers have to understand the basic concepts to create different types of visual metaphor stimuli for their experiments. For the creation/development of the present mock advertisement, researchers have followed the definitions provided by Forceville [23] and Phillips & McQuarrie [21]. Their juxtaposition, Fusion, and replacement visual metaphor definitions were applied to create the present context-specific proxy ad for home loan advertisements. The rationale behind such selection was the wide applicability of their framework. Several researchers have used their framework for advertisement studies in various other fields, viz., food products, packaging products, utility products, etc. [16, 18, 24, 28, 29]. However, its use in the financial (banking/non-banking) sector was not explored. The

Creating a Proxy Advertisement for Cognitive Response

769

studies/definitions/descriptions provided by other researchers remained at the conceptual levels only and were not put into practice/application. Image Collection The second sub-phase of this Planning phase comprises a critical stage of gathering a collection of relevant images. Primarily, for this purpose, it is advisable to collect/fetch one’s own images from nearby locations, vicinity, etc. It will help evade copyright infringement, royalty matters, etc. However, the images from the ‘Open Sources’ can also be procured for ease and simplification. The selected images (searched and gathered) can further be used to compose different types of metaphors. In the present case study, relevant open-source Portable Network Graphics (PNG) format images were gathered and used. These were selected for the practical reason that the PNG format images remain of the same quality, no matter how many times those are edited and saved in the file. These images would not become blurry or distorted; thus, PNGs become an ideal choice for logos and graphs. The images closely related to the ‘Home Loan’ product were selected for further proxy advertisement development for the present study. An image of 1) a home/house depicting the intended need-based product, 2) a key depicting ownership/possession, and 3) a hand depicting help/lending of support for becoming capable of having future ownership (in the form of a loan) was chosen. Template (Design) Background Once the context, preferred components (tools/techniques), alteration scheme, and images are finalized, there lies the need to select an appropriate design template. The design template and its background must comprehend the selected context, images, etc., to convey a better understanding and must go well with those. Likewise, each set of advertisements’ backgrounds was considered extremely important in this proxy ad. Researchers in this proxy ad have kept the background flat; a flat background would be helpful to focus on the primary visual element and its visual structure. Moreover, the planning for the color selection of the background is also essential. In the present study, every set of advertisements created used similar backgrounds, hues, saturation, and tones to minimize the distraction of focused visual elements. 3.2 Phase II – Creation As already discussed in Phase - I, context, visual representation scheme, images, design template background, and color scheme were selected and finalized. In the second Phase, termed the ‘Creation Phase,’ several different proxy/mock advertisement sets were developed using various visual representation schemes. The position alterations, image alterations, and combination alterations of the selected images were done to achieve various proxy advertisements for the ‘Home Loan’ product. These were attained via the rigorous use of juxtaposition, Fusion, and replacement of visual metaphors. This Phase comprises two sub-phases: 1) Preliminary creation and 2) Final creation (re-designing and improvement).

770

P. Das et al.

Fig. 2. A context-specific proxy advertisement created for the ‘Home Loan’ product of the banking (financial/ non-financial) sector (Source: Author)

Preliminary Creation (Sub-phase I) This Phase comprises ad development based on certain pre-defined compositions depicting the advertisement’s intended depiction/understanding/underlying meaning/message. In practical concerns, following the researcher’s definitions of various types of visual metaphors, the development of the background comes first, followed by the arrangement of the visual structure. Figure 2, by way of an example of a ‘Home Loan’ advertisement, demonstrates the process of creating/developing various proxy advertisements. In this figure, under the juxtaposition visual metaphor, the fact of two things being seen or placed close together has contrasting effects. The house and key were placed side by side in a way that create a visual structure of juxtaposition metaphor [refer sub-phase I, (a)]. A fusion visual metaphor, also known as a hybrid or synthesized metaphor, combines the product (target) with what it is being compared with (source) to form a single visual element. For example, in attaining a fusion visual metaphor, the key is combined to make it an integral part of a house; it seems the key is the door of that house, creating a visual structure of a fusion visual metaphor [refer sub-phase I, (b)]. Another alteration can be achieved via replacement, i.e., the process of substituting one visual element for another so that the latter’s absence is felt in the former [21]. For example, the house vanished entirely in the replacement visual metaphor. Instead of a hand, the key vertically held upon the hand formed a replacement of the house [refer sub-phase I, (c)]. As discussed, three different proxy advertisements based on similar visual elements were created using various visual metaphors. These can invoke several sensory perceptions, and such responses help the researcher assess the advertisement’s usability and comprehensibility. Once these alternate proxy ads were created, the researchers carried out a pilot test to check/determine their preliminary understanding and comprehensibility. It helped the

Creating a Proxy Advertisement for Cognitive Response

771

researchers determine the scope of further improvements for enhancing the comprehensibility of the created proxy advertisements in the context of better adaptability with the context under consideration (Home Loan product). A pilot test was conducted with 20 students of design studies background (06 pursuing B.Des. Degree and 14 pursuing M.Des.) to have their opinion about the understanding of the meaning being conveyed by the images and co-relation towards the ‘Home Loan’ product. They conveyed their inability to have a stronger co-relation/connection with the ‘Home Loan’ product by witnessing the created proxy ads. They uttered the missing feature of logo and typography as the probable reason for this disconnect. As per their opinion, financial products must have logo and typology components to be more comprehensible. The researchers took their feedback as a strong inclination for improving/re-designing proxy ads and decided to adhere. Final Creation (Re-designing and Improvement) (Sub-phase II) In the purview of the feedback and suggestions received, the researchers further explored the brand communication/articulation-related aspects followed in the advertisement research. It revealed that a logo is a distinguishing feature that expresses a company’s views and ideals [30]. Generally, it is a standard practice to put the brand name and logo on the ads, which appear in virtually all corporate and brand communications, and these are considered essential parts of the brand’s identity [31, 32]. Thus, the researchers, having firm affirmation, decided to put a logo and product label on the proxy ads and re-designed those as per the steps followed earlier. For this, the the researchers created a fictitious logo named “Excellent Bank”. The fictitious logo was used to reduce the possibility of having a biased response that may occur on account of witnessing the known logo and relating it to their brand identity. Moreover, to avoid misleading respondents’ perceptions, all proxy ads created comprise the same font family, font size, and color scheme. Researchers included the ‘paper plane’ image in the logo considering the fact that in the context of banking, the ‘paper plane’ symbolizes belief, optimism, and desire; the motto line of the logo reads, “We safeguard and protect your money,” trying to establish a connection between consumers and the bank. Thus, utilizing all these new features, three new proxy ads were created for further use of cognitive responses and subsequent evaluation under VMS. [refer sub-phase II, (d-f)]. 3.3 Phase III – Validation Three different proxy ads created in the previous Phase can act as a specific stimulus and can invoke various cognitive responses, and these can be further studied in the context of VMS under advertisement research under observation. In order to accomplish the useful/meaningful creation of a proxy advertisement, it is necessary to validate the stimuli. The persons/practitioners of visual studies/visual communication belonging to various walks of life, viz. Designers, design faculty, and design researchers, may be engaged in the task. They can well assess the merits/demerits of the developed proxy ads and validate the created ads’ credibility. Once the experts validate the proxy ads created (following the above-mentioned process), they may further be put for rigorous field trial and testing with vigor.

772

P. Das et al.

In the current research, the proxy ads created were validated by engaging the three professionals from academic and non-academic backgrounds. Table 1 below represents the demographic details of the professionals engaged in the task. In order to verify/validate the accuracy of the proxy advertisement, the stimulus was presented to each of the experts. Each specialist provided insightful criticism of the stimulus and suggestions, which were included in the alteration/re-creation of the advertisements and afterward exhibited to validate their effectiveness. After several iterations, a panel of three specialists/experts were pleased with the proxy advertisement, generated a common consensus, and approved further testing. Finally, the three best proxy ads were created [refer sub-phase II, (d-f)], and these may be further used for cognitive response evaluation studies. Table 1. Demographic details of the experts engaged in the validation process in Phase – III (Source: Author) S. No

Designation/ Expertise Area

Department

Affiliation

1

Faculty

Design

Kala Bhavana, Santiniketan, Bolpur, Birbhum, West Bengal, India

2

Senior Research Scholar (SRF)

Humanities & Social Science

Indian Institute of Technology (IIT) Kharagpur, India

3

Designer

Graphic Design

Ogilvy, Kolkata, India

4 Discussion and Future Scope of Work 4.1 Insights from the Creation/Development Process In advertisement studies, particularly VMS, advertisements are required for conducting cognitive response evaluations to assess the comprehensibility and usefulness of the ads. The ads based on copyrighted material might pose several legal complicacies. To evade such unintentional legal complicacies, one can create one’s own proxy/mock advertisement. The current research demonstrates a step-by-step process for creating one’s own proxy/mock advertisement. It is a three-tier process comprising three different phases that are further divided into several sub-phases. The current research elucidates all those phases and sub-phases minutely and provides several insights through the process followed. A detailed proxy/mock advertisement creation process, ranging from planning to validation, has been demonstrated in the current research. The presently adopted methodology is in substantiation to other visual metaphor-related studies encompassing various

Creating a Proxy Advertisement for Cognitive Response

773

attributes of VMS, cognitive response evaluation, and advertisement management [4–7, 10, 12, 21, 23, 29, 31]. A proactive participatory approach was instrumental in creating proxy ads, involved design students in the creation phase, and engaged the visual communication experts in the validation phase. It kept all the stakeholders involved in the brainstorming session and gave their suggestions, feedback, and opinions on the current research. It helped create/generate three well-defined and validated proxy ads to be used for further cognitive response evaluation studies. Juxtaposition, Fusion, and Replacement visual metaphors defined by Forceville [23] and Phillips & McQuarrie [21] were applied to create the present context-specific proxy ad for ‘Home Loan’ advertisements. The adopted method of selecting context, related images, design creation process, and other associated parameters for VMS studies is in corroboration with other VMS and advertisement studies [10, 24, 30]. In general, the current research elucidates the significant phases and attributes of the creation/development process of proxy/mock advertisement in detail and in a structured framework. This framework, by way of an example (home loan product of financial institutions), illustrates a step-by-step process involved in creating the proxy ad, and it can be used as a guideline/protocol to develop one’s own proxy/ mock ads that may be well suitable for conducting one’ own original and authentic research and may help evade from any legal complicacies arising out of IPR infringement issues. Generally, the students engaged in design research, visual management studies, advertisement, and marketing management are more likely to approach the objective of creating a proxy advertisement methodically as a direct consequence of the educational framework that has been proposed in the current research. Moreover, the findings/insights of the current study and adopted methodology are also helpful for simplifying the process of creating proxy advertisements and determining the stimuli sought for scientific investigations. As discussed in the initial sections, proxy ads have become the industry standard in the field of advertisement research since they provide researchers the option to develop an assessment that is more useful in determining whether or not an advertisement is understandable and usable. As such, the designers, design researchers, and students have the creative abilities and skills necessary to provide the scope for a large range of stimuli; they are proactively/increasingly involved/engaged in proxy advertisement creation and development [17, 21, 23, 30, 32]. The proxy ads they created following the proposed guidelines/ framework may further be used to gauge the likely performance of different advertising strategies by testing them with a sample of the target audience/user groups/intended population samples and predicting their successful interventions. In order to test the effects of various ad variables on consumer behavior, these proxy ads can be employed in experimental designs. Further, the researchers may measure different marketing initiatives’ success using original and authentic proxy advertisements created through a standard and well-implementable process. In addition, by utilizing such proxy advertisements and the inferences drawn, the researchers can determine which types of advertising are most effective. To conclude, it is pertinent to recognize the importance and usefulness of the proxy advertisement as a valuable tool in the development of commercials, as they permit researchers to test out different approaches in a controlled environment before releasing them to the general public and, as such

774

P. Das et al.

the current research envisaging their creation in a well-articulated and implementable format is worth. In furtherance, as the created proxy advertisement addresses the user needs and uses context, it is expected that the proposed framework and proxy ads created using it will be highly beneficial for the VMS studies in the context of advertisement research and will benefit the business houses, start-ups, government agencies and public at large. 4.2 Limitations and Future Scope Although the researchers in the current research had aptly implemented the participatory creation/design process, ranging from defining the critical components to validating the created proxy advertisement, there are a few limitations with the current study. The current research is limited to a visual metaphor analysis scheme/definitions as proposed by Forceville [23] and Phillips & McQuarrie [21]. It uses their definitions of Juxtaposition, Fusion, and Replacement visual metaphor and is based only on three visual elements: home, key, and hand. More Visual elements may be chosen for other research studies, and standard definitions of visual metaphors provided by other researchers may be considered to have more rigorous explorations in the creation of proxy advertisements. Secondly, the present study focuses on validating the final proxy ads with three professionals and considering their opinions to freeze the proxy ads. It may not be a suitable choice to do so. More professionals may be engaged in further studies to evaluate and validate the proxy ads being developed. Similarly, in Phase II, sub-phase I design students were engaged in determining the initial comprehensibility of the preliminary ad, and their feedback was incorporated to re-design the improvised ads. Professionals with higher experience may also be engaged at this step to have more critical inputs, and the quality of final proxy ads may be enhanced to the highest content thereof. Overall, the present research has well articulated the proxy/mock advertisement creation process by way of an example. It can be easily replicated by novice and inexperienced design students/designers searching for a guideline/protocol/framework to create their own proxy advertisement for their further specific research purposes. They may use similar approaches to design mock advertisements and predict their usability by conducting VMS.

5 Conclusion By deploying a proactive participatory proxy ad creation approach plausibly, the researchers have devised an easily replicable and adaptable framework for other proxy ad creation needed for VMS studies. The presently followed and demonstrated approach considering various aspects related to visual metaphors, their use, and practical application for creating context-specific proxy ads would be very much beneficial to young researchers, designers, visual study proactivists, and advertisement/marketing managers looking to create their own proxy ads for their intended context-specific purposes.

Creating a Proxy Advertisement for Cognitive Response

775

References 1. Sama R (2019) Impact of media advertisements on consumer behaviour. J Creative Commun 14(1):54–68. https://doi.org/10.1177/0973258618822624 2. Samuel S, Anita T (2023) The relationship between trends in technology use and repurchase intention. Int J Data Netw Sci 7(1):449–456. https://doi.org/10.5267/j.ijdns.2022.9.001 3. Ye M, Dahana WD, Baumann C, Li X (2023) When does media multitasking induce store visit and conversion? the influence of motivational factors. Electron Commer Res Appl 59:101256. https://doi.org/10.1016/j.elerap.2023.101256 4. Williams J, Hartnett N, Trinh G (2023) Finding creative drivers of advertising effectiveness with modern data analysis. Int J Mark Res 65(4):423–447. https://doi.org/10.1177/147078 53221134258 5. Huang W., Thienmongkol R, Weeranakin N (2022) Research on the achievements of thai advertising content creation and social media channels and cross-cultural communication in China. In: 2022, 11th international conference on information communication and applications (ICICA). IEEE, pp 69–72. https://doi.org/10.1109/ICICA56942.2022.00020 6. Ramapriya M, Sudhamathi S (2019) Factors influencing effectiveness of online advertisement towards consumer purchase decision. Int J Recent Technol Eng (IJRTE), 8(26):714–717. https://doi.org/10.35940/ijrte.B1134.0782S619 7. Gu C, et al (2022) What do users care about? research on user behavior of mobile interactive video advertising. Heliyon 8(10). https://doi.org/10.1016/j.heliyon.2022.e10910 8. Bian Q, Forsythe S (2012) Purchase intention for luxury brands: a cross-cultural comparison. J Bus Res 65(10):1443–1451. https://doi.org/10.1016/j.jbusres.2011.10.010 9. Bauer HH, Sauer NE, Becker C (2006) Investigating the relationship between product involvement and consumer decision-making styles. J Consum Behav 5(4):342–354. https://doi.org/ 10.1002/cb.185 10. Frolova S (2014) The role of advertising in promoting a product. https://www.theseus.fi/bit stream/handle/10024/80777/Frolova_Svetlana.pdf 11. Kadry A (2016) The persuasive effect of using visual metaphors in advertising design. Int Des J 6(1):33–40. https://www.faa-design.com/files/6/17/6-1-kadry.pdf 12. Birdsell BJ, Tatsuta N, Nakamura H (2022) Effects of visual metaphors on enhancing the power of advertisements. J Psychol Behav Sci 5. https://iafor.org/journal/iafor-journal-of-psy chology-and-the-behavioral-sciences/volume-8-issue-1/article-1/ 13. Huang WH, Hsieh SY (2023) The impact of animal metaphors on consumer response to courtesy advertising. J Retail Consum Serv 75:103485. https://doi.org/10.1016/j.jretconser. 2023.103485 14. Söderlund C (2023) Recognizing non-designers’ contribution in the process of designing information on visual management boards: a metaphorical approach. Design Sci 9:e18. https:// doi.org/10.1017/dsj.2023.14 15. Madsen MJ (2018) A Q-method study of visual metaphors in advertising. Brigham Young University. https://scholarsarchive.byu.edu/cgi/viewcontent.cgi?article=7975&context=etd 16. Margariti K, Boutsouki C, Hatzithomas L, Zotos Y (2019) Visual metaphors in food advertising: a cross-cultural study. Food Res Int 115:338–351. https://doi.org/10.1016/j.foodres. 2018.11.030 17. Das P, Majhi M (2021) Gender role portrayal in indian advertisement: a review. In: International conference of the indian society of ergonomics Cham: Springer International Publishing, pp 461–471. https://doi.org/10.1007/978-3-030-94277-9_40 18. Gkiouzepas L, Hogg MK (2011) Articulating a new framework for visual metaphors in advertising. J Advert 40(1):103–120. https://doi.org/10.2753/JOA0091-3367400107

776

P. Das et al.

19. Lagerwerf L, Hooijdonk CM, Korenberg A (2012) Processing visual rhetoric in advertisements: Interpretations determined by verbal anchoring and visual structure. J Pragmat 44(13):1836–1852. https://doi.org/10.1016/j.pragma.2012.08.009 20. Phillips BJ, McQuarrie EF (2002) The development, change, and transformation of rhetorical style in magazine advertisements 1954–1999. J Advert 31(4):1–13. https://doi.org/10.1080/ 00913367.2002.10673681 21. Phillips BJ, McQuarrie EF (2004) Beyond visual metaphor: a new typology of visual rhetoric in advertising. Mark Theory 4(1–2):113–136. https://doi.org/10.1177/147059310404408 22. Phillips BJ (2003) Understanding visual metaphor in advertising. Persuasive imagery: a consumer response perspective 297–310. https://doi.org/10.4324/9781410607256 23. Forceville C (2002) Pictorial metaphor in advertising. Routledge, London. https://doi.org/10. 4324/9780203064252 24. Chang CT, Wu YC, Lee YK, Chu XY (2018) Right metaphor, right place: choosing a visual metaphor based on product type and consumer differences. Int J Advert 37(2):309–336. https://doi.org/10.1080/02650487.2016.1240468 25. Chang CT, Yen CT (2013) Missing ingredients in metaphor advertising: the right formula of metaphor type, product type, and need for cognition. J Advert 42(1):80–94. https://doi.org/ 10.1080/00913367.2012.749090 26. Singh G, Karmakar S (2023) Identification of appropriate tools and techniques for ergonomic evaluation in FMCG industrial shop floor. In: International Conference on Research into Design. Singapore: Springer Nature Singapore, pp 103–116. https://doi.org/10.1007/978-98199-0293-4_9 27. Singh G., Karmakar S (2021) Scope of improvement in assembly-line of FMCG industries through ergonomic design. In: Design for Tomorrow, vol 3, Springer, Singapore, pp 201–214. https://doi.org/10.1007/978-981-16-0084-5_16 28. Dehay EK, Landwehr JR (2019) A MAP for effective advertising: the metaphoric advertising processing model. AMS Review 9(3–4):289–303. https://doi.org/10.1007/s13162-0180131-1 29. Lundmark C (2005) Metaphor and creativity in British magazine advertising (Doctoral dissertation, Luleå tekniska universitet). https://www.diva-portal.org/smash/get/diva2:999922/ FULLTEXT01.pdf 30. Adîr V, Adîr G, Pascu NE (2014) How to design a logo. Procedia Soc Behav Sci 122:140–144. https://doi.org/10.1016/j.sbspro.2014.01.1316 31. Schechter AH (1993) Measuring the value of corporate and brand logos. Design Manag J (Former Series) 4(1):33–39. https://doi.org/10.1111/j.1948-7169.1993.tb00124.x 32. Henderson PW, Cote JA (1998) Guidelines for selecting or modifying logos. J Mark 62(2):14– 30. https://doi.org/10.1177/0022242998062002

A Contribution of a Graphic Design Intern at Sika Portugal Jorge Fonseca1 and Rita Coelho2(B) 1 ESMAD, Polytechnic of Porto, rua D, Sancho I, 4480-876 Vila do Conde, Portugal

[email protected]

2 uniMAD, ESMAD, Polytechnic of Porto, rua D. Sancho I, 4480-876 Vila do Conde, Portugal

[email protected]

Abstract. This article focuses on the development of an internship that took place in the master’s degree in design at the School of Media Arts and Design of the Poly-technic Institute of Porto with the company Sika, which is focused on the construction market. The objective of this internship was to create visual solutions for innovative purposes, including 3D design and communication design projects. In this essay, we examine part of the visual results of this Master in Design internship, attempting to debate them within the context of visual identity standards’ constraints as means to, not only guide and push further the creativity of an internship design student but also, as a means to reflect on the process of broadening the academic capabilities of a design student into the professional realm. The design internship served to rethink the company’s internal functions and selfimage. Methodologically, an experimental and empirical study was carried out utilizing design methodologies. Following that, visuals were created to clarify the functions and locations inside the departments of this global corporation’s main building in Portugal. The results, which have a cohesive language, respecting Sika’s visual identity, were well received by the corporate team and respond in a unified manner and as a method of grasping the intricacy of the company’s structure while offering a larger sense of connection to the full in-house workforce. Keywords: Graphic Design · 3D Rendering · Visual Identity

1 Introduction The current project emerged from an academic internship between the Master’s in Design and the international company Sika taken at the second and final year in this Master’s Degree in Design at the School of Media Arts and Design, Porto Polytechnic. Sika is one of the leading construction material suppliers in Portugal, specialized in chemicals. With a significant presence in the portuguese market, the company carries out an extensive marketing campaign each year to expand its reach, thus requiring a wellthought-out and structured design plan. This opportunity emerged within Sika’s marketing and design department, with a special focus on graphic design and its approach to social media. The main objectives © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 777–786, 2024. https://doi.org/10.1007/978-3-031-47281-7_63

778

J. Fonseca and R. Coelho

were to respect the tone of voice of Sika, with a systemic approach to easily captivate visitors to its website and physical products (packaging for large surfaces), maintaining a consistent visual language that would reflect a continuous line in their communication, while also establishing a feeling of belonging to its employers and attracting new, younger customers. Sika Portugal is a subsidiary of Sika AG, globally headquartered in Zurich, Switzerland and is a multinational group with a global presence in over 100 countries. Sika Portugal holds a prominent position in the Portuguese market and offers a wide range of solutions for the construction and industrial sectors. It specializes in chemical products. Particularly, Sika Portugal is based in Vila Nova de Gaia, in the Arrábida area. This location houses most of the necessary functions, except for production. The company comprises various departments, including customer support, sales management, marketing, commercial, product engineering, human resources, IT, finance, and, most importantly, the executive management. In the next section, we will define our design methodologies applied in this intership, in order to accomplish a contribution to the communication for a wider and also younger audience with visual consistency aligned with Sika’s main values and purposes. In the second section, entitled Project development and proposal, we will clarify our design within the scope of visual identity, combining information obtained through drawing, 3D technologies and a restricted color palette and typeface (Fig. 1 and Fig. 2).

Fig. 1. Sika logo

Fig. 2. Sika Portugal headquarters building, Vila Nova de Gaia

1.1 Methodologies Sika Portugal has a graphic manual that establishes all the guidelines and usage standards for the development and creation of graphic material within the company, ensuring maximum consistency and precision. We have therefore based this internship contribution on this theme, focusing on enhancing the graphic communication of Sika Portugal over a four-month period. Within the scope of this internship in a large-scale company, it was necessary to consider strategies to enhance both digital and physical interactions and, consequently, boost sales. In essence, the goal was to excel in corporate graphic communication. Before explaining the process and methodologies undertaken at the beginning of this internship, it is important to note that the objectives were focused on the construction

A Contribution of a Graphic Design Intern at Sika Portugal

779

sector, excluding the maritime and automotive industries of Sika, which were not within the internship scope. Analysing some of the references that inspired this internship, we must mention UserCentred Design (UCD). This approach places the user at the centre of the design process, ensuring that the solutions developed meet the needs and expectations of the end users. To achieve this, we researched the users, their behaviours, desires, and pain points. This understanding was then translated into insights that guided the design process, resulting in more intuitive and effective products and services. Starting with the analysis of its target audience, we found that Sika, is a specialist in the commercialization of chemical products for construction and industry, and caters to several groups of people, including construction professionals, architects, engineers, contractors, distributors, and building owners. These professionals are involved in construction and maintenance projects in which Sika’s products are used to ensure durability, safety, and structural efficiency. Based on the analysis of the target audience, we proceeded to identify personas. Through conversations with the sales department, it was discerned that there are primarily two personas when it comes to seeking Sika’s products. The first persona consists of construction professionals who have a good understanding of the field and know what they need when they visit large retailers or Sika distributors. They already have a clear idea of the type of adhesives or sealants they require, or the specific waterproofing membrane they need. Communicating the products to this group is relatively straightforward, allowing some room to focus on highlighting the new technologies that differentiate Sika from the rest of the market and showcasing the innovations that catch the attention of construction professionals. The second persona identified is the so-called “do-it-yourselfer,” as referred to in large retailers. These individuals are looking to carry out repairs or projects at home on weekends or in their spare time. Generally, they are unsure about the products they need to address their specific problem, providing an opportunity for Sika to communicate its products effectively. With these two primary groups of people who frequently seek Sika’s products identified, it became easier to devise a visual communication strategy and approach.

2 Project Development and Proposal Visual identity is crucial for brand recognition and communication. It serves as a strategic tool that helps organizations convey their values, personality, and offerings to their target audience. However, visual identity design works with constraints. The presence of a graphic manual in Sika is evident, with a predominant color scheme of yellow with orange tones, and a defined typography for all works, using the font Klavika, only varying in the weights within the same font family. This ensures a complete and clean layout. The Sika logo is a fundamental part of the company’s visual identity and should be used in accordance with specific guidelines, maintaining its proportion, alignment, and visual integrity. There are two important rules: application on a yellow background and alignment in the bottom right corner, adhering to the precision defined in the graphic manual. In addition, Sika closely monitors the performance of its activities on social media

780

J. Fonseca and R. Coelho

platforms, analysing metrics and feedback to evaluate the effectiveness of the strategies employed. This continuous analysis allows the company to adjust its approaches and optimize its actions, ensuring a consistent and impactful presence on social media. These activities are carried out by the Marketing department, always reporting to the other departments to assess their effectiveness (Fig. 3 and Fig. 4).

Fig. 3. Sika Visual Identity Guidelines

Fig. 4. Sika Visual Identity Guidelines

The main objective of this project, particularly, was to develop visuals that conveyed a relaxed atmosphere, placing less emphasis on architectural representation and focusing more on identifying the departments within the building. Additionally, the pictogram had to adhere to the brand guidelines, using yellow and orange tones, creating another “Sika world” style, a unique atmosphere. Joan Costa in its seminal book Design para os olhos: marca, cor, identidade, sinalética highlights that every corporation needs to comunicate its uniqueness and its presence, for itself and for its customers (Costa 201). For Sika, a palette composed of yellow, orange and red tones, a robust geometric approach, together with curved, floating and simple lines was a way to signify both the quality and precision of its industrial products and specialized services and its commitment to a wider audience. The following images is work done by the main author. We started by conducting in-depth research on the building, analysing its structure, department layouts, and architectural features. Started sketching a 2D aerial perspective based on images from Google Maps (Fig. 5 and Fig. 6).

Fig. 5. Aerial Perspective Sika Portugal

Fig. 6. Sika 2D Building Sketch by the author

Based on the aerial perspective, the outlines of the building were used to create its silhouette, aiming for greater readability. From this initial sketch, we proceeded to

A Contribution of a Graphic Design Intern at Sika Portugal

781

designate the three most prominent floors of the building, representing each department and the areas they occupied (Fig. 7).

Fig. 7. 2D Floor Plan of Sika and building elevation with ground floor, third and fourth floors belonging to Sika Portugal, by the author.

Starting with rough sketches on paper, in order to loosely draw the 3D volume without using correct isometric grids, it gave us a sense of the space available to work with, and we quickly realized that the lower right hand corner of the building appeared too short (Fig. 8 and Fig. 9).

Fig. 8. 2D third Plan of Sika building, by the author

Fig. 9. 2D fourth Plan of Sika building, by the author

Creating a colour palette that aligned with the brand’s context, we proceeded to divide each department with its corresponding label, excluding common areas such as restrooms and lunchrooms. The goal was to create an illustration rather than a realistic architectural floor plan. Based on these plans, we took the opportunity to represent the internship location on

782

J. Fonseca and R. Coelho

the 3rd floor, surrounded by the commercial and sales area, where it took place and the communication was facilitated through direct and physical proximity, working together to drive innovation. It’s important to note that the 1st and 2nd floors are primarily occupied by other companies, spaces subleased to Sika. The idea of an isometric perspective immediately came to mind, especially since it hadn’t been done at Sika before. It was a style that had also been used and proposed by the intern in other projects during this internship, maintaining visual consistency. This project came about after the initial stages of said internship (Fig. 10 and Fig. 11).

Fig. 10. Sika Building Sketch

Fig. 11. Digital Illustration of Sika Building

The digital representation of the building was created using Illustrator, which facilitated the next step of 3D modelling. Keen on developing the work even further, we moved on to Blender (Fig. 12 and Fig. 13). During the 3D modelling phase, we utilized some pre-existing 3D models from Sika’s database that had been used in previous projects. For example, the yacht model proved to be a time-saving asset.

A Contribution of a Graphic Design Intern at Sika Portugal

Fig. 12. Unrendered Floor 0 Perspective

783

Fig. 13. Floor 0 Perspective

The focus was on the color palette and lighting, as aimed to establish the “Sika world” concept. This was the main goal we wanted to achieve (Fig. 14, Fig. 15, Fig. 16, Fig. 17).

Fig. 14. Unrendered Floor 3 Perspective

Fig. 15. Floor 3 Perspective

Fig. 16. Unrendered Floor 4 Perspective

Fig. 17. Floor 4 Perspective

As we progressed through the modelling and rendering stages, we refined the scale, volume, and lighting and incorporated spiral staircases, resembling those found in the actual building. Emphasis was placed on the most important objects, representing each

784

J. Fonseca and R. Coelho

department, in a freer and almost cartoon-like style. Aimed to break free from the constraints of architectural floor plan drawings. Moving forward, the final phase involved merging the three images onto a physical support with captions, preparing it for printing. The objective was to draw the icons in 2D. Drawing only the silhouettes of each icon was an elaborate process. It allowed to effectively communicate these elements on the final support, ensuring that viewers can identify what they are seeing. This exercise proved beneficial for learning process and supported as we are still fine-tuning the final images; it will later be shown from digital screens in the Sika building in Vila Nova de Gaia (Fig. 18 and Fig. 19).

Fig. 18. Banner for Sika’s website, by the author

Fig. 19. Banner for Sika’s website, by the author

Furthermore, the intern also conceptualized and developed these banners, aimed to differentiate industrial construction and reflect the company’s visual identity. With a distinctive orange style, the banners were used to convey Sika’s presence in the industry, highlighting its innovation and modernity. They were strategically positioned at trade fairs and industry events, effectively communicating Sika’s unique positioning. Taking the opportunity to show that the same style Sika world was used in other projects maintaining the coherence of the company, helping to raise the bar in the digital world. The orange coming from the Sika logo also lends itself, due to the color’s symbology, to a young, fresh and fun identification, giving the emotional side so necessary to brands (Raposo 2009; Travis 2000). This was essential in defining the created graphics, following the established guidelines. The intern contributed to the creation of visual elements aligned with Sika Portugal

A Contribution of a Graphic Design Intern at Sika Portugal

785

corporate identity, ensuring consistency and coherence across all graphic applications. In addition, constructive feedback was provided for the continuous improvement of the graphic standards manual. Through a user-centered design approach and strategic use of persuasive tech-niques, the banners communicated Sika’s innovative solutions and expertise. Regular feedback loops with Sika’s marketing team ensured seamless alignment with the company’s overall brand strategy.

3 Final Considerations During this internship at Sika, the opportunity was validated to engage in enriching experiences and gain practical knowledge that complemented academic education. Throughout this journey, we were able to apply concepts to real life situations, developing essential skills and competencies for a future professional career. One of the key observations we can make is the value of continuous learning. It was constantly encouraged to seek knowledge beyond direct responsibilities. This allowed us to understand the importance of staying updated in a constantly evolving market. The opportunity to work in a team was also a significant aspect of the internship experience. Collaborating with experienced professionals provided with a comprehensive insight into visual identity, design, marketing practices and methodologies. The exchange of ideas and knowledge shared resulted in more solid and enriching projects. Having analysed the graphic outcomes of this internship, we feel that constraints fostered design creativity by pushing the design intern to find innovative solutions. When faced with limitations, the design intern was encouraged to explore unconventional approaches, experiment with alternative design elements, and consider novel interpretations of brand attributes. Sika’s visual constraints served as creative catalysts, prompting the exploration of uncharted territories. Subsequently, the visuals produced clarify the functions within the various departments of this global company in a cohesive means of understanding the complexity of the building and giving a wider sense of belonging to the whole in-house team. Sika’s house style, in terms of color palette and aim in simplicity, was respected, and challenged in an innovative approach, once the design student and intern – with a fresh perspective – successfully proposed and developed 3D renderings of synthetic images that play with the idea of a Sika world: Sika’s products and services. The results differ from existing approaches in their visual portfolio but maintain coherence with Sika’s values and objectives. This design internship provided the master student-intern with a strong understanding of the needs and expectations of younger wider audiences and, being relatively new to the design field and more closely connected to contemporary user preferences, it also provided a hands-on experience in applying the academic skills and knowledge to realworld projects, while also arranging valuable insights into designing for diverse user groups. This design contribution became validated by actively participating in Sika realworld design projects, it provided an outside view that challenged existing paradigms, fostered collaboration, and pushed the boundaries of visual communication. As outcomes, we should also highlight the fact that the analyzed visual products were well received in Sika’s Portuguese headquarters and the finished internship has resulted in

786

J. Fonseca and R. Coelho

landing a job post for the design student, once the master is completed. As forthcoming developments are concerned, we would like to deepen the design of the pictograms, in addition to other visual and digital products, aligned with a cohesive visual structure that broadens Sika’s communication. As possible extensions of the results and additional research opportunities that arise from this study, we would like to highlight 3D renderings of Sika’s products and their correct usage and placement, through images and animations that portray an integrated and coherent system to be developed in future research. The hallmark of the student’s work at Sika was the creation of isometric 3D images of Sika building, this innovative approach significantly deviated from conventional graphic design practices within its corporate sector. Traditionally, companies tended to rely on static, two-dimensional imagery for branding and marketing purposes. The incorporation of isometric 3D graphics, especially in an industrial context, is a departure from the norm and sets this visual communication apart from Sika’s competitors. The orange and yellow color scheme, derived from its logo, created a seamless integration with its visual identity. This consistent and harmonious use of colors across different departments and sectors reinforced the Sika image as a cohesive and unified entity. The results of this project offer numerous possibilities for future extensions: the Sika World style can be further expanded and integrated into various marketing collateral, from brochures to digital campaigns, ensuring a consistent and memorable brand presence. In summary, the internship was seen – both by the students and its mentors at Sika and at ESMAD – as a valuable experience. Through the activities carried out, design and marketing skills were promoted, fresh perspectives, user-centric approaches, and innovative ideas contributed to working in a leading company in the industry sector. The integration of the interns’ insights and contributions lead to enhanced visual communication from within the company as well as for its external targets and stakeholders. Additionally, it also permitted the seizure and enforcement of essential knowledge for the student’s professional journey.

References Chaves N (1988) La imagen corporativa Coelho R (2022) Flexible ID: to adapt is to resist. In: Raposo D (ed), Design, visual communication and branding. Cambridge Scholars Publishing, UK, pp 82–107 Costa J (2011) Design para os olhos: marca, cor, identidade, sinalética. Dinalivro, Lisboa Moura M (2018) O Design que o design não vê. Orfeu Negro, Lisboa Olins W (2008) Wally olins: the brand handbook. Thames & Hudson, UK Raposo D (2019) Communicating visually. Cambridge Scholars Publishing, UK Raposo D (2009) Design de identidade e imagem corporativa. In: I+Diseño: Revista internacional de investigación, innovación y desarrollo en diseño, vol 1, p 194 Travis D (2000) Emotional branding: how successful brands gain the irrational edge. Prima Venture, Califórnia

The Expressivity of the Typographic Composition Elisabete Rolo(B)

and João Gomes

CIAUD, Research Centre for Architecture, Urbanism and Design, Lisbon School of Architecture, Universidade de Lisboa, Lisboa, Portugal [email protected], [email protected]

Abstract. In the early stages of teaching graphic design, in addition to the typographic rules, it is essential to train for the mastery of spatial composition and typographic expressivity. In this context – using the methodology of critical literature review – it seems vital to focus on the various manifestations that, throughout the history of graphic design, assumed the word as expressive matter – namely, visual poetry, Futurism, Dadaism, and Postmodernism. Furthermore, besides establishing this historical compilation, we also reflect on the factors that influence the expressive character of typography, trying to identify a set of fundamental principles to transmit in initial education in graphic design. Finally, we also seek to critically reflect on an exercise of an initial training course in graphic design that addresses the topics mentioned, pointing out suggestions for improvement to enhance the applicability of the knowledge extracted from the exercise to works of greater complexity. Keywords: Graphic Design · Typography · Typographic expressivity · Typographic composition

1 Introduction “In the 21st century, graphic design is ubiquitous; it is a major component of our complex print and electronic information systems. (…) Technology’s relentless advance has dramatically changed how graphic designs are created and distributed to a mass audience. However, the fundamental role of the graphic designer—giving expressive form and clarity of content to communicative messages—remains the same” (Meggs, s.d.). “Typography is the face of language” (Lupton 2006 [2004]) and “exists to honour its content” (Bringhurst 2004 [1992]) are two of the most expressive aphorisms about typography. Graphic design essentially works with written content expressed through typography. Therefore, typography will always be essential and inseparable from disciplinary practice because it performs multiple functions. Through its formal characteristics, it arouses varied feelings in readers and communicates beyond the semantic dimension: it confers personality and identity, builds recognition, establishes hierarchies of information, and puts in symbiosis verbal and visual language (Heller & Ilic 2012; Hyndman 2016). © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 787–796, 2024. https://doi.org/10.1007/978-3-031-47281-7_64

788

E. Rolo and J. Gomes

“Teaching a student graphic design before teaching her type and typography is like teaching a baby to walk before she crawls. (…) Type is the formal expression of writing, and writing is the physical representation of language. Type is the lingua franca of graphic design – a vessel that holds the codes that represent ideas that convey meanings that trigger understanding. Type is the single most important graphic design element, and typography is the most consequential course (or set of courses) a student can take as an undergraduate”. (Heller 2004, p. vii). Most of the knowledge transmitted in the teaching of typography consists of disciplinary nomenclature and composition rules inherited from 500 years of tradition and whose primary purpose is the organization of information. However, in the initial training of graphic designers, it is also essential to approach typography as an element with the potential for expressive composition.

2 Expressive Typographic Composition In a very simplified way, typography and graphic design intend to fulfill two types of purpose: expressivity and legibility. Then, both characteristics are present in all works, but in some, legibility is more important, and in others, it is expressivity that has more weight. In cases where the occupation of typographic space aims to achieve legibility, the grid is a primary tool for structuring and arranging elements, as we studied in the article “The Typographic Grid in the Editorial Project: An Essential Resource to the Graphic Consistency and Perception” (Rolo 2018). However, in the scope of the present article, we do not dwell on the grid. We intend a freer and more exploratory approach focused on expressivity. As Timothy Samara states, structuring through the grid is not always the best way to serve the content “(…) sometimes content must ignore structure altogether to create specific kinds of emotional reaction, or to invoke an important narrative association; sometimes it requires a more complex intellectual involvement on the part of the audience in order to resonate more profoundly. (…) Exploring alternative means for organizing and visualizing content often creates opportunities to transcend its literal, pragmatic meaning and to engage and inform with a richer, more specific, and uniquely memorable, visual language” (Samara 2017). This approach to design and typography may neglect legibility, but this will not be problematic because, as Cramsie states, “Each of us is predisposed to scan our environment in search of meaning, and the strength of this predisposition enables us to read text even under quite challenging conditions.” (Cramsie 2010, p. 300). 2.1 The Expressive Use of Typography in History In the history of graphic design, we can identify several moments in which typography was used in a fundamentally expressive way. One of the first manifestations in which we can perceive this is visual poetry, which, according to Meggs (2016), dates back at least to the work of the Greek poet Symmias of Rhodes (c. 3rd century BC) – “Called standard poetry, the verse that explored this idea often took the form of religious objects or symbols.” (Meggs 2009, p. 321). However, this type of poetry was prevalent and evolved,

The Expressivity of the Typographic Composition

789

the works of Stéphane Mallarmé (1842–1898) and Guillaume Apollinaire (1880–1918) being paradigmatic examples. In 1897, Mallarmé published the poem “Un Coup de dés” (A Throw of Dice), consisting of seven hundred words arranged over twenty pages and composed in a varied range of type: uppercase, lowercase, roman, and italics. Instead of the traditional composition of the poem surrounded by white, empty margins, Mallarmé used the white space as if it were “silence” to reinforce the text’s meaning. Moreover, instead of arranging the words sequentially, he placed them in unexpected positions on the page to reinforce their expressiveness. He thus managed to create different reading rhythms. In 1917, Apollinaire coined the term “Calligramme”, combining the words “calligraphy” and “ideogram” (MOMA, n.d.). The following year, he published the book “Calligrammes” – a compilation of visual poems combining letters to form a drawing, figure, or pictogram. In these poems, Apollinaire explored the potential fusion of poetry and painting, introducing the concept of simultaneity in typography linked to the time and sequence of the printed page (Meggs 2006, p. 322). “The materiality of the letters, their graphic shapes, rhythm, and flow all work together to add a visual dimension to the poem.” (Eskilson 2007, p. 149). The poem “Il Pleut” (It Rains) also integrated in this compilation but published for the first time in 1916 in the periodical “SIC” (“Sons, Idées, Couleurs” - “Sounds, ideas, colors!”), demonstrates well this type of work. “The diagonally slanting letters of the poem create a visual likeness of falling rain, its subject, exemplifying Apollinaire’s astute play at the juncture of visual, verbal, and auditory modes of expression” (MOMA, n.d.). In visual poetry in Portugal, the works of Salette Tavares (1922–1994), E.M. de Melo e Castro (1932–2020), and Ana Hatherly (1929–2015) stand out. Jorge dos Reis (2012) uses the term “typographic poetry” to designate this type of manifestation, including graphic visual poetry, concrete poetry, and experimental poetry. Another moment in the history of design in which we can perceive the use of typography as an expressive matter is Futurism. Created in 1909 by journalist and poet Filippo Marinetti (1876–1944), it aimed to create a revolutionary change in Italian society that would free it from historical classicism and allow it to compete in the modern industrial world (Eskilson 2007, p. 158). “Futurism was one of the first radical aesthetic movements to achieve a complete rupture with traditional forms.” (Friedl 1998, p. 52). Its works manifested the cult of speed and the machine and, in the graphic context, artists “liberated themselves from tradition, animated their pages with dynamic, non-linear compositions achieved by collaging words and letters” (Meggs 2006, p. 251) that intended to be “words of freedom”.1 ) and express noise, movement, and speed or visually reinforce the ideas in the text. Two of the most recognized works of Futurism that influenced graphic design are “Zang Tumb Tuuumb” (1914) and “Les mots en liberté futuristes” (1919), in which

1 From the Italian “Parole in Libertà” title of Marinetti’s book, dated 1914. It was a kind of

“verbal painting” intended to celebrate the then-recent battle of Tripoli. In it, “the poet tried to find visual equivalents for sounds through different word formats and sizes” (Hollis, 2000, p. 36).

790

E. Rolo and J. Gomes

Marinetti makes use of the materiality of words to reinforce the message, similarly to Mallarmé (Cramsie 2010). With some points of rapprochement to Futurism, Dadaism emerged in 1916 as an anti-war protest movement, with anti-art, shock, and absurdity demonstrations. Since it defended the ideal of anti-art, Dadaism never had a very defined style, and it was in graphic design that it presented greater consistency (Cramsie 2010, p. 180). It was characterized by the combination of diversified fonts and the miscellany of cut-outs and images in collages that conveyed high tension, as well as the use of photomontage (Drucker & McVarish 2009, p. 192) – techniques that were, at the time, quite disruptive. The avant-garde art movements reflected in graphic design with the Bauhaus, the New Typography, and the Swiss Style (in a very general way). In these, typography was seen as more functional and organized, not exploiting the potential of expressive composition so much. In the Swiss Style, there is a primacy of rules, and typography is always associated with the grid, which does not promote freedom. However, after the years dominated by the Swiss Style, there were again frequent manifestations with typographic compositions that were freer of conventions and more in the service of expressiveness. The work of Herb Lubalin (1918–1981) is a case in point. Lubalin developed his career in the USA between 1939 and 1980 and revolutionized the American editorial and advertising landscape. He considered himself a “graphic expressionist” who used typography and lettering as a creative means of expressing an idea and provoking an emotional response from the observer (Graphéine 2022). Similarly, Robert Massin (1925–2020) became notable in the 1960s for his experimental typography that revealed affinities with Futurist and Dadaist typography and materialized in dynamic compositions using letters in a visual and concrete way. He worked extensively in editions of poetry and plays for the French publishing house Gallimard and stood out in work developed for the plays of Ionesco, linked to the theatre of the absurd (Meggs 2009, pp. 571–572). For example, in his design for Ionesco’s “Délire à deux” (Delirium for two), the words take on an expressionist character – they leap, run, overlap, and blur into inkblots in a calligraphic homage to the surreal, nonrepresentational ideas of piece. We can claim that his work anticipates the spatial possibilities inherent in the computer graphics of the 1980s (Meggs 2009, pp. 571–572). The period of postmodernism in design was marked by opposition to the cold formalism of the modernist tradition and was prolific in works that explored expressive typographic composition. We can state that this movement had several directions: firstly, some relaxation of the rules carried out by Swiss-style designers. Then there was new wave typography, which began in Switzerland with Wolfgang Weingart (1941–2021), the exuberance linked to the Memphis Group, the retro approach that recovered expressions from the interwar period, and the electronic revolution brought about by the introduction of the computer in the 1980s (Meggs & Purvis 2006, p. 605).

The Expressivity of the Typographic Composition

791

In this context, the work of Wolfgang Weingart (1941–2021) stands out, not only as a designer but also as a lecturer at the Basel School of Design. While not wholly renouncing the Swiss Style, he questioned the typography of absolute order and sharpness, rejecting the right angle as the exclusive principle of organization. Instead, he arrived at a more complex, jovial, and intuitive design, with great richness of visual effects, testing, at times, the limits of legibility (Meggs and Purvis 2009, pp. 601, 605) (Cramsie 2010, p. 298). In the wake of this type of expression, we should mention the works of Dan Friedman (b. 1945), April Greiman (b. 1948), Paula Scher (b. 1948), and Katherine McCoy (b. 1945) (for the Cranbrook Academy of Art) – here already taking advantage of the potentialities of the computer. We must also mention the works of David Carson (b. 1955) and Neville Brody (b. 1957). With different methods and plasticity, we must also mention the British Alan Kitching (b. 1940), who has taken advantage of letterpress printing with wood and metal characters in a more independent work, but in which the expressiveness of typography also takes precedence.

3 Teaching Expressivity in Typographic Composition 3.1 Some Principles of Typographic Composition To better convey the practical knowledge of expressive typesetting, besides knowing the historical references, it is helpful to analyze some introductory works to the Graphic Design discipline that theoretically address the subject. Thus, we reviewed the books Carter 1999; Lupton, and Philips 2015; Poulin 2012; and Samara 2004. We chose these books because they address the topic of introduction to graphic design from a didactic perspective. In them, we can identify some crucial topics to initiate students in the arrangement of elements in space, which can be considered fundamental principles of graphic design. In Carter’s work (1999), we find the factors: Balance, Direction, Grouping, Proximity, Repetition, Rhythm, and Rotation. In the work of Poulin (2012), we can find the factors: Movement, Space, Balance, Symmetry, Asymmetry, Tension, Contrast, and Framing. In the work of Samara (2004), we identify the factors of Texture and Rhythm. Finally, in the work of Lupton and Phillips (2015), we identified the factors: Rhythm and Balance, Scale, Texture, Framing, Layers, and Patterns. Based on the analysis of these factors, we sought to create a synthesis capable of simplifying the knowledge. Thus, we propose to divide between factors that relate to space and factors that relate to size (also both interrelated). Regarding space-related factors, we included two features – balance (symmetry/asymmetry) and rhythm (repetition, proximity, and direction). In the factors related to size, we highlight scale and framing.

792

E. Rolo and J. Gomes

“Balance occurs when visual elements within a composition are equally distributed and arranged to communicate a feeling of stability and harmony.” (Poulin 2012, p. 113). In balance, we must consider mainly two strategies: symmetry and asymmetry. Symmetry is based on the distribution of elements on at least two sides along a common axis, which inherently conveys stability. However, the balance need not be static (Lupton & Phillips 2015, p. 49). Asymmetry, also called ‘dynamic balance’, occurs when elements are intentionally arranged unevenly in composition and appear random and dynamic (Poulin 2012, p. 117). Rhythm is a strong, regular, repeated pattern: the beating of drums, the pounding of rain, and the sound of footsteps. Speech, music, and dance use rhythm to express form over time. Graphic designers use rhythm in the construction of static images and in books, magazines, and motion graphics with duration and sequence. While pattern design typically employs uninterrupted repetition, most graphic design forms seek rhythms punctuated with change and variation (Lupton & Phillips 2015, p. 49). Through the repetition of identical elements, we can achieve a certain rhythm. However, this repetition occurs in typography with distinct and contrasting elements (Carter 1999, p. 49). Directly related to repetition is proximity, which offers many expressive possibilities, given that typographic elements can range from overlapping to very spaced. Similarly, the direction factor also allows interfering with rhythm – western typographic conventions dictate that typography is displayed horizontally, with a reading direction from left to right. By rotating the characters, lines, or blocks of text at other angles, very different and expressive reading and perception rhythms are obtained (Carter 1999). Scale mainly refers to the subjective perception of the observer and the relationship between the various elements in the same visual field. When we seek legibility, scale is essential, as it has to allow the text to be easily readable at the distance the graphic object is meant to be read. However, the perception of scale may be conditioned by the context of the objects. A graphic element can appear larger or smaller depending on the size, positioning, and color of the elements surrounding it. When the elements are all the same size, the design looks simple or boring. Size contrast can create a sense of tension, as well as a sense of depth and movement, where small shapes tend to recede, and large ones move forward (Lupton & Phillips 2015, p. 62). Directly related to scale, we have the framing, which allows us to reinforce its effects. For the best preparation of students in the disciplines of introduction to graphic design, in addition to the theoretical approach (and its illustration with examples) of the parameters we mentioned above, it is also essential to teach the principles of visual perception of Gestalt – “prägnanz”, figure-background, similarity, proximity, continuity, closure – that constitute basic knowledge for any discipline in the field of visual arts. 3.2 An Exercise for Initial Training in Expressive Typographic Composition In the context of a curricular unit for introduction to Graphic Design in an undergraduate course in design, we have proposed the following exercise: From a list of poems provided, choose one to work on. 1. choose a stanza and compose it by introducing the element space to emphasize its meaning, always maintaining the same size in the letters.

The Expressivity of the Typographic Composition

793

2. Use the variable size (one word or several with different sizes, or even just one letter, or several letters with another size). 3. Compiling all the knowledge acquired in the previous paragraphs, dramatize the poem’s content. Then, compose it expressively, using only variables and typographic resources. This exploratory exercise focuses on the various strategies of typographic composition with a view to expressivity concerning the occupation of space as a visual field. It puts into practice the theoretical concepts described above. It has several types of practical applicability, especially in the design of graphic objects in which expressivity is the main requirement of the work, such as posters or book covers, for example. In Figs. 1, 2 and 3 can observe the proposals of some students. Observing a wide range of responses to this exercise over several years, we conclude that it could benefit from becoming more concrete, with students considering each of the basic principles – balance, rhythm, scale, framing – separately, according to more precise guidelines. On the other hand, the students showed more difficulties in the first step of the proposed exercise, mentioning that using only one variable – space – is too restrictive, so it is worth considering starting with less limiting situations.

Fig. 1. Working with space. Source: School work from Martha Reis.

794

E. Rolo and J. Gomes

Fig. 2. Working with size. Source: School work from Carlota Vieira.

Fig. 3. Working with space and size. Source: School work from Vera Sousa.

The Expressivity of the Typographic Composition

795

4 Final Remarks Through this study, we could systematize a reflection on the expressivity of typographic composition. The main moments in the history of communication design in which the word was assumed as expressive matter – graphic poetry, Futurism, Dadaism, and Postmodernism – are important visual references to transmit in the first levels of graphic design teaching. Remembering their protagonists and focusing on the reference works of these movements allows us to establish relationships and acquire an essential global awareness of the evolution of this theme. On the other hand, focusing on several works that address the basic principles of expressiveness in graphic and typographic composition allows us to establish paralellisms and synthesize essential knowledge. Since this research aims to improve teaching, reflection on the exercise that most directly addresses the topic under study allowed us to conclude that, in the future, it will be essential to transmit the historical and theoretical knowledge addressed to students in a simple and profusely illustrated way. In addition, it will also be essential to make some changes in the assignment, to improve the students performance and learning process. Acknowledgments. This work is financed by national funds through FCT – Fundação para a Ciência. e a Tecnologia, I.P., under the Strategic Project with the references UIDB/04008/2020 and UIDP/04008/2020.

References Bringhurst R (2004) The Elements of Typographic Style. Hartley & Marks Publishers Carter R (1999) Tipografia experimental. Destarte Cramsie P (2010) The story of graphic design: from the invention of writing to the birth of digital design. The British Library Drucker J, McVarish E (2009) Graphic design history: a critical guide. Pearson Prentice Hall Eskilson SJ (2007) Graphic design a new history. Laurence King Publishing Ltd. Fawcett-Tang R, Jury D (eds) (2007) New Typographic Design. Lawrence King Friedl F, Ott N, Stein B (1998) Typography – when, who, how. Typographie – wann, wer, wie. typographie – quand, qui, comment. köneman Graphic design | Art, Typography, & Layout Techniques. Encyclopaedia Britannica, homepage, https://www.britannica.com/art/graphic-design, last accessed 2023/05/23 Herb Lubalin, the letter as an image. Graphéine – Agence de Communication Paris Lyon, homepage, https://www.grapheine.com/en/history-of-graphic-design/herb-lubalin-the-letteras-image, last accessed 2022/07/20 Heller S (sd) Robert Massin. Design Observer, homepage, http://designobserver.com/feature/rob ert-massin/40185, last accessed 2022/03/22 Heller S (2004) Design literacy: understanding graphic design (2nd ed). Allworth Press Heller S, Anderson G (2021) Type speaks: a lexicon of expressive, emotional, and symbolic typefaces. Abrams Heller S, Ili´c M (2012) Stop, think, go, do: how typography & graphic design influence behavior. Rockport Publishers Hollis R (2000) Design Gráfico: Uma História Concisa. São Paulo: Martins Fontes Hyndman S (2016) Why fonts matter. Virgin Books Lupton E (2006) Pensar com tipos. Cosac Naify

796

E. Rolo and J. Gomes

Lupton E, Phillips JC (2015) Graphic design: the new basics (second edition, revised and expanded). Princeton Architectural Press, Maryland Institute College of Art McCormick L (2013) Playing with type: 50 graphic experiments for exploring typographic design principles. Rockport Publishers Meggs PB (sd) The digital Revolution. Encyclopaedia Britannica, homepage, https://www.britan nica.com/art/graphic-design/The-digital-revolution, last accessed 2019/07/01 Meggs PB, Purvis A (2009) História do Design Gráfico. Cosac Naify Meggs PB, Purvis AW (2006) Meggs’ history of graphic design. J. Wiley & Sons MoMA | Inventing Abstraction | Guillaume Apollinaire | «Il Pleut» (The rain). SIC no. 12 (December 1916), homepage, https://www.moma.org/interactives/exhibitions/2012/inventing abstraction/?work=17, last accessed 2023/02/28 Poulin R (2012) The language of graphic design: an illustrated handbook for understanding fundamental design principles. Rockport Reis JD (2012) Três movimentos da letra: O desenho da escrita em Portugal. Biblioteca Nacional de Portugal Rolo, E.: The Typographic Grid in the Editorial Project: An Essential Resource to the Graphic Consistency and Perception. In: Bagnara S, Tartaglia R, Albolino S, Alexander T, Fujita Y (eds) IEA 2018, vol 827. AISC. Springer, Cham (2019). https://doi.org/10.1007/978-3-31996059-3 Samara T (2004) Typography Workbook: a real-world guide to using type in graphic design. Rockport Publishers Samara T (2017) Making and breaking the grid: a graphic design layout workshop. Rockport Publishers Samara T (2020) Design elements: understanding the rules and knowing when to break them (Third edition). Rockport Publishers Torres R (2021) Arquivo Digital da PO.EX – Poesia Experimental Portuguesa, homepage https:// po-ex.net/, last accessed 2023/03/28.

A Study of the Relationship Between Image Schema and Visual Identity Design Xin Wang1 and Albert Young Choi2(B) 1 Graduate School, College of Design, Hanyang University ERICA, Ansan, South Korea 2 Faculty of Graduate School, Communication Design, College of Design, Hanyang University

ERICA, Ansan, South Korea [email protected]

Abstract. In today’s digital age, consumers are inundated with an overwhelming amount of brand information. Effective visual identity design that can convey a brand’s value and information to its audience and promote interaction between consumers and the brand is crucial. Image schema in cognitive psychology are a type of analogical or metaphorical thinking pattern that can help people understand complex concepts and information, making them valuable for design research. This article aims to explore the relationship between image schema and visual identity design, with the goal of opening up new directions for research in the field. The paper argues that by understanding the image schema that support people’s perceptions of the world and incorporating them into visual identity design, designers can create visual metaphors that resonate with people’s underlying cognitive frameworks, resulting in more attractive and memorable brand identities. To verify this viewpoint, test experiments were conducted in this study. The test results have shown that, compared with designs that do not conform to the image schema theory, designs that conform to this theory are more effective in conveying information to consumers. This proves that using image schema in visual identity design can convey certain abstract information and have a positive effect on attracting audiences. Keywords: Image Schema · Visual Identity Design · Visual Metaphor

1 Introduction 1.1 Background and Purpose of the Study Visual identity design is an important component of the overall branding strategy of a company, as it helps establish a strong and consistent visual image across all marketing channels and touchpoints. Effective visual identity also helps increase brand awareness and ultimately drive business growth. How to better conduct visual identity design has long been a hot topic of concern for many brands, designers and scholars. Increasingly, scholars have found that the theories or results of psychology can promote the development of design research and guide design practice. Image schema in cognitive psychology is a kind of analogical or metaphorical thinking pattern that can help people © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 797–808, 2024. https://doi.org/10.1007/978-3-031-47281-7_65

798

X. Wang and A. Y. Choi

understand complex concepts and information. Image schema is not only widely used in linguistics but also has many studies in the field of design. However, there are little literatures on the relationship between image schema and visual identity design. This study attempts to find the correlation between image schema and visual identity design and prove the positive role of image schema in visual identity design practice, thereby opening up new directions for the research of visual identity design. 1.2 Scope and Method of Research This study employed qualitative and quantitative research methods. Firstly, through literature review, the concepts of image schema and visual identity design were clarified and points out the theoretical correlation between the two elements - visual metaphor. Then, a test experimental method was used to select brand logos that did not conform to this theory, among a rapidly developing and culturally diverse Chinese market, with the important element of logo in visual identity design as the object. Twenty designers were interviewed to evaluate these logos and based on the evaluation results and theoretical basis, the logos were redesigned. Subsequently, 255 Chinese consumers were surveyed using questionnaires to compare the logos before and after redesign. Finally, based on the analysis of the survey results, the effectiveness of image schema theory in various aspects was compared with designs that did not conform to this theory. To ensure the scientific validity of the data and analysis results, reliability analysis and validity analysis were also conducted using the SPSSAU platform to statistically analyze the reliability of the sample measurement results and the effectiveness of the measurement improvements. This study explains the relationship between image schema and visual identity design, and recommends the use of image schema in visual identity design.

2 Literature Review 2.1 Image Schema The concept of image schema was introduced by linguist and cognitive scientist George Lakoff and philosopher Mark Johnson in their seminal work “Metaphors We Live By,” published in 1980. In this book, they argued that image schemas were basic cognitive structures that underlie metaphorical expressions in language. Image schemas are described as abstract patterns of perceptual and motor experiences that are present in our interactions with the physical world [1]. For example, the image schema of “updown” arises from our experiences of gravity and the way our bodies move in space. Similarly, the image schema of “the container” arises from our experiences with physical containers like boxes, bags, and jars. Lakoff and Johnson went on to develop the theory of conceptual metaphor, which posits that many of our abstract concepts are based on metaphorical mappings from more concrete image schemas [2]. For example, the metaphor “argument is war” maps the image schema of conflict onto the abstract concept of an argument. Image schema is a cognitive structure that represents the repeated patterns of experience we use to organize and understand our sensory experiences. It is a fundamental

A Study of the Relationship Between Image Schema and Visual Identity

799

pattern of perception and movement experience that constitutes the basis of human understanding of the world. These patterns are formed through repetitive and consistent bodily experiences and are considered to be pre-linguistic, cross-cultural, and universal. Their exact nature and organization may vary from individual to individual and culture to culture, as they are influenced by our personal experiences and cultural backgrounds. The formation process of the image schema is shown in Fig. 1.

Fig. 1. The process of constructing an image schema. (Drawn by the author)

The following is a summary and explanation of the characteristics of image schema, please refer to Table 1. Table 1. The characteristics of image schema. Characteristics

Definition

Universality

Image schemas exist universally, appearing in various cultures and contexts, and are widely recognized and understood

Symbolism

Image schemas are symbolic and often represent deeper meanings and values, and are often nonverbal

Multiplicity

Image schemas often have various interpretations and meanings and can convey different meanings to different people

Self-System

Image schemas can form a whole by being interrelated and can form a self-system

Emotionality

Image schemas often have strong emotional colors and can evoke emotional empathy and sympathy in people

Transcendence of Time and Space

Image schemas are not limited by the constraints of time and space and can transcend different historical periods and cultural regions

The concept of image schema has been extensively discussed in cognitive linguistics, which is a subfield of linguistics that focuses on the relationship between language and thought. Some of the influential works that describe image schema include: “Philosophy

800

X. Wang and A. Y. Choi

in the Flesh” by George Lakoff and Mark Johnson (1999) [3]; “The Body in the Mind” by Mark Johnson (2013) [4]; “Conceptual Structures: Information Processing in Mind and Machine” by John Sowa (1984) [5]; “The Structure of Time: Language, Meaning and Temporal Cognition” by Evans, V. F. (2000) [6]. These works describe image schema as a set of primitive mental structures that are grounded in our bodily experiences and serve as the foundation for more abstract thought processes. Image schemas can be thought of as mental templates that help us make sense of the world around us and organize our experiences. For example, the “container” image schema underlies our understanding of spatial relationships and is used to understand concepts such as containment, boundaries, and limits. The “path” image schema is used to understand motion, directionality, and causality. In the field of design, especially in product design, graphic design, and interface design, there have been studies on the application of image schema. Markussen, T., & Krogh, P. G. (2008) proposed the use of “aesthetic interaction design” in their book “Mapping cultural frame shifting in interaction design with blending theory,” which involves designing products and interfaces that interact with the body and promote bodily awareness [7]. The authors believe that this approach can lead to more attractive and effective designs. Meanwhile, Van Der Lugt, R. (2002) explored the use of diagrams in design to capture and convey image schema in his work “Developing brainsketching: a graphic tool for generating ideas.” [8] The author believes that diagrams can be a useful tool for designers to understand complex image schema and incorporate them into their designs. In “Schemata, gambits and precedent: some factors in design expertise” (2004), Lawson, B. studied the role of image schema in design expertise and suggests that expert designers can use their knowledge of image schema to generate new design solutions [9]. The author suggests that the development of design expertise may involve acquiring and refining image schema knowledge. Tversky, B. (2002) analyzed the use of sketches in design in his technical report “What do sketches say about thinking” and suggest that sketches often reflect the use of image schema in problem-solving [10]. The author recommends that designers can benefit from explicitly considering the image schema behind sketches and using them as tools for generating new ideas. Overall, these studies suggest that image schema can be a useful tool for designers, its metaphorical nature enabling them to create more appealing and effective designs. By leveraging the cognitive foundations of mental imagery, designers can develop new design approaches that are based on human perception and interaction with the surrounding world. 2.2 Visual Identity Design The concept of visual identity design appears in many books about branding, design, and marketing. In “Designing Brand Identity,” Alina Wheeler defines visual identity as “the visible elements that represent how a brand communicates and expresses itself to the world [11].” She emphasizes that visual identity is more than just a logo; it includes typography, color, imagery, and other design elements that collectively create a cohesive brand image. In “Logo Design Love: A Guide to Creating Iconic Brand Identities,” David Airey defines visual identity as “the combination of a logo, color, typography, photography, and other design elements that a company uses to communicate visual

A Study of the Relationship Between Image Schema and Visual Identity

801

information about itself to the world [12].” He stresses the importance of creating a strong visual identity that is memorable, timeless, and relevant to the target audience. In “The Brand Gap: How to Bridge the Distance Between Business Strategy and Design,” Marty Neumeier defines visual identity as”the sum total of a brand’s look, feel, and behavior [13].”He emphasizes that visual identity is an important component of overall brand recognition, and its design should reflect the brand’s personality, values, and purpose. In “Brand Thinking and Other Noble Pursuits,” Debbie Millman defines visual identity as “the combination of design elements that represent a brand, including its logo, typography, color palette, and other visual assets [14].” In summary, visual identity is the graphic representation of an organization, brand, or individual used to convey specific information or imagery to its audience. It is composed of various visual elements, such as logos, colors, typography, images, and other design elements, which create a consistent and cohesive look and feel across all communication materials. A strong visual identity is crucial for establishing brand awareness and leaving a memorable impression on the audience. It helps differentiate the brand from competitors and through visual metaphors and other ways to communicate the brand’s values and personality to the audience. Visual identity design is the process of creating the visual elements that make up the foundation of brand identity. It involves designing a unique and memorable logo, selecting colors and typography that reflect the brand’s personality, and creating a consistent visual language that can be applied to all brand communications. In addition, visual identity design should follow certain principles, which can guide the creation of successful and effective visual identity. These principles have been summarized by referencing the views of many renowned scholars, such as Don Norman and Jakob Nielsen (2013) [15], as well as Paul Rand (1996) [16]. These principles are referenced in Table 2. Table 2. Principles of Visual Identity Design. Principles

Definition

Simplicity

Visual identity should be simple and easy to understand, without unnecessary complexity or confusion

Memorability

Visual identity should be memorable and distinctive, helping the brand or organization stand out in the minds of its audience

Communicability

Visual identity should have the ability to effectively communicate the intended message or meaning to the target audience

Uniqueness

Visual identity should be distinct and unique, standing out from competitors and creating a strong association with the brand

Flexibility

Visual identity should be flexible enough to adapt to different media and environments, while still maintaining its core elements

Consistency

Visual identity should be consistent with the brand’s personality, values, and mission, conveying a clear and consistent message

802

X. Wang and A. Y. Choi

3 Image Schema and Visual Identity Design After combing through concepts and theories, as well as conducting investigations into current research, it is not difficult to find that image schema and visual identity design are related, as image schema provide the conceptual foundation for effective visual identity design. The important link between the two is the visual metaphor. Image schema is a foundational cognitive structure for people to perceive and understand the world, including people’s understanding of abstract concepts. Image schema is the basis of visual metaphor, which enables visual elements (such as color, shape, line and space) to communicate abstract concepts or emotions through visual metaphor. On the other hand, visual identity refers to the visual representation of a brand or entity. Designers can use image schema to create visual identities that more effectively convey intended information and elicit desired emotional responses from consumers. By understanding the potential image schema associated with specific concepts or brands, designers can use visual elements such as color, shape, and composition to create consistent and recognizable patterns that align with the brand’s values and personality. For example, the image schema of “up” is typically associated with positive emotions such as happiness and success, which is why many brands use upward arrows or other upward-leaning visual elements in their logos and brand materials. Similarly, the image schema of “the container” is associated with concepts such as containment, protection, and organization, which is why many brands use boxed or enclosed visual elements in their logos and brand materials. Overall, image schema and visual identity design are closely related because image schema provides a conceptual framework for creating an effective visual identity, conveys the desired message through visual metaphors, and elicits the desired emotional response from consumers.

4 Experiment Although there is theoretical evidence for the correlation between the two, practical design experience has shown insufficient evidence for their effectiveness. In light of this, our study conducted experiments, which were limited to the Chinese market. The reason for choosing the Chinese market is due to its rapidly growing economy, large population, and unique culture, which can provide valuable insights for understanding the perception of visual identity design in different environments and for its application. The variable of setting logo design is due to the inherent importance of logo design as a core element in brand visual identity design. The experiment is divided into five stages. Stage 1: Selection of Test Subjects. Choose a brand logo design that is representative and not in line with the theory among the logo designs currently circulating in the Chinese market. The “Haoxiangni” brand logo is affiliated with Haoxiangni Health Food Co., Ltd. The company was founded in 1992 and is primarily engaged in the research, procurement, production, and sales of healthy foods such as red dates, freeze-dried products, nuts, and dried fruits [17]. In recent years, the Haoxiangni brand has developed into a leader in China’s red date industry, with its products primarily based on red dates and catering to

A Study of the Relationship Between Image Schema and Visual Identity

803

people of all ages, enjoying widespread popularity. The brand’s mission is to “provide healthy food for those who understand health and seek it,” following a development strategy as the “leader of the new generation of healthy food,” and focusing on the highend positioning of “Haoxiangni - Premium Red Dates.”As such an outstanding health snack brand, extra care should be taken in the design of its logo. Twenty visual design professionals with an average work experience of more than three years were gathered to evaluate and analyze the logo based on the characteristics of the image-schema theory. This was done to ensure and confirm that the logo does not conform to the theory. The analysis results are summarized in Table 3. Table 3. Haoxiangni brand’s current logo and analysis results. Brand Logo

Evaluation Criteria

Analysis Results

Universality

It cannot be universally recognized and clearly understood

Symbolism

It cannot represent deeper meaning and value, or even identify the industry attributes of the brand

Multiplicity

It cannot convey certain meanings without looking at the text message, and has no explanatory effect on the brand

Self-System

The overall design is not coordinated and not related to other design expressions of the brand

Emotionality

Does not have emotional color, and it is difficult to evoke emotional resonance

Transcendence of Time and Space It is not recommended for long-term use, and it cannot help the brand to expand across borders

Stage 2: Logo Redesign. Based on the analysis results of numerous designers, the author extracted the brand’s image schema and redesigned the brand logo accordingly. The author also invited the 20 designers mentioned above to evaluate the redesign from a designer’s perspective, ensuring the authority and reliability of the design outcome. (See Fig. 2). The new logo design adopts the form of dates instead of the graphics of the old logo, which metaphorically represents the attributes of the industry and the characteristics of the product. A calligraphic font is used to emphasize the brand name, and combined with the form of green leaves, it forms the main body of the logo together with the graphic of the date. The product concept is embellished in English below the Chinese name.

804

X. Wang and A. Y. Choi

Fig. 2. Redesigned brand logo. (Designed and created by the author)

Stage 3: Project Testing. The project is based on the redesign of the “Haoxiangni” brand logo to verify the hypothesis that using image schemata in design results in higher scores in terms of information conveyance and attractiveness compared to designs that do not use this theory. The independent variables are the previous logo design and the updated logo design of the “Haoxiangni” brand. The dependent variables include simplicity, memorability, communicability, uniqueness, flexibility and consistency. The survey includes understanding the response of consumers who are familiar or unfamiliar with the brand towards the logo. The evaluation of the project uses a five-point scale based on the application of visual recognition design principles summarized in the theoretical investigation, as shown in Table 4. Table 4. Evaluation criteria and questionnaire content. Evaluation criteria

Survey questions

Simplicity

Is the logo visually simple and appealing to you?

Memorability

How memorable is this logo?

Communicability

How well does this logo communicate the brand’s message?

Uniqueness

How unique is the logo compared to other logos in the same industry?

Flexibility

How well does the logo be easily recognizable in any medium?

Consistency

How well does the logo align with the values and mission of the brand?

Stage 4: Analysis of Results. Based on the original and redesigned solutions, a Likert scale was used to evaluate each question on a scale of 1–5. To grasp the overall situation, the differences between the results of the two surveys were displayed as a radar chart. According to Table 5, the scores of the logo designed with image schematic design have been improved in all six evaluation criteria and the overall average score has also been increased. Except for consistency, which has a difference of 0.63 points compared to the score before the redesign, the scores of other aspects are relatively balanced, basically maintained between 0.38 and 0.5. This may be because the image schema design conveys certain abstract information in visual recognition design, which resonates with consumers’ cognitive understanding.

A Study of the Relationship Between Image Schema and Visual Identity

805

Table 5. Survey Results Score. Evaluation criteria

Current logo

Redesigned logo

Score Difference

Simplicity

3.36

3.86

0.5

Memorability

3.4

3.78

0.38

Communicability

3.38

3.84

0.46

Uniqueness

3.29

3.8

0.51

Flexibility

3.37

3.79

0.42

Consistency

3.24

3.87

0.63

Average score

3.34

3.83

0.49

As shown in Fig. 3, the redesigned logo excels in various aspects compared to the previous one, especially in brand consistency, design simplicity, and information communication. This proves that the use of image schema based design is more effective in conveying brand information and concepts.

Fig. 3. Likert scale comparison before and after redesign. (Created by the author based on the data).

Stage 5: Analysis of Validation Results. This study used the SPSSAU platform to perform statistical analysis on the collected data to explore the differences between two logos. In order to understand the demographic characteristics of the survey participants, frequency analysis was conducted, and the Cronbach’s Alpha coefficient was used to determine the reliability of the scale used. The demographic characteristics of the experimental participants used for analysis are shown in Table 5. Of the total 255 people, 192 were male, accounting for 75.29%, and 63 were female, accounting for 24.71%. The age range of the respondents was mainly between 18–35 and 36–55, which also reflects their consumption capacity. In terms of occupation, the ratio of students and professionals

806

X. Wang and A. Y. Choi

was balanced, with the highest percentage being “other” at 48.63%, mainly involving self-employed individuals and freelancers. In the process of advancing understanding, it has been found that this group of respondents has a wide and comprehensive range of expertise, which is why they tend to pay more attention to wellness and health. (See Table 6.).

Table 6. Demographic characteristics. Variables

Classification

Frequency

Ratio

Brand familiarity

Yes

154

60.39%

No

101

39.61%

Gender

Male

192

75.29%

Age

Career

Female

63

24.71%

Non-binary

0

0%

0 ~ 17

1

0.3%

18 ~ 35

124

48.63%

36 ~ 55

119

46.67%

56 and above

11

4.31%

Student

56

21.96%

Employed

75

29.41%

Other

124

48.63%

To confirm the feasibility of the six-dimensional questions used in the study, an Exploratory Factor Analysis (EFA) was conducted to extract exploratory factors. To examine whether the factors have internal consistency and reliability, a reliability analysis was conducted on each question used in the analysis. The results showed that the KMO value was 0.940, indicating strong correlations between variables, and the data was suitable for factor analysis. The result of Bartlett’s sphericity test was 4953.690, with p = 0.000 and df = 120.000, which indicated that the correlations between standard scale variables were recognized, and an overall factor analysis of the logo design scheme could be conducted. The reliability test showed that the measured sample responses for simplicity, memorability, communicability, uniqueness, flexibility, and consistency had a Cronbach’s Alpha of 0.965, indicating high reliability. Summarized in Tables 7 and 8. Table 7. The results of the reliability analysis. Sample size

Number of items

Cronbach’s Alpha

255

12

0.965

A Study of the Relationship Between Image Schema and Visual Identity

807

Table 8. The results of validity analysis. Variables

Measurement items

Factor 1

Factor 2

Commonality

Current logo

6

Simplicity

0.29

0.88

0.854

Memorability

0.31

0.89

0.887

Communicability

0.33

0.88

0.876

Uniqueness

0.30

0.91

0.910

Flexibility

0.32

0.88

0.878

Consistency

0.30

0.91

0.914

Redesigned logo

6

Simplicity

0.89

0.33

0.896

Memorability

0.87

0.37

0.900

Communicability

0.91

0.28

0.911

Uniqueness

0.89

0.33

0.906

Flexibility

0.90

0.31

0.903

Consistency

0.92

0.26

0.916

Eigenroot value (before rotation)

8.70

2.05

-

Variance explained % (before rotation)

72.48%

17.12%

-

Cumulative variance explained % (before rotation)

72.48%

89.60%

-

Eigenroot value (after rotation)

5.40

5.35

-

Variance explained % (after rotation)

45.03%

44.57%

-

Cumulative variance explained % (after rotation) 45.03%

89.60%

KMO

0.952

-

Bartlett

4611.362

-

df

66.000

-

p

-

-

5 Conclusion On a broader level, the motivation of this article is to observe the relationship between image schema theory and visual identity design. This study uses a literature review to summarize how image schema provides a conceptual foundation for effective visual identity design. For empirical investigation, the study employs logo redesign tests to capture the feasibility of applying image schema to visual identity design. Through a survey, the study explores how using image schema can convey abstract information in visual identity design, and ensures the validity and reliability of the survey through reliability and validity analyses. The study found that image schema provides a conceptual prerequisite for visual identity design. By understanding the image schema that support people’s worldview and using these image schema in visual identity design, brand information can be conveyed

808

X. Wang and A. Y. Choi

more effectively. Furthermore, resonating with people’s potential cognitive frameworks through visual metaphors in the design helps create more attractive and memorable brand identities. Overall, the study’s findings highlight the relationship between image schema and visual identity design, emphasizing the importance of image schema in both research and practice. Based on the availability of data, further research on how to extract image schema and better measure their practicality as a visual identity design language would greatly benefit the future development of brand identity.

References 1. Lakoff G, Johnson M (1980) Metaphors We Live By. University of Chicago, Chicago 2. Lakoff G, Johnson M (2020) Conceptual metaphor in everyday language. In: Shaping Entrepreneurship Research, pp 475–504. Routledge 3. Lakoff G, Johnson M (1999) Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought. Basic Books 4. Johnson M (2013) The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. University of Chicago press 5. Sowa JF (1984) Conceptual Structures: Information Processing in Mind and Machine. Addison-Wesley Longman Publishing Co., Inc. 6. Evans VF (2000) The Structure of Time: Language, Meaning and Temporal Cognition. Georgetown University 7. Markussen T, Krogh PG (2008) Mapping cultural frame shifting in interaction design with blending theory. Int J Des 2(2):5–17 8. Van Der Lugt R (2002) Developing brainsketching: a graphic tool for generating ideas. In: 7th European Conference on Creativity and Innovation, Enschede, pp 239–253. Twente University Press 9. Lawson B (2004) Schemata, gambits and precedent: some factors in design expertise. Des Stud 25(5):443–457 10. Tversky B (2002) What do sketches say about thinking. In: 2002 AAAI Spring Symposium, Sketch Understanding Workshop, Stanford University, AAAI Technical Report SS-02-08, vol 148, p 151 11. Wheeler A (2017) Designing Brand Identity: An Essential Guide for the Whole Branding Team. John Wiley & Sons 12. Airey D (2009) Logo Design Love: A Guide to Creating Iconic Brand Identities. New Riders 13. Neumeier M (2003) The Brand Gap: How to Bridge the Distance Between Business Strategy and Design 14. Millman D (2011) Brand Thinking and Other Noble Pursuits. Skyhorse Publishing Inc. 15. Nielsen J, Norman D (2014) The Definition of User Experience. Nielsen Norman Group, 191 16. Rand P (2017) Design, Form, and Chaos. Yale University Press 17. No Author. (n.d.). About Haoxiangni. http://www.haoxiangni.cn/about.html

Implications of New Media in Brands’ Visual Identities: A Systematic Literature Review Sofia Almeida(B)

and Catarina Lelis

DIGIMEDIA: Digital Media and Interaction; Department of Communication and Art, University of Aveiro, Aveiro, Portugal [email protected]

Abstract. The constant metamorphoses and challenges created by new media and its language led to visual identities playing a crucial role in creating emotions and establishing strong bonds with the audiences. Therefore, a brand’s visual identity is increasingly imperative for its communication and strategic success. To better understand the current implications of new media in visual identities, a systematic literature review was carried out via SCOPUS database and grounded on PRISMA as method. As a result, this analysis highlights scientific contributions with origin in Anglo-Saxon countries and China. Four main themes were identified: user experience, user behaviour, visual/sensorial communication, and globalisation. It was also found that the term “multi” was often present (e.g., multi-experiences and multi-channels). In conclusion, it became clear that research on this topic is very limited and that there is room for much to be done. Understanding the value of the new media in a brand’s visual identity (namely artificial intelligence) may open new avenues of research, creation, and development of digital and dynamic identities, and of emotional relationships—which may arguably become crucial for the sustainability and future of brands in the hybrid spheres of reality that lie ahead. Keywords: Visual Identity · Brand Design · New Media · Dynamic Identities · Smartness · Artificial Intelligence

1 Introduction The countless studies which have been accompanying the evolution of branding in recent years prove that a brand is a constantly evolving living organism that follows the social, economic, and technological transformation. The recent update of the definition published by the American Association [1], assumes brand as a valuable intangible asset. Once it allows the creation of distinctive images and associations in the minds of audiences, creating economic value while integrating factors such as innovation and quality of services that contribute to the success, notoriety, and reputation of organisations. However, whilst owning a visual identity, a brand is not just a symbolic representation of an organisation. It does not only fulfil the purpose of identifying it and differentiating © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 809–825, 2024. https://doi.org/10.1007/978-3-031-47281-7_66

810

S. Almeida and C. Lelis

it from its counterparts. On the contrary, it is a strategic resource used to communicate an essence, vision, mission, and value promise. The visual identity (VI) is the graphic representation of such essence, a strategy drawn by the organisation to achieve their aspirations and create relationships and connections to win the public over [2, 3]. To promote the brand experience and, consequently, to create brand’s soul. Thus, a visual identity is representation of a certain social, historical, economic, and technological context, which currently accompanies the constant mutations of the of digital information and a society in rapid globalisation. The beginning of the century was marked by a paradigm shift in communication, the so-called new media era, with the advent of the Internet. The new media include social networks and mobile applications, tangible and intangible interfaces, among others. We now live in the era of digital media and internet technology, the effective engine of communication [4]. This paradigm shift has created countless possibilities and new means of communicating, consequently reshaping the user-brand relationship. The user is no longer a mere passive observer and interpreter, having become an active participant who interacts directly with the brand [5]. In this period marked by the transition of communication from offline to online [2], Kreutz (2012) identifies two distinct brand communication strategies: the conventional (rigid and standardised) and the unconventional (mutable and dynamic). More recently, studies by Lelis and Kreutz (2021), concluded that current brands are flexible, promoting greater interaction with the user, to keep up with the dynamism of today’s society [6]. The constant metamorphosis and challenges created by the new media prompts visual identity to take a crucial role in the communication and strategic success of a brand. Since language is abundantly visual, which justifies its huge magnetising power in global society. This research intends to understand how brands’ visual identities have adapted to the new media context (see Table 1). Thus, it is important to recognise which visual identity strategies are used by contemporary brands, and how they are taking advantage of new media to increase their value, reputation, and positioning in the global market. Table 1. Formulation of the research question using PICO tool. Source: Methley et al. (2014) [7]. Research Question

How did brands’ visual identity adapt to the new media?

Problem

The brand’s visual identity is not designed for the current digital context

Interest

Behaviour of a brands’ visual identity on digital platforms

COntext

New Media

2 Systematic Literature Review Process To begin this exploratory research in a structured, objective, and delimited way, a systematic literature review (SLR) was carried out in two different stages.

Implications of New Media in Brands’ Visual Identities: A Systematic Literature

811

Grounded on Problem, Interest, and COntext (PICO) model, (see Table 1) to help the formulation of a research question, and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) as methodological protocol. The purpose of this SLR was to assess what has been published and indexed about brands’ visual identity and their behaviour in the context of new media. 2.1 First Stage To carry out this research, SCOPUS database was used. Relevance criteria were defined to extract publications focused on image and brand’s visual identity in the context of new media, defined by keywords in English, using Boolean operators, as presented in Table 2. Table 2. Search parameters used for retrieving and selecting articles—1st stage of the SLR. Search Query

Scopus

“Visual identity” OR brand OR AND “New media” OR “Digital media” OR 696 “brand image” OR “dynamic brand” Transmedia OR “flexible brand” OR “mutant brand”

Fig. 1. Flow diagram of the process employed on the first stage of the systematic literature review, following PRISMA. Source Page et al. (2021) [8].

Figure 1 presents the article selection process and identifies the inclusion and exclusion factors, following the PRISMA Protocol [8].

812

S. Almeida and C. Lelis

A total of 696 records were identified of which 99 were screened after examining the variables, the type of sample and the language. This led to excluding records due to non-relevant keywords, non-final articles, published in languages other than English and Portuguese (the ones the authors thoroughly master), and in other venues besides academic journals. Finally, 42 publications were considered eligible for more careful reading. Of these, only eight articles were selected to be included in the meta-analysis, after performing a scrutiny of their abstracts and introductions. Among the exclusion criteria used in this selection, were articles related to fashion, journalism, and advertising. 2.2 Second Stage Nonetheless, after completion of the previous SLR exercise, the results were ambiguous, because of the broadness of the keyword “new media”. Hence, a further search was performed, focused on concepts that were found as most frequent during the SLR’s first stage such as “smartness” and artificial intelligence (AI) (Table 3). Resorting to SCOPUS, the process applied was the same (Fig. 2). The criteria being focused on publications that related brands’ visual identity in the contexts of or operating via smartness features and AI mechanisms. Table 3. Search parameters used for retrieving and selecting articles—2nd stage of the SLR. Search Query “Visual identity” OR brand* OR” Dynamic brand” OR “mutant brand”

Scopus AND

smartness* OR “artificial intelligence”

988

Again, according to PRISMA [8], out of 988 records, a first screening led to 291 records after excluding all retrieved results in press or ongoing research (which would fail in presenting applied methods and results), short or non-articles (e.g., reports, conference proceedings, etc.), published in languages other than English and Portuguese, and published before 2015. Further filtering excluded all results considered from different subject areas, with keywords outside the field and unavailable texts (e.g., due to lacking institutional access). Hence, the second search allowed the identification of additional readings by evaluating the title, abstract and keywords, using the following screening criteria: i) articles that explicitly state their research questions or objectives ii) with a research context within AI and smartness iii) focus on the topic of visual brand identity. Fifty texts were considered eligible but only eight were deemed to be focused on the theme and were eventually analysed.

Implications of New Media in Brands’ Visual Identities: A Systematic Literature

813

Fig. 2. Flow diagram of the process employed on the second stage of the systematic literature review, following PRISMA [8].

3 Findings and Meta-Analysis To achieve a better analysis of the collected articles, and after a repeated review of each one of them, the data was extensively summarised (see Appendix 1 and Appendix 2). As a result of the filtering implemented, this analysis highlights scientific contributions from the last thirteen years, with origin in Anglo-Saxon countries and China, providing clues as to what has been studied and the path and evolution of the theme topic of visual brand identity in the context of the new media. It was possible to ascertain that research on the topics under study is quite limited. 3.1 First Stage of the SLR From the first stage of the SLR, the main goals of the articles analysed the brand design in the digital context, namely the logo design process, focused on user’s behaviour in digital context. The methodology used is mostly quantitative, with only two of the eight articles analysed using qualitative methodology. The main results identified four main themes: 1) User experience; 2) User behaviour; 3) Visual/Sensorial Communication; 4) Globalisation. By analysing these themes, it was possible to conclude that the term that best defines a visual identity, in this context, is MULTI. Due to the various advantages and challenges that the digital society and new

814

S. Almeida and C. Lelis

media constantly pose to brands, resulting from its interaction with the user: multiplatform, multi-sensations, multi-experiences, multi-tasks, multi-channels, multimedia, among others. 3.2 Second Stage of the SLR According to the second SLR exercise the main goals of the selected articles focused on brand experience, namely in the impact of the Artificial Intelligence in brands. Qualitative and logic analysis were most frequently used, with only three articles using quantitative methodology. The results noticeable that the concept of smartness is directly linked to places and territories: smart cities, city brand, smart place, smart campus, place branding, smart region, among other combinations (n = 20). On the other hand, the concept of AI is mostly related to intelligent voice assistants: smart assistant, smart voice (n = 15).

4 Conclusion It can be established that the evolution of brands visual identity in the context of new media is related to and dependent on both technological progress and networked means of communication. On the one hand, these are the driving force not only for socioeconomic changes, but also for changes in the communication paradigms; on the other hand, technology is centred on the human, a curious and dissatisfied being by nature, and therefore leading to constant behavioural changes. From this point of view, it is plausible to reinforce the conviction that, through visual identity, a brand is a living organism. Its sustainability must accompany the surrounding mutations, in a flexible and dynamic way, adapting itself and surviving in new digital contexts. It is therefore a concept that must be constantly adapted, renewed, and redesigned. Considering the research question of this article: “How did the brands’ visual identity adapt to the new media?”, the results indicate that the trend has been the adjust of brands to the new media context. Rather than involving a sound design process, involving a sound design process, involving strategy, creativity and the advancements of communication and information technologies, to consistently create visual identities that the evidence the digital imperative and a multi-led positioning. It’s important to highlight that a quantitative methodological tendency was found. Which reveals that the existent research on the topic of visual brand identity its restricted to statistics and traditional models, leaving out the emotional aspects. Nevertheless, the emotional field have become crucial with the advent of the artificial intelligence. Hence, there’s a gap in the methodology applied, witch at the same time represents a research window of opportunity. As the design is a privileged bond for establishing relations with the user, can undoubtedly become fundamental to create positive emotions contributing to the wellbeing and improved quality of life, It is urgent to understand that communication must make people feel in a unique and global way.

Implications of New Media in Brands’ Visual Identities: A Systematic Literature

815

As living organisms, the user and the brand mutually influence one another in a continuous system of renewal and alteration of behaviours, values, experiences. The user is dynamic, meaning that the brand and its visual identity must also be dynamic. In the same way that the user has decision-making power, a brand can also establish relationships through more dynamic communication, doing so via its VI. In this context, current brand VIs can be lively, flexible, creative, innovative—if they want, if they need. Through design, they can incorporate the emotional experience, in an image-based resource that should be simple and direct, easy to decode. Design will help brands contemplate the digital, the different platforms and interactions, integrating graphics, text, sound, and animation combined, using multimedia technology and data-related features to create multisensory experiences. Finally, all this will only be successful if there is coherent, consistent, and integrated behaviour of the brand in all points of contact with the user, whether offline or online. Understanding the value of the new media in a brand’s visual identity may open new avenues of research, creation, and development of emotional relationships—which may arguably become crucial for the sustainability and future of brands in the hybrid spheres of reality that lie ahead, and may, without a doubt, become fundamental for sustainability and for improving the quality of life of future generations.

Appendix 1. 1st stage of the SLR - data collected and summarised, following the PRISMA method [8] Author/Affiliation Goals

Methods

Main results

[9] United Kingdom

- brand design: strong, clear, and short names - logo design: square because of the shape of mobile apps - algorithms for colour choice - questionnaire to 100

- the brand design should convey the emotional tone, touch users’ spiritual needs, awaken their emotional magnetism, to deepen the interpretation of the brand - the findings reveal that the suggested brand design approach can elevate brand acclaim by 7.3% and increase users’ emotional experiences

- context of the new 5G media era - brand design and its impact on users’ emotional experience - aim of changing traditional brand design and providing new insights into the logo creation process

(continued)

816

S. Almeida and C. Lelis

(continued) Author/Affiliation Goals

Methods

Main results

[10] China

- resolve the issue that, in the digital age, traditional static visual images no longer satisfy users’ needs - aims to investigate the structure and execution of logo design in the context of new media. Multiple types of multi-dimensional spatial effects have taken the role of static, two-dimensional restrictions

- proposes a dynamic logo design system based on encoded neural networks - creation of logo design algorithms, using software (and hardware platform - carrying out questionnaires to designers between 24 and 30 years old

- future design development will unavoidably follow a dynamic trend, or direction - they are more humanized and richer in forms of expression, and their meaning is to create a series of new visual languages, adapt to the needs of the future development, valuing the emotional and interactive characteristics - dynamic logo design incorporates various media, such as sound, text, graphics, and animation, increasing the connotation and meaning of the logo

[11] China

- focuses on online brand image design strategy in an international context - global updating traditional brands - creating successful online brands in the new media age

- analysis and algorithmic application of the brand image model (ASP Net method)

- online brand design strategy must reflect the international era - user experience, reputation, and public opinion have become determining factors in brand communication strategies, with direct impact on brand image - the brand identity strategies must be reviewed systematically to keep up with rapid technological developments (continued)

Implications of New Media in Brands’ Visual Identities: A Systematic Literature (continued) Author/Affiliation Goals

Methods

Main results

817

[4] China

- analyses visual communication from a new angle and direction - the development and innovation fostered by media have given visual communication a new language and a new form of expression - analyses visual communication and design through the lens of digital media in contemporary scenography

- uses scientific and systematic methods - applies the method of multisensory expression in modern visual design, then analyses the application of digital media technology in visual communication design

- new media have had a significant impact on traditional print media, breaking the monotony of the traditional and making modern visual design more vivid and intuitive - the multisensory expression made possible by modern visual design through new technologies gives it a prominent role in the communication of information - dynamic visual design distributes and integrates information on different screen formats, contributing to the diffusion of information through easy and direct communication

[12] South Africa

- aims to respond to the absence of a structure, or model, that integrates the various communication channels in the public domain - despite technological advances and changes in communication, the traditional communication channels have enormous emphasis

- qualitative study - 8 in-depth interviews with channel and media managers, from leading global agencies identified through the Forrester Institute Report

- the success of the brand communication depends on the development of an integrated communication plan, which a continuous management for a brand communication is effective - the communication strategy plan of brand creation must: recognize the importance of the target audience’s profile when relating to the brand; integrate both new and traditional media; considered the advantages and interaction between the user and the brand in the different types of contact and platforms (continued)

818

S. Almeida and C. Lelis

(continued) Author/Affiliation Goals

Methods

Main results

[13] - creation of theoretical Australia/ Ireland model that aims to explain how social media content can influence user behaviour and contribute to the creation of strong connections in the online and dynamic social media context. It is a tool that can be used to stimulate positive behaviour and relationships between user and social networks, avoiding negative relationships

- based on the Uses and Gratifications Theory (UGT), the model of Creating positive stimuli to create engagement in users’ relationships with social networks (SMEB) was created, which is nothing more than a typology and behaviours of relationships between social networks and users

- the brand experience consists of multiple interactions with the user and depends on the context. Social networks have changed consumer behaviour - user is now an active player, not limited to the consumption of information and communication, but gets involved and has an active voice in the selection of information . The behaviour typology of the model identifies 7 types: co- creation, positive contribution, consumption, dormancy, detachment, negative contribution and codestruction

[14] United States

- exploratory research - media multitasking from media monotasking - impacts of branding targeted at the unconscious: Strong brands act as cognitive triggers and have impact on consumer behaviour

- a strong brand with a consistent visual identity, unified, hyper- focused, with a single message, easy to decode and memorize is pointed as a solution to the lack of attention of digital users - the visual identity must be transversal and coherent in all contact points, to create synergic relationships between the digital and real environments

- analyses the shift in consumer behaviour and the lack of focus they have due to concurrent digital multi- use, and it raises awareness of the need for a strong, straightforward visual identity with a single, clear, and direct message that the customer can memorize

(continued)

Implications of New Media in Brands’ Visual Identities: A Systematic Literature (continued) Author/Affiliation Goals [15] United Kingdom

- user acceptance and behaviour in the new multichannel scenario - how technology is changing user behaviour and the relationship with brands - Axciom marketing and technology company commissioned a study, focusing on the relationships between users and brands

819

Methods

Main results

-research commissioned by Acxiom, conducted in July 2011. - two observation panels were created: 1) 1014 UK users responded to a 10-question questionnaire; 2) survey of 222 marketing professionals

- technology is changing the way users interact with brands, and users accept and like this new paradigm - the users control the brand-individual relationship, they choose the information - despite the growing use of new digital the traditional brand is still highly valued and should not be forgotten - brands need to understand when and where to communicate with users to create strong connections and consistent, positive relationships

820

S. Almeida and C. Lelis

Appendix 2.—2nd Stage of the SLR - Data Collected and Summarised, Following the PRISMA Method [8].

[16] South Korea

Author/Affiliation

Goals Methods results Methods

- “using new e-commerce users as the participants, this study investigated the structural relationship between brand experience, brand e motional factors (brand attachment and brand love), brand loyalty, and repurchase intention with brand love as the mediator”

- structured questionnaire - based on studies and a survey of 310 respondents from China

- “the four dimensions of brand experience have a significant positive correlation with brand emotion, With brand cognitive experience having the greatest impact on consumer brand emotion. Second, the influence Of brand emotion on brand loyalty is positive and significant, and brand attachment has a stronger influence than brand love on brand loyalty. In addition, brand loyalty has a positive effect on repurchase intention Finally, brand love plays a mediating role on the relationship between brand attachment and brand loyalty. To enhance customers’ brand attachment and love for e- commerce platforms, companies must enhance customers’ interest and curiosity in their products (continued)

Implications of New Media in Brands’ Visual Identities: A Systematic Literature

821

(continued) Author/Affiliation

Goals Methods results Methods

[17] United Kingdom

- importance of research in “smart visual identities”, which function as a valuable and comprehensive branding resource

- hermeneutics - qualitative analyses - explore stage

- propose a definition of “smart brand”, suggesting that individuals can increase their design literacy while utilizing a smart learning environment by using highly relatable brand identities

[18] Australia Spain Saudi Arabi Japan Malaysia

- the idea of an artificially intelligent city is as a potential successor to the well-known smart city brand, where a city’s smartness is now closely linked to the adoption of workable technical solutions, such as AI

- review of the status of AI literature, research, developments, trends, and applications

- urban politicians, managers, and planners can use the obtained insights to guarantee that AI is implemented properly in our cities, and the identified crucial questions give academics ideas for future research and development

[19] USA Brazil

- this study aims to investigate how effective tourism branding methods have been applied to institutional websites to inform industry stakeholders about pertinent values, advantages, and features

- semiotic analysis of two smart tourism-related sites - the study analyses verbal and visual cues that are brand related

- the research shows how brand features integrated into websites convey a brand identity and support certain approaches to smart tourism. Both businesses present a ecotopian vision of smart places with comparable signs, but they use distinct approaches to encourage stakeholder buy- in (continued)

822

S. Almeida and C. Lelis

(continued) Author/Affiliation

Goals Methods results Methods

[20] United Kingdom Russia Italy

- this essay’s goal is to investigate how creativity and innovation function as critical indicators of smartness in the marketing of cities and travel destinations

- qualitative and logical deductive approach - comparing ‘smart cities’ branding both inside and outside of Europe: Milan (Italy) and Tomsk (Russia) are shown and contrasted

- smart destination branding strategies should be peoplecentred, based on people participation, creativity, and innovation as drivers of smart urban development - innovative branding tactics are supported by the intertwined relationship between creative efforts, smart cities/destinations, and innovation

[21] India

- “aims to identify which AI technologies are impacting brand identity; explores AI applications in brand promotions; and Investigates the influence of AI on brand equity”

- meta synthesis for testing prepositions

- the dimensions of brand identity and brand equity have been redefined because of the meta-synthesis of the research - however, there is still a lot of buzz about artificial intelligence (AI) and how it will affect brand management - future study might therefore examine how AI affects various models of brand equity and identity (continued)

Implications of New Media in Brands’ Visual Identities: A Systematic Literature

823

(continued) Author/Affiliation

Goals Methods results Methods

[22] India Spain

- “builds the appraisal theory of emotions on robots, and employs three online experiments and one field study to demonstrate that customers have different emotional responses to the three types of AI”

- between-subject online experiment - 3 online experiments focusing on mechanical, thinking, and feeling AI

- “feeling” AI relates more strongly to positive emotions than mechanical AI” - “feeling AI and thinking AI increase spending and loyalty intention through customers’ positive emotions” - “none of the three intelligences are directly related to negative emotions; perceived robot autonomy is an important mediator in these relationships”

[23] Malaysia Indonesia

- examine questions: “1) how smartness in tourism smart destination influence city branding, 2) what capability needed by the stakeholders to improve city brand equity, 3) how entrepreneurship opportunity will be created”

- an integration model of interconnection capabilities based on brand equity has been put out to address these queries

- the study’s conclusions will aid the tourism industry’s ecosystem in identifying opportunities to capitalize on city branding, forge competitive advantages, and foster entrepreneurship

References 1. American Marketing Association (2021). What is Marketing? The Definition of Marketing. https://www.ama.org/the-definition-of-marketing-what-is-marketing/ 2. de Kreutz EA (2018) Contruindo marcas mutantes Revista Latinoamericana de Comunicación, vol 119. https://revistachasqui.org/index.php/chasqui/article/view/120/0. Accessed 1 Nov 2022. 3. Williams RL, Omar M (2014) How branding process activities impact brand equity within Higher Education Institutions. J Market High Educ 24(1):1–10. https://doi.org/10.1080/088 41241.2014.920567 4. Guan X, Wang K (2022) Visual communication design using machine vision and digital media communication technology. Wireless Commun. Mobile Comput. 2022:1–11. https://doi.org/ 10.1155/2022/6235913

824

S. Almeida and C. Lelis

5. Lelis C, Kreutz EA (2022) The HOW behind the story: a framework for the design of brand narratives. In: Design, Visual Communication and Branding. Cambridge Scholars Publishing, Newcastle 6. Lelis C, de Kreutz EA (2021) The Realms of Participation in Visual Identity Design. Corporate Reputation Review. https://doi.org/10.1057/s41299-021-00134-4 7. Methley AM, Campbell S, Chew-Graham C, McNally R, CheraghiSohi S (2014) PICO, PICOS and SPIDER: A comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res 14(1):1–10. https://doi.org/10.1186/ S12913-014-0579-0/TABLES/7 8. Page MJ et al (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372. https://doi.org/10.1136/BMJ.N71 9. Li X (2022) Brand design in the era of 5g new media and its impact on consumers’ emotional experience. Front Psychol. 13:4388. https://doi.org/10.3389/fpsyg.2022.956490 10. Yang Z (2022) Dynamic logo design system of network media art based on convolutional neural network. Mobile Inform Syst 2022. https://doi.org/10.1155/2022/3247229 11. Fu L (2022) A brand image design service model using the visual communication technology under the background of internationalization. Wirel Commun Mob Comput 2022. https://doi. org/10.1155/2022/5922967 12. Belling C, Enslin C, Bozas A (2021) A proposed media channel framework for integrated brand communication planning. Communitas 26:20–32. https://doi.org/10.18820/24150525/ COMM.V26.2 13. Dolan R, Conduit J, Fahy J, Goodman S (2015) Social media engagement behaviour: a uses and gratifications perspective. J Strat Market 24(3–4):261–277. https://doi.org/10.1080/096 5254X.2015.1095222 14. Brasel SA (2012) How focused identities can help brands navigate a changing media landscape. Bus Horiz 55(3):283–291. https://doi.org/10.1016/J.BUSHOR.2012.01.005 15. Dudgeon M (2011) Tug of love: How technology is changing the relationship between consumers and brands- and what marketers can do about it. J Database Mark Cust Strategy Manag 18(4):313–321. https://doi.org/10.1057/DBM.2011.30/TABLES/7 16. Ding Y, Tu R, Xu Y, Park SK (2022) Repurchase intentions of new e-commerce users in the COVID-19 context: the mediation role of brand love. Front Psychol 13:4729. https://doi.org/ 10.3389/FPSYG.2022.968722/BIBTEX 17. Lelis C (2021) Smart Brands and Identities: building friendly bridges between Design and Smartness. International Journal on Interaction Design & Architectures, no. Pedagogical Approaches, Ludic and Co-Design Strategies & Tools supporting Smart Learning Ecosystems and Smart Education 18. Yigitcanlar T et al (2020) Can Building ‘Artificially Intelligent Cities’ Safeguard Humanity from Natural Disasters, Pandemics, and Other Catastrophes? An Urban Scholar’s Perspective. https://doi.org/10.3390/s20102988 19. Gretzel U, Collier de Menfonça M (2019) Smart destination brands: semiotic analysis of visual and verbal signs. Int J Tourism Cities 5(4):560–580. https://doi.org/10.1108/IJTC-092019-0159 20. Trinchini L, Kolodii NA, Goncharova NA, Baggio R (2019) Creativity, innovation and smartness in destination branding. Int J Tourism Cities 5(4):529–543. https://doi.org/10.1108/IJTC08-2019-0116/FULL/PDF 21. Kolla N, Kumar MG (2019) Meta-synthesis on artificial intelligence (AI): imperatives for branding. Int J Recent Technol Eng 8(3):2251–2255. https://doi.org/10.35940/IJRTE.B3268. 098319

Implications of New Media in Brands’ Visual Identities: A Systematic Literature

825

22. Schepers J, Belanche D, Casaló LV, Flavián C (2022) How smart should a service robot be? J Serv Res 25(4):565–582. https://doi.org/10.1177/10946705221107704 23. Basbeth F, Abd Ghani NH, Sedyowidodo U (2018) Smart destination branding: the need for new capability and opportunities for entrepreneurship. In: Proceeding - 2018 International Conference on ICT for Smart Society: Inno- vation Toward Smart Society and Society 5.0, ICISS 2018. https://doi.org/10.1109/ICTSS.2018.8549943

Intervention of Product Design in E-Commerce: Contribution to a Sustainable Proposal in Supermarket Delivery Services João Tinoco1

, Cláudia Lima1,2(B)

, and Rui Mendonça1,2

1 Faculty of Fine Arts, University of Porto, Av. de Rodrigues de Freitas 265, 4049–021 Porto,

Portugal [email protected] 2 ID+ – Faculty of Fine Arts, University of Porto, Av. de Rodrigues de Freitas 265, 4049–021 Porto, Portugal

Abstract. There has been an exponential growth in supermarket e-commerce services in recent years. Delivery services, due to their peculiarities and requirements, are causing a significant increase in single-use plastics. Given the pollution caused by excessive plastic waste, solutions must be created to minimize the consumption of single-use plastics. The main goal of this case study is to propose a new stage in the supermarket ecommerce services, replacing single-use plastic used for transporting orders with reusable products with high longevity. Mixed methods were applied for this research including analysis of packaging methods in Portuguese supermarkets, analysis of effective delivery solutions from supermarkets worldwide as well as participant observation of the picking process in one of the Continente warehouses. Intrinsic problems were identified that compromise the service effectiveness, efficiency, and sustainability. A set of components was designed to adapt to the equipment used by Continente and other national and international supermarkets and stores. This set is capable of reconciling, organizing, and protecting the ordered products, eliminating the need for plastic transport bags and other single-use elements used for preservation purposes. The conjunction of designers, engineers, and specialists in the elastomers and thermoplastics industry, as well as the support from experienced operations employees of the e-commerce service, make this proposal evidence of an adequate solution, capable of being mass-produced at low costs, providing significant improvements to the service. Keywords: Product Design · Sonae · E-commerce · Groceries Transport · Sustainability

1 Introduction This article reports a case study from a project for the master’s degree in Product and Industrial Design, of the Faculty of Fine Arts and the Faculty of Engineering, University of Porto. The project was carried out within an internship done at Sonae MC. The initial © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 826–839, 2024. https://doi.org/10.1007/978-3-031-47281-7_67

Intervention of Product Design in E-Commerce

827

phases of the project had contributions of 20 students of the master’s degree and a Design professor. Several presentations were done in class during the year for project analysis and discussion. These presentations were in the presence of Sonae MC representatives from the administration, quality management and e-commerce operations, as well as specialists in plastic injection from other companies. The project started from an invitation from Sonae MC to improve the e-commerce service of Continente. It was aimed to identify unsustainable or inefficient practices in the service process to solve them through Product Design. Continente Online is part of the Continente supermarket owned by Sonae MC. According to the Statista Database, in 2020 Continente Online was the Portuguese leader in e-commerce sales in the personal care and food market. This case study regards the exponential growth in supermarket e-commerce with the excessive consumption of single-use plastics and its consequences for the environment. The main goal is to provide a new proposal in supermarket e-commerce services that replaces the single-use plastic used for preserving the products ordered in the transportation phase with reusable elements of high longevity.

2 Methodology Mixed methods were applied sequentially to identify and understand the problem and create a more viable, effective, efficient, and sustainable solution. Initially, a documentary analysis was carried out through scientific articles, reports, and statistical data to understand the growth of e-commerce, with a deeper analysis of the food retail sector and the pollution from plastic packaging. In the next phase, an analysis of the packaging from Portuguese supermarket delivery services was made. Several online test deliveries with products of different types were ordered from Continente, Auchan, Froiz, El Corte Inglés and Intermarché supermarkets to understand the existing strategies or constraints when responding to the various requirements for preservation. Afterwards, participant observation was held at the Continente Gaia Jardim warehouse and made it possible to examine the entire picking process (separation and preparation of orders). The layout of the warehouse, the equipment and tools used, as well as interactions between employees and the system, were analyzed. Informal interviews were carried out with employees, to understand the difficulties and needs in completing the various tasks. An analysis of solutions currently implemented in supermarket delivery services worldwide was also included. Different systems, strategies and equipment were observed, with the aim of expanding knowledge regarding existing solutions applied for greater efficiency, effectiveness, and sustainability. Based on the findings, a new set of components was designed with the view to eliminate single-use plastics without compromising the delivery timings and preservation of the goods transported.

828

J. Tinoco et al.

3 The Growth of E-Commerce For M. Dhanabhakya [1] e-commerce (EC) is the process of buying and selling over the Internet or conducting any transaction that involves a transfer of ownership or rights to products or services through a computer-mediated network without the use of paper documents. According to Santos et al. [2], EC emerged through the advances of the Internet in the 1990s. In this decade, a series of opportunities arose, namely in communication. Some companies realized the potential of the Internet and used it as a means of interacting with their customers: they started to share information about their products and later developed ways of receiving orders and distribution. Jimenez et al. [3] states that society has an increasingly overloaded lifestyle, in which people are looking for the fastest, easiest, and cheapest way to carry out everyday tasks. Consequently, traditional retailers have become a less advantageous alternative for purchasing a huge variety of products when compared to EC. EC allows an easy purchase, regardless of the customer’s location. Sun and Finnie [4] noted that: There are three important facts that are revolutionizing the way we shop, first of all, internet connection is becoming accessible for more people around the world; the second fact is that mobile devices are making their technology user-friendly with the purpose that anyone, no matter their age, can be able to use it, and finally, it can be seen that for new generations, technology is part of their daily lives, they are “technology natives”, they can see, understand and use internet as something which is inherent. Due to advancement in technology, the transactions through any digital media are becoming a must for almost any company that provides a product or a service. For Duch-Brown et al. [5] EC continues to grow due to the rapid spread of information and the evolution of communication technologies as well as the massive adoption of the internet in recent years. The growing role of EC has resulted in major structural changes that are leading to a huge restructuring of manufacturing, marketing, and product acquisition processes. At the end of 2019, the COVID-19 pandemic emerged, exposing society to a series of threats and challenges that required various interventions to contain the spread of the virus. Certain restrictions, like going from place to place and physical store changes, significantly hampered access to essential goods. There were also psychological factors, highlighting the anxiety of going to public spaces, resulting from the potential health risk. Consequently, many consumers have been forced to change their behavior and consumption habits. The recurrence of EC services increased exponentially, with many experiencing them for the first time. In 2022, after the emergence of society from the pandemic, there was a permanence of behavioral consumption trends previously created with confinement. According to data available in the Statista Database (2022) in Portugal, from 2021 to 2022 an increase of 15.8% was recorded in EC revenues and a growth of 57% is expected until 2025. 3.1 E-Commerce in Food Retail Behavioral changes resulting from the COVID-19 pandemic led to an exponential growth of EC in the food industry. At the end of the pandemic, this sector was no exception concerning the permanence of consumption trends caused by confinement. With the data available on the Office for National Statistics, it was possible to observe that, despite

Intervention of Product Design in E-Commerce

829

the discontinuity of the implemented restrictions that made it difficult to go to physical food stores, although there was a decline, sales remained significantly higher compared to 2019 (see Fig. 1).

Fig. 1. Share of food retail sales made online in Great Britain from January 2018 to October 2022 (Office for National Statistics). Chart adapted by the author.

Lu & Reardon [6] state that food retail has constantly evolved over the last century, from traditional local stores to supermarkets, to EC. The rapid growth of EC is due to the expansion of economic benefits and range of products, reduced transaction costs and the fact that it allows home ordering and delivery. According to Siragusa & Tumino [7], EC services in the food sector have several peculiarities. First, the composition of the order is different from most other retail sectors. A typical supermarket online delivery order is made up of multiple one-piece lines belonging to various typologies, including frozen, fresh, and dry. Secondly, the distribution system for supermarkets is defined by a set of specific needs, with emphasis on distribution centers, which are generally located close to customers’ homes, to reduce delivery time. Thirdly, this sector has an almost zero rate of order return. For supermarket EC services to be able to establish an efficient and profitable business, due to their high complexity, especially when it comes to food products, an implementation of operating systems is required to guarantee that the products arrive preserved at the customer’s home within the timing of delivery. Food packaging has been a fundamental factor in the efficiency of these services, significantly optimizing the picking and transport of supermarket orders. 3.2 E-commerce in Food Retail Plastic is an extremely important material in our daily lives. In recent years, it has provided a considerable increase in our quality of life, provided an exponential evolution in various sectors, and aided in solving or mitigating various problems for society. Traditional food packages are passive barriers designed to mitigate the adverse effects of the environment on the product [8]. Plastics are highly diverse and are the most common materials in food packaging [9]. Arora and Padua [10] noted that:

830

J. Tinoco et al.

Over the past decades, polymers have replaced conventional materials (metals, ceramics, paper) in packaging applications due to their functionality, lightweight, ease of processing, and low cost. The use of synthetic polymers is ubiquitous in food packaging where they provide mechanical, chemical, and microbial protection from the environment and allow product display. For Marsh & Bugusu [11], several functions increase the importance of food packaging: the containment and prevention of food waste, protection and packaging, the possibility of tracking products in production, processing and distribution are other possible advantageous functions. Ease of handling, disposal, product visibility, sealing or microwave resistance are all contributors to packaging convenience. The packaging turns out to be the face of the product, being a crucial element in sales. It can improve its image and differentiate it from the competition as well as provide the necessary information about the product to the consumer and indicators of impenetrability (when the packaging is opened, it undergoes irreversible changes to warn the consumer). Due to its properties, plastic can respond to the various functions and requirements that food packaging generally needs. Hence, it is ubiquitous in supermarkets and traditional food retail stores, and indispensable in containment, protection, and transport. Currently, there is a huge diversity of plastics with a wide variety of characteristics, allowing its production in rigid or flexible forms, depending on the application. Packaging sometimes combines different polymers to respond to aesthetic and functional properties. Pieces with different sizes, physical and optical features, as well as complex shapes can be produced, offering considerable design freedom. Plastic also enables production processes to be optimized, with packaging able to be processed, formed, food inserted and sealed on a single production line. According to data from Conversio Market and Strategy GmbH, plastic packaging accounted for 44% of plastic use in 2021. 3.3 Plastic Pollution Chen et al. [12] state that single-use plastics are generally derived from polymers produced from petroleum, being effective and cheap materials for food packaging. However, in addition to typically coming from non-renewable sources, the vast majority of these are not biodegradable and therefore persist in the environment. In 2021, the Association Plastics Europe (2022) reported that around 390.7 million tons of plastic were produced worldwide. Recycling and production of plastics based on biological attributes represented only 9.8% of the plastic produced in the world. According to Jambeck et al. [13], about 9 million tons of plastic enter the ocean annually. This problem is recognized as a global threat to ecosystems, marine life and human health. Due to its high durability, it is estimated that it takes hundreds to thousands of years for plastic to decompose. Most of this waste comes from single-use plastics, such as plastic bags, which have a low return value and are difficult to degrade. The exponential growth of EC services in supermarkets, due to their peculiarities and requirements, is causing a significant increase in single-use plastic consumption. Given the problems caused by plastic waste, mainly due to its presence in the oceans, solutions must be created to reduce or eliminate the use of single-use plastics in EC services from supermarkets.

Intervention of Product Design in E-Commerce

831

4 Packaging Analysis from Test Orders When analyzing Continente Online orders, it was noticed that all products were in good condition, however, there was excessive use of single-use materials in their packaging, It was found an unnecessary amount of plastic bags and other single-use materials in the packaging of fragile products: bubble wrap, promotional magazine paper, cardboard boxes and Styrofoam boards were used in excessive amounts to protect and pack the products. Several detergent lids were sealed with exaggerated amounts of plastic tape. Finally, it was also found that the packaging of the same products varied from order to order, that is, there was not one single packaging procedure established (see Fig. 2).

Fig. 2. Order from Continente Online made in September of 2021. Photographs taken by the project team.

In the orders to other Portuguese supermarkets, it was verified that products were delivered in good condition. Although there are similarities in the transport solutions with Continente, in certain supermarkets, different packaging strategies were found. In the case of Intermarché, better use of the useful space of the RPCs (reusable plastic containers) was registered, minimizing the volume of orders, and reducing the movement of products. There was also a recurrence of paper, replacing plastic, as in the case of paper transport bags (Intermarché) and the use of promotional magazines as protective packaging for fragile products (Auchan). In some cases, ecologically worse solutions have been found. At Froiz, instead of being sealed with sticky tape, the detergents were placed individually in a tied plastic bag. To protect products, orders from El Corte Inglés, were inserted in plastic foam nets. The main problem identified, common to all supermarkets observed, was the excessive use of single-use materials, especially plastic. It is considered that a greater reconciliation of products per transport bag is needed, as well as a smaller amount of plastic for sealing and protection purposes (see Fig. 3).

Fig. 3. Examples of excessive use of single-use materials in orders from Auchan, El Corte Inglés and Froiz made in October 2021. Photographs taken by the project team.

832

J. Tinoco et al.

5 Participant Observation at Continente’s Picking Warehouse The picking of Continente’s EC service is carried out in warehouses specifically designed for the process or in conventional Continente stores. In specialized warehouses for picking, thanks to the specification of the layout, organization, tools and employees, the collection of products is carried out much more efficiently and effectively than in conventional stores. Participant observation made it possible to understand how Continente’s picking process works, as well as to identify various associated problems. This phase is organized by three different parameters: the green boxes, the picking process, and the identified problems. 5.1 Green Boxes The green boxes are rental RPCs used by Sonae MC in stores and in the Continente EC service that belongs to a reuse system called Pool service from IFCO and CHEP. In Continente’s EC service, all orders are placed in green plastic boxes that provide easy transport, greater protection, packaging, and separation. This equipment is highly resistant and durable, it has a fitting design that allows its stacking, forming towers, without compromising its structural rigidity. As a result, orders are organized and packaged in vans, significantly reducing product damage during the transportation phase. Another great advantage of these RPCs is that they are foldable. This functionality allows a significant reduction of the occupied volume when not in use. The following panel shows the three green boxes used in Continente ‘s EC service (see Fig. 4).

Fig. 4. Panel with the characteristics of the three green boxes used in Continente’s EC service (taken from the IFCO website in September 2022- https://www.ifco.com/pt/explore-os-nossosprodutos/). Panel adapted by the author.

After delivery of the order, the boxes are folded (compacted) and stacked in towers, occupying less volume in the van. Finally, with the orders delivered, the boxes are returned to the picking warehouse or conventional store warehouse for new order cycles. IFCO’s Pool service also includes a cleaning process. The green boxes are transported to an IFCO RPC washing center located in Azambuja (Lisbon). Next to the washing center, there is an IFCO warehouse where the clean boxes are stored for later distribution back to the Continente facilities. 5.2 Picking Process The warehouse at Continente Gaia Jardim is made up of two large buildings with wide aisles. The products are distributed across different zones. Each of the zones has one or more types of products where one or more pickers (picking employees) are located.

Intervention of Product Design in E-Commerce

833

When an order is placed on the Continente online platform, it is sent to a picking system, where the order will be collected. The pickers are spread across the various warehouse areas and work simultaneously to pick up products from the orders placed. Each picker carries a cart on wheels with 6 or 9 empty green boxes, where the products will be inserted. In front of the cart, there is a tablet with a built-in system that indicates which products to collect from the respective area. Most of the time, each picker oversees several orders at the same time, picking these orders simultaneously in that area. The picker moves with the cart through the aisles organized by typology. The products are collected and placed in plastic bags and then inserted in green boxes. Certain products require extra tasks when being collected: grocery products (the most common type) need to be bagged and weighed; bubble plastic sheets are inserted inside some egg cartons; products that have a high probability of leaking on the trip, namely detergents are sealed with several layers of adhesive tape. 5.3 Report from Pickers Through informal interviews with pickers of the Continente Gaia Jardim warehouse, the team managed to acquire important aspects of their daily needs and difficulties. When asked which was the main problem of the process, most pickers referred to the effort and time required for doing several packaging tasks, namely sealing detergents with adhesive tape, and coating every glass bottle. They claimed that new equipment should be implemented to accelerate and facilitate the process. Pickers also complained about the need to insert most ordered products in plastic bags, and for fragile products, to tie them to the IFCO green box handle. Some pickers suggested that there should exist an element in their carts to organize, separate and consolidate the products during the product collection with less time and effort. 5.4 Identified Problems Issues in Product Stowage. In the picking process, the products are placed loosely in green boxes, except those considered fragile. due to oscillations resulting from braking, cornering, acceleration and turbulence of the van, the products often collide with each other or against the walls of the green box. hence, they occasionally arrive damaged at the customer’s home. The empty space between the products in the green box is the main reason for their damage during transport, as it allows movements created from the oscillations of the van, which result in impacts. The Emptier Space There is in the green box, the more likely the products will be heavily impacted during transportation. most of these rpcs are not transported completely full. if the green boxes were filled, the products would have no room for movement. Due to the requirement for a maximum capacity weight per box (15 kg) and the current picking system on continente, it is rare for a green box to be transported full. Inefficient Use of the Green Boxes’ Volume. As aforementioned, the order is divided into tasks with different specificities depending on the items and by areas of the store or picking warehouse. For example, a customer who ordered fruits and detergents: since the product categories are found in different places in the store or warehouse, two employees

834

J. Tinoco et al.

will be responsible for their collection. thus, this order will use two boxes, instead of just one. a box would have enough space to transport all the products in the order, however, due to the picking system, two boxes will be used. This problem increases the delivery time and the employee effort required to perform the various tasks because there are more boxes to be transported, loaded, unloaded, and compacted. An inefficient use of the van’s cargo capacity results in a lot of waste, namely the increase in fuel consumption and the wear and tear on vehicles. This inefficiency forces the use of a greater number of vehicles or trips to respond to the same number of requests from a given area, increasing the environmental impact. Time and Effort Required on Existing Preservation Methods. On Continente Online there are frequently ordered products that, given the current system, require specific methods for the product to arrive at the customer’s home in good condition. Products such as eggs, glass bottles and certain grocery items are more likely to be damaged in transit. For this reason, fragile products are wrapped with single-use materials. Continente uses elements like bubble wrap sheets to protect fragile products. Moreover, plastic bags containing them are often tied to the handle of the green box to reduce product swaying during the transportation phase. Despite reducing the swaying of products inside the box, strong impacts persist during transport which occasionally result in damage to the products. In addition to fragile products, there are others like liquid detergents, which, due to their packaging, their content is highly prone to leaking during the transport of the order. Through the experience of processing the service, employees identified several products that leaked frequently. The products leaked for two reasons: i) with impacts on other products and on the walls of the box, ii) the constant pressure on the lid exerted by the movements of the liquid itself. To solve this problem, the lids of products at risk of leaking are sealed with several layers of plastic tape. Although this method prevents the spillage of the liquid, it requires a lot of time, both for its processing by the employee and for its removal by the customer after delivery of the order. Additionally, it requires high consumption of single-use materials. Excessive Consumption of Single-Use Materials. As aforementioned, there was excessive consumption of plastic tape to seal the lids of products at risk of leaking. In addition, other single-use materials such as bubble wrap, cardboard, and promotional magazine paper were used to protect products during transport. Moreover, another element stood out: the single-use plastic bag. Plastic bags from Continente Online have the function of packaging and separating products as well as facilitating their transport. Most ordered products are placed in plastic bags. Procedures for separating products into bags, following the rules of good practice and food safety legislation, significantly increase the number of plastic bag consumption. This excessive use of single-use materials in the Continente Online deliveries has become a major concern for the customers and the company due to the environmental impact caused. Sonae MC applied procedures to reduce this impact: customers have the option of returning plastic bags to Continente for recycling. However, less than 10% of plastic bags are returned.

Intervention of Product Design in E-Commerce

835

6 Analysis of Existing Solutions To avoid damage to products, several supermarkets applied solutions to prevent the movement of RPCs in transport. Tesco and Morrisons (United Kingdom) have attached RPC frames to conventional cargo vans. These support structures have slides that allow the RPCs to slide like drawers. This system results in less time and effort required to unload orders, as well as a lower probability of products arriving at home damaged. There are also support structures for RPCs independent of the cargo vans, which have adequate dimensions to prevent them from moving during the trip. These have wheels to allow large sets of RPCs to be easily transported. The Picnic (Netherlands) order loading system allows for easy and quick insertion of orders into the transport van. Intending to mitigate the environmental impact of the EC service, several supermarkets have implemented parcel delivery strategies that significantly reduce the consumption of single-use plastics. Instacard (United States), Woolworths (Australia) and Food Panda (Asia) are examples of supermarkets where products are placed in reusable bags, without single-use bags. Upon delivery, the reusable bags are left with the customer, where an extra cost is applied that can be returned to the store or in the next order. Therefore, disposable transport bags are not used, and the delivery time is shorter compared to conventional supermarket EC services. In Mandirituba (Brazil) and Morrisons (United Kingdom), ordered products are placed directly in the RPCs without transporting plastic bags. Employees carefully organize and consolidate the products, trying to keep them compact and placing the heaviest ones underneath. Upon arriving at the destination, the employee delivers the RPCs to the customer and waits for all the products to be collected and for the box to be returned. With this solution, in addition to not using any transport bags, through consolidation, the number of RPCs per order is lower and the products have less risk of being damaged during transport.

7 Defined Guidelines of the Design Proposal Based on the findings, a set of guidelines were defined for the design of a new proposal for Continente EC service: Exclude or Reduce the Need for Disposable Materials for Transporting Orders: To reduce the use of disposable materials, without compromising the orders and respecting the legislation in force and the defined good practice standards, separate certain products during the collection and transport phases of the service. Stowing Products: Deficiencies were found in the stowing of the products and the preservation methods proved to be ineffective and inefficient. The solution should stow the products better to reduce possible damage or spillage of contents during transport. Be Adaptable to the Ordered Products: Considering that Continente Online covers the entire Continente range, products with different dimensions, formats and weights are sold. The solution must be adaptable to any product or set of products ordered.

836

J. Tinoco et al.

Be Resistant and Durable: The resistance and durability of the equipment to be developed are crucial characteristics for an adequate solution. The requirement for the degree of resistance would depend on its entire cycle, involving several factors that the equipment could be exposed to, namely exposure to weight, force, impacts or even cleaning processes. The high durability of the equipment would result in a longer useful life, that is, less need for replacement and less waste. Be Easy and Fast to Use: The solution should require little time and effort to be processed by employees and customers to optimize the service. Reduce Wasted Space for Orders: Wasted order space, resulting from poor use of the RPCs’ available volume, requires additional vehicles or journeys to meet the same number of orders for a given area. Reducing wasted space lessens the environmental impact and brings economic benefits to the company. Once the guidelines were defined, brainstorming, sketching, 3D modeling (and prototype testing) of different design models were carried out with positive and negative points evaluated. At this stage, the project team acquired feedback from several specialists in the plastic injection industry to avoid constraints and lower costs envisaging the mass production of the design proposal.

8 Proposed Solution The solution consists of a set of components that adapt to the RPC from IFCO used by Continente Online. This is divided into two products that work together: the elastic band and the height extension. For this solution, it is proposed that the RPCs are given to the customer when delivering the order, applying an extra cost, which can be refunded by returning the boxes to the store or to the delivery employee, in the next order. The elastic band (see Fig. 5) protects, reconciles, and stows the products in the RPCs. This is divided into two groups of elements: the elastic membrane and the fitting plates. The docking plates are fitted to the RPCs previously without compromising their compactness. This plate is a single piece of plastic with a hole that locks the membrane shaft through a strong fit. The membrane is made of a highly elastic elastomer and has a plastic shaft at each end where two plug-ins are located. When packing the products, the employee places the products in the corners of the box and places one or more elastic membranes as needed. After use, the membranes are easy to remove and store.

Fig. 5. Render of the elastic band and photograph of the prototype. Images by the author.

Intervention of Product Design in E-Commerce

837

The height extension (see Fig. 6) is a component that fits over the top of the RPC, allowing other RPCs to be placed on top. It allows employees to position tall products vertically, being an advantage of transport in particular for glass bottles or detergents at risk of leaking. The product is made of polypropylene and has a similar structure to the IFCO RPC. It is coloured red to be easily identified by employees. It is proposed that the height extension is applied in RPCs whose products exceed its height during the picking process. Right before delivering the RPCs to the client, the driver can easily remove the height extension by pulling it out of the RPC. After the delivery, this element can be folded, minimizing storage space.

Fig. 6. Render of the height extension and photograph of the prototype. Images by the author.

It is considered that this solution can extinguish the plastic bags used for transport and other single-use protective and spill-resistant plastics. In addition, it solves the remaining identified problems and brings several other advantages to the service: it stows, reconciles, and protects the products; eliminates the risk of detergents leaking; provides more efficient use of the useful volume of the green boxes, significantly reducing the number of boxes per order; it has an intuitive handling, similar to the existing equipment, making it easier for the employee to adapt; it is adaptable to existing equipment and cleaning system; it takes up little space when stored. Additionally, it is suggested the application of this solution should be accompanied by modifications in the EC system from Continente envisaging a more suitable service, not only regarding the picking process but also the layout and equipment from stores and warehouses as well as changes in the Continente Online website. This solution should be also accompanied by a marketing campaign that values the implemented solution by raising client awareness of the problems caused by single-use plastics.

9 Final Considerations Through this study, it is concluded that in supermarket home delivery services there is often excessive consumption of single-use materials, mainly plastics, which contributes significantly to its ecological footprint. The exponential growth of EC requires changes to be implemented in supermarket services to reduce their environmental impacts. The main goal of this study is to contribute to a more sustainable EC service for supermarkets, replacing single-use materials used in the transport of orders by reusable

838

J. Tinoco et al.

elements of high longevity. This case study showed the potential of Design to improve the sustainability, efficiency, and effectiveness of EC services in supermarkets. The multidisciplinary team of designers, engineers, specialists in the elastomers and plastics injection industry as well as the feedback from experienced employees of Continente’s EC service make this proposal evidence of a pertinent solution, capable of being mass-produced at lower costs and contributing to a better stage in supermarket home delivery services. For the next phases, the product designed will be prototyped and submitted to several real operating tests in picking warehouses or stores. It is crucial to evaluate the solution’s relationship with locations, equipment, and people throughout the whole process in order to identify new needs or aspects that can be improved. Acknowledgements. The authors acknowledge the institutions involved in this project, the University of Porto, Sonae MC and their professionals, as well as the students that participated in the initial stages of the project. This project is funded by the master’s degree in Product and Industrial Design, of the Faculty of Fine Arts and the Faculty of Engineering, University of Porto.

References 1. Dhanabhakya, M.: Directorate of distance education dde-where innovation is a way of life master of commerce M.Com-Second Year Course Code: 59 Paper Code: MCOM2005 ECOMMERCE (2005) 2. dos Santos VF, Sabino LR, Morais GM, Goncalves CA (2017) E-Commerce: A Short History Follow-up on Possible Trends. Int J Bus Adm 8(7):130. https://doi.org/10.5430/ijba.v8n7p130 3. Jimenez D, Valdes S, Salinas M (2019) Popularity comparison between e-commerce and traditional retail business. Int. J. Technol. Bus. 1(1):10–16. https://doi.org/10.5281/zenodo. 2591555 4. Sun Z, Finnie GR (2004) Intelligent techniques in e-commerce. Springer, Berlin 5. Duch-Brown N, Grzybowski L, Romahn A, Verboven F (2017) The impact of online sales on consumers and firms. Evidence from consumer electronics. Int J Ind Organ 52:30–62. https:// doi.org/10.1016/j.ijindorg.2017.01.009 6. Lu L, Reardon T (2018) An economic model of the evolution of food retail and supply chains from traditional shops to supermarkets to e-commerce. Am J Agric Econ 100(5):1320–1335. https://doi.org/10.1093/ajae/aay056 7. Siragusa C, Tumino A (2022) E-grocery: comparing the environmental impacts of the online and offline purchasing processes. Int J Log Res Appl 25(8):1164–1190. https://doi.org/10. 1080/13675567.2021.1892041 8. Brody, A. L., Bugusu, B., Han, J. H., Sand, C. K., McHugh, T. H.: Innovative food packaging solutions. In Journal of Food Science 73(8) (2008). https://doi.org/10.1111/j.1750-3841.2008. 00933.x 9. Geueke B, Groh K, Muncke J (2018) Food packaging in the circular economy: Overview of chemical safety aspects for commonly used materials. J. Cleaner Prod. 193:491–505. https:// doi.org/10.1016/j.jclepro.2018.05.005 10. Arora, A., Padua, G. W.: Review: Nanocomposites in food packaging. In Journal of Food Science 75(1) (2010). https://doi.org/10.1111/j.1750-3841.2009.01456.x 11. Marsh K., Bugusu B.: Food packaging - Roles, materials, and environmental issues: Scientific status summary. In Journal of Food Science 72 (3) (2007)

Intervention of Product Design in E-Commerce

839

12. Chen Y, Awasthi AK, Wei F, Tan Q, Li J (2021) Single-use plastics: production, usage, disposal, and adverse impacts. Sci Total Environ 752:141772. https://doi.org/10.1016/j.scitot env.2020.141772 13. Jambeck J., Geyer R., Wilcox., Siegler T., Perryman M., Andrady A., Narayan R., Law K.: Plastic waste inputs from land into the ocean. In Science 347(6223), pp. 768–771, (2015) 14. BCSD. Case Study: Pool Service (2013). https://bcsdportugal.org/wp-content/uploads/2013/ 10/2015-CS-SONAEMC-PoolService1.pdf

Adapting Advanced Typographic Taxonomy Principles for Modular Stencil System Classification João Francisco Rodrigues Gomes(B)

and João Aranda Brandão

CIAUD, Research Centre for Architecture, Urbanism and Design, Lisbon School of Architecture, Universidade de Lisboa, Rua Sá Nogueira, Pólo Universitário do Alto da Ajuda, 1349-063 Lisbon, Portugal [email protected]

Abstract. Modular stencil systems have long been an important part of the typographer’s toolset and, more recently, have experienced a resurgence as part of typography teaching toolkits. Constant advances in fast prototyping and manufacturing technologies, combined with ongoing research on their history and development, may further popularize these tools on educational and creative settings. Their classification, for evaluation and comparison purposes, is paramount to their successful implementation on those settings, and the complexity and variety of their corpus, on account of their modules being tailored to typography and lettering and offering levels of flexibility comparable to those found in cuttingedge variable typefaces, necessarily follows that of the contemporary typographic corpus. As such, we propose the repurposing of extant advanced typographic taxonomy systems, namely those especially tailored for modular, geometric and variable typography. These are already based on a fine-grained understanding of typographic anatomy, and include categories, such as those related to grids and to module variety and shapes, also particularly relevant to stencil systems. In order to tackle the complexity of these stencil systems, we follow a theoretical approach, through a non-interventionist methodology of qualitative research, via literature review and case study analysis, selecting these cases based on their current and immediate commercial availability and/or potential licensing options, and their grounding on ongoing or recently published research. Keywords: Type Design · Typeface Classification · Typographic Taxonomy · Modular Fonts · Stencil Systems · Typography Teaching · Syllabus Development

1 Introduction Teaching typography and type design has long been a complex affair, due to the fine detail pertaining to the alphabet’s history and structure, which must be quickly assimilated by design students [1]. On the theoretical side, various approaches are used, including those letterform design theories based on calligraphy, namely the ones advanced by Johnston [2, 3] and Noordzij [4], one of their core tenets being the intimate relationship with the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 840–855, 2024. https://doi.org/10.1007/978-3-031-47281-7_68

Adapting Advanced Typographic Taxonomy Principles for Modular Stencil

841

tools used by artisans and the emphasis on strokes. These fostered the revival of the historical link between calligraphy and conventional moveable type and, especially with the latter’s parametric three-axis cube model, further enhance the understanding of type design with an added level of abstraction, in the form of combinatorial design principles which would be hard to realize with said traditional tools. On the practical side, however and due to logistical and/or curricular limitations [5], either calligraphy practice or advanced digital type design may not always be an option, or not to the extent necessary for a proper, organic and sustained acquisition of a sufficient knowledge of typographic anatomy. As an answer to this issue, typography and type design teachers have commonly resorted to modular type design exercises [6–11], as they make the learning more simple and fun [12]. The systems most recently proposed (see Fig. 1) have been either fully digital – such as FontStruct [13–15] –, hybrid – such as Q Project [16, 17] – or stencil-based – such as RUHA [18] –, and their emphasis on modules, and especially the latter two’s outline-based mode of operation, make them especially suited as primers to the leading outline-based type design approach followed by default in all popular specialty software packages. Moreover, stencil-based systems allow for expressive qualities not easily emulated in simpler digital solutions (see Fig. 2) and bridge the gap left by potentially rushed and, thus, less-than-ideal calligraphy practice, in that they also entail immediate, physical experimentation with traditional tools and processes – in this case, those of lettering and graphic arts – and even with other more exotic materials and resources, with all the pedagogical benefits such a manual and playful approach offers [19–21].

Fig. 1. Q Project (top-left) [17], RUHA (top-right) [the authors], Super Veloz (bottom-left) [22], Experimental Type (center-bottom) [23], and TIMOS (bottom-right) [9]

If we add the fact that they are grounded on sound, and in some cases ongoing research, this growing corpus of stencil systems seems to show immense promise, especially in learning environments and institutions where there is already a tradition of manual, craft-like exercises and the manufacturing infrastructure needed to reproduce

842

J. F. R. Gomes and J. A. Brandão

Fig. 2. RUHA’s expressive flexibility in action [the authors]

the necessary parts through licensing agreements, or the means to otherwise acquire them in finished form. Nevertheless, not all these stencil systems are created equal, and a dedicated evaluation process thereof, which transcends a mere review of the associated literature, is, thus, required before any of them are chosen for integration in teaching plans. Many factors should be taken in consideration, namely their material characteristics and durability, their ease of use and versatility, and especially their relevance vis-à-vis the syllabus and the aforementioned letterform design theories contained therein. Considering the existence of inherently expandable typographic taxonomy systems already based on, or at the very least compatible with and adaptable to those theories, it naturally follows that those systems are also prime candidates for expansion into tools for analysis and classification of both the material and technical substance of the stencil systems under review. Thanks to the infinitely customizable and fine-grained nature of advanced typographic taxonomy systems, stencil systems may, thus, be analyzed at the level of the modules they offer and their potential from a typographic design standpoint, as well as the composition aids and letterform construction methods they support or even actively foster.

2 Stencil Systems: An Overview 2.1 Modularity in the Alphabet Advanced typographic stencil systems rely on the breakdown of the alphabet into the most basic components of its anatomy, in the form of discrete modules, and should ideally put into evidence its functional roles and the interaction between them. In order to better identify those factors in stencil systems, it is, therefore, useful to provide some context on their structuring and nomenclature in traditional letterform design techniques, according to the aforementioned theories. Evolving from the simple monolinear and later more subtly finished nature of the Greek alphabet, Roman lettering archetypes were based on brush-based strokes and finishings [23], and further rationalized according to geometric grids [24] and transposed to different media and tools, both manual and mechanized, throughout history. Despite the considerable formal variation across the currently dominant Latin archetypes – Roman Square Capitals, Carolingian Minuscules and 15th century Papal Chancery hand-based Italic letters –, and even now-defunct or otherwise niche archetypes – Textura, Fraktur, Schwabacher, Sütterlin, etc. –, their constituents can generally be interpreted, on a more abstract level, as a system of junctions [25] or roles [26, 27], and, on a more concrete level, as a system of structural anatomical components and non-essential – but stylistically crucial – finishings [28] (see Fig. 3).

Adapting Advanced Typographic Taxonomy Principles for Modular Stencil

843

Regarding the expression of the strokes themselves, these may either be understood and idealized (or literally produced) according to said conventional, calligraphy- or epigraphy-based theories – wherein they follow median lines, or skeletal forms [2, 4, 29] –, as a series of outlines which are then filled in [30–32], as an assemblage of discrete, adjacent [33, 34] and/or overlapping [35] geometric modules, or even as spaces filled in pre-determined grids [36–38]; all these categories, especially the last two, can actually overlap, and give way to modular-specific letterform construction approaches such as monolinear, formal, hybrid, etc. [39].

Fig. 3. Frutiger’s theory of junctions (left) [25] and Cabral’s summary of serif classification (right) [28] [adapted and translated by the authors]

2.2 Modularity in Stencils For the purposes of teaching the inner workings of letterform anatomy, when we refer to modularity in stencils, we are not exactly including modular systems of interlocking or otherwise framed and fully formed-character stencil plates [40], which may be useful still as a design or teaching tool but fall outside of the scope of said purposes; we are, instead, focusing on stencil systems based on the multi-part letter principles set forth by des Billettes and followed later on in stencils aimed at sign writing, in systems such as Econosign and Stencillor, in plastic lettering guides [41] and, indeed, in the more specialized cases we will be analyzing shortly. Regarding these, Gamonal-Arroyo [12] offers us his own comprehensive review of geometric modularity and grids in the latin alphabet and of other historical stencil systems alike, including the even more compact and universal Plaque Decoupée Universelle by Joseph A. David [42], an early example of the push towards standardization in design succinctly explained by Albers in 1926 [43–46]. In addition to a review of bespoke contemporary stencil systems targeted at typography teaching, such as the aforementioned RUHA, Gamonal-Arroyo proposes and executes the transposition and simplification into stencil form of historical modular typesetting systems, such as Nebiolo foundry’s Fregio Mecano, Iranzo foundry’s Figuras Geometricas or Trochut’s Super Tipo Veloz (see Fig. 1), all of which, much like Kombinationsschrift „3“ and its extant incarnations, were only materialized in more traditional, letterpress-bound form. The latter system, here called just Super Veloz likely due to the fact that it is no longer based on lead type, is notable for the successful pedagogical experiments made by Gamonal-Arroyo with both an inexperienced audience of children aged 5–12, which harkens back to Soennecken’s Kindergarten-bound geometric Fröbel-Gaben for letterform construction [47], and an experienced audience of adults, whose usage of advanced lettering techniques further proved the added flexibility of the system besides the obvious freedom it already offered over its lead counterpart, thanks to the easy module rotation,

844

J. F. R. Gomes and J. A. Brandão

mirroring and superposition operations it affords by design. Both this system, with its emphasis on extreme variations in contrast and very obvious calligraphic references, and RUHA, with the same tenets plus the added variety on finishings and ornamentation, as well as alignment notches, a module indexing system and a grid, seem to be comprehensive, flexible and compact enough to reach these goals. On the more strictly geometrical and abstract side of the spectrum, Slanted’s Experimental Type stencils [48] offers (see Fig. 1), with its three plates, a great variety of assorted structural and vaguely ornamental shapes, but nothing in the way of composition/design aids, either on the plates themselves or in the magazine that prompted its production; furthermore, there is some redundancy and wasted potential in that some shapes could decomposed into simpler modules, and the shapes themselves do not seem to adhere to a common scale or metric standard. TIMOS [8, 9], on the other hand, while being considerably more limited in terms of modules offered, is more cohesive and programmatic in its shape choices – i.e. they suggest specific treatments for stroke corners and other anatomical elements – and strictly square-grid-based. Furthermore, said grid is on full display by virtue of being etched on its surface, and is complemented by alignment notches like those found on RUHA or on des Billettes’s own historical specimens. Straddling these two universes is Typotheque’s Q Project, by far the most diverse of them all, as each of its plates – Q Shapes 01, Q Shapes 02 and Q Shapes 03 – effectively correspond to a traditional Roman Square Capitals pattern, a monolinear pattern and a modular and geometric pattern (see Fig. 1). In the first plate’s case, what the system gains in compactness loses in clarity and flexibility, as the shapes are not laid out according to their position vis-à-vis the corresponding letter’s anatomy or to typometric guidelines – which are both literally and virtually non-existent – and their proximity precludes the usage of any technique where ink might easily bleed onto the nearest modules and may compromise its structural stability, as a break between modules labelled “i” and “v” attests; on the other hand, the familiarity of these shapes to even a moderately proficient typography student may mitigate some of its quirks to a certain extent. In the second plate’s case, the reduced set of modules avoids said structural problems, and the alignment of many of the modules should make composition much easier for many of the letterforms. And finally, in the third plate’s case, the structure seems to still be solid enough and alignment considerably easier. Common to all these elements is the absence of printed guidelines or alignment notches and the usage of an alphabetical and single-digit module labelling system, which may be confusing as it clashes with the actual characters to which construction they contribute to, effectively resulting in a typographic Stroop Effect. Notably absent from plates Q Shapes 02 and Q Shapes 03 are also the more complex ornamental finishings found in their digital counterparts, with only the most basic and geometric ones being available. Finally, even though plate Q Shapes 01 is not at all congruent with the rest of the system, and the latter two may be used in conjunction in a way apparently not envisioned in the digital system due to, once again, the structural modules not sharing a common scale or metric standard, each of them is internally cohesive.

Adapting Advanced Typographic Taxonomy Principles for Modular Stencil

845

3 Modular Stencil System Classification: An Adaptation 3.1 Typographic Taxonomy Systems Overview As a vital complement to the overview hereby presented, the most promising cases out of these shall be further analyzed according to a standardized framework that covers, on one hand, their potential – intended by default and extended by predictable subversion of the rules given or suggested by their design – from a finalized letterform family (i.e. a quasi-typeface) standpoint, and, on the other hand, their stencil-specific characteristics. Such a framework, if properly set up, will allow for finer comparisons between these cases, and should necessarily draw from the best practices in respectively the research fields of traditional typographic taxonomy and of stencil systems. Interestingly, the leading experts on each of these areas, Catherine Dixon and the already cited Eric Kindel, cooperated in the publication Typeform Dialogues, on the eponymous interactive project developed at the Central Saint Martins College of Art & Design, London, with the latter author indicating the eminent Nicolete Gray as the precursor to an all-inclusive philosophy when originally setting up, along with Nicholas Biddulph, the Central Lettering Record at one of its predecessor institutions, the Central School of Art and Design [49]. Dixon would, then, follow on Gray’s footsteps and, while tackling the ever-increasing variety of specimens hosted by said record by the mid-1990s, develop a tentative – but untenable – more complex yet traditional taxonomy system, followed by an advanced, descriptive and inherently expandable one based on the sources of inspiration behind a particular set of letterforms – be they certain tools or processes, traditional hands, figurative elements, etc. –, on their particular formal attributes – namely identifying details on certain characters –, and on the recurrent stylistic patterns – or trends, be they contemporary or transhistorical – that arise from the combination of all those factors [50], the latter constituting quasi-classifications. Since it is, as mentioned, descriptive in nature, this system not only allows for the creation of brand new categories on the fly, but also offers an extremely fine level of detail, as it allows for any specimen to be categorized under several categories at the same time – which, especially in more recent extended versions thereof, is of extreme importance for modular typography, where traditional sources may be given a very specific, modular and grid-based treatment [39], or for variable typefaces, where any given specimen is no longar static and may, in fact, cover an entire range between two or more traditional classifications [51]. As these advanced taxonomy systems are geared towards the categorization of fullyformed typefaces, they already contain the headers and sub-menus necessary for the categorization of any system – letterpress-based, stencil-based, digital, etc. – with lettering production in mind; even if, as mentioned before, some degree of subversion is allowed by such lettering systems, the stated or implicit goals of their creators can serve us as the lowest-common-denominator for their categorization. Likewise, those advanced taxonomy systems, especially the variant extended and geared towards modular typography, are also designed to accommodate very specific elements such as modules and grids, which make an even more unavoidable appearance on stencil-based systems. As for their usefulness to evaluate stencil-based systems as teaching tools, the fact that

846

J. F. R. Gomes and J. A. Brandão

these taxonomy systems are built from the ground up to also analyze construction methods grounded on specific letterform design theories makes any of their users especially mindful of those. 3.2 Proposed Extensions and Adaptations to an Existing Advanced System While it would be tempting to use, virtually unchanged, one of these advanced typographic taxonomy systems for this particular use case, such an approach would, however, be misguided and insufficient. Much like with modular and variable typefaces before them, the creation of some stencil-specific extensions is also naturally in order. Thus, taking Brandão & Gomes’s extended variant [39] as a starting point, we propose the addition or adjustment of the following stencil-specific headers and respective submenus to it: • The header Material Characteristics, as those inform the user of what exactly the system offers and how durable it may be, with Number of Plates, Size of Plates – including thickness –, Material – e.g. acrylic/PP/PE-LD/PE-HD/card stock/metal –, Texture, Process of Manufacture – e.g. punch-cutting/laser-cutting/water-jetcutting/CNC machining –, and Process of Application of Design Aids to Surface – if applicable, e.g. silkscreening/etching/engraving – as sub-menus; • The header Design Aids (when applicable), as they can greatly enhance the user experience of such as system and make self-evident the intended constraints and design goals behind it, and Module Naming/Labeling – e.g. numerical or alphabetical –, Character Reference – whether the characters whose construction each module is used for are referenced next to the latter –, Anatomical Reference – whether the anatomical elements whose construction each module is used for are referenced next to the latter – Grids, and Alignment Notches – as designed and used, just like mentioned before, in des Billettes’s historical stencils – as sub-menus; • Due to the usually wide variety of modules offered in these systems, the addition, to the already existent Modules header, of module-specific sub-menus, and corresponding sub-sub-menus such as Module Construction, Module Shape, Structure Detailing, Direct Reference to Tool, and Anatomical Function(s) – i.e. if any given module corresponds to one or several typical letterform components, be they structural or decorative; • And finally, taking a page from Kindel’s own analysis of these systems, the addition of the qualifiers Adaptive Form and Natural Form [41] to the sub-menu Approach of the header Construction, as these seem to be a determining factor both in character design and in the establishment of particular stencil-specific patterns. Notably, there is some overlap between the Modules’ sub-sub-menus and other headers and sub-menus, but the fact that certain modules have some functions does not mean that they completely fulfil them or that other variations aren’t possible or even explicitly suggested, in a similar spirit to that of Albers’s Kombinationsschrift „3“ [33, 34], of experiments by Crouwel [38] and Schrofer [37], or of modern variable typefaces [51].

Adapting Advanced Typographic Taxonomy Principles for Modular Stencil

847

4 Case study Analysis 4.1 Case Selection Criteria The analysis process herein proposed will be part of a wider, doctorate research project and, as such, will cover not only all the aforementioned cases, but more specimens which may be found or created within said research’s deadlines. However, for the sake of demonstration and validation of the framework advanced for it, a narrower selection is preferable. Thus, a more general case study selection criteria, according to the details presented in the overview in 3.2, must be followed. These criteria are based on three core tenets: variety/flexibility, pedagogical value due to the adherence to relevant letterform design theories, and predictable ease-of-use. Thus, the three case studies selected for analysis were RUHA, TIMOS and Super Veloz. 4.2 Case N.º 1: RUHA • Material Characteristics number of plates: 1; size of plate (incl. thickness), in mm.: 420 × 320 × 8,2 (open), 240 × 320 × 1,6 (closed); material: PP (Priplak® Opaline 800µ mandarine); texture: rough on the obverse, smooth matte on the verse; process of application of design aids to surface: silkscreen, on the obverse • Design Aids module naming: yes, numerical; character reference: yes; anatomical reference: yes; grids: yes; alignment notches: yes, for the baseline and uppercase height on the top row, and for the baseline, x-height, ascender height and descender height in the middle row • Construction approach: adaptive form, derived from hard alloy lead type characters; structural detailing: non-cursive, straight, unconnected characters, with high contrast, inspired in the French model of the Romain du Roi and Modern Didone typefaces; direct reference to tool: conventional, 18th –19th century French typography and flexible, pointed nibs; character sets: upper- and lowercase, including diacritics and both ornamented and non-ornamented variants of all finishings and some structural components. • Grid kind: horizontal, fixed fields on the y axis, without field subdivision; uppercase/ascender height subdivided into 10 equidistant units (7 of which corresponding to the x-height, and 3 additional units for the descender height), repeated across three rows of modules, and four different horizontal space guidelines (word space, and large, medium and small letter spaces); module snapping: yes, except for finishings; character snapping: yes

848

J. F. R. Gomes and J. A. Brandão

• Shapes overall treatment: uppercase characters adhere to a classic, Roman Square Capitals model, and most lowercase characters present a rounded, high-contrast look and with generous counters when constructed with the default thin serifs.; curves: most characters feature perfect arc rounded external corners with constant but bespoke radiuses; stems: (basic) straight with parallel edges; (ornamented) with square lozenge notches cut on either side • Modules number: 79; 1–6: vertical stems, all rectangular; 7, 24 and 26: short, medium and long serifs, all rectangular; 8–11, 20 and 22: uppercase curves, except for 11, also used in lowercase g, and 22, also used in lowercase t; 12–19: horizontal bars, all rectangular; 21: top stroke for exclamation point; 23: lower-right stroke of uppercase G, rectangular with a diagonal notch; 25 and 27–29: uppercase diagonal strokes; 30 and 76–77: small, medium and large lozenges, all square; 30 is tittle/dieresis-sized; 31 and 78–79: small, medium and large circles, all perfect; 31 is tittle/dieresis-sized; 32–37: diacritic marks (half-tilde, acute/grave, half-ring, lower-half-cedilla/ogonek, upper-half-ogonek, upperhalf-cedilla); 38–42: lowercase vertical stems, all rectangular, except 39 (ornamental, notched on either side) and 40 (notched on lower-right corner for connection with bowls); 43–48: lowercase curves; 50–53: lowercase diagonal strokes; 54–59: bracketed serifs (57 without spur); 60–61: serifs with vertical spur; 62 and 70–73: wedge terminal and serifs; 63–65: bifurcating terminal and serifs; 66–69: Tuscan terminals and serifs; 74–75: ornamental circle-diamond-circle and double-circle terminals/serifs • Modelling contrast: very high; axis of contrast: vertical; transition: gradual • Terminals baseline/general: horizontal, straight cuts by default (ornaments optional); x-height: same as the baseline; ascenders and descenders: same as the baseline • Proportion characters, both uppercase and lowercase, are approximately square, with a relatively short x-height; relative internal proportions: ascenders, descenders and uppercase height relatively short • Weight heavy • Key Modules 39 (ornamental vertical stem, notched on either side); 62–74 (wedge, bifurcating, tuscan and geometric terminals and serifs)

Adapting Advanced Typographic Taxonomy Principles for Modular Stencil

849

• Decoration conventional, spur, wedge, bifurcating, Tuscan or geometric terminals and serifs. 4.3 Case N.º 2: TIMOS • Material Characteristics number of plates: 1; size of plate (incl. Thickness), in mm.: approx. 195 × 165 × 15; material: transparent acrylic; texture: smooth on both sides; process of application of design aids to surface: engraved • Design Aids module naming: yes, numerical; character reference: no; anatomical reference: no; grids: yes; alignment notches: yes, one for each grid unit • Construction approach: natural form, derived from modular and geometric lead type characters; structural detailing: non-cursive, straight characters; direct reference to tool: modular letterpress typography, metal, glass, ceramics, etc.; character sets: upper- and lowercase, figures, diacritic marks and a few larger reference characters • Grid kind: square, regular, with no field subdivision; module snapping: yes; character snapping: yes • Shapes overall treatment: monoline or with variable contrast according to the number of modules used on each axis; curves: several different styles of inner and outer finishings offered, such as corners with varying radiuses, and square triangles and chamfers; stems: straight with parallel edges • Modules number: 23; 1: 1 × 1 unit square; 2–5: assorted square-based shapes; 6, 12 and 18: 1 × 1 unit squares with 1 unit-, ½ unit- and ½ unit-radius rounded corners; 7, 13 and 19:1 unit-, ½ unit- and unit-radius internal rounded corners; 8: 1 × 1 unit squares with 1 unit-radius rounded corner and ½ unit-diameter circle counter; 9: teardrop shape ( unit-diameter circle with square corner); 10: 1 unit-diameter circle; 11: 1 × 1 unit square with all internal corners rounded forming a circle, with bridges; 14 and 20: 1 × 1 unit squares with one large and small chamfered corner; 15 and 21: large and small square triangles; 16–17: 1 unit-width chevrons with inner and outer side cut flat; 22: unit-shape with top corners rounded forming a semicircle; 23: 1 1 unit-width × unit-width × unit-shape with truncated semicircle

850

J. F. R. Gomes and J. A. Brandão

• Modelling contrast: null by default, or variable, depending on the number of modules used on each axis; axis of contrast: none by default, or vertical/horizontal depending on the number of modules used on each axis; transition: none, gradual, abrupt or instant, depending on the size of characters and on the modules used • Terminals baseline/general: horizontal, straight cuts by default (constructed serifs in larger sizes optional); x-height: same as the baseline; ascenders and descenders: same as the baseline. • Proportion on their reference sheet presentation, characters, both uppercase and lowercase, are approximately square, with a relatively short x-height; relative internal proportions: ascenders, descenders and uppercase height relatively short • Weight regular by default, or variable, depending on character size and the number of modules used on each axis • Key Modules 8 (1 × 1 unit squares with 1 unit-radius rounded corner and ½ unit-diameter circle counter); 16–17 (positive and negative half-chevrons); 22–23 (positive and negative semi-circle terminals) • Decoration n/a 4.4 Case N.º 3: Super Veloz • Material Characteristics number of plates: 1; size of plates, in mm.: 180 × 230 × 2/3; material: transparent PMMA; texture: smooth on both sides; process of application of design aids to surface: engraving, on the obverse • Design Aids other: typographic point ruler on the left edge, and metric ruler on the right edge. • Construction approach: natural form, derived from modular and geometric lead type characters; structural detailing: hybrid (cursive and non-cursive), mostly straight characters with flowing thin strokes; direct reference to tool: modular letterpress typography and flexible steel pointed nib; character sets: uppercase/monocase

Adapting Advanced Typographic Taxonomy Principles for Modular Stencil

851

• Grid n/a • Shapes overall treatment: monoline or highly contrasted when two weights are present on the same; curves: approximately round, except for some more oval modules; stems: straight with parallel edges • Modules number: 69; from left to right, top to bottom: 1: thick vertical stem, rectangular; 2: black diagonal stroke; 3–4 and 11–12: black curved strokes, with transition(s) from thin horizontal bars; 5: medium vertical stem, rectangular; 6: medium diagonal stroke; 7– 8 and 15–16: medium curved strokes, with transition(s) from thin horizontal bars; 9: black diagonal to vertical stem transition; 10: black curved shoulder to vertical stem transition; 13: medium curved shoulder to vertical stem transition; 14: medium diagonal to vertical stem transition; 17: thin horizontal bar and diagonal stroke crossing; 18–19: thin diagonal stroke and horizontal bar joins; 20: light vertical stem; 21–22 and 24: light curved strokes, with transition(s) from thin horizontal bars; 23: light diagonal stroke; 25: light half-oval stroke, ¾ height; 26: thin vertical stroke; 27: thin vertical bracket; 28–29 and 34–35: thin curved strokes, monolinear; 30–31: thin diagonal strokes; 32: thin curved shoulder to vertical stem transition; 33: thin diagonal to vertical stem transition; 36: thin half-circle stroke, ¾ height, monolinear; 37–59: assorted thin strokes, corners, intersections and joins, ½ height, monolinear; 60–64: assorted thin strokes, corners, and joins, 35% height; 65: circle, 35% height; 66: square diamond, 35% height; 67: square, 35% height; 68: quarter circle, 35% height; 69: quarter arc, inner radius half of external radius, 35: height • Modelling contrast: high, medium or null, depending on the combination of modules used; axis of contrast: vertical; transition: abrupt • Terminals baseline/general: horizontal, straight cuts, or curved, depending on modules used; xheight: same as the baseline; ascenders and descenders: same as the baseline • Proportion characters are relatively tall; relative internal proportions: ornamental strokes and finishings can be made to be extra tall or wide • Weight thin, light, medium or black, depending on the modules used

852

J. F. R. Gomes and J. A. Brandão

• Key Modules 37–43 and 48–53 (highly detailed thin strokes for calligraphic-style monolinear ornaments) • Decoration Calligraphic-style, monolinear ornament

5 Conclusions After the two-stage analysis of these three systems, we can clearly see they exist in a continuum between simplicity and complexity, absolute freedom and a certain level of prescriptive restraint. In either case, whether the end-user has too much choice – which can induce paralysis [52] – on the blank page or on the stencil plate, an experienced guide can, indeed, make a crucial difference, and it is imperious that this expert deeply knows and understands the systems being put into use. The very act of picking those systems is, itself, a choice, and some research is always needed in assisting teachers through that process. These expandable frameworks, as has been posited before, can help them, as they have helped and, seeing how they categorically answer some of our needs in related ongoing research and upcoming teaching endeavors, will still help us, in that goal, even if they are not being used in the classical, commercial-bound applications typeface classification systems are known for. Indeed, they show promise in an academic setting, and we accordingly extol creators of these tools to anticipate that need and preemptively categorize and describe their own systems in a more standardized fashion. In so doing, teachers may be able to better ascertain whether any given system is suitable for a particular syllabus. Case in point: TIMOS may constitute a good primer for beginners, where notions of detailed typographic anatomy may not yet be a priority, whereas RUHA could serve as the basis for a more advanced exercise on finishings and their crucial effect on style, with Super Veloz being a faithful – if simplified – reproduction of letterpress-based modular illustration and an excellent tool for creative expression even beyond mere learning exercises, just like its conventional predecessor was designed for. These conclusions are self-evident from the analysis of these systems module by module, and the opposite exercise of working backwards from a set of curricular demands and needs and determining which elements, both structural – i.e. the design of modules themselves – and ancillary – the composition/design aids and any accompanying literature –, any new or extant stencil system needs to offer may be equally valid and, thus, benefit from the usage of such advanced taxonomy systems. Moreover, the latter systems can, indeed, be made known to end users – the students –, especially considering how both the stencil plates and the exercises to be conducted with them have the potential to make them also more self-evident and palpable than any theory lecture can. We feel confident that this methodology of analysis may decisively contribute to the process of creation of new stencil-based tools and/or of integration of extant ones into teaching toolkits, but also as a teaching tool in and of itself (even if in adapted form, perhaps in the form of short and informal interactions with students), and

Adapting Advanced Typographic Taxonomy Principles for Modular Stencil

853

shall publish our findings in parallel with or as part of the research project that motivated this paper in the first place. Acknowledgments. This work is financed by national funds through FCT – Fundação para a Ciência e a Tecnologia, I.P., under the Strategic Project with the references UIDB/04008/2020 and UIDP/04008/2020. The corresponding author’s current PhD research project, “Tipografia Modular: Aplicações Práticas Enquanto Ferramenta Pedagógica”, with the reference SFRH/BD/06696/2020, is financed by national funds through FCT – Fundação para a Ciência e a Tecnologia, I.P. The authors would like to acknowledge and thank CIAUD, Research Centre for Architecture, Urbanism and Design, Lisbon School of Architecture, Universidade de Lisboa and its staff for their kind support.

References 1. Barnbrook J (2004) The stroke of a pen. In: Heller S (ed) The education of a typographer. Allworth Press, New York, NY, pp 195–199 2. Johnston E (1971) Formal penmanship and other papers. Lund Humphries, London 3. Johnston E (1906) Writing & illuminating & lettering. Macmillan, New York 4. Noordzij G (2005) The stroke: theory of writing. Hyphen Press, London 5. Amado P, Silva AC, Napoleão E (2021) Kickstarting type design education with SLOType. In: Martins N, Brandão D (eds) Advances in design and digital communication: proceedings of the 4th international conference on design and digital communication, Digicom 2020, November 5–7, 2020, Barcelos, Portugal. Springer International Publishing, Cham, pp 191–201. https:// doi.org/10.1007/978-3-030-61671-7_18 6. Carvalho S, Bernau K (2008) Typography needs you: a modular type design workshop 7. Workshop de Caligrafia e Desenho Digital de Tipos da FAUL. https://www.fa.ulisboa. pt/index.php/pt/agenda/noticias-2/item/694-workshop-de-caligrafia-e-desenho-digital-detipos-da-fa. Accessed 28 Feb 2020 8. Gonçalves Â, Quelhas V (2021) TIMOS set: a modular speculation. In: Silveira Gomes A, Morgado A, Santos R (eds) Encontro de Tipografia – Livro de Resumos/Typography Meeting – Book of Abstracts. pp. 46–47. ATIPO – Associação de Tipografia de Portugal, Caldas da Rainha 9. Gonçalves  (2023) TIMOS – ESMAD. https://angelogoncalves.pt/timos-esmad. Accessed 20 April 2023 10. Pigford AJ (2020) Modular Wood Type. http://www.digfy.com/portfolio/items/modularwood-type/. Accessed 28 Feb 2020 11. Treacy T (2020) modular type workshop with ashley john pigford for graphic design students at vcu. https://pointedpress.tumblr.com/post/105195946705/modular-type-workshopwith-ashley-john-pigford-for. Accessed 20 April 2020 12. Gamonal-Arroyo R (2019) The construction of the letter: stencils as a creative and didactic tool. In: Moreira L, Sanches M (eds) 9th Typography Meeting/9.o Encontro de Tipografia. pp. 298–309. ATIPO – Associação de Tipografia de Portugal, Tomar 13. Meek R (2021) FontStruct (2008) – Online Creative Community. https://web.archive.org/ web/20201109213816/http://robmeek.com/project/fontstruct/. Accessed 8 Jan 2021 14. 2009 Interactive Annual – FontStruct. https://web.archive.org/web/20090523040744/http:// www.commarts.com/interactive/cai09/FontStruct.html. Accessed 8 Jan 2021

854

J. F. R. Gomes and J. A. Brandão

15. Meek R (2021) FontStruct | Build, Share, Download Fonts, https://fontstruct.com. Accessed 8 Jan 2021 16. The Q Project, an open-ended typographic play system. https://www.typotheque.com/blog/ the_q_project_an_open-ended_typographic_play_system. Accessed 9 Jan 2021 17. Q-Project (2021) Typoteque, Den Haag 18. Morgado ALM, Santos R, Dias RR, Dantas R (2020) Ruha Stencil : Tipos das Letras. http:// tiposdasletras.com/index.php/stencil/ruha-stencil/. Accessed 24 Jan 2020 19. Golec M (2005) Memory, instinct, and design: beyond Paul Rand’s “play principle.” In: Heller S (ed) The education of a graphic designer. Allworth Press, New York, pp 163–167 20. Bil’ak P (2020) The Importance of Play. https://www.typotheque.com/articles/the_import ance_of_play. Accessed 16 Dec 2020 21. Gruendler S (2020) It’s not how we teach typography, it’s how we learn typography. https://www.commarts.com/columns/it-s-not-how-we-teach-typography-it-s-how-welearn-typography. Accessed 15 Dec 2020 22. Gamonal R (2023) Plantilla Super Veloz. https://familiaplomez.bigcartel.com/product/planti lla-superveloz. Accessed 20 April 2023 23. Gray N (1986) A history of lettering. Phaidon, Oxford 24. Brandão J, Gomes J (2020) Modularity and grids in letterforms across calligraphy, conventional type and geometric type teaching. In: Rebelo F, Soares M (eds) Advances in ergonomics in design: proceedings of the AHFE 2020 virtual conference on ergonomics in design, July 16–20, 2020, USA. Springer International Publishing, Cham, pp 231–238. https://doi.org/10. 1007/978-3-030-51038-1_33 25. Frutiger A (1989) Signs and symbols: their design and meaning. Van Nostrand Reinhold, New York 26. Hofstadter DR (1995) Fluid concepts & creative analogies. Basic Books, New York 27. McGraw GEJr (1995) (Indiana U. of C.S.: Letter Spirit (part one): Emergent high-level perception of letters using fluid concepts. http://goosie.cogsci.indiana.edu/farg/mcgrawg/thesis. html 28. Cabral TO (2014) Tipos de sucesso. http://hdl.handle.net/10400.5/7190. 10.400.5/9545 29. Majoor M (2018) My Type Design Philosophy. https://www.typotheque.com/articles/my_ type_design_philosophy. Accessed 21 Feb 2018 30. Morison S (1994) Pacioli’s classic roman alphabet. Dover Publications, New York 31. Tory G (1529) Champ fleury, Au quel est contenu Lart et Science de la deue et vraye Proportion des Lettres Attiques, quon dit autrement Lettres Antiques, et vulgairement Lettres Romaines, proportionnees selon le Corps et Visage humain. Ce livre est privilegie pour dix ans par le Roy nostre sire et est a vendre a Paris sus Petit Pont a Lenseigne du Pot Casse par Maistre Geofroy Tory de Bourges Libraire et autheur du dict livre et par Giles Gourmont, aussi libraire, demourant en la rue…, Paris 32. Simonneau L (1716) Constructions des lettres. https://archive.org/details/constructionsdes00 simo 33. Albers J (1931) kombinationsschrift „3“. Bauhaus 1 Z. Für Gestalt. 4, 3–4 34. Albers J (2014) Combination Script “3.” In: Martinez de Guereñu L, Toledo M, Fontán del Junco M (eds) Josef Albers: minimal means, maximum effect. pp 215–216. Fundación Juan March, Madrid 35. Dürer A (1525) Underweysung der Messung, mit dem Zirckel und Richtscheyt, in Linien ebnen unnd gantzen Corporen. Hieronymus Andreae, Nürnberg. https://doi.org/10.3931/erara-9108 36. Schrofer JW (1987) Letters op Maat. Lecturis, Eindhoven 37. Huygen F (2014) Jurriaan Schrofer – graphic designer, pioneer of photobooks, art director, teacher, art manager. Environmental Artist. Valiz, Amsterdam

Adapting Advanced Typographic Taxonomy Principles for Modular Stencil

855

38. Huygen F (2015) Wim Crouwel Modernist. Lecturis, Eindhoven 39. Brandão J, dos Reis J, Duarte JG (2020) A critical overview of modular and geometric type design categorization in typographic taxonomy systems. In: Rebelo F, Soares M (eds) Advances in ergonomics in design: proceedings of the AHFE 2020 virtual conference on ergonomics in design, July 16–20, 2020, USA. Springer International Publishing, Cham, pp 239–246. https://doi.org/10.1007/978-3-030-51038-1_34 40. Kindel E (2006) Patents progress: the Adjustable Stencil. J Print Hist Soc 65–92 41. Kindel E (2003) Recollecting stencil letters. In: Stiff P (ed) Typography papers 5. The University of Reading, Reading, Department of Typography & Graphic Communication, pp 65–101 42. Kindel E (2007) The ‘Plaque Découpeé Universelle’: a geometric sanserif in 1870s Paris. In: Stiff P (ed) Typography papers 7. Hyphen Press, London, pp 71–80 43. Albers J (1926) Zur Ökonomie der Schriftform. Offset- Buch- Werbekunst. 3:395–397 44. Albers J (2014) On the economy of typeface. In: Martinez de Guereñu L, Toledo M, Fontán del Junco M (eds) Josef Albers: minimal means, maximum effect. p 209. Fundación Juan March, Madrid 45. Albers J (1926) Zur Schablonenschrift. Offset- Buch- Werbekunst. 3:397 46. Albers J (2014) On stencil typeface. In: Martinez de Guereñu L, Toledo M, Fontán del Junco M (eds) Josef Albers: minimal means, maximum effect. pp 210–211. Fundación Juan March, Madrid 47. Pool A (2010) FF DIN Round: digital block letters. https://fonter.files.wordpress.com/2010/ 09/ff_din_round.pdf 48. Slanted Publishers: Experimental Type Stencils. https://www.slanted.de/product/experimen tal-type-stencils/. Accessed 20 April 2023 49. Kindel E (2018) Eminents observed: a century of writing, lettering, type and typography at the Central School, London. In: Kindel E (ed) Typeform dialogues. Hyphen Press, London, pp 51–88 50. Dixon C (2018) Systematizing the platypus: a perspective on type design classification. In: Kindel E (ed) Typeform dialogues. Hyphen Press, London, pp 88–133 51. Gomes JFR, Brandão JA, Cabral TO, Rolo E (2022) The Role of Advanced Typographic Taxonomy Systems Vis-à-Vis Modular, Variable and Parametric Typography. Presented at the – The European Conference on Arts, Design and Education 2022 September 5. https:// doi.org/10.22492/issn.2758-0989.2022.19 52. Schwartz B (2005) The Paradox of Choice: Why More is Less

Embracing a Collaborative Practice with a Stakeholder: A Challenge Extended from the Academy to the Labour Market Carla Cadete(B) Lusófona University, Rua Augusto Rosa, 24, 4000-098 Porto, Portugal [email protected]

Abstract. This article focuses on collaborative practice 2022 between the academy, with a 3rd-year un-dergraduate student in Communication Design from Porto Lusófona University (UL), his De-sign professor and a publishing company – Book Cover. The author of the selected illustrations and student, Joel Maia, was asked to design illustrations for 14 book covers in the Jules Verne Collection. The covers were featured in two Portuguese newspapers, Jornal de Notícias in Porto and Diário de Notícias in Lisbon, over 14 weekends. Later, the covers were available in bookstores and at the Lisbon and Porto Book Fair. The article is divided into four main parts: introduction, literature review, collaborative practice methodology and final considerations. This pedagogical practice proved to be a valuable opportunity for several reasons, – financial support to the selected student and strong motivation for them to continue and complete their studies within a supervised and supportive framework; an opportunity to keep the student engaged and provide valuable skills for the job market, enhance his portfolio, offer an opportunity for recognition, visibility and foster creativity and innovation. Moreover, allows a close relationship between teacher, student and client, nurturing a collaborative learning environment like a design studio to learn through real-life projects. The publisher capitalized on the student’s availability within the specified timeframe and obtained 14 highly creative illustrations, significantly reducing their workload. This partnership proved to be a great example of a co-creation initiative that resulted in successful outcomes. Finally, it benefits everyone involved and demonstrates the potential for collaboration to equip students and promote participation in innovative practices by creating a dynamic, vibrant, and enriching learning environment. Keywords: Collaborative practice · Communication Design Bachelor · Book Cover Publisher · Jules Verne Collection · Editorial Design · Cover Design

1 Introduction The article highlights a successful partnership between the academy and a publisher. This kind of initiative can significantly improve cooperation between educators and higher education students, increasing the confidence and motivation of all stakeholders © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 856–872, 2024. https://doi.org/10.1007/978-3-031-47281-7_69

Embracing a Collaborative Practice with a Stakeholder: A Challenge

857

to implement more collaborative practices in academia. At the Communication Design Bachelor program, we actively promote collaborative practices, including guest lectures, workshops, and studio tours, as additional avenues to enhance our curriculum and contribute to our student’s knowledge, skills, and overall success. Our course is designed as a collaborative environment where students, tutors, and practitioners can work together to explore and analyse topics. We aim to equip our students with the necessary tools to succeed in the academy and their future careers through these initiatives. This project represents our commitment to preparing our students for the real world. Also, collaborating with stakeholders exemplifies a different challenge from typical coursework, instilling a sense of purpose and urgency beyond simply achieving good grades. “The future will present students with novel problems and opportunities. Awareness that the world will continue to change can be built into curricula and pedagogy by intentionally cultivating learners’ capacities for problem-recognition and problem-solving.” (UNESCO 2021, p. 51). This pedagogical practice was a collaborative project between the academy, a 3rd-year undergraduate student in Communication Design from Porto Lusófona University (UL) and a publishing company – Book Cover. In 2022, the invited student, Joel Maia, created 14 book cover illustrations for Jules Verne Collection. The books were initially featured in two Portuguese daily newspapers, – Jornal de Notícias (Porto) and Diário de Notícias (Lisbon), from September 3rd to December 3rd, 2022 and later in national bookstores and book fairs (from May 2023). Book Cover is a small publishing company founded in September 2017 in Porto, Portugal. The company has two exceptional collections: Essentials of Portuguese Literature and Foreign Literature. With a catalogue boasting over 150 publications, Book Cover is committed to publishing influential books while supporting emerging Portuguese and international authors. Over 14 weeks, Joel Maia (the student) was tasked with showcasing skills in managing pressure, being responsible, and employing a methodical approach to the project. This design project required extensive research and analysis of the market, the content of the books and the pursuit of rigorous deadlines. This was because newspapers and books were spread nationally every Saturday for 14 weeks. The project provided the student with an engaging experience and equipped him with valuable skills for the job market. It also offered a chance for recognition and payment, added value to his portfolio and included illustration credits in published books. Tragically, some students were compelled to abandon their educational pursuits entirely due to a lack of financial means. Students hailing from economically disadvantaged regions in the northern interior of the country were particularly susceptible to financial difficulties owing to limited financial support and elevated unemployment rates. The publisher proposed collaborating with this class in response to this urgent matter. Many students in this course were grappling with these very financial challenges—the proposal aimed to provide financial assistance to these students. Moreover, allows a close relationship between teacher, student and publisher, fostering a collaborative learning environment like a design studio to learn through real-life projects. Preparing students for the job market can be challenging, especially when handling stakeholder relationships. The initiative turned out to be a success both academically and editorially, making it a great achievement.

858

C. Cadete

2 Literature Review In today’s world, individuals face numerous challenges, particularly in the job market. Therefore, we must equip our students with the necessary skills to respond effectively to these challenges. The first cycle of studies in Communication Design should focus on comprehensive training, providing students with the ability to respond to the multiple areas in which Communication Design currently operates: identity, editorial, illustration, packaging, infographics, typography and multimedia. We focus on developing transversal skills and interdisciplinary projects across multiple areas of knowledge, emphasising participatory projects through cooperation with stakeholders, teamwork models and participation in international competitions. With the globalisation, the role of higher education is changing. Universities must equip their students with diverse skills and capacities to succeed in today’s complex society (Chan 2016). Universities must develop and share knowledge and skills and shape a forwardthinking perspective to benefit society (Tromp & Hekkert 2021). Students in higher education should be equipped for life beyond the classroom. According to the UNESCO Report 2021, “Young people need strong support upon educational completion to be integrated into labour markets and contribute to their communities and societies according to their potential.” (2021, p. 42). It is urgent to improve incentives and support for effective teaching. Margolin argues, “If the social projects be involved in the pedagogical objectives, students eventually believe that they would be able to carry out work inside and outside the classroom.” (2014, p. 68). The academy should facilitate dialogue and collaboration between local and global companies and industries to better prepare students for the job market. Another report by the UNESCO International Institute for Higher Education in Latin America and the Caribbean (IESALC), based on the views of 25 global higher education experts who were invited to participate, “Collaboration between stakeholders would be essential: the flexible movement of students and lecturers is key to opening up education as well as supporting professional collaboration.” (UNESCO IESALC 2021, p. 24). The experts consulted also agreed on the fundamental message that the strength of higher education lies in diversity. For this, it should employ plural ways of knowing and doing. (UNESCO IESALC 2021). Following a communication on European Strategy for Universities from the European Commission, “There is a need to tackle in a systemic and comprehensive way academic careers, responding to the call of the Council to promote flexible and attractive career structures and improve working conditions.” (European Commission 2022, p. 12). In addition, the same report concluded, “Cooperation among universities and with the industrial ecosystems is mutually beneficial in this respect, with the higher education sector supporting skills development for industry and the business sector, in addition to personal development purposes.” (European Commission 2022, p. 13). As stated in Shaping Inclusive and Responsive University Strategies (SIRUS), cofunded in 1996 by the European Commission’s Lifelong Learning Programme, “Time and money are usually of the essence for the non-traditional students. Therefore, universities need to engage with local employers and identify – in cooperation with stakeholders – how higher education can support and up-grade the knowledge

Embracing a Collaborative Practice with a Stakeholder: A Challenge

859

and skills of the regional workforce for the mutual benefit of the student, the university and the employer. Ideally, such cooperation with stakeholders is a central part of the institutional strategic orientation.” (Smidt & Sursock 2011, p. 23). Improving the education system to be more responsive to the needs of the labor market is crucial to support the employability of graduates “(…) one of the key purposes of introducing the three-cycle system across Europe was to develop first cycle qualifications that the labour market would accept.” (Sursock & Smidt 2010, pp. 39–40). A Higher Education Stakeholder Forum organised by the Directorate for Education and Skills that supports the OECD’s new work on higher education acknowledges, “The in-depth analysis of labour market relevance and outcomes of higher education systems project recognises that one of the main objectives of higher education is to provide graduates with the skills needed to succeed in the labour market.” (OCDE 2017, p. 15). In this process, all stages of education have an essential role to play, teachers, students and employees. However, the student must always be the focus, “Studentcentred learning refers to pedagogies focused on the learner and what is learned, rather than on the teacher and what is taught (Sursock, Smidt and Korhonen 2011). In other words, the learning process is not just about transferring and reproducing knowledge but about deeper understanding and critical thinking (Sursock, Smidt and Korhonen 2011). Furthermore, “The learners ‘construct’ their meaning by proactive, independent learning, discovery and reflection. Assessment is generally formative and feedback continuous.” (Smidt & Sursock 2011, p. 16). Universities must review their programmes to equip students with innovative skills by integrating learning across disciplines and co-working with other courses and stakeholders, “Learning and innovation skills are increasingly being recognised as the distinguishing factors that separate students who are prepared for the complex life and work environments of the 21st century from those who are not.” (Mansilla & Gardner 2007). Scholars should be aware that besides the first steps are starting to be taken in this direction and the subject is starting to be talked about in universities, academics need to acknowledge that while progress is being made and discussions are being had in universities about this matter, there is still much work to be done in terms of reforming course programs and in classrooms, “However, there is still much work to be done, and a great opportunity for a new generation of young and committed designers to complete the transition (…)” (Coleman, Clarkson, Dong & Cassim 2007, p. 8). The European Union launched the Europe recovery plan (European Commission 2020), in which measures to revitalise and support organisations will be implemented. In this regard, design plays a crucial role in adapting to new requirements and must provide comprehensive solutions that promote innovation, employment, and value creation for all stakeholders. Steen (2012) argues that people who participate in participatory design projects must develop the virtues of cooperation, curiosity, creativity, empowerment and reflexivity to become participatory design virtuosos. Steen (2012) provides valuable insights into the virtues individuals participating in participatory design (PD) projects should cultivate to excel in this collaborative process. These virtues contribute to becoming PD virtuosos, who can effectively engage in the design process and create meaningful outcomes. These virtues are cooperation, curiosity, creativity, empowerment, and reflexivity. Cooperation is essential for successful PD,

860

C. Cadete

as it fosters a collaborative and inclusive environment where diverse perspectives and contributions are valued, enriching the design process. Curiosity plays a significant role, driving participants to empathize with others’ experiences and learn jointly. By understanding the needs and challenges of users and stakeholders, they gain deeper insights and create more meaningful solutions. Creativity empowers PD to envision and materialize innovative ideas. PD virtuosos are unafraid to experiment and take risks, leading to the co-creation of novel products and services that address identified needs. Empowerment involves sharing power and fostering an environment where everyone’s voice is respected and valued, fostering commitment to the design process. Reflexivity is the ability to self-reflect and adapt during the design process by continuously evaluating their thoughts, feelings, and actions, embracing feedback and remaining open to new insights. When participants embody these virtues, they can work together to create innovative and meaningful solutions that meet the needs and desires of users. For all that has been pointed out, it is crucial to prepare students for the job market by establishing solid relationships with stakeholders and integrating real-world projects into the curricula. In the Design field, creating new briefings with real projects, guest lectures, workshops, seminars, and study visits should be offered as supplementary tools to enhance the knowledge taught in the classroom. Course programs should also be reviewed and adapted accordingly. Collaborating with stakeholders and engaging in participatory projects can be a significant challenge for the academy when it comes to enhancing students’ skills for the job market, “We, therefore, also would like to encourage both sides – representatives of business and academia – to deal with the current uncertainty in interdisciplinary collaboration by intensifying the dialogue between research and practice.” (Augsten & Gekeler 2017, p. 1068).

3 Collaborative Practice On the publisher’s first meeting, the student was provided with a detailed explanation regarding the books and the market in which they would be sold. Considering the book’s complexity and the three distinct environments where the books would be sold is crucial. The first is the paper shop, where the book would be sold along with the newspaper to an older audience. This environment has two key sales components: low prices and book collection (some people are passionate about collecting, especially when the covers are visually appealing). The second environment is bookstores, and the third is book fairs, where the book is sold in a space dedicated to books and faces more competition from peers regarding themes, design covers and price. As a result, the graphic image of the book cover and its cost are constantly compared. Ensuring the book’s cover stands out in both environments is crucial. Over 14 weeks, Joel Maia (3rd-year undergraduate student) was tasked with showcasing skills in managing pressure, being responsible, and employing a methodical approach to the project. This design project required extensive research and analysis of the market, the content of the books and the pursuit of rigorous deadlines. This was because newspapers and books were distributed nationally every Saturday for 14 weeks. Concerned with the diverse audience, limited production time, and low-price book, it is essential to create illustrations with broad appeal and a simple but attractive design

Embracing a Collaborative Practice with a Stakeholder: A Challenge

861

and avoid expensive and time-consuming techniques like varnishing and using special papers or cuts. Instead, focus on creating visually engaging illustrations that can be easily reproduced without compromising the quality of the final product. Doing so allows one to appeal to a broader audience and keep production costs low while delivering a high-quality book. 3.1 Briefing and Context The cliente is Jornal de Notícias (Porto), Diário de Notícias (Lisbon) and bookstores. The collection target is more than 8 years. The briefing asked for designing covers for 14 Classic Jules Verne Collection titles—a collaborative project between the Design degree at Lusófona University and Book Cover publishing. Book size 140 × 210 mm. Design author (teacher and author of this article), the author of the selected illustrations, and student Joel Maia. A weekly edition in two Portuguese daily newspapers – Jornal de Notícias, Diário de Notícias and the bookstores. The 14 titles published were: A Journey to the Center of the Earth, Around the World in 80 Days, Michael Strogoff, The English at the North Pole, Desert of the Sea, Five Weeks in a Balloon, North Against South, Doctor Ox, Adventures Of three Englishmen and three Russians, The South Star, The Underground City, The Floating City, From the Earth to Moon and 20.000 Leagues Under the Seas. While speaking with the editor about the financial difficulties of many students and their families. The challenges included difficulties in paying for tuition, transportation, food, and necessary educational materials. Consequently, many students were forced to work on weekends or balance their studies with part-time employment. Tragically, some students had no choice but to abandon their educational pursuits due to a lack of financial resources. Students from economically disadvantaged regions in the interior north of the country faced a higher prevalence of financial difficulties due to limited financial support and higher unemployment rates. Moreover, in some cases, the student’s family members, who pay their fees, died, leaving them vulnerable. In response to this pressing issue, the publisher proposed a partnership with the Communication Design course at Lusófona University. Many students in this class are facing this particular difficulty. The idea was to offer financial aid to deserving students by engaging them in illustrating the Júlio Verne collection. A group of senior students was selected for this project, and they received guidance and support from both their teacher and the editor. The task assigned to them was to create illustrations for the title “Journey to the Center of the Earth” within a strict time frame of 6 h, spread across two classes. Many students simplify their projects by using typography or patterns, due to time constraints or difficulty with drawing (Fig. 1). The submissions were then evaluated by the editor, the marketing director of Jornal de Notícias, and the professor, with the best representation being chosen. This particular initiative not only provided much-needed financial assistance to the selected student, but it also served as a powerful motivation for them to continue and complete their studies. The student’s proposals were evaluated based on creativity, applicability to the 14th titles, visual impact, readability, and target audience. By offering such opportunities, the publisher and the university demonstrated their commitment to supporting needy students and empowering them to overcome financial obstacles on their educational journey. Once the best proposal was chosen, the student

862

C. Cadete

selected proceeded to share additional illustrations and projects to ensure the quality of the work. The collaboration between the Design Professor (author of this article) and the publisher existed for several years as the author of the logo and other design cover collections for the same publisher. Due to the project’s high level of responsibility, tight deadlines, and the student’s lack of experience, a professional designer must work closely with him to ensure successful co-creation. As a result, a team comprised professionals from two national daily newspapers, the publisher, and the academy (professor and student). This proposal easily fits into the curricular unit since the programmatic objectives focus on editorial projects and illustration. This partnership between the academy and publisher was a unique opportunity, resulting from specific circumstances, and is not expected to be repeated. As previously discussed, ethics and deontology play a vital role in this case. Since design is typically an exclusive profession for qualified professionals, universities must ensure that students refrain from competing directly with professionals but gain educational experiences. The central aim of involving students in this collaborative project was to offer them a valuable educational opportunity. This specific initiative served a dual purpose: financial support to the selected student and strongly motivated them to continue and complete their studies within a supervised and supportive framework. To uphold ethical and moral standards, it was essential to balance providing students with hands-on learning experiences and ensuring that the project did not compromise their education or create an unfair competitive environment vis-à-vis seasoned professionals. Consequently, transparency, well-defined guidelines, and an unwavering focus on educational goals were essential guiding principles throughout the collaborations, all while safeguarding the integrity of the design profession. The project served as a learning opportunity rather than a professional competition. Full disclosure of any pre-existing relationships, such as the one between the Design Professor and the publisher, was observed. The role of the professor was primarily that of a mentor and educator dedicated to steering the students towards skill development and learning. At no point did the professor engage in actions that favoured the interests of the student or the professionals at the expense of the educational objectives.

Fig. 1. Due to time constraints or difficulty with drawing, many students simplify their projects using typography or patterns.

Embracing a Collaborative Practice with a Stakeholder: A Challenge

863

The project rigorously adhered to the professional standards and ethical codes pertinent to the design industry. This encompassed a steadfast commitment to respecting intellectual property rights, maintaining client integrity, and upholding the principles of professional integrity. 3.2 Collaborative Value At the Communication Design Bachelor program, we actively promote collaborative practices, including guest lectures, workshops, and studio tours, as additional avenues to enhance our curriculum and contribute to our student’s knowledge, skills, and overall success. This particular participatory project served as a means to: • Enhance Engagement: By involving students in real-life projects and collaborative initiatives, we fostered a higher level of engagement. This hands-on experience allowed them to apply their skills and knowledge, increasing their commitment to their education; • Develop Marketable Skills: Collaborative practices provide students with valuable skills that are highly sought after in the job market. By working closely with professionals and clients, students gained practical experience and developed competencies for their future careers; • Expand Professional Networks: Through collaborations with external partners, such as the publishing company in this case, students had the opportunity to connect and network with professionals in their field. Building these relationships can open doors to future internships, job opportunities, and mentorship, ultimately enhancing their professional development; • Foster Creativity and Innovation: Collaborative practices often involve interdisciplinary teamwork and the exchange of ideas. This environment stimulates creativity and encourages innovative thinking as students learn from each other’s perspectives and explore new approaches to problem-solving; • Provide Recognition and Visibility: By having their work featured in newspapers, bookstores, and book fairs, students gained recognition and exposure for their talent and skills. This recognition boosts their confidence and helps establish their reputation within the industry; • Overall, promoting collaborative practices within our program creates a rich learning environment that prepares students for the realities of the professional world while nurturing their creativity, skills, and professional networks; • In addition to the illustrations, Joel was consistently informed of the decision-making process and closely involved in the Design process and Pre-Press stages, allowing him to understand the project as a whole comprehensively; • This partnership proved to be mutually beneficial. The publisher capitalized on the student’s availability within the specified timeframe and obtained 14 highly creative illustrations, significantly reducing their workload. Meanwhile, the student had the opportunity to collaborate with a publisher, seeing their work printed and sold in paper shops alongside two national newspapers and bookstores nationwide. However, the professor and editor had to guide to ensure that the project was delivered on time and met the required standards, given the student’s relative lack of experience;

864

C. Cadete

• As a result of this collaboration, the student received illustration credits in the published books, with their name printed on 14 different collection editions. Their achievements serve as a role model and inspiration for their peers; • The book collection was extensively publicised through weekly advertisements on the front and interior pages of both newspapers and on their respective websites and Instagram accounts (Figs. 2 and 3); • The success of this collaborative practice is evident in the significant number of books sold and the positive feedback received from readers, who shared their enthusiasm at the Lisbon Book Fair and on social media platforms, including Book Cover’s Instagram. Readers and enthusiasts praised the originality of the book covers. Furthermore, the book collection was featured on a Portuguese TV program on Sic Notícias in Programa cujo o nome estamos legalmente impedidos de dizer, having highlighted the cover’s originality and the low price, providing access to all, hosted by journalist Carlos Vaz Marques, on Saturday, October 31, 2022, garnering attention for its originality and affordable pricing, making it accessible to all (Fig. 4); • The publisher has reported that readers expressing enthusiasm for the books on Instagram have greatly appreciated the graphic image collection. Booksellers at the Lisbon and Porto Book Fair have echoed the same sentiment, and the FNAC store’s management highlighted the complete Jules Verne collection during May and June 2023 at their checkouts (Figs. 5 and 6).

Fig. 2. Every week, advertisements promoting the book collection were featured on both newspapers’ front and interior pages. Also on the newspaper’s website and Instagram. Font: Images from the author.

Fig. 3. The book collection was advertised weekly on the newspaper’s website, as well as on both its social media pages and Instagram. Font: Images from the author.

Embracing a Collaborative Practice with a Stakeholder: A Challenge

865

Fig. 4. The book collection was showcased on a Portuguese TV program – Sic Notícias, by Carlos Vaz Marques journalist, on Saturday, October 31, 2022, in Programa cujo o nome estamos legalmente impedidos de dizer. Font: Images from the author.

3.3 Methodology The Design Thinking methodology was used to guide the creative process in the following steps: problem definition, ideation, prototype, and implementation (Brown 2005, 2009; Ideo 2015; Lupton & Philipps 2011; Tschimmel 2011; Wyatt & Brown 2010). Lusófona Design students commonly use this methodology in their projects, so Joel found it easy to apply it to his project. Problem Definition: During the initial meeting, the publisher and the teacher discussed the importance of compromising on creating 14 illustrations for 14 book covers that must be consistent throughout the campaign. It was emphasised that weekly distribution should be done with fidelity and rigour to avoid any risks to the project, the publisher, and the client (Jornal de Notícias and Diário de Notícias). Additionally, it was analysed and discussed some other edited covers of the Jules Verne collection, considering both positive and negative attributes. Factors such as legibility, creativity, alignment with the collection’s identity, and unity with the content determine whether they add value to the narrative.

Fig. 5. Porto Book Fair, Jules Verne Collection, on September 4, 2023 (left side). Readers expressed enthusiasm for the book collection on Instagram after visiting the Lisbon Book Fair, on Mai 25, 2023 (right side). Font: Images from the author.

866

C. Cadete

Fig. 6. On May 25, 2023, the Jules Verne collection was displayed at Bertrand and Porto Fnac bookstores (from left to right). Font: Images from the author.

The main objective of the editor was to create a contemporary collection that would stand out from others, focusing on simplicity while ensuring visual appeal. The core team for this project consisted of the publisher, the professor, and the student responsible for creating the illustrations. The project followed a strict timeline, with the goal of producing one illustration per week for a continuous period of 14 weeks. Each team member had specific commitments weekly. The publisher was responsible for providing a synopsis of the title every week and actively participating in meetings in person or through virtual platforms like Zoom to monitor and guide the work progress. Ideation and Prototype: The professor oversaw the book design, including the cover and interior layout, and continuously guided the student’s work. Additionally, the professor sometimes acted as a bridge between the publisher and the student, facilitating communication when the publisher’s availability was limited. This collaborative team approach ensured effective coordination and adherence to the project timeline. By fulfilling their respective roles and maintaining regular communication, the team successfully executed the project, achieving the editor’s vision of a contemporary collection that is distinct, visually appealing, and different from other publications. When designing the book cover, it was important to prioritise simplicity to emphasize the illustration. The title and author name, always in white, could change position according to the illustration shape while maintaining the collection’s identity. For the inside book, it was crucial to consider the intended age group (up to eight years old) and ensure that the typography is easily readable, taking into account form, scale, and leading, as both children and older people may read it easier—Adobe Garamond Pro Regular, size 11.5pt, leading 15,5pt. The choice of Offset Cream paper for the interior is deliberate, as its non-white colour reduces the contrast with black text, resulting in a less visually taxing reading experience. To begin his research, the student analyzed Jules Verne’s covers edited by other publishers and read their synopses. He also created a list of keywords, developed a mind map and created a mood board for each title (Fig. 7).

Embracing a Collaborative Practice with a Stakeholder: A Challenge

867

An exercise that starts with a divergent process to create a large range of options, “If the convergent phase of problem-solving is what drives us toward solutions, the objective of divergent thinking is to multiply options to create choices.” (Brown 2009, p. 67) Joel developed the creative process and ideas with several drawings in the sketchbook. Brainstorming is an exercise that starts with a divergent approach to creating an extensive range of options. A practice that generates more ideas. According to Brown, “When I use drawing to express an idea, I get different results than if I try to express it with words, and I usually get to them more quickly.” (Brown 2009, p. 81). These adventure novels took place during the latter part of the 19th century; Joel needs to research this period to ensure the correct drawing of a train, a boat, a balloon, a tzar suit, etc. Also, strategic questions were created to evaluate each title’s ability to respond to the narrative, creativity, and appeal while considering if the illustrations enhance the book content. To further develop his ideas, he sketched out one or more concepts and met weekly, either face-to-face or via Zoom, to present his work and receive guidance. Since the editor’s main idea was to create a contemporary collection that must be simple but visually appealing, it was decided by the team that one of the design tools to be explored should be colour. To align with the unique narratives of each edition, a palette comprising 14 solid and flat colours was meticulously curated. The selection process considered the specific stories featured in each collection edition. The illustration technique should be easy since the available time was reduced. At this stage, the student made some proposals with different techniques and materials; however, after analysing all of them, the digital technique with simple shapes and flat colours was chosen, as it responds more effectively to the objectives and the available time (Fig. 8).

Fig. 7. Mind map and mood board for the first edition (from left to right). Font: Images from the author.

Implementation (Problem-Solution): After the professor, publisher, and newspaper’s marketing team approve an illustration, it is delivered to the professor for the layout and graphic composition. Cover typography should be simple and easy to read, with adaptable spacing that complements each illustration better. The focus of the design was to create contrast and draw attention first to the image and then to the title. Just one object should be represented. To create a harmonious contrast and balance, Joel highlighted the colours and shapes of the image. The colours were studied according to the theory of Johannes Itten (1974) of the seven

868

C. Cadete

Fig. 8. Joel proposed different illustration techniques. The last one, on the right side, using a digital process, was selected. Font: Images from the student.

contrasts that generate harmony: Contrast of hue, saturation, light and dark, warm and cool, Complementary Contrast, Simultaneous contrast and Contrast of extension. This theory is taught to students in the first year of their degree program. The covers were printed with five colours to achieve a solid and vibrant background. A Pantone colour was selected for the background, while the illustration was printed using four colours (CMYK). In addition to the illustration, the student, at all stages of the project, was always informed of the decisions and accompanied the Design process and the Pre-Press to have a global understanding of the project. The covers were printed in 240 gr. Cardboard, with 70 mm large flaps. The author’s biography was printed on the first flap, while the second flap listed all the titles in the collection (Fig. 9). The publisher and the newspaper´s marketing decided that the first 13 issues of the collection would be printed in softcover and the last in hardcover, as this last work would be published in December—a premium book intended for a Christmas gift. For the value of this book to be economical, the endpapers needed to be false – the first and last pages of the inside were glued directly to the cover. The publisher and marketing team of the newspapers agreed that the first 13 collection issues would be printed on softcover. In contrast, the final issue, 20,000 Leagues Under the Sea, would be printed on hardcover. This decision was made because the book would be released in early December and is of high quality, making it a suitable Christmas gift (Fig. 10). The cover for A Journey to the Center of the Earth was intentionally designed to highlight a creature from the ocean’s depths rather than the story’s main characters - a German scientist, his nephew, and his goddaughter habitually featured in other covers. The animal was depicted using simple geometric shapes and contrasting colors, such as green water and small red triangles, creating a dynamic effect on the cover. This created a dynamic effect on the cover, while the title in white stood out on the left side, striking a balance between dynamic and static, light and dark, and warm and cool elements. For Michael Strogoff , the cover’s color scheme emphasised a hue contrast, featuring the vibrant red of the czar coat against the background, creating a harmonious visual effect. In The English at the North Pole and Desert of the Sea, Jules Verne’s character, Captain Hatteras, known for his experience in the Poles, was deliberately avoided to differentiate from previous covers. Instead, a mountain of ice became the central element, with different objects atop the mountain - the English flag in one cover and a small polar

Embracing a Collaborative Practice with a Stakeholder: A Challenge

869

bear in the other. The analogous color scheme of white and various shades of blue united the two covers, and the use of light and dark contrast with the hue. The proposed cover for Five Weeks in a Balloon was approved quickly. It features a red balloon set against a warm orange background, creating a hue contrast. The blue background complements the red balloon, providing a warm and cool contrast. The crop position at the top of the cover adds to the dynamic effect. In Dr Ox, the illustration of the evil character is portrayed in an austere manner, donning dark glasses to symbolise the surrounding mystery in the narrative. The warm orange background contrasted with the cold blue of the character’s suit, achieving harmony on the cover. Regarding Adventures of Three Englishmen and Three Russians, the previous covers predominantly featured human figures, leading to the decision to avoid this approach. Instead, the shield and spears of African warriors were chosen, representing the dangers faced by the international commission in the desert, jungles, and African savannas.

Fig. 9. Pre-Press – flattened cover for printing, final Pdf file. Font: Image from the author.

Fig. 10. The collection’s first 13 editions were printed in softcover, while the last issue, 20,000 Leagues Under the Sea, was printed in hardcover. Font: Images from the author.

870

C. Cadete

In The Floating City, the initial design with a side view of the boat lacked dynamism. The second version successfully achieved the desired dynamism by featuring a bottomup perspective, a right-side cut, and transparent, overlapping smoke stains from the boat’s chimney. The intense red color reflected the narrative of a cursed ship and forbidden love, using the hue contrast of the red color. In regards to 20,000 Leagues Under the Seas, the emphasis was placed on the Nautilus submarine rather than typical underwater scenes with divers and sea creatures. However, there was a problem with the initial design, as the submarine was mistakenly depicted in the air instead of being submerged. To fix this, a transparent stain was added to show its submersion in the ocean, resulting in the final cover design. The book covers for various Jules Verne novels feature different illustrations depending on the title. For example, Around the World in 80 Days has an illustration on the left side, while Michael Strogoff has it on the right. Five Weeks in a Balloon has it on the top, and Dr Ox, The English at the North Pole, Desert of the Sea, and North Against South have it on the bottom. A Journey to the Center of the Earth has it in the middle, and the author. In all covers the title and author are adjusted accordingly. This creates a dynamic collection that is still coherent in its design. When exposed, the 14 titles easily stand out from all the other due to the simple shapes and contrasting but harmonious colors in the set. Final Consideration As highlighted in the article, the partnership between the academy and the publisher shows the transformative potential of collaboration between the academy and the publisher in equipping students and fostering their engagement in innovative practices. This collaborative project has provided students with valuable practical experience, allowing them to refine their skills and develop highly relevant competencies for their future careers and providing financial support to the selected student. It strongly motivated them to continue and complete their studies within a supervised and supportive framework. At the Communication Design Bachelor program, we actively promote collaborative practices, including guest lectures, workshops, and studio tours, as additional avenues to enhance students’ curriculum and contribute to their knowledge and skills, preparing them for the job market and overall success. By engaging in participatory projects like this, we aim to improve student engagement, cultivate marketable skills, expand professional networks, foster creativity and innovation, and provide recognition and visibility. Our course is a collaborative space where students, tutors, and practitioners can work, explore, and analyze together. Collaborating with stakeholders presents a unique challenge beyond typical coursework, as it instils a sense of purpose and urgency beyond achieving good grades. This co-creation work has demonstrated mutual benefits, significantly improving cooperation between higher education institutions and industry partners. It boosts confidence and motivation to implement more collaborative practices in the academy. The initiative has created a dynamic, vibrant, and enriching learning environment, resulting in successful outcomes. This successful collaborative project at the graphic level has generated positive results regarding reader reception and sales. It has proven beneficial not only for the students involved but also for the editor and the newspapers. The project’s

Embracing a Collaborative Practice with a Stakeholder: A Challenge

871

visually appealing design and cohesive approach have resonated with readers, increasing their engagement and interest in the books. This, in turn, has positively impacted sales and the overall reception of the publications. The collaborative effort has created a win-win situation for all parties involved, showcasing the power and effectiveness of such partnerships in graphic design. Finally, by incorporating and expanding the number of collaborative projects with partners, Design course programs effectively bridge the gap between the academy and the industry. This integration fosters a stronger connection between theory and practice, preparing students for the real-world challenges and expectations they will encounter in their professional careers. Acknowledgements. This work was funded by Fundação para a Ciência e Tecnologia (FCT) under HEI-Lab R&D Unit (UIDB/05380/2020).

Conflict of Interest. The author declares that there is no conflict of interest.

References Augsten A, Gekeler M (2017) From a master of crafts to a facilitator of innovation. How the increasing importance of creative collaboration requires new ways of teaching design. Design J 20(sup1):S1058–S1071. https://doi.org/10.1080/14606925.2017.1353049 Brown T (2005) Strategy by design. In: Fast company. special issue masters of design, pp 2–4 Brown T (2009) Change by design: how design thinking transforms organisations and inspires innovation. Harper Collins, New York Chan RY (2016) Understanding the purpose of higher education: An analysis of the economic and social benefits for completing a college degree. J Educ Policy Plan Admin 6(5):1–40 Coleman R, Clarkson J, Dong H, Cassim J (2007) Design for inclusivity – a practical guide to accessible, innovative and user-centred design. Routledge, England European Commission. Europe’s moment: Repair and Prepare for the Next Generation. https:// ec.europa.eu/commission/presscorner/detail/en/ip_20_940. Accessed 3 May 2023 European Commission. Communication on European Strategy for Universities. https://educat ion.ec.europa.eu/sites/default/files/2022-01/communication-european-strategy-for-universit ies-graphic-version.pdf. Accessed 3 May 2023 Ideo: The Field Guide to Human-Centered Design. 1st edn (2015). ISBN: 978-0-9914063-1-9, http://www.designkit.org/. Accessed 21 May 2023 Itten J (1974) The art of color. Wiley, New York Lupton, E., Phillips, J.C.: Graphic Design Thinking: Beyond Brainstorming. Princeton Architectural Press (2011) Margolin V (2014) Design e Risco de Mudança. ESAD de Matosinhos. Vila do Conde: Verso da História (2014) Mansilla VB, Gardner H (2007) From teaching globalization to nurturing global consciousness. In: Suarez-Orozco MM (ed) Learning in the global erainternational perspectives on globalization and education. University of California Press, pp 46–66. https://doi.org/10.1525/california/978 0520254343.003.0002

872

C. Cadete

OECD. Higher Education Stakeholder Dialogue Enhancing Higher Education System Performance. Meeting of the Informal Working Group on Higher Education, 4–5 December, OECD Conference Centre, Paris. https://one.oecd.org/document/EDU/EDPC(2017)45/en/pdf. Accessed 3 May 2023 Smidt H, Sursock A (2011) Engaging in Lifelong Learning: Shaping Inclusive and Responsive University Strategies Steen M (2012) Virtues in participatory design: cooperation, curiosity, creativity, empowerment and reflexivity. Sci. Eng. Ethics 19. https://doi.org/10.1007/s11948-012-9380-9 Sursock A, Smidt H (2010) Commission, European. Trends 2010: A Decade of Change in European Higher Education. https://eua.eu/downloads/publications/trends%202010%20a%20decade% 20of%20change%20in%20european%20higher%20education.pdf. Accessed 4 May 2023 Tschimmel K (2011) Processos Criativos. A emergência de ideias na perspectiva sistémica da criatividade. ESAD, Matosinhos Tromp N, Hekkert P (2021) Designing for society – products and services for a better world. Bloomsbury, New York UNESCO. Reimagining our futures together – A new social education. Report from the International Commission of the Futures of Education, Paris. ISBN 978-92-3-100478-0. https://une sdoc.unesco.org/ark:/48223/pf0000379707. Accessed 3 May 2023 UNESCO. Thinking Higher and Beyond: Perspectives on the futures of higher education to 2050. Report from the International Institute for Higher Education in Latin America and the Caribbean (IESALC). ISBN 978-980-7175-57-9. https://www.iesalc.unesco.org/eng/wpcontent/uploads/2021/05/Thinking-Higher-and-Beyond_EN-_Format_FINAL.pdf. Accessed 3 May 2023 Wyatt J, Brown T. Design Thinking for Social Innovation. Stanford Social Innovation Review, Graduate School of Business. https://myweb.uiowa.edu/dlgould/plugin/documents/Design_ Thinking_for_Social_Innovation.pdf

Defining the Emotional Design Stages of Package Designs by Analyzing the Melatonin Gummy Candy Packaging Xie Zeyuan1 and Albert Young Choi2(B) 1 Graduate School, College of Design, Hanyang University ERICA, Ansan 15588, South Korea 2 Faculty of Graduate School, Communication Design, College of Design,

Hanyang University ERICA, Ansan 15588, South Korea [email protected]

Abstract. This study employs a research approach that combines qualitative and quantitative methods, utilizing the three dimensions of emotional design as analytical tools to analyzes the packaging design of melatonin gummies from ten prominent brands available in the United States, Australia, and Brazil. Subsequently, delves into the exploration section, aiming to comprehend, evaluate, and quantify the dimensions of design. Specific recommendations are provided for various aspects of melatonin gummy packaging design, including color schemes, element selection, and opening mechanisms. The aim of this paper is to offer novel design foundations and an evaluation framework for melatonin gummy packaging. Based on the analysis and research findings, it identifies patterns in the graphical and color elements of current melatonin gummy packaging designs. The analysis illustrates how brands leverage emotional design across three dimensions to evoke associations with sleep, providing a suitable emotional experience. This paper aims to underscore the empathy embedded in design and guide the direction of future iterations of melatonin gummy packaging. Keywords: The Emotional Design · Multi-level · The Constraint Design

1 Introduction 1.1 Research Background The prevalence of sleep disorders is increasing among modern individuals due to the impact of changing environmental factors and the multifaceted pressures of daily life. These disorders are typically characterized by recurrent and chronic symptoms, including difficulty falling asleep, shallow or fragmented sleep patterns, frequent nocturnal awakenings, and early morning awakenings [1]. Melatonin is an endogenous neurohormone that regulates the sleep-wake cycle [2]. An increasing number of people are seeking melatonin supplements to improve sleep quality. Melatonin gummies have become a popular over-the-counter sleep aid. Between 2016 and 2020, the sales in the United © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 873–886, 2024. https://doi.org/10.1007/978-3-031-47281-7_70

874

X. Zeyuan and A. Y. Choi

States increased by 150% [3]. To enhance sales and improve the user experience, melatonin supplements are often presented in the form of gummies, chocolates, or oral sprays. The sweet and innocuous appearance of these products, resembling candies, may lead even adults to underestimate their role as sleep aids, potentially relaxing their vigilance regarding the efficacy of melatonin gummies for sleep. Unfortunately, such candy-like packaging of melatonin has not given enough consideration to households with children, resulting in numerous cases of accidental ingestion by children. During the COVID-19 pandemic, there was an increase in cases of pediatric melatonin exposure. In 2020, melatonin became the most frequently ingested substance among children reported to U.S. national poison control centers. Children are at increased risk of melatonin exposure due to the widespread use and popularity of melatonin supplements. In 2021, the U.S. Poison Control Centers received over 52,000 calls, reporting concerns over children ingesting significant amounts of melatonin supplements, representing a sixfold increase from approximately a decade ago [4]. Calls related to pediatric melatonin ingestion increased by 530% from 2012 to 2021, resulting in 27,795 emergency department and clinic visits, 4,097 hospitalizations, 287 intensive care unit admissions, and 2 deaths [5]. These data suggest that designers should comprehensively assess the packaging of melatonin gummies, including the shape, taste, color, and emotional changes experienced by users. 1.2 Research Contents, Methods and Purpose The study focuses on the packaging design of melatonin gummies, selecting the packaging of the top 10 well-known and bestselling melatonin gummy brands in the current market for analysis. The research employs a literature review approach, reviewing relevant literature to understand the current state of research and the theoretical foundations related to the topic. A combination of qualitative and quantitative research techniques is used, primarily utilizing the three stages of emotional design as measurement criteria. The aim is to identify the emotional factors and emotional needs associated with users of melatonin gummy products. Subsequently, the study enters the discussion phase, exploring the effectiveness, limitations, and potential areas for improvement of the research methodology, while also proposing recommendations for future research. Research Purposes: Firstly, to gain a comprehensive understanding of the concept of emotional design and apply emotional design models to analyze products and brands. The analysis aims to examine whether emotional design is universally applicable to the packaging of medical and healthcare products. Secondly, to comprehend the distinct perspectives that emotional design brings to the packaging of pharmaceutical products. This involves explicitly highlighting the focus areas of the instinctive, behavioral, and reflective levels in emotional design, which need special attention when designing packaging for sleep aid supplements. The study proposes that emotional design can make the packaging of specialized pharmaceutical products more personalized, diverse, and humane. Thirdly, to analyze pharmaceutical packaging through the lens of emotional design theory, assessing and judging whether the current packaging designs for melatonin products on the market are the result of a comprehensive application of the multi-dimensional emotional design principles. The study argues that melatonin packaging is distinct from

Defining the Emotional Design Stages of Package Designs

875

both food and traditional pharmaceutical packaging. It aims to concretize the dimensions of packaging design based on the fundamental concepts of emotional design and to infuse humanistic care into the packaging of specialized medical products providing methods. Ultimately, the study seeks to provide new design foundations and establish a new evaluation system for packaging design. Given this background, melatonin, as a dietary supplement, needs to consider emotional factors when designing its packaging. However, the concealed and innocuouslooking packaging of melatonin has led to an increase in cases of accidental ingestion by children. This calls for designers to convey humanistic care and the ingenuity of emotional design in packaging while reflecting on packaging safety. Different layers and dimensions of emotional communication should be integrated into the design to engage users in a sensory dialogue, conveying a sense of trust and reliability.

2 Theoretical Background 2.1 The Emotional Design In an era of rapid technological advancement, emphasis has often been placed on rationality, with the belief that research, design, and judgment should be based on reason. Designers were expected to overcome their subjective emotions by relying on rational and logical thinking. Emotions and moods have long been regarded as animalistic and instinctual aspects of human behavior [6]. However, emotional design, as an interdisciplinary field that combines psychology, neuroscience, and design [7], highlights the positive role of users’ emotions in product design [8]. Designers are now required to evoke positive emotional responses in users through product design to enhance the overall user experience. Design philosophy has gradually transitioned from Sullivan’s “form follows function” to Esslinger’s “form follows emotion” [9], reflecting the trend of increased attention to user emotions in product design. In simpler terms, emotional design aims to stimulate users in a way that elicits emotional fluctuations and emotional connections. It leverages product functionality, operational behaviors, or product characteristics to evoke emotional awakening and identification, ultimately leading users to form a unique perception of the product [10]. Factors like the visual design of the product, often the first point of consumer-product interaction, are utilized to elicit desired emotions [11]. Emotions are directed at something, and they are generated by actions taking place exterior to the body or thoughts, internal depiction of memories and experiences [12]. Emotional design requires a method that emphasizes evaluation and attention. While emotions may be subjective, the emotional process is universal, and individuals with shared concerns and evaluations tend to experience similar emotions regarding specific products [13]. Therefore, it is essential to define the term “emotion” and delineate the scope of emotions. In everyday life, people often use words like “affection,” “feeling,” “emotion,” and “mood” to describe psychological activities. However, in emotional design, the definition of emotion refers to the entire process of user interaction with a product, starting from initial contact to the lasting memories it leaves.

876

X. Zeyuan and A. Y. Choi

2.2 The Multi-Level Emotional Design Scholars have categorized emotions into multiple levels for better understanding. In “Designing for Emotion,” author Aarron Walter relates emotional design to Maslow’s Hierarchy of Human Needs and introduces the Hierarchy of User Needs theory. According to this theory, product traits can be categorized as functional, reliable, usable, and pleasurable. Emotional design falls into the highest level, “pleasurable.“ Like Maslow’s hierarchy of needs, Aarron suggests that users must satisfy lower-level needs before pursuing higher-level ones [14]. Donald Norman, combining insights from cognitive psychology, proposed three levels of emotional design. He divided human psychology into three facets: visceral, behavioral, and reflective, which correspond to the three levels of emotional design: visceral level design (instinctive level design), behavioral level design, and reflective level design. In contrast to Aarron, Norman posits that in any design, the three distinct levels of visceral, behavioral, and reflective are intricately intertwined. There may not be a clear hierarchy of fulfillment. No design can be entirely independent of these three levels. And it is crucial to understand how these three components interact with emotions and cognition [10]. Firstly, the visceral level represents people’s initial response to a product. It is the most instinctive and animalistic response in humans, transcending race and culture. “The instinctive level precedes consciousness and thought, it forms the basis for the appearance elements and first impressions. The instinctive level design places greater emphasis on the initial impression the product gives people, focusing on aspects such as the product’s appearance and tactile qualities.“ At the instinctive level, aspects of a product such as its color [15], shape, size, material [16,17], etc., are the most fundamental factors influencing emotions. Experiments suggest that the instinctive level demonstrates that aesthetically pleasing appearances stimulate the brain, enhance creativity, increase tolerance for minor difficulties, and mitigate the negative aspects of a product [18]. The behavioral level design depends on the functionality of the product and its ease of use. The behavioral level design is the stage where the functionality of the product is realized. While achieving well-functioning design may seem straightforward among the myriad standards that products must meet. It is quite challenging. Implicit user needs are not as evident as one might imagine. HLB emphasizes that the true challenge of product design lies in “ultimately understanding those needs of the user that are not satisfied or explicitly stated [10]. “ The case of melatonin gummy ingestion illustrates that designers need to optimize behavioral level design to avoid some problems. The reflective level design is about the meaning of the product to the user. It involves users’ feelings and reflections when using the product and after use. Users complete their sense of identification with the product and fundamental value judgments through reflection. At this stage, designers should capture memory points that induce emotional connections through user reflection on the product or experience. The reflective level exists only in the realm of human cognition, making it a battleground for outstanding design works. The multilayered emotional design provides a fundamental design methodology for product packaging design. Emotional design requires designers to consider the integrated

Defining the Emotional Design Stages of Package Designs

877

impact of every touchpoint at each level when designing a product. If applied to packaging design, it should simultaneously possess the following three aspects: 1, Evoke the user’s instinctive response at first sight, creating an initial favorable impression. 2, Align with user behavioral habits during usage, providing essential functionality for smooth use. 3, Connect with the user on a deeper emotional level, triggering associations and resonance, thereby forming unique memories related to the usage process or the brand. This necessitates that a designer’s scope of design encompasses not only an attractive appearance and functional ease of use but also extends to a higher spiritual level, aiding users in forming memories or building expectations about the product. 2.3 The Constraint in the Design Constraint design refers to the design approach that guides user behavior by restricting actions during the design process. Constraints can be categorized into two types: physical constraints and psychological constraints. Physical constraints limit user actions through physical means, while psychological constraints influence user behavior through symbols, conventions, mappings, or suggestions [19]. Some also refer to this as “obstacle design”. Physical constraint design often restricts users’ habitual actions or interrupts their user experience. It raises concerns that the absence of behavioral level design in emotional design may render the entire product meaningless. Norman illustrates an interesting point through case studies. When users have a negative user experience with a product, businesses can improve it through later benign interactions or reflection layer design. Remarkably, users tend to have a more favorable perception of businesses or brands with emotional touchpoints than a perfectly smooth and error-free user experience. This enhances their sense of memory and closeness. The square toilet paper roll design by Shigeru Ban is considered a classic case of redesign and serves as an excellent example of emotional and constraint design. Ban replaced the traditional circular roll of toilet paper with a square one and introduced an obstacle in the rotation, making the process of using toilet paper less smooth [20]. Due to the non-smooth behavioral level, users’ attention is drawn, arousing their curiosity and guiding them into the reflection level. Users contemplate why this operation differs from their daily life experiences, conveying the designer’s intention to promote paper conservation. This is achieved by setting physical constraints in the behavioral level, leading to the perception of the reflective level (Fig. 1). Psychological constraints are commonplace in daily life, such as prohibition and toxic signs. These are often displayed using symbols like a cross or menacing skull icons, highlighted in red and yellow to indicate danger. Products like laundry detergents and mouthwashes incorporate bright colors like pink and blue, strong appetite suppressant colors [21], or highly stimulating fragrances, leveraging human instincts to avoid risk. Young children have a well-developed ability to detect and reject bitterness [22]. To ensure safety, bittering agents or appetite-suppressing dyes are added to oral medications to prevent accidental ingestion. Designers employ psychology to protect users by implementing psychological constraints. Users typically expect a seamless experience, but an unsmooth behavioral experience triggers perception at the reflective level. It provides a targeted and feasible framework

878

X. Zeyuan and A. Y. Choi

Fig. 1. The design of Shigeru Ban’s sanitary toilet paper holds significance in terms of innovation, critique, and constraints.

to help consumers make rational decisions with minimal cognitive load. By introducing obstacles in the behavioral layer and disrupting the smooth user process, users are prompted to think, engage in a dialogue with designers, and become part of the narrative system created by designers. When users accurately grasp the designer’s ingenuity, it triggers deep thinking and forms memory points about the product. By first setting constraints and then creating a reverse design, emotional fluctuations in users are easily aroused. Users immediately establish an emotional connection with such meaningful design works.

3 Case Study on Packaging Design of Melatonin Gummies 3.1 The Particularity of Melatonin Packaging Design Melatonin Packaging Design is Different from Previous Food Packaging Designs (Table 1). Table 1. Comparative analysis of food packaging and melatonin gummy packaging.

Target Audience:

Food packages

Melatonin gummy packages

The target audience is typically broad and encompasses consumers of various age groups

Primarily target patients with sleep disorders, necessitating consideration for the protection of special groups such as the elderly, children, and pregnant women (continued)

Defining the Emotional Design Stages of Package Designs

879

Table 1. (continued) Food packages

Melatonin gummy packages

Selling Locations:

The places where packaging is encountered are more public, with a wider audience. Retailers manipulate consumer emotions, increase product exposure, and boost sales through the design of selling spaces, including lighting, music, fragrance, and shelf placement [23]

Melatonin gummies belong to a specialized category of health supplements, and their selling locations are not as widespread as those for food products

Perspective of Conveyance and Guidance:

Aims to capture people’s attention at the instinctive level through factors such as packaging colors, shapes, sounds, textures, smells, and sizes [24]. The primary direction of conveying information is to recreate and emphasize the appearance, color, and texture of food through packaging design, thereby stimulating consumers’ desire to consume, and promoting sales. From a behavioral perspective, packaging should fulfill basic functions such as preservation, sealing, and ease of opening, using visual cues to guide consumer behavior

In contrast, the instinctive level design of health supplements does not require bright colors or exaggerated graphics. It does not aim to maximize consumer attention but rather to differentiate the product within its category. From a safety perspective, melatonin packaging design should not lead non-sleep-disturbed consumers to use the product, nor should it exaggerate the efficiency of sleep aid supplements. From a behavioral perspective, packaging should offer basic functions like preservation, sealing, and ease of opening, while also considering safety in design at the reflective level

Labeling and Information Perspective:

Food packaging design needs to present information such as ingredient lists, expiration dates, and calorie content

Melatonin packaging should clearly state dosage, timing of use, efficacy, usage recommendations, and precautions to guide consumers correctly (continued)

880

X. Zeyuan and A. Y. Choi Table 1. (continued) Food packages

Narrative and Emotional Food packaging design can Communication Perspective: convey higher emotional threshold concepts like energy, joy, celebration, or reunion. Packaging is used to evoke positive emotions in consumers

Melatonin gummy packages Melatonin gummy packaging design primarily focuses on the direction and scale of design. While emphasizing the pharmacological function of the product, it should also consider the soft emotional design aspects such as comforting, reassuring, and supporting patients from the perspective of doctors and patients who frequently encounter the medication

Melatonin Packaging Design also Differs from Traditional Pharmaceutical Packaging. Pharmaceutical packaging primarily serves patients and healthcare professionals, prioritizing the accurate transmission of information. Therefore, ease of recognition and functionality take precedence over aesthetics in pharmaceutical packaging design. Melatonin supplement packaging can highlight its health function and is allowed more autonomy and design freedom than pharmaceutical packaging, meaning there can be significant differences between products. In summary, melatonin gummy packaging falls between food packaging design and pharmaceutical packaging design. It needs to differentiate itself from similar sleep aid health products through design while conveying essential information. Melatonin gummy packaging should strike a balance between the two, differentiating itself from competitors through packaging design and fulfilling its role of providing basic information while offering comfort to patients.

3.2 Multi-level Emotional Design Analysis of the Melatonin Gummy Brand Tables 2, 3, 4 analyzes ten well-known brands of melatonin gummy in the market today and analyzes how each brand triggers users’ emotions at various stages from the perspective of three design levels of emotional design. 3.3 Summary of the Emotional Design of Melatonin Gummy Brand According to Table 5, it is easy to discern both the commonalities and individualities in the current design of melatonin gummies. Simultaneously, it is apparent that there are strengths and weaknesses in the current market packaging designs for melatonin gummies. Given the uniqueness of melatonin gummies, manufacturers aim to convey emotions such as trust, assurance, calmness, serenity, and spirituality through packaging design.

Defining the Emotional Design Stages of Package Designs

881

Table 2. Analysis of the emotional design of the well-known brand of Melatonin gummies. Name

Vita fusion

Natrol

Olly

Zarbee's Naturals

Origin Packaging

The U.S.

The U.S.

The U.S.

The U.S.

Visceral level design

Material: Color:

Pattern/ element:

Behavioral level design Reflective level design

TransparTransparent Transparent Transparent plastic ent plastic plastic plastic Purple, Purple, Purple, white Dark bule, yellow green, white and orange red Moon, Color blocks Fruits, sleepA bee in a nightcap. stars, and straw- ing moon, The stars, the moon. clouds, berry clouds, circles gummies. gummy candies. The safety cap design is adopted to prevent children from ingestion. It does not adopt the form of separate packaging. None of them has much more of an operational interaction. The elements in the design of instinctive level use stars, moon, clouds, nightcap and the circle can convey information about the biological clock, night, peace of mind and calm. The dark packaging is decorated with bright colors and the light design can relieve the anxiety and tension of patients taking melatonin and help them fall asleep. On the behavioral level, through the design of a safety cover, users can be triggered to trust the quality of the product and have the idea of appropriately taking it.

Overall, among the numerous products currently on the market, only the instinctive level is relatively strong, with some brands creating a reassuring narrative atmosphere for users solely through external packaging. The shortcoming at the instinctive level design is the lack of psychological constraints. For example, most products opt for transparent plastic bottles, allowing everyone to see the appearance of the gummies inside. Without changes in the instinctive level design or the color and shape of the gummies, there is still a lack of psychological constraints for children. The behavioral and reflective levels still hold significant potential, and designers are encouraged to enhance the narrative by introducing physical constraints and other creative elements in the design. This will help users form a strong emotional connection with the product and complete the narrative between the user and the product.

882

X. Zeyuan and A. Y. Choi

Table 3. Analysis of the emotional design of the well-known brand of Melatonin gummies. Name Origin Packaging

Visceral level design

Behavioral level design Reflective level design

Material:

Sundown The U.S.

Nature Made The U.S.

Transparent plastic

Transparent plastic

Color:

Orange, red

Yellow, red

Pattern/element:

Fruits, square

Fruits, color block.

The safety cap design is adopted to prevent children from ingestion. It does not adopt the form of separate packaging. None of them has much more of an operational interaction. The instinctive level design mainly uses the orange color to echo the product name sunset, triggering the idea that people should rest after sunset. A strawberry motif is chosen to illustrate the flavor of the gummies. Overall yellow and orange are not strongly associated with sleep, and the design is not very suitable for melatonin. Thus, it is difficult to reach the reflective level of the patient.

4 Improvement Direction of Packaging Design of Melatonin Gummies The entire process of emotional design can be viewed as a process in which designers encode information, which is then decoded by consumers. When the design encoding and consumer decoding align perfectly, it indicates a design process that maintains emotional continuity. At the instinctual level, colors, patterns, and shapes are utilized to construct a narrative backdrop and create a narrative atmosphere. Colors such as deep blue or deep purple, associated with trust, comfort, and tranquility, may be chosen as primary color schemes to create a narrative atmosphere that aligns with tranquility and peaceful sleep for patients. Brighter colors like yellow or green, used in smaller areas, can reduce the weight of the design, add a sense of lightness and delight, alleviate the heaviness and burden associated with medication, and facilitate product differentiation. The selection of graphics that evoke sleep-related imagery, such as starry nights, pillows, pajamas, or sleep caps, is used to construct a narrative backdrop and create an atmosphere. Psychological constraints should be added at the instinctual level, which can be accomplished using color, symbols, implications, and symbolic methods, or by altering the color and scent of the gummies. At the behavioral level, users can be guided into a dialogical atmosphere through simple physical constraint designs. In behavioral design, designers can enhance the enjoyment of opening, using, and storing melatonin gummies by introducing simple operations. By adding the enjoyment of operation, the multidimensional experience of

Defining the Emotional Design Stages of Package Designs

883

Table 4. Analysis of the emotional design of the well-known brand of Melatonin gummies. Name Origin Packaging

Nordic The U.S.

Garden of life The U.S.

Nature’s Way The Australia

Herbaland The Brazil

Visceral level design

Material:

Opaque plastic bottle

Opaque plastic bottle

Opaque plastic bottle

Opaque bottle

Color:

White, blue

White, purple

White, purple

White, blue

Pattern/ element:

Fruits, circle

Fruits, color block.

Fruits, moon, color block.

Fruits, moon, clouds, stars

plastic

Behavior level design

The safety cap design is adopted to prevent children from ingestion. It does not adopt the form of separate packaging. None of them has much more of an operational interaction.

Reflective level design

Instinctive level design large area of pure white with blue purple auxiliary color matching, transfer quiet, pure feeling. The traditional medicine packaging material contains gummies, which is difficult to distinguish from medicine, and easy to gives patients the psychological burden of 'taking medicine'. Sealed packaging is used to prevent the gummy from deteriorating by light; Purple flowers highlight the natural herbal properties of the product, natural herbs give people peace of mind, purely natural, with small side effects.

packaging can be enhanced, deepening the depth and meaning of healthcare product packaging, and making the product more dialogical and narrative. Adding some simple physical constraints to protect consumers and guide patients to take the correct dosage is recommended. Positive interactions can provide comfort and confidence to patients, offering a sense of care during the operation, allowing patients to feel cared for by the designer, and establishing a preconceived notion that the brand is an image of trustworthiness and empathy. It is advisable to adopt a separate packaging model that ensures freshness and taste while adding a new level of constraint design to the behavioral level to prevent users from overdosing and developing drug dependence. The addition of a safety cap may make many users feel interrupted during their initial use. Therefore, how to evoke deeper resonance in the reflection level through this obstacle setting is an innovative point and design point for designers. Adding the constraint design of special category packaging reveals the scale and temperature of the design, and it is the embodiment of the designer’s empathy and sense of responsibility. At the reflection level, deepening the product’s sense of humor, profundity, care, and other emotional factors accompany users to complete the experience and strengthen

884

X. Zeyuan and A. Y. Choi Table 5. The visual design element of the Melatonin gummy brand.

Visceral Design

Visual elements

Case study

Emotional narrative and expression

Dominant tone

Purple, dark blue, orange, white

In color psychology, color gives people calm and stable psychological feelings, and sets the tone for the following narrative

Auxiliary color

Yellow and green

Relieve the psychological burden of taking medicine and increase the sense of relaxation and pleasure. Easy to distinguish similar products

Pattern

The stars, the moon, the night, the sunset

Fit in with the mood of sleep, create peace of mind, quiet narrative atmosphere

Figure

Circle

Life cycle, the biological cycle

Behavioral Design

Most have a safety cover design. Did not take the form of packaging. The procedure for taking medication does not adjust well to the atmosphere of sleep. There are no further operates lead the user into the narrative context

Reflective Design

The narrative is not well completed, there is a lot of potential space

emotional connections. The reflection level design is a determining factor in judging the excellence of a product. For product design targeting specific groups, emotional factors need to be considered. Emotional design provides a fundamental methodology for packaging design. It requires a comprehensive consideration of emotional connections between products and customers through instinctual, behavioral, and reflective level designs. For melatonin gummy packaging design, instinctual-level design can use images, graphics, colors, etc., to help consumers construct a narrative backdrop, creating an imaginal narrative atmosphere. At the behavioral level, users are encouraged to join the background of the dialogue with the designer through hands-on operations. In reflection level design, packaging can carry and convey more emotional information, forming a narrative design through visual language.

5 Conclusion This article elucidates the significant role of emotional design through the analysis of melatonin gummy packaging design. It provides a paradigm for analyzing the packaging design of melatonin gummies using a multi-dimensional emotional design model. Factors across the instinctual level, behavioral level, and reflection level- complement and

Defining the Emotional Design Stages of Package Designs

885

permeate each other, stimulating user emotions and conveying the warmth of design. It is understood that packaging design for special medical and healthcare products requires a different emphasis on emotional aspects, quantifying design scales, and emphasizing the importance of incorporating psychological and physical constraint design in the behavioral level for sleep aid healthcare products. The article highlights the addition of operational delight and an expanded dimension to packaging through physical constraints. However, this article has certain limitations as it cannot ascertain specific data regarding children’s ingestion caused by different brands of packaging, thus preventing the formulation of more targeted recommendations. It is hoped that this gap can be addressed, and the hypotheses validated in future research. The paper aims to provide new design principles and evaluation criteria for melatonin gummy packaging design, pinpoint new design priorities, and suggest directions for future iterations. Emotional design and constraint design provide theoretical support in practice for packaging design. They help designers understand that while designing products to fulfill the basic functionality of packaging, they should also design from the perspective of evoking user emotions.

References 1. Wang Y, Salas RME (2021) Approach to common sleep disorders. Semin Neurol 41(6):781– 794 2. Karasek M, Winczyk K (2006) Melatonin in humans. J Physiol Pharmacol 57(5):19–39 3. Koopman-Verhoeff ME, van den Dries MA, van Seters JJ, Luijk MPCM, Tiemeier H, Luik AI (2019) Association of sleep problems and melatonin use in school-aged children. JAMA Pediatr 173(9):883–885 4. Gummin DD, Mowry JB, Beuhler MC, et al (2021) 2020 annual report of the American Association of Poison Control Centers’ National Poison Data System (NPDS): 38th annual report. Clin Toxicol 59(12): 1282–1501 5. Lelak K, Vohra V, Neuman MI, Toce MS, Sethuraman U (2022) Pediatric melatonin ingestions—United States, 2012–2021. MMWR Morbid Mortal Weekly Report 71(22):725–729 6. Picard RW (1997) Affective computing, 1st edn. Massachusetts Institute of Technology, Cambridge 7. Triberti S, Chirico A, La Rocca G, et al (2017) Developing emotional design: emotions as cognitive processes and their role in the design of interactive technologies. Front Psychol 8:1773 8. Hekkert P (2014) Aesthetic responses to design: a battle of impulses. The Cambridge handbook of the psychology of aesthetics and the arts. 1st edn. Cambridge University Press, Cambridge 9. Fay S (1999) Frog: form follows emotion, 1st edn. Watson-Guptill Pubns, New York 10. Norman DA (2004) Emotional design: why we love (or hate) everyday things.Basic books, 2nd edn. New York 11. Bloch PH (1995) Seeking the ideal form: product design and consumer response. J. Mark 59(3), 16–29 12. Van Gorp T, Adams E (2012) Design for emotion. Elsevier, Amsterdam 13. Desmet PM, Porcelijn R, Van Dijk MB (2007) Emotional design; application of a researchbased design approach. Knowl Technol Policy 20(3):141–155 14. Walter A, Spool JM (2011) Designing for emotion. A book apart, New York

886

X. Zeyuan and A. Y. Choi

15. Chebat JC, Morrin M (2007) Colors and cultures: exploring the effects of mall décor on consumer perceptions. J Bus Res 60(3):189–196 16. Kim D, Boradkar P (2022) Sensibility design. In: Proceedings of the IDSA National Design Education Conference, pp 17–20. New York 17. Ürgen C (2006) The use and importance of emotional design in contemporary design practice. Izmir Institute of Technology, Turkey 18. Campbell A, Pisterman S (1996) A fitting approach to interactive service design: the importance of emotional needs. Design Manage J (Former Ser) 7(4):10–14 19. Lidwell W, Holden K, Butler J (2010) Constraint. In: Universal Principles of Design, 3rd edn. Rockport Publishers, Massachusetts 20. Hara K (2008) Designing design, baden, 1st edn. Lars Müller Publishers, Switzerland 21. Zellner DA (2013) Color-odor interactions: a review and model. Chemosens Percept 6:155– 169 22. Kajiura H, Cowart BJ, Beauchamp GK (1992) Early developmental change in bitter taste responses in human infants. Develop Psychobiol J Int Soc Develop Psychobiol 25(5):375–386 23. Baker J, Levy M, Grewal D.: An experimental approach to making retail store environmental decisions. J Retail 68(4):445 24. Togawa T, Park J, Ishii H, Deng X (2019) A packaging visual-gustatory correspondence effect: using visual packaging design to influence flavor perception and healthy eating decisions. J Retail 95(4):204–218

Audiovisual Design and Communication

Alcântaras – O Som é Um Sistema: Expanded Poetics of Production/Reception of Audio and Visual Scintillations of Sound as a System Ramusyo Brasil1

and Cesário Alves2(B)

1 Instituto Politécnico de Bragança, Campus Santa Apolónia, Bragança, Portugal 2 Politécnico do Porto - Escola Superior de Media Artes e Design, Rua D. Sancho I, n.º 981,

Vila do Conde, Portugal [email protected]

Abstract. Alcântaras, as the name suggests (bridge, in Arabic), proposes the creation of bridges that connect dispersed temporalities and technologies, but with the same matrix: territoriality and shared ethnicity. A certain State of Nature of living matters that demands, more than an opposition to the Rule of Law, a certain Ecology of Knowledge. A palm tree in the wind, music that spreads in the tidal breeze, by connecting image and sound, ancestral and emerging technologies, nature and culture, body and sound, we want to show the sound that emerges from the wind and the images heard by light, both in ethereal vibrational movement. Alcântara, in Maranhão, Brazil, which was once the scene of the commercial development of cotton crops, today pulses with reggae sound systems and Creole drums coming from the quilombola communities that were created during and after the end of the colonial era. Such communities are spaces of resistance and maintenance of very refined practices of a cultural framework that involves religiosity, art and life, and body politics. This essay reflects on the process that permeates this artistic investigation, by revealing sparkling/sound excerpts from a strong amalgam of black culture, hidden in a recess on the coast of Maranhão, which find in the reunion of the visions of a collective of artists and at the intersection of their practices, a unique and unrepeatable synthesis. Keywords: Performance · Sound Arts · Visual Arts · Creative Processes · Artistic Research · Documentary

1 Introduction and Problematization The artistic investigation proposal of Alcântaras – O Som é um Sistema is a revisitation of the territory that was once a Portuguese colony and that today is one of the biggest granaries of Brazilian black culture, the city of Alcântara1 . 1 Alcântara is a city with typical Portuguese architecture from the 18th and 19th centuries, in

which there are more than 150 rural quilombola communities, which are community lands of black populations enslaved by the Portuguese colonial process, in an area that also houses the Aerospace Launch Base of Alcântara, in the State of Maranhão, northeast of Brazil. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 889–900, 2024. https://doi.org/10.1007/978-3-031-47281-7_71

890

R. Brasil and C. Alves

Between 1780 and 1820, Alcântara was one of the richest cities in the Portuguese crown in Brazil. From a nautical chart that managed to reach Portugal in just 22 days, and a large production of cotton for the textile industry, the city was very rich, even richer than the capital of Maranhão, São Luís. it survives on tourism from cultural heritage and colonial architecture, from the important Festas do Divino Espírito Santo (Holly Spirit) and São Benedito (Saint Benedict), in May and August, respectively. Alcântara experiences the effervescence of art and culture, many artists live in the city or spend time developing artistic projects. One of Alcântara’s greatest assets is its quilombola communities. Source of a strong tradition of Afro-Brazilian culture, it is the scene of genuinely Maranhão popular manifestations, such as the Tambor de Crioula2 . The Tambor de Crioula is an eminently rhythmic music and dance present in Maranhão. Characterized by the tambor crivador, tambor meião and tambor grande (small, medium, and large drum), played by the coreiros (choir men) and danced by the coreiras (choir women), who perform in a circle and, above all, in front of the drums (Figs. 1 and 4). The research process that guided the project Alcântaras - O Som é um Sistema consisted precisely in tracing relationships between the nautical cartography of the colonial period and the sound and musical flow that traversed the black cultures dispersed across the Atlantic, starting in Africa and spreading out through Jamaica, other Caribbean countries, Brazil, Spain and Portugal, through processes of communication, technology and cultural transmission. The research involves an initial experience of the places and the documentary recording of the visual and sound landscape, which includes photographs, films, audio recordings of oral statements and other concrete sounds, followed by a conceptual and experimental development which is concerned with the various political, historical and spiritual dimensions of this territory. To assemble the threads that draw this cultural and aesthetic configuration, we make use of the sound arts in conjunction with the media arts, to create an expanded experience of receiving the audiovisual work, incorporating in the process the desire to adapt to different spaces/times of dis-semination: the space/time of the exhibition/installation; the website’s virtual space/time; the editorial space/time; the space/time of the performance. None of the forms that the work takes is, therefore, definitive, being part of an open process that reaches in the disciplinary intersection and the diversity of perspectives of its creators, something unique that can only derive from collaboration and encounter. The aim is to establish a dialogue open to contingency and transformation with the audiences and places where the work is presented, providing moments of dialogue, training and collective creation that encourage transformations in the work. We start from these images and sounds captured in the quilombola communities of Alcântara and other cities in the Baixada Maranhense (seaside Maranhão), to observe 2 The main game takes place between the tambor grande, who is the soloist, while the tambor

crivador and the meião provide the rhythmic basis of the demonstration, and the coreira (choir woman) who is in front of the big drum, in a game of correlations and of question and answer between the solo of the large drum and the movements of the choir highlighted in front of it. The coreiras give way to each other in front of the drums through the umbigada (bellybutton touch). The singing takes place in a question-and-answer format, starting from a base phrase that emerges in different variations throughout the song, led by the choir of those present.

Alcântaras – O Som É Um Sistema: Expanded Poetics of Production/reception

891

Fig. 1. Tambor grande, tambor meião and tambor crivador, instruments that form Tambor de Crioula. Photographed in Alcântara, Maranhão, Brazil. Cesario Alves, 2019

and feel the reggae radiolas, the sound system parties, as well as the popular cultures of Tambor de Crioula, bambaê3 and bumba-boi4 that also mark the rhythm of local identities (Figs. 2 and 3). At the same measure, in the state of Maranhão, the dynamics of the festa or festação (festivity) are marked by music, dance, exchanges and the liberating ecstasy of the movement of the body together with the drink, the laughter, the touch, the being-together and feeling the self, to live fully, in that ambience, the manifestation of the id-entity in its highest astral. The idea of id-entity, hyphenated in this way, alludes to the possibility of understanding this category as the mixture, or even the synthesis, between the Freudian id, where, according to Freud, the energies that, in opposition to the I of language, “(…) contains the passions” (p. 31) [1], together with the notion of entity of the Afro-Brazilian religions: an image of thought that brings with it the strength of the engagement of the body, in the possession of this medium guided by archetypes and spiritual energies that encompass the assumption and resemblance, necessary in the sense of collective work and ontological perception that every popular manifestation evokes at its core. 3 The bambaê is a cultural manifestation from Maranhão characterized by the sound of the boxes

of the Divine (drums), played by women. The ballet is danced in pairs and in a circular fashion. 4 Bumba-meu-boi, or simply bumba-boi, is the best-known cultural manifestation of Maranhão.

It goes back to the cattle cycle of the caboclo peasantry, in which African, indigenous, and European cultures mix in a mestizo synthesis of local cultural practices, updated annually during the São João (saint John) festivals, in the month of June, in Maranhão.

892

R. Brasil and C. Alves

Fig. 2. Page left: Cláudio do Espírito Santo, inhabitant of Alcântara, guide of the region and voice of one audio testimonial5 . Page right: Radiola in a neighbor’s house, near the artistic residency in Alcântara. Cesario Alves, 2019

As everything seems to point out, the uniqueness of a body is linked to the identity of its actions in an environment and the incessant flow of images that not only identify it in relation to other living beings but make it able to survive. All of this would also be related to the dramaturgy of a body, since everything is resolved the moment it happens. A present that carries history and points to the future, but that is organized at every moment, creating new nexus of meaning. (p. 80). [2] Thus, body technologies and technologies for the body, such as those of the Tambor de Crioula and radiolas, evoke forms of knowing-being and space-time coordinates that reorganize the modes of perception of the lived experience, constituting themselves as advanced epistemologies from being to the world, to life as technology and aesthetics of existence. The project Alcântaras - o Som é um Sistema aimed at two general objectives: (1) to produce, through a process of applied research, the realization of a body of documentary work on image and sound, which allows deepening previous research on the confluence of sonic cultures in the territory under study and its ramifications to other

5 The audio recording can be heard in the film, from minute 7´19´´ to 10´50´´ and from 14´19´´

to 15´49´´ in the following Youtube link: https://youtu.be/0-MuZJ8FxlA.

Alcântaras – O Som É Um Sistema: Expanded Poetics of Production/reception

893

Fig. 3. Frame extracted from a film by Ramusyo Brasil, 2015.

cultures and places6 ; (2) Starting from this base and in order to share and expand the knowledge it encompasses, think methodologically and collectively, the artistic and language potential of sound and visual arts, in the context of installation, performance and publication, incorporating edited and repeatable elements, but also elements of contingency, appropriation and reaction to the space, time and conditions of each presentation (Fig. 6). Alcantaras - o som é um sistema: the process. Thus, this obsessive permanence in the creative, this methodological amplification in the search for territory, transforms the artistic practice and, by condensation, the artistic project, into an experience of creative nomadism. (p. 102) [3] The first audiovisual and literary records that are part of this work were made in Alcântara – Maranhão, in 2010 by Ramusyo Brasil. Ramusyo concluded his doctorate 6 The activities of the project “Alcântaras – o som é um sistema” were presented while still in the

development process (performance) at the Festival Arca, Arquivo e Cinema, in Montemor-oNovo, Portugal, in November 2021, at the Galería Central (exhibition) and at the Aula Magna (performance) of the Universidad de Málaga, in addition to Librería Suburbia (performance + presentation of the zine), in Málaga – Spain, in June and July 2022, followed by presentations of the performance and publication at Associação VIC in the city of Aveiro and at Espaço Cultural MINA, in Bragança, within the CITA (International Cycle of Artistic Transversalities) in October 2022.

894

R. Brasil and C. Alves

Fig. 4. Frame extracted from a film by Ramusyo Brasil, 2019.

in social sciences (PUC-SP / 2011) with the thesis “Reggae in the Brazilian Caribe” that focuses on the presence and influence of Reggae in the sonic culture of São Luís, in Maranhão, that also nourished this project. The other decisive moment was the artistic residency opportunity of Cesário Alves in Alcântara - Maranhão in September 2019, within the Residency of Scientific and Artistic Exchanges / RICA, which resulted in multiple audio and visual records, another point of view from which it was possible to imagine a collaboration with Ramusyo Brasil, which would later become enriched with the work of the artists Mariana Sardon and Gustavo Dias and the museologist Carollina Ramos, which took place between 2020 and 2022. The beginning of the process is due to the artistic residency carried out by Cesário Alves, artist and lecturer of photography studies at the School of Media Artes and Design, of the Polytechnic of Porto, Portugal, between July and September 2019, through Financing of FAPEMA, the Foundation for the Support of Research, Scientific and Technological Development of the state of Maranhão, for the project NAVE NUVEM – Advanced Nucleus of Excellence of Nucleus of Visualities, Aesthetics and Mediations, from the publication of the open call for International Cooperation 2017 (COOPI). The project was coordinated by researcher Marcus Ramusyo de Almeida Brazil and has as partner institutions the Nucleus of Research and Image Production (NUPPI) of the Federal Institute of Education, Science and Technology of Maranhão (IFMA), Brazil, and the School of Media Artes and Design, of the Polytechnic of Porto, in Portugal.

Alcântaras – O Som É Um Sistema: Expanded Poetics of Production/reception

895

The experience of Cesário Alves was an immersion in the universe of Alcântara, in its sound and visual landscapes, in its strong popular culture of the Tambor de Crioula and reggae sound systems, through a process of documentary observation, which implied as much the registration of photographs, video and sound through smartphone, hub of contemporary technological hybridization, but also with the quality of digital reflex systems and photography with silver salts film, with 120mm format. The artist also produced and curated a workshop/exhibition entitled “Photography without a camera: between the document and pictorial representation”, with the participation of students from the class of Production, Creation and Teaching with Images specializing in Art, Media and Education, from IFMA, at the Historical and Artistic Museum of Maranhão (MHAM), in São Luís do Maranhão. In this artistic research through practice, the participants worked with the technique of printing photograms in photosensitive developing black and white paper. The photograms provide a platform for intuitive plastic expression with the potential both for symbolic and abstract representation and for serving as documentary evidence of the shape of things. In this way, the pictorial results of this collective experience, as produced and exhibited in public in the Historical and Artistic Museum of Maranhão (MHAM), reveal not only the vestiges of the objects shapes, but also the unrepeatable moments of the gestures that illuminated them (Fig. 5).

Fig. 5. Page left: Hands of the last lace maker in Alcântara, Dona Joralda Ribeiro, also voice of one of the audio testimonials. Page right: Photogram from the workshop with the students of IFMA at the Historical and Artistic Museum of Maranhão (MHAM), in São Luís do Maranhão. Cesário Alves, 2019.

896

R. Brasil and C. Alves

Ramusyo Brasil, in turn, has a large collection of image and sound recordings of cultural manifestations from Maranhão, made in contexts of other research projects, such as the Tambor de Crioula, the bambaê, bumba-boi, among others. When confronting the archives of their experiences, the two researchers envisioned common paths of artistic investigation and possibilities for disciplinary crossings. When viewing the archives together, the idea of Alcântaras emerged: bridges of views, landscapes and diverse temporalities over a certain latitude that borders the Atlantic, which reflects and incorporates many other latitudes. Sound, music, dance, are communicating dimensions of ancient waves that crossed the Atlantic, creating a cultural environment that is both profuse and identity-connected. (…) knows how to establish vinculum who has knowledge of everything, or who knows at least the nature, disposition, inclination, habit, utility and purpose of that particular reality that should be connected by vinculum. (p. 25) [4]. The idea of vinculum established from symbolic exchanges is important to the present audio and visual project, in the sense that we understand sound as a technological, social and cultural system, in which social groups and subjectivities work creatively around the ethereal language of the body and the production of sounds, in the form of music or simply textures and/or noises, in its audible form, but also in its visual representations. in the tessitura between body, sound and image production lies the core of the audiovisual exercise proposed here, in the search for a certain sonic dominance of the living materials

Fig. 6. Audiovisual performance in Málaga, Spain, 2022

Alcântaras – O Som É Um Sistema: Expanded Poetics of Production/reception

897

at play, light presents itself in its becoming rhythm and sound as a flickering of the senses, in deep synesthetic symbiosis present in the flows of the assumption of the body as a medium of language, as stated by henriques: Sonic Dominance occurs when and where the sonic medium displaces the usual or normal dominance of the visual medium. With sonic dominance sound has the near monopoly of attention. The aural sensory modality becomes the sensory modality rather than one among the others of seeing, smelling, touching and tasting. (…) With the sonic, as I will discuss, environment, context and the combination of the senses are all particularly vital. (p. 452) [5]

2 Methodology (…) sometimes it’s nice to reach out and touch a sound. (6). The notion of sound tactility is evoked here as a method of listening. “The finger isn’t obsolete. The ear isn’t obsolete” (7). the sound is diaphanous and haptic, It allows the passage of light and contains a dimension of touch, in the fingers as a sound gesture, in the body as an emanation of sound and in the pressure of the loudspeaker frequencies on the air that moves and passes through us. The collective sound creation involved in this project includes composition and interpretation with musical instruments, such as the electric bass, in which the amplified sound of the struck strings provides its rhythmic pulse that guides the other interventions. another dimension of tactility is explored with electronic sound synthesis instruments, built and adapted by the artists themselves, so that their flow of electricity can be influenced by variations in light and body movement, but also through touch pressure. on contact electrodes, conditioning amplitudes and frequencies. The printed publication/zine produced to document and expand the work is also a tactile materialization that represents the image archive, but also the sound, alowing the hearing and seeing of frames from the film, photographs and transcriptions of recorded speeches, the sound as an illocutionary act in the image. What brings the correspondences between sound and image closer together is the double characteristic that gives them form: the manifestation of the inscription of time in recorded sound or photographic production, as well as the charge of electromagnetic waves that transmit latent energy that is converted into image, into sound, in image-sound, from the perspective of an audio-vision7 , as formulated by Chion. Here, the inscription of time provides in the sound-visual performance the notion of a single instant, even in the processes of sound, rhythmic and visual patterns, which gives rise to difference in repetition, in what Deleuze [8]. Proposed in the transcendence of difference and its radical immanence to the experience of the subjects, on which only 7 Audio-Vision: The type of perception proper to the experience of film and television, but often

in real life as well, wherein the image is the conscious focus of attention, but one to which the sound supplies at every moment a series of effects, sensations, and meanings which often, by means of a projective phenomenon I call added value, are credited to the image and seem to emanate naturally from it. [9]

898

R. Brasil and C. Alves

temporal changes and becoming affect. The methodology, therefore, wants to understand how this relationship enables the construction of an experience of difference/event. Therefore, it is essential to think about technology from its material base, its modes of production, as well as the disruptive character of massive urban culture, from the perspective of notions such as continuous and discontinuous [10]. These dimensions are observable in the engagement of bodies, performances and forms of orality [11], shaping the construction of aesthetics of existence [12] in the context of Jamaican sound systems and Maranhão radio stations [13]. From a theoretical-methodological point of view, the project used Benjamin’s method of dialectical assembly [14], which favors the assembly of fragments with the intention of forming a critical constellation, a dialectical image.

3 Results As a result of the research and creation process, we created an installation exhibition that displayed and contextualized the archives of images and sounds, a sound-visual performance, to make it possible to see and hear the images, a publication/zine as a permanent materialization of the process experienced, and a website8 , which served as a space for descriptive memory in the process of constant remaking of the project. Looking for interaction with the communities where the work was presented, we carried out on two occasions in different cities, the TapeLoop workshop, with the artist Mariana Sardon, from which participants created magnetic tape loops to integrate into sound design, production projects or musical performance. The artist uses these devices to record concrete sounds, later manipulated on modified tape players, to produce effects of repetition, acceleration or dragging of time, incorporating them into her participation in performance. She also contributed to this project with the construction and adaptation of electronic audio generation circuits, such as the synthesizer known in the reggae universe as ‘Dub Siren’, bringing to this artistic investigation the technological and aesthetic questioning of concrete music and field recordings, allied to the potential of modular and granular synthesizers (Fig. 7). From the crossing of the sensibilities of Mariana Sardon and Cesário Alves, the instruments created and used in this research, acquire a unique signature that crosses sound design with the visual, also mirrored in the conception of the zine publication, which documents and expands the entire project and whose pages illustrate this article. In this sense, the idea was to create an immersive environment of image and sound that would bring connections between ancestral sounds and current modes of culture and technologies, which involve an entire transit of black culture in colonial and post-colonial flows and refluxes, which resulted into beautiful and powerful aesthetic experiences. A state of sound well-being that is provided by the sensitive experience of ambience, frequencies, repetitions (loops), passages and correlations of forces between volumes and durations. This poetics of sound was incorporated into the content of images, both still (photographs) and moving (films), which sought a poetics of sound in images, as suggested by Lissovsky in his text Photography is a song [15]. Thus, one of the possible final considerations of this research concerns the intermediality of technical images and electronically amplified or generated sounds, which offer 8 https://omuca.wordpress.com/alcantaras-O-som-E-um-sistema/

Alcântaras – O Som É Um Sistema: Expanded Poetics of Production/reception

899

Fig. 7. Instruments built by Mariana Sardon and Cesário Alves, 2022.

possibilities for different readings, reverberations and tensions, according to the places in which they are inserted. This reinforces the experiential and performative character of this proposal, in which, with each movement of mediation with the chosen supports, new fields of meaning open, making other syntheses of language possible, an audiovision implicated in a phenomenology of the event that connects past (archives), present (performativity of sound and image produced live) and becoming (improvisation and projection). We use our bodies as conductors/enunciators of sound, while we are touched by sound and transport it, feeding it back to to the system that generated it, so that it can be transformed again.

References 1. Freud, S. Obras completas – volume 16: o eu e o id, “autobiografia” e outros textos (1923 – 1925). Companhia das Letras, São Paulo (2011) 2. Greiner, C. O corpo: pistas para estudos indisciplinares. Annablume, São Paulo (2013) 3. Cañizares S (2004) Proyecto artístico y territorio. Editorial Universidad de Granada, Granada 4. Bruno, G. Os vínculos. Hedra, São Paulo, (2012) 5. Henriques, J. Sonic dominance and the reggae sound system session. In Bull, M.; Back, L. Auditory culture reader. Berg, Oxford, (2003) 6. Collins N (2020) Handmade electronic music: the art of hardware hacking. Routledge, London 7. Behrman D (2020) Foreword In Handmade electronic music: the art of hardware hacking. Routledge, London

900

R. Brasil and C. Alves

8. Deleuze, G. Diferença e repetição. Paz & Terra, São Paulo (2018) 9. Chion, M. La descrption du son dans l’analyse áudio-visuelle, Homepage, http://michelchion. com/texts, Accessed 7 Aug 2023 10. Lotman Y (1999) Cultura y explosión: lo previsible y lo imprevisible en los procesos de cambio social. Gedisa Editorial, Barcelona 11. Zumthor, P. Performance, recepção, leitura. Cosac Naify, São Paulo (2007) 12. Foucault M (2012) Ditos e escritos: ética, sexualidade, política. Forense Universitária, Rio de Janeiro 13. Brasil, R. O reggae no Caribe brasileiro. Pitomba, São Luís (2014) 14. Benjamin, W. Passagens. Imprensa Oficial & UFMG, São Paulo, (2009) 15. Revista Zum Homepage, https://revistazum.com.br/radar/fotografia-e-musica/, Accessed 29 May 2023

The Production of Effects of Presence in Live Audio-Visual Performances Frederico Dinis(B) CEIS20 - Centre for Interdisciplinary Studies, University of Coimbra, Coimbra, Portugal [email protected]

Abstract. Performance has changed from a transgressive function to an artistic genre among others. This happened not only because the socio-cultural environment changed but also because conceptions of the body changed with contact with the technology itself. As a result, performance has become a crossroads of artistic disciplines, where unconventional dialogues converge in the desire to embark on an experiment that integrates the performer with an intertextual, multisensory, technological and experiential approach to the event. In this context, audio-visual performances are characterised by the incorporation of alternative aesthetics and technological advances, moving away from the traditional focus on the performer’s body and expanding into other media, especially sound and images. In addition to this expansion into other media, live performance is coconstructed by the physical presence of performers and spectators. The performative moments can have a transformative effect as spectators experience the materiality of a live performance as an ephemeral event. This materiality of the ephemeral can also be achieved by incorporating technological means during the performative moment, thus reinforcing a performativity of sound and image in which sound and image negotiate their meaning. In this essay, therefore, we will explore how live audio-visual performances have the capacity to stimulate new interpretations of artworks and generate a sense of presence through technological mediation, based on a series of artistic practices and audio-visual performative moments. Keywords: Presence · Audiovisuality · Performance

1 Introduction The idea of performance can be understood from a variety of angles, from different disciplinary perspectives, artistic domains or cultural settings (Wardrip-Fruin 2006). It is this conceptual flexibility and range of creative methodologies that make performance, as an action presented to an audience (Carlson 2004), rich in potential for exploration. This exploration focuses particularly on its multiple interpretations, and we aim to formulate a perspective that emphasises the intersection of sound and visual elements, memory and sensory media as a theme for both theoretical analysis and creative recognition. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 901–910, 2024. https://doi.org/10.1007/978-3-031-47281-7_72

902

F. Dinis

Performance has become a crossover point between the arts, bringing together unorthodox modes of discourse with a shared desire to embark on an experimental journey that engages artists with an intertextual, multisensory, technological and experiential approach to the event (Dinis 2020). In this context, audio-visual performances are characterised by embracing alternative aesthetics and technological advances, moving away from the traditional emphasis on the performer’s body and extending their reach to other media, particularly sound and image (Dinis 2020). In addition to this extension to other media, live performance is co-constructed by the bodily presence of performers and the audience (Dinis 2020), and performative moments can have a transformative effect as the audience experiences the materiality of a live performance as an ephemeral event (Fischer-Lichte 2019). This materiality of the ephemeral can also be achieved through the inclusion of technological media during the performative moment (Bay-Cheng et al. 2015). It reinforces the performativity of sound and image in which sounds and images negotiate their meaning (Harbison 2019). These experiences with sounds and images remain fundamental in the contemporary context since they are linked to the concept of synaesthesia as a relationship between the senses a concept that is even more present in audio-visual practices due to the capacity of technology to produce simultaneous stimuli for the different senses. The interrelationship between sound and visual media emerges as a mechanism for creating new ways of shaping our perceptions. It fosters dialogical connections that can stimulate audiences to explore different contexts through manipulating sound and image, where one follows the other rather than being seamlessly blended during reception. When sound and image converge, they open up a wider range of potential relationships, creating alternative narratives that, during the performative experience, transcend their individual sonic or visual nature. Instead, they materialise as ephemeral amalgamations facilitated by technological media (Dinis 2020). Sound and visual media develop their combinatory potential, conceptually expanding live audio-visual performances. When sound and image simultaneously engage both senses within a unified aesthetic-narrative context, an articulation emerges that engages the audience and inspires new interpretations (Dinis 2020). The discourse on how live audio-visual performances can stimulate new interpretations and poetic understandings of works of art brings to the forefront questions about the generation of presence effects mediated by technology. Consequently, it becomes imperative to explore these domains of audio-visual mediation through methodologies that facilitate the generation of presence effects during audio-visual performative moments. Starting from a series of artistic practices and audio-visual performative moments that have been developed in recent years1 , we explore and deepen the synergy between sound and image. Our aim is to establish connections and relationships that highlight the underlying concepts, intentions and overall coherence inherent to artistic practices that incorporate these two media (Dinis 2020). 1 Available at: https://fredericodinis.wordpress.com/performance/p-a-r/. This website showcases

the documentation of the research-creation process and the performative moments. Collectively, these materials provide a more profound and illustrative understanding of the trajectories explored in this essay. For this reason, they are integral components of the creative process itself, emphasizing its essence as a reflective practice.

The Production of Effects of Presence in Live Audio-Visual

903

2 The Effects of Presence Over time, the evolution of technology and art has facilitated a convergence of different artistic forms. This convergence has enabled audio-visual artists to create a distinct aesthetic characterised by the fundamental elements of sound, image and ‘in motion’ (Bruno 2022). In this sense, sound and visual media evolve in a combinatory potential with other disciplines, thus conceptually expanding live audio-visual performance. By presenting sound and moving images in confluence, the aesthetic experience of ‘live’ becomes more intense, creating a complementarity between sound and visual means, resulting in an immersive2 experience. The increasing integration of new media and technologies into the performance arts has given rise to diverse artistic forms and practices. This transformation is also evident in the realm of theory, as reflected in the considerable number of anthologies and texts published on the subject in recent decades3 . These approaches typically range from considering the use of new technologies and media as a discontinuity or something 2 Richard Wagner attempted to evoke this particular effect in audiences through his concept of

the Gesamtkunstwerk (total work of art), which has similarities in its underlying principles to the concept of multimedia proposed by Packer & Jordan (2001). 3 These publications include the following: Liveness: Performance in a Mediatized Culture (1999; rev. 2008) by P. Auslander; Remediation: Understanding new media (2000) by J.D. Bolter & R. Grusin; Multimedia: From Wagner to Virtual Reality (2001) by R. Packer & K. Jordon; The New Media Book (2002) by D. Harries; Prefiguring Cyberculture: An Intellectual History (2002) by D. Tofts, A. Jonson & A. Cavallaro; The New Media Reader (2003) by N. Wardrip-Fruin & N. Monfort; Performance and Technology: Practices of Virtual Embodiment and Interactivity (2006) by S. Broadhurst & J. Machon; Intermediality in Theatre and Performance (2006) by F. Chapple & C. Kattenbelt; Theatre and Performance in Digital Culture: From Simulation to Embeddedness (2006) by M. Causey; Postdramatic theatre (2006) by H-T. Lehmann; Digital Performance: A History of New Media in Theater, Dance, Performance Art and Installation (2007) by S. Dixon; Closer: Performance, Technologies, Phenomenology (2007) by S. Kozel; Multi-Media: video–installation–performance (2007) by N. Kaye; A Philosophy of Computer Art (2009) by D. Lopes; New media: A critical introduction (2009) by M. Lister, J. Dovey, S. Giddings, K. Kelly & I. Grant; Entangled: Technology and the Transformation of Performance (2010) by C. Salter; Mapping Intermediality in Performance (2010) by S. Bay-Cheng, C. Kattenbelt, A. Lavender & R. Nelson; Cyborg Theatre: Corporeal/Technological Intersections in Multimedia Performance (2011) by J. Parker-Starbuck; Performing Mixed Reality (2011) by S. Benford & G. Giannachi; Materializing new media: Embodiment in information aesthetics (2011) by A. Munster; Multimedia Performance (2012) by R. Klich & E. Scheer; The Johns Hopkins guide to digital media (2014) by M-L. Ryan, L. Emerson & B. J. Robertson; The Rhythmic Event: Art, Media, and the Sonic (2014) by E. Ikoniadou; Embodied Avatars: Genealogies of Black Feminist Art and Performance (2015) by U. McMillan; Black Performance on the Outskirts of the Left (2017) by M. Gaines; Transmission in Motion: The Technologizing of Dance (2016) by M. Bleeker; Performance in the Twenty-First Century: Theatres of Engagement (2016) by A. Lavender; The Delayed Present: Media-induced Tempor(e)alities & Techno-traumatic Irritations of “the Contemporary” (2017) by W. Ernst; Intermedial Theater: Performance Philosophy, Transversal Poetics, and the Future of Affect (2017) by B. Reynolds; Immersive Embodiment: Theatres of Mislocalized Sensation (2019) by L. Jarvis; Digital Theatre: The Making and Meaning of Live Mediated Performance (2020) by N. Masura; among others.

904

F. Dinis

entirely new incorporated into artistic practices to seeing it as a continuation of the ongoing technological integration in the artistic production process (Dinis 2020). For those who understand this integration as a discontinuity, the use of technology opens the door to the emergence and growth of new artistic possibilities (Saltz 2013). Conversely, other authors see these practices as the unfolding of a pre-existing potential, pointing out that the performance arts have historically used the most advanced technology available at the time to enhance the ‘spectacle’ of productions (Dixon 2007). The various perspectives presented here reflect the development of a significant discussion on the consequences of the incorporation of new media in reshaping artistic expressions and methods. Among the key issues explored in these discussions is the questioning of the concept of presence and its transformation when subjected to technological mediation. This debate seems to establish a (apparent) perceived dichotomy between what is considered ‘live’ and what is ‘mediated’. The development of the artistic movements associated with performance art, within the framework of its historical development and prevailing modes of expression, emphasises the importance of the performer’s presence as an immediately tangible experience. This presence becomes essential in the interaction between the audience and the performer and ultimately serves as the central objective of the performance. In Nightsea Crossing (1981–1987), for example, Marina Abramovic and Ulay (1986) seek to achieve an “absolute state of presence”, defined by Fischer-Lichte (2012: 115) as “radical presence”, or, in other words, “appearing and being perceived as an embodied mind”. In this sense, performance, understood as a unique experience and marked in its ephemeral appearance by its character as a ‘live’ event, seems to share that quality that Benjamin (1969) defined as ‘aura’, through the presence of the artist’s body, itself transformed into an artistic object. The discussion surrounding performance art often highlights its defining attribute, which is the ‘live’ nature, as it cannot be replicated, contrasting it with ‘mediated’ art. This position is advocated by Phelan (1993) when he states: The only life of performance is in the present. Performance cannot be stored, recorded, documented or otherwise participate in the circulation of representations of representations: if it does, it becomes something other than performance. (...) [When] performance attempts to enter the economy of reproduction, it betrays and diminishes the promise of its own ontology. The ’being’ of performance, as the ontology of subjectivity proposed (here), is achieved through its disappearance. (Phelan 1993: 146) The ontology of performance is paradoxically expressed through both presence and absence, which is why performance cannot ultimately fulfil the promise of absolute presence. Auslander (1999) is critical of Phelan’s theory because: (...) we cannot treat the qualities traditionally assigned to ’live’ performance, which supposedly distinguish it from technologically mediated performance, as inherent or ontological characteristics. Rather, they are phenomenologically and historically defined. (Auslander 1999: 108)

The Production of Effects of Presence in Live Audio-Visual

905

In arguing that the concept of ‘live’ is in fact a result of mediation rather than the reverse (Auslander 1999), it becomes clear that historically the notion of ‘live’ only emerged following the development of reproduction techniques. Consequently, ‘live’ defines itself as that which can be captured or recorded, establishing a clear interdependence with and reliance on mediation. For Auslander (1999), Phelan’s assertion is unfounded, (...) not only because it is not clear that ’live’ performance possesses a distinct ontology, but also because it is not a question of performance entering the economy of reproduction, since it has always been there. (...) the very concept of ’live’ performance presupposes that of reproduction, (...) and ’live’ can only exist within an economy of reproduction. (Auslander 1999: 54) Furthermore, Auslander (1999) observes a growing tendency to progressively incorporate mediation into ‘live’ events and provides an understanding of why performance theory discourse insists on positioning performance as primarily ideological rather than as a mediated or technologically influenced art form. It’s important to recognise that despite the emphasis on performance as an event that occurs in real-time, performance is closely intertwined with various mediating media and processes, particularly through the use of images. However, for those who value the ‘live’ aspect of performance, mediation is seen as a factor that diminishes the presence of the performer. But this weakness is not always seen as a negative. Fuchs (1985) cites as an example the Mabou Mines collective’s production of Beckett’s Come and Go (1975) because it “shattered the usual expectations of physical presence and contact between actors and audience” (Fuchs 1985: 164). For the author, the theatre at that moment was characterised as a “theatre of absence”, characterized by the “failure of the theatrical enterprise of the spontaneous word with its logocentric claims of origin, authority, authenticity, in short, of Presence” (Fuchs 1985:172). The increasing role of mediation in the performance arts, particularly in the 1990s, signified the postmodern shift away from the concept of presence in the realm of theory. This shift involved relocating the central subject and fragmenting narratives, which eventually undermined the authoritative status of textual works. Another approach can be found in the work of The Wooster Group, where the actors do not construct traditional characters, but instead engage in actions that draw on their individual characteristics. This method creates a unique persona, and a deliberate strategy with the aim of “dismantling the notion of presence and challenging the established structures of authority within the performance” (Auslander 1994: 47). Furthermore, the deconstruction of presence can also be attributed to the collective’s conceptual methodology. The depersonalisation aspect is also noteworthy. Rather than relying on the direct representation of a pre-existing dramaturgical text, they choose to craft their own texts and performance scripts, thus contributing to the dismantling of traditional concepts of presence4 . 4 These works are often the product of combining, collaging, and arranging various components,

such as moving images, computer programming, lighting, sound, and dance. This results in a complex and fluid network where text, media, and performance converge.

906

F. Dinis

The employment of new media to deconstruct presence has gained increased support, as proposed by authors such as Pontbriand (1982), Fuchs (1985), Auslander (1999), and Féral (1985, 1992), due to the growing recognition of the experience of presence in theatre, performance, and the arts. This recognition highlights the essential role of technological mediation in the production of presence. As Pontbriand (1982: 155–156) points out “presence no longer depends on materiality, but on the display value of the work of art, its multiplicity and its accessibility”. Giannachi & Kaye (2011: 3) challenge the “dominant view in performance studies of the relationship between performance and technology, which associates ‘mediating’ technologies with the displacement and elimination of the performer’s ‘presence’”. Presence, understood as ‘being before’, ‘in front of’ something or someone who is ‘other’, always involves a sense of multiplicity and otherness and unfolds in the dynamic interplay between the creation and execution of a performance, as well as in the reception of that presence. For these authors, “presence is best observed as a doing [to perform], produced in the act, and not as a function of a particular medium or in relation to an ‘intrinsic’ value of certain modes of representation, or simply as a privileged term and thus in opposition to absence” (Giannachi & Kaye 2011: 19–20). The increasingly dominant role of technology in performance is also questioned by Cusack (2007): In a theatrical environment where cutting edge technology can now realistically represent, on an unprecedented level, anything the theatrical author imagines, what happens to the oldest component of theatre: the body and the living presence of the performer? (Cusack 2007: 52) This interrogation presupposes the idea of the performer’s live body as an immediate presence, and therefore of greater importance, rather than the view that a body on stage is always a sign, a representation, that is, a body that is always mediated. Therefore, the central question doesn’t concern the transformation of the performer’s physical body and immediate presence, but, as articulated by Weems (2016), it concerns the challenge of reimagining what ‘live’ means in a deeply interconnected digital world, where experiences of presence are being increasingly intermediated for both performers and audiences. Presence is not limited to a certain form of media or a single living organism. Rather, it arises from performative occurrences that can occur in both live and mediated contexts (Giannachi, Kaye & Shanks 2012). And the relationship between performative, live and mediated moments can be characterised according to a set of characteristics that define the concept of multimedia (Packer & Jordan 2001), namely: integration, interactivity, hypermedia, immersion and narrativity (Dinis 2020). Technological tools in artistic expression correspond with Wagner’s notion of a total work of art, enabling the amalgamation of multiple artistic forms within one piece. New media promotes discussion and connectivity between different artistic fields. While functioning as supplemental means for reproducing artistic creations, these technological media, such as audio-visual or multimedia, simultaneously evolve into distinct and independent art forms. Their link to other arts is mainly through their shared origins,

The Production of Effects of Presence in Live Audio-Visual

907

and as such technological media, especially audio-visual, should be seen as the bridge between artistic objects rather than their final artistic aim. Audio-visual creations, enabled by their medium, act as extensions of an individual’s emotions and senses, allowing the opportunity to experience sensations that may otherwise be unavailable. The role of performers is demonstrated through the intricate interplay between live and mediated performances, in which a delicate balance is struck between the isolation of physical bodies and the encounter with technology. This presence is interwoven intricately into the interdependence and complementarity of physical and mediated presences. It is not mutually exclusive or antagonistic to these entities. The dynamics of this relationship become apparent during the performance and the reception of the presence in the performative moments.

3 The Performative Moments After exploring the connection of presence in a mediated performance, it is essential to reflect on the notions of performative space and time within the realm of performative practice. According to Goldberg (2007), (...) a work of art could completely free itself from architecture [or space]. (...) [Since] this kind of work should change the viewer’s perception of the museum landscape and the urban landscape, it should challenge [the public] to question the situations in which they normally view art. (Goldberg 2007: 196-197) In this context, any space where an action can occur in the presence of at least one person can be deemed a performative space (Alvarez 2004; Rancière 2010). However, a detailed understanding of the notion of performative space, especially with regard to its temporal aspects, is essential in order to truly qualify it as such when it comes to performance. The experience of space, both inside and outside the ‘theatrical’ building, operates from two conceptions related to the performative space: (i) space perceived as an empty space waiting to be occupied, and (ii) space perceived as invisible, boundless and intertwined with its participants, characterised by coordinates, movements and paths. In the second conception, space is not something to be filled, but rather to be extended and explored (Dinis 2020). These two nearly opposing concepts of space correspond to two distinct approaches in defining it: ‘the external objective space’ and ‘the gestural space’ (Pavis 2003). Pavis (2003) considers ‘the external objective space’ as a visible, frontal and often fillable and descriptive space. In addition to this space, the author describes what he calls ‘the gestural space’, defining it as the space created by the presence, scenic position and displacements of the performers, that is, a space projected and outlined by the performer, induced by his presence and an evolving space that can expand or contract. In addition to the conceptions of ‘performative space’, expressed in the concepts of ‘external objective space’ and ‘gestural space’ (Pavis 2003), it is also important to highlight the diversity of alternative performative spaces and the transformation of spaces originally designed for different purposes into performance venues. These spatial concepts are closely linked with their own exclusive terminology, in which terms such as

908

F. Dinis

performance and improvisation expand the parameters of what is considered a performance space (Artaud 1996; Brook 2008; Rancière 2010; Ikoniadou 2014; Harbison 2019). This opens up new possibilities, recursive procedures, repetitions, non-linear frameworks, simultaneous events and the fusion of different languages. In this context, time, performance space, and the interaction between performer and audience are intimately linked, allowing for greater creative expression. This collaboration frequently involves multiple media working together and exploring new geographic and territorial dimensions, ultimately creating new performative experiences (Dinis 2020). These performative moments are self-contained expressions that represent the intersection between performers and their audience through artistic work and engagement. Therefore, live audio-visual performances represent the moment itself as a form of artistic expression. Although technology allows for endless repetition, the performative moment is inherently distinct and singular, comparable to a transient encounter. For, as Dinis (2020) emphasises: This quality of unrepeatability is not the result of the random or the unexpected, as is often the case between rehearsed and improvised performances. This quality results from the unrepeatable dynamic between the audience and the performers, in the way the audience reacts (even emotionally) to the sound and visual work of the performers, and also to the dynamic between the performative space, the moment, and the performer himself. (Dinis 2020: 132) The nature of the performative moment is significantly affected by the characteristics of the space. The range of venues that host these transient practices is diverse, including galleries, theatres, cinemas and museums as well as outdoor settings, discovered spaces and other locations temporarily converted into artistic contexts. However, it is relevant to note that space is not limited to physical presence, as many events take place for an audience that is virtually present (Dinis 2020). During live audio-visual performances, sound and image merge in a coherent expressive process that shapes the generation of presence-related effects and reinforces the representation of the artwork in the minds of the audience. The synergy between sound and image also stimulates the emergence of new narratives, making them richer and more immersive in artistic representation. This phenomenon illustrates the interconnection between space and time, creating new possibilities for performance to be explored and experienced within the realm of live audio-visual performances.

4 Conclusions The concept of performance has been interpreted through many different lenses, and shaped by a variety of disciplinary perspectives, artistic fields and cultural backgrounds. The inherent flexibility of this concept and the variety of creative approaches involved make performance, as an action presented before an audience, an ideal place for exploration, especially when sound and visual media converge. This amalgamation of elements offers significant potential for experimentation and innovation.

The Production of Effects of Presence in Live Audio-Visual

909

The integration of technology into different media has facilitated the emergence of performance as a prominent mode of artistic expression in recent decades. This development has led to a departure from the traditional focus on the body and performer, leading to an exploration of other media, including sound and image. Within the context of decentralization, this essay examines the relationship between performer and spectator, drawing insights from a range of artistic practices and audio-visual performative moments that have been developed in recent years, all mediated through sound and image. Through this exploration of audio-visual mediation, we have observed that presence is generated in live performative instances and is not tied to a specific medium or physical body. Sound and image act as key expressive mechanisms in the production of these manifestations of presence, contributing significantly to the production of meaning and the articulation of artistic intent. Therefore, audio-visual media ought to be viewed as bridges that interconnect artistic elements, rather than as the ultimate artistic destination. They enable exchange and interaction between distinct artistic domains, fostering a convergence of creative realities. Performative moments catalyse the emergence of new opportunities, iterative processes, non-linear frameworks, simultaneous occurrences, and the fusion of artistic languages. In this context, time, performative space, and the performativity shared between performer and spectator interconnect with greater freedom enabled by the collaboration of multiple media. During live audio-visual performances, sound and image converge flawlessly as instruments for expression, moulding the design of presence effects and enhancing the audience’s perception of the work of art. The coherence between sound and live visuals also triggers the generation of new kinds of narratives, adding depth and immersive qualities to the artistic representation. This interaction reflects the interdependence of space and time, leading to new ways of exploring performance, which are actively developed and experimented with in live audio-visual presentations. In this context, audience engagement goes beyond everyday physical limitations, resulting in mechanisms that are in constant motion. It blurs supposedly fixed boundaries and utilizes bodily experiences to catalyse spatial exploration. This crossing of boundaries, observed during the performative moment, brings together sound and image in a tangible expression of ephemerality made possible through the use of technological tools. Acknowledgement. This work is funded by national funds through the FCT – Fundação para a Ciência e a Tecnologia, I.P., in the scope of the project UIDB/00460/2020.

References Alvarez JC (ed) (2004) Museu Nacional do Teatro - Roteiro. IMC, Lisboa Artaud A (1996) O Teatro e o seu Duplo. Fenda, Lisboa Auslander P (1994) Presence and resistance. University of Michigan Press, Michigan Auslander P (1999) Liveness: performance in a mediatized culture. Routledge, New York

910

F. Dinis

Bay-Cheng S, Parker-Starbuck J, Saltz DZ (2015) Performance and media: taxonomies for a changing field. University of Michigan Press, Ann Arbor Benjamin, W.: Das Kunstwerk im Zeitalter seiner technischen Reproduzierbarkeit. Suhrkamp, Frankfurt am Main (1969) Brook P (2008) O Espaço Vazio. Orfeu Negro, Lisboa Bruno G (2022) Atmospheres of projection: environmentality in art and screen media. University of Chicago Press, Chicago Carlson M (2004) What is performance? In: Bial H, Brady S (eds) The Performance Studies Reader. Routledge, New York, pp 68–72 Cusack T (2007) Is it live?. Stage Directions Nov, pp 52–54. https://goo.gl/HDbYDm. Accessed 28 April 2023 Dinis F (2020) Sinuous Sensations Hypnotic Emotions: Contemporary sound and visual performance (PhD Thesis). Faculty of Arts and Humanities, University of Coimbra, Coimbra Dixon S (2007) Digital performance: a history of new media in theater, dance, performance art, and installation. The MIT Press, Cambridge Féral J (1985) Performance et Théâtralité, le Sujet Desmistifié. In: Théâtralité, Écriture et Mise en Scène, pp 125–140. Hurtubise, Quèbec Féral J (1992) What is left of performance art? Autopsy of a function. Birth of a genre. Perform Issue(s) Happen Body Spectacle, Virt Real 14(2):142–162 Fischer-Lichte E (2012) Appearing as embodied mind: defining a weak, a strong and a radical concept of presence. In: Giannachi G, Kaye N, Shanks M (orgs), Archeologies of presence: art, performance and the persistence of being, pp 103–118. Routledge, New York Fischer-Lichte E (2019) Estética do Performativo. Orfeu Negro, Lisboa Fuchs E (1985) Presence and the revenge of writing. Perform Arts J 9(2/3):163–173 Giannachi G, Kaye N (2011) Performing Presence: Between the live and the simulated. Manchester University Press, Manchester Giannachi G, Kaye N, Shanks M (eds) (2012) Archaeologies of presence: art, performance and the persistence of being. Routledge, New York Goldberg RL (2007) Performance art: from futurism to the present. Thames and Hudson, London Harbison I (2019) Performing image. MIT Press, Cambridge Ikoniadou E (2014) The rhythmic event: art, media, and the sonic. MIT Press, Cambridge Packer R, Jordan K (eds) (2001) Multimedia: from Wagner to virtual reality. Norton & Company, New York Pavis P (2003) A análise dos espetáculos. Perspetiva, São Paulo Phelan P (1993) Unmarked: the politics of performance. Routledge, New York Pontbriand C (1982) The eye finds no fixed point on which to rest. Modern Drama 25(1):154–162 Rancière J (2010) O Espectador Emancipado. Orfeu Negro, Lisboa Saltz D (2013) Media, technology, and performance. Theatre J Baltimore 65:421–432 Wardrip-Fruin N (2006) Expressive processing: on process-intensive literature and digital media (PhD Thesis). Brown University, Providence, Rhode Island Weems M (2016) Art Talk: an evening with Marianne Weems. The New School. https://youtu.be/ MnZtnmOSpwk. Accessed 28 April 2023

Perceptions About the Impact of Audiovisual Storytelling Campaigns on Young Consumers: An Exploratory Study with Higher Education Students Teresa Gouveia1(B)

, Ana Oliveira2 , Nídia Salomé Morais1

, and Rui Raposo3

1 Centro de Estudos em Educação e Inovação, Escola Superior de Educação, Politécnico de

Viseu, Viseu, Portugal {tgouveia,salome}@esev.ipv.pt 2 Instituto Politécnico de Viseu - Escola Superior de Educação, Rua Maximiano Aragão, 3504-501 Viseu, Portugal [email protected] 3 Departamento de Comunicação e Arte, Universidade de Aveiro, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal [email protected]

Abstract. As society advances and changes, so do its perceptions and behaviours. The quality of a product or service and its price are no longer the only determining factors in the purchase decision. It is increasingly evident in the consumer’s search for brands that match their values. Thus, storytelling emerges as a persuasive technique that excels in the transmission of emotional narratives that arouse feelings in the consumer. In this sense, this study aims to understand the influence of audiovisual storytelling on the behavior of higher education students: in their consumption practices and their perceptions of brand awareness. The study of a quantitative nature made use of a questionnaire survey that obtained answers from 92 students at the School of Education of Viseu, belonging to the 3 years of the Degree in Advertising and Public Relations, the Degree in Social Communication and the Degree in Plastic Arts and Multimedia. The results obtained allow us to understand that, in its generality, audiovisual storytelling exerts a positive influence on the young consumer, having a relevant role in their consumption practices and generating trust and identification with the brands. The analysed data seem to indicate that the use of this technique leads to the attribution of meaning to the brand, leading, consequently, to an increase of notoriety in the consumer’s perception. Keywords: Audiovisual Storytelling · Students · Branded Content

1 Introduction By embracing cultural currents and societal standards, companies endeavour to connect with selective patrons who crave authenticity. Connecting the dots between the brain and brand experiences requires our primary focus. Emerging within this paradigm, branded © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 911–921, 2024. https://doi.org/10.1007/978-3-031-47281-7_73

912

T. Gouveia et al.

content represents an innovative tactic to pique curiosity gently. Instead, of endowing consumers with information, entertainment, or amusement (Orozco et al. 2015), the communication involved should surpass just presenting a value proposal. Integration becomes an art form when combining brands with compelling narratives, making them irresistible to consumers (Araújo 2018). A tailor-made plan should be developed bearing in mind the intricacies and sensitivities of the target population and how they relate to the particular setting in which they reside (Massarolo & Mesquita 2014). Brands build strong ties by leveraging content and storytelling techniques; consumers view them favourably as dependable entities (Rocha 2019). According to Reske & Souto (2018) and Garcia (2017), enterprises leverage storytelling and branded content to provide unique brand encounters. Content that offers value beyond direct sales can help create a more enduring bond between brands and their audiences, as observed in the research of Carim & Alves (2014) and Covaleski & Siqueira (2017). By examining the research findings, the author uncovers the influence of audiovisual storytelling on the conduct and viewing propensities of college students. Determining how audiovisual storytelling campaigns affect the emotions of young consumers, along with identifying the most engrossing aspects of these campaigns, was the primary focus.

2 Brand Content and Audiovisual Storytelling As brands increasingly interact with society and culture, they incorporate a wide range of experiences and lifestyles. This shift places a strong emphasis on emotions, which play a vital role in building meaningful psychological connections and crafting unforgettable brand experiences. Consequently, consumers begin to form expectations and interests related to these brands (Gomez et al. 2011). This contemporary approach to communication, championed by consumer goods companies, revolves around the consumer as the central figure. It creates an opening for branded goods, and within this framework, branded content transforms its messaging from being deceptive to genuinely exciting. Communication now strives to provide information that goes beyond the mere product or service, sparking genuine interest among consumers (Orozco et al. 2015). The concept of brand content encompasses a diverse array of forms, all aimed at fostering trust and nurturing enduring relationships between brands and consumers (Hardy 2017). The objective behind developing products associated with a brand is to incorporate qualities that extend beyond mere utility. It involves restructuring the communication process with the audience, inviting them into a narrative that transcends the products or services themselves (Covaleski 2015). Branded products emerge from the desire to blend advertising with entertainment, forging strong brand-consumer connections. This can take various forms, including interactive social media games, tailor-made shows, short films, and captivating stories, all enabling brands to craft compelling communication strategies (Covaleski & Siqueira 2017). At the core of successful branded content lies the ability to capture the audience’s attention. Storytelling, with its intimate and relatable appeal, fosters deep and meaningful connections among audiences (Bonsignore & Sassoon 2014). Integrating products or services seamlessly into narratives effectively captures and engages the consumer audience (Araújo 2018). Storytelling, as a structured communication technique, artfully stirs

Perceptions About the Impact of Audiovisual Storytelling Campaigns

913

emotions by presenting conflicts and revealing profound truths (Núñez 2007). It can be approached from various angles, be it pragmatic, pictorial, or poetic, where technique and art beautifully merge (Xavier 2015). These strategies must align with the unique characteristics of each communication medium (Massarolo & Mesquita 2014). In today’s ever-evolving landscape, advertising faces the challenge of adapting to technological advancements and meeting the demands of a more discerning and engaged audience. This shift, driven by factors such as media fragmentation, consumers’ appetite for intricate and innovative content, and shortened attention spans, necessitates the creation of content capable of effectively competing (Tomaz 2019). Overcoming this attention crisis presents significant obstacles, as the sheer volume of advertising messages and the risk of oversaturation can lead to unintended consequences, misrepresenting campaign ideals and corporate values (Ribaric 2014). Television advertising, with its extensive reach, remains an influential medium, leveraging the power of audiovisual formats to convey messages effectively (Fank 2017). To adapt to the growing demand and cost pressures, advertisers have embraced shorter commercial durations, resulting in more concise and focused messaging (Vieira 2009). Emotionally engaging consumers in this environment has led to the widespread adoption of storytelling (Garcia 2017). By incorporating specific aspects of the audience’s reality and seamlessly integrating them into the advertising context, storytelling bridges the gap between consumers’ daily lives and the narratives presented in advertisements (Camargo 2013). This narrative construction artfully employs rhetorical devices and subjective shots to mirror an optimistic world, aligning with consumers’ social and cultural contexts and their pursuit of well-being (Ribaric 2014). To address this, advertisers must prioritize consumer aspirations throughout the development of advertising campaigns, ensuring a close alignment between the audience and the brand (Araújo 2018). In this context, the audiovisual format plays a pivotal role in storytelling, enabling the seamless integration of imagery, sound, and movement to enhance message retention and comprehension (Vieira 2009). Paired with the art of storytelling, this technique serves as a powerful communication tool for conveying a brand through narrative (Reske & Souto 2018). Storytelling, as a branding strategy, aims not only to forge a connection with the product but also to establish a profound identification with the brand itself (Carim & Alves 2014). This technique effortlessly captures attention by evoking emotions and inviting audiences to delve into the subtle nuances of the narrative (Ribaric 2014; Fank 2017). Through storytelling, brands can create emotional bonds and foster consumer engagement, effectively addressing latent needs while solidifying their brand identity (Rocha 2019). This results in a brand experience that deeply resonates with consumers, leading to a lasting impact and strengthening the brand’s position in the hearts and minds of its target audience.

3 Study and Participants The primary objective of the study presented in this article was to investigate the influence of audiovisual storytelling on the consumption behaviours of young individuals within a Higher Education Institution in Portugal.

914

T. Gouveia et al.

Methodologically, this research is appropriately characterized as an exploratory study, given its intended purpose of acquiring information and insights to enhance the understanding of the research problem at hand. Specifically, the study sought to achieve the following objectives: (1) Identify the decisive factors that impact the purchasing decisions of young consumers, with a focus on discerning the role of audiovisual storytelling among these factors; (2) Analyse the emotional responses elicited in Higher Education students by audiovisual storytelling campaigns; (3) Enumerate the characteristics of these campaigns that most effectively stimulate these responses, to assess their genuine impact on consumer behaviour; (4) Examine whether audiovisual storytelling campaigns exert any discernible influence on consumption practices, including heightened trust in a brand or influencing the choice of one brand over another. Following an initial phase dedicated to an in-depth examination of pertinent concepts and information about the study’s theme, which facilitated the development of the theoretical framework and the rationale behind methodological choices, the study progressed to the meticulous selection of storytelling campaigns. Subsequently, the construction of the data collection instrument ensued. The audiovisual storytelling campaigns subjected to analysis in this study were those conducted by the three predominant telecommunications operators in Portugal, namely, Vodafone Portugal, NOS, and MEO, during the Christmas season of 2019. Specifically, the campaigns chosen for scrutiny included “Neste Natal abra o presente” (2019), “A descoberta” (2019), and “MEO || Natal” (2019), as depicted in Fig. 1.

Fig. 1. Images from the analysed campaigns.

Having these campaigns as a reference, we proceeded to implement an online questionnaire survey organized into the following sections: (1) Definition of the consumer profile; (2) Exploration of the decisive factors to consumption; (3) Visualization of audiovisual storytelling campaigns associated to the main telecommunication operators in Portugal; (4) Impact of the audiovisual storytelling campaigns of telecommunication operators on the consumer. The study participants consisted of 92 students from the School of Education at the Polytechnic Institute of Viseu, enrolled in bachelor’s degree programs in Advertising and Public Relations, Social Communication, and Plastic Arts and Multimedia for the academic year 2019/2020. In terms of gender distribution (as shown in Table 1), 70 students identified as female (76.1%), while 22 identified as male (23.9%). The most represented age group in the study ranged between 17 and 22 years of age, accounting for 84 students (91.3%), followed by those aged 23 to 27, totalling 8 students (8.7%). As for the place of residence of the respondents, the majority hailed from the city of Viseu

Perceptions About the Impact of Audiovisual Storytelling Campaigns

915

(36 students – 39.1%), followed by students who, while not residing in the city itself, belonged to the broader district of Viseu (19 students – 20.7%). Table 1. Participants’ characterization.

Gender Age Region

Course

n

%

Female

70

76.1

Male

22

23.9

17–22 years old

84

91.3

23–27 years old

8

8.7

Viseu (City)

36

39.1

District of Viseu

19

20.7

Other Locations

37

40.2

Social Communication

67

72.7

Advertising and Public Relations

18

19.6

Plastic Arts and Multimedia

7

7.6

The majority of participants, specifically 67 students (72.8%), indicated their enrolment in the Degree program in Social Communication. This was followed by the Degree program in Advertising and Public Relations, with 18 students (19.6%), while the Degree program in Plastic Arts and Multimedia had the lowest representation, comprising 7 students (7.6%). Regarding their academic year, the distribution was as follows: 38 students (41.3%) were in their 2nd year, followed by 28 students (30.4%) in their 3rd year, and 26 students (28.3%) in their 1st year.

4 Results 4.1 Visualization of Audiovisual Storytelling Campaigns In the section of the questionnaire titled “Visualization of Audiovisual Storytelling campaigns associated with the primary telecommunication operators in Portugal,” respondents were provided with the opportunity to view the three selected campaigns, alluding to the Christmas season of 2019, orchestrated by the leading telecommunication operators in Portugal, namely, Vodafone Portugal, NOS, and MEO. The inclusion of these campaigns within the questionnaire served the purpose of acquainting the respondents with storytelling in practical application. This inclusion ensured that, in the event of respondents not being familiar with these campaigns, they would have the opportunity to gain insight into the specific campaigns in question. Upon examination of Table 2, it becomes evident that a significant proportion of the respondents had prior knowledge of the three storytelling campaigns under scrutiny. More specifically, 75 respondents (81.5%) were already acquainted with the Vodafone

916

T. Gouveia et al.

Portugal campaign, 62 students (67.4%) were already familiar with the NOS campaign, and 55 respondents (59.8%) were likewise versed in the MEO campaign. These findings, which underscore a pre-existing awareness of the selected campaigns, align with expectations, considering the advertising strategies commonly employed by Portuguese telecom operators, particularly during the Christmas season, and their widespread presence across diverse online platforms. Table 2. Familiarity with the Audiovisual Storytelling campaigns analyzed. n

%

Yes

75

81.5

No

17

18.5

Already knew MEO’s campaign?

Yes

55

59.8

No

37

40.2

Are you familiar with the NOS campaign?

Yes

62

67.4

No

30

32.6

Did you know the Vodafone Portugal campaign?

While the questionnaire encompassed the option for participants to view the designated campaigns, the ensuing results illuminate that these audiovisual storytelling campaigns had already registered within the consciousness of the students. They were not only recognized but also recalled by the participants. Given this context, it was deemed pertinent to ascertain the emotional responses elicited by these campaigns among the respondents, as illustrated in Fig. 2.

Fig. 2. Word cloud about the feelings associated with the campaigns.

As evident from the results presented in Table 3, the most frequently cited emotional response by the students was the experience of emotion itself, mentioned a total of 151 times. This was closely followed by instances of the respondents expressing empathy towards the brand that had orchestrated the campaign, with 80 mentions. Additionally, a notable number of students indicated experiencing feelings of well-being (70 mentions)

Perceptions About the Impact of Audiovisual Storytelling Campaigns

917

and nostalgia (68 mentions). The sentiment of hope was expressed 66 times, while satisfaction was conveyed 62 times. It is worth emphasizing that evoking empathy towards the brand associated with the campaign aligns with one of the core objectives of a branding strategy underpinned by storytelling. Such a strategy seeks to enhance the brand’s image within the public consciousness. By eliciting emotions such as emotion itself, empathy, and well-being, the branding strategy, enriched by storytelling, engenders an association wherein the brand, when recalled by the consumer, becomes intertwined with positive sentiments, thus augmenting its overall image. Less frequently mentioned were feelings related to admiration (48 mentions) and enthusiasm (29 mentions) concerning the brand linked with the campaign. These sentiments, while less prevalent, also play a role in shaping the consumer’s perception of the brand and contribute to the nuanced emotional landscape evoked by storytelling-driven branding strategies. Table 3. Feelings associated with the campaigns under analysis. Vodafone Campaign

NOS Campaign

MEO Campaign

Emotion

57

52

42

151

Empathy

30

27

23

80

Well-being

19

29

22

70

Nostalgia

27

25

16

68

Hope

19

26

21

66

Satisfaction

17

19

26

62

Admiration

15

16

17

48

Enthusiasm

3

13

13

29

Upon analysing the data in the previous table, which looks into brands and the associated emotions, it becomes evident that Vodafone and NOS played a pivotal role in evoking the emotion of “emotion.“ They garnered 57 and 52 mentions, respectively, indicating their prowess in connecting with consumers on an emotional level. Additionally, both Vodafone and NOS excelled in eliciting empathy, with Vodafone receiving 30 mentions and NOS 27. In the realms of “well-being” and “hope,” the NOS campaign stood out, amassing 29 and 26 mentions, respectively. When it comes to invoking nostalgia, Vodafone and NOS demonstrated a neck-and-neck performance, with 27 and 25 mentions, respectively, from the students. In contrast, MEO’s storytelling campaign set itself apart by receiving 26 mentions for its ability to convey “satisfaction” to the participants. Interestingly, all three operators consistently inspired “admiration.“ However, Vodafone’s campaign seemed to generate the least “enthusiasm” among students. Audiovisual storytelling is characterized by its adept integration of various elements, each meticulously designed to collectively engage consumers. In this context, critical factors such as “Emotional Involvement,” “Script/Narrative,” “Soundtrack,” “Event Triggering,” “Characterization,” and “Visual Composition (including elements like colours

918

T. Gouveia et al.

and settings)” play a pivotal role in captivating the audience. To gain deeper insights into these attributes, one of the questionnaire items aimed to pinpoint the distinctive features of audiovisual storytelling campaigns by Portuguese telecom operators that resonated most with respondents. The data, as elucidated in Table 4, underscores the prominence of “Emotional Involvement,” with 166 mentions, closely followed by “Soundtrack” with 127 mentions, and “Script/Narrative” with 122 mentions. Table 4. Characteristics of the audiovisual storytelling campaigns. Campaign Vodafone

Campaign NOS

Campaign MEO

Emotional Involvement

73

50

43

166

Script/Narrative

34

39

49

122

Soundtrack

42

46

39

127

Event Trigger

26

38

27

91

Characters

17

33

25

75

Picture Composition

35

28

23

86

Other Options

1

1

3

5

Undoubtedly, the soundtrack emerges as one of the most conspicuous elements when it comes to infusing emotional resonance into audiovisual content, often wielding the power to define the prevailing mood elicited by these campaigns. Simultaneously, the script and narrative structure wield significant influence over the success of an audiovisual storytelling campaign. These elements serve as the narrative backbone, dictating the trajectory of the story. Their meticulous and strategic construction assumes pivotal importance in facilitating the triumph of a branding strategy underpinned by storytelling. Additional components such as the orchestration of pivotal events, the delineation of compelling characters, and the meticulous composition of visual elements also impact respondents’ perceptions. While these facets garnered comparatively fewer mentions, they nevertheless constitute integral components of any audiovisual storytelling campaign. However, they might appear inconspicuous to consumers due to their subtle nature. In contrast, it is worth noting that some respondents indicated not receiving any discernible stimulus from the presented storytelling campaigns. In essence, these observations emphasize the multifaceted nature of audiovisual storytelling, underscoring that its success is contingent upon a harmonious orchestration of various elements, each contributing to the overall impact experienced by the audience. 4.2 Influence of Audiovisual Storytelling on Brand Perception Within the purview of this research, it was deemed relevant to explore whether audiovisual storytelling campaigns foster sentiments of trust towards the telecommunication operators under examination and whether this perceived sentiment influences the selection of a particular operator over its competitors. A review of the findings presented in

Perceptions About the Impact of Audiovisual Storytelling Campaigns

919

Table 5 enables the following conclusions: A total of 61 respondents (66.3%) acknowledge that audiovisual storytelling campaigns indeed cultivate a sense of trust in Portuguese telecom operators. However, it is noteworthy that 20 students (21.7%) do not experience increased trust in telecom operators due to audiovisual storytelling. Furthermore, 11 students (12%) remain undecided, lacking a definitive perception regarding whether audiovisual storytelling campaigns engender a sense of trust. These insights shed light on the nuanced role that storytelling plays in shaping consumer trust dynamics within the telecommunications sector. Table 5. Students’ perceptions about audiovisual storytelling campaigns.

Audiovisual storytelling campaigns create a feeling of trust towards the telecom operator that promotes them

Yes

n

%

61

66.3

No

20

21.7

Don’t know

11

12.0

Audiovisual storytelling campaigns are a Yes factor with weight in the decision to choose No a telecom operator over the competition Don’t know

44

47.8

This type of audiovisual storytelling Yes campaign influences the consumption No practices of young people in my age group Don’t know

35

38.0

13

14.1

49

53.3

18

19.6

25

27.2

In light of the findings acquired, as elucidated in Table 5, it is discernible that while 61 respondents acknowledge an augmented sense of trust in telecom operators, only 44 of them (47.8%) affirmed that storytelling campaigns significantly factor into their choice of one operator over others. A substantial subset of participants, comprising 35 students (38%), does not accord these campaigns significant importance during decision-making moments, while 13 students (14.1%) marked the response option “Don’t know.“ These observations highlight the complex interplay between storytelling campaigns and the decision-making processes of consumers within the telecommunications sector. Moreover, within a broader context, this research sought to evaluate whether audiovisual storytelling campaigns exert an influence on the consumption practices of young individuals within the same age group. As per the results elucidated in Table 5, a majority of respondents, specifically 49 individuals (53.3%), perceive an impact of audiovisual storytelling campaigns on the consumption behaviours of their peers within the same age bracket. In contrast, approximately 25 respondents (27.2%) express uncertainty regarding their response to this question, while 18 respondents (19.6%) assert that audiovisual storytelling exerts no discernible influence on the consumption practices of young people within the same age cohort. These findings illuminate the intricate dynamics surrounding the influence of storytelling campaigns on the consumption behaviours of young individuals, underscoring the nuances inherent in this relationship.

920

T. Gouveia et al.

5 Conclusions This article presents a study investigating the impact of audiovisual storytelling on the consumption behaviour of young students in a Portuguese Higher Education Institution. As an exploratory study, our research aimed to identify key factors influencing the purchase decisions of young consumers and analyse the emotional responses elicited by audiovisual storytelling campaigns. We also examined the campaign attributes that resonated most with students, assessing their impact on consumer behaviour, specifically within the context of Portuguese telecom operators: NOS, Vodafone Portugal, and MEO. Our study unequivocally confirms that audiovisual storytelling campaigns significantly influence consumption practices, fostering increased trust in brands and influencing brand selection. The study involved 92 students from the School of Education at the Polytechnic Institute of Viseu, predominantly female, aged 17 to 22. Through exposure to telecom operators’ 2019 Christmas campaigns, participants gained first-hand experience of storytelling’s impact on consumption decisions. The findings reveal that audiovisual storytelling positively transforms the consumption practices of young students. Participants consistently reported heightened excitement, empathetic connections with brands, and an overall sense of well-being when engaging with these campaigns. Campaigns characterized by emotional engagement, quality soundtracks, and well-structured narratives were most appealing. The study also observed an increase in brand trust and a preference for specific brands. However, it’s important to note that this study, limited to a specific institution in Portugal, offers valuable insights but represents only a partial view. Future research should explore the efficacy of audiovisual storytelling across different age groups and consumption contexts, providing a more comprehensive understanding of its multifaceted influence. Acknowledgements. This work is funded by National Funds through the FCT - Foundation for Science and Technology, I.P., within the scope of the project Refª UIDB/05507/2020. Furthermore, we would like to thank the Centre for Studies in Education and Innovation (CI&DEI) and the Polytechnic of Viseu for their support.

References Araújo M (2018) Publicidade audiovisual e a intertextualidade com o cinema: um estudo de recepção publicitária (dissertação). Universidade Federal de Pernambuco, Recife. https://rep ositorio.ufpe.br/handle/123456789/35883 Bonsignore P, Sassoon J (2014) Branded Content. La nuova frontiera della comunicazione d’impresa (F. Angeli (ed.)). https://www.francoangeli.it/Ricerca/Ricerca_collana_Libri.aspx? Collana=640 de Camargo HW (2013) Mito e filme publicitário: Estruturas de significação. SciELO-EDUEL Carim M, Alves G (2014) Storytelling E O Histórico Do Filme Publicitário: Análise Do Comercial “Always #Likeagirl.” Revista Comunicando, vol 3, pp 203–219. https://docplayer.com.br/ 10279664-Storytelling-e-o-historico-do-filme-publicitariouma-analise-do-comercial-familiamoderna-1

Perceptions About the Impact of Audiovisual Storytelling Campaigns

921

Costa M (2013) Narrativas transmidiáticas como estratégias publicitárias. Cadernos de Comunicação 17(1):211–240. https://periodicos.ufsm.br/ccomunicacao/article/view/10897 Covaleski RL (2015) Conteúdo de marca e entretenimento: narrativas híbridas. Comunicação Mídia e Consumo 12(34):107. https://doi.org/10.18568/1983-7070.1234107-123 Covaleski RL, Siqueira OAS (2017) Conteúdo de Marca Audiovisual e regimes interacionais: reflexões sobre o engajamento digital do consumidor. Intercom: Revista Brasileira de Ciências Da Comunicação, 40(2):61–75. https://doi.org/10.1590/1809-5844201724 Frank M (2017) Persuasão através da emoção: O Storytelling como recurso para animações publicitárias (dissertação). UPF - Faculdade de Artes e Comunicação (FAC).http://repositorio.upf. br/handle/riupf/1028 Garcia TP (2017) Storytelling na Publicidade On-line: Análise de engajamento do vídeo The New Volvo XC60 – Moments (dissertação). UPF - Faculdade de Artes e Comunicação (FAC). http:// repositorio.upf.br/handle/riupf/1278 Gomez L, Olhats M, Pólo C (2011) Fashion Branding: uma relação emocional com o consumidor. ModaPalavra E-Peri Ódico 4(8). http://www.revistas.udesc.br/index.php/modapalavra/article/ view/7820 Hardy J (2017) Commentary: branded content and media-marketing convergence. Polit Econ Commun 5(1):81–87. http://www.polecom.org/index.php/polecom/article/view/79/274 Massarolo JC, Mesquita D (2014) Estratégias Contemporâneas Do Storytelling para Múltiplas Telas. Revista Latinoamericana de Ciencias De La Comunicación 11:48–57. http://revista.pub alaic.org/index.php/alaic/article/viewFile/564/336 McSill J (2014) 5 lições de storytelling: fatos, ficção e fantasia. DVS Editora, São Paulo Núñez A (2007) Será mejor que lo cuentes. Ediciones Urano, Barcelona Orozco G, Aguado J, Feijoo C, Martínez I (2015) La publicidad ante el reto digital. Revista de Pensamiento Sobre Comunicación, Tecnología y Sociedad. http://www.revistasculturales.com/ revistas/28/telos-revista-de-pensamientosociedad-y-tecnologia/num/99 Reske DL, Souto CB (2018) Storytelling: Emoção No Filme Publicitário Do Zaffari. Disciplinarum Scientia 19(1):101–117. https://periodicos.ufn.edu.br/index.php/disciplinarumALC/ article/view/2473/2134 Ribaric M (2014) A publicidade audiovisual na sociedade de consumo. Revista Temática 9(5). https://www.researchgate.net/publication/319939702_A_publicidade_audiovisual_na_s ociedade_de_consumo Rocha MDS (2019) As estratégias de storytelling no branding: mais do que contar histórias é fazer histórias. Um case da marca Red Bull. https://repositorio.ucs.br/11338/5681 Rodrigues C (2007) Elementos do filme publicitário: A construção do sentido crazy. Cadernos de Semiótica Aplicada 5(2). https://periodicos.fclar.unesp.br/casa/article/view/547/468 dos S Tomaz SF (2019) Storytelling: a narrativa publicitária To Give More e as reminiscências cinematográficas (dissertação). Instituto Politécnico de Lisboa, Escola Superior de Comunicação Social. http://hdl.handle.net/10400.21/11195 Vieira CVF (2009) A Narrativa audiovisual publicitária: a forma comercial e a forma social (dissertação). Universidade do Minho - Instituto de Ciências Sociais. https://repositorium. sdum.uminho.pt/bitstream/1822/9770/1/tese.pdf Xavier A (2015) Storytelling. Histórias que deixam marcas (1a edição). Best Business

Optimizing Accessibility Features for Deaf Community in Bilingual Videos: A User-Centered Design Analysis Jéssica Rodrigues Esteves(B) , Luis Rocha , Berenice Gonçalves , Giselle Merino , Eugenio Merino , and Júlio Monteiro Teixeira Design Department, Federal University of Santa Catarina, Florianópolis, SC, Brazil [email protected] Abstract. Bilingual videos in Brazilian Sign Language/Portuguese are intended to provide access to information for deaf users through accessibility features such as captions and simultaneous translation. However, recent research has highlighted the potential benefits of incorporating visual stimuli and strategies, such as symbols, illustrations, icons, pictograms, and animations, to enhance information representation and facilitate content understanding for this user group. Accordingly, this study aims to analyze a bilingual video with a user-centered focus on accessibility features for deaf users. The methodological procedures involved a thorough literature review on user-centered design for the deaf, followed by a field analysis on YouTube to identify the most suitable video for investigation. The selected video was evaluated by the researchers using a model for assessing the design of bilingual educational resources, and semi-structured interviews were conducted with two deaf users to elicit their perceptions of the video’s accessibility features. The results revealed that the accessibility features, such as captions and simultaneous translation, were deemed sufficient for users to comprehend the information presented. However, feedback from the participants suggested potential areas for improvement in the layout and organization of the video to enhance its accessibility. The findings of this study have implications for the design of bilingual educational resources and the development of user-centered approaches to enhance accessibility for the deaf community. Keywords: User-centered design · Deaf user · Bilingual videos

1 Introduction The development of digital media technology has opened up new possibilities for design, challenging traditional graphic design to create an interactive presentation of information (Bonsiepe 2019). Digital media, also referred to as new media, emerged from a reconfiguration of traditional media’s properties, combining both old and new conventions while adapting to the medium in which it is used (Manovich 2001). The media encompasses four key aspects: configuration, including individual, multiple, or multimedia; types, such as text, graphics, or animation; rhythm, synchronous or asynchronous content; and editing, real-time or edited (Paul 2007). According to Manovich (2001), digital media can take many forms, including digital photography, films, games, websites, and videos. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 922–935, 2024. https://doi.org/10.1007/978-3-031-47281-7_74

Optimizing Accessibility Features for Deaf Community

923

Videos are a powerful medium for conveying information, and engaging audiences through both sight and sound. Millions of users on social media platforms actively watch and share videos, with over two billion registered users on YouTube alone, representing one-third of active internet users. However, many videos lack accessibility features for individuals who are blind or deaf. According to the Brazilian Institute of Geography and Statistics (IBGE 2020), over 10 million people in Brazil have some form of hearing impairment, equivalent to 5% of the population. Among them, 2.7 million are completely deaf. The World Health Organization (WHO 2010) estimates that globally, 900 million people may develop hearing loss by 2050. In December 2020, YouTube Brazil launched a campaign to promote the production of accessible videos in celebration of the International Day of Persons with Disabilities, on December 3. The campaign is part of the “Google Accessibility Sprint Brazil” program, which plans to implement several accessibility projects in 2021 (MWPT 2020). Nine content creators participated in the campaign and made digital accessibility features available in their videos, such as sign language interpretation, audio description, and captions. While the simultaneous translation of sign language and Portuguese is a feature that makes the video bilingual for deaf users, other visual features can be used to enhance the understanding of information in digital media. Researchers from various fields having presented findings from media studies in pursuit of accessibility for the deaf, despite that, in practice, the issue remains neglected due to factors such as lack of time, interest, or knowledge during project development (Vieira 2019). The author has proposed seven principles for designing digital media aimed at deaf users – linguistic redundancy, context, protagonism, belonging, autonomy, visuality, and relevance – to provide a positive experience for them. Based on the aforementioned points, this study’s research question is: How do deaf users, that have Brazilian Sign Language (Libras) as their first language, understand informational content in a Portuguese/Libras bilingual video? Therefore, the objective of this article is to analyze a bilingual video with accessibility features aimed at deaf users. To accomplish this an informative video with digital accessibility features was selected from the “Google Accessibility Sprint Brazil” program to be evaluated from the perspective of deaf users.

2 User-Centered Design for Deaf User-centered design (UCD) is an approach that focuses on ensuring that the user is at the center of the design process, building a solid understanding of how they will interact with the product, and the characteristics of that interaction (Elmansy 2015). Creating products intended for human use is a complex process that requires knowledge from different fields. Therefore, involving the user in the design process is considered a strategy to propose solutions that meet the specific needs of the audience, providing usability and satisfactory user experiences in performing their tasks (Cybis, Betiol, & Faust 2015). When developing a user-centered project, designers need to consider human capabilities, such as sensory (vision, hearing, touch, smell, and taste), cognitive (thinking and communication), and motor (reach, location, and dexterity) abilities, as well as temporal

924

J. R. Esteves et al.

and social dimensions. However, deaf users have compromised sensory abilities due to hearing loss, which impacts their cognitive abilities, particularly their thinking and communication skills. Brazilian deaf users communicate through Brazilian Sign Language (Libras), which has been recognized as a legal means of communication and expression since 2002, as determined by Law No. 10,436 (Souza 2015). The Deaf community includes deaf individuals, sign language translation and interpretation professionals, and others who live with the deaf and their culture, using Libras (Brasil 2002). Like oral languages, sign languages have arisen spontaneously from the interaction between people and allow for the expression of any type of concept – whether it be descriptive, emotional, rational, literal, metaphorical, concrete, or abstract (Brito et al. 1997). That is why sign languages do not originate, as commonly believed, from oral languages. Instead, they have their own distinct grammar, structure, and set of rules. For illustrative purposes, it is worth noting that Brazilian Sign Language (LIBRAS), utilized within the deaf community of Brazil, exhibits marked linguistic divergence from Portuguese Sign Language (LGP), which serves as the primary mode of communication among the deaf community in Portugal. Libras is defined by the use of signs and classifiers (Vieira 2019), which are linguistic resources of Libras grammar used as visual strategies to describe the characteristics of images, objects, animals, people, and others, according to how the user sees them (Campello 2008). In addition, rhythm, slow and fast movements, and facial and body expressions are other elements used in Libras to replace hearing and speech in communication (Vieira 2019). Therefore, sign languages differ from oral languages by using the visual-spatial modality and not the oral-auditory modality, so that signs are articulated spatially – through hands and facial and body expressions – and perceived visually – through vision (Gesser 2009). Therefore, sign languages use space and the dimensions it offers in the constitution of its phonological, morphological, syntactic, and semantic mechanisms to convey meanings, which are perceived by its users through the same spatial dimensions (Gesser 2009). The current educational concept is Bilingualism, that is, the coexistence of Brazilian Sign Language as the first language and written Portuguese as the second language, to meet the different functions that each presents in the daily life of the deaf person (Quadros 2008; Machado 2009). In the last ten years, the production of Digital Educational Resources (DERs), such as videos, multimedia systems, interactive websites, and online learning environments, aimed at the deaf public has increased (Moraes, Gonçalves & Bergmann 2018). However, recommendations for the production of digital media focused on deaf users are scarce and are included in accessibility research aimed at users with all kinds of needs or a specific area (Petrie, Weber & Fisher 2005; Lapolli, Vanzin & Ulbricht 2013; Guimarães 2014; Flor, Vanzin & Ulbricht 2013; Silva 2013; Araújo 2012). In this context, Moraes and Gonçalves (2021) present a model for evaluating bilingual educational resources, explained below. 2.1 Model for Evaluating the Design of Bilingual DERs Moraes and Gonçalves (2021) propose a Model for the Evaluation of bilingual Digital Educational Resources (DERs). These resources are divided into five categories:

Optimizing Accessibility Features for Deaf Community

925

Pedagogical Context, Deaf Culture, Translation and Interpretation, Digital Media, and Interface Design. The Pedagogical Context category assesses whether an educational resource is suitable for the intended pedagogical objectives and considers the cognitive and emotional characteristics of the user. The Deaf Culture category raises concerns about the use of Libras and Portuguese, visual culture, and whether the resource is appropriate for the cognitive characteristics of bilingual deaf students. The Translation and Interpretation category evaluates the quality of translation and interpretation, as well as the interaction between the presenter and visual elements in bilingual educational resources. The Digital Media category focuses on the digital media elements used in educational resources, and the Interface Design category examines the visual layout of the interface and its usability in digital educational resources.

Fig. 1. Expanded configuration of the Model for evaluating the design of Bilingual DERs (Libras/Portuguese). Source: Moraes & Gonçalves (2021).

This model aims to assist with prognostic assessments of bilingual resources in a non-linear and customizable way. Evaluators can start with any category, and the criteria are adaptable depending on the resource being evaluated (Fig. 1). The purpose of this research is to evaluate the quality of digital media elements in educational resources, such as text, graphics, images, videos, animations, and games. In this research, we used the Digital Media category, which is divided into static, dynamic, and interactive media (Moraes & Gonçalves 2021). Based on the criteria, the authors developed a checklist to support the assessment, either by a teacher, a user, or a project team. In this research, we used this checklist to evaluate a video.

926

J. R. Esteves et al.

2.2 Principles for User-Centered Media Design for the Deaf Vieira (2019) proposed principles for digital media design focused on deaf users, divided into three levels, explained below. At the first level, there are principles such as linguistic redundancy, context, visuality, autonomy, protagonism, relevance, and belonging. The second level presents the three human aspects associated with deaf users when viewing information, with nine ramifications, namely: cognitive (language, attention, memory, reasoning); emotional (identification, action, motivation); and physical (vision and gesturing). Finally, the third level lists the user’s characterization dimensions and their ramifications: cultural (visual experience, linguistic, family, literature, visual arts, social life, sports, policies, and materials); linguistic (Libras, bilingualism, SignWriting, sign translation and interpretation professional); and visual (assistive technology, symbol, typography, and color). The principles generated by Vieira (2019) show respect and value for the deaf language, based on the perspective of the deaf community itself, as provided by the User-Centered Design methodology, by placing the human being at the center of the process and allowing active participation (Elmansy 2015; Cybis, Betiol & Faust 2015). Given this research’s objective, it is necessary to explain the principles of usercentered design for digital media targeted at the Deaf community. Principle 1, linguistic redundancy, focuses on ensuring that media content is available in similar levels of information and detail across different modalities, including Libras (Brazilian Sign Language), written Portuguese, and visual languages such as illustrations, infographics, animations, and videos. Principle 2, context, refers to providing the user with the necessary background information or explanations to facilitate understanding of the content. This can involve the use of glossaries, narrative techniques, human interpreters (as opposed to avatars, icons, or other non-human elements), and other strategies. Principle 3, protagonism, acknowledges the deaf person’s agency and potential to actively participate in the development of media content by commenting, testing, choosing, building, and making decisions as the main author of their story (Vieira 2019). Thus, involving users in all stages of project development is crucial. Principle 4, belonging, stresses the importance of using elements that represent the deaf person’s sociocultural context, enabling them to identify with and feel empathy for their language and culture. Principle 5, autonomy, recognizes the user’s power to make choices and act independently, by providing control over the media’s actions and minimizing the need for external assistance. Principle 6, visuality, emphasizes the use of visual resources to convey information, instead of relying on sound or text alone. Finally, Principle 7, relevance, focuses on organizing information in a clear and hierarchical way, so that the deaf person can easily identify and understand the most important parts of the content. After analyzing these principles, it was found that linguistic redundancy, belonging, autonomy, visuality, and relevance have specific recommendations for audiovisual media, making them particularly relevant for analyzing videos posted on social networks. On the other hand, the context principle was not considered in this study since the content is informative and includes accessibility features for all users, regardless of prior knowledge. Similarly, the protagonism principle relates to a stage in media development and falls outside the scope of this research.

Optimizing Accessibility Features for Deaf Community

927

3 Methodology This study aims to explore User-Centered Design principles to improve accessibility for the Deaf community. Specifically, the focus is on visualizing information through an accessible bilingual video format, which is structured into four procedures as shown in Fig. 2. Through qualitative and applied research methods, this study seeks to identify effective strategies for creating inclusive and user-friendly visual content for Deaf audiences. The technical procedures of the study started with a literature review. Then, an exploratory field analysis was conducted to identify accessible bilingual videos available on YouTube Brazil to celebrate the International Day of Persons with Disabilities. Fifteen videos, published on different content creator channels, were identified. For the analysis, an “education” themed video was selected, with a maximum length of 5 min, to avoid overwhelming the user during the analysis. The authors elected a video from the Drauzio Varela channel. This particular video, chosen from among various options, boasted the highest viewership count and presented content that was notably more comprehensive and informative.

Fig. 2. Research procedures.

The selected video was evaluated by the researchers using the Model for the Evaluation of Bilingual DERs Design (Moraes and Gonçalves 2021), which is designed for the prognostic evaluation of Bilingual Digital Educational Resources (DERs) by researchers, teachers, and designers in the field. After the video evaluation, semi-structured interviews with participants were carried out. The participants invited to participate in the research are students from the Federal Institute of Maranhão (IFMA), enrolled in the Visual Arts Teaching program at the São Luís Centro Histórico Campus, as well as the Electronics and Electromechanics technical programs at the São Luís Monte Castelo Campus. Contact with the students was made via WhatsApp. These students are part of the list of students assisted by the institution’s Special Needs Assistance Center (NAPNE), which was obtained in 2019 through a formal request to the sector, aiming to recruit students interested in participating in research on inclusion and accessibility topics. Out of the 5 students invited 2 accepted to participate in the interviews. They are deaf individuals who have Libras as their first language and Portuguese as their second language. Both participants also had experience watching videos on YouTube. After acceptance, the Informed Consent Form (ICF) was sent, and a date was scheduled for the evaluation. The interviews were mediated by a Libras interpreter from IFMA.

928

J. R. Esteves et al.

The interviews were conducted via Google Meet between February 2nd and 17th1 , 2021, at the previously agreed-upon times. The research involved the presence of two researchers – co-authors of this article -, a Libras interpreter, and a deaf user. After the user watched the video, the moderator started the interview. Finally, the interviews were transcribed, analyzed using content analysis (Bardin 2015), synthesized, and grouped into categories related to the principles for media design centered on deaf users (Vieira 2019) using a chart that marked the responses of each user.

4 Bilingual Video Analysis: Findings and Implications This study focuses on digital media in video format available on YouTube. The video, titled “Homosexuality [Videos for Everyone]”, was posted on December 3, 2020, on Dr. Drauzio Varella’s channel, which has 2.67 million subscribers. It has received 48,615 views and was created as part of a YouTube campaign celebrating the International Day of Persons with Disabilities on December 3. As a result, the video includes sign language interpretation, audio description, and captions. 4.1 Bilingual Video Evaluation The video has an informative nature and deals with the topic of homosexuality, in which the doctor explains, in a colloquial and scientific way, about taboos surrounding the theme. In terms of the visual organization of the layout, the video has only one set containing three main elements, highlighted in Fig. 3. The video does not include any other visual stimuli or resources. Another relevant aspect to be highlighted is that the video was adapted to be accessible through resources such as Portuguese captions and simultaneous translation in Libras, as its original version without digital accessibility was posted on the content creator’s channel in 2014. Therefore, the video was not originally conceived to be inclusive but was adapted later for accessibility.

Fig. 3. Visual organization of the video “Homossexuality [Videos For Everyone]”. Source: https:// www.youtube.com/watch?v=rQ7FjVX1fAI&t=1s&ab_channel=DrauzioVarella

1 It is important to highlight that the interviews were carried out during the Covid pandemic in

Brazil, which made face-to-face contact between participants and researchers impossible.

Optimizing Accessibility Features for Deaf Community

929

In order to evaluate the quality of the video, we used the Model for Evaluating the design of Bilingual DERs (Moraes & Gonçalves 2021), considering two criteria for digital media. Each criterion was assessed based on three aspects (yes, no, not applicable). The summary of the evaluation is available in Table 1. Table 1. Summary of the video evaluation. DIGITAL MEDIA Category composed by criteria that assess the quality of the elements of digital media that make up educational resources: text, graphics, images, videos, a nimations, games, etc. Dynamic Media Criteria: They refer to the elements of videos, animations, audio, and video graphics. Criterion 4: Presentation and Quality of Videos

Yes

No

N/A

Yes

No

N/A

Yes

No

N/A

Yes

No

N/A

a) Does the video have good resolution, lighting, focus, contrast, and chroma key cropping? b) Is the video quality and framing capable of capturing the details of hands, eyes, mouth, and movements? c) Is the presenter/interpreter of the resource properly sized in conformity with the rest of the video? d) Is the average duration of the DER on video around 10 minutes? For example, videos that are too long fatigue the user. Criterion 5: Quality of footage a) Are the lighting, costumes, and image of the presenters maintained throughout the scenes? b) Are camera changes and cuts carefully done, so that they do not impair the understanding of sign language? Criterion 6: Adequacy of the script and narrative a) Are the script and narrative lines constructed according toLibras’ linearity? For example, the objects are presented first, then the scene, and then the actions. b) The script prioritizes the development of a playful, light, and dialogic narrative. Criterion 7: Quality of animations a) Are the animations consistent with the theme of the video? For example, animations for adult audiences can be playful, but not childish. b) Do animations make proper use of animation principles: timing, anticipation, continuity, etc.? c) Are the animations clear and concise? That is, they avoid details and effects that might distract the user.

The video is of good quality, with a clear resolution, appropriate lighting, focus, contrast, chroma key cutout, proper framing, and a duration of less than 10 min, meeting criterion 4. Additionally, the video features quality lighting and costume, satisfying criterion 5. Its narrative line is well-constructed, presenting a light and dialogic approach, fulfilling criterion 6. However, the video does not satisfy criterion 7 as it lacks animated visual resources or animation principles, being shot in a single take without cuts. The evaluation of the video using the Bilingual DERs design evaluation model, focusing on the criteria of digital media (Moraes & Gonçalves 2021), revealed that it does not meet all the listed criteria, which may make it less appealing or understandable

930

J. R. Esteves et al.

for deaf users. Therefore, to determine if criterion 7 impacts user comprehension of the information, semi-structured interviews were conducted, as described below. 4.2 Interviews with Participants The semi-structured interviews with participants revealed their perceptions regarding the accessibility features available in the video, as well as suggestions for changes to improve the viewing experience. Therefore, the interviews addressed the principles of linguistic redundancy, belonging, autonomy, visuality, and relevance, which have specific recommendations for audiovisual media (Vieira 2019), as detailed below. Regarding principle 1, linguistic redundancy, it was analyzed whether the user could understand the media content with equivalent levels of information and detail – Brazilian Sign Language (Libras), written Portuguese, and completely visual languages (illustrations, infographics, animations, among others). Of the visual resources mentioned, the media analyzed has translation in Libras and subtitles in Portuguese. Regarding the resources used in the video that aided in understanding the content, both interviewees identified simultaneous translation with the Libras Interpreter and subtitles in Portuguese. Although each of the interviewees used one of the resources more prominently, both emphasized that the simultaneous presence of the Libras Interpreter and Portuguese subtitles is fundamental for deaf users with different needs to understand the content, as there are deaf people who are not fluent in Portuguese and, therefore, prefer Libras; while others are fluent in both languages and can choose the resource that is most accessible to their needs. Regarding principle 4, belonging, the video uses a Libras Interpreter for simultaneous translation instead of automatic avatars (Vieira 2019), which is preferred by deaf users. Both participants highlighted that they felt represented in the video due to the presence of the Libras Interpreter, which corroborates with Vieira (2019), who emphasizes the importance of using representative elements of the deaf socio-cultural context so that they can identify and generate empathy regarding their language and culture. Principle 6, visuality, deals with the preferential use of visual stimuli and strategies such as images (symbols, illustrations, icons, pictograms, animations) to represent information and facilitate the understanding of content. Additionally, the principle also deals with recording videos with the actor signaling from the front, wearing simple clothes and without accessories so that the signals can be clearly seen (Vieira 2019, p. 210). The video does not present the mentioned visual strategies. When questioned about other visual elements that could be used to improve the experience of understanding the information, the participants could not cite other examples. Finally, regarding principle 7, relevance, both participants highlighted the importance of the presence of the Libras Interpreter in the video, which is more relevant than subtitles. According to E2, most deaf people prefer simultaneous translation in Libras over subtitles. Regarding the layout organization, E1 highlighted again the need for a greater proportion between the size of the video presenter and the Libras Interpreter to maintain a greater balance ratio. The results of the interviews were summarized and grouped into categories related to the principles for designing media centered on the deaf user (Vieira 2019), available in Table 2. The interviewees are identified by the acronyms E1 (interviewee 1) and E2

Optimizing Accessibility Features for Deaf Community

931

(interviewee 2). The markings (✓) in the E1 and E2 columns refer to the positive response regarding the aspect shown in the Themes column. Table 2. Summary of the interview responses from the participants. PRINCIPLES

E1

E2

Positive experience when watching the video

✓

✓

The video has accessibility

✓

✓

Identification of the use of Portuguese subtitles

✓

✓

Identification of the Libras Interpreter

✓

✓

Principle 1 – Linguistic redundancy

✓

Portuguese subtitles were the most relevant feature The Libras Interpreter was the most relevant feature

✓

Principle 4 – Belonging Identification of deaf people with the video

✓

Half-screen division between the Libras Interpreter and the hearing presenter

✓

✓

Principle 5 – Autonomy Use or non-use of subtitles

✓

Principle 6 – Visuality ✓

✓

Felt interested in the video content

✓

✓

The video presents a good organization of elements in the video layout

✓

✓

Positive experience of viewing the Libras Interpreter Principle 7 – Relevance

It is evident that increasing the window for simultaneous translation through the Libras Interpreter directly influences the understanding of the content by participants, corroborating with Brito et al. (1997) and Gesser (2009). The cited authors emphasize the visual-spatial modality of Libras since the signs are articulated spatially through the hands, and facial and bodily expressions, and are visually perceived through sight. Therefore, the larger the window for simultaneous Libras translation, the better the user will be able to visualize the content. According to the interviewees, the ideal would be for the Libras interpreter window to occupy half of the video space, as shown in Fig. 4. Regarding this research, another aspect to be highlighted is that participants watched the video using mobile devices (smartphone and tablet), which further compromises the visualization of the translation window. Based on users interviews, it was identified that the video has accessibility features and is easily understandable for deaf users. However, the video does not incorporate the principles for digital/dynamic media design proposed by Vieira (2019) and Moraes & Gonçalves (2021), such as: I) dividing the interpreter’s translation window with equal importance to the listener’s window, split evenly on the screen; II) using visual strategies

932

J. R. Esteves et al.

Fig. 4. Suggested layout for Portuguese/Libras bilingual video.

(symbols, illustrations, icons, pictograms, images) to represent information and make the content more engaging; and III) improving the quality of animations through animated visual resources or animation principles. Analyzing the video from a user-centered design perspective, it is emphasized that the production of digital media for the deaf should place the user at the center of the process (Elmansy 2015) to propose solutions that meet their needs (Cybis, Betiol & Faust 2015). The interviewees claimed to see the deaf community as a very restricted group. However, by feeling included in the conception process of didactic materials for the deaf, they realize how they can collaborate to improve the conditions of people with disabilities. From this perspective, it is worth noting that the analyzed video was not designed to be accessible to all types of users because, as demonstrated in the evaluation, its first version for listeners was posted in 2014. In order to meet the needs of deaf users, resources such as Portuguese subtitles and simultaneous translation into Libras were used to make it accessible. However, recent research shows the use of images and/or other visual stimuli such as graphic symbols, icons, photography, illustrations, and animations (Vieira 2019; Moraes & Gonçalves 2021) to represent information and facilitate understanding of the content, which were not used in the video. During the interview, participants could not point out which other visual resources could be used to make the video more engaging, probably due to a lack of knowledge and exposure to accessible bilingual videos with these features.

5 Conclusion Bilingual Portuguese/Libras videos can reduce barriers and provide access to information for both deaf and hearing users. However, it is important to ensure that the accessibility features are sufficient for deaf users to understand the information presented. In this study, we conducted an analysis from two perspectives. First, we used the Model for Evaluating the design of Bilingual DERs (Moraes & Gonçalves 2021) to assess the

Optimizing Accessibility Features for Deaf Community

933

adequacy of accessibility features provided in a video. Second, we conducted interviews with two deaf participants based on principles for designing media centered on the deaf user (Vieira 2019). We analyzed the video ourselves to identify any issues related to the quality of the recording, narrative, and animation. To evaluate the perceptions of deaf users, we conducted semi-structured interviews with two participants who watched the video and provided feedback based on principles for designing media with a focus on deaf users. The results showed that although the video aimed to be accessible for deaf users, it did not meet all of the recommended criteria due to being adapted with Portuguese subtitles and Libras interpreters, rather than designed inclusively from the start. However, the lack of adherence to these criteria did not impact the understanding of the video’s content, as reported by the participants. During the interviews, technical difficulties with internet speeds made communication challenging between the mediator, interpreter, and participants. Given the Covid-19 pandemic circumstances, remote interviews were the sole viable method to carry out the research. Despite these issues, we consider remote interviews with Libras interpreters to be the most appropriate method for qualitatively evaluating digital media designed for deaf users. The feedback obtained from the interviews allowed us to identify suggestions and reflections that would not have been possible through other methods. However, a checklist of principles for designing digital media with a focus on users could optimize the evaluation process for remote interviews. Based on research results, we created a suggested layout for bilingual Portuguese/Libras videos, in which the interpreter occupies half the video space to improve translation comprehension. This layout may also contribute to a sense of hierarchy and belonging for deaf users. When the presenter and interpreter share the same video space, deaf users can feel represented in the video. We emphasize the importance of involving users at all stages of digital media construction or adaptation, as offering accessibility options for people with diverse abilities empowers and provides autonomy to users. The analyzes cannot be generalized, as it was not possible to increase the number of interviewers due to the limitations imposed during the pandemic period, and the difficulties in reconciling the schedules of participants and sign language interpreters. In future research, we will conduct interviews with a larger number of users to thoroughly assess this scenario. Acknowledgments. This work was carried out with the support of the Federal Institute of Education, Science and Technology of Maranhão (IFMA) and funded by the Coordination for the Improvement of Higher Education Personnel – Brazil (CAPES) – Funding Code 001.

References Araújo ECJ (2012) Acessibilidade para surdos em ambientes de mobile learning: recomendações para interfaces. Master’s thesis, Programa de Pós-Graduação em Engenharia e Gestão do Conhecimento, Universidade Federal de Pernambuco Bardin L (2015) Análise de conteúdo. Edições, Lisboa 70: 225

934

J. R. Esteves et al.

Bittencourt MC (2002) Estudos de percursos acessíveis aos portadores de necessidades especiais em espaços abertos na cidade de Maringá. Master’s thesis, Programa de Pós-Graduação em Engenharia de Produção, Universidade Federal de Santa Catarina Bonsiepe G (2019) Design, cultura e sociedade, 2nd edn. Editora Blucher Brasil (2002) Lei nº 10.436, 24 de abril de 2002. Dispõe sobre a Língua Brasileira de Sinais Libras e dá outras providências. Diário Oficial da República Federativa do Brasil. http://www. planalto.gov.br/ccivil_03/leis/2002/l10436.htm Brito LF (1997) A Língua Brasileira de Sinais. In: BRASIL, Ministério da Educação. Deficiência Auditiva. Série Atualidade Pedagógicas. SEESP, Brasília Campello AR (2008) Aspectos da visualidade na educação de surdos. Doctoral dissertation, Programa de Pós-Graduação em Educação, Universidade Federal de Santa Catarina Cybis W, Betiol AH, Faust R (2015) Ergonomia e Usabilidade: Conhecimentos, Métodos e Aplicações, 3rd edn. Novatec Elmansy R (2015) Characteristics of human centered design. Designorate. http://www.design orate.com/characteristics-of-human-centered-design/ Flor CS, Vanzin T, Ulbricht V (2013) Recomendações da WCAG 2.0 (2008) e a acessibilidade de surdos em conteúdos da Web. Revista Brasileira de Educação Especial 19(2):161–168 Gesser A (2009) Libras? Que língua é essa?: Crenças e preconceitos em torno da língua de sinais e da realidade surda. Parábola Ed Guimarães APN (2014) Recomendações para avaliação da experiência de usuário em aplicativos móveis para surdos. Dissertação de mestrado, Programa de Pós-Graduação em Informática, Universidade Federal da Paraíba IBGE (2020) Pesquisa Nacional de Saúde: 2019. Atenção primária à saúde e informações antropométricas. Brasil / IBGE, Coordenação de Trabalho e Rendimento. IBGE, Rio de Janeiro. https://biblioteca.ibge.gov.br/visualizacao/livros/liv101758.pdf. Accessed 22 Apr 2021 Lapolli M, Vanzin T, Ulbricht VR (2013) Infográficos na Web: uma Proposta Centrada no Usuário Surdo. HFD 2(4):2–11 Machado PC (2009) Diferença cultural e educação bilíngue: as narrativas dos professores surdos sobre questões curriculares. Tese de doutorado, Programa de Pós-Graduação em Educação, Universidade Federal de Santa Catarina Manovich L (2001) The language of new media. The MIT Press Moraes LM, Gonçalves BS (2001) Bilingual digital educational resources design: a model for assessment and supporting checklist. Estudos em Design 29(3) Moraes L, Gonçalves B, Bergmann J (2018) Recursos Educacionais Digitais para Estudantes Surdos: Uma possível classificação. Educação Gráfica 22(3):44–62. http://www.educacaograf ica.inf.br/wp-content/uploads/2019/02/7_RECURSOS-EDUCACIONAIS.docx.pdf MWPT (2020) Movimento Web para Todos. Web para Todos e YouTube Brasil mobilizam grandes canais pela acessibilidade digital. Recuperado em 19 de janeiro de 2020, de. https://mwpt.com. br/web-para-todos-e-youtube-brasil-mobilizam-grandes-canais-pela-acessibilidade-digital/ Paul N (2007) Elementos das narrativas digitais. In: Ferrari P (Org.) Hipertexto Hipermídia: as novas ferramentas da comunicação digital, pp 121–139. Contexto Petrie HL, Weber G, Fisher W (2005) Personalization, interaction, and navigation in rich multimedia documents for print-disabled users. IBM Syst J 44(3):629–635 Quadros RM (Org.) (2008) Estudos Surdos III. Série Pesquisas. AraraAzul da Silva EL, Menezes EM (2001) Metodologia da pesquisa e elaboração de dissertação, 3rd edn. Laboratório de Ensino a Distância da UFSC, Florianópolis Silva RAL (2013) Recomendações para acessibilidade aos surdos de auxílio aos designers na criação na implementação de ambientes web. (Dissertação de mestrado). Programa de PósGraduação em Informática, Universidade Federal do Paraná, Curitiba, PR, Brasil

Optimizing Accessibility Features for Deaf Community

935

Souza LC (2015) Recomendações para cursos online em língua portuguesa com foco na integração de alunos surdos. (Tese de doutorado). Programa de Pós-Graduação em Estudos Linguísticos, Faculdade de Letras, Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brasil Vieira FM (2019) Princípios para o design de mídia digital com foco no usuário surdo. (Tese de doutorado). Orientador Gilson Braviano WHO (2010) Measuring health and disability manual for the world health organization disability assessment schedule 2.0. Retrieved from Geneva: http://apps.who.int/iris/bitstream/10665/ 43974/1/9789241547598_eng.pdf. Accessed 29 Jul 2016

Surfing the Landscape of Physical Computing Platforms to Develop Electroacoustic Musical Instruments Filipe Lopes1(B) and Sérgio Coutinho2 1 CIPEM/INET-Md, uniMAD, Escola Superior de Media Artes e Design, Instituto Politécnico

do Porto, Rua D. Sancho I, 4480-876 Vila do Conde, Portugal [email protected] 2 CIPEM/INET-Md, Instituto Politécnico do Porto, Rua Dr. Roberto Frias, 712, 4200-465 Porto, Portugal [email protected]

Abstract. There has been an increasing number, development, and availability of platforms choices to prototype electroacoustic instruments in the last decade. This demonstrates a growing interest in physical computing among the “technologicalmusic” community. Bela and Daisy are nowadays among the most popular and widespread platforms specifically developed for out-of-the-box music technology purposes. Each of these platforms has its own unique features, capabilities, and community, providing options for different preferences and requirements and exhibiting an increasing adaptability in terms of computer music languages. This raises the question: which one is the ideal to develop an electroacoustic instrument? In this article, we present an overview of these platforms, among others that have been used since 2000, from the viewpoint of the performer-luthier at the development stage for an electroacoustic instrument to perform music in the wilderness. In addition, based on a specific project the authors are developing, we propose some design guidelines which considers the musical and technological possibilities and implications of these platforms. Keywords: Electroacoustic Instrument · Physical Computing · Performer-Luthier

1 Introduction The history of electroacoustic instruments can generally be traced back to the beginning of the twentieth century, when electricity started to be used to produce sound in form of modular synthesizers or more self-contained approaches such as the theremin. With the arrival of stable digital technologies in the sixties, and increasing computational power in the following decades, the possibilities to merge old, recent, and fresh technologies to design electronic musical instruments was enormous and appealing to the “technological-music” community, specifically when the advent of fast, inexpensive digital chips, made possible realtime audio processing. Furthermore, as soon as hardware © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 936–949, 2024. https://doi.org/10.1007/978-3-031-47281-7_75

Surfing the Landscape of Physical Computing Platforms

937

started to shrink into affordable microchips and microcontrollers back in the eighties, it made possible the development of musical instruments featuring programmable electronic components coupled with analog (i.e. electronic) parts. One of the most famous instrument combining analog and digital components was the Yamaha DX7, which featured at the time the groundbreaking frequency modulation synthesis developed by John Chowning [1]. By this time, it was clear for the “technological-music” community that in the following decade it would be possible to develop original electroacoustic instruments (i.e. musical instruments featuring electronic and acoustic elements) combining microchips, sensors and actuators, coded in high-level computer music languages, therefore, an opportunity to prototype and develop unseen interfaces and musical instruments. It is not a surprise that by the early 2000’s a conference so specifically in its topic like the New Interfaces for Musical Expression (NIME) came to existence, directly born after a workshop in a conference on Human Factors in Computing Systems [2]. Likewise, it is no surprise that with the technological advances in the decade of 2010, conferences like the Ubiquitous Music - born in 2010 - or the International Conference on Live Interfaces - born in 2012 - also came into existence. Since NIME and conferences alike were born, many discussions within these conferences have evolved around the topics of expressivity, audio technologies, machine learning, interaction, tactility, pedagogy, performance, sensors and actuators, laptop music performance, mobile and web technologies, artificial intelligence, among many others. Furthermore, according to our experience, the people engaged in these activities and conferences are not just people from the commercial industry but mostly from academia (e.g. PhD students, Master students, Research Labs) or hobbyists. For the purposes of this paper, we will use the term perfomer-luthier to refer to the people that possess know-how and experience in music performance and composition, together with musical computing know-how, willing to develop digital and/or electroacoustic interfaces and musical instruments. Since 2005 up until 2012, roughly speaking until Raspberry Pi came alive, in the realm of electroacoustic instruments developed by perfomer-luthiers, many were developed using either the widespread and popular platform Arduino with its technical constraints [3] or by developing specific circuits with commonly available microchips (e.g. ATmel) [4]. While there are many computer music languages dedicated to sound and music computing that are enduring since the 80’s, e.g. MaxMSP, Supercollider, Pd [5], the same cannot not be said for interfaces developed for music expression and the hardware used to developed such interfaces and instruments. The problem of obsolesce and the lack of time to document, devise a plan for preservation and longtime technological viability are probably some of the reasons for that [6]. Let us persist on the software-hardware comparison to make a point: circa until the Arduino platform got popular, it’s fair to say that developing hardware was something that demanded precise skills and certain laboratory infra-structures in order to devise an electroacoustic instrument, therefore, comparable to the precise know-how and skill demands of coding in extremely low-level languages (e.g. assembler). In the last decade, however, we have seen the arising of platforms (e.g. Bela, Daisy) that can be situated within the landscape of physical computing, i.e. physical systems that encompass sensors and actuators that respond and interact with the world, specifically manufactured

938

F. Lopes and S. Coutinho

for performers, musicians, composers and all the community engaged and interested in music technology. These platforms have overcome a range of technological problems that made life difficult a decade ago, e.g. latency [7], for people interested in developing electroacoustic instruments, thus, platforms such as Bela can be compared to a high level language (e.g. Pd, Supercollider, python, javascript,) in terms of accessibility, computational performance and demanded skills, which is to say fitted, accessible and appealing for the performer-luthier profile. This “technological-situation” is something new and exciting. It has changed dramatically the environment to develop electroacoustic instruments as nowadays one can spend much more time in experimentations and discussions about the artistic and sonic aims, so, less time worried and taking into account the technological and computational restrictions (e.g. exaggerated latency, audio codecs, power consumption). This also means that while developing an electroacoustic instrument, the people involved will not be so affected by technological bias, thus, their creative choices and workflow will not be so consciously or unconsciously shaped by technology. Altogether, the performer-luthier can focus on inventing and building an electroacoustic instrument more than on programming and debugging. The question of what platform to choose for your specific project, however, becomes a thing. Which platform from the landscape of possibilities should one choose to develop an electroacoustic instrument? This type of question has been asked in the past for educational purposes [8], for software purposes (e.g. should I use MaxMPS, Supercollider or Pd?) when someone wanted to develop a digital musical instrument [9] and it might be the same case when someone has to choose a Digital Audio Workstation to learn (e.g. Reaper, Ableton Live, Logic, Pro Tools). It seems a natural thing. Our aim with this paper is to describe how we came to define criteria to choose a platform to develop the Gaia, an electroacoustic instrument that is being developed to perform music in the wilderness. The development of this electroacoustic instrument is part of a larger research project entitled WeNature [54] which addresses the relationship between Music and Nature from different facets. We will share our design guidelines for this electroacoustic instrument, describe some of the most used and recent physical computing platforms for audio and music production that have surfaced, as well as to provide case studies that relate both of our aims within the research project and the technological landscape we have at our disposal.

2 Platforms In recent years many different platforms have emerged in the electronic music communities which has been making possible to develop portable and self-contained electroacoustic instruments. While platforms like Arduino and Teensy have been used extensively and have a long tradition within this field, we will concentrate on the more recent platforms specifically able to perform real time music. Having said that, it is not the case that Arduino and more generic embedded platforms are not used anymore to prototype music projects or are unsuited. Recently, Cullen [10] released a book combining both the learning of the fundamentals of electronics and audio electronics, approaching subjects like MIDI and digital audio signal processing within the framework of Arduino, thus, making it possible for musicians to develop their projects on an Arduino based approach.

Surfing the Landscape of Physical Computing Platforms

939

2.1 Technology and Musical Expression In the pursuit of new ways for musical expression, and other artistic practices, technology has always been a major influence in the search for new methods and possibilities. From carving wood, drying and using animal skin to construct a kora (i.e. traditional west african musical instrument), to advanced digital audio synthesis and manipulation, if we understand tools and processes as technology, it becomes clear that music and musical instruments have always been in direct relationship and dependent on complex technological processes and mechanisms. For instance, the complexity of a “simple” musical instrument (e.g. didgeridoo) is actually rather complex. The materials used in its construction, the way vibrations are induced and the acoustical properties that modulate timbral characteristics are intrinsically connected with the sonic result and the way the instrument is activated, thus, its technology. The case of computer music (i.e. digital audio) is a particular one. The unstoppable technological development train (we will avoid the word progress) have taken us to a time and situation where everything is a computer. They are ubiquitous and can be used by anyone with basic “digital carving” skills. Computer music uses the computational resources to “oscillate digital data”, and the musical result is digital oscillations translated to electrical oscillations and, finally, air oscillations, presenting us a vast array of timbral possibilities to be explored. This clairvoyance, originally remarked by the great Max Mathews [11], still holds. Nonetheless, we are living in a time when it is possible to expand on that idea to include the merging of acoustic instrument technology together with affordable and accessible software and hardware technologies, thus, “living” the same feeling of “no theoretical limitations to the performance (…) of musical sounds” [11] within the context of the contemporary performer-luthier. In the following paragraphs we will trace a landscape about the physical computing platforms at our disposal, together with specific technological and computational information, to share and get acquainted with the technological information that makes up these platforms. We will approach the platforms in terms of technical characteristics, demanded skills to program and examples of musical instruments that were developed based on them. 2.2 Computing Interaction with the World The way the digital world can interact with the physical world is through what is called, not surprisingly, physical computing. Interaction here means to sense the world using sensors (e.g. microphone, light dependent resistor, ultrasonic) and to actuate on the world via actuators (e.g. loudspeakers, light, motors). The way that this signal chain and system interacts with the world can be designed and coded, thus, we can talk about designing interactive systems. At the root of this lies the computer which can be addressed in terms of microprocessors, microcontrollers or embedded platforms which integrates both microprocessors and microcontrollers (e.g. Arduino).

940

F. Lopes and S. Coutinho

2.3 Embedded Platforms Two aspects are important and needed when the performer-luthier decides to use embedded platforms for audio. One aspect is technical: some embedded platforms congregate specific parts with better characteristics for dealing with audio signals and audio processing, e.g. better analog-to-digital and digital-to-analog converters (i.e. electronic components that translate the digital domain to the electrical domain and vice-versa), better architecture for faster sensor readings and audio processing tasks. Another aspect is availability of documentation for audio practices: online help resources, guides, personal blogs with notes, forums, and community online platforms like Discord, along with the available software libraries. When both these aspects are met, these technologies become a natural candidate for electroacoustic instrument development. On the other hand, choosing the right development platform can lead to a vast sea of specs and features which leads to new doubts. For purposes of this paper, we will consider two types of embedded platforms: physical computing embedded platforms that normally require external modules, or components, for audio applications; and embedded audio platforms that gather all the needs for audio applications and usually have dedicated documentation for that purpose. The physical computing embedded platforms we selected were: Arduino, micro:bit, NodeMCU ESP32, Raspberry Pi, and Teensy; the selected embedded audio platforms were: Akso, Axoloti, Bela Board, Daisy. We will not discuss the platforms Open Minds Toolkit and pikoADK because they are still in development stages, although we recognize these will soon be part of the landscape we are now addressing. Furthermore, there are other devices we will not include in this analysis because although one can edit and run programming scripts, they are not considered development platforms because they don’t have general purpose input and output pins available (GPIO) (e.g. Organelle and Norns, AudioMoth). Finally, most of the information we gathered about the devices comes directly from the official website of the company that makes/invented these devices. Arduino Probably the most popular platform, it has a vast product range, but we will focus on the following versions: Nano, Uno, and Mega. The main difference between them is the number of available pins for input/output. Considering Arduino Uno R3, based on the 8-bit AVR chip ATmega328P (Clock Speed: 16 MHz), it has sixteen digital input/output pins (six of them capable of PWM output) and six analog inputs. It is equipped with a USB connection and its renewed IDE lets you code and upload your programs with minimum effort. Although it doesn’t come with a digital-to-analog converter, it can be connected to external modules and components more audio oriented via its GPIO or UART, I2C and SPI (communication protocols), nonetheless, its core processor might not be powerful enough for digital audio processing (e.g. filtering, sample manipulation). Arduino might not be the best choice for tasks like realtime audio processing. Conversely, being one of the first development platforms and due to its “open-sourceness” it is probably the most documented board and any example for usage can be probably found doing a quick online search. It is low-cost and easy to find in any seller, perfect for learning electronics

Surfing the Landscape of Physical Computing Platforms

941

and programming, experimenting around sensors and buttons, generate simple lowresolution tones and melodies, but probably not the best choice for audio applications “out of the box”. Some example projects developed using Arduino include: Handheld Drum Machine [12], Bleep Drum Machine [13], ArduTouch [14] and SJS-One [15]. Micro:bit Micro:bit is a tiny development board, created by BCC in partnership with technology companies and educational organizations to teach coding and electronics to students and beginners. It features Bluetooth connectivity, a LED matrix display, buttons, and sensors like an accelerometer and compass. The micro:bit is easy to program using languages like Microsoft MakeCode, MicroPython, and Scratch, and is widely used in schools for projects ranging from simple games to complex robotics and IoT applications. Micro:bit is a not the best choice for complex projects within the field of electroacoustic instruments, however, it is a fantastic board for educational purposes. Some example projects developed using micro:bit include: MINI.MU music gloves [16], a synthesizer and tone generator [17], TS-DET1 [18] and a Melody-Synthesizer [19]. NodeMCU ESP32 NodeMCU ESP32 is similar to the Arduino but it is IoT oriented (i.e. Internet of Things collective devices connected via network). It features powerful resources and some builtin functions suited for wireless and network communication i.e. WiFi and Bluetooth “onboard”. The ESP23 has two cores, 32-bit 80MHz to 240MHz adjustable clock rate. It has thirty-six general-purpose input output pins and ten of those can be used for capacitive touch inputs (this is a particularly interesting feature when thinking about interfacing and interaction). It has eighteen analog-to-digital converters with twelve-bit resolution and two eight-bit digital-to-analog converters. NodeMCU ESP32 it is not designed for audio applications out of the box, although external modules can be used to enhance the outputs and make them suitable for audio. It can be hooked up to an SD-Card module and play audio files, but it might not be suited for digital audio processing. Some example projects developed using NodeMCU ESP32 include: a synthesizer [20], a polyphonic MIDI synthesizer [21], Corwin Point [22] and a CTAG Strampler [23]. Raspberry Pi Probably the most popular small-size computer, with general-purpose digital input/output pins onboard, and like most computers it runs an operating system (normally Raspbian OS). Perfect for physical computing, it allows the control its IO pins through python scripts, and it can run pretty much the most common audio programming environments natively, like Pd or SuperCollider. Although the Raspberry Pi comes with built-in audio output through a minijack connector, it’s not well suited for realtime audio processing applications “out of the box”. For that purpose, some complex implementations are needed along with an external audio device. It lacks analog inputs although it is possible to hook up a module capable of reading analog signals. For the

942

F. Lopes and S. Coutinho

complexity of setting up the basic environment for computer audio development, there are some projects that might improve the ease of usage, like blokas.io (https://blokas.io/). Blokas.io developed and share on their website the “Patchbox OS – Raspberry Pi OS for Audio Projects”. The Patchbox OS is a custom Linux distribution specially designed for Raspberry Pi based audio projects. It comes pre-configured for low latency audio performance and pre-installed audio software. Besides the OS distribution improved for audio development they also have available for sale “Pisound – Raspberry Pi Sound Card & MIDI Interface”. Pisound is an ultra-low latency high-quality sound card and MIDI interface specially designed for Raspberry Pi. Some example projects developed using Raspberry Pi include: a Piano Launchpie [24], Nina [25], Zynthian [26] and a Trill Guitar [27]. Teensy The Teensy (4.0) is an Arduino-compatible development board equipped with a 32-bit ARM Cortex-M7 that runs at 600 MHz. It can be compared with the Arduino and the NodeMCU ESP32. It runs without an operating system but with more computational power. With a higher cost and higher power consumption, it packs the needs for digital audio computation (e.g. signal processing, filters, fast fourier transform). It has forty digital input/output pins and fourteen analog input pins but is not equipped with an audio digital-to-analog converter on board. Although there are Audio Adaptor Boards for Teensy, it comes with common communication ports (e.g. serial, I2C, and others), making it possible to connect USB devices to it. Along with the documentation available, there are available useful libraries, like the Teensy Audio Library which features polyphonic playback, recording, synthesis, analysis, effects, filtering, mixing, multiple simultaneous I/O, and flexible internal signal routing. All audio is CD quality (16 bits, 44.1 kHz) and streams automatically as your Arduino Sketch Runs. Overall, it is not designed for “out of the box” audio processing and basic knowledge of electronics and programming are helpful to use it. Some example projects developed using Teensy include: Omniphone [28], a DrumMachine [29], Monolith [30] and a Synth [31]. Axoloti “Axoloti Core” was a development board with 24bit/96kHz stereo input and output (analog-to-digital and digital-to-analog converters) that at the time we are writing this paper, is no longer available and out of production. It packed a 168MHz STM32F427 microcontroller and it was suitable for digital audio processing. Being designed for audio applications, it had onboard MIDI input and output, an SDCard slot, and onboard USB host port for connecting USB-MIDI compliant devices. Axoloti was well documented, but it could only be programmed through a dedicated software named the Axoloti Patcher (resembles Pd and MaxMSP). After editing, patches could be stored in the Axoloti Core, and run without a computer connection. The Akso development board is inspired by and compatible with Axoloti Core but ramps up CPU performance and memory capacity by about four times.

Surfing the Landscape of Physical Computing Platforms

943

Some example projects developed using Axoloti include: a Synth [32], a Drum-Synth [33] and a Axo-Control [34]. Bela Board This embedded audio platform relies on a BeagleBone Black that, like a Raspberry Pi, is a low-cost small-sized computer. The Bela cape can be seen as a shield, that enables specific functions for dealing with audio and signal processing. The BeagleBone has an AM335x 1GHz ARM Cortex-A8 processor with 512 MB RAM memory, 4 GB of internal memory. With Ethernet, USB connection (USB-MIDI compliant), and SDCard slot. Although it lacks a Wi-Fi connection, it can be easily connected to an external common USB adapter, and it runs an operating system. Together with its cape, it can be seen as one standalone audio development platform that provides ultra-low latency in audio processing. It is equipped with two audio inputs and two audio outputs (both 16 bit / 44100 Hz), sixteen digital input/output pins, eight analog inputs and eight analog outputs (16 bit analog to digital and digital to analog converters), and a two-channel 1.1W 8 speaker output. Like most commonly used embedded platforms, it started as an academic project, and due to its features and flexibility, it’s been used for not just educational purposes but also for developing new musical instruments and other audiorelated experiments. Bela provides, on their website, extensive “how to” guides and a vast amount of “easy-to-understand” documentation. Within its browser-based IDE, it can run multiple programming languages scripts (C + +, Pure Data, SuperCollider, Csound, and more) and it features an in-browser oscilloscope for visualizing, examining and understanding your sensor signals, and GUI functionality. Some example projects developed using Bela include: Kuplen [35], Brassynth [36], Crescente [37] and Neod [38]. Daisy Daisy is a small, single, board designed for developing standalone high-quality audio projects. It has a stereo audio input and output at 96kHz/24-bit, which makes it a highaudio-quality embedded platform. Its processor, an ARM Cortex-M7 MCU, runs at 480 MHz and has 64 MB RAM memory, it is capable of realtime audio processing. It has an SD Card slot and supports common communication protocols such as SPI, UART (for connecting MIDI), and I2C. On top of the stereo audio input and output, Daisy has thirty-one general-purpose input/output pins, twelve analog-to-digital converters (16-bit), and two digital-to-analog 12-bit converters. It can be programable in C + +, Arduino, Max/MSP gen ~ and Pd. Daisy provides good documentation, “how to” guides, and videos. Its community its growing and there is a well-maintained Daisy community server on Discord. Some example projects developed using Daisy include: Chompi Club [39], Ribbon Synth [40], Pollen(8) [41] and Vocode-O-Matic [42].

3 Prototype Stages Imagining an electroacoustic instrument is not a linear process as it lies at the crossroads of arts, technology, sound art, music composition and engineering. Having said this, derived from our experience in prototyping instruments and interfaces [43–46]

944

F. Lopes and S. Coutinho

along with the information gathered from the literature, we believe it makes sense to establish two different stages when it comes to choose a platform to develop an electroacoustic instrument: First, the general prototype stage (GPS); Second, the special prototype stage (SPS). Both are stages of development and experimentation towards the development of an electroacoustic instrument, thus, stages taken before the final construction. Furthermore, the proposal of these stages is informed by our practice, thus, they might be agglutinated or even completely different for other type and experienced performer-luthier. 3.1 General Prototype Stage (GPS) This stage can be defined by an informal and experimental phase in which the array of commonly used platforms and music computing languages are, roughly speaking, valid. This considers that these technologies are all comparable and similar in terms of computational performance vs musical output, therefore, quick sketches for code, hardware assembly and musical experimentations can be tried out. While the overall aim of the research and the musical-theoretical background will be already shaping such experimentations, many of the specific things (e.g. sampling vs synthesis, sensors to be used, shape of the instrument) are open for discovery. For this reason, the technological choices can be based on the familiarity with the platforms (e.g. I have already used Arduino in the past so I will use Arduino, I have already used Bela so I will use Bela), availability (e.g. Do I have Arduinos in the lab? Do I have sensors in the lab?) or accessibility (e.g. the project does not has funding so I will use free and open-source tools). 3.2 Special Prototype Stage (SPS) This stage supports more traditional and formal methods in terms of research methodology. It features a follow up reflection about the GPS stage which, in turn, should have helped to make clearer the desired musical-sonic-voice of the instrument and the way the performer interacts with the body of the instrument (e.g. faders, knobs, pressure sensors, gyroscopes, tactile feedback). This information will make it clearer what technological and computational demands are necessary (e.g. latency above or below 10 ms), therefore, helpful to decide about the platforms which can perform efficiently in terms of computation (e.g. is the platform able to compute 40 oscillators in real time?) and physical-computing demands (e.g. I need to use I2C sensors, I need it to be running on rechargeable batteries). In possession of this information and the desired equipment, one can then take the necessary time to assemble everything into a stable self-contained prototype and start a new trial phase, probably in cooperation with professional musicians to get a better reading about the performance, both at the level of music and technology. To make sense of the data collected at this stage (e.g. direct observation, interviews), one can use research methods commonly used in Human to Computer Interaction (HCI), specifically looping the process of implementing and testing, where implementation means prototyping ideas for sensors, actuators and coding means mapping strategies, followed by tests conducted by either the authors or invited musicians. Overall, it seems interesting to consider a heuristic approach [47] at this stage.

Surfing the Landscape of Physical Computing Platforms

945

4 Design Principles At the moment that we are writing this paper we have finished the GPS stage. We’ve used commonly used platforms to do experimentations such as Arduino, Raspberry Pi and Bela Mini. These are all platforms we have available in our lab. The principles that we are going to state are based on the empirical impressions we gathered from these experiments, the knowledge we gathered from the literature [8] and the recent concerns within the NIME community that consider ecological and sustainability issues [48]. We’ve used the word “principles” instead of “criteria” because our electroacoustic instrument is still not built - we are now starting the SPS stage -. Principles offer guidelines to help anyone imagining something that is still does not exists, as opposed to criteria, which is derived from something that already exists. That being said, before delving into our design principles, we will offer a quick overview of the WeNature research project as to better contextualize the design principles we will propose. 4.1 WeNature: A Quick Overview What does it mean to perform and experience music in the wilderness? Our assumption is that performing and experiencing music in the wilderness has the power to strengthen our collective relationship to nature, hence, “music can provide a sounding model for the renewal of human consciousness and culture” [49]. Starting from that question and assumption, WeNAture proposes to develop extensive theoretical research and a philosophy about music and nature. We believe we will be able to delve into the endless topic of “what is music” and what we understand as Music, by musicking [50] in the wilderness. Our intention is to perform music which will resonate with nature and develop a compositional approach and an electroacoustic instrument which enables performers to integrate in real time the specificities of a wilderness environment. We believe “the more the environment as material informs the sound processing, the more the two will blend into each other” [51], thus, we believe we will be able to grasp new intuitions about Music and Nature by musicking in wilderness spaces. WeNature, which started on January 2023, combines and builds upon previous projects which looked at plants, nature, space, and music in diverse ways [52, 53]. 4.2 Seven Principles • Low Latency: Low latency is an essential feature if one wants the instrument to react to the performer actions fast enough. This, in turn, is essential to imprint a sense of intimacy and instrumentality. We want the performer and the audience to feel they are witnessing realtime musical creation as distinct to the manipulation of pre-recorded or pre-planned-sequenced events. • Able To Produce Acoustic Sound: since our research encompasses the study of traditional and indigenous instruments that are used to play in outdoor situations, we want to include in our electroacoustic instrument a resonant box, strings, a tube, and a mouthpiece.

946

F. Lopes and S. Coutinho

• Self-Contained and Rechargeable: Considering that ecological and sustainability is an important issue of today’s agenda, we will adopt technology to power and recharge the electroacoustic instrument, as well as research on do-it-yourself techniques to develop the electronic components. • Mid-size and Portable: Given that we want to perform in wilderness spaces, it is important to develop an instrument that we can carry and play anywhere. Furthermore, a mid-size instrument will enable us to include the possibility to use physical movement gestures (e.g. bend to the side, hug the instrument) to induce and/or manipulate audio via sensors (e.g. gyroscope, weight). • Able to Synthesize Sounds and Sample sounds: Along the lines of electroacoustic music tradition, we want to be able to implement standard synthesis techniques (e.g. fm synthesis, granular synthesis, subtractive synthesis) together with sampling techniques (e.g. record and play sounds in real time, musique concrete) in order to blend (or not) the sound of the instrument with the surrounding acoustic environment. • Able to Collect Data From the Surrounding Environment: Using a wide array of sensors to analyze the environment (e.g. gps, moisture, humidity, temperature), we want to be able to use such information for modulation purposes, thus, inducing randomness during the performance. • Able to Include Artificial Intelligence: While we are not considering AI or machine learning as a prominent feature in our instrument, we are considering including them along the signal chain. Some technological devices we are envisioning are e.g. Nano 33 BLE Sense Rev2.

5 Conclusions This paper reports the research we have done to answer the following questions: what platforms should we use to develop our electroacoustic instrument and based on what principles? We’ve defined two stages within a larger experimental phase: the first is the General Prototype Stage and the second is the Special Prototype Stage. For the first stage, since nowadays the many physical-computing platforms are powerful and relatively easy to sketch and prototype for the common performer-luthier, we think that any commonly used platforms will be fit for experimentation purposes. Nonetheless, two concepts became clear to us during this process which may add extra criteria to consider: 1 - friendliness: the performer-luthier should use a platform and a computing music language that he/she is comfortable with, in addition to some helpful features such as ease to add sensors/actuators, a strong online community and good documentation. 2 - sustainability: if we take into account that Arduino came to existence in 2005, that many Arduino friendly sensors/actuators appeared in the following years and that it has been 18 years since then, we realized that many performer-luthiers, such as we, and many research and fab labs have accumulated and have stored a lot of electronic components, microchips, microcontrollers, batteries, and similar electronic things. While many of these are or will become obsolete soon, the use of these components at the GPS stage (or even in the final instrument design) signifies less things being bought i.e., less things being accumulated or used once, less demand for the industry to produce such components. This might seem a minor thing, but it was enlightening to us as soon as

Surfing the Landscape of Physical Computing Platforms

947

we looked at our shelves, drawers and compartments and notice the number of things accumulated and used once or twice (and imagining the same for many performerluthiers). We believe we are at a special time for electroacoustic instrument development. The accessibility, computation power and ease to use embedded platforms for audio processing creates the perfect scenario for a big number of electroacoustic instruments to surface, be disseminated, replicated, thus, to develop a repertoire and a pedagogy (i.e. grow a tradition). Hopefully this situation will also present the “world” to new music, new music ensembles and a refreshed idea about what an orchestra can be. Acknowledgments. This research is supported by the project UIDB/00472/2020, funded by Portuguese Foundation for Science and Technology (FCT) through portuguese national funds (PIDDAC). The authors would like to thank CIPEM/INET-md, ESMAD/P. Porto, ESE/P. Porto and João Ferreira (Wasted).

References 1. Chowning JM (1973) The synthesis of complex audio spectra by means of frequency modulation. J Audio Eng Soc 21(7):526–534 2. Jensenius AR, Lyons MJ (eds) (2017) A nime reader: fifteen years of new interfaces for musical expression. In: Jensenius AR, Lyons M (eds) New Interfaces For Musical Expression, vol 3. Springer (2017) 3. Bianchi AJ, Queiroz M (2013) Real time digital audio processing using Arduino. In: Proceedings of the sound and music computing conference, pp 538–545, Stockholm, Sweden 4. Kapur A, Lazier AJ, Davidson P, Wilson RS, Cook PR (2004) The electronic sitar controller. In: Proceedings of the new interfaces for musical expression international conference, pp 7–12 5. Wang G (2012) A history of programming and music. In: Collins N, d’Escrivan J (eds) The Cambridge companion to electronic music. Cambridge Press, pp 55–86. https://doi.org/10. 1017/CCOL9780521868617.006 6. Pennycook B (2008) Who will turn the knobs when I die? Organised Sound 13(3):199–208. https://doi.org/10.1017/S1355771808000290 7. McPherson AP, Jack RH, Moro G (2016) Action-sound latency: are our tools fast enough? In: Proceedings of new interfaces for musical expression, pp 20–25. Brisbane, Australia (2016) 8. Charles JF, Solares CC, Toro-Tobón C, Willette A (2018) Using the axoloti embedded sound processing platform to foster experimentation and creativity. In: Proceedings of new interfaces for musical expression, pp 204–205. Blacksburg, Virginia 9. McPherson A, Tahıro˘glu K (2020) Idiomatic patterns and aesthetic influence in computer music languages. Organised Sound 25(1):53–63 10. Cullen C (2020) Learn audio electronics with arduino: practical audio circuits with arduino control. CRC Press (2020) 11. Mathews MV (1963) The digital computer as a musical instrument: a computer can be programmed to play” instrumental” music, to aid the composer, or to compose unaided. Science 142(3592):553–557 12. Create Digital Music. https://cdm.link/2017/02/heres-cool-handheld-drum-machine-canbuild-arduino/. Accessed 21 Mar 2023 13. GitHub. https://github.com/BleepLabs/Bleep-Drum. Accessed 21 Mar 2023

948 14. 15. 16. 17. 18. 19.

20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44.

45. 46.

F. Lopes and S. Coutinho GitHub. https://github.com/maltman23/ArduTouch. Accessed 21 Mar 2023 GitHub. https://github.com/stg/SyltJamSynth2011. Accessed 21 Mar 2023 Adafruit. https://www.adafruit.com/product/4141. Accessed 20 Mar 2023 Youtube. https://www.youtube.com/watch?v=cKL0n1TkbEU. Accessed 20 Mar 2023 Turriscandura. https://turiscandurra.com/2022/04/ts-det1-microbit-keypad-synth-with-det uning/. Accessed 20 Mar 2023 Element14. https://community.element14.com/learn/learning-center/stem-academy/mic robit/b/blog/posts/project-bbc-micro-bit-8-button-melody-synthesizer-by-cabe. Accessed 20 Mar 2023 Vanheusden. https://www.vanheusden.com/electronics/ESP32-MIDI-synth/. Accessed 18 Mar 2023 GitHub. https://github.com/marcel-licence/esp32_basic_synth. Accessed 18 Mar 2023 Hackster. https://www.hackster.io/news/corwin-point-is-an-esp32-based-synthesizer-fre eform-circuit-sculpture-a3efbd50d398. Accessed 18 Mar 2023 GitHub. https://github.com/ctag-fh-kiel/ctag-straempler. Accessed 18 Mar 2023 Youtube. https://www.youtube.com/watch?v=hRomgKe1OsM. Accessed 17 Mar 2023 Create Digital Music. https://cdm.link/2022/08/nina-12-voice-analog-polysynth-with-motori zed-controls-now-in-preorders/. Accessed 17 Mar 2023 Zynthian Homepage. https://zynthian.org/. Accessed 21 Mar 2023 Bela Blog. https://blog.bela.io/creating-a-guitar-with-trill-and-raspberrypi/. Accessed 28 Mar 2023 Create Digital Music. https://cdm.link/2020/10/omniphone-inclusive-design-music-therapy/, https://arvidandmarie.com/?p=366. Accessed 28 Mar 2023 GitHub. https://github.com/albnys/Drum-Machine. Accessed 28 Mar 2023 Youtube. https://www.youtube.com/watch?v=gL5llIO3NcU. Accessed 28 Mar 2023 Notes and Volts. https://www.notesandvolts.com/2021/02/teensy-synth-extra-2-goldensynth.html. Accessed 28 Mar 2023 Matrix Synth. https://www.matrixsynth.com/2020/01/axoloti-synth-with-enclosure-and. html. Accessed 29 Mar 2023 Otsohavanto. https://otsohavanto.net/axoloti-drum-synth/. Accessed 29 Mar 2023 Create Digital Music. https://cdm.link/2016/02/axoloti-board-makes-sound-box-the-size-ofa-game-controller/. Accessed 28 Mar 2023 Bela Blog. https://blog.bela.io/kuplen-physical-modelling/. Accessed 29 Sep 2023 Bela Blog. https://blog.bela.io/brassynth/. Accessed 29 Mar 2023 Bela Blog. https://blog.bela.io/crescente-electroacoustic-instrument/. Accessed 28 Mar 2023 Bela Blog. https://blog.bela.io/neod-microtonal-instrument/. Accessed 28 Mar 2023 Gear News. https://www.gearnews.com/chompi-club-adorable-daisy-powered-sampler-andlooper/. Accessed 30 Mar 2023 Youtube. https://www.youtube.com/watch?v=7t8ggCvSLRs. Accessed 30 Mar 2023 Hammong Eggs Music. http://hammondeggsmusic.ca/daisy/pollen8.html. Accessed 30 Mar 2023 Look Mum No Computer. https://lookmumnocomputer.discourse.group/t/vocode-o-maticd31-for-daisy-seed-a-31-terts-band-vocoder/1936/1/. Accessed 30 Mar 2023 Berru Homepage. https://berru.pt. Accessed 10 Apr 2023 Ferreira H, Lopes F (2022) Crescente: a new musical instrument for live performance. In: International Conference on New Interfaces for Musical Expression [Internet] (2022). https:// nime.pubpub.org/pub/b3fkwcg7 Lopes F, Rodrigues P (2020) Plantorumori–first report. In: Proceedings of the 5th international conference on live interfaces, pp 49–55. Norwegian University of Science and Technology Lopes F (2019) Old new technologies# 1. In: Proceedings of the 9th international conference on digital and interactive arts

Surfing the Landscape of Physical Computing Platforms

949

47. Nielsen J, Molich R (1990) Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 249–256 48. Masu R, Melbye AP, Sullivan J, Jensenius AR (2021) NIME and the environment: toward a more sustainable NIME practice. In: Proceedings of the international 21st conference on new interfaces for musical expression 49. Herzogenrath B (2012) The farthest place: the music of John Luther Adams. UPNE 50. Small C (1988) Musicking: The meanings of performing and listening. Wesleyan University Press 51. Greie-Ripatti A, Bovermann T (2017) Instrumentality in Sonic Wild{er}ness. In: Bovermann T, Campo A, Egermann H, Hardjowirogo S-I, Weinzierl S (eds) Musical Instruments in the 21st Century. Springer, Singapore, pp 243–262. https://doi.org/10.1007/978-981-10-29516_16 52. Lopes F, Rodrigues PM (2021) Musicking with plants. Perspectives on music, sound and musicology: Research, Education and Practice, pp 133–152 (2021) 53. Lopes F, Guedes C (2020) Composing music with a space. Perspect. New Music 58(1):5–22 54. https://filipelopes.net/wenature. Accessed 2 Sep 2023

Co-streaming: From a Community-Driven Audiovisual Format to a New Consumption Channel in Esports Diego Martín Muñoz(B) Universidad Pontificia de Salamanca, 37002 Salamanca, Spain [email protected]

Abstract. The co-streaming technique is emerging as a new form of digital consumption in e-sports. Its latest major appearance corresponds to the development of the technique in the League of Legends World Cup 2022, through different content creators selected by language. In fact, its importance is beginning to expand to other entertainment industries such as in the premiere of the animated series ‘Arcane’, or in the audiovisual exploitation of the novel and media phenomenon of the ‘Kings League’. The positioning of content creators as a key figure in simultaneous and parallel retransmissions to the main ones, has become a strategy for generating and exploiting audiences. Given its success and expansion, this article aims to relate this novel form of consumption with its first manifestations, which had been developed with a community character and which were produced in a rudimentary way, without showing images and only providing synchronization tools to the community of viewers. In this way, a concrete classification of these manifestations can be established, in order to delimit, differentiate and rename the co-streaming phenomenon. Keywords: Co-streaming · esport · streamer · audience · format

1 Introduction Since the emergence of the Internet in society, including its normalization and democratization as a tool for everyday use, audiovisual consumption has seen its main ecosystem modified. In Spain, for example, it has gone from the exclusive predominance of television at the beginning of the 1990s, with the monopoly established in the generalist government channels, to a myriad of content coming from various online platforms(Nosty 2017). From a video compilation website, such as YouTube, to other streaming services. Whether of movies or music, such as Netflix, HBO or Spotify, (Romay Zamora 2020) which are characterized by broadcasting pre-recorded content already catalogued in their databases; as well as live broadcasts, on Twitch, the preferred platform in the West, or even on YouTube, which has also included this service among its functions, more recently (FundéuRAE 2013). The technological revolution brought about by the advent of the Internet has constituted a new paradigm that has influenced all aspects of society (Castells 2003). For © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 950–962, 2024. https://doi.org/10.1007/978-3-031-47281-7_76

Co-streaming: From a Community-Driven Audiovisual Format

951

example, in the video game sector, this tool has become essential in history, being one of the turning points in the development of electronic sports, which although they had already taken their first steps with primitive tournaments, the arrival of the Internet in the 90’s, established a new paradigm for players, who began to compete against each other, instead of against computer artificial intelligence. A milestone that will mark the future of the e-sports industry (Antón Roncero 2018). At first, and with the emergence of the first professional competitions, thematic cable TV channels were the main means of transmission of these contents, and in fact, they marked a style book in the production and realization of e-sports, but gradually, their audiovisual exploitation was turned to the Internet and the streaming technique. Regardless of whether they belonged to one publishing house or another, or whether their organization and retransmission depended on external companies, the main e-sports events settled on live streaming websites. This is how the main competitions have been configured since 2010, on titles such as League of Legends, Dota2 and Counter Strike: Global Offensive, which have also marked the development of the industry today, due to their longevity and validity. Just as e-sports content has been structured over the years through the streaming technique, so has video game content. At first, on platforms such as YouTube as gameplays or tutorials of the latest titles, and later it is also one of the main protagonists on live streaming platforms. As a result of the online exploitation of video games in general, and e-sports in particular, an audiovisual format has emerged from the community that has become a new consumption channel, co-streaming. Where professional video game competitions and content creators have formed an amalgam that has given new aspects to audiovisual productions.

2 Methodology This research article is organized through the construction of a historical context and theoretical framework around the co-streaming technique. In this way, it provides a glimpse of its origins as a format proposed by the community of content creators. It also structures the process by which some of the publishing houses responsible for the most popular e-sports in the world are beginning to enable and allow certain streamers to make parallel retransmissions of their major championships. Likewise, it is possible to determine, classify and define these broadcasts for future studies that can be derived from the co-streaming technique. One of the difficulties involved in this research work is the lack of scientific literature on this tool, so we will resort to different broadcasts that have been produced to determine and specify the evolution that has occurred. Moreover, as it is a format created in a communitarian way, it is impossible to establish a concrete starting point and to refer to all the broadcasts that have taken place, so special attention will be paid to the most representative and mediatic ones, in different e-sports, and in different countries. In this way, it will be possible to establish an evolutionary line of co-streaming, in order to investigate the main objective of this research work: the definition and classification of the technique.

952

D. Martín Muñoz

In addition, special interest will also be placed on the functioning of the co-streaming technique, especially in its early stages, since it was carried out through specific tools to unite audio and image in a synchronized manner.

3 Internet and Content Creators The first co-streamings were not authorized by the publishers, so they were produced independently. It was done through a multitude of streamers, from the most recognized, to less media figures of the audiovisual ecosystem.. Therefore, the first starting point in the construction of a historical context and theoretical framework on the co-streaming technique is the figure of the content creator. In this way, knowing the functioning of the figures that carry it out, it is possible to understand the phenomenon and its prolegomena. The arrival of the Internet, and the technological innovations that have allowed its empowerment as a communicative tool, has been crucial in the configuration of the figure of content creators. While it is true that this role could be dated earlier in any other profile whose purpose is the creation of any kind of work, the purpose of having a clear target audience, which is the one that forms the community of viewers, who share tastes, skills, customs and even language, and this connection is possible thanks to the Internet tool, is what establishes the differentiation and the turning point in the definition of the figure of the content creator. Creation of identities and sense of identification and belonging (Castells 2000). The Internet is the tool that allows the creator to connect with his community, but the platform that enables this can vary. From social networks such as Twitter, which uses a specific number of characters for users to transmit a text message; Instagram with creations based on photographic resources; or Facebook, which was born as a form of connection between students, and currently experiments with other forms of entertainment; to video compilation websites such as YouTube, which allow the connection of the creator with his community through notifications to subscribers of a particular channel, with the warning of a new content. 3.1 Video on Demand Platforms Despite the different types of content creators, the one that establishes a direct relationship with the co-streaming phenomenon is the one that produces content around that topic, and that does it on a similar platform or of the same nature. And these characteristics are those that YouTube fulfills. It is true that some, such as Vimeo or Dailymotion, have the same characteristics, but due to the impact and repercussion of the former, it will be the one on which the content in this section will be organized. The video web repository was born in 2005 with the aim of combating the need to share videos, since at that time, the most common thing to do was to download them from web pages and play them later on the computer. With the arrival of this platform, videos began to be hosted on the web and were distributed and shared among users with Internet access, without the need for additional software tools or extensive storage space. The success of the platform resulted in 8 million daily visits before its official launch,

Co-streaming: From a Community-Driven Audiovisual Format

953

which led it to be bought by Google in November 2006 for 1650 million dollars (Cloud 2006; López 2018). The subject matter of the videos uploaded to the platform was varied. However, most of them had a very specific characteristic: everyday life. The content generated on YouTube ranged from the behavior of users’ pets, cooking recipes, handicrafts, beauty treatments, motor sports, to content generated by large companies, trailers for the latest movies or the latest songs on the music scene…(DIVIMOVE 2015). Among these themes, one of the main ones had to do with video games, which due to the technological evolution they underwent from the 1980s, went from arcade machines in bars, cafes and arcades, to family homes, with the arrival of the first consoles and personal computers (Kent 2016). This everydayness of the titles gave it the ability to be the protagonist of many of the contents generated on YouTube, and today, it still remains one of the pillars of the video-on-demand company. In fact, according to the company’s 2021 trends report, video games have become pop culture, and this is reflected in the ‘Gaming’s’ category, which is among the Top 10. Minecraft, the most viewed title on the platform has featured in two of the most popular videos on the platform with over 100 million combined views. From this own report, all content related through video games is categorized through the statement “audiences seek community and storytelling through gaming” (Shipley & Szmidt 2021). This context on YouTube has been built through the content creators who have generated their videos and gradually have been configuring and building their community of viewers. It also maintains communication with them through the tools provided by the platform itself, such as the feedback received in the comments of the videos, with the likes or dislikes that users leave on their content, or through the audiovisual impact of the channel in general. All of this also shapes and influences the number of subscribers

Fig. 1. Content creator ‘PewdiePie’ in one of his YouTube videos.

954

D. Martín Muñoz

that a youtuber has and who will be the ones notified when the latter uploads a video again (Berzosa 2017). In the figure above, we can see Felix Arvid Ulf Kjellberg better known as ‘PewDiePie’ one of the most relevant youtubers of the scene, in a gameplay of the video game, previously mentioned, Minecraft. This content creator began his career on the video-ondemand platform in 2010, and although he currently also uses other platforms to make his live content, he built his community of viewers mostly through the early years on YouTube, where he currently has 111 million subscribers (PewDiePie 2023). The case of ‘PewDiePie’ is one of many. In what can be considered the ‘star system’ of the world youtubers (Berzosa 2017; Stokel-Walker 2019). Some of the content creators who have also developed their activity on YouTube, attracting large communities of viewers are: ‘MrBeast’, the individual user with the most subscribers in the world, with 148 million; German Garmendia with his channel known as ‘HolaSoyGerman’ especially in the Latin American community, with 43 million subscribers; as well as Luis Fernando Flores, Salvadoran youtuber known as ‘Fernanfloo’; RubiusOMG’ from the Spanishspeaking community, like ‘Vegeta777’, although both have moved their usual work of content creation to live platforms, they began their professional career on the video on demand platform, where they have 40 and 33 million subscribers, respectively. These cases are due only to individual creators and not to companies, broadcasters or large companies that use YouTube with another perspective and purpose. Although these youtubers do not have a direct relationship with e-sports and its broadcasts, they do define the role of content creator, of the ‘star system’ of Internet consumption and of a content that has as main protagonists these figures, who represent an indispensable element of the ecosystem. And this can be extrapolated to e-sports broadcasts through co-streamings.

4 Esports Broadcasts Despite the fact that nowadays, e-sports are mostly consumed through streaming platforms, and in the case of the West through Twitch, television has also played a fundamental role in the development of professional competitive video games. In addition to helping the normalization of a stigmatized sector, it has also created a stylebook that has shaped a very specific operation when it comes to e-sports coverage (Hiltscher & Scholz 2015). Regardless of the platform on which the content is broadcasted, the operation of esports broadcasts has a specific path: The publishing house of the video game, creates a professional competitive event or gives permission to a specialized external company to organize it, then they themselves, or the company that manages the event, is responsible for the broadcasting of the program, and all the aspects that make it up, such as logistics and production, so that the show is produced according to the established standards, After this celebration, the broadcasting rights are transferred or sold to the specific platform that is deemed appropriate -currently streaming websites have monopolized this service, after that, the event signal returns either to the videogame publisher -in many occasions this signal is managed through local companies with commercial agreements with the parent company-, or to the event organizer, who through a specific channel and the

Co-streaming: From a Community-Driven Audiovisual Format

955

chosen platform takes the content to the viewer. In short: The company organizes an esports event, passes the content through a platform and broadcasts it through a particular channel on the web, that of the signal operator (Newzoo 2022). One of the keys to the audiovisual exploitation of e-sports is the relationship between the videogame publisher and its competition, and the degree of involvement in it. If they use external companies that manage and organize the competitive scene of the title, it can be described as an open model; if, on the contrary, it is the publishing house that takes full charge of the competition of its video game, we can say that it is a closed model (Antón Roncero 2018, p.315). The importance of this classification in the different video games lies in the choice of a different audiovisual exploitation strategy depending on the type of model used by each company with its product. In addition, another key to the audiovisual exploitation of an e-sport is its signal operator. In many cases these are either sections of the parent company or the company organizing the event itself, but in other cases and for very specific audiovisual markets, or those that have a specific specialty -see language- the main responsible decide to use a local company that can serve as a transmission vehicle. For example, in Spain and much of Latin America, the main operator of the e-sports signal is the Liga de Videojuegos Profesional, -here in after LVP-, which belongs to the Mediapro audiovisual group, and in addition to organizing its own competitions on some titles at the state level, it is also the manager of the signal. In e-sports such as League of Legends or Valorant, it is the main responsible for broadcasting the competitions held worldwide or continental level in Spanish, for the members of the Spanish-speaking community (Liga de Videojuegos Profesional 2022). These are the characteristics and requirements of e-sports broadcasts in general. Always depending on the rights of the specific publishing house and establishing permissions and authorizations for organizations or retransmissions.

Fig. 2. CS: GO Final of ESL One Katowice 2015

In the image, you can see the final of the ESL ONE competition of 2015, on the video game CS: GO, which is organized by the eponymous company. In this case and

956

D. Martín Muñoz

following the structure outlined above: The publisher of the title Counter Strike: Global Offensive, is Valve, which maintains an open management model on the competitive structure of its title. In this way, it allows the company ESL to organize the tournament and take charge of the audiovisual coverage of the same. This competition could be seen through Twitch in the channels enabled by the company responsible, and operated by different sections of the parent company, to bring the signal in different languages (ESL España 2015). In these broadcasts, the players involved are publishers, tournament organizers, platforms and signal operators. In this type of e-sports broadcasts, which are the most classic and common, content creators do not enter. At least, not in an authorized way. 4.1 Co-streaming With a broadcasting scheme focused on the audiovisual rights of e-sports and the commercial agreements of the companies, the role of the content creator, who had built his community of viewers around one or several specific titles, could not enjoy competitive games of the highest technical, tactical and strategic level with the regular users of his channel. A content that, due to the subject matter they have dealt with since their beginnings on video-on-demand platforms, could fit into their style and content grid. Many of the content creators that emerged through platforms such as YouTube, transferred or complemented their content creation on live broadcasting websites, such as Twitch -in fact many of them reuse it on both platforms-, and as we have seen above, it is on streaming platforms where e-sports have found their habitat and their way to reach their viewers, practically exclusively. For this reason, and given the impossibility of being able to show the image of the games through their live broadcast channels, many of them began to make content qualified as “reaction to the corresponding e-sports event” in this way, viewers could know the opinions of the creator of specific content. Despite this, viewers were still unable to see the content generated by a League of Legends or Dota2 game, they could only enjoy the comments made by the streamer, and in some cases view the game on another screen, although the synchronization of the signals was complicated. Unofficial This strategy is based on showing the streamer’s reaction and comments on content that he cannot show to his followers. What we qualify as an unofficial format. Publishers or tournament organizers set limitations on the content they can broadcast from competitions, other agents, so many content creators chose to show only their reaction to that event. After this, and the popularization of this type of format, tools and techniques were created to improve the experience. Such as the creation of communicative spaces that allowed the synchronization of signals directly by the consumer. For example, content creators began to use graphics that gave clues or reflected the exact moment, through a stopwatch that marked the same time the game was taking. In this way, followers could synchronize the signals in a more efficient way. Despite this, live streaming platforms offer limitations in terms of timeline manipulation, so exact synchronization was often not possible. At least on streaming websites. On YouTube for example, which was born as a video repository website, but ended up

Co-streaming: From a Community-Driven Audiovisual Format

957

including live broadcasting options among its tools, became part of the ecosystem of e-sports broadcasting, and this website does allow interaction with the timeline of the broadcast, which made it easier for users to synchronize the signals. The unofficial co-streaming technique, therefore, requires two broadcasting signals: first, the one of the competitions itself that the user wants to watch; and secondly, the one of the streamers who comments on aspects or gives opinions about the game itself. These signals can be synchronized manually by the users, or through web pages or specific software that allows it. This is the case of Liveviewsync, which offers the possibility of synchronizing broadcasts through basic time controls.

Fig. 3. Liveviewsync webcast synchronization interface

Through this platform, the user can synchronize the two broadcasts. On the left side of the screen you can select the channel corresponding to the competition that the user wants to consume, which will be reflected in the left box; on the right side you can do the same but with the content creator or streamer who is reacting to the game in his live. In addition, you can also select and choose the chat in which to interact with other users, either belonging to the competition or to the content creator. With the time controls below the first screen, the user can modify the timeline of the video of the competition and thus synchronize the two corresponding signals (Ayora 2020). Once we have unraveled the functioning of the costreaming that occurs unofficially, we can highlight some of the broadcasts with this dynamic and that have been consumed by many users through the platform shown in Fig. 3. One of the most mediatic unofficial co-streamings took place in the Spanish audiovisual ecosystem and it was through the content creator Ibai Llanos. Llanos, who had previously worked as an e-sports caster in the LVP, in the video game League of Legends almost exclusively, after leaving this operator, began to develop his professional profile as a streamer, and one of the first major broadcasts he made was the co-streaming of the League of Legends world championship in 2020, which he called “Radio Worlds” (Esportmaníacos 2020). In this event, the Spanish streamer had former colleagues from her time in the Professional Video Game League, other content creators, and experts of the League of Legends video game to accompany her in the broadcast.

958

D. Martín Muñoz

In addition to this rebroadcast, one of the most important e-sports programs on the scene, “Esportmaníacos” also made its own co-streaming of the same event – it should be noted that in addition to making different audiovisual content on e-sports, this channel is a regular of the unofficial co-streaming, through many of the collaborators of its program. With this, together with “Radio Worlds” and the official broadcast in Spanish by the LVP, all audience records were broken to date, reaching almost 200,000 viewers (Díaz 2020). Official As a result of many of these demonstrations being done through synchronization tools, some publishers decided to start allowing some content creators to broadcast their eSports games through their personal streaming channels. In this way, there was no need to use two different broadcasts and their corresponding synchronization. It was an official signal with the corresponding content creator’s commentary. One of the main companies that started with the authorization of these co-streamings was Riot Games, developer of video games such as League of Legends and Valorant, some of the most popular e-sports at European level (ISFE ESPORTS 2021). The first manifestation of official co-streaming took place in 2018, at the League of Legends regional qualifiers in the North American region, and after the pilot experience it was extended to the competition called Rift Rivals, an international League of Legends tournament that pitted the best teams from North America and Europe -competitive regions of eSports- although the streamers authorized to carry out coverage of the event were limited to users and teams from the region called LCS -denomination of the top League of Legends league in North America- (LOLESPORTS STAFF 2019). Since then, this new content broadcasting channel has become popular on the North American continent and has taken place in subsequent editions of the top league competition with the collaboration of teams, content creators, and even former players who have participated in the development of co-streamings (LCS 2022; LOL ESPORTS STAFF 2020; LOLESPORTS STAFF 2021). As in League of Legends, Riot Games, which is also the developer of the video game Valorant, has also allowed the development of co-streamings in the professional competitions of this title. In addition, it has followed a strategy similar to that used in League of Legends, as it has started with pilot tests in North American competitions, and is gradually applying them to other competitions and regions worldwide (Riot Games 2021; Valorant Championship Tour 2021, 2022). In this way, one of the main publishers of the e-sports scene is beginning to enable co-streamings as a new way of consumption. In addition, in its role and development in other countries it has also applied it: in League of Legends competitions belonging to the European Emea Masters, which is composed of regional leagues circumscribed to specific countries or areas in the European continent from 2022. In addition, it has been extended to competitions in this same category in 2023. The indications for its celebration, on the part of the publishing house, was the organization of co-streamings through the teams participating in the leagues, represented by a content creator; the same characteristics that have also been authorized for the realization of this type of coverage in the League of Legends European Championship – known as LEC – the maximum competition of the continent, although with the particularity of having only been authorized for two of the participating teams in the competition: KOI, a recently

Co-streaming: From a Community-Driven Audiovisual Format

959

created team belonging to the Kosmos business group, along with content creator Ibai Llanos; and Team Heretics, founded by the also streamer ‘Grefg’. Surely, as a strategy to follow their followers in Spain, which is the nationality of both teams and where most of their followers are located. Although the largest manifestation of co-streaming to date has occurred at the League of Legends World Cup Final in 2022, with the permission and authorization of the publisher. The ‘Worlds’ as it is popularly known among eSports fans, is the most awaited event by the eSports public every year and is capable of generating audiences in the millions worldwide, which has earned it on several occasions, including the one held in 2022, as the most popular esports tournament in the world (Murko 2022). And one of the main reasons for the popularity of e-sports, has been the realization of different co-streamings of the final. Riot Games authorized five content creators, representing English, Chinese, Korean and Spanish, to cover the event. (LoL Esports 2022). This has influenced the amount of viewership generated by the final, which has been the most watched in the history of eSports with more than 5 million viewers worldwide -not counting the Chinese audience due to methodological aspects in audience measurement-, with a wide contribution from parallel broadcasts (Esports Charts 2022). The content creators who have carried out these retransmissions have been: Sykkuno, in English; Rita, in Chinese; Mingyo Kim and Sangho Lee, in Korean; and Ibai Llanos, in Spanish. The latter, who has been and is the protagonist of several co-streamings, put together a team of participants for the broadcast that included the content creator and Sergio ‘Knekro’ García, the historical commentator of League of Legends in Spanish, Alejandro ‘Kuentin’ Cotrina, and the player of the Mad Lions team, Javier ‘Elyoya’ Prades. In Spain, in addition to the co-stream by content creator Ibai Llanos, there has also been the official broadcast by the signal operator, the LVP. Between the two streams, almost half a million viewers have been reached on average, with the peak audience reaching over 600,000 viewers (Twitch Tracker, 2022b, 2022a). In this way, it has surpassed the 2019 edition as the most watched production in Spain so far, almost tripling the data obtained (LVP 2019).

5 Discussion and Conclusions The main objective stipulated in the methodology of this communication was to recognize co-streaming as one of the ways of audiovisual exploitation in e-sports, and also, knowing what and how it has been its evolution, from a format created by the community of content creators, to have the ability to discuss and differentiate the different categories that can be produced from it and that can help in future research work. What we have called unofficial co-streaming because it does not have the necessary permissions granted by the publisher of the video game, we have detected the importance of the words “synchronized” for its importance in the consumption for the viewer, “independent” for its belonging to specific content creators and not to companies dedicated or linked to the audio-visual exploitation of e-sports, and “broadcast” for its character of content transmitted but unrelated to the retransmission to which they refer. Therefore,

960

D. Martín Muñoz

Fig. 4. Promotional poster for the co-streaming of Ibai Llanos for the League of Legends World Cup final

the term “independent synchronized broadcasting” seems to us to be the most appropriate for events produced through co-streaming without the authorization of the relevant companies. Therefore, and in an exclusive manner, the manifestations that we were naming as official, we can now name them only as co-streamings, since there is no coincidence, as there was before. Table 1. Distinction of the term co-streaming Co-streaming

Renaming

Official

Co-streaming

Unofficial

Independent synchronized broadcasting

In addition, the evolution of the co-streaming format with the first manifestations through synchronization with external tools, towards the technique currently used and defined exclusively as co-streaming, can serve as a basis for future works that follow this line of research, with the need to provide other contents that nurture the evolution and that can even establish more specific time periods.

Co-streaming: From a Community-Driven Audiovisual Format

961

References Antón Roncero M (2018) Los deportes electrónicos (esports): el espectáculo de las competiciones de videojuegos. Universidad Complutense de Madrid Ayora V (2020) Cómo ver en directo los Worlds con Ibai totalmente sincronizados – Movistar eSports. Diario AS. https://esports.as.com/worlds-2020/como-ver-directo-ibai-worlds_0_1 398160175.html Berzosa M (2017) Youtubers y otras especies. https://docreader.readspeaker.com/docreader/?jsm ode=1&cid=bvqgq&lang=%27es_es&url=https%3A%2F%2Fpubliadmin.fundaciontelefo nica.com%2Fmedia%2Fpublicaciones%2F581%2FYoutubers_y_otras_especies.pdf&_ga=2. 163800191.1769618526.1572955228-2052655585.1552556195&r Castells M (2000) Internet y la Sociedad Red (Universitat Oberta de Catalunya, Ed.). Universitat Oberta de Catalunya Castells M (2003) La dimensión cultural de internet. Andalucía Educativa 36:7–10 Cloud J (2006) The YouTube Gurus – TIME. TIME, 25 December 2006. https://content.time.com/ time/magazine/article/0,9171,1570795-5,00.html Díaz C (2020) Los datos de audiencia de la final de Worlds 2020 - Movistar eSports. Diario AS. https://esports.as.com/worlds-2020/millones-personas-pico-final-Worlds_0_1406 259363.html DIVIMOVE (2015) Contenidos que triunfan en youtube divimove. Slideshare, 8 July 2015. https:// es.slideshare.net/OptimediaSpain/contenidos-que-triunfan-en-youtube-divimove ESL España (2015) ESL One Katowice rompe los récords de visualizaciones de CS:GO - News | ESL Play. ESL Play, 27 March 2015. https://play.eslgaming.com/news/254758/ESL-One-Kat owice-rompe-los-r-cords-de-visualizaciones-de-CS-GO/ Esportmaníacos (2020) Ibai y la G2 House van a dar los Worlds en modo radio - Esportmaniacos. Esportmaníacos, 18 September 2020. https://www.esportmaniacos.com/lol/ibai-y-la-g2house-van-a-dar-los-worlds-en-modo-radio/ Esports Charts (2022) LoL Worlds 2022 – Viewership and Detailed Stats. Esports Charts. https:// escharts.com/tournaments/lol/2022-world-championship FundéuRAE (2013) «en directo» y «en continuo», alternativas a «streaming» | Fundéu. Fundéu. https://www.fundeu.es/recomendacion/en-directo-y-en-continuo-alternativas-a-streaming/ Hiltscher J, Scholz TM (eds) (2015) 2013/14 eSports Yearbook. Books on Demand GmbH ISFE ESPORTS (2021) The guide to esports Kent SL (2016) La gran historia de los videojuegos (SA Ediciones B (ed.)), 1°edn. Three Rivers Press LCS (2022) LCS en Twitter: “Watch the #LCS Super Week with our featured co-streamers starting at 2:30PM PT/5:30PM PT today!, 25 February 2022. https://t.co/Aca0ArNJZ3”/Twitter. Twitter. https://twitter.com/lcsofficial/status/1497285584175304705 Liga de Videojuegos Profesional (2022) ¿Que es la LVP? | Somos la Liga de Videojuegos Profesional. Lvpglobal. https://lvp.global/quienes-somos/ LOL ESPORTS STAFF (2020) Co-streaming returns for the 2020 LCS Summer Split. Lolesports, 5 June 2020. https://lolesports.com/article/co-streaming-returns-for-the-2020-lcs-summer-split/ blt68e116af3db30bf4 LOLESPORTS STAFF (2019) Co-Streaming Pilot Program Coming to Rift Rivals. Nexus League of Legends. https://nexus.leagueoflegends.com/en-us/2019/06/co-streaming-pilot-pro gram-coming-to-rift-rivals/ LOLESPORTS STAFF (2021) 2021 LCS Co-streaming Policy. LoL Esports, 15 January 2021. https://lolesports.com/article/2021-lcs-co-streaming-policy/bltdb637d457b785a73 López JM (2018) La historia de YouTube antes de Google. Hipertextual, 8 December 2018. https:// hipertextual.com/2018/12/nacimiento-youtube

962

D. Martín Muñoz

LVP (2019) Récord histórico de espectadores en la final de los Worlds. Lvp.Global, 11 November 2019. https://lvp.global/final-worlds-audiencias-lvp/ Murko D (2022) Most popular esports tournaments in 2022. Esports Charts, 21 December 2022. https://escharts.com/news/most-popular-esports-tournaments-2022 Newzoo (2022) Global Esports & Live Streaming Market Report. https://resources.newzoo. com/hubfs/Reports/Esports/2022_Newzoo_Free_Global_Esports_Live_Streaming_Market_ Report.pdf?utm_medium=email&_hsmi=212483853&_hsenc=p2ANqtz-9hyzZzkLnQvA lATpRvK0ou-K3YU1blYYi5K0w8vcWH4rIQPxYR8pOVVwKQRmVQ_xiN6y8WMXCiG G0zJnekQxTc5JCiLA&utm_content=212483853&utm_source=hs_automation Nosty BD (2017) Diez años que cambiaron los medios (Fundación Telefónica (ed)). Ariel. https:// www.fundaciontelefonica.com/cultura-digital/publicaciones/602/ PewDiePie (2023) PewDiePie - YouTube. YouTube. https://www.youtube.com/@PewDiePie Games R (2021) NA VCT Watch Party Guidelines. Play Valorant, 7 October 2021. https://playva lorant.com/en-us/news/esports/na-vct-watch-party-guidelines/ Romay Zamora M (2020) CONSUMO TELEVISIVOS DE LOS ESPAÑOLES HACIA LAS NUEVAS PLATAFORMAS DE STREAMING: NETFLIX [Universidad Pontificia de Comillas]. https://repositorio.comillas.edu/xmlui/bitstream/handle/11531/36977/ CAMBIO HABITOS CONSUMO TELEVISIVOS DE ESPANOLES HACIA PLATAFORMAS DE STREAMING NETFLIX - 201604345.pdf?sequence=2&isAllowed=y Shipley K, Szmidt N (2021) 2021 year-end YouTube video trends. Think with Google, November 2021. https://www.thinkwithgoogle.com/marketing-strategies/video/2021-youtube-trends/ Stokel-Walker C (2019) YouTubers: How YouTube shook up TV and created a new generation of stars. Canbury Press. https://books.google.es/books?hl=es&lr=&id=mMKWDwAAQ BAJ&oi=fnd&pg=PP1&dq=star+system+youtubers&ots=1wSjm9hOt_&sig=NewDwSl6K 1JFGlMIc1zJdCxnbZU#v=onepage&q=star%20system%20youtubers&f=false Twitch Tracker (2022a) ibai - Stream Nov 05, 2022 - Stats on viewers, followers, subscribers; VOD and clips · TwitchTracker. Twitch Tracker, 6 November 2022. https://twitchtracker.com/ ibai/streams/41466720779 Twitch Tracker (2022b) LVPes - Stream Nov 05, 2022 - Stats on viewers, followers, subscribers; VOD and clips TwitchTracker. Twitch Tracker, 6 November 2022. https://twitchtracker.com/ lvpes/streams/46267082652 Valorant Championship Tour (2021) VALORANT Champions Tour en Twitter: “Let’s get the party started! Meet your hosts for the official #VALORANTMasters Berlin watch parties , 3 September 2021. https://twitter.com/ValorantEsports/status/1433807918604849156?s=20 Valorant Championship Tour (2022) VALORANT Champions Tour en Twitter: “Watch with us or with your favorites! Check out the #VALORANTChampions Watch Party hosts, 28 August 2022. https://t.co/WrbOugTx7o”/Twitter. Twitter. https://twitter.com/ValorantEsports/status/ 1563949794028097537

Directional Acoustic Vehicle Alerting Systems: A Review of the Literature and Simulation Model for a Shared Zone Closed Car Park Frederico Pereira(B) and Emanuel Sousa Human-Technology Interaction and Robotics” Domain, Centre of Computer Graphics, Campus de Azurem, 4800 058 Guimaraes, Portugal [email protected]

Abstract. The widespread adoption of electric vehicles presents an opportunity to reduce the contribution of road traffic to noise pollution. However, concerns have been raised about the increased danger that quieter vehicles pose to other road users, as they may be less detectable. To address these concerns, Acoustic Vehicle Alerting Systems (AVAS) are currently legally mandatory in many parts of the U.S. and Europe. However, the addition of noise from AVAS has sparked controversy regarding missed opportunities for a quieter environment. To mitigate this, various researchers have developed and evaluated concepts for a directional sound source to be used in AVAS, which could address the issue of detectability while minimizing noise pollution. This work reviews the existing literature on vehicle directional alerting concepts and evaluate their effectiveness in the particularly concerning case of a shared zone closed car park, using an acoustic simulation model. Sound levels are measured for scenarios with directional AVAS and conventional AVAS (non-directional). Keywords: AVAS · Electric vehicles · noise

1 Introduction The recognition of the damaging contribution of Internal Combustion Engines (ICE) vehicles emissions to atmosphere pollution has accelerated the development of the electric power unit technology. In the U.S., electric vehicles (EV) has since 2011 grown from to 0.2% of total sales to 4.6% in 2021, and some estimates foresee it reaching 40% to 50% by 2030 [1, 2]. Along with the prospective decrease of air pollution in urban environments, the popularization of electric vehicles brings an opportunity for a significant reduction of traffic noise (with noise here designating unwanted or unnecessary sounds, i.e. of negative subjective impression). The adverse effects of traffic noise have long been acknowledged, as evidenced by (simplistic) regulations existing in the Roman Empire, which restricted circulation of chariots in stone streets during nighttime [3]. Indeed, sources of noise, artificial or natural, have always existed, but not with the intensity experienced in modern urban © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 963–975, 2024. https://doi.org/10.1007/978-3-031-47281-7_77

964

F. Pereira and E. Sousa

environments, with a significant share resulting from road traffic [4] For these settings, research on the impact of sound pollution to well-being has identified road traffic noise as a major cause for annoyance [5–7]. Notably, studies evaluating the differential of annoyance resulting from ICE to EV vehicles revealed that annoyance ratings were generally lower for EV vehicles [8]. However, quieter vehicles present a two-fold challenge when considering the utility that pedestrians have grown accustomed to take from combustion engine noise. These “annoying” sounds end up conveying important information, serving as alert signals, calling the attention to approaching vehicles and thus providing a sense of safety. EVs emit lower levels of noise, which makes them more difficult to detect by pedestrians in their surroundings, particularly at speeds below 30 km/h. In such situations, EVs can be up to 10 dBA quieter than ICE vehicles. As a result, pedestrians may only detect EVs when they are less than 5 m away, whereas an ICE vehicle could be detected at a distance of about 50 m. Background noise is a major factor in masking the sound of EVs, especially in scenarios with high background noise and low-speed traffic, such as shared spaces like car parks, where the issue of detectability is exacerbated [9, 10]. As pointed out by [11] the risk of pedestrian accident with an EV is double that of with ICE vehicles. Despite a lack of consensus [12, 13] the issue has been recognized, and due to pedestrian safety concerns (with particular attention to the visual impaired population), regulations have been set in place requiring manufacturers to install an Acoustic Vehicle Alerting System (AVAS) [14, 15], a loudspeaker based system that generates a sound when the vehicle drives below a certain velocity, in order to alert pedestrians to its presence. The proposed regulations are not free of controversy. Research on the noise pollution effects of generalized EVs adoption, showed that there would be a significant noise reduction (contrasting with ICE) for velocities below 30 km/h, although the use of AVAS turns this reduction less pronounced [16]. The trade-off between mitigating noise pollution and ensuring pedestrian safety through the use of AVAS has been a contentious issue, with critics arguing that AVAS limits the potential for a quieter environment. These arguments prompted a discussion about alternatives that can address the issue of detectability without compromising noise reduction. Some views point to a change in behavior of EV drivers, while others suggest an adaptation from pedestrians in their road crossing practices [17]. However, the adoption of AVAS provides an opportunity to reassess the effectiveness and acceptability of vehicle alerting systems by implementing sound design strategies that address the specific problem at hand. These aim to create alerting sounds that ensure high detectability while minimizing annoyance. Currently implemented AVAS have demonstrated good results on improving vehicle detection by pedestrians [18, 19], but space for improvement is identified in the subjective ratings for sound preferences [8, 20]. Regulations in place set the sound design requirements’ focus on the frequency content and noise levels for AVAS solutions, but little to no guidelines are given for devices’ radiation patterns (a requirement is set out for stationary vehicles in [15] that emphasizes energy in the 800–1250 Hz range). The ability to shape sound directionality to attend to specific needs can and should be accounted for in sound design strategies.

Directional Acoustic Vehicle Alerting Systems

965

Directional systems may be play a crucial role on the successful deployment of vehicle alerting systems for pedestrians [21]. The objective of this study is to provide an overview of the potential solutions for directional systems for vehicle auditory warnings. A simple case scenario of a shared space is investigated to assess the prospective benefits of such a system to noise pollution reduction.

2 AVAS Directional Solutions 2.1 Literature Review Directional sound sources have been extensively investigated and are well documented in the literature, with classical examples being loudspeaker array technologies for sound reinforcement. A share of the recent decades’ investigation was dedicated to the topic of private sound-zones. Researchers developed various loudspeaker array beamforming and beamsteering techniques to explore the ability to increase the sound energy in a restricted direction, such as when transmitting information to a specific target location. Beamforming relies on the combination of output signals from non-directional sources to form a directional source, and beamsteering is based on the relative phase manipulation of the individual sources, granting the ability to steer the focused energy beam. The underlying basic principle is the constructive/destructive interference caused by the collective interaction of the various sources emitted energy. Target locations of focus are commonly deemed bright zones, and dark-zones the ranges of minimized energy [22, 23]. The ability to control the radiation spatial pattern of a vehicle warning sound affords the possibility to focus the radiated energy into restricted directional ranges, as for example, in the direction of vehicle motion, or even, towards an identified vulnerable target in risk of collision, while limiting sound energy in directions of other road users (Fig. 1). Aware of the potential benefits for safety and noise reduction that a directional warning system could offer, Pompei in 2001 patented a concept design with applicability to vehicles [24]. Similarly, directional warning systems were a central topic of the pioneering research project eVADER [25] and a concept for a “next generation” sound alarm system precluding directional sound radiation was investigated and developed. In a 2013 paper [26], Berkhoff and Rots, acknowledging the risk posed by quiet EVs foreseen the ideal warning system, one that would inform a target while minimizing noise pollution. Based on a broadside 8-element loudspeaker array, the authors explored different signal processing algorithms for a real-time beamforming system. Following this primary study, an eVADER consortium 2014 publication [19] examines, through numerical simulations, the application of a 6-element directional system to be installed in the front bumper of an electric vehicle (Fig. 2). Cheer and colleagues put forward an analysis of two popular loudspeaker array configurations, a broadside linear array and an end-fire array [27]. Noting the higher efficiency of the end-fire configuration but predicting adoption resistance from manufacturers due to its cost and impracticality, the authors suggested an alternative low-cost solution for an end-fire configuration containing only one loudspeaker. On the basis of

966

F. Pereira and E. Sousa

Fig. 1. Scheme of a directional system, with focus of energy in the bright zone, at the frontal direction (gradient fill) and minimization of energy in the dark zone (pattern fill)

Fig. 2. Example depiction of a front bumper installed array for directional sound radiation. Sound levels are to be maximized in a target zone and minimized in remaining directions.

its operation is a perforated pipe that is attached to the loudspeaker, with a number of holes along its length. While the loudspeaker driver generated sound wave travels along the pipe, each hole mimics an individual sound source with a delay relative to its distance to the driver. An eVADER proposed system prototype was installed on the front bumper of a passenger vehicle (Nissan Leaf). It comprised a sophisticated software for the control of a broadside configuration array of loudspeakers capable of directional sound radiation. The proposed concept included environmental sound monitoring (from 4 microphones) and

Directional Acoustic Vehicle Alerting Systems

967

pedestrian detection (cameras system), enabling an automated adjustment of beam direction, directivity, and loudness levels for the emitted auditory alarm. In a 2014 publication [28] details and testing results were described. The solution employed a 6-loudspeaker array with non-uniform spacing, managed by a DSP system employing sound-power minimization beamforming algorithms. During the development stage, instead of loudspeakers, inertial mass shakers were considered due to their advantages over the former, which included better protection against the environment and easy installation, while maintaining high quality sound output. However, this solution requires a more powerful amplifier and its acoustic performance dependance on panel and other car body parts characteristics, led to the discarding of the shakers option. Instead, loudspeakers were installed in the final prototype. Results of directivity measurements were satisfactory, with the system achieving 10 dBA attenuation from the lobe maximum in a −60 to + 60 steer direction angles (measured at 5 m radius), and 20 dBA attenuation behind the vehicle [29]. One of the tasks of the eVADER project was to design an alert sound for pedestrian warning. To this end experiments were carried (in a virtual environment), where subjects would judge several sounds for detectability and annoyance. The most appropriate sound was found to be 6 dB quieter and less annoying than regular ICE vehicles noise [30]. The eVADER project was quite comprehensive and pioneered the application of directional sound alarm systems for EVs, however, in the following years, (as far as these authors are concerned) it did not produce publications reporting experimental listening tests on pedestrian perception/detectability, nor results on the effective sound pollution reduction achievable by the conceptualized system. Inspired by the eVADER project, Lee and colleagues [31] contemplated an Active Pedestrian Alerting System to be installed in a vehicle. An array of 6-loudspeakers was designed with an operation range complying with European and U.S. vehicle alarm regulations. An optimization process was based upon three principles, SPL contrast at target, driver’s comfort, and alarm regulations. The study conducted, through numerical simulations, a detailed analysis on the influence of various speaker parts on sound emission, as well as evaluation of vertical positioning of the array at the front bumper of a vehicle. Speaker parts were found to importantly affect system response, and a positioning at lower height was deemed favorable, however, in the publication, distances to the ground were not specified. A prototype was installed in a vehicle and measurements revealed efficient directivity patterns, with plots showing approximately 20 dB difference for frontal directions, and 10–15 dB for lateral angles steering. Yet, as in previous studies, unfortunately, perceptual validation listening tasks were not published. (Reasons for the absence of these listening tasks may pertain to the challenging design of an experimental task, that would involve a directional source and dynamic source-listener interaction in a controlled environment?). On the wake of previous investigations, Kournoutos and Cheer [32–34] advanced a series of publications seeking to offer a less expensive and more robust solution for a vehicle directional warning sound system. The authors proposed two alternatives, a low-cost end-fire array employing a single loudspeaker (a system previously investigated in 2013), and a structural array consisting of 6 dynamic inertial actuators acting upon a rectangular aluminum panel integrated to the car body. The latter, having a superior

968

F. Pereira and E. Sousa

robustness, is based on sound radiation by inflicting a controlled vibration of the panel through adjustment of relative amplitude and phase of the actuators. A parametric analysis identified the actuators’ number, layout, panel material, geometry and boundary conditions play a determining role in the resulting vibration patterns. Better directivity is achieved with more actuators and longer panels, steering performance is better when employing regular spacing between actuators, while frontal performance benefits from uneven spacing. Analytical models were formulated for both concepts and implemented in a simulation software. Radiation patterns for the two models were put against the solution for a conventional 6-loudspeaker broadside array. The analytical model for the structural actuators solution is defined with 6 actuators with identical geometrical disposition as in the broadside loudspeaker array. Model results showed that loudspeaker array and structural actuators maintain good directivity throughout the frequency range from 40 Hz to 5 kHz, displaying a spread of energy in the higher frequency range (>1 kHz). It is noted that the structural actuators pattern, in comparison with the loudspeaker array, displays a contrast low frequency limit at about 150 Hz, but a better focus performance in the higher range. The end-fire array shows low directivity up to 945 Hz, with better performance until 2 kHz, from where side lobes start to be quite noticeable [32]. Further investigating the structural actuator solution, to validate the simulation results a simple physical prototype of the concept was built and measured. The performance of the prototype matched well the simulation results from 400 Hz to 3 kHz, outside this range expected deviations were noticeable, more so in the lower frequencies [35]. Subsequently, a concept design was implemented in a commercial vehicle and different install locations tested (hood, front bumper, and front door). Determining a targeted bright zone width of 36o and a steering of up to 72o, directivity measurements were conducted in an anechoic chamber using a circular array of microphones. The front bumper installation was found to be preferred due to its cost/efficiency balance. The collected data showed that a directional behavior is well established within 200 Hz–5 kHz, maintaining an acoustic contrast over 10 dB (an identical behavior as previously indicated by the simple prototype measurement, and analytical model simulation). Overall, measurement results proved the proposed structural array mounted on the vehicle front bumper as a feasible, identical performance low-cost alternative for a loudspeaker based directional warning sound system [34]. Although comprehensive and rigorous measurements were carried out for an objective characterization of the sound radiation, (as far as the authors are aware), no perceptual studies with human subjects were conducted. In recent years, several patents have been filed for directional AVAS systems (as confirmed by a Google Patents search) but despite this surge, at the time of writing, no EV production model includes an AVAS with directional radiation capability.

3 Case Scenario: Shared Zones The identified concerns over the lower noise emitted by EVs and the arguments against AVAS adoption are of particular interest in pedestrian and vehicles traffic shared zones. Generally, in these conditions, vehicles drive at lower speeds, and interaction with other road users is frequent. In the case of closed spaces, as often are public car parks, strategies

Directional Acoustic Vehicle Alerting Systems

969

to limit the buildup of energy due to long reverberation times, an aggravation factor to the nuisance and detection issues, may benefit from directional AVAS systems. As to grasp the beneficial contribution that a directional AVAS might have in mitigating sound pollution in such scenario, a simulation model of an underground car park was developed. The model geometry and materials represent a real-world car park located in the city of Guimaraes (Portugal). Field measurements were taken to assess acoustic conditions and validation of the implemented model. Reverberation time RT60, 1 kHz reference was measured at 4.4 s, during busy hours, position averaged SPL background measurements gave Leq = 59.5 dBA (10 min. Recording). 3.1 Model Simulation

Fig. 3. Analytically derived (normalized) power pattern (dB) for a broadside transducer array of 6 elements with 13.7 cm spacing. From top, left to right, 250 Hz, 500 Hz, 1 kHz, 2 kHz, 4 kHz and at bottom right 5 kHz. A baffle is introduced, restricting results to one hemifield.

970

F. Pereira and E. Sousa

Acoustic scenarios of vehicle traffic with and without directional AVAS were simulated using as sound stimuli for vehicle noise 250 Hz–5 kHz band limited white noise. Taking into account current AVAS legislations[14, 15] sources’ sound power was defined as to produce at 1m distance frontal direction an SPL of 62 dBA, directional radiation was modeled considering a 6-loudspeaker broadside array installed on a vehicle’s front bumper. Directional radiation pattern was determined based on the array configuration analytical model and conclusions of Kournoutos and Cheer prototype investigations [34] (Fig. 3). Most currently implemented (conventional) AVAS use one or two loudspeakers under the vehicle bonnet. The car body, effectively works as a baffle, shielding energy propagation towards its rear. The modeled conventional (non-directional) AVAS radiation pattern was established in agreement with existent literature denoting current systems as having no significant directivity [31, 36, 37]. The modeled pattern is in essence a uniform frontal hemispherical sound radiation, with an attenuation of −20 dBA towards the rear (Fig. 4).

Fig. 4. Modeled freefield sound radiation (horizontal plane) for a directional AVAS (left) and for a non-directional AVAS (right). SPL levels in dB.

Simulated number and position of concurrently moving vehicles was established from field visual inspection, with four vehicles considered. The simulation was implemented in the i-Simpa acoustic simulation software [38]. Noise sources are modeled at a height of 0.5 m. A grid of 14 receivers is included in the simulation as sound level measuring points, these are at a height of 1.2 m, and have an omnidirectional pattern. No physical dimensions are attributed to neither sources or receivers. Four scenarios were evaluated: a) 14 SPL A-weighted values are measured for the four vehicles simulating directional AVAS sources b) 14 SPL A-weighted values are measured for the four vehicles simulating conventional AVAS

Directional Acoustic Vehicle Alerting Systems

971

Fig. 5. Simulation model plan layout, showing the 14 receiver points and the 4 vehicle noise sources with respective moving directions (arrows). Vehicle 1 is denoted with a rectangle (red). Dimensions are in meters, ceiling height of 4 m, all surfaces are modeled as concrete (0.1 absorption coefficient).

c) 14 SPL A-weighted values are measured for one vehicle simulating directional AVAS (vehicle 1) d) 14 SPL A-weighted values are measured for one simulating conventional AVAS (vehicle 1) 3.2 Results and Discussion From the evaluated scenarios a comparison of measured sound levels was conducted. Scenarios A and B intend to represent a typical condition of various moving vehicles. The intention is to assess background noise differences that may result from directional vs. conventional AVAS sound sources. A single-number level is taken as the averaged 14 grid points measurements. The comparison yields a difference of 3.6 dBA, an attenuation for the directional sound sources scenario A. Other scenarios of interest are those in which an approaching vehicle needs to warn road users who are in its path or proximity and therefore at risk of collision. These are the focus of analyses for scenarios C and D. In these proposed cases, receivers 1 and 4 are not in the moving path of vehicle 1, yet receiver 1 is nearby (see Fig. 5). Comparing levels produced by directional and conventional AVAS, show that at receiver 1, a level of 56 dBA is measured for conventional AVAS (Fig. 6 bottom right), while an attenuation of 5.5 dBA is registered for the directional AVAS (Fig. 6 bottom left). For receiver 4,

972

F. Pereira and E. Sousa

Fig. 6. Simulation model plan layout. Evaluated scenarios A, B in top row, left to right. Scenarios, C and D at the bottom row, left to right.

which most certainly is out of danger, a level of 53 dBA is measured for conventional AVAS, and an attenuation of 7.9 dBA is achieved with a directional source. Furthermore, from scenarios C and D SPL map departures is noticeable that for relatively far receivers a directional source effectively mitigates contributions to sound pollution. Results are summarized in Tables 1 and 2. Table 1. Summarized results for scenarios A (directional source) and B (conventional source) Receiver

Scenario A Level (dBA)

Scenario B Level (dBA)

Difference dBA

14 points averaged

50.9

54.5

3.6

Table 2. Summarized results for scenarios C (directional source) and D (conventional source) Receiver

Scenario C Level (dBA)

Scenario D Level (dBA)

Difference (dBA)

1

51.1

56.6

5.5

4

45.3

53.2

7.9

Directional Acoustic Vehicle Alerting Systems

973

4 Conclusions The present study offers an overview of recent research on AVAS warning systems for electric vehicles, with a focus on mitigating noise pollution while addressing concerns regarding the reduced audibility of these vehicles. In the first section, we discuss the reasons for concern regarding the quietness of electric vehicles. In the subsequent section, we present a literature review of AVAS solutions. This review emphasizes two distinct radiation pattern design approaches: omnidirectional radiation and directional radiation. Lastly, in the final part of the study, we conduct a simulation within a closed car park to assess the impact of an AVAS solution featuring a directional radiation pattern. The car park model is based on an existing one, and four scenarios derived from field observations and measurements are implemented. The results demonstrate that directional solutions effectively offer an opportunity for mitigating noise pollution without compromising the audibility of electric vehicles to pedestrians and other road users nearby. We measured attenuations in the range of 3 to 8 dBA, indicating that sound design incorporating AVAS directional radiation pattern holds promise for achieving both increased detectability and reduced road traffic noise pollution. In light of these observations, we propose that AVAS regulations should include more specific recommendations regarding pattern directivity design.

5 Further Work and Limitations The present study is based on the conclusions of existent literature discussing the implementation of directional warning systems. It is noteworthy that there is not a vast amount of research on the subject. Particularly, reduced number of research exists reporting directivity measurements of currently implemented AVAS, a limitation that may hinder detailed comparisons to directional solutions. In the present work, a model of a car park was implemented, and an acoustic simulation performed in the software iSimpa. Limitations of this work include the no consideration for parked vehicles and geometry details, as well as simulation parameters and i-Simpa SPPS sound particles tracing method restrictions. Further work on assessment of the benefits of directional AVAS are to incorporate perceptual testing in Virtual Reality environments and acoustic simulations integrating sound wave phenomena.

References 1. Global EV Data Explorer (n.d.). https://www.iea.org/data-and-statistics/data-tools/global-evdata-explorer; 2. EV Chargers: How many do we need? (n.d.). https://press.spglobal.com/2023-01-09-EV-Cha rgers-How-many-do-we-need 3. Fritsch A (n.d.) SOUNDS & SILENCE: RECLAIMING SILENT SPACE 4. Theakston F, Weltgesundheitsorganisation (eds) (2011) Burden of disease from environmental noise: Quantification of healthy life years lost in Europe. World Health Organization, Regional Office for Europe

974

F. Pereira and E. Sousa

5. Okokon E, Turunen A, Ung-Lanki S, Vartiainen A-K, Tiittanen P, Lanki T (2015) Road-traffic noise: annoyance, risk perception, and noise sensitivity in the finish adult population. Int J Environ Res Public Health 12(6):5712–5734. https://doi.org/10.3390/ijerph120605712 6. Guski R, Schreckenberg D, Schuemer R (2017) WHO environmental noise guidelines for the European region: a systematic review on environmental noise and annoyance. Int J Environ Res Public Health 14(12):1539. https://doi.org/10.3390/ijerph14121539 7. Brink M et al (2019) A survey on exposure-response relationships for road, rail, and aircraft noise annoyance: differences between continuous and intermittent noise. Environ Int 125:277– 290. https://doi.org/10.1016/j.envint.2019.01.043 8. Altinsoy ME (2022) The evaluation of conventional, electric and hybrid electric passenger car pass-by noise annoyance using psychoacoustical properties. Appl Sci 12(10):5146. https:// doi.org/10.3390/app12105146 9. Misdariis N, Pardo L-F (2017) The sound of silence of electric vehicles – Issues and answers 10. Pardo-Ferreira MC, Torrecilla-García JA, de las Heras-Rosas C, Rubio-Romero JC (2020) New risk situations related to low noise from electric vehicles: perception of workers as pedestrians and other vehicle drivers. Int J Environ Res Public Health 17(18):6701. https:// doi.org/10.3390/ijerph17186701 11. Hanna R (2009) Incidence of pedestrian & bicyclist crashes by hybrid electric passenger vehicles. Technical report: vol. DOT HS 811 204. U.S. Department of Transportation NHTSA. https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/811204 12. Muirhead M, Walter LK (n.d.) Analysis of STATS19 data to examine the relationship between the rate of vehicle accidents involving pedestrians and type approval noise levels 13. Tataris K (2014) Incidence rates of pedestrian and bicyclist crashes by hybrid electric passenger vehicles: an update. Ann Emerg Med 64(2):195–196. https://doi.org/10.1016/j.anneme rgmed.2014.05.017 14. Commission Delegated Regulation (EU) 2017/1576—Of 26 June 2017—Amending Regulation (EU) No 540 / 2014 of the European Parliament and of the Council as regards the Acoustic Vehicle Alerting System requirements for vehicle EU-type approval. (n.d.) 15. Dortch MH (2016) Federal motor vehicle safety standards; minimum sound requirements for hybrid and electric vehicles. Federal Register NHTSA DOT 16. Campello-Vicente H, Peral-Orts R, Campillo-Davo N, Velasco-Sanchez E (2017) The effect of electric vehicles on urban noise maps. Appl Acoust 116:59–64. https://doi.org/10.1016/j. apacoust.2016.09.018 17. Sandberg U, Goubert L, Mioduszewski P (2010) Are vehicles driven in electric mode so quiet that they need acoustic warning signals? 18. Roan MJ, Neurauter L, Song M, Miller M (2021) Probability of detection of electric vehicles with and without added warning sounds. J Acoust Soc Am 149(1):599–611. https://doi.org/ 10.1121/10.0003386 19. Poveda-Martínez P, Peral-Orts R, Campillo-Davo N, Nescolarde-Selva J, Lloret-Climent M, Ramis-Soriano J (2017) Study of the effectiveness of electric vehicle warning sounds depending on the urban environment. Appl Acoust 116:317–328. https://doi.org/10.1016/j.apacoust. 2016.10.003 20. Lee SK, Lee SM, Shin T, Han M (2017) Objective evaluation of the sound quality of the warning sound of electric vehicles with a consideration of the masking effect: annoyance and detectability. Int J Automot Technol 18(4):699–705. https://doi.org/10.1007/s12239-0170069-6 21. AVAS – How to move from successful legislation to successful implementation and what are the next steps (n.d.).https://www.euroblind.org/newsletter/2022/february/en/avas-how-movesuccessful-legislation-successful-implementation-and-what 22. Azar L, Shi Y, Wooh S-C (2000) Beam focusing behavior of linear phased arrays. NDT E Int 33(3):189–198. https://doi.org/10.1016/S0963-8695(99)00043-2

Directional Acoustic Vehicle Alerting Systems

975

23. Chang J-H, Lee C-H, Park J-Y, Kim Y-H (2009) A realization of sound focused personal audio system using acoustic contrast control. J Acoust Soc Am 125(4):2091–2097. https:// doi.org/10.1121/1.3082114 24. Pompei FJ (2001) Directional acoustic alerting system. Patent no. US7106180B1. https://pat ents.google.com/patent/US7106180B1/en?q=(Directional+acoustic+alerting+system)&oq= Directional+acoustic+alerting+system 25. EVADER: Electric vehicle alert for detection and emergency response (n.d.). https://cordis. europa.eu/project/id/285095 26. Berkhoff, A. P. (2013). Directional sound sources using real-time beamforming control. https://repository.tno.nl//islandora/object/uuid:90ce29f9-5eff-4882-83ef-595644676df4 27. Cheer J, Birchall T, Clark P, Moran J, Elliott SJ, Fazi FM (2013) Design and implementation of a directive sound field for an electric car warning sound. In: Proceedings of the institute of acoustics, vol 35 28. Quinn D, Mitchell J, Clark P (2014) Development of a next-generation audible pedestrian alert system for EVs having minimal impact on environmental noise levels project eVADER 29. Berkhoff AP (2014) Source arrays for directional and non-directional sound generation 30. Robart R, et al (2013) EVADER: A perceptual approach to finding minimum warning sound requirements for quiet cars 31. Lee J, et al (2018) Experimental and numerical study on speaker design of active pedestrian alerting system (APAS) in hybrid and pure electric vehicles. 2018-01-1550. https://doi.org/ 10.4271/2018-01-1550 32. Kournoutos N, Cheer J, Elliott SJ (2018) The design of a low-cost directional warning sound system for electric vehicles 33. Kournoutos N, Cheer J (2019) Design and realisation of a directional electric vehicle warning sound system, 040001. https://doi.org/10.1121/2.0001224 34. Kournoutos N, Cheer J (2020) Investigation of a directional warning sound system for electric vehicles based on structural vibration. J Acoust Soc Am 148(2):588–598. https://doi.org/10. 1121/10.0001681 35. Kournoutos N, Cheer J (2020) A system for controlling the directivity of sound radiated from a structure. J Acoust Soc Am 147(1):231–241. https://doi.org/10.1121/10.0000589 36. Orts RP, Vicente HC, Davo NC, Martínez PP, Soriano JR (2016) Experimental study of the noise directivity of a hybrid electric vehicle 37. Campillo-Davo N, et al (n.d.) Numerical analysis of the exterior acoustic field emitted by an industrial ev equipped with avas system 38. I-Simpa an open source software for 3D sound propagation modelling (n.d.). https://i-simpa. univ-gustave-eiffel.fr

Author Index

A Almeida, Flávio 639 Almeida, Sofia 809 Almendra, Rita 336, 629 Alvelos, Heitor 470 Alves, Cesário 889 Amaro, Ana Carla 651 Angari, Roberta 615 Antunes, Ines P. 639 Antunes, Maria Fernanda 26 Aparo, Ermanno 336 Ayanoglu, Hande 487 B Baki, Ramisha 122 Baldi, Vania 287 Banik, Gobinda 765 Barbas, Maria Potes 695 Barreto, Luís 15 Barreto, Susana 470 Barros, Paulo 677 Beça, Pedro 402 Berbel, Noemy 629 Bertolotti, Elisa 706 Bonini Lessing, Emanuela F. 371 Botelho-Francisco, Rodrigo Eduardo Branco, Paulo Duarte 695 Brandão, João Aranda 840 Brasil, Ramusyo 889 Brito-Costa, Sónia 26 Buono, Mario 413, 555 Burrows, Alison 498 C Cadete, Carla 856 Canina, Marita 508 Capece, Sonia 413, 555 Caputo, Francesco 413 Cardoso, Patrícia Azinheira Cardoso, Pedro 108, 357

520

Carleo, Salvatore 608 Carrega, Jorge 79 Carvalhais, Miguel 37, 108 Carvalho, Mariana Alves 487 Carvalho, Vítor 184 Cerejo, Joana 37 Chatterjee, Abhishek 470 Choi, Albert Young 542, 719, 753, 797, 873 Clemente, Violeta 569 Coelho, Rita 777 Colaço, Inês 390 Cordeiro, Bruno 381 Coutinho, Sérgio 936 Curado, António 677 Curralo, Ana Filomena 15, 677

425

D d’Orey, Rui 79 da Silva, Elton Rubens Vieira 402 da Veiga, Pedro Alves 470 Das, Partha 765 de Almeida Magalhães, Artur Porto 594 de Almeida, Pedro Duarte 594 De Luca, Vanessa 706 Di Roma, Annalisa 530 Dinis, Frederico 901 Drumeva, Krasimira 251 Duarte, Emília 165 Dutton, Emanuele 498 E Espada, Sílvia 26 Esteves, Jéssica Rodrigues 53, 922 Estima, Jacinto 136 Ezingmüller, Caroline 569 F Fan, Jieqiong 542 Fernandes, Andreia 224 Fernandes, Catarina 15

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 N. Martins and D. Brandão (Eds.): DIGICOM 2023, SSDI 35, pp. 977–979, 2024. https://doi.org/10.1007/978-3-031-47281-7

978

Author Index

Ferreira, Luís 508 Fonseca, Jorge 777 Freitas, Inês 677

Losciale, Piera 530 Loureiro, Eduarda 584

J Jahangir, Raiyan 94 Júnior, Demerval Gomes S. 136

M Mahmud, Hasan 122 Majhi, Manoj 765 Manique, Ana 459 Marques, Rita 174 Marques, Tiago 629 Martelo, Rita 301 Martín Muñoz, Diego 950 Martins, Alexandre 79 Martins, Nuno 184 Masseno, Manuel 174 Matos, Demétrio 184 Meirinhos, Manuel 311 Mendonça, Rui 390, 826 Merino, Eugenio 922 Merino, Giselle 922 Miranda, Daniel 184 Mohim, Nasif Shahriar 94 Monestier, Eva 508 Morais, Nídia Salomé 911 Morais, Rodrigo 449 Morgado, Aprígio 459 Mumu, Afnan 122 Muñoz Martínez, Víctor Fernando

K Kamal, Mahedi 94 Kaminska, Dorota 3 khan, Imran 211

N Noriega, Paulo 174 Norogrando, Rafaela 200 Nunes, Maria João 665

L L.Vilaça, João 241 Lankadari, Sajini 732 Lanza, Dario 67 Laska-Lesniewicz, Anna 3 Leite, Luís 224 Lelis, Catarina 345, 809 Lima, Cláudia 390, 826 Lima, Cláudia Raquel 470 Lima, Octávio 241 Lombardi, Ilaria 555 Lopes, Filipe 936 Lopes, Inês 174 Lopes, Lígia 390 Lopes, Sérgio Ivan 677

O Oliveira, Ana 911 Oliveira, Eliza 287 Oliveira, Lídia 211, 287, 425, 651

G Gago, José 629 Giugliano, Giovanna 413 Gomes, João 787 Gomes, João Francisco Rodrigues Gonçalves, Berenice 922 Gonçalves, Berenice Santos 53 Gouveia, Teresa 911 Grieger, Jenifer Daiane 425 Guzman, Bernardo Meza 345 H Hernández-Ramírez, Rodrigo 487 I Islam, Iyolita 94 Islam, Md Tariquel 94 Islam, Muhammad Nazrul

840

136, 438, 449,

94, 122

P Pache de Faria, Clara 200 Paiva, Sara 15 Pappalettera, Giovanni 530 Parreira, Suzana 323, 381 Parrilli, Davide M. 438 Peixoto, Sara 184 Penedos-Santiago, Eliana 470 Pereira, Frederico 963 Pereira, Jorge Brandão 470

555

Author Index

979

Pombo, Fátima 569 Pontillo, Gabriele 615 Providência, Francisco 152 Q Quintero, Edgar Nunes

345

R Rafael, Sónia 261 Rahman, Mouneeta 122 Ranzo, Patrizia 608 Raposo, Rui 3, 911 Rebelo, Francisco 165, 174 Reigoto, Sara 276 Reis, Lara 165 Reis, Matilde 261 Reis, Miriam 569 Reisinho, Pedro 3 Rocha, Luis 922 Rodrigo, Buddhika 732 Rolo, Elisabete 665, 787 Rolo, Ema 629 Rosa, Carlos 639 S Sá Couto, Pedro 108 Santo, Pedro Espírito 520 Scarcelli, Alessandra 530 Senese, Vincenzo Paolo 555 Silva, Bruno Mendes da 79 Silva, Verónica 449 Singh, Gurdeep 765

Soares, Liliana 276, 336 Sousa, Emanuel 963 Sousa, Fernando 152 Sultana, Zinia 94 T Teixeira, Ana Rita 26 Teixeira, Júlio Monteiro 922 Terroso, Miguel 241 Ting, Yang 719 Tinoco, João 826 Trigueiros, Paula 498 Tschimmel, Katja 569 V Vairinhos, Mário 3 Vezzani, Valentina 508 Viana, Óscar T. 241 Vicente, Fernanda 311 Vieira, Andreia Teles 695 Vieira, Maria Júlia 357 Vilar, Elisângela 174 W Wang, Xin 797 Wei, Wenbo 753 Z Zannat, Riyadil 122 Zeyuan, Xie 873 Zhao, An 753 Zwolinski, Grzegorz 3